I finished this project last year and it’s been ‘dying’ to be posted. My goal was to take point clouds obtained from two Kinects and register them into one coordinate system. After the registration process was completed, the new point cloud would be much more robust with many of the obstructed blank spots filled in.
The project follows a simple algorithm.
- Use libfreenect to obtain point clouds from both Kinects
- Use OpenGL to display the point clouds in an interactice virtual environment
- Use OpenCV to display the RGB streams for the user to select correspondences
- Calculate the transformations using Procrustes Analysis and the correspondence matrices
- Apply the translation and rotation to the point clouds visualized in OpenGL
The implementation of this algorithm can be found here.
After a few weeks of stagnant development with the project, I made a bet with my friends and advisor that I could finish the project in one weekend before I left for a vacation. Below is the video which resulted from my sleepless weekend hackathon.
After the project was finished. I used it to complete what is called an “Honor’s Thesis” here at my University. It is an undergraduate research project, which once defended successfully allows the student to graduate “with honors” on their diploma. Looking back on the thesis now I would have done things differently – but isn’t that almost always the case. None-the-less it was a mile stone in my life and it is my work.
Let me know if you’d like a copy of it and I’ll be happy to send it to you!
I watched the video and, whoa! Very nice stuff! I would like to see the copy of your thesis, if possible
Thanks and good job!
Thanks for the feedback! I’m working on adapting this to some interesting immersive visualizations for my master’s work. I sent the paper, let me know what you think!
Looks cool. Did you implement your 3D render for the point cloud with openGL. Could you send me the code of that part?
Thanks for the feedback Brian. Yes I did, the project is hosted on my github – https://github.com/mathnathan/Kinect-Registration
Thanks for sharing, Nathan. Really nice job. Looking forward to your Thesis.
Hi,
I’m interested with your project about using this dual kinect
Can you share your thesis to me as well ?
Looking forward to your Thesis
Thank you
I have tried your code which is downloaded from https://github.com/mathnathan/Kinect-Registration and built in my Ubuntu 12.04 and using OpenCV2.4
But, I got an error like :
”
[22%] Built target freenect
[22%] Built target freenect_sync
Linking C shared library ../../../../lib/libfreenect_cv.so
/usr/bin/ld: cannot find -lcv
collect2: ld returned 1 exit status
make[2]: *** [lib/libfreenect_cv.so.0.0.1] Error 1
make[1]: *** [include/libfreenect/wrappers/opencv/CMakeFiles/freenect_cv.dir/all] Error 2
make: *** [all] Error 2
”
Have tried with other PC , but still got the same result as above
Is there any suggestion to solve this problem ?
Hey Edmond, It’s nice to see your interest in my project. I haven’t looked at it in a very long time. Perhaps tomorrow after work I will have sometime to sit down and try to recompile it in Ubuntu 12.04. I’ll keep you posted, and be expecting an email shortly with my thesis. Cheers!
Thank you for your prompt reply.
Btw, I’m using your “libfreenect” which is inside of the folder “include” of Kinect – Registration
When I tried building with “cmake ..” , it seems needs two other folders like “example” and “fakenect” instead of “include” folder.
Also, it needs the folder of “cpp” for the inside of folder “wrappers”.
So, I copy the folders from https://github.com/OpenKinect/libfreenect then put it in inside of the folder your “libfreenect”.
Then, I got the result like as above
But, I got an error like :
”
[22%] Built target freenect
[22%] Built target freenect_sync
Linking C shared library ../../../../lib/libfreenect_cv.so
/usr/bin/ld: cannot find -lcv
collect2: ld returned 1 exit status
make[2]: *** [lib/libfreenect_cv.so.0.0.1] Error 1
make[1]: *** [include/libfreenect/wrappers/opencv/CMakeFiles/freenect_cv.dir/all] Error 2
make: *** [all] Error 2
”
Hope it is clear.
Looking forward to see your thesis and the message for solving this problem.
Hi!Thats so awesome!I’m doing a final year (undergrad project too!) in point clouds.Its completely a new field for me and I’m overwhelmed by what platform to work with.I have a windows 64bit and am not sure whether to use cpp or Matlab?May I please get a copy of your thesis?Thanks!
Hi Nathan, I am doing something similar, may I have a look on your thesis.
Thanks very much!
Hi Nathan,
This looks very interesting. I’m working on geomorphological applications, either with multiple Kinects or one Kinect from multiple positions. I’d be fascinated to read your thesis, as it looks like you may save duplication of effort.
Ben
Pingback: Registration of Multiple Kinects | Visualization Lab at FSU
Nice work, thanks for sharing! Have you considered using 2D features like SIFT to automatize correspondence search?
I am very interested in a copy of your thesis!
Thanks!
Thanks Alex. Actually, YES! That was my first approach and in my thesis I outline the process. Unfortunately it was not as robust as the interactive matching and I needed results. I have discontinued working on this, however people in the lab here in my department have picked up my work and made some very interesting progress. We now use the PCL library and do registration using segmented planes from the image. We’ll be sure to blog about it soon on the Visualization Lab website. I’ll take your email and update you when we do. Thanks for reaching out Alex.
Hey Ben, thanks for reaching out. Sorry for the delayed reply on things, I’ve been slaving away on my graduate thesis. My work is old news now. We’ve made big progress since then on registration and 3D recording, I’ll take your email and update you when we post some of our work on the Lab’s blog. Cheers!