Registering Two Point Clouds

I finished this project last year and it’s been ‘dying’ to be posted. My goal was to take point clouds obtained from two Kinects and register them into one coordinate system. After the registration process was completed, the new point cloud would be much more robust with many of the obstructed blank spots filled in.

The project follows a simple algorithm.

  1. Use libfreenect to obtain point clouds from both Kinects
  2. Use OpenGL to display the point clouds in an interactice virtual environment
  3. Use OpenCV to display the RGB streams for the user to select correspondences
  4. Calculate the transformations using Procrustes Analysis and the correspondence matrices
  5. Apply the translation and rotation to the point clouds visualized in OpenGL

The implementation of this algorithm can be found here.

After a few weeks of stagnant development with the project, I made a bet with my friends and advisor that I could finish the project in one weekend before I left for a vacation. Below is the video which resulted from my sleepless weekend hackathon.

After the project was finished. I used it to complete what is called an “Honor’s Thesis” here at my University. It is an undergraduate research project, which once defended successfully allows the student to graduate “with honors” on their diploma. Looking back on the thesis now I would have done things differently – but isn’t that almost always the case. None-the-less it was a mile stone in my life and it is my work.

Let me know if you’d like a copy of it and I’ll be happy to send it to you!

This entry was posted in 3D Scene Reconstruction. Bookmark the permalink.

20 Responses to Registering Two Point Clouds

  1. Giseli says:

    I watched the video and, whoa! Very nice stuff! I would like to see the copy of your thesis, if possible :) Thanks and good job!

  2. Nathan Crock says:

    Thanks for the feedback! I’m working on adapting this to some interesting immersive visualizations for my master’s work. I sent the paper, let me know what you think!

  3. brian lee says:

    Looks cool. Did you implement your 3D render for the point cloud with openGL. Could you send me the code of that part?

  4. Nathan Crock says:

    Thanks for the feedback Brian. Yes I did, the project is hosted on my github –

  5. Brian lee says:

    Thanks for sharing, Nathan. Really nice job. Looking forward to your Thesis.

  6. Edmond Ngantung says:

    I’m interested with your project about using this dual kinect
    Can you share your thesis to me as well ?
    Looking forward to your Thesis
    Thank you

  7. Edmond Ngantung says:

    I have tried your code which is downloaded from and built in my Ubuntu 12.04 and using OpenCV2.4
    But, I got an error like :

    [22%] Built target freenect
    [22%] Built target freenect_sync
    Linking C shared library ../../../../lib/
    /usr/bin/ld: cannot find -lcv
    collect2: ld returned 1 exit status
    make[2]: *** [lib/] Error 1
    make[1]: *** [include/libfreenect/wrappers/opencv/CMakeFiles/freenect_cv.dir/all] Error 2
    make: *** [all] Error 2

    Have tried with other PC , but still got the same result as above
    Is there any suggestion to solve this problem ?

  8. Nathan Crock says:

    Hey Edmond, It’s nice to see your interest in my project. I haven’t looked at it in a very long time. Perhaps tomorrow after work I will have sometime to sit down and try to recompile it in Ubuntu 12.04. I’ll keep you posted, and be expecting an email shortly with my thesis. Cheers!

  9. Edmond Ngantung says:

    Thank you for your prompt reply.
    Btw, I’m using your “libfreenect” which is inside of the folder “include” of Kinect – Registration
    When I tried building with “cmake ..” , it seems needs two other folders like “example” and “fakenect” instead of “include” folder.
    Also, it needs the folder of “cpp” for the inside of folder “wrappers”.
    So, I copy the folders from then put it in inside of the folder your “libfreenect”.
    Then, I got the result like as above
    But, I got an error like :
    [22%] Built target freenect
    [22%] Built target freenect_sync
    Linking C shared library ../../../../lib/
    /usr/bin/ld: cannot find -lcv
    collect2: ld returned 1 exit status
    make[2]: *** [lib/] Error 1
    make[1]: *** [include/libfreenect/wrappers/opencv/CMakeFiles/freenect_cv.dir/all] Error 2
    make: *** [all] Error 2
    Hope it is clear.
    Looking forward to see your thesis and the message for solving this problem.

  10. Bhavani says:

    Hi!Thats so awesome!I’m doing a final year (undergrad project too!) in point clouds.Its completely a new field for me and I’m overwhelmed by what platform to work with.I have a windows 64bit and am not sure whether to use cpp or Matlab?May I please get a copy of your thesis?Thanks!

  11. Lin says:

    Hi Nathan, I am doing something similar, may I have a look on your thesis.
    Thanks very much!

  12. Ben Evans says:

    Hi Nathan,

    This looks very interesting. I’m working on geomorphological applications, either with multiple Kinects or one Kinect from multiple positions. I’d be fascinated to read your thesis, as it looks like you may save duplication of effort.

  13. Pingback: Registration of Multiple Kinects | Visualization Lab at FSU

  14. Alex says:

    Nice work, thanks for sharing! Have you considered using 2D features like SIFT to automatize correspondence search?
    I am very interested in a copy of your thesis!

  15. Nathan Crock says:

    Thanks Alex. Actually, YES! That was my first approach and in my thesis I outline the process. Unfortunately it was not as robust as the interactive matching and I needed results. I have discontinued working on this, however people in the lab here in my department have picked up my work and made some very interesting progress. We now use the PCL library and do registration using segmented planes from the image. We’ll be sure to blog about it soon on the Visualization Lab website. I’ll take your email and update you when we do. Thanks for reaching out Alex.

  16. Nathan Crock says:

    Hey Ben, thanks for reaching out. Sorry for the delayed reply on things, I’ve been slaving away on my graduate thesis. My work is old news now. We’ve made big progress since then on registration and 3D recording, I’ll take your email and update you when we post some of our work on the Lab’s blog. Cheers!

  17. Keith says:

    Hi, I am very interested in your theory, could u plz send it to me?

  18. James Chen says:

    Very nice work!
    I would like to see the copy of your thesis
    However, I’m working on similar project with two Kinects using PCL (
    Do you have any suggestion or advise about that?
    Thank you!

  19. Tanaji Kamble says:

    Hello Nathan,
    U done really nice work.. i am very much curious to know how u done that..i want to go through ur graduation thesis and updated work in detail.. if possible n time permits to u den let me know what will be the potential applications for 3D reconstruction using multiple depth cameras..i am eagerly waiting for ur reply…my self also want to work with multiple depth cameras but unfortunately i am unfamiliar with linux i have plan to work on windows platform using point cloud library..

  20. Tanaji says:

    Hi Nathan,

    I like your work its very interesting. Iā€™d like to read your thesis, can send me on my mail..

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>