Calibration and overlay of RGBD with a thermal camera

I have a thermal camera that I would like to overlay automatically to the output of a RGBD camera (e.g., Kinect).

I can retrieve the intrinsic parameters of the thermal camera by using a special board I built, that allows the thermal camera to give as output a series of points similar to the ones obtained with a visible sensor and a typical calibration chessboard.

Once the intrinsic parameters of the thermal camera are obtained, what would the next steps be to align all the cameras? It's important to take into account that the two cameras deeply differ from each other, because they operate in different wavelengths.

I suggest that you obtain both the intrinsic and extrinsic parameters for both cameras. The extrinsic parameters will include information like the offset/distance between the two cameras and the relationship between their two headings, which is needed to relate their images. (If only the relative position and orientation/heading of the two cameras is known, that will be enough.)

Once you have these parameters, you'll be able to map each pixel in the RGBD image to its corresponding pixel in the thermal image. This will let you align those two images. Then, depending upon the specific setup, you might need to do fine-grained image stabilization (e.g., if they are not on tripods).

It's not clear why the different wavelengths would have any impact on alignment of the two images.

See also https://en.wikipedia.org/wiki/Camera_resectioning.

StackExchange Computer Science Q#44838, answer score: 2