Camera Calibration

Every camera has a unique camera sensor and camera lens. When an image is captured by the camera, it has its own distortion characteristics. There are two types of distortion – radial and tangential distortion. This distortion results in a curved image being captured rather than a flat image. A problem brought about by this fact is the camera is not truly aware of the position various objects are in the space around the drone. A curved, distorted image would create the illusion that an object is a certain distance away from the drone but in reality its location is slightly offset. This is an issue if we need to maneuver the drone to point in the direction of a detected face. Hence, we need to undergo camera calibration to obtain the camera matrix. This camera matrix determines the amount of curvature the image undergoes in all axes.

Hence, we need to undergo camera calibration in order to generate this matrix. OpenCV is able to do this by detecting a chessboard pattern. The figure below shows various pictures of the same chessboard pattern captured at different angles. The pictures had to be as close to the lens as possible in order for the calibration script to generate the most accurate matrix. The further away an object is from the camera, the less distortion and curvature it undergoes. Hence, it should be located as close to the camera lens as possible.

By feeding about 15-20 pictures into the camera calibration script, it was able to detect the chessboard pattern and based on the curvature of the lines joining the points along the chessboard, it was able to generate a camera matrix which contains several coefficients responsible for undistorting the image. By feeding in multiple images, the matrix obtained was far more accurate in terms of undistorting the image reliably. The camera matrix generated is as follows:

With this camera matrix, we can now proceed with image flattening.