LiFCal Online Light Field Camera Calibration via Bundle Adjustment

Aymeric Fleith1,2
Doaa Ahmed2
Daniel Cremers1
Niclas Zeller2
1Technical University of Munich, Munich, Germany
2Karlsruhe University of Applied Sciences, Karlsruhe, Germany
Metric depth estimation
Process overview: Raw images undergo camera calibration via plenoptic bundle adjustment. This yields a metric camera model used to compute a totally focused image and depth map from new raw images, enabling accurate metric depth measurement.

Abstract

We propose LiFCal, a novel geometric online calibration pipeline for MLA-based light field cameras. LiFCal accurately determines model parameters from a moving camera sequence without precise calibration targets, integrating arbitrary metric scaling constraints. It optimizes intrinsic parameters of the light field camera model, the 3D coordinates of a sparse set of scene points and camera poses in a single bundle adjustment defined directly on micro image points.

We show that LiFCal can reliably and repeatably calibrate a focused plenoptic camera using different input sequences, providing intrinsic camera parameters extremely close to state-of-the-art methods, while offering two main advantages: it can be applied in a target-free scene, and it is implemented online in a complete and continuous pipeline.

Furthermore, we demonstrate the quality of the obtained camera parameters in downstream tasks like depth estimation and SLAM.

Pipeline

Pipeline
Flowchart of the online calibration algorithm for the focused plenoptic camera. Images are acquired from the camera and are first used to initialize the parameters. Next, a complete bundle adjustment for the plenoptic camera model is performed.

Poster

Poster

License Terms

LiFCal was developed in collaboration between the Technical University of Munich and the Karlsruhe University of Applied Sciences. The code is open-source under a GNU General Public License Version 3 (GPLv3).

BibTeX

@inproceedings{Fleith2024LiFCal,
        title     = {LiFCal: Online Light Field Camera Calibration via Bundle Adjustment},
        author    = {Fleith, Aymeric and Ahmed, Doaa and Cremers, Daniel and Zeller, Niclas},
        booktitle = {German Conference on Pattern Recognition (GCPR)},
        year      = {2024},
    }