Hybrid multi-view reconstruction by Jump-Diffusion

Florent Lafarge (1, 2) , Renaud Keriven (2, 3) , Mathieu Brédif (4) , Hoang-Hiep Vu (2, 3)

1 ARIANA - Inverse problems in earth monitoring
2 imagine [Marne-la-Vallée]
3 LIGM - Laboratoire d'Informatique Gaspard-Monge
4 MATIS - Méthodes d'Analyses pour le Traitement d'Images et la Stéréorestitution

We propose a multi-view stereo reconstruction algorithm which recovers urban scenes as a combination of meshes and geometric primitives. It provides a compact model while preserving details: irregular elements such as statues and ornaments are described by meshes whereas regular structures such as columns and walls are described by primitives (planes, spheres, cylinders, cones and tori). A Jump-Diffusion process is designed to sample these two types of elements simultaneously. The quality of a reconstruction is measured by a multi-object energy model which takes into account both photo-consistency and semantic considerations (i.e. geometry and shape layout). The sampler is embedded into an iterative refinement procedure which provides an increasingly accurate hybrid representation. Experimental results on complex urban structures and large scenes are presented and compared to multi-view based meshing algorithms.

Mots clés: Multi-view Stereo; surface reconstruction; Jump-diffusion; primitive extraction; hybrid representation

Domaines: Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Florent Lafarge, Renaud Keriven, Mathieu Brédif, Hoang-Hiep Vu. Hybrid multi-view reconstruction by Jump-Diffusion. CVPR 2010, Twenty-Third IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun 2010, San Francisco, United States. pp.350-357, ⟨10.1109/CVPR.2010.5540193⟩. ⟨hal-00654411⟩