Large-scale 3D Modeling from Crowdsourced Data
Large-scale image-based 3D modeling has been a major goal of computer vision, enabling a wide range of applications including virtual reality, image-based localization, and autonomous navigation. One of the most diverse data sources for modeling is Internet photo collections. In the last decade, the computer vision community has made tremendous progress in large-scale structure-from-motion and multi-view stereo from Internet datasets. However, utilizing this wealth of information for 3D modeling remains a challenging problem due to the ever-increasing amount of image data. In a short period of time, research in large-scale modeling has progressed from modeling using several thousand images, to modeling from city-scale datasets of several million, and recently to reconstructing an Internet-scale dataset comprising 100 million images. This tutorial will present the main underlying technologies enabling these innovations.
- Motivation and basic principles
- Building blocks of image-based 3D reconstruction
- Challenges and opportunities of crowd-sourced data
- Single- and two-view image characteristics
- Local features
- Feature matching
- Projective geometry
- Epipolar geometry
- Large-scale visual data association
- Image indexing
- Image registration
- Connected-component discovery
- Sparse modeling
- Bundle adjustment
- Challenges in large-Scale modeling
- Dense modeling
- Depth and normal estimation
- Fusion and surface generation
jmf (at) cs.unc.edu
Associate Professor, UNC Chapel Hill
edunn (at) stevens.edu
Associate Professor, Stevens Institute of Technology
marc.pollefeys (at) inf.ethz.ch
Full Professor, ETH Zürich
Director of Science, Microsoft HoloLens
jared.heinly (at) urcventures.com
Senior Research Engineer, URC Ventures
Johannes L. Schönberger
jsch (at) inf.ethz.ch
Ph.D. Student, ETH Zürich
- Johannes L. Schönberger, Jan-Michael Frahm. "Structure-from-Motion Revisited", CVPR 2016.
- Jared Heinly, Johannes L. Schönberger, Enrique Dunn, Jan-Michael Frahm. "Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset)", CVPR 2015.
- Johannes L Schönberger, Filip Radenovi, Ondrej Chum, Jan-Michael Frahm. "From Single Image Query to Detailed 3D Reconstruction", CVPR 2015.
- Jan-Michael Frahm, Pierre Georgel, David Gallup, Tim Johnson, Rahul Raguram, Changchang Wu, Yi-Hung Jen, Enrique Dunn, Brian Clipp, Svetlana Lazebnik, Marc Pollefeys. "Building Rome on a Cloudless Day", ECCV 2010.
- Jan-Michael Frahm, Marc Pollefeys, Svetlana Lazebnik, Christopher Zach, David Gallup, Brian Clipp, Rahul Raguram, Changchang Wu, Tim Johnson. "Fast Robust Large-scale Mapping from Video and Internet Photo Collections", In special issue "100 years of ISPRS" of the ISPRS Journal of Photogrammetry and Remote Sensing.
- Enliang Zheng, Enrique Dunn, Valdimir Jojic, Jan-Michael Frahm. "PatchMatch Based Joint View Selection and Depthmap Estimation", CVPR 2014.
- David Gallup, Marc Pollefeys, Jan-Michael Frahm. "3D Reconstruction using an n-LayerHeightmap", DAGM 2010.
- Johannes L. Schönberger, Enliang Zheng, Marc Pollefeys, Jan-Michael Frahm. "Pixelwise View Selection for Unstructured Multi-View Stereo", ECCV 2016.