Header logo is

Semantic Visual Localization


Conference Paper


Robust visual localization under a wide range of viewing conditions is a fundamental problem in computer vision. Handling the difficult cases of this problem is not only very challenging but also of high practical relevance, eg, in the context of life-long localization for augmented reality or autonomous robots. In this paper, we propose a novel approach based on a joint 3D geometric and semantic understanding of the world, enabling it to succeed under conditions where previous approaches failed. Our method leverages a novel generative model for descriptor learning, trained on semantic scene completion as an auxiliary task. The resulting 3D descriptors are robust to missing observations by encoding high-level 3D geometric and semantic information. Experiments on several challenging large-scale localization datasets demonstrate reliable localization under extreme viewpoint, illumination, and geometry changes.

Author(s): Johannes Schönberger and Marc Pollefeys and Andreas Geiger and Torsten Sattler
Book Title: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2018
Publisher: IEEE Computer Society

Department(s): Autonomous Vision
Research Project(s): Global Localization and Affordance Learning
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Event Place: Salt Lake City, USA

Links: pdf


  title = {Semantic Visual Localization },
  author = {Sch{\"o}nberger, Johannes and Pollefeys, Marc and Geiger, Andreas and Sattler, Torsten},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  publisher = {IEEE Computer Society},
  year = {2018}