Automatic Enrichment of Indoor 3D Models using a Deep Learning Approach based on Single Images with Unknown Camera Poses

Research output: Contribution to journalResearch articleContributedpeer-review


  • M. Jarzabek-Rychard - , Wroclaw University of Environmental and Life Sciences, TUD Dresden University of Technology (Author)
  • H. G. Maas - , TUD Dresden University of Technology (Author)


3D building modeling is a diverse field of research with a multitude of challenges, where data integration is an inherent component. The intensively growing market of BIM-related consumer applications requires methods and algorithms that enable efficient updates of existing 3D models without the need for cost-intensive data capturing and repetitive reconstruction processes. We propose a novel approach for semantic enrichment of existing indoor models by window objects, based on amateur camera RGB images with unknown exterior orientation parameters. The core idea of the approach is the parallel estimation of image camera poses with semantic recognition of target objects and their automatic mapping onto a 3D vector model. The presented solution goes beyond pure texture matching and links deep learning detection techniques with camera pose estimation and 3D reconstruction. To evaluate the performance of our procedure, we compare the estimated camera parameters with reference data, obtaining median values of 13.8 cm for the camera position and 1.1° for its orientation. Furthermore, a quality of 3D mapping is assessed based on the comparison to the reference 3D point cloud. All the windows presented in the data source were detected successfully, with a mean distance between both point sets equal to 3.6 cm. The experimental results prove that the presented approach achieves accurate integration of objects extracted from single images with an input 3D model, allowing for an effective increase of its semantic coverage.


Original languageEnglish
Pages (from-to)1-7
Number of pages7
JournalISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Issue numberW1-2022
Publication statusPublished - 3 Feb 2022
Externally publishedYes


Title2022 Measurement, Visualisation and Processing in BIM for Design and Construction Management II
Duration7 - 8 February 2022
CountryCzech Republic

External IDs

Mendeley 7a2a0fb0-91ce-3a43-97f6-5376539b0aa4
WOS 000860711400001



  • Building Information Model (BIM), camera pose estimation, deep learning, object recognition, texture mapping

Library keywords