Construction API

Con-API’s Computer Vision Research

Transforming visual data into actionable insights for construction and infrastructure projects.

Our Computer Vision division develops novel algorithms and systems to extract rich, structured information from site imagery, videos, and 3D scans. Our solutions combine deep learning, BIM integration, and 3D reconstruction to develop automated systems for the detection, measurement, and monitoring of construction elements and site conditions. These methods represent the foundation of many of our Progress Monitoring, Safety Monitoring, and Operation Monitoring applications which requires comprehensive data from raw sources.

Research themes:

• 2D/3D object detection and segmentation
• Semantic enrichment of BIM models
• Structure-from-motion and photogrammetry
• Domain adaptation from synthetic to real imagery
• Vision-based data fusion with IoT and robotics

Featured Representative Works

1. Lin, P. C., Lin, J. J., and S. H. Hsieh (2023). “Construction Site Scaffolding Completeness Detection Based on Mask RCNN and Hough Transform,” Proceedings of the 30th EG-ICE, July 4-7, 2023, London, UK, 235-244. (arXiv) ❐

This paper presents a safety inspection method, utilizing deep learning-based object detection and geometric feature extraction, to automatically assess the completeness of scaffolding structures on construction sites. Missing components (e.g., cross braces) are a frequent cause of safety violations but manually inspection is time-consuming and labor-intensive.

The proposed method combines Mask R-CNN, a deep learning–based object detection framework, with Hough Transform geometry detection to identify both the scaffolding frames and their cross braces from only site photographs. By comparing detected configurations against the expected complete setup, the system can flag incomplete scaffolding without any manual input. The non-invasive and automated proposal has the potential to greatly reduce inspection time, cut labor costs, and improve on-site safety compliance.

---

2. Pal, A., Lin, J. J.*, Hsieh, S. H., & Golparvar-Fard, M. (2024). Activity-level construction progress monitoring through semantic segmentation of 3D-informed orthographic images. Automation in Construction, 157, 105157. (SCI) ❐

This paper introduces the novel ALPMS (Activity-Level Progress Monitoring System) framework which tracks the percent completion of construction tasks by activity. This method can provide granular and quantitative information unlike traditional binary methods. Using new solutions such as projective transformation or NeRF (Neural Radiance Fields), orthographic projections are semantically segmented to detect the current stage of each building element.

Site photographs are generated into as-built 3D point clouds and is aligned with the project’s 4D BIM. The progress metrics is then passed back into the BIM for 3D visualization and schedule updates. Validated on two building projects, ALPMS achieved high accuracy, reporting activity-level completion with less than 6% mean absolute error.

Conference Highlights

Yu, P., Cheng, P.C., Lin, J. J. and Wang, L. (2023). “Estimating Urban Heat Island Effect through Building Façade System and Form Detection on Street View and Aerial Images,” Proceedings of the 30th European Group for Intelligent Computing in Engineering (EG-ICE 2023), July 4-7, 2023, London, UK, 235-244.

Pal, A., Lin, J. J., and S. H. Hsieh (2021). “Semantic Segmentation of Superpixels for Vision-based Automated Construction Progress Reporting,” Proceedings of the 25th Symposium on Construction Engineering and Management, Paper No. 141, July 16, 2021, Taipei, Taiwan.

Project Involvement within this field

營建工地智慧視覺監視與自動報告系統的研發
Research and development of intelligent visual monitoring and automatic reporting system for construction sites.
2022.Nov – 2024.Nov. Co-PI
Sponsor: National Science and Technology Council (NSTC)