How Artificial Intelligence Is Improving Efficiency on Our Construction Sites

In terms of digitalisation, the construction industry has the reputation of being rather slow and of failing to exploit the enormous untapped potential of digital technologies. This assessment is unwarranted, however, because the sleeping giant has long been awake. Besides the well-established digital method of Building Information Modelling (BIM), the industry is already using many other innovative digital solutions. And when it comes to the development of such solutions, the STRABAG Group is right on the cutting edge. A good example of this is the ESKIMO research project.

Daniele Roos

ZÜBLIN Subdivision Stuttgart

The ESKIMO research project addresses specific AI-related digitalisation hurdles in the construction industry with the cross-disciplinary networking of eleven partners from research and science, the IT industry and construction. As part of the project, ZÜBLIN has made two construction projects in Germany available for the collection of training data. Active support in the form of know-how and experience also comes from the digitalisation professionals at STRABAG Innovation & Digitalisation.

The project is being funded by the German Ministry of Education and Research. It was officially launched on 1 April 2020 and is expected to end in March 2022 after a period of 24 months. The project’s focus lies on the use of AI in construction with the aim of significantly boosting efficiency in construction management. To this end, three usage scenarios are of particular interest:

Technical usage scenario

Automated defect documentation can significantly speed up site inspections and the process of acceptance while helping to improve the quality of the work. By simply pointing a smartphone or tablet with the ESKIMO solution installed at a building defect, an automatic suggestion appears as to which kind of defect it could be. If the suggestion is confirmed, ESKIMO handles all further steps automatically. A helmet camera system can be used to automatically map the defect in the 3D model. The site management still retains full control at all times and can take corrective action if necessary.

The goals at a glance:

Automatic classification of visual defects (stains, cracks, scratches)
Automatic mapping of a defect in the plan and in the 3D model
Automatic subcontractor suggestion regarding a defect
Automatic recording and mapping of 360° image in the model for documentation purposes

Commercial usage scenario

An intelligent commercial quality assurance process is used to carry out a performance comparison based on the BIM model and the as-is conditions. Regular as-is documentation makes it possible to determine the time of installation of the individual building elements and to compare this with the planning schedule in the model. The project generates vast quantities of documents that have to be sifted, sorted and filed. ESKIMO tackles this problem and trains an artificial intelligence to classify the documents (data sheets, invoices, delivery notes/measurements) and to extract characteristics from these documents. Data sheets are assigned to the individual spaces. The as-is conditions are recorded by the AI and compared in the model. The site manager has access to all the data necessary for invoice verification (invoice, measurement and as-is conditions).

The goals at a glance:

Up-to-date space plan at all times
Automatic construction status documentation
Partially automated invoice verification

Logistics usage scenario

Using a special helmet mount, all areas of the construction site are surveyed with 360° camera technology. Images are made at regular intervals, e.g. every two days or once a week, always from the same locations. The result is a continuous documentation of the changing conditions of potential areas for installations and equipment on the construction site. These areas can be viewed on a computer in virtual walk-through inspections with the possibility of tracking the use and availability of the areas over time. The survey should cover not only areas used for site installations and equipment but also container fill levels, as the ESKIMO project will make it possible to automatically locate waste and disposal containers and to record their fill levels in order to derive and initiate the necessary logistics operations.

The goals at a glance:

Plan of potential site installations/equipment areas and their dimensions/size
Optimised navigation on the construction site
Target/actual comparison with logistics operations

All usage scenarios are founded on the same basic principle: the intelligent interpretation of the as-is conditions makes it possible to obtain and process information from imaging data. Deep neural networks, which learn from images with the help of AI algorithms, are used for machine image processing.

Putting theory to practice

What may sound a bit complicated at first becomes easier to understand when looking at a typical problem from ESKIMO project. Let’s say we want to determine our position in a space such as a room. We know which room we are in, but not exactly where. So we look for certain fixed elements that are also visible in the BIM model. To solve this task, windows, doors and edges are detected, the AI’s predictions are processed by a “normal“ algorithm and a topological comparison is made between the as-is conditions and the BIM model with the aim of determining our exact position. To test the AI, a neural network previously trained with thousands of images is shown other images that were not used during training.

The results are impressive: The AI recognises doors and windows with a probability of up to 99 %. However, because of the sunlight shining into the room, the AI also incorrectly detects two doors on the opposite wall.

There are several possible ways to deal with this problem. We could use a different architecture, adjust the training parameters (hyperparameters) or try to find better parameters for the problem.

The future is now

You can find out how we solved this problem in our next update. Because even if the ESKIMO system is still under development, it already shows that AI-assisted construction is no longer a distant dream but is becoming a reality right at our own doorstep.

Stay tuned!

Research Project ESKIMO

Project partners:

Actimage GmbH
Bauunternehmung Karl Gemünden GmbH & Co. KG
Frankfurt Economics AG
Fraunhofer IOSB
Hochschule Darmstadt
Karlsruher Institut für Technologie
Open Experience GmbH
PMG Projektraum Management GmbH
Ed. Züblin AG

Project duration:

April 2020 – March 2022

_{Photos: pickup – stock.adobe.com / baranozdemir – istockphoto.com}