The ESKIMO research project addresses specific AI-related digitalisation hurdles in the construction industry with the cross-disciplinary networking of eleven partners from research and science, the IT industry and construction. As part of the project, ZÜBLIN has made two construction projects in Germany available for the collection of training data. Active support in the form of know-how and experience also comes from the digitalisation professionals at STRABAG Innovation & Digitalisation.
The project is being funded by the German Ministry of Education and Research. It was officially launched on 1 April 2020 and is expected to end in March 2022 after a period of 24 months. The project’s focus lies on the use of AI in construction with the aim of significantly boosting efficiency in construction management. To this end, three usage scenarios are of particular interest:
Technical usage scenario
Automated defect documentation can significantly speed up site inspections and the process of acceptance while helping to improve the quality of the work. By simply pointing a smartphone or tablet with the ESKIMO solution installed at a building defect, an automatic suggestion appears as to which kind of defect it could be. If the suggestion is confirmed, ESKIMO handles all further steps automatically. A helmet camera system can be used to automatically map the defect in the 3D model. The site management still retains full control at all times and can take corrective action if necessary.
The goals at a glance:
- Automatic classification of visual defects (stains, cracks, scratches)
- Automatic mapping of a defect in the plan and in the 3D model
- Automatic subcontractor suggestion regarding a defect
- Automatic recording and mapping of 360° image in the model for documentation purposes
Commercial usage scenario
An intelligent commercial quality assurance process is used to carry out a performance comparison based on the BIM model and the as-is conditions. Regular as-is documentation makes it possible to determine the time of installation of the individual building elements and to compare this with the planning schedule in the model. The project generates vast quantities of documents that have to be sifted, sorted and filed. ESKIMO tackles this problem and trains an artificial intelligence to classify the documents (data sheets, invoices, delivery notes/measurements) and to extract characteristics from these documents. Data sheets are assigned to the individual spaces. The as-is conditions are recorded by the AI and compared in the model. The site manager has access to all the data necessary for invoice verification (invoice, measurement and as-is conditions).
The goals at a glance:
- Up-to-date space plan at all times
- Automatic construction status documentation
- Partially automated invoice verification
Logistics usage scenario
Using a special helmet mount, all areas of the construction site are surveyed with 360° camera technology. Images are made at regular intervals, e.g. every two days or once a week, always from the same locations. The result is a continuous documentation of the changing conditions of potential areas for installations and equipment on the construction site. These areas can be viewed on a computer in virtual walk-through inspections with the possibility of tracking the use and availability of the areas over time. The survey should cover not only areas used for site installations and equipment but also container fill levels, as the ESKIMO project will make it possible to automatically locate waste and disposal containers and to record their fill levels in order to derive and initiate the necessary logistics operations.
The goals at a glance:
- Plan of potential site installations/equipment areas and their dimensions/size
- Optimised navigation on the construction site
- Target/actual comparison with logistics operations
All usage scenarios are founded on the same basic principle: the intelligent interpretation of the as-is conditions makes it possible to obtain and process information from imaging data. Deep neural networks, which learn from images with the help of AI algorithms, are used for machine image processing.
Putting theory to practice
What may sound a bit complicated at first becomes easier to understand when looking at a typical problem from ESKIMO project. Let’s say we want to determine our position in a space such as a room. We know which room we are in, but not exactly where. So we look for certain fixed elements that are also visible in the BIM model. To solve this task, windows, doors and edges are detected, the AI’s predictions are processed by a “normal“ algorithm and a topological comparison is made between the as-is conditions and the BIM model with the aim of determining our exact position. To test the AI, a neural network previously trained with thousands of images is shown other images that were not used during training.
The results are impressive: The AI recognises doors and windows with a probability of up to 99 %. However, because of the sunlight shining into the room, the AI also incorrectly detects two doors on the opposite wall.
There are several possible ways to deal with this problem. We could use a different architecture, adjust the training parameters (hyperparameters) or try to find better parameters for the problem.
The future is now
You can find out how we solved this problem in our next update. Because even if the ESKIMO system is still under development, it already shows that AI-assisted construction is no longer a distant dream but is becoming a reality right at our own doorstep.
Research Project ESKIMO
- Actimage GmbH
- Bauunternehmung Karl Gemünden GmbH & Co. KG
- Frankfurt Economics AG
- Fraunhofer IOSB
- Hochschule Darmstadt
- Karlsruher Institut für Technologie
- Open Experience GmbH
- PMG Projektraum Management GmbH
- Ed. Züblin AG
April 2020 – March 2022
Photos: pickup – stock.adobe.com / baranozdemir – istockphoto.com