Element assembly with HoloLens 2

Increasing efficiency in element assembly with spatial construction plans

Project Idea

schaerholzbau ag in Altbüron is a modern timber construction company that builds houses according to ecological, economic and social principles. The planning of the building projects is done completely digitally using the cadwork software. Based on these construction plans, machines and employees saw and mill individual wooden beams into the desired shape. These are assembled by hand in the production hall to form transportable elements. During the construction of the building, these elements are installed as wall, floor or roof components.

The elements are usually different in shape and vary in complexity. For this reason, employees often have to consult the printed construction plan when assembling the elements on the workbench. There is potential for optimization in this process step.

Since the planning of the buildings and the components is already done digitally and in 3D, these plans could also be displayed spatially in Mixed Reality. This idea gave rise to the ElementBau app for HoloLens 2. For the first time, the app enables a completely digital workflow from CAD planning to the assembly on the work bench.

With the use of this digital tool, a significant acceleration of the process step was measured.

Features of the Mixed Reality App

The app projects the construction plan three-dimensionally and in life-size on the workbench and supports the employees during assembly:

• The component selected by gesture or voice command is highlighted in the virtual model.
• A mini model of the element can be freely rotated and zoomed.
• The virtually displayed parts can be repositioned to examine complex interior components.
• A floating parts list displays the different parts of the element. This list is fixed in space or can be summoned with a gesture if required.
• Information is only displayed where it is useful for the employee and does not restrict the field of view while working.

Technical Background

The construction of the buildings and the planning of the individual elements is done in cadwork. A plugin created by Design-to-Production, adds a QR code to the printed plan, converts the parts into 3D models (FBX) and uploads them to an Azure Blob Storage.

Just as before, the construction plans of the elements are transmitted to a CNC-Maschine in the production hall, which automatically saws and mills the pieces into the right shape.

Employees begin work on a new element by scanning the QR code on the printed blueprint. Using the information in the printed code, the app downloads the digital assembly plan from the cloud and projects it spatially on the workbench in Mixed Reality. The natural interactions with the holographic content, rely on the functions of the MRTK library. During work, employees can also interact with the app using voice commands - this speech recognition uses Azure Cognitive Services in the background.