How do medical teams react when seconds decide between life and death? Simulaid combines real-life emergency simulation with augmented reality, eye tracking and the Situation Awareness Global Assessment Technique (SAGAT). The system reveals what would otherwise remain invisible – how well teams perceive their environment, interpret information correctly and anticipate the next steps. The aim is not only to train the interaction of perception, communication and action in extreme situations, but also to make it measurable.
The project combines a physical city model and digital extension to create a hybrid system:
Virtual auscultation via button instead of stethoscope – sounds directly in the headphones
SAGAT questions (Situation Awareness Queries) in the middle of training – scenario freezes briefly, background is blurred
Eye Tracking tracking to record eye movements and focus of attention
Eye tracking to record eye movements and focus of attention
Two pilot cases:
Acute coronary syndrome (NSTEMI) – patient with chest pressure, progressive instability
Anaphylaxis after insect bite – bicycle courier with increasing shortness of breath and allergic reaction
The AR environment runs on HoloLens 2 and is controlled via a central web interface.
A Unity-based system displays the virtual effects directly in the participants' field of vision, while a SignalR network in the background keeps the connection between the command room and the glasses synchronised.
Data on eye movements, interactions and SAGAT responses is stored in Azure CosmosDB and can be analysed for scientific evaluation.
The result is a robust platform that seamlessly combines real simulation, AR and data analysis.
Simulaid is currently in the pilot phase.
In the next steps, the system will be expanded to:
intergrate additional medical scenarios,
automate data analysis (eye tracking × response behaviour),
and display training feedback in real time.
In the long term, Simulaid is set to become a versatile tool for training, further education, research and quality assurance in the medical field.
A system that not only trains skills, but also raises awareness.
