Simulaid – Mixed Reality Training for critical situations
Simulaid is a research project aimed at developing a mixed reality application that supports medical students in training for various emergency scenarios. It is integrated into the existing simulation environment at Bern University Hospital and enhances it with immersive elements. On the one hand, clinical pictures are visually displayed at various stages, and on the other hand, the students' interaction and communication skills are analysed and explored using interactive elements. The aim is not only to train the interplay of perception, communication and action in extreme situations, but also to make it accessible as a basis for reflection in the learning process.
The project combines real medical simulation with digital MR enhancements, creating a hybrid training system in the research environment.
Project background
- Mixed reality (MR)-supported simulation with visual and auditory cues (e.g. redness, cyanosis, breathing sounds)
- Interactive questions during training: Training supervisors can display situational questions to test students' situational awareness
- Eye tracking: Recording of eye movements and focus of attention
- Real-time control: Escalation levels can be adjusted live via the training manager's command room
- Pilot cases: Acute coronary syndrome, anaphylaxis after insect sting
- Technical setup: Application on Microsoft HoloLens 2 (Unity-based)
- External control via a web-based user interface for adjusting escalation levels, scenarios and interaction questions
- Data collection: Storage of eye-tracking data, communication histories and responses for analysis, monitoring and further processing
- Analytical capability: Combination of eye-tracking and interaction data for precise evaluation of perception and decision-making behaviour
Project goals
- Co-creation with training management: Development in close collaboration with senior physicians at the Emergency Centre of Bern University Hospital
- Scientific evaluation: Research and evaluation by Bern University Hospital
- Immersive and intuitive learning: Development of comprehensible emergency scenarios with a focus on communication, audiovisual enhancements, systematic tracking and intuitive teaching
- Integration: Development for existing infrastructure