XR Interaction Design

⁠
`Nanome`⁠
⁠

⁠

Loading…⁠

⁠

XR Public Projects

Augmented First Responder (A1R)

⁠

2021

Platform: Magic Leap One (AR Headset)

Lead Designer

Skills used: SME Research, Concepts, Mockups, Modeling, Unity,

Spatial.io⁠

for prototyping

⁠

Winner Phase 2 NIST CHARIoT Challenge⁠

⁠

Using the latest in AR technology and 3D UI/UX research, the team designed holograms for visualizing real-time environment data to convey actionable information to first responders (e.g., firefighters, EMS, and law enforcement personnel). Our AR solution includes holograms for incident command perspective and heads-up displays (HUD) for a wildfire scenario. This video showcases the first responder on-device experience.

⁠

EMERGENCY RESPONSE FOR THE HEALTH CARE SYSTEM

INNOVATION CHALLENGE

An Augmented Reality based Communication Technology for providers internally and externally within a health care system to share information and collaborate with each other.

In the design of our proposal, we interviewed multiple subject matter experts to identify the key challenges for first responders and the results came back unanimously – clear communication of critical information.

⁠

First Responder AR HUD View concept

⁠

Incident Command Environmental Interface Concept

Thus we propose using augmented reality to improve communication and coordination of situational information between emergency team members and Emergency Healthcare Command Centers to prevent on-site confusion. Additionally, improve response times and effective management and faster decision-making to prevent and mitigate first responder and civilian injury and to save more lives.

‍AR Hardware for First Responders

First Responders will use smartglass headsets and/or mobile devices (e.g. smartphones or tablets) for receiving and sending information through voice commands or via hand gestures.

AR User Interface (Gaze, Gesture and Voice Control)

A1R Heads Up Display (HUD) User Interface (UI) comprises simple graphic elements to convey critical information to the first responders, such as vitals, navigation, placemarks for HazMat, team requests or commands, messages, warnings, and tasks.

Emergency Healthcare Command User Interface (UI)

A1R Emergency Healthcare Command System will convey critical information to the incident commander such as the location and status of deployed resources and personnel, messages and warnings. Emergency Healthcare Commander could be able to use smartglass as UI in future implementations.

AR based A1R solution offers a state-of-the art user interface (UI) to send and receive clear communication of information about the emergency response team; collect and analyze the real-time environment data; and identify and assess the resources at risk, and assist healthcare personnel in making decisions to take effective actions in chaotic situations to improve crisis outcomes and reduce the number of lives lost.

HoloTRIAGE

⁠

Winner Microsoft Reactor Hackathon⁠

⁠

2020

Platform: Hololens 2 (AR Headset)

Lead Designer, Lead Developer

Skills Utilized: Mockups, 3D Modeling, MRTK, Unity

HoloTRIAGE represents a new paradigm for first response and disaster medicine that augments and expands the current medical and healthcare training industries. Using the convergence of AI, Avatars, Multi-Haptics and XR into one integrated holistic platform. XR allows virtual victims and rescue equipment in real context. AI enables automated training of protocols and assessment of trainees. Avatars allow remote instructors to record themselves and be present in realtime. Haptics enable trainees to feel sensory and motor feedback. HoloTRIAGE reduces costs, vastly simplifies deployment, expands scale, enhances efficiency, saves time, massively portable, and democratizes access to training for professionals and non-medical communities.

⁠