Mobile Conversational AI with Medical Data Intelligence

Virtual Collaborator.

Nursing Assistant


Lead Designer

Product Owners

GE Healthcare

Parsons New School

Parsons New School 



The Virtual Collaborator is a patient-centric platform for nurses and hospitalists in the ICU for monitoring sepsis prone patients and promoting rapid response team decision making. Using the Virtual Collaborator, Ezra follows a patient from admission to discharge and can extract clinical data from the electronic medical record (EMR) and lab reports. Nurses can ask for specific information or generate graphs like you would any team member.

Ezra can pre-empt a request by providing vital signs or lab results as soon as it has been recorded in the EMR. The HCP team can also request information from Ezra. Lastly, Ezra can keep you and the team aware of using pre-built and personal notifications of vital/lab abnormalities, reminders, and event callouts.

Inital Collaboration Sketches with Clinical Specialist, Designers, Developers and Product owners on userflow and navigation

Artboard 4

Concept sketches for the Virtual Collaborator

UX Solutions

Natural Collaboration

Chat elements should be visual & quick to understand. Communication with Ezra should be clear & straightforward.

Simple Navigation

Users should know how to go back to home screen or how to reach patients.

Layout Hierarchy

Screens layout should be uncluttered, easily accessible with information clearly presented

Contextual Notifications

We only notify users when necessary to limit alarm fatigue.

Promoting Mobile Collaboration

 We focus on two major pain points: data access and communication, where clinical staff can have easy access to actionable information. Using Natual Language Processing, we made Ezra, a chatbot who can quickly get hospital data as well as can generate unique data visualizations. We focused on releasing staff from cumbersome computer stations they're forced to return to for any actionable data.

Virtual Collaborator 1

Left to Right: Patient List, Patient Chat Channel, Requesting Custom Graphs, Private Chat

Actionable Information on Hand

One of the many challenges we faced on the project was balancing introducing new workflows with the demanding environment in the ICU. Virtual Collaborator combines traditional data dialogue methods while identifying subtle ways to add new techniques. This helps in two ways by also hinting at the functionality and knowledge base of Ezra for the chat.

Each page provides in-depth clinical information from vitals, labs, meds, and order. We promote actionable information first without losing detail using iconography, color, and passive interactions with data points.

"Up to 49% of clinician time is spent interacting with the EMR."

- Zanotti-Cavazzoni, S.l

Left to Right: Vitals Expanded, Lab Results, Medication Delivery, Patient Summary

Awareness w/out disruption

With Sepsis, response time is a critical factor, so keeping them alert with information while balancing the fact nurses overwhelmed with notification fatigue. We faced this by allowing nurses to develop their own and standardizing the priority.

From the patient list, you can see how active a patient channel is. Nurses can snooze notifications if busy highly requested in the rapid space. 

Clinicians and care teams would be able to set there own notifications related to specific changes to clinical status. These could be based on published early warning scores or your facility’s protocol.

Building our Design System

Unique for our application. I made our design system to promote the feeling of a clean environment, flexible. The design system was made into three parts: the visual design, which focuses on building a design language promoting the GE message of constancy, and clarity.

Second is the data visualization design system with a focus on actionable information first. Colors are consistent with hospital standards to reduce cognitive load, and allowing for touch navigation for our graphs will promote readability even on data-heavy visualizations.

Lastly is the NLP (Natural Language Processing). Instead of searching the EMR, our chatbot Ezra (working name) brings the information to you. Our responses are designed in both visuals and language to promote actionable information. Reduce cognitive load. At the same time, it is an adaptive structure to handle the 3,000 unique requests and correlating responses.

Artboard 9

Design System vr. 2.0 of the Virtual Collaborator

Project Hindsight

We faced many challenges in this project, working with the constraints of regulations on relatively new technologies (NLP, Chatbots, Messaging) and the difficulties of introducing users to them. Through the design process, we created many creative solutions that are not just innovative but valuable. We believe these methods will last for many years, balancing the delicate workflows our users have built. Our challenge was making a product that is official, secure, and improves upon the workflows.

With our app's new workflow, including new technologies for most nurses, user testing was a priority. Over six months, we ran five in-person interview sessions with over 100 medical professionals. Here we would gauge insights into feature desirability and usability. These sessions were an essential way to expose crucial unknowns and encourage new approaches. It also reconfirmed a great excitement in our approach with many interviewee's wanting to use the prototype in their teams.

The Virtual Collaborator was shown at HIMMS 2019 with much interest and will be used for testing in hospitals over the next year. Product Site