Overview

Position – UX Designer, Visual Artist     Categories – Sketch, AI, Conversational Interface     Timeline – 5 Weeks

Familiar is an AI chatbot built as a supportive companion. This piece was developed as part of the Interface Cognitive A.I. Collab working with design leaders at IBM, Han-Shen Chen (now Lead Product Designer at Microsoft), and Bell Labs, Wei Wei (UX Designer). Familiar is an app to help users relieve stress and negative emotions built up throughout their day by being there to listen and talk to you. Your Familiar is a self-reflection of your own personal health, changing and growing with your own mood which it picks up through talking with you.

David Utt: UX Designer, Visual Artist     Shubhank Sahay: Animation, User Researcher     Yi Tang: System Architect

Explore the Project Deck.

Project Goals

Building on the interaction points of pet ownership and therapy. Our team found these 4 goals to be the best way we can improve a person’s health by preventing self-isolation.

  • Conversation: Familiar reaches out to you to talk when in need or when you are most free. Unlike games which use buttons to interact users take care and feed familiar through conversation.
  • Goal Building: Each familiar is unique and has a set of needs that you have to take care of: Food, Hygiene, and Mood each are satisfied by a set of user’s tasks learned through interactions and changes in their familiar’s status.
  • Engagement: Familiar is highly customizable with multiple methods of animating and demonstrating emotions/interest. Including text-based conversation familiar emotes, physically changes, and reacts.
  • Self Reflection: By talking with Familiar it reacts as a personal reflection. The goal being as your familiar improves in health our user’s social life improves as well, being more open to conversations and being around others escaping from isolation.

Target User

In our first 2 weeks, we researched cases, symptoms, and treatments of depression and self-isolation from various papers and conducted interviews. During this process, we interviewed caretakers of these individuals such as family members and friends. In addition, interviewing those who have previously suffered this condition to understand both what it was like with depression via. self-isolating one’s self and how they were able to get through it. From this data, we developed the persona of Jessica, a stressed college student, who works very hard but causing her to have few friends and no one to talk to about her day to day stresses.

Experience User Flow

Our group focused on the user to pet interaction using multiple methods of engagement including machine learning, character design, and therapy techniques. Our familiar grows with you and in turn, reflects your emotions. The diagram below showing the conversational loop that we built.

Familiar starts a conversation with you by reaching out as a notification depending on three factors: when you are most comfortable, scheduled a chat, or your pet is in need of care.
When talking to your Familiar it will ask questions about your day, tell jokes, and reference concerns based on new and previous knowledge.
Using IBM Watson Speech to Text, Text to Speech and Sentiment Analysis. Familiar builds a memory of concerns through events and emotions correlating setting moods and needs depending on what it learns through conversations with you.
Based on your input, Familiar will react with emotes, sounds, and body language to reflect your own mood. In addition, will reference previous conversations to see if noted concerns have improved or changed.
Besides talking to your own familiar, your’s will seek out and find others nearby and through friends so your can branch out to meet new people and break through self-isloation.

Design and Visual Style

To make sure that people would want to engage initially with our familiar our visual style is very bright, colorful, and playful. We tested with users of various sexes and ages to make sure that the product was still attractive to all audiences. The semi-flat form also provides us the ability to animate multiple features easily with pre-built emotional states and emotes to provide context to the conversation itself creating another level of depth to the interaction.

Machine Mechanism

To create a realistic and engaging conversation with your familiar our machine learning system is a 4 step process. First engaging the users by calling out to them at a set time or when pet needs are serious. Second, reveal user’s task via pet’s condition and emotions. Third, get user input via Watson’s speech to text analysis and respond accordingly based on text to speech API and condition matching in Waston Conversational mapping. Lastly recording data and feedback setting updated conditions for the next conversation.

This flow has multiple expansions detailed below explaining how the analysis works based on a database of set concerns and emotions related to them. Each of these concerns has related keywords of which there correlating emotional statements are run through a sentimental analysis logging concerns and reflecting back confort, encouragement, or curiosity via the visual style based on the conditions meet. Thesis concerns can then later be referred back to on another day showing remembrance and interest to help improve localized concern categories.

Digital Prototype & Presentation

As part of the final project of the IBM Cognitive AI collaboration, we presented our research and prototype to various colleagues in the field of AI interfaces and UX design. Familiar was considered extremely successful and well regarded by the group, I hope to continue this project after review with my other group members Shu and Yi expanding familiar into possibility as a physical toy-like interface and further detailing out the system’s capabilities using IBM Watson’s conversational interface and node.

Want to chat? Contact me.

This case study is over, but I’ve prepared a few more so let’s keep going.