CO-OP: mHealth Tools for Tracking Everyday Illness Experiences

The past decade has witnessed unprecedented advances in mobile sensor technologies to support continuous passive sensing as well as active sensing (in which the human initiates self-reported observations). However, these applications are designed and intended for use by a single person, who is either collecting personal data about themself or a care recipient. This research explores the design of mobile technologies to support multiple individuals who are collaboratively collecting, reconciliating, and consolidating personal data on a single person.

  • Project Date: Sep 2018 - Mar 2020
  • Affiliations:  Georgia Institute of Technology, Children's Healthcare of Atlanta, Emory University Hospital
  • Funding: NSF #1652302
  • My Role:  Study design, study protocol development, relationship building with clinical partners, data collection, data analysis, prototype design, tool development, design material development, and paper writeup
  • Collaborators:  Jungwook Park (Research Software Engineer), Udaya Lakshmi (Graduate Research Assistant), Kimberly Do (Research Assistant), Rosa Arriaga (Co-Investigator), Sampath Prahalad (Lead Rheumatologist), Thomas Olson (Lead Oncologist) and Lauren Wilcox (Principal Investigator)


During complex treatment regimens such as chemotherapy, adolescent patients must self-report their observations of symptoms and treatment side effects for their care team to inform decisions about ongoing treatment. This communication is challenging because patients, parents and clinicians have unmatched experiences, conceptions and linguistic representations of indicators of health. Yet patients' overt reliance on parental caregivers to manage and communicate their illness experience could only result in a quality of care that is based on an approximation of their actual health status.


The goal of this research is to understand how to accommodate both patient and caregiver’s observational accounts of the illness and support their collaborative construction of patient illness narratives.

  • Expert knowledge elicitation with 12 clinicians
  • Co-design with 13 families
  • Diary probe with 12 families
  • Semi-structured interviews
  • Mobile ecological momentary assessment (mEMA)

Characterizing Collaborative Patient-Generated Data

Co-design is a powerful design method that democratizes the design process by directly involving the intended users and stakeholders as co-partners to envision and conceive of a technology design through multiple design activities. Through co-design, I was able to characterize the language (visual representations) of patient illness narratives that matter for the adolescent population.

Expert Knowledge Elicitation Study

Before engaging with patients, I hypothesized that experienced, practicing clinicians would have already navigated the challenging problem of reconciling child- and caregiver-reported observations of symptoms and side effects. To get a better understanding of their perspectives about collaborative symptom tracking, I first conducted a knowledge elicitation study with 12 oncologists. I was particularly interested in the types of data they would like to have reported by the patient alone or together with their caregiver.

The survey instrument I used in the knowledge elicitation study included a range of physical, psychological and behavioral concepts. Clinicians were asked to circle T/P or both to indicate from whom they wanted the data to come.

Results of the survey show a clear divide between data clinicians prefer to have collected by patients alone (blue, 1st column) vs. one that is collected in tandem with their parental caregivers (green, 1st column).

Further interviews with oncologists and nurse practitioners suggested that clinicians were aware of the need to track both patients' and parents' observations, independently.

"I think that a teen should report lack of appetite. But the parent should help report what they're actually eating because lack of appetite is subjective [...] More like parents for checking, confirming."

Solid Tumor Oncologist

However, clinicians still relied on parental self-reports to make clinical decisions due to their lack of trust in child patients or opted for the worst-case (over-reports) to avoid a misdiagnosis. The only time clinicians specifically asked patients and their caregiver to track symptoms is when they heard discrepant reports between the two.

These findings suggested a need to design a tool to support independent self-reports of patients and their caregivers.

Storyboarding Activity

Before designing the intervention, I first needed to know the best way to support patients communicate their illness experiences. Informed by research on the positive impact of images on symptom communication, and through storyboard-based co-design with patient families, I found the right set of building blocks (Visual ODLs) to help young patients reconstruct various aspects of their daily living.

By providing a visual language, I was able to leverage teens' familiarity and favorable attitude towards using a visual conversational medium while capitalizing on their recognition over recall (thereby lessening cognitive burden on patients affected by 'brain fog').

I also found the need for technology to:

  • Scaffold the process of encoding and articulating symptomatic experiences through representation of how symptoms affect patients’ ability to engage in daily activities
  • Allow full expression of these experiences through the use of varied forms of media data representations
  • Support distinct roles which family caregivers can serve in tracking the patient experience.

Diary Probe

While I understood how patients and family caregivers wanted to utilize the language (visual representations) of illness to communicate symptoms, I needed to know how they would actually use it in their daily living in order to inform the design of a functional mobile health app.

In collaboration with a Udaya Lakshmi, I designed a diary probe packet, consisting of stickers (with visuals), diary booklets, and an instant film camera (if needed). In a 2 week diary probe deployment with 12 families, I found:

  • While patients attended to personally-felt symptoms, parents were better at capturing contextual details (e.g., location, timing) and activities around the illness experience.
  • Patients were able to develop better symptom literacy in the act of tracking (with illustrations)
  • Patients were able to coordinate the exchange of sensitive symptoms with their parents outside of the typical context of face-to-face communication.

Various sketches provided by child patient (C, in green) and parent (P, in orange) participants of the diary study.

Analysis of these sketches showed that parents (P) identified the anatomical region of the affected area whereas child patients (C) were more likely to add detailed elaborations of sensations onto the diagrams beyond indicating the physical location.

Lessons Learned

All three studies allowed me to identify three important strategies to support patients and family caregivers to record and track their daily experiences during treatment:

  1. Tailor the representation of illness experience to match the patient’s cognitive and linguistic needs
  2. Incorporate caregiver observations in ways that complement experiences that are unique to the patient.
  3. Provide rich means to elaborate on patients' illness experience in the context of everyday life.

CO-OP Prototype Design


The design of pictographic representations of activities and physical and emotional experiences evolved over the course of two years since the beginning of this project. I started from hand-drawn paper sketches, computer-generated illustrations (in collaboration with Udaya Lakshmi), and finalized the designs with pixel-perfect vector renderings. To complete the vector renderings, I hired a professional graphic designer from Upwork and continued to iterate on the designs to generate diverse and inclusive sets of iconography.

CO-OP User Interface

In collaboration with Jung Wook Park, I created CO-OP, an interactive tablet mHealth application that integrates self-reported observations with passively collected data such as tablet usage, location, and social context. The application can generate rich perspectives on the patient's illness experiences, including captured media data and collateral information about when and where side effects are occurring.

Distinct User Experience for Patient and Family Members

Early investigations (see above) showed that there is a need to support different experiences for patients and family caregivers. This informed our decision to create unique UI flows for each, where we asked slightly different questions and also tailored the text descriptions to match the current user.

UI Flow for Patient Users

UI Flow for Family Users

CO-OP Demo

CO-OP Deployment and Evaluation Plan

My research goal was to understand how families can work together to co-construct patients' illness narratives in everyday life. To do this, I employed mobile ecological momentary assessment (mEMA) methods that are geared towards achieving high ecological validity by placing the data collection activities in the hands of patients and their parental caregivers, in their natural setting.

My plan was to deploy the CO-OP app with 20 patient-parent pairs in-the-wild over a 3 week period and apply quantitative and qualitative assessments—derived from mobile sensor data, daily mEMA, surveys and interviews—to determine adherence, engagement, and patients’ self-efficacy to manage their illness. However, the project was put to rest due to an unfortunate series of circumstances (i.e., delay in IRB approval due to merging process among two hospitals, addressing critical items in IRB audit report, emergence of COVID-19). See our onboarding slide deck for more detailed information about the study protocol.

This research (if continued) will contribute computational techniques that can inform Machine Learning (ML) models to automate data triangulation between active and passively collected patient data. With this knowledge, we can predict symptom occurrence based on its immediate context, such as co-occurring activities, location, and time of day. We can also identify opportune moments to prompt the user to make an entry upon completing a routine. One envisioned use case is to mitigate the user's logging burden by automatically suggesting a set of likely symptoms at a specified time or location.

ACM Digital Library Using Diaries to Probe the Illness Experiences of Adolescent Patients and Parental Caregivers.
Matthew K. Hong, Udaya Lakshmi, Kimberly Do, Sampath Prahalad, Thomas Olson, Rosa Arriaga, and Lauren Wilcox. Proceedings of the 38th Annual ACM Conference on Human Factors in Computing Systems (CHI 2020), Honolulu, HI, USA, 2020 (24.3% acceptance rate).
ACM Digital Library Visual ODLs: Co-Designing Patient-Generated Observations of Daily Living to Support Data-Driven Conversations in Pediatric Care.
Matthew K. Hong, Udaya Lakshmi, Thomas Olson, and Lauren Wilcox. Proceedings of the 36th Annual ACM Conference on Human Factors in Computing Systems (CHI 2018), Montréal, Québec, Canada, 2018 (25% acceptance rate).
WISH Just-in-Time Design: In Situ Methods for Capturing and Articulating Adolescents’ Illness Experiences.
Matthew K. Hong, Udaya Lakshmi and Lauren Wilcox. Workshop on Interactive Systems in Healthcare (WISH 2017), Washington, DC, USA, 2017.