A Gamified Smartphone App to Support Engagement in Care and Medication Adherence for HIV-Positive Young Men Who Have Sex With Men (AllyQuest): Development and Pilot Study

Introduction

HIV Infection Among Young Men Who Have Sex With Men in the United States

In the United States, men who have sex with men (MSM) experience the highest rates of new HIV diagnoses, with young MSM (YMSM) and MSM of color continuing to be significantly impacted [1]. Although the number of diagnoses of HIV infection among MSM remained stable, from 2010 to 2014, the number of diagnoses among MSM in the age range of 13 to 24 years increased by 6% [2]. National HIV Behavioral Surveillance data on MSM from 20 cities found that among black MSM in the age range of 18 to 24 years tested in 2014, 26% were HIV positive, compared with 3% of white MSM. This disparity in HIV prevalence between black and white MSM increased from 2008 to 2014, especially among YMSM [3]. Young Hispanic or Latino MSM had a 20% increase in numbers of diagnoses of HIV infection from 2010-2014 [2].

HIV also disproportionately impacts YMSM across the HIV care continuum, with disparities in linkage, retention, antiretroviral therapy (ART) adherence, and viral suppression [4,5]. YMSM have documented low levels of ART adherence, impeding their likelihood of achieving viral suppression [6]. One study among 13 sites in the US Adolescent Trials Network found only 7% of diagnosed youth (81% male, 72% black, and 70% gay or bisexual) achieved viral suppression [5], which was substantially lower than the estimated 50% viral suppression for all age groups [4,5]. Alarmingly, a recent study of 991 HIV-infected YMSM (aged 15-26 years) found that 69.4% had a detectable viral load; two-thirds of whom (n=458) reported recent condomless anal sex [7].

Barriers to Adherence Among Youth

Individual-level stated barriers to adherence among youth include forgetting, not feeling like taking medication, and not wanting to be reminded of HIV [8]. Additional factors contributing to suboptimal adherence and viral suppression among youth, including YMSM, include low medication adherence self-efficacy [9], psychological distress (depression and anxiety) [10-13], substance use (alcohol, marijuana, and other drugs) [10,13,14], structural barriers (eg, homelessness and health insurance) [15], low social support [11,16], and HIV-related stigma [10,11,17]. Multiple factors are often present, and these syndemics are associated with greater likelihood of nonadherence and detectable viral load in a dose-response nature [10,18] that also shows disproportionate impact on minority MSM [18].

There are only a few published interventions focused on supporting ART adherence among YMSM [19,20]. A 2017 systematic review of interventions along the HIV care continuum identified 117 medication adherence interventions. Only 9 (9/117, 8%) focused on adolescents or youth, and only 2 were specifically designed for MSM [19]. A 2014 review of adherence interventions meeting the Centers for Disease Control and Prevention’s criteria for evidence-based interventions found none that exclusively focused on MSM or youth [21]. Due to the often marginalized and stigmatized status that many YMSM living with HIV endure, there is a need for the development of tailored interventions that account for the unique and challenging circumstances they face.

Tailored interventions have been found to produce higher rates of behavior change and maintenance than nontailored programs in a variety of health domains, including HIV [22-24]. Technology-based interventions—particularly mobile health (mHealth) platforms—can provide tailored adherence interventions and allow YMSM to engage and connect with others [25-27]. mHealth tools also offer the capacity to design and deliver tailored content that best meets the specific HIV management challenges faced by YMSM and each individual end user [25]. The fidelity to intervention delivery provided by mHealth and the market saturation of mobile technology ownership across socioeconomic strata [28,30] also provides a high-impact platform that can be taken to scale across and beyond the United States. Daily mobile phone–based contact is acceptable to youth living with HIV and is associated with improved adherence [29]. In this paper, we describe the development, usability evaluation, and subsequent pilot testing of AllyQuest, a tailored mobile phone app to increase engagement in HIV care, ART adherence, and social support.

Methods

Intervention Development (September 2015-May 2016)

AllyQuest is a novel, theoretically-based, mobile phone app intervention designed to improve engagement in care and ART adherence and social support among HIV-positive YMSM (target age: 16-24 years). AllyQuest development was guided by evidence-based risk reduction and medication adherence interventions [30-32]; health behavior change theories, including social cognitive theory (SCT) [33,34]; narrative communication (eg, storytelling) [35-38]; and the principles of persuasive technology [39]. AllyQuest addresses key principles of SCT, including (1) Observational learning by participating in daily activities, (2) Modeling and vicarious experiences (observing and participating in daily discussions, exploration of narrative “choose-your-own-adventure” stories), (3) Self-efficacy and verbal persuasion from expert sources (multimedia knowledge center and tailored messages), and (4) Reinforcements (virtual rewards and achievements) [33,34-40]. Narrative storytelling relies on the modeling of behaviors by similar others and has been shown to foster self-reflection and influence disease self-management [41-43]. The Fogg Behavior Model (FBM) [39] of persuasive technology informed the development of Ayogo’s Empower Platform, the operating system on which AllyQuest was developed. According to the FBM, the principal factors to promote behavior change using technology include triggers, ability, and motivation. The design of AllyQuest enhances motivation and skills and provides triggers to encourage positive behaviors. App notifications are triggers for healthy behaviors. Regular behavioral self-report prompts serve as additional triggers and help participants establish healthy habits. Ability is increased through knowledge and by identifying small steps toward target behavioral goals (eg, understanding side effects and knowing how to fill a prescription). Participants also get tips from others who are dealing with similar issues and through narrative stories within the app that reinforce the consequences of healthy and unhealthy behaviors. App motivators include social support, rewards, goal setting, and achievements.

Concepts for AllyQuest design and overall “look and feel” were informed by research we conducted with HIV-positive YMSM to understand technology utilization, the barriers and facilitators to ART adherence, and the use of an app to assist with adherence [26,44,45]. Gamification elements incorporated into AllyQuest included the ability to “level up,” earn and redeem in-app virtual currency, and the ability to unlock app features. Throughout development, we worked closely with two YMSM youth advisory boards, composed of eight HIV-positive YMSM, located in Durham, North Carolina and Chicago, Illinois. Youth advisory board activities included 13 in-person sessions and seven rounds of Web-based surveys.

Usability Testing (June 2016-August 2016)

Usability testing was conducted according to established usability guidelines [46] one-on-one with eight (nonyouth advisory board) HIV-positive YMSM aged 16 to 24 years. Participants were guided through app installation on their personal phones. They were asked to explore the app and complete specified tasks within the app without study staff assistance. While exploring the app, participants were asked to “think aloud” and provide a running commentary of their thought processes while performing the tasks [47]. The concurrent think aloud method was chosen to elicit real-time feedback and emotional responses [46,48,49]. Participants then received a guided tour of all app features. Participants were asked about their initial impressions of the app and completed a posttest survey to assess user experience. Participants were then asked to use the app daily for 1 week to assess ongoing functionality, monitor for any technical issues, ensure content comprehension, evaluate intervention features, and describe overall impressions of app relevance and appeal. To facilitate social connectivity, youth advisory board members also used the app during the usability testing period. A semistructured phone interview was conducted at the end of the 1-week testing period.

Intervention Refinement (August 2016-October 2016)

The research team collated all participant feedback into a usability report that was presented to the technological partner. Usability testing revealed several addressable technical bugs and user experience issues that were resolved. Usability participants also provided specific recommendations for content edits and expansion. The research team addressed as many of these suggestions as possible before pilot launch, and the remaining suggestions were prioritized for the next iteration of development.

Pilot Evaluation (October 2016-January 2017)

A 4-week pilot trial was then conducted with 20 HIV-positive YMSM to evaluate intervention feasibility and acceptability. Participants were recruited from a clinic in Chicago that primarily serves impoverished communities. Participants completed a pretest survey and then staff assisted with app download to participants’ phones. After 4 weeks of use, participants completed an online posttest survey and a phone-based qualitative interview.

Pilot Trial Measures

Sociodemographic Items

Sociodemographic items assessed age, race or ethnicity, education, income, homelessness, health insurance, and sexual identity.

Feasibility

Usage data was captured through in-app analytics and included number of times per day or week participants accessed the app and average time spent using app, daily number of activities completed and daily discussion questions answered, content of posts, and number of health-focused daily quests completed.

Acceptability

System Usability Scale (SUS) [50] is a 10-item, 5-point Likert scale of subjective assessments of usability. The SUS provides a global measure of system satisfaction and subscales of usability and learnability. For this trial, 9 of the 10 items were used, as one question was deemed duplicative.

Client Satisfaction Questionnaire-8 (CSQ-8) was used to assess global intervention satisfaction. The CSQ-8 has eight items (quality of app, kind of service received from app, app met needs, recommend app to a friend, amount of help received from app, effectiveness of app for dealing with health problem, overall satisfaction, and willingness to use the app again). These domains are assessed on a 4-point response scale with individually specified anchors. Participant responses are scored from 1 to 4, and thus, the possible total scores range from 8 to 32. Higher scores indicate greater satisfaction. The CSQ-8 has demonstrated high internal consistency across a large number of studies and has been used to evaluate technology-based interventions [51-55].

HIV Self-Management

We developed four questions to assess domains of HIV-specific self-management after versus before the 1-month pilot trial. These included feeling connected to others with HIV, knowledge about HIV, ability to effectively manage HIV, and ability to reliably take ART. All outcomes were measured on a 5-point Likert scale, ranging from 1=“Much less” to 5=“Much more.”

Pilot Trial Analysis

Frequencies and measures of central tendency (means, medians, and SDs) were calculated to describe the sample in terms of sociodemographics and acceptability, feasibility, and 4-week retention. Chi-square tests, t tests, and correlations were examined to provide preliminary effect estimates. Exact statistical tests were used where possible to account for the small sample size. The Spearman rank order correlation coefficient was calculated to assess the magnitude of association between app usage and HIV self-management outcomes, where a value of 0 indicates no correlation, and higher values indicate stronger association between variables [56]. The Spearman rank order correlation is a nonparametric alternative to the Pearson correlation [57] based on ranks instead of absolute values and is less sensitive to outliers and nonnormal variable distributions. Statistical analyses were conducted using SAS (SAS Institute Inc) software version 9.4 for Windows.

Results

Intervention Development

Youth advisory board members provided feedback on (1) Intervention structure and format (eg, organization of the intervention, appropriateness and appeal of language and images, and ease of navigation); (2) Intervention content and activities (eg, comprehension, acceptability, and relevance); and (3) Overall app impressions (eg, utility, interest, and enjoyment). Youth advisory board members’ feedback informed content development in all app components (Table 1).

Usability Testing

Usability testing revealed several addressable technical bugs and user experience issues. Given the time frame and cost considerations with app development, we prioritized bug fixes and user experience issues. Although users had other suggestions for app improvement (eg, making the daily discussion more like Facebook and using different imagery in the profile area), these changes were deemed by the research team and technical partners to not likely impact pilot trial outcomes in a meaningful way and given development cost, were prioritized for a later study.

App content was also reviewed by usability participants who provided specific recommendations for areas of the app that could be expanded (brain builders and daily discussion posts) and edited (collection stories and daily quests). The research team worked to address as many of these suggestions as possible before the pilot launch. Figures 1-6 provides screenshots of the fully developed AllyQuest app.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

Pilot

The mean age was 21.8 years (range: 19-24), 95% (19/20) were nonwhite, 95% (19/20) identified as gay, 25% (5/20) had not completed high school, 65% (13/20) were currently employed, and 25% (5/20) reported homelessness in the past 6 months (Table 2). Most (16/20, 80%) participants had been diagnosed in the past year, all reported being engaged in care, and 95% (19/20) were currently prescribed ART. One-month retention was 85%, (17/20) though all pilot participants engaged in app use during the trial.

App Feasibility

The mean total time of app use was 158.4 min (SD 114.1), and range was 13 to 441 min. There was a mean of 21.2 days of use with a mean of 19.4 days of logging medication (Figure 7). App usage declined over the course of the trial, with a mean of 4.3, 3.4, 3.0, and 2.8 days of usage during weeks 1, 2, 3 and 4, respectively. Although participants were told that they only needed to use the app for 4 weeks, their access to AllyQuest was not discontinued until the final participant completed the trial. A total of 14 participants continued to use the app after their 4-week pilot trial period ended. There was a total of 17 knowledge center articles available during the pilot. Participants read a mean of 8.3 articles (range: 0-17). A total of 45 daily discussion questions were developed by the study team that appeared to participants on consecutive days during the pilot trial (eg, a participant enrolled on October 1 would see daily discussion topic #1, whereas a participant enrolling on October 7 would see would see daily discussion topic #7). If a participant did not log on, then they would not see the daily discussion topic that day but could navigate back to those conversations to comment. There were 222 posts to the daily discussion social wall, with a median of 5 posts (range: 1-11) for each daily discussion question. Most users (16/20) posted at least once during the 4-week pilot (median: 5.5 posts/person, range: 0-41 posts). The questions that received the most posts included “What is one goal you have for your health?” (11 posts), “How do you start the safe sex conversation?” (8 posts); “How do you deal with people who react badly to your status?” (7 posts), “How did you incorporate taking meds into your routine?” (7 posts), “What qualities do you appreciate in a healthcare provider?” (7 posts,) and “How has HIV changed your romantic life?” (7 posts).

‎View this figure

App Acceptability

Acceptability ratings were high. Overall, participants found the app easy to use and navigate, not intrusive, and few reported technical issues (Table 3). The mean score on the CSQ-8 was 27.8 (SD 5.9). Most rated the quality of the app as excellent (n=10) or good (n=7), and overall, 15/17 were satisfied with the app. Overall, 16/17 participants felt they were getting the kind of service they wanted from the app, 14/17 felt that the app met most or almost all of their needs, 15/17 reported being mostly or very satisfied with the amount of help they received from the app, and 15/17 felt the app helped them deal more effectively with their HIV. Most (16/17) would recommend the app to a friend if they were in need of similar help, and 16/17 would use the app again.

Qualitative exit interviews (n=17) identified areas for app improvement, including the need for additional tailoring and personalization. Overall, AllyQuest met the needs and expectations of its users, as expressed by some users:

But, you know, it did help me to overall accept that I have to take this prescription because it’s only going to help me in the long run. So it was good as far as helping me remember to take the medicine.
[Participant 1004]

Being that I’m not much of a public speaker, in a sense, or an extroverted personality, the app kinda allowed me to, you know, bolster my genuine personal personality in a discrete manner, it was enjoyable.
[Participant 1008]

Next I know I was just on this thing all day. My friends are like, “You’re always on your phone.” I’m like, “Oh yeah, it’s this app. I really can’t talk about it’s my personal life.” They’re like, “Oh, okay.” So I’m just always on my phone.
[Participant 1007]

The daily discussion and medication tracker were users’ favorite features. The discussions gave users a safe space to give and receive advice and made them feel less alone:

When I would read other people’s comments on the little discussion panel thing—I would feel like I wasn’t so alone...it made me feel like I was a part of a community that understood how hard it is dealing with something like that, because HIV is, like—it’s something that’s serious, and I always thought it was a death sentence because when I first found out I had it, I literally thought I was gonna die.
[Participant 1020]

You’re talking to profiles and it’s really, really cool cause I get a lot of feedback that I would need in my life because I don’t talk about it with a lot of people. So this is a way that I get to keep it private and to myself and also get help. So it’s really been amazing.
[Participant 1007]

The medication tracker helped users establish a medication strategy and normalize taking medications daily, as described by one user:

I’ve downloaded other apps where like a medicine tracker...it was just a very basic thing. And also I’ve had discussion forums on my phone where it just dove clean into something that was, like, I’m not gonna talk about that just yet. So I think the app was, like, right there in the middle. It offered the avenue to go deeper into the conversation and it offered just a safe space at the same time.
[Participant 1008]

Overall, users liked and trusted the health information on the app. Overall, they felt that the tone and content in the narratives was important and relevant. One participant described how these stories influenced his relationship with his partner:

Interviewer: How did the stories motivate you in the app?

Participant: It kinda motivated me to be more open with my partner...I would keep a lotta things from them. Like, I would go through stressful times and I wouldn’t tell them...but after seeing how a lotta that stuff played out, like in the [app] stories, I try to start opening up more, and I think we’re gonna stay together for a long time.
[Participant 1002]

Participants did comment that they wanted more information regarding sexual health, relationships, mental health, and nutrition.

HIV Self-Management

Higher levels of app usage were positively correlated with HIV self-management outcomes, and there was a statistically significant (P<.05) positive association between the number of days logged into the app and knowledge and confidence in the ability to reliably take HIV medication (Table 4). Although statistical power was limited because of the small sample size, results are promising for a full intervention trial.

Discussion

Principal Findings

In this paper, we describe the development of a novel, theory-based ART adherence app for YMSM that showed strong acceptability, feasibility, and preliminary impact on HIV self-management outcomes. Prior work has established that technology-based interventions—particularly mHealth platforms—can provide tailored adherence interventions and allow YMSM to engage and connect with others [25-27]. Furthermore, daily mobile phone–based contact is acceptable to youth living with HIV and is associated with improved adherence [29]. Although several mHealth adherence studies for MSM are underway or under development [65,66], to our knowledge, there are no currently available interventions for YMSM that include the range of features incorporated in AllyQuest or are poised to utilize technology in a similar highly innovative and engaging way.

HIV-positive YMSM were instrumental in informing all stages of AllyQuest development. Prior research with HIV-positive YMSM informed the inclusion of essential elements in the initial AllyQuest prototype, including information on both HIV-related issues and general health and wellness [64], anonymity and privacy features [58,59,64], medication reminders and tailored adherence strategies [59,63,67-69], and provision of social support and a connection with others [61,70]. Youth advisory board and usability participants provided further feedback that was incorporated in an agile way to allow for ongoing modifications and enhancements. The result of this process is a user-centered, highly engaging, multicomponent care support app.

AllyQuest accommodates different learning styles, motivations, and needs among YMSM through features including app-guided tailoring of content, personalized messages, and inclusion of game-based elements. Gamification uses game design components in nontraditional gaming contexts, thus providing opportunities for a greater level of engagement of participants in online behavioral interventions [60,71]. Interventions can utilize gamification to deliver highly engaging content and promote participant interactions both within and outside the intervention, thus increasing the potential for health behavior change. To optimize intervention engagement and impact, AllyQuest integrates health-related challenges, rewards, social connectivity, and “unlocking” character-driven narratives [60,71].

AllyQuest was built on an established platform developed by our technology partner, Ayogo. In a world of ever-shrinking resources, developing apps for ART adherence de novo may not take advantage of prior work done to improve treatment adherence in other disease states, thus failing to capitalize on lessons learned and software assets developed. Furthermore, collaborating with technology partners with established products allows for some degree of cost-sharing, ensuring that the app will be updated as needed and can be scaled up if proven effective. However, this process still requires active engagement with the target population to ensure adaptations are both developmentally and culturally congruent with their needs. Allowing sufficient time for iterative adjustments to the intervention is critical.

Understanding the full spectrum of app feasibility and acceptability before large-scale efficacy testing is essential. This entails measuring and subsequently aligning multiple streams of both in app (metrics of use and participant postings) and out of app (pre- and posttest surveys and qualitative exit interviews) data. Ensuring the app has a robust back-end data system to capture all app analytics is a crucial piece that should be prioritized early in development. In this study, feasibility and acceptability metrics aligned with our primary HIV self-management outcomes, increasing the likelihood that AllyQuest t could in fact impact long-term HIV ART adherence among HIV-positive YMSM. However, identifying additional strategies to ensure consistent and sustained app engagement should be considered. Integration of features that provide users who may not respond to technology-only solutions (eg, two-way text messaging or video counseling sessions with an adherence counselor) or stepped transition to in-person interventions should be considered in future studies.

Limitations

This study is not without limitations. Data on HIV self-management outcomes were self-reported and were only measured among participants who received the intervention. Due to the limited scope and length of this pilot study, enrolling a control group would not have been feasible, and measuring changes in biologic outcomes would not have been clinically meaningful. Statistical tests should be interpreted with caution, given the sample size. Findings may not be generalizable to YMSM from other sociodemographic backgrounds or geographic locations. Although we attempted to enroll diverse youth, the majority of participants were YMSM of color. Given the disproportionate impact of HIV among YMSM of color in the United States, these youth represent the population in highest need of interventions. Finally, we had 3 participants who did not complete their follow-up survey or interview, though all 3 participants logged on to the app and used it during the pilot study. Additional engagement and retention strategies will be important particularly when evaluating the impact of the app on out-of-care youth.

Conclusions

This small pilot trial confirmed that an interactive app is feasible and acceptable to YMSM as a tool to address ART adherence. Future work should build on the promising data from this trial to test AllyQuest in a larger, diverse sample to assess intervention efficacy for improving ART adherence and increasing sustained viral suppression. If a highly scalable technology such as AllyQuest could ultimately demonstrate effectiveness in implementation studies, it would be a powerful tool for realizing the individual and public health benefits of biomedical advances in prevention and care therapies. Furthermore, placing these tailored technologies in the hands of YMSM offers an approach to HIV self-management that may empower youth as they establish optimal HIV care engagement habits for the future.