We conducted a systematic review of quantitative evidence designed to assess eHealth literacy as a study variable in PLWH. Owing to the heterogeneity relative to study designs and statistical analysis approaches among the included studies, we synthesized the study findings rather than conducting a meta-analysis.

Studies were screened to assess their relevance for our review. Specifically, the following inclusion criteria were used papers that used a quantitative study design (including descriptive, correlational, quasi-experimental, or experimental); papers including eHealth literacy as a study variable; and papers including participants with HIV or AIDS. Our initial search was not limited by the age of study participants or sex to maximize the breadth of the study findings. In addition, we included any study that reported quantitative findings relevant to the review question. Studies from around the globe were included, as were studies conducted in various settings, including community or health system settings.

Notably, only studies written in English were included. Studies were excluded if full-texts were unavailable (ie, conference abstracts), they were not quantitative designs, or they reported protocol only with no measured outcomes.

In consultation with a medical librarian, peer-reviewed journal papers were searched systematically in PubMed, EMBASE, CINAHL, Scopus, Web of Science, and Cochrane databases using variations of MeSH terms—methodological interest (ie, measurement of eHealth literacy as a study variable), population of interest (ie, PLWH), and study design of interest (ie, quantitative) to identify relevant papers published in English before April 27, 2017. In addition, a manual search of reference lists in selected papers was completed. Multimedia Appendix 1 provides a full search strategy for the database searches. Papers and abstracts were excluded if they did not address the population, design, or variable of interest.

At the conclusion of the study selection process, 1 reviewing author extracted data from the studies using a standard template. The initial data extraction captured both the study characteristics (eg, setting, participants, type of study design, and eHealth literacy measure) and key findings from each study. In addition, other team members reviewed the studies and extracted data relating to key findings. Extracted findings were compared and discussed until all discrepancies were resolved.

We assessed the rigor of the underlying evidence base for the review by developing an overview of key methodological characteristics, including the study design, sample size and strategy, study setting, and year of publication. No studies were excluded on the basis of the quality assessment. Rather, the quality assessment process was conducted independently by 2 raters using the Joanna Briggs Institute quality appraisal tools based specifically on study designs, randomized controlled trials (RCTs) [22], quasi-experimental [22], and cross-sectional [23] studies to identify strengths and weaknesses in study methodologies and guide the interpretation and assessment of study findings.

Figure 1 presents a detailed outline of the paper selection process. Our initial database search in April 2017 resulted in 116 citations. After removing duplicates, 56 titles with abstracts were reviewed independently for relevance by 2 authors (among HH, LES, and SG). The third author resolved conflicts in the inclusion of papers. Overall, 26 papers passed on to the next full-text review process. Of 26 full-text papers that were reviewed independently by 2 authors, 7 were deemed eligible. Reasons for exclusion included study design not quantitative (n=6), patient population not PLWH (n=5), duplicated paper (n=2), eHealth literacy not measured (n=2), full paper not found (n=2), and wrong format of paper (ie, not a journal paper; n=2).

Overall, the studies appraised in this review achieved, at least, the assessment criteria, but the quality varied across individual studies. Although 1 RCT scored 8 of 13 [24] and 1 of 3 quasi-experimental studies scored 6 of 9 [25], they exhibited strengthened validity of causal inferences by comparing the control and intervention groups. In addition, 2 quasi-experimental studies scored 6 of 9 [26,27] and lacked a comparison group to determine pre-post intervention effects. One cross-sectional study earned a perfect score of 8 of 8 [28]; the remaining 2 earned 4 and 6, respectively [29,30]; potential confounding factors were not identified in these 2 studies. In addition, 2 of 7 studies did not use a validated standard measure of eHealth literacy but collected participants’ basic literacy skills [24,29].

Furthermore, an interrater agreement rate was calculated [31]. The resulting statistic indicated substantial agreement (average interrater agreement rate, 69%) [32]. For items where discrepancies occurred between raters, we resolved them by interrater discussion. Table 1 shows consensual scores of the quality assessment.

Tables 2 and 3 summarize the main characteristics of studies included in this review. All 7 included studies were published from 2010 to 2016. Of these, 4 studies were conducted in the United States [25-28], 2 in Uganda [24,29], and 1 in Peru [30]. Various study designs used were cross-sectional [28-30], quasi-experimental [25-27], and RCT [24]. Of note, 2 studies identified a theoretical or conceptual framework used in their research [24,26].

Study participants were recruited from a variety of settings as follows: community-based HIV/AIDS organizations [26,28-30], HIV hospital settings [24,27], and both community-based and hospital settings [25]. Overall, HIV-infected adults aged >18 years were included; 1 study included only people who had advanced immunosuppression (CD4⁺ [cluster of differentiation 4] T-cell count <350 cells/mm³ and taking ARTs for, at least, 4 years) [29], 1 included women only [28], and 1 involved men who have sex with men and transgender women [30]. The sample sizes ranged from 18 to 895.

Among studies that included women, most had a majority of female participants (56%-100%) but 2 [25,26] included only 9% and 29% of females in their study samples, respectively. Studies in the United States tended to include a large proportion of African American or black (57%-63%) participants [25-28] in which more than half (54.3%) of the study sample was Caucasian. All but 1 study [30] reported low educational levels with 37%-65% of participants having less than high school education. The baseline access to mobile phones, computers, and the internet was fairly high among participants in the United States, Uganda, and Peru. In the United States, 87.3%-88.9% used a smartphone, [25,28], 58.7%-88.9% used a home computer or tablet [25,28], 72.2% had regular access to the internet [27], and 66.7% used the internet daily [25]. Similarly, in Uganda, 81.8%-82.8% of study participants owned a mobile phone [24,29]. Krishnan et al reported that 59.6% of participants in Peru had access to a standard cell phone, 30.1% had access to a smartphone, 37.3% used landlines, and 35.4% accessed a laptop or desktop computer [30].

In this review, 5 of 7 studies defined eHealth literacy. Most studies [25-28] defined eHealth literacy as the capacity to find, process, understand, and apply health information to make appropriate health decisions. Blackstock et al [28] specified that this information must come from an electronic source. Kim et al [29] simply defined health literacy as the ability to read and write.

In addition, 3 studies conducted in the United States [25,27,28] measured eHealth literacy using the eHealth Literacy Scale (eHEALS), a self-evaluation tool comprising 8 items with a 5-point Likert scale. eHEALS measures the participants’ level of knowledge, comfort, and skills in utilizing the internet or electronic health information to solve health problems [33]. In addition to assessing the ability to utilize internet-based health information using eHEALS, Woods et al [25] determined participants’ general literacy, numeracy levels, and HIV-associated knowledge using a battery, including the Test of Online Pharmacy Skills (TOPS), Test of Online Health Records Navigation (TOHRN), Rapid Estimate of Adult Literacy in Medicine), HIV Knowledge 18, Expanded Numeracy Scale, Short Assessment of Health Literacy, Test of Functional Health Literacy in Adults (TOFHLA) reading comprehension, and Newest Vital Sign.

Moreover, Ownby et al [26] used the full-length version of TOFHLA [34] to measure basic reading and numeracy abilities to understand the verbal and written information commonly used in actual health care settings. Krishnan et al [30] used a short version of TOFHLA [35] in Spanish for screening patient literacy levels in health care settings in Peru. In Uganda, Siedner et al [24] and Kim et al [29] evaluated the feasibility and effect of a mobile phone-based short message service (SMS) text message intervention on the adherence to HIV treatment. eHealth literacy was assessed by study-tailored questions by asking participants to read a full sentence in their local language at enrollment; for example, “Are you able to read and/or write?” along with mobile phone availability [24,29].

Overall, varying scales with differing scoring systems were used to determine the level of eHealth literacy. High eHealth literacy scores among PLWH ranged from 52.4% to 87% in study samples with the majority of studies finding high eHealth literacy among 65%-80% of participants. Such a wide variance arose because high literacy was defined differently in each study, ranging from the ability to read a complete sentence [24] to a TOFHLA score >75 [26], and an eHEALS score greater than the median [28]. Kim et al [29] simply asked about the ability to read and write and reported on differences in participant demographic characteristics by literacy; they found that men are more likely to be literate and use a cell phone than women, AOR 2.81 (95% CI 1.83-4.30), and employed participants are more likely to be literate and use a cell phone than those with no income, AOR 2.35 (95% CI 1.23-4.49).

The acceptability of eHealth interventions was measured in 3 studies [27,29,30]. Nearly all (91.7%) patients with high eHealth literacy supported their providers’ use of SMS text messaging communication for reminders or to check health status in contrast to only 38.8% of PLWH who were not literate or did not own a cell phone (P<.001) [29]. Daily electronic medication adherence reminders were preferred over weekly or monthly [30]. Furthermore, perceptions of the ability to use the internet and eHealth literacy levels increased significantly after administration of a brief computer and eHealth class (P<.05 and P<.01, respectively) [27].

In this review, 6 of 7 studies examined the associations between eHealth literacy and a variety of health outcomes in PLWH. The 2 studies that measured the relationship between eHealth literacy and HIV-related behavior reported conflicting results. In Blackstock et al [28], higher eHealth literacy was found to be associated with more significant HIV transmission risk behaviors among women living with HIV, including vaginal or anal intercourse without a condom and illicit drug use in the past 30 days, adjusted for income and perceived health status, AOR 3.90 (95% CI 1.05-14.56). The authors suggested the complexities of eHealth literacy across unique social contexts as a possible explanation for the unexpected finding.

In contrast, following an electronically delivered health literacy intervention targeting HIV-related health literacy on medication adherence, participants in Ownby et al [26] self-reported increased knowledge about barriers to adherence and medication misconceptions (P=.02) as well as adherence behavioral skills, including using reminders, scheduling medications with other daily activities, and soliciting social support (P=.02). Data were collected 3 months apart; however, no control group was included in this study.

Siedner et al [24] examined participant retention in HIV care by measuring attendance at return-to-clinic appointments in accordance with instructions. Following an intervention that involved providing test results through SMS text messages, 60.8% of participants returned to the clinic when provided instructions through SMS text messages [24]. The ability to read a complete sentence on enrollment was independently associated with an accurate identification of the message sent, AOR 4.54 (95% CI 1.42-14.47; P=.01), and return to the clinic within 7 days of the first transmitted SMS text message, AOR 3.81 (95% CI 1.61-9.03; P=.002) [24]. In addition, the ability to access an SMS text message on enrollment was independently associated with returning to the clinic within 7 days of the SMS text message notification, AOR 4.90 (95% CI 1.06-22.61; P=.04) [24].

The relationship between eHealth literacy and HIV treatment adherence was mixed. Literacy was inversely associated with ART adherence, which was measured by Kim et al [29] as the self-reported number of missed doses per month (86.4% adherence among literate PLWH with a phone vs 90.6% adherence among not literate PLWH or those with no phone; AOR=1.76; 95% CI 1.12-2.77; P=.007). Krishnan et al [30] found no significant differences between patients with optional and suboptimal adherence in their access to communication technology overall; however, a significant difference was observed for mHealth acceptance among participants with and without optimal ART adherence (P<.01); for example, participants with poor adherence were less likely to be interested in anonymous internet interaction with a health professional to discuss HIV-related issues compared with participants with optimal adherence (P<.001) [30]. Ownby et al [26] attempted to improve the rates of adherence with an electronically delivered health literacy intervention; after this intervention, the adherence increased by 2.3% overall, resulting in the statistical significance among participants who were <95%, <90%, and <85% adherent (P=.01,.009,.04, respectively) but not among those in adherence categories of ≤75% [26]. These conflicting results about the relationship between the adherence and eHealth literacy might have been, in part, because of the complexities of measuring the adherence primarily with self-report as well as the nuanced differences between participants exhibiting high- and low-level adherence.

Because only 2 studies assessed participants’ HIV viral load under dissimilar study settings, we were unable to determine the association between eHealth literacy and HIV viral suppression [25,29]. Woods at al [25] reported, among a small sample of 46 HIV-infected participants with and without HIV-associated neurocognitive disorders, poorer performance in Web-based health care navigation tasks was associated with fewer years of education (ρ=.49, P=.003), higher plasma HIV viral load (ρ=–.47, P=.006), less frequent computer and internet use (P<.05), not owning a smartphone (P<.05), and higher anxiety related to using a computer (P<.05). According to Kim et al [29], in a large-scale study (n=895) with participants having advanced immunosuppression, however, the proportion of participants with an HIV viral load of >1000 copies/mL did not differ between literate phone owners (9%) and phone users who could not read and write (5.7%, P=.09).