This paper is in the following e-collection/theme issue:

Reviews on Public Health Technology and Innovation (70) JMIR Theme Issue: COVID-19 Special Issue (2461) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1520) Digital Health Reviews (1326) Outbreak and Pandemic Preparedness and Management (1743) JMH Theme Issue: COVID-19 and Mental Health: Impact and Interventions (386) Reviews in Digital Mental Health (247)

Published on in Vol 9 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/43730, first published .
The Usability, Feasibility, Acceptability, and Efficacy of Digital Mental Health Services in the COVID-19 Pandemic: Scoping Review, Systematic Review, and Meta-analysis

The Usability, Feasibility, Acceptability, and Efficacy of Digital Mental Health Services in the COVID-19 Pandemic: Scoping Review, Systematic Review, and Meta-analysis

The Usability, Feasibility, Acceptability, and Efficacy of Digital Mental Health Services in the COVID-19 Pandemic: Scoping Review, Systematic Review, and Meta-analysis

Authors of this article:

Shaoling Zhong1 Author Orcid Image ;   Xinhu Yang1 Author Orcid Image ;   Zihua Pan1 Author Orcid Image ;   Yu Fan1 Author Orcid Image ;   Yanan Chen1 Author Orcid Image ;   Xin Yu1 Author Orcid Image ;   Liang Zhou1 Author Orcid Image
Article Authors Cited by (18) Tweetations (7) Metrics

    Review

    The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China

    Corresponding Author:

    Liang Zhou, MD

    The Affiliated Brain Hospital of Guangzhou Medical University

    36 Mingxin Road

    Liwan District

    Guangzhou, 510370

    China

    Phone: 86 81268045

    Email: liangzhou_csu@vip.163.com


    Background: After the rapid spread of the novel SARS-CoV-2, the short-term and long-term mental health impacts of the pandemic on the public, in particular on susceptible individuals, have been reported worldwide. Although digital mental health services expand accessibility while removing many barriers to in-person therapy, their usability, feasibility, acceptability, and efficacy require continued monitoring during the initial phase of the pandemic and its aftermath.

    Objective: In this study, we aimed to understand what mental health services are offered, whether they are practical or acceptable, and to what extent digital mental health services are effective in response to the COVID-19 pandemic across high-income and low- and middle-income countries.

    Methods: This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the PRISMA Extension for Scoping Reviews (PRISMA-ScR) guideline. We implemented searches in PubMed (MEDLINE), Embase, PsycINFO, and Cochrane databases for studies that were published between December 2019 and November 2021 and that involved the use of digital mental health services. Two review authors screened, assessed, and extracted studies independently. The protocol was registered on the International Prospective Register of Systematic Reviews.

    Results: This review identified 7506 articles through database searching. In total, 65 (0.9%) studies from 18 countries with 67,884 participants were eligible for the scoping review. Of the 65 studies, 16 (24.6%) were included in the meta-analysis. A total of 15 (23.1%) studies measured the usability; 31 (47.7%) studies evaluated the feasibility; 29 (44.6%) studies assessed the acceptability; and 51 (78.5%) studies assessed the efficacy. Web-based programs (21/65, 32.3%), videoconferencing platforms (16/65, 24.6%), smartphone apps (14/65, 21.5%), and SMS text messaging (5/65, 7.7%) were the main techniques. Psychotherapy (44/65, 67.7%) followed by psychoeducation (6/65, 9.2%) and psychological support (5/65, 7.7%) were commonly used. The results of the meta-analysis showed that digital mental health interventions were associated with a small reduction in depressive symptoms (standardized mean difference=−0.49; 95% CI −0.74 to −0.24; P<.001) and a moderate reduction in anxiety symptoms (standardized mean difference=−0.66; 95% CI −1.23 to −1.0; P=.02) significantly.

    Conclusions: The findings suggest that digital mental health interventions may be practical and helpful for the general population, at-risk individuals, and patients with preexisting mental disorders across high-income and middle-income countries. An expanded research agenda is needed to apply different strategies for addressing diverse psychological needs and develop integrated mental health services in the post–COVID-19 era.

    Trial Registration: PROSPERO CRD42022307695; https://tinyurl.com/2jcuwjym

    JMIR Public Health Surveill 2023;9:e43730

    doi:10.2196/43730

    Keywords

    digital medicine; COVID-19; mental health services; psychological well-being; COVID-19 pandemic 


    Background

    As of October 5, 2022, there were 624.4 million confirmed cases of COVID-19 spread over 228 countries and territories, and COVID-19 has claimed the lives of 6.6 million individuals [1]. After the rapid spread of the novel SARS-CoV-2, known risk factors for mental health impacts have been reported worldwide. Fear of the virus and containment strategies might challenge psychological well-being [2]. Social isolation, loneliness, unemployment, and loss of income after the incidence of COVID-19 have become common. These risk factors might result in mental health problems, such as anxiety, depression, and insomnia, particularly in susceptible populations that include patients with COVID-19, first-line health professionals, and older adults [3]. In a cross-sectional study by Lai et al in China [4], 50% (634/1257) of health care workers reported experiencing depressive symptoms; 45% (560/1257) had anxiety; and 34% (427/1257) reported experiencing insomnia. Increased symptoms of mental health disorders and limited access to mental health care services and social support have also been reported in people with preexisting mental health disorders [5]. Furthermore, emerging reports suggested the possibility of the long-term effects of pandemics on psychological well-being [6,7].

    Although most evidence has been on the negative mental health impacts of COVID-19, a parallel area of research interest has explored how mental health services change when exposed to such stressors [8]. Web-based digital mental health services expand accessibility while removing many barriers to in-person therapy; thus, digital mental health services might be a solution in response to the challenge. Recent efforts in implementing digital mental health services have shown promising applications based on efficacy results from randomized controlled trials (RCTs). In a systematic review [9], internet-based cognitive behavioral therapy (CBT) was linked to a higher reduction in depressive symptoms at posttreatment than treatment as usual and the waiting list. Usability, feasibility, acceptability, and efficacy results at the time of pandemics, however, require continued monitoring. Knowing what mental health care service is available, whether it is feasible and acceptable, and to what extent digital mental health services take effect is crucial to inform policy decisions, such as the delivery, implementation, and target areas for applying mental health resources during the pandemic and its aftermath.

    Objective

    Against this background, the objective of this scoping review and systematic review was to qualify and quantify the usability, feasibility, acceptability, and efficacy of digital mental health interventions applied for infectious disease outbreaks in the initial phase of the pandemic. We focused on the assessment of whether each of the applied techniques was usable, feasible, acceptable, and effective. We also separately synthesized data from high-income countries and low- and middle-income countries (LMICs). As there is no standard definition of usability, feasibility, acceptability, and efficacy, we considered comprehensive and broad definitions of these terms. We defined usability as program use, user engagement, and the ease of use of the services being tested, as followed in prior research [10,11]. We measured feasibility by attrition, attendance, adherence, retention, and qualitative feedback. We defined acceptability as user satisfaction, intent to continue use, and the perceived appropriateness of the intervention [12]. We defined efficacy as the intended effects and effect size estimation of the services [12].


    Overview

    This report includes a scoping review of studies on digital mental health interventions applied for infectious disease outbreaks and a systematic review of studies assessing commonly reported mental distresses identified in the scoping review. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the PRISMA Extension for Scoping Reviews (PRISMA-ScR) guideline when appropriate (Multimedia Appendix 1).

    Data Search

    We sought to include original studies focusing on digital mental health services applied during or in response to COVID-19. We implemented searches in PubMed (MEDLINE), Embase, PsycINFO, and Cochrane between December 2019 and November 2021 using a combination of search terms relating to both digital mental health services and COVID-19. The complete search strategy is provided in Multimedia Appendix 2. We performed a manual search of the reference lists of relevant reviews, eligible studies, and relevant conference abstracts.

    Study Selection

    We removed duplicate citations across electronic databases before the screening. Initially, pairs of 2 review authors (SZ, XY, ZP, and YF) independently screened the titles and abstracts for eligibility. We referred to full texts if titles and abstracts did not provide information for eligibility based on the inclusion and exclusion criteria. The full texts of the potentially included studies were screened by 1 review author and checked by another author for agreement. In case of disagreements, the opinion of a third reviewer was sought, and a consensus was reached through discussion.

    We included qualitative, quantitative, and mixed methods studies that involved the use of digital mental health services. Eligible interventions included health education, evaluation, consulting, helplines, psychological interventions, and telemedicine. The intervention was delivered by any means of digital technology, including SMS text messaging, smartphones, websites, social media, videoconferencing, wearable devices, or mobile apps. We excluded comments, protocol papers, systematic reviews, preprints, conference papers, and studies that did not provide any qualitative or quantitative data for defined outcomes. We excluded studies that were conducted before December 2019. There was no exclusion for age, gender, ethnicity, socioeconomic status, professions (health workers or not), or any type of mental disorder (clinical or nonclinical). For the meta-analysis, we included parallel RCTs and crossover RCTs that provided adequate data for analyses, using the same aforementioned criteria.

    Data Extraction

    We developed a data extraction form that could be used by all review authors, as recommended by the Joanna Briggs Institute Scoping Review methodological guidance [13]. The data extraction form included information on study characteristics (authors, publication year, study design, study period, geographic regions, and study aim), participant characteristics (age, gender, sample size, and type of mental illness), digital mental health services characteristics (types of techniques and type of mental health services), and outcome characteristics (type of outcomes and their measurements). Owing to the diverse outcomes identified across eligible studies, we extracted the study results in 2 phases. In phase 1, we extracted the key findings for all reported outcomes. For effectiveness data, we extracted the results for significant differences between digital mental health services and comparison groups or trends between postintervention and preintervention outcome measurements for digital mental health services. In phase 2, we identified RCTs that reported similar digital mental health services and outcomes for the systematic review and meta-analysis. Here, effectiveness data from the comparison groups and intervention arms were retrieved using separate extraction forms. The data were extracted by 2 reviewers (XY and ZP) independently. Disagreements were resolved by discussion. A third reviewer (SZ) was involved when necessary.

    Risk of Bias Assessment

    Two reviewers (XY and ZP) independently assessed the risk of bias using the Cochrane Risk of Bias (RoB 2) tool for all included RCTs. Disagreements were resolved by discussions. In case of uncertainties, a third reviewer (SZ) was consulted. The 5 domains included bias owing to the assignment of intervention, adherence to the intervention, missing outcome data, outcome measurement, and selection of reported outcome that were measured. Studies were rated as “low” risk, “some concerns,” or “high” risk.

    Data Synthesis

    Owing to the nature of the research question and the heterogeneity of the studies, we synthesized qualitative and quantitative results separately. For the scoping review, we descriptively summarized the included studies with a narrative synthesis of individual studies. We summarized the overall number of included studies, the total number of study populations and their basic characteristics, countries, study design, type of digital mental health services, and the objectives and findings of the included studies. We clustered the studies by the type of techniques used and location (high-income countries vs LMICs). For each type of technique, the findings on usability, feasibility, acceptability, and effectiveness data were narratively synthesized by a textual approach.

    For meta-analyses, we synthesized findings on similar efficacy outcomes that were measured by >3 RCTs. Outcomes, such as loneliness, burnout, or psychological well-being, that were measured by 1 or 2 RCTs were not synthesized. We pooled continuous outcomes as standardized mean differences (SMD) with 95% of CIs using a random-effects meta-analysis. The values of 0.2, 0.5, and 0.8 in SMDs denote minor, medium, and large effects, respectively [14]. Cochran Q and the I2 statistic were used to assess heterogeneity. An I2 value of ≥60% can be considered a substantial level of inconsistency across studies [15]. The analyses were stratified according to the reported outcomes. For each of these, we pooled the evidence from all eligible RCTs, regardless of the techniques applied, measurements used, and population studied. We conducted subgroup analyses to examine whether there were differences in outcomes on the basis of sample size (as a dichotomous variable), measurement of the outcome, location, duration of follow-up, and duration of intervention. We also performed meta-regression analyses to explore the effect size by sample size (as a continuous variable), age, and proportion of females in the population studied. We used funnel plots and ran the Egger test to look for asymmetry to detect publication bias that might impair the validity of our findings. We performed all analyses with R statistical software (version 4.2.2; R Foundation for Statistical Computing) [16].


    Study Selection

    The PRISMA flowchart shows the study selection process (Figure 1). This review identified 7506 articles that were identified through database searching. After removing duplicate records, we screened 5118 (68.19%) records and identified 65 (1.27%) eligible studies for the scoping review, and the results were synthesized narratively. Of the 19 RCTs, 17 (89%) studies reported depressive symptoms and 18 (95%) studies reported anxiety symptoms as the outcome. In total, 3 (16%) studies were not included in the meta-analyses; 2 (67%) studies [17,18] that reported on depression and anxiety disorders did not provide adequate data, and 1 (33%) study that reported anxiety scores using state anxiety was with a very large estimate [19]. Accordingly, of the total 65 eligible studies, 16 (25%) studies that reported depressive symptoms or anxiety symptoms were included in the meta-analysis.

    ‎
    Figure 1. Flowchart of study selection.
    View this figure

    Study Characteristics

    Table 1 presents the characteristics of participants and studies included in the review. The 65 included studies were conducted in the following 18 different countries: 24 (37%) studies from the United States (participants: 54,245/67,884, 79.9%), 8 (12%) studies from China (1114/67,884, 1.6%), 6 (9%) studies from Canada (5490/67,884, 8.1%), 5 (8%) studies from Italy (315/67,884, 0.5%), 3 (5%) studies from Australia (1004/67,884, 1.5%), 3 (5%) studies from France (2235/67,884, 3.3%), 2 (3%) studies from India (654/67,884, 1%), 2 (3%) studies from Malaysia (123/67,884, 0.2%), 2 (3%) studies from Greece (164/67,884, 0.2%), 2 (3%) studies from Turkey (121/67,884, 0.2%), and further 9 (14%) studies were conducted in other countries (ie, Dominican Republic, Israel, Japan, Oman, South Africa, Spain, Sweden, and the United Kingdom). Of these, 49 (75%) studies were from high-income countries, and 16 (25%) studies were conducted in middle-income countries. None of the included studies reported data from low-income countries. The participant pool consisted of 67,884 participants; most (44,125/67,884, 65%) were female. There were 2663 (3.9%) people with a mental illness or mental distress, 713 (1.1%) people with or suspected of COVID-19, and 748 (1.1%) hospital workers.

    Of the 65 studies, a total of 15 (23.1%) studies measured usability; 31 (47.7%) evaluated feasibility; 29 (44.6%) assessed acceptability; and 51 (78.5%) assessed efficacy.

    Web-based programs (21/65, 32.3%), videoconferencing platforms (16/65, 24.6%), smartphone apps (14/65, 21.5%), and SMS text messaging (5/65, 7.7%) were the main techniques applied. There were other studies that used social media (3/65, 4.6%), hotlines (3/65, 4.6%), a chatbot (1/65, 1.5%), virtual reality (VR; 1/65, 1.5%), and robotic telemedicine (1/65, 1.5%). Most studies provided psychotherapy (44/65, 67.7%), followed by psychoeducation (6/65, 9.2%), psychological support (5/65, 7.7%), psychological assessments (4, 6.1%), and psychiatric clinical services (4/65, 6.1%).

    Table 1. Characteristics of participants and studies included in the qualitative systematic review.
    Author and yearDesign and study typeSample size, nAge (years), mean (SD)aFemale, %Intervention vs control group
    Aguilera et al [20]Pre- and postintervention evaluation; quantitative30333.3 (11.0)78A texting intervention (StayWell at Home)
    Agyapong et al [21]Pre- and postintervention evaluation; quantitative276776.7% were aged between 26 and 6088Text4Hope subscribers who received once-daily supportive SMS text messages vs Text4Hope subscribers who did not receive messages
    Al-Alawi et al [18]RCTb; quantitative4628.51 (8.7)78Therapist-guided web-based therapy vs newsletter received via email containing self-help information and tips to cope with distress
    Al-Refae et al [22]RCT; quantitative16525.24 (8.74)78.8Smartphone intervention (Serene) vs waiting list
    Bantjes et al [23]Pre and postintervention evaluation; quantitative17522.2 (4.9)85.4A web-based group CBTc intervention
    Ben-Zeev et al [24]RCT; quantitative31537.89 (11.64)83.8Smartphone app (CORE) vs waiting list
    Brouzos et al [25]Pre- and postintervention evaluation; quantitative8233.07 (9.55)78Web-based positive psychology intervention (“Staying Home—Feeling Positive”) vs no intervention
    Casas [26]Pre- and postintervention evaluation; mixed methods1539.52 (13.57)73.3Written exposure therapy delivered via telehealth
    Chandra et al [27]Pre- and postintervention evaluation; mixed methods64349.57 (15.23)37.2Mental health support telecounseling
    Coifman et al [28]Pre- and postintervention evaluation; quantitative2845.33 (9.6)75Daily coping toolkit intervention
    Comer et al [29]RCT; quantitative40Children=6.2 (1.8); parents=38.39 (4.5)Children: 72.5; parents: 75iCALM Telehealth Program vs waiting list
    Craig et al [30]Pre- and postintervention evaluation; mixed methods9622.3 (4.1)17.7AFFIRM online group vs waiting list
    Detweiler Guarino et al [31]Pre- and postintervention evaluation; quantitative248444 (15.2)53.2The PATH program
    Dincer and Inangil [32]RCT; quantitative7233.46 (9.6)88.9Received emotional freedom techniques vs stayed comfortable in a calm and tranquil environment
    Esentürk and Yarımkaya [33]Pre- and postintervention evaluation; mixed methods14Children=12.07; parents=51.4Children=42.9; parents=64.2WhatsApp-based physical activity intervention
    Fiol-DeRoque et al [34]RCT; quantitative48241.37 (10.4)83.2PsyCovidApp vs a general control app
    Gabrielli et al [35]Pre- and postintervention evaluation; mixed methods7120.6 (2.4)68Healthy-coping intervention via chatbot
    Geoffroy et al [36]Postintervention evaluation;14932.7 (11.0)86Covid-Psy hotline
    Gordon et al [37]Pre- and postintervention evaluation; mixed methods9951.4 (14.7)86A guided imagery mobile app
    Graziano et al [38]Pre- and postintervention evaluation; quantitative30Patients=22.5 (6.9); parents=37 (6.3)Patient=56.3; parents=92.9Telehealth psychological interventions
    Gromatsky et al [39]Pre- and postintervention evaluation; mixed methods2054.2 (11.95)15VA CONNECT
    Guan et al [19]RCT; quantitative9518.63 (0.75)28.4Self-compassion writing induction condition vs a writing control condition (to think about a negative event)
    Guo et al [40]Pre- and postintervention evaluation; quantitative508Intervention=47.0 (4.3), control=46.6 (4.2)84.1Web-based education program vs no intervention
    Held et al [41]Pre- and postintervention evaluation; mixed methods2—d0Intensive cognitive processing therapy–based program
    Hom et al [42]Postintervention evaluation; mixed methods2339.04 (16.15)52.2Virtual partial hospital program
    Hosseinzadeh Asl [43]RCT; quantitative4933.06 (6.02)55.1Mindfulness-based intervention vs waiting list
    Hu et al [44]Pilot RCT; quantitative4842.6 (15.8)54.2WeChat-based psychological interventions vs conventional nursing
    Ibrahim et al [45]Pre- and postintervention evaluation; quantitative4363.4167.4Virtual group exercise
    Jaworski et al [46]Postintervention evaluation; quantitative49,287——COVID coach
    Kahlon et al [47]RCT; quantitative24069.1 (12.1)79Receive calls vs no calls
    Karagiozi et al [48]RCT; quantitative8244.5 (10.6)—Web-based intervention group vs onsite intervention
    Kim [49]RCT; quantitative8041.7 (13.8)56.25Teleacupressure self-practice group vs web-based communication group
    Kolbe et al [50]Postintervention evaluation; mixed methods13 patients and 11 staff——Virtual reality
    Lazzaroni et al [51]Pre- and postintervention evaluation; quantitative50Between 13 and 2484Eye movement desensitization and reprocessing intervention group
    Lima et al [52]Postintervention evaluation; mixed methodsHealthy volunteers=10; older adults and people with dementia=1Healthy volunteers=21-59; older adults and people with dementia=not mentionedHealthy volunteers=50; older adults and people with dementia=0Robotic telemedicine
    Maldonado [53]Pre- and postintervention evaluation; quantitative4044.1 (13)82.5Online intervention group
    McKeon et al [54]Pre- and postintervention evaluation; quantitative1164.4 (4.3)82Mental health–informed lifestyle intervention
    Nauphal et al [55]Pre- and postintervention evaluation; quantitative536.2 (9.5)40Web-based CBT group
    Ortiz et al [56]Pre- and postintervention evaluation; mixed methods38346.9 (16.8)70.1Texting intervention group
    Parolin et al [57]Pre- and postintervention evaluation; mixed methods13433.20 (10.61)86Italia Ti Ascolto intervention group
    Philip et al [58]Pre- and postintervention evaluation; quantitative206943.52 (13.94)65.1KANOPEE group
    Puspitasari et al [59]Pre- and postintervention evaluation; quantitative7636.6 (13.4)86Adult Transitions Program teletherapy group
    Rojas et al [60]Pre- and postintervention evaluation; mixed methods1—0Brief CBT for suicide prevention
    Shalaby et al [61]Postintervention evaluation; quantitative203244.58 (13.45)88Text4Hope intervention group
    Shapira et al [62]Pilot RCT; quantitative8272 (5.63)—A short-term digital group vs a waitlist control group
    Sharrock et al [63]Pre- and postintervention evaluation; quantitative90437.83 (12.64)67.1iCBTe group during COVID-19 vs iCBT group before COVID-19
    Song et al [64]Pre- and postintervention evaluation; quantitative12934.64 (9.11)69Mobile internet CBT group vs a waitlist control group
    Sosa Lovera et al [65]Postintervention evaluation; quantitative4973273COVID-19 helpline group
    Sturgill et al [66]Pre- and postintervention evaluation; quantitative9919.9 (1.94)69Ajivar activities intervention + routine mental wellness instruction group vs group that received routine mental wellness instruction
    Suffoletto et al [67]Pilot RCT; quantitative5218.7 (0.46)86.5MoST-MHf Intervention vs enhanced usual care group
    Summers, Wu, and Taylor [68]Pre- and postintervention evaluation; quantitative27349.6 (9.24)59.3Gro Health app users
    Sun et al [69]Pilot RCT; mixed methods11422.21 (2.67)73.7Mindfulness-based mHealthg group vs social support–based mHealth group
    Tarquinio et al [70]Pre- and postintervention evaluation; quantitative1733.2 (4.10)100Eye Movement Desensitization and Reprocessing intervention group
    Vallefuoco et al [71]Postintervention evaluation; quantitative30——18 therapists and 12 parents of children with ASDh
    van Agteren et al [72]Pre- and postintervention evaluation; quantitative8938.67 (13.06)66Internet-based group mental health intervention
    Wagner [73]Postintervention evaluation; mixed methodsChildren: 204; clinicians: 9Children=27.54 months (5.36 months)23ASD-PEDS intervention group
    Wahlund et al [74]RCT; mixed methods67046 (13.50)81.6Web-based psychological intervention vs waiting list
    Wasil et al [75]Pre- and postintervention evaluation; quantitative18931.04 (8.91)72.99COMET intervention
    Wei et al [17]RCT; quantitative2644.65 (12.09)38.5An internet-based integrated intervention vs support care
    Wood et al [76]Postintervention evaluation; mixed methods726.9 (4.8)—Group teletherapy
    Ying et al [77]Pre- and postintervention evaluation; quantitative13773.39 (7.37)68.5iCBT group
    Zepeda et al [78]Pre- and postintervention evaluation; mixed methods279.5667iCOPE
    Zhang et al [79]Pilot RCT; quantitative5750.12 (6.79)47.1WeChat-based psychological interventions vs waitlist group
    Zimmerman et al [80]Postintervention evaluation; quantitative48037.1 (14)67.5Telehealth program vs partial in-person program before the COVID-19 outbreak
    Van Lieshout et al [81]RCT; quantitative40331.8 (4.40)100Web-based CBT + treatment as usual vs treatment as usual

    aWhere mean (SD) was not available in the original study, mean or ranges were used where applicable.

    bRCT: randomized controlled trial.

    cCBT: cognitive behavioral therapy.

    dNot applicable.

    eiCBT: internet-based congnitive behavioral therapy.

    fMoST-MH: Mobile Support Tool for Mental Health.

    gmHealth: mobile health.

    hASD: autism spectrum disorder.

    Qualitative Synthesis of Results

    Overview

    This section summarizes the findings from 65 studies that reported the feasibility, acceptability, and effectiveness of digital mental health services during the COVID-19 pandemic. The synthesis was organized according to the type of digital technique used. Table 2 presents a summary of usability, feasibility, acceptability, and efficacy by the types of techniques in high-income countries and LMICs. Table S1 (Multimedia Appendix 3 [17-81]) presents summaries of individual studies on mental health interventions and major findings and their measurements.

    Table 2. Summaries of usability, feasibility, acceptability, and efficacy by type of techniques in high-income countries (HICs) and low- and middle-income countries (LMICs).
    Techniques and groupsUsabilityFeasibilityAcceptabilityEfficacy
    Videoconferencing platforms

    		HICs (n=11)—a
    • Three studies reported a completion rate between 92% and 100%.
      • ‎
    • Three studies reported qualitative feedback of highly perceived benefits.
      • ‎
    • Three studies reported technology-related challenges.
      • ‎
    • Five studies reported high levels of overall satisfaction.
      • ‎
    • Nine studies reported psychotherapy improved depressive levels and 5 studies found improved anxiety symptoms.
      • ‎
    • One study found no effects on decreasing anxiety symptoms.
      • ‎

    		LMICs (n=5)—
    • Two studies reported the mean attendance ranges from 6.4 (SD 2.8) to 10.26 (SD 7.02).
      • ‎
    • One study reported high levels of satisfaction.
      • ‎
    • Three studies reported improvements in depressive symptoms or negative effects.
      • ‎
    • Two studies reported no effects on decreasing depression levels.
      • ‎
    Web-based programs

    		HICs (n=17)
    • Two studies reported a significant increase in registered users.
      • ‎
    • One study reported a high rating in usability.
      • ‎
    • Five studies reported the completed rate ranged from 30.5% to 85%.
      • ‎
    • Three studies reported retention rate, which ranged from 30.5% to 85%.
      • ‎
    • One study reported high attrition rate of 13%.
      • ‎
    • One study reported strong attendance.
      • ‎
    • Two studies reported that retentions were strong.
      • ‎
    • Two studies reported high ratings of feasibility, and one study reported technical difficulty.
      • ‎
    • Five studies reported high satisfaction.
      • ‎
    • One study reported that acceptability scores were significantly better than average.
      • ‎
    • Three studies reported high acceptability and rated the interventions as acceptable, helpful, appropriate, and positive.
      • ‎
    • Eight studies reported improvement in anxiety symptoms and anxiety-related social impairment neither with nor without controls.
      • ‎
    • Two studies indicated no effects on reducing anxiety.
      • ‎
    • Five studies indicated improvement in depression symptoms and depression severity.
      • ‎
    • One study reported no difference in depressive symptoms when compared with onsite groups.
      • ‎
    • Three studies reported improved stress-related symptoms.
      • ‎
    • Two studies reported reductions in COVID-19–related worry.
      • ‎
    • One study indicated improvements in insomnia.
      • ‎

    		LMICs (n=4)
    • One study reported the mean time spent was 35.63 (SD 25.41) min.
      • ‎
    • One study reported a completion rate of 87.4%.
      • ‎
    • One study reported high attendance and high retention rates (91.2%).
      • ‎
    • One study reported high ratings of acceptability and the other a high level of satisfaction.
      • ‎
    • Four studies indicated improvement in anxiety symptoms neither with no controls nor control groups.
      • ‎
    • Two studies reported improvement in depressive symptoms with no controls.
      • ‎
    • One study reported a reduction in stress and burnout.
      • ‎
    Smartphone apps

    		HICs (n=11)
    • Two studies reported high usability scores. Two studies reported user engagement. The time ranged from 36.7 min over 12 weeks to 1424 min over 14 weeks.
      • ‎
    • One study reported the number of days retained was 42.44 (SD 44.40).
      • ‎
    • Three studies reported a retention rate that ranged from 28.3% to 85.7%.
      • ‎
    • Two studies rated the intervention as acceptable, and 2 studies rated the intervention as satisfied.
      • ‎
    • One rated the intervention as good.
      • ‎
    • One rated the intervention as helpful.
      • ‎
    • Four studies reported reduced depressive symptoms compared with the preintervention and waitlist groups.
      • ‎
    • Five studies reported similar improvements in anxiety symptoms compared with the preintervention, waitlist group, and a control app with limited access to psychoeducational content.
      • ‎

    		LMICs (n=3)——
    • One study indicated high satisfaction and would recommend the service. One study reported helpful and enjoyable.
      • ‎
    • Two studies reported improvements in depressive symptoms, insomnia, psychological flexibility, and self-compassion, either with no controls or in comparison with the waitlist group.
      • ‎
    Texting

    		HICs (n=5)
    • One study reported high usability.
      • ‎
    • Two studies reported a completion rate of 16% and 78%, respectively.
      • ‎
    • One study reported a high level of overall satisfaction.
      • ‎
    • Four studies reported an improved mood rating was observed at posttreatment assessment either with or without the control group.
      • ‎

    		LMICs (n=0)————
    Social media

    		HICs (n=1)—
    • One study reported high retention.
      • ‎
    • One study reported high acceptability.
      • ‎
    • One study reported an effect on psychological distress, quality of life, functioning, loneliness, and physical activity without a control group.
      • ‎

    		LMICs (n=2)
    • One study reported an average user time of 18.7 hours.
      • ‎
    		——
    • Two studies reported improvements in anxiety and depressive symptoms when compared with either the usual care or the waitlist group.
      • ‎
    Hotline and telephone calls

    		HICs (n=2)
    • One study reported 5.73 (SD 3.22) calls/day.
      • ‎
    • One study reported a dropout rate of 7.5%.
      • ‎
    		—
    • One study reported improvements in loneliness, depression, anxiety, and general mental health.
      • ‎

    		LMICs (n=1)——
    • One study reported high satisfaction, and the participants would use again, and would recommend it to others.
      • ‎
    		—
    Robotic telemedicine and VRb

    		HICs (n=2)
    • One study reported an average of 78 (SD 24.8) times.
      • ‎
    • One reported a completion rate of 58%.
      • ‎
    • One study reported high satisfaction.
      • ‎
    • One study reported a decrease in anxiety symptoms and stress symptoms.
      • ‎

    		LMICs (n=1)
    • One study reported an overall positive impression of the multimodal robotic system.
      • ‎
    • One study reported the need to adjust some features.
      • ‎
    		——

    aNo evidence found.

    bVR: virtual reality.

    Web-Based Programs

    Overall, 21 (32.3%; 5480/67,884, 8.1% out of total participants) out of the 65 studies included in this scoping review used web-based programs. In 17 studies that were conducted in high-income countries, a significant increase in registered users (n=2) and a high rating in usability (n=1) were reported. The completion rate ranged from 30.5% to 85% (n=5); the retention rate ranged from 30.5% to 85% (n=3); and the attrition rate was 13% (n=1). Quality checks of feasibility reported strong attendance (n=1), strong retentions (n=2), and high ratings of feasibility (n=2). Studies presented preliminary support for the feasibility of web-based programs yet with possible technical difficulty (n=1). In terms of acceptability, there was high satisfaction (n=5); the interventions were perceived to be acceptable, helpful, appropriate, and positive (n=3); and the acceptability scores were significantly better than average (n=1). Most studies, either when no control group is involved or when compared with the control group, reported improvements in anxiety symptoms and anxiety-related social impairment (n=8), while no effects on reducing anxiety were reported as well (n=2). The same patterns were found in outcomes measuring depression. Most studies reported improvements in depressive symptoms and depression severity (n=5), and there was no difference in depressive symptoms when the intervention group was compared with onsite groups (n=1). Improvements in stress-related symptoms (n=3), COVID-19–related worry (n=2), and insomnia (n=1) were identified.

    In 4 studies conducted in LMICs, the mean time spent was 35.6 (SD 25.4) minutes (n=1) and the completion rate was 87.4% (n=1). High attendance and high retention rates (91.2%) were reported (n=1). With regard to acceptability, high ratings of acceptability and a high level of satisfaction (n=2) were reported. Studies, either when no control group is involved or when compared with the waitlist group, reported improvements in anxiety symptoms (n=4); some studies reported improvements in depressive symptoms (n=2), and 1 study reported reduction in stress and burnout.

    Videoconferencing Platforms

    Among the 65 studies, 16 (24.6%; participants: 1973/67,884, 2.9%) used teleconferencing platforms, including Zoom [30,39,49,55,59,60,62,72,73,81], Skype [25], Microsoft Teams [1,49], Google Meet [45], Tencent Meeting [19], and other platforms [41,42]. In 11 studies that were conducted in high-income countries, a completion rate between 92% and 100% (n=3), qualitative feedback of high perceived benefits (n=3), and technology-related challenges (n=3) were reported in terms of feasibility. High levels of satisfaction were reported (n=5). Studies reported improvements in depressive levels (n=9) and anxiety symptoms (n=5). No effects on decreasing anxiety symptoms were reported in one study. In 5 studies conducted in LMICs, the mean attendance ranged from 6.4 (SD 2.8) to 10.26 (SD 7.02; n=2). High levels of satisfaction (n=1) were reported. There were improvements in depressive symptoms and negative affect (n=3), yet no effects on decreasing depression levels were identified (n=2). No evidence was found in usability for LMICs or high-income countries.

    Smartphone Apps

    Of the 65 studies, 14 (21.5%; participants: 53,786/67,884; 79.2%) developed smartphone apps to deliver mental health services. In 11 studies conducted in high-income countries, studies reported high usability scores (n=2), and user engagement time ranged from 36.7 minutes over 12 weeks to 1424 minutes over 14 weeks (n=2). Feasibility was reported in terms of the average number of days retained (mean 42.4, SD 44.4; n=1), and the retention rate ranged from 28.3% to 85.7% (n=3). In terms of acceptability, the interventions were rated as “acceptable” (n=2), “satisfied” (n=2), “good” (n=1), and “helpful” (n=1). Studies reported a reduction in depressive symptoms compared with the preintervention and waitlist group (n=4). Similar improvements were reported in anxiety symptoms compared with preintervention, waitlist group, and a control app with limited access to psychoeducational contents (n=5). In 4 studies in LMICs, no evidence was found for usability and feasibility. Acceptability was reported as high satisfaction (n=1) and helpful and enjoyable (n=1). There were improvements in depressive symptoms, insomnia, psychological flexibility, and self-compassion, either when no control group is involved or when compared with the waitlist group (n=2).

    SMS Text Messaging

    Of the 65 studies, 5 (7.7%; participants: 5537/67,884, 8.1%) used the SMS text messaging technique. All studies were conducted in high-income countries. Studies reported high usability of ecological momentary intervention (n=1), and the completion rate ranged from 16% to 78%. An overall high satisfaction was reported (n=1). Studies suggested that an improved mood rating was observed at posttreatment assessments either with or without the control group (n=4). No evidence was found of studies conducted in LMICs for the technique of SMS text messaging.

    Social Media

    The use of social media as a technique to deliver mental health services was described in 3 (4.6%; participants: 116/67,884, 0.2%) out of 65 studies [44,54,79]. In one study conducted in high-income countries, high retention, high acceptability, and an effect on multiple outcomes (psychological distress, quality of life, functioning, loneliness, and physical activity) were reported (n=1). In the 2 studies conducted in LMICs, an average user time of 18.7 hours (n=1) was reported. Evidence suggested improvements in anxiety and depressive symptoms when compared with either usual care or the waitlist group (n=2). No evidence was found for other outcomes.

    Hotline and Telephone Calls

    Of the 65 studies, 3 (4.6%; participants: 886/67,884, 1.3%) reported using hotlines and telephone calls to deliver mental health services during the COVID-19 pandemic [36,47,65]. In 2 studies conducted in high-income countries, an average of 5.73 (SD 3.22) calls per day was reported (n=1). A dropout rate of 7.5% was reported (n=1), and there were improvements in loneliness, depression, anxiety, and general mental health, as reported in the same study (n=1). In one study that was conducted in LMICs, participants reported high satisfaction and mentioned that they would use the service again and would recommend it to others (n=1). No evidence was found in terms of usability, feasibility, or efficacy for the techniques of hotline and telephone calls by the studies conducted in LMICs.

    Robotic Telemedicine and VR

    Of the 65 studies, 3 (4.6%; participants: 106,67,884, 0.2%) used robotic telemedicine and VR. Two studies conducted in high-income countries reported an average use of 78 (SD 24.8) times (n=1) and a completion rate of 58% (n=1). High satisfaction was reported (n=1). Evidence suggested a decrease in anxiety symptoms and stress symptoms for all participants after the intervention (n=1). In 1 study conducted in LMICs, an overall positive impression of the multimodal robotic system was reported (n=1), yet the need to adjust some features was suggested (n=1). No evidence for acceptability or efficacy was found by studies conducted in LMICs.

    Meta-analyses

    Of the 65 studies, 15 (23.1%) that reported depressive symptoms were included in the meta-analysis. Digital mental health interventions were associated with a small significant reduction in depressive symptoms (SMD −0.49, 95% CI −.74 to −.24; P<.001; Figure 2) with substantial heterogeneity (I2=87%, 95% CI 80%-91%; Q=107.57; P<.001).

    The effectiveness of digital mental health interventions on anxiety symptoms was reported in 15 RCTs. The pooled SMD showed a moderate and significant effect of these interventions in reducing anxiety symptoms (SMD=−0.66, 95% CI −1.23 to −.10; P=.02; Figure 3) with substantial heterogeneity (I2=93%, 95% CI 90%-95%; Q=207.22; P<.001).

    We examined the sources of heterogeneity by studying covariates across studies (Tables 3 and 4). Subgroup analyses revealed that reduced depressive symptoms were associated with the measurement of depression (using Depression Anxiety Stress Scales-21: SMD=−0.13, 95% CI −0.31 to 0.06 vs other measurements: SMD=−0.61, 95% CI −0.91 to −0.30; Q=7.04; P=.008). Reduced anxiety symptoms were not associated with sample size, location, measurement of anxiety, duration of follow-up, or the duration of intervention.

    Meta-regression analyses are reported in Multimedia Appendix 4. There was no significant association between SMDs of depression and sample size, age, and proportion of females in the study population. Meta-regression analyses did not reveal differences in the SMDs of anxiety depending on sample size, age, and proportion of females in the study population.

    For the main analyses of depression and anxiety, we observed evidence of funnel plot asymmetry, which could suggest no publication bias (Egger test: P=.57 and P=.50, respectively; Multimedia Appendix 5).

    ‎
    Figure 2. Effectiveness of digital mental health interventions in reducing depressive symptoms. BDI-II: Beck Depression Inventory-II; BSI: Brief Symptom Inventory; CCAPS: College Counseling Center Assessment of Psychological Symptoms; DASS-21: Depression Anxiety Stress Scales-21; EPDS: Edinburgh Postnatal Depression Scale; HAMD: Hamilton Rating Scale for Depression; MADRS-S: Montgomery Åsberg Depression Rating Scale-Self rated; PHQ: Patient Health Questionnaire; and SDS: Self-rating Depression Scale; SMD: standardized mean difference.
    View this figure
    ‎
    Figure 3. Effectiveness of digital mental health interventions in reducing anxiety symptoms. BDI-II: Beck Depression Inventory-II; BSI: Brief Symptom Inventory; CBCL: Child Behavior Checklist; CCAPS: College Counseling Center Assessment of Psychological Symptoms; DASS-21: Depression Anxiety Stress Scales-21; GAD-7: Generalized Anxiety Disorder-7; HAMA: Hamilton Anxiety Rating Scale; SAS: Self-rating Anxiety Scale; SMD: standardized mean difference; and STAI: State-Trait Anxiety Inventory.
    View this figure
    Table 3. Effect size of digital mental health services for reducing depressive symptoms during the COVID-19 pandemic according to study characteristics.
    GroupStudies, nSMDa (95% CI)QP value
    Sample size0.03.86

    		≥818−0.51 (−0.86 to −0.16)


    		<817−0.46 (−0.84 to −0.09)

    Measurement of depression7.04.008

    		DASS-21b4−0.13 (−0.31 to 0.06)


    		Other measurementsc11−0.61 (−0.91 to −0.30)

    Location0.41.52

    		High-income countries11−0.53 (−0.83 to −0.24)


    		Low- and middle-income countries4−0.35 (−0.82 to 0.10)

    Duration of follow-up1.23.27

    		Short-term12−0.55 (−0.84 to −0.26)


    		Long-term3−0.28 (−0.66 to −0.09)

    Duration of intervention0.84.36

    		≥4 weeks10−0.56 (−0.90 to −0.23)


    		<4 weeks5−0.34 (−0.68 to −0.02)

    aSMD: standardized mean difference.

    bDASS−21: Depression Anxiety Stress Scales-21.

    cOther measurements include Beck Depression Inventory-II, Brief Symptom Inventory, College Counseling Center Assessment of Psychological Symptoms, Edinburgh Postnatal Depression Scale, Hamilton Rating Scale for Depression, Montgomery Åsberg Depression Rating Scale—Self rated, Patient Health Questionnaire, and Self-rating Depression Scale.

    Table 4. Effect size of digital mental health services for reducing anxiety symptoms during the COVID-19 pandemic according to study characteristics.
    GroupStudies, nSMDa (95% CI)QP value
    Sample size0.24.62

    		≥817−0.54 (−1.01 to −0.07)


    		<818−0.85 (−1.99 to 0.29)

    Location0.49.48

    		High-income countries10−0.49 (−0.66 to −0.13)


    		LMICsb5−1.17 (−3.02 to 0.68)

    Measurement of anxiety10.95

    		GAD-7c5−0.68 (−1.31 to −0.04)


    		Other measurementsd10−0.71 (−1.61 to 0.19)

    Measurement of anxiety22.06.15

    		DASS-21e4−0.23 (−0.38 to −0.08)


    		Other measurements11−0.85 (−1.68 to −0.02)

    Duration of follow-up1.14.29

    		Short-term12−0.80 (−1.55 to −0.06)


    		Long-term3−0.29 (−0.86 to 0.29)

    Duration of intervention0.58.45

    		≥4 weeks9−0.47 (−0.91 to −0.04)


    		<4 weeks6−1.07 (−2.56 to 0.41)

    a SMD: standardized mean difference.

    bLMICs: low- and middle-income countries.

    cGAD-7: Generalized Anxiety Disorder-7.

    dOther measurements including Beck Anxiety Inventory, Brief Symptom Inventory, Child Behavior Checklist, College Counseling Center Assessment of Psychological Symptoms, Hamilton Anxiety Rating Scale, Self-rating Anxiety Scale, and State-Trait Anxiety Inventory.

    eDASS−21: Depression Anxiety Stress Scales-21.

    Risk of Bias Assessments

    For the meta-analysis, we evaluated the risk of bias for the included RCTs, and the results are presented in Multimedia Appendix 6 [22,24,29,32,34,43,44,47-49,62,67,69,74,79,81]. Overall, the included studies showed low risk of bias. Of the 16 RCTs included, 6 (38%) studies were rated “low” risk; 7 (44%) studies were rated “some concerns”; and 3 (19%) studies were rated “high” risk. Owing to the absence of previously published analysis plans, bias in reporting was assessed as “some concerns” for all 16 studies.


    Principal Findings

    To our knowledge, this is the most comprehensive review of the evidence examining the feasibility, acceptability, and efficacy of digital mental health services during the COVID-19 pandemic. The 65 studies identified in the scoping review reported using videoconferencing platforms, web-based programs, smartphone apps, SMS text messaging, social media, hotline, robotic telemedicine, and VR to deliver mental health services among 67,884 participants across 18 countries. Psychotherapy, psychoeducation, and psychological support were among the most common mental health services delivered during the infectious disease outbreak. Overall, digital mental health services delivered during the initial phase of COVID-19 were usable, feasible, and acceptable and improved psychological well-being in participants across the included studies in high-income countries and LMICs. Sixteen RCTs that reported comparable outcomes, including for depression or anxiety, were systematically reviewed and meta-analyzed. The results indicated that digital mental health interventions were significantly associated with a small reduction in depressive symptoms and a moderate reduction in anxiety symptoms. The significance and effect sizes of the interventions differed among the measurements of the depression.

    Our study suggested that the feasibility, acceptability, and effectiveness of digital mental health services during the pandemic were similar to those before the pandemic [82,83], indicating the potential for digital intervention to respond to the mental health needs of the people during the pandemic. These needs include addressing mental health problems among people experiencing COVID-19–related symptoms of anxiety, depression, and stress; patients with suspected or confirmed COVID-19; health workers; and patients with preexisting mental disorders. The advantages of digital mental health services in promoting psychological well-being depend on the ability to overcome the physical distance between people with mental health needs and providers and increasing the accessibility of mental health services. Although the use of digital mental health services has long been advocated, it has not yet been widely adopted because of a lack of support from both service users and health care professionals. In most nations, the situation has changed after the COVID-19 pandemic [84]. In China, for example, increased use of web-based mental health assessments, psychoeducation, and psychological counseling were observed during the COVID-19 pandemic [85].

    Different strategies should be used for applying suitable techniques to digital mental health services. In our review, we found that smartphone app outreach was used by most service users (79.2%). Individuals with mild to moderate mental health symptoms, which may be addressed by psychosocial support and psychological interventions based on a stepped-care approach, are the target populations for the expanded use of smartphone apps to improve the delivery of mental health services. The technology with broad reach could also be used for psychoeducation [46], which is necessary for the support and prevention of mental disorders. Previous research has found that using apps or other web-based treatments for the older adult population can be challenging. Our study found that older adults have positive attitudes toward and may benefit from the electronic device–based mental health services, including videoconferencing platforms [45,62], social media [54], and web-based programs [77]. We suggest that smartphone apps and other web-based interventions should be considered for the older age population if technical assistance could be provided for people with difficulties with these devices. Telephone calls that are easy to use should also be considered for older adults [47].

    The use of digital mental health services was reported across high-income countries and middle-income countries, but there were no studies from low-income countries. Digital mental health services have great potential in LMICs because of the limited availability of mental health services to cover the population needs [86] along with diminished health system capacity during the pandemic. Promising findings were reported for digital mental health services (videoconferencing platforms, web-based programs, and smartphone apps) in most high-income countries and some middle-income countries. However, these techniques rely on the internet and may be difficult to replicate in low-income areas. In addition, limited studies have examined the use of SMS text messaging and telephone calls in middle-income countries. Techniques that do not rely on the internet are a priority in low-resource areas. For instance, owing to their great accessibility and low cost of use, SMS text messaging and hotline might be considered for use in resource-limited or isolated locations. The benefits of delivering digital mental health services in remote and low-resource areas, in terms of potential impact and resource savings, must be considered alongside the feasibility of achieving high coverage of mental health services.

    The finding that digital mental health services are associated with the amelioration of anxiety and depressive symptoms is consistent with previous evidence examining their effects [9,87]. Subgroup and meta-regression analyses allowed us to identify the effects of the study and population characteristics. For instance, there was no evidence of differences in SMD based on sample size. We found that depression measures contributed to heterogeneity. The posttreatment effects of digital mental health interventions differed from Depression Anxiety Stress Scales-21 (DASS-21) and other measurements. Although the current evidence supports the short-term effectiveness of most digital mental health services, a few studies have evaluated long-term outcomes after the interventions [43,69,79]. The long-term benefits of receiving digital mental health services should be assessed through practical evaluations. Along with the viability of reaching high coverage, it is important to weigh the potential impacts and resource savings of providing digital mental health care in rural and underresourced regions.

    Our review sheds light on future research. First, further efforts should be focused on how digital mental health services can be integrated with existing traditional treatment modalities to form a complementary and integrated service model. In particular, the integration of medication and digital services remains to be further validated for safety and feasibility. Second, most studies included in our review were digital psychotherapy interventions, suggesting that digital technologies may have facilitated the accessibility of psychotherapy during the pandemic. We found evidence supporting the efficacious delivery of psychotherapies, either guided [62,77] or self-help therapy [31,34,64,74]. In a study by Al-Alawi et al [18], therapist-guided web-based therapy was associated with greater effectiveness than self-help therapy for people with anxiety and depressive symptoms during the COVID-19 pandemic. Future studies should focus on determining whether people require a guided web-based psychotherapy approach and promote individualized treatment to ensure the optimal allocation of treatment resources. Third, despite the diversity of mental disorders, only a few are covered in this review. Depression and anxiety were the most common outcomes measured. Thus, the role of digital services for many severe mental illnesses, including schizophrenia and bipolar disorder, needs to be further examined.

    Our study has implications for policy development in health care. During a pandemic, digital mental health technology can be used as a backup to address psychological demands and broaden service coverage. We urge policy makers to create future industry regulatory laws so that the digital health sector can grow in a way that provides security and effectiveness testing. The development of digital health infrastructure, including information transmission facilities and technical testing methods for data security issues, is also a crucial prerequisite for the future development of digital mental health. If digital mental health services are to continue, there is a need to specify the minimum level of privacy and security that is acceptable. For example, in the United States, digital mental health services have rapidly made changes in response to policies and regulations concerning confidentiality and privacy to promote telehealth delivery of care [88].

    Limitations

    Our study has several limitations that should be noted. First, owing to the nature of our research question, the scoping review demonstrated substantial heterogeneity. In the pool of included studies, there was a high heterogeneity in terms of study design, type of techniques, type of mental health services, intervention components, target conditions, populations, and outcomes. Most studies did not have comparisons, and their effectiveness remains to be examined in future studies. For meta-analyses, we found only a few factors that contribute to heterogeneity between studies. The subgroup analyses were based on outcomes and limited by the variations in the type of techniques and type of mental health services. As a result, the findings should be interpreted with caution and read in terms of implications for future research.

    Second, most participants in our scoping review were from high-income nations, primarily the United States and European countries. There is an urgent need for more independent research that examines the viability, acceptability, and efficacy of digital mental health as a response to pandemics in LMICs, particularly in low-income countries.

    Third, although the current evidence supports the short-term effectiveness of most digital mental health services, only 2 studies have evaluated the long-term outcomes. A practical evaluation of the long-term benefits of digital mental health services should be undertaken, particularly in urban contexts.

    Fourth, this study did not include preprint servers in the search, which may lead to the overestimation of our findings. However, a methodological research study of systematic reviews and meta-analyses revealed that including unpublished data (such as preprints) may add new sources of bias, although it does not change the results of reviews [89]. In addition, there is a possibility that the findings in preprints will be changed further, which leads to the results of the review being amended as well.

    Finally, the target of this study was to evaluate the usability, feasibility, acceptability, and initial efficacy of digital mental health services in the initial phase of the pandemic. Therefore, we limited the data search to the first 2 years of the pandemic. We recommend that future studies should assess whether these services will remain the same or be further refined during other phases of the pandemic.

    Conclusions

    In summary, we have reported a range of digital mental health services delivered during the initial phase of the COVID-19 pandemic. We found that digital mental health services were usable, feasible, acceptable, and effective in response to mental health needs in the initial phase of pandemic across high-income countries and middle-income countries. Our findings also highlight the amelioration of depressive and anxiety symptoms based on the pooled results of the RCTs. To date, the strategies to lessen the COVID-19 pandemic’s negative impacts on mental health remain a subject of much debate. We suggest that policy makers develop, implement, and assess different strategies for addressing the long-term mental health needs of the pandemic and its aftermath. Digital mental health services as novel strategies for promoting psychological well-being can purposively address physical barriers and provide a promising element for future integrated mental health service models in the post–COVID-19 era.

    Acknowledgments

    This study was funded by the National Natural Science Foundation of China (grant 82101575) and Science and Technology Program of Guangzhou (grant 202102020702).

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklists.

    DOCX File , 54 KB
    ‎
    Multimedia Appendix 2

    Search databases and strategy.

    DOCX File , 34 KB
    ‎
    Multimedia Appendix 3

    Summary of the findings of individual studies included in the qualitative systematic review.

    DOCX File , 145 KB
    ‎
    Multimedia Appendix 4

    Results of the meta-regression for depression and anxiety.

    DOCX File , 324 KB
    ‎
    Multimedia Appendix 5

    Publication bias of meta-analyses.

    DOCX File , 114 KB
    ‎
    Multimedia Appendix 6

    Risk of Bias across randomized studies for meta-analyses.

    DOCX File , 534 KB
    1. COVID-19 Coronavirus pandemic. World-O-Meter.   URL: https://www.worldometers.info/coronavirus/ [accessed 2022-10-05]
    2. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar 14;395(10227):912-920 [FREE Full text] [CrossRef] [Medline]
    3. Moreno C, Wykes T, Galderisi S, Nordentoft M, Crossley N, Jones N, et al. How mental health care should change as a consequence of the COVID-19 pandemic. The Lancet Psychiatry 2020 Sep;7(9):813-824 [FREE Full text] [CrossRef] [Medline]
    4. Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors Associated With Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019. JAMA Netw Open 2020 Mar 02;3(3):e203976 [FREE Full text] [CrossRef] [Medline]
    5. Yao H, Chen J, Xu Y. Patients with mental health disorders in the COVID-19 epidemic. The Lancet Psychiatry 2020 Apr;7(4):e21 [FREE Full text] [CrossRef] [Medline]
    6. Lu Z, Shi L, Que J, Zheng Y, Wang Q, Liu W, et al. Long-term psychological profile of general population following COVID-19 outbreak: symptom trajectories and evolution of psychopathological network. Epidemiol Psychiatr Sci 2022 Sep 27;31:e69 [FREE Full text] [CrossRef] [Medline]
    7. Mak IW, Chu CM, Pan PC, Yiu MG, Chan VL. Long-term psychiatric morbidities among SARS survivors. Gen Hosp Psychiatry 2009;31(4):318-326 [FREE Full text] [CrossRef] [Medline]
    8. Yue J, Yan W, Sun Y, Yuan K, Su S, Han Y, et al. Mental health services for infectious disease outbreaks including COVID-19: a rapid systematic review. Psychol. Med 2020 Nov 05;50(15):2498-2513. [CrossRef]
    9. Karyotaki E, Efthimiou O, Miguel C, Bermpohl FM, Furukawa TA, Cuijpers P, Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis. JAMA Psychiatry 2021 Apr 01;78(4):361-371 [FREE Full text] [CrossRef] [Medline]
    10. 11. Measure performance. Australian Government Digital Transformation Agency.   URL: https:/​/www.​dta.gov.au/​help-and-advice/​digital-service-standard/​digital-service-standard-criteria/​11-measure-performance [accessed 2023-01-04]
    11. Heynsbergh N, Heckel L, Botti M, Livingston PM. Feasibility, useability and acceptability of technology-based interventions for informal cancer carers: a systematic review. BMC Cancer 2018 Mar 02;18(1):244 [FREE Full text] [CrossRef] [Medline]
    12. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
    13. Peters MD, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth 2020 Oct;18(10):2119-2126. [CrossRef] [Medline]
    14. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Cambridge, Massachusetts, United States: Academic Press; 1969.
    15. Deeks J, Higgins J, Altman D. Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions version 6.3. Hoboken, New Jersey: Wiley; 2022.
    16. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2019.
    17. Wei N, Huang B, Lu S, Hu J, Zhou X, Hu C, et al. Efficacy of internet-based integrated intervention on depression and anxiety symptoms in patients with COVID-19. Journal of Zhejiang University 2020 May;21(5):400-404 [FREE Full text] [CrossRef] [Medline]
    18. Al-Alawi M, McCall RK, Sultan A, Al Balushi N, Al-Mahrouqi T, Al Ghailani A, et al. Efficacy of a six-week-long therapist-guided online therapy versus self-help internet-based therapy for COVID-19-induced anxiety and depression: open-label, pragmatic, randomized controlled trial. JMIR Ment Health 2021 Feb 12;8(2):e26683 [FREE Full text] [CrossRef] [Medline]
    19. Guan F, Wu Y, Ren W, Zhang P, Jing B, Xu Z, et al. Self-compassion and the mitigation of negative affect in the era of social distancing. Mindfulness 2021 Jun 28;12(9):2184-2195 [FREE Full text] [CrossRef] [Medline]
    20. Aguilera A, Hernandez-Ramos R, Haro-Ramos AY, Boone CE, Luo TC, Xu J, et al. A text messaging intervention (StayWell at home) to counteract depression and anxiety during COVID-19 social distancing: pre-post study. JMIR Ment Health 2021 Nov 01;8(11):e25298 [FREE Full text] [CrossRef] [Medline]
    21. Agyapong VI, Shalaby R, Hrabok M, Vuong W, Noble JM, Gusnowski A, et al. Mental health outreach via supportive text messages during the COVID-19 pandemic: improved mental health and reduced suicidal ideation after six weeks in subscribers of Text4Hope compared to a control population. Int J Environ Res Public Health 2021 Feb 23;18(4):2157 [FREE Full text] [CrossRef] [Medline]
    22. Al-Refae M, Al-Refae A, Munroe M, Sardella NA, Ferrari M. A self-compassion and mindfulness-based cognitive mobile intervention (Serene) for depression, anxiety, and stress: promoting adaptive emotional regulation and wisdom. Front Psychol 2021 Mar 22;12:648087 [FREE Full text] [CrossRef] [Medline]
    23. Bantjes J, Kazdin AE, Cuijpers P, Breet E, Dunn-Coetzee M, Davids C, et al. A web-based group cognitive behavioral therapy intervention for symptoms of anxiety and depression among university students: open-label, pragmatic trial. JMIR Ment Health 2021 May 27;8(5):e27400 [FREE Full text] [CrossRef] [Medline]
    24. Ben-Zeev D, Chander A, Tauscher J, Buck B, Nepal S, Campbell A, et al. A smartphone intervention for people with serious mental illness: fully remote randomized controlled trial of CORE. J Med Internet Res 2021 Nov 12;23(11):e29201 [FREE Full text] [CrossRef] [Medline]
    25. Brouzos A, Vassilopoulos SP, Baourda VC, Tassi C, Stavrou V, Moschou K, et al. "Staying Home - Feeling Positive": Effectiveness of an on-line positive psychology group intervention during the COVID-19 pandemic. Curr Psychol 2021 Mar 20:1-13 [FREE Full text] [CrossRef] [Medline]
    26. Casas J. Evaluating the feasibility and acceptability of written exposure therapy delivered via telehealth for the treatment of post-traumatic stress disorder. University of Nevada, Reno. 2021.   URL: https:/​/www.​proquest.com/​openview/​22c4a4bc7c2d0a10c02d7b3270b566a2/​1?pq-origsite=gscholar&cbl=18750&diss=y [accessed 2022-09-05]
    27. Chandra M, Rai CB, Sandhu VK, Kumari N, Vishnoi S, Aman S, et al. Mhealth based mental health support counselling service for covid-19 suspect and positive patients in isolation facilities. J Indian Academy Clin Med 2021;22(1-2):6-11.
    28. Coifman KG, Disabato DD, Seah TH, Ostrowski-Delahanty S, Palmieri PA, Delahanty DL, et al. Boosting positive mood in medical and emergency personnel during the COVID-19 pandemic: preliminary evidence of efficacy, feasibility and acceptability of a novel online ambulatory intervention. Occup Environ Med 2021 Apr 26;78(8):555-557. [CrossRef] [Medline]
    29. Comer JS, Furr JM, Del Busto C, Silva K, Hong N, Poznanski B, et al. Therapist-led, internet-delivered treatment for early child social anxiety: a waitlist-controlled evaluation of the iCALM telehealth program. Behav Ther 2021 Sep;52(5):1171-1187. [CrossRef] [Medline]
    30. Craig SL, Leung VW, Pascoe R, Pang N, Iacono G, Austin A, et al. AFFIRM Online: utilising an affirmative cognitive-behavioural digital intervention to improve mental health, access, and engagement among LGBTQA+ youth and young adults. Int J Environ Res Public Health 2021 Feb 05;18(4):1541 [FREE Full text] [CrossRef] [Medline]
    31. Detweiler Guarino I, Cowan DR, Fellows AM, Buckey JC. Use of a self-guided computerized cognitive behavioral tool during COVID-19: evaluation study. JMIR Form Res 2021 May 31;5(5):e26989 [FREE Full text] [CrossRef] [Medline]
    32. Dincer B, Inangil D. The effect of Emotional Freedom Techniques on nurses' stress, anxiety, and burnout levels during the COVID-19 pandemic: a randomized controlled trial. Explore 2021 Mar;17(2):109-114 [FREE Full text] [CrossRef] [Medline]
    33. Esentürk OK, Yarımkaya E. Whatsapp-based physical activity intervention for children with autism spectrum disorder during the novel coronavirus (COVID-19) pandemic: a feasibility trial. Adapt Phys Activ Q 2021 Oct 01;38(4):569-584. [CrossRef] [Medline]
    34. Fiol-DeRoque MA, Serrano-Ripoll MJ, Jiménez R, Zamanillo-Campos R, Yáñez-Juan AM, Bennasar-Veny M, et al. A mobile phone-based intervention to reduce mental health problems in health care workers during the COVID-19 pandemic (PsyCovidApp): randomized controlled trial. JMIR Mhealth Uhealth 2021 May 18;9(5):e27039 [FREE Full text] [CrossRef] [Medline]
    35. Gabrielli S, Rizzi S, Bassi G, Carbone S, Maimone R, Marchesoni M, et al. Engagement and effectiveness of a healthy-coping intervention via chatbot for university students during the COVID-19 pandemic: mixed methods proof-of-concept study. JMIR Mhealth Uhealth 2021 May 28;9(5):e27965 [FREE Full text] [CrossRef] [Medline]
    36. Geoffroy PA, Le Goanvic V, Sabbagh O, Richoux C, Weinstein A, Dufayet G, et al. Psychological support system for hospital workers during the Covid-19 outbreak: rapid design and implementation of the Covid-Psy hotline. Front Psychiatry 2020 May 28;11:511 [FREE Full text] [CrossRef] [Medline]
    37. Gordon JS, Sbarra D, Armin J, Pace TW, Gniady C, Barraza Y. Use of a guided imagery mobile app (See Me Serene) to reduce COVID-19-related stress: pilot feasibility study. JMIR Form Res 2021 Oct 04;5(10):e32353 [FREE Full text] [CrossRef] [Medline]
    38. Graziano S, Boldrini F, Righelli D, Milo F, Lucidi V, Quittner A, et al. Psychological interventions during COVID pandemic: telehealth for individuals with cystic fibrosis and caregivers. Pediatr Pulmonol 2021 Jul 27;56(7):1976-1984 [FREE Full text] [CrossRef] [Medline]
    39. Gromatsky M, Sullivan SR, Mitchell EL, Spears AP, Edwards ER, Goodman M. Feasibility and acceptability of VA CONNECT: caring for our nation's needs electronically during the COVID-19 transition. Psychiatry Res 2021 Feb;296:113700 [FREE Full text] [CrossRef] [Medline]
    40. Guo L, Wu M, Zhu Z, Zhang L, Peng S, Li W, et al. Effectiveness and influencing factors of online education for caregivers of patients with eating disorders during COVID-19 pandemic in China. Eur Eat Disord Rev 2020 Nov 27;28(6):816-825 [FREE Full text] [CrossRef] [Medline]
    41. Held P, Klassen BJ, Coleman JA, Thompson K, Rydberg TS, Van Horn R. Delivering intensive PTSD treatment virtually: the development of a 2-week intensive cognitive processing therapy-based program in response to COVID-19. Cogn Behav Pract 2021 Nov;28(4):543-554 [FREE Full text] [CrossRef] [Medline]
    42. Hom MA, Weiss RB, Millman ZB, Christensen K, Lewis EJ, Cho S, et al. Development of a virtual partial hospital program for an acute psychiatric population: lessons learned and future directions for telepsychotherapy. J Psychother Integration 2020 Jun;30(2):366-382. [CrossRef]
    43. Hosseinzadeh Asl NR. A randomized controlled trial of a mindfulness-based intervention in social workers working during the COVID-19 crisis. Curr Psychol 2022 Aug 09;41(11):8192-8199 [FREE Full text] [CrossRef] [Medline]
    44. Hu J, Cai Z, Ma X. Effects of WeChat-based psychological interventions on the mental health of patients with suspected new coronavirus pneumonia: a pilot study. Jpn J Nurs Sci 2021 Oct 17;18(4):e12429 [FREE Full text] [CrossRef] [Medline]
    45. Ibrahim A, Chong MC, Khoo S, Wong LP, Chung I, Tan MP. Virtual group exercises and psychological status among community-dwelling older adults during the COVID-19 pandemic-a feasibility study. Geriatrics (Basel) 2021 Mar 22;6(1):31 [FREE Full text] [CrossRef] [Medline]
    46. Jaworski BK, Taylor K, Ramsey KM, Heinz A, Steinmetz S, Pagano I, et al. Exploring usage of COVID coach, a public mental health app designed for the COVID-19 pandemic: evaluation of analytics data. J Med Internet Res 2021 Mar 01;23(3):e26559 [FREE Full text] [CrossRef] [Medline]
    47. Kahlon MK, Aksan N, Aubrey R, Clark N, Cowley-Morillo M, Jacobs EA, et al. Effect of layperson-delivered, empathy-focused program of telephone calls on loneliness, depression, and anxiety among adults during the COVID-19 pandemic: a randomized clinical trial. JAMA Psychiatry 2021 Jun 01;78(6):616-622 [FREE Full text] [CrossRef] [Medline]
    48. Karagiozi K, Margaritidou P, Tsatali M, Marina M, Dimitriou T, Apostolidis H, et al. Comparison of on site versus online psycho education groups and reducing caregiver burden. Clin Gerontol 2022 Jul 05;45(5):1330-1340. [CrossRef] [Medline]
    49. Kim YJ. The effect of tele-acupressure self-practice for mental health and wellbeing in the community during COVID-19. Current Psychiatry Res Rev 2021 Jan 15;16(4):267-274. [CrossRef]
    50. Kolbe L, Jaywant A, Gupta A, Vanderlind WM, Jabbour G. Use of virtual reality in the inpatient rehabilitation of COVID-19 patients. Gen Hosp Psychiatry 2021 Jul;71:76-81 [FREE Full text] [CrossRef] [Medline]
    51. Lazzaroni E, Invernizzi R, Fogliato E, Pagani M, Maslovaric G. Coronavirus disease 2019 emergency and remote eye movement desensitization and reprocessing group therapy with adolescents and young adults: overcoming lockdown with the butterfly hug. Front Psychol 2021;12:701381 [FREE Full text] [CrossRef] [Medline]
    52. Lima MR, Wairagkar M, Natarajan N, Vaitheswaran S, Vaidyanathan R. Robotic telemedicine for mental health: a multimodal approach to improve human-robot engagement. Front Robot AI 2021 Mar 18;8:618866 [FREE Full text] [CrossRef] [Medline]
    53. Maldonado J. The effect of an online mental health promotion program during COVID-19 doctor of nursing practice projects. Andrews University. 2020 Oct.   URL: https://digitalcommons.andrews.edu/dnp/10/ [accessed 2022-09-10]
    54. McKeon G, Tiedemann A, Sherrington C, Teasdale S, Mastrogiovanni C, Wells R, et al. Feasibility of an online, mental health-informed lifestyle program for people aged 60+ years during the COVID-19 pandemic. Health Promot J Austr 2022 Jul 19;33(3):545-552. [CrossRef] [Medline]
    55. Nauphal M, Swetlitz C, Smith L, Rosellini AJ. A preliminary examination of the acceptability, feasibility, and effectiveness of a telehealth cognitive-behavioral therapy group for social anxiety disorder. Cognit Behavioral Pract 2021 Nov;28(4):730-742. [CrossRef]
    56. Ortiz R, Southwick L, Schneider R, Klinger EV, Pelullo A, Guntuku SC, et al. Improving mood through community connection and resources using an interactive digital platform: development and usability study. JMIR Ment Health 2021 Feb 26;8(2):e25834 [FREE Full text] [CrossRef] [Medline]
    57. Parolin L, Benzi I, Fanti E, Milesi A, Cipresso P, Preti E. Italia Ti Ascolto [Italy, I am listening]: an app-based group psychological intervention during the COVID-19 pandemic. OSF Preprints. 2021.   URL: https://doi.org/10.4081/ripppo.2021.517 [accessed 2022-08-30]
    58. Philip P, Dupuy L, Morin CM, de Sevin E, Bioulac S, Taillard J, et al. Smartphone-based virtual agents to help individuals with sleep concerns during COVID-19 confinement: feasibility study. J Med Internet Res 2020 Dec 18;22(12):e24268 [FREE Full text] [CrossRef] [Medline]
    59. Puspitasari AJ, Heredia D, Gentry M, Sawchuk C, Theobald B, Moore W, et al. Rapid adoption and implementation of telehealth group psychotherapy during COVID 19: practical strategies and recommendations. Cogn Behav Pract 2021 Nov;28(4):492-506 [FREE Full text] [CrossRef] [Medline]
    60. Rojas SM, Gold SD, Bryan CJ, Pruitt LD, Felker BL, Reger MA. Brief cognitive-behavioral therapy for suicide prevention (BCBT-SP) via video telehealth: a case example during the COVID-19 outbreak. Cogn Behav Pract 2022 May;29(2):446-453 [FREE Full text] [CrossRef] [Medline]
    61. Shalaby R, Vuong W, Hrabok M, Gusnowski A, Mrklas K, Li D, et al. Gender differences in satisfaction with a text messaging program (Text4Hope) and anticipated receptivity to technology-based health support during the COVID-19 pandemic: cross-sectional survey study. JMIR Mhealth Uhealth 2021 Apr 15;9(4):e24184 [FREE Full text] [CrossRef] [Medline]
    62. Shapira S, Yeshua-Katz D, Cohn-Schwartz E, Aharonson-Daniel L, Sarid O, Clarfield AM. A pilot randomized controlled trial of a group intervention via Zoom to relieve loneliness and depressive symptoms among older persons during the COVID-19 outbreak. Internet Interv 2021 Apr;24:100368 [FREE Full text] [CrossRef] [Medline]
    63. Sharrock MJ, Mahoney AE, Haskelberg H, Millard M, Newby JM. The uptake and outcomes of internet-based cognitive behavioural therapy for health anxiety symptoms during the COVID-19 pandemic. J Anxiety Disord 2021 Dec;84:102494 [FREE Full text] [CrossRef] [Medline]
    64. Song J, Jiang R, Chen N, Qu W, Liu D, Zhang M, et al. Self-help cognitive behavioral therapy application for COVID-19-related mental health problems: a longitudinal trial. Asian J Psychiatr 2021 Jun;60:102656 [FREE Full text] [CrossRef] [Medline]
    65. Sosa Lovera A, Ureña AJ, Arias J, Araujo Rodríguez A, Canario Guzmán JA. Psychological helpline in response to the COVID-19 pandemic in the Dominican Republic. Couns Psychother Res 2022 Jun 29;22(2):534-541 [FREE Full text] [CrossRef] [Medline]
    66. Sturgill R, Martinasek M, Schmidt T, Goyal R. A novel artificial intelligence-powered emotional intelligence and mindfulness app (Ajivar) for the college student population during the COVID-19 pandemic: quantitative questionnaire study. JMIR Form Res 2021 Jan 05;5(1):e25372 [FREE Full text] [CrossRef] [Medline]
    67. Suffoletto B, Goldstein T, Gotkiewicz D, Gotkiewicz E, George B, Brent D. Acceptability, engagement, and effects of a mobile digital intervention to support mental health for young adults transitioning to college: pilot randomized controlled trial. JMIR Form Res 2021 Oct 14;5(10):e32271 [FREE Full text] [CrossRef] [Medline]
    68. Summers C, Wu P, Taylor AJ. Supporting mental health during the COVID-19 pandemic using a digital behavior change intervention: an open-label, single-arm, pre-post intervention study. JMIR Form Res 2021 Oct 06;5(10):e31273 [FREE Full text] [CrossRef] [Medline]
    69. Sun S, Lin D, Goldberg S, Shen Z, Chen P, Qiao S, et al. A mindfulness-based mobile health (mHealth) intervention among psychologically distressed university students in quarantine during the COVID-19 pandemic: a randomized controlled trial. J Couns Psychol 2022 Mar;69(2):157-171. [CrossRef] [Medline]
    70. Tarquinio C, Brennstuhl M, Rydberg JA, Bassan F, Peter L, Tarquinio CL, et al. EMDR in telemental health counseling for healthcare workers caring for COVID-19 patients: a pilot study. Issues Ment Health Nurs 2021 Jan 14;42(1):3-14. [CrossRef] [Medline]
    71. Vallefuoco E, Purpura G, Gison G, Bonifacio A, Tagliabue L, Broggi F, et al. A multidisciplinary telerehabilitation approach for supporting social interaction in autism spectrum disorder families: an Italian digital platform in response to COVID-19. Brain Sci 2021 Oct 25;11(11):1404 [FREE Full text] [CrossRef] [Medline]
    72. van Agteren J, Ali K, Fassnacht DB, Iasiello M, Furber G, Howard A, et al. Testing the differential impact of an internet-based mental health intervention on outcomes of well-being and psychological distress during COVID-19: uncontrolled intervention study. JMIR Ment Health 2021 Sep 15;8(9):e28044 [FREE Full text] [CrossRef] [Medline]
    73. Wagner L, Corona LL, Weitlauf AS, Marsh KL, Berman AF, Broderick NA, et al. Use of the TELE-ASD-PEDS for autism evaluations in response to COVID-19: preliminary outcomes and clinician acceptability. J Autism Dev Disord 2021 Sep 30;51(9):3063-3072 [FREE Full text] [CrossRef] [Medline]
    74. Wahlund T, Mataix-Cols D, Olofsdotter Lauri K, de Schipper E, Ljótsson B, Aspvall K, et al. Brief online cognitive behavioural intervention for dysfunctional worry related to the COVID-19 pandemic: a randomised controlled trial. Psychother Psychosom 2021;90(3):191-199 [FREE Full text] [CrossRef] [Medline]
    75. Wasil AR, Taylor ME, Franzen RE, Steinberg JS, DeRubeis RJ. Promoting graduate student mental health during COVID-19: acceptability, feasibility, and perceived utility of an online single-session intervention. Front Psychol 2021 Apr 7;12:569785 [FREE Full text] [CrossRef] [Medline]
    76. Wood HJ, Gannon JM, Chengappa KR, Sarpal DK. Group teletherapy for first-episode psychosis: piloting its integration with coordinated specialty care during the COVID-19 pandemic. Psychol Psychother 2021 Jun 20;94(2):382-389. [CrossRef] [Medline]
    77. Ying Y, Ji Y, Kong F, Chen Q, Lv Y, Hou Y, et al. Internet-based cognitive behavioral therapy for psychological distress in older adults without cognitive impairment living in nursing homes during the COVID-19 pandemic: a feasibility study. Internet Interv 2021 Dec;26:100461 [FREE Full text] [CrossRef] [Medline]
    78. Zepeda MS, Deighton S, Markova V, Madsen JW, Racine N. iCOPE with COVID-19: a brief telemental health intervention for children and adolescents during the COVID-19 pandemic. Cogn Behav Pract 2021 Oct 22 [FREE Full text] [CrossRef] [Medline]
    79. Zhang H, Zhang A, Liu C, Xiao J, Wang K. A brief online mindfulness-based group intervention for psychological distress among chinese residents during COVID-19: a pilot randomized controlled trial. Mindfulness (N Y) 2021;12(6):1502-1512 [FREE Full text] [CrossRef] [Medline]
    80. Zimmerman M, Benjamin I, Tirpak JW, D'Avanzato C. Patient satisfaction with partial hospital telehealth treatment during the COVID-19 pandemic: comparison to in-person treatment. Psychiatry Res 2021 Jul;301:113966 [FREE Full text] [CrossRef] [Medline]
    81. Van Lieshout RJ, Layton H, Savoy CD, Brown JS, Ferro MA, Streiner DL, et al. Effect of online 1-day cognitive behavioral therapy-based workshops plus usual care vs usual care alone for postpartum depression: a randomized clinical trial. JAMA Psychiatry 2021 Nov 01;78(11):1200-1207 [FREE Full text] [CrossRef] [Medline]
    82. Zhang X, Lewis S, Firth J, Chen X, Bucci S. Digital mental health in China: a systematic review. Psychol. Med 2021 Sep 28;51(15):2552-2570. [CrossRef]
    83. Naslund JA, Aschbrenner KA, Araya R, Marsch LA, Unützer J, Patel V, et al. Digital technology for treating and preventing mental disorders in low-income and middle-income countries: a narrative review of the literature. Lancet Psychiatry 2017 Jun;4(6):486-500. [CrossRef]
    84. Shore JH, Schneck CD, Mishkind MC. Telepsychiatry and the coronavirus disease 2019 pandemic-current and future outcomes of the rapid virtualization of psychiatric care. JAMA Psychiatry 2020 Dec 01;77(12):1211-1212. [CrossRef] [Medline]
    85. Liu S, Yang L, Zhang C, Xiang Y, Liu Z, Hu S, et al. Online mental health services in China during the COVID-19 outbreak. Lancet Psychiatry 2020 Apr;7(4):e17-e18. [CrossRef] [Medline]
    86. Merchant R, Torous J, Rodriguez-Villa E, Naslund JA. Digital technology for management of severe mental disorders in low-income and middle-income countries. Curr Opin Psychiatry 2020 Sep;33(5):501-507 [FREE Full text] [CrossRef] [Medline]
    87. Sin J, Galeazzi G, McGregor E, Collom J, Taylor A, Barrett B, et al. Digital interventions for screening and treating common mental disorders or symptoms of common mental illness in adults: systematic review and meta-analysis. J Med Internet Res 2020 Sep 02;22(9):e20581 [FREE Full text] [CrossRef] [Medline]
    88. Bartels SJ, Baggett TP, Freudenreich O, Bird BL. COVID-19 emergency reforms in massachusetts to support behavioral health care and reduce mortality of people with serious mental illness. Psychiatr Serv 2020 Oct 01;71(10):1078-1081. [CrossRef] [Medline]
    89. Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, OPEN consortium. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One 2017;12(4):e0176210 [FREE Full text] [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    DASS-21: Depression Anxiety Stress Scales-21
    LMICs: low- and middle-income countries
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PRISMA-ScR: PRISMA Extension for Scoping Reviews
    RCT: randomized controlled trial
    SMD: standardized mean difference
    VR: virtual reality

    Edited by Y Khader; submitted 22.10.22; peer-reviewed by T Sathish, M Musker, C Leung; comments to author 02.12.22; revised version received 09.01.23; accepted 11.01.23; published 13.02.23

    Copyright

    ©Shaoling Zhong, Xinhu Yang, Zihua Pan, Yu Fan, Yanan Chen, Xin Yu, Liang Zhou. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 13.02.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.