Background: After the rapid spread of the novel SARS-CoV-2, the short-term and long-term mental health impacts of the pandemic on the public, in particular on susceptible individuals, have been reported worldwide. Although digital mental health services expand accessibility while removing many barriers to in-person therapy, their usability, feasibility, acceptability, and efficacy require continued monitoring during the initial phase of the pandemic and its aftermath.
Objective: In this study, we aimed to understand what mental health services are offered, whether they are practical or acceptable, and to what extent digital mental health services are effective in response to the COVID-19 pandemic across high-income and low- and middle-income countries.
Methods: This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the PRISMA Extension for Scoping Reviews (PRISMA-ScR) guideline. We implemented searches in PubMed (MEDLINE), Embase, PsycINFO, and Cochrane databases for studies that were published between December 2019 and November 2021 and that involved the use of digital mental health services. Two review authors screened, assessed, and extracted studies independently. The protocol was registered on the International Prospective Register of Systematic Reviews.
Results: This review identified 7506 articles through database searching. In total, 65 (0.9%) studies from 18 countries with 67,884 participants were eligible for the scoping review. Of the 65 studies, 16 (24.6%) were included in the meta-analysis. A total of 15 (23.1%) studies measured the usability; 31 (47.7%) studies evaluated the feasibility; 29 (44.6%) studies assessed the acceptability; and 51 (78.5%) studies assessed the efficacy. Web-based programs (21/65, 32.3%), videoconferencing platforms (16/65, 24.6%), smartphone apps (14/65, 21.5%), and SMS text messaging (5/65, 7.7%) were the main techniques. Psychotherapy (44/65, 67.7%) followed by psychoeducation (6/65, 9.2%) and psychological support (5/65, 7.7%) were commonly used. The results of the meta-analysis showed that digital mental health interventions were associated with a small reduction in depressive symptoms (standardized mean difference=−0.49; 95% CI −0.74 to −0.24; P<.001) and a moderate reduction in anxiety symptoms (standardized mean difference=−0.66; 95% CI −1.23 to −1.0; P=.02) significantly.
Conclusions: The findings suggest that digital mental health interventions may be practical and helpful for the general population, at-risk individuals, and patients with preexisting mental disorders across high-income and middle-income countries. An expanded research agenda is needed to apply different strategies for addressing diverse psychological needs and develop integrated mental health services in the post–COVID-19 era.
Trial Registration: PROSPERO CRD42022307695; https://tinyurl.com/2jcuwjym
As of October 5, 2022, there were 624.4 million confirmed cases of COVID-19 spread over 228 countries and territories, and COVID-19 has claimed the lives of 6.6 million individuals . After the rapid spread of the novel SARS-CoV-2, known risk factors for mental health impacts have been reported worldwide. Fear of the virus and containment strategies might challenge psychological well-being [ ]. Social isolation, loneliness, unemployment, and loss of income after the incidence of COVID-19 have become common. These risk factors might result in mental health problems, such as anxiety, depression, and insomnia, particularly in susceptible populations that include patients with COVID-19, first-line health professionals, and older adults [ ]. In a cross-sectional study by Lai et al in China [ ], 50% (634/1257) of health care workers reported experiencing depressive symptoms; 45% (560/1257) had anxiety; and 34% (427/1257) reported experiencing insomnia. Increased symptoms of mental health disorders and limited access to mental health care services and social support have also been reported in people with preexisting mental health disorders [ ]. Furthermore, emerging reports suggested the possibility of the long-term effects of pandemics on psychological well-being [ , ].
Although most evidence has been on the negative mental health impacts of COVID-19, a parallel area of research interest has explored how mental health services change when exposed to such stressors . Web-based digital mental health services expand accessibility while removing many barriers to in-person therapy; thus, digital mental health services might be a solution in response to the challenge. Recent efforts in implementing digital mental health services have shown promising applications based on efficacy results from randomized controlled trials (RCTs). In a systematic review [ ], internet-based cognitive behavioral therapy (CBT) was linked to a higher reduction in depressive symptoms at posttreatment than treatment as usual and the waiting list. Usability, feasibility, acceptability, and efficacy results at the time of pandemics, however, require continued monitoring. Knowing what mental health care service is available, whether it is feasible and acceptable, and to what extent digital mental health services take effect is crucial to inform policy decisions, such as the delivery, implementation, and target areas for applying mental health resources during the pandemic and its aftermath.
Against this background, the objective of this scoping review and systematic review was to qualify and quantify the usability, feasibility, acceptability, and efficacy of digital mental health interventions applied for infectious disease outbreaks in the initial phase of the pandemic. We focused on the assessment of whether each of the applied techniques was usable, feasible, acceptable, and effective. We also separately synthesized data from high-income countries and low- and middle-income countries (LMICs). As there is no standard definition of usability, feasibility, acceptability, and efficacy, we considered comprehensive and broad definitions of these terms. We defined usability as program use, user engagement, and the ease of use of the services being tested, as followed in prior research [, ]. We measured feasibility by attrition, attendance, adherence, retention, and qualitative feedback. We defined acceptability as user satisfaction, intent to continue use, and the perceived appropriateness of the intervention [ ]. We defined efficacy as the intended effects and effect size estimation of the services [ ].
This report includes a scoping review of studies on digital mental health interventions applied for infectious disease outbreaks and a systematic review of studies assessing commonly reported mental distresses identified in the scoping review. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the PRISMA Extension for Scoping Reviews (PRISMA-ScR) guideline when appropriate ().
We sought to include original studies focusing on digital mental health services applied during or in response to COVID-19. We implemented searches in PubMed (MEDLINE), Embase, PsycINFO, and Cochrane between December 2019 and November 2021 using a combination of search terms relating to both digital mental health services and COVID-19. The complete search strategy is provided in. We performed a manual search of the reference lists of relevant reviews, eligible studies, and relevant conference abstracts.
We removed duplicate citations across electronic databases before the screening. Initially, pairs of 2 review authors (SZ, XY, ZP, and YF) independently screened the titles and abstracts for eligibility. We referred to full texts if titles and abstracts did not provide information for eligibility based on the inclusion and exclusion criteria. The full texts of the potentially included studies were screened by 1 review author and checked by another author for agreement. In case of disagreements, the opinion of a third reviewer was sought, and a consensus was reached through discussion.
We included qualitative, quantitative, and mixed methods studies that involved the use of digital mental health services. Eligible interventions included health education, evaluation, consulting, helplines, psychological interventions, and telemedicine. The intervention was delivered by any means of digital technology, including SMS text messaging, smartphones, websites, social media, videoconferencing, wearable devices, or mobile apps. We excluded comments, protocol papers, systematic reviews, preprints, conference papers, and studies that did not provide any qualitative or quantitative data for defined outcomes. We excluded studies that were conducted before December 2019. There was no exclusion for age, gender, ethnicity, socioeconomic status, professions (health workers or not), or any type of mental disorder (clinical or nonclinical). For the meta-analysis, we included parallel RCTs and crossover RCTs that provided adequate data for analyses, using the same aforementioned criteria.
We developed a data extraction form that could be used by all review authors, as recommended by the Joanna Briggs Institute Scoping Review methodological guidance . The data extraction form included information on study characteristics (authors, publication year, study design, study period, geographic regions, and study aim), participant characteristics (age, gender, sample size, and type of mental illness), digital mental health services characteristics (types of techniques and type of mental health services), and outcome characteristics (type of outcomes and their measurements). Owing to the diverse outcomes identified across eligible studies, we extracted the study results in 2 phases. In phase 1, we extracted the key findings for all reported outcomes. For effectiveness data, we extracted the results for significant differences between digital mental health services and comparison groups or trends between postintervention and preintervention outcome measurements for digital mental health services. In phase 2, we identified RCTs that reported similar digital mental health services and outcomes for the systematic review and meta-analysis. Here, effectiveness data from the comparison groups and intervention arms were retrieved using separate extraction forms. The data were extracted by 2 reviewers (XY and ZP) independently. Disagreements were resolved by discussion. A third reviewer (SZ) was involved when necessary.
Risk of Bias Assessment
Two reviewers (XY and ZP) independently assessed the risk of bias using the Cochrane Risk of Bias (RoB 2) tool for all included RCTs. Disagreements were resolved by discussions. In case of uncertainties, a third reviewer (SZ) was consulted. The 5 domains included bias owing to the assignment of intervention, adherence to the intervention, missing outcome data, outcome measurement, and selection of reported outcome that were measured. Studies were rated as “low” risk, “some concerns,” or “high” risk.
Owing to the nature of the research question and the heterogeneity of the studies, we synthesized qualitative and quantitative results separately. For the scoping review, we descriptively summarized the included studies with a narrative synthesis of individual studies. We summarized the overall number of included studies, the total number of study populations and their basic characteristics, countries, study design, type of digital mental health services, and the objectives and findings of the included studies. We clustered the studies by the type of techniques used and location (high-income countries vs LMICs). For each type of technique, the findings on usability, feasibility, acceptability, and effectiveness data were narratively synthesized by a textual approach.
For meta-analyses, we synthesized findings on similar efficacy outcomes that were measured by >3 RCTs. Outcomes, such as loneliness, burnout, or psychological well-being, that were measured by 1 or 2 RCTs were not synthesized. We pooled continuous outcomes as standardized mean differences (SMD) with 95% of CIs using a random-effects meta-analysis. The values of 0.2, 0.5, and 0.8 in SMDs denote minor, medium, and large effects, respectively . Cochran Q and the I2 statistic were used to assess heterogeneity. An I2 value of ≥60% can be considered a substantial level of inconsistency across studies [ ]. The analyses were stratified according to the reported outcomes. For each of these, we pooled the evidence from all eligible RCTs, regardless of the techniques applied, measurements used, and population studied. We conducted subgroup analyses to examine whether there were differences in outcomes on the basis of sample size (as a dichotomous variable), measurement of the outcome, location, duration of follow-up, and duration of intervention. We also performed meta-regression analyses to explore the effect size by sample size (as a continuous variable), age, and proportion of females in the population studied. We used funnel plots and ran the Egger test to look for asymmetry to detect publication bias that might impair the validity of our findings. We performed all analyses with R statistical software (version 4.2.2; R Foundation for Statistical Computing) [ ].
The PRISMA flowchart shows the study selection process (). This review identified 7506 articles that were identified through database searching. After removing duplicate records, we screened 5118 (68.19%) records and identified 65 (1.27%) eligible studies for the scoping review, and the results were synthesized narratively. Of the 19 RCTs, 17 (89%) studies reported depressive symptoms and 18 (95%) studies reported anxiety symptoms as the outcome. In total, 3 (16%) studies were not included in the meta-analyses; 2 (67%) studies [ , ] that reported on depression and anxiety disorders did not provide adequate data, and 1 (33%) study that reported anxiety scores using state anxiety was with a very large estimate [ ]. Accordingly, of the total 65 eligible studies, 16 (25%) studies that reported depressive symptoms or anxiety symptoms were included in the meta-analysis.
presents the characteristics of participants and studies included in the review. The 65 included studies were conducted in the following 18 different countries: 24 (37%) studies from the United States (participants: 54,245/67,884, 79.9%), 8 (12%) studies from China (1114/67,884, 1.6%), 6 (9%) studies from Canada (5490/67,884, 8.1%), 5 (8%) studies from Italy (315/67,884, 0.5%), 3 (5%) studies from Australia (1004/67,884, 1.5%), 3 (5%) studies from France (2235/67,884, 3.3%), 2 (3%) studies from India (654/67,884, 1%), 2 (3%) studies from Malaysia (123/67,884, 0.2%), 2 (3%) studies from Greece (164/67,884, 0.2%), 2 (3%) studies from Turkey (121/67,884, 0.2%), and further 9 (14%) studies were conducted in other countries (ie, Dominican Republic, Israel, Japan, Oman, South Africa, Spain, Sweden, and the United Kingdom). Of these, 49 (75%) studies were from high-income countries, and 16 (25%) studies were conducted in middle-income countries. None of the included studies reported data from low-income countries. The participant pool consisted of 67,884 participants; most (44,125/67,884, 65%) were female. There were 2663 (3.9%) people with a mental illness or mental distress, 713 (1.1%) people with or suspected of COVID-19, and 748 (1.1%) hospital workers.
Of the 65 studies, a total of 15 (23.1%) studies measured usability; 31 (47.7%) evaluated feasibility; 29 (44.6%) assessed acceptability; and 51 (78.5%) assessed efficacy.
Web-based programs (21/65, 32.3%), videoconferencing platforms (16/65, 24.6%), smartphone apps (14/65, 21.5%), and SMS text messaging (5/65, 7.7%) were the main techniques applied. There were other studies that used social media (3/65, 4.6%), hotlines (3/65, 4.6%), a chatbot (1/65, 1.5%), virtual reality (VR; 1/65, 1.5%), and robotic telemedicine (1/65, 1.5%). Most studies provided psychotherapy (44/65, 67.7%), followed by psychoeducation (6/65, 9.2%), psychological support (5/65, 7.7%), psychological assessments (4, 6.1%), and psychiatric clinical services (4/65, 6.1%).
|Author and year||Design and study type||Sample size, n||Age (years), mean (SD)a||Female, %||Intervention vs control group|
|Aguilera et al ||Pre- and postintervention evaluation; quantitative||303||33.3 (11.0)||78||A texting intervention (StayWell at Home)|
|Agyapong et al ||Pre- and postintervention evaluation; quantitative||2767||76.7% were aged between 26 and 60||88||Text4Hope subscribers who received once-daily supportive SMS text messages vs Text4Hope subscribers who did not receive messages|
|Al-Alawi et al ||RCTb; quantitative||46||28.51 (8.7)||78||Therapist-guided web-based therapy vs newsletter received via email containing self-help information and tips to cope with distress|
|Al-Refae et al ||RCT; quantitative||165||25.24 (8.74)||78.8||Smartphone intervention (Serene) vs waiting list|
|Bantjes et al ||Pre and postintervention evaluation; quantitative||175||22.2 (4.9)||85.4||A web-based group CBTc intervention|
|Ben-Zeev et al ||RCT; quantitative||315||37.89 (11.64)||83.8||Smartphone app (CORE) vs waiting list|
|Brouzos et al ||Pre- and postintervention evaluation; quantitative||82||33.07 (9.55)||78||Web-based positive psychology intervention (“Staying Home—Feeling Positive”) vs no intervention|
|Casas ||Pre- and postintervention evaluation; mixed methods||15||39.52 (13.57)||73.3||Written exposure therapy delivered via telehealth|
|Chandra et al ||Pre- and postintervention evaluation; mixed methods||643||49.57 (15.23)||37.2||Mental health support telecounseling|
|Coifman et al ||Pre- and postintervention evaluation; quantitative||28||45.33 (9.6)||75||Daily coping toolkit intervention|
|Comer et al ||RCT; quantitative||40||Children=6.2 (1.8); parents=38.39 (4.5)||Children: 72.5; parents: 75||iCALM Telehealth Program vs waiting list|
|Craig et al ||Pre- and postintervention evaluation; mixed methods||96||22.3 (4.1)||17.7||AFFIRM online group vs waiting list|
|Detweiler Guarino et al ||Pre- and postintervention evaluation; quantitative||2484||44 (15.2)||53.2||The PATH program|
|Dincer and Inangil ||RCT; quantitative||72||33.46 (9.6)||88.9||Received emotional freedom techniques vs stayed comfortable in a calm and tranquil environment|
|Esentürk and Yarımkaya ||Pre- and postintervention evaluation; mixed methods||14||Children=12.07; parents=51.4||Children=42.9; parents=64.2||WhatsApp-based physical activity intervention|
|Fiol-DeRoque et al ||RCT; quantitative||482||41.37 (10.4)||83.2||PsyCovidApp vs a general control app|
|Gabrielli et al ||Pre- and postintervention evaluation; mixed methods||71||20.6 (2.4)||68||Healthy-coping intervention via chatbot|
|Geoffroy et al ||Postintervention evaluation;||149||32.7 (11.0)||86||Covid-Psy hotline|
|Gordon et al ||Pre- and postintervention evaluation; mixed methods||99||51.4 (14.7)||86||A guided imagery mobile app|
|Graziano et al ||Pre- and postintervention evaluation; quantitative||30||Patients=22.5 (6.9); parents=37 (6.3)||Patient=56.3; parents=92.9||Telehealth psychological interventions|
|Gromatsky et al ||Pre- and postintervention evaluation; mixed methods||20||54.2 (11.95)||15||VA CONNECT|
|Guan et al ||RCT; quantitative||95||18.63 (0.75)||28.4||Self-compassion writing induction condition vs a writing control condition (to think about a negative event)|
|Guo et al ||Pre- and postintervention evaluation; quantitative||508||Intervention=47.0 (4.3), control=46.6 (4.2)||84.1||Web-based education program vs no intervention|
|Held et al ||Pre- and postintervention evaluation; mixed methods||2||—d||0||Intensive cognitive processing therapy–based program|
|Hom et al ||Postintervention evaluation; mixed methods||23||39.04 (16.15)||52.2||Virtual partial hospital program|
|Hosseinzadeh Asl ||RCT; quantitative||49||33.06 (6.02)||55.1||Mindfulness-based intervention vs waiting list|
|Hu et al ||Pilot RCT; quantitative||48||42.6 (15.8)||54.2||WeChat-based psychological interventions vs conventional nursing|
|Ibrahim et al ||Pre- and postintervention evaluation; quantitative||43||63.41||67.4||Virtual group exercise|
|Jaworski et al ||Postintervention evaluation; quantitative||49,287||—||—||COVID coach|
|Kahlon et al ||RCT; quantitative||240||69.1 (12.1)||79||Receive calls vs no calls|
|Karagiozi et al ||RCT; quantitative||82||44.5 (10.6)||—||Web-based intervention group vs onsite intervention|
|Kim ||RCT; quantitative||80||41.7 (13.8)||56.25||Teleacupressure self-practice group vs web-based communication group|
|Kolbe et al ||Postintervention evaluation; mixed methods||13 patients and 11 staff||—||—||Virtual reality|
|Lazzaroni et al ||Pre- and postintervention evaluation; quantitative||50||Between 13 and 24||84||Eye movement desensitization and reprocessing intervention group|
|Lima et al ||Postintervention evaluation; mixed methods||Healthy volunteers=10; older adults and people with dementia=1||Healthy volunteers=21-59; older adults and people with dementia=not mentioned||Healthy volunteers=50; older adults and people with dementia=0||Robotic telemedicine|
|Maldonado ||Pre- and postintervention evaluation; quantitative||40||44.1 (13)||82.5||Online intervention group|
|McKeon et al ||Pre- and postintervention evaluation; quantitative||11||64.4 (4.3)||82||Mental health–informed lifestyle intervention|
|Nauphal et al ||Pre- and postintervention evaluation; quantitative||5||36.2 (9.5)||40||Web-based CBT group|
|Ortiz et al ||Pre- and postintervention evaluation; mixed methods||383||46.9 (16.8)||70.1||Texting intervention group|
|Parolin et al ||Pre- and postintervention evaluation; mixed methods||134||33.20 (10.61)||86||Italia Ti Ascolto intervention group|
|Philip et al ||Pre- and postintervention evaluation; quantitative||2069||43.52 (13.94)||65.1||KANOPEE group|
|Puspitasari et al ||Pre- and postintervention evaluation; quantitative||76||36.6 (13.4)||86||Adult Transitions Program teletherapy group|
|Rojas et al ||Pre- and postintervention evaluation; mixed methods||1||—||0||Brief CBT for suicide prevention|
|Shalaby et al ||Postintervention evaluation; quantitative||2032||44.58 (13.45)||88||Text4Hope intervention group|
|Shapira et al ||Pilot RCT; quantitative||82||72 (5.63)||—||A short-term digital group vs a waitlist control group|
|Sharrock et al ||Pre- and postintervention evaluation; quantitative||904||37.83 (12.64)||67.1||iCBTe group during COVID-19 vs iCBT group before COVID-19|
|Song et al ||Pre- and postintervention evaluation; quantitative||129||34.64 (9.11)||69||Mobile internet CBT group vs a waitlist control group|
|Sosa Lovera et al ||Postintervention evaluation; quantitative||497||32||73||COVID-19 helpline group|
|Sturgill et al ||Pre- and postintervention evaluation; quantitative||99||19.9 (1.94)||69||Ajivar activities intervention + routine mental wellness instruction group vs group that received routine mental wellness instruction|
|Suffoletto et al ||Pilot RCT; quantitative||52||18.7 (0.46)||86.5||MoST-MHf Intervention vs enhanced usual care group|
|Summers, Wu, and Taylor ||Pre- and postintervention evaluation; quantitative||273||49.6 (9.24)||59.3||Gro Health app users|
|Sun et al ||Pilot RCT; mixed methods||114||22.21 (2.67)||73.7||Mindfulness-based mHealthg group vs social support–based mHealth group|
|Tarquinio et al ||Pre- and postintervention evaluation; quantitative||17||33.2 (4.10)||100||Eye Movement Desensitization and Reprocessing intervention group|
|Vallefuoco et al ||Postintervention evaluation; quantitative||30||—||—||18 therapists and 12 parents of children with ASDh|
|van Agteren et al ||Pre- and postintervention evaluation; quantitative||89||38.67 (13.06)||66||Internet-based group mental health intervention|
|Wagner ||Postintervention evaluation; mixed methods||Children: 204; clinicians: 9||Children=27.54 months (5.36 months)||23||ASD-PEDS intervention group|
|Wahlund et al ||RCT; mixed methods||670||46 (13.50)||81.6||Web-based psychological intervention vs waiting list|
|Wasil et al ||Pre- and postintervention evaluation; quantitative||189||31.04 (8.91)||72.99||COMET intervention|
|Wei et al ||RCT; quantitative||26||44.65 (12.09)||38.5||An internet-based integrated intervention vs support care|
|Wood et al ||Postintervention evaluation; mixed methods||7||26.9 (4.8)||—||Group teletherapy|
|Ying et al ||Pre- and postintervention evaluation; quantitative||137||73.39 (7.37)||68.5||iCBT group|
|Zepeda et al ||Pre- and postintervention evaluation; mixed methods||27||9.56||67||iCOPE|
|Zhang et al ||Pilot RCT; quantitative||57||50.12 (6.79)||47.1||WeChat-based psychological interventions vs waitlist group|
|Zimmerman et al ||Postintervention evaluation; quantitative||480||37.1 (14)||67.5||Telehealth program vs partial in-person program before the COVID-19 outbreak|
|Van Lieshout et al ||RCT; quantitative||403||31.8 (4.40)||100||Web-based CBT + treatment as usual vs treatment as usual|
aWhere mean (SD) was not available in the original study, mean or ranges were used where applicable.
bRCT: randomized controlled trial.
cCBT: cognitive behavioral therapy.
eiCBT: internet-based congnitive behavioral therapy.
fMoST-MH: Mobile Support Tool for Mental Health.
gmHealth: mobile health.
hASD: autism spectrum disorder.
Qualitative Synthesis of Results
This section summarizes the findings from 65 studies that reported the feasibility, acceptability, and effectiveness of digital mental health services during the COVID-19 pandemic. The synthesis was organized according to the type of digital technique used.presents a summary of usability, feasibility, acceptability, and efficacy by the types of techniques in high-income countries and LMICs. Table S1 ( [ - ]) presents summaries of individual studies on mental health interventions and major findings and their measurements.
|Techniques and groups||Usability||Feasibility||Acceptability||Efficacy|
|Hotline and telephone calls|
|Robotic telemedicine and VRb|
aNo evidence found.
bVR: virtual reality.
Overall, 21 (32.3%; 5480/67,884, 8.1% out of total participants) out of the 65 studies included in this scoping review used web-based programs. In 17 studies that were conducted in high-income countries, a significant increase in registered users (n=2) and a high rating in usability (n=1) were reported. The completion rate ranged from 30.5% to 85% (n=5); the retention rate ranged from 30.5% to 85% (n=3); and the attrition rate was 13% (n=1). Quality checks of feasibility reported strong attendance (n=1), strong retentions (n=2), and high ratings of feasibility (n=2). Studies presented preliminary support for the feasibility of web-based programs yet with possible technical difficulty (n=1). In terms of acceptability, there was high satisfaction (n=5); the interventions were perceived to be acceptable, helpful, appropriate, and positive (n=3); and the acceptability scores were significantly better than average (n=1). Most studies, either when no control group is involved or when compared with the control group, reported improvements in anxiety symptoms and anxiety-related social impairment (n=8), while no effects on reducing anxiety were reported as well (n=2). The same patterns were found in outcomes measuring depression. Most studies reported improvements in depressive symptoms and depression severity (n=5), and there was no difference in depressive symptoms when the intervention group was compared with onsite groups (n=1). Improvements in stress-related symptoms (n=3), COVID-19–related worry (n=2), and insomnia (n=1) were identified.
In 4 studies conducted in LMICs, the mean time spent was 35.6 (SD 25.4) minutes (n=1) and the completion rate was 87.4% (n=1). High attendance and high retention rates (91.2%) were reported (n=1). With regard to acceptability, high ratings of acceptability and a high level of satisfaction (n=2) were reported. Studies, either when no control group is involved or when compared with the waitlist group, reported improvements in anxiety symptoms (n=4); some studies reported improvements in depressive symptoms (n=2), and 1 study reported reduction in stress and burnout.
Among the 65 studies, 16 (24.6%; participants: 1973/67,884, 2.9%) used teleconferencing platforms, including Zoom [, , , , , , , , , ], Skype [ ], Microsoft Teams [ , ], Google Meet [ ], Tencent Meeting [ ], and other platforms [ , ]. In 11 studies that were conducted in high-income countries, a completion rate between 92% and 100% (n=3), qualitative feedback of high perceived benefits (n=3), and technology-related challenges (n=3) were reported in terms of feasibility. High levels of satisfaction were reported (n=5). Studies reported improvements in depressive levels (n=9) and anxiety symptoms (n=5). No effects on decreasing anxiety symptoms were reported in one study. In 5 studies conducted in LMICs, the mean attendance ranged from 6.4 (SD 2.8) to 10.26 (SD 7.02; n=2). High levels of satisfaction (n=1) were reported. There were improvements in depressive symptoms and negative affect (n=3), yet no effects on decreasing depression levels were identified (n=2). No evidence was found in usability for LMICs or high-income countries.
Of the 65 studies, 14 (21.5%; participants: 53,786/67,884; 79.2%) developed smartphone apps to deliver mental health services. In 11 studies conducted in high-income countries, studies reported high usability scores (n=2), and user engagement time ranged from 36.7 minutes over 12 weeks to 1424 minutes over 14 weeks (n=2). Feasibility was reported in terms of the average number of days retained (mean 42.4, SD 44.4; n=1), and the retention rate ranged from 28.3% to 85.7% (n=3). In terms of acceptability, the interventions were rated as “acceptable” (n=2), “satisfied” (n=2), “good” (n=1), and “helpful” (n=1). Studies reported a reduction in depressive symptoms compared with the preintervention and waitlist group (n=4). Similar improvements were reported in anxiety symptoms compared with preintervention, waitlist group, and a control app with limited access to psychoeducational contents (n=5). In 4 studies in LMICs, no evidence was found for usability and feasibility. Acceptability was reported as high satisfaction (n=1) and helpful and enjoyable (n=1). There were improvements in depressive symptoms, insomnia, psychological flexibility, and self-compassion, either when no control group is involved or when compared with the waitlist group (n=2).
SMS Text Messaging
Of the 65 studies, 5 (7.7%; participants: 5537/67,884, 8.1%) used the SMS text messaging technique. All studies were conducted in high-income countries. Studies reported high usability of ecological momentary intervention (n=1), and the completion rate ranged from 16% to 78%. An overall high satisfaction was reported (n=1). Studies suggested that an improved mood rating was observed at posttreatment assessments either with or without the control group (n=4). No evidence was found of studies conducted in LMICs for the technique of SMS text messaging.
The use of social media as a technique to deliver mental health services was described in 3 (4.6%; participants: 116/67,884, 0.2%) out of 65 studies [, , ]. In one study conducted in high-income countries, high retention, high acceptability, and an effect on multiple outcomes (psychological distress, quality of life, functioning, loneliness, and physical activity) were reported (n=1). In the 2 studies conducted in LMICs, an average user time of 18.7 hours (n=1) was reported. Evidence suggested improvements in anxiety and depressive symptoms when compared with either usual care or the waitlist group (n=2). No evidence was found for other outcomes.
Hotline and Telephone Calls
Of the 65 studies, 3 (4.6%; participants: 886/67,884, 1.3%) reported using hotlines and telephone calls to deliver mental health services during the COVID-19 pandemic [, , ]. In 2 studies conducted in high-income countries, an average of 5.73 (SD 3.22) calls per day was reported (n=1). A dropout rate of 7.5% was reported (n=1), and there were improvements in loneliness, depression, anxiety, and general mental health, as reported in the same study (n=1). In one study that was conducted in LMICs, participants reported high satisfaction and mentioned that they would use the service again and would recommend it to others (n=1). No evidence was found in terms of usability, feasibility, or efficacy for the techniques of hotline and telephone calls by the studies conducted in LMICs.
Robotic Telemedicine and VR
Of the 65 studies, 3 (4.6%; participants: 106,67,884, 0.2%) used robotic telemedicine and VR. Two studies conducted in high-income countries reported an average use of 78 (SD 24.8) times (n=1) and a completion rate of 58% (n=1). High satisfaction was reported (n=1). Evidence suggested a decrease in anxiety symptoms and stress symptoms for all participants after the intervention (n=1). In 1 study conducted in LMICs, an overall positive impression of the multimodal robotic system was reported (n=1), yet the need to adjust some features was suggested (n=1). No evidence for acceptability or efficacy was found by studies conducted in LMICs.
Of the 65 studies, 15 (23.1%) that reported depressive symptoms were included in the meta-analysis. Digital mental health interventions were associated with a small significant reduction in depressive symptoms (SMD −0.49, 95% CI −.74 to −.24; P<.001;) with substantial heterogeneity (I2=87%, 95% CI 80%-91%; Q=107.57; P<.001).
The effectiveness of digital mental health interventions on anxiety symptoms was reported in 15 RCTs. The pooled SMD showed a moderate and significant effect of these interventions in reducing anxiety symptoms (SMD=−0.66, 95% CI −1.23 to −.10; P=.02;) with substantial heterogeneity (I2=93%, 95% CI 90%-95%; Q=207.22; P<.001).
We examined the sources of heterogeneity by studying covariates across studies (and ). Subgroup analyses revealed that reduced depressive symptoms were associated with the measurement of depression (using Depression Anxiety Stress Scales-21: SMD=−0.13, 95% CI −0.31 to 0.06 vs other measurements: SMD=−0.61, 95% CI −0.91 to −0.30; Q=7.04; P=.008). Reduced anxiety symptoms were not associated with sample size, location, measurement of anxiety, duration of follow-up, or the duration of intervention.
Meta-regression analyses are reported in. There was no significant association between SMDs of depression and sample size, age, and proportion of females in the study population. Meta-regression analyses did not reveal differences in the SMDs of anxiety depending on sample size, age, and proportion of females in the study population.
For the main analyses of depression and anxiety, we observed evidence of funnel plot asymmetry, which could suggest no publication bias (Egger test: P=.57 and P=.50, respectively;).
|Group||Studies, n||SMDa (95% CI)||Q||P value|
|≥81||8||−0.51 (−0.86 to −0.16)|
|<81||7||−0.46 (−0.84 to −0.09)|
|Measurement of depression||7.04||.008|
|DASS-21b||4||−0.13 (−0.31 to 0.06)|
|Other measurementsc||11||−0.61 (−0.91 to −0.30)|
|High-income countries||11||−0.53 (−0.83 to −0.24)|
|Low- and middle-income countries||4||−0.35 (−0.82 to 0.10)|
|Duration of follow-up||1.23||.27|
|Short-term||12||−0.55 (−0.84 to −0.26)|
|Long-term||3||−0.28 (−0.66 to −0.09)|
|Duration of intervention||0.84||.36|
|≥4 weeks||10||−0.56 (−0.90 to −0.23)|
|<4 weeks||5||−0.34 (−0.68 to −0.02)|
aSMD: standardized mean difference.
bDASS−21: Depression Anxiety Stress Scales-21.
cOther measurements include Beck Depression Inventory-II, Brief Symptom Inventory, College Counseling Center Assessment of Psychological Symptoms, Edinburgh Postnatal Depression Scale, Hamilton Rating Scale for Depression, Montgomery Åsberg Depression Rating Scale—Self rated, Patient Health Questionnaire, and Self-rating Depression Scale.
|Group||Studies, n||SMDa (95% CI)||Q||P value|
|≥81||7||−0.54 (−1.01 to −0.07)|
|<81||8||−0.85 (−1.99 to 0.29)|
|High-income countries||10||−0.49 (−0.66 to −0.13)|
|LMICsb||5||−1.17 (−3.02 to 0.68)|
|Measurement of anxiety1||0||.95|
|GAD-7c||5||−0.68 (−1.31 to −0.04)|
|Other measurementsd||10||−0.71 (−1.61 to 0.19)|
|Measurement of anxiety2||2.06||.15|
|DASS-21e||4||−0.23 (−0.38 to −0.08)|
|Other measurements||11||−0.85 (−1.68 to −0.02)|
|Duration of follow-up||1.14||.29|
|Short-term||12||−0.80 (−1.55 to −0.06)|
|Long-term||3||−0.29 (−0.86 to 0.29)|
|Duration of intervention||0.58||.45|
|≥4 weeks||9||−0.47 (−0.91 to −0.04)|
|<4 weeks||6||−1.07 (−2.56 to 0.41)|
a SMD: standardized mean difference.
bLMICs: low- and middle-income countries.
cGAD-7: Generalized Anxiety Disorder-7.
dOther measurements including Beck Anxiety Inventory, Brief Symptom Inventory, Child Behavior Checklist, College Counseling Center Assessment of Psychological Symptoms, Hamilton Anxiety Rating Scale, Self-rating Anxiety Scale, and State-Trait Anxiety Inventory.
eDASS−21: Depression Anxiety Stress Scales-21.
Risk of Bias Assessments
For the meta-analysis, we evaluated the risk of bias for the included RCTs, and the results are presented in[ , , , , , , , - , , , , , , ]. Overall, the included studies showed low risk of bias. Of the 16 RCTs included, 6 (38%) studies were rated “low” risk; 7 (44%) studies were rated “some concerns”; and 3 (19%) studies were rated “high” risk. Owing to the absence of previously published analysis plans, bias in reporting was assessed as “some concerns” for all 16 studies.
To our knowledge, this is the most comprehensive review of the evidence examining the feasibility, acceptability, and efficacy of digital mental health services during the COVID-19 pandemic. The 65 studies identified in the scoping review reported using videoconferencing platforms, web-based programs, smartphone apps, SMS text messaging, social media, hotline, robotic telemedicine, and VR to deliver mental health services among 67,884 participants across 18 countries. Psychotherapy, psychoeducation, and psychological support were among the most common mental health services delivered during the infectious disease outbreak. Overall, digital mental health services delivered during the initial phase of COVID-19 were usable, feasible, and acceptable and improved psychological well-being in participants across the included studies in high-income countries and LMICs. Sixteen RCTs that reported comparable outcomes, including for depression or anxiety, were systematically reviewed and meta-analyzed. The results indicated that digital mental health interventions were significantly associated with a small reduction in depressive symptoms and a moderate reduction in anxiety symptoms. The significance and effect sizes of the interventions differed among the measurements of the depression.
Our study suggested that the feasibility, acceptability, and effectiveness of digital mental health services during the pandemic were similar to those before the pandemic [, ], indicating the potential for digital intervention to respond to the mental health needs of the people during the pandemic. These needs include addressing mental health problems among people experiencing COVID-19–related symptoms of anxiety, depression, and stress; patients with suspected or confirmed COVID-19; health workers; and patients with preexisting mental disorders. The advantages of digital mental health services in promoting psychological well-being depend on the ability to overcome the physical distance between people with mental health needs and providers and increasing the accessibility of mental health services. Although the use of digital mental health services has long been advocated, it has not yet been widely adopted because of a lack of support from both service users and health care professionals. In most nations, the situation has changed after the COVID-19 pandemic [ ]. In China, for example, increased use of web-based mental health assessments, psychoeducation, and psychological counseling were observed during the COVID-19 pandemic [ ].
Different strategies should be used for applying suitable techniques to digital mental health services. In our review, we found that smartphone app outreach was used by most service users (79.2%). Individuals with mild to moderate mental health symptoms, which may be addressed by psychosocial support and psychological interventions based on a stepped-care approach, are the target populations for the expanded use of smartphone apps to improve the delivery of mental health services. The technology with broad reach could also be used for psychoeducation , which is necessary for the support and prevention of mental disorders. Previous research has found that using apps or other web-based treatments for the older adult population can be challenging. Our study found that older adults have positive attitudes toward and may benefit from the electronic device–based mental health services, including videoconferencing platforms [ , ], social media [ ], and web-based programs [ ]. We suggest that smartphone apps and other web-based interventions should be considered for the older age population if technical assistance could be provided for people with difficulties with these devices. Telephone calls that are easy to use should also be considered for older adults [ ].
The use of digital mental health services was reported across high-income countries and middle-income countries, but there were no studies from low-income countries. Digital mental health services have great potential in LMICs because of the limited availability of mental health services to cover the population needs  along with diminished health system capacity during the pandemic. Promising findings were reported for digital mental health services (videoconferencing platforms, web-based programs, and smartphone apps) in most high-income countries and some middle-income countries. However, these techniques rely on the internet and may be difficult to replicate in low-income areas. In addition, limited studies have examined the use of SMS text messaging and telephone calls in middle-income countries. Techniques that do not rely on the internet are a priority in low-resource areas. For instance, owing to their great accessibility and low cost of use, SMS text messaging and hotline might be considered for use in resource-limited or isolated locations. The benefits of delivering digital mental health services in remote and low-resource areas, in terms of potential impact and resource savings, must be considered alongside the feasibility of achieving high coverage of mental health services.
The finding that digital mental health services are associated with the amelioration of anxiety and depressive symptoms is consistent with previous evidence examining their effects [, ]. Subgroup and meta-regression analyses allowed us to identify the effects of the study and population characteristics. For instance, there was no evidence of differences in SMD based on sample size. We found that depression measures contributed to heterogeneity. The posttreatment effects of digital mental health interventions differed from Depression Anxiety Stress Scales-21 (DASS-21) and other measurements. Although the current evidence supports the short-term effectiveness of most digital mental health services, a few studies have evaluated long-term outcomes after the interventions [ , , ]. The long-term benefits of receiving digital mental health services should be assessed through practical evaluations. Along with the viability of reaching high coverage, it is important to weigh the potential impacts and resource savings of providing digital mental health care in rural and underresourced regions.
Our review sheds light on future research. First, further efforts should be focused on how digital mental health services can be integrated with existing traditional treatment modalities to form a complementary and integrated service model. In particular, the integration of medication and digital services remains to be further validated for safety and feasibility. Second, most studies included in our review were digital psychotherapy interventions, suggesting that digital technologies may have facilitated the accessibility of psychotherapy during the pandemic. We found evidence supporting the efficacious delivery of psychotherapies, either guided [, ] or self-help therapy [ , , , ]. In a study by Al-Alawi et al [ ], therapist-guided web-based therapy was associated with greater effectiveness than self-help therapy for people with anxiety and depressive symptoms during the COVID-19 pandemic. Future studies should focus on determining whether people require a guided web-based psychotherapy approach and promote individualized treatment to ensure the optimal allocation of treatment resources. Third, despite the diversity of mental disorders, only a few are covered in this review. Depression and anxiety were the most common outcomes measured. Thus, the role of digital services for many severe mental illnesses, including schizophrenia and bipolar disorder, needs to be further examined.
Our study has implications for policy development in health care. During a pandemic, digital mental health technology can be used as a backup to address psychological demands and broaden service coverage. We urge policy makers to create future industry regulatory laws so that the digital health sector can grow in a way that provides security and effectiveness testing. The development of digital health infrastructure, including information transmission facilities and technical testing methods for data security issues, is also a crucial prerequisite for the future development of digital mental health. If digital mental health services are to continue, there is a need to specify the minimum level of privacy and security that is acceptable. For example, in the United States, digital mental health services have rapidly made changes in response to policies and regulations concerning confidentiality and privacy to promote telehealth delivery of care .
Our study has several limitations that should be noted. First, owing to the nature of our research question, the scoping review demonstrated substantial heterogeneity. In the pool of included studies, there was a high heterogeneity in terms of study design, type of techniques, type of mental health services, intervention components, target conditions, populations, and outcomes. Most studies did not have comparisons, and their effectiveness remains to be examined in future studies. For meta-analyses, we found only a few factors that contribute to heterogeneity between studies. The subgroup analyses were based on outcomes and limited by the variations in the type of techniques and type of mental health services. As a result, the findings should be interpreted with caution and read in terms of implications for future research.
Second, most participants in our scoping review were from high-income nations, primarily the United States and European countries. There is an urgent need for more independent research that examines the viability, acceptability, and efficacy of digital mental health as a response to pandemics in LMICs, particularly in low-income countries.
Third, although the current evidence supports the short-term effectiveness of most digital mental health services, only 2 studies have evaluated the long-term outcomes. A practical evaluation of the long-term benefits of digital mental health services should be undertaken, particularly in urban contexts.
Fourth, this study did not include preprint servers in the search, which may lead to the overestimation of our findings. However, a methodological research study of systematic reviews and meta-analyses revealed that including unpublished data (such as preprints) may add new sources of bias, although it does not change the results of reviews . In addition, there is a possibility that the findings in preprints will be changed further, which leads to the results of the review being amended as well.
Finally, the target of this study was to evaluate the usability, feasibility, acceptability, and initial efficacy of digital mental health services in the initial phase of the pandemic. Therefore, we limited the data search to the first 2 years of the pandemic. We recommend that future studies should assess whether these services will remain the same or be further refined during other phases of the pandemic.
In summary, we have reported a range of digital mental health services delivered during the initial phase of the COVID-19 pandemic. We found that digital mental health services were usable, feasible, acceptable, and effective in response to mental health needs in the initial phase of pandemic across high-income countries and middle-income countries. Our findings also highlight the amelioration of depressive and anxiety symptoms based on the pooled results of the RCTs. To date, the strategies to lessen the COVID-19 pandemic’s negative impacts on mental health remain a subject of much debate. We suggest that policy makers develop, implement, and assess different strategies for addressing the long-term mental health needs of the pandemic and its aftermath. Digital mental health services as novel strategies for promoting psychological well-being can purposively address physical barriers and provide a promising element for future integrated mental health service models in the post–COVID-19 era.
This study was funded by the National Natural Science Foundation of China (grant 82101575) and Science and Technology Program of Guangzhou (grant 202102020702).
Conflicts of Interest
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklists.DOCX File , 54 KB
Search databases and strategy.DOCX File , 34 KB
Summary of the findings of individual studies included in the qualitative systematic review.DOCX File , 145 KB
Results of the meta-regression for depression and anxiety.DOCX File , 324 KB
Publication bias of meta-analyses.DOCX File , 114 KB
Risk of Bias across randomized studies for meta-analyses.DOCX File , 534 KB
- COVID-19 Coronavirus pandemic. World-O-Meter. URL: https://www.worldometers.info/coronavirus/ [accessed 2022-10-05]
- Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar 14;395(10227):912-920 [FREE Full text] [CrossRef] [Medline]
- Moreno C, Wykes T, Galderisi S, Nordentoft M, Crossley N, Jones N, et al. How mental health care should change as a consequence of the COVID-19 pandemic. The Lancet Psychiatry 2020 Sep;7(9):813-824 [FREE Full text] [CrossRef] [Medline]
- Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors Associated With Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019. JAMA Netw Open 2020 Mar 02;3(3):e203976 [FREE Full text] [CrossRef] [Medline]
- Yao H, Chen J, Xu Y. Patients with mental health disorders in the COVID-19 epidemic. The Lancet Psychiatry 2020 Apr;7(4):e21 [FREE Full text] [CrossRef] [Medline]
- Lu Z, Shi L, Que J, Zheng Y, Wang Q, Liu W, et al. Long-term psychological profile of general population following COVID-19 outbreak: symptom trajectories and evolution of psychopathological network. Epidemiol Psychiatr Sci 2022 Sep 27;31:e69 [FREE Full text] [CrossRef] [Medline]
- Mak IW, Chu CM, Pan PC, Yiu MG, Chan VL. Long-term psychiatric morbidities among SARS survivors. Gen Hosp Psychiatry 2009;31(4):318-326 [FREE Full text] [CrossRef] [Medline]
- Yue J, Yan W, Sun Y, Yuan K, Su S, Han Y, et al. Mental health services for infectious disease outbreaks including COVID-19: a rapid systematic review. Psychol. Med 2020 Nov 05;50(15):2498-2513. [CrossRef]
- Karyotaki E, Efthimiou O, Miguel C, Bermpohl FM, Furukawa TA, Cuijpers P, Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis. JAMA Psychiatry 2021 Apr 01;78(4):361-371 [FREE Full text] [CrossRef] [Medline]
- 11. Measure performance. Australian Government Digital Transformation Agency. URL: https://www.dta.gov.au/help-and-advice/digital-service-standard/digital-service-standard-criteria/11-measure-performance [accessed 2023-01-04]
- Heynsbergh N, Heckel L, Botti M, Livingston PM. Feasibility, useability and acceptability of technology-based interventions for informal cancer carers: a systematic review. BMC Cancer 2018 Mar 02;18(1):244 [FREE Full text] [CrossRef] [Medline]
- Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
- Peters MD, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth 2020 Oct;18(10):2119-2126. [CrossRef] [Medline]
- Cohen J. Statistical Power Analysis for the Behavioral Sciences. Cambridge, Massachusetts, United States: Academic Press; 1969.
- Deeks J, Higgins J, Altman D. Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions version 6.3. Hoboken, New Jersey: Wiley; 2022.
- R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2019.
- Wei N, Huang B, Lu S, Hu J, Zhou X, Hu C, et al. Efficacy of internet-based integrated intervention on depression and anxiety symptoms in patients with COVID-19. Journal of Zhejiang University 2020 May;21(5):400-404 [FREE Full text] [CrossRef] [Medline]
- Al-Alawi M, McCall RK, Sultan A, Al Balushi N, Al-Mahrouqi T, Al Ghailani A, et al. Efficacy of a six-week-long therapist-guided online therapy versus self-help internet-based therapy for COVID-19-induced anxiety and depression: open-label, pragmatic, randomized controlled trial. JMIR Ment Health 2021 Feb 12;8(2):e26683 [FREE Full text] [CrossRef] [Medline]
- Guan F, Wu Y, Ren W, Zhang P, Jing B, Xu Z, et al. Self-compassion and the mitigation of negative affect in the era of social distancing. Mindfulness 2021 Jun 28;12(9):2184-2195 [FREE Full text] [CrossRef] [Medline]
- Aguilera A, Hernandez-Ramos R, Haro-Ramos AY, Boone CE, Luo TC, Xu J, et al. A text messaging intervention (StayWell at home) to counteract depression and anxiety during COVID-19 social distancing: pre-post study. JMIR Ment Health 2021 Nov 01;8(11):e25298 [FREE Full text] [CrossRef] [Medline]
- Agyapong VI, Shalaby R, Hrabok M, Vuong W, Noble JM, Gusnowski A, et al. Mental health outreach via supportive text messages during the COVID-19 pandemic: improved mental health and reduced suicidal ideation after six weeks in subscribers of Text4Hope compared to a control population. Int J Environ Res Public Health 2021 Feb 23;18(4):2157 [FREE Full text] [CrossRef] [Medline]
- Al-Refae M, Al-Refae A, Munroe M, Sardella NA, Ferrari M. A self-compassion and mindfulness-based cognitive mobile intervention (Serene) for depression, anxiety, and stress: promoting adaptive emotional regulation and wisdom. Front Psychol 2021 Mar 22;12:648087 [FREE Full text] [CrossRef] [Medline]
- Bantjes J, Kazdin AE, Cuijpers P, Breet E, Dunn-Coetzee M, Davids C, et al. A web-based group cognitive behavioral therapy intervention for symptoms of anxiety and depression among university students: open-label, pragmatic trial. JMIR Ment Health 2021 May 27;8(5):e27400 [FREE Full text] [CrossRef] [Medline]
- Ben-Zeev D, Chander A, Tauscher J, Buck B, Nepal S, Campbell A, et al. A smartphone intervention for people with serious mental illness: fully remote randomized controlled trial of CORE. J Med Internet Res 2021 Nov 12;23(11):e29201 [FREE Full text] [CrossRef] [Medline]
- Brouzos A, Vassilopoulos SP, Baourda VC, Tassi C, Stavrou V, Moschou K, et al. "Staying Home - Feeling Positive": Effectiveness of an on-line positive psychology group intervention during the COVID-19 pandemic. Curr Psychol 2021 Mar 20:1-13 [FREE Full text] [CrossRef] [Medline]
- Casas J. Evaluating the feasibility and acceptability of written exposure therapy delivered via telehealth for the treatment of post-traumatic stress disorder. University of Nevada, Reno. 2021. URL: https://www.proquest.com/openview/22c4a4bc7c2d0a10c02d7b3270b566a2/1?pq-origsite=gscholar&cbl=18750&diss=y [accessed 2022-09-05]
- Chandra M, Rai CB, Sandhu VK, Kumari N, Vishnoi S, Aman S, et al. Mhealth based mental health support counselling service for covid-19 suspect and positive patients in isolation facilities. J Indian Academy Clin Med 2021;22(1-2):6-11.
- Coifman KG, Disabato DD, Seah TH, Ostrowski-Delahanty S, Palmieri PA, Delahanty DL, et al. Boosting positive mood in medical and emergency personnel during the COVID-19 pandemic: preliminary evidence of efficacy, feasibility and acceptability of a novel online ambulatory intervention. Occup Environ Med 2021 Apr 26;78(8):555-557. [CrossRef] [Medline]
- Comer JS, Furr JM, Del Busto C, Silva K, Hong N, Poznanski B, et al. Therapist-led, internet-delivered treatment for early child social anxiety: a waitlist-controlled evaluation of the iCALM telehealth program. Behav Ther 2021 Sep;52(5):1171-1187. [CrossRef] [Medline]
- Craig SL, Leung VW, Pascoe R, Pang N, Iacono G, Austin A, et al. AFFIRM Online: utilising an affirmative cognitive-behavioural digital intervention to improve mental health, access, and engagement among LGBTQA+ youth and young adults. Int J Environ Res Public Health 2021 Feb 05;18(4):1541 [FREE Full text] [CrossRef] [Medline]
- Detweiler Guarino I, Cowan DR, Fellows AM, Buckey JC. Use of a self-guided computerized cognitive behavioral tool during COVID-19: evaluation study. JMIR Form Res 2021 May 31;5(5):e26989 [FREE Full text] [CrossRef] [Medline]
- Dincer B, Inangil D. The effect of Emotional Freedom Techniques on nurses' stress, anxiety, and burnout levels during the COVID-19 pandemic: a randomized controlled trial. Explore 2021 Mar;17(2):109-114 [FREE Full text] [CrossRef] [Medline]
- Esentürk OK, Yarımkaya E. Whatsapp-based physical activity intervention for children with autism spectrum disorder during the novel coronavirus (COVID-19) pandemic: a feasibility trial. Adapt Phys Activ Q 2021 Oct 01;38(4):569-584. [CrossRef] [Medline]
- Fiol-DeRoque MA, Serrano-Ripoll MJ, Jiménez R, Zamanillo-Campos R, Yáñez-Juan AM, Bennasar-Veny M, et al. A mobile phone-based intervention to reduce mental health problems in health care workers during the COVID-19 pandemic (PsyCovidApp): randomized controlled trial. JMIR Mhealth Uhealth 2021 May 18;9(5):e27039 [FREE Full text] [CrossRef] [Medline]
- Gabrielli S, Rizzi S, Bassi G, Carbone S, Maimone R, Marchesoni M, et al. Engagement and effectiveness of a healthy-coping intervention via chatbot for university students during the COVID-19 pandemic: mixed methods proof-of-concept study. JMIR Mhealth Uhealth 2021 May 28;9(5):e27965 [FREE Full text] [CrossRef] [Medline]
- Geoffroy PA, Le Goanvic V, Sabbagh O, Richoux C, Weinstein A, Dufayet G, et al. Psychological support system for hospital workers during the Covid-19 outbreak: rapid design and implementation of the Covid-Psy hotline. Front Psychiatry 2020 May 28;11:511 [FREE Full text] [CrossRef] [Medline]
- Gordon JS, Sbarra D, Armin J, Pace TW, Gniady C, Barraza Y. Use of a guided imagery mobile app (See Me Serene) to reduce COVID-19-related stress: pilot feasibility study. JMIR Form Res 2021 Oct 04;5(10):e32353 [FREE Full text] [CrossRef] [Medline]
- Graziano S, Boldrini F, Righelli D, Milo F, Lucidi V, Quittner A, et al. Psychological interventions during COVID pandemic: telehealth for individuals with cystic fibrosis and caregivers. Pediatr Pulmonol 2021 Jul 27;56(7):1976-1984 [FREE Full text] [CrossRef] [Medline]
- Gromatsky M, Sullivan SR, Mitchell EL, Spears AP, Edwards ER, Goodman M. Feasibility and acceptability of VA CONNECT: caring for our nation's needs electronically during the COVID-19 transition. Psychiatry Res 2021 Feb;296:113700 [FREE Full text] [CrossRef] [Medline]
- Guo L, Wu M, Zhu Z, Zhang L, Peng S, Li W, et al. Effectiveness and influencing factors of online education for caregivers of patients with eating disorders during COVID-19 pandemic in China. Eur Eat Disord Rev 2020 Nov 27;28(6):816-825 [FREE Full text] [CrossRef] [Medline]
- Held P, Klassen BJ, Coleman JA, Thompson K, Rydberg TS, Van Horn R. Delivering intensive PTSD treatment virtually: the development of a 2-week intensive cognitive processing therapy-based program in response to COVID-19. Cogn Behav Pract 2021 Nov;28(4):543-554 [FREE Full text] [CrossRef] [Medline]
- Hom MA, Weiss RB, Millman ZB, Christensen K, Lewis EJ, Cho S, et al. Development of a virtual partial hospital program for an acute psychiatric population: lessons learned and future directions for telepsychotherapy. J Psychother Integration 2020 Jun;30(2):366-382. [CrossRef]
- Hosseinzadeh Asl NR. A randomized controlled trial of a mindfulness-based intervention in social workers working during the COVID-19 crisis. Curr Psychol 2022 Aug 09;41(11):8192-8199 [FREE Full text] [CrossRef] [Medline]
- Hu J, Cai Z, Ma X. Effects of WeChat-based psychological interventions on the mental health of patients with suspected new coronavirus pneumonia: a pilot study. Jpn J Nurs Sci 2021 Oct 17;18(4):e12429 [FREE Full text] [CrossRef] [Medline]
- Ibrahim A, Chong MC, Khoo S, Wong LP, Chung I, Tan MP. Virtual group exercises and psychological status among community-dwelling older adults during the COVID-19 pandemic-a feasibility study. Geriatrics (Basel) 2021 Mar 22;6(1):31 [FREE Full text] [CrossRef] [Medline]
- Jaworski BK, Taylor K, Ramsey KM, Heinz A, Steinmetz S, Pagano I, et al. Exploring usage of COVID coach, a public mental health app designed for the COVID-19 pandemic: evaluation of analytics data. J Med Internet Res 2021 Mar 01;23(3):e26559 [FREE Full text] [CrossRef] [Medline]
- Kahlon MK, Aksan N, Aubrey R, Clark N, Cowley-Morillo M, Jacobs EA, et al. Effect of layperson-delivered, empathy-focused program of telephone calls on loneliness, depression, and anxiety among adults during the COVID-19 pandemic: a randomized clinical trial. JAMA Psychiatry 2021 Jun 01;78(6):616-622 [FREE Full text] [CrossRef] [Medline]
- Karagiozi K, Margaritidou P, Tsatali M, Marina M, Dimitriou T, Apostolidis H, et al. Comparison of on site versus online psycho education groups and reducing caregiver burden. Clin Gerontol 2022 Jul 05;45(5):1330-1340. [CrossRef] [Medline]
- Kim YJ. The effect of tele-acupressure self-practice for mental health and wellbeing in the community during COVID-19. Current Psychiatry Res Rev 2021 Jan 15;16(4):267-274. [CrossRef]
- Kolbe L, Jaywant A, Gupta A, Vanderlind WM, Jabbour G. Use of virtual reality in the inpatient rehabilitation of COVID-19 patients. Gen Hosp Psychiatry 2021 Jul;71:76-81 [FREE Full text] [CrossRef] [Medline]
- Lazzaroni E, Invernizzi R, Fogliato E, Pagani M, Maslovaric G. Coronavirus disease 2019 emergency and remote eye movement desensitization and reprocessing group therapy with adolescents and young adults: overcoming lockdown with the butterfly hug. Front Psychol 2021;12:701381 [FREE Full text] [CrossRef] [Medline]
- Lima MR, Wairagkar M, Natarajan N, Vaitheswaran S, Vaidyanathan R. Robotic telemedicine for mental health: a multimodal approach to improve human-robot engagement. Front Robot AI 2021 Mar 18;8:618866 [FREE Full text] [CrossRef] [Medline]
- Maldonado J. The effect of an online mental health promotion program during COVID-19 doctor of nursing practice projects. Andrews University. 2020 Oct. URL: https://digitalcommons.andrews.edu/dnp/10/ [accessed 2022-09-10]
- McKeon G, Tiedemann A, Sherrington C, Teasdale S, Mastrogiovanni C, Wells R, et al. Feasibility of an online, mental health-informed lifestyle program for people aged 60+ years during the COVID-19 pandemic. Health Promot J Austr 2022 Jul 19;33(3):545-552. [CrossRef] [Medline]
- Nauphal M, Swetlitz C, Smith L, Rosellini AJ. A preliminary examination of the acceptability, feasibility, and effectiveness of a telehealth cognitive-behavioral therapy group for social anxiety disorder. Cognit Behavioral Pract 2021 Nov;28(4):730-742. [CrossRef]
- Ortiz R, Southwick L, Schneider R, Klinger EV, Pelullo A, Guntuku SC, et al. Improving mood through community connection and resources using an interactive digital platform: development and usability study. JMIR Ment Health 2021 Feb 26;8(2):e25834 [FREE Full text] [CrossRef] [Medline]
- Parolin L, Benzi I, Fanti E, Milesi A, Cipresso P, Preti E. Italia Ti Ascolto [Italy, I am listening]: an app-based group psychological intervention during the COVID-19 pandemic. OSF Preprints. 2021. URL: https://doi.org/10.4081/ripppo.2021.517 [accessed 2022-08-30]
- Philip P, Dupuy L, Morin CM, de Sevin E, Bioulac S, Taillard J, et al. Smartphone-based virtual agents to help individuals with sleep concerns during COVID-19 confinement: feasibility study. J Med Internet Res 2020 Dec 18;22(12):e24268 [FREE Full text] [CrossRef] [Medline]
- Puspitasari AJ, Heredia D, Gentry M, Sawchuk C, Theobald B, Moore W, et al. Rapid adoption and implementation of telehealth group psychotherapy during COVID 19: practical strategies and recommendations. Cogn Behav Pract 2021 Nov;28(4):492-506 [FREE Full text] [CrossRef] [Medline]
- Rojas SM, Gold SD, Bryan CJ, Pruitt LD, Felker BL, Reger MA. Brief cognitive-behavioral therapy for suicide prevention (BCBT-SP) via video telehealth: a case example during the COVID-19 outbreak. Cogn Behav Pract 2022 May;29(2):446-453 [FREE Full text] [CrossRef] [Medline]
- Shalaby R, Vuong W, Hrabok M, Gusnowski A, Mrklas K, Li D, et al. Gender differences in satisfaction with a text messaging program (Text4Hope) and anticipated receptivity to technology-based health support during the COVID-19 pandemic: cross-sectional survey study. JMIR Mhealth Uhealth 2021 Apr 15;9(4):e24184 [FREE Full text] [CrossRef] [Medline]
- Shapira S, Yeshua-Katz D, Cohn-Schwartz E, Aharonson-Daniel L, Sarid O, Clarfield AM. A pilot randomized controlled trial of a group intervention via Zoom to relieve loneliness and depressive symptoms among older persons during the COVID-19 outbreak. Internet Interv 2021 Apr;24:100368 [FREE Full text] [CrossRef] [Medline]
- Sharrock MJ, Mahoney AE, Haskelberg H, Millard M, Newby JM. The uptake and outcomes of internet-based cognitive behavioural therapy for health anxiety symptoms during the COVID-19 pandemic. J Anxiety Disord 2021 Dec;84:102494 [FREE Full text] [CrossRef] [Medline]
- Song J, Jiang R, Chen N, Qu W, Liu D, Zhang M, et al. Self-help cognitive behavioral therapy application for COVID-19-related mental health problems: a longitudinal trial. Asian J Psychiatr 2021 Jun;60:102656 [FREE Full text] [CrossRef] [Medline]
- Sosa Lovera A, Ureña AJ, Arias J, Araujo Rodríguez A, Canario Guzmán JA. Psychological helpline in response to the COVID-19 pandemic in the Dominican Republic. Couns Psychother Res 2022 Jun 29;22(2):534-541 [FREE Full text] [CrossRef] [Medline]
- Sturgill R, Martinasek M, Schmidt T, Goyal R. A novel artificial intelligence-powered emotional intelligence and mindfulness app (Ajivar) for the college student population during the COVID-19 pandemic: quantitative questionnaire study. JMIR Form Res 2021 Jan 05;5(1):e25372 [FREE Full text] [CrossRef] [Medline]
- Suffoletto B, Goldstein T, Gotkiewicz D, Gotkiewicz E, George B, Brent D. Acceptability, engagement, and effects of a mobile digital intervention to support mental health for young adults transitioning to college: pilot randomized controlled trial. JMIR Form Res 2021 Oct 14;5(10):e32271 [FREE Full text] [CrossRef] [Medline]
- Summers C, Wu P, Taylor AJ. Supporting mental health during the COVID-19 pandemic using a digital behavior change intervention: an open-label, single-arm, pre-post intervention study. JMIR Form Res 2021 Oct 06;5(10):e31273 [FREE Full text] [CrossRef] [Medline]
- Sun S, Lin D, Goldberg S, Shen Z, Chen P, Qiao S, et al. A mindfulness-based mobile health (mHealth) intervention among psychologically distressed university students in quarantine during the COVID-19 pandemic: a randomized controlled trial. J Couns Psychol 2022 Mar;69(2):157-171. [CrossRef] [Medline]
- Tarquinio C, Brennstuhl M, Rydberg JA, Bassan F, Peter L, Tarquinio CL, et al. EMDR in telemental health counseling for healthcare workers caring for COVID-19 patients: a pilot study. Issues Ment Health Nurs 2021 Jan 14;42(1):3-14. [CrossRef] [Medline]
- Vallefuoco E, Purpura G, Gison G, Bonifacio A, Tagliabue L, Broggi F, et al. A multidisciplinary telerehabilitation approach for supporting social interaction in autism spectrum disorder families: an Italian digital platform in response to COVID-19. Brain Sci 2021 Oct 25;11(11):1404 [FREE Full text] [CrossRef] [Medline]
- van Agteren J, Ali K, Fassnacht DB, Iasiello M, Furber G, Howard A, et al. Testing the differential impact of an internet-based mental health intervention on outcomes of well-being and psychological distress during COVID-19: uncontrolled intervention study. JMIR Ment Health 2021 Sep 15;8(9):e28044 [FREE Full text] [CrossRef] [Medline]
- Wagner L, Corona LL, Weitlauf AS, Marsh KL, Berman AF, Broderick NA, et al. Use of the TELE-ASD-PEDS for autism evaluations in response to COVID-19: preliminary outcomes and clinician acceptability. J Autism Dev Disord 2021 Sep 30;51(9):3063-3072 [FREE Full text] [CrossRef] [Medline]
- Wahlund T, Mataix-Cols D, Olofsdotter Lauri K, de Schipper E, Ljótsson B, Aspvall K, et al. Brief online cognitive behavioural intervention for dysfunctional worry related to the COVID-19 pandemic: a randomised controlled trial. Psychother Psychosom 2021;90(3):191-199 [FREE Full text] [CrossRef] [Medline]
- Wasil AR, Taylor ME, Franzen RE, Steinberg JS, DeRubeis RJ. Promoting graduate student mental health during COVID-19: acceptability, feasibility, and perceived utility of an online single-session intervention. Front Psychol 2021 Apr 7;12:569785 [FREE Full text] [CrossRef] [Medline]
- Wood HJ, Gannon JM, Chengappa KR, Sarpal DK. Group teletherapy for first-episode psychosis: piloting its integration with coordinated specialty care during the COVID-19 pandemic. Psychol Psychother 2021 Jun 20;94(2):382-389. [CrossRef] [Medline]
- Ying Y, Ji Y, Kong F, Chen Q, Lv Y, Hou Y, et al. Internet-based cognitive behavioral therapy for psychological distress in older adults without cognitive impairment living in nursing homes during the COVID-19 pandemic: a feasibility study. Internet Interv 2021 Dec;26:100461 [FREE Full text] [CrossRef] [Medline]
- Zepeda MS, Deighton S, Markova V, Madsen JW, Racine N. iCOPE with COVID-19: a brief telemental health intervention for children and adolescents during the COVID-19 pandemic. Cogn Behav Pract 2021 Oct 22 [FREE Full text] [CrossRef] [Medline]
- Zhang H, Zhang A, Liu C, Xiao J, Wang K. A brief online mindfulness-based group intervention for psychological distress among chinese residents during COVID-19: a pilot randomized controlled trial. Mindfulness (N Y) 2021;12(6):1502-1512 [FREE Full text] [CrossRef] [Medline]
- Zimmerman M, Benjamin I, Tirpak JW, D'Avanzato C. Patient satisfaction with partial hospital telehealth treatment during the COVID-19 pandemic: comparison to in-person treatment. Psychiatry Res 2021 Jul;301:113966 [FREE Full text] [CrossRef] [Medline]
- Van Lieshout RJ, Layton H, Savoy CD, Brown JS, Ferro MA, Streiner DL, et al. Effect of online 1-day cognitive behavioral therapy-based workshops plus usual care vs usual care alone for postpartum depression: a randomized clinical trial. JAMA Psychiatry 2021 Nov 01;78(11):1200-1207 [FREE Full text] [CrossRef] [Medline]
- Zhang X, Lewis S, Firth J, Chen X, Bucci S. Digital mental health in China: a systematic review. Psychol. Med 2021 Sep 28;51(15):2552-2570. [CrossRef]
- Naslund JA, Aschbrenner KA, Araya R, Marsch LA, Unützer J, Patel V, et al. Digital technology for treating and preventing mental disorders in low-income and middle-income countries: a narrative review of the literature. Lancet Psychiatry 2017 Jun;4(6):486-500. [CrossRef]
- Shore JH, Schneck CD, Mishkind MC. Telepsychiatry and the coronavirus disease 2019 pandemic-current and future outcomes of the rapid virtualization of psychiatric care. JAMA Psychiatry 2020 Dec 01;77(12):1211-1212. [CrossRef] [Medline]
- Liu S, Yang L, Zhang C, Xiang Y, Liu Z, Hu S, et al. Online mental health services in China during the COVID-19 outbreak. Lancet Psychiatry 2020 Apr;7(4):e17-e18. [CrossRef] [Medline]
- Merchant R, Torous J, Rodriguez-Villa E, Naslund JA. Digital technology for management of severe mental disorders in low-income and middle-income countries. Curr Opin Psychiatry 2020 Sep;33(5):501-507 [FREE Full text] [CrossRef] [Medline]
- Sin J, Galeazzi G, McGregor E, Collom J, Taylor A, Barrett B, et al. Digital interventions for screening and treating common mental disorders or symptoms of common mental illness in adults: systematic review and meta-analysis. J Med Internet Res 2020 Sep 02;22(9):e20581 [FREE Full text] [CrossRef] [Medline]
- Bartels SJ, Baggett TP, Freudenreich O, Bird BL. COVID-19 emergency reforms in massachusetts to support behavioral health care and reduce mortality of people with serious mental illness. Psychiatr Serv 2020 Oct 01;71(10):1078-1081. [CrossRef] [Medline]
- Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, OPEN consortium. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One 2017;12(4):e0176210 [FREE Full text] [CrossRef] [Medline]
|CBT: cognitive behavioral therapy|
|DASS-21: Depression Anxiety Stress Scales-21|
|LMICs: low- and middle-income countries|
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|PRISMA-ScR: PRISMA Extension for Scoping Reviews|
|RCT: randomized controlled trial|
|SMD: standardized mean difference|
|VR: virtual reality|
Edited by Y Khader; submitted 22.10.22; peer-reviewed by T Sathish, M Musker, C Leung; comments to author 02.12.22; revised version received 09.01.23; accepted 11.01.23; published 13.02.23Copyright
©Shaoling Zhong, Xinhu Yang, Zihua Pan, Yu Fan, Yanan Chen, Xin Yu, Liang Zhou. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 13.02.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.