This paper is in the following e-collection/theme issue:

COVID-19 Vaccination (126) JMIR Theme Issue: COVID-19 Special Issue (2388) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1481) Outbreak and Pandemic Preparedness and Management (1673) Hygiene and Infection Prevention (297) Infectious Diseases (non-STD/STI) (1478) Vaccination and Immunization in the Digital Age (343) HIV/AIDS/STI Prevention and Care (816)

Published on in Vol 8, No 6 (2022): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/33995, first published .
Reasons for COVID-19 Vaccine Hesitancy Among Chinese People Living With HIV/AIDS: Structural Equation Modeling Analysis

Reasons for COVID-19 Vaccine Hesitancy Among Chinese People Living With HIV/AIDS: Structural Equation Modeling Analysis

Reasons for COVID-19 Vaccine Hesitancy Among Chinese People Living With HIV/AIDS: Structural Equation Modeling Analysis

Authors of this article:

Yan Yao1 Author Orcid Image ;   Ruiyu Chai1 Author Orcid Image ;   Jianzhou Yang2 Author Orcid Image ;   Xiangjun Zhang3 Author Orcid Image ;   Xiaojie Huang4 Author Orcid Image ;   Maohe Yu5 Author Orcid Image ;   Geng-feng Fu6 Author Orcid Image ;   Guanghua Lan7 Author Orcid Image ;   Ying Qiao8 Author Orcid Image ;   Qidi Zhou9 Author Orcid Image ;   Shuyue Li10 Author Orcid Image ;   Junjie Xu11 Author Orcid Image
Article Authors Cited by (16) Tweetations (3) Metrics

    Original Paper

    1Department of Epidemiology and Biostatistics, School of Public Health, Jilin University, Changchun, China

    2Department of Preventive Medicine, Changzhi Medical College, Changzhi, China

    3Department of Public Health, University of Tennessee, Knoxville, TN, United States

    4Beijing Youan Hospital, Capital Medical University, Beijing, China

    5Tianjin Centers for Disease Control and Prevention, Tianjin, China

    6Jiangsu Provincial Center for Disease Control and Prevention, Nanjing, China

    7Guangxi Center for Disease Prevention and Control, Nanning, China

    8The Second Hospital of Huhhot, Huhhot, China

    9Department of Emergency Medicine, Shenzhen Hospital, Peking University, Shenzhen, China

    10Changchun Maternity Hospital, Changchun, China

    11Clinical Research Academy, Peking University Shenzhen Hospital, Peking University, Shenzhen, China

    *these authors contributed equally

    Corresponding Author:

    Junjie Xu, PhD

    Clinical Research Academy

    Peking University Shenzhen Hospital

    Peking University

    1120, Lianhua Road

    Futian District

    Shenzhen, 518036

    China

    Phone: 86 755 8392333

    Email: xjjcmu@163.com


    Related ArticleThis is a corrected version. See correction statement in: https://publichealth.jmir.org/2022/7/e40910

    Background: Many countries and organizations recommended people living with HIV/AIDS (PLWHA) receive the COVID-19 vaccine. However, vaccine hesitancy still exists and becomes a barrier for promoting COVID-19 vaccination among PLWHA.

    Objective: This study aims to investigate factors that contributed to COVID-19 vaccine hesitancy among PLWHA.

    Methods: The study used a multicenter cross-sectional design and an online survey mode. We recruited PLWHA aged 18-65 years from 5 metropolitan cities in China between January 2021 and February 2021. Participants completed an online survey through Golden Data, a widely used encrypted web-based survey platform. Multiple linear regression models were used to assess the background characteristics in relation to COVID-19 vaccine hesitancy, and structural equation modeling was performed to assess the relationships among perceived benefits, perceived risks, self-efficacy, subjective norms, and COVID-19 vaccine hesitancy.

    Results: Among 1735 participants, 41.61% (722/1735) reported COVID-19 vaccine hesitancy. Older age, no other vaccinations in the past 3 years, and having chronic disease history were positively associated with COVID-19 vaccine hesitancy. Structural equation modeling revealed a direct relationship of perceived benefits, perceived risks, and subjective norms with self-efficacy and vaccine hesitancy and an indirect relationship of perceived benefits, perceived risks, and subjective norms with vaccine hesitancy. Moreover, self-efficacy toward COVID-19 vaccination was low. PLWHA had concerns of HIV disclosure during COVID-19 vaccination. Family member support could have an impact on COVID-19 vaccination decision-making.

    Conclusions: COVID-19 vaccine hesitancy was high among PLWHA in China. To reduce COVID-19 vaccine hesitancy, programs and strategies should be adopted to eliminate the concerns for COVID-19 vaccination, disseminate accurate information on the safety and efficacy of the COVID-19 vaccine, encourage family member support for COVID-19 vaccination, and improve PLWHA’s trust of medical professionals.

    JMIR Public Health Surveill 2022;8(6):e33995

    doi:10.2196/33995

    Keywords

    COVID-19 vaccine; vaccine hesitancy; PLWHA; structural equation modeling 


    The COVID-19 pandemic has become a global health challenge and poses a serious health threat [1]. Compared with the HIV-negative population, people living with HIV/AIDS (PLWHA) with a weakened immune condition or with comorbidities have an increased risk of having poorer outcomes from COVID-19 [2]. Moreover, PLWHA who are immunocompromised are more likely to have a more severe illness and a longer disease course from COVID-19 [3-5]. Some longitudinal studies have reported that PLWHA have higher COVID-19 mortality than the HIV-negative population [6-8]. Therefore, it is critical for PLWHA to receive vaccines to prevent COVID-19. Many countries and organizations recommended PLWHA to receive a COVID-19 vaccine [9-14]. The Joint United Nations Programme on HIV/AIDS has declared that the COVID-19 vaccines authorized by regulators can significantly reduce the risks of severe COVID-19 illness and death and are safe for most people, including PLWHA [11]. The UK Department of Health and US Centers for Disease Control and Prevention released guidance that recommended PLWHA, regardless of CD4 count, should be vaccinated against COVID-19 [12,13]. In March 2021, China launched an updated COVID-19 guideline that also recommended PLWHA receive COVID-19 vaccines [14].

    Although the evidence on the side effects and protective efficacy of COVID-19 vaccination in PLWHA is insufficient, some studies have shown that COVID-19 vaccine hesitancy is higher among PLWHA than HIV-negative people [15,16]. For example, a study in British Columbia, Canada, showed that 65.2% of PLWHA reported intending to receive a COVID-19 vaccine recommended and available to them, which was lower than HIV-negative people (79.6%). That is to say, COVID-19 vaccine hesitancy is higher among PLWHA than HIV-negative people [15]. In a cross-sectional study conducted in Beijing, China, the rate of COVID-19 vaccine hesitancy of the PLWHA population was 27.5%, which was higher than HIV-negative people (17.75%) [16]. Hence, vaccine hesitancy exists and becomes a barrier for promoting COVID-19 vaccination among PLWHA. Vaccine hesitancy was defined by the World Health Organization as the delay in acceptance or refusal of vaccination despite the availability of vaccination services [17]. Vaccine hesitancy was listed as one of the top 10 global health threats in 2019 [18].

    It is urgently needed to eliminate COVID-19 vaccine hesitancy and improve the coverage rate for PLWHA who might encounter more barriers and have more concerns about COVID-19 vaccination. A recent study reported that vaccine hesitancy was influenced by various factors, such as perceived benefits and risks of a vaccine, perceived safety of a vaccine, confidence in a vaccine, attitudes toward a vaccine, and an individual’s demand [17,19-21]. Perceived vaccine safety was reported as an essential factor that can lead to a vaccination decision [22]. In other words, people who perceive a vaccine as unsafe are more likely to refuse or delay vaccination (vaccine hesitancy) [23]. Perceived risks of a vaccine could also result in vaccine hesitancy [24]. Moreover, a recent French study that investigated COVID-19 vaccine hesitancy in PLWHA indicated a high hesitancy rate of 28.7%, and PLWHA had concerns about serious side effects of COVID-19 vaccination [25]. Another study that investigated vaccine hesitancy among African American PLWHA demonstrated that people trusted some COVID-19 vaccination sources, such as social service and health care providers, more than others [26].

    Although previous studies determined some factors that were associated with PLWHA’s vaccine hesitancy, complex relationships among multiple factors might exist but remain unassessed. A structural equation modeling (SEM) approach that provides a flexible framework to analyze multiple variables and takes into consideration relationships among variables could provide a more compelling explanation of COVID-19 vaccine hesitancy. However, there is a lack of research investigating the factors correlated with PLWHA’s vaccine hesitancy through SEM. Therefore, we designed a survey that investigated factors associated with COVID-19 vaccine hesitancy among PLWHA using SEM. Some factors that were reported in the literature were assessed and included in the model, such as perceived benefits, perceived risks, self-efficacy, and subjective norms. The findings of the study aimed to provide valuable evidence for a deep understanding of COVID-19 vaccine hesitancy, therefore contribute to policy making and programming efforts with the goals of addressing vaccine hesitancy and promoting COVID-19 vaccination among PLWHA.


    Study Design

    The study used a multicenter cross-sectional design and an online survey mode. Recruitment was conducted in 5 large cities from 4 regions of China between January 2021 and February 2021. These cities included 2 in the North (Tianjin and Beijing), 1 in the Northeast (Hohhot), 1 in the East (Nanjing), and 1 in the South (Nanning). To achieve the study objectives, we have set up the following criteria for cities to be qualified and included in this study: (1) must have community-based organizations (CBOs) providing services to PLWHA; (2) each city has a minimum of 3000 reported PLWHA; (3) COVID-19 vaccination was first scaled up in these sites; and (4) there is an adequate supply of vaccines in these sites.

    We used 2 methods to calculate the sample size.

    The first one was based on the estimation of the rate of COVID-19 vaccine hesitancy among the PLWHA population based on the clustering sample method. According to a cross-sectional study conducted in Beijing, China, the rate of COVID-19 vaccine hesitancy of the PLWHA population was 27.5% [16]. We first estimated the sample size using the following sample size formula from a simple randomized sampling method. The α is the significance level; if α was 0.05, Z1-ɑ/2 could be assumed to be 1.96. δ is the allowable error, and was considered to be 0.05. The p, or the estimated COVID hesitancy rate in the PLWHA population, was considered to be 27.5%. Then, we used the design effect (deff) to further calculate the sample size based on a clustering sampling method. The deff was defined as the ratio of the variance, taking into account the clustering sample design and the variance of a simple random sample design with the same number of observations, deff was considered to be 2 based on previous studies [27-29]. Eventually, a sample size of 613 was initially generated based on a clustering sample study design. A minimum sample size of 852 was acquired after taking into consideration the no response rate of participants (20%) and the portion of unacceptable responses (10%). The sample size formula was expressed as:

    For the second sample size calculation, Nunnally [30] believed that the minimum sample size should be at least 10 times the analyzed variables to conduct a SEM analysis. There are 23 variables in this study without considering background characteristics, so a sample size of 230 was initially needed. A minimum sample size of 320 was acquired after taking into consideration the no response rate of participants (20%) and the portion of unacceptable responses (10%).

    Last, we used 852 as the minimum sample size of this study.

    Participants

    Eligible participants were individuals aged 18-65 years who had been diagnosed with HIV or AIDS and were living in 1 of the 5 cities. Exclusion criteria included (1) being illiterate and unable to complete the online questionnaire and (2) PLWHA who had been vaccinated against COVID-19.

    Recruitment and Data Collection

    We recruited participants mainly through CBOs, which provide services mainly to PLWHA and have been cooperating closely with HIV clinical service providers in the 5 study sites. In China, HIV outreach services to PLWHA have been transferred from government agencies to CBOs [31]. At present, CBOs are the primary providers of these routine tasks. In addition, a large portion of PLWHAs is followed up by CBOs. The questionnaire survey was carried out using Golden Data, a commonly used, encrypted, web-based survey platform. Each participant took about 13-15 minutes to complete this survey. An electronic coupon with a value of 20 Chinese yuan (US $3.10) was sent to the participant upon completion. The database we used was protected by a password and could only be accessed by designated research team members. More detailed information about recruitment and data collection can be found in Multimedia Appendix 1.

    Instrumentation

    A panel consisting of public health researchers, psychologists, clinicians, CBO staff, and participant representatives was assembled to develop the questionnaire for the study. Ten participant representatives responded to the online survey and provided feedback for improvement. The panel revised and finalized the questionnaire based on the pilot responses and the feedback. The 10 participants were not included in the final analyses of the study.

    The questionnaire included the following sections: (1) background characteristics (eg, including sociodemographic characteristics, presence of chronic conditions, history of other vaccination in the past 3 years, HIV disease information), (2) vaccine hesitancy, (3) perceived risks, (4) self-efficacy, (5) perceived benefits, and (6) subjective norms. Constructs (2) to (6) were assessed using a 5-point Likert scale. Latent variables that may have direct or indirect relationships with COVID-19 vaccine hesitancy were also measured according to the hypotheses.

    In this study, vaccination hesitancy was defined as the proportion of respondents who reported “definitely not” or “probably not” or “unsure” to undergo the COVID-19 vaccination program based on a recent peer-reviewed study by Fisher et al [32].

    Study Hypotheses

    Based on the literature, we proposed the following study hypotheses:

    • Hypothesis 1: Perceived benefit is negatively associated with COVID-19 vaccine hesitancy (the higher the perceived benefits, the lower the degree of COVID-19 vaccine hesitancy).
    • Hypothesis 2: Perceived risk is positively associated with COVID-19 vaccine hesitancy (the higher the perceived risks, the higher the degree of COVID-19 vaccine hesitancy).
    • Hypothesis 3: Subjective norms are negatively associated with COVID-19 vaccine hesitancy (the higher the subjective norms, the lower the degree of COVID-19 vaccine hesitancy).
    • Hypothesis 4: Self-efficacy is negatively associated with COVID-19 vaccine hesitancy (the higher the self-efficacy, the lower the degree of COVID-19 vaccine hesitancy).
    • Hypothesis 5: Self-efficacy is positively associated with perceived benefits (the higher the self-efficacy, the higher the degree of perceived benefits).
    • Hypothesis 6: Self-efficacy is negatively associated with perceived risks (the higher the self-efficacy, the lower the degree of perceived risks).
    • Hypothesis 7: Self-efficacy is positively associated with subjective norms (the higher the self-efficacy, the higher the degree of subjective norms).

    Statistical Analysis

    Descriptive statistics were performed to summarize the background characteristics associated with and frequencies of COVID-19 vaccine hesitancy. The total average scores and dimensional average scores for vaccine hesitancy, perceived benefits, perceived risks, self-efficacy, and subjective norms were generated. A 1-way ANOVA test was used to identify the factors predicting COVID-19 vaccine hesitancy. SPSS software (version 24.0; IBM Corporation) was used to perform all data analyses. The significance level was set at a 2-tailed P value of <.05.

    Model Analysis

    First, means and standard deviations were generated to describe the basic information; skewness and kurtosis were computed to describe the distribution of the data. Furthermore, we used Amos 24.0 to construct the SEM and used the nonparametric percentile bootstrap method of bias correction to test the indirect relationships.

    Ethics Approval

    The institutional review boards of the Changzhi Medical College (RT2021003) approved this study. Respondents were informed that their participation was voluntary, and consent was implied by completion of the questionnaire.


    Background Characteristics

    A total of 1883 PLWHA completed the online survey from the 5 metropolitan cities. We excluded 148 participants who had been vaccinated for COVID-19. A total of 1735 participants were included in this study. The majority of the participants were 18-39 years old (1285/1735, 74.06%) and identified themselves as male (1638/1735, 94.41%; Table 1). In terms of relationship status, education, and employment status, 67.44% (1170/1735) of participants were currently single, 62.25% (1080/1735) had received a college education or higher, and 69.91% (1213/1735) had a full-time job. Only 77.22% (1339/1735) of the participants had basic health insurance. Moreover, 17.35% (301/1735) of the participants received their HIV diagnosis within 1 year prior, 97.58% (1693/1735) of the participants were on antiretroviral therapy (ART), 70.55% (1224/1735) reported they had an undetectable viral load, and 46.86% (813/1735) reported their CD4 T cell counts were above 500 cells/µL.

    Table 1. Background characteristics of participants (n=1735).
    Sociodemographic characteristics and chronic disease and HIV-related indicatorsResults, n (%)
    Age group (years)

    		18-29523 (30.14)

    		30-39762 (43.92)

    		40-49325 (18.73)

    		≥50125 (7.20)
    Gender at birth

    		Male1638 (94.41)

    		Female97 (5.59)
    Gender identity

    		Male1420 (81.84)

    		Female164 (9.45)

    		Transgender146 (8.41)

    		Others5 (0.29)
    Relationship status

    		Currently single1170 (67.44)

    		Cohabited/married with a same-sex partner236 (13.60)

    		Cohabited/married with an opposite-sex partner329 (18.96)
    Highest education level attained

    		Junior high or below277 (15.97)

    		Senior high or equivalent378 (21.79)

    		College and above1080 (62.25)
    Employment status

    		Full-time1213 (69.91)

    		Part-time/unemployed/retired/students/others522 (30.09)
    Monthly personal income (Chinese yuan/US $)

    		No fixed income204 (11.76)

    		<1000/15494 (5.42)

    		1000-2999/154-462230 (13.26)

    		3000-4999/462-770501 (28.88)

    		5000-6999/770-1078338 (19.48)

    		7000-9999/1078-1540)174 (10.03)

    		≥10,000/1540)194 (11.18)
    Type of health insurance

    		No197 (11.35)

    		Basic health insurance only1339 (77.18)

    		Commercial health insurance only35 (2.02)

    		Both basic and commercial health insurance157 (9.05)

    		Others7 (0.40)
    Study site

    		Beijing495 (28.53)

    		Tianjin320 (18.44)

    		Nanjing313 (18.04)

    		Hohhot315 (18.16)

    		Nanning292 (16.83)
    Current tobacco use

    		No1253 (72.22)

    		Yes482 (27.78)
    Current alcohol use

    		No1395 (80.40)

    		Yes340 (19.60)
    Self-reported BMI (kg/m2)

    		<18.5155 (8.93)

    		18.5-23.91128 (65.01)

    		24.0-27.9364 (20.98)

    		≥2888 (5.07)
    Presence of chronic disease conditions (not including HIV)

    		No1157 (66.69)

    		Yes578 (33.31)
    Medication use for treating chronic diseases (not including HIV)

    		No1639 (94.47)

    		Yes96 (5.53)
    History of other vaccinations in the past 3 years

    		No1324 (76.31)

    		Yes411 (23.69)
    Time since HIV diagnosis (years)

    		≤1301 (17.35)

    		2-5806 (46.46)

    		>5628 (36.20)
    On antiretroviral therapy

    		No42 (2.42)

    		Yes1693 (97.58)
    HIV viral load in the most recent episode of testing (copies/mL)

    		Undetectable (<50)1224 (70.55)

    		Detectable (≥50)197 (11.35)

    		Not sure314 (18.10)
    CD4+ T cell count in the most recent episode of testing, cells/mm3

    		>500813 (46.86)

    		350-499354 (20.40)

    		200-349177 (10.20)

    		<20059 (3.40)

    		Unknown332 (19.14)

    Attitudes Toward COVID-19 Vaccines

    Regarding the responses to the statement “the likelihood of receiving free COVID-19 vaccination in the future,” 58.4% (1013/1735) of the participants responded that they would accept. Only 2.2% (38/1735) of the participants responded that they would definitely not get vaccinated, 6.7% (116/1735) of the participants said they would probably not get vaccinated, and 32.7% (568/1735) of the participants said they were unsure. In total, 41.6% (722/1735) of participants had vaccine hesitancy (Table S1 in Multimedia Appendix 2).

    Among the 722 participants who hesitated to be vaccinated, when they were asked about factors affecting their vaccine willingness, a majority (482/722, 66.8%) of participants demonstrated concerns about a possible influence on ART, and 65% (469/722) had concerns about a possible influence on HIV disease status, that is the HIV disease would progress abnormally, including a rebound of viral load, or a decrease of absolute CD4+ T cell counts after COVID-19 vaccination. Moreover, 57.6% (416/722) of participants had concerns about the possible side effects of the COVID-19 vaccine. Nearly one-half of the participants (332/722, 46%) demonstrated fear of HIV disclosure. Many participants (308/722, 42.7%) had concerns that ART might affect the effectiveness of the vaccine, 40.3% (291/722) of participants worried that their HIV status might affect the effectiveness of the vaccine, and 22.9% (165/722) had concerns about the vaccine effectiveness alone. A small number of participants (15/722, 2.1%) reported other factors that were associated with their vaccine hesitancy (Figure 1).

    ‎
    Figure 1. Self-reported reasons of COVID-19 vaccine hesitancy in people living with HIV/AIDS (PLWHA; n=722). ART: antiretroviral therapy.
    View this figure

    Vaccine Hesitancy and Background Characteristics

    A 1-way ANOVA was used to assess differences in COVID-19 vaccine hesitancy scores among participants with different demographic characteristics. Compared with the group aged 18-29 years old, participants over 29 years old were more hesitant to get the COVID-19 vaccine (P=.009; Table S2 in Multimedia Appendix 2). Participants who had chronic diseases or a chronic disease history were more hesitant than those who did not have (PLWHA without chronic diseases: 2.62 vs PLWHA with chronic diseases or history: 2.42; P<.001). PLWHA who did not have other vaccinations in the past 3 years were more hesitant than the ones who did (eg, without other vaccinations: 2.35 vs with other vaccinations: 2.21; P=.01).

    The significant variables in the univariate analyses were included in the multiple linear regression model. Multiple linear regression analyses identified that the tolerance of independent variables was greater than 0.1, and the variance expansion factor ranged from 1.01 to 1.40. All were less than 10, which indicated there was no multicollinearity and the results of the linear regression model were reliable.

    The results of the multivariate linear regression analyses showed that, in general, older age (except for the group aged 40-49 years) was positively associated with COVID-19 vaccine hesitancy. Having received another vaccination in the past 3 years was negatively correlated with COVID-19 vaccine hesitancy (β=–0.07, P=.01; Table 2). Having chronic diseases or a chronic disease history was positively correlated with COVID-19 vaccine hesitancy (β=2.77, P=.01).

    Table 2. Multivariate analyses of vaccine hesitancy by background characteristics.
    CharacteristicsUnstandardized coefficientStandardized coefficient (β)t value (df)P valueCollinearity statistics

    		BSE


    		ToleranceVIFa
    Intercept2.220.04N/Ab50.23<.001N/AN/A
    Age (years)

    		18-29RefcRefRefRef (3)RefRefRef

    		30-390.120.050.062.23 (1).030.711.40

    		40-490.100.070.041.47 (1).140.731.37

    		≥500.230.100.062.35 (1).020.831.20
    Presence of chronic disease conditions0.140.050.072.77 (1).010.931.07
    History of other vaccinations in the past 3 years–0.150.05–0.07–2.86 (1)<.0011.001.01

    aVIF: variance inflation factor.

    bN/A: not applicable.

    cRef: reference.

    Correlation Matrix

    The results showed a negative correlation between perceived benefits and vaccine hesitancy and a positive correlation between perceived risks and vaccine hesitancy (both P<.001; Table S3 in Multimedia Appendix 2). Self-efficacy and subjective norms were negatively correlated with vaccine hesitancy (both P<.001).

    Measurement Scores

    Generally, when the absolute value of the skewness coefficient of an observation variable is <3 and the absolute value of the kurtosis coefficient is <8, the data can be regarded as having a normal distribution. According to the kurtosis results (from –1.29 to 1.45) and skewness (from –1.23 to 0.72), the study data were normally distributed.

    The mean self-efficacy score was the lowest of all indicators; in other words, participants’ confidence of COVID-19 vaccination was generally low. The mean perceived risk was the highest. In addition, the concern about HIV disclosure showed the highest mean score among all perceived risks. Moreover, PLWHA would accept the suggestions of family members on COVID-19 vaccination. However, recommendations from a HIV-positive peer and medical professionals had less influence on the acceptance of a COVID-19 vaccine (Table S4 in Multimedia Appendix 2).

    Results of Confirmatory Factor Analysis and SEM

    This study hypothesized that perceived risks, perceived benefits, self-efficacy, and subjective norms were associated with COVID-19 vaccine hesitancy; therefore, these 4 factors were included in the SEM to explore their direct or indirect relationships with vaccine hesitancy.

    Confirmatory Factor Analysis

    Confirmatory factor analysis was used to confirm that each latent factor was being measured appropriately. We used the root mean square error of approximation, normed fit index (NFI), incremental fit index (IFI), Tucker-Lewis index (TLI), and comparative fit index (CFI) to assess whether the model was fit appropriately. The values of NFI, IFI, TLI, and CFI were 0.93, 0.94, 0.93, and 0.94, respectively (all >0.90). The results showed that the hypothesized model had an adequate fit (Table S5 in Multimedia Appendix 2).

    Table 3 showed that the factor loadings for the items were between 0.52 and 0.92 (above 0.5), and the Cronbach α values were between 0.85 and 0.92. It indicated that this online survey had good reliability. The average variance extracted (AVE) and the construct reliability were above 0.5 and 0.7, respectively, which indicated that the convergent validity and internal consistency of this survey were good. According to the discriminant validity analysis, all square roots of AVE were higher than the correlation values, which indicated a good evaluation (Table 4).

    Table 3. Results of reliability and convergent validity analyses.
    Constructs and itemsMeasuresEstimateP valueCronbach αAVEaCRb
    Perceived benefits0.860.530.87

    		PB1COVID-19 vaccination is effective in improving immune function.0.65<.001



    		PB2COVID-19 vaccination is effective in reducing your risk of SARS-CoV-2 infection.0.75<.001



    		PB3COVID-19 vaccination is effective in reducing mortality caused by COVID-19.0.86<.001



    		PB4COVID-19 vaccination is effective in reducing the severity of COVID-19.0.81<.001



    		PB5COVID-19 vaccination is effective in reducing the risk of spreading.0.70<.001



    		PB6Getting COVID-19 vaccination can make you feel relieved.0.52<.001


    Perceived risks0.900.580.91

    		PR1COVID-19 vaccination has severe side effects.0.81<.001



    		PR2COVID-19 vaccination uptake has a significant negative influence on the effectiveness of ARTc.0.76<.001



    		PR3COVID-19 vaccination uptake can reduce immunity.0.70<.001



    		PR4You have concerns about the risk of exposing your PLWHAd identity when taking up the COVID-19 vaccine.0.69<.001



    		PR5COVID-19 vaccination uptake can bring trouble/psychological pressure.0.84<.001



    		PR6COVID-19 vaccination uptake may not produce protective antibodies due to HIV infection.0.80<.001



    		PR7The side effects of COVID-19 vaccination are severer for PLWHA than those without HIV infection.0.70<.001


    Self-efficacy0.920.700.92

    		SFE1You will take up the COVID-19 vaccine even if it interrupts your daily routine.0.79<.001



    		SFE2You will get the COVID-19 vaccine even when you do not feel well.0.81<.001



    		SFE3You will get the COVID-19 vaccine even if the side effects would affect your daily activities.0.92<.001



    		SFE4You will get the COVID-19 vaccine even if HIV infection would reduce its effectiveness.0.84<.001



    		SFE5You will get the COVID-19 vaccine even if it reduces the effectiveness of ART.0.82<.001


    Subjective norms
    		0.850.600.86

    		SN1Your family members will support you to get the COVID-19 vaccine.0.88<.001



    		SN2Your HIV-infected friends will support you to get the COVID-19 vaccine.0.84<.001



    		SN3Medical professionals will support you to get the COVID-19 vaccine.0.74<.001



    		SN4CBOe workers will support you to get the COVID-19 vaccine.0.62<.001


    aAVE: average variance extracted.

    bCR: construct reliability.

    cART: antiretroviral therapy.

    dPLWHA: person living with HIV/AIDS.

    eCBO: community-based organization.

    Table 4. Display discriminant validity analysis.
    ConstructsPerceived benefitsPerceived risksSelf-efficacySubjective norms
    Perceived benefits0.530.320.340.19
    Perceived risks0.320.58–0.16–0.19
    Self-efficacy0.34–0.160.700.41
    Subjective norms0.19–0.190.410.60
    The square root of AVEa0.720.760.840.78

    aAVE: average variance extracted.

    Structural Equation Modeling

    Table 5 shows that the results supported hypothesis 1 (H1) to hypothesis 7 (H7). In other words, respondents who had higher perceived benefits, lower perceived risks, higher self-efficacy, and more support from social networks were more willing to receive the COVID-19 vaccine or were less hesitant to be vaccinated against COVID-19. Perceived benefits, perceived risks, and subjective norms yielded significant direct effects on self-efficacy (β=0.35; β=–0.25; β=0.30, respectively; all P<.001). The relationship between perceived benefits and vaccine hesitancy was partially mediated by self-efficacy (β=0.03, P<.001). The relationship between perceived risks and vaccine hesitancy also was partially mediated by self-efficacy (β=0.08, P<.001). Similarly, the relationship between subjective norms and vaccine hesitancy was partially mediated by self-efficacy (β=–0.29, P<.001). Furthermore, there were direct relationships between perceived benefits, perceived risks, and subjective norms and COVID-19 vaccine hesitancy (β=–0.15; β=–0.08; β=–0.29; β=–0.20, respectively; all P<.001). SEM results are visualized in Figure 2.

    Table 5. Estimation results of the COVID-19 vaccine hesitancy model.
    HypothesisUnstandardized path coefficientStandardized path coefficientSECRaP valueSupport
    H1:PBb-VHc–0.17–0.150.03–5.33<.001Yes
    H2:PRd-VH0.070.080.022.96<.001Yes
    H3:SNe-VH–0.44–0.290.04–11.00.003Yes
    H4:SFEf-VH–0.17–0.200.02–7.37<.001Yes
    H5:PB-SFE0.490.350.0012.20<.001Yes
    H6:PR-SFE–0.24–0.220.03–8.21<.001Yes
    H7:SN-SFE0.560.300.0511.72<.001Yes

    aCR: critical ratio.

    bPB: perceived benefits.

    cVH: vaccine hesitancy.

    dPR: perceived risk.

    eSN: subjective norms.

    fSFE: self-efficacy.

    ‎
    Figure 2. Structural equation modeling results on relationships of perceived benefits (PB), perceived risks (PR), subjective norms (SN), self-efficacy (SFE), and COVID-19 vaccine hesitancy (VH).
    View this figure

    Principal Findings

    This study utilized SEM to investigate relationships among perceived benefits, perceived risks, self-efficacy, subjective norms, and COVID-19 vaccine hesitancy. The study found a high COVID-19 vaccine hesitancy rate among PLWHA in China. Factors associated with vaccine hesitancy were age, a history of chronic diseases, and other vaccinations in the past 3 years. In addition, confidence in COVID-19 vaccination showed the lowest mean of all measured indicators, while perceived risks had the highest mean score. People were highly concerned about possible HIV disclosure during the COVID-19 vaccination. The findings of this study provided valuable information on designing a COVID-19 vaccination campaign addressing possible barriers and improving COVID-19 acceptance among PLWHA.

    In this study, 41.61% (722/1735) of participants reported COVID-19 vaccine hesitancy. The rate was higher than the result of 16.4% generated by a previous nationwide online survey in China [33]. Although the updated Chinese guideline included PLWHA for COVID-19 vaccination, PLWHA might have a higher vaccine hesitancy rate than the general population because of concerns about HIV disclosure, interactions with ART and HIV disease, side effects, and others. Moreover, the vaccine hesitancy rate was higher than that of PLWHA in other nations and regions. Various factors might contribute to the difference, such as sociocultural factors, national policy and guidance, and types of vaccines.

    This study found that vaccine hesitancy was associated with age, and the relationship showed an inverted U-shaped curve. Except for the group aged 40-49 years, older participants showed higher vaccine hesitancy than the younger group. This finding was consistent with a recent French study [34]. Moreover, we found other vaccinations in the past 3 years and a history of chronic diseases were significant predictors of COVID-19 vaccine hesitancy. PLWHA who did not have other vaccinations in the past 3 years and had a history of chronic diseases were more hesitant be vaccinated against COVID-19. The findings could help promote COVID-19 vaccination among PLWHA. More detailed guidelines on COVID-19 vaccination for people with chronic diseases could be widely disseminated to the public and health care providers. PLWHA and HIV doctors must work on managing chronic diseases and eliminating concerns on COVID-19 vaccination.

    We found perceived benefits, perceived risks, and subjective norms yielded significant direct effects on self-efficacy and COVID-19 vaccine hesitancy. The relationships between perceived benefits, perceived risks, subjective norms, and vaccine hesitancy were partially mediated by self-efficacy. The SEM results showed that the higher the perceived benefits, the higher the self-efficacy and the lower the degree of hesitation. Therefore, in order to reduce vaccine hesitation in PLWHA, an education campaign could be developed to provide evidence of the safety and effectiveness of the vaccine, highlighting the latest COVID-19 vaccination guidelines for PLWHA, and informing about the benefits of COVID-19 vaccination from both population and individual perspectives. Previous studies also have highlighted that the safety and efficacy of the COVID-19 vaccine were associated with individuals’ vaccine hesitancy [35,36].

    Perceived risks included participants’ perceptions on vaccine safety and the fear of HIV disclosure. The SEM results showed that the higher the perceived risks, the lower the self-efficacy and the higher the degree of hesitation. Moreover, the fear of HIV disclosure during COVID-19 vaccination was a major concern. HIV stigma exists, and people might hesitate to disclose their HIV status when they receive a COVID-19 vaccine. Unintentional HIV disclosure and related stigma might aggravate their psychological burden [8,37,38]. Some strategies could be proposed to address COVID-19 vaccine hesitancy; for example, HIV clinics could collaborate with COVID-19 vaccination sites to provide COVID-19 vaccines to PLWHA. Health care providers at COVID-19 vaccination sites could underline and inform people about a protocol while protecting individuals’ information and privacy.

    Subjective norms included the support of family members, HIV-infected friends, medical professionals, and CBO workers. The SEM analysis results showed that, with a higher score for subjective norms, the higher the self-efficacy and the lower the degree of hesitation. PLWHA would prefer to accept suggestions regarding COVID-19 vaccination from the support of their family members. On the other hand, the support of an HIV-positive person and medical professionals showed less influence on PLWHA’s decision making. It showed that PLWHA need the strength of their families. COVID-19 vaccination programs based on PLWHA families could be implemented to improve self-efficacy and reduce vaccine hesitancy in PLWHA through family support and mobilization. Although professional medical providers were one of the most trusted groups that could influence vaccine decision making [39], PLWHA could distrust medical staff because of HIV-related stigma and other reasons [40].

    This study had limitations. First, this was a cross-sectional study, so no causality was established. Second, this survey was conducted in PLWHA from 5 large Chinese cities; therefore, the results may not be generalizable to PLWHA in China as COVID-19 vaccine availability, COVID-19 vaccine education, and regional policies and programs might be different among cities and regions. Third, because most of the reported PLWHAs in the 5 selected cities were male, the participants were also majority male. This may influence medical hesitancy, as women are more likely to access medical care. Fourth, because policies and guidelines related to the COVID-19 vaccine have been changing frequently, people’s attitudes about COVID-19 vaccination may vary. Therefore, the findings were sensitive to some factors, such as political and vaccine-related circumstances. Fifth, most measurements in this study were self-constructed and adopted from existing measurements in the general population. The internal validity of these scales was acceptable. However, external validation data were unavailable. Finally, this study did not use random sampling based on the sampling framework, which cannot represent the current situation regarding the vaccination willingness of the entire PLWHA population in China. The extrapolation of the research results needs to be cautious.

    Conclusions

    COVID-19 vaccine hesitancy was high among PLWHA in China. To reduce vaccine hesitation and increase vaccine coverage in PLWHA, social sectors, health facilities, and local communities must work on joint efforts and collaborations to implement strategies and programs that increase COVID-19 vaccination efficacy and eliminate barriers.

    Acknowledgments

    We would like to thank all participants in this study and the staff who contributed during the data collection process. We are also thankful to the Changzhi Medical College and National Institute of Mental Health of the National Institutes of Health for funding this project.

    This study was funded the Academic technology leader project of Changzhi Medical College (Grant No. XSQ201902). The funders had no role in study design, collection, analysis or interpretation of the data, writing the manuscript, or the decision to submit the paper for publication.

    Authors' Contributions

    RC, YY, and JX conceptualized the study and wrote, reviewed, and edited the manuscript. RC, XH, XZ, SL, and YY designed the methodology. JY, XH, MY, GF, GL, QY, and QZ curated the data. RC, JY, and YY performed the formal analysis. JY, XH, MY, GF, and GL performed project administration. XH and JY obtained the resources. QZ, SL,YY, and JX supervised the study. YY and RC contributed equally as first authors. JY, XZ, SL and JX contributed equally as corresponding authors. All authors have read and approved the final manuscript as submitted.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    The process of specific recruitment and data collection.

    DOCX File , 13 KB
    ‎
    Multimedia Appendix 2

    Supplementary tables.

    DOCX File , 34 KB
    1. WHO Coronavirus (COVID-19) Dashboard. World Health Organization.   URL: https://covid19.who.int/ [accessed 2022-05-24]
    2. Mathur R, Rentsch CT, Morton CE, Hulme WJ, Schultze A, MacKenna B, OpenSAFELY Collaborative. Ethnic differences in SARS-CoV-2 infection and COVID-19-related hospitalisation, intensive care unit admission, and death in 17 million adults in England: an observational cohort study using the OpenSAFELY platform. Lancet 2021 May 08;397(10286):1711-1724. [CrossRef] [Medline]
    3. Choi B, Choudhary MC, Regan J, Sparks JA, Padera RF, Qiu X, et al. Persistence and Evolution of SARS-CoV-2 in an Immunocompromised Host. N Engl J Med 2020 Dec 03;383(23):2291-2293 [FREE Full text] [CrossRef] [Medline]
    4. Hensley MK, Bain WG, Jacobs J, Nambulli S, Parikh U, Cillo A, et al. Intractable Coronavirus Disease 2019 (COVID-19) and Prolonged Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Replication in a Chimeric Antigen Receptor-Modified T-Cell Therapy Recipient: A Case Study. Clin Infect Dis 2021 Aug 02;73(3):e815-e821 [FREE Full text] [CrossRef] [Medline]
    5. Khatamzas E, Rehn A, Münchhoff M, Hellmuth J, Gaitzsch E, Weiglein T, et al. Emergence of multiple SARS-CoV-2 mutations in an immunocompromised host. medRxiv. Preprint published on January 15, 2021 2020. [CrossRef]
    6. Bhaskaran K, Rentsch CT, MacKenna B, Schultze A, Mehrkar A, Bates CJ, et al. HIV infection and COVID-19 death: a population-based cohort analysis of UK primary care data and linked national death registrations within the OpenSAFELY platform. The Lancet HIV 2021 Jan;8(1):e24-e32. [CrossRef]
    7. Western Cape Department of Health in collaboration with the National Institute for Communicable Diseases‚ South Africa. Risk Factors for Coronavirus Disease 2019 (COVID-19) Death in a Population Cohort Study from the Western Cape Province, South Africa. Clin Infect Dis 2021 Oct 05;73(7):e2005-e2015 [FREE Full text] [CrossRef] [Medline]
    8. Tran BX, Phan HT, Latkin CA, Nguyen HLT, Hoang CL, Ho CSH, et al. Understanding Global HIV Stigma and Discrimination: Are Contextual Factors Sufficiently Studied? (GAP). Int J Environ Res Public Health 2019 May 29;16(11):1 [FREE Full text] [CrossRef] [Medline]
    9. Canada’s COVID-19 Immunization Plan: Saving Lives and Livelihoods. Public Health Agency of Canada.   URL: https:/​/www.​canada.ca/​content/​dam/​phac-aspc/​documents/​services/​diseases/​2019-novel-coronavirus-infection/​canadas-reponse/​canadas-covid-19-immunization-plan-en.​pdf [accessed 2022-05-24]
    10. Recommendations on the use of COVID-19 vaccines. Government of Canada. 2021 Oct 22.   URL: https:/​/www.​canada.ca/​en/​public-health/​services/​immunization/​national-advisory-committee-on-immunization-naci/​recommendations-use-covid-19-vaccines.​html [accessed 2022-05-24]
    11. COVID-19 vaccines and HIV. UNAIDS. 2021 Jun 01.   URL: https://www.unaids.org/en/resources/documents/2021/covid19 -vaccines-and-hiv [accessed 2022-05-24]
    12. British HIV Association. 2021 Jan 11.   URL: https://www.bhiva.org/SARS-CoV-2-vaccine-advice-for-adults-living-with-HIV- update [accessed 2022-05-24]
    13. HIV and COVID-19 Basics. Centers for Disease Control and Prevention.   URL: https://www.cdc.gov/hiv/basics/covid-19.html [accessed 2022-05-24]
    14. COVID-19 Vaccination Technical Guidelines (First Edition). National Health Commission of the People's Republic of China. 2021 Mar 29.   URL: http://www.nhc.gov.cn/xcs/yqfkdt/202103/c2febfd04fc5498f916b1be080905771.shtml [accessed 2022-05-24]
    15. Kaida A, Brotto LA, Murray MCM, Côté HCF, Albert AY, Nicholson V, et al. Intention to Receive a COVID-19 Vaccine by HIV Status Among a Population-Based Sample of Women and Gender Diverse Individuals in British Columbia, Canada. AIDS Behav 2022 Jul;26(7):2242-2255 [FREE Full text] [CrossRef] [Medline]
    16. Liu Y, Han J, Li X, Chen D, Zhao X, Qiu Y, et al. COVID-19 Vaccination in People Living with HIV (PLWH) in China: A Cross Sectional Study of Vaccine Hesitancy, Safety, and Immunogenicity. Vaccines (Basel) 2021 Dec 09;9(12):1458 [FREE Full text] [CrossRef] [Medline]
    17. Report of the SAGE Working Group on vaccine hesitancy. World Health Organization. 2014 Nov 12.   URL: https:/​/cdn.​who.int/​media/​docs/​default-source/​immunization/​sage/​2014/​october/​sage-working-group-revised-report-vaccine-hesitancy.​pdf?sfvrsn=240a7c1c_4 [accessed 2022-05-24]
    18. Ten threats to global health in 2019. Pan American Health Organization. 2019 Jan 17.   URL: https://www.paho.org/en/news/17-1-2019-ten-threats-global-health-2019 [accessed 2022-05-24]
    19. Betsch C, Schmid P, Heinemeier D, Korn L, Holtmann C, Böhm R. Beyond confidence: Development of a measure assessing the 5C psychological antecedents of vaccination. PLoS One 2018;13(12):e0208601 [FREE Full text] [CrossRef] [Medline]
    20. Hou Z, Tong Y, Du F, Lu L, Zhao S, Yu K, et al. Assessing COVID-19 Vaccine Hesitancy, Confidence, and Public Engagement: A Global Social Listening Study. J Med Internet Res 2021 Jun 11;23(6):e27632 [FREE Full text] [CrossRef] [Medline]
    21. Sallam M. COVID-19 Vaccine Hesitancy Worldwide: A Concise Systematic Review of Vaccine Acceptance Rates. Vaccines (Basel) 2021 Feb 16;9(2):1 [FREE Full text] [CrossRef] [Medline]
    22. Wang J, Lu X, Lai X, Lyu Y, Zhang H, Fenghuang Y, et al. The Changing Acceptance of COVID-19 Vaccination in Different Epidemic Phases in China: A Longitudinal Study. Vaccines (Basel) 2021 Feb 25;9(3):191 [FREE Full text] [CrossRef] [Medline]
    23. MacDonald NE, SAGE Working Group on Vaccine Hesitancy. Vaccine hesitancy: Definition, scope and determinants. Vaccine 2015 Aug 14;33(34):4161-4164 [FREE Full text] [CrossRef] [Medline]
    24. Lin C, Tu P, Beitsch LM. Confidence and Receptivity for COVID-19 Vaccines: A Rapid Systematic Review. Vaccines (Basel) 2020 Dec 30;9(1):16 [FREE Full text] [CrossRef] [Medline]
    25. Vallée A, Fourn E, Majerholc C, Touche P, Zucman D. COVID-19 Vaccine Hesitancy among French People Living with HIV. Vaccines (Basel) 2021 Mar 24;9(4):302 [FREE Full text] [CrossRef] [Medline]
    26. Bogart LM, Ojikutu BO, Tyagi K, Klein DJ, Mutchler MG, Dong L, et al. COVID-19 Related Medical Mistrust, Health Impacts, and Potential Vaccine Hesitancy Among Black Americans Living With HIV. J Acquir Immune Defic Syndr 2021 Feb 01;86(2):200-207 [FREE Full text] [CrossRef] [Medline]
    27. Carlin JB, Hocking J. Design of cross-sectional surveys using cluster sampling: an overview with Australian case studies. Aust N Z J Public Health 1999 Oct;23(5):546-551. [CrossRef] [Medline]
    28. Ferreira AC, Teixeira AL, Silveira MF, Carneiro M. Quality of life predictors for people living with HIV/AIDS in an impoverished region of Brazil. Rev Soc Bras Med Trop 2018;51(6):743-751 [FREE Full text] [CrossRef] [Medline]
    29. Knox SA, Chondros P. Observed intra-cluster correlation coefficients in a cluster survey sample of patient encounters in general practice in Australia. BMC Med Res Methodol 2004 Dec 22;4(1):30 [FREE Full text] [CrossRef] [Medline]
    30. Nunnally JC, Bernstein IH. Psychometric Theory. New York, NY: McGraw-Hill; 1994.
    31. Lau JT, Wang Z, Kim Y, Li J, Gu J, Mo PK, et al. Low sustainability, poor governance, and other challenges encountered by grassroots non-governmental organizations targeting HIV prevention for men who have sex with men in China - a nation-wide study. AIDS Care 2017 Dec 08;29(12):1480-1490. [CrossRef] [Medline]
    32. Fisher KA, Bloomstone SJ, Walder J, Crawford S, Fouayzi H, Mazor KM. Attitudes Toward a Potential SARS-CoV-2 Vaccine : A Survey of U.S. Adults. Ann Intern Med 2020 Dec 15;173(12):964-973 [FREE Full text] [CrossRef] [Medline]
    33. Lin Y, Hu Z, Zhao Q, Alias H, Danaee M, Wong LP. Understanding COVID-19 vaccine demand and hesitancy: A nationwide online survey in China. PLoS Negl Trop Dis 2020 Dec;14(12):e0008961 [FREE Full text] [CrossRef] [Medline]
    34. Schwarzinger M, Watson V, Arwidson P, Alla F, Luchini S. COVID-19 vaccine hesitancy in a representative working-age population in France: a survey experiment based on vaccine characteristics. The Lancet Public Health 2021 Apr;6(4):e210-e221. [CrossRef]
    35. Del Riccio M, Boccalini S, Rigon L, Biamonte MA, Albora G, Giorgetti D, et al. Factors Influencing SARS-CoV-2 Vaccine Acceptance and Hesitancy in a Population-Based Sample in Italy. Vaccines (Basel) 2021 Jun 10;9(6):633 [FREE Full text] [CrossRef] [Medline]
    36. Edwards B, Biddle N, Gray M, Sollis K. COVID-19 vaccine hesitancy and resistance: Correlates in a nationally representative longitudinal survey of the Australian population. PLoS One 2021;16(3):e0248892 [FREE Full text] [CrossRef] [Medline]
    37. Agenda for Zero Discrimination in Health-Care Settings. UNAIDS. 2017 Feb 15.   URL: https://www.unaids.org/en/resources/documents/2017/2017-agenda-zero-discrimination-health-care [accessed 2022-05-24]
    38. Jackson-Best F, Edwards N. Stigma and intersectionality: a systematic review of systematic reviews across HIV/AIDS, mental illness, and physical disability. BMC Public Health 2018 Jul 27;18(1):919 [FREE Full text] [CrossRef] [Medline]
    39. Hildreth JEK, Alcendor DJ. Targeting COVID-19 Vaccine Hesitancy in Minority Populations in the US: Implications for Herd Immunity. Vaccines (Basel) 2021 May 11;9(5):1 [FREE Full text] [CrossRef] [Medline]
    40. Senyurek G, Kavas MV, Ulman YI. Lived experiences of people living with HIV: a descriptive qualitative analysis of their perceptions of themselves, their social spheres, healthcare professionals and the challenges they face daily. BMC Public Health 2021 May 12;21(1):904 [FREE Full text] [CrossRef] [Medline]

    ‎
    ART: antiretroviral therapy
    AVE: average variance extracted
    CBO: community-based organization
    CFI: comparative fit index
    deff: design effect
    IFI: incremental fit index
    NFI: normed fit index
    PLWHA: people living with HIV/AIDS
    SEM: structural equation modeling
    TLI: Tucker-Lewis index

    Edited by T Sanchez, A Mavragani; submitted 03.10.21; peer-reviewed by M Wambura, J Opoku; comments to author 24.01.22; revised version received 13.02.22; accepted 27.04.22; published 30.06.22

    Copyright

    ©Yan Yao, Ruiyu Chai, Jianzhou Yang, Xiangjun Zhang, Xiaojie Huang, Maohe Yu, Geng-feng Fu, Guanghua Lan, Ying Qiao, Qidi Zhou, Shuyue Li, Junjie Xu. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 30.06.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.