This paper is in the following e-collection/theme issue:

Outbreak and Pandemic Preparedness and Management (1727) JMIR Theme Issue: COVID-19 Special Issue (2452) Research Instruments, Questionnaires, and Tools (576) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1517) Infectious Diseases (non-STD/STI) (1575) JMH Theme Issue: COVID-19 and Mental Health: Impact and Interventions (383) Hygiene and Infection Prevention (317) Information Seeking, Information Needs (465)

Published on in Vol 8, No 9 (2022): September

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/34675, first published .
Psychometric Properties of the COVID-19 Pandemic Fatigue Scale: Cross-sectional Online Survey Study

Psychometric Properties of the COVID-19 Pandemic Fatigue Scale: Cross-sectional Online Survey Study

Psychometric Properties of the COVID-19 Pandemic Fatigue Scale: Cross-sectional Online Survey Study

Authors of this article:

Carmen Rodriguez-Blazquez1 Author Orcid Image ;   Maria Romay-Barja2 Author Orcid Image ;   Maria Falcon3 Author Orcid Image ;   Alba Ayala4 Author Orcid Image ;   Maria João Forjaz1 Author Orcid Image
Article Authors Cited by (14) Tweetations (12) Metrics

    Original Paper

    1National Epidemiology Center, Carlos III Health Institute, Madrid, Spain

    2Nacional Center of Tropical Diseases, Carlos III Health Institute, Madrid, Spain

    3School of Medicine, University of Murcia, Murcia, Spain

    4Department of Statistics, Carlos III University, Getafe, Spain

    Corresponding Author:

    Maria Falcon, PhD

    School of Medicine

    University of Murcia

    Campus de Espinardo

    Murcia, 31000

    Spain

    Phone: 34 868887583

    Email: falcon@um.es


    Background: Pandemic fatigue is defined as feelings of demotivation to follow preventive measures against COVID-19, together with decreased trust in government and frequency of information-seeking behaviors.

    Objective: This study aims to analyze the psychometric properties of the COVID-19–specific pandemic fatigue scale according to classical test theory (CTT) and Rasch model approaches in the general Spanish population.

    Methods: This was a cross-sectional study in a representative sample of 1018 adults who completed an online survey in November 2020 in the framework of the COVID-19 Snapshot Monitoring (COSMO)-Spain project. The assessments included the 6-item COVID-19 Pandemic Fatigue Scale (CPFS) and other COVID-19–related variables: COVID-19 infection, adherence to preventive behaviors, information-seeking behavior, self-efficacy, worry, and cognitive and affective risk perception. Data quality, acceptability, reliability, and validity were analyzed according to CTT, and the fit to the Rasch model, unidimensionality, appropriateness of the response scale, item local independency, reliability (person-separation index [PSI]), and item-person distribution were also calculated.

    Results: The mean CPFS score was 17.06 (SD 5.04, range 6-30), with higher scores for women, younger participants, participants who never seek information on COVID-19, those who think they would contract a mild disease in case of infection, those with higher level of worry about coronavirus/COVID-19, and those who felt depressed or felt the coronavirus/COVID-19 is spreading slowly (all P<.01). The Cronbach alpha for the CPFS was 0.74. In the confirmatory factor analysis, one factor was identified (root mean square error of approximation [RMSEA]=.02; comparative fit index [CFI]=.99; χ25=8.06, P=.15). The CPFS showed good fit to the Rasch model (χ 224=42.025, P=.01, PSI=.642), unidimensionality (binomial 95% CI –.005 to .045), and item local independency.

    Conclusions: Our results suggest that the CPFS has moderate reliability and internal consistency and it is composed of a single dimension. It is a useful tool to ascertain the level of pandemic fatigue in the general population, which may help to guide the communication and information strategies to face the COVID-19 pandemic.

    JMIR Public Health Surveill 2022;8(9):e34675

    doi:10.2196/34675

    Keywords

    COVID-19; pandemic fatigue; psychometric properties; Rasch analysis; validation; online survey; pandemic; fatigue; mental health; information seeking; health information 


    The world has become familiar with the term “pandemic fatigue” in the context of COVID-19 [1-3]. This term has been used to describe different phenomena related to psychological distress and demotivation to follow preventive measures, as well as decreased trust in the government and frequency of information-seeking behaviors [3].

    In 2020, the World Health Organization (WHO) defined pandemic fatigue as a “demotivation to follow recommended protective behaviors, emerging gradually over time and affected by a number of emotions, experiences and perceptions,” proposing a framework on how to “maintain and reinvigorate” people’s motivation to comply with COVID-19 response policies [4]. The WHO proposes that pandemic fatigue is expressed through an increasing number of people not sufficiently following or accepting recommendations and restrictions or decreasing their effort to keep themselves informed about the pandemic [4].

    The restrictions adopted by the authorities to tackle this public health crisis have saved many lives but have also affected the mental and physical well-being of the population, social cohesion, economic stability, and community resilience [5]. After more than 2 years of restrictions, fatigue is an expected and natural response [4], and several authors have already measured how support of and compliance with nonpharmaceutical interventions (NPIs) have decreased in Spain [6] and worldwide as the pandemic evolves [7-12].

    In Spain, compliance with health authorities’ recommendations has been high [13], and the vaccination campaign has been a success [14,15]. Even so, compliance with NPIs remains important and will be in the future, as long as new variants continue to be a threat [16]. On the other hand, social, psychological, and economic consequences of the pandemic will continue over time, so the availability of a tool that measures pandemic fatigue is crucial.

    Another challenge that public health authorities face is to keep the population informed in the context of a health crisis of unpredictable duration. Disinterest and information fatigue might be an obstacle for adherence to NPIs to combat the pandemic [17]. In Spain, there is a national strategy to improve health communication and address pandemic fatigue [18]. Public health communication strategies should focus on raising awareness in the event of future outbreaks and new restrictions [17].

    Lilleholt et al [19], in 2021, conceptualized pandemic fatigue to represent a general demotivation toward following COVID-19–related health protective behaviors and staying informed about the development of the pandemic. The authors developed and validated the COVID-19 Pandemic Fatigue Scale (CPFS) with the aim of identifying who experiences it, analyzing related emotions and perceptions, and shedding light on the relationship between pandemic fatigue and health protective behaviors.

    The aim of this study was to assess the psychometric properties of the Spanish version of the CPFS to measure pandemic fatigue in the Spanish general population, administered online, using 2 complementary methodological approaches: classic test theory (CTT) and the Rasch model. In addition to reliability and internal validity, Rasch analysis provides unique information such as differential item functioning for population groups and adequacy of the response scale. Finally, scales that fit the Rasch model provide results in a linear scale.


    Design and Procedures

    This is a cross-sectional, observational, nationwide study with survey data collected using an online questionnaire. This study is part of a larger project, the COVID-19 Snapshot Monitoring (COSMO)-SPAIN project [20], based on the COSMO tool developed by the WHO Europe Regional Office [6,21], with the aim of monitoring the knowledge, attitudes, compliance with the preventive measures, and risk perception of the Spanish population toward the COVID-19 pandemic, as well as informing COVID-19 outbreak response measures, including policies, interventions, and communications. More details can be found in the protocol of the COSMO-Spain study [22].

    A nationally representative sample of 1018 subjects living in Spain was recruited. The sample was stratified to match the Spanish general population in terms of age, education, gender, and area of residence. A research company invited the potential participants who fit the inclusion criteria (both sexes, aged 18 years or older, and being able to answer an online questionnaire) and carried out the survey. The research market company has a panel of 157,535 members from the Spanish population. They contacted panel members who fit the inclusion criteria by email; 2655 invitations were sent, 1777 members participated in the survey (response rate 67%), and 1020 complete questionnaires were obtained. The data were collected between November 24, 2020, and November 27, 2020, at the end of the “second pandemic wave” in Spain. During that period, 60,462 cases of COVID-19 were detected, with a cumulative incidence of 128.6 over 14 days [23]. Mobility restrictions and capacity limitations in commercial establishments were present in different Spanish regions.

    Ethical Review

    The Ethics Committee of Carlos III Health Institute (CEI PI 59-2020-v2) approved the study protocol. The survey was anonymous, and the research company provided data with no identifying information to the researchers. Participants were informed on the purpose and characteristics of the study and provided informed consent by clicking a box.

    Variables

    Online Survey

    The online survey included questions about participants’ sociodemographic characteristics: sex (male, female), age, education (highest level of education attained: incomplete primary or less, primary, secondary, high school, and university), area of residence (village, 2000 to 50,000; town, 50,000 to 400,000; city, >400,000), and employment situation (working, student, domestic care, retired/pensioner, long-term unemployed, unemployed, or Spanish temporary employment regulation due to COVID-19).

    CPFS

    The CPFS is a self-reported questionnaire based on the original version by Lilleholt et al [19]. It asks about demotivation toward COVID-19–related health-protective behaviors and staying informed about the development of the pandemic. The CPFS includes 6 items rated from 1 (strongly disagree) to 5 (strongly agree): “I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc.,” “I feel strained from following all of the behavioral regulations and recommendations around COVID-19,” “I am sick of hearing about COVID-19,” “I am tired of restraining myself to save those who are most vulnerable to COVID-19,” “when friends or family members talk about COVID-19, I try to change the subject because I do not want to talk about it anymore,” and “I am losing my spirit to fight against COVID-19.” The total CPFS score is obtained by summing the items, with a maximum of 30 points that is indicative of a higher degree of COVID-19 pandemic fatigue. In the original study, it reached Cronbach α values of 0.83 and 0.87 (Danish and German studies, respectively) [19].

    Other Variables

    Other variables were included, as described in the following paragraphs (see also Multimedia Appendix 1).

    COVID-19 infection was ascertained using the question “To your knowledge, are you, or have you been, infected with COVID-19?”, with yes/no response options.

    Adherence to preventive behaviors was assessed by asking how frequently respondents carried out a list of 12 measures to prevent infection from coronavirus/COVID-19, with 1 (never) to 5 (always) as scoring options. The listed behaviors included use of face masks: (1) using face masks following the recommendations and (2) wearing face masks in the presence of relatives and friends. It also included questions on hygienic behavior: (3) ventilating closed spaces; (4) using hydro alcoholic gel or disinfectants; (5) disinfecting surfaces; (6) washing hands; and (7) avoiding touching eyes, nose, and mouth with unwashed hands. Finally, it included physical distancing: (8) avoiding public transportation, (9) ensuring physical distancing, (10) avoiding social or family events, (11) not visiting relatives and friends if they are in quarantine, and (12) avoiding crowded spaces. The total number of preventive behaviors was calculated for each participant, computing the scores 4 and 5 as a positive response (=1) and summing them to obtain a score ranging from 0 to 12. The same procedure was applied to calculate the score for each type of preventive behavior.

    Information-seeking behavior was assessed by asking respondents about the frequency of searching for information on coronavirus/COVID-19, answered on a scale from 1 (never) to 5 (several times a day).

    Perceived self-efficacy was surveyed using the question “avoiding an infection with coronavirus/COVID-19 in the current situation is...?”, with a response scale from 1 (very difficult) to 5 (very easy). This question, addressing self-assessed COVID-19 self-protection and avoidance ability, has been adapted from a previous study [24] by the original authors of the COSMO survey [21].

    Level of worry about the coronavirus/COVID-19 in general was collected using a response scale from 1 (do not worry at all) to 5 (worry a lot).

    Cognitive risk perception was measured using a question on perceived probability of getting infected with coronavirus/COVID-19, answered from 1 (very unlikely) to 5 (very likely), and a question on how severe would contracting the coronavirus/COVID-19 be for you, answered on a scale from 1 (not severe) to 5 (very severe) [19]. Both items were multiplied to obtain the value of the cognitive risk perception, ranging from 1 to 25, with higher scores indicative of higher risk perception.

    Affective risk perception was collected using “the coronavirus/COVID-19 to me feels...,” including 3 items: speed of propagation, with a response scale ranging from 1 (spreading slowly) and 5 (spreading fast); fear, scored from 1 (not fear-inducing) to 5 (fear-inducing); and mood, with a scale from 1 (it does not affect my mood) to 5 (makes me feel depressed) [19]. The responses to these questions were summed, obtaining a score that ranged from 3 to 15, with higher values indicating higher affective risk perception.

    All items were originally in English and were translated by professional translators, reviewed and slightly modified by the COSMO-Spain team to adapt them to the Spanish context.

    Data Analysis

    Variables were summarized using descriptive statistics, including central tendency and dispersion measures (mean, median, and SD) and frequency and percentages, depending on their format.

    Since the total CPFS score fit a normal distribution (Shapiro-Wilk test, P=.26), parametric statistics were used. According to the CTT [25], the following psychometric properties of the CPFS were analyzed: data quality and acceptability, structural validity, hypotheses testing (construct validity), and internal consistency.

    Data quality and acceptability were computed by the mean, median, SD, and range of the observed versus theoretical values; skewness (criterion: −1 to +1); floor and ceiling effects (criterion: ≤15%) of the CPFS items; and total score [26].

    For structural validity, exploratory (EFA) and confirmatory factor analyses (CFA) were used. For EFA, a principal component analysis with varimax rotation was applied. CFA used maximum likelihood estimations. A root mean squared error of approximation (RMSEA) ≤0.06 and comparative fit index (CFI) >0.9 indicated a good fit to the model [27].

    Hypotheses testing (construct validity) included convergent and discriminative validity. Convergent validity was analyzed using Pearson correlation coefficients to ascertain the association of pandemic fatigue with related continuous variables: age, number and type of protective behaviors, and cognitive and affective risk perception. Following the literature [4,19], moderate-to-high correlation coefficients (r≥0.30 and r≥0.60) [28] between CPFS and these variables were hypothesized. Regarding discriminative (known groups) validity, mean differences in total CPFS score in the sample grouped by relevant variables were calculated, using ANOVA and Student t tests. The following hypotheses were established, according to the literature [4,12]: a higher CPFS score would be reached in younger participants; those with lower education levels, risk perception, perceived severity, self-efficacy, level of worry, or information-seeking behavior; and those with higher levels of depression.

    Internal consistency was examined by computing the Cronbach α coefficient (criterion ≥0.70), item total corrected correlations (standard r ≥0.40), interitem correlations, and the item homogeneity index (criterion >.30) [29].

    The Rasch model, one of the most used applications of item response theory, was also applied to complete the information on the measurement properties of the CPFS provided by the CTT. According to the Rasch model, the answer to a certain item is a function between the person’s ability (level of pandemic fatigue) and the item’s difficulty (level of construct represented by that item), expressed in logits [30]. The following measurement properties were assessed: fit to the Rasch model, unidimensionality, appropriateness of the response scale, item local independency, reliability (person-separation index [PSI]), and item-person distribution. There are excellent tutorials and examples explaining the Rasch analysis process [31,32].

    Since small deviations from the Rasch model are signaled as statistically significant when using large sample sizes, resulting in unnecessary model modifications, a random sample of 300 was drawn. This sample size allows for stable estimates regardless of targeting [33]. Fit to the Rasch model was considered when there was a nonsignificant chi-square test using Bonferroni correction for number of items (P>.008) [31]. Also, fit residuals were expected to be within the interval of –2.5 to +2.5 and item and person estimates to follow a normal distribution with a mean of 0 and SD of 1. Modifications were performed iteratively until model fit is achieved. PSI measures reliability and is interpreted similarly to Cronbach alpha. Threshold is the point of equal answer probability between 2 adjacent response categories. In case of disordered thresholds, adjacent response categories were collapsed. Unidimensionality was checked using a principal component analysis of residuals and then comparing person estimates with a binomial test; a lower bound of the 95% CI should be ≤.05 [34,35]. Local item independency, or the degree to which 1 item response does not lead to the response to another item, was analyzed though the correlation matrix of the residuals [36]. Differential item functioning (DIF) occurs when, for the same construct level, 2 or more sample groups answer in a statistically different way [37]. DIF was inspected through ANOVA by the following groups: age (groups defined by the median: ≤46 years; >46 years), gender, and education level (low: up to 14 years old; medium: secondary or professional training; high: university). Finally, the person-item threshold distribution was visually inspected.

    CTT analysis was performed using SPSS 27.0 (IBM Corp, Armonk, NY) and Rasch analysis using RUMM2030 statistical software.


    The sample was formed by the same number of men and women, with a mean age 46.1 (SD 14.2, range 18-85) years (Table 1). Most participants were working (577/1018, 56.7%), 27.7% (282/1018) of them in a setting with a moderate risk of infection.

    Table 1. Sociodemographic characteristics of the sample (n=1018) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
    VariablesResults, n (%)
    Sex

    		Women509 (50.0)

    		Men509 (50.0)
    Age groups (years)

    		18-29177 (17.4)

    		30-44301 (29.6)

    		45-60336 (33.0)

    		≥61204 (20.0)
    Education level

    		Incomplete primary or less31 (3.0)

    		Primary240 (23.6)

    		Secondary308 (30.3)

    		University439 (43.1)
    Employment

    		Working577 (56.7)

    		Student41 (4.0)

    		Homemaker32 (3.1)

    		Retired/pensioner186 (18.3)

    		Long-term unemployed100 (9.8)

    		Unemployed or ERTEa82 (8.1)
    Type of work

    		With high risk of contagion101 (9.9)

    		With moderate risk of contagion282 (27.7)

    		No risk69 (6.8)

    		Telework102 (10.0)

    		Health care staff23 (2.3)

    aERTE: Spanish Temporary Employment Regulation due to COVID-19; in Spanish, “expediente de regulación temporal de empleo.”

    Psychometric Properties According to CTT

    Table 2 shows the data quality and acceptability analysis of the CPFS. The mean total CPFS score was 17.06 (median 17.0, SD 5.04, range 6-30). Skewness of the total CPFS score was .13. All items reached the expected score range (1-5), and most of them showed a floor effect, especially items 6 and 4. Ceiling effect was marked in items 1 and 3. The total PFS score did not present floor or ceiling effects.

    Table 2. Data quality and acceptability of the COVID-19 Pandemic Fatigue Scale (CPFS) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
    CPFS itemMean (SD)MedianMinimum to maximumFloor effect, %Ceiling effect, %
    1. I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc.3.88 (1.23)4.001-57.343.3
    2. I feel strained from following all of the behavioral regulations and recommendations around COVID-192.77 (1.32)3.001-524.212.5
    3. I am sick of hearing about COVID-193.52 (1.28)4.001-59.830.3
    4. I am tired of restraining myself to save those who are most vulnerable to COVID-192.07 (1.27)2.001-548.37.2
    5. When friends or family members talk about COVID-19, I try to change the subject because I do not want to talk about it anymore2.81 (1.30)3.001-521.613.5
    6. I am losing my spirit to fight against COVID-192.01 (1.20)2.001-549.75.1
    CPFS Total17.06 (5.04)17.006-301.91.4

    Regarding structural validity, EFA identified 2 factors with a correlation coefficient of .70, explaining 62.6% of variance. Figure 1 shows the path diagram of the CPFS using CFA. The 1-factor model obtained an RMSEA of .02 and CFI of .99 (χ25=8.06, P=.15).

    ‎
    Figure 1. Path diagram of the COVID-19 Pandemic Fatigue Scale (CPFS) model in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020. CPFS items: 1. I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc.; 2. I feel strained from following all of the behavioral regulations and recommendations around COVID-19; 3. I am sick of hearing about COVID-19; 4. I am tired of restraining myself to save those who are most vulnerable to COVID-19; 5. When friends or family members talk about COVID-19, I try to change the subject because I do not want to talk about it anymore; 6. I am losing my spirit to fight against COVID-19.
    View this figure

    The construct validity results of the CPFS appear in Table 3. CPFS scores were significantly higher for women, younger participants, participants who never seek information on COVID-19, those who think they would contract a mild disease in case of infection, those with lower level of worry about coronavirus/COVID-19, those who felt depressed, or participants who felt the coronavirus/COVID-19 was spreading slowly. The CPFS correlated with age (r=–0.20; P<.001), number of preventive measures (r=–0.16; P<.001), use of face masks and physical distancing (r=–0.12; P<.001), hygienic behavior (r=–0.13; P<.001), and affective risk perception (r=0.07; P=.03).

    Internal consistency statistics are displayed in Table 4. The Cronbach α was .74, item homogeneity was .33, and item-total corrected correlation ranged from r=0.42 (item 1) to r=0.56 (item 3). Intercorrelation between items ranged from r=0.17 (item 1 with items 4 and 6) to r=0.51 (item 1 with items 3 and 5).

    Table 3. Construct validity of the COVID-19 Pandemic Fatigue Scale (CPFS) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
    VariablesNumber of participants, nCPFS scores, mean (SD)P valueaCorrelation with total CPFSbP value
    Sex

    		Men50916.59 (4.75).003—c—

    		Women50917.53 (5.28)——
    Age (years)d———–0.20<.001
    Age group (years)

    		18-2917718.90 (5.07)<.001——

    		30-4430117.32 (4.93)——

    		45-6033616.51 (5.21)——

    		≥6120415.99 (4.42)——
    Education level

    		Primary or less7817.67 (5.53).47——

    		Secondary50116.92 (4.97)——

    		University43917.10 (5.04)——
    COVID-19 infection (past or present)

    		Yes6917.45 (4.72).40——

    		No94917.03 (5.07)——
    Number of preventive behaviorsd———–0.16<.001
    Type of preventive behaviors

    		Use of face masksd———–0.12<.001

    		Hygienic behaviord———–0.13<.001

    		Physical distancingd———–0.12<.001
    Information-seeking behavior

    		Never (1-2)27018.53 (5.08)<.001——

    		Occasionally (3)40216.67 (4.45)——

    		Several times a day (4-5)34616.35 (5.43)——
    Perceived self-efficacy: for me, avoiding an infection with coronavirus/COVID-19 is...?

    		Very difficult/difficult (1-2)24617.52 (5.22).19——

    		Neutral (3)52817.01 (4.72)——

    		Easy/very easy (4-5)24416.70 (5.50)——
    General level of worry

    		Not worry at all (1-2)11618.81 (5.56)<.001——

    		Moderate (3)30117.55 (4.66)——

    		Worry a lot (4-5)60116.47 (5.02)——
    Cognitive risk perceptiond———–0.01.85
    Perceived probability of getting infected with coronavirus/COVID-19

    		Very unlikely/unlikely (1-2)26817.24 (5.29).03——

    		Neutral (3)48716.65 (4.63)——

    		Likely/very likely (4-5)26317.62 (5.46)——
    How severe would contracting the coronavirus/ COVID-19 be for you

    		Very mild/mild (1-2)17617.59 (5.67).03——

    		Normal (3)48017.28 (4.50)——

    		Severe/very severe (4-5)36216.51 (5.35)——
    Affective risk perceptiond———0.07.03
    The coronavirus/COVID-19 to me feels…

    		It is spreading slowly (1-2)3119.58 (6.37).01——

    		Neutral (3)18517.36 (4.55)——

    		It is spreading fast (4-5)80216.89 (5.07)——
    The coronavirus/ COVID-19 to me feels…

    		Not fear-inducing (1-2)26517.02 (5.40).96——

    		Neutral (3)34417.12 (4.61)——

    		Fear-inducing (4-5)40917.03 (5.16)——
    The coronavirus/ COVID-19 to me feels…

    		It does not affect my mood (1-2)27515.97 (4.94)<.001——

    		Neutral (3)31916.76 (4.60)——

    		Depressed (4-5)42417.99 (5.26)——

    aStudent t and ANOVA tests with Bonferroni correction.

    bPearson correlation coefficients for continuous variables.

    cNot applicable.

    dContinuous variable.

    Table 4. Internal consistency of the COVID-19 Pandemic Fatigue Scale (CPFS) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
    CPFS itemITCCaCronbach α after item deletionIntercorrelations



    		Item 1Item 2Item 3Item 4Item 5Item 6
    1. I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc..42.73—b0.210.510.170.370.17
    2. I feel strained from following all of the behavioral regulations and recommendations around COVID-19.46.720.21—0.290.370.300.39
    3. I am sick of hearing about COVID-19.56.690.510.29—0.230.510.31
    4. I am tired of restraining myself to save those who are most vulnerable to COVID-19.42.720.170.370.23—0.260.43
    5. When friends or family members talk about COVID-19, I try to change the subject because I do not want to talk about it anymore.54.690.370.300.510.26—0.37
    6. I am losing my spirit to fight against COVID-19.50.700.170.390.310.430.37—

    aITCC: item total corrected correlation.

    bNot applicable.

    Psychometric Properties According to the Rasch Model

    The Rasch analysis showed that all items displayed disordered thresholds. After reducing the response options to scales with 2 to 4 points, according to the item, data showed a good fit to the Rasch model (χ 224=42.025; P=.01; PSI=.642; Table 5), unidimensionality (binomial 95% CI: –005 to .045), and item local independency.

    Table 5. Goodness of fit to the Rasch Model of the COVID-19 Pandemic Fatigue Scale (CPFS) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
     Attribute  CriteriaCPFS
    Item fit residual

    		Mean0–.189

    		SD1.852
    Person fit residual

    		Mean0–.285

    		SD1.736
    Item-trait, χ2 (df)Low42.025 (24)
    Interaction P valueNSa.0128
    PSIb>0.70.642
    Unidimensionality

    		Independent t tests<5%2.00%
     95% CI binomial*c.042-.091

    aNS: nonsignificant.

    bPSI: Personal Separation Index

    cLower bound should be ≤.05.

    Table 6 presents the fit at the item level. Item 1 (“I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc.”) showed DIF by age, with older adults overestimating pandemic fatigue (Figure 2). No DIF was observed by sex or education level. The person-item threshold distribution was close to normality, with no floor or ceiling effects and item threshold locations ranging from –2 to 2 logits. There was a lack of items representing persons with lower and higher pandemic fatigue levels (Figure 3).

    Table 6. Individual item fit of the COVID-19 Pandemic Fatigue Scale (CPFS) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
    ItemLocationStandard errorFit residualχ24P value
    1. I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc.–1.596.0991.1778.672.07
    2. I feel strained from following all of the behavioral regulations and recommendations around COVID-190.289.0740.0295.762.22
    3. I am sick of hearing about COVID-19–1.346.097–1.17910.429.03
    4. I am tired of restraining myself to save those who are most vulnerable to COVID-191.777.183–0.7667.832.10
    5. When friends or family members talk about COVID-19, I try to change the subject because I do not want to talk about it anymore–0.220.0900.2513.163.53
    6. I am losing my spirit to fight against COVID-191.095.088–0.6476.167.19
    ‎
    Figure 2. Differential item functioning for item 1, by age groups defined by the median (≤46 years, >46 years) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020.
    View this figure
    ‎
    Figure 3. Person-item threshold distribution of the COVID-19 Pandemic Fatigue Scale (CPFS) in the COVID-19 Snapshot Monitoring (COSMO)-Spain study, November 2020. Note: the distribution of persons (top part) and item thresholds (bottom part) locations are shown on the same logit scale. Persons with higher fatigue level and more difficult thresholds are represented on the right.
    View this figure

    Pandemic fatigue is an important barrier to implementing NPIs, and monitoring its population levels requires the availability of valid and reliable measures [4]. This is the first validation study of the Spanish version of the CPFS scale in a representative sample of the Spanish general population, as part of an international behavioral insights initiative, the COSMO study [6,21]. The use of 2 complementary methodological approaches, the CTT and Rasch model, provides a robust testing of the psychometric properties of the scale.

    Principal Findings

    Results indicate that the Spanish population reported a moderate-to-high level of pandemic fatigue during the second pandemic wave (mean 17.06 points on a 24-point scale). Our data are in accordance with those of another Spanish study that reported a moderate level of pandemic fatigue (around 3 on a 1-4 scale) in the same period, using a different scale [38].

    The CPFS displayed moderate reliability and internal consistency, allowing for group comparisons. Both CFA and Rasch analysis supported that the scale measures a single construct, indicating that the items may be summed to provide a meaningful total score. In addition, Rasch analysis allows converting the raw scores into a true interval scale, supporting the calculation of change scores and use of parametric statistics. The Rasch analysis results also indicate that respondents were not able to distinguish between all levels of the 5-point response scale. If these results are confirmed in further studies, the response scale may be simplified. However, there is no need to change the way the scale is administered, only how it is coded.

    Items 1 (I am tired of all the COVID-19 discussions in TV shows, newspapers and radio programs, etc.) and 3 (I am sick of hearing about COVID-19) had a ceiling effect, indicating that a high percentage of respondents scored the highest level. This is consistent with the findings from the Rasch analysis, where these items presented the lowest locations, meaning that people with low levels of pandemic fatigue will easily endorse items 1 and 3. On the contrary, items 4 and 6 are endorsed by people with higher levels of pandemic fatigue. The item hierarchy indicates that, when people start feeling some pandemic fatigue, they will first feel tired of COVID-19 discussions in the media (item 1). Only respondents with very high levels of pandemic fatigue will acknowledge that they are tired of restraining themselves to save those who are most vulnerable to COVID-19 (item 6). These results support the content validity of the scale.

    One item presented a bias by age, with older adults overestimating pandemic fatigue scores in the same construct level. If further research confirms these results, separate item 1 locations may be calculated for each age groups. In the meanwhile, differences by age should be interpreted cautiously.

    Comparison With Prior Work

    The known-groups validity results showed that young people reported higher levels of pandemic fatigue. This may be explained by a higher impact of NPI on their social lives, which, in Spain, takes place mostly outside of home. Moreover, young people suffer from asymptomatic or mild disease if infected; they are less likely to adhere to preventive measures [8] and have a decreased risk perception [39]. As a result, they play a crucial role in the increase of incidence rates in several countries, such as in Spain during the summer of 2021 [40]. Younger age was also found to be associated with greater risk of decreasing or diminished interest and avoidance of news about COVID-19 [41]. Therefore, it is necessary to develop campaigns and information strategies specifically addressed to this group of population to overcome these difficulties.

    The absence of DIF by sex indicates that the observed significant sex differences are not due to an item bias. We found higher pandemic fatigue scores in women than men, consistent with a study reporting that women are less likely to sustain long-term confinement [42]. However, another study reported higher levels of pandemic fatigue in Spanish men than in women [38]. These discrepancies might be due to how pandemic fatigue was measured in different studies.

    In general, our hypotheses about convergent and discriminative validity were supported by the results. As explained in the previous pararaphs, women and young people showed higher levels of pandemic fatigue. Other variables associated with pandemic fatigue were the number and type of preventive behaviors, although with low correlation coefficients. Although pandemic fatigue impacts the adherence to protective behaviors, as stated in the definition by the WHO [4], in Spain, the levels of compliance with the main protective measures (use of face masks, washing hands, and social distance) were very high [13], and the use of face masks was compulsory at the time of data gathering. These results, similar to those in other countries [2], could be an explanation for the low correlation of the CPFS with preventive measures.

    Decreased information-seeking behavior is another consequence of pandemic fatigue, and, as our results suggest, people who never or almost never look for information on COVID-19 scored significantly higher on the CPFS. Related to this, items 1 and 3 of the CPFS, which enquire about “information fatigue,” reached the highest mean scores. However, it is difficult to judge if the pandemic fatigue caused the decrease in information-seeking behavior. As hypothesized, people who reported higher levels of pandemic fatigue were those with lower levels of concern, who perceived they are unlikely to be infected, who believed the disease was spreading slowly, who thought they would experience mild disease if infected, or those with depression. Although no causal inferences can be inferred here, other studies have found that less fear of COVID-19 predicted diminished interest in or avoidance of COVID-19 news [41], which is part of the pandemic fatigue definition. In addition, information avoidance predicted a reluctance to engage in COVID-19 preventive behaviors in China [43]. Information avoidance was found to be related to more negative attitudes toward information searching, negative affective responses to risk, and perceived information overload [44].

    Limitations and Strengths

    This study has some limitations. First, we used a cross-sectional design, which provides data from the specific time of an evolving pandemic. Second, data were collected using an online survey, which might not reach minority, hard-to-reach population groups. However, the representative sample provides strength for the external validity of the study.

    This study presents information on the measurement properties of the CPFS to measure pandemic fatigue in a valid and reliable way. Study strengths include the use of a representative population sample and both CTT and Rasch model methods. Results indicate that the CPFS is useful to monitor the level of population pandemic fatigue from the perspective of individuals. Being formed by only 6 items, the questionnaire is quick to apply, while providing good-quality measurement data for group comparisons. The use of the CPFS could help identify groups of people at risk of higher pandemic fatigue and the design of adequate intervention programs and information campaigns addressed at them.

    Conclusions

    In conclusion, the Spanish version of the CPFS is a promising questionnaire to measure pandemic fatigue at the population level, an important public health implication. Its strengths include that it is a brief, unidimensional scale, with a reliability level that allows for group comparisons, absence of bias by gender or education level, and satisfactory validity. As weaknesses and room for improvement, the reliability of the CPFS is not suitable for comparisons at the individual level; 1 item presented bias by age, and there was a lower than expected association with some behavioral aspects.

    Further research is needed to test DIF by country. This is very important considering that the CPFS is being used in the WHO behavioral insight survey, in which more than 30 countries are participating. In addition, information on the scale’s sensitivity to change will be very useful to monitor changes due to pandemic progression and public health interventions. In addition, studies of the associated factors to pandemic fatigue in Spain and other countries, measured through the CPFS scale, would be very useful to design public health interventions to prevent and ease pandemic fatigue. Still, our study suggests that younger adults, women, and people with lower risk perception are more susceptible to presenting with higher levels of fatigue. Communication strategies targeted at these groups will likely have a positive impact on lowering pandemic fatigue [18] and, consequently, increase adherence to protection measures.

    Acknowledgments

    The research was funded by the Carlos III Health Institute. The funder had no role in the study design; collection, analysis, and interpretation of data; writing of the paper; or decision to submit for publication.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Description of the variables.

    DOCX File , 13 KB
    1. Michie S, West R, Harvey N. The concept of "fatigue" in tackling covid-19. BMJ 2020 Nov 02;371:m4171. [CrossRef] [Medline]
    2. Reicher S, Drury J. Pandemic fatigue? How adherence to covid-19 regulations has been misrepresented and why it matters. BMJ 2021 Jan 18;372:n137. [CrossRef] [Medline]
    3. Meichtry S, Sugden J, Barnett A. Pandemic Fatigue Is Real—And It’s Spreading. The Wall Street Journal. 2020 Oct 26.   URL: https:/​/www.​wsj.com/​articles/​pandemic-fatigue-is-realand-its-spreading-11603704601?reflink=desktopwebshare_permalink [accessed 2022-08-08]
    4. Regional Office for Europe. Pandemic fatigue: reinvigorating the public to prevent COVID-19: policy considerations for Member States in the WHO European Region. World Health Organization. Regional Office for Europe. 2020.   URL: https://apps.who.int/iris/handle/10665/335820 [accessed 2022-08-08]
    5. Mukhtar S. Psychological health during the coronavirus disease 2019 pandemic outbreak. Int J Soc Psychiatry 2020 Aug 21;66(5):512-516 [FREE Full text] [CrossRef] [Medline]
    6. Rodríguez-Blázquez C, Romay-Barja M, Falcón M, Ayala A, Forjaz MJ. The COSMO-Spain survey: three first rounds of the WHO behavioral insights tool. Front Public Health 2021 May 31;9:678926 [FREE Full text] [CrossRef] [Medline]
    7. Petherick A, Goldszmidt R, Andrade EB, Furst R, Hale T, Pott A, et al. A worldwide assessment of changes in adherence to COVID-19 protective behaviours and hypothesized pandemic fatigue. Nat Hum Behav 2021 Sep;5(9):1145-1160. [CrossRef] [Medline]
    8. Wright L, Fancourt D. Do predictors of adherence to pandemic guidelines change over time? A panel study of 22,000 UK adults during the COVID-19 pandemic. Prev Med 2021 Dec;153:106713 [FREE Full text] [CrossRef] [Medline]
    9. Franzen A, Wöhner F. Fatigue during the COVID-19 pandemic: Evidence of social distancing adherence from a panel study of young adults in Switzerland. PLoS One 2021 Dec 10;16(12):e0261276 [FREE Full text] [CrossRef] [Medline]
    10. Łaszewska A, Helter T, Simon J. Perceptions of Covid-19 lockdowns and related public health measures in Austria: a longitudinal online survey. BMC Public Health 2021 Aug 04;21(1):1502 [FREE Full text] [CrossRef] [Medline]
    11. Haktanir A, Can N, Seki T, Kurnaz MF, Dilmaç B. Do we experience pandemic fatigue? current state, predictors, and prevention. Curr Psychol 2021 Oct 20:1-12 [FREE Full text] [CrossRef] [Medline]
    12. MacIntyre CR, Nguyen P, Chughtai AA, Trent M, Gerber B, Steinhofel K, et al. Mask use, risk-mitigation behaviours and pandemic fatigue during the COVID-19 pandemic in five cities in Australia, the UK and USA: A cross-sectional survey. Int J Infect Dis 2021 May;106:199-207 [FREE Full text] [CrossRef] [Medline]
    13. Beca-Martínez MT, Romay-Barja M, Falcón-Romero M, Rodríguez-Blázquez C, Benito-Llanes A, Forjaz MJ. Compliance with the main preventive measures of COVID-19 in Spain: The role of knowledge, attitudes, practices, and risk perception. Transbound Emerg Dis 2022 Jul 10;69(4):e871-e882 [FREE Full text] [CrossRef] [Medline]
    14. Barandalla I, Alvarez C, Barreiro P, de Mendoza C, González-Crespo R, Soriano V. Impact of scaling up SARS-CoV-2 vaccination on COVID-19 hospitalizations in Spain. Int J Infect Dis 2021 Nov;112:81-88 [FREE Full text] [CrossRef] [Medline]
    15. Amiel S. How struggling Spain became one of Europe's vaccination champions. EuroNews. 2021 Mar 09.   URL: https:/​/www.​euronews.com/​my-europe/​2021/​09/​03/​how-struggling-spain-became-one-of-europe-s-vaccination-champions [accessed 2022-08-08]
    16. de Bruin M, Suk J, Baggio M, Blomquist S, Falcon M, Forjaz M, et al. Behavioural insights and the evolving COVID-19 pandemic. Euro Surveill 2022 May;27(18):2100615 [FREE Full text] [CrossRef] [Medline]
    17. Link E, Rosset M, Freytag A. Patterns of online information seeking and avoidance about SARS-CoV-2 and COVID-19. EJHC 2022 Apr 21;3(1):53-75. [CrossRef]
    18. Recomendaciones sobre Estrategias Comunicativas frente a la Fatiga Pandémica. Consejo Interterritorial del Sistema Nacional de Salud. 2021 Feb 03.   URL: https:/​/www.​sanidad.gob.es/​profesionales/​saludPublica/​ccayes/​alertasActual/​nCov/​documentos/​Recomendaciones_estrategias_comunicacion_fatiga_pandemica.​pdf [accessed 2022-08-08]
    19. Lilleholt L, Zettler I, Betsch C, Böhm R. Pandemic fatigue: measurement, correlates, and consequences. PsyArXiv. Preprint posted online on December 20, 2020 [FREE Full text] [CrossRef]
    20. Monitorización del comportamiento y las actitudes de la población relacionadas con la COVID-19 en España (COSMO-SPAIN). COSMO-Spain.   URL: https://portalcne.isciii.es/cosmo-spain/ [accessed 2022-08-08]
    21. Betsch C, Wieler LH, Habersaat K. Monitoring behavioural insights related to COVID-19. The Lancet 2020 Apr;395(10232):1255-1256. [CrossRef]
    22. Forjaz M, Romay Barja M, Falcón Romero M, Rodriguez-Blazquez C. Spain COVID-19 Snapshot MOnitoring (COSMO Spain): Monitoring knowledge, risk perceptions, preventive behaviours, and public trust in the current coronavirus outbreak in Spain. PsychArchives 2021:1 [FREE Full text] [CrossRef]
    23. Equipo C. Informe nº 54. Situación de COVID-19 en España. Casos diagnosticados a partir 10 de mayo. RENAVE. CNE. CNM (ISCIII). Instituto de Salud Carlos III. 2020 Nov 25.   URL: https:/​/www.​isciii.es/​QueHacemos/​Servicios/​VigilanciaSaludPublicaRENAVE/​EnfermedadesTransmisibles/​Documents/​INFORMES/​Informes%20COVID-19/​Informe%20COVID-19.​%20N%c2%ba%2054_25%20de%20noviembre%20de%202020.​pdf [accessed 2022-08-08]
    24. Renner B, Schwarzer R. The motivation to eat a healthy diet: How intenders and nonintenders differ in terms of risk perception, outcome expectancies, self-efficacy, and nutrition behavior. Polish Psychological Bulletin 2005;36(1):15 [FREE Full text]
    25. Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002 May;11(3):193-205. [CrossRef] [Medline]
    26. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 1995 Aug;4(4):293-307. [CrossRef]
    27. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 1999 Jan;6(1):1-55. [CrossRef]
    28. Fisk JD, Brown MG, Sketris IS, Metz LM, Murray TJ, Stadnyk KJ. A comparison of health utility measures for the evaluation of multiple sclerosis treatments. J Neurol Neurosurg Psychiatry 2005 Jan 01;76(1):58-63 [FREE Full text] [CrossRef] [Medline]
    29. Jenkinson C, Fitzpatrick R. Cross-cultural evaluation of the short form 8-item Parkinson's Disease Questionnaire (PDQ-8): results from America, Canada, Japan, Italy and Spain. Parkinsonism Relat Disord 2007 Feb;13(1):22-28. [CrossRef] [Medline]
    30. Rasch G. Probabilistic Models for Some Intelligence and Attainment Tests. San Diego, CA: Mesa Press; 1993:773-702.
    31. Tennant A, Conaghan PG. The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Rheum 2007 Dec 15;57(8):1358-1362 [FREE Full text] [CrossRef] [Medline]
    32. Pallant JF, Tennant A. An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS). Br J Clin Psychol 2007 Mar 24;46(Pt 1):1-18. [CrossRef] [Medline]
    33. Linacre J. Sample size and item calibration or person measure stability. Rasch Measurement Transactions 1994;7(4):328 [FREE Full text]
    34. Smith EJ. Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. J Appl Meas 2002;3(2):205-231. [Medline]
    35. Tennant A, Pallant JF. Unidimensionality Matters! (A Tale of Two Smiths?). Rasch Measurement Transactions 2006;20(1):1048-1051 [FREE Full text]
    36. Baghaei P. Local dependency and Rasch measures. Rasch Measurement Transactions 2007;21(3):1105-1106 [FREE Full text]
    37. Tennant A, Pallant J. DIF matters: A practical approach to test if Differential Item Functioning makes a difference. Rasch Measurement Transactions 2007;20(4):1082-1084 [FREE Full text]
    38. Cuadrado E, Maldonado MA, Tabernero C, Arenas A, Castillo-Mayén R, Luque B. Construction and validation of a brief pandemic fatigue scale in the context of the coronavirus-19 public health crisis. Int J Public Health 2021 Aug 30;66:1604260 [FREE Full text] [CrossRef] [Medline]
    39. Maytin L, Maytin J, Agarwal P, Krenitsky A, Krenitsky J, Epstein RS. Attitudes and perceptions toward COVID-19 digital surveillance: survey of young adults in the United States. JMIR Form Res 2021 Jan 08;5(1):e23000 [FREE Full text] [CrossRef] [Medline]
    40. Spain's COVID-19 incidence rate rises, but officials see signs of hope. Reuters. 2021 Jul 26.   URL: https:/​/www.​reuters.com/​world/​europe/​spains-covid-19-incidence-rate-rises-officials-see-signs-hope-2021-07-26/​ [accessed 2022-08-08]
    41. Buneviciene I, Bunevicius R, Bagdonas S, Bunevicius A. COVID-19 media fatigue: predictors of decreasing interest and avoidance of COVID-19-related news. Public Health 2021 Jul;196:124-128 [FREE Full text] [CrossRef] [Medline]
    42. Martinez-Garcia M, Rabasa A, Barber X, Polotskaya K, Roomp K, Oliver N. Key factors affecting people's unwillingness to be confined during the COVID-19 pandemic in Spain: a large-scale population study. Sci Rep 2021 Sep 20;11(1):18626 [FREE Full text] [CrossRef] [Medline]
    43. Song S, Yao X, Wen N. What motivates Chinese consumers to avoid information about the COVID-19 pandemic?: The perspective of the stimulus-organism-response model. Inf Process Manag 2021 Jan;58(1):102407 [FREE Full text] [CrossRef] [Medline]
    44. Link E. Information avoidance during health crises: Predictors of avoiding information about the COVID-19 pandemic among german news consumers. Inf Process Manag 2021 Nov;58(6):102714 [FREE Full text] [CrossRef] [Medline]

    ‎
    CFA: confirmatory factor analyses
    CFI: comparative fit index
    COSMO: COVID-19 Snapshot Monitoring
    CPFS: COVID-19 Pandemic Fatigue Scale
    CTT: classical test theory
    DIF: differential item functioning
    EFA: exploratory factor analyses
    NPI: nonpharmaceutical intervention
    PSI: person-separation index
    RMSEA: root mean square error of approximation
    WHO: World Health Organization

    Edited by T Sanchez, A Mavragani; submitted 03.11.21; peer-reviewed by S Song, J Voss; comments to author 30.04.22; revised version received 18.05.22; accepted 28.06.22; published 08.09.22

    Copyright

    ©Carmen Rodriguez-Blazquez, Maria Romay-Barja, Maria Falcon, Alba Ayala, Maria João Forjaz. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 08.09.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.