We used data from the TT-C randomized control trial, which included Black young adults aged 18‐29 years who were unvaccinated or insufficiently vaccinated against COVID-19 in Alabama, Georgia, and North Carolina. Participants were recruited online or via social media and randomized 1:1 to either the TT-C DHI, a self-directed asynchronous app developed to enable Black young adults in Southern United States to make decisions about COVID-19 vaccine receipt via nonstigmatizing and tailored messaging, or an information standard-of-care control consisting of COVID-19 vaccine materials adapted from the Centers for Disease Control and Prevention for three months. Surveys were completed at baseline, one month, and three months after randomization. Additional details about the study, which was conducted between March and December 2023, have been published previously [30] .

Five baseline variables related to general vaccine attitudes at baseline were used as latent indicators in the LPA: vaccine hesitancy, vaccine confidence, vaccine-related conspiracy beliefs, vaccine knowledge, and group-based medical mistrust. Each variable was assessed using a previously validated scale with a standardized continuous score as input using z-scores, such that the mean score transformed to zero to remove the impact of any baseline differences in means or variances across the variables before they were used in the LPA. This approach helps mitigate the influence of baseline distribution differences in the clustering process by making the variables comparable. The measures used in this LPA were prespecified in the study protocol and tested in formative surveys with Black young adults [30,33]. Vaccine hesitancy was a mean score of a 9-item scale with Likert responses ranging from 1‐5, where higher scores indicated greater hesitancy [34]. Vaccine confidence was a mean score of a 7-item scale with Likert responses ranging from 1‐5, where higher scores indicated greater confidence [35]. Vaccine-related conspiracy beliefs were a mean score of an adapted 5-item scale with Likert responses ranging from 1‐4, where higher scores indicated greater belief in conspiracy [36]. Vaccine knowledge was a sum score of an adapted 4-item scale with “yes” or “no” responses for each item, where higher scores indicated greater knowledge [37]. Group-based medical mistrust was a sum score of a 12-item scale with Likert responses ranging from 1‐5, where higher scores indicated greater mistrust [38]. For mean scores, if a response was missing for more than one scale item, the mean score was set as missing and not calculated. For summed scores, if a response was missing for any scale item, the summed score was set as missing and not calculated.

Once latent profiles were established, differences in several baseline sociodemographic variables and other measures were assessed between the profiles. These included self-reported: sex assigned at birth, gender, age, education (ie, 9th-12th grade; no diploma; high school graduate or GED completed; some college level or technical or vocational degree; bachelor’s degree; other advanced degree), political affiliation (ie, Democrat, Republican, Independent, other), employment status (eg, full-time; part-time; student; not working; disabled, permanently or temporarily), annual income, insurance coverage (eg, public, private, none), social support (continuous scale) [39], and previous flu and human papillomavirus vaccination.

The primary outcome was uptake of a COVID-19 vaccine anytime during the 3-month follow-up period, based on uploaded vaccine cards or, if not uploaded, self-report at one month and three months. Secondary outcomes included vaccine hesitancy, vaccine confidence, vaccine-related conspiracy beliefs, and vaccine knowledge, which were reported at 3 months. These secondary measures were assessed in the same manner as the baseline latent indicators explained above. Latent profiles were used to assess effect measure modification (EMM) of the TT-C DHI on both primary and secondary outcomes.

Latent profile analysis allows us to gain insights into “hidden” subgroups in a study sample using person-centered mixture modeling. This approach models how indicator variables relate to each other by estimating different profiles (or classes) and determining the best-fitting model [40,41]. In LPA, model parameters include the probability of an individual belonging to a given profile, and each profile can have its own set of means and variances.

We evaluated two different model specifications: (1) equal variance and covariance across profiles and (2) varied variances and covariances across profiles. For each specification, we tested models with two, three, and four profiles. Model fit was assessed using an analytical hierarchy process based on the following information criteria: Akaike information criterion, Bayesian information criterion, Approximate Weight of Evidence, consistent likelihood criterion, and Kullback information criterion [42]. Lower values indicate a better fit for each criterion. Additional criteria including the sample size-adjusted Bayesian information criterion, entropy, and the bootstrap likelihood ratio (BLRT) were used to assess overall model quality. For these criteria, a lower sample size-adjusted Bayesian information criterion, higher entropy, and statistically significant BLRT indicate a good fit. The best model was selected using an analytical hierarchical process that balanced low values of Akaike information criterion, Bayesian information Criterion, Approximate Weight of Evidence, consistent likelihood criterion, and Kullback information criterion, which indicate a more parsimonious model, with fewer parameters while still accounting for data complexity, with higher entropy to ensure distinct profile separation and a significant BLRT to confirm robustness [42]. After model selection, we assigned study participants to the profile with the highest posterior probability based on their pattern of responses across the variables. Descriptive statistics were then calculated for the sociodemographic and additional variables within each profile. Differences in continuous variables across profiles were tested using ANOVA, while differences in categorical variables were assessed with the χ² test of independence.

The primary outcome of vaccine uptake at three months was assessed using a logistic regression model. The odds ratio approximates the risk ratio, as the probability of vaccine uptake was not common (<15%). For secondary outcomes—vaccine hesitancy, vaccine confidence, and vaccine-related conspiracy beliefs—we used linear regression to determine differences in outcomes at three months. Poisson regression was used to test for differences in the secondary outcome of vaccine knowledge score at three months. The main effects of the TT-C DHI intervention on vaccine uptake and vaccine attitudes will be reported in the primary manuscript of the study (Budhwani et al, PhD, unpublished data, 2024). The EMM of the effect of the TT-C DHI intervention on primary and secondary outcomes by profile group was assessed in this study by including the study arm and latent profile at baseline as both main effects and interaction terms. The EMM assesses whether an intervention effect (ie, the effect of the intervention on the outcome in the intervention group compared to the control group) differs across a variable of interest (in this case, the LPA groups). Interaction terms with a P value <.10 were considered indicators of significant effect measure modification. For all models, the state of residence was controlled to account for its inclusion in the stratified randomization study design. We did not control for other variables as they were balanced by the randomized design. All analyses were conducted using R software (version 4.4.1; R Foundation for Statistical Computing) [43].

A sensitivity analysis was conducted using latent class analysis (LCA), typically used with categorical variables, where all indicator variables were dichotomized at appropriate cut-points for each validated scale [44]. We tested LCA models with two, three, and four classes and used similar criteria to assess fit. We then compared the number of classes identified by LCA to the number of profiles identified by LPA, as well as the total variance explained and general sociodemographic differences across the classes/profiles. We chose to report LPA as the main analysis to maintain the continuous nature of the originally validated measures used as indicator variables and because of the added model flexibility of LPA with varying variances and covariances across profiles.

All study materials and procedures were reviewed and approved by the Institutional Review Boards of the University of North Carolina, Chapel Hill (21‐1746) and Florida State University (STUDY00003617). Informed consent was collected digitally from all study participants before data collection and randomization. All data were deidentified to safeguard participant information.

Using the prespecified fit criteria with the analytic hierarchy process, we found that the model with three profiles and varied variances and covariances across profiles provided the best fit (Table 1). The profiles in Figure 1 were named based on the mean values of the five indicator variables, all of which were significantly different (P<.001) between the profiles (Table 2). The vaccine-receptive group (Profile 2) included 124 participants (34%) and had the lowest mean scores for medical mistrust, conspiracy beliefs, and vaccine hesitancy and the highest mean score for vaccine confidence and knowledge of vaccines compared to other profiles. The vaccine-resistant group (Profile 3) included 81 participants (23%) and had the highest mean score for medical mistrust, conspiracy beliefs, and vaccine hesitancy and the lowest mean score for confidence and knowledge of vaccines compared to other profiles. The vaccine-neutral group (Profile 1) included 155 participants (43%) and had mean scores between those of the vaccine-receptive and vaccine-resistant groups for all indicator variables.

Individuals in the vaccine-receptive group was significantly more likely to have been vaccinated for the flu in the past 12 months (n=40, 41%) compared with those in the vaccine-neutral (n=33, 29%) and the vaccine-resistant (n=7, 15%) groups (P=.01). Additionally, the vaccine-receptive group was more likely to report Democratic political affiliation (n=74, 60%) compared to the vaccine-neutral (n=70, 45%) and vaccinate-resistant (n=27, 33%) groups (P=.001). Reported annual income and insurance coverage were highest among the vaccine-receptive group; however, there were no significant differences in educational attainment or employment status across profiles. The vaccine-receptive group also tended to be younger (mean age, 23.3, SD 3.7) years and reported more social support (mean score, 4.1, SD 0.8) compared with both the vaccine-neutral (mean age, 23.6, SD 3.3 years; mean social support score, 3.8, SD, 0.9) and vaccine-resistant (mean age, 24.5, SD 3.1 years; mean social support score, 3.6, SD 1.1) groups (P=.05).

Effect measure modification analysis showed that the TT-C DHI did not have a different effect on vaccine uptake across latent profile subgroups (Table 3). For secondary outcomes, EMM showed that the TT-C DHI had differing effects among those identified as vaccine-resistant compared with those classified as vaccine-receptive at baseline for the outcomes of vaccine hesitancy (interaction P=.07), vaccine confidence (interaction P=.06), and vaccine-related conspiracy beliefs (interaction P=.04) at 3 months (Table 3). There were also differing effects between vaccine-neutral and vaccine-receptive groups from baseline for the outcomes of vaccine hesitancy (interaction P=.05) and vaccine confidence (interaction P=.05) at 3 months. The vaccine-resistant group showed the greatest effect from the TT-C DHI, with a reduced mean vaccine hesitancy score of −0.40 (95% CI −0.76 to −0.37), an increased mean vaccine confidence score of 0.39 (95% CI 0.02 to 0.75), and a reduced conspiracy belief score of −0.47 (95% CI −0.86 to −0.09). The vaccine-neutral subgroup experienced moderate effects, including a reduced mean vaccine hesitancy score of −0.36 (95% CI −0.52 to −0.19) and an increased mean vaccine confidence score of 0.35 (95% CI 0.18 to 0.51). The TT-C DHI had no significant effect on any of the secondary vaccine-related outcomes among the vaccine-receptive subgroup.

When using the prespecified fit criteria with the analytic hierarchy process, we found the LCA model with the best fit to include three classes, similar to the LPA results. Similar to our LPA results, the three classes represented subgroups of study participants who were either: (1) vaccine-receptive, with low mean scores for mistrust, conspiracy beliefs, or hesitancy and high mean scores for knowledge and confidence; (2) vaccine-resistant, with high mean scores for mistrust, conspiracy beliefs, and hesitancy and low mean scores for knowledge or confidence; or (3) somewhere in between the two, as vaccine-neutral. The classes from the LCA had similar differences as the profiles in the LPA analysis with respect to political affiliation, income, insurance coverage, and flu vaccination. Thus, LCA results reinforce the findings from the LPA.