Racial Composition of Social Environments Over the Life Course Using the Pictorial Racial Composition Measure: Development and Validation Study

Introduction

Background

Research investigating the impact of racial segregation on health has typically focused on neighborhood racial composition [1-9]. Some scholars argue that this is due to racial segregation being an essential driver of health disparities [8,10]. However, evidence demonstrates the potential positive impacts of racial segregation [11,12]. For example, in a North Carolina study of Black church members, living in areas with a high proportion of Black individuals was associated with high physical activity [12]. Another example can be found in a study of New York City residents that demonstrated lower rates of all-cause mortality, coronary heart disease, and cardiovascular disease among older racialized Black people living in areas predominantly populated by Black people [13]. These mixed findings may result from different measures of racial segregation used across studies.

Most studies have used census data [1-5,7,8] or investigator-adapted questions [14-18] to measure racial segregation. Census data allow researchers to calculate the percentage of Black individuals in a census region and use this as an independent variable in regression analyses. Others have operationalized census data to construct local segregation-based measures such as the Index of Concentration at the Extremes [19,20], the dissimilarity index [21,22], the isolation index [23,24], the interaction index [23,25], the location quotient of residential segregation [26,27], and the Getis-Ord Gi* statistic [28].

Although census data can objectively measure neighborhood composition, their geographical ranges may be too broad and may not represent what study participants perceive as their neighborhood [16,29]. Therefore, some researchers use investigator-adapted survey items instead of census data to measure racial segregation [14-18]. Perceived neighborhood racial composition (PNRC) is an investigator-adapted and self-reported measure of the racial composition of a participant’s community [14]. The PNRC measure is limited in several ways. First, it focuses on a single neighborhood aspect [15]. Second, it does not address the difference that an individual may experience on a daily basis when moving between their home and school (or work) [16]. Moving between neighborhoods with varying levels of racial segregation can impact individual outcomes [30,31]. Proximity to predominantly White neighborhoods provides access to better public resources such as parks and pools and lower crime rates [30,31]. In contrast, predominantly Black neighborhoods often remain underdeveloped and underresourced [30,31]. However, proximity to predominantly White neighborhoods may also increase exposure to anti-Black discrimination such as racial profiling [30,31]. Finally, how it is asked on surveys varies across studies.

A measure that can account for differences across multiple social environments at different life stages will be helpful to social scientists. Analyzing the racial composition of social environments over the life course may help in identifying a susceptible period of racial segregation that has a more pronounced deleterious health effect than other periods [32-34]. This can guide public health efforts and interventions aimed at dismantling structural racism, broadening the understanding of psychosocial determinants of health, and elucidating the effects of racism at different developmental stages among populations considered marginalized [35-43].

Another limitation of census-based measures is the lack of validation in populations with low health literacy. Improving health literacy has become a global public health concern prioritized by clinicians, genetic counselors, and health communication specialists [44-46]. Limited health literacy has been associated with increased rates of emergency department visits, unintentional medication nonadherence, and all-cause mortality [47-55]. Creating a visual assessment of the racial composition of social environments provides researchers with an accessible survey measure to use in low health literacy populations, potentially improving patient care, engagement, and education.

An extension of the PNRC approach is needed that can (1) serve as a uniform measure across disciplines, (2) capture multiple social environments, (3) measure the racial composition of these social environments during 1 study visit, and (4) accommodate participants with limited health literacy. Measuring the racial composition over the life course is possible with census data and the PNRC measure. However, both approaches would require a longitudinal design, which is susceptible to participant dropout. Measuring the racial composition of life course environments is crucial because racial segregation is multidimensional, interconnected with racism, operates across various social settings, and has cumulative effects [10,30,56,57].

Objective

We propose a novel approach to measuring the racial composition of social environments over the life course. The Pictorial Racial Composition Measure (PRCM) is a 10-item questionnaire of pictures representing social environments across adolescence and adulthood. The PRCM is intended to differentiate the racial composition of multiple social environments. We develop and validate the PRCM using data from cognitive interviews and surveys administered to a medically underserved patient population. The written version of this racial composition measure was previously validated against census data [14] and has been applied to study the impact of racial segregation on hypertension diagnosis [58] and health literacy [59,60]. This study aimed to validate the pictorial version against the written version.

Methods

Setting

This study was conducted at the Center for Outpatient Health (COH) primary care clinic at the Barnes-Jewish Hospital.

Ethical Considerations

The Human Research Protection Office at the Washington University School of Medicine approved the primary data collection (201212029), and the New York University Institutional Review Board approved the secondary data analysis (IRB-FY2017-213). Participants provided verbal and written consent. This study received a waiver of documentation of consent because all the study data were deidentified.

Study Recruitment and Survey Administration

Figure 1 outlines the convenience sampling approach used to recruit study participants. Study recruitment details have been described elsewhere [55,58,61-65], but in brief, trained research assistants (RAs) approached 4243 individuals in the COH waiting room between July 1, 2013, and April 29, 2014. Eligible patients were aged at least 18 years, a COH patient, and English speaking. Of the 4243 individuals, 41.32% (1753/4243) refused to participate, 26.16% (1110/4243) were ineligible, and 32.52% (1380/4243) provided verbal and written consent to participate in the study.

‎

RAs asked the participants to complete a self-administered written questionnaire and a verbally administered survey component. Surveys were administered on different days of the week and at different times of the day. Of the 1380 participants who gave consent, 73.19% (1010/1380) completed the survey. Survey completion was measured as (1) full completion of the written component or (2) three-fourths completion of the written component and full completion of the verbal component. The primary reason for incomplete surveys was inadequate time between the start of the survey and when the clinic was ready to begin the patient evaluation. Survey respondents were similar to the underlying COH primary care clinic patient population concerning sex, age, race, and location in St Louis. The final analytic sample consisted of 549 respondents who identified as non-Hispanic White or Black (including multiracial) and completed both the written and the pictorial measures. The pilot phase included 241 respondents, and the main phase included 308 respondents.

Measurement Development Stages

Development of the PRCM occurred across the pilot and main phases. From July 1 to August 6, 2013 (pilot phase), the PRCM was self-administered as part of the written survey component. During this phase, RAs also conducted cognitive interviews with 13 participants. From August 13, 2013, to April 30, 2014 (main phase), RAs verbally administered the PRCM to each participant in private COH conference rooms. We refined the PRCM using insights from cognitive interviews and focus groups (data not shown). This refinement leverages mixed methods research and cognitive response interviews, which are important components in developing survey measures, especially among populations with varying levels of health literacy [66].

Cognitive Interviews

We used cognitive interviews to improve the validity of the PRCM. One advantage of using this technique is the higher potential for accurately assessing the racial composition of social environments over the life course [67]. Another advantage is the systematic identification and reduction of measurement errors during the measurement development stage [68]. Cognitive interview participants were selected from the clinic population. We conducted 13 cognitive interviews and stopped once we reached saturation. We achieved saturation in the data with <10 participants but continued with a few more interviews to ensure we had captured important perspectives.

RAs conducted cognitive interviews with 10 Black participants during the pilot phase. RAs presented 10 sets of pictures to these participants. They asked them to select the image that best described the racial composition of the following social environments: current or most recent workplace, place of worship, high school (HS), HS classroom, junior HS, junior HS classroom, current neighborhood, neighborhood growing up, current block, and block growing up. Participants recommended changes to the PRCM. We report selected recommendations and responses in the Results section.

Racial Composition Measures

Written Measure

The written survey component included an item adapted from the Behavioral Risk Factor Surveillance System [69] and the National Survey of Black Americans [70] to measure racial composition. This item asked participants to identify the racial composition of 10 social environments from their past and present: neighborhoods and blocks (adolescent and current), schools and classrooms (junior high and HS), workplace, and place of worship. The 8 possible responses included: mostly Black, about half Black, some Black, mostly White, about half White, some White, did not live or attend school in the United States (only applicable to school environments), and not applicable. We categorized participants’ responses into mostly Black, half Black, or mostly White. Participants who indicated that they did not experience the environment in the United States or that the environment was not applicable to them were excluded from the analysis for that environment.

Pictorial Measure

To construct the PRCM, we used an adapted version of the “neighborhood cards” developed by Krysan and Farley [56]. Figures S1-S10 in Multimedia Appendix 1 show the adapted measures used in the main phase. Different versions of the adapted measure were used during the pilot phase, but they were revised for the main phase based on results from the cognitive interviews and focus groups. RAs presented this pictorial measure to the participants and asked them to select the approximate racial composition of 10 social environments. The 7 possible responses to the pictorial measure included: all Black, 70% Black, 50% Black, 30% Black, 10% Black, all White, and not applicable. As with the written measure, we collapsed the answer choices into the same 3 categories: all Black and 70% Black were categorized as mostly Black, 50% Black and 30% Black were categorized as half Black, and 10% Black and all White were categorized as mostly White. Those who selected not applicable were not included in the analysis for that environment.

Demographics and Health Literacy

Self-reported demographic characteristics included sex (male or female), race (non-Hispanic White or Black), age, current county (St Louis City, St Louis County, or other), education (less than HS, HS or General Educational Diploma, or more than HS), employment status (currently in the workforce or not), and income (<US $9999, US $10,000-$19,999, or >US $20,000). Health literacy was assessed using the Rapid Estimate of Adult Literacy in Medicine–Revised (REALM-R) [71,72] and the Newest Vital Sign (NVS) [73].

Analytic Strategy

We calculated several validity metrics by comparing written and PRCMs (Textbox 1). For each social environment and survey phase (pilot and main), we computed positive versus negative pairwise comparisons: (1) mostly Black versus all other categories, (2) half Black versus all other categories, and (3) mostly White versus all other categories. We calculated the following validity metrics for each pairwise comparison: sensitivity, specificity, correct classification rate (CCR), positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR–), false positive rate, and false negative rate. All validity statistics were calculated using SAS and STAT software (version 9.4; SAS Institute Inc).

Results

Cognitive Interviews

Table 1 presents descriptive statistics of the 10 Black cognitive interview participants and their recommended changes to the PRCM. Most of the participants were female (7/10, 70%), aged at least 26 years (9/10, 90%), HS graduates (9/10, 90%), and earning <US $30,000 (9/10, 90%). In addition, the proportions of those with inadequate health literacy differed between the REALM-R and the NVS.

Among the 10 social environments, the initial items for 3 environments had the lowest number of respondents who recommended changes be made: place of worship, current workplace, and current block. These results indicate that respondents experienced the least confusion when seeing those pictures. For example, when asked to select the picture that best described the racial makeup of their place of worship, 1 respondent answered, “it’s asking the race that I worship with, that I go to church with. It’s an all-black church.” This demonstrates that these set of pictures were easy to interpret.

In contrast, participants reported that the past and current neighborhood pictures were the most difficult to interpret. Reasons for this included the indistinguishable color of the houses, the small size of the houses, and the placement of trees. For example, 1 person asked, “Are those supposed to be trees?...It’s kind of confusing.” Another respondent found the pictures unclear, stating, “Now this one is kind of confusing to me as far as these little houses are concerned, because I’m not quite sure if the ones that are painted black mean that black people live there and the ones painted white means white people.” Respondents suggested altering the picture to “make the houses bigger” and “adding people” to make it clearer.

Survey Data: Analytic Sample Characteristics

As shown in Table 2, most of the analytic sample was female (363/549, 66.1%), Black (364/549, 66.3%), and aged >44 years (387/549, 70.5%). Approximately half (257/549, 46.8%) of the participants lived in St Louis City, and 82.1% (451/549) of the participants had at least a HS education. Most participants reported earning <US $20,000 (354/549, 64.4%) and not being in the workforce (408/549, 74.3%). A higher proportion of the sample was categorized as having inadequate health literacy using the NVS than the REALM-R.

Pilot Phase

Validity statistics for the pictorial against the written racial composition measure during the pilot phase are summarized in Table 3. Across all 10 social environments, both dichotomizations of mostly Black versus other categories and mostly White versus other categories exhibited a moderate-to-high sensitivity, specificity, CCR, PPV, and NPV. The LR+ values were >1, and the LR– values were close to 0. The false positive and negative rates were low to moderate. These values indicate that using either the mostly Black versus other categories or the mostly White versus other categories dichotomizations may provide accurate and reliable measures of racial composition across the 10 social environments.

The half Black versus other categories dichotomization yielded a good specificity, CCR, NPV, and false positive rate. However, we observed wide ranges for sensitivity, PPV, and false negative rate. The half Black LR+ ranged from 1 to 13, and the half Black LR– ranged from 0.2 to 1.0. These findings suggest half Black dichotomization may be less accurate and reliable than the mostly Black and mostly White dichotomizations when measuring the racial composition of the 10 social environments.

Main Phase

Table 4 compares the validity statistics for the pictorial against the written racial composition measure during the main phase. Across all 10 social environments, the dichotomizations of mostly Black versus other categories and mostly White versus other categories yielded a moderate-to-high sensitivity, specificity, CCR, PPV, and NPV. The LR+ values were >1, and the LR– values were close to 0. These results were similar to the pilot phase results. However, the false positive and negative rates in the main phase slightly differed from those in the pilot phase. Overall, these findings further support that using either the mostly Black versus other categories or the mostly White versus other categories dichotomizations may provide accurate and reliable measures of racial composition across the 10 social environments.

The specificity, CCR, NPV, LR+, and LR– were relatively good for the half Black versus other categories dichotomization. However, this dichotomization demonstrated poor validity regarding its sensitivity, PPV, and false negative rate. These values suggest that the half Black versus other categories dichotomization may not be as accurate and reliable as the mostly Black versus other categories and mostly White versus other categories dichotomizations.

Discussion

Principal Findings

We developed and validated the PRCM, a measure of the racial composition of social environments over the life course. Pilot and main phase results across all 10 social environments showed that the mostly Black and mostly White dichotomizations exhibited moderate-to-high scores for sensitivity, specificity, CCR, PPV, and NPV. These dichotomizations also yielded acceptable values for LR+ and LR– and false positive and negative rates. These findings support that the mostly Black and mostly White dichotomizations may provide accurate and reliable measures of racial composition across the 10 social environments.

The PRCM provides an added window into understanding how racial segregation has impacted minoritized communities and can be used to investigate interpersonal, institutional, structural, and systemic racism [74,75]. Although further empirical research is needed from a generalizable population, the mostly Black and mostly White dichotomization findings provide preliminary evidence that an optimal cut-off point exists for the categorization of segregation in social environments. We validated a pictorial measure in a population composed mainly of those with low health literacy according to the NVS. Thus, we provide a novel and accessible resource for clinical researchers to measure the racial composition of their patient’s social environments. Analyses using the PRCM may lead to a more comprehensive understanding of how segregated social environments over the life course affect health outcomes.

The mostly Black and mostly White dichotomizations performed better than the half Black dichotomization, possibly because the older study population (387/549, 70.5% aged at least 45 years) may have been more likely to remember predominantly Black or White social environments compared to 50% Black environments. In addition, we grouped 50% Black and 30% Black together for the half Black dichotomization. It may be that 30% Black is too far from 50%, potentially contributing to the poor validation of the half Black dichotomization. Overall, these results were similar to those of sensitivity analyses stratified by race (data not shown). There was also a smaller number of participants reporting half Black environments, providing limited data upon which to validate this level.

All explored social environments exhibited good validity metrics for mostly Black and mostly White dichotomizations. However, neighborhood growing up, current neighborhood, and current workplace tended to perform the lowest. This could stem from the images on the pictorial cards for those environments. The neighborhood cards showed houses, which might not resonate with participants who lived in alternative housing arrangements such as apartments or mobile homes. Similarly, the workplace pictorial cards displayed office desks with computers. It is possible that some participants may have never worked in an office setting.

Comparison With Prior Work

Other approaches used to measure racial segregation include the Index of Concentration at the Extremes [19,20], the dissimilarity index [21,22], the isolation index [23,24], the interaction index [23,25], the location quotient of residential segregation [26,27], and the Getis-Ord Gi* statistic [28]. The Index of Concentration at the Extremes assesses the degree to which individuals in an area are privileged [19,20]. The dissimilarity index is the proportion of individuals considered marginalized needing to relocate to create an equal racial and ethnic distribution within the population [21,22]. The isolation index estimates the spatial proximity between individuals of the same racial category living in the same areas [23,24]. The interaction index estimates the spatial proximity between individuals of different racial categories residing in the same areas [23,25]. The location quotient of residential segregation compares the racial and ethnic distribution of a smaller geographic region to that of a larger geographic region [26,27]. The Getis-Ord Gi* statistic measures each neighborhood’s racial composition and compares it to surrounding neighborhoods [28].

Despite the advantages of these census-based approaches, none can measure the potentially varying racial compositions of multiple social environments across the life course. These census-based approaches may capture regions that do not match what participants consider as their neighborhood [16,29]. An example is a Black participant living in a census region but attending a predominantly White school or a racially integrated church in another census region. Another shortcoming is that these census-based measurements have yet to be validated in low health literacy populations. Finally, repeated assessments of racial segregation over time for each study participant are prone to missing data and may require data imputation techniques (eg, last observation carried forward and multiple imputation) [76].

The validity of the mostly Black and mostly White dichotomizations of the PRCM demonstrates that this measure can assess the racial composition of social environments spanning adolescence and adulthood without depending on census data. Moreover, the PRCM allows the participant to self-report the racial composition of their social environments, negating the potential misclassification of what participants consider their neighborhood. Another advantage of the PRCM is that it can be administered at a single study visit, reducing the need to impute data for responses over time. In addition, because it is picture based, the PRCM may help diversify survey question types and reduce respondent fatigue. Finally, the PRCM was validated using a mixed methods approach in a medically underserved patient population with varied health literacy levels, including those with limited health literacy. However, convenience sampling from a single midwestern primary care clinic may limit generalizability, creating the need to validate the PRCM in other populations.

While we have presented cognitive interview data from Black participants used to inform the PRCM, we also collected data from a few non-Hispanic White participants. We recruited a subset of participants from the COH patient population that was approximately 30% White. However, we reached saturation early in the cognitive interviews and decided to stop recruitment with a lower proportion of White cognitive interview participants. Data were analyzed stratified by race, and the interview data from White participants were concordant with the themes observed with data from Black participants. As the PRCM is refined or being adapted for other studies, additional cognitive interviewing would be helpful. In addition, there were some modifications based on the cognitive interviews to the instrument and the mode of administration from the pilot phase to the main phase. While these changes were made to improve the measure, some of these changes may not have had the intended results and will require additional refinement in future research. We do not have any empirical data to support this, but we hypothesize that changing the mode of administration from written self-administered in the pilot phase to verbally administered by RAs in the main phase may have impacted the validity, with participants more likely to report socially desirable answers to the RAs.

Social, clinical, and public health scientists can use PRCM data in a variety of ways in regression models. One example is creating a composite score, summing the number of mostly Black (or mostly White) social environments a study participant reported, and estimating the effect of reporting an additional mostly Black or mostly White environment on the outcome of interest. Another way researchers can use the PRCM data is to calculate the number of discordant matches between the racial composition of each social environment and the study participant’s race. A third way is to administer the PRCM to children, their parents, and grandparents, assessing the potential generational effects of racial segregation. We encourage scientists to use color images (print or digital) when administering the PRCM.

Racial residential integration is a subject of increasing interest among social scientists [77-79]. A systematic review of racial residential integration spanning 60 years yielded 3 main findings [78]. First, the concept of integration has recently focused on racial composition, shifting away from racial and social integration [78]. Second, the measurement methods for integration have changed over time [78]. Third, racial residential integration has evolved from the Black-White dichotomy to including a broader range of races and ethnicities (eg, Asian and multiethnic) [78]. The authors of this review also note that inconsistent census cutoffs hinder the comparison of integration effects across studies [78]. To unify the study of racial residential integration, the authors recommend developing “a multidimensional theoretical framework for residential integration that includes both racial composition and social interactions” [78]. The PRCM contributes to this goal by helping us understand the range of racial segregation and integration, especially in residential, educational, workplace, and religious contexts.

Limitations and Future Work

Although we developed a novel measure of racial segregation, we must recognize the limitations of the PRCM and this study. This work was conducted in St Louis, Missouri, where the majority of the population identify as non-Hispanic White alone or Black alone. Given the limited diversity of the community from which the sample was taken, the response options were meant to represent Black and White neighborhoods. This Black and White comparison is a major limitation and requires validation in other populations outside of St Louis before adaptation and use.

The PRCM includes pictures of non-Hispanic White and Black social environments. Thus, it cannot measure compositions of other races and ethnicities, such as Asian or American Indian, limiting its capacity to capture the full racial and ethnic composition of social environments. While this is appropriate for the context of this study (St Louis, Missouri), extensions of the PRCM beyond St Louis, Missouri, or those communities with similar racial compositions should incorporate other racial and ethnic identities to enhance the inclusivity of this measure.

The PRCM is limited in its ability to evaluate segregated social environment experiences in other groups considered historically marginalized (eg, Hispanic and multiracial). Recent research shows that the upward mobility of Hispanic people is more similar to that of individuals racialized as White than of those racialized as Black [80]. Future research should extend the PRCM scope to be adapted for use in multiracial populations beyond those primarily composed of individuals racialized as Black or White.

The binary (White vs Black) measurement limits the PRCM’s ability to measure racially integrated social environments. The PRCM is a subjective measure susceptible to recall bias and misclassification. However, the validity results from this study suggest that bias and misclassification were reduced. We also think that the pictures help people remember each social environment. From the interview transcripts, one participant stated as follows: “It reminds me of my neighborhood. Honestly there were no whites around, period. None.” Another participant noted the following: “I am having to remember to relate it back to here (taps something) and I’m looking at it now...” Future studies should include empirical data on whether the pictures help participants remember.

The PRCM is a retrospective cross-sectional measure, which cannot infer causality on an outcome. We lacked information on the zip codes where participants grew up, which prevented us from conducting historical analyses. This highlights an area for future research to support longitudinal studies. Convenience sampling was used to recruit study participants. To circumvent sampling error and bias, RAs recruited participants at different times of the day and on different weekdays. We also acknowledge that using 2 subjective methods has its limitations, such as potential same source bias, telescoping effects, and memory distortions. However, standard validation approaches state that new metrics should be compared against the current gold standard, and in this case, the written measure is the current gold standard for subjective racial compositions across social environments. Moreover, people’s perceptions of their environments are also important to measure in addition to objective data, as perceptions may affect some outcomes.

Another limitation was combining Black and multiracial participants into 1 group. Given the small number of multiracial participants, we decided to include them in the analysis instead of excluding them. Most (7/10, 70%) of the multiracial participants were in the pilot sample, with only 3 (30%) in the main phase sample. The multiracial sample was too small to make inferences as an individual group or to create any real bias among the group of Black respondents. In future research, it will be important to validate the measure among individuals with other racial identities. Finally, the PRCM lacks the ability to assess social interactions between races.

We examined the current and past racial compositions of neighborhoods and blocks to obtain a life course examination of residential contexts over time. Future work could examine if adding additional key time points and being specific about ages would improve the life course measurement. Furthermore, we developed a measure that could be used in a sample with varying levels of health literacy, particularly among those with limited health literacy.

Many of the stratified analyses by the NVS and REALM-R showed that no statistically significant differences were found when comparing the concordance of the pictorial and written measures to objective census-based measures of the racial composition of participants’ current neighborhood and HS (Multimedia Appendix 2). However, NVS analyses yielded substantial differences in concordance between the pictorial measure and the objective census-based measure of participants’ current neighborhood racial composition (Multimedia Appendix 2). There are consistent differences in measures of health literacy across the literature [61,81,82]. This is because different measures capture different aspects of health literacy [61,81,82]. In particular, the NVS is a measure that includes numeracy, while the REALM-R is focused on written words [71-73]. We included both measures in our study to capture these differences.

Our selection of patients from the COH primary care clinic was purposeful in that measures are rarely validated among medically underserved populations with varying levels of health literacy. Using the principles of universal design, we believe a measure that is valid and reliable in this population is likely valid among the general US population, but further work is needed to obtain validation metrics in other populations. For example, patients seeking care in a safety net primary care setting may differ from the larger population. Thus, further work is needed to validate the PRCM for use in community contexts.

Conclusions

Using mixed methods of cognitive interviews (qualitative) and survey (quantitative) data, we developed and validated the PRCM. This pictorial measure increases our ability to examine racial composition in multiple environments across the life course among medically underserved populations. The PRCM can serve as a uniform measure of racial segregation used across disciplines. It can also be used with other measures to inform equitable intervention and prevention efforts, thus improving the lives of those impacted by structural racism.