This study was a cohort study. This study followed the Transparent reporting of multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines.

The cohort was part of the China Stroke High-risk Population Screening and Intervention Program (CSHPSIP), an ongoing nationwide population-based program [5,11]. The study participants were recruited from the screening site in the Jinfeng District of Yinchuan City, Ningxia Hui Autonomous Region. A total of 22,259 community residents were enrolled in 2013 from one urban and rural monitoring site, and the outcome ascertainment was completed in 2023. The inclusion criteria for the screening participants are individuals who are aged 40 or older, permanent residents (those who have lived in the area for 6 months or more), and those who voluntarily participate by signing an informed consent form [12]. Patients younger than 40 years, those with a history of stroke or TIA, or individuals recruited after 2014 were excluded from the study cohort. The data quality control process is detailed in the Multimedia Appendix 1.

Based on the “Stroke Screening and Prevention Technical Specifications” promulgated by the National Health and Family Planning Commission’s Stroke Screening and Prevention Engineering Committee, the following risk factors were assessed: hypertension, heart disease, smoking, dyslipidemia, diabetes, physical inactivity, overweight, and FHS. The detailed criteria for each risk factor are shown in Table S1 in Multimedia Appendix 1 which is also available on the China Stroke and Cardiovascular Disease website.

The criteria for classifying individuals into high-, medium-, and low-risk stroke groups were as follows: the high-risk group was defined as having 3 or more risk factors; the medium-risk group was characterized by 3 or less risk factors along with a history of chronic diseases (such as hypertension, diabetes, and heart disease); and the low-risk group was defined as having 3 or less risk factors without any history of chronic diseases (Table S1 in Multimedia Appendix 1) [13]. For high-risk individuals, follow-up visits are conducted by primary health care institutions at 6 months and 12 months after the initial assessment. For moderate-risk populations, primary health care institutions conduct a single follow-up visit at 12 months to evaluate and address their associated risk factors.

We recorded stroke as an endpoint event by searching electronic hospitalization records in Ningxia in June 2023. Stroke was identified using the diagnostic code I60-63 from the International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10).

Logistic regression was used to determine the odds ratios (ORs) of every risk factor for incident stroke [14]. The basic equation for regression with multiple independent variables is:

Y is the estimated continuous outcome; α is the intercept. This is considered a constant value; β is the beta coefficients; and X_i is each risk factor.

The Rothman-Keller model, initially developed by Kenneth J. Rothman and David B. Keller in the early 1970s, was designed to assess the combined impact of tobacco and alcohol consumption on the risk of oral and pharyngeal cancers [15]. This model provides a statistical framework that enables researchers to quantify both the independent and joint contributions of various risk factors to disease risk. It has been adapted and applied to a wide range of health conditions and diseases, including early-onset colorectal cancer and mild cognitive impairment [16,17]. Its flexibility in considering both additive and multiplicative effects of risk factors makes it a valuable tool for public health research and individual risk prediction.

The Rothman-Keller model uses the binomial distribution function method for risk classification. It calculates the benchmark proportion of incidence and risk scores based on the population exposure rate and OR of each risk factor. In addition, it estimates an individual’s relative risk of developing a disease by calculating their combined risk scores. The parameters of the Rothman-Keller model are calculated as follows:

The population exposure rate of every risk factor was derived from the literature [18]. The OR of exposure to a risk factor was sourced from the logistic regression model. Data for 100,000 participants were randomly generated by the binomial distribution functions of risk factors collected from the literature to identify nodes of high, medium, and low risk in models. The exposure rate of a risk factor in the study population was P₀. We generate 100,000 random P values of 0: 1, P<P₀ was recorded as 1 (ie, exposure), and P>P₀ was recorded as 0 (ie, nonexposure). Each risk factor simulates a column of data, summarizing the exposure of 100,000 community residents to each risk factor.

The ethics review committee of The People’s Hospital of Ningxia Hui Autonomous Region approved this study (approval number: 2020-KY-053). Patients provided informed consent for using the data. Data were deidentified. No compensation was provided.

A total of 22,259 community residents were recruited in 2013. After excluding individuals with a history of stroke (n=313), and TIA (n=384), as well as participants younger than 40 years old (n=195) and those recruited after 2014 (n=9576), 11,791 participants were included in the follow-up cohort. After a 10-year follow-up period, 99 participants were lost to follow-up. Finally, a total of 11,692 eligible participants were included in the final analysis (Figure 1). A total of 767 participants (6.6%) had a stroke by the end of the follow-up period. Based on the “8+2” stroke risk score, the 10-year stroke incidence among the 3 stroke risk groups of the community residents was as follows: low-risk group 4.47% (n=8908, 398 stroke cases); medium-risk group 14.71% (n=1074, 158 stroke cases); and high-risk group 12.34% (n=1710, 211 stroke cases) (Table 1). Kaplan-Meier survival curves are shown in Figure S1 in Multimedia Appendix 1.

The baseline incidence ratio (ρ) of 8 factors was obtained to calculate the population attributable risk percentage (PAR%) through a previous study [18]. The OR values of these factors were assessed using a logistic regression model. The parameters of the logistic model and the Rothman-Keller model are shown in Table 2.

After ranking the stroke incidence risk based on a dataset of 100,000 random entries, 2 nodes were selected for subdividing the risk groups into low-risk, medium-risk, and high-risk categories using the logistic regression model: node a (ID=25844, risk prediction score=0.06168760) and node b (ID=77778, risk prediction score=0.16451650). In addition, for the Rothman-Keller model, 2 other nodes were chosen for the same purpose: node A (ID=25426, risk prediction score=0.0021993391) and node B (ID=64553, risk prediction score =0.0047898060) (Figure S2 in Multimedia Appendix 1).

The sensitivity, specificity, and AUC of the “8+2” stroke risk score, the Rothman-Keller model, and the logistic regression model are presented in Table 3. The logistic regression model and the Rothman-Keller model demonstrated significant differences in AUC values compared to the “8+2” stroke risk score (Z=2.60, P<.05; Z=3.47, P<.05, respectively). However, no difference was observed between the logistic regression model and the Rothman-Keller model (Z=0.688, P>.05). The comparison of receiver operating characteristic curve is shown in Figure S3 in Multimedia Appendix 1.

From the NRI, we found that the majority of participants remained at the same level of risk for developing a stroke as predicted by the “8+2” stroke risk score, the logistic regression model, and the Rothman-Keller model (ie, along the diagonal from the lower left to the upper right). However, some community residents were reclassified as having a different level of risk for developing stroke (Figure S4 in Multimedia Appendix 1). According to the “8+2” stroke risk score, the NRI for reclassification of stroke events by the Rothman-Keller model was 7.8%, while the NRI for nonstroke events was −2.7%. The absolute NRI was then estimated to be 0.051 (P=.01), calculated using the sum of the net estimated for individuals who developed a stroke and those who did not. The NRI for the logistic regression model was 1.7% for stroke events and −0.7% for nonstroke events, resulting in an absolute NRI of 0.010 (P=.62) .

This cohort study followed 11,692 individuals aged 40 years and older for a duration of 10 years. According to the “8+2” stroke risk score, the stroke incidence in the low-risk (n=8908), medium-risk (n=1074), and high-risk groups (n=1710) was 4.5%, 14.7%, and 12.3%, respectively. We developed a logistic regression model and a Rothman-Keller model to validate and optimize the risk score. Through a comparative analysis of the performance of the 3 models, we found that the Rothman-Keller model exhibited the best performance.

We verified the efficacy of the model using an actual database. There was no significant difference in the AUC values between the logistic regression model and the Rothman-Keller model. To evaluate model performance more accurately, we used the NRI for a more in-depth analysis. The NRI assesses the effects of low-, medium-, and high-risk reclassification for both stroke and nonstroke events, resulting in a net reclassification that provides a more accurate estimate than that obtained with other approaches [23]. Positive values of the stroke NRI indicate that the model effectively identifies patients with stroke, enabling physicians to initiate targeted detection or treatment to prevent stroke events. In contrast, a decrease in the NRI for nonstroke events suggests that community residents with a low or medium risk as determined by the “8+2” stroke risk score may actually be at a higher risk of having a stroke. Based on the overall NRI, we conclude that the Rothman-Keller model enhances the reclassification of both stroke and nonstroke events [25]. This finding aligns with the results of previous studies. Researchers used the Rothman-Keller model to predict the likelihood of mild cognitive impairment in older Chinese individuals. Upon validation with actual population data, it was found that the model had appropriate accuracy and performed well in terms of predictive efficacy [16,26]. The model can be adjusted and optimized based on new research data and epidemiological changes, thereby maintaining its predictive power in a timely manner. Its methodology can be applied to risk assessments for other populations and chronic diseases, demonstrating significant universality.

There are many model studies aimed at predicting stroke risk [27-30]. For instance, SCORE2 is a risk assessment tool developed using extensive data from a large number of European populations. It is designed to evaluate the risk of cardiovascular disease in both men and women across 4 distinct risk areas in Europe within a 10-year period [31]. While this tool is widely applied, it has limitations in terms of ethnicity and geography. Significant prediction errors may occur when applying it to populations with considerable differences [9,32]. A study estimated the 10-year risk of stroke based on a cohort analysis and found that factors such as age, systolic blood pressure, diastolic blood pressure, FHS, atrial fibrillation, diabetes, and others can significantly predict the incidence of stroke [27]. Compared with other models, this study developed a Rothman-Keller model based on questionnaire information to identify new risk nodes through simulation datasets, which provided a basis for stroke prevention. Furthermore, the model’s predictive power and accuracy were verified using real-world data. Logistic regression analysis indicated that smoking, heart disease, dyslipidemia, and physical inactivity were not related to stroke, which may be attributed to variations in demographic and stroke subtypes differences. This result was consistent with the results of previous Mendelian randomization studies [33-37].

The limitations of this study are as follows: first, the dataset used for verification only included participants from only one region. Studies in other provinces are necessary to evaluate the efficacy of our model. When research results are extrapolated to other populations with significant differences, it may be essential to consider the exposure rates of risk factors and OR or RR values for those populations in order to update and optimize the model. This process enhances the predictive accuracy and applicability of the Rothman-Keller model. Second, we were unable to perform a subgroup analysis on the various types of stroke. The primary objective of the program is to identify and intervene with high-risk populations to prevent the occurrence of stroke or reduce its risk, rather than focusing on the subtypes of ischemic stroke and hemorrhagic stroke. Furthermore, given the limitations of medical resources and the acceptability of screening, the program may prioritize the implementation of more accessible and universal preventive measures. These measures include controlling blood pressure, quitting smoking, and increasing physical activity, all of which are effective in preventing both ischemic stroke and hemorrhagic stroke [38,39]. Third, during the decade, participants may have received lifestyle interventions, pharmaceutical treatments, and early clinical treatment that influenced the incidence of stroke events. However, in the medium- and high-risk groups for stroke, these patients represent a certain proportion. Our model primarily assesses the variations in the initial screening judgments, and the outcome events remain consistent across different models, making it unlikely to influence the study results. Finally, the low sensitivity observed in this study may be attributed to the lack of several important stroke prediction factors from the risk scoring scale, thereby limiting its predictive ability. In 2021, the Guidelines for Stroke Prevention and Treatment in China recommended including homocysteine testing in routine screening and conducting carotid artery examinations for high-risk populations when conditions permit. Our research findings further support this recommendation. Although this study has some limitations, it also presents several advantages. The diagnostic criteria for risk factors in this project are based on relevant guidelines and standards established by the China Health Commission. Staff members undergo standardized training and assessment, and only those who pass the assessment are qualified to conduct screening tasks. Therefore, the identification of risk factors in this study had high accuracy and credibility. We used the binomial distribution of risk factors to construct a random dataset of 100,000 community residents, which allowed us to determine the high-, medium-, and low-risk boundary values of the models. The variables in the model were easily obtained and predicted estimates could be derived through straightforward calculations. We included a substantial number of community residences from the CSHPSIP over a decade for external validation, which exhibited good discrimination and calibration.

In conclusion, the Rothman-Keller model may improve the predictive efficacy of stroke screening models. In the future, verification will need to be carried out in a wider population and combined with more risk factors. The Rothman-Keller model for assessing individualized stroke risk, combined with interactive information platforms for health education, is beneficial for decreasing the incidence of stroke among high-risk groups.