Predicting Risky Sexual Behavior Among College Students Through Machine Learning Approaches: Cross-sectional Analysis of Individual Data From 1264 Universities in 31 Provinces in China

Introduction

Risky sexual behavior (RSB) is defined as sexual activities that are more likely to lead to the risk of sexually transmitted infections (STIs) and unwanted pregnancies [1], including sex without contraceptive use and sex with an ineffective or unsafe contraceptive method. It is a serious issue among college students worldwide, especially in low- and middle-income countries [2,3]. Among 5 types of contraceptive methods under the classification criterion from the World Health Organization (WHO) [4], condoms and hormonal contraceptive methods are regarded as safe and highly effective for adolescents [5]. Studies have shown that consistent condom use is low among college students [6]. In previous research, it was estimated that approximately 40% of students did not use a condom during their last sexual encounter [7]. Casual sex and sex with multiple partners have also been regarded as RSBs [8-10]. Nearly half of the college students had casual sex experience [11,12], and approximately 50.7% of students were sexually active, with 42.3% of students having multiple sexual partners [13]. Thus, to help reduce the influence of RSB, it is important to intervene and prevent it among college students by identifying relevant risk factors and making RSB predictions.

A number of association studies have focused on the risk factors for RSB. A systematic review of 30 papers summarized 11 aspects of risk factors of RSB including sociodemographics, gender roles, substance use, and partner characteristics [14]. It was widely validated that being male [15,16], drinking alcohol [17], experiencing poverty, and experiencing peer pressure [18] are significant risk factors for RSB [19,20]. Mental health has also been linked to RSB, with higher depression resulting in more RSB and STIs [20,21]. In addition, numerous public health and sociological studies have found that romantic relationship status [22,23], sexual knowledge [1], and sexual attitude have a significant influence on RSB [24,25]. However, few studies have fully used these identified factors to make RSB predictions.

Previous predictive studies on RSB have mostly been based on conventional regression models, which have high limitations of assumption and less ideal effects on RSB prediction. To fill such gaps, machine learning (ML) offers a possible alternative for factor identification and outcome prediction. In the past few years, a large number of studies have emerged using ML to predict the occurrence of STIs, which have achieved ideal performance [24,25]. However, RSB, as the main transmission route for STIs, has gained little attention from ML for prediction.

The objective of this study was to develop an ML-based model to precisely predict RSB in college students. Through a cross-province survey in China, this study aimed to develop a series of ML models to predict different types of RSB among Chinese college students. By comparison, we adopted the optimal model to identify key risk factors to help recognize and predict college students’ engagement in RSB, thus facilitating more precise intervention and prevention.

Methods

Participants and Research Procedures

We conducted a large-scale and internet-based survey, the National College Student Survey on Sexual and Reproductive Health in 2020 (NCSS-SRH 2020), sponsored by the China Family Planning Association (CFPA). Through multistage sampling from November 2019 to February 2020, a total of 55,757 Chinese college students from 241 universities completed the questionnaire survey. Voluntary participants were recruited using snowball sampling, and informed consent was obtained from each participant before completing the survey.

Among all participants, 1177 (2.11%) were excluded for either failing the attention check questions, ignoring the informed consent, or being outside the age range for college students (15-24 years old according to a standard definition of late adolescence and young adulthood by WHO) [26,27]. Samples with duplicated answers and variables with missing values over 5% were deleted. For the remaining variables, missing values were imputed with the use of multiple imputation. Due to the constraint of sexual orientation and past sexual experience, 8794 (15.78%) self-identified heterosexual students with sexual intercourse experience were finally included in the analyses.

RSB Outcomes

We classified RSB from 4 perspectives: (1) whether contraception was used, (2) whether the contraceptive method was effective and safe, (3) whether participants engaged in casual sex or sex with multiple partners, and (4) the integration of the former 3 perspectives.

Contraception use was evaluated through 2 dimensions. Regarding frequency, contraception use was measured by the question “Do you use contraception while having sex every time you have sexual intercourse?” Regarding practices, contraception use was measured by the question “Did you/your partner use contraception the last time you had sex?” For both questions, a “no” response was considered to indicate RSB. According to WHO guidance, 4 contraceptive types with over 10 specific methods were investigated as contraception use: hormonal contraceptive methods, intrauterine devices, emergency contraception, and condoms [4].

The effectiveness and safety of contraception was mainly determined by the method of contraception. If someone reported using “emergency contraception,” “external ejaculation,” or a “safe period” for contraception, they were considered a member of the RSB group. The effectiveness and safety of contraception were also evaluated through the frequency and practice dimensions, which were, respectively, measured by the question of whether such approaches were taken every time or the last time of sexual intercourse.

Casual or multiple sex was assessed with the following questions: “Have you ever had sex through ‘booty calls’, ‘one-night stands’, ‘buying sex’, or ‘sex with multiple partners’?” As before, a “yes” response was considered RSB.

Finally, integrated RSB dimension was the combination of contraception use, the safety of contraception, and casual sex or sex with multiple partners. As long as 1 of these 3 types of behaviors occurred, we considered the student to have met the criteria for integrated RSB.

Predictors

We included 126 potential predictors, including baseline characteristics (sex, age, ethnicity, religion, income, parental information, etc), daily habits (exercise, appearance and popularity, mobile phone addiction, alcohol and tobacco consumption, etc), physical and mental health status, relationship status, sexual knowledge, sexual education, sexual attitude, previous sexual experience, experience of sexual harassment and assault, etc. Among them, the age of the participant, the age of the participant’s partner, the income and expenditure of the participant, and the frequency of sports were treated as continuous variables. Other variables, including the degree of agreement, the frequency of participation, and the order of evaluation, were treated as either binary variables or ordered categorical variables.

Continuous variables were standardized, and categorical variables were split into multidimensional Boolean values before applying ML models. For each type of RSB, we used ML models to select and identify key variables to predict RSB among college students.

Statistical Analyses

For descriptive statistics of baseline characteristics, continuous data were presented as the mean (SD) or the median (IQR), and the Student t-test or the Kruskal-Wallis test was applied depending on the normality distribution. Categorical data were presented in the form of counts with percentages, and the chi-square test or the Fisher exact test was applied. P<.05 in a 2-tailed test was considered statistically significant in these tests. In addition, we developed multivariable mixed models with a logit link function to learn the specific linear relationship between RSB and the key variables screened by ML. The results were presented in the form of point estimates of coefficients and corresponding 95% CIs, and statistical significance was accepted when P<.05.

Model Development and Validation

We used 8 ML approaches: multiple logistic regression (MLR), naive Bayes (BYS), linear discriminant analysis (LDA), random forest (RF), gradient boosting machine (GBM), extreme gradient boosting (XGBoost), deep learning (DL), and the ensemble model. The ensemble model used the average values of all other models’ predictive values to perform classification. The data set was split into a training set and a test set randomly in a ratio of 8:2, with 7035 (80%) samples in the training set and 1759 (20%) samples in the test set. Our models were built on the training set and then applied to the test set for RSB prediction. Model discrimination was assessed through the receiver operator characteristic (ROC) curve, and model performance was assessed through accuracy, precision, recall, F₁-score, the area under the receiver operator characteristic curve (AUROC), and the root-mean-square error (RMSE) calculated on the test set, which were presented in the form of the mean (SD).

Through comparison, we chose the optimal model for RSB prediction and included the most predictable variables in the model. The appropriate number of predictable variables was determined by the turning points of the model performance indicators. If all indicators did not change significantly through statistical testing when a new variable was added, we considered the turning point to have been reached.

To obtain the optimal performance for each model, we adopted the minimum distance (MD) method to select cut-off points to discriminate predictive values into 0 or 1. The MD method regarded the point closest to (0,1) on the ROC curve as the optimal cut-off point. To ensure the reliability and minimize the sensitivity of the results, we used a 10-fold cross-validation method to select the tuning hyperparameters as well as cut-off points. In addition, we repeated this process 10 times to minimize the influence of accidental circumstances.

All models were generated using R version 4.0.3 (R Core Team and the R Foundation for Statistical Computing). We used the glmnet package for MLR, the e1071 package for BYS, the MASS package for LDA, the randomForest package for the RF, the gbm package for GBM, the xgboost package for XGBoost, and the h20 package for DL.

Ethical Considerations

Ethical approval was obtained from the Institutional Review Board of Tsinghua University (IRB no. 20190083). All participants provided informed consent online, which was set before answering the questionnaire and emphasized the autonomy of participating and the ability to withdraw at any time. The privacy of personal information was protected throughout the study via anonymous data collection, and confidentiality was maintained by asking participants to provide honest answers. Eligible participation in this survey was voluntary and was not compensated.

Results

Baseline Characteristics

The baseline characteristics of the participants are shown in Table 1. Our sample covered all provincial-level administrative regions in China and achieved a relatively good balance in the sample division of eastern, central, western, and northeastern regions (n=4758, 54.11%, n=1484, 16.88%, n=2140, 24.33%, and n=412, 4.69%, respectively), as well as the sex ratio (males: n=3918, 44.55%; females: n=4876, 55.45%). A total of 8794 students were included, among which 3682 (41.87%) did not use contraception every time they engaged in sexual intercourse, 3602 (40.96%) did not use safe or effective contraceptive methods every time they engaged in sexual intercourse, 1157 (13.16%) had casual sex or sex with multiple partners, and 5328 (60.59%) had experience with at least 1 of those 3 former behaviors before. In addition, 307 (3.49%) participants did not use contraception and 2140 (24.33%) used ineffective or unsafe contraceptive methods during their last sexual intercourse, the details of which are shown in Multimedia Appendix 1. The distribution of basic characteristics differed greatly in terms of RSB. Between the 2 groups with and without integrated RSB, the region of residence, sex, age, ethnicity, religious beliefs, urbanization of hometown, left-behind experience, migration experience, and self-assessment of family finances were significantly different (P<.05) among college students.

Model Performance and Validation

To precisely identify RSB among Chinese college students, we used various ML models to execute RSB prediction on the test data set to choose the optimal model. The model performance for the 4 types of RSB is presented in Table 2, and the model performance for the other 2 types of RSB based on the last sexual intercourse is presented in Multimedia Appendix 1. Through the comparison from multiple rounds of experiments, it was obvious that some ML models had better efficiency than the traditional multilinear logistic regression model. Compared to the performance of MLR (for which the average AUROCs were 0.76, 0.71, 0.91, and 0.79, respectively), XGBoost, GBM, and the RF presented better performance in terms of accuracy, the F₁-score, and the AUROC.

To better understand model discrimination, we plotted ROC curves of all models on 4 types of RSB in Figure 1. Similarly, ROC curves of the other 2 types of RSB based on the last time of sexual intercourse are presented in Multimedia Appendix 1. It could be inferred that the curves of XGBoost and GBM were above the other curves, which suggests that these 2 models outperformed the others. Their average results were similar (average AUROCs were 0.77, 0.72, 0.94, and 0.80, respectively, in GBM and 0.78, 0.72, 0.94, and 0.80, respectively, in XGBoost). The ensemble model also played an effective role in predicting different types of RSB (average AUROCs were 0.77, 0.72, 0.93, and 0.80, respectively).

‎

Variable Selection and Prediction for RSB

Through the comprehensive comparison of the evaluation indicators considering both efficiency and robustness, XGBoost was chosen to form the predicting model. To comprehensively predict RSB, we took integrated RSB as the outcome to select important variables, which were ranked in the order of importance according to the XGBoost model. Multimedia Appendix 1 shows the trend of 6 indicators as the number of variables increases, where points represent the average performance and lines represent the range. The turning points of the 6 indicators were 5th, 8th, 7th, 7th, 6th, and 12th, respectively. Thus, we finally chose 12 key variables for our prediction model, as presented in Figure 2. It could be inferred that RSB has multiple types of influencing factors, including relationship status, sexual knowledge, sexual attitudes, and previous sexual experience. The final prediction model we developed could estimate not only the probability of a student engaging in RSB but also the kind of RSB they were more likely to engage in.

To explore the specific association between the outcome variables and the predictive factors identified through XGBoost, we finally performed MLR, and the results of integrated RSB are presented in Table 3. It could be inferred that the lack of sexual knowledge and a liberal sexual attitude significantly increased the risk of RSB. In addition, regarding intimate relationship status, compared to the non-single group, the single group had a higher risk of RSB. Previous sexual experience also had a great influence on RSB. The greater the number of people participants had sexual intercourse with, the higher their risk of RSB. The more convenient the availability of contraceptives was, the less risk students would show RSB. The results of the other three types of RSB are shown in Multimedia Appendix 1.

‎

Discussion

Principal Results

This study validated the effectiveness of ML models in predicting RSB among college students through comparisons of multiple models. Among various ML models, XGBoost performed the best in this task, with a higher accuracy, precision, F₁-score, and AUROC performance than others. Thus, we eventually used XGBoost to identify the 12 most predictive factors for total RSB, including relationship status, sexual knowledge, sexual attitudes, and previous sexual experience. This systematic process of data modeling as well as the accuracy of the final results indicated that ML approaches could have considerable value in RSB prediction and intervention among college students.

ML models were substantially superior to conventional regressions and should be recommended for more practical applications. Compared to the AUROC values of the MLR model (0.76, 0.71, 0.91, and 0.79 on the 4 types of RSB, respectively), XGBoost had a much higher effect in terms of AUROC values (0.78, 0.72, 0.94, and 0.80 on the 4 types of RSB, respectively). However, the capacity of explanation of MLR was nonnegligible. Thus, we finally used an MLR model again to investigate the linear association between outcomes and those important risk factors identified through XGBoost.

In this study, demographic characteristics and socioeconomic status were found to be significant factors of RSBs. Adolescents who were from an ethnic minority background, held religious beliefs, or had a lower family financial status tended to engage in more RSBs. This may be due to the lack of sexual education resources or specific religious customs. In addition, it was worth noting that students who had migration experience or were from rural hometowns were associated with increased RSBs. This finding was consistent with previous research and reflects the persisting gap in adolescent sexual and reproductive health between urban and rural areas [28,29]. Correlated with a lower level of education and socioeconomic status, rural-to-urban adolescents had less exposure to sexual knowledge and sex education [28,30].

In accordance with previous studies, romantic relationships were highly associated with RSB. It was found that college students with a romantic relationship had a significantly higher probability of using condoms during vaginal sex, oral sex, and anal sex [23]. On the one hand, students not in romantic relationships usually had fewer condom-carrying practices and a higher occurrence of unplanned sex. On the other hand, according to Rosenthal et al [31], rather than being concerned with the risk of STIs, students not in romantic relationships paid more attention to building intimacy through RSB, especially during casual sexual encounters. This is a worrisome mechanism since these students are more inclined to be unaware of each other’s health status, which could lead to a considerable risk of STIs.

This study also validated the role of sexual attitude in RSB. The more tolerant the students were toward condom nonuse and one-night stand, the higher their probability of engaging in RSB. In fact, it was widely validated that sexual attitude plays the most predictive influence in predicting RSB [32,33], which could be illustrated through the theory of planned behavior (TPB) that attitude is an activator of behavior [34]. Thus, sexual education should be facilitated to emphasize the importance and necessity of safe sex and dispel misunderstandings about it.

In addition, students’ previous sexual experience also had a large influence on the possibility of RSB, such as the number of sexual partners, the decision maker regarding the contraceptive method, and whether to take photos/videos during sex. This finding is also rational since well-practiced behavior would more likely recur due to the natural automation of initiates and controls [35].

Limitations

There are some limitations in this study. First, all outcomes and predictors were self-reported by participants, which may have caused recall bias and nonresponse bias. Since some questions were sensitive, participants may not have been willing to provide correct answers, such as on previous sexual experience, experience of sexual harassment, and assault. Second, the measurement of contraception use did not differentiate between the types of contraceptive methods used by the study population. Different contraceptive methods serve different functions. Condom use can prevent both STIs and unwanted pregnancies, while hormonal contraception is only effective for pregnancy prevention. Although the vast majority of participants used condoms for contraception, the predictive accuracy of contraceptive effectiveness may still have been obscured in this study because of the lack of differentiation across methods. Third, the definition of casual sex and sex with multiple partners was not well specified. Though these 2 types of behavior were undoubtedly validated as RSBs, the risks of contracting STIs and unwanted pregnancy can be kept relatively low with correct condom use. Fourth, although ML models have better predictive performance than traditional regression models, their explanatory performance is much weaker. We performed MLR to compensate for this drawback, but multicollinearity may exist among those selected risk factors, which could lead to inaccurate estimation. In particular, we had a large scale of many questions, with much similarity among them. Thus, questions could be divided into clusters to decrease the variable dimensions as well as strengthen the explanatory power. Fifth, since our model relied on a cross-sectional questionnaire, the outcomes and predictors were questioned concurrently. Under the same time window, it is difficult to identify the sequence of events. There may be causal inversion problems between predictors and outcomes, and thus, the prospective predictive efficiency of the model is hard to validate.

Comparison With Prior Work

There are considerable methodological, theoretical, and practical implications of this study. From the methodological aspect, we adopted a scientific and rigorous process to generate an RSB-predictive model using ML methods, which constitutes a research gap and urgent work to be done. The selection of the model, the adjustment of parameters, the comparison of indicators, and the finalization of variable numbers are of high reference value in the methodology. From a theoretical perspective, we identified a series of risk factors for RSB. We provided additional evidence for the association of demographic characteristics and socioeconomic status with RSB. Critical factors influencing RSB were also explored, including sexual attitude, sexual knowledge, relationship status, and sexual experience. Through the results presented, a comprehensive and evidence-based guideline was formed to facilitate more precise interventions and prevent RSB among adolescents and young adults. From a practical perspective, we developed a predictive model to help identify RSB among college students. Due to privacy concerns and the stigmatization of sexual behavior, it is often difficult to investigate the real prevalence of RSB among adolescents. Using the 12 predictors identified here, the model can predict not only the probability of a student engaging in RSB but also what kind of RSB they are more likely to engage in. With such a model, our study allows for more targeted intervention and prevention of RSB in students before they contract STIs, and thus, these students will be better able to avoid the various negative consequences of RSB, including STIs and unwanted pregnancy.

Conclusion

In summary, our study confirmed that ML approaches, especially XGBoost, have greater predictive effects for RSB than traditional regression models. Such ML-based assessment tools could generate new applications with considerable practical value, which would promote health at both the individual and the public level in the future.