Data from the 2019 cycle of OSDUHS were used for analyses. OSDUHS is a biennial cross-sectional voluntary survey conducted among students in grades 7-12 attending publicly funded schools within Ontario, with the goal of collecting data on their physical, mental, and social well-being and the prevalence of self-reported risk behaviors such as gambling and drug use. OSDUHS data have been widely used to help direct policy decision-making, public programming aimed at supporting youth, and setting priorities for enhancing the health of youth [24]. Administered since 1977, OSDUHS uses a 2-stage cluster sample design based on a random selection of schools stratified by region and school level (elementary and secondary) and a random selection of classes within each school [24].

The 2019 cycle of OSDUHS, administered in 263 schools across 47 school boards within Ontario, had a sample of 14,142 students. The cycle administered 4 versions of the survey: 2 versions for elementary students (grades 7-8) and 2 for secondary school students (grades 9-12). Each version consisted of a set of core questions that were common across all 4 surveys regarding demographics, perception of school experience, family life, substance use, physical activity, hours of sleep, and other physical and mental health indicators. Responses to these core questions constituted the set of features used in our analysis. Survey weights were not used in our analysis. Further details about the 2019 OSDUHS are publicly available [24].

Our primary outcome of interest was student self-reported level of agreement to the question “I feel enthusiastic about my future”[25], answered on a 4-point Likert scale (0: strongly disagree to 3: strongly agree). This single Likert-type item of self-reported enthusiasm was used as an outcome given its simplicity and interpretability when building a machine learning model, while also being cognizant of its limitations in being able to capture the multidimensional nature of the well-being construct. This question was posed to all students, maximizing the included sample, and enthusiasm is a known contributor to well-being and positive affect [14,15]. Furthermore, the Likert scale is a fundamental psychometric tool used to quantify qualitative attributes of the human experience [26]. A Likert scale style survey is easier to administer and minimizes the participant burden, increasing the likelihood of survey completion. Of the total sample, 13,661 student participants provided a response to self-reported enthusiasm and were included in our modelling (481/14,142, 3.5% missing). Raw values and 2 transformations of this variable were used to fit 3 different models, as described in the Statistical Modelling section below. Likert-style outcomes have been used in previous machine learning classification studies [27,28].

A complete set of 50 variables was selected from the aforementioned common core questions to maximize the number of student responses that could be included for model training and testing. A range of data types were used in the analysis, including numeric, ordinal, and unordered categorical. Variables such as household composition (family members present in the home environment), racial background, and geographic region within Ontario were one-hot encoded (the process of splitting a categorical variable into multiple binary dummy variables with either yes or no membership), a method often used for categorical variables in tree-based learning models. Ordinal variables were treated as numeric (the machine learning algorithm selected for analysis was XGBoost, later described, which is a nonlinear model and therefore makes no scalar assumptions about levels of ordinal variables). Some variables such as the 10-item responses to “How often did you drink alcohol in the last 12 months?” were collapsed into binary responses (yes vs no) corresponding to whether any alcohol was consumed in the last 12 months. These collapsed variables were created by the OSDUHS team and were provided to us in the original OSDUHS data set [24]. This was the case for all questions pertaining to substance use and was done based on the very low response rates—largely in categories of substance use—for extreme values of some variables.

Additional processing involved re-encoding and collapsing of categories for ease of interpretation. First, any response option that could be interpreted as a “no” was collapsed into a common category. For example, the 2 response options “use internet, but not social media,” and “don't use the internet” were combined to “don’t use social media” for the question “How many hours do you typically spend on social media?” Second, the weight of the student was considered to be the average of the 2-kg range options provided to students. Third, the many categories for language spoken were condensed to categories of “English only,” “French only,” “English and French only,” and “other multiple languages.” All encoded ordered categorical variables were shifted to start with a value of zero. Finally, the remaining categorical variables were one-hot encoded (0=no, 1=yes for category membership), with “don’t know” or “unsure” responses being treated as missing. Crucially, XGBoost was designed to manage missing values by learning appropriate tree branches during the training process and implementing a default mechanism for evaluating new data, which negates the need for a priori imputation. A complete descriptive list of the input variables used in our analysis can be found in Table S1 in Multimedia Appendix 1. Variable transformations are detailed in Table S2 in Multimedia Appendix 1.

Python (v 3.8.13; Python Software Foundation) was used for all analyses. Three classification models were generated using the XGBoost algorithm [20], with the outcome being class labels derived from self-reported enthusiasm, modelled by our set of 50 input variables. XGBoost is a decision tree–based algorithm, whereby a set of trees are initialized and iterated during model training to improve fit. The weighted average of the output of each tree within the set is considered as the prediction of the trained model. XGBoost was selected due to its ability to handle linear, nonlinear, and interactive effects between predictor variables.

A total of 3 classification models were built, each for a different grouping of responses to our primary outcome of enthusiasm. These 3 models were specified as follows:

In the binary classification of model 3, a sensitivity analysis to demonstrate ambivalence is reported in Multimedia Appendix 1. The data were divided into 2 nonoverlapping subsets through random sampling, stratified by outcome categories (to maintain balance in outcome groups), with 80% as training data and 20% as withheld test data to evaluate unbiased model performance. Random oversampling of the outcome group with fewer observations—within only the training data—was used to help mitigate distributional imbalances (using the imbalanced-learn Python library). The area under the receiver operating characteristic curve (AUROC) was used to evaluate the model’s accuracy on the validation data during the fitting process due to its scale and classification-threshold invariance. XGBoost automatically handled missing data by assignment of a default direction at each decision tree node such that loss of AUROC was minimized. The optimal hyperparameters for the XGBoost algorithm, which are values that control the XGBoost process and are selected before model training, were selected through Bayesian optimization using the Hyperopt Python library [30]. Hyperparameters that were optimized can be seen in Table S3 in Multimedia Appendix 1. Accuracy, precision, and recall were also calculated for each model within test data, and confusion matrices were used to visualize model performance.

Given that XGBoost is not inherently interpretable, the importance of each input feature in model classification was determined by calculating the absolute mean SHAP values [21] based on the test data. A positive SHAP value indicated that the variable had a positive influence on the outcome (pushing the model classification toward enthusiastic), with a greater magnitude indicative of a greater impact on the model output. In addition, following our previously published approach [22], SHAP values for the interactions among the top 15 input features were calculated to determine the importance of interaction among these variables on classification in comparison to the importance of the individual variables. SHAP analysis was performed on all models; however, the best model based on the aforementioned metrics was selected to present detailed results and structure discussion.

The 2019 OSDUHS was approved by the institutional review boards at the Center for Addiction and Mental Health, York University, and 34 Ontario school boards. Schools were randomly selected. After seeking approval from relevant school boards, the randomly selected schools were invited to participate in the survey. Schools that could not participate were replaced by schools within the same stratum. Once a school was approved, 1 or 2 classes in the relevant grades were randomly selected from a master list of all classes. Students in the selected classes were given a parental consent–student assent form to take home to parents, which explained the survey’s purpose and method. Only students with parental consent could participate. Students completed the survey in the classroom during regular school hours. The survey data were anonymous [24]. Generative artificial intelligence was not used in the creation of this manuscript. Analyses of OSDUHS data were approved by the CAMH Research Ethics Board (CAMH 099/2019).

Table 1 summarizes the demographic characteristics of the survey participants included in our study. The sample had a mean age of 14.9 (SD 1.8) years, was well balanced for biological sex assigned at birth (7612/13,661, 55.72% female), and included a majority of individuals who self-identified as White (8617/13,661, 63.08%).

Table 2 summarizes the key performance metrics for the 3 models built using XGBoost: the multiclass classification of all enthusiasm responses (model 1), the binary classification of enthusiastic respondents against all others (model 2), and the binary classification of enthusiastic versus not enthusiastic respondents, excluding ambivalent classes (model 3). For each of the 3 models, the accuracy, precision, and recall metrics were highly similar, being within 1% of each other, indicating that the model classified a similar number of false positives as false negatives [31]. The AUROC metrics were 0.68 for model 1, 0.75 for model 2, and 0.86 for model 3. The AUROC metrics are shown in Table 2, which highlights the calculated performance metrics for each model on the withheld test data. Model 3 was selected as the best classification model for self-reported enthusiasm using the set of 50 input features, as it had the best performance across accuracy (0.81, 95% CI 0.79-0.83), precision (0.82, 95% CI 0.80-0.84), and recall (0.81, 95% CI 0.79-0.83) in held-out test data. The confusion matrix depicting model 3 performance is shown in Figure 1B. Confusion matrices for models 1 and 2 are available in Figures S1-S2 in Multimedia Appendix 1. Optimal hyperparameters for our trained model are provided in Table S3 in Multimedia Appendix 1. The superior performance of this model when compared to those of model 1 (accuracy 0.53, 95% CI 0.51-0.55) and model 2 (accuracy 0.68, 95% CI 0.66-0.70) is likely due to the exclusion of potentially ambivalent respondents, polarizing the sample and offering more contrast between classes for training.

SHAP analysis ranked the most important features used by the model for classification. Figure 1B shows the top 20 input variables, as determined by the mean of the absolute values of their SHAP values in the test data. Figure 2 illustrates the impact of the response of each individual student on model outputs for the top 5 input features within the test data. The most important contributor to youth enthusiasm based on SHAP analysis was self-rated physical health. In general, themes surrounding physical health, family relationships, and school experience appeared among the top input features across all 3 models. Details for the SHAP values of the top 5 input features in each model are listed in Table 3. A complete list of SHAP values for model 3 is available in Table S4 in Multimedia Appendix 1.

Examination of the impact of pairwise variable interactions on model classification indicated that the interactions between input features did not substantially affect the model output, with mean absolute SHAP values for these interactions being lower than the mean absolute SHAP values of any of the top 15 most important explanatory variables in each of the 3 tested models. For our top model (model 3), the most important interaction with a SHAP value of 0.054 was between “talk about your problems or feelings with your parents” and “self-rated physical health.” Variable interaction data are presented in Figures S3-S8 in Multimedia Appendix 1.