For the objective of this study, data from the 2018, 2019, and 2020 CCHS and Canadian Urban Environment (CANUE) were used. CCHS is an annual national population health survey conducted by Statistics Canada to gather data on health status, health care use, and health determinants at the health region levels of geography. The CCHS covers the household population from 12 years of age and older living in the 10 provinces and 3 territories. The CCHS sample is selected using different frames according to the age group. For the adult population (18 y and older), the sample of households is selected from an area frame. For the youth population (12 to 17 y old) a list frame is used to select persons. The area frame used by the Labour Force Survey is used as a sampling frame for the adult population. The Labour Force Survey uses a 2-stage sample design. In the first stage, a sample of primary sampling units, corresponding to geographical regions called clusters, is selected. In each selected primary sampling unit, a sample of dwellings is drawn at the second stage. To sample persons for the youth population between the ages of 12 and 17 years, the CCHS uses a list frame created from the Canadian Child Benefit files. The sample for the youth population is selected from a list of individuals. For adult participants, 1 person is selected per household using varying probabilities taking into account the age and the household composition. Each year the CCHS collects information on approximately 65,000 Canadians (60,000 aged 18 y and older and 5000 aged 12 to 17 y) [15]. The CCHS uses 2 separate computer-assisted interviewing applications to collect data, 1 for telephone interviews (computer-assisted telephone interviewing) and 1 for personal interviews (computer-assisted personal interviewing). Approximately 25% of these completed cases were conducted in person using computer-assisted personal interviewing, and the other 75% were conducted over the phone using computer-assisted telephone interviewing. Starting in 2015, the CCHS undertook significant redesign in sampling and the content [15]. Data about the outcome variables of the proposed study (perceived and unmet mental heath needs) were not collected in 2015, 2016, and 2017 CCHS, therefore, we will analyze the 2018, 2019, and 2020 CCHS data.

CANUE is a Canadian Institutes of Health Research–funded collaboration that focuses on developing robust methods for producing measures of urban form that capture a wide range of characteristics for every postal code in Canada [16]. It has produced a unique repository of standardized metrics of urban, suburban, and rural characteristics. CANUE shares a range of data that can be directly linked with population health surveys through postal codes, including neighborhood-level material and social deprivation [17], marginalization [18], and walkability. CANUE is updated every 5 years. The most recent CANUE update was in 2016. The CANUE data can be obtained from CANUE consortium at no costs for research, and is linkable to the CCHS by postal codes.

In the CCHS, perceived and unmet mental health needs were measured by the Meadows et al [19] Perceived Needs for Care Questionnaire (PNCQ). The PNCQ was designed and field tested for the Australian National Survey for Mental Health and Wellbeing which was commissioned by the Federal Government of Australia [19]. The PNCQ assessed four types of help for problems with emotions, mental health, or the use of alcohol or drugs: (1) information about problems, treatments, or services; (2) medication; (3) counseling or therapy; and (4) other mental health services. Respondents were asked which types of help they had received in the past 12 months. For each type received, they were asked if they felt they had received enough. For each type of help not received, they were asked if they felt it was needed. Based on the PNCQ, the following binary outcome variables can be derived: [9,20]

This is a secondary data analysis using the deidentified data collected by Statistics Canada. The data analysis was carried out at one of the Research Data Centers across the country, located on the campus of Dalhousie University. Researchers who conducted data analysis had to go through security clearances and become “deemed employees” of Statistics Canada. The results were vetted by data analysts of the Research Data Center to ensure privacy and confidentiality. CCHS participants gave informed consent to Statistics Canada for use of their data in accordance with the Statistics Act and no compensation was provided to the CCHS participants. For these reasons, ethics review was waived by the health research ethics committee of Dalhousie University.

The 2018 CCHS data were used to develop the prediction models. The demographic and socioeconomic characteristics of the participants in the 2018 CCHS are presented in Table 1. In the 2018 CCHS, 17.82% reported perceived mental health need and 3.81% reported unmet mental health need. In 2019, the proportion of perceived and unmet mental health needs was 18.04% and 3.91%, respectively; in 2020, the proportion of perceived and unmet mental health needs was 18.1% and 3.92%, respectively.

The individual based models with the best performance for predicting perceived mental health need are in supplemental tables (Tables S1 [Atlantic region], S2 [Central region], and S3 [Western region] in Multimedia Appendix 1). Sex, age, self-reported mental health, physician diagnosed mood and anxiety disorders, and self-reported life stress and life satisfaction were the predictors in the 3 regional models. Perceived work stress and household food insecurity were in the models for the Atlantic and Central regions, but not for the Western region. However, immigrants, smoking, or problematic drinking and material deprivation were predictors in the models for the Atlantic and the Western region, not for the Central region. The individual based models had good discriminative power with C statistics over 0.83 (Tables S4-S6 in Multimedia Appendix 1) and good calibration by comparing observed and predicted proportions.

Converting the individual based models into synthetic estimation models, the predicted proportion of perceived mental health need in each province was estimated. The absolute difference between the observed and predicted proportions in 2018 was less than 1% for all provinces (Tables 2-4), except Saskatchewan (3.37%). This is not surprising as the models were developed using the 2018 CCHS data. To validate the models, we applied the synthetic models directly in 2019 and 2020 data. The extent to which the predicted or synthetic estimates agree with observed proportions varied by regions (Tables 2-4). The models had the best performance in Ontario and British Columbia (absolute difference <1% in both 2019 and 2020), followed by Quebec (absolute difference=1.54% in 2019 and 0.54% in 2020). The models for Nova Scotia, New Brunswick, Saskatchewan, and Alberta had 1 absolute difference being less than 1% and the other greater than 2%. The absolute differences between observed and predicted proportions in Newfoundland and Labrador (−4.16% for perceived needs in 2020) and Prince Edward Island (4.58% for perceived needs in 2019) were larger than those in other provinces. When applying the models in the 10 selected health regions, the data showed that the model performed well in health regions in Ontario (first 2 digits with 35) and in Quebec (first 2 digits with 24), supported by the relatively small absolute differences between observed and predicted proportions (Table 5). Health region 1304 had the largest absolute differences in 2019 and 2020.

The individual based models with the best performance for predicting unmet mental health need are in Tables S7 (Atlantic region), S8 (Central region), and S9 (Western region) in Multimedia Appendix 1. As seen from the tables, sex, age, marital status, self-reported mental health, self-reported life stress, and low sense of belonging to the community were the common predictors in the models for the 3 regions. Household income was a predictor only in the model for Central region; professional diagnosed mood disorders, problematic drinking, and smoking and social deprivation were the factors specific for the model for the Atlantic region. The individual based models had good discriminative power with C statistics over 0.77 (Tables S10-S12 in Multimedia Appendix 1) and good calibration by comparing observed and predicted proportions.

The individual based models were converted into synthetic estimation models, and the predicted proportion of perceived mental health need in each province was estimated (Tables 2-4). The data demonstrated that the models for unmet mental health need performed well in most of the provinces with the absolute difference between observed and predicted proportions being less than 1%, except for Prince Edward Island, Nova Scotia, and Saskatchewan. When applying the models in the 10 selected health regions, the data showed that the models calibrated well with data in all selected health regions with absolute difference between observed and predicted proportions being less than 1% (Table S13 in Multimedia Appendix 1).

The 3 waves of CCHS showed that the proportion of the Canadian population that perceived the need for mental health care was between 17.82% and 18.1%, and about 4% reported unmet mental health need. This study demonstrated the feasibility of integrating individual and community level data to build informative synthetic models for perceived and unmet mental health care needs at the population level. The models performed well in predicting the outcomes at both the provincial and health regional levels, particularly in populous provinces such as Ontario, Quebec, and British Columbia. The absolute differences between the observed and predicted proportions of perceived and unmet mental health needs in Ontario, Quebec, and British Columbia were less than 1%. For the rest of the provinces, the results showed that the models at provincial and health regional levels predicting unmet mental health needs had better calibration than the models predicting perceived mental health need.

This study had several limitations. First, although the predictors in the models were associated with the outcomes, causal inferences cannot be made. The goal of the prediction models is to identify a key set of factors that in combination are best predictive of the outcomes. The models are not to test hypothesis or make inferences about etiology. Second, the relationships between the selected factors and the outcomes are complex. The logistic regression model is a linear function. Although we found no evidence of interactions among the selected predictors, nonlinear relationships between some predictors and the outcomes are still possible. Future studies may test if models using machine learning techniques have better performance. Third, the selection of candidate predictors was limited by what CCHS collected. It is possible that other factors are associated with the outcomes of this study, but were not measured in CCHS, therefore could not be examined in this study. Finally, the models appeared to perform well at regional and provincial levels. Relatively large absolute differences between predicted and observed proportions of perceived and unmet needs were observed in some health regions (eg, health region 1304). Caution should be exercised for using them at the health region level.

The CCHS used a nationwide representative sample of the Canadian household population. However, we found that a model developed for the whole population performed well only in the most populous provinces such as Ontario and Quebec, but not in the rest of the provinces. Developing models for specific regions appeared to be a better approach. The regional models have some common predictors, including sex, age, self-reported mental health, physician diagnosed mood and anxiety disorders, self-reported life stress and life satisfaction for perceived mental health need, and sex, age, marital status, self-reported mental health, self-reported life stress, and low sense of belonging to the community for unmet mental health need. These factors have been found to be significantly associated with mental health problems [33] in the literature. On the other hand, the coefficients associated with these factors varied by regional models and these models also contained predictors that are regionally specific. This finding indicates that there are common factors in the mechanisms underlying perceived and unmet mental health needs across geographic regions and populations. The extent to which these factors influence perceived and unmet mental health needs may differ due to the population characteristics of these provinces and regions. The differences in population characteristics and the distribution of the outcomes across the regions may also contribute to the finding that the models contained region specific predictors.

We have not found studies on predicting perceived and unmet mental health needs at the population level. Therefore, a direct comparison with previous studies is not possible. There are several studies predicting county or state level suicide and severe mental illness. For example, Kandula et al [34] modelled county-level suicide risk in the United States using county-level predictors derived from 8 different databases of different sources (government programs, health surveys, or private organizations). For some predictors such as the prevalence of major depressive episode, only state-level estimates were available and these estimates were extrapolated to the counties [34]. Hudson [30] explored the utility of regression synthetic estimation model that incorporated individual data from the National Comorbidity Survey, census and hospital administrative data to predict state-level prevalence of severe mental illness. The advantages of these population risk prediction models are the use of community level predictors from existing sources or published research and the ability of adapting the models in local context. Notably, our study used the regression synthetic estimation modeling approach. We used CCHS data. The use of a single data source may improve the efficiency of data analysis, data access, and eventually of the decision-making process.

One critical element of building risk prediction models is assessing model performance and model validation. This is to ensure that the developed model is accurate and has good performance in different populations or at different time periods. In this study, we developed the models using the 2018 CCHS data, and validated the models using the data from 2019 and 2020 CCHS. Furthermore, we validated the models in a random sample of health regions. These models were designed to estimate population proportions and to identify regions with high levels of mental health need and unmet mental health need; these models were not to be used by clinicians to identify high-risk individuals. Therefore, the focus of model performance assessment can be different. Kandula et al [34] used symmetric proportional error (observed deaths–predicted deaths)/(observed deaths+predicted deaths) to quantify model calibration. Hudson [30] calculated the absolute difference between the predicted and observed prevalence of severe mental illness. In this study, we calculated the absolute difference between the observed and model predicted proportions of perceived and unmet mental health needs. The absolute difference indicates the extent to which the model prediction deviates from the observed value. However, there is no consensus about what an optimal threshold of absolute difference should be. There may be other indicators that are useful for assessing the performance of population level models. Consultations with policy and decision makers (ie, end users of the models) would be helpful to understand what indicators are informative about model performance, and the level of the model error that is acceptable.

The results of this study are expected to have implications for population mental health planning. Few would deny that resources allocation should be partly driven by needs, and needs assessments typically require the knowledge of potential changes in prevalence estimates and in local population profiles, for example, their demographics, diagnoses, and mental health services use. The prediction models developed by this study will allow decision makers and mental health services planners to forecast the proportions of perceived and unmet mental health needs in the years to come at the provincial (state) and health regional (county) levels based on the potential changes in local population profiles. Such profile changes may be estimated using health administrative data and national population census data. Additionally, region-specific estimates can help categorize health regions—for example, regions with relatively stable mental health needs especially those that remained in the highest or lowest groups, or regions in which the largest year-to-year changes are observed—and hence help identify areas with a greater need of preventive resources, or conversely identify areas where interventions seem to be effective.

In conclusion, accurate forecast of perceived and unmet mental health needs in the population can allow policy and decision makers and mental health services planners to categorize regions or communities that are at high need and to monitor changes so that they may mobilize resources and interventions to the right populations and the right places at the right time. Regularly collected population health data such as those from the CCHS are readily accessible to policy and decision makers and mental health services planners. The models are particularly useful for service planners at the health regional level because population health surveys usually do not contain sufficient numbers of participants at the health regional level. Future studies are needed to identify methods to improve prediction for regions with small numbers of residents.