The Use of Online Consultation Systems or Remote Consulting in England Characterized Through the Primary Care Health Records of 53 Million People in the OpenSAFELY Platform: Retrospective Cohort Study

Introduction

Background

The National Health Service (NHS) England Digital First Primary Care (DFPC) program [1] has led the work on enabling the implementation and improvement of the use of digital tools in general practice, including the use of online consultation (OC) systems (see Textbox 1 for OC system clarification) and video consultation (VC) systems. The NHS Long Term Plan [2], published in January 2019 before the COVID-19 pandemic, committed to ensuring that every patient in England has the right to digital-first primary care by 2023-2024; the 5-year general practitioner (GP) contract reform framework to support the NHS Long Term Plan specifically committed for all practices to offer OC and VC systems by 2021. Legislation that came into force in October 2021 requires all NHS GP practices to make an OC system available to their patients [3]. The COVID-19 pandemic response significantly accelerated the adoption and need for such systems. An internal NHS England data collection of information provided by OC system suppliers (rapidly established in April 2020) [4] showed that as of June 21, 2021, approximately 79% of the practices in England had OC system capability in place. This was believed to stand closer to 95% when considering gaps in data collection [5]. In early 2021, there were approximately 10 weekly OC submissions per 1000 population, although underscored by local variation. Most recently, the NHS England Delivery Plan for Recovering Access to Primary Care, published in early May 2023, outlined the ambition for general practices to move to a fairer, safer, and more sustainable model of general practice termed the Modern General Practice Access Model [6]. An important enabler of this model is the use of OC systems for improving access and supporting care navigation and triage in general practice.

Given the rapid nationwide adoption and use of OC systems and other digital tools (including alongside and in response to the COVID-19 pandemic), there is a need to better understand how they are used, implemented, and generate impact. Monitoring and evaluation have been stood up by the DFPC team as well as independent studies. Specifically, commissioned work includes research questions comparing different models of implementation and different types of OC systems; analysis of patient experience; and analysis of the impact of digital tools on outcomes such as prescribing patterns, accident and emergency department attendances, and emergency admissions [9,10]. However, activity-monitoring data sets such as General Practice Appointments Data (GPAD) [11] or the internal NHS England data collection from OC system suppliers are aggregate and do not capture sociodemographics, clinical history, or patient pathways. The data available in GPAD are classified as experimental data due to variations in practice coverage. GPAD presents details of patient appointments that are recorded in the general practice appointments system, rather than the totality of interactions. As such, it cannot be considered to be a complete view of general practice activity. Specifically, it does not currently capture all triage activity or all appointments following from online requests to practices because they are often managed within other IT products. The GPAD data set also does not currently include all enhanced-access evening and weekend appointments, which are managed by other appointment books. In terms of consultation appointment mode, it only includes a broad video or online category (including VCs, videoconferencing, written OCs, and the like), which is returned with variable completion and standardization. By contrast, more targeted pilots and evaluation studies have been designed that do access richer quantitative or qualitative information [9,10,12,13], but these tend to be localized to a single supplier or set of general practices; therefore, they may not be nationally representative or provide a full picture along the pathway.

Objectives

OpenSAFELY is a new secure analytics platform for electronic health records (EHRs) in the NHS, created to deliver urgent insights during the global COVID-19 emergency. The platform uses a novel approach for enhanced security and timely data access that avoids the need to migrate large volumes of disclosive pseudonymized patient data outside of the secure environments managed by the EHR software companies (eg, The Phoenix Partnership [TPP] and Egton Medical Information Systems [EMIS]); instead, it relies on trusted analysts to run computations and analysis on near–real-time pseudonymized patient records still held inside the data centers and secure cloud environments of EHR companies. With the approval of NHS England, we conducted a service evaluation using the NHS England OpenSAFELY COVID-19 research platform. In this particular study, we explore EHR coding activity that is related to the use of OC systems and written OCs (that is, responses delivered by SMS text messages or online messages) in general practices via OpenSAFELY. By using primary care EHR system data (Textbox 1), it is possible to leverage nationally representative, longitudinal patient cohort data regarding clinical and administrative encounters and to analyze this in the context of other factors such as geography, sociodemographics, clinical characteristics, and other health care interactions. Given the limited pre-existing standardization and insight into the coding of submissions received via an OC system and the coding of the mode of consultation, we set out to focus on the following aims:

• Understanding OC system and written OC coding use and prevalence in primary care records by codes of interest (OC system-relevant as shorthand)
• Understanding the variation in OC system–relevant coding use over time, before and after the start of the COVID-19 pandemic, and in terms of interpractice variation
• Understanding broad demographics and past clinical history of patients with OC system–relevant coding activity

This exploratory analysis may help inform further research and evaluation questions and their feasibility, including large-scale impact evaluation studies.

Methods

Study Design

We conducted a retrospective cohort study using general practice primary care EHR data from all GP practices in England with EHR vendors TPP and EMIS as suppliers. This analysis project is part of a “ways of working” pilot to onboard into OpenSAFELY any new approved users or researchers (including NHS England analysts) [14].

Data Source

All data were linked, stored, and analyzed securely within the OpenSAFELY platform [15], which is a data analytics platform created with the approval of NHS England to address urgent COVID-19 research questions. Data records used in this study are pseudonymized general practice primary care EHR data from practices in England that are supplied by the vendors TPP and EMIS. These contain data such as diagnoses, medications, and sociodemographic characteristics. Similarly pseudonymized data sets from other data providers are securely provided to the EHR vendor and linked to the primary care data, such as information on care home status. No free-text data are included.

The TPP database analyzed via OpenSAFELY (OpenSAFELY-TPP) is based on 24.2 million people currently registered with 2546 GP surgeries using TPP SystmOne software, while the EMIS database analyzed within OpenSAFELY (OpenSAFELY-EMIS) is based on 32.6 million people currently registered with 3821 GP surgeries using EMIS. Together, these represent approximately 99% of the practices. Most of the outcomes in this study explore OpenSAFELY-TPP, although a more fixed-scope overall coding use and prevalence characterization was extended to OpenSAFELY-EMIS as well, based on what was feasible in its early operational days (Multimedia Appendix 1).

OpenSAFELY-TPP and OpenSAFELY-EMIS were used side by side to identify the use of OC-relevant activity in the period surrounding the pandemic start (from January 2019 to December 2020) in terms of individual code use and practice coverage. OpenSAFELY-TPP, covering approximately 40% of practices, was used to understand trends in coding activity over time (before and after the start of the COVID-19 pandemic), interpractice variation, associations with sociodemographic factors, and associations with clinical history factors.

For benchmarking and triangulation, data from the national OC and VC system supplier collection are also used [4]. This data collection was stood up rapidly at the start of the pandemic and includes aggregate use data taken directly from the participating OC system suppliers. This contains daily information from August 2020 onward, derived from the daily collection files. It can be extended as a weekly trend back to April 2020. No information on demographics, clinical history, or pathway is part of its specifications. On completeness, an audit undertaken on March 31, 2021, suggested that approximately an additional 10% of practices were using an OC system supplier that did not contribute to the national collection. As of September 2021, there were 5 out of 20 suppliers that had not submitted their data, meaning that metrics on total national use will be understated, which should be taken into account when benchmarking. However, the situation had been evolving because several suppliers were working toward submitting their data. Practices using either TPP or EMIS as their EHR system were identified by linking to a separate data collection, Patient Online Management Information (POMI) [16].

Coding Systems

In general practice, staff record information about patients using clinical coding systems such as Systematized Medical Nomenclature for Medicine–Clinical Terminology (SNOMED CT) and the Dictionary of Medicines and Devices. The TPP system is fully compliant with SNOMED-CT, with GPs using it in their front-end interactions with EHR systems, having previously used Clinical Terms Version 3 (CTV3) before the NHS-wide standard was adopted. OpenSAFELY can query the records using either CTV3 or SNOMED-CT, which allows flexibility on querying some past activity that cannot be easily mapped to SNOMED-CT.

Approach to Deciding Codes for Interrogation

We could not ascertain the existence of a nationally consistent and standardized codelist for activity associated with OC systems or with the carrying out of a written OC, although NHS England is undertaking work to facilitate standardized coding; for example, SNOMED-CT codes [17] for OC systems were agreed upon in 2022. The historic absence of such codelists relates to a number of reasons, including the recency and pace of technology implementation and adoption, the lack of standardization of terminology and appropriate codes around digital forms of interaction with a practice (by route and mode), and the disparity between supplier systems and templates. As such, a custom SNOMED-CT codelist was created on OpenCodelists with existing relevant codes. This is available for inspection and reuse by anyone [18]. The codes are also provided in Table 1.

In terms of criteria, the term online consultations is ambiguous and can include submissions and requests received from patients via an OC system (route of access) or a consultation mode using written electronic messaging (appointment mode). Codes of interest were those deemed to be associated with either submissions made using an OC system (route) or written OCs (mode). Keywords included in the search were “consultation (procedure),” “econsultation,” “indirect encounter,” “online,” “remote triage,” “telemedicine,” and “telepractice.”

The codelist was developed using the following steps: (1) browse the SNOMED-CT Term Browser [9] for relevant keywords and children concepts (SNOMED-CT is a hierarchical terminology system where individual concepts can have parent concepts as well as children concepts that are essentially subtypes) and find their CTV3 equivalents (refset), if listed; (2) browse the NHS Digital CTV3 to SNOMED-CT Mapping Lookup [10] for relevant keywords and children concepts, and find their SNOMED-CT equivalents, if listed; (3) browse local TPP codes [11]; (4) pragmatically browse the literature, online resources, and white publications for further code indications [11-15]; and (5) obtain clinical and program input via the DFPC program on the initially found codes of interest (long list), as well as further codes, to arrive at a refined list.

Recorded Code Activity and Practice Coverage Over Time (TPP)

The coding activity over time was characterized by individual code. The interpractice variation was also assessed.

Cohort

Using OpenSAFELY-TPP, for each week or month (period) of the study, the population of interest was defined as those aged ≥1 year (an early design choice to discard the possibility of 0 representing a null return), alive, and registered at the start of this period. Patients were assigned to the practice with which they were registered within this period. In turn, any activity (OC-relevant codes and GP consultations) a patient had in this period was assigned to their practice of registration. This yielded a study population of >23 million patients, relating to 2551 TPP practices.

Analysis

We extracted the number of times each code was recorded (1) during the period from January 2019 to December 2020 at monthly intervals and (2) from the period of January 6, 2020, to March 22, 2021, at weekly intervals. The OpenSAFELY weekly data were specifically generated for contextualization and benchmarking with a relevant weekly measure of OC submissions from a separate data source: the national OC or VC supplier data collection. The OpenSAFELY monthly data were used for all further analysis. Absolute count instances and rates per 1000 registered practice patient population were computed [19]. Practice coverage—the number of practices with at least 1 instance of the code over the 2-year period—was also calculated, at both national and regional levels [20]. We also calculated the rate at which certain codes were recorded per 1000 registered patients at a general practice level (among practices with any instance over the 2-year period) following the methods described in the NHS England National General Practice Improvement Programme [1]. We computed the deciles, median, and interdecile range for February, April, September, and December 2020 (ie, quarterly and for the start of the pandemic period).

The general practice consultation activity over this same period was also recorded for context; this uses a purpose-built function on OpenSAFELY rather than relying solely on counting code instances [19]: cohortextractor.patients.with_gp_consultations() captures GP-patient interactions, whether in person or by telephone or video call. The concept of a “consultation” in EHR systems is generally broader and might include things such as updating a telephone number with the receptionist. It captures events such as interactions and “consultations” differently from what is captured in the GPAD data set from GP appointment systems. It might also—but will not necessarily—capture interactions from OC-related events. In the analysis in this paper, the metrics and shorthand terminologies “GP consultation” and “GP-patient interaction”—whether in the Methods section, figures, tables, or descriptions—will refer to those obtained via the aforementioned OpenSAFELY EHR method, with the relevant caveats.

Sociodemographics of Patients With Relevant Coding Activity (TPP)

Sociodemographic characteristics were characterized for patients with any OC-relevant coding instance.

Cohort

The population cohort was defined as all those registered with a single TPP GP practice between January 2019 and December 2020, resulting in a cohort of >20 million patients. The following characteristics were recorded, typically based on January 2019 status: ethnicity (based on ethnicity codelists), gender, age, care home status [21], household size, practice registered with and associated region, rurality of place of residence, disability status (learning disabilities and intellectual disabilities codelists created from the Quality and Outcomes Framework register), and Index of Multiple Deprivation 2019 quintile [22]. These characteristics come mainly from the primary care records themselves. While these are detailed and longitudinal, they can be incomplete regarding patient characteristics. Instead of excluding incomplete records, missing characteristic information is recorded as “unknown.”

The Index of Multiple Deprivation is a national statistic and the official measure of relative deprivation experienced by people living across England [22]. It is calculated for every lower layer super output area (approximately 1500 people) in England, and the overall rank is derived by combining and weighting the following domains: income; employment; health deprivation and disability; education, skills, and training; crime; barriers to housing and services; and living environment. In this study, each patient was assigned a deprivation rank and quintile according to the lower layer super output area they reside in.

Analysis

When producing summary statistics, the study population was divided into whether the patients had had any recorded OC-relevant coding instance (at least 1 match for any of the short-listed OC codes in the January 2019-December 2020 period). Summary statistics were also computed for patients who had had any GP consultation in this same period. For a given sociodemographic or geographic dimension at a time, we computed the following for the 2-year period: (1) the instance rate of codes (number of code instances, standardized per 1000 registered practice patient population), and (2) the coverage rate of codes (portion of population with at least 1 code instance).

Clinical History of Patients With eConsultation Activity (TPP)

In further follow-up exploratory analysis, the clinical history of patients with eConsultation activity was investigated.

Cohort

The cohort included TPP practice–registered patients with a single practice between March 1, 2019, and February 28, 2021. Occurrences of the eConsultation code or of GP-patient interaction were recorded. Only patients in practices with non-nil eConsultation coverage were considered. Age and gender were captured. Clinical history flags per patient were dictated by whether each patient had any recorded occurrence from the individual codelists before March 2019. These were chosen to broadly align with the most prevalent long-term conditions according to the NHS England Population and Person Insight dashboard and framework [23]. The codelists were as follows: hypertension, asthma, chronic respiratory disease other than asthma, osteoarthritis, depression, diabetes, chronic heart disease, cancer, atrial fibrillation, stroke, peripheral arterial disease, heart failure, chronic kidney disease, and serious mental illness.

Analysis

Results, in terms of the historic prevalence of individual clinical conditions among those with an eConsultation code in either the prepandemic (March 2019-February 2020) or pandemic (March 2020-February 2021) period were captured. Prevalence figures were also tabulated alongside those of two relevant comparator subcohorts (only within practices using the eConsultation code at all in this period): (1) all the remaining practice population, and (2) the remaining practice population with a recorded activity of a GP consultation in this period. A multivariate logistic regression model was used to assess which clinical conditions were associated with higher adjusted odds of having had an eConsultation code recorded among the population in those practices that had had a GP consultation or eConsultation coding activity. Age groups and gender were also included as a first order case-mix adjustment. Further confounding (whether from interactions or other omitted characteristics) may remain.

Recorded Code Activity and Practice Coverage (EMIS and TPP)

For the analysis of coding activity leveraging both EMIS and TPP (about 99% of the GP practices), a more fixed-scope exploration was implemented as a compromise, given the OpenSAFELY-EMIS functionality and server availability, which was in its earlier stages. The results can be found in Multimedia Appendix 1 (“Coding activity prevalence [EMIS]”).

Cohort

We defined the population of interest as those aged ≥1 year; alive; and registered as of January 1, 2019. This resulted in cohorts of >23 million and >30 million patients for TPP and EMIS, respectively. Patients were assigned to the practice they were registered with at the start. In turn, any activity (codes associated with OC systems, remote consultations, or with the EHR query on GP consultations) a patient had in this 2-year period was assigned to the initial practice of registration, rather than having activity reassigned to a new practice if the patient moved (this was necessary due to practical OpenSAFELY-EMIS functionality considerations). This differs from the TPP coverage analysis in the previous subsections, which reflected month-on-month fluctuations in patient registration for each patient and reassigned activity accordingly.

Analysis

A table comparing EMIS and TPP statistics under similar conditions is presented in Multimedia Appendix 1. We show the number of times each code was recorded during the period from January 2019 to December 2020, aggregated over the 2 years. Values are given per 1000 registered population and also per 1000 registered patient population in practices where the code was recorded. To mitigate practice disclosure, results are not shown where a code was found in <5 practices.

Software and Reproducibility

Data management was performed using Python (version 3.8.0; Python Software Foundation), with analysis carried out using R (R Foundation for Statistical Computing). All code for the OpenSAFELY platform for data management, analysis, and secure code execution is shared for review and reuse under open licenses [24]. All project code used for data management and analysis is also shared openly for review and reuse under the Massachusetts Institute of Technology license [25]. The developed codelist is publicly available as well [18]. The project code and codelists were transmitted to the OpenSAFELY-TPP platform within the TPP secure environment and to the OpenSAFELY-EMIS platform within the EMIS secure environment for local execution against pseudonymized data, that is, without the need for the researcher to locally see or move pseudonymized records. After a review for the disclosiveness of aggregate results in a secure server, the cleared aggregate results were released into final openly available outputs in the project repository. Although detailed pseudonymized patient data are potentially reidentifiable and therefore not shared (nor was it directly viewed when undertaking this research, given the platform design outlined previously), further queries to, or analysis against, the same pseudonymized records can be made through a new project request to the OpenSAFELY framework [14].

Ethical Considerations

NHS England is the data controller for OpenSAFELY-EMIS and OpenSAFELY-TPP, EMIS and TPP are the data processors, and all study authors using OpenSAFELY have the approval of NHS England. This implementation of OpenSAFELY is hosted within EMIS and TPP environments that are accredited to the ISO 27001 information security standard and are NHS Information Governance Toolkit compliant [26,27].

Patient data have been pseudonymized for analysis and linkage using industry-standard cryptographic hashing techniques; all pseudonymized data sets transmitted for linkage onto OpenSAFELY are encrypted; access to the platform is via a virtual private network connection, restricted to a small group of researchers; the researchers hold contracts with NHS England and only access the platform to initiate database queries and statistical models; all database activity is logged; and only aggregate statistical outputs leave the platform environment following best practice for the anonymization of results, such as statistical disclosure control for low cell counts [28].

The OpenSAFELY research platform adheres to the obligations of the UK General Data Protection Regulation and the Data Protection Act 2018. In March 2020, the Secretary of State for Health and Social Care used powers under the UK Health Service (Control of Patient Information) Regulations 2002 to require organizations to process confidential patient information for the purposes of protecting public health, providing health care services to the public, and monitoring and managing the COVID-19 outbreak and incidents of exposure; this sets aside the requirement for patient consent [29].

Taken together, these provide the legal bases to link patient data sets on the OpenSAFELY platform. GP practices, from which the primary care data are obtained, are required to share relevant health information to support the public health response to the pandemic and have been informed of the OpenSAFELY analytics platform. This study has been approved by internal NHS England processes as per openly available policies [30,31]. NHS England service evaluations or audits in OpenSAFELY are currently not required to have ethics approval. This study was supported by MB, DFPC National Clinical Director, as senior sponsor.

Patient and Public Involvement

We have developed a publicly available website [15] through which we invite any patient or member of the public to contact us regarding this study or the broader OpenSAFELY project.

Results

The results are presented in the following subsections for TPP-based practices and cohorts. The results for the combined OpenSAFELY-TPP and OpenSAFELY-EMIS analysis can be found in Multimedia Appendix 1; an aid to interpretation is given in Textbox 2.

Weekly Coding Activity and Contextualization With Rapid Supplier Data Collection

Figure 1A depicts the absolute instances of the code eConsultation, as well as of all SNOMED CT codes combined. It reflects coding activity through all TPP practices during the period from January 6, 2020, to March 22, 2021. However, different practices may approach the coding of different activity associated with the use of OC systems differently, especially based on the OC system supplier in place, chosen OC pathways, and implementation maturity [32].

Separately, Figure 1B depicts the total OC submissions during the period from April 27, 2020, to March 22, 2021, according to the NHS England OC or VC data collection from system suppliers (rapid and aggregate) [4]. This is likely still an underestimation due to data completeness issues.

The shapes of the 2 graphs over time look the same in terms of peaks and troughs, and both have a similar overall significant rising profile. If we assume that each OC submission should generate at least 1 activity code in the primary care systems, then the data suggest that coding activity in general practice IT clinical systems is not fully tracking all OC activity. The 2 data sources do track different but related activity. This is addressed in the Methods and Discussion sections.

‎

Codes in Use and Practice Coverage

Figure 2 shows the portion of practices that had at least 1 instance of the respective SNOMED CT codes over the 2-year period. Breakdowns by region are presented in Figures S1 and S2 in Multimedia Appendix 1.

Of the 18 codes, 12 (67%) returned no results in TPP. The last column of Table 1 indicates this. The SNOMED CT codes for which instances were found also correspond to those where both (1) a CTV3 mapping was available when specifying the codelists, and (2) CTV3 querying had activity recorded (not shown). The practice coverage, in decreasing order, was as follows: eConsultation via online application (1068881000000101; 2157/2551, 84.56% of the practices), telemedicine consultation with patient (448337001; 1822/2551, 71.42%), email sent to patient (401271004; 1782/2551, 69.85%), remote consultation encounter type (325871000000103; 1286/2551, 50.41%), alert received from telehealth monitoring system (699249000; 1122/2551, 43.98%), and encounter by computer link (185320006; 580/2551, 22.74%).

‎

Monthly Trends in Coding Activity

While the previous figures focused on coverage, Figure 3 shows the monthly use of the various codes (coding activity) during the period from January 2019 to December 2020. Values are given as a rate (per 1000 cohort population). The entire cohort population is considered, rather than just patients in practices where each code was recorded. The rate of GP consultation events is also given for context (the practice coverage is near complete at >99%, as expected). Absolute counts are shown in Figure S3 in Multimedia Appendix 1.

The codes with the highest activity, ordered by highest monthly peak, are presented in Textbox 3. We have also plotted the monthly rate of (overall) GP consultations regardless of the modality in the TPP practices. This stood broadly at >400 consultations per 1000 patients in 2019. The dip is seen around April 2020. Recovery occurred, with October 2020 registering the second highest monthly rate (519 consultations per 1000 patients) after October 2019 (532 consultations per 1000 patients; Figure 3A).

‎

Interpractice Variation in Monthly Coding Activity Trends

To better convey the coding activity over time and in terms of interpractice variation, the median and decile trends were created for contextual highlighting of GP consultations (Figure 4), eConsultation (Figure 5), telemedicine consultation with patient (Figure 6), and aggregate of all codes (Figure 7). Deciles illustrate variation across practices for a given metric in a more compact form; for each time point, practices are sorted and ranked from lowest to highest activity, with points that define the top 10%, 20%, ..., 80%, 90% plotted for each month. The 50% decile (ie, the median) is shown as a continuous bold line. Figure 7 shows that there is considerable variation in coding activity levels across practices, with the top deciles of practices gaining much of their activity around the start of the pandemic.

‎

‎

‎

‎

Demographic Patterns in OC Coding Activity

Broadly, Table 2 shows that the cohort with at least 1 OC-relevant coding instance has a statistically significant higher preponderance of female patients; has a higher relative preponderance of patients aged 18 to 40 years, followed by those aged 40 to 50 years and those aged 50 to 60 years; skews more toward White patients; and skews more toward those who are least deprived. The P value (indicating significance) is shown as well as the difference in relative distribution (in percentage points) to understand effect size. Further comparison with the population with GP consultations is given in Tables S1-S7 in Multimedia Appendix 1.

While instances of codes associated with OC systems or remote OCs cannot be compared directly to GP consultation figures or GPAD figures, it is useful to look at relative values across levels of a given sociodemographic characteristic. As such, for further contextualization of the results in Table 2, the distribution of the full study population based on whether they had general practice consultations (as opposed to not) is given in Multimedia Appendix 1 (“Further data: TPP cohort sociodemographic characteristics”). The differential patterns between patients with an OC instance and those with wider general practice interaction, by sociodemographic characteristics, are shown through indicators of rate and coverage. The effect size by individual sociodemographic characteristics is further highlighted in the tables (Multimedia Appendix 1) through standardizing the indicator value for each group of interest by that of the reference group (for ethnicity=White and for deprivation=5 [least deprived]). Patterns are similar to those found when comparing OC activity with the full cohort population in Table 2, albeit in a less pronounced way, both in rates and coverage. Specifically, we still observe a higher relative preponderance of OC-coded activity in patients aged 18 to 40 years and a higher preponderance among those who are least deprived, White, or female. Some other subcohorts are small; therefore, figures and differential patterns require caution in drawing firm conclusions, given the underlying uncertainty.

Patterns in Clinical History for Patients With eConsultation Coding Activity

The cohort of patients who, during the period from March 2020 to February 2021, had eConsultation activity coded in their records was characterized overall by a lower prevalence (clinical history) of most long-term conditions compared to the remaining population with recorded GP consultation activity during this period (Table 3). Notable exceptions were asthma and depression, where 20.17% (164,868/817,547) and 22.81% (186,523/817,547), respectively, of the eConsultation patients had a clinical history of these compared to 17.21% (1,551,624/9,018,200) and 19.46% (1,754,903/9,018,200), respectively, for other patients with general GP-patient interactions. The comparison against the full population in these practices (rather than just patients therein with GP consultation recorded activity) is given in Table S8 in Multimedia Appendix 1, producing a starker difference for asthma and depression but a more homogeneous profile otherwise (indicating that OC patients resemble more the general population, with respect to clinical history). The tabulation for the prepandemic eConsultation activity is given in Table S9 in Multimedia Appendix 1.

From the previous subsection, we have also seen that activity skews to a younger and female-tilted profile, which may be masking some of the clinical history profile characteristics. Table 4 shows that, when considering all clinical history conditions simultaneously, while also factoring in age and gender (first order adjustment), the clinical characteristics that were more prevalent among those with eConsultation activity recorded are those listed in Table 5 (odds ratio >1 and significant). Other clinical history conditions were mainly less prevalent with these adjustments.

Discussion

Summary

Using OpenSAFELY-TPP and OpenSAFELY-EMIS, we were able to generate data on clinical coding activity relevant to OC systems and remote monitoring across approximately 99% of practices and >53 million patient records. We observed large variation in coding instance rates among practices in England and between the 2 EHR systems. For TPP practices (circa 40% of the practices in England), we explored further the trends and variation in coding activity related to digital forms of interaction. Coding activity increased rapidly during the study period, with a marked increase and acceleration at the start of the pandemic and the first lockdown. Furthermore, we found population subcohorts—both in terms of sociodemographics and clinical history—against which the recorded instance rates were higher.

Above all, this work highlighted that more needs to be done to consolidate and harmonize activity definition (by route and mode) and coding practices associated with the use of OC systems in general practice. Work underway by NHS England (eg, the creation of SNOMED CT codes for OC systems), alongside work to standardize the definitions of demand, capacity, and activity and drive improvements in the user interface of IT systems to support improvements in the consistency of coding and data quality, begin to address this need. As another example, regular open automated reporting could be established via OpenSAFELY, as has been done elsewhere [20], which monitors the coverage and instance rates of relevant codes over time so that central and regional NHS managers or GP practices can review the coding quality over time on its own and through triangulation with local sources or OC system supplier data.

Findings in the Context and Comparison of Existing Evidence

Of the 18 codes identified, 6 were in active use in TPP practices. The ones used by more practices were, in order, eConsultation via online application (2157/2551, 84.56% of practices), telemedicine consultation with patient (1822/2551, 71.42%), and email sent to patient (1782/2551, 69.85%). In the analysis extension to OpenSAFELY-EMIS, its practices also had registered activity for these codes, but their volumes were quite different, indicating likely differences in the digital systems of different suppliers, coding approaches, or population served. The code eConsultation via online application has been explicitly linked to OCs or triages, specifically by supplier eConsult [34] and North of England Commissioning Support in their SystmOne guidance [35]. The code consultation via multimedia encounter type, detected in EMIS, has been suggested for use in NHS England total triage guidance [36].

Coding activity related to digital forms of interaction picked up rapidly from the start of the COVID-19 pandemic and the first lockdown (from March 2020 onward). In the second semester of 2020, >9 monthly eConsultation coding events per 1000 registered population were registered compared to <1 per 1000 a year prior. This broad rising trend observed for OC system codes was consistent with the rising trend in weekly OC submissions as captured in the NHS England OC or VC supplier data collection [4], including broadly in terms of the peaks and troughs. Increased adoption and use was expected over the period, given the prepandemic (2019) commitment for all practices to implement and offer OCs and VCs by 2021, with a further legal mandate coming into force in October 2021. The pandemic response likely further accelerated this implementation and also favored increased use, given the substantial shift away from face-to-face appointments due to infection, prevention, and control measures as well as national guidance [36-38]. Although the pandemic was declared over in May 2023 [39], the use of digital forms of GP practice interaction and consultation will continue to be delivered and even likely increase, given the NHS England Delivery Plan for Recovering Access to Primary Care (2023) [6] and the key role played by OC systems in the Modern General Practice Access Model outlined by the plan.

It was noted that NHS England OC or VC and coding event indicators, while similar in trends and patterns, did differ in absolute values. This may relate to a range of reasons, including codes such as eConsultation only being triggered downstream from what is considered a submission in an OC system, certain practices or OC systems not yet using dedicated codes, certain practices or OC systems using codes that are different or broader than those studied here, how an OC submission is defined within the supplier data collection, practices using some codes for clinical activity but not for administrative activity, and certain practices potentially not recording all OC activity in the GP IT clinical system (eg, repeat requests or some practices managing web-based requests and subsequent appointments within their OC system rather than the GP IT clinical system).

Over 2019 and 2020, respectively, 227,429 and 3,323,333 OC system–relevant codes were found in the TPP cohort. When contextualized with wider GP consultation or interaction coding, this corresponded to 1.8 OC system codes for every 1000 GP consultation codes in 2019 and 27.9 OC system codes for every 1000 GP consultation codes in 2020. Although a direct interpretation of this and other publication sources is complicated due to lack of coding standardization, interpretation, and completeness issues, this relative level of OC system use (either route of access or written consultation mode) is broadly in line with observations from other sources. GPAD is the central statistics publication on appointments that have taken place in general practice, showing for instance, record number of >30 million recorded appointments in November 2021. In terms of appointment mode, video, videoconference, or online are all captured under a single category, video/online, as per the GPAD specification. This was done to ensure greater consistency between EHR suppliers that submit data for GPAD publication, given the data quality and variability in the ways the submitting suppliers define the mode types, capture them, and report them [40]. In 2019 and 2020, respectively, 5.6 and 4.6 in 1000 appointments had been found to be recorded with respect to mode as either video, videoconference, or online (ie, written consultation), although, as mentioned, publication caveats that are related to how the mode of consultation is recorded and defined, field completion, and the effect of the pandemic warrant caution and could mean that levels are understated and conflated [41]. Much of the partial or total triage activity is also not reflected in the collected and published GPAD statistics. Further on appointment mode, in the self-reported GP Patient Survey 2020 and 2021 editions—each mainly covering experiences of the year prior—0.2% and 2.6%, respectively, of those who had booked an appointment said that they received an appointment to speak to someone on the web (eg, video or written) [42].

Interpractice variation was large, reflecting the recent nature of OC system implementation adoption and use: December 2020 saw the median practice have 0.9 recorded codes per 1000 population compared to approximately 36 for the highest decile of practices.

When compared to the full practice population or to patients who had consulted their general practice at all, the cohort of patients who had any recorded coding activity in 2019-2020 tended to skew toward female patients, White patients, and those least deprived. With respect to age, there was a higher relative preponderance of patients aged 18 to 40 years. Although focused on different types of general practice online service access (booking appointments, ordering prescriptions, and viewing records and not an OC system request or consultation), analysis of 2018 and 2019 GP Patient Survey data [43,44] showed analogous evidence of a strong deprivation gradient in the awareness and use of services in favor of those least deprived as well as a reduction in awareness and use for patients aged >75 years. Ethnicity was also associated with variability. A 2019 cross-sectional West Midlands self-administered survey also showed variation in the use and awareness of these 3 services with demographics, namely, lower levels with greater deprivation and with being male (awareness of prescriptions and the awareness and use of online appointment booking) [45]. Patterns by gender and age are also broadly in line with those found in a previous pilot in the South West, using eConsult [13]. The results may also be reflective of the greater implementation challenges, such as the time, capacity, and support required to embed the use of digital tools in practices working in the most challenging circumstances and the highest areas of deprivation, communications, and language to support people to navigate access points and general practice involvement in local commissioning decisions that were made at pace due to the urgency of responding to the pandemic, alongside support with patient factors such as health and digital literacy as well as confidence with, and access to, digital devices and data. Furthermore, there is variation in the design, functionality, and interoperability of OC systems, which may impact the usability and accessibility of different systems. This study does not explore the rates of use across different demographic characteristics between different types of OC systems.

We also found an overall lower prevalence of most long-term conditions in patients who had an eConsultation compared to those with other GP consultation interactions recorded in the same period—in this case, the period from March 2020 to February 2021. In part, this overall lower prevalence may reflect the inherent nature of the intended OC submissions themselves, which are not exclusively focused on the need for a traditional GP consultation but reflect more general population needs such as administrative tasks as well as digitally enabled routine checks and queries.

We found early indications that patients with eConsultation coding activity were more prevalent in cohorts of patients with a clinical history of asthma, depression, or heart conditions when compared to the cohort of other patients with any overall GP consultation activity in the same period. This increased activity may be reflecting not only expected increased general health use in such cohorts but also the prioritized development and adoption (due to the size of the target population or suitability for chronic condition management) of tailored eConsultation solutions and formularies for these cohorts of patients with long-term conditions [46]; for instance, solutions for mental health, self-administered psychometric tests such as the General Anxiety Disorder-7 questionnaire or the Patient Health Questionnaire-9 that can be later reviewed by health professionals, online self-referral routes, and online modes of delivery for talking therapies [47,48] can be supported by, or made interoperable with, an OC system.

More broadly, when considering the aforementioned profiling, it is important to note that the OC system user profile may not be generalizable and may be very dependent on the practice-by-practice model. For instance, the user profile may be influenced by how the OC system has been implemented, to whom and for what conditions the practices have promoted the OC option, the type and design of the OC system, the ease of finding and navigating the OC system, and staff confidence in using digital tools and explaining them to patients. There may also be differences in the user profile in practices with high use of OC systems compared to practices with low use. The authors of the 2016 South West England pilot study did note that the practices involved in the early pilot had fewer patients with long-term health conditions than practices in the rest of England, reflecting potentially greater early-adopter appetite or capability by such practices [13]. Recent studies [49,50] also noted the importance of a complex interplay of multiple determinants of good access, including factors such as service responsiveness, flexibility and choice (eg, between digital and nondigital access routes and consultation types), the type and complexity of the problem, continuity, communication, and the wider social context.

Strengths and Limitations

The key strength of the study relies on the scale and completeness of the underlying record-level EHR data. With OpenSAFELY-TPP, analysis can be run directly on the full data set of raw, single-event clinical events, including tests, treatments, diagnostic events, and other diagnostic and sociodemographic information, covering approximately 40% of the practices in England and equating to >20 million patients, if considering those with a stable practice of registration over the study period. When used along with OpenSAFELY-EMIS, this extends to about 99% of practices. Linkage to secondary care and mortality information is also built in. In comparison with other general practice setting data sources, the Clinical Practice Research Datalink data set holds records on only a sample of patients across 2 databases, while the General Practice Extraction Service data set held by NHS Digital contains fewer data items for each individual patient. NHS England holds a number of record-level commissioning data sets, but these are primarily focused on capturing secondary care activity, specifically across inpatient, outpatient, and emergency settings. NHS England open publications such as GPAD can only give a high-level view of a proportion of activity, with no insight into clinical or sociodemographic factors, quality, or the full workload of general practices. Furthermore, these publications group coarsely all forms of eConsultation as mode of consultation (video or written) and do not provide an easy route to follow GP access types and pathways, although GPAD improvements are being rolled out, including on standardizing appointment categories [51]. Another key strength is the transparency and reproducibility of the analysis undertaken. All code for the platform, data management, and analysis is shared openly on GitHub, allowing for peer review and reusability under open licenses.

In terms of key limitations, as highlighted when describing the codelists in the Methods section, a data-driven approach is taken that relies on SNOMED CT codes and mapping to historical CTV3 hierarchy. Caution must be applied when interpreting the results because not all activity from the use of the OC system is captured in the GP IT clinical system, the codes examined are not specific, and it is unclear whether they are being used to describe either submissions made using an OC system or the consultation mode by written online message or another remote consultation modality. In addition, practices vary in their coding practice, and therefore the use of the codes is unlikely to be consistent. The improvement in coding definition and quality will directly enable potential improvements in insight from OpenSAFELY (and from any other data assets leveraging EHR systems, such as General Practice Extraction Service and Clinical Practice Research Datalink).

Opportunities for Future Research

The first area to highlight relates to coding quality. By highlighting characteristics and gaps in coding approaches and activity, the insights from this study support work underway with system suppliers to improve coding guidelines and implementation. By regularly monitoring and reviewing the coding activity and reviewing it in context with further data sources and evidence on OC system implementation and use, coding quality could be proactively tracked and improved. This can be done through reduced burden for administrative and clinical staff, given that analysis can be executed in a single framework from re-executable code. As OpenSAFELY encourages curation of thematic and open codelists, the existing codelist can be managed and updated in line with these developments and such that, when relevant, it becomes increasingly aligned with ongoing specification work for GPAD and the forthcoming NHS England OC system supplier data collection being stood up.

Second, forward-looking analysis will be able to leverage insight brought by upcoming SNOMED CT codes that better define and differentiate OC system submissions and written OCs and that differentiate triage in terms of request type (clinical and administrative), the mode of consultation (written OC, video, telephone, and face-to-face), and professional type, allowing for a more journey- or pathway-centric view of digital-first primary care activity. The wider operationalization of OpenSAFELY-EMIS also means that, alongside OpenSAFELY-TPP, studies will be able to draw more comprehensively on data encompassing >99% of the English population [52].

The third area to highlight relates to future opportunities to use OpenSAFELY—alongside a refreshed curated codelist tailored to maximize the tracking of underlying patient activity, namely the code eConsultation—for large-scale cohort or longitudinal studies that help inform continuous improvement and evaluation on service use and patient outcomes. Examples of research questions include the creation of an observatory of inequalities, where a strategic metric is chosen for tracking variation by a key protected characteristic; the characterization of the types of demands and requests submitted by OC system users; and pathway analysis to support impact evaluation protocols (eg, illustratively, the impact of online access and comparisons of modes of consultations on the continuity of care; the rates of accident and emergency department attendance and admissions; secondary care referrals; and general practice appointments use by role and modality, reuse, and prescribing) [9,10,53]. As for the assessment of real-world pilot studies in specific practices (eg, through counterfactual analysis), by design, OpenSAFELY outputs with disclosive practice identifiers cannot be extracted, and querying on practice identifier is not yet straightforward, given pseudonymization. However, there is currently an ongoing cluster randomized controlled trial project where such matching is being carried out in the background by TPP. Subject to ensuring that such functionality would still comply with information governance and data protection—with appropriate processes, controls, and safeguards in place—the ability to define a pilot (intervention) and counterfactual group in the OpenSAFELY study design could in future be extended to external collaborators through simplified platform functionality.

Conclusions

Insights from this study increase the understanding of the implementation and use of OC systems and written OCs in terms of implementation, trends, and variation. Alongside operational data and evaluation studies, this can support the evidence base around models of OC system implementation and differential patterns of access and uptake. Current gaps in coding practice are also highlighted and can therefore support conversations with practices, OC system suppliers, and EHR suppliers on ensuring consistent and widespread coding practices. Further work that could be leveraged via OpenSAFELY includes key metric monitoring, such as coding quality, coding activity, or variation. Furthermore, the design and implementation of large-scale impact evaluation studies can be considered to understand the types of demands and characteristics of OC system users and how the use of OC systems affects outcomes such as the continuity of care as well as the type and modality of general practice consultation use or unplanned urgent and emergency care.