The global population of forcibly displaced persons, including refugees, asylum seekers, and internally displaced persons, has surpassed 100 million people [1]. Both human-made and natural disasters can break down traditional centers of health care delivery, which may leave displaced persons with gaps in medical care [2]. Displaced persons who leave their countries of origin are often highly mobile between the time of displacement and resettlement [1], which can create challenges for continuity of care, including management of chronic medical conditions, provision of psychiatric and mental health services, and prevention and treatment of communicable diseases.

Humanitarian organizations that provide medical care for displaced persons do so with few resources, limited technological infrastructure, and high patient and staff turnover [3]. Staff turnover may pose serious challenges to maintaining continuity of care for displaced persons. Health information, if properly leveraged, can guide public health interventions for these populations [4,5]. More humanitarian relief organizations are transitioning their operations from paper to electronic medical records, but clinical documentation in ad hoc relief clinics is typically not standardized, nor conducive to analysis [2]. Specifically, clinicians working in resource-limited settings, particularly in disaster response scenarios, may not have time to consistently document all aspects of history-taking and the physical examination, and they may not themselves formally assign International Classification of Diseases, Tenth Revision (ICD-10) codes to patient encounters [4,5]. Quality of care in these settings can be greatly enhanced if data from clinical documentation can be effectively analyzed and interpreted [6,7]. Electronic health records designed for austere environments may also be an untapped resource to enhance population health and guide public health interventions, for example, by contributing to systems of syndromic surveillance for early warning of infectious disease outbreaks in humanitarian settings [7-9].

Past research has evaluated the ability for free-text clinical documentation to be analyzed with high efficiency, both by manual review and using natural language processing [10,11]. Studies using machine learning methods have used free-text clinical documentation, mostly from high-income settings, to categorize symptom presentations and classify diagnosed chronic diseases [11,12]. However, few studies have evaluated free-text clinical documentation in humanitarian relief settings where clinicians are under particularly high time pressures and operate with limited resources. There is a growing recognition that clinical documentation from humanitarian relief settings could provide much-needed data to measure the outcomes and improve the performance of humanitarian action [13]. To address this gap, we developed and tested a methodology of assigning medical diagnosis codes to free-text medical records collected from a humanitarian clinic at the Mexico-US border.

From an original sample of 8546 encounters, we found that 99.0% (n=8460) were assigned at least 1 ICD-10 code and therefore met inclusion criteria for analysis. These represented 5623 unique patients and 2774 unique diagnosis codes (Table 1). The diagnoses included communicable and noncommunicable illnesses (Table 2). The most common diagnoses included management of hypertension, diabetes mellitus, headaches, and encounters to receive prescription refills for chronic conditions. Common diagnoses that were likely related to infectious diseases during this study period included cough, fever, and urinary tract infection. The mean word count for free-text encounters was 20.5 words, with a median of 14 (IQR 2‐28) words per encounter. A sample of the free-text clinical documentation and diagnosis code assignments are shown in Table 3. There were 58.78% (4973/8460) encounters where both coders assigned 1 diagnosis code, 18.56% (1570/8460) encounters where both coders assigned multiple diagnosis codes, and 22.66% (1917/8460) encounters with a mixed number of codes assigned.

Of the 4973 encounters with a single code, 62.26% (n=3096) were concordant and 37.74% (n=1877) were discordant. Of the 1877 discordant encounters, there were 68.67% (n=1289) of instances in which 1 coder was concordant with the arbitrator, and 31.33% (n=588) of instances that had a unique diagnosis assigned by the arbitrator which was not assigned by either coder. Therefore, only 11.82% (588/4973) single diagnosis encounters had a diagnosis that was not assigned by either of the initial 2 coders.

Of the 1570 encounters with multiple diagnosis codes, 13.69% (n=215) had complete concordance between the 2 coders, 64.65% (n=1015) were partially concordant, and 21.66% (n=340) were completely discordant. Of the 340 completely discordant encounters, there were 22.94% (n=78) of instances in which 1 coder was concordant with the arbitrator, 61.47% (n=209) of instances in which 1 coder was partially concordant with the arbitrator, and 15.59% (n=53) of instances that had unique diagnosis codes assigned by the arbitrator which were not assigned by either coder. Therefore, only 3.38% (53/1570) of multidiagnosis encounters had a diagnosis that was not assigned by either of the initial 2 coders.

The most frequent diagnoses in concordant clinical encounters were hypertension, acute upper respiratory tract infection, acute nasopharyngitis, viral infection, urinary tract infection, cough, and diabetes mellitus. The most frequent diagnoses in discordant clinical encounters were hypertension, headache, repeat prescription, diabetes mellitus, cough, abdominal pain, and history of diseases of the circulatory system. The frequency of concordance was not significantly different across sex categories: concordance was observed between 38.15% (1985/5203) of encounters with female patients and 39.73% (1294/3257) of encounters with male patients (P=.34). Concordance in diagnosis codes in encounters across age strata ranged from 30.66% to 42.91%. Clinical encounters with older patients (those older than 40 years) had lower concordance between coders than those with other age groups (P<.001). Concordance ranged from 30.43% to 46.05% in encounters across different countries of origin (Table 1).

Encounters with fewer words were more frequently coded in a concordant manner than encounters with more words (Table 1). Specifically, encounters with fewer than 2 words were coded concordantly 75.04% (1572/2095) of the time whereas encounters with greater than 28 words were coded concordantly 13.72% (300/2186) of the time. We found that ICD-10 codes with less specificity (eg, Z00: Encounter for a General Examination Without Abnormal Findings) were more frequently assigned to encounters with 0 or 1 words.

In this analysis of clinical records from ad hoc humanitarian clinics serving displaced persons migrating in Mexico, we developed a system to assign medical diagnosis codes to clinical encounters. This system relied on entry-level medical coders to independently review encounters and then more senior medical coders to arbitrate differences between their assigned ICD-10 codes. Our dataset included 8460 clinical encounters over a 16-month study period, representing 5623 unique patients. We found that the first 2 medical coders demonstrated concordance in more than 60% of their assignments of single- and multidiagnosis encounters. In cases where they differed, additional review by a third, more experienced coder allowed for resolution of most of the discordance. This novel method can be applied to other circumstances where it is necessary to review medical information collected through the course of medical relief operations.

Our analysis suggests that this approach is reliable across most patient demographic categories. As could be expected, encounters with multiple diagnoses were less frequently coded in a completely concordant manner than encounters with single diagnoses. Encounters with older patients in our sample had a lower proportion of their diagnosis codes completely concordant between assignments. This could be indicative of more comorbidities or more complex medical diagnoses in patients who are older. It is also possible that older adults had more frequent multidiagnosis encounters than younger patients, which could produce discrepancies in the diagnosis assigned based on the free-text medical encounter. While many of the same diagnoses were frequently assigned in both concordantly and discordantly coded encounters, some nonspecific diagnoses (eg, abdominal pain and history of disease of circulatory system) were more commonly assigned in discordantly coded encounters. There was also variation observed between the proportion of encounters with concordant diagnosis assignments across patients of different countries of origin; however, these differences were relatively small. It is possible that there are differences in complexity of the clinical documentation between people with different medical conditions and migration experiences. It is also possible that linguistic or communication barriers were present for some patients based on their country of origin, which may have impacted the clarity with which clinicians documented their evaluation.

We also noted a difference in how medical encounters were coded based on the length of free-text documentation in the medical record. Interestingly, clinical encounters with more text were less frequently coded in a concordant manner than encounters with fewer words. This may be because in encounters with few words, the clinicians frequently just stated the diagnosis rather than any of the clinical findings (eg, “pneumonia” or “hypertension”) whereas lengthy documentation in other encounters may describe several findings that could be coded in different ways. It is also likely that clinical encounters with lengthy documentation may systematically differ from less thoroughly documented encounters as the former may include greater complexity in their presentation, disease process, or management considerations.

To address these challenges, future adaptations of this approach should prioritize enhanced training for coders focused on complex cases, particularly those involving multidiagnosis encounters or nonspecific presentations common in low-resource settings. Future adaptations could also incorporate training on culturally specific presentations of diseases and variations in medical terminology across demographic groups [16]. This could help coders better understand and interpret free-text medical records originating from diverse populations. Collaborating with clinicians to improve the clarity and consistency of free-text documentation would further reduce ambiguities and support more accurate assignments, although this may not be feasible in many austere environments. Finally, periodic audits of coding concordance stratified by demographic characteristics could provide valuable feedback to ensure equity in coding across diverse patient populations.

There is growing use of electronic health records among relief organizations serving displaced persons worldwide [11,12,14]. Past reports from humanitarian settings have identified challenges in the implementation of ICD-10 coding into existing health records, including the time required for data entry, staff turnover, and the operational capacity for data analysis [13,17,18]. The approach used in this study can be used to characterize medical diagnoses from electronic health record systems in disaster response and other austere settings with a high level of consistency [15]. The strengths of this approach include the reliability in coding and the ability to have records reviewed and coded remotely while relief operations are ongoing. It also has applicability to diverse geographic and sociopolitical contexts where clinicians work in low-resource environments. It should be noted that we have no way to ascertain diagnosis accuracy (ie, the “true” illnesses and conditions present in the clinical encounters studied). Our method of completing multiple reviews of each encounter presents challenges for efficiency with high-volume clinical operations. Future work may leverage large language models to more efficiently code clinical encounters, but this would require validation before implementation [12]. As with other operations involving electronic health records, patient privacy and data security are also paramount concerns [16].

There are limitations in this analysis and approach. First, given that this analysis was conducted retrospectively, we cannot compare the accuracy of diagnostic codes with the reference standard of clinician designation, nor is this information readily available from clinics in austere environments. In addition, because of the limited diagnostic capabilities in the clinical sites studied, most of the diagnoses were based on the bedside examination (ie, history and examination), which resulted in nonspecific diagnoses being assigned.

In summary, we developed a system to assign ICD-10 codes to thousands of clinical encounters for displaced people in transit at the Mexico-US border, with a variety of clinical presentations for a diverse patient population. The simple methodological approach described in this report holds potential to be adapted to other disaster response and resource-limited settings to advance the provision of clinical care and population health research. Scale-up of the electronic health record technology described in this study can improve patient safety by allowing program managers to monitor quality of care of diagnoses across patient populations. It can also provide critical information to public health officials regarding risk and clusters of transmissible infections, trauma and injuries, and outbreak preparedness.