%0 Journal Article
%@ 2369-2960
%I JMIR Publications
%V 8
%N 3
%P e36119
%T Identifying COVID-19 Outbreaks From Contact-Tracing Interview Forms for Public Health Departments: Development of a Natural Language Processing Pipeline
%A Caskey,John
%A McConnell,Iain L
%A Oguss,Madeline
%A Dligach,Dmitriy
%A Kulikoff,Rachel
%A Grogan,Brittany
%A Gibson,Crystal
%A Wimmer,Elizabeth
%A DeSalvo,Traci E
%A Nyakoe-Nyasani,Edwin E
%A Churpek,Matthew M
%A Afshar,Majid
%+ University of Wisconsin–Madison, 1685 Highland Avenue, 5158 Medical Foundation Centennial Building, Madison, WI, 53705, United States, 1 3125459462, majid.afshar@wisc.edu
%K natural language processing
%K public health informatics
%K named entity recognition
%K contact tracing
%K COVID-19
%K outbreaks
%K neural language model
%K disease surveillance
%K digital health
%K electronic surveillance
%K public health
%K digital surveillance tool
%D 2022
%7 8.3.2022
%9 Original Paper
%J JMIR Public Health Surveill
%G English
%X Background: In Wisconsin, COVID-19 case interview forms contain free-text fields that need to be mined to identify potential outbreaks for targeted policy making. We developed an automated pipeline to ingest the free text into a pretrained neural language model to identify businesses and facilities as outbreaks. Objective: We aimed to examine the precision and recall of our natural language processing pipeline against existing outbreaks and potentially new clusters. Methods: Data on cases of COVID-19 were extracted from the Wisconsin Electronic Disease Surveillance System (WEDSS) for Dane County between July 1, 2020, and June 30, 2021. Features from the case interview forms were fed into a Bidirectional Encoder Representations from Transformers (BERT) model that was fine-tuned for named entity recognition (NER). We also developed a novel location-mapping tool to provide addresses for relevant NER. Precision and recall were measured against manually verified outbreaks and valid addresses in WEDSS. Results: There were 46,798 cases of COVID-19, with 4,183,273 total BERT tokens and 15,051 unique tokens. The recall and precision of the NER tool were 0.67 (95% CI 0.66-0.68) and 0.55 (95% CI 0.54-0.57), respectively. For the location-mapping tool, the recall and precision were 0.93 (95% CI 0.92-0.95) and 0.93 (95% CI 0.92-0.95), respectively. Across monthly intervals, the NER tool identified more potential clusters than were verified in WEDSS. Conclusions: We developed a novel pipeline of tools that identified existing outbreaks and novel clusters with associated addresses. Our pipeline ingests data from a statewide database and may be deployed to assist local health departments for targeted interventions. 
%M 35144241
%R 10.2196/36119
%U https://publichealth.jmir.org/2022/3/e36119
%U https://doi.org/10.2196/36119
%U http://www.ncbi.nlm.nih.gov/pubmed/35144241