Phoenix Newsletter - October 2025 President’s Message: Enduring Commitments in a Time of Change Read President’s Message: Enduring Commitments in a Time of Change
Home Research Research Library Natural Language Processing Improves Reliable Identification of COVID-19 Compared to Diagnostic Codes Alone Natural Language Processing Improves Reliable Identification of COVID-19 Compared to Diagnostic Codes Alone 2025 Author(s) Hendrix, Nathaniel, Parikh, Rishi V, Taskier, Madeline, Walter, Grace, Phillips, Robert L, and Rehkopf, David H Topic(s) Role of Primary Care, Achieving Health System Goals, and What Family Physicians Do Keyword(s) Health Information Technology (HIT), Measurement, Population Health, and Quality Of Care Volume American Journal of Epidemiology Source American Journal of Epidemiology Observational COVID-19 studies often rely on diagnostic codes, but their accuracy and potential for differential misclassification across patient subgroups are unclear. In this proof of concept study, we examined age, race, and ethnicity as predictors of differential misclassification by comparing the classification accuracy of diagnostic codes to classifiers based on natural language processing (NLP) of clinical notes. We assessed differential misclassification in two primary care-based samples from the American Family Cohort: first, a cohort of 5000 patients with COVID-19 status assessed by physicians based on notes; and second, 21,659 patients (out of 1,560,564) who received COVID-specific antivirals. Using annotated note data, we trained and tested three NLP classifiers (tree-based, recurrent neural network, and transformer-based). Approximately 63% of likely COVID-19 patients in the two samples had a documented ICD-10 code for COVID-19. Sensitivity was highest among younger patients (68.6% for <18 years versus 60.6% for those 75+), and for Hispanic patients (68.0% versus 58.5% for Black/African American patients). The tree-based classifier had the highest area under the ROC curve (0.92), although it was less accurate among older patients. NLP performance drastically worsened predicting data collected post-training. While NLP may improve cohort identification, frequent retraining is likely needed to capture changing documentation. ABFM Research Read all 2013 Specialty board certification in the United States: issues and evidence Go to Specialty board certification in the United States: issues and evidence 2019 Comparing Comprehensiveness in Primary Care Specialties and Their Effects on Healthcare Costs and Hospitalizations in Medicare Beneficiaries Go to Comparing Comprehensiveness in Primary Care Specialties and Their Effects on Healthcare Costs and Hospitalizations in Medicare Beneficiaries 2019 Patient and Clinician Perceptions of Prediabetes: A Mixed-Methods Primary Care Study Go to Patient and Clinician Perceptions of Prediabetes: A Mixed-Methods Primary Care Study 2019 An Oral History Interview With Leon Gross, PhD Go to An Oral History Interview With Leon Gross, PhD
Author(s) Hendrix, Nathaniel, Parikh, Rishi V, Taskier, Madeline, Walter, Grace, Phillips, Robert L, and Rehkopf, David H Topic(s) Role of Primary Care, Achieving Health System Goals, and What Family Physicians Do Keyword(s) Health Information Technology (HIT), Measurement, Population Health, and Quality Of Care Volume American Journal of Epidemiology Source American Journal of Epidemiology
ABFM Research Read all 2013 Specialty board certification in the United States: issues and evidence Go to Specialty board certification in the United States: issues and evidence 2019 Comparing Comprehensiveness in Primary Care Specialties and Their Effects on Healthcare Costs and Hospitalizations in Medicare Beneficiaries Go to Comparing Comprehensiveness in Primary Care Specialties and Their Effects on Healthcare Costs and Hospitalizations in Medicare Beneficiaries 2019 Patient and Clinician Perceptions of Prediabetes: A Mixed-Methods Primary Care Study Go to Patient and Clinician Perceptions of Prediabetes: A Mixed-Methods Primary Care Study 2019 An Oral History Interview With Leon Gross, PhD Go to An Oral History Interview With Leon Gross, PhD
2013 Specialty board certification in the United States: issues and evidence Go to Specialty board certification in the United States: issues and evidence
2019 Comparing Comprehensiveness in Primary Care Specialties and Their Effects on Healthcare Costs and Hospitalizations in Medicare Beneficiaries Go to Comparing Comprehensiveness in Primary Care Specialties and Their Effects on Healthcare Costs and Hospitalizations in Medicare Beneficiaries
2019 Patient and Clinician Perceptions of Prediabetes: A Mixed-Methods Primary Care Study Go to Patient and Clinician Perceptions of Prediabetes: A Mixed-Methods Primary Care Study
2019 An Oral History Interview With Leon Gross, PhD Go to An Oral History Interview With Leon Gross, PhD