Diplomate Spotlight Opening Doors with Board Certification: A Conversation with Long Standing Diplomate Joseph Cook Read Opening Doors with Board Certification: A Conversation with Long Standing Diplomate Joseph Cook
Phoenix Newsletter - July 2025 Available Now: 2026 5-Year Cycle Registration Read Available Now: 2026 5-Year Cycle Registration
Home Research Research Library Natural Language Processing Improves Reliable Identification of COVID-19 Compared to Diagnostic Codes Alone Natural Language Processing Improves Reliable Identification of COVID-19 Compared to Diagnostic Codes Alone 2025 Author(s) Hendrix, Nathaniel, Parikh, Rishi V, Taskier, Madeline, Walter, Grace, Phillips, Robert L, and Rehkopf, David H Topic(s) Role of Primary Care, Achieving Health System Goals, and What Family Physicians Do Keyword(s) Health Information Technology (HIT), Measurement, Population Health, and Quality Of Care Volume American Journal of Epidemiology Source American Journal of Epidemiology Observational COVID-19 studies often rely on diagnostic codes, but their accuracy and potential for differential misclassification across patient subgroups are unclear. In this proof of concept study, we examined age, race, and ethnicity as predictors of differential misclassification by comparing the classification accuracy of diagnostic codes to classifiers based on natural language processing (NLP) of clinical notes. We assessed differential misclassification in two primary care-based samples from the American Family Cohort: first, a cohort of 5000 patients with COVID-19 status assessed by physicians based on notes; and second, 21,659 patients (out of 1,560,564) who received COVID-specific antivirals. Using annotated note data, we trained and tested three NLP classifiers (tree-based, recurrent neural network, and transformer-based). Approximately 63% of likely COVID-19 patients in the two samples had a documented ICD-10 code for COVID-19. Sensitivity was highest among younger patients (68.6% for <18 years versus 60.6% for those 75+), and for Hispanic patients (68.0% versus 58.5% for Black/African American patients). The tree-based classifier had the highest area under the ROC curve (0.92), although it was less accurate among older patients. NLP performance drastically worsened predicting data collected post-training. While NLP may improve cohort identification, frequent retraining is likely needed to capture changing documentation. ABFM Research Read all 2013 Most family physicians work routinely with nurse practitioners, physician assistants, or certified nurse midwives Go to Most family physicians work routinely with nurse practitioners, physician assistants, or certified nurse midwives 2017 Intention Versus Reality: Family Medicine Residency Graduates’ Intention to Practice Obstetrics Go to Intention Versus Reality: Family Medicine Residency Graduates’ Intention to Practice Obstetrics 2022 Competencies for the Use of Artificial Intelligence in Primary Care Go to Competencies for the Use of Artificial Intelligence in Primary Care 2023 Trends of Geriatric Certification and Practice Patterns of Family Physicians: 1988-2019 Go to Trends of Geriatric Certification and Practice Patterns of Family Physicians: 1988-2019
Author(s) Hendrix, Nathaniel, Parikh, Rishi V, Taskier, Madeline, Walter, Grace, Phillips, Robert L, and Rehkopf, David H Topic(s) Role of Primary Care, Achieving Health System Goals, and What Family Physicians Do Keyword(s) Health Information Technology (HIT), Measurement, Population Health, and Quality Of Care Volume American Journal of Epidemiology Source American Journal of Epidemiology
ABFM Research Read all 2013 Most family physicians work routinely with nurse practitioners, physician assistants, or certified nurse midwives Go to Most family physicians work routinely with nurse practitioners, physician assistants, or certified nurse midwives 2017 Intention Versus Reality: Family Medicine Residency Graduates’ Intention to Practice Obstetrics Go to Intention Versus Reality: Family Medicine Residency Graduates’ Intention to Practice Obstetrics 2022 Competencies for the Use of Artificial Intelligence in Primary Care Go to Competencies for the Use of Artificial Intelligence in Primary Care 2023 Trends of Geriatric Certification and Practice Patterns of Family Physicians: 1988-2019 Go to Trends of Geriatric Certification and Practice Patterns of Family Physicians: 1988-2019
2013 Most family physicians work routinely with nurse practitioners, physician assistants, or certified nurse midwives Go to Most family physicians work routinely with nurse practitioners, physician assistants, or certified nurse midwives
2017 Intention Versus Reality: Family Medicine Residency Graduates’ Intention to Practice Obstetrics Go to Intention Versus Reality: Family Medicine Residency Graduates’ Intention to Practice Obstetrics
2022 Competencies for the Use of Artificial Intelligence in Primary Care Go to Competencies for the Use of Artificial Intelligence in Primary Care
2023 Trends of Geriatric Certification and Practice Patterns of Family Physicians: 1988-2019 Go to Trends of Geriatric Certification and Practice Patterns of Family Physicians: 1988-2019