Original Research Spotlight: Why do we still need physicians when powerful AI tools are available?

A recent American Board of Family Medicine (ABFM) study in the Journal of the American Board of Family Medicine (JABFM), Performance Evaluation of the Generative Pre-trained Transformer (GPT-4) on the Family Medicine In-Training Examination, compared the performance of GPT-4  on the 2022 ABFM In-Training Exam to its predecessor, GPT 3.5, and the national cohort of family residents. Researchers utilized both quantitative and qualitative analysis.

Both GPT-3.5 and GPT-4 exhibited response patterns analogous to those of medical residents, particularly in relation to question difficulty. GPT-4 showed significant improvements over GPT-3.5, with a 28% (84% vs 56%) increase in accuracy; however, the study highlights the essential role of physicians’ critical thinking and lifelong learning skills.

Check out more of ABFM’s latest research below and in our research library, or follow us on LinkedIn where we regularly highlight research relevant to family medicine. ABFM conducts research devoted to creating, evaluating, and maintaining cutting edge certification methods and to advancing the scientific basis of family medicine.

The ABFM studies listed below offer important findings that can inform policy and practice for the betterment of family medicine, including novel efforts to rank usability of electronic health records, change total investment in primary care, explore 25% declines in family physicians’ self-reported panel size, and demonstrate the need to add 57,000 primary care clinicians above the current baseline to meet population demand.

Achieving Health System Goals

• Measuring Primary Healthcare Spending
• Impact of Response Bias in Three Surveys on Primary Care Providers’ Experiences with Electronic Health Records
• Small Independent Primary Care Practices Serving Socially Vulnerable Urban Populations
• Policy Briefs With Personality: How to Innovatively Disseminate Evidence for Advocacy
• Electronic Health Record Usability, Satisfaction, and Burnout for Family Physicians
• US Primary Care Workforce Growth: A Decade of Limited Progress, and Projected Needs Through 2040
• Only One Quarter of Family Physicians Are Very Satisfied with Their Electronic Health Records Platform

Education and Training

• National Family Medicine Residency Outcomes Project Methodology
• Prevalence and Predictors of Burnout Among Resident Family Physicians
• Setting the Target: Comparing Family Medicine Among US Allopathic Target Schools
• How Early Career Family Medicine Women Physicians Negotiate Their First Job After Residency

Family Medicine Certification

• Certifying Boards Can Provide Knowledge that Shapes Policy
• Performance Evaluation of the Generative Pre-trained Transformer (GPT-4) on the Family Medicine In-Training Examination
• The Effect of Spaced Repetition on Learning and Knowledge Transfer in a Large Cohort of Practicing Physicians
• Sexual Misconduct by Board Certified Family Physicians

Role of Primary Care

• Celebrating 25 Years of High-Quality Family Medicine and Primary Care Policy Research
• Insights From a New National Academies Report on Caregiving
• The Disproportionate Impact of Primary Care Disruption and Telehealth Utilization During COVID-19
• Post-COVID Conditions in US Primary Care: A PRIME Registry Comparison of Patients With COVID-19, Influenza-Like Illness, and Wellness Visits

What Family Physicians Do

• Self-Reported Panel Size Among Family Physicians Declined by Over 25% Over a Decade (2013-2022).
• Underlying Reasons for Primary Care Visits Where Chlamydia Testing Was Performed in the United States, 2019-2022
• Insights From a New National Academies Report on Caregiving