Associations Between Medical Education Assessments and American Board of Family Medicine Certification Examination Score and Failure to Obtain Certification

Purpose Family medicine residency programs can be cited for low pass or take rates on the American Board of Family Medicine (ABFM) certification examination, and the relationships among standardized medical education assessments and performance on board certification examinations and eventual board certification have not been comprehensively studied. The objective of this study was to evaluate the associations of all required standardized examinations in medical education with ABFM certification examination scores and eventual ABFM certification. Method All graduates of U.S. MD-granting family medicine residency programs from 2008 to 2012 were included. Data on ABFM certification examination score, ABFM certification status (as of December 31, 2014), Medical College Admission Test (MCAT) section scores, undergraduate grade point average, all United States Medical Licensing Examination (USMLE) Step scores, and all ABFM in-training examination scores were linked. Nested logistic and linear regression models, controlling for clustering by residency program, determined associations between assessments and both certification examination scores and board certification status. As many international medical graduates (IMGs) do not take the MCAT, separate models for U.S. medical graduates (USMG) and IMGs were run. Results The study sample was 15,902 family medicine graduates, of whom 92.1% (14,648/15,902) obtained board certification. In models for both IMGs and USMGs, the addition of more recent assessments weakened the associations of earlier assessments. USMLE Step 2 Clinical Knowledge was predictive of certification examination scores and certification status in all models in which it was included. Conclusions For family medicine residents, more recent assessments generally have stronger associations with board certification score and status than earlier assessments. Solely using medical school admissions (grade point average and MCAT) and licensure (USMLE) scores for resident selection may not adequately predict ultimate board certification. Throughout the course of medical education and training, aspiring physicians’ application of knowledge is tested multiple times. The Medical College Admission Test (MCAT) is required for admission to all U.S. MD-granting medical schools. Undergraduate grade point average (GPA), both overall and in the sciences, is also crucial for medical school admission. Following medical school matriculation, students must pass 3 successive steps of the United States Medical Licensing Examination (USMLE) to obtain a full medical license. Step 1 is often completed before beginning the third year of medical school, and the 2 parts of Step 2 are often completed in the fourth year. For international medical graduates (IMGs), passing Step 1 and both parts of Step 2 are requirements for certification by the Educational Commission for Foreign Medical Graduates (ECFMG), which is necessary for entrance into Accreditation Council for Graduate Medical Education (ACGME)-accredited residency programs. Step 3 is usually taken early in residency but can be completed any time before independent practice. These scores, except Step 3 scores, form the core of the objective standardized assessments available to residency directors when making decisions regarding whom to accept into their programs. Despite these examinations being required components of the medical education landscape for decades, it remains unknown which are the best predictors of board certification examination performance and eventual board certification. Family medicine residency program directors are especially concerned about their potential residents’ ability to pass the American Board of Family Medicine (ABFM) Family Medicine Certification Examination as, during the time of our study (2008–2012), the ACGME required that family medicine residencies have a rolling 5-year 90% ABFM certification examination pass rate and a rolling 5-year 95% take rate.1 Having a low certification examination pass rate is one of the most common citations received by family medicine residencies2 and may be seen as a marker of low-quality training.

Read More