Diplomate Spotlight Opening Doors with Board Certification: A Conversation with Long Standing Diplomate Joseph Cook Read Opening Doors with Board Certification: A Conversation with Long Standing Diplomate Joseph Cook
Phoenix Newsletter - July 2025 Available Now: 2026 5-Year Cycle Registration Read Available Now: 2026 5-Year Cycle Registration
Home Research Research Library Performance on the American Board of Family Medicine (ABFM) certification examination: are superior test-taking skills alone sufficient to pass? Performance on the American Board of Family Medicine (ABFM) certification examination: are superior test-taking skills alone sufficient to pass? 2011 Author(s) O’Neill, Thomas R, Royal, Kenneth D, and Puffer, James C Topic(s) Family Medicine Certification Keyword(s) Psychometrics Volume Journal of the American Board of Family Medicine Source Journal of the American Board of Family Medicine INTRODUCTION: Certification examinations used by American specialty boards have been the sine qua non for demonstrating the knowledge sufficient for attainment of board certification in the United States for more than 75 years. Some people contend that the examination is predominantly a test of superior test-taking skills rather than of family medicine decision-making ability. In an effort to explore the validity of this assertion, we administered the American Board of Family Medicine (ABFM) Certification to examinees who had demonstrated proficiency in taking standardized tests but had limited medical knowledge. METHODS: Four nonphysician experts in the field of measurement and testing were administered one version of the 2009 ABFM certification examination. Scaled scores were calculated for each examinee, and psychometric analyses were performed on the examinees responses to examination items and compared with the performance of physicians who took the same examination. RESULTS: The minimum passing threshold for the examination was a scaled score of 390, corresponding to 57.7% to 61.0% of questions answered correctly, depending on the version of the examination. The 4 nonphysician examinees performed poorly, with scaled scores that ranged from 20 to 160 (mean, 87.5; SD, 57.4). The number of questions answered correctly ranged from 24.0% to 35.1% (mean, 29.2%; SD, 0.05%). Rasch analyses of the examination items revealed that the nonphysician examinees were more likely to use guessing strategies in an effort to answer questions correctly. Distracter analysis suggest near-complete randomness in the nonphysician responses. CONCLUSIONS: Though all 4 nonphysician examinees performed better than would have been predicted by chance alone, none performed well enough to even fall within 8 SE below the passing thresholds; their performance was far below that of almost all physicians who completed the examination. Given that the nonphysicians relied heavily on the identifying cues in the phrasing of items and the manner in which response options were presented, the results affirm the notion that the ABFM certification examination is not primarily a measure of generic test-taking ability but measures information critical to the estimation of a family physician’s knowledge sufficient for certification. Item analysis confirmed that items were well written, provided minimal cueing, and required medical knowledge to answer correctly. ABFM Research Read all 2015 Family Physicians with a Certificate of Added Qualifications (CAQs) in Sports Medicine Spend the Majority of Their Time Practicing Sports Medicine Go to Family Physicians with a Certificate of Added Qualifications (CAQs) in Sports Medicine Spend the Majority of Their Time Practicing Sports Medicine 2020 Helping Family Physicians Keep Up to Date: A Next Step in the Pursuit of Mastery Go to Helping Family Physicians Keep Up to Date: A Next Step in the Pursuit of Mastery 2022 Family Physician Racial Identity: An Analysis of “Other” Race Selection and Implications for Future Data Collection Go to Family Physician Racial Identity: An Analysis of “Other” Race Selection and Implications for Future Data Collection 2020 Shaping Keystones in a Time of Transformation: ABFM’s Efforts to Advance Leadership & Scholarship in Family Medicine Go to Shaping Keystones in a Time of Transformation: ABFM’s Efforts to Advance Leadership & Scholarship in Family Medicine
Author(s) O’Neill, Thomas R, Royal, Kenneth D, and Puffer, James C Topic(s) Family Medicine Certification Keyword(s) Psychometrics Volume Journal of the American Board of Family Medicine Source Journal of the American Board of Family Medicine
ABFM Research Read all 2015 Family Physicians with a Certificate of Added Qualifications (CAQs) in Sports Medicine Spend the Majority of Their Time Practicing Sports Medicine Go to Family Physicians with a Certificate of Added Qualifications (CAQs) in Sports Medicine Spend the Majority of Their Time Practicing Sports Medicine 2020 Helping Family Physicians Keep Up to Date: A Next Step in the Pursuit of Mastery Go to Helping Family Physicians Keep Up to Date: A Next Step in the Pursuit of Mastery 2022 Family Physician Racial Identity: An Analysis of “Other” Race Selection and Implications for Future Data Collection Go to Family Physician Racial Identity: An Analysis of “Other” Race Selection and Implications for Future Data Collection 2020 Shaping Keystones in a Time of Transformation: ABFM’s Efforts to Advance Leadership & Scholarship in Family Medicine Go to Shaping Keystones in a Time of Transformation: ABFM’s Efforts to Advance Leadership & Scholarship in Family Medicine
2015 Family Physicians with a Certificate of Added Qualifications (CAQs) in Sports Medicine Spend the Majority of Their Time Practicing Sports Medicine Go to Family Physicians with a Certificate of Added Qualifications (CAQs) in Sports Medicine Spend the Majority of Their Time Practicing Sports Medicine
2020 Helping Family Physicians Keep Up to Date: A Next Step in the Pursuit of Mastery Go to Helping Family Physicians Keep Up to Date: A Next Step in the Pursuit of Mastery
2022 Family Physician Racial Identity: An Analysis of “Other” Race Selection and Implications for Future Data Collection Go to Family Physician Racial Identity: An Analysis of “Other” Race Selection and Implications for Future Data Collection
2020 Shaping Keystones in a Time of Transformation: ABFM’s Efforts to Advance Leadership & Scholarship in Family Medicine Go to Shaping Keystones in a Time of Transformation: ABFM’s Efforts to Advance Leadership & Scholarship in Family Medicine