The reliability of ABFM examinations: implications for test-takers

A common theme among family physicians that have repeatedly performed poorly on the ABFM Maintenance of Certification (MC-FP) Examination is the complaint that they received a score that was identical, or almost identical to their score on a previous administration of the exam. From their perspective, it is a mystery as to why they received the exact same score (or a very similar score), despite additional study time and preparation. Often, physicians assume a mixup has occurred and ask if it is possible that results have erroneously been provided from their previous attempt. After a psychometric review, it is clear that there is no mistake at all. In fact, we anticipate many test-takers will receive a comparable score on future attempts at successfully taking the exam. We base this anticipation on the psychometric concept of reliability.