The reliability of American Board of Family Medicine examinations: implications for test takers

A common theme among family physicians who have performed poorly repeatedly on the American Board of Family Medicine (ABFM) Maintenance of Certification (MC-FP) examination is the complaint that they received a score that was identical, or almost identical, to their score on a previous administration of the examination. From their perspective, why they received the exact same score (or a very similar score), despite additional study time and preparation, is a mystery. Often, physicians assume a mix-up has occurred and ask if it is possible that results have been provided erroneously from their previous attempt. After a psychometric review, it is clear that there is no mistake at all. In fact, we anticipate many test takers will receive a comparable score on future successful attempts at taking the examination. We base this anticipation on the psychometric concept of reliability.