Understanding the “sum of subtest to overall score discrepancy” on the MC-FP examination

When high-stakes examinations, such as the American Board of Family Medicine’s (ABFM) Maintenance of Certifi cation-in Family Practice (MC-FP) examination are administered, candidates and diplomates are keenly interested in the accuracy of their test scores, especially when their scores are close to, but below, the pass/fail cut point. In some instances, candidates will attempt to reverse engineer their scores using the information provided on the score report in an effort to verify the “weighted sum of the subtest scores” is congruent with the overall test score. Any discrepancy might become alarming to the candidate, providing a seemingly legitimate reason to believe the overall score was inaccurate, thus prompting a phone call to the ABFM for further investigation and clarifi cation. Historically, such a mistake in scoring has never been found; however, a statistical phenomenon that we will describe below could make it appear so. We would like to explain this phenomenon so that examinees who attempt to reverse engineer their score reports will better understand the “sum of subtest to overall score discrepancy” phenomenon.