The Consequential Validity of ABFM Examinations

Measurement scholar, Samuel Messick, defines validity as “an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores….”1 (p13) Messick’s definition of validity differed from previous validity theorists in that he acknowledged test scores often affect social policy, and thus argued social consequences should be examined. Messick referred to this form of validity as “consequential validity.” Shepard2,3 further clarified social consequences to include both the positive/negative and intended/unintended consequences that may result from score-based inferences. The purpose of this article is to discuss consequential validity as it pertains to American Board of Family Medicine (ABFM) examinations.