Conceptual Foundations: The Evidence Behind Continuing Certification

A recently published study synthesized findings from over 600 articles to describe the foundational evidence behind the need for continuous certification.

At A Glance

The whitepaper revealed four central themes: cognitive skills need to be kept current; self-assessment is not enough; testing enhances learning and retention, and goals and consequences motivate behavior. Researchers from the University of Pittsburgh, who led this study, have published multiple articles discussing the science behind the change from a one-day exam to longitudinal assessment, the likely outcomes, as well as other insights.

Cognitive Skills Need to be Kept Current

Over the course of training, physicians develop significant knowledge and expertise. However, existing literature clearly supports the need for individuals to engage in appropriate training and study to keep these cognitive skills current. Equally, it reinforces the idea that, in the absence of such training and study, cognitive skills will decline over time. Specifically, it supports the idea that decay of cognitive skills can manifest itself in the form of lower quality care as physicians get further from residency training. Evidence also suggests that knowledge or skills gained may not remain accessible to physicians for a number of reasons. These include an absence of study, and also the presence of other similar knowledge or skills that compete in what is brought to mind through a process called interference. Cognitive decline can also interfere with a task fundamental to clinical care, the ability to retrieve knowledge fast enough to be useful. Clinicians also face standards of care that continuously evolve over time and it is a considerable challenge to keep up with these changes. As such, evidence supports the idea that longitudinal assessment has the potential to serve not only an evaluation mechanism but also to promote learning and retention serving to maintain existing levels of knowledge and increasing awareness of new standards of care.

Self-Assessment Is Not Enough

Physician learners have asked why cognitive skills cannot be kept current through self-assessment and self-directed remediation in areas of weakness. Although the existing literature shows that learners have some ability to self-assess strengths and weaknesses, considerable systematic biases in self-appraisal exist. Learners across disciplines tend to overestimate how much they will remember after learning. Of greater concern, is that it is the relatively poor performers in a domain that are particularly unaware of their poor performance or capacity. Furthermore, individuals frequently employ ineffective learning strategies, prioritizing activities that are more enjoyable (e.g., interesting subject matter or easier topics) over more pertinent learning activities directed at areas of weakness. Evidence from both basic and applied science on the difficulty of accurately measuring one’s own knowledge and cognitive skills suggest that physicians should be at least partly guided in their choices of learning 6 activities and affirms the potential peril to public stakeholders if learners retain absolute control over the subject matter studied to maintain cognitive skills.

Testing Enhances Learning and Retention

Meta–analysis studies provide robust evidence that testing is a powerful device for learning and retaining cognitive skills. Meta-analysis also provides robust evidence that taking a test serves as a strong learning experience in its own right because it allows learners to practice retrieval from memory. That testing aids learning has been demonstrated across numerous domains, including medicine. Testing is especially helpful when information must be retained and retrievable for long periods of time, as is the case for physicians. Evidence suggests that the effects of testing are boosted by spacing tests over time, and that testing benefits not only rote memorization, but also complex thinking, such as diagnostic classification and clinical reasoning. Additionally, the learning benefits of testing can transfer to related material that was not studied, demonstrating value beyond the narrowly-learned subject matter that is directly assessed. Feedback, specifically providing and explaining correct answers, makes learning more effective and, when applicable, should be provided to learners. Contrary to some lay concerns, making an error on a test has not been shown to be harmful to long-term retention, as long as feedback on why the answer is wrong is provided.

Goals and Consequences Motivate

Assessment can also serve as an important motivator. Physicians will be more motivated to study and practice their skills when there are clear consequences (i.e., benefit) for doing so and clear costs for not doing so. Physicians learn and retain more when they expect to be tested, thus testing should be difficult enough to engender deeper and more effective learning. Emphasizing how maintenance of medical expertise aligns with physicians’ values (e.g., their best intentions in treating patients), and offering some degree of autonomy/control over learning content may foster intrinsic motivation within physicians. Research supports the tendency for intrinsically motivated individuals to work harder and persist longer in the face of difficulty, adopt better learning strategies, and procrastinate less. It additionally verifies that people learn and perform better when motivated by their own values and interests rather than strictly external rewards. There is reason to believe that an external framework offered by a tailored longitudinal assessment program may aid in these aspects, given evidence suggesting that physicians are highly motivated individuals.

Conceptual Foundations Publications

Researchers from the University of Pittsburgh Department of Psychology led the Conceptual Foundations project and have published a series of articles that cover the science behind Maintenance of Certification (with more to come).

Read the whitepaper

Conceptual Foundations for Designing Continuing Certification Assessments for Physicians

Conceptual Foundations