research Performance Evaluation of the Generative Pre-trained Transformer (GPT-4) on the Family Medicine In-Training Examination Read Performance Evaluation of the Generative Pre-trained Transformer (GPT-4) on the Family Medicine In-Training Examination
post Perceiving Structures: Social Medicine Immersion in the Bronx Read Perceiving Structures: Social Medicine Immersion in the Bronx
post International Advocacy Takes Center Stage in JABFM Read International Advocacy Takes Center Stage in JABFM
Home Research Featured Work Conceptual Foundations Research The Evidence Behind Continuing Certification The Conceptual Foundations study synthesized findings from over 600 articles to describe the foundational evidence behind the need for continuous certification. View Infographic At A Glance The whitepaper revealed four central themes: cognitive skills need to be kept current; self-assessment is not enough; testing enhances learning and retention, and goals and consequences motivate behavior. Researchers from the University of Pittsburgh, who led this study, have published multiple articles discussing the science behind the change from a one-day exam to longitudinal assessment, the likely outcomes, as well as other insights. Learn More Below 1: Cognitive Skills Need to Be Kept Current Over the course of their professional careers, physicians develop significant knowledge and expertise. The existing literature clearly supports the need for individuals to engage in appropriate study and assessment to keep these cognitive skills current: The decay of cognitive skills can manifest itself in the form of lower quality care as physicians get further from formal training. Knowledge or skills gained may not remain accessible to physicians for a number of reasons, including an absence of study and self-assessment. Cognitive decline can also interfere with a task fundamental to clinical care – the ability to retrieve knowledge fast enough to be useful. Clinicians face standards of care that continuously evolve over time, and it is a considerable challenge to keep up with these changes. Longitudinal assessment serves as a method of evaluation. It also may promote learning and retention serving to maintain existing levels of knowledge and increasing awareness of new standards of care. 2: Self-Assessment Is Not Enough Physician learners have asked why cognitive skills cannot be kept current through self-assessment and self-directed remediation in areas of weakness. Although the existing literature shows that learners have some ability to self-assess strengths and weaknesses, considerable systematic biases in self-appraisal exist: Learners across disciplines tend to overestimate how much they will remember after learning. Poor performers are often unaware of their poor performance or capacity. Individuals frequently employ ineffective learning strategies, prioritizing activities that are more enjoyable (e.g., interesting subject matter or familiar topics) over more pertinent learning activities directed at areas of weakness. Evidence from both basic and applied science on the difficulty of accurately measuring one’s own knowledge and cognitive skills suggest that physicians should use objective evidence to guide their choice of learning activities in order improve care. 3: Testing Enhances Learning and Retention Meta–analysis studies provide robust evidence that testing is a powerful device for learning and retaining cognitive skills, and that taking a test serves as a strong learning experience in its own right because it allows learners to practice retrieval from memory. The theory that testing aids learning has been demonstrated across numerous domains, including medicine. Testing is especially helpful when information must be retained and retrievable for long periods of time, as is the case for physicians. The effects of testing are boosted by spacing tests over time, and that testing benefits not only rote memorization, but also complex thinking, such as diagnostic classification and clinical reasoning. The learning benefits of testing can transfer to related material that was not studied, demonstrating value beyond the narrowly-learned subject matter that is directly assessed. Feedback, specifically providing and explaining correct answers, makes learning more effective and, when applicable, should be provided to learners. Contrary to some lay concerns, making an error on a test has not been shown to be harmful to long-term retention, as long as feedback on why the answer is wrong is provided. 4: Goals and Consequences Motivate Assessment can also serve as an important motivator: Physicians learn and retain more when they expect to be tested, thus testing should be difficult enough to engender deeper and more effective learning. Intrinsically motivated individuals to work harder and persist longer in the face of difficulty, adopt better learning strategies, and procrastinate less. People learn and perform better when motivated by their own values and interests rather than strictly external rewards. Physicians tend to be more motivated to study when there are clear consequences for not doing so. There is reason to believe that an external framework offered by a tailored longitudinal assessment program may aid in these aspects, given evidence suggesting that physicians are highly motivated individuals. VIEW INFOGRAPHIC Conceptual Foundations Publications Researchers from the University of Pittsburgh Department of Psychology led the Conceptual Foundations project and have published a series of articles that cover the science behind Maintenance of Certification (with more to come). Expand All Collapse All Cognitive perspectives on maintaining physician’s medical expertise: I. Reimagining Maintenance of Certification to promote lifelong learning Rottman, B. M., Caddick, Z. A., Nokes-Malach, T. J., & Fraundorf, S. H. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: I. Reimagining maintenance of certification to promote lifelong learning. Cognitive Research: Principles & Implications, 8, 46. Read the paper Cognitive perspectives on maintaining physicians’ medical expertise: II. Acquiring, maintaining, and updating cognitive skills Caddick, Z. A., Fraundorf, S. H., Rottman; B. M., & Nokes-Malach, T. J. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: II. Acquiring, maintaining, and updating cognitive skills. Cognitive Research: Principles & Implications, 8, 47. Read the paper Cognitive perspectives on maintaining physicians’ medical expertise: III. Strengths and weaknesses of self-assessment Fraundorf, S.H., Caddick, Z.A., Nokes-Malach T.J., & Rottman, B.M. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: III. Strengths and weaknesses of self-assessment. Cognitive Research: Principles & Implications, 8, 58. Read the paper Cognitive perspectives on maintaining physicians’ medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention Fraundorf, S.H., Caddick, Z.A., Nokes-Malach, T.J., Rottman, B.M. (2023) Cognitive perspectives on maintain physicians’ medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention, 8, 53. Read the paper Cognitive perspectives on maintaining physicians medical expertise: V. Using a motivational framework to understand the benefits and cost of testing Nokes-Malach, T. J., Fraundorf, S. H., Caddick, Z. A., & Rottman; B. M. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: V. Using a motivational framework to understand the benefits and costs of testing. Cognitive Research: Principles & Implications, 8, 64. Read the paper Read the Whitepaper Conceptual Foundations for Designing Continuing Certification Assessments for Physicians Conceptual Foundations
At A Glance The whitepaper revealed four central themes: cognitive skills need to be kept current; self-assessment is not enough; testing enhances learning and retention, and goals and consequences motivate behavior. Researchers from the University of Pittsburgh, who led this study, have published multiple articles discussing the science behind the change from a one-day exam to longitudinal assessment, the likely outcomes, as well as other insights. Learn More Below
Conceptual Foundations Publications Researchers from the University of Pittsburgh Department of Psychology led the Conceptual Foundations project and have published a series of articles that cover the science behind Maintenance of Certification (with more to come). Expand All Collapse All Cognitive perspectives on maintaining physician’s medical expertise: I. Reimagining Maintenance of Certification to promote lifelong learning Rottman, B. M., Caddick, Z. A., Nokes-Malach, T. J., & Fraundorf, S. H. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: I. Reimagining maintenance of certification to promote lifelong learning. Cognitive Research: Principles & Implications, 8, 46. Read the paper Cognitive perspectives on maintaining physicians’ medical expertise: II. Acquiring, maintaining, and updating cognitive skills Caddick, Z. A., Fraundorf, S. H., Rottman; B. M., & Nokes-Malach, T. J. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: II. Acquiring, maintaining, and updating cognitive skills. Cognitive Research: Principles & Implications, 8, 47. Read the paper Cognitive perspectives on maintaining physicians’ medical expertise: III. Strengths and weaknesses of self-assessment Fraundorf, S.H., Caddick, Z.A., Nokes-Malach T.J., & Rottman, B.M. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: III. Strengths and weaknesses of self-assessment. Cognitive Research: Principles & Implications, 8, 58. Read the paper Cognitive perspectives on maintaining physicians’ medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention Fraundorf, S.H., Caddick, Z.A., Nokes-Malach, T.J., Rottman, B.M. (2023) Cognitive perspectives on maintain physicians’ medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention, 8, 53. Read the paper Cognitive perspectives on maintaining physicians medical expertise: V. Using a motivational framework to understand the benefits and cost of testing Nokes-Malach, T. J., Fraundorf, S. H., Caddick, Z. A., & Rottman; B. M. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: V. Using a motivational framework to understand the benefits and costs of testing. Cognitive Research: Principles & Implications, 8, 64. Read the paper
Cognitive perspectives on maintaining physician’s medical expertise: I. Reimagining Maintenance of Certification to promote lifelong learning Rottman, B. M., Caddick, Z. A., Nokes-Malach, T. J., & Fraundorf, S. H. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: I. Reimagining maintenance of certification to promote lifelong learning. Cognitive Research: Principles & Implications, 8, 46. Read the paper
Cognitive perspectives on maintaining physicians’ medical expertise: II. Acquiring, maintaining, and updating cognitive skills Caddick, Z. A., Fraundorf, S. H., Rottman; B. M., & Nokes-Malach, T. J. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: II. Acquiring, maintaining, and updating cognitive skills. Cognitive Research: Principles & Implications, 8, 47. Read the paper
Cognitive perspectives on maintaining physicians’ medical expertise: III. Strengths and weaknesses of self-assessment Fraundorf, S.H., Caddick, Z.A., Nokes-Malach T.J., & Rottman, B.M. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: III. Strengths and weaknesses of self-assessment. Cognitive Research: Principles & Implications, 8, 58. Read the paper
Cognitive perspectives on maintaining physicians’ medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention Fraundorf, S.H., Caddick, Z.A., Nokes-Malach, T.J., Rottman, B.M. (2023) Cognitive perspectives on maintain physicians’ medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention, 8, 53. Read the paper
Cognitive perspectives on maintaining physicians medical expertise: V. Using a motivational framework to understand the benefits and cost of testing Nokes-Malach, T. J., Fraundorf, S. H., Caddick, Z. A., & Rottman; B. M. (2023). Cognitive perspectives on maintaining physicians’ medical expertise: V. Using a motivational framework to understand the benefits and costs of testing. Cognitive Research: Principles & Implications, 8, 64. Read the paper
Read the Whitepaper Conceptual Foundations for Designing Continuing Certification Assessments for Physicians Conceptual Foundations