Stages of Milestones Implementation: A Template Analysis of 16 Programs Across 4 Specialties

ABSTRACT Background: Since 2013, US residency programs have used the competency-based framework of the Milestones to report resident progress and to provide feedback to residents. The implementation of Milestones-based assessments, clinical competency committee (CCC) meetings, and processes for providing feedback varies among programs and warrants systematic examination across specialties. Objective: We sought to determine how varying assessment, CCC, and feedback implementation strategies result in different outcomes in resource expenditure and stakeholder engagement, and to explore the contextual forces that moderate these outcomes. Methods: From 2017 to 2018, interviews were conducted of program directors, CCC chairs, and residents in emergency medicine (EM), internal medicine (IM), pediatrics, and family medicine (FM), querying their experiences with Milestone processes in their respective programs. Interview transcripts were coded using template analysis, with the initial template derived from previous research. The research team conducted iterative consensus meetings to ensure that the evolving template accurately represented phenomena described by interviewees. Results: Forty-four individuals were interviewed across 16 programs (5 EM, 4 IM, 5 pediatrics, 3 FM). We identified 3 stages of Milestone-process implementation, including a resource-intensive early stage, an increasingly efficient transition stage, and a final stage for fine-tuning. Conclusions: Residency program leaders can use these findings to place their programs along an implementation continuum and gain an understanding of the strategies that have enabled their peers to progress to improved efficiency and increased resident and faculty engagement.