Discriminatory ability of milestones: An analysis of milestone variability by obstetrics and gynecology subspecialty

Background: Little evidence exists regarding Accreditation Council for Graduate Medical
Education (ACGME) milestone discriminatory ability. This short report describes variability in
milestone scores by category to determine their utility in discerning high and low performers in
a single Obstetrics and Gynecology residency.
Methods: A Clinical Competency Committee (CCC) design was implemented with four
subcommittees, each responsible for a predetermined milestones subset: Obstetrics,
Gynecology, Ambulatory Practice, and Professional Activities. Milestone scores for 44 residents
per year over four biannual evaluation cycles (2014-2016) were evaluated, for a total of 176
independent evaluations.
Results: Findings indicate that discriminatory ability, assessed by variability between resident
scores, differed by subcommittee. Subcommittees that were primarily tasked with evaluating
clinical- and procedural-based milestones demonstrated lower discriminatory ability among
trainees.
Conclusion: Greater Professional Activity milestone variability indicates better differentiation;
future research should determine correlation of these findings with other professionalism
performance metrics and novel intervention strategies.