On February 12, 2020, the organization behind the United States Medical Licensing Examination, or USMLE, announced that within the next two years, they will stop reporting the results of the Step 1 exam as a 3-digit score. Instead, examinees will simply receive a mark of pass or fail.
Medical students currently take Step 1 about midway through their education (traditionally after the first two years of preclinical study, though many schools have switched to having their students sit the exam after three years instead). Discussions about a potential change of this sort have been ongoing for years, both within the NBME and the FSMB (which jointly administer the test), and within the medical community at large. It comes as little surprise, therefore, that the decision has finally been announced.
In their justification for the change, the NBME/FSMB placed a good deal of emphasis on the negative impact that reporting Step 1 scores has on the wellbeing of the students taking the exams. This is, speaking from experience, a worthy place for emphasis; I took Step 1 less than two years ago, and I can attest to the stress inherent in taking a test for which the results can have such a large impact on one’s career. However, I worry that focusing exclusively on this particular stress risks obscuring some of the reasons for the test’s importance. Why, after all, do students like me find the test so stressful? The reason, of course, is because we believe, rightly, that our numeric score will impact our chances of securing the residency position of our choice. The 3-digit Step 1 score will be included prominently in our residency applications and is one of the major factors influencing our success.
The results of the NRMP’s 2018 Program Director Survey make this plain. The fact of the matter is that the Step 1 score is used to make resident selection decisions. In fact, of all the factors that go into choosing which applicants to select for an interview, the Step 1 score (or COMLEX Level 1 score, the equivalent for DO students) is the one used most often by program directors. Indeed, 94% cite the Step 1 score as important, with directors giving it an average importance rating of 4.1 out of 5. Even after an applicant has been interviewed, the Step 1 score informs program directors’ ranking list (though post-interview, “Interactions with faculty during interview and visit” and “interpersonal skills” take precedence).
But whether or not Step 1 was designed to help program directors identify applicants likely to be successful residents — and the NBME/FSMB has made it clear that was not the intent — there is some evidence that it is useful for that purpose. A 2013 meta-analysis found positive correlations between USMLE scores and measures of resident performance, including both examination-based and evaluation-based outcome measures. Other studies have also found positive correlations between Step 1 scores and residency performance, though overall the research is spotty and the results conflicting.
Even if you find the research suggesting that Step 1 scores can help predict success in residency unconvincing, it remains the case that nearly all residency program directors have historically considered the numeric Step 1 score a valuable tool. In its absence, what will the landscape of residency applications look like?
The hope, it seems, is that emphasis will shift to non-standardized factors such as medical school grades, letters of recommendation, and research and extracurricular activities. Setting aside the question of whether non-standardized factors are more or less predictive of residency success than the Step 1 score, the problem is their subjectivity and the difficulty of comparing across students from different schools. This subjectivity has led some to speculate that, absent an objective measure, selection committees will default to preferring applicants who attend schools considered more rigorous or prestigious. Another possibility is that the importance of standardized test performance will merely shift from Step 1 to Step 2. (Notably, the NBME/FSMB shows no sign of this being the intention, though they did make a point of noting that Step 2 results will continue to be reported as a 3-digit score.)
Whatever the effects of this change on residency program directors, I fear it will not have its intended effect on medical students. The fundamental source of the stress and anxiety that we attach to taking Step exams is the stress and anxiety of applying to residency in general. The residency application process gets more competitive every year, and making Step 1 pass/fail won’t change that. I suspect that, like an algebraic equation, reducing the competition associated with the USMLE will merely increase the competition associated with other parts of the application, leaving the sum total of anxiety unchanged. It may manifest as a fight to get your name on the most publications, or (even more) cutthroat competition for honors on rotations — but as long as the number of applicants increases and the number of residency positions remains stagnant, the stress is here to stay.
Austin Reifel is a fourth-year medical student. He can be contacted on Twitter. Austin is a 2019–2020 Doximity Fellow.
