Board certification exams need a makeover. In a recent survey of anesthesiology residents on Objective Structured Clinical examination (OSCE), an overwhelming majority said it did not add any value to their training or prepare them for life after residency. In the study, 96% of respondents said it should be stalled, with 60% of those respondents saying it should be scrapped altogether. Even though the survey focused on anesthesiology, the results highlight issues of evaluation bias and the unproven value of all board certifications.
American Board of Anesthesiology (ABA) is unique among the American Board of Medical Specialties (ABMS) members to introduce OSCE for candidates who have completed residency since 2016. While all the 24 member boards administer a computerized “written” examination of multiple-choice questions, 13 boards also administer oral examinations. One of the members, psychiatry, notably eliminated oral boards in 2016 and decided to restrict to only written examinations. Among the reasons for this elimination were cultural and language barriers seen as potential factors contributing to evaluation bias. If there is one specialty where communication and professionalism is important, it is probably psychiatry. Yet, the subjective nature of oral boards was recognized as an unavoidable downside that outweighed any purported benefits.
Being a private company, the ABA or any ABMS member is allowed to introduce any changes in their certification process as they see appropriate. All the ABMS members have their unique goals, with the goals of the ABA being 1) to serve the public interest by advancing the standards of anesthesiology practice through certification and maintenance of certification programs, and 2) advancing the highest standards of the practice of anesthesiology.
Considering these examinations are responsible for major stress and notable burnout, distress, and depression among residents and first year graduates, any resulting certification must be shown to improve the patient care. Unfortunately, no study has ever shown that board certification improves patient care. Other than Silber et al., no one has even attempted to study such relationship. In this study, 8,894 patients were anesthetized by mid-career anesthesiologists. These anesthesiologists were 11–25 years removed from medical school graduation and lacked board certification. The 8,000 patients were compared with a much larger cohort, about 61,000 patients. But the outcome studied (death rate within 30 days of admission, in-hospital complication rate, and the failure-to-rescue rate) had little to do with anesthesia delivery.
If the only argument for continuing this practice of certification (initial and maintenance of certification) is that insurers expect it or that it lessens the disciplinary actions, that is wholly insufficient. For example, the relationship between performance on the American Board of Physical Medicine and Rehabilitation primary certification exams and the risk of subsequent disciplinary actions by state medical boards over a physician's career was analyzed to support the conclusion that written or oral examination failure increased the risk of subsequent disciplinary action by 5.77-fold. Zhou et al., provided similar conclusions to support the effectiveness of oral specialty certification examinations in anesthesiology. Similarly, although Kopp et al. , observed that obtaining board certification was associated with a lower rate of receiving severe license actions from a state medical boards among surgeons, they also noted that more research is needed to link the certifications to outcomes. However, factors such as alcohol and substance abuse, inappropriate prescribing practices, inappropriate contact with patients, and fraud are responsible for more than two thirds of disciplinary actions and these have nothing to do with board certifications. Demonstration of competence in oral exams cannot be directly equated with reduced disciplinary proceedings.
In the absence of any concrete data from independent agencies (nearly all the studies supporting the value of certifications are from ABMS member board directors) that associate certifications to better and safer patient care, focus needs to be on improving the value of existing exams and reduce unnecessary burden and not increase untested layers of examination. In this context, I suggest that the ABMS should ask the following questions:
1. What should be the nature of these examinations and do they bring in any benefit to our patients?
2. Who should be conducting these examinations?
3. Finally, who should be performing research to examine the value of these certifications?
In conclusion, it is essential that the certifying authorities provide evidence of improved patient care among board certified providers. Additional layers or changes in the examination must be proven to accomplish this goal. Use of surrogate markers, especially in the form of association between license disciplinary actions from medical boards, is wholly inadequate. It is important to know that both residents and experienced attendings are already burdened with significant service commitments. As a result, their time and effort should be directed toward providing better patient care and any certifications (including recertification) should assist in achieving that goal. Otherwise, it would be construed as wasted time and energy, which is better utilized for patient care. Finally, it is essential that board directors should also include residents and all research must be conducted by independent nonprofit organizations.
How would you revamp the OSCE? Share in the comments.
Dr. Goudra is a content creator and owner of the website IatroBoards, a multiple-choice question bank for anesthesiology residents and medical students preparing for USMLE and COMLEX.
Image by Intpro / GettyImages
