Board certification exam - we need CHANGE!!

[deleted1088441]

Given recent changes to USMLE and MOC, we need to look at the board certification exam too! Look at this interesting new viewpoint piece that argues for this!

It is a single high-stakes exam that could thwart the career you have been working towards for years. It costs thousands of dollars and hasn’t been proven as a marker of predicting a good clinician. Its failure rate is almost universally higher than USMLE exams, and things have only worsened during COVID.

Given the ongoing changes to USMLE and MOC, we need to look at the board certification exam too. Direct link to article here as well: https://rdcu.be/cdwTF

---

Members don't see this ad.

 

[ValentinNarcisse]

Given recent changes to USMLE and MOC, we need to look at the board certification exam too! Look at this interesting new viewpoint piece that argues for this!

It is a single high-stakes exam that could thwart the career you have been working towards for years. It costs thousands of dollars and hasn’t been proven as a marker of predicting a good clinician. Its failure rate is almost universally higher than USMLE exams, and things have only worsened during COVID.

Given the ongoing changes to USMLE and MOC, we need to look at the board certification exam too. Direct link to article here as well: https://rdcu.be/cdwTF

Some may disagree, but I would say attitudes like this are fueling the decline of America. Lowering standards of admission for medical school, removing USMLE scores, and now arguing for removal of board exams only allows inadequate and underperforming students and residents to move forward in the process. We should be selecting for the elite, not promoting those who could not adequately prepare for exams they knew were coming years in advance. Everyone knows the rule of the game up front, prepare accordingly.

 

[LucidSplash]

Nothing REALLY forces you to take the boards. All the boards are is a declaration of a minimum level of competence in a specialty. Minimum. Level. Most hospitals will not give you privileges if you aren’t board eligible or BC. But you can set up your own shop, be PP, found a surgery center, etc. I understand complaints about MOC expense and time. But no one asks you your score on the board exam once you are in practice. I do not understand the argument against having a marker for a minimum level of competence and then requiring a minimum level of ongoing education to maintain that marker.

 

[NotAProgDirector]

I don't understand how the changes to USMLE have anything to do with this. USMLE was changed to Pass/Fail. Board cert is already pass/fail (basically) -- you do get a score in IM but nothing is ever done with it. It's already p/f. And in IM, you can already use UTD during the exam.

 

[Obnoxious Dad]

Some may disagree, but I would say attitudes like this are fueling the decline of America. Lowering standards of admission for medical school, removing USMLE scores, and now arguing for removal of board exams only allows inadequate and underperforming students and residents to move forward in the process. We should be selecting for the elite, not promoting those who could not adequately prepare for exams they knew were coming years in advance. Everyone knows the rule of the game up front, prepare accordingly.

I would say that this post represents the reason we now spend 18% of the U.S. GDP on health care in a normal year and still face shortages of nurses, physicians, dentists, dental hygienists, and pharmacists. The health care professions in the U.S. have erected one barrier after another to professional practice without any empirical data to show that these barriers improve care or reduce cost. The Flexner boys are still at work.

We've closed almost all of the inexpensive but highly effective hospital based RN diploma programs. We've lengthened the education of pharmacists from 5 years to eight years in spite of the fact that advances in computer technology, packaging and supply chain management have made the job easier and less risky. In spite of the fact that anesthesia is much safer than it was 30 years ago, thanks to improved protocols and technology, its now takes three years beyond getting an RN and two years of icu time to become a CRNA. Dentists have fought the use of dental therapists everywhere. It now takes two years to learn how to learn how to clean teeth.

 

[deleted1088441]

Some may disagree, but I would say attitudes like this are fueling the decline of America. Lowering standards of admission for medical school, removing USMLE scores, and now arguing for removal of board exams only allows inadequate and underperforming students and residents to move forward in the process. We should be selecting for the elite, not promoting those who could not adequately prepare for exams they knew were coming years in advance. Everyone knows the rule of the game up front, prepare accordingly.

I think the point is to figure out evidence based ways to evaluate physicians. Continuing to do something just because that is the way it has always been done is not a strong argument. You can have a high bar of evaluation without a high-stakes exam that has little evidence of actually being a good marker of competency.

 

[deleted1088441]

I don't understand how the changes to USMLE have anything to do with this. USMLE was changed to Pass/Fail. Board cert is already pass/fail (basically) -- you do get a score in IM but nothing is ever done with it. It's already p/f. And in IM, you can already use UTD during the exa

 

[deleted1088441]

I don't understand how the changes to USMLE have anything to do with this. USMLE was changed to Pass/Fail. Board cert is already pass/fail (basically) -- you do get a score in IM but nothing is ever done with it. It's already p/f. And in IM, you can already use UTD during the exa

I think its looking at the greater picture of what can happen when you critically look at how we evaluate ourselves along the way, and to go one level deeper (which this article does not address) - what are the forces preventing change, particularly where the money flows.

I don't understand how the changes to USMLE have anything to do with this. USMLE was changed to Pass/Fail. Board cert is already pass/fail (basically) -- you do get a score in IM but nothing is ever done with it. It's already p/f. And in IM, you can already use UTD during the exam

 

[NotAProgDirector]

I totally agree that a MCQ exam is a very imperfect way of addressing skill as a clinician, especially for procedural based specialties. Even in IM, I've had residents who graduated where I was 100% confident in their skills, and they failed the exam. So it is imperfect.

But I do worry about replacing it with something "better". The ABIM tried to do exactly that with it's MOC program. When initially launched, it included getting patient feedback, submitting evidence of practice improvement activity, etc. It was well intentioned, but an enormous PITA for unclear benefit. Before I'm willing to abandon what we have, I'd like to know what you're recommending we replace it with.

As mentioned, the ABIM exam is now "open book" (a huge plus IMHO), they listened to feedback and dumped most of the most onerous parts of the MOC, trialed a 2 year mini-exam which was another disaster and has been suspended, and now are working on some sort of an ongoing MOC process (perhaps) without a 10 year exam.

Should just graduating from an ACGME program grant "board ceritfication"? That's an interesting question. I could probably support that if there was a decent MOC program that was required from the start.

 

[deleted1088441]

Open book is an interesting option - did not know that was the case for ABIM, I think that is a step in the right direction. Certainly not that way for peds now, or many others I am aware of. I'm not sure that just graduating from an ACGME program would suffice, but its an interesting argument. I think graduation along with a more robust milestone project along the way that has real markers that cannot just "checked off" could be interesting.

 

[sunshinefl]

Open book is an interesting option - did not know that was the case for ABIM, I think that is a step in the right direction. Certainly not that way for peds now, or many others I am aware of. I'm not sure that just graduating from an ACGME program would suffice, but its an interesting argument. I think graduation along with a more robust milestone project along the way that has real markers that cannot just "checked off" could be interesting.

Like what? What would that look like?

 

[SLC]

I don't understand how the changes to USMLE have anything to do with this. USMLE was changed to Pass/Fail. Board cert is already pass/fail (basically) -- you do get a score in IM but nothing is ever done with it. It's already p/f. And in IM, you can already use UTD during the exam.

Wait what? You can use UpToDate during the ABIM exam?

Crazy if true.

 

[sunshinefl]

Wait, is that true?

 

[deleted1088441]

Wait what? You can use UpToDate during the ABIM exam?

Crazy if true.

i think that person is mistaken - my understanding is its only MOC where you can do that

 

[deleted1088441]

Wait what? You can use UpToDate during the ABIM exam?

Crazy if true.

i think that person is mistaken - my understanding is its only MOC where you can do that

 

[mvenus929]

Open book is an interesting option - did not know that was the case for ABIM, I think that is a step in the right direction. Certainly not that way for peds now, or many others I am aware of. I'm not sure that just graduating from an ACGME program would suffice, but its an interesting argument. I think graduation along with a more robust milestone project along the way that has real markers that cannot just "checked off" could be interesting.

A more robust milestone project? Since you're apparently peds--what is your issue with the current milestone project?

Also keep in mind that any new system is going to require manpower. If you shift that manpower to the residencies, they need additional funding for administrative work. Direct observation is a fabulous tool, but it's so difficult to get enough direct observations to determine if someone is competent due to staff time, patient comfort, etc.

Canada is working on competency-based medical education and has been for years (I don't know all the details and haven't felt like digging through the literature to figure it out). The thought is that people don't all need the same amount of time to become competent in a skill, so you focus on the skills you need help with and check off the skills you're already proficient at. Part of the problem is that we don't do well with feedback and don't have enough people who are good at doing it.

 

[Dr.LeoSpaceman]

Open book is an interesting option - did not know that was the case for ABIM,

I'm always suspicious when people who don't seem to know/understand the details of the process show up and claim it needs to be blown up.

What if it's not the process of the board exam, but the candidate selection and education that precedes it?

 

[deleted1088441]

I'm always suspicious when people who don't seem to know/understand the details of the process show up and claim it needs to be blown up.

What if it's not the process of the board exam, but the candidate selection and education that precedes it?

A more robust milestone project? Since you're apparently peds--what is your issue with the current milestone project?

Also keep in mind that any new system is going to require manpower. If you shift that manpower to the residencies, they need additional funding for administrative work. Direct observation is a fabulous tool, but it's so difficult to get enough direct observations to determine if someone is competent due to staff time, patient comfort, etc.

Canada is working on competency-based medical education and has been for years (I don't know all the details and haven't felt like digging through the literature to figure it out). The thought is that people don't all need the same amount of time to become competent in a skill, so you focus on the skills you need help with and check off the skills you're already proficient at. Part of the problem is that we don't do well with feedback and don't have enough people who are good at doing it.

No issue with the current milestone project - if you look in the article, it talks about incorporating a more robust milestone project into a board certification process.

 

[deleted1088441]

I'm always suspicious when people who don't seem to know/understand the details of the process show up and claim it needs to be blown up.

What if it's not the process of the board exam, but the candidate selection and education that precedes it?

The person who wrote that ABIM allows up to date in the initial certification exam was mistaken. Well, obviously people agree the entire process is imperfect, which is why you are seeing changes in USMLE, residency evaluation (milestone project), and MOC - so why not look at the certification exam too?

 

[mvenus929]

No issue with the current milestone project - if you look in the article, it talks about incorporating a more robust milestone project into a board certification process.

Yeah, I'm not clicking on a link that I can't tell where it directs me, especially on an open forum.

So, how do you make the milestone project more robust, and how do you incorporate it into board certification? Still sounds like you're putting the onus on the residency program, which means we need better assessment tools to be able to determine that residents are meeting those milestones, which means more manpower.

You come in like this is a huge thing that you can change overnight without realizing that people are already working on new ways to ensure that graduating residents will provide safe and competent care to patients. Board certification is one component of that. And, as mentioned above, USMLE going pass/fail is not equivalent to change board certification, which is already (essentially) pass/fail--literally no one cares what score I got on my Peds Boards. They just want to know if I'm certified. And the MOC discussion is still ongoing and frought with controversy since it's essentially a money-making scheme for boards with no direct correlation (that I'm aware) of patient care. Not to mention the fact that we don't have great ways of assessing if someone is providing good care to someone unless they are egregiously not providing said care.

 

[deleted1088441]

Yeah, I'm not clicking on a link that I can't tell where it directs me, especially on an open forum.

So, how do you make the milestone project more robust, and how do you incorporate it into board certification? Still sounds like you're putting the onus on the residency program, which means we need better assessment tools to be able to determine that residents are meeting those milestones, which means more manpower.

You come in like this is a huge thing that you can change overnight without realizing that people are already working on new ways to ensure that graduating residents will provide safe and competent care to patients. Board certification is one component of that. And, as mentioned above, USMLE going pass/fail is not equivalent to change board certification, which is already (essentially) pass/fail--literally no one cares what score I got on my Peds Boards. They just want to know if I'm certified. And the MOC discussion is still ongoing and frought with controversy since it's essentially a money-making scheme for boards with no direct correlation (that I'm aware) of patient care. Not to mention the fact that we don't have great ways of assessing if someone is providing good care to someone unless they are egregiously not providing said care.

Not sure why you are taking an oppositional tone, it honestly sounds like we are on the same page about most of this. Should the onus not be largely on the residency program? Isn't that the point of residency? To train physicians in a specific field and make them competent in that field? If so, why not get them more involved in the certification process? It will hold programs more accountable. The money is there to do it - its just flowing to the wrong people. Why instead rely on a high-stakes exam that is quite frankly a terrible marker for anything other than one's performance on the exam? You are right no one cares about your score - because everyone understands how useless the exam is (and there is data to support this). Any no one is talking about an overnight change - just sparking a conversation - what is wrong with that?

 

[Dr.LeoSpaceman]

The person who wrote that ABIM allows up to date in the initial certification exam was mistaken. Well, obviously people agree the entire process is imperfect, which is why you are seeing changes in USMLE, residency evaluation (milestone project), and MOC - so why not look at the certification exam too?

Yes, I'm fine with evaluating board examinations. But I think the problem I have is that I get the sense our goals in re-evaluating are different. I reflexively disagree with a position that something is deserved just by "putting in the time". And to me, that is how this reads:

It is a single high-stakes exam that could thwart the career you have been working towards for years.

The point is not to redesign in order to get 100% pass rate. In my mind the goal is to create a meaningful assessment that ensures people meet some level of baseline competence. In that setting, some people might fail. While that's unfortunate, it's preferable to churning out physicians of questionable competence.

And I think that's why USMLE reform is positive. Over the past decade, it's morphed from a way to make sure everyone possesses similar baseline knowledge to some sort of medical aptitude/placement exam.

 

[deleted1088441]

Yes, I'm fine with evaluating board examinations. But I think the problem I have is that I get the sense our goals in re-evaluating are different. I reflexively disagree with a position that something is deserved just by "putting in the time". And to me, that is how this reads:



The point is not to redesign in order to get 100% pass rate. In my mind the goal is to create a meaningful assessment that ensures people meet some level of baseline competence. In that setting, some people might fail. While that's unfortunate, it's preferable to churning out physicians of questionable competence.

And I think that's why USMLE reform is positive. Over the past decade, it's morphed from a way to make sure everyone possesses similar baseline knowledge to some sort of medical aptitude/placement exam.

I don't disagree with anything you say - and agree it shouldn't be a free pass just because you've "sweated it out" prior. I think there are a number of reasons to reform the process - including the variance of pass rates of the board exam in any given year - its between 81-95% - so on the other hand, if you haven't been weeded out by the time the board certification rolls around - and suddenly 10-20% of people are failing - there is an issue in the evaluation process - and I do not think you can put it all on the board exam itself, but it should take some of the onus - but the entire process needs to constantly be re-evaluated.

 

[agranulocytosis]

i think that person is mistaken - my understanding is its only MOC where you can do that

Initial certification is still closed book. I've taken 3 initial certification exams since 2017 (IM, Pulm, CC), all have been closed book and thankfully passed first time. Recertification has access to Up To Date, q2 years or q10 years from my understanding.

As a reply to the initial poster: the pass rates for the IM and IM subspecialty boards is generally >95%. They're essentially "gimme" tests. That is, if sitting for 6-8 hours nearly falling asleep taking a test is a "gimme".

If you're arguing about the difficulty of the test, it's unfounded. But if you're arguing against the cost of the test and the money-making racket the ABIM has going on, then you're onto something.

All told, I've spent at least $6k in test fees just to be able to practice. Thanks, ABIM.

 

[deleted1088441]

Initial certification is still closed book. I've taken 3 initial certification exams since 2017 (IM, Pulm, CC), all have been closed book and thankfully passed first time. Recertification has access to Up To Date, q2 years or q10 years from my understanding.

As a reply to the initial poster: the pass rates for the IM and IM subspecialty boards is generally >95%. They're essentially "gimme" tests. That is, if sitting for 6-8 hours nearly falling asleep taking a test is a "gimme".

If you're arguing about the difficulty of the test, it's unfounded. But if you're arguing against the cost of the test and the money-making racket the ABIM has going on, then you're onto something.

All told, I've spent at least $6k in test fees just to be able to practice. Thanks, ABIM.

your statistic about 95% pass is incorrect - ABIM is 89-90s, peds 81-93, surgery closer to low 80s. Suspect it will be lower for all this year given COVID, But anyhow - you are correct that most pass. Check out the original article to get a better sense, but you are also correct that perhaps the most infuriating factor is the money racket that has been set up - not something discussed in this article, but worth adding to the discussion.

 

[Dr.LeoSpaceman]

surgery closer to low 80s

Incorrect. First time pass rate for the qualifying exam (multiple choice) is ~95% while the first time pass rate for the certifying exam (oral exam) is 80-85% (data). The combined first-attempt pass rate was ~87% in a recent cohort of 600 GS residents And this does not account for the number of people who will eventually pass on subsequent attempts.

So again, I'm having a hard time pinning down your specific argument as it relates to the board exams. They're too hard? They don't adequately assess what they're meant to assess? They are biased against specific types of examinees? The time and money involved on the part of the examinee isn't commensurate with the perceived value?

EDIT: Sidenote, I think the oral exam is honestly one of the better ways to test the things boards are interested in assessing. Especially now that the ABS has focused on standardizing the exam to the point where it's not just two white-hairs quizzing you on the esoterica that springs to their minds.

 

[Raryn]

I don't understand how the changes to USMLE have anything to do with this. USMLE was changed to Pass/Fail. Board cert is already pass/fail (basically) -- you do get a score in IM but nothing is ever done with it. It's already p/f.

This. No one but the examinee and (optionally - decision made by the examinee) the PD of their former program can even see the score.

And in IM, you can already use UTD during the exam.

Only for the recertification exam. UTD during the initial exam isn't a thing. That said, the ABIM initial exam isn't all that bad.

 

[mvenus929]

Not sure why you are taking an oppositional tone, it honestly sounds like we are on the same page about most of this. Should the onus not be largely on the residency program? Isn't that the point of residency? To train physicians in a specific field and make them competent in that field? If so, why not get them more involved in the certification process? It will hold programs more accountable. The money is there to do it - its just flowing to the wrong people. Why instead rely on a high-stakes exam that is quite frankly a terrible marker for anything other than one's performance on the exam? You are right no one cares about your score - because everyone understands how useless the exam is (and there is data to support this). Any no one is talking about an overnight change - just sparking a conversation - what is wrong with that?

I'm sorry I'm coming across as oppositional. Rather, I'm intending to challenge and determine exactly what you want changed, as someone above said, and encourage you to do more reading (as people much smarter than me are already doing extensive work in this area).

If it's how we become board certified, then how would you like to change things? You mentioned not liking a single test--so should we break it up into multiple tests? Or is it the test itself isn't good for you? Using competency-based medical education will help with this, but what do we do when someone needs more time to become competent? It doesn't work very well in our current residency system where you get x amount of funding for y years to complete your residency and reduced funding after that, and residents are employees--at what point do we say that someone isn't making satisfactory progress?

Is it that board certification doesn't predict good clinical practice? Well, then we need to define what good clinical practice is, and how to assess it, and figure out what predicts it (is it more personality/inherent skill? Is it exposure to certain things? Is it the circumstances in which they work? Is it specialty-specific or across specialties?) The only information I'm personally aware of about USMLE scores and future performance relates to either board passing rates or to sanctions by the medical board, but there are a lot of people who don't provide 'good' clinical care that aren't 'bad' enough to warrant a sanction from the board (in large part because their style of practice genuinely works for some people). I honestly don't know the data for specialty specific board exams, but also challenge that different skills are needed to work in different situations. I need a different skill set as peds endo than you do as PEM, than my friend does as a general pediatrician. But we all sit for the same basic board exam (and then the subspecialists among us sit for another one 2+ years later).

 

[NotAProgDirector]

Thought I had posted this yesterday:

Sorry for the bad info. You're correct, it's only the ABIM recert that is open book, and perhaps only the 2 year "check in". In any case, I think making the entire exam open book makes sense -- but I don't think that will actually help people pass. Usually, people who are failing run into time troubles -- so won't really have time to look up more stuff. And you need to recognize what you don't know, and find it quickly to be of value. So I think it's a good idea, but I wonder if it would really change anything.

If we make residency programs required to do "enhanced milestones" (whatever that is), you can be sure that 100% of everyone will meet those milestones at graduation. If not, residents would need to be extended (else their career comes to a complete halt, with no way to improve) and since programs will get assessed on their ability to graduate people on time, you'll get 100%. If we want board certification to mean something, then it has to be done by an outside agency.

The "milestones" are a joke. They are worthless. Yes, I said that.

 

[rokshana]

Wait what? You can use UpToDate during the ABIM exam?

Crazy if true.

Yes you can for recertification

 

[Raryn]

Thought I had posted this yesterday:

Sorry for the bad info. You're correct, it's only the ABIM recert that is open book, and perhaps only the 2 year "check in". In any case, I think making the entire exam open book makes sense -- but I don't think that will actually help people pass. Usually, people who are failing run into time troubles -- so won't really have time to look up more stuff. And you need to recognize what you don't know, and find it quickly to be of value. So I think it's a good idea, but I wonder if it would really change anything.

UpToDate is available for both the 2 year and the 10-year option. I *think* for the 10-year option it's only available for 3 of the 4 blocks. I'm also unsure of whether it will be available for the upcoming "longitudinal assessment" which will replace the 2-year option in 2022ish.