- Joined
- Aug 23, 2014
- Messages
- 3,268
- Reaction score
- 6,624
Sounds like the field is SOL. there is zero true leadership at the very top.
I don't know what you all thought was going to happen at the circus that is ASTRO.
PDs are mostly junior faculty trying to climb the ladder of the popularity contest that is academics, which means playing politics and carefully navigating pathologic personalities. Challenge the ABR's big lie? To what end? I am sure many mean well and want to help but can't risk self-immolation.
Walking around this meeting and overhearing conversation, I have never felt so out of place. Our field is filled with truly odd, insecure, and jealous people. Academics attract the oddest of the odd. What is a normal person to do? I suppose go present a retrospective review "poster" to a group of people who don't care. What a waste. I know that there are a handful of us who can see through all the b.s., and as much as I'd like to believe we are really a silent majority, I sadly not really sure anymore.
The ABR wants to break it down between big and small programs. Why stop there guys? Why not break it down between sex, race, age, socioeconomic status, undergrad institution, political affiliation, marriage status, etc. I am sure if they looked hard enough they could find all sorts of shocking ways to group people who failed. But there's only one grouping they care about.
The next step? We need to fix this system that allows the ABR to administer physics/bio repeat exams the day before the clinical. That is absurd and setting people up for failure.
These exams test how well you can cram for a test and memorize a load of useless and easily forgettable material, not how competent you are. It's step 1 all over again. We all know that. The ABR will never admit it. The idea you could pull a PGY-5 aside in clinic on any random day and expect him to be able to perfectly draw the MAPK/ERK pathway is laughable.
What we can do is keep pushing them to separate the dates between the physics/bio and clinical exam. There should be a one month buffer between them, minimum. There is no reason to have them on sequential days unless the ABR wants to purposely punish and set a higher bar for failing candidates. We should ask the ABR to administer the bio/physics exam in July and the clinical in August. There is no reasonable objection they could offer to this. We need to push them to admit that sequential exams are unfairly punitive and is a problem that is easily solved by separating the exams by a month. This solves the biggest problem, and we should focus on this before moving on to the higher level problem of the exam content and scoring.
Reality? Not going to happen. They want the class of 2019 to have repeat failures on bio/physics, new failures on clinical, and a 2-3 year delay in getting to orals. With higher bars at each step, including orals, hopefully getting a handful of us past the 6 year hard limit on board eligibility. They used these exams to identify their "small number" of residents that they want forced out of the field (or at least into the locums pool driving rates even further down) and build Wallner's proscription list of programs that need to be shut down.
This scandal has killed any enthusiasm I had left after enduring the pointless tribulations of residency. It's really sad. I know people are suffering through some serious depression and anxiety over this affecting their personal and professional lives. Not even the tiniest effort to help or compromise from the ABR. Not a shred of compassion, even. Instead they patronize and humiliate the 60% of us not in the big programs. I'm so over it. ASTRO is a joke and I doubt I'll come back any time soon after I graduate.
Angoff method may have been influenced by increased clinical physician input into the exams.
Couple things I heard second-hand:
Wallner wasn't there.
Kachnic defended the position as said above. There was apparently some discussion of potentially moving the re-take to sometime a little bit earlier than clinical (so it wasn't back to back days), but nothing finalized.
Kachnic apparently frustrated that she's getting blasted for this as she's just the second in command for this.
No plans for mid year re-take.
Kachnic shut down suggestions of "simply increase the pass rate" to 85-90%.
Small programs (< 6 residents) only had statistically significant results compared to large programs (> 16 residents). There were 4 columns in terms of resident split, only the first and 4th column were 'significantly' apart.
Apparently some maybe better resources for study might come out? However, apparently when they state that 'cancer biology' or 'immunology' are testable concepts, they mean ALL of cancer biology and immunology.
Angoff method may have been influenced by increased clinical physician input into the exams.
No analysis of what percentage of old exam questions (40-60% any given year) were correctely answered.
My takeaways:
What a joke.
Apparently we're all only supposed to train at MSKCC, MDACC, HROP, Stanford, Emory, or other places with 16+ residents. Or expand programs until they hit 16 residents, at which point we'll just osmosis all the rad bio and physics we need from each other.
Kachnic is likely just the fall gal given Wallner didn't even have the decency to show up.
Better have a PhD in both cancer biology AND immunology AND whatever other topics are listed in the 'study guide' if you wanna be sure you know enough material to pass.
Wish somebody had discussed the hit piece Wallner and Kachnic co-authored in PRO as a conspiracy.
If your PD went to ASTRO, I encourage you to ask them directly what the proposed solution to any of this will be, as they can potentially post more details.
. Just waiting for the impossible clinical boards this year to double down on the class of 2019's "stupidity".
Can't even imagine how malignant the oral boards will be in a couple years....traditionally that was the one test most of us worried about.
At this rate, they might as well just keep baseball bats in the examination hotel rooms for the examiners... would be a lot quicker for the examinees.
Can someone please elaborate on how the angoff score may have been done incorrectly this year? Did they change the point value for questions that have been used in the past? That should absolutely be illegal for them to do.
I still don’t understand how/why we have no power to force the ABR to demonstrate that this was a fair exam. The Onus should be on them. It shouldn’t be a “take our word for it” situation. If they changed the value of questions compared to years prior (which wouldnt surprising) or if our class performed similarly on questions used on prior exams they should be forced to re-evaluate.
For anyone who hasn't, listen to Kachni's ARRO presentation. Its recorded on the virtual meeting. Mind boggling.
What session? There are like 50 arro sessions
For anyone who hasn't, listen to Kachni's ARRO presentation. Its recorded on the virtual meeting. Mind boggling.
I have a hard time believing she knew that was going to be recorded.
14 minutes into the "Applying Jobs and Entering Practice: A panel discussion"
A few thoughts:
1) Shout out to the woman who asked the first question in the session after LK spoke. Her question was succinct and exceedingly reasonable. For those who didn't attend the session (I wasn't able to attend) or listen to the 'virtual meeting' presentation, she asked LK if the ABR had compared the performance on the 'recycled' questions questions which appeared on the 2018 exams and the prior exams. In other words, did 2018 examinees answer these questions correctly at the same rate as prior years? LK did not have an answer (she "left out the slides") but assured us additional results would be presented during ADROP (the whisperings did not include this in their review).
2) LK's explicit acknowledgement of recalls playing a role in larger programs is really, really frustrating. More residents = more people to 'inform' radiobiology instructors after taking rad bio and physics = better instruction in larger programs = better pass rates in larger programs. I have trouble believing she would say this out loud knowing the session was recorded. I'm not in the the pro-lawsuit group because I hold out hope this thing will correct itself when enough PDs/chairs complain, but this is the kind of thing that very strongly suggests an unfair advantage for certain residents in larger programs.
3) The alleged disconnect between ABR and ASTRO is a farce. It's the same people who are on committees and task forces and research groups. It's like they pretend there's a firewall and they don't talk to each other professionally or otherwise. As such, they can shrug off the (very reasonable) request for better exam prep materials.
Then again, maybe these three points are really just cyber bullying...
The whole idea of residents taking a rad bio exam, and then telling their instructors "hey you really should cover these additional 5-10 topics, since they were on the test this year and we didn't learn about them" is in the same spirit as a recall, right? Why is that felt to be OK?
I still want to see the data that stratifies exam results based on presence of a faculty member on the exam committee. I imagine that will also be positive. Obviously doesn't fit their narrative of 'hurr hurr small programs need to get geud', but that's what anybody who knows anything about research would ask in regards to their 'conclusions'.
"Maybe this is a phenomenon of larger programs I don’t know, but the residents talk. You know… you’re not suppose to have recalls and there are no recalls but you kind of talk about what is on the exam, and a couple years back as cancer biology was coming on the exam our residents would say to our radiobiologist that teaches the course… you know… you really need to cover this. So, we get not all programs have this so we will go back with all of the stakeholders to make sure what you need to prepare will be there."
Lisa Kachnic, ASTRO 2018
"Maybe this is a phenomenon of larger programs I don’t know, but the residents talk. You know… you’re not suppose to have recalls and there are no recalls but you kind of talk about what is on the exam, and a couple years back as cancer biology was coming on the exam our residents would say to our radiobiologist that teaches the course… you know… you really need to cover this. So, we get not all programs have this so we will go back with all of the stakeholders to make sure what you need to prepare will be there."
Lisa Kachnic, ASTRO 2018
It was in the news several years ago for radiologyDoes the ABR actually talk about how residents create and save recalls for all the written board exams? Was that discussed at all at the ASTRO session?
What she described is the definition of a recall. If an examinee tells their professor what is on the exam to help future students that is a recall. I am not sure how anyone could argue otherwise.Kachnic: "You know… you’re not suppose to have recalls and there are no recalls"
There are no recalls? What if there is proof there were recalls? Would that be a big deal? How old would recalls have to be to "prove" there's still a problem with the exam?
The whole idea of residents taking a rad bio exam, and then telling their instructors "hey you really should cover these additional 5-10 topics, since they were on the test this year and we didn't learn about them" is in the same spirit as a recall, right? Why is that felt to be OK?
.
Would folks rather have 20 spots be divided up between MDA and MSK or MDA and MSK have 16 spots total between them and the next 4 go to random DO school’s rad onc department staffed by 4 attendings?
Not the situation in RO actually.
More like mdacc and msk getting 24 spots now and a bunch of random smaller university and community programs opening up 4-6 spot programs as well
Personally, I think the bottom 150-200 community rad spots should be eliminated because they do not provide adequate training.
A few off the top of my head: New York Methodist, Allegheny general, Beaumont (although to be fair, that's really an outlier in terms of its reputation).What community programs are there? .
200 spots would be all of them
What can we do? I think a couple of things.
1. Mainstream media has been interested in the recall story before. No reason to think they wouldn't be again, especially since the ABR has gotten busted on it before. They wouldn't care about doctors failing a hard test. They would care about large programs conspiring to shut down the little ones by implementing cheating methods that benefit programs with larger numbers of residents. Especially when a maldistribution problem has been well-documented and the small programs are the ones that are filling the rural and midwestern need for rad oncs. Now the ABR wants everyone to be training at the big programs? Harvard grads are not typically going to work in Salina, KS.
2. We need to quantify recall use. What's out there, what are people doing, what kind of strategies are these programs using? If you know something, speak up. Use a VPN and post anonymously.
3. Someone could post recalled questions publicly and anonymously on a different website like reddit or something. This is really the only thing we can do to actually fight back (and it's no surprise the ABR forbids it as it is the only power we have to take back some control). Making the recalls public does two things: 1. It forces the ABR to create a fair test by generating 100% new questions each year, and 2. It levels the playing field for everyone and prevents under-qualified residents from passing while qualified residents fail.
If we can prove that recalls were used, there may be a legitimate argument to invalidating this year's test.
Unlike diagnostic radiology, rad onc doesn't really have a problem with programs flagrantly not training residents well. In general with only a few exceptions in the past, you'll be equally competent no matter what program you go to.
This assertion by the ABR that resident performance at bigger programs is due to poorer teaching and residency complacency is a complete lie. Resident quality is excellent throughout the field with little variation. We don't have IMGs, FMGs, DOs. Most residents came from top medical schools with top USMLE scores. It just isn't so. The ABR is lying.
What IS different between big programs and small programs is the capacity for exam recall use, as Lisa Kachnic inadvertently admitted when she said what we all already knew. I tried to bring this up earlier in the thread and nobody really seemed to care that much, but I think what really came from the ASTRO "meeting" was that the recalls are the issue.
My program of 4 residents does not have recalls. We don't "report back" to our instructors. Only one person takes the test each year. I barely failed rad bio. Now that we know 60% of the exam is recycled, I feel confident that had I had access to those recalls and used them, I would have passed. Likewise, I feel confident that many who passed using recalls would have otherwise failed.
The ABR has allowed this problem to happen. By recycling questions, they engender the creation of recalls. Period. If the ABR actually cared about people cheating, then they would write new tests each year. But they don't. They hate the small programs as evident by their raving publications on the matter, so it's just peachy that big programs are doing this. What this has done is reward the dishonest (cheating) residents and programs and harm the honest people. Recall use falsely elevates exam scores and misleads the test creators by biasing their opinions on what we "would know" per the Angoff method (funny how Lisa Kachnic wouldn't comment on any shortcomings of this method namely the glaringly obvious problem of over-confidence bias). As a result the honest people are punished by creep in question difficulty.
What can we do? I think a couple of things.
1. Mainstream media has been interested in the recall story before. No reason to think they wouldn't be again, especially since the ABR has gotten busted on it before. They wouldn't care about doctors failing a hard test. They would care about large programs conspiring to shut down the little ones by implementing cheating methods that benefit programs with larger numbers of residents. Especially when a maldistribution problem has been well-documented and the small programs are the ones that are filling the rural and midwestern need for rad oncs. Now the ABR wants everyone to be training at the big programs? Harvard grads are not typically going to work in Salina, KS.
2. We need to quantify recall use. What's out there, what are people doing, what kind of strategies are these programs using? If you know something, speak up. Use a VPN and post anonymously.
3. Someone could post recalled questions publicly and anonymously on a different website like reddit or something. This is really the only thing we can do to actually fight back (and it's no surprise the ABR forbids it as it is the only power we have to take back some control). Making the recalls public does two things: 1. It forces the ABR to create a fair test by generating 100% new questions each year, and 2. It levels the playing field for everyone and prevents under-qualified residents from passing while qualified residents fail.
Keeping recalls private allows cheating. Making them public prevents it.
I have never seen an ABR recall. If you have this, the power to do something is in your hands. We need whistle-blowers, people that are willing to make the recalls public and talk to the media. This is not ok, and the ABR has no right to bully us the way they are.
If we can prove that recalls were used, there may be a legitimate argument to invalidating this year's test.
It's probably the same issue in RO as well from a clinical standpoint. I trained at a smaller academic program in the middle of nowhere but it was well established with plenty of pathology including peds, srs/sbrt, interstitial gyn/h&n/prostate brachy etc.I am not familiar with radonc training but familiar with DR and IR training from the research I’ve did.
While a small community DR program may train people “adequately”, they do not have the breadth of pathology to allow great training. They maybe lacking in modalities like cardiac MRs for example, and their IRs may not do work like interventional oncology.
I am very sympathetic to your situation, KHE88, but do you think a 4 person program truly have no gaps in experience compared to training at centers with 12-16 residents?
It seems like to me that they are basically trying to reduce spots with their agenda. It’s a really sinister way to do it, but could this be ultimatey beneficial to the job market?