Rad onc supervision, the epilogue

[TheWallnerus]

Was looking at SDN year-end stuff, favorite threads, etc, and this 4 year old post from @elementaryschooleconomics… the smartest poster on SDN… struck me:

I would place a footnote on this - CMS hit us totally randomly with the supervision rule change on November 1st, with uncertain but almost without question negative effects for most of us.

Blame CMS for the mid-interview season timing.

Let us harken back in our collective minds to what a sea change moment this seemed poised to be. CMS was preparing to make a blanket change to supervision levels in all hospital outpatient departments in America come Jan 1, 2020. I also would have written this was going to result in “without question negative effects for most of us.”

However I think 2019 is calling and wants its negative effects back.

I think this is an important discussion to have if only because I can remember how sky-is-falling we all were at the time. But the sky didn’t fall. What happened? Several things in my opinion…

1) ASTRO immediately discovered that image guided radiation therapy was not a therapy but a diagnostic test (and hence why ASTRO no longer calls IGRT “IGRT” anymore but instead “image guidance”).
2) Much of rad onc seemed to have ignored the potential permissiveness of blanket general supervision.
3) The change didn’t affect freestanding (yet in the meantime we have virtual supervision, yada yada).
4) A hodgepodge of varying state laws and restrictive LCDs have helped to make the Supervision Change a dog with more bark than bite?

Suffice it to say I don’t think there have been ANY negative effects from the several supervision changes in rad onc since 2019. No negative patient effects, most importantly, and no negative job market effects. This is my hypothesis… prove me wrong.

Let us learn from history.

Discuss.

---

Members don't see this ad.

 

[elementaryschooleconomics]

Was looking at SDN year-end stuff, favorite threads, etc, and this 4 year old post from @elementaryschooleconomics… the smartest poster on SDN… struck me:



Let us harken back in our collective minds to what a sea change moment this seemed poised to be. CMS was preparing to make a blanket change to supervision levels in all hospital outpatient departments in America come Jan 1, 2020. I also would have written this was going to result in “without question negative effects for most of us.”

However I think 2019 is calling and wants its negative effects back.

I think this is an important discussion to have if only because I can remember how sky-is-falling we all were at the time. But the sky didn’t fall. What happened? Several things in my opinion…

1) ASTRO immediately discovered that image guided radiation therapy was not a therapy but a diagnostic test (and hence why ASTRO no longer calls IGRT “IGRT” anymore but instead “image guidance”).
2) Much of rad onc seemed to have ignored the potential permissiveness of blanket general supervision.
3) The change didn’t affect freestanding (yet in the meantime we have virtual supervision, yada yada).
4) A hodgepodge of varying state laws and restrictive LCDs have helped to make the Supervision Change a dog with more bark than bite?

Suffice it to say I don’t think there have been ANY negative effects from the several supervision changes in rad onc since 2019. No negative patient effects, most importantly, and no negative job market effects. This is my hypothesis… prove me wrong.

Let us learn from history.

Discuss.

*struts around, adjusts Fedora*

In all seriousness -

I've seen zero negative data, but that's also an artifact of limited data in general. I know a crew at Sloan is shopping around a manuscript on their experience and I'm told it confirms no safety issues, but I'm waiting for the actual paper like Christmas morning.

In the meantime, I've gone looking at...basically all of the telemedicine literature, ever. Most of it centers around patient perception and experience, or trainee education, etc. But I've been unable to find like...any threat to any form of safety in any field of medicine.

I would love to hear people's experiences and opinions on this, though.

 

[elementaryschooleconomics]

Just to also go find what Past Me wrote for Current Me to read:

On the role of known and anonymous forums in Radiation Oncology

I’ve been following the Google Spreadsheet this year and have refrained from commenting because I do actually believe that it should be a space reserved for current M4s discussing the match. However, there are many instances of people saying “ignore the SDN trolls and listen to your...

forums.studentdoctor.net

God, those were the days eh?

On November 11th, 2019, I hadn't heard the word "coronavirus" since I took USMLE Step 1 in 2010.

While, obviously, the pandemic was the most Black Swan event ever, the Black Swan of Black Swans, it's hard to overstate how much it shifted the course of Radiation Oncology.

Reading my own post today, in 2024: I completely agree with myself at the time, given the information I had.

And this is the reason I basically refuse to make predictions more than 6 months in the future.

 

[communitydoc13]

any threat to any form of safety in any field of medicine

This will never come.

Safety is incredibly hard to measure in reference to rare events (clinically significant XRT misadministration or commercial airline disasters). I have no doubt that the safety protocols employed in the recent Japan airlines crash were "unproven".

One would need to use proxy measures, like clinically insignificant XRT misadministration or number of safety tracking tools used (a terrible measure).

 

[NotMattSpraker]

*struts around, adjusts Fedora*

In all seriousness -

I've seen zero negative data, but that's also an artifact of limited data in general. I know a crew at Sloan is shopping around a manuscript on their experience and I'm told it confirms no safety issues, but I'm waiting for the actual paper like Christmas morning.

In the meantime, I've gone looking at...basically all of the telemedicine literature, ever. Most of it centers around patient perception and experience, or trainee education, etc. But I've been unable to find like...any threat to any form of safety in any field of medicine.

I would love to hear people's experiences and opinions on this, though.

I am assuming we are now talking both about supervision and telemedicine in clinic. I believe the Sloan paper is about OTVs.

I have thought about this a lot as a user of remote coverage of SBRT and adaptive (published) and doing a fair bit of telemedicine in a lot of settings in the clinic.

The papers I am on suggest that remote coverage of complex procedures is "the same" as being at the machine. I do not agree. That said, people probably vary a lot in their comfort. I like to go to the machine if the case is not straight forward, so that could be comfort over "safety".

If you look at the literature on incident learning in radiation oncology and add in the new Cherenkov imaging data, radiotherapy is exceptionally safe. The risk of an error that reaches the patient has to be less than 5% (could be as low as 1%) and I would guess the vast majority of these cases are only marginally clinically relevant at best.

Even if you do an extreme thought experiment where you do a geographic miss on the majority of early stage NSCLC SBRT fractions in your clinic, you likely will not observe a very high rate of toxicity or treatment failure. Bad radiotherapy is very hard to detect.

So, I do not think the absence of errors in the literature proves that loosening supervision is "safe". I don't think anyone will be able to show it's unsafe. Its going to be all about experience and comfort, as you said.

I can't resist being pedantic here, I have to point out we don't even have an outcomes based definition of safe or good radiotherapy delivery! Our entire quality/safety paradigm hinges on peer review of clinical decision making/target design and technical QA of beam quality.

 

[evilbooyaa]

Suffice it to say I don’t think there have been ANY negative effects from the several supervision changes in rad onc since 2019. No negative patient effects, most importantly, and no negative job market effects. This is my hypothesis… prove me wrong.

Let us learn from history.

Discuss.

There will never be a quantifiable detriment to supervision changes. We can't even 'prove' that people are Bad rad oncs in terms of EBRT, let alone something as minor as supervision changes.

Most jobs that hire are employed. Most employed jobs are not willing to pull the trigger on general supervision despite the rules changes. Those who were doing general a few days a week were likely CAH and were exempted even prior to the rule. Thus, there will be no effect on the job market unless hospitals, en masse, start allowing slower departments to go to a general supervision rule. There are likely anecdotes out there already, but it would have to become main stream.

And you'd probably need to see an academic institution do it first (or perhaps multiple) before the floodgates opened.

*struts around, adjusts Fedora*

In all seriousness -

I've seen zero negative data, but that's also an artifact of limited data in general. I know a crew at Sloan is shopping around a manuscript on their experience and I'm told it confirms no safety issues, but I'm waiting for the actual paper like Christmas morning.

In the meantime, I've gone looking at...basically all of the telemedicine literature, ever. Most of it centers around patient perception and experience, or trainee education, etc. But I've been unable to find like...any threat to any form of safety in any field of medicine.

I would love to hear people's experiences and opinions on this, though.

In regards to bolded: Do you really think Sloan would publish a paper suggesting that something that they do is NOT the best thing ever? If there were a single safety issue identified, the entire project would be buried and never see the light of dead. I can predict the conclusion of the paper right now:

"Telemedicine for OTVs is safe, effective, and convenient for patients to have as an option for their OTVs."

+/- some hedging on only doing it for breast/prostate/whatever disease sites they did, +/- them stating that this is only safe and effective at SLOAN not at some dinky community practice.

But man, can't believe 2019 was only 4ish years ago. Feels a hell of a lot longer than that.

 

[elementaryschooleconomics]

This will never come.

Safety is incredibly hard to measure in reference to rare events (clinically significant XRT misadministration or commercial airline disasters). I have no doubt that the safety protocols employed in the recent Japan airlines crash were "unproven".

One would need to use proxy measures, like clinically insignificant XRT misadministration or number of safety tracking tools used (a terrible measure).

Well...

It will "never come" in the form of evidence we can all universally agree upon, sure.

I guess, as @NotMattSpraker pointed out, we have to be very precise in what we're talking about here in the first place.

We basically have, over the last ~25 years, three distinct "eras" of supervision:

2000-2009 was the transition/bizarro era. You had the development and widespread adoption of both IMRT and routine OBI (with CBCT). From 2006-2009 you had this murky "personal vs direct" supervision debate, which is where the infamous MIMA/NY Times whistleblower case was born.

From 2010-2020, you had the direct supervision era.

From 2020-2024, we're in the general and direct virtual era.

I would argue we can ignore the pre-2010 events because it was too tumultuous.

But it's at least plausible we could build a database with:

Cohort A: January 1st, 2017 - December 31st, 2019 patients
Cohort B: April 1st, 2020 - January 1st, 2024 patients

In the short term, we could compare safety events, whistleblower complaints/claims, disciplinary actions related to radiotherapy (for anyone involved, not just the doctor), etc.

In a few years we could compare outcomes, or we could at least compare outcomes for patients treated March-December 2019 with March-December 2021 (2020 is probably a year with a lot of confounding).

At this point I might just start building this study myself, because to do it well would require data from hospitals and medical boards etc not immediately available to the public.

In regards to bolded: Do you really think Sloan would publish a paper suggesting that something that they do is NOT the best thing ever? If there were a single safety issue identified, the entire project would be buried and never see the light of dead. I can predict the conclusion of the paper right now:

"Telemedicine for OTVs is safe, effective, and convenient for patients to have as an option for their OTVs."

Oh yeah I mean...I already know that's the conclusion of the paper bahahaha, but I'm hopeful that the methods can be even marginally generalized for the rest of us!

 

[communitydoc13]

In the short term, we could compare safety events, whistleblower complaints/claims, disciplinary actions related to radiotherapy (for anyone involved, not just the doctor), etc.

It'll be something, just not something I would take seriously.

Safety events may be higher in a good clinic with a low barrier to reporting such events and a robust reporting system than in a bad clinic with none of the above.

Whistleblower complaints always go down when the regulatory environment becomes more lax (less to blow about)…same with disciplinary actions.

I agree with you to keep the years as close together as possible. It is a worthwhile project if an imperfect one.

 

[elementaryschooleconomics]

It'll be something, just not something I would take seriously.

Safety events may be higher in a good clinic with a low barrier to reporting such events and a robust reporting system than in a bad clinic with none of the above.

Whistleblower complaints always go down when the regulatory environment becomes more lax (less to blow about)…same with disciplinary actions.

I agree with you to keep the years as close together as possible. It is a worthwhile project if an imperfect one.

I keep hoping someone who isn't me does it

(I suspect it will be me)

 

[NotMattSpraker]

I keep hoping someone who isn't me does it

(I suspect it will be me)

All super low volume with unclear connections to the health of the patient. That said, I guess you don't need to compare statistically. You could just go by like vibes or dreams or whatever, it all seems publishable these days.

https://www.linkedin.com/pulse/video-consultations-useful-dupuytrens-radiotherapy-clinics-shaffer-zlf6e%3FtrackingId=yeOVGmm9n%252Fh7DECKLXucZw%253D%253D/?trackingId=yeOVGmm9n%2Fh7DECKLXucZw%3D%3D

Thought this was interesting. A blog about the telemedicine experience of Dr. Shaffer for dupuytrens. 5% had the wrong diagnosis! I have no idea how referrals work or anything, but if 5% of my consults had the wrong diagnosis Id probably have a talk with my admins.

 

[elementaryschooleconomics]

All super low volume with unclear connections to the health of the patient. That said, I guess you don't need to compare statistically. You could just go by like vibes or dreams or whatever, it all seems publishable these days.

https://www.linkedin.com/pulse/video-consultations-useful-dupuytrens-radiotherapy-clinics-shaffer-zlf6e%3FtrackingId=yeOVGmm9n%252Fh7DECKLXucZw%253D%253D/?trackingId=yeOVGmm9n%2Fh7DECKLXucZw%3D%3D

Thought this was interesting. A blog about the telemedicine experience of Dr. Shaffer for dupuytrens. 5% had the wrong diagnosis! I have no idea how referrals work or anything, but if 5% of my consults had the wrong diagnosis Id probably have a talk with my admins.

...

...

...

I'd consider only 5% of my consults with the wrong diagnosis a 5-year goal for my neck of the woods.