Step 1 P/F: Decision

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Okay, so you can actually predict how much you have to limit applications using the ERAS data. Take Internal Medicine for example. If we look at the ERAS data from 2010-2020 and use multivariate regression, we get the following equation:

Applications/Program = 0.02 * (Total Applicants) + 13.32 * (Applications/Person)

So, if we limited IM applications from 40 -> 20, that would reduce total applications from 780.4 -> 514. That's a big difference, but residencies will still need to use screening measures to cut down the application load because of the huge number of medical students applying.

For orthopedics, it's different, the equation is:

Applications/Program = 0.01 * (Total Applicants) + 7.37 * (Applications/Person)

So, if you cut the number of applicants/person from 80->30 programs, that would result in 550->230 applications/program on average, which is actually pretty manageable and could work. Only issue is there is a lot of variability right now with some programs receiving only 200 apps and others receiving like 800, so I can see applicants receiving more interviews but falling much farther down their match lists, and some less desirable programs not matching at all.
Wow, didnt realize that even IM had doubled the applications per capita in under a decade.

If this trend continues we would be over 1000 applications per program in just a few years.

Members don't see this ad.
 
It does add up, in the sense that the quality of their average cohort would be better. Right now if you're a middling program it's smart to be playing defensive. You have to either 1) interview an absurd number of superstars to fill a single slot at your program or 2) yield protect against strong apps because it's not safe to trust they'll rank you well.

You can safely protect yourself from having to SOAP, but only by either way over-inviting and hoping to get lucky or by intentionally hamstringing the quality of your interviewees to ensure you fill.

Imagine, instead, that you knew every single application to your program was seriously considering you because they only have a few precious applications to send. Boom, now you can safely interview all your highly desired applicants and not have to play the yield game. You still fill your program, but now you do it with less apps to review, less wasted interviews and more certainty of applicant interest.
This was my original point. What I was saying doesn't add up is that people are being shut out of competitive specialties because other people over-apply and over-interview. Every spot fills no matter what—the top applicants go on a lot of wasted interviews. Limiting or not limiting the number of apps should not change the number of matches as long as people apply realistically, which is the entire point.

There would be a few people who shut themselves out of the specialty because they overestimate the competitiveness of their application and apply to too many top programs with their limited apps, but I don't think it's anyone's responsibility but theirs, and I think that would mostly self-correct within a couple of application cycles.
 
  • Like
Reactions: 2 users
This was my original point. What I was saying doesn't add up is that people are being shut out of competitive specialties because other people over-apply and over-interview. Every spot fills no matter what—the top applicants go on a lot of wasted interviews. Limiting or not limiting the number of apps should not change the number of matches as long as people apply realistically, which is the entire point.

There would be a few people who shut themselves out of the specialty because they overestimate the competitiveness of their application and apply to too many top programs with their limited apps, but I don't think it's anyone's responsibility but theirs, and I think that would mostly self-correct within a couple of application cycles.
Oh, yeah, to use Carmody's terms the problem with Application Fever was never that it was shutting people out of any specialties. But it was the driving force for Step 1 Mania. That's why people want caps (or why I do at least)
 
  • Like
Reactions: 3 users
Members don't see this ad :)
Will never happen.

Definitely think its low probability, but not 0. but the argument can be made that the current disruption of life can have an impact on student performance and emotion during this period.

Personally, I think it would be a very tough message for the USMLE to sell to students who have built their pre-clinical education around being well prepared for Step 1. I couldn't find data on test takers by month, but enough have already taken the test where the 2022 cohort will have a greater ratio of scored to P/F test takers than it would have if implemented in a few years.
 
Definitely think its low probability, but not 0. but the argument can be made that the current disruption of life can have an impact on student performance and emotion during this period.

Personally, I think it would be a very tough message for the USMLE to sell to students who have built their pre-clinical education around being well prepared for Step 1. I couldn't find data on test takers by month, but enough have already taken the test where the 2022 cohort will have a greater ratio of scored to P/F test takers than it would have if implemented in a few years.
Except, you're literally just supposed to stay home and study, so it's the perfect dedicated? I don't understand this argument and it'll only come from people who don't want to put in the work.
 
  • Like
Reactions: 5 users
Amid COVID-19, make USMLE Step 1 pass/fail now

Seems like there is a possibility of learning about how P/F will impact residency applications sooner than we thought

This idea is absolute nonsense. Everyone is in the same boat more or less (RIP to the people that have to take it 6 months to a year in advance now), so it's up to each person to figure out how they'll continue to prepare for this exam and adjust. Easier said than done, but still. Gotta do what you gotta do.
 
  • Like
Reactions: 2 users
Except, you're literally just supposed to stay home and study, so it's the perfect dedicated? I don't understand this argument and it'll only come from people who don't want to put in the work.


This idea is absolute nonsense. Everyone is in the same boat more or less (RIP to the people that have to take it 6 months to a year in advance now), so it's up to each person to figure out how they'll continue to prepare for this exam and adjust. Easier said than done, but still. Gotta do what you gotta do.


I don't think it will be an open/shut conversation at the USMLE. Most students are going into self quarantine anyways, and in a vacuum it could be an advantage. But the truth is there will certainly be students who are personally affected - whether it is their health, family health, financial, or other. So those students may not have an advantage, and I think the USMLE will consider that amongst other factors when weighing the option.

In general, with the current landscape of the country/world (ie a whole lot of uncertainty)- I don't think we can count on anything happening as one would expect/hope
 
  • Like
Reactions: 1 user
A lot of students would want an extended dedicated period to max out their score, this doesn't seem too tragic...?
 
I don't think it will be an open/shut conversation at the USMLE. Most students are going into self quarantine anyways, and in a vacuum it could be an advantage. But the truth is there will certainly be students who are personally affected - whether it is their health, family health, financial, or other. So those students may not have an advantage, and I think the USMLE will consider that amongst other factors when weighing the option.

In general, with the current landscape of the country/world (ie a whole lot of uncertainty)- I don't think we can count on anything happening as one would expect/hope
I just disagree with this mindset. If you get sick, push back your test. People get sick every year with things, they don't get an excuse to have their Step be P/F because crap happened to them. That's life...stuff happens.
 
  • Like
Reactions: 2 users
Surprise surprise, one of the authors of the article is a med student at Yale (couldn't find out anything about the other one).
 
  • Like
Reactions: 9 users
Members don't see this ad :)
No. Two woke 4th years publishing an opinion piece on a blog will have no impact on the timing of Step 1 going P/F. They're just making sure no tragedy goes to waste.
Their argument gives me a headache.

URMs get an advantage for getting into undergrad and then for getting into med school, and now you want to "level" the playing field for residency??
How long do you want to have your hand held for???
 
  • Like
  • Sad
Reactions: 5 users
A lot of students would want an extended dedicated period to max out their score, this doesn't seem too tragic...?

Some are peeved I'm sure, especially those who were within a week or 2 of testing. Would be tough to maintain your "training" if you've exhausted all your QBanks and now have to maintain for an indefinite amount of time until Prometric reopens and things get back to normal.
 
  • Like
Reactions: 1 users
Surprise surprise, one of the authors of the article is a med student at Yale (couldn't find out anything about the other one).

I immediately knew that was the case, lol.
 
  • Like
Reactions: 2 users
I just disagree with this mindset. If you get sick, push back your test. People get sick every year with things, they don't get an excuse to have their Step be P/F because crap happened to them. That's life...stuff happens.


Yea, if someone got sick and had to change their test, that's life. But when entire states and countries are shutting down for indefinite periods of time, that's entirely unprecedented and changes how decisions are made.

Again, I don't think this would go down and I selfishly hope it doesn't. But I can't ignore the possibility of the USMLE doing something completely off the wall here.
 
  • Like
Reactions: 1 user
Yea, if someone got sick and had to change their test, that's life. But when entire states and countries are shutting down for indefinite periods of time, that's entirely unprecedented and changes how decisions are made.

Again, I don't think this would go down and I selfishly hope it doesn't. But I can't ignore the possibility of the USMLE doing something completely off the wall here.

The only thing the NBME is going to be doing, is sitting back and counting their money
 
  • Like
  • Haha
Reactions: 5 users
Again, I don't think this would go down and I selfishly hope it doesn't. But I can't ignore the possibility of the USMLE doing something completely off the wall here.
We are on the same page here. I don't trust NBME for anything, but I do think this would cause a lot of uproar if people were literally days away from taking their exam and were suddenly told it would be P/F. People need to be prepared for that way before dedicated hits.
 


They took no time squashing that notion!
 
  • Like
  • Love
Reactions: 8 users
Was poking around the USMLE website and found some interesting data on pass rates every year that I hadn't seen before. Very interesting trend:

1586378735115.png


Any DO (or international) able to give some insight on what's been responsible for such amazing gains in the last 10-15 years? Did admissions criteria for DO become much more competitive? Did COMLEX and curricula change to align more with USMLE? Did it take a decade for Uworld and First Aid to become equally popular with DO/IMG student bodies?
 
  • Like
  • Wow
Reactions: 2 users
Was poking around the USMLE website and found some interesting data on pass rates every year that I hadn't seen before. Very interesting trend:

View attachment 301588

Any DO (or international) able to give some insight on what's been responsible for such amazing gains in the last 10-15 years? Did admissions criteria for DO become much more competitive? Did COMLEX and curricula change to align more with USMLE? Did it take a decade for Uworld and First Aid to become equally popular with DO/IMG student bodies?
Mix of those things. DO admission criteria has gone up significatnly since the early 2000s. Still a good amount below MD, but at this point average MD stats are just overkill lol. Once your MCAT is 50th%ile your chance of passing USMLE on first attempt is in the mid 90%s. Also the merger. Around 2015 when it was annoucned it was clear USMLE would become a soft requirement for many programs since the AOA match was going away.
 
  • Like
Reactions: 5 users
Was poking around the USMLE website and found some interesting data on pass rates every year that I hadn't seen before. Very interesting trend:

View attachment 301588

Any DO (or international) able to give some insight on what's been responsible for such amazing gains in the last 10-15 years? Did admissions criteria for DO become much more competitive? Did COMLEX and curricula change to align more with USMLE? Did it take a decade for Uworld and First Aid to become equally popular with DO/IMG student bodies?

There’s a small crowd on here who felt the move to P/F was intentionally anti DO. Before seeing this chart, I thought they were being conspiratorial. I’m more sympathetic to that perspective now.
 
  • Like
Reactions: 1 user
Any DO (or international) able to give some insight on what's been responsible for such amazing gains in the last 10-15 years? Did admissions criteria for DO become much more competitive? Did COMLEX and curricula change to align more with USMLE? Did it take a decade for Uworld and First Aid to become equally popular with DO/IMG student bodies?
Mix of those things. DO admission criteria has gone up significatnly since the early 2000s. Still a good amount below MD, but at this point average MD stats are just overkill lol. Once your MCAT is 50th%ile your chance of passing USMLE on first attempt is in the mid 90%s. Also the merger. Around 2015 when it was annoucned it was clear USMLE would become a soft requirement for many programs since the AOA match was going away.

Basically this. The average DO student has gotten better academically. As MD admissions has gotten more and more competitive there are a growing number of borderline MD candidates at DO schools with lofty goals.

For example the last time I looked at my undergrad med school admittance statistics the average MD matriculant had a 3.8/514.... the DO average was like a 3.6/507 or something like that. A lot of students with the national matriculant averages for MD only getting DO acceptances.

The caveat I will add, is that I think the pass rate would drop slightly if every DO had to take it.
 
  • Like
Reactions: 3 users
Was poking around the USMLE website and found some interesting data on pass rates every year that I hadn't seen before. Very interesting trend:

View attachment 301588

Any DO (or international) able to give some insight on what's been responsible for such amazing gains in the last 10-15 years? Did admissions criteria for DO become much more competitive? Did COMLEX and curricula change to align more with USMLE? Did it take a decade for Uworld and First Aid to become equally popular with DO/IMG student bodies?
In an age of increasing competition, there’s considerably more incentive to try to do well on step. Prior to the merger announcement, our absolute best students were throwing everything into the comlex for that 700+ score and ignoring USMLE. That was the best way to gun for something competitive. If you wanted to do ortho 10 years ago, USMLE was mostly a waste of time.

Now our best students are taking it.
 
  • Like
Reactions: 2 users
The caveat I will add, is that I think the pass rate would drop slightly if every DO had to take it.

Exactly, because how many DOs are even taking it now? Like what's the percentage? Currently, the people gunning for surgical subs/derm/IR and the people that want to keep all options open (location, academic, etc) take step and prepare like their life depends on it. The people pursuing primary care, which is most DOs, don't have to take step, so I assume they don't.
 
Exactly, because how many DOs are even taking it now? Like what's the percentage? Currently, the people gunning for surgical subs/derm/IR and the people that want to keep all options open (location, academic, etc) take step and prepare like their life depends on it. The people pursuing primary care, which is most DOs, don't have to take step, so I assume they don't.

I think it's like 50-60% or something like that. I saw the number published somewhere but don't care enough to go digging for it lol
 
  • Like
Reactions: 1 user
I have the number handy, it was 4,136 DO students in 2018 when there were about 7,000 2nd year DO students. So a majority now (~60%) are taking it!

Good point about it probably being the upper half of the distribution that is bothering to take the Step. Even so, that trend is incredible! It totally explains to me why there was such anti-DO bias in the past (they were passing at around the rate of IMGs), but also looks like good evidence that a studious DO student in the modern era is meeting the bar just as readily as MD students are. Together with the merger, maybe those DO biases will soon be a thing of the past
 
  • Like
Reactions: 5 users
Good point about it probably being the upper half of the distribution that is bothering to take the Step. Even so, that trend is incredible! It totally explains to me why there was such anti-DO bias in the past (they were passing at around the rate of IMGs), but also looks like good evidence that a studious DO student in the modern era is meeting the bar just as readily as MD students are. Together with the merger, maybe those DO biases will soon be a thing of the past

~300 hours of magic in the curriculum probably hasn’t helped
 
  • Like
Reactions: 1 user
The attendings training and teaching you on the wards overwhelmingly scored in the 200s-210s or lower, if they took it at all. Yet we have no problem respecting them, or at least, I don't. The suggestion that the existence of the test was the important part, not the content itself, just underscores how badly we needed this change. Having everyone memorize digits of Pi to find who is the most studious should feel like an effective argument ad absurdum, not like an actual good idea.


I'll add that it's not just a maldistribution of applications, but also of who gets interviewed. A shocking fraction of interviews gets absorbed by a small handful of applicants. In the worst cases like Family Medicine and Internal Medicine, a full 50% of interviews are being taken by only 7-12% of applicants.

bOQt2JD.png



The opportunity cost of this inequity is huge. So, so many interviews are getting wasted on a handful of superstars that are all likely to land in their first few ranks.

Sorry, late to the game here, the conv has moved on. But I can't help but comment on this. The paper is a really interesting look at application numbers, but these graphs and conclusions make no sense to me. Let's look at the statement "7-12% of applicants get 50% of interviews". The lower numbers in those graphs are "all ERAS applications", which includes people who get no interviews at all. Lots of those are IMG's who are not competitive and should simply be excluded from this discussion. But I'm not convinced the bigger number, 25%, is correct either. The authors calcuated their findings from 2016, so I used the same data set. Here's the data for IM:

1586401267582.png


First columns are the number of ranks, followed by the number of people who matched and didn't match with those ranks, and then the sum of those in Column D, followed by the percent of the total. So 18% of applicants ranked 16+ programs, and by the author's definition were hoarders.

Column F is (D x A), which is the total number of interviews consumed by that group. The total number of interviews is at the bottom, and Col G is the percent of interviews. So, that 18% had 26% of the interviews. In a random world, they would have had 18%, so their difference is 8%. Nowhere near as big as the authors are suggesting.

To get exactly to that graph above, Col H is a running total of interviews summed from the bottom to the top, and col I is the percent of the total. You can see that the groups of 13 - 16 ranks, when summed together, use 54% of the interview spots. That last column is the sum of applicants (Col D) for that same range, and the last number is the percent of the total. So, by these calculations, in IM 40% of the applicants accounted for 54% of the interviews.

You'll get no argument from me that application inflation overloads programs, and that programs might decide to decline to interview candidates that are "too good" for them. An application limit is one way to address this, but a limit low enough to actually make a difference would be problematic for applicants. A limit of 50 would still likely yield the same behavior from programs -- all an application tells me is "I'm in your top 50" which isn't very helpful
 
  • Like
  • Hmm
Reactions: 6 users
Sorry, late to the game here, the conv has moved on. But I can't help but comment on this. The paper is a really interesting look at application numbers, but these graphs and conclusions make no sense to me. Let's look at the statement "7-12% of applicants get 50% of interviews". The lower numbers in those graphs are "all ERAS applications", which includes people who get no interviews at all. Lots of those are IMG's who are not competitive and should simply be excluded from this discussion. But I'm not convinced the bigger number, 25%, is correct either. The authors calcuated their findings from 2016, so I used the same data set. Here's the data for IM:

View attachment 301625

First columns are the number of ranks, followed by the number of people who matched and didn't match with those ranks, and then the sum of those in Column D, followed by the percent of the total. So 18% of applicants ranked 16+ programs, and by the author's definition were hoarders.

Column F is (D x A), which is the total number of interviews consumed by that group. The total number of interviews is at the bottom, and Col G is the percent of interviews. So, that 18% had 26% of the interviews. In a random world, they would have had 18%, so their difference is 8%. Nowhere near as big as the authors are suggesting.

To get exactly to that graph above, Col H is a running total of interviews summed from the bottom to the top, and col I is the percent of the total. You can see that the groups of 13 - 16 ranks, when summed together, use 54% of the interview spots. That last column is the sum of applicants (Col D) for that same range, and the last number is the percent of the total. So, by these calculations, in IM 40% of the applicants accounted for 54% of the interviews.

You'll get no argument from me that application inflation overloads programs, and that programs might decide to decline to interview candidates that are "too good" for them. An application limit is one way to address this, but a limit low enough to actually make a difference would be problematic for applicants. A limit of 50 would still likely yield the same behavior from programs -- all an application tells me is "I'm in your top 50" which isn't very helpful
Very interesting. I'll try to run something similar for other specialties as well

Sent from my Pixel using Tapatalk
 
Any update on p/f for current M1’s? Kinda sucks still not knowing whether we should focus on zanki or class rank
 
Any update on p/f for current M1’s? Kinda sucks still not knowing whether we should focus on zanki or class rank
Nahh. I think they’re focused on figuring out how to let M2/M3s take Step 1/2 by May-June rn lol
 
Well then. I’ll just adjust either way when the time comes. I’m happy I’m an M1 and not an M2 or 3 right now
M2s are fine. Longer dedicated. Us M3s are totally screwed
 
  • Like
Reactions: 1 users
Hi guys, I am MD class of 2024 and I have a few questions. Do you all have any suggestions for what I should be doing in order to ensure I am able to match at a competitive residency when the time comes? Since STEP is now P/F and grades are too.. I am a little anxious about what I should be focusing on. Should I be focusing on joining on campus organizations? Volunteering? Doing as much research as possible? Please let me know what you guys think as I want to have a game plan before I start classes. I am going to a middle-tier MD school.
 
Hi guys, I am MD class of 2024 and I have a few questions. Do you all have any suggestions for what I should be doing in order to ensure I am able to match at a competitive residency when the time comes? Since STEP is now P/F and grades are too.. I am a little anxious about what I should be focusing on. Should I be focusing on joining on campus organizations? Volunteering? Doing as much research as possible? Please let me know what you guys think as I want to have a game plan before I start classes. I am going to a middle-tier MD school.
Bingo. That and staying on top of your studies. I believe Step 2CK will eventually be p/f, but once Step 1 is p/f and CK is still scored, there will be a lot of people who underachieve on CK because they will have underestimated the knowledge base that proper, thorough Step 1 studying provides. Do not do that to yourself by ignoring Step 1 because it's p/f. The best way to do well on Step 2CK is to do well on Step 1.

This got me thinking: it's possible that, when Step 1 goes p/f, the Step 2CK average score will actually go down for this reason. I know this will obviously be balanced against increased importance of CK, extra motivation, etc., but if you miss the chance to put the hay in the barn for Step 1, it will be hard to do really well on CK.
 
  • Like
Reactions: 2 users
I could have done about 30% of the work to just pass med school instead of trying. If they make STEP1 and CK pass fail, and schools like Yale have no preclinical or clinical grades, then the residency selection process has moved to undergrad.

Go to a school that inflates. And destroy the MCAT. More important than ever for premeds in the future. Elites now control the game. They are flexing. And they will use the excuse the test is racist, sexist, classist, and homophobic, if they have to. They want to erode meritocracy.

If anything residency selection should be school blind, then that would really throw a wrench into all of this. Going to a top school should give the advantages of more opportunities to succeed rather than that plus basically a guaranteed ticket to a top residency, regardless of performance, especially in the context of nationally standardized criteria.
 
Last edited:
  • Like
Reactions: 7 users
Still be AOA. Still be perfect. If you are a mid tier.

AOA has been Made Great Again. You have no choice now. Kill yourself for those PhD exams. The flood gates have been opened to intraclass gunnerism again.

The era of even semi egalitarianism is coming to a close. The door will really shut with CK, but there is still a little bit of hope.

District T20 has begun the war. May the odds ever be in your favor.
 
  • Like
  • Haha
Reactions: 4 users
Was poking around the USMLE website and found some interesting data on pass rates every year that I hadn't seen before. Very interesting trend:

View attachment 301588

Any DO (or international) able to give some insight on what's been responsible for such amazing gains in the last 10-15 years? Did admissions criteria for DO become much more competitive? Did COMLEX and curricula change to align more with USMLE? Did it take a decade for Uworld and First Aid to become equally popular with DO/IMG student bodies?

UFAP and the all of Anki Decks have made information access an even playing field. Study strategies are also quite streamlined. A 250 now is a matter of space bar monkeying to victory along with doing tons of questions. Nothing else is relevant.
 
  • Like
Reactions: 4 users
UFAP and the all of Anki Decks have made information access an even playing field. Study strategies are also quite streamlined. A 250 now is a matter of space bar monkeying to victory along with doing tons of questions. Nothing else is relevant.
Unfortunate for Step to go P/F so quickly after the residency merger. With DO takers on track to equal or outperform MD takers, a decade of mixing with similar Step scores could have helped knock down some bias
 
  • Like
  • Sad
Reactions: 6 users
Still be AOA. Still be perfect. If you are a mid tier.

AOA has been Made Great Again. You have no choice now. Kill yourself for those PhD exams. The flood gates have been opened to intraclass gunnerism again.

The era of even semi egalitarianism is coming to a close. The door will really shut with CK, but there is still a little bit of hope.

District T20 has begun the war. May the odds ever be in your favor.

This is another one of the bad sides of this decision. AOA is a stupid thing that amounts to little more than a popularity contest at many schools. That it holds any weight whatsoever in residency selection is ridiculous
 
  • Like
Reactions: 2 users
This is another one of the bad sides of this decision. AOA is a stupid thing that amounts to little more than a popularity contest at many schools. That it holds any weight whatsoever in residency selection is ridiculous
Sorry but can you explain what AOA means?
 
Terrible idea. Absolutely not. NO. Nope.
 
This is another one of the bad sides of this decision. AOA is a stupid thing that amounts to little more than a popularity contest at many schools. That it holds any weight whatsoever in residency selection is ridiculous
I disagree. I think it carries a net benefit despite whatever flaws come with clinical grades and election criteria. If we accept that merit-based competition between peers exists for certain specialties and for certain programs within all specialties, we should accept that students should have ways to demonstrate merit compared to their peers.

As more and more schools move to P/F everything, P/F Step 1, not even putting full comments on MSPEs, there's almost nothing left for a student to point to that constitutes credible evidence of excellent performance in med school.

Not everyone is the same, and there's nothing wrong with acknowledging that. Med school performance is not a reflection of someone's worth as a person. But in a competitive field like medicine, I think it's wrong to legislate for equality of outcomes among a given class at a school when there is nearly true equality of opportunities at matriculation, probably more than in any other profession. And I think it's flatly wrong to abolish recognition of excellent performance in med school because others' feelings may be hurt or they may suffer emotional stress during a critical time like the start of M4. That is BS.
 
Top