dear class of 2012

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
I'm pretty confident that Step 2 is a waaaay bigger thing among medical students than it is Program Directors. It can be a plus if you have a really bad Step 1 score, but for 95% of us, it just isn't going to matter any.

If you have a 220 on your Step 1 and shoot up considerably on your Step 2, no one is going to equate you with someone who scored 235 on your Step 1.

I wouldn't overly think the Step 2 thing. Taking it early is mainly advantageous just for the sake of PDs knowing it's one less nagging thing they have to worry about you potentially failing and making their life difficult.

Members don't see this ad.
 
you can also take step 2 early just for the sake of being done with it. if you take it anytime after you submit eras and before mid jan or so the scores will be back so that you have the option of sending it to programs if you think it might help. the nice thing about being done with step 2 at that point is that you don't have it hanging over your head after interviews so you can really relax a bit.
 
When I was interviewing for residency many PDs stated in my interview:

"Well, I see you took step 2. And did well. That is VERY good and a DEFINITE point in your favor!"

There are programs that won't rank you if you don't have a passing step 2 score so if you take it late and fail, you won't be ranked. If you take it early, at least you have time to retake before rank lists are due.
 
I've done a quick anecdotal analysis of my program's use of Step 2 in the application evaluation process to see how often the Step 2 score helped an applicant, hurt an applicant, or made no difference to the application:

For applicants who were seriously considered but ultimately not offered an interview, having the Step 2 score helped 34, hurt 4, and made no difference for 21 of them.

For applicants who ultimately were offered an interview (78 total), having the Step 2 score helped 18--and we ended up matching with two of them. Eight were invited and ultimately ranked without releasing their Step 2 scores.
 
Putting more weight on Step 1 than Step 2 has never made any sense to me at all. I thought we were, for the most part, supposed to be clinicians.

Maybe someone on the program admin side can explain.
 
Then hook them with the "zomg that's amazing I've always had an interest in undeserved populations with HIV, that is SO unique I don't think anyone else does it like you guys!! Like, you have a clinic. And you give them da HAARTz. Amazing."

Yes, this process is THAT lame. Accept. Embrace. Match. Think of it as a first date.

What if I like playing hard to get on first dates? :laugh:
 
To Summarize Step 2 Options and how it may or may not affect you.

1) You bombed Step 1 and barely passed. I would take Step 2 to have it available by ERAS submission but you have to attempt to rock it.
Pros: If you do rock it, you might save yourself from some low score screening.
Cons: Do poorly and now you've got multiple low scores, which is arguably worse than 1 low score.

2) You have a mediocreish Step 1 score but have done really well on shelf exams and think you can rock Step 2. Take it so it's either ready to submit with ERAS or soon after Sept 1 based on your level of anxiety and whether you're comfortable with controlling who can see it if you're worried about the score.
Pros: Do well on Step 2 and perhaps your app gets taken out of an auto-reject based on score pile at a more competitive program
Cons: Do the same on Step 2 and now you have a trend of non interesting scores but it probably won't burn you too much. Do worse and then you burn yourself.


3) You did just fine on Step 1 but it's nothing that makes you standout. You can go either way but definitely take it after you submit ERAS so you can control who sees the score.
Pros: Maybe an impressive score will catch someone's attention at some where more competitive.
Cons: Only if you do poorly and are forced to submit it based on the game changer rule below.

4) You rocked Step 1. Take whenever you want unless the game changer applies, you have nothing to worry about just whatever you do take it so the score comes out after you submit ERAS so you can control it's release, no need to risk making that beautiful score look slightly less beautiful.
Pros: You're a smarty pants, there are no cons here.

Game changer: Make sure you look at all the programs you want to apply to and see their Step 2 requirements. Many of the Cali schools are starting to require it for ranking so you may just need to take it and send it to everyone.

Overtime: Your scores follow you to fellowship too. I had a faculty member who is also a fellowship director for a competitive fellowship at my school tell me she definitely looks at Step 2.
 
Last edited:
To Summarize Step 2 Options and how it may or may not affect you.

1) You bombed Step 1 and barely passed. I would take Step 2 to have it available by ERAS submission but you have to attempt to rock it.
Pros: If you do rock it, you might save yourself from some low score screening.
Cons: Do poorly and now you've got multiple low scores, which is arguably worse than 1 low score.

2) You have a mediocreish Step 1 score but have done really well on shelf exams and think you can rock Step 2. Take it so it's either ready to submit with ERAS or soon after Sept 1 based on your level of anxiety and whether you're comfortable with controlling who can see it if you're worried about the score.
Pros: Do well on Step 2 and perhaps your app gets taken out of an auto-reject based on score pile at a more competitive program
Cons: Do the same on Step 2 and now you have a trend of non interesting scores but it probably won't burn you too much. Do worse and then you burn yourself.


3) You did just fine on Step 1 but it's nothing that makes you standout. You can go either way but definitely take it after you submit ERAS so you can control who sees the score.
Pros: Maybe an impressive score will catch someone's attention at some where more competitive.
Cons: Only if you do poorly and are forced to submit it based on the game changer rule below.

4) You rocked Step 1. Take whenever you want unless the game changer applies, you have nothing to worry about just whatever you do take it so the score comes out after you submit ERAS so you can control it's release, no need to risk making that beautiful score look slightly less beautiful.
Pros: You're a smarty pants, there are no cons here.

Game changer: Make sure you look at all the programs you want to apply to and see their Step 2 requirements. Many of the Cali schools are starting to require it for ranking so you may just need to take it and send it to everyone.

Overtime: Your scores follow you to fellowship too. I had a faculty member who is also a fellowship director for a competitive fellowship at my school tell me she definitely looks at Step 2.

Agree with this post. However I would emphasize that the Steps have a lot less impact on fellowship than they did for residency. Sort of like some residencies ask for college GPA or MCAT scores, but don't really use them in any significant way. You have a 3-7 year more recent residency track record to draw from, so nobody is going to care if you squeaked by on Step 2 8 years ago. And the bulk of folks won't be doing fellowships anyhow, unless the job market in certain specialties remains tight.
 
Interpret the Step 2 advice appropriately - it is very field-specific.

In ENT, I did not have my Step 2 scores in for the first half of my interviews, and not a single interviewer commented on it. When my scores were in during the second half of my interviews, not a single interviewer commented on it. I think the competitive specialties tend to be much more Step 1-weighted. And even then, it was more of a "your academic achievements speak for themselves, now we want to know about you" sort of thing. Talk to PGY-1/PGY-2s in your home program... they're close to the process and probably the most "in the know" for what helps and doesnt for your specific field.

And most of all, keep an open mind and keep up your confidence throughout the process. Interview season tends to invoke a lot of "imposter syndrome" if you compare yourself to other applicants... be confident in yourself. I ended up matching at a program which I ranked highly but considered "out of my league". It can happen!
 
Interpret the Step 2 advice appropriately - it is very field-specific.

In ENT, I did not have my Step 2 scores in for the first half of my interviews, and not a single interviewer commented on it. When my scores were in during the second half of my interviews, not a single interviewer commented on it. I think the competitive specialties tend to be much more Step 1-weighted. And even then, it was more of a "your academic achievements speak for themselves, now we want to know about you" sort of thing. Talk to PGY-1/PGY-2s in your home program... they're close to the process and probably the most "in the know" for what helps and doesnt for your specific field.

I agree with this and I too will enter a ENT residency, I just have one additional comment. I had step 2 ready for viewing during the second half of my interviews. No one commented on the fact that I didn't have it during December (I matched at a place that I interviewed at in December). In fact, when I had my Step 2 score in January (~260) 3 people at 2 separate programs actually made it a point to tell me that it didn't matter how I did on step 2, without provocation. It seemed like Step 1 was an interview make or break criterion, one of many, and that step 2 wasn't part of any equation (and yes, there are equations... sadly).
 
I'm still curious why they put so much weight on an exam that has so little to do with clinical medicine.

I'd say convenience. It's the only standardized test that everyone has taken (almost all schools require a passing score after so many months in 3rd year). It's a simple quantitative number and people love looking at the numbers, because it is easy. It's definitely not a good measure of how students will perform with clinical/surgical skills, but it does correlate with how they'll do on in service exams.
 
I'd say convenience. It's the only standardized test that everyone has taken (almost all schools require a passing score after so many months in 3rd year). It's a simple quantitative number and people love looking at the numbers, because it is easy. It's definitely not a good measure of how students will perform with clinical/surgical skills, but it does correlate with how they'll do on in service exams.

I would beg to note that although there is a correlation, it doesn't account largely for the correlation...

http://www.ncbi.nlm.nih.gov/pubmed/21036416
 
Thanks for all the useful info in this thread. Keep it coming :D
 
If you:

(a) are in the bottom-1/3 of your class, and

(b) rotated at a program where they liked you more than you liked them,

then have a serious talk with your dean before making your ROL.

The traditional (and generally solid) advice that you should rank programs in order of your preference might not apply to you.

The scramble this year was brutal. There were hundreds more people in it than there were available slots. :thumbdown:

In my case, the dean said that the particular program where I rotated (where the residents seemed unhappy) wrote me a very strong recommendation letter, and that I likely would have matched there if they'd been at the top of my ROL rather than at the bottom of it. It is possible that he only said this to make me feel better. Talking with him about it before it was too late to do anything about it would've made be feel better, also, and I may have been able to match in the specialty I wanted.

I'm fine with where I ended up, but I only got that because I was lucky. Anecdote: 2nd day of scramble, I got an e-mail from a program telling me that I made the short-list of 50 people they were considering for their 1 open slot. I didn't know this program existed until I applied to a crapload of places on the first day. Seriously.

That isn't where I ended up, btw. I'm just saying: the scramble sucks and will only get suckier with the sizes and number of med school classes growing faster than the sizes and number of residency programs.
 
Is that really true? Ranking a program a certain way can prevent you from matching?

I dunno about that...

As long as you rank ANY program, you have a chance of matching there, ONLY if you were ranked high enough or if OTHERS high on their (the program's) rank list decided NOT to rank the program (or ranked them super low).

I ranked according to where I wanted to go (in order), and I still matched... not at my top spots, but I matched. I was happy with the program at my interview, so I'm cool with it. Only reason I didn't rank it as highly was due to location (the NE, I'm a southerner). Still, I'm happy.

Maybe I didn't understand the algorithm, but from everything I read, it really doesn't matter WHERE you rank a program.. as long as you RANKED the program.


I thought where you rank a program had no bearing unless a certain program ranked you high enough?


If you:

(a) are in the bottom-1/3 of your class, and

(b) rotated at a program where they liked you more than you liked them,

then have a serious talk with your dean before making your ROL.

The traditional (and generally solid) advice that you should rank programs in order of your preference might not apply to you.

The scramble this year was brutal. There were hundreds more people in it than there were available slots. :thumbdown:

In my case, the dean said that the particular program where I rotated (where the residents seemed unhappy) wrote me a very strong recommendation letter, and that I likely would have matched there if they'd been at the top of my ROL rather than at the bottom of it. It is possible that he only said this to make me feel better. Talking with him about it before it was too late to do anything about it would've made be feel better, also, and I may have been able to match in the specialty I wanted.

I'm fine with where I ended up, but I only got that because I was lucky. Anecdote: 2nd day of scramble, I got an e-mail from a program telling me that I made the short-list of 50 people they were considering for their 1 open slot. I didn't know this program existed until I applied to a crapload of places on the first day. Seriously.

That isn't where I ended up, btw. I'm just saying: the scramble sucks and will only get suckier with the sizes and number of med school classes growing faster than the sizes and number of residency programs.
 
You should always make you ROL in order of PREFERENCE. If you ranked them 1 or 10 it doesn't change your rank on programs list only yours....if they ranked you at a number to match then you wil go there if you did not match at your higher preferred programs.
 
Hopefully the Dean understood that, and was just softening the suckage.

Either way, glad you got a spot. :thumbup:
 
If you:

(a) are in the bottom-1/3 of your class, and

(b) rotated at a program where they liked you more than you liked them,

then have a serious talk with your dean before making your ROL.

The traditional (and generally solid) advice that you should rank programs in order of your preference might not apply to you.

The scramble this year was brutal. There were hundreds more people in it than there were available slots. :thumbdown:

In my case, the dean said that the particular program where I rotated (where the residents seemed unhappy) wrote me a very strong recommendation letter, and that I likely would have matched there if they'd been at the top of my ROL rather than at the bottom of it. It is possible that he only said this to make me feel better. Talking with him about it before it was too late to do anything about it would've made be feel better, also, and I may have been able to match in the specialty I wanted.

I'm fine with where I ended up, but I only got that because I was lucky. Anecdote: 2nd day of scramble, I got an e-mail from a program telling me that I made the short-list of 50 people they were considering for their 1 open slot. I didn't know this program existed until I applied to a crapload of places on the first day. Seriously.

That isn't where I ended up, btw. I'm just saying: the scramble sucks and will only get suckier with the sizes and number of med school classes growing faster than the sizes and number of residency programs.

This makes zero sense and is bad advice.

You have absolutely nothing to lose by ranking programs in terms of preference. If a program you ranked highly isn't interested in you, then you simply drop to the next one on your list, and then the next one, and so on.....like others have said, the way you rank a program has no bearing on how they will rank you.

Please learn how the match works before you start posting horrible advice.
 
In my case, the dean said that the particular program where I rotated (where the residents seemed unhappy) wrote me a very strong recommendation letter, and that I likely would have matched there if they'd been at the top of my ROL rather than at the bottom of it. It is possible that he only said this to make me feel better. Talking with him about it before it was too late to do anything about it would've made be feel better, also, and I may have been able to match in the specialty I wanted.

100% false. If you were ranked highly enough to match there, you would have matched there whether they were first or last on your list, so long as you ranked them (given that you didn't match anywhere higher on your list).
 
In my case, the dean said that the particular program where I rotated (where the residents seemed unhappy) wrote me a very strong recommendation letter, and that I likely would have matched there if they'd been at the top of my ROL rather than at the bottom of it. It is possible that he only said this to make me feel better.
Sorry to hear that you had to scramble, but if it helps, your dean is incorrect. Either he meant for that to make you feel better or he doesn't understand how the match works. Your rank order list doesn't affect whether you match or not. If they had ranked you highly you would have wound up there if none of the other higher programs had ranked you highly.

But I think you're better off at a program where people are happy and treated well. I hope you enjoy the place that you wound up at and it all turns out to be for the best in the long run.
 
I will give you my list of things I think are crucial, but deferoxamine nailed most of them already.

1. Buy a nice suit. I would buy 2. Brooks Brothers outlets will give you 2 for $500.
2. Leave your goddamn phone in the car!
3. Depending on your field, do an early away, and get a LOR from someone big.
4. Apply to as many programs as you can afford.
5. Have your application complete before Sept 1st. Leave a spot open for that away letter when it arrives.
6. Practice your 30 second blurb on the following: "tell me a little about you," "why did you choose ___," "tell me what you did in this research project." Make your answers concise, don't bore someone with specific pathways, and don't look rehearsed. Just be natural. Notice I said 30 seconds!
 
How many interviews can you squeeze into Dec and Jan?
 
How many interviews can you squeeze into Dec and Jan?

It depends on where the interviews are located. If you are in a large city with several programs, you could probably do 3-4 if the stars line up correctly and all the programs are offering interviews that week. If you are traveling, you have to consider the probability of delays (especially since you're traveling during the winter) and factor in time for rest. I'd recommend 2 per week max, and never on consecutive days unless they are very close to each other. It's hard to present yourself at your best, most interested, when you're tired. Tired applicants can easily come across as unfocused or disinterested, and in my experience interviewers (residents as well as faculty) don't usually give applicants benefit of the doubt -- they assume disinterest.

The sad truth is that there are only so many interview days in the year, and only a few programs will coordinate with each other. Applicants end up having to make difficult choices among excellent programs offering interviews on the same days.
 
Fewer in December, the last 10 days or so tend to be interview free due to holidays, whereas only the first few days of January are out of the question.

I had January off. I did 10 interviews that month. Not easy but not impossible. Scheduling is stressful and a snowstorm can really screw you.
 
If you:

(a) are in the bottom-1/3 of your class, and

(b) rotated at a program where they liked you more than you liked them,

then have a serious talk with your dean before making your ROL.

The traditional (and generally solid) advice that you should rank programs in order of your preference might not apply to you.

100% false. If you were ranked highly enough to match there, you would have matched there whether they were first or last on your list, so long as you ranked them (given that you didn't match anywhere higher on your list).
Bolded for emphasis. Absolutely false advice in billy1343's post. Rank where you want; if they don't want you, they won't rank you. Then you can move down on your list and get ranked by the place that does want you. No harm, no foul.
 
How many interviews can you squeeze into Dec and Jan?

As many as you can schedule. I went on 14 interviews, 12 in Nov/Dec, 1 in Oct, 1 in Jan.

I did up to 4 interviews/week which is...rough. It was especially bad when there was a lot of traveling going on between them. I would limit it to 2-3 a week and try to group by place if possible.

That being said, a lot of interviews in a short period of time can be done and can work out well. I matched at a place that was my 3rd interview in 3 days! Just try to know you're own limitations.

Good luck!
 
If you:

(a) are in the bottom-1/3 of your class, and

(b) rotated at a program where they liked you more than you liked them,

then have a serious talk with your dean before making your ROL.

The traditional (and generally solid) advice that you should rank programs in order of your preference might not apply to you.

The scramble this year was brutal. There were hundreds more people in it than there were available slots. :thumbdown:

In my case, the dean said that the particular program where I rotated (where the residents seemed unhappy) wrote me a very strong recommendation letter, and that I likely would have matched there if they'd been at the top of my ROL rather than at the bottom of it. It is possible that he only said this to make me feel better. Talking with him about it before it was too late to do anything about it would've made be feel better, also, and I may have been able to match in the specialty I wanted.

This makes zero sense and is bad advice.

Please learn how the match works before you start posting horrible advice.

100% false.

Before you guys make such strong statements, realize that YOU do not, beyond a shadow of a doubt, know how the match works.

The NRMP website gives us an idea of how the algorithm works, but NONE of us knows what the algorithm is. I, and some others (as mentioned by Billy), suspect that there is a line in the current algorithm which might do something to the above effect. [If tl;dr, skip to last paragraph]

The algorithm is mathematical, and works on a non-Nash Equilibrium (think: Beautiful Mind). Id venture to guess that most of us don't begin to understand the complexity of the mathemetics, myself included. But here's some tidbits of insight to make you think... and ultimately, the red statement above might actually hold some water.

Fact: Over the years there have been (at least) two different algorithms in use for the Match. The switch from the first to the second algorithm was made because it was determined that, with the large numbers involved, it did not matter where an Applicant ranked a program. Matches were computed and found to be in favor of the programs preferences. A second algorithm was devised which, according to the mathematics and probably simulations, better "favored the applicant"

For the next few thoughts, please think like a physicist. As the apocryphal physicist replied when asked to describe a cow he said, "consider a sphere...". Dont think in precise numbers - think in general terms. The Match is working with ever-increasing numbers of applicants, and mathematical algorithms work within a particular range of numbers. Even the algorithm which, at any given time, keeps your Facebook Wall nicely all-inclusive of all your friends, yet makes sure you see the "important" status updates, this algorithm starts to break down when you have >250 friends. So bear with me, if you'd like to understand the above suggestion.

Over the years, there has been an rapidly increasing number of Applicants in the match, and a relatively stable number of programs. It is possible that the mathematics, and non-Nash game theory, works differently in this situation. Think of it this way - Youve got, more or less, the same groups of Applicants interviewing at a handful of places. Top US grads keep seeing each other on interviews, Mid-tier grads keep seeing each other.... What you end up with is pockets of similar ROLs.

Now, take any one program - say JHU Internal Med. Say they interview 250 applicants. Say 200 of those applicants rank JHU at #1. Now, it simply does not matter that these 200 applicants ranked JHU #1. JHU will fill based solely on their own ROL.

Combine the three phenomena - similar pockets of applicants, and relatively similar ROLs, and coveted programs filling entirely under their own ROL. and see it like a physicist sees a cow. Each residency program can be pegged at #1 literally hundreds of times. But, Each residency program only has a single #1 Applicant. Realize that again, a similar phenomenon, of the Applicants ROL being inconsequential, could occur in the mathematical output.... especially as the Applicant drops down on their ROL, the in-consequentiality could increase. "But the algorithm 'favors the applicant'", you say. Yes... all the applicants... equally... not just you.

I bring this up because I had a similar discussion with faculty. Faculty have noticed that in the last couple of years, students match within their top4 as per the usual ranking-advice and outcomes, OR, they match somewhere at 12-13-14. But, there seems to be nothing in between. It seems, that the applicants match where they want, or they match where they are wanted. But theres nothing in between. I brought up the usual ranking-advice... rank where you prefer, and you'll drop down one by one. The response was - "You wont match in the middle of your list because you didnt rank them high enough, but many others did." It followed that, the reason people seemed to match in the teens was only because the Program ranked them #1 or 2, and no applicant could unseat them. And so, the above advice in red was posited.

If you've bothered to read this long post. I'd urge you not to make the assumption that you know what the Match algorithm is, and that it will work the same with the recent increases in applications. The best advice for a mediocre applicant may in fact be - give some consideration to where you will be ranked.

As Billy suggested above - If you are an unimpressive applicant and would rather match low than have to scramble - Ranking a program that you think you've got a good chance with higher up might "lock you in". It might not be the case that you'd just as well have matched there even if you ranked them low. Remember: the current alogrithm was tweaked to "favor the applicant", when sheer numbers seem to favor the programs. That probably means - favor the applicants expressed choices, rather than match the most applicants. The usual thought that one will match if they were ranked high by the program (due to the second-pass of unmatched applicants unseating tentative-match holders from program where they are ranked higher) can not be so simplistic. Were it so simple, the 46.9K applicants would continue to bump each other off until Match results resembled the mere 3.6K programs' ROLs. The current version of the Algorithm "favors the applicant". So it must contain some branch in the decision tree which gives more weight to where the Applicant ranked the program - such that when an applicant ranks a program low, they don't match there, until & unless they are ranked very (~#1) high by the program.
 
Last edited:
Reductio, the match algorithm is pretty simple, not some crazy non-Nash equation.

The algorithm starts at your #1 program. If you don't match at #1, your #2 program takes the place of your #1 spot. The process repeats until your ranks are done. If you rank 100 programs, and program #99 ranked you to match, you are GUARANTEED to match at program 99 (if you fall that far). The algorithm places no weight based on how high you rank a program.

The difference with the old algorithm was that it would start from the programs side, rather than the applicants side.

I think the phenomenon you described of people either matching at 1-4 or 11-15 is because programs 5-9 are likely to be of similar competitiveness to 1-4, and applicants are likely overqualified for 11-15 (included them as "safeties"). So, maybe a student fell because of a red flag, or bad interview skills. they are likely to fall to programs they are overqualified for.

Anyways, rank programs in the order you want to go. There is absolutely no benefit to ranking a safety program higher to increase your chances to match
 
Last edited by a moderator:
Reductio, the match algorithm is pretty simple, not some crazy non-Nash equation.

The algorithm starts at your #1 program. If you don't match at #1, your #2 program takes the place of your #1 spot. The process repeats until your ranks are done. If you rank 100 programs, and program #99 ranked you to match, you are GUARANTEED to match at program 99 (if you fall that far). The algorithm places no weight based on how high you rank a program.

The difference with the old algorithm was that it would start from the programs side, rather than the applicants side.

I think the phenomenon you described of people either matching at 1-4 or 11-15 is because programs 5-9 are likely to be of similar competitiveness to 1-4, and applicants are likely overqualified for 11-15 (included them as "safeties"). So, maybe a student fell because of a red flag, or bad interview skills. they are likely to fall to programs they are overqualified for.

Anyways, rank programs in the order you want to go. There is absolutely no benefit to ranking a safety program higher to increase your chances to match


Bear with me here... Say you fell to #99. The reason you did match was because you were ranked "to match", (say #1) by the program. Say I ranked the same program at #1, but I matched lower at my #75. Reason must be, that once you fell to #99, you bumped me off because you were ranked #1 by the program. Even though I preferred the program more than you did, you still matched there because you were "ranked to match". So, that doesn't really favor the applicant (me). It actually favors the program, because they got their #1 (you), as opposed to someone who wanted them (me). You might say that it favored the applicant from your perspective. But I dont think it can work like that...

In this (the commonly held) scenario, as far as decision points go, it (1)favored the program by taking you over me, and (2) un-favored me by dropping me lower on my list though I preferred the program more than you did. It favored the program over the applicant.

That's why I dont think the commonly understood version of the algorithm can be complete.
 
Bear with me here... Say you fell to #99. The reason you did match was because you were ranked "to match", (say #1) by the program. Say I ranked the same program at #1, but I matched lower at my #75. Reason must be, that once you fell to #99, you bumped me off because you were ranked #1 by the program. Even though I preferred the program more than you did, you still matched there because you were "ranked to match". So, that doesn't really favor the applicant (me). It actually favors the program, because they got their #1 (you), as opposed to someone who wanted them (me). You might say that it favored the applicant from your perspective. But I dont think it can work like that...

In this (the commonly held) scenario, as far as decision points go, it (1)favored the program by taking you over me, and (2) un-favored me by dropping me lower on my list though I preferred the program more than you did. It favored the program over the applicant.

That's why I dont think the commonly understood version of the algorithm can be complete.

Well, in your scenario it favored one applicant at the expense of another. That doesn't mean the algorithm doesn't favor applicants, in fact, The algorithm maximized the amount of people matching overall because both applicants were able to match somewhere in your scenario. The kid who matched at #99 may have bumped you from your list, but he was able to match SOMEWHERE because it only takes one program to get in somewhere, which is how it should work, IMO.

The match doesn't favor applicants in that every applicant gets what they want. It is still a match, close to 50-50 weight on the applicant and the programs side. The applicant gets a slight edge because they get the first say in the order of programs to choose.

Either way, all I wanted to convey was that you don't stand to lose anything by ranking a program you consider a safety program as last on your rank list. You don't increase your overall chances to match by putting it higher.
 
Last edited by a moderator:
The scores on these tests (Step 1 or 2) mean very, very little.

On the USMLE, the Standard Deviation is ~20. This means that everyone scoring from 210 to 230 is statistically the exact same score (within 1 SD of the mean of 220).

The COMLEX is even worse with a Standard Deviation of 80 (but a wider point spread). On the COMLEX a 460 to 540 are the same score (within 1 SD of the mean of 500).

So, if there is no statistical difference between Candidate A with a 210 and Candidate B with a 230, why do we care so much about these stupid scores. I mean, I guess there is a difference between a 200 and a 240, but so few people get those scores. Most of us are in the 210-230 range, and are thus, the same. Hmmm, maybe that's why we all wind up making good doctors....
 
Last edited:
The scores on these tests (Step 1 or 2) mean very, very little.

On the USMLE, the Standard Deviation is ~20. This means that everyone scoring from 200 to 240 is statistically the exact same score (within 1 SD of the mean of 220).

The COMLEX is even worse with a Standard Deviation of 80 (but a wider point spread). On the COMLEX a 420 to 580 are the same score (within 1 SD of the mean of 500).

So, if there is no statistical difference between Candidate A with a 200 and Candidate B with a 240, why do we care so much about these stupid scores. I mean, I guess there is a difference between a 180 and a 260, but so few people get those scores. Most of us are in the 200-240 range, and are thus, the same. Hmmm, maybe that's why we all wind up making good doctors....

The scores certainly matter for gaining interview offers for competitive residencies at university programs. 200-240 (or 2 standard deviations based on your number) is a huge range, covering 68% of all test takers. I think the student who scored in the 84th %ile has more medical knowledge than the one who scored in the 16th %ile. Good luck getting radiology or derm interviews at big academic centers with a 200 on Step 1.

I agree with you that the test is a terrible, terrible way to predict who will make good doctors, but the test has been shown to correlate with residents ability to pass the boards. That is why these tests are used as screening tools. Competitive programs can hand pick their residents, so they only interview a group of people who they know are most likely to pass the boards. Not that there aren't other ways to compensate for a low step 1 and gain interviews, but the test is important for programs.
 
The scores certainly matter for gaining interview offers for competitive residencies at university programs. 200-240 (or 2 standard deviations based on your number) is a huge range, covering 68% of all test takers. I think the student who scored in the 84th %ile has more medical knowledge than the one who scored in the 16th %ile. Good luck getting radiology or derm interviews at big academic centers with a 200 on Step 1.

I agree with you that the test is a terrible, terrible way to predict who will make good doctors, but the test has been shown to correlate with residents ability to pass the boards. That is why these tests are used as screening tools. Competitive programs can hand pick their residents, so they only interview a group of people who they know are most likely to pass the boards. Not that there aren't other ways to compensate for a low step 1 and gain interviews, but the test is important for programs.

Oh, sorry. I was forgetting to split the SD...my above post has been corrected, although my original point still stands. If there's no difference between a 210 and a 230, why do we pretend like there is?
 
Oh, sorry. I was forgetting to split the SD...my above post has been corrected, although my original point still stands. If there's no difference between a 210 and a 230, why do we pretend like there is?

Because program directors believe that there is a difference. You are fooling yourself if you think they don't care about it. Hell, I'm willing to bet theres a big difference between a 230 and a 229 in their eyes...

As I said before, I agree that the test is meaningless to predict who will be a good doctor. But it actually does a very good job of predicting ability to pass the boards. The test isn't a fluke like the MCAT; if you take it twice, you aren't going to go from a 210 to a 230, and a 230 correlates with a better chance to pass board exams (especially tough ones like radiology, dermatology), than a 210 does.

To say that the scores mean very little is just plain wrong, because many who aspire to match in plastics, rad onc, and derm have to start thinking about other career choices if they don't score highly enough on the test.
 
Last edited by a moderator:
Because program directors believe that there is a difference. You are fooling yourself if you think they don't care about it. Hell, I'm willing to bet theres a big difference between a 230 and a 229 in their eyes...

As I said before, I agree that the test is meaningless to predict who will be a good doctor. But it actually does a very good job of predicting ability to pass the boards. The test isn't a fluke like the MCAT; if you take it twice, you aren't going to go from a 210 to a 230, and a 230 correlates with a better chance to pass board exams (especially tough ones like radiology, dermatology), than a 210 does.

To say that the scores mean very little is just plain wrong, because many who aspire to match in plastics, rad onc, and derm have to start thinking about other career choices if they don't score highly enough on the test.

You're misunderstanding me. I know people CARE about the score. Statistically speaking, though, PDs shouldn't. There is no statistical difference between a score of 210 and one of 230.
 
You're misunderstanding me. I know people CARE about the score. Statistically speaking, though, PDs shouldn't. There is no statistical difference between a score of 210 and one of 230.

I screw up basic statistics all the time, but I believe you've confused Standard Deviation / Standard Error here.

All tests can be described in terms of reliability and validity. Reliability = how accurate is the test measure, or how likely is a certain individual to get the same (or similar) score if they take the test repeatedly. Validity = how well does a score on the test relate to whatever you're trying to predict with the test.

When comparing two scores, we need to address both reliability and validity.

The "reliability" of the USMLE would ask -- if the person who scored a 210 and a 230 were to take the test repeatedly (without additional studying, etc), how likely would it be that they would get the same score? In answering that question, we need the Standard Error of Measurement, which predicts exactly that. Note that the USMLE "predicts" this as they cannot have people take the test repeatedly. Instead, they have multiple questions that are similar, and compare how likely the similar questions are answered the same. In any case, the SEM for the 3 digit score is 4-8 points. Even at the extreme of 8, 210+8=218 and 230-8=222, these two scores are "significantly different" based upon the reliability of the test.

The Standard Deviation, on the other hand measures the overall distribution of the population of test scores. It would be used to calculate a percentile, for example.

Whether the test has any "Validity", meaning that even though a 230 is a "statistically better" score than a 210 whther that translates into being a better physician, is a difficult question to answer.

I am prepared to be flamed / corrected by anyone who actually understands statistics.
 
I screw up basic statistics all the time, but I believe you've confused Standard Deviation / Standard Error here.

Standard Error is just an estimate of Standard Deviation. I hate it, but wikipedia is the clearest explanation I can find without my stats textbooks handy:

http://en.wikipedia.org/wiki/Standard_deviation
http://en.wikipedia.org/wiki/Standard_error_(statistics)

The entry on SD has this to say:

In science, researchers commonly report the standard deviation of experimental data, and only effects that fall far outside the range of standard deviation are considered statistically significant – normal random error or variation in the measurements is in this way distinguished from causal variation.

Which is how I learned SD, and how I'm interpreting it above. I would also welcome the corrections of any REAL statisticians, as I only took 2 classes in it, and they have both since been replaced with med school.

That being said, my point still stands that doctors (and program directors) are NOT statisticians, so how can we be expected to come up with correct conclusions based on the scores thrown at us by the shelf exams, without a statistician on staff, that is.

I mean, we still have PD's and Students who think the 2 digit score is a percentile or percent correct. I mean, come on.
 
In answering that question, we need the Standard Error of Measurement, which predicts exactly that. Note that the USMLE "predicts" this as they cannot have people take the test repeatedly. Instead, they have multiple questions that are similar, and compare how likely the similar questions are answered the same.



so THATS why they did that... +pissed+
 
Standard Error is just an estimate of Standard Deviation. I hate it, but wikipedia is the clearest explanation I can find without my stats textbooks handy:

http://en.wikipedia.org/wiki/Standard_deviation
http://en.wikipedia.org/wiki/Standard_error_(statistics)

The entry on SD has this to say:



Which is how I learned SD, and how I'm interpreting it above. I would also welcome the corrections of any REAL statisticians, as I only took 2 classes in it, and they have both since been replaced with med school.

That being said, my point still stands that doctors (and program directors) are NOT statisticians, so how can we be expected to come up with correct conclusions based on the scores thrown at us by the shelf exams, without a statistician on staff, that is.

I mean, we still have PD's and Students who think the 2 digit score is a percentile or percent correct. I mean, come on.

I don't mean to be abrasive dig (but given you are acting like a complete dick to aPD I have no problem with it), the amount of defense you take over this issue makes it seem like you have some major insecurities over your performance on Step 1.

I think everyone here has agreed with your initial point: Step 1 is useless to predict performance as a physician.

You are confusing some major concepts here: reliability, validity and significance.

Reliability:
The ability of a measurement (or exam score) to be consistent if repeated by an individual

Step 1 is very good at this

Validity:
The ability of a measurement corresponds to the real world (ie. predicting future job performance)

Step 1 doesn't claim to do this

Statistical significance :

The chance that the outcome was not the result of random chance (with 95% certainty if your alpha is .05).

So, in a test like Step 1, if you score a 210, there is a 95% chance that the score wasn't a result of random chance. Same with a 230. The scores are what they are.

Sure, there is a 5% chance you got a 210 because of a random screwup, but that doesn't mean there is no true difference between a 210 and 230.
 
Last edited by a moderator:
I don't mean to be abrasive dig (but given you are acting like a complete dick to aPD I have no problem with it), the amount of defense you take over this issue makes it seem like you have some major insecurities over your performance on the Step 1.

I think everyone here has agreed with your initial point: Step 1 is useless to predict performance as a physician.



The Standard Deviation is a measure of variance from a mean value. It is NOT a measure of statistical significance.

Statistical significance ONLY refers to the chance that the outcome was not the result of random chance (with 95% certainty if p<.05). So, in a test like Step 1, if you score a

So, you are arguing that a 210 and 230 on Step one are not statistically significant, and therefore, due to random chance.

I'm really sorry if it's coming off that way...must be all the board studying.

I get what you're saying, but we use the SD to compute the confidence interval.

Regardless, I'm not saying that I understand statistics well enough to fully analyze what the scores do or do not mean...but I am saying that I don't really think anyone does and we rely on them too heavily when comparing applicants. Great docs can have crappy scores and vice versa. So, why compare that way?

I know the counter-argument..."If we don't use the Board Scores, what else do we use?"...but I think that's the job ahead: to find something better.
 
Let me put this another way:

-Imagine a normal distribution of Americans and their height. Say, the mean is 5'5 and the standard deviation is 4 inches. Now, you have a person who is 5'1 and a person who is 5'9 (each within one standard deviation). Are you really going to argue that there is no difference in height between these two people just because there is no statistically significant difference?

There is a real difference between upper and lower limits of standard deviations in a normal distribution, whether a standardized test or height measurement.
 
Let me put this another way:

-Imagine a normal distribution of Americans and their height. Say, the mean is 5'5 and the standard deviation is 4 inches. Now, you have a person who is 5'1 and a person who is 5'9 (each within one standard deviation). Are you really going to argue that there is no difference in height between these two people just because there is no statistically significant difference?

There is a real difference between upper and lower limits of standard deviations in a normal distribution, whether a standardized test or height measurement.

That sounds right to me...but while there is an absolute height difference, as you pointed out, it's not statistically significant...and if it's not statistically significant, who cares. Would you use a treatment based on a study that showed that the use of that drug was not statistically significant?

There might be a real difference in the numbers (or height, or scores), but if that difference isn't statistically significant, then what does it matter. Does being tall (or having a high Step 1/2 score) make you a better doctor? No? Then why do we care so much?

Oh right, because it's all about how likely it is that you'll pass Step 3, or your specialty boards. If that's the case, maybe we should stop designing all of our tests to predict how well we'll pass the next test, and start designing them (including the final one) to make better physicians.

I'm really sorry that I'm ranting a bit...I'll stop, promise. I'm just sick of studying Step 2 zebras and would rather get back to things that actually matter. :D I just wish people would stop basing so many decisions on these (relatively) worthless tests. Competency, yes. Our entire lives and careers...not so much. It's a good test, but not that good.
 
I don't mean to be abrasive dig (but given you are acting like a complete dick to aPD I have no problem with it),

Ive been reading the SDN for years, and just recently decided to post. Im regretting it already.

Only on the SDN do we call out program directors, and say that we'd rather not have their opinion. Then we ask for a statistician... blatantly assuming that said PD doesnt have an graduate degree in epidemiology or biostatistics.

Standard deviation is a measure of variability

p values measure statistical significance

SD is only a good measure of statistical significance when youre saying that Applicant X is "significantly" better than average. In which case, Applicant X must be 2 Standard Deviations above the mean, which if I remember correctly, is score of 240 on the USMLE.

EDIT: 2SD above the mean could be as much as 260+ on the USMLE, depending on the round of testing. Seems like usually, 2SD above the mean is 240-260+, depending...

All you 240 kids don't feel like such hot-shots now, do you??
 
Last edited:
Ive been reading the SDN for years, and just recently decided to post. Im regretting it already.

Only on the SDN do we call out program directors, and say that we'd rather not have their opinion. Then we ask for a statistician... blatantly assuming that said PD doesnt have an graduate degree in epidemiology or biostatistics.

I wasn't trying to "call out" anyone. Nor did I say I'd rather not have their opinion. I would love their opinion. I was just trying to start a discussion about whether the test is a good statistical judge of a medical student, which I don't think it is.

I'm sure there are many PD's who understand this much better than I do, and I'm also sure that aPD is one of them. I've always found aPD's posts informative and helpful. I'm also sure that there are PD's with good statistical training, which is ultimately why I posted, hoping someone could shed some light on why we care so much about this test, when it seems, to me, to not be a very good statistical instrument. I'm hoping someone with more experience can point out my stats ignorance.

Thank you for your supporting stats points though! That seems to correlate well to my buried stats memories...

Anyways, it's ultimately a silly and pointless discussion. The scores DO matter to PDs, so I better get back to work. I do hope that one day, though, medical students are judged based on how good they'll be as clinicians and less on if they know every Zebra in Harrison's.

We are fast approaching (or past) the point where it's possible to memorize every medical fact known to man. The emphasis in testing needs to shift away from rote memorization and application to identifying, interpreting, and applying information and relating to patients well...

...There I go again. Back to work. :p
 
Anyways, it's ultimately a silly and pointless discussion. The scores DO matter to PDs, so I better get back to work. I do hope that one day, though, medical students are judged based on how good they'll be as clinicians and less on if they know every Zebra in Harrison's.

We are fast approaching (or past) the point where it's possible to memorize every medical fact known to man. The emphasis in testing needs to shift away from rote memorization and application to identifying, interpreting, and applying information and relating to patients well...

...There I go again. Back to work. :p

EVERYBODY agrees with you on this. The trouble is, we/PDs have no wayto tell how good someone will be as a clinician - especially at a 20 minute residency interview. Even LORs mean very little, because they probably all say the same things. And, what what one attending might think makes a good clinician is very different from what another might think.

So maybe PDs figure "All I need to know is - can this guy 1) be trained/educated, 2)pass the boards 3) work well with us." The rest is up to us.

Now that the thread has been totally derailed.....
 
First, and most importantly, no one is "calling me out" here. I don't see anything disrespectful about anyone's response to my post here. And, my ego is plenty healthy so that I can handle criticism. Plus, I can always block anyone who really annoys me.

Anyhow, I do NOT have a PhD in statistics. In fact, as stated, I suck at statistics. My entire statistics experience includes:

1. A stats course in high school (extra math course for us math nerds)
2. Some sort of biostats in medical school, which was much less rigorous than #1
3. This website, which completely oversimplifies the issue.

I have probably misspoken about standard error. Still, we are all missing something here.

Let's take the example of height. If you measure the height of all adults in the US, you'll get a nice normal distribution. It will have a mean (the average height), and a standard deviation which describes how much "spread" is in the population. As mentioned above, let's assume that the mean is 5'5" and the SD is 4". That would mean that 68% of the population would be between 5'1" and 5'9", and 95% would be between 4'9" and 6'0". All is good with the world. However, this does not mean that someone who is 4'9" is not a "statistically different" height than someone who is 6'0".

Here's why:

Let's say I want to measure you. I take my measuring tape, and find that you're 5'6". Then, someone else measures you with the same tape, and finds you're 5'7". A third person gets 5'5 1/2". And so on... That's because, no one really can know exactly how tall you are. Everytime you get measured, there's a bit of error involved. However, after multiple measurements, all of those measurements plotted will form a normal distibution, the mean of which is a best estimate of your "true" height. The more measurements I make, the more accurately I can predict your true height. The normal distribution of these multiple "samples" of your height have a mean and an SD also -- and that SD is what I meant when I said "standard error". To tell whether two different people are the "same" height or not, you need to compare the differences of their heights to the SD (or standard error) of the MEASUREMENTS, not the SD/SE of the population.

The same is true for the USMLE. If you want to know if two scores are statistically different, you need to know the error in the measurement (i.e. how likely your score is to represent your "true" score), not the distribution of the population. The USMLE score report does report both of these -- the SD of the popuation is about 20 points, and the standard error of measurement is 4-8 points.

Again, I'm certain I'm using the wrong words to describe this.
 
EVERYBODY agrees with you on this. The trouble is, we/PDs have no wayto tell how good someone will be as a clinician - especially at a 20 minute residency interview. Even LORs mean very little, because they probably all say the same things. And, what what one attending might think makes a good clinician is very different from what another might think.

So maybe PDs figure "All I need to know is - can this guy 1) be trained/educated, 2)pass the boards 3) work well with us." The rest is up to us.

Now that the thread has been totally derailed.....

Nah. Boards weigh in heavily.

Even stellar LORs from big names within the hospital/program you're applying doesn't matter as much. Boards and sometimes the school you come from can make quite a bit of difference.

PDs like to say "this is what I want" and name those 3, but that's just interview-day fluff.
 
Nah. Boards weigh in heavily.

Even stellar LORs from big names within the hospital/program you're applying doesn't matter as much. Boards and sometimes the school you come from can make quite a bit of difference.

PDs like to say "this is what I want" and name those 3, but that's just interview-day fluff.

I think you missed the point.

Step 1 and school are useful and universal screening tools because there's simply no way to interview everyone who applies. They also function as surrogates for the "educatable/trainable" and "board passing" issues. If you can get into (and graduate from) a good school and score well on standardized tests, the answer to those two is most likely "yes."

The 20 minute interview is then left to determine if the applicant is a sociopathic d-bag or not (which is point 3).
 
Top