Important Considerations for Diversity in the Selection of Dermatology Applicants

[Oussedik]

Some of you might like the recent manuscript I published in JAMA Dermatology on rethinking the paradigm on how dermatology applicants are selected. Even though I completed a research fellowship in dermatology, I think medical students should not be disadvantaged in the residency selection process for not having completed a research fellowship. See link below (redirects to JAMA Derm)

Diversity in the Selection of Dermatology Applicants

---

Members don't see this ad.

 

[werd]

congrats on the publication. hopefully research fellowship applicants will not outcompete others so routinely in the future.

 

[dermmom17]

Thanks. Interesting ideas. Diversity and lack of research year between 3rd and 4th year could have been reason my family member didn't match.

 

[Sophia88]

Glad someone brought attention to this important issue. At many institutions sometimes multiple years of unpaid fellowships are becoming the new norm. I encountered this as a med student in an area where cost of living was ~40k for the year. Nearly impossible for someone survive on a unpaid year unless they have significant money saved or family support. The path to becoming a physician is expensive and long enough we don't need to add even more hoops to jump through.

 

[Tamahawk]

Some of you might like the recent manuscript I published in JAMA Dermatology on rethinking the paradigm on how dermatology applicants are selected. Even though I completed a research fellowship in dermatology, I think medical students should not be disadvantaged in the residency selection process for not having completed a research fellowship. See link below (redirects to JAMA Derm)

Diversity in the Selection of Dermatology Applicants

Great article, sir! Thanks for sharing and raising this important topic for discussion in the medical literature and on this board. I am one of the folks that agree that taking an extra year helps your chances, so I took out loans to be able to do so. I feel that it did certainly help in my application process, as many programs positively commented upon my research year productivity during interviews, and I was able to secure a better LOR than I otherwise would've if I just did a 1 month derm rotation.

While I agree it is unfair that I was at an institution that had great access to mentors and research opportunities, and unfair that I was able to take a loan to take the year off, I don't know that this is something that can or necessarily should be "corrected for" during the application process. That extra year of research did give me a stronger background, more publications and genuinely closer relationships with my home institution faculty; the Derm faculty got the opportunity to see my work ethic as well. To NOT take into consideration these types of experiences would be akin to NOT taking into account the medical school that one attends (economic status certainly affects where people can get into and/or afford to matriculate), grades or USMLE scores during clinical years (since rich kids can afford resources that poorer ones cannot), or away rotations (because rich kids can more easily afford to travel). More bluntly stated:
1) Being rich is helpful along every step of the way, but doesn't necessarily assure you of success.
2) Being poor makes every step much harder, but doesn't rule out the possibility of success.
3) While we're being 100, minorities in medical school are poorer, on average, than their white counterparts.

I am neither white nor rich, but did recognize what it takes to be able to be competitive in this inherently unfair system. As I tell my children daily, the world isn't fair...it's up to you to figure out the rules of the game and adapt around them.

 

[Oussedik]

I am neither white nor rich, but did recognize what it takes to be able to be competitive in this inherently unfair system. As I tell my children daily, the world isn't fair...it's up to you to figure out the rules of the game and adapt around them.

I couldn't agree more!

 

[lampshade4u]

and if you already have X number of publications is there any merit in a research year? If two students both have exact same application except one took a research year and the other didn't, who is at the advantage? I would say the student who got the same work done without having to take the year off to do it...

 

[deleted480308]

It's a competition for a highly sought after spot, you have to outrun the competition and right now that usually means an absurd score or a research year. It is what it is...

 

[lampshade4u]

It's a competition for a highly sought after spot, you have to outrun the competition and right now that usually means an absurd score or a research year. It is what it is...

but would you say the research year is valuable in that you did a research year or that you can publish more? If I already have 25+ publications in the field, is there any sense in doing a research year?

 

[deleted480308]

but would you say the research year is valuable in that you did a research year or that you can publish more? If I already have 25+ publications in the field, is there any sense in doing a research year?

I was replying more to the OP.

If you have an absurdly high score and a lot of research, the only benefit I would see to doing a research year would be if it was with someone who made residency selection decisions so you could get an "in" with them. But I'm not a derm guy, that's just observational

 

[DermADVISOR]

Glad someone brought attention to this important issue. At many institutions sometimes multiple years of unpaid fellowships are becoming the new norm. I encountered this as a med student in an area where cost of living was ~40k for the year. Nearly impossible for someone survive on a unpaid year unless they have significant money saved or family support. The path to becoming a physician is expensive and long enough we don't need to add even more hoops to jump through.

Yes

 

[DermADVISOR]

1) Being rich is helpful along every step of the way, but doesn't necessarily assure you of success.
2) Being poor makes every step much harder, but doesn't rule out the possibility of success.
3) While we're being 100, minorities in medical school are poorer, on average, than their white counterparts.

I am neither white nor rich, but did recognize what it takes to be able to be competitive in this inherently unfair system. As I tell my children daily, the world isn't fair...it's up to you to figure out the rules of the game and adapt around them.

Your statements aren't 100% accurate. Without money in some circumstances like this one, "adapt around them" isn't possible. Not saying it is wrong or wrong the way things are.

 

[dermathalon]

Fact of the matter is that removing the research year fellowship component is just not possible and is borderline absurd. If a student works super hard over the four years and publishes a ton, then a research fellowship is not likely needed. How many of these types of students do you see...not many. It's just not the same as a research fellowship. You are basically asking someone to remove the hard work and dedication of a student working over the course of a year on projects and the success that they achieve. While I respect the viewpoint of the OP, it is utopian. Let's first work on getting rid of Step 1 filters (which are purely ridiculous as they have little bearing on being a good doctor) or the love for Step 1 scores. Just because you take a year off does not mean that you will get pubs. On the other hand, if you bust your ass you will get a lot of pubs if you have the right mentor and the right attitude. That's the resident that I want and who cares if they go into research or not. It's just that they have that work ethic in them. If students have more pubs, they have more pubs. It is what it is. That said, disadvantaged student grants need to be made more available but that's not helpful to make these suggestions for the student that is applying today or in the next year. What are we going to tell them to do? Naturally, to take a year off and focus on all of the benefits that it brings. Hopefully, faculty have the ability to fund or perhaps they can set aside salary to at least help the students make rent (you can do this in tax-deductible fashion in a university)...that will tell you how seriously the faculty are willing to support a student. Departments to make an effort to support medical students if they are serious about underrepresented minorities Put your money where your mouth is. The real solution is to support these students and be creative to open up funds if you think a student is worth it. And yes, you will be indebted to a student that works really really hard and does a good job. That's called real life and choosing someone with great work ethic. If they flub the interview then it's on them. Take the horse to water but....you know the rest.

Here is the harshest truth: inside of a residency selection meeting, the committee doesn't have to answer to anybody. They can rank whomever they want. We can scream diversity until we are blue but nothing changes until we change our mindset. Stop the deranged love for Step 1 and we will start to see more diversity in residencies and be better off for it. Take a look at that student with at Step in the 210s or 220s that has truly struggled and has a heap of humility and empathy and you'll pick an amazing doc (who will pass the boards as well...we have the crazy notion that they won't pass the boards..not true because it's all about work ethic at that point). I know a lot of folks that got 220s and passed the boards just fine. If we are willing to give this break to PhDs (many of whom will never go into research anyway) then why not give concessions to those students that can make an impact in their communities that will make dermatology better off with greater diversity.

However, setting a blind eye to the research year is not going to solve the issue nor is it fair to those students who bust their ass during this year.

 

[mohsable]

Stop the deranged love for Step 1 and we will start to see more diversity in residencies and be better off for it. Take a look at that student with at Step in the 210s or 220s that has truly struggled and has a heap of humility and empathy and you'll pick an amazing doc (who will pass the boards as well...we have the crazy notion that they won't pass the boards..not true because it's all about work ethic at that point). I know a lot of folks that got 220s and passed the boards just fine.

I agree with most of your points about how a research fellowship/year can showcase a student’s work ethic and should not be discounted. However, I’m struggling to understand the logic behind your arguments about step scores. It seems that you believe a low step 1 score leads to or is associated with humility/empathy, work ethic, and passing the boards. However, other than personal anecdotes, I’m not sure there is much evidence to support these claims.

For instance, even if your argument is true, they don’t exclude the possibility that a high step 1 score can be achieved by a humble/empathetic person with a strong work ethic who will pass the boards as well. In fact, the person with the high step 1 score has proven more in terms of work ethic and ability to score well on exams. In terms of humility, I’m not convinced that having a person humbled by their low score is more favorable than having a person with a high score who feels that their hard work has paid off and is now even more driven for future tests.

The comments about diversity are interesting but flawed. Having a mix of step 1 scores in a program doesn’t necessarily improve diversity, just as having only high step 1 scorers doesn’t necessarily reduce diversity. Sure, some people with low step 1 scores may come from less privileged backgrounds, but so too can people with high scores.

To dive a little deeper, I get the arguments about “systematic oppression,” but I do feel like once students are in medical school it’s a more level playing field. Are the low step 1 scoring students disadvantaged in some way in their studying? Are they working a million side jobs and unable to afford First Aid or uworld? I pose these questions because for the most part, I fail to see the connection between low scoring students and their supposed backgrounds. The only real conclusion I can draw is that they probably don’t score well on standardized tests.

Ultimately, if passing the boards is of utmost importance to programs, why should they take a gamble?

 

[dermathalon]

Thanks for continuing this discussion because I think it's an important one.

High step score + humility are wonderful and nothing I said precludes that. What I said was "Take a look at that student with at Step in the 210s or 220s that has truly struggled and has a heap of humility and empathy and you'll pick an amazing doc." My point isn't that people with high step scores couldn't be humble. Humility and empathy are characteristics and not learned from scoring low on Step 1. I see that you are looking at it as being humbled by the low step score. That couldn't be further from my point. My point is that you can find amazing docs from the lower step score pool who have the qualities that are needed to be outstanding derms but nothing precludes the high step scoring students from having these qualities too. On the flip side, there will be many students that have a low step score and do not have humility either. Low step score does not automatically mean that there is humilty and empathy. Those qualities are built from life experiences and not from simply scoring low on step 1 (I see you misinterpreted otherwise).

I never choose against a student that has scored really well on step 1, done amazing in school, has humility and empathy, tons of research...basically a rock star student...who wouldn't want someone like that? But those students are far and few in between.

Diversity does not refer to step 1 diversity. It refers to diversity overall. I don't care about having a mix of step 1 score for the sake of having a diversity of step 1 scores (again, I see that you were interpreting that I was referring to diversity in terms of step 1 scores). So on that point, I'm in agreement. I'm talking about overall diversity.

The answer to your question: "Are the low step 1 scoring students disadvantaged in some way in their studying?" is maybe and that is the point. I think the biggest erroneous belief is that all medical students are on a level playing field. They are not. This is not specific to medicine. It's utopian to think that they are. We want to idealize that we are all on the same level playing field but we are not. That doesn't mean that you handicap the grades. You still put everyone in the same grading schema. But you do have to account for it and acknowledge that it truly exists. People have very different living environments, different stresses at home, different responsibilities and this is not automatically erased by getting into medical school.

Finally, the question of gambling is an interesting one because it is an anecdotal argument. In fact, studies have shown that those that come in with an especially low board score have a lower fail rate probably because they are scared to fail the boards. It's the ones that don't bother studying that fail. Some of this research came out of Ohio State so you can look up those manuscripts for yourself. Steps are correlated with in training exam scores but there was not much correlation when looking at other factors that are more subjective that are vital to being a good dermatologist. Also, correlation with ITEs does not mean people will fail the boards more. ITE are reported as a continuous percentage while boards are pass/fail. Very few fail and it's not because their step 1s were low. There are other factors involved there. I know some that failed because they didn't study, some got sick, some had anxiety issues...nothing to do with cognitive ability (for what it's worth, all were high scorers on Step 1). Sure, someone with a low step 1 could also fail but my point is that it's more correlated to other factors than simply a step 1 score. In fact, the studies in derm have shown that the correlation to step 1 only accounts for about 26% of the variability in ITE scores whereas the rest of the variation is attributed to other factors.

My question is why would I gamble on someone that is difficult to work with? This can be someone that has either a high or low board score so I'm not biasing against it. My point is that board scores can't measure this so why are we missing out on potentially great people out of the gate with these cutoffs.

I scored high 99% on the step 1 but I'm not enamored with these tests. If anything the studies have shown that there is more of a correlation to step 2 rather than step 1 when thinking about boards, but such as study has not been done in derm. Even then, the correlation appears weak. Thankfully our patients are not multiple choice exams.

 

[mohsable]

You bring up good points, but you misunderstood my points about diversity. I too am referring to overall diversity. You seem to suggest that accepting students with low step 1 scores would somehow improve diversity compared to the status quo. My argument is that even among the high scoring students you would already have a significant degree of diversity (i.e., high scoring students are not all the same in terms of backgrounds and experiences), such that programs do not really need to go searching very far for acceptable candidates. In short, I believe your argument only holds true if programs are sacrificing diversity by favoring high scores--which I am not convinced is true.

Your answer to my question about whether low scoring students are disadvantaged in some way in their studying is still problematic in my opinion. Without downplaying the very real differences in living environments, life stressors, and responsibilities that people face, I don't believe there is a way to truly "account for it" in the selection process. Low scores occur for many reasons, among which include those aforementioned differences, but also poor test taking ability, subpar preparation, and sometimes bad luck. Furthermore, if a student has willingly or unwillingly allowed the stressors in their life to affect them in such a way that they score poorly on a board exam, what's to say these same stressors don't negatively influence them in every subsequent test they take or their clinical performance during their training and beyond? That's the gamble that programs face, as I mentioned previously.

Do you have the citations for the studies that show those with low board scores have lower fail rates? It does seem believable, but it is not something I am familiar with. Like you mentioned, all the studies I have seen, including the OSU study in JAAD 2011, have found moderate correlation between USMLE and ITE scores. Obviously, step 1 scores do not correlate with being a good dermatologist. But, neither do medical school prestige, publications, letter writers/connections, or many other factors that are also considered. I just don't think any other identifiable/tangible factor accounts for more variability in ITE scores than the step 1 score, which is why it is used in this way to protect programs.

I agree that there are some amazing people among the group that will be eliminated from consideration on the basis of their scores. However, if there are enough diverse, interesting, likable, and high scoring people to fill most of the residency spots, the programs don't actually experience enough of a loss (if any) that would justify changing the current system. To me, the only true loss is felt by low scoring students.

 

[dermathalon]

I respect your position on this and you bring up very valid points. I think the point where we disagree is the "diversity" in our opinions.

I personally think you do lose out when you use step 1 filtering because we know that there are many factors involved in regards to making a good dermatologist. If we are so quick to tell our patients that treatment X is not a justified treatment when there is poor evidence for it then why are we quick to assume that step 1 scores are a good predictor of passing the boards when there is poor evidence for that. Seems like we choose a side that suits us.

So I do feel that using Step 1 as a filter is not justified when we know that there are many different factors with Step 1 as nothing more than an ok correlation (and only to the ITE and not the boards itself). We choose step 1 because it's easy. Many tell me that it is because "we have to start somewhere" which to me sounds like laziness when other approaches could have been used as well. We love it when computers can do the work for us. No one said it would be easy to look through applications but I do believe we are missing out on fantastic students that may have scored below 230 or 240 on a one-time multiple choice test.

My contention is that it is not a loss felt by the students but that it is a loss felt by society. The fact is that we are having diversity conversations come up now when we have been using the Step 1 cutoff approach for years. If it worked, we would not be having conversations about diversity. Instead, we would be basking in the glory of how great we do it in derm. IMHO, we are clearly missing the boat since that is not the case and I feel that Step 1 cutoffs are a mistake. Has anyone looked at the diversity of the pool of applicants that are rejected from cutoffs vs those that are not? Perhaps that is worth doing if it has not been done yet. I fear that we are pre-selecting a big chunk of applicants to never make it into derm, especially with a metric that is only modest at best. That doesn't seem very evidence-based to me and seems more bias-based. My contention is let's look at the whole pool and not a pre-stratified pool. If we feel that faculty are too busy to spend time on this, maybe that reflects our priorities and perhaps we need to pay closer attention to that. I do not believe that the diversity is sufficiently reflected in those that score high step 1 and my understanding (please correct me if I'm wrong) is that you do. Therein, lies our difference in opinion. I'm sure that our final pool of interviewees may not be that different since many of the amazing applicants will have high step 1 but I'm guessing I'll have more than a few of the lower step 1 scorers that have something amazing to bring to the table too that we would have missed out on from a cutoff.

In regards to the evidence regarding step 1 scores and board failures, the interesting line from that OSU 2011 study is: "We were unable to analyze if there is an association between Step 1 scores and board passage/failure as a result of the small number of board failures." I think it is telling when you can't find enough board failures to make a correlation. They published another manuscript showing poor correlation between step 1 and non-cognitive abilities, which I think are very important for an effective physician and dermatologist in such a social specialty that is based on frequent patient interactions and nimble social contexts from room to room.

 

[mohsable]

Similarly, I think your viewpoint is important, well thought out, and should be a point of discussion among programs. I agree that using step 1 as a filter is not the ideal solution, but it may be the best/only current solution for standardizing academic achievement between different medical schools as well as for giving a starting point for addressing the issue of over-applying in dermatology. In my opinion, the root of the problem is not the step 1 cutoff, but that too many applicants are applying to too many programs. Somehow, it became acceptable for the majority of applicants, even highly competitive "rock star" applicants, to apply to basically every program. This culture spawns not only financial waste on the part of the applicant (while disadvantaging applicants who cannot afford to apply to $3,000+ worth of programs) but also a logistic nightmare on the part of the program. It's not that the faculty are too busy to spend time on reviewing applications, but that there are too many applications from dart-throwing applicants with little to no true interest in their program to be worth their time.

I responded to your initial comment because I wanted to point out that step 1 cutoffs are not at the center of the issue and that the relationship between step 1 scores and diversity is yet to be determined . I don't think anyone denies that step 1 is an important thing to consider when assessing an applicant's ability to score well on standardized exams. The issue is that the importance of step 1 has increased in relation to other more important factors (including diversity of background and experiences) once it became impossible to give a thorough evaluation to each application. Personally, I'm all for restrictions on the number of programs that applicants can apply to, as I think it would create a more thoughtful and comprehensive application process for applicants and program. Perhaps then the need for filters of any sort would be eliminated.

 

[dermathalon]

I wholeheartedly agree with you that we have way too much dart throwing and that is something that we definitely agree on. Regardless of our views on Step 1, I think you hit the nail on the head with the point of limiting applications. I think the challenge is that people feel so backed up against a wall with the current way derm applications go that they are not willing to risk losing even one more interview and try to soak up as many interviews as possible (can't blame anyone in the current system). If they made that rule change it really would be a solution to allow better review of each application. I've long advocated for rock-star students that I've advised to not do the dart throwing and keep it focused and they all match fine and in very strong programs too. You've articulated the point much better than I have in the past.

 

[doctalaughs]

You guys are too PC.

The ONLY objective thing is test scores and to a lesser extent, grades. You can cry/whine all you want that one group or person is more disadvantaged, or "might" have made a good doctor or "is overlooked" but that's completely subjective.

The fact is that the U.S. is probably the MOST forgiving country of objective scores in the world - many/most other countries decide whether you can be a doctor (or engineer or whatever) by scores in HIGH SCHOOL.

If you are still whining about scores being unfair by med school I don't know what to say. And of course if you have low scores it makes sense that the hard work of a year fellowship is seen as a "plus" to make up for it. Should we start randomly selecting applicants for residency, or med school, or whatever based on their facebook posts? Sheesh. I guess this is why we have one of the most ineffective and inefficient educational systems in the world.

 

[dermathalon]

Should we start randomly selecting applicants for residency, or med school, or whatever based on their facebook posts? Sheesh. I guess this is why we have one of the most ineffective and inefficient educational systems in the world.

LOL, the facebook comment was comedic... Where is sore eye asses when we need him/her?

Suffice it to say that a lot of people strive to come to the "most ineffective and inefficient educational" system for all careers and walks of life because they actually get another chance and we put out very strong doctors. Our system is pretty darn good (albeit expensive) for university and professional education . That's another thread so not going to drag this point out.

Many countries have quotas set up for past injustices that are more outright present than what we have in the US (there are several examples around the world of this). There is nuance in every system. To each his own. I think we can do better.

Research years will always count because # of pubs is an objective measure that is believed by many to roll sustained dedication and sustained hard work into a metric (whether you agree or not). If you publish a lot then people will overlook the need for a research year. That said, it's not a revelation that those that take a research year are likely to get more pubs.

 

[doctalaughs]

LOL, the facebook comment was comedic... Where is sore eye asses when we need him/her?

Suffice it to say that a lot of people strive to come to the "most ineffective and inefficient educational" system for all careers and walks of life because they actually get another chance and we put out very strong doctors. Our system is pretty darn good (albeit expensive) for university and professional education . That's another thread so not going to drag this point out.

Many countries have quotas set up for past injustices that are more outright present than what we have in the US (there are several examples around the world of this). There is nuance in every system. To each his own. I think we can do better.

Research years will always count because # of pubs is an objective measure that is believed by many to roll sustained dedication and sustained hard work into a metric (whether you agree or not). If you publish a lot then people will overlook the need for a research year. That said, it's not a revelation that those that take a research year are likely to get more pubs.

We are saying the same thing. Ignoring concrete objective thing (like scores, number of pubs, doing a research fellowship) is silly. Posters above are saying we are "obsessed" with scores and suggest that fellowship should not be considered; instead we should look harder at that kid with very low objective measures more carefully based on subjective things like "background growing up" and "humility" and "diversity."

Whatever you think of the US education system, we give more "second chances" and "looking at the whole person" than any system in the world, and still people whine.

 

[dermisit]

I respect your position on this and you bring up very valid points. I think the point where we disagree is the "diversity" in our opinions.

I personally think you do lose out when you use step 1 filtering because we know that there are many factors involved in regards to making a good dermatologist. If we are so quick to tell our patients that treatment X is not a justified treatment when there is poor evidence for it then why are we quick to assume that step 1 scores are a good predictor of passing the boards when there is poor evidence for that. Seems like we choose a side that suits us.

So I do feel that using Step 1 as a filter is not justified when we know that there are many different factors with Step 1 as nothing more than an ok correlation (and only to the ITE and not the boards itself). We choose step 1 because it's easy. Many tell me that it is because "we have to start somewhere" which to me sounds like laziness when other approaches could have been used as well. We love it when computers can do the work for us. No one said it would be easy to look through applications but I do believe we are missing out on fantastic students that may have scored below 230 or 240 on a one-time multiple choice test.

My contention is that it is not a loss felt by the students but that it is a loss felt by society. The fact is that we are having diversity conversations come up now when we have been using the Step 1 cutoff approach for years. If it worked, we would not be having conversations about diversity. Instead, we would be basking in the glory of how great we do it in derm. IMHO, we are clearly missing the boat since that is not the case and I feel that Step 1 cutoffs are a mistake. Has anyone looked at the diversity of the pool of applicants that are rejected from cutoffs vs those that are not? Perhaps that is worth doing if it has not been done yet. I fear that we are pre-selecting a big chunk of applicants to never make it into derm, especially with a metric that is only modest at best. That doesn't seem very evidence-based to me and seems more bias-based. My contention is let's look at the whole pool and not a pre-stratified pool. If we feel that faculty are too busy to spend time on this, maybe that reflects our priorities and perhaps we need to pay closer attention to that. I do not believe that the diversity is sufficiently reflected in those that score high step 1 and my understanding (please correct me if I'm wrong) is that you do. Therein, lies our difference in opinion. I'm sure that our final pool of interviewees may not be that different since many of the amazing applicants will have high step 1 but I'm guessing I'll have more than a few of the lower step 1 scorers that have something amazing to bring to the table too that we would have missed out on from a cutoff.

In regards to the evidence regarding step 1 scores and board failures, the interesting line from that OSU 2011 study is: "We were unable to analyze if there is an association between Step 1 scores and board passage/failure as a result of the small number of board failures." I think it is telling when you can't find enough board failures to make a correlation. They published another manuscript showing poor correlation between step 1 and non-cognitive abilities, which I think are very important for an effective physician and dermatologist in such a social specialty that is based on frequent patient interactions and nimble social contexts from room to room.

I think your logic is flawed. Let me ask you a few questions. Using your thought process, would you say that an applicant trying to match into neurosurgery with a step score of 220 may end up a better neurosurgeon than one with 265 because not all "factors" have been accounted for them being a good neurosurgery candidate? Is the student with the 220 the same caliber as the 265? Will the 220 contribute to the field of neurosurgery the same as the 265?

There's a reason why the exams exist in the first place. Obviously some people are advantaged in their education/background/wealth. On average however, we can reliably use exams like the step to gauge or predict the capacity for a student to do well. All those other "factors" don't correlate as well to being a good doctor, or at least I haven't seen any studies to suggest that being the case.

What it seems like you are suggesting is that cognitive abilities should be trumped by or on par with social intelligence/relatability when selecting candidates. This is in trend with the socializing of many aspects of society, and is detrimental to medical and scientific fields. There are plenty of disadvantaged students from very poor countries, much worse than America, who end up doing extremely well on the Step. They are disadvantaged in the truest sense, yet are able to perform. Them doing well is evidence that disadvantaged does not always equate to performance, but sheer will power to work hard.

I am not blind to the need for there to be diversity in medicine, as evidence does show that diversity can result in better outcomes, but that shouldn't be solved by lowering the standards by which candidates are chosen. That's an issue that starts with instilling work ethic and guidance in children from a young age.

 

[dermathalon]

I think your logic is flawed. Let me ask you a few questions. Using your thought process, would you say that an applicant trying to match into neurosurgery with a step score of 220 may end up a better neurosurgeon than one with 265 because not all "factors" have been accounted for them being a good neurosurgery candidate? Is the student with the 220 the same caliber as the 265? Will the 220 contribute to the field of neurosurgery the same as the 265?

There's a reason why the exams exist in the first place. Obviously some people are advantaged in their education/background/wealth. On average however, we can reliably use exams like the step to gauge or predict the capacity for a student to do well. All those other "factors" don't correlate as well to being a good doctor, or at least I haven't seen any studies to suggest that being the case.

Sounds like you ascribe quite a lot of value to Step 1. Why are we talking about neurosurgery on a derm thread?

My answer to your specific questions are:

Using your thought process, would you say that an applicant trying to match into neurosurgery with a step score of 220 may end up a better neurosurgeon than one with 265 because not all "factors" have been accounted for them being a good neurosurgery candidate?

No, they wouldn't be better or worse based on a low or high step score. That's the point.

Is the student with the 220 the same caliber as the 265?

We don't know. Have to look at the whole application. Step 1 is just part of the story and not good information on its own as you have posed the question.

Will the 220 contribute to the field of neurosurgery the same as the 265?

I don't know and neither do you. I fail to see the logic in this question as "contribution to a field" involves a lot more steps. I don't know what definition you are using but many academicians (which I do not think is the only way to define contribution) use publications and academic output. There are MD/PhDs candidates and highly published MD candidates that get into derm that do not have high scores and some (not all of those with publications coming in will go on to careers focused on academia) may contribute academically. We already make concessions for people with academic output so that step scores don't matter as much. So the answer to your question is "nobody knows because it's the wrong metric."

Cognitive abilities should be on par with social intelligence/relatability when selecting candidates. Cognitive abilities include being able to make an accurate diagnosis in the setting of a real patient and this is not the same as taking a multiple choice standardized exam. I think social intelligence is part of cognitive abilities in medicine (may not be the same in other fields) because we have to make many gray area diagnoses, choices, and treatments in complex cases.

The part about socializing society is a non-sequitor.

 

[reno911]

Sounds like you ascribe quite a lot of value to Step 1. Why are we talking about neurosurgery on a derm thread?

My answer to your specific questions are:


No, they wouldn't be better or worse based on a low or high step score. That's the point.


We don't know. Have to look at the whole application. Step 1 is just part of the story and not good information on its own as you have posed the question.


I don't know and neither do you. I fail to see the logic in this question as "contribution to a field" involves a lot more steps. I don't know what definition you are using but many academicians (which I do not think is the only way to define contribution) use publications and academic output. There are MD/PhDs candidates and highly published MD candidates that get into derm that do not have high scores and some (not all of those with publications coming in will go on to careers focused on academia) may contribute academically. We already make concessions for people with academic output so that step scores don't matter as much. So the answer to your question is "nobody knows because it's the wrong metric."

Cognitive abilities should be on par with social intelligence/relatability when selecting candidates. Cognitive abilities include being able to make an accurate diagnosis in the setting of a real patient and this is not the same as taking a multiple choice standardized exam. I think social intelligence is part of cognitive abilities in medicine (may not be the same in other fields) because we have to make many gray area diagnoses, choices, and treatments in complex cases.

The part about socializing society is a non-sequitor.

I'm not sure what your point is. I doubt anyone is advocating looking at only step scores. And while I'm sure it hasn't been proven that people with higher step scores become better dermatologists (or whatever), it doesn't mean we can make an educated guess about that. It's similar to the process used in making the "gray area diagnoses" that you refer to.

Let's say you had two groups of applicants who were completely identical in terms of grades, pubs, and everything else. The only difference is that one group had an average step score of 265 and the second group had an average of 220. You really don't think that the higher scoring group would be more successful (however you choose to define that) on average? Do you actually think that it's a complete non-factor and both groups will turn out equally well? Not close, but actually equal. Sure doing well on a multiple choice exam is not the same as making a real-life diagnosis. But you don't think there is any positive correlation between the two?

The step score is just one piece of data. Dismissing it too much is bad. Placing too much emphasis on it is bad too. How much is too much? That's a subjective determination that is hard to get complete agreement on.

 

[doctalaughs]

There are multiple studies showing step scores are a good predictor of residency performance and pass rate of real boards. If that's not an indication of "being a competent doctor" then it's going to impossible to design a study convincing you. Of course it's a single piece of data (and yes, a 210 candidate COULD be a better doctor than a 265 candidate), but it's a very important piece of data and considered rightly so. Even if you think boards don't matter for being a good doctor (which is sort of idiotic) -- the programs need their residents to pass easily, and therefore makes sense they want good USMLE scores.

Most of these studies aren't in dermatology (because it's hard to get a large N in a small field) but no reason to think the results would be any different.

West J Emerg Med. 2017 Apr;18(3):544-549. doi: 10.5811/westjem.2016.12.32478. Epub 2017 Feb 7.
USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study.

USMLE and Otolaryngology: Predicting Board Performance.
Puscas L, Chang CWD, Lee HJ, Diaz R, Miller R.

United States Medical Licensing Examination Step 1 and 2 Scores Predict Neuroradiology Fellowship Success.
Yousem IJ, Liu L, Aygun N, Yousem DM.
J Am Coll Radiol. 2016 Apr;13(4):438-44.e2. doi: 10.1016/j.jacr.2015.10.024. Epub 2016 Feb 28.

Is USMLE Step 1 score a valid predictor of success in surgical residency?
Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM.

General surgery resident remediation and attrition: a multi-institutional study.
Yaghoubian A, Galante J, Kaji A, Reeves M, Melcher M, Salim A, Dolich M, de Virgilio C.
Arch Surg. 2012 Sep;147(9):829-33. doi: 10.1001/archsurg.2012.1676.

USMLE step 1 scores as a significant predictor of future board passage in pediatrics.
McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT.
Ambul Pediatr. 2007 Mar-Apr;7(2):192-5.

 

[dermathalon]

My point was a simple one and please don't make it seem more complicated than it is. Don't use Step 1 as a singular filter for the computer cut. That's it if you want to increase diversity. Who said board scores don't matter at all? Please don't attack a straw-man. My point is that we can do better with our filters and not that we should eliminate test scores altogether.

As for the studies, if the point was to be evidence-based, then let's be evdience-based instead of just reading titles:

USMLE step 1 scores as a significant predictor of future board passage in pediatrics.
McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT.
Ambul Pediatr. 2007 Mar-Apr;7(2):192-5.
Scores higher than 220 was correlated with higher pass rates. 220.

Is USMLE Step 1 score a valid predictor of success in surgical residency?
Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM.
"...trainees with an average above the mean had a higher first-time pass rate (P = .04)." Above the mean was the comparator.

USMLE and Otolaryngology: Predicting Board Performance.
Puscas L, Chang CWD, Lee HJ, Diaz R, Miller R.
This study showed that Step 1 scores predicted a trend for passing the boards. However, the authors point out that their model was not very good at actually predicting a failure. In looking at the 402 residents with Step 1 and 2 scores, of whom 21 failed the WQE on the first attempt, the model only correctly predicted 4 of them as failing.

West J Emerg Med. 2017 Apr;18(3):544-549. doi: 10.5811/westjem.2016.12.32478. Epub 2017 Feb 7.
USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study.
Step 1 score of 227, Step 2 CK score of 225 and composite score of 444 predicted a 95% chance of passing both boards. Also they found that Step 2 was a better predictor than Step 1.

General surgery resident remediation and attrition: a multi-institutional study.
Yaghoubian A, Galante J, Kaji A, Reeves M, Melcher M, Salim A, Dolich M, de Virgilio C.
Arch Surg. 2012 Sep;147(9):829-33. doi: 10.1001/archsurg.2012.1676.
This study does not even look at board pass rates. If the point is resident attrition, this is a study on surgical residents which is comparing apples to oranges.

United States Medical Licensing Examination Step 1 and 2 Scores Predict Neuroradiology Fellowship Success.
Yousem IJ, Liu L, Aygun N, Yousem DM.
J Am Coll Radiol. 2016 Apr;13(4):438-44.e2. doi: 10.1016/j.jacr.2015.10.024. Epub 2016 Feb 28.
This study is apples to oranges and the average step scores were lower than what are pushed in derm.

All these studies show that, yes, Step 1 is correlated with better board pass rates but these correlations should not be misconstrued. The scores that lead to this prediction are lower (close to the mean in multiple studies) than what we are looking at in derm and only bolsters my point that it's diminishing returns after a certain point. Also, the models are only modest correlations at best, showing that other factors should be taken into account along with test scores.

I think I've expressed my point and looks like some are taking it to mean that we should throw out test scores altogether. You read what you want to read. My point is that applications should be read beyond a computer cut on the Step 1 score to increase diversity. That's my point. Look at all of the applicants or just be up front and save applicants their money. If you are a staunch believer in Step 1 as a cutoff, prove it by posting it on your website instead of hiding behind anonymous debates on SDN. I personally do not believe that taking this approach will increase diversity though. The point of this thread was increasing diversity.

I'll let others have the last word on this one and thanks for engaging in the conversation.

 

[doctalaughs]

My point was a simple one and please don't make it seem more complicated than it is. Don't use Step 1 as a singular filter for the computer cut. That's it if you want to increase diversity. Who said board scores don't matter at all? Please don't attack a straw-man. My point is that we can do better with our filters and not that we should eliminate test scores altogether.

As for the studies, if the point was to be evidence-based, then let's be evdience-based instead of just reading titles:

USMLE step 1 scores as a significant predictor of future board passage in pediatrics.
McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT.
Ambul Pediatr. 2007 Mar-Apr;7(2):192-5.
Scores higher than 220 was correlated with higher pass rates. 220.

Is USMLE Step 1 score a valid predictor of success in surgical residency?
Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM.
"...trainees with an average above the mean had a higher first-time pass rate (P = .04)." Above the mean was the comparator.

USMLE and Otolaryngology: Predicting Board Performance.
Puscas L, Chang CWD, Lee HJ, Diaz R, Miller R.
This study showed that Step 1 scores predicted a trend for passing the boards. However, the authors point out that their model was not very good at actually predicting a failure. In looking at the 402 residents with Step 1 and 2 scores, of whom 21 failed the WQE on the first attempt, the model only correctly predicted 4 of them as failing.

West J Emerg Med. 2017 Apr;18(3):544-549. doi: 10.5811/westjem.2016.12.32478. Epub 2017 Feb 7.
USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study.
Step 1 score of 227, Step 2 CK score of 225 and composite score of 444 predicted a 95% chance of passing both boards. Also they found that Step 2 was a better predictor than Step 1.

General surgery resident remediation and attrition: a multi-institutional study.
Yaghoubian A, Galante J, Kaji A, Reeves M, Melcher M, Salim A, Dolich M, de Virgilio C.
Arch Surg. 2012 Sep;147(9):829-33. doi: 10.1001/archsurg.2012.1676.
This study does not even look at board pass rates. If the point is resident attrition, this is a study on surgical residents which is comparing apples to oranges.

United States Medical Licensing Examination Step 1 and 2 Scores Predict Neuroradiology Fellowship Success.
Yousem IJ, Liu L, Aygun N, Yousem DM.
J Am Coll Radiol. 2016 Apr;13(4):438-44.e2. doi: 10.1016/j.jacr.2015.10.024. Epub 2016 Feb 28.
This study is apples to oranges and the average step scores were lower than what are pushed in derm.

All these studies show that, yes, Step 1 is correlated with better board pass rates but these correlations should not be misconstrued. The scores that lead to this prediction are lower (close to the mean in multiple studies) than what we are looking at in derm and only bolsters my point that it's diminishing returns after a certain point. Also, the models are only modest correlations at best, showing that other factors should be taken into account along with test scores.

I think I've expressed my point and looks like some are taking it to mean that we should throw out test scores altogether. You read what you want to read. My point is that applications should be read beyond a computer cut on the Step 1 score to increase diversity. That's my point. Look at all of the applicants or just be up front and save applicants their money. If you are a staunch believer in Step 1 as a cutoff, prove it by posting it on your website instead of hiding behind anonymous debates on SDN. I personally do not believe that taking this approach will increase diversity though. The point of this thread was increasing diversity.

I'll let others have the last word on this one and thanks for engaging in the conversation.

So you want residency selection comittees to look at all applications regardless of score in the name of diversity? Should we carefully pore over the application with a 195 step 1 because there is a 1 in a million chance we'll decide to discard one of 265s (that we could easily fill the whole class with)?

No one is suggesting scores are the only factor considered. Like I said, the USA already gives non-objective factors way more consideration than the rest of the world. However, having reasonable cutoffs is an efficient and fair way to pick a class. You can argue it should be 230 instead of 240 or whatever, but also understand 1 more thing -- for every person with a "low" score that you pick in the name of "diversity," you are denying a spot to someone else. That is why I'm totally against non-objective fluffly, subjective ways to lower the bar -ie same argument with affirmative action.

Instead, how about we find ways so that a more "diverse" group of candidates score equally highly so they get equal consideration.

 

[ActinicKeratosis]

Step 1 is really the only truly objective measure in medical school. Step 1 screens reward those that are the hardest working/most intelligent. I think that makes sense. There are plenty of diverse applicants that score very well on Step 1.

 

[reno911]

Dermathalon,

If your points are that
1. Eliminating a cutoff will probably lead to an increase diversity
2. If programs have a cutoff they should post it and make it clear.

then, those are pretty banal and I would not argue with either of them. Any arguments on those points would be fairly uninteresting. You're basically having a conversation with yourself.

Obviously eliminating a cutoff will likely increase diversity. The real issue is whether the value of increased diversity is worth taking people with lower scores. Now that's actually an interesting argument, and it is something that people in this thread are attempting to discuss.

And if a program has cutoffs, they should post them. Why would they have applicants waste time and money applying for no reason? When I was in academia I argued very strongly for doing this, but the Chair and the PD just didn't want to, so I was unsuccessful. Their reasons for not doing so were very weak.

So, I guess that I can see why you might bring this up as what you are suggesting is not what is generally done at most programs, however I don't think anyone here has argued against that specifically (I could have missed it though. I'm too lazy to read this whole thing to check).

 

[dermathalon]

Reno, since you specifically made some points I wanted to respond to you. Yes, those are banal points yet as you found out, people are still resistant to them. You have accurately stated my points except that my stance (and the reason that I brought up the conversation about Step 1 in the first place) is that I think looking at all the applications will still allow us to maintain quality in residents AND increase diversity too. It's not just about increasing diversity. I don't think you lose on quality by removing high computer cuts on Step 1 and the research supports that. I highly doubt we are looking at many students with 195s. Most of them will sit in the 220s and above with some dipping into the 210 and rarely 200s. At end of the day, everyone will use step scores as part of the overall assessment (it's silly to think that step scores would be disregarded). The opinions and preferences are split no matter which side of the equation you stand on. I know a bunch of colleagues on derm residency selection committees that believe high step 1 filters are very important and just as many that sit on derm residency selection committees that are opposed to filtering based on step 1. I see more and more faculty opposing the step 1 filters than in previous years.

Cutoffs should be posted or it borders on unethical when students on tight budgets are paying for each application to be reviewed if the application is just going through a computer cut anyway.

 

[doctalaughs]

Reno, since you specifically made some points I wanted to respond to you. Yes, those are banal points yet as you found out, people are still resistant to them. You have accurately stated my points except that my stance (and the reason that I brought up the conversation about Step 1 in the first place) is that I think looking at all the applications will still allow us to maintain quality in residents AND increase diversity too. It's not just about increasing diversity. I don't think you lose on quality by removing high computer cuts on Step 1 and the research supports that. I highly doubt we are looking at many students with 195s. Most of them will sit in the 220s and above with some dipping into the 210 and rarely 200s. At end of the day, everyone will use step scores as part of the overall assessment (it's silly to think that step scores would be disregarded). The opinions and preferences are split no matter which side of the equation you stand on. I know a bunch of colleagues on derm residency selection committees that believe high step 1 filters are very important and just as many that sit on derm residency selection committees that are opposed to filtering based on step 1. I see more and more faculty opposing the step 1 filters than in previous years.

Cutoffs should be posted or it borders on unethical when students on tight budgets are paying for each application to be reviewed if the application is just going through a computer cut anyway.

I have no problems with programs posting their cutoffs.

However, you first were arguing for elimination of cutoffs but now are moving the goalposts and arguing for lowering the cutoffs. As I stated, reasonable people can argue what the cutoff should be but ultimately that is up to the program, as it should be, because they can still easily fill their classes with a super-high cut.

Regarding the evidence that quality "can be maintained" or raised while lowering the average scores -- you don't have any. Because that has not been specifically studied. The absence of evidence does't mean you can make that assertion. You are correct that beyond 220/230 or so there is no specific POSITIVE study showing quality of a doctor increases. However it's a logical fallacy to assume it doesn't happen, because you also have no NEGATIVE study.

Even if you go strictly by the evidence available it's hard to argue lowering the cutoff below 220. And again, even if we look at all applications it's unclear to me whether you are arguing to give an applicant "extra points" for diversity ahead of a candidate with higher scores, or you are arguing we are somehow missing applicants with low scores but some other amazing achievement (won nobel prize, discovered cure for melanoma etc).

 

[dermathalon]

I do not support cutoffs and have not wavered on that. Give extra points for other achievements and I surely hope your bar is not nobel prizes and discovering the cure for cancer. Publications, extraordinary policy experience, and many other extraordinary features may be present. You'll be surprised what you find in an applicant when you remove the computer cut.

Let me remind you that you are the one that originally brought up these studies suggesting that there was a correlation through the entire scale...

 

[doctalaughs]

I do not support cutoffs and have not wavered on that. Give extra points for other achievements and I surely hope your bar is not nobel prizes and discovering the cure for cancer. Publications, extraordinary policy experience, and many other extraordinary features may be present. You'll be surprised what you find in an applicant when you remove the computer cut.

Let me remind you that you are the one that originally brought up these studies suggesting that there was a correlation through the entire scale...

What you advocate for is pretty uncontroversial if impractical. However, selection comittees already do what you are saying (give extra points for pubs, experiences and extracurriculars). To say that looking at all those apps in the 195-220 range is going to increase diversity or quality is pretty dubious though. The reason I say this is that it's unlikely that those with super-low scores (for a field like derm) are going to have SUCH amazing "non-score" credentials as to boost them into competition with the 250-270 range. After all, the 230s are those that might have such great experiences they might edge out a few "boring 265s."

Again, I thought you were advocating extra points for diversity alone which I'm against. Points for other work/life experience is already done. If you eliminate cutoffs won't change much (except waste hundreds of hours of faculty time which costs the institutions a LOT of money).

 

[WartyShawty]

You guys are so annoying. Bunch of whiny humblebragging snobs and knowitalls. You're all like a persistent wart.

 

[WartyShawty]

Talking about scores is like talking about the name brand of your clothes. Classless

 

[WartyShawty]

And as always, you somehow find some way to digress from the entire point of the OP. So let's can it, and talk about diversity sans scores.

 

[doctalaughs]

And as always, you somehow find some way to digress from the entire point of the OP. So let's can it, and talk about diversity sans scores.

So then contribute to the conversation in a manner you would like to steer the conversation .... instead of juvenile name-calling?

 

[sore eye asses]

I think you've all missed a really important selection factor. With globalthermonuclear war on the horizon it's clear we need to enrich for candidates with high board scores in the radiation oncology portion of step 1. Special consideration for those candidates who live in close proximity to a uranium mine or have expressed strong interest in isotope enrichment. Extra points for best comb over and tweet frequency. Only then can we ensure our survival in this brave new world. Those loser candidates who fail to meet these standards will all be shipped to Elon Musk's Martian colony to live out the rest of their lame existence staring at the barren erythematous slag and treating Martian scabies (gross and intractable).

 

[reno911]

Dermathalon,

As I suspected, there is not really much to argue with what you are saying.

1. No one here in this thread is against programs posting their cutoffs, so there's no discussion to be had.

2. The idea that eliminating cutoffs will increase diversity is likely true. Although the magnitude of the effect is unclear. If every program eliminated cutoffs, whether there are 50 more under-represented minorities offered a derm slot or just one, in either case, diversity will have been increased. It sounds like you think there will be a significant increase in diversity whereas doctalaughs, for example, thinks there will be very little. Good luck resolving that.

3. The only thing that's not trivial is your claim that eliminating cutoffs will lead to increased diversity without a reduction in "quality" (whatever that means). But you must admit that your claim here is speculative. No matter how you choose to define quality, your claim is very difficult to prove. However, it would also be very difficult to prove you wrong. The best that you're going to get is competing speculation.

 

[lampshade4u]

but then what do you say to the white kid who busted hard to score 260 that loses their place to someone else with a lower quality application because of their skin color? Like someone above said, the playing field once you hit medical school is about as even as it gets. There are plenty of loans and clubs and deans of diversity and scholarships available in far more abundance for those of "diverse" backgrounds, that extending the ideas of affirmative action into residency selection is just going a bit too far IMO.

 

[Helicases]

but then what do you say to the white kid who busted hard to score 260 that loses their place to someone else with a lower quality application because of their skin color? Like someone above said, the playing field once you hit medical school is about as even as it gets. There are plenty of loans and clubs and deans of diversity and scholarships available in far more abundance for those of "diverse" backgrounds, that extending the ideas of affirmative action into residency selection is just going a bit too far IMO.

I want to challenge your assertion that a white kid who works hard and earns a 260 can even lose their place to a minority with a "lower application." Derm is not a given for anyone. You tell the white kid to continue working hard and striving for dermatology if that is his or her passion. There are plenty of examples of people from all sorts of backgrounds, countries, ethnicities, and medical schools who applied more than once to their desired field. Who is to say anyone has a place in the specialty? Like grades, there are subjective metrics that go into the match. Sometimes they favor the white candidate with a 260 and sometimes they don't. It is important to stay humble.

 

[lampshade4u]

I want to challenge your assertion that a white kid who works hard and earns a 260 can even lose their place to a minority with a "lower application." Derm is not a given for anyone. You tell the white kid to continue working hard and striving for dermatology if that is his or her passion. There are plenty of examples of people from all sorts of backgrounds, countries, ethnicities, and medical schools who applied more than once to their desired field. Who is to say anyone has a place in the specialty? Like grades, there are subjective metrics that go into the match. Sometimes they favor the white candidate with a 260 and sometimes they don't. It is important to stay humble.

is lowering the bar or choosing anyone over anyone else on the basis of skin color (especially when they are less qualified) in the name of diversity ok?

 

[dermathalon]

I've seen plenty of cases where it's Causacian students that have struggled with homelessness, lived on food stamps, and being immigrants (yes, immigrants come in all colors). Diversity is so much more than just skin color. Stay humble.