Basic Modeling:
Went through the online SDN and other online boards and made a file of self-reported scores comparing them to UWorld Averages, NBME Scores, and UWorld Self Assesment Scores (UWSA). USMLE-Rx did their own model I think based on survey for all steps previously (listed here for convience). In all, I was able to collect aprox 130 separate date points including initial first time UW percentages, second time through UW averages, NBMEs, and UWSAs. Not all data was available for all final scores (ie not everyone did an NBME or a UWSA but in no category were there less than 30individual values). Good ranges were seen in all categories, Uworld averages ranged from 50 to 82% for first time through, second time from 63 to 93, UWSAs from 176 to 256, and NBMEs from 280 to 670. All posts from which original data were obtained I only collected from 2014 onward. For simplicty, I stuck to only linear models (except for UWorld Second time through given better model yields and comparatively limited data) though in reality, there's likely some non-linear aspects I don't really feel like nerding out on after just finishing this BS exam. And if you're still reading welcome to the geek club and you should probablly just run your numbers below and get back to studying... anyway...
For UWorld 1st time Percentage to scores:
Estimated Score = (1.685 * UWorld Percent Correct) + 113.2
63% likely yielding Final Score of 219
Estimated Passing with 197 = 50% correct
For UWorld 2nd time Percentage to scores:
Estimated Score = (64.41 * (UWorld Percent Correct ^ 0.2764))
86% likely yielding Final Score of 220
Estimated Passing with 197 = 58%
For USMLE-Rx per their estimates:
Estimated Score = (2.64 * USMLE-Rx Percent Correct) + 47
66% likely yielding Final Score 221
Estimated Passing with 197 = 57%
For NBME to scores:
Estimated Score = (0.099 * NBME Three Digit Score) + 170.31
510 likely yielding Final Score 220
Estimated Passing with 197 = 270
For UWSA to scores:
Estimated Score = (0.418 * UWSA) + 130.44
216 likely yielding Final Score 220
Estimated Passing with 197 = 160
Take Away Points from Model Estimates:
1) If you're consistently scoring above 57% on Uworld, you take an NBME and score over 400, or you have a UWorld Self Assesment score over 190, chances are pretty good you'll pass.
2) UWSA is a decent predictor but vastly underestimates scores. The R2 values for UWSAs was about 50% (ie 50% of the variation form mean was explained by the modeled outcome). That's not bad but given a 173 computes to an actual score of 202, it's more likely to freak people out than be a truly useful estimate of score. This is particularly true of lower but passing scores, once into the 220s, the UWSA seems to be a pretty accurate predictor, but at lower scores, a really bad one.
Important Footnotes (Please read before DMing me with complaints):
1) There's probably at least some "publication bias" here in that people who fail don't seem to be posting their UWorld scores. That potentially makes these equations over-value a score towards passing.
2) It should be noted that there's a difference between assessment tools and teaching tools. In other words UWorld, USMLE-Rx and Qbanks are teaching tools, NBME and UWSA are assessment tools. So technically the two should not be equated but given the UWSA seemingly is not a great predictor, and NBMEs seem to under estimate performance, finding a correlation between knowledge base and score outcomes is seemingly kinda hard.
3) These models don't take into account the reaction or preparation related to a given score. In other words, brilliant people may have awesome UW scores, blow off the exam and do average. Stupid people (me) may do crap but then bust ass and pull out a passing score. To what degree additional studying and preparation play in determining your actual outcome is completely unknown. The only thing according to the data collected here, is that people who seem to do UWorld a Second time appear to have final scores averaging somewhere between 2 and 4 points higher when matched to people who only did it once with similar 1st time scores (big whop right).
4) Per Mark Twain via Benjarmin Disraeli there are three types of lies; Lies, Damn Lies, and Statistics. Though all of these models were fairly strong and interestingly consistent across data fields (People with a listed UW average and NBME or UWorld score seemed to get similar close prdicitions from each individual score), they are purely estimates and anyone who even remotely followed the 2016 Election results knows estimates can, on occasion, be way off. Do not message me with smart-aleky stuff, overly critical, or inside baseball stats/modeling stuff, trust me, you win, you're way smarter than I am, I'm only trying to help.
Having run this and my numbers, after several months of prep, I decided screw it and went to take the damn thing.
My Own Experience PASS (210)
My Study Prep: Terrible and historically bad test taker with mid/lower prior step scores. If all of the preceeding discussion seems to display vast amounts of overthinking, you likely have at least some insight into just one of the many reasons why I've always done poorly on exams (E.g; am I supposed to think this hard for the answer? Should I give aspirin before the STEMI guy goes to the cath lab? Am I delaying him from getting to the cath lab if I say give him asprin first, does this question imply I even have access to a cath lab? etc, etc.) Did not take first two years in residency given family issues and working a more malignant residency. 1st time UW average was 53% (yikes), did it again to 68%, then sort of a third to mid 70s though I strangely recalled few of the actual questions each time through (read; again not the smartest guy on the blogs here). I did USMLE-Rx twice, first time 58%, second time 68%, did some Kaplan questions but not many at 68% and completed the Boards Vitals set with 68%. I did all this because I did lousy on a UWSA, 173, and an NBME 260 (shockingly low) after a good two months of studying uworld. For CCS did UWorld cases and then prior to exam did the 100 CCS Cases for Mac/PC. The low scores, extended time from med school and prior steps are the reason for the more effort and studying than you'll likely see here or (hopefully, fingers crossed) is necessary. For Reading material, I did Secrets Book x1, MTB x2, Doctors in Training x1, and made personal notes from Qbanks and that reading/viewing.
Day 1: Standard fair randomness. Stats plays a decent role but was more or less equal to some oddball ethics questions which easily could have been answered multiple ways. I felt ok with the stats having done the UW Stats and made my own notes. The Secrets book has a decent stats section, First Aid can also be helpful for this. I felt far less comfortable with the Step 1 type questions (histology, biochem, etc). I felt ok after the first day, not great, and was largely resigned to failing.
Day 2: I actually felt the multiple choice questions were somewhat harder. The difficulty of additional clinical training is that you start disagreeing with what is likely the right answer and have to continually ask yourself which answer they actually want you to click. I didn't feel good after the multiple choice questions but had planned to try to do better on CCS. I studied hard for the CCS and felt it kinda threw me for a loop. The CCS software is slow and lags (exactly similar to NBME software). I took the approach to do as much as possible with each patient while doing what was stated as necessary (all old folks get a pneumovax, varicella and influenza vaccine and all young women get an hpv vaccine and pap smear sometime later regardless of presentation, anyone with any type of cough gets a sputum gram stain, etc, etc). Because the software is slow, I frequently ran out of time while waiting for the orders to be confirmed after typing them in. I wanted to try to compensate for what I imagined would be poor MCQ performance with a strong CCS showing, but I honestly don't know if this is the right approach or not. If you read Cruch CCS, it seems like all cases have basic criteria you need to meet and that's it. Same more of less with UWorld. The CCS Cases online stressed basically ticking every possibly box possible which is sort of what I shot for so we'll see. If I fail and have to do this again having noticed a lousy CCS score, I'll let you know and won't take this approach again. I certainly had a poor guy I left with an untreated condition which it occurred to me how to fix in the car ride home, and a case of two that ended before the likely diagnostic result came back confirmatory. That makes me nervous but again, well see. In the end.
I feel like this could go either way, I only need to pass, but have a strong suspicion I failed just based on my numbers are soft and how I felt walking out. Can't sleep so writing this
Will update with my score but had such a horrible time studying for this exam, and obviously used these boards extensively so felt the need to give back. Best of luck, keep in touch.
