Once 9PM EST hits tomorrow...

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

NewMD86

New Member
5+ Year Member
2+ Year Member
Joined
Jan 13, 2017
Messages
10
Reaction score
2
Does anyone know when the NRMP Algorithm starts running initially? Immediately? The next day? The following week?

I know it runs multiple times to ensure accuracy and validity.

Thanks in advance to any and all that know this answer.

Members don't see this ad.
 
It doesn't matter when or how it is run, the only thing that matters is the final product. It's very much a black box behind the scenes.
 
  • Like
Reactions: 1 user
Does anyone know when the NRMP Algorithm starts running initially? Immediately? The next day? The following week?

I know it runs multiple times to ensure accuracy and validity.

Thanks in advance to any and all that know this answer.
The actual algorithm can probably be run by a $400 desktop in 15 minutes. It's not a particularly computationally difficult one.

They do do some quality control, reach out to people that may have accidentally uncertified at the last minute, run it a few times just to make sure nothing is totally screwed up...

But I still have no clue what they do the majority of the three weeks.
 
  • Like
Reactions: 3 users
The actual algorithm can probably be run by a $400 desktop in 15 minutes. It's not a particularly computationally difficult one.

They do do some quality control, reach out to people that may have accidentally uncertified at the last minute, run it a few times just to make sure nothing is totally screwed up...

But I still have no clue what they do the majority of the three weeks.

What they do in those three weeks is the million dollar question. Haha.

Thanks for your input.

I think I read somewhere that the algorithm gets run in about 30 seconds (a YouTube video)?

Again, thanks for your reply.
 
  • Like
Reactions: 1 user
You could probably pay a couple motivated high-schoolers to do it by hand with pen/paper in 3 weeks. (Note: only slight exaggeration, the algorithm is relatively simple, but humans make mistakes)
 
This question is asked every year. We specifically discussed it with the NRMP the year that SOAP came out, because SOAP extended the amount of time needed by the NRMP and the question was whether ROL's would be due earlier, or match day would be later.

Running the algorithm is ridic fast, seconds or less. As mentioned, it's a very straightforward process, and with "only" 30,000 participants it's relatively small as far as computer processing goes. What takes so much time is:

1. All of the data is moved from the web based system to their in house system. The match program isn't part of their web system.
2. All of the data is checked for errors. I assume the main "error" they are looking for is a missing rank list. Of the 30,000 people, some idiot will decide to change their rank list at 8:50PM on the due date. And their computer / internet will crash, and by the time they've fixed it, it will be 9:02. Although I can't say for certain, I expect the NRMP might allow those people to "recertify" their last list (no changes allowed). Same with programs -- if someone forgets to certify a list, they probably allow that. The key is the NRMP wants the match to work well, not torture people for screwing up the deadline.
3. They run the match. Seconds.
4. They might run the match several times to find the best result. Without couples, the match algorithm would result in the same answer every time, there would be one "best" result. With couples included, it's more complicated and it's possible that multiple solutions will be found. 99% (or more) of the result will be the same, but a few people might end up in different spots. You would think that it would be easy to pick the "best" solution, but it isn't. Let's say in a second solution one person ends up 5 spots higher, but 4 other people end up 1 spot lower. Or let's say that 5 people match higher on their list, but one person doesn't match at all. Are those "better" solutions?
5. They then double check the solution, by hand. They pull some number of applicants and check to see that it's impossible that they match any higher on their rank lists.
6. Once that's all done the results are final. Now they get ready for SOAP. Given some of the problems in the first year of SOAP, this is no easy task.

Bottom line, I'd rather they took 3 weeks and got it right. The urology match (in the early 2000's) was screwed up and it was a huge debacle. There's a real benefit to ensuring the results are accurate. Plus, it's nice to know, to the second, when you'll get your results -- no need to sit around and wait for a random message.
 
  • Like
Reactions: 9 users
This question is asked every year. We specifically discussed it with the NRMP the year that SOAP came out, because SOAP extended the amount of time needed by the NRMP and the question was whether ROL's would be due earlier, or match day would be later.

Running the algorithm is ridic fast, seconds or less. As mentioned, it's a very straightforward process, and with "only" 30,000 participants it's relatively small as far as computer processing goes. What takes so much time is:

1. All of the data is moved from the web based system to their in house system. The match program isn't part of their web system.
2. All of the data is checked for errors. I assume the main "error" they are looking for is a missing rank list. Of the 30,000 people, some idiot will decide to change their rank list at 8:50PM on the due date. And their computer / internet will crash, and by the time they've fixed it, it will be 9:02. Although I can't say for certain, I expect the NRMP might allow those people to "recertify" their last list (no changes allowed). Same with programs -- if someone forgets to certify a list, they probably allow that. The key is the NRMP wants the match to work well, not torture people for screwing up the deadline.
3. They run the match. Seconds.
4. They might run the match several times to find the best result. Without couples, the match algorithm would result in the same answer every time, there would be one "best" result. With couples included, it's more complicated and it's possible that multiple solutions will be found. 99% (or more) of the result will be the same, but a few people might end up in different spots. You would think that it would be easy to pick the "best" solution, but it isn't. Let's say in a second solution one person ends up 5 spots higher, but 4 other people end up 1 spot lower. Or let's say that 5 people match higher on their list, but one person doesn't match at all. Are those "better" solutions?
5. They then double check the solution, by hand. They pull some number of applicants and check to see that it's impossible that they match any higher on their rank lists.
6. Once that's all done the results are final. Now they get ready for SOAP. Given some of the problems in the first year of SOAP, this is no easy task.

Bottom line, I'd rather they took 3 weeks and got it right. The urology match (in the early 2000's) was screwed up and it was a huge debacle. There's a real benefit to ensuring the results are accurate. Plus, it's nice to know, to the second, when you'll get your results -- no need to sit around and wait for a random message.

Heard the same regarding time to run the algorithm. Don't remember where I read it, but I think the article I read said that it takes less than half a second to run.

Re your #1: Someone in the radiology forum posted that they kept changing their rank list last second and didn't certify in time... so apparently stuff like that actually happens. Will be interesting to hear what nrmp does about a case like that.

Re your #4: I don't understand how running the same algorith with the same inputs and criteria can result in different results. If you run it five times, shouldn't you have five identical outcomes? Maybe I'm just missing something here, but I don't see how couples can 'disrupt' the algorithm.

R
 
Re your #4: I don't understand how running the same algorith with the same inputs and criteria can result in different results. If you run it five times, shouldn't you have five identical outcomes? Maybe I'm just missing something here, but I don't see how couples can 'disrupt' the algorithm.

R

In a perfect 1:1 matching, the order doesn't matter. That is, if you run all the applicants in any order you should end up with identical results. Throwing couples into the mix, theoretically you might get a little different results running applicants in a different order. The chance of this making a difference should be vanishingly small, but when there's 50,000 people being matched, maybe a few might be changed. Can't say I completely understand the reasoning, but there's some technical papers where the NRMP put it out.
 
Here's why:

Let's imagine a match without couples. Your rank list is being processed in the middle somewhere. We start with your #1. Sadly, someone has been "preliminary matched" into all the spots, and the person in the last spot is higher than you on the program's match list by 1 rank. That's too bad, you're not going to your #1 program. We move on to your #2 program, where you end up matching. Not too bad, #2 it is. Note that it doesn't matter what order the lists were processed in -- if you had been processed earlier you would have been prelim matched to your #1, but ultimately you would have been bumped out by the others and pushed into your #2. And there's no point in going back to reprocess your list again later, because the only possibility is that people even higher on the program's rank list have pushed other people out. It's impossible for a spot to open for you.

Now let's imagine the same match with couples. Same thing happens, you don't get your #1 and get placed in your #2. But now, someone who was prelim matched to your #1 gets bumped out because their partner gets bumped out of their spot. So now there's an opening for you, but your list has already been processed. This makes the algorithm more complicated -- because once a list is processed, it's possible that going back later might allow for a better match. I'm 100% certain that the NRMP manages this issue -- the simple way would be to match everyone, and then go back and reprocess people's lists to see if a higher match is possible, and then do that repeatedly until no one moves. But I think (can't prove) that this opens the possibility of multiple solutions. Again, before anyone panics, 99.9% of everything is going to be the same, the differences are going to be very, very small.

In fact, now that I think about it, imagine two couples applying to two programs each with 1 spot. Theoretically, 2 people should match and two should not. Here are the rank lists (I did not include the unmatched options):

Man's Best Hospital
D
C
B
A

St. Elsewhere
A
B
C
D


A/B
MBH/St E
St E / MBH

C/D
St E/MBH
MBH/St E

What happens? First we process A/B's list, and match A to MBH and B to St E.
Now we process C/D list. Their first choice doesn't happen, because C is below B on St E's list. And their second choice doesn't happen, because D is below B on St E's list. So A/B match, and C/D don't

But let;s say we process C/D first. So now C matches to St E, and D to MBH. Now we process A/B list But both options will be rejected because A and B are both below C on MBH's list. So now C/D match, and A/B don't.

This is a tie situation. Again, this is going to happen very rarely. The larger the match gets, the less likely this outcome is. But it's possible. Note that listing the unmatched options will fix this -- if all unmatched options are listed, D will match at MBH and A will match at St E.
 
  • Like
Reactions: 1 user
I've seen the Great AUA Disaster mentioned several times, but I've never been able to find what exactly happened. Anyone familiar with what their match failure was?
 
I've seen the Great AUA Disaster mentioned several times, but I've never been able to find what exactly happened. Anyone familiar with what their match failure was?
2005 AUA match, they somehow inputted a fair bit of the data backwards. Or ran it backwards. Told everyone their results, people were a bit surprised but no big deal... at first.

Except a handful of programs went unfilled. Top programs in the country. At least a couple reached out to applicants to find out why they were ranked so low, and found out that the people ended up matching below them on their list. This should be impossible. So the program directors contacted the AUA, who audited the result and realized something went pear-shaped. They emailed everyone, voided the match result, and reran the match a few days later. The vast majority of people then ended up at the same place or even higher on their rank list, though a handful ended up lower down.

There was a *great* thread on the urology match website where you could read the reactions people were posting as this was going on. Total cluster----. It was around still even a couple years ago, but it looks like they deleted their pre-2009 archives. If you're interested, this is an archive of what the forum looked like, but a lot of the posts seem to be missing. The match irregularities thread is the big one, as is match results take 2.
 
Last edited:
  • Like
Reactions: 1 users
Top