It makes a ton of sense to do it retroactively.
Put yourself in the shoes of a PD at a competitive surgical program in 2022. You'd become completely familiar and reliant on Step 1 as your major yardstick, and Step 2 hasn't really been able to replace Step 1 in the short time it's been absent. Most of your applicants are coming from Pass/Fail preclinical curriculums, and many have clinical grades that are inflated, subjective, and generally not that useful. Everyone has a big pile of worthless case reports and middle authorships on small chart reviews. Everyone's MSPE raves about them and refuses to give ranks, or even quartiles anymore.
Along comes Mr. GapYears, who took some time off for research and has a 261 listed for Step 1. Wow! You know what that means!
His friend from the year below him in school, Mr. NoGap, has an extremely similar application with research years prior to med school but a Pass for Step 1.
You rank Mr. GapYears a lot better, and he matches at your program, while Mr. NoGap doesn't. In fact, Mr. NoGap falls surprisingly far down his rank list and is very upset.
Has Mr. NoGap been treated unfairly here? How do we know he didn't get a 270 and was simply unable to show so? Would you be comfortable arguing about it in court?
Seems much safer to just draw a deadline after all the current scoreholders have matched (sans MSTP), and then report everybody and anybody on ERAS as just Pass.