New AAMC practice exam curve?

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

Dirtybird

Full Member
7+ Year Member
Joined
Jan 9, 2014
Messages
176
Reaction score
122
Anybody else notice their scaled scores change for AAMC FL 1?

My score dropped 2 points compared to the score I got immediately after taking the exam last week, 1 point each from the CARS and P/S sections.

Members don't see this ad.
 
I just took it yesterday and was bummed with my score. Seems weird that in the verbal section I got 74% correct, but was 61st percentile and only got a 125...

If this is the case every two questions would be a whole MCAT point to my overall score. (132 in max so 7 more points were available to me in this section, but I only missed 14 questions so 7/14 = 2)

upload_2017-5-9_18-13-58.png

upload_2017-5-9_18-13-27.png
 

Attachments

  • upload_2017-5-9_18-11-21.png
    upload_2017-5-9_18-11-21.png
    41.9 KB · Views: 367
Members don't see this ad :)
I just took it yesterday and was bummed with my score. Seems weird that in the verbal section I got 74% correct, but was 61st percentile and only got a 125...

If this is the case every two questions would be a whole MCAT point to my overall score. (132 in max so 7 more points were available to me in this section, but I only missed 14 questions so 7/14 = 2)

View attachment 218705
View attachment 218704
I feel like the scoring is off. I got 95 percentile on CARS and it was a 129. This doesn't make any sense to me...I would think it would at least be a 130.

Also, you seem to be right about the 2 questions per point thing for this exam. I got 47 out of 59 correct on P/S. Before they dropped scores, I had a 127. Now, they dropped it to a 126. You got a 127 with 49 questions correct, so this makes sense.

However, on C/P I got 2 more questions right than you and we still got the same score.
 
I had a 127 on P/S and they dropped it to 126. I had a 130 for C/P and it got bumped up to a 131. Overall score didn't end up changing

I'm also at 95 percentile CARS and 129 scaled score. Most of the charts I see say it's a 130. Weird
 
Without a doubt the CARS and PS curves are brutal for AAMC FL1. I spoke to AAMC regarding the matter and they stated that the curve was based on "all of the attempts students have made on AAMC FL1".

I guess some students will review the material, and kind of go at a leisurely pace - which might represent why the curves for those sections are brutal.
 
  • Like
Reactions: 1 users
Without a doubt the CARS and PS curves are brutal for AAMC FL1. I spoke to AAMC regarding the matter and they stated that the curve was based on "all of the attempts students have made on AAMC FL1".

I guess some students will review the material, and kind of go at a leisurely pace - which might represent why the curves for those sections are brutal.

Sounds like the B/B and P/C scales should be on the harsher side as well then. And shouldn't the percentile change but not the scaled score?

Is this true for FL2 as well?
 
Sounds like the B/B and P/C scales should be on the harsher side as well then. And shouldn't the percentile change but not the scaled score?

Is this true for FL2 as well?

Not 100% sure regarding the exact details; Just relaying the message I got.

Also, prior to the scaling of AMCL FL1, there was a slight difference between the CARS scores for FL1 and FL2. In other words... getting 34 right would have correlated with a higher score for FL 1 than FL2. Now they seem to be more even.

I don't like the shift very much. The shift is based on students practice exams. This will make gauging actual MCAT scores more difficult... as prior to FL1 scaling, students (on average according to SDN and Reddit) were scoring 2 points higher on the real deal compared to FL1/2 average. Now we might see a slightly more dramatic (2-3 pt) increase.
 
Does this mean anything for the actual MCAT? Like they *gulp* making it harder?

I don't suspect so. Scoring (to my knowledge) is scaled - based on other students' performances. As long as you can compete with other students... the MCAT will spare you lol.
 
yeah, the new curve for CARS is rough. How often will they keep doing a new scaled score conversion? Is it every year, forever, or just till the new exam gets all worked out. The average is 500 now, does that mean it won't move anymore?
 
If they're scaling it for all the students attempts, then it's being skewed because of people taking the *same test* multiple times. I don't know why people do that, but they do. Then they just remember answers and get higher scores, which skews the percentiles.
 
  • Like
Reactions: 2 users
If they're scaling it for all the students attempts, then it's being skewed because of people taking the *same test* multiple times. I don't know why people do that, but they do. Then they just remember answers and get higher scores, which skews the percentiles.
I do not think that is how this works. Do you have any evidence to back this statement up? I've heard that the AAMC had 1000s of questions it can use in an MCAT. Do you know how many tests you would have to take for your ability to memorize anything to make an impact? The AAMC also only let's us take the exam a maximum of 7 times, period.
 
Members don't see this ad :)
I do not think that is how this works. Do you have any evidence to back this statement up? I've heard that the AAMC had 1000s of questions it can use in an MCAT. Do you know how many tests you would have to take for your ability to memorize anything to make an impact? The AAMC also only let's us take the exam a maximum of 7 times, period.
I actually think it's a good theory. Any sort of familiarity will raise a score for subsequent attempts.
 
I do not think that is how this works. Do you have any evidence to back this statement up? I've heard that the AAMC had 1000s of questions it can use in an MCAT. Do you know how many tests you would have to take for your ability to memorize anything to make an impact? The AAMC also only let's us take the exam a maximum of 7 times, period.

Pretty sure the practice exams, qbanks, and section banks always have the same questions. Everybody is tested on the same questions, retaking the practice material doesn't change up the questions or even their order. Notice how people ask questions like "how did you solve exam 2 B/B #23" and whatnot on sdn and reddit.

edit: I think you may have confused practice exams he/she was referring to with the actual mcat
 
  • Like
Reactions: 2 users
Yeah, I was referring to practice exams. I've seen people list their practice exam scores like "AAMC FL1, attempt 1: 72%, attempt 2: 96%".. It throws things off, unless maybe the AAMC is only using first attempts for scaling? That's the only way that multiple attempts wouldn't skew things. Or maybe we're just all getting smarter. I don't know about all of you, but I feel like I'm getting dumber as I get closer to my actual MCAT. o_O
 
  • Like
Reactions: 1 user
Yeah, I was referring to practice exams. I've seen people list their practice exam scores like "AAMC FL1, attempt 1: 72%, attempt 2: 96%".. It throws things off, unless maybe the AAMC is only using first attempts for scaling? That's the only way that multiple attempts wouldn't skew things. Or maybe we're just all getting smarter. I don't know about all of you, but I feel like I'm getting dumber as I get closer to my actual MCAT. o_O
Couldn't agree more.
 
Yeah, I was referring to practice exams. I've seen people list their practice exam scores like "AAMC FL1, attempt 1: 72%, attempt 2: 96%".. It throws things off, unless maybe the AAMC is only using first attempts for scaling? That's the only way that multiple attempts wouldn't skew things. Or maybe we're just all getting smarter. I don't know about all of you, but I feel like I'm getting dumber as I get closer to my actual MCAT. o_O


Same! I feel like I know nothing
 
I can 99.99999% assure y'all that the AAMC doesn't scale the practice tests based on performance on people taking the practice exam. There are too many confounding variables to produce data from exams that would be taken in a strict and controlled environment (i.e. testing conditions - no outside help/timing/etc).

The adjustments are a reflection of the new distribution of scores on actual administrations.
 
Gained 1 point in C/P (127 to 128), lost 1 point in CARS (124 to 123), and gained 1 point in B/B (126 to 127). Overall 502 to 503, and am testing 5/19. I know I do not have to score as well as you pre-meds to get in, but still nervous knowing my acceptances are pending my score and scholarships pending 500+.
 
Yeah, I was referring to practice exams. I've seen people list their practice exam scores like "AAMC FL1, attempt 1: 72%, attempt 2: 96%".. It throws things off, unless maybe the AAMC is only using first attempts for scaling? That's the only way that multiple attempts wouldn't skew things. Or maybe we're just all getting smarter. I don't know about all of you, but I feel like I'm getting dumber as I get closer to my actual MCAT. o_O
People's repeated attempts have nothing to do with the practice test scaled scores. When these tests were released, they came with a specific raw to scaled score conversion that fits for just that exam. These new curves are the latest for the actual MCAT. These new scales do not replace the scaled scores on the practice tests. Those are still the same as they have always been.
 
People's repeated attempts have nothing to do with the practice test scaled scores. When these tests were released, they came with a specific raw to scaled score conversion that fits for just that exam. These new curves are the latest for the actual MCAT. These new scales do not replace the scaled scores on the practice tests. Those are still the same as they have always been.
This is both true and false. True that when the tests were released, they came out with a specific raw to scaled score conversion. False that repeated attempts don't affect score. Yes, it's true that let's say 50th percentile will always be a 125 (not accurate, just made up a number) on the scale, but what exactly qualifies as 50th percentile changes as more people take the exam, and is certainly skewed by retakers who do better because they remember half the answers. If the average is 40/59 questions correct, then someone who got those 40 questions correct will be 50th percentile and score a 125. However, if a bunch of people retake, remember most of the answers, and then the average becomes 47/59, a person who got 40 questions correct will now drop to < 50th percentile. Their scaled score will then drop.
 
This is both true and false. True that when the tests were released, they came out with a specific raw to scaled score conversion. False that repeated attempts don't affect score. Yes, it's true that let's say 50th percentile will always be a 125 (not accurate, just made up a number) on the scale, but what exactly qualifies as 50th percentile changes as more people take the exam, and is certainly skewed by retakers who do better because they remember half the answers. If the average is 40/59 questions correct, then someone who got those 40 questions correct will be 50th percentile and score a 125. However, if a bunch of people retake, remember most of the answers, and then the average becomes 47/59, a person who got 40 questions correct will now drop to < 50th percentile. Their scaled score will then drop.


Disagree here - there is no way to produce a reliable scaled score from a administrations of a practice exam. There are way too many variables to account for that result in a non-standardized score. What if someone takes the test with an open book on try #2 and scores a 528? How would the AAMC systematically eliminate that outlier?

The only reason why the scale for FL1 changed was because the AAMC acquired new data from the proctoring of the actual exam and could reassess the scale they produced for FL1. The questions on that exam were undoubtedly pulled from the same pool of questions used to produce the first actual forms of MCAT2015.
 
  • Like
Reactions: 1 user
Disagree here - there is no way to produce a reliable scaled score from a administrations of a practice exam. There are way too many variables to account for that result in a non-standardized score. What if someone takes the test with an open book on try #2 and scores a 528? How would the AAMC systematically eliminate that outlier?

The only reason why the scale for FL1 changed was because the AAMC acquired new data from the proctoring of the actual exam and could reassess the scale they produced for FL1. The questions on that exam were undoubtedly pulled from the same pool of questions used to produce the first actual forms of MCAT2015.
Yeah, its pretty scary how confused some people seem to be about a straightforward process.
 
Yeah, its pretty scary how confused some people seem to be about a straightforward process.
I don't know if I would refer to it as "scary"....this is the info I got from multiple people, not something I made up on my own. It may seem straightforward to you, but no need to belittle those of us who are confused.
 
  • Like
Reactions: 1 user
I don't think we're confused. Maybe we're approaching this from different angles, but in any case we're all just assuming things about how the process works. Since they scale each proctored exam differently, it doesn't make sense that they would apply any proctored exam scale to a practice exam. The only reason I could see for them to change the scale of a practice exam after it's been in circulation for nearly two years is that people are doing better on it overall now than they were last year or the year before. We evolve, they evolve. Red queen hypothesis and all that.
 
  • Like
Reactions: 1 user
I don't know if I would refer to it as "scary"....this is the info I got from multiple people, not something I made up on my own. It may seem straightforward to you, but no need to belittle those of us who are confused.
You are right, that did come across condescending, I apologize. I am very skittish when it comes to misinformation spread on here, so I got too insulting.

IMO the raw-to-scaled score process is something only the AAMC will ever know the details on. The AAMC says the reason they have had to update the scales so often since 2015 is because they are collecting data to determine proper scales. That is why last year the MCAT average was about a 502, and now it is a 500, which is where they want it. The AAMC will want to change the exam questions or type of passages they use to keep these scaled scores constant, otherwise the exam becomes useless as an objective admissions tool.

I want to avoid having anyone believe that the scaled scores for the MCAT will always change, or that people taking a practice test, or even the real MCAT over and over will change these score conversions. they will not. The practice test scales have not really changed, and the AAMC's goal is to keep the scaled scores consistent, so that the average score is a 500, and the average section score is about 125.

Sorry for the tone, thanks for the conversation.
 
  • Like
Reactions: 1 user
I see what you're saying. The original discussion came up because they did change the scaling scores for FL1. I took it and had a 519, then 6 weeks later they changed the scale and my 130 in P/S became a 129 and they changed my score to a 518. This happened to most people (everyone?) who took it before the change - their scores went down after the fact because the scale was changed.
 
I see what you're saying. The original discussion came up because they did change the scaling scores for FL1. I took it and had a 519, then 6 weeks later they changed the scale and my 130 in P/S became a 129 and they changed my score to a 518. This happened to most people (everyone?) who took it before the change - their scores went down after the fact because the scale was changed.
Mine went up 1, which I found odd.
 
wow! So basically we just have no idea what they're doing. But they definitely did change the scale for FL1 this month, and who knows if they will do it again in the future? I don't really care anymore since I already took my real exam last Friday, but it would be nice to have a bit more transparency. Also, I should probably be writing my personal statement and not be distracted by AAMC data the entire 27 days I have left before scores are released.. :laugh:
 
You are right, that did come across condescending, I apologize. I am very skittish when it comes to misinformation spread on here, so I got too insulting.

IMO the raw-to-scaled score process is something only the AAMC will ever know the details on. The AAMC says the reason they have had to update the scales so often since 2015 is because they are collecting data to determine proper scales. That is why last year the MCAT average was about a 502, and now it is a 500, which is where they want it. The AAMC will want to change the exam questions or type of passages they use to keep these scaled scores constant, otherwise the exam becomes useless as an objective admissions tool.

I want to avoid having anyone believe that the scaled scores for the MCAT will always change, or that people taking a practice test, or even the real MCAT over and over will change these score conversions. they will not. The practice test scales have not really changed, and the AAMC's goal is to keep the scaled scores consistent, so that the average score is a 500, and the average section score is about 125.

Sorry for the tone, thanks for the conversation.
It's no problem, I understand your perspective. Thank you for your insight :)
 
  • Like
Reactions: 1 user
Top