FOR THOSE OF YOU WHO HAVE QUESTIONS REGARDING PERCENTILE INFO, I THOUGHT THIS INFO MIGHT HELP. IT IS TAKEN FROM BOTH NBME.ORG AND NBOME.ORG
COMLEX INFO:
What types of score reports will candidates and schools receive from their computer-based COMLEX-USA examinations? When and how will the scores be reported?
For the computer-based COMLEX-USA examinations, candidates will still receive an official printed copy of their score reports from the NBOME by mail. The student score report will provide 3-digit standard scores, 2-digit standard scores, and a pass/fail designation. The 2-digit scores are linear transformations of the 3-digit scores so that 75 is designated as the minimum 2-digit passing score. Percentile rank will no longer be reported due to the fact that candidates will be taking the examination throughout the year.
USMLE INFO:
USMLE SCORE REPORTING: PERCENTILE INFORMATION DISCONTINUED
May 1999
Policy Change
As of May 1999, percentile information is no longer provided in connection with reports of USMLE scores.
Background
The primary focus of USMLE is on the licensure decision, and Steps 1, 2, and 3 scores are used in this process. The scores for each administration of a USMLE Step are equated so that a given two-digit or three-digit score always represents the same level of examinee performance for that Step. In other words, a score of 200 on one administration of a Step indicates the equivalent level of examinee performance as a score of 200 on any other administration of the same Step. This equivalence holds even if the pass-fail standard is changed, which permits comparing performance across time.
It is important also to remember that the two-digit score shown on USMLE transcripts is not a percentile. The two-digit score is a total test score that is designed to meet the requirements of many state licensing authorities. The two-digit score scale is one on which a 75 is always the minimum passing score. However, a given two-digit score may represent a different level of performance if the two administrations were subject to different pass/fail standards.
Percentiles are different from the two- and three-digit equated scores in that they can only be interpreted in the context of the examinee (norm) group upon which they are based. When the norm group changes, the percentile for a given score will change and percentiles based on different groups cannot be compared. Although the performance of large groups of examinees typically does not change dramatically from one year to the next, over a longer period of time there have been substantial changes. For example, in the late 1980s, applications to medical schools showed a marked decline, and this was reflected by lower performance on the initial administrations of USMLE as compared to the present when group USMLE performance has improved. As a result, identical three-digit scores are associated with significantly different percentiles if those percentiles are based upon the examinees from the different periods.
Problems with Comparison of Percentile Ranks
It is important to note that the above phenomenon has little impact in the licensure context for which USMLE was designed. Nevertheless, it also is clear that USMLE scores are used by third parties for a number of different reasons, with a heavy reliance upon corresponding percentile data. It has come to our attention that, in these secondary uses of USMLE data, failure to appreciate fully the relative nature of percentiles has caused a number of problems. The three most common are:
1. Student A took Step 1 in 1994 and reports a score of 210 and a percentile of 58. Student B took Step 1 in 1996 and reports a score of 210 and a percentile of 48.
Are both reports correct? Yes. Each student used the percentile conversion charts developed for the cohort which included the examination that he/she took. In each case the reference group was different and so the percentiles calculated for the same score were different.
Which student had better performance on Step 1? Both students scored 210, so their performance was identical. The percentiles, although "correct" imply that Student A "did better" on Step 1 but, in fact, the two students performed equally well.
2. Student A took Step 1 in 1994 and reports a score of 206 and a percentile of 50. Student B took Step 1 in 1996, and reports a score of 210 and a percentile of 48.
Are both reports correct? Yes. Each student used the percentile conversion charts developed for the cohort which included the examination that he/she took. In each case the reference group was different.
Which student had better performance on Step 1? Although Student A fell into a higher percentile rank because a different norm group was used, Student B actually had a slightly better performance on Step 1 as represented by the three-digit score. It should be noted that the difference between the scores is very small (i.e., approximately 0.2 standard deviations) and probably reflects no practical difference.
3. A student took Step 1 in 1994. He/she reports a score of 218 and a percentile of 73. A medical school official refers to a USMLE percentile conversion table and finds that the student's percentile is 63.
Are both percentiles correct? Yes. Both the student and school official have obtained accurate but apparently inconsistent information as a result of using percentile charts based on different examination cohorts. This is because the percentile for the Step 1 score varies depending on the norm group used to calculate the percentiles. The applicant was in the 73rd percentile of those who took Step 1 in 1994 and the 63rd percentile compared to those who took Step 1 in 1995 and 1996.
These examples clearly show a potential for misinterpretation of the scores if the user is not familiar with: a) the difference between two- or three-digit equated scores and percentile information, and b) the examinee group upon which the percentile information is based.
Summary
Percentiles are not meaningful when attempting to rank order students whose percentile equivalents were calculated using different examinee groups. Utilizing percentiles can lead to the types of errors illustrated above. It is for these reasons that percentile information is no longer available in connection with USMLE scores.
