Your individual grades in the preclinical years by themselves don't really matter as much, as PM&R residency programs are much more focused on personality fit, board scores, LORs, and your performance in clinical rotations. In addition, there are variations in the preclinical curriculum among schools based on how you are graded (mandatory attendance) or due to some being more boards focused and clinically relevant than others which could make your grades more of a moot point.
However, your grades could reflect an underlining problem. Are you getting a good grasp of the preclinical material? Focus on learning the material the best you can for Step 1 and Step 2 and be prepared to work hard on clinical rotations.
That being said, PM&R isn't the most competitive specialty out there (but I hear it's getting more competitive). I also didn't have the highest preclinical grades (though with decent board scores and clinical grades) and still managed to get plenty of interviews.