I think a big thing with PM&R is the commitment to the specialty. That doesn't mean, however, that an individual has no shot if they figure out late that they are interested. I think your board scores are fine, but board scores are just one item to help you get your foot in the door for an interview, you'll also need letters that advocate for your to do PM&R, a good personal statement, etc. It is nice to also have "experience" in the field: PM&R rotations (both inpatient and outpatient), community service (adaptive sports, Special Olympics, sports event coverage), medical societies (AAPM&R is free for students, as is AANEM. I believe AAP costs a little bit), networking at conferences (the same aforementioned societies), research (anything from case reports all the way to double-blinded, etc, studies), and publications from that research.
I think you first need to ask yourself: why PM&R? What aspect(s) do you like about PM&R? Do you like the patient population (everything from sports injuries to chronic pain patients to traumatic brain/spinal cord injuries to children with disabilities to amputees)? What is your end goal, fellowship/general?