That goes both ways. If I were evaluating two jobs that were otherwise close to equal, I would go with the one that had enough wisdom, experience, empathy, and common courtesy to discourage me from wearing a suit for an interview. And, to be honest, even if the jobs weren't that equal, I'd give the "down to earth" group that didn't bother wasting my time with superficial appearances a huge bump in the rankings.
This isn't the 1950s. We aren't trying to claw our way up to middle management.
When you get right down to it, an anesthesiologist wearing a long white ("doctor's") coat to an interview would be absolutely ridiculous...and yet it would be less ridiculous than a business suit.