Fair enough.
If derm is a superficial field (as some naive people think), then cosmetic derm is the ultimate in superficiality. It's a business where one can make a great deal of money, and it's usually cash pay. If you were to express interest in cosmetics then you might appear shallow and thus uninterested in medical derm. This, naturally, would be off putting to a program director. That's the reasoning as I understand it.
One of the paradoxical things is that many of the cosmetic dermatologists that I know in academic settings are extremely intelligent, very capable physicians who are well respected by their peers. But, if a derm resident came through for interviews and said their career goal would be to be just like [insert famous, well-respected cosmetic dermatologist here], they would almost certainly never get a residency spot.
The sad thing that while there really is a lot of cool basic science behind many laser and cosmetic procedures, and it is conceivable that someone may have a genuine academic interest in such things, most cosmetic dermatologists are in it largely for the money. So, if one expresses an interest in cosmetics, he will be presumed to be greedy and shallow despite any claims and evidence to the contrary. It's unfortunate, but that's the way it is.