You're correct in that any time off is going to be nice, but having time off somewhere you enjoy is even better. During my first year of school, I really didn't care much for the city where I went to school and as a result most of my free time was spent at home or leaving the city. During second year, I came to really like where I was, and so time off meant going out and enjoying what the city had to offer. I think the second approach is the superior one.
Generally speaking, southerners are a pretty pleasant group. The whole Southern Hospitality stereotype had to start somewhere. That said, I wish I could say that we had erased all of our racist leanings. But, I suspect it isn't as bad as many people fear when moving here. Also, and this is going to sound bad, but each minority group will be treated differently by the types of people who care about such things. That will be offset some when you consider that most med students won't likely frequent the more rural types of places where such sentiment is strongest.
If worse comes to worse, Greenville is only 30 minutes away and its a very nice place (I'm moving there in 2 weeks).