You'll really need to shadow some neurosurgeons to find out for yourself because people will tell you different things. But it certainly takes demands on your time; as much as probably any other specialty.
Procedures can be long and arduous. Outcomes at times can be poor. The patients on the floor can be very sick and require a lot of attention. Because of the nature of the nervous system and consequences of fiddling with it, the threat of lawsuit is always present. There is more than a fair share of neurosurgeons that think their career sucks and regret choosing this field. But there are also many that love what they do and have a great family life.
I think a lot depends on the practive environment and the type of cases performed. Expectation has a lot to do with it also. However, I could forsee in the future some neurosurgeons doing soley minimally invasive spinal procedures on an out-patient basis and living a very, very good life. But for now, it's not that way.
Working 12 hour days, every day; working more weekends than not, even in private practice is not unheard of. Even if there your school does not have a neurosurgery program see if you can find some neurosurgeons out in private practice that will let you follow them around for a day or two to see if it's for you. It can be tough to make that decision since your exposure may be very short, and, more than likely you'll have second thoughts regardless of what specialty you choose.