One concern I would have about seeing patients by yourself at 11 pm is the liability and safety issues of seeing patients alone at a time of night when absolutely nobody else would be around.
I'd very much agree with this.
A concern for outpatient I'd have is seeing patients who may not be stable in the office. I wouldn't worry so much about anxiety or minor depression cases, but cases where the patient has a past history of violence, antisocial PD, histrionic PD, Borderline PD, psychosis &/or mania with a history of dangerous behavior can be working without a safety net.
In residency, I've known 3 cases where patients decompensated in a manner that would've warranted security show up, and Haldol IM be given, but since it was outpatient, the police had to be called, and they didn't show up for about 15 minutes. During that time, the patient was doing things that were not safe, but the staff & doctor could only sit there & try in vain to verbally redirect the patient, while the patient was doing things like screaming & overturning a table with about $100 worth of pharm rep food, while the pharm rep is in horror hiding under another table in the room, while suggesting to the staff to give the guy geodon (yeah that was the funny part of the story. They should've grabbed the pharm rep & threw him at the patient).
And I know of one situation in an (P)ACT team where someone decompensated & pulled a knife on the resident & mental health worker accompanying the resident. The health worker actually karate kicked the knife out of the patient's hand, and the patient grabbed another knife. They decided to split after that and called the police.
IMHO any outpatient office that sees the above type of patient needs a safety alert system, where the doctor could signal someone else to possibly call 9-1-1, & possibly contact law enforcement ahead of time & have preplanned strategies to work with them should things go awry.
Outpatient office violence is rare, but it can happen, and when it does, if there's no precautions, everyone is caught with their pants down. The incidents I mentioned above happened in the course of 4 years. That's 4 incidents in outpatient in 4 years. That IMHO warrants planning.