Don't forget all the office phone calls that you have to review and return. Don't forget all the forms you have to fill out on your patients behalf (prior authorizations, FMLA, disability, sports camp physical, work physical, school physical, etc). Don't forget the insurance appeals/denials for tests/imaging that you want done, which may involve peer-to-peer review. You also have to fight rejection claims from insurance companies who do not want to pay you. You also have to have time to review labs and imaging results that you ordered on your patients. And that's just the medical aspect.
Are you using an EMR (which cost hundreds of thousands of dollars)? If not, the government will penalized you with decreasing reimbursement. Are you attesting to meaningful use requirements? Are you up to date since there are new requirements every year? If not, the government will penalized you by decreasing reimbursement. Are you involved in quality assurance/PQRS? If not, the government will penalized you by decreasing reimbursement. Are you, your staff, as well as the business/EMR HIPAA compliant? Are you prepare for random audits (whether for meaningful use, or Medicare RAC, or insurance audits)? What about random IRS audits since you are a small business owner with a lot of potential business deductions?
Do you have an office nurse or medical assistant? How about a good office manager (who can help with the above - don't want to skimp on salary or you can lose a good office manager)? Do you offer health care? Retirement? Remember, they are W2 employees while you are a private business owner ... so you have to pay the employer portion of their social security and medicare taxes. Don't forget the usual business insurances for the building, non-medically related business liability, etc.
Are your diabetics well controlled? Are your asthmatics well controlled? Are your patients getting all age appropriate cancer screening? What percentage of your patients still smoke? The future trend is to punish (or "reward") PCPs who have "good results" and punish doctors whose patients still smoke, eat fried food, and go into COPD exacerbation/DKA/CHF every other weekend ... by decreasing reimbursement
Oh, and while you deal with the above, don't forget to do some Maintenance of Certification stuff to maintain your board certification. And in most states, you also have to do some CMEs as well.
*one reason why I'm glad I'm not in primary care. I joined an established group that had everything already up and running, but starting out on your own, there are a lot to consider ... the above are just a small sample