Disclosure: I am currently a full time senior ER clinician at a corporate hospital that also has ECC residencies among other specialty internship/residencies, so I'll give you my take on it all based on my experience at this place. This certainly would not apply to all hospitals.
QOL will vary quite a bit. In my hospital, CC only takes overnight transfers and referrals from primaries from 7a-4p. If the primary didn't call first, it arrives at 4:01p, or they just plain don't want it, it gets dumped on me on ER. They are far better protected from being stuck late or overloaded with cases than we ER doctors are at the same hospital. In one of the sister hospitals, the CC service will leave by 2pm on weekends if they are in a lull, leaving me to receive anything else they would otherwise have taken in addition to all other ER cases. So where I'm at, the grass often seems greener on the other side.
You'd get paid crap as a resident, but can make quite a nice salary in private practice - academia, probably not so much. My DACVECCs earn commission on their own cases in addition to the cases their residents are taking lead on. When you think about it....each hospitalized case that stays for more than 24 hours and actually gets a good workup is going to be $3-5,000, getting higher as patients stay longer or are sicker. A DKA will easily hit 5-7000 by discharge, higher if it stays a really long time. . I'd estimate that our CC service has 10-15 inpatients as a daily average. on top of receiving That is a lotttt of money. I believe our specialists make 20% production on commissionable charges, but we don't have the same contract as ER docs so I don't know the full details, but they are certainly earning more than most ER docs in the area. How much more, or how much less, would depend on whatever your ER contract/pay structure would be. I get straight salary and probably make 1/3-1/2 of the boarded CC's without having an intern and resident on my service to do client comms, paperwork, etc. The highest ER salary I've heard of in my area is 200k with production, and that clinic also has their ER doctors doing surgery (we don't at my clinic..thank jeebus).
TLDR: IMO, the salary of a criticalist can be bomb in private practice. For academia, you could fairly easily look up the salaries of clinicians at public institutions - when I had looked out of curiosity a few years ago, most of the clinicians at my vet school were earning 80-120k/year with no production regardless of specialty. That's kind of salary is almost guaranteed for ER, so in that sense, it may not be super beneficial to specialize. QOL will depend highly on how your clinic functions.
It is also worth mentioning (this is something that I feel strongly about given my current situation) that as an ER doc you will honestly be doing a lot of the same level of medicine as a criticalist/CC resident without the pay. If a referred case gets here at 4:01pm, it's on me, regardless of how near-death it is. I run more codes than the CC service does. I am taking care of CC's patients overnight when they are home sleeping. We don't wait for CC to return in the morning to start treating a DKA. From my point of view, if you can stomach 3 years of $30,000, the extensive reading outside of work, and sitting for boards, I would recommend you strongly consider getting boarded. You'd get better pay for very similar cases/workflow compared to ER. However, that only really applies for private practice. If you dream of academia, it may not be worth it.