Correct--while you're in school your interest on federal (and most private) loans doesn't capitalize. It capitalizes upon the start of repayment (generally six months after graduation), and then I think it capitalizes quarterly (perhaps yearly) after that while in repayment. If you enter IBR or PAYE, the interest doesn't capitalize again until you leave IBR or PAYE. Which is a huge benefit for those programs if you will be spending a couple years in residency making minimal (IBR/PAYE) payments that don't cover all the interest that accrues.
As far as how much gets taxed, I'm not positive, but I believe that the money forgiven is considered income, so if you have $250,000, assume you'll be paying taxes at the highest tax bracket, probably close to 30% in total. I think you have to pay state and federal taxes, as well as social security and medicare (FICA). I'm not 100% sure though. I read somewhere else on this board that the government takes your assets into consideration. If that's the case, it's probably worth talking to a tax lawyer at some point if you plan to benefit from forgiveness through IBR or PAYE.
Once again, I am not positive about how IBR/PAYE forgiveness gets taxed, because honestly I want my loans paid off within 20 years. I think it's too big a risk to assume those programs will still be around (despite me thinking there's a fair chance they will be). That would just be way too much interest building up over 20 or 25 years, and if the programs did disappear, or the amount forgiven was limited, then I'd really be in the hole.
With all that said, I would only borrow what you need, with the assumption that you'll be paying back every penny yourself. Assume loan forgiveness won't be around, because there's a fair chance it won't be. Honestly, I don't think anyone should take out loans with the idea that they will get them forgiven. My wife and I owe quite a bit, and if she stops working when we have kids, all that loan payback falls on me. My repayment burden will be extremely high in that "worst case scenario", (have kids, wife stops working and never goes back) but it's doable when I run the numbers. I'm doing IBR while in residency, and as then paying as much additional income as we can towards our highest interest private loans (ironically they're at rates much lower than the federal loans, but then if PSLF sticks around, I'd hate to have spent my money paying off money that could get forgiven...)
Hopefully these forgiveness programs will still be around for you (and me) to benefit from, but no one knows for sure. So borrow as little as you can, and you'll be very thankful for it once you begin repayment. I really wish I could talk to my younger self--I thought I understood the magnitude of how much I was borrowing and how much I'd owe when I started my post-bac and then medical school, but I didn't fully understand until I finished school and started repayment.