I recently visited my sister and had the pleasure of sitting on her Balance Ball Chair while working at my laptop (it travels with me wherever I go). Although it was a little weird at first, I got the hang of it quickly and found myself sitting with such perfect posture. I can see how, if not accustomed to sitting on a ball, your low back muscles could get tired from finally working and feel tight after sitting on the ball for hours. The lower spinal muscles get weak with all the slumping we do, but you can't really slump when using a balance ball chair, so those muscles that actually support your vertebra work!!!
The chair is made by lifestyle company Gaiam (they make solar energy products now too!!!). Gaiam designed the chair with chiropractic pioneer Dr. Randy Weinzoff, as a tool for strengthening core muscles and improving spinal alignment. The 52 cm ball is removable (imagine all the exercises you can do with the ball when it is out of the chair) and the chair is designed for folks between 5'1" and 5'11''.
The only downsides I can see are that you can't do the chair squat desk stretch in this chair and that it is not the most attractive piece of furniture out there.
Buy it from Gaiam directly for the sale price of $80, marked down from $100.