Do you also have an acid neutralizer? That is a common setup with well water to have have both the softener and the neutralizer--it's a second smaller tank. And, with well water's pH of less than 7.0, it's VERY likely, because otherwise, over time, the acidity will weaken the pipes in your house.

If so, that answers Chem_Geek's question right there.

Either way, unless you bring in other water, you wil have to do a lot of the T/A reduction. Be VERY careful not to add ANY calcium until T/A is in the proper 80-125 range for nomal pH or your water could go cloudy.