It partly depends on the quality of the plaster, but nearly everyone says that being within +/- 0.3 is perfectly fine and some say that +/- 0.6 is OK as well. Any problems with being too negative are going to take a while to show up anyway unless the pH is very low. As for scaling with being too positive, most pools don't see that showing up until at least +0.7 or for others +1.0. However, in saltwater chlorine generator cells it shows up a lot more because the pH near the hydrogen gas generation plate is high so such pools can be run at a somewhat negative saturation index.

I'll bet you aren't varying by more than 0.3 anyway. It sounds, however, like your pH is changing more than it should. If your pH tends to rise, then your TA is probably too high.