This will work, albeit slowly--the new water should have 0 CYA in it, so by draining the old and diluting with fresh, the CYA reading will come down. Depending on the CH of your fill water, it should also bring your calcium down, as well.

Out of curiousity, how were you maintaining any chlorine in the pool with no stabilizer? I have a sneaking suspicion that the pool store gave you some wrong info.....

Janet