Just to update, I chiseled out the old deck anchors and cemeted in new brass ones. I expect brass and a PVC ladder will solve the issue. After the cement dried I found that I was off 1/2 inch or so on the location of the anchors so the ladder doesn't rest flat against the pool wall but it not too bad.

I still wouldn't trade the SWG for anything.

Dave