I've been using Zeolite, and I do Add stuff through the skimmer, it has NEVER caused a problem.
I do understand why you'd have to backwash on first use, the zeobrite/lite is a mineral and during transport it actually grinds against itself creating a fine powder, this powder needs to be eliminated and not blown into your pool.
However, if it goes into your pool, just shut the pump off for a day or two, let it settle (even adding some floculant will help bind it with the rest of the foreign stuff), add some DE into the skimmer and vac it out - the DE will catch it and you can backwash it out.