it may also be some type of pollen - yours sure looks like it, it's too light to be clay dust.
Do you have tiny specs floating on the surface ocasionally too? that would likely be pollen too. The stuff may just be too fine for the sand filter, so it's recirculating, and of course, over time, your pool surface is picking up more and more of it.