Sound's like dead algae. I don't know what sort of vacuum you have or why you can't vacuum it up.

To vacuum it easily you need:

1. A wheeled vac head (a brush head stirs it up, unless you've got super-suction)
2. Good suction
3. Patience to move slowly enough so you DON'T stir it up.

We do it every week, on a 200,000 gallon pool.
Brush type: Poolmaster 27514 Triangular Vinyl Liner Vacuum
Wheeled type, medium duty: Poolmaster 27200 Heavy-Duty Flexible Metal Handle Vacuum
Wheeled type, light duty: Swimline 8150 Weighted Flex Vacuum Head, Blue
Hydro Tools 8365 8- to 16-Foot Adjustable Fluted Premium Fluted Telescopic Pool Pole, White
Poolmaster 33430 Premium Vacuum Hose with Swivel Cuff, 30-Feet by 1-1/2-Inch