My basic test for Algae vs dirt is the touch method - meaning, touch it and see, if it's slimy, 95% chance it's algae.
What you're describing does sound exactly like the trouble algae I referred to back a page. If you're getting it on the walls, it's another sure sign, dirt follows the laws of gravity - meaning it'll go straight down - algae, being a living organsm doesn't have to follow those laws. (like spiderman)
if it's first showing in the areas where less circulation exists and then looks like that area is growing, once again, a likely sign it's algae.
Dirt will keep accumulating in those areas too, but won't really grow it's deposit or settling area into water that's moving more adjacent to the stagnant water.

I've also tried this little test with this particular algae as well.
In my case it always starts on the side of the steps away from my return jet (bad circulation area).
Turn off pump to stop water movement
I've done this from outside the pool and it's MUCH easier because I have a "grabber" (for other purposes), but I don't expect everyone to own one.
You could also rig a broom handle with something like a stiff wire that will support the container for you.
Try taking a baby food sized container full of bleach into the pool - you can keep the lid off because keeping it upright under water seems to leave the bleach in it if it's moved really slow.
slowly move towards the problem area so as not to ciculte the water near it.
Once centered about 1 foot above the area, slowly flip the container allowing the bleach to settle over and onto that spot.
Come back 15 minutes later and see what the spot looks like - if it's dirt - like clay dust, it'll look the same, if it's algae, the color likely changed allready and it might be getting cloudy - recheck at half an hour after.