What pollen doesn't get filtered out will eventually sink to the bottom and clump together into yellowish-brownish areas which might make you think you have algae. It's a very, very soft substance and will swirl up and dissipate at even the gentlest water movement. When you brush it, it looks like a cloud of dust in the water.
It vacuums up easily if you go slowly with the vacuum so you don't stir it up. During pollen season you may have to vac daily, or keep brushing and keep the pump running in the hope of filtering it out.
We've had great success using the solar cover which has kept most, but not all, pollen out of the water. I had to give up on the skimmer socks when pollen was heaviest because the amount of it in the sock had begun to impede water flow from the return. But I did vacuum daily.