Algae, if not green is dead and could be whitish. Floating on the surface could be either dead algae or other type of debris. Pollen is yellow. If you use skimmer socks, the pollen will be caught in the sock (This is what I use). Also, pollen has a tendency to sit on the water line and pool wall surface (again yellow fine as it is pollen). When I manually vacuum, the amount of pollen caught that is sitting on the floor is very heavy, so my guess is you have pollen and not dead algae.
Even if you water is clear, algae can exist. Brushing walls as routine maintenance is important. How are you testing your water? Please add your signature as this helps other members with your pool information.