Say you've got a wood pile -- that's phosphates (and nitrates) -- representing algae nutrients.
Say you've got sparks flying into that wood pile -- those sparks are live algae blown into the pool.
Say you're spraying that wood pile with a fine mist of water -- that's chlorine.
So long as you've got enough water to put out the sparks (i.e. enough active chlorine to kill algae), then the size of the wood pile (i.e. phosphates) is irrelevant. You won't start a fire.
If the active chlorine level gets too low so doesn't put out the sparks and they instead turn into flames, then it will take a higher active chlorine level (a lot more water) to put out the resulting fire (algae bloom). However, once that fire is out, you can lower the active chlorine level back down to the level needed to put out the sparks.
Yes, you can use a phosphate remover to lower the amount of wood in the wood pile, but it's not necessary so long as you maintain the proper active chlorine level. So a phosphate remover needs to be viewed in the same way as an algaecide -- it's insurance if you are unable or unwilling to maintain a proper FC/CYA level, but it's not necessary. And, of course, it's extra cost. And as was pointed out, it's not 100% reliable because if the chlorine level gets to zero, then bacteria can convert organic phosphates (which phosphate removers don't touch) into orthophosphate that algae uses for growth.