... There is what appears to be an overflow ... port in the skimmer? ...
Take a look at the two photos in this post.
Is that second hole in the bottom of the skimmer what you are talking about when you say "overflow"? If so, that second hole most likely goes to the floor drains, and your float valve (aka diverter valve) is missing (see the second photo). Without the float valve, the floor drains are non-operational.
How often are you backwashing when it's on filter-mode vs recirculate?
Okay good. Wanted to make sure you had that filter going as often as you can. Just curious: what does a glass of pool water look like? Can you see through it?
Leave the glass out and see if you get any precipitate. Again, just curious.