UPDATE!
The chlorine tabs helped a bit, so I called in the pool company and we drained the pool, they did an acid wash, and the plaster came up pretty clean -
Not spotless, but enough for the summer, at least -
Recommendation was to refinish, at some point, with mini pebble and that would...