It sounds like your pool is swimmable. I think my approach would be to manage the pH down to about 7.0 and bring the FC up to about 50% greater than what we suggest in the FC/CYA chart. Then maintain that condition for a couple of weeks. I think you will get some dissolving of that stuff on the bottom of your pool. Not sure just what it is but you will be attacking it on two fronts.