Please do not hijack another's thread......it is very confusing. here is your new thread.
I have this problem as well -- Bought this house in July of last year, has a DE filter which is pretty uncommon for this area, I've had a pool with a sand filter so I thought it was a problem with the filter (pool inspector told us we had a sand filter). Had another "expert" come and look, he said that it was just fine "dirt" that was missing the filter and to backwash every few days. I did a "big" cleaning of the filter in October -- looked like it had not been cleaned in quite a while -- I had a really hard time getting the DE and debris out of the bottom of the filter but after three hours of putting the filter back together (it was missing parts and several nuts had rusted) I turned on the pool and things were fine, but just a few days later the "dirt" was back.
We regularly have the chemicals checked and haven't had an algae problem (we get full sun so we keep a REALLY good eye on it). Vacuuming gets rid of the dirt, like the above poster, but just a few hours later it is back. We also have a water loss issue, although we can't figure out exactly why but possibly an issue with the spider gasket, that may or may not be part of the issue. We've also had to recently had to replace the "main drain" line due to a tree root invasion but the "dirt" problem didn't get better or worse.
Pool is approx 15 years old, 20K gallons, freeform IG gunite, looks like everything is original and has a DE filter.