Some here in this MEGA thread have reported that the black treads have made marks on their plaster. I haven't noticed anything yet but my pool is very dirty still ( 2 cycles in and still needs work). If it isn't dirty or there isn't much debris, I'd continue to brush it to stir up the plaster dust so your filter can get it (and then after it's cured, a filter cleaning will likely be needed FYI). I will say though, leaves left on curing plaster can cause staining so don't let any debris sit for too long.
My 2 cents....