This is a very common start-up procedure, especially for exposed aggregate finishes. Your builder usually does this whole procedure.
Leave the pump off at first.
Add 3 1/2 gallons muriatic acid per 10,000 gallons and then brush the entire pool. The acid will help dissolve the plaster dust so you don't need to vacuum.
Brush twice daily but don't vacuum or add chlorine until plaster dust is gone. For dark plaster only brush lighter colored or white areas.
When all plaster dust is gone, neutralize the acid with soda ash/washing soda
by using a base demand test, wait 24 hours and repeat if needed.
When the PH reaches 7.2 or higher, check all equipment and turn on the circulation system. Clean the filter when the pressure increases 10 psi. Add sanitizer in small amounts at first until the desired level is obtained.