After just completing a build by owner myself, I am now very familiar with the builder costs and I recommend you stick to a payment structure that allows you to always have the upper hand. If a builder doesn't want to agree to terms favorable TO YOU, go find another. If things go bad in the middle of the build and you have reason to terminate the contract with the builder due to bridge of contract, its easier to just go find a new builder to finish the job with the money you still have on the bank than go to court and try to get the money you already paid to be returned.
See below. Builders like to combine many of these steps together for their convenience, but I recommend you stick to your guns and pay as followes (for inground pools). Note, if you can break these payments down into even more installments with more detailed milestones even better. MartinKennedy was able to do a great job at that, but that is not likely the case:
5% at signing
15% when pool is excavated AND rebar/form is complete
15% when electrical AND plumbing rough is complete
15% when gunite/shotcrete is complete
15% when backfills, deck, coping and tiles are complete
15% when equipment is installed, final electrical and final plumbing is complete.
15% when pool is plastered
5% AFTER FINAL INSPECTION is passed and job is site is 100% clean and completed.
I don't get why builders consider the job finished with the plaster. The job is finished when the final inspection is passed and the job is site is ready for a party!