You most definitely should not give the builder all of the money, and technically you don't have to pay
any of the final amount just yet. I can only offer you how this would work in California, so you may have to do a little research to see how much of this applies in FL.
"Pool fill" is reasonably unambiguous. But "start up" is not, not at all. That could mean just about anything. In general, the burden of making a contract clear lies with the author, or rather the "supplier" of the contract, and in this case that is your builder. It also falls to him because he is the "professional party" and should have the experience and qualifications to supply a proper contract, something consumers don't typically know how to do. In CA, the format and contents of a construction contract is very strictly regulated, and it is the contractors lawful duty to make sure his contract is legal and "up to snuff" in all areas. If it is not reasonably clear to you what "start up" means, then that is his fault.
"Start up" could mean when the pump starts up, or it could also mean when the "start up" procedure is completed, which some would define as the actual chemical start up process, which could conceivably take a day, a week, or a month. Or it could mean just about anything else. I followed the startup procedure as defined by the National Plasters Council, which is aptly named "Swimming Pool Start-Up Procedures", which took 28 days according to their guide (
here and
here). It's not unreasonable to use their definition of start up, as they have been around a very long time, as have their start-up procedures. They are well known and could be considered an industry standard.
The point being, the contractual term "start up" was not properly defined in the contract, and your builder doesn't have any more rights to defining it now than you do. So you are at an impasse, and so the logical solution, at least the first to try, is to negotiate.
Explain your position, you could even state that you assumed that term meant the completion of the chemical start up process (which is what it
should mean, IMO), and then make him an offer, or wait until he does first, and then counter with whatever you can live with. One way to go would be to pencil out the monetary value of the outstanding work and materials, and subtract that from the amount he is seeking. Pay him the balance, and withhold the rest until all the outstanding work is done to your satisfaction. This is reasonable, and fair, and puts no undue financial burden on either of you. Any judge would also find that reasonable. If he squawks, then offer to settle the matter in court, and see how he likes that idea.
If he is owed more than the remaining work, then you are holding the cards. He could retaliate, and drag things out, or take you up on your offer to go to court, or short you in some way (like lessening the quality of the outstanding labor or materials), etc, so this move is not without risk. But, IMO, that risk is less than the risk of handing a contractor final payment before the work is done, because everything I just listed, or worse, could happen anyway.
As a sidebar, I would also insist the the pool pass final inspection (if there is a permit) and that all subcontractors furnish you a lien release for the work they each performed. Neglecting to tie up those loose ends can come back to bite you, both in terms of the build, and the money you might owe someone.
Be fair, but be firm. Resist any sort of emotional expressions. Stay calm, don't be confrontational. State the facts. State your proposal. Make it clear that you want the pool completed, and that you sincerely want him to get paid in full as soon as possible, as soon as the work is done. No reasonable person could fault that. Leave the rest to him. Do it in writing if you can. Have a witness present if you can't (not a relative or spouse). Follow up any in-person discussion(s) with an email that outlines everything that was said and agreed to. Good luck.