If the change is still conforming there should be no troubles, and in the worst case would require an amendment to the plans. On other construction I've seen this done with handwritten notes and arrows. However as hinted with "no fly zones"... if you happened to violate a property line setback, or violate the angle of repose of an adjacent structure, it could get bad for you.
In our case we added a step and changed some curves on the fly when they shot the shell. All inside the perimeter of the pool shown on the plans. The inspector never said a word about it. However, in other (non pool) construction we once made some minor changes to a structure and the inspector made us get a letter from the architect before OK'ing it. In the end it is often at the sole discretion of the inspector... so my policy to inspectors is to ALWAYS be friendly and polite, NEVER be evasive, but DON'T volunteer information they have not asked for.