I am going on vacation next week. I'm planning on bumping up Cholorine with bleach before we leave but after a few days I was going to have my dog sitter add a floater with some pucks to get me through the last three days of vacation. My question is she will be turning my pump on in the am and off in the evening. Should the floater be pulled out when the pump is off?

With a floater it doesn't need to be removed unless there's a good possibility of it getting stuck close to something it would bleach or corrode. There's not much of a chance that will happen unless you have some special circumstances.

When i used to float pucks I would put the "float" under my solar cover and in front of return and it would hold there nicely. During day chlorine would be distributed nicely and kept away from liner and be out in the pool when it sat there overnight just soaking (I leave my pump off at night). If no solar cover you could always turn it into a "buoy" instead of a float and tie an anchor to it ...