These are actually pretty nice to have around the yard if you can get over the psychological aspect of their size and their curiosity. They keep those noisy cicadas at bay pretty well.
Cicada killers are almost never reported to sting humans. Of course they can do it like all wasps, but you really do have to mess with them quite a bit to irritate them, quite a bit. They are fairly curious insects and will often fly at you and around you, more or less, just to see what you are up to.
If you are freaked out by them and want to rid yourself of them, here is what you can do. First off, you need to locate their nest entrance. Just watch them fly and see where they land. Once they land, they will head to the nest which is going to be a 1/2" or so diameter hole in the ground with dirt piled around the edge. To treat it, get you some 10% Sevin dust or diazinon (diazinon is probably better) and dust their nest hole. Be sure to get a little inside and around the hole. They need to touch the dust to get killed. This will kill them in a day or so. Spraying the hole with wasp spray generally doesn't do much because they tend to go pretty far into the ground, farther than you could effectively reach with the spray. Of course, you can try to spray them directly as soon as they land but you must get the spray directly on them in order for it to work. The spray can also harm plants too.
You may need to treat again next year. At the bottom of the nest hole is another larval cicada killer wasp that may emerge next year. At this time of year, there are only two wasps in each nest. Mama and her baby. Later in the year, males appear. They are territorial and somewhat aggressive but they have no stinger.