Yikes. The ground is so hard and almost impossible to dig down more. I was hoping with concrete blocks and the sand and sand cove we would be ok.
If you have clay, it'll be rock hard when it's really dry.
What I ended up doing was finding ways to soften the clay by getting it wet, but not making it muddy. If it becomes muddy, it becomes sticky and just becomes a messy disaster. This means taking your time. Soaking it with a hose isn't going to get deep enough and will make the top mud.
A sprinkler misting it over night slowly and then allowing things to dry for a few hours may help greatly, but you need to keep a close eye on it. Do not let water pool in it anywhere.
Damp clay is really nice to work with. Dry clay is like concrete, literally, you can often break tools. Wet clay is a nightmare, just as bad as dry clay really.
Also, I found I could backfill using clay and tamping it and then allowing it to get fairly wet and then tamp it again. Backfilling DOES work,
when done correctly. When done wrong, forget it, you're tearing the pool down and doing it over. This can come in very handy if you find you dug a block down an inch too far etc. Instead of digging the rest down an inch, backfill, tamp, get it wet, tamp, get it wet etc and make sure it's 100% packed and check everything again. DO NOT just toss loose soil in the hole and call it a day, that'll settle, a lot. A simple 2x4 on end can actually tamp really good because it's such a small surface area, but it takes a lot of time. But is reasonable to use for an area the size of a patio block.
In your case you need to keep digging unfortunately until most of that rail is firmly on the ground. If there's a minor gap here and there go ahead and fill it in with soil and pack it, but what you have now isn't going to cut it.