A 1/2 day with a small machine like a Bobcat S70 would do the trick in no time. If you've never used one, the learning curve is literally about 20 minutes or so. If you have a rental place close, a half day should do it and make things a whole lot easier for you than doing it by hand. I rented a larger S130 for about $300 a day (a few times) , so depending on your area a half day may be about 1/2 that.
Or . . .
For the amount of "depth" you have there, you may consider renting a small John Deere (JD 17 ZTS) trac hoe. It's got a small 8" x 40" blade on the front that you could "skim down" your high side ground for the pool bottom. and then you could use the back hoe and "even out" / cut back that large drop off and get it ready for a small stackable retaining wall or a gradual slope.
I think a half day if you make good time, would be all you need.
I had to rent a larger JD Track hoe 3 times and the Bobcat twice due to my rear yard. (My home is built on a hill and the back yard was terraced with 8x8 landscape timbers (20 years old) and it was a very horrible job.
I "cut down" an area of my rear yard appx 6 feet deep in one section and then prepared a spot for a 33x15 A/G that will essentially be "in ground" / at deck level. It was a TON OF WORK for the site work but came out really good.
I'll be posting some pics in case anyone was interested.