I don't have a lot of experience with household/pool pipe backfilling, but I worked in a power plant for over 30 years and we had a lot of buried piping. this may be overkill, but we always filled from 2 inches below the piping to 2 inches above the pipe with sand. We found that backfilling with natives soil, gravel or stone led to additional leaks. When there is flow thru the pipe, you get some vibration, eventually a stone against the pipe wears thru the pipe and you have a leak.
When my pool was installed , I insisted of packing the pipes in sand. Again, that may be overkill but it's cheap insurance.