Yeah, I would have a few concerns about the weight, roots, and water drainage on top of the retaining wall. It really depends how the wall was built and the strength of the wall both to vertical loads down but more importantly to horizontal loads pushing the wall outward. I think problems will not show for a few years out.
I have retaining walls around my pool area that are about 20 years old and some needed to be rebuilt as the wall began leaning. My walls are not directly along the pool like your is but along terraced areas.
Only time will tell if your builder built the wall to handle the load the trees add to it.
@Rich D @jimmythegreek thoughts?