TA will not harm swimmers nor hardware.
Do you have scale forming anywhere?
Normally, if you manage the pH in the 7's, the TA will take care of itself.
The one reason to push TA lower is to reduce pH rise rate, or if you have a SWCG and it is creating scale inside it.