If I were concerned about Chlorine, I'd not drink tap water, nor maintain my pool, or use my pool, clean my bathrooms, or sport the whitest of white laundry.
I do or have all of the above. So no, I'm not concerned.
It is rumored that pretty much everything causes cancer. From power lines...