Beautiful place! Good luck!
Coming from a construction background I'm going to weigh in on this one.
I would absolutely, positively NOT tear up any of that floor slab unless I had no other choice. A mid-century house on a slab, especially in New England, has a very high chance of having asbestos wrapped pipes and ducts in the floors. It was very common back in the day and will cost you a fortune if you have to mitigate and encapsulate that due to someone accidentally hitting one. We're doing a project now for this exact reason.
If you're concerned about mold in the walls, it's easy enough to find evidence of it without doing a lot of work. Take off a few switch plate covers. Look closely at the plaster surrounding the boxes. If you see dampness or discoloration, that's a clue. Check carefully around the glass. Look for rust, discoloration, old water staining. Look at all the paint for peeling, streaking and touchups especially in the high corners and near windows. Those are clues. If the humidity in that room is very high, the walls will condense especially if they're plaster. You can pick up a moisture probe at Home Depot cheap. You'll make a few pinholes in the plaster but you'll have an answer about the walls and cavities.
You didn't say the size of the room or the pool but if you know, please post those up. I'd want this room zoned separately from the main HVAC system. Several ways you can do it. Either with a zone control system or on it's own system. With the high ceiling, amount of exterior glass, skylights and humidity levels in that room I'd recommend a completely separate system over zoning. It will cost you more to do it that way but it will be worth it if you can.
Being in New England, your heating demand is going to be far more than the cooling demand. Since the current AC system isn't functioning, I'd get a GOOD HVAC guy out there to take a look at what you have and give you some options. Emphasis on the V! Too many in the industry do not understand proper ventilation. Get several prices. Correct airflow will be just as important as humidity control. Do NOT let them oversize the AC. Bigger is not always better and especially not in this situation. Whoever comes out needs to break out a psychrometer and actually measure the humidity levels. If they don't, show them the door. If they won't actually discuss what they propose and explain "why", show them the door. You don't want the guy that comes out and just says 4 ton AC here already, OK here's your price.
Another thought on ventilation if you get the house. Consider adding two large ceiling fans. Slow turning commercial type fans. You don't need a lot of movement but just enough to circulate any dead air space above the beams and would help solve any moisture issues up there. We've used several from Big Rear Fans and they're my go-to for high quality, quiet and non-industrial looking fans. I think two would balance the room better than one or three from the photos.
With radiant floor heating, I'm presuming a natural gas or oil fired boiler. Fuel cost and efficiency wise, natural gas is much better. Discuss with the HVAC guy the possibility of going to a dual fuel system. May or may not be possible with the current setup but I'd strongly consider it. If you put in heat pumps and not straight AC systems, the added cost is negligible but you'll have a few advantages. The units can be wired to only run for AC and not use the heat function or you can run heat mode in the shoulder seasons and switch to the radiant floor in colder temps.
I'd also highly recommend getting the AHRI numbers on any new HVAC equipment. With the new SEER2 ratings that took effect in January, MANY equipment ratings and matchups are no longer meeting EnergyStar and the federal tax credit requirements. I'd be looking at the Carrier Greenspeed systems and the Bosch M20 systems, both with an added dehumidifier. Bosch also makes a very nice high efficiency Combi boiler.
If I can answer any questions, feel free to DM.
