Hi, Welcome to Healthcare Magic Forum.
There is no harm if you drink water at room temperature. If you feel nauseated, you can boil the water, bring it down to room temperature and then use it. It is always better, if you drink warm water or a filtered water during pregnancy
to prevent infections.