It has to increase. Maybe you are not getting a true resting heart rate
as a baseline. To be trully resting, take it as soon as you wake up and before you get up or move around. Also, remember that people who are atheletes have exercised thier heart to the point that it becomes very efficient. Meaning that it has accomplished pushing a large amount of blood through the body with each beat. Meaning that the heart rate can be low but accomplish alot....that can be a sign of a true athelete. You didn't say how much you exercise....nevertheless....when you exercise the body demands more oxygen
, etc, so the heart MUST increase in rate....or you'll pass out.