Your doctor is correct for several reasons.
As pointed out correctly in prior answers there should be adequate time between readings else
venous congestion will give erroneous readings.
I suggest that you take your pressure readings near the same time. For instance at 8PM, when you have been sitting relaxed for some time. Remember that our BP is affected by what we are doing, how we are sitting, and
stress levels. During sexual intercourse a systolic (the high number) reading of 180-190 is quite normal. So it is not the "excursions" or occasional highs, rather it is the overall average when the body is calm, supported, when the cuff is at the same height as the heart (important with wrist cuff devices). Even for instance if you take your BP at 8PM on Mondays and Wednesdays, there will be differences, and there will be between the left and right arm. So use the same arm, and look for "the trend" or the average.
Looking at the average of the two numbers it is likely that your doctor has you on a med regime that is appropriate and working. I would suggest though that you stop
smoking (if you do) lose weight (if you need to) develop a cardio-vascular exercise program (seach target heart range calculator), make sure you eat plenty of foods with potassium, and if you are black, East Indian, or Hispanic, try cutting back on salt, as it may make a significant difference.