There is very remote chance of pregnancy
if you touched the top of vagina with semen smeared penis.
But for this to happen,you're wife should have been ovulating.
There does exist a micro possibility of conception but that would be a very rare thing.
I think there is no point in loosing your sleep over his matter.
Just wait for next periods and if they are absent,then go for pregnancy test.
Any decision should be taken after that.