I think you both need a keener set of tests to ascertain whether there is actually infertility
or not. Your husband somewhere along the line might need to consult a urologist
too. Its only after all these that we can come to a comclusion and draw up a better management plan. I suggest you see your gynecologist/obstetrician for a proper investigations a and do urologic consultation for your husband.