From history it seems that there is no proper diagnosis made about your problem.
There might be having some infection in testicles and seminal vesicles.
One side testes is smaller so one should find out of cause of having small testicle.
You may require to go for semen testing as well.Consult sexologist
and get examined.
ok and take care.