IF the fibroid
is distorting the uterine cavity
or it is extending up to cornual region then you may find difficulty in being pregnant.
Sometimes there is no difficulty even with bigger fibroids.
Get your other investigations done like HSG , hysteroscopy
to see the tubal patency and the cavity of uterus