Sorry to hear about your problem but I am glad that you are seeking medical attention for the same.
A family history of breast cancer
does put you at a higher risk of getting the disease but regular screening can help you detect it early.
An inversted nipple for so long does not confer any risk of cancer. Is the discoloration of the nipple on the other side associated with itching as well? When was the last time you had a mammogram?
I would advise you to visit a breast surgeon and have your breasts evaluated and also get an imaging test done.
Hope everything turns out to be normal.