Sores in vagina. Could it be an allergy from the soap?

Hello, about a year ago I noticed I had three sores on my vagina just on the inside of my lips . They were sore and painful, the looked almost like a blister . Pea sized. They had a head with an indent on it and the center was dark colored. I had been to the gynecologist prior to that and tested negative for everything and hasn t been sexually active since. I had used st. Ives citrus soap so I thought that might have caused it. A year later, I still have a little white place where one of them was. It is not painful or itchy, just a small white place left from it. What was it and is it serious?