I think you need one gynaecological examination. According to your discription , you might be having bartholin's cyst
of vagina. This might have been infected. You require a kind of antibiotic course and then if problem persists, go for excision and evacuation of the same. Do not worry as it is a minor procedure.