Thanks for the query.
Yes, that could be bartholin abscess
You better consult gynecologist get examined and take treatment. Because antibiotic treatment is needed to control the infection.
Household remedies may not help always and can also increase the symptoms. So better to go for medical treatment.