Welcome to HCM,
You are right it seems that you are having some hair follicle infection
causing this problem.
Take some anti inflammatory medicine like ibuprofen-paracetamol
To recover from repeated infection go for antibiotic medicine course.
Do not try to press upon and try to remove pus, it will aggravate the condition.
Apply some antibiotic cream inside the nostril
Ok and bye.