Hi Welcome to Health Care Magic
What you are describing me may be Skin Tag. If inflamed by scratching/ puling will cause pain.
You can do few thing.
Tie it at the base with string and It will cut off. Mean while use Ibuprofen
600mg 3 times a day.
2. Consult Dermatologist
. He will cauterize with nitric oxide or laser
3. If you go to Surgeon he will cut it ans suture it.