hello and welcome to HCM,
You are right about the spot. It is known as tonsillolith
or a tonsil stone. Tonsils have small holes on their surface which we call the crypts. Over time in some these get collected with deposits and harden. The recurrent episodes of infection also accelerate the process.
If this is the first time this has happened you do not have to worry, see an ENT and the doctor will remove them for you. However if you are getting this problems more often then you may undergo a minor tonsillectomy
to completely cure you of this problem!!
hope you recover soon but please do not meddle with the brush
Dr Sriram Nathan