That piece might have gone down but you should see an ent specialist
and get done a Flexible fiberoptic nasopharyngolaryngoscopy.
To see for the piece of rubber whether it is still there upto the level of vocal cords or not .
If you do not have difficulty in respiration then thats the good part.
But try not to get late and see a doctor as soon as possible and get examined.
in doubt do reply