Thanks for your query,based on the facts that you have posted it appears that you have what is called as Infected Sebaceous cyst
on shaft of the penis.
The penile skin is rich in sebaceous glands and hence prone to get more sebaceous cyst due to accumulation of sebum beneath the skin.
Normally they fade away without treatment however they need to be treated if increase in size or get infected.
Please take broad spectrum antibiotic like Cefexine along with anti inflammatory drug like Diclofenac
twice daily.for a week .
If persists even after taking antibiotics you will need to consult qualified General Surgeon
for clinical evaluation and further treatment.
If needed it may need to get excised in Toto (Completely along with the sac)under local anaesthesia.