Welcome to health care magic forum
Six weeks cast after minor or hairline crack is sufficient for healing the fracture
.Swelling after removal of cast is generally unusual. as your limb`s swelling is increasing .you must get the X-ray repeated ,keep the limb elevated & do movements of toes regularly.If X-ray shows any thing ,consult the orthosurgeon again.
Wish you good health