Thanks for posting your query
You had not mentioned exact what fracture
& whether is displaced or not & what type of treatment you had get.
Now three weeks are passed, swelling may occur if your limb is not elevated most of time.
So, you have to visit your doctor for clinical exam and x-ray if needed to confirm whether fracture is united or not.