Get a check x ray
done to assess for healing status. Usually bridging with callus
(new bone Occurs in 4- 6 weeks) but re-modelling may take upto 6-9 weeks or longer,, your Doc.should be able to tell you objectively after check X ray whether it is safe to remove. In all probability it should be but after cast removal some wrist support may have to be worn for few days, with fomentation and gradual wrist exercises that you would be prescribed. Support will be a must if there is local tenderness on fracture
area indicating support for few days more.