Welcome to health care magic.
From your description, it seems to me that you may be having recurrent angioedema
. There are several causes for including food, medications, infections, infestations and familial. Since you get it infrequently, I suggest you to keep a food diary on the day you get the swelling. Note down everything that you have ingested that day and after a month, just check if you have food item in common and try to eliminate that to see if the swelling episodes subside.
You can also take an anti histamine
on the day you get the swelling for faster relief.
Hope this helped you.