The major cause of fatty liver
disease is alcohol as you must have already noted. Since the liver is the major organ responsible for alcohol digestion or detoxification
it turns to accumulate a lot of fat converted from the high calories contained in alcohol. Ones you are already having fatty liver due to alcohol, it is advisable to stop completely alcohol as recurrence is most likely and continues fatty liver will definitely lead to hepatitis and hence consequently liver failure
which when it complicates results to irreversible damage to the liver such as a fulminant hepatitis
or hepatic failure/ liver cirrhosis/hepatocellular carcinoma
These complications which started initially as fatty liver (even though you see it as mild diseases) resulted to liver failure and subsequently early death.
I totally agree with your doctor to stop alcohol. You can reduce in take but that would not still remedy the situation. Your liver is prone to fatty liver, it is best you quit drinking.
Even if your fatty liver is of non-alcoholic origin, same counseling still holds
Hope this helps