I understand your feelings
On second thoughts,
You are right
You must give maximum attention to your child
They need your caring throughout day
Affection is all that they want now
It is advisable to allow her to sleep with you upto the age of tleast eight.
Then you shall think of independance to her
Tell your husband so and take him to a child psychologist
if need be.