Kick him to the curb. Don't make the same mistake my mother did - she stayed married to my ******* of a dad for 20 years, and she never forgave herself.
If he isn't being good to your child, your child will grow up with a father, yes, but she will grow to hate him. If there are bruises, etc, take pictures and show them to the police or at least keep them to remember why you shouldn't be with him. He may have been charming or cute or funny at some point, but I don't find it cute/funny/charming when someone is abusive to their family. I wish you good luck and hope you find happiness.