My 5 year old stepson absolutley hates me?

Me and my new husband have known each other since we were 5, he gets on with my children, and i get on really well with his daughter, but when his son comes to visit, even though he is 5 he scares me, the child's mum is an ex junky but still smokes, i've tried to make allowences , but if looks could kill, well thats what it like. the mum has been with her partner since the child was 1, please advise