In the UK there was a tragic case of Baby P (Peter)...who was regularly tortured and and beaten, which led to his death at just 18 months old or so...
It brought down a (govt) social worker unit director, and a hospital Dr that failed to pick-up the kids back was broken... she did say he was particularly sullen and withdrawn for a bay his age though (doh)...??
The man whom did this too the child was not the baby's natural father.... so I think its fair to state that in some instances (not as infrequent as you might think) men with base aggressive feral natures, with psychopathic tendencies can be overtly hostile to children not of their own making...
(Whether showing atypical baby behaviour or not)
It's an awful aspect of life that for me is way too common..... but what the answer is with this, well there are no simple easy answers...?
As these men can have their own brutal history's with dysfunctional family backgrounds too...
How a child is nurtured or not...does have impacts on the future adult....