Ferrer was preparing to serve in the second set when the child started crying loudly. He decided to play through the distraction, and ended up losing the point. In a moment of frustration, he hit a ball towards the source of the noise (which, again, was baby). The ball came nowhere near the child.
Now, hitting a ball at a crying baby won’t make you the most sympathetic of figures. Ferrer was a foreign player, playing an American, in America, and hit a tennis ball at an American baby. Yes, he was booed loudly.
But as Yahoo! notes, bringing a baby to a tennis match, a setting where silence is key, was probably not the best idea. Just saying.