A man has been charged with causing death by dangerous driving after a three-week-old baby died in a car crash.