Gender stereotypes take root early. According to a study in the journal Child Development, girls' and boys' attitudes about math begin to diverge as early as second grade.
The researchers studied 247 children in Seattle-area grade schools: "Boys associated math with their own gender while girls associated math with boys. In the self-concept test, boys identified themselves with math more than girls did."
Source: Cvencek, D., Meltzoff, A. N. and Greenwald, A. G. (2011), "Math–Gender Stereotypes in Elementary School Children." Child Development, 82: 766–779.