Replicating and extending a meta-analysis to determine whether test design affects the magnitude of sex differences seen in mathematics

To trust meta-analyses as a tool, we need to be able to replicate and update findings. To exhibit this, I have replicated and extended findings in Lindberg et al. (2010) – a meta-analysis of the size of sex differences in mathematics. Specifically, I have focused on the effect of question type for children in 11th and 12th grades. I re-analysed research from the original paper covering 1990-2007 and included work from 2007-2015. I retrieved 3593 new records; 314 of which met the inclusion criteria and 46 were eligible for inclusion – 29 of which were new. These studies yielded 99 effect sizes and a sample of 281,193 (152,620 female). Using a random-effects model, I found that studies that only used multiple-choice (d=.03) were larger than those that did not (d=.18) and that the subgroups were different Q(2)=15.4 (all ps, < .001). This study intends to show the importance of test design for education and provoke discussion about how existing meta-analyses can be replicated and extended.