Speakers
Description
In meta-analysis, the Q statistic is traditionally used for testing the hypothesis of homogeneity of the parametric effect sizes of the set of studies. Several critiques have been posed to that test, especially when applied to the standardized mean difference (g). Among them, that the weights are based on estimated, not true, variances, that the variances of the estimates correlate with the own g values, and that it is assumed a wrong distribution of g (normal) although it is actually a linear transformation of a Student’s t. We present an improved test of homogeneity of g values based in the Mixture Model of Suero et al (in press) in which most of the problems highlighted are solved or greatly reduced. Specifically, the variances of g in the studies are independent of the own g values, and the true distribution of g is acknowledged and transformed to a normal distribution. Although the variances are still estimated, we show that their impact in the performance of the test is negligible under the Mixture Model. We present the results of an extensive Monte Carlo simulation to assess the performance of the classical Q test, an alternative that use weights based only on the samples sizes (effective sample size, Qñ) and two normalizing transformations (those of Johnson-Welch and Laubscher). The results show that whereas the classical Q test yields a too low rate of type I errors and Qñ an unacceptable large rate, the two normalizing transformations yield rates within a comfortable 4% –6 % range. Furthermore, the rate of correct rejections (estimated power) is always higher than that of the Q test. Taking all the results together, we conclude recommending the new test for meta-analysis of g values, using the estimates provided by the Mixture Model and the normalizing transformation of Johnson-Welch.
