XI Conference – European Congress of Methodology

Reporting of standard deviations and pre-post correlations: implications for effect size estimation in meta-analysis.

23 Jul 2025, 11:30

30m

Poster Statistical analyses Poster Session 1

Abstract

One key advancement in the development of meta-analysis methodology is the standardization of the effect size (ES), which allows for the integration of studies under a common metric, but raises the question of which standard deviation should be used. In repeated measures designs, where the ES is defined as the standardized mean change, some authors recommend using the standard deviation of the difference scores (SD). However, some studies do not report the SD, leading to the exclusion of such studies. To prevent the loss of valuable data, the SD can be estimated from the pre-test and post-test variances, along with the pre-post correlation coefficient. This correlation is also necessary to estimate the variance of the ES, which is then used in the model weights. A common challenge is that some studies do not report this correlation, requiring researchers to impute it in some way. A widely used approach is conducting sensitivity analyses by imputing different correlation values and comparing the resulting estimates, although some authors recommend using the average correlation reported in previous studies. We analysed a dataset of primary studies from multiple meta-analyses in clinical psychology to determine the percentage of studies that report the SD and the pre-post correlation, as well as the range of reported correlation values. Finally, we conducted a sensitivity analysis by recalculating the combined ES for each meta-analysis using the mean correlation reported and the most commonly observed values. The implications of these findings are discussed.
Funding: MICIU/AEI /10.13039/501100011033/ and FEDER funds, European Union, grant no. PID2022-137328NB-I00