XI Conference – European Congress of Methodology

Bootstrap-F and adjusted F-tests in split-plot designs: The effect of non-sphericity and heterogeneity on Type I error

23 Jul 2025, 17:00

30m

Faculty of Social Sciences and Communication. (The Pyramid)/. - Foyer (Faculty of Social Sciences and Communication. (The Pyramid))

Poster Design/Research methods Poster Session 2

Speakers

Roser Bono (University of Barcelona)Dr Guillermo Vallejo (University of Oviedo) F. Javier García-Castro (Universidad Loyola Andalucía) Rafael Alarcón (University of Malaga) Jaume Arnau (University of Barcelona) María J. Blanca (University of Malaga)

Abstract

Background. Adjusted F-tests are not robust to simultaneous violation of the assumptions of sphericity and homogeneity of covariance matrices when group sizes are unequal. As an alternative, the bootstrap-F (B-F) method has been proposed. Objective. The aim of the study was to analyse the robustness of the Greenhouse-Geisser (F-GG) and Huynh-Feldt (F-HF) tests and the B-F method in split-plot designs under non-sphericity and heterogeneity of covariance matrices between groups. It is hypothesized that the B-F method will outperform the F-GG and F-HF tests. Method. A simulation study was conducted using a split-plot design with two groups and three repeated measures under normal distribution. The manipulated variables were: (a) total sample size, (b) group size, (c) epsilon value (Greenhouse-Geisser estimation), (d) coefficient of sample size variation, (e) homogeneity/heterogeneity of covariance matrices and (f) pairing between group size and covariance matrices (null, positive and negative). Type I error rates for time and interaction effects were computed and results were interpreted according to Bradley's liberal criterion, whereby a procedure is robust if the Type I error rate is between 2.5% and 7.5%. Results. Adjusted F-tests for time and interaction effects were conservative with positive pairing (variance heterogeneity) and a moderate or high coefficient of sample size variation. They were liberal with negative pairing (variance heterogeneity) and a moderate or high coefficient of sample size variation. The B-F method was robust in all conditions when the coefficient of sample size variation was low or moderate. However, with a high coefficient of sample size variation and negative pairing, the B-F method was robust only when N > 20. Non-sphericity had little effect on the Type I error rates of all statistics. Conclusions. The B-F method showed greater robustness than did the F-GG and F-HF tests, making it a valid option for analysing split-plot designs under heterogeneity of covariance matrices in the conditions studied here. Sample size above 20 is recommended. This research was supported by grant PID2020-113191GB-I00 from the MCIN/AEI/10.13039/501100011033.