Speakers
Abstract
Three-way interaction models are powerful analytical tools for investigating how combinations of multiple factors influence outcomes across fields like management, psychology, and the social sciences. Unlike two-way interactions, which focus on how a single moderator affects the relationship between two variables, three-way interactions enable researchers to analyse the joint effects of three predictors. Statistically, three-way interactions are analysed using regression models that include the main effects of all three predictors, the pairwise interactions between each combination of predictors, and a three-way interaction product term created by multiplying the values of all three predictors. While effective, this approach is inherently sensitive to deviations in the data, as the multiplicative nature of product terms amplifies even small deviations in the predictors. This sensitivity makes three-way interaction models particularly vulnerable to issues that compromise data integrity. Consequently, our study focuses on measurement error and outliers, as these are among the most critical factors affecting the validity and reliability of interaction analyses.
Measurement error in the predictor variables becomes exacerbated in the product term, attenuating true effects and decreasing statistical power. Outliers distort interaction estimates disproportionately, as even moderate values in one or multiple predictors can result in extreme product term values due to the multiplicative logic of the product term. While two-way interactions may still yield meaningful results under such conditions, that might not be the case for three-way interactions. Our Monte Carlo simulations demonstrated that measurement error in the predictor variables significantly reduces the reliability of the product term, leading to diminished power, downward-biased regression coefficients, and smaller effect sizes. Outliers were found to exert disproportionate influence by creating extreme product term values, resulting in spurious interactions or obscured genuine effects. These challenges were exacerbated in smaller samples and smaller effect sizes.
To address these issues, we recommend tailored approaches: Power analyses that explicitly incorporate predictor reliabilities to ensure adequate sample sizes for detecting interaction effects, even with measurement error present in the data. Scale shortening, often used to address practical constraints like survey space or participant fatigue, should be approached with caution. Our findings indicate that reducing scales to the commonly used minimum of three items significantly undermines statistical power for three-way interactions, even for high sample sizes. For addressing outliers, diagnostic techniques focusing on the dynamics of product term are essential. Initial evaluations of product term distributions via histograms can identify extreme values, while targeted methods like DFBETAS effectively pinpoint data points exerting disproportionate influence on interaction regression coefficients. Unlike more general detection methods like Cook’s Distance, DFBETAS directly assesses the impact of individual observations on parameters affected by outliers. Non-linear transformations and robust MM-regression were less effective in mitigating distortions caused by outliers in our simulations, and we therefore do not recommend their use for outlier diagnostics.
Overall, this study contributes to the methodological toolkit for analysing three-way interaction models, addressing critical gaps in the literature and equipping researchers with strategies to handle the challenges posed by measurement error and outliers.
| Oral presentation | Unpacking the Impact of Measurement Error and Outliers on Three-Way Interactions: Evidence from Monte Carlo Simulations with Best-Practice Recommendations |
|---|---|
| Author | Christina Andres, Torsten Biemann, Jeremy Dawson |
| Affiliation | University of Mannheim, University of Sheffield |
| Keywords | Three-way interactions; Measurement Error; Outliers |
