XI Conference – European Congress of Methodology

A Comparison of Crossed-Random Investigations of Educational Leadership in Psychology and Economics

23 Jul 2025, 10:30

15m

Oral Presentation Design/Research methods Session 11: "Education, Accesibility and Methodological Critique"

Abstract

One methodological challenge in social sciences is the understanding of practices in different areas that study the same topics. We address the interdisciplinary understanding between psychology and economy in research about educational leadership. A recent call has been made to extend multilevel modeling (MLM) into econometric work (Oshchepkov & Shirokanova, 2022), which we do with the case of crossed-random school data about grades and teacher ratings of principal leadership- nested both in principals and in schools. The goal of our work is to compare the treatment of crossed random data, and the causal inference claims that can be made, in the typical psychological approach versus the economical approach. Both economists and psychologists study educational leadership and often see for a goal to make causal inference (Höfler et al., 2024, Martin et al., 2021). The different expressions of analysis and conclusions and the terminology in respective tradition do however hinder conversation and knowledge exchange. We aim to promote mutual understanding between the two approaches.
The crossed-random data structure occurs when two different units partly overlap at one and the same level of the data hierarchy. In our example, the overlapping units are schools and principals. Since principals quite frequently change their jobs in our longitudinal data of 3445 principals, they do not entirely overlap, which enables analysis of the magnitude of principal’s influence on their schools. The crossed random context for data in educational leadership research serves as an example of an analysis that would be managed differently with research methods typical to psychology versus economic research. With a psychological approach, as well as in organizational research (Eckardt, 2021), an MLM approach would be employed (e.g, Snijders & Bosker). In the MLM approach, the variance of data at different levels, like students and schools, is explicitly modelled. In the econometric approach, the variance that stems from a hierarchical data structure instead is ascribed to the error variance compound of the model, by means of fixed or random effects (e.g., Böhlmark et al., 2015). We compare the analytical frameworks and the conclusions that can be made to understand whether they differ.
We focus on two aspects for the comparison of analytical procedures. They are estimation methods and the levels of analysis at which interpretations are made. The estimation method in MLM is often maximum likelihood (e.g., Rockwood, 2020) whilst econometric models employ least squares estimation techniques (Greene, 2019). The main procedural difference caused by the different estimation techniques is perhaps the simultaneous estimation enabled my maximum likelihood. The simultaneous estimation considers the interdependencies of parameters which lend accuracy to standard deviations and confidence intervals. Meanwhile economists do rigorous work on each estimate e.g., with weights, that are not easily integrated in the MLM framework. Our comparison addresses the potential consequences of the different estimation techniques through comparison of models in our example data and follow-up simulations. For the levels of analysis at which interpretations are applied in the different analytical frameworks, we discuss feasibility relative to research aims and to policy making.