XI Conference – European Congress of Methodology

Evaluating Methodological Quality in Football Studies: An Application of the MQSOM

25 Jul 2025, 08:45

15m

Oral Presentation Applications/Substantive areas Session 22 : "Methodological Quality and Modeling in Behavioral"

Abstract

Introduction: observational methodology has been widely used in football research during the last decades, due to its low level of intervention and independence from standardized measurement tools. Observational methodology allows the systematic recording, quantification and analysis of player behaviors on the field. As scientific production based on observational methodology increases, there is a growing need to evaluate its methodological quality. A Methodological Quality Scale for Studies Based on Observational Methodology (MQSOM), a tool to measure the methodological quality of these studies, has recently been validated with adequate psychometric properties (RMSEA = 0.000, NNFI = 1, GFI = .98, AGFI = .97). The MQSOM comprises a second-order factor of Methodological quality (MQ; ω = .87; D = .55) containing two first-order factors: Quality of design (D1; 6 items; ω = .90; D = .46; ICC = .933 - .967) and Quality of measurement and analysis (D2; 5 items; ω = .68; D = .67; ICC = .797 - .988). Objective: this study presents the results of a systematic mixed-method review that applies the MQSOM to primary studies of observational methodology on football. Methods: descriptive statistics by country of affiliation, journal, object of study and event observed are presented. In addition, an analysis of proportions was performed to identify differences in terms of procedural characteristics. Finally, a two-stage cluster analysis was performed. Results: the analysis of proportions showed significant differences in terms of type of observation, coding manual specification, type of data, observation instrument, recording, control and analysis software, type of parameter, data quality control and analysis used. In addition, two-step cluster analysis produced five methodological quality profiles ranked in decreasing order by MQSOM score. Profile 1 exhibited high levels of methodological quality (GD = 0.81; D1 = 0.78; D2 = 0.89), Profile 2 exhibited moderate-to-high levels of methodological quality (GD = 0.76; D1 = 0.70; D2 = 0.83), Profile 3 exhibited low-to-high levels of methodological quality (GD = 0.56; D1 = 0.37; D2 = 0.79), Profile 4 exhibited low-to-moderate levels of methodological quality (GD = 0.46; D1 = 0.26; D2 = 0.70), and Profile 5 exhibited low levels of methodological quality (GD = 0.24; D1 = 0.06; D2 = 0.45). Conclusions: this application of MQSOM allows to obtain an updated snapshot of the observational methodology applied in football. This work highlights future improvements in terms of methodological quality and presents MQSOM as a valuable tool to assess the quality of sports intervention programs based on observational methodology.