XI Conference – European Congress of Methodology

A Mixed-Method Network Analysis Approach for Enhancing the Development of Assessment Instruments and Obtaining Validity Evidence: A Questionnaire to Measure Police Officers Attitudes towards Intervention in Gender-Based Violence

23 Jul 2025, 15:00

15m

Oral Presentation Measurement Session 16 : "Mixed methods and Behavior assessment"

Abstract

In this study, we propose a mixed-method network analysis approach to enhance the development of measurement instruments and gather robust validity evidence. By integrating semantic and psychometric networks, this method uncovers the relational dynamics between behaviors (semantic definitions) and items (final instrument version) for obtaining content validity evidence and internal structure validity, while it also identifies key elements critical for advancing the development process of the instrument. Specifically, the mixed method approach is illustrated by two studies carried out in the development of an instrument to assess police attitudes toward intervention in gender-based violence. The first study compiled six focus groups involving 36 specialized and non-specialized gender-based violence police officers and applied a mixed-method semantic network analysis that combined qualitative data (i.e., themes extracted from police discussions on their perspectives about intervention in gender-based violence) and quantitative data (i.e., co-occurrence of these themes within police discussions). Multidimensional scaling (MDS) was used to visualize the network along with node and network centrality indices. The goal was to obtain domain definition validity evidence and identify key behaviors shaping police attitudes toward intervention in gender-based violence. The second study drew on 272 police responses to the 32-item first version of the APIVG-S, administered online via the Unipark platform. It aimed to detect key items and obtain validity evidence based on the internal structure. For psychometric network analysis, we estimated partial correlation networks using Spearman’s rank-correlation and the glasso regularization method, with a tuning parameter set to 0.5. The accuracy and stability of the edge estimates were assessed through nonparametric bootstrapping. On the one hand, the semantic network analysis revealed that attitudinal behaviors were grouped along two theoretical dimensions (reactive and proactive), providing validity evidence for domain definition. Similarly, the behavior “conventional policing in gender-based violence” emerged as the most central node in the semantic definition, highlighting which behaviors warrant a greater number of items in the table of specifications. On the other hand, the psychometric network analysis showed a medium level of connectivity (sparsity index of 0.544) and provided validity evidence based on the internal structure, as the most interconnected items corresponded to the same theoretical dimensions. The network was found to be relatively accurate and stable. The benefits of the mixed-method network analysis approach are discussed in terms of the joint interpretation of qualitative and quantitative data, incorporating the experiences of the target population (and not only experts) into the instrument development process, and identifying different indicators of the importance of behaviors in the semantic definition and the instrument’s items.