XI Conference – European Congress of Methodology

Session

Symposium : "Bridging Psychometrics and Artificial Intelligence"

25 Jul 2025, 08:30

327. Optimizing LLM Embeddings for Automatic Item Development and Validation

25/07/2025, 08:30

Oral Presentation

Large Language Models (LLMs) have shown promise in text clustering and dimensionality analysis through embeddings, yet their potential for optimization remains largely unexplored. We conducted a comprehensive simulation study to enhance the accuracy of LLM embeddings in trait mapping using Dynamic Exploratory Graph Analysis (Dynamic EGA). The simulation generated 200 items across 4 traits of...

328. Does the result of an in-silica structural validity matches the structural validity computed in human-gathered data?

25/07/2025, 08:50

Oral Presentation

The rapid advancement of large language models (LLMs) has enabled automated psychological scale development, yet questions remain about the correspondence between in-silica and human-gathered validation. This study examines whether structural validity metrics computed during automated item development match empirical validation results. Using AI-GENIE (Automatic Item Generation and Validation...

329. How Block Types and Social Desirability Shape Forced-Choice Questionnaire Automatic Assembly

25/07/2025, 09:10

Oral Presentation

The construction of forced-choice questionnaires often relies on item banks with single-stimulus or Likert-type items. In its simplest form, items must be paired to create a desired number of blocks. A key challenge in this process is pairing items while accounting for factors such as item polarity and social desirability, which can impact the quality of the measures. Recent combinatorial...

330. Predicting Item Response Theory Parameters from the Semantic Space of Computational Language Models

25/07/2025, 09:30

Oral Presentation

Parallel to the development of new technologies, computational language models have emerged as automated tools for analyzing semantic relationships between linguistic units. Due to their success in performing human-like tasks, such as vocabulary tests and sentiment analysis, interest in the practical applications of these models has grown exponentially, resulting in the development of larger...

164. Bridging Psychometrics and Artificial Intelligence

Measurement

Symposium