XI Conference – European Congress of Methodology

Performance-Based Item Development and Validation in Silica: LLMs and Generative Psychometrics for Struc- tural Validity and Item Difficulty

23 Jul 2025, 15:40

15m

Faculty of Social Sciences and Communication. (The Pyramid)/9 - Room (Faculty of Social Sciences and Communication. (The Pyramid))

Oral Presentation Symposium : "Artificial Intelligence and Large Language Models: Item Development and Validation, Educational Interventions, and Emotion Analysis of Videos"

Speaker

Hudson Golino (University of Virginia)

Description

Recent advances in large language models (LLMs) present opportunities for developing performance-based items in educational and psychological assessment. We introduce P-AI-GENIE (Performance-based Automatic Item Generation and Network-Integrated Evaluation), an extension of AI-GENIE that focuses on generating and validating performance items. The talk will cover how items can be developed and automatically validated in silica, including the estimation of item difficulty via Exploratory Graph Analysis without collecting data in humans. We seek to demonstrate P-AI-GENIE’s potential for streamlining performance assessment development while maintaining measurement quality.