XI Conference – European Congress of Methodology

Standards and Guidelines for Educational and Psychological Assessment: Continuing the Conversation

Not scheduled

1h 30m

EAM2025

Symposium Measurement Symposium : "Standards and Guidelines for Educational and Psychological Assessment: Continuing the Conversation"

Speaker

Stephen Sireci (University of Massachusetts Amherst)

Abstract

Standards play a crucial role in guiding practices in Educational and Psychological Assessment. Various professional associations continuously update guidelines to support practitioners in assessment-related processes, leading to the emergence of different approaches. However, how do these associations gather information to propose new standards? To what extent do they consider users’ experiences? Do their ultimate goals differ? Moreover, how can professionals navigate and integrate the diverse guidelines available? This discussion will explore the complementarity between different approaches and examine ways to support users in effectively applying multiple standards.

Abstract

The Standards for Educational and Psychological Testing have been published by the American Psychological Association, the American Educational Research Association, and the National Council on Education since the 1950s. They are currently under revision, and the forthcoming version, is expected to be published in 2026. In this presentation, a member of the Joint Committee revising the Standards will discuss the difficult questions and issues that are being considered for the revision. These issues include whether the five sources of validity evidence are sufficient and appropriate for guiding practitioners, the distinction and overlap between validity and fairness, the desire to make the Standards more authoritative, and the desire to make the Standards more understandable to lay audiences. The presenter will take questions from the audience to understand what other issues members of EAM think are important to consider in revising the Standards.

Abstract

In this symposium/panel session, four leaders in test development and validation will provide presentations and answer questions regarding cutting-edge issues related to standards and guidelines for educational and psychological testing. The first three presenters will deliver 15-minute talks on specific standards and guidelines coming from the USA, OECD, and EFPA. Next, a renown discussant will present commentary on the three presentations, pose questions to the audience, and facilitate a discussion among the panel members and the audience. This will be an important session for participants to engage with leading psychometricians in the field and contribute to the improvement of standards and guidelines in educational and psychological testing.

Abstract

The value of the Programme for International Student Assessment (PISA) in informing evidence-based policymaking relies on the degree of precision with which population-level statistics are estimated and reported. But also, the degree to which those aggregate statistics can be meaningfully compared (e.g., country-level mean scores) and the interpretations made based on those comparisons are valid for the intended purposes. Although validity, comparability, and reliability are important components of fairness, they do not address all issues of fairness in assessment. Fairness provides an important additional lens for ensuring the validity of research, policies, and all other aspects of a testing program to promote positive, intended outcomes and minimize negative ones. PISA Technical Standards serve as a set of criteria for post hoc data adjudication (decisions on whether the data for a specific country are of sufficient quality for inclusion in the international reports), but do not address aspects of assessment quality like fairness, validity, reliability, and comparability. This presentation will describe the principles behind the new PISA Quality Standards to deliver relevant, rigorous, and transparent information to policymakers through assessment instruments that provide fair, valid, and reliable data comparable across cultural settings, time, and groups. The presentation will focus on major threats and suggested guidelines for ensuring fairness, validity, comparability, and reliability. The presentation will stress the similarities and differences with other professional standards.

Abstract

Psychological tests are essential tools that help psychologists make decisions about people. The Board of Assessment (BoA) of the European Federation of Psychologists' Associations (EFPA) has various projects aimed at improving tests and testing practices across Europe and beyond. In this presentation, we share two BoA projects. The first is the BoA’s flagship project, which focuses on tests: the Test Review Model. This model provides a systematic framework for reviewing and assessing psychological tests based on several criteria (materials, reliability, validity, norms, etc.). It has recently been updated to incorporate aspects of digital assessments, inclusivity, and diversity, among others. The second project focuses on test user standards, defining the competencies and skills required to ensure proper test use. In this presentation, we will highlight the main features of these projects, emphasizing their impact and the challenges of implementation.