Speaker
Abstract
In psychological assessment, a popular test design is administering items in order of progressively increasing difficulty, referred to as a progressive test. Prime examples include subtests of the well-known Wechsler intelligence test batteries. Progressive tests are based on the concept of a Guttman scale, which facilitates intuitive score interpretation, and the application of stopping rules in test administration for higher efficiency and less test burden. However, in practice, stopping rules often lack a well-documented empirical basis and justification due to the absence of clear standards and guidelines, raising general concerns about fairness and validity. To facilitate evidence-based decision-making for setting appropriate stopping rules, we propose a transparent approach that charts the impact of varied alternative stopping rules on Accuracy (e.g., test outcomes at the individual and group level, and norm tables) and Efficiency (e.g., person-specific test length). These A-E charts are based on retro-actively applying stopping rules to normative data administered without a stopping rule or with a default stopping rule. An empirical working example is used to illustrate the proposed toolkit. We show that a universal stopping rule likely does not exist and that the optimal rule varies as a function of the desired efficiency-accuracy trade-off suitable for the intended test use and target population. The proposed approach provides a pragmatic solution for practitioners, researchers, test developers, and test publishers to rethink the existing stopping rules, systematically evaluate the alternatives, and set appropriate rules.
| Oral presentation | Setting Stopping Rules for Progressive Tests: A Practical and Transparent Toolkit |
|---|---|
| Author | Jianan Chen, Ellen Irén Brinchmann and Johan Braeken |
| Affiliation | University of Oslo |
| Keywords | stopping/discontinue rule, test design |
