Speakers
Abstract
Numerical reasoning (NR) is a key competence associated with higher academic performance, especially in specific fields of study. Higher education has proven to be highly relevant in academic tasks involving critical thinking, analytical thinking, and problem-solving. Assessing NR in the university admission process would provide crucial insights into the candidate’s profile and enable universities to implement targeted actions to foster and enhance these skills. The measurement of numerical reasoning traditionally involves the administration of fixed-length questionnaires, with the corresponding risks they may entail in high-stakes settings (e.g., cheating). In this scenario, computerized adaptive testing (CAT) offers an alternative to overcome some drawbacks of conventional testing methods while providing more efficient measurements.
This study aimed to construct and analyze the psychometric properties of a first set of items to adaptively measure NR that will form part of a larger bank currently under construction. Based on a previous pilot study conducted with university students, different logical reasoning schemes were established to generate items according to five difficulty levels. This initial pool was administered to a huge sample of undergraduate students, and preliminary analyses were conducted to select the items with better psychometric properties for subsequent pool calibration. After that, a post-hoc simulation study was carried out to assess the performance of the CAT. The CAT is expected to be efficient and highly accurate in measuring NR using a small tailored set of pool items. Finally, some suggestions are offered regarding the different phases involving the construction of the item pool.
| Oral presentation | First steps in the construction of a new item pool to adaptively measure numerical reasoning in university students |
|---|---|
| Author | Juan F. Luesia |
| Affiliation | Universidad Loyola Andalucía |
| Keywords | Numerical-reasoning, item-pool, computerized-adaptive-testing(CAT), academic-performance, university-students |
