Description
Background. Careless and insufficient effort responding (C/IER) occurs when respondents fail to give sufficient attention to item content, which leads to poor-quality data (Podsakoff et al., 2012). There are several methods to detect this phenomenon, one being Instructed Response Items (IRI), valued for its simplicity, robust metric properties, and ability to identify different C/IER patterns (Kam & Chan, 2018). While detecting C/IER is a crucial first step, deciding how to address this phenomenon once identified is equally important, as this choice can determine the extent of its impact on data quality.
Objectives. This study compares four strategies for managing C/IER and their impact on the psychometric properties of questionnaires, specifically reliability and validity evidence based on the internal structure: (1) using the total sample without adjustments, (2) excluding careless respondents to create a “clean”sample, (3) retaining the total sample while treating C/IER as a control variable, and (4) retaining the total sample while treating C/IER as a moderating variable.
Methods. We use simulated data based on the Big Five Questionnaire (Caprara et al., 1993) and the Maslach Burnout Inventory (Maslach & Jackson, 1981). A total of 180 conditions are manipulated, with variations in variables such as Severity of C/IER (25%, 50%, 75%, 100%), Percentage of C/IER (0%, 8%, 24%), or Sample Size (150, 300, 700). For each condition, 100 replications are run.
Expected results and Conclusions. Based on previous studies with empirical data (Tomás et al., 2023), we anticipate that using C/IER as a moderating variable (4) will be the most effective strategy. In contrast, using the total sample without adjustments (1) will likely be the least effective, given that C/IER is ignored. Regarding the exclusion of careless respondents (2), we anticipate a reduction in statistical power and its subsequent impact on the psychometric properties. As for (3) using C/IER as a control variable, based on previous empirical research examining its impact on questionnaire psychometric properties (Tomás et al., 2023) and substantive research models results (Tomás et al., 2025), we expect this strategy to be a less effective approach for addressing CR. We will provide recommendations for managing C/IER, helping to mitigate its impact on data quality
in applied research.
This study has been developed within the research project PID2022-141339NB-I00, funded by MCIU /AEI
/10.13039/501100011033 / and by FEDER A way to make Europe, EU
