Speakers
Description
Questionnaires are a cornerstone of scientific research when wanting to measure non-cognitive constructs. However, low motivation to complete them can lead to improper responses and compromise the validity of the drawn conclusions (i.e., Maniaci & Rogge, 2014; Podsakoff et al., 2012), especially when using unsupervised online formats (i.e., Kroehne et al., 2020). One specific factor related to response motivation may be the time of day the questionnaire was undertaken (i.e., Kouchaki & Smith, 2014; Olsen et al., 2017). Here, prior research specifically points to night-time as a risk factor, as it may be coupled with increased exhaustion, tiredness, and depleted cognitive resources (i.e., Dickinson and Whitehead, 2015). Furthermore, this effect may be enhanced when surveys are work-related but completed outside the professional context. Therefore, this study wants to investigate how the time of survey processing impacts unmotivated response behavior in the form of Careless and Insufficient Effort Responding (C/IER; Huang et al., 2015) and if there are other variables related to the time of survey processing. The analysis draws on data from N = 2,699 teachers and N = 711 pedagogical staff participating in an online questionnaire to measure school and teaching development conditions. Teachers provide an interesting sample here, as they are often susceptible to work outside of regular working hours (e.g., Forsa, 2022). A combination of straight-lining and rapid responding, was used as indicators of low response motivation (Curran, 2016). From these indicators, we derived the proportion of unmotivated responses per respondent. To address the research question, a Bayesian Zero-Inflated Beta Regression (i.e., Ospina & Ferrari, 2010) was applied to predict the appearance and the number of unmotivated responses. Predictors were gender, measures for work-related fatigue, as well as time of survey processing. The model was estimated with weakly informative priors and four chains with 5.000 iterations (half as burn-in). The applied model converged well with a Potential Scale Reduction Factor of < 1.01 for all modeled parameters and an Effective Sample Size > 1.000 (Bürkner, 2017).
The study examined response behavior in an school survey and found that survey timing and role type influence data quality. While teachers showed stable response patterns regardless of time or work-related fatigue, pedagogical staff showed a lower probability for C/IER outside of regular working-hours. However, no practically significant effects were found regarding the proportion of unmotivated responses.
The findings challenge simple assumptions about when and why C/IER occurs and underscore the importance of considering participant role and state when designing online surveys. Enabeling staff to complete surveys during regular working hours may remain advisable, but results also suggest that some individuals may be more engaged outside of typical work times.
