QualTurk aims to increase the quality of survey data collected from Amazon Mechanical Turk (MTurk) by flagging respondents who fail quality checks. The system is designed to support Mechanical Turk in conjunction with Qualtrics.
How it works
MTurk workers are automatically registered by this system and forwarded to your Qualtrics survey. The quality checks include i. survey completion above a minimum time threshold (optional), ii. correctly answering an arbitrary set of pre-screening questions (optional), iii. correctly answering two specific questions that test if respondents read instructions (required). The system also ensures that workers cannot repeat the same HIT. The system provides workers with a completion code and optional debrief text. The code enables them to complete the HIT.
What you get
QualTurk provides data on which quality check each worker failed, if any (csv file). All workers receive a completion code (except repeat takers who are not eligible), which is also visible on MTurk. The code itself reflects if there was any quality issue and which one. Based on this information, you can decide who to pay on MTurk. Your Qualtrics survey will include responses from lower quality workers, but they can be filtered out by cross-checking with the QualTurk dataset on who passed which quality checks.