Survey fraud is a serious problem that can have a significant impact on the quality of market research data – the very life blood of the insights industry. And it is a growing problem. Some industry commentators now believe that up to a third of the data being collected is being discarding because of quality concerns and panel fraud. This is a big deal.
There are several different ways to commit survey fraud, including:
- Impersonating respondents: This is when someone pretends to be someone else in order to take a survey. This can be done by using a fake name, address, or phone number and using a VPN that suggests they are coming from a location different to where they really are.
- Responding multiple times: This is when someone takes a survey more than once. This can be done by using different email addresses and/or phone numbers and/or manipulating the survey link.
- Giving false answers: This is when someone intentionally gives false answers to survey questions, often when they want to be able to qualify for a survey and reward. For example, when they declare that they have just bought a new car, when they don’t even drive.
- ‘Bots’ filling out online surveys.
This kind of survey fraud can be difficult to detect, but there are ways to reduce the risk of it happening to you. Here are some practical methods to consider:
Five steps during survey set-up . . .
- Always try to keep surveys short, engaging and to the point. Even genuine respondents will get bored if the survey is too long or too dull and may start to give less reliable answers as they just try to finish the survey. This may be unintentional deception, but bad data is bad data – and this will affect project quality just the same.
- If you are buying in sample, use reputable panel companies that require their members to validate their email addresses and use two-factor authentication. Ask your panel provider about how their members are verified.
- Use Google’s CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart). This is a system intended to distinguish humans from bot activity. It works by presenting tasks (such as recognising and inputting displayed characters) that are straightforward for humans but challenging for bots, enhancing the reliability of the responses. See our KB article on adding this at the start of your online surveys.
- Limit survey access. One way to ensure your respondents are genuine is to restrict access to surveys using unique URLs that contain access codes and/or participant IDs. This makes it much harder for bots or duplicate respondents to gain unauthorized access. Always encrypt your links, to prevent respondents from editing the links and using them multiple times. See this KB article for more details of prettifying and encrypting survey links.
- Use smart questionnaire design. Strategic questions, including “red herring” questions, can catch out bots.
- For instance, it could be a simple instruction like, “Please select the third response for this question”. A human that is paying attention will understand this and obey it, but bots (and humans) that are simply pattern-matching or picking answers at random would mostly fail, thereby revealing their non-human nature.
- Including two similar questions throughout your survey can help reveal any inconsistencies in answers. For example, adding questions such as “in what year were you born?” and “please tell me your age?” at the beginning and end of the survey can help you distinguish authentic and fraudulent responses.
Five steps once the survey is in field . . .
- Conduct both automated and manual data checks. It’s crucial to examine data for unusual response patterns, duplicate entries, or inconsistent data that may suggest fraud. Straight-lining is an obvious example of this.
- Open questions are often a good place to start when searching for fraudulent activity. Check for repeated answers or general nonsensical answers or odd formatting including the use of bullet points. A neat trick is to check the number of keystrokes on the page and the number of characters in the open question box. If the number of characters in the open question box is higher than the number of keystrokes, then you know someone, or something, has done a cut and paste with the response rather than typed something out. We posted a recent blog article on this topic and provided the code in this KB article.
- Use time-based response limits: automated processes and cheats will fill surveys faster than genuine respondents. Placing pragmatic time limits on survey completion can expose bots and cheats, who can be rejected, leaving only the respondents whose time to respond fits what we’d expect. In Askia you can put time checks anywhere in the survey, so you can test timings on a sectional basis or even question by question. You can also prevent page navigation for a specified duration, to try to prevent people speeding through each question. The JavaScript to perform this trick is described in this KB article.
- Check IP Addresses: All devices connected to the internet use an IP address. Through tracking and monitoring this data, you can look for multiple responses from the same IP address, which could flag potential fraud. You can also check that the geolocation of the respondent matches where the respondent claims they are living. Many fraudsters will be using VPNs that disguise their IP Address, but not all of them.
- Check Device Fingerprints: This process refers to identifying a respondent’s device based on its unique settings, configurations or activity, helping to track and flag suspicious or recurring device usage in your surveys. Askia gathers this metadata from a respondent, including the device type, screen size, browser and OS version. There are also some very good third-party solutions out there that provide additional digital fingerprinting safeguards and stop a suspected fraudulent respondent from even starting the survey.
In conclusion, although no single method can eliminate the risk of fraud, employing a combination of these techniques can significantly mitigate the risks and maintain the quality of your survey data. As with any anti-fraud measures, the techniques will need to keep evolving to stay one step ahead of the fraudsters, and Askia remains committed to doing all we can as a survey software provider.
