Online survey platforms are now an everyday tool for academic studies, but scientists are becoming more alert to the influence of artificial intelligence on the answers they collect. Platforms such as Prolific and Amazon Mechanical Turk are popular because they pay people small sums to answer questions for studies. The number of participants makes them appealing to researchers who need quick access to diverse respondents.
Anne-Marie Nussberger at the Max Planck Institute for Human Development in Berlin told colleagues that she had been shocked to see signs of AI use in her own studies. Her team began checking how often survey participants were relying on tools such as ChatGPT. The concern is that automated replies could pollute the quality of data that social scientists and psychologists depend on.
The problem goes further than spotting an unusual turn of phrase. Nussberger and others fear that the growing use of AI could distort the results of behavioural research. If participants are copying in machine-generated answers, then the data stops representing genuine human opinions. This undermines the very foundation of the surveys that academics use to study public behaviour and social attitudes.
How Common Is AI Use In Online Research?
A study led by Janet Xu, assistant professor of organisational behaviour at Stanford Graduate School of Business, examined how often participants admit to using large language models. Along with Simone Zhang of New York University and AJ Alvero of Cornell University, Xu surveyed around 800 Prolific users. They found that nearly 1/3 said they had used AI tools to answer at least some survey questions.
The results were mixed. 2/3 of those surveyed said they had never turned to AI when writing answers. About one quarter admitted they sometimes used it, while fewer than 10% reported using it very frequently. The most common reason was that people found it easier to express their thoughts with help from a chatbot.
Xu explained that answers generated with AI looked noticeably different from human ones. They were longer, cleaner, and lacked the sarcasm or sharpness often found in authentic responses. The smooth tone made the replies sound artificial. “When you do a survey and people write back, there’s usually some amount of snark,” she explained.
More from Artificial Intelligence
- How Are Startups Using Synthetic Data to Train Early AI Models?
- How Does Data Set Training Influence the Performance of AI?
- How to Detect and Prevent AI Hallucinations In Your Applications
- Will Sam Altman Be Remembered as a Villain or a Hero of Humanity?
- What Is An AI-Native Startup?
- Can AI Enhance The Effectiveness Of Online Learning Platforms?
- Coding with Copilots: Are Developers Becoming Architects Instead of Builders?
- OpenAI Unveils GPT-5: Not Quite AGI, but Leaps and Bounds Forward
What Are The Risks For Data Quality?
The Stanford-led study pointed out that those who avoid AI often do so because they feel it would be dishonest. Some said it would be cheating the researchers. This concern about authenticity shows how trust and validity in research are at stake.
AI-generated responses also tend to use more neutral and abstract wording. In studies before ChatGPT’s release in 2022, people expressed more emotional and concrete language, even when it came to sensitive topics such as race or politics. Machines, on the other hand, flatten those differences. This shift could lead to the dilution of diversity in survey results.
Xu mentioned that if too many people hand over their opinions to AI, the overall findings could present a false sense of harmony. For example, workplace surveys on discrimination could end up looking more positive than reality, making it harder to spot problems. Researchers may then draw misleading conclusions about social attitudes or workplace culture.
What Can Researchers Do About It?
The paper made the argument that asking participants directly to avoid using AI can help. Another method is to use software features that block copy and paste, or even ask respondents to record their voices instead of writing. These measures make it harder for people to paste in machine-written replies.
There are also lessons for academics themselves. Many people who said they used AI explained that they did so because they found instructions confusing or too demanding. When surveys are long or unclear, participants are more likely to turn to chatbots. Designing shorter and clearer questionnaires could reduce the temptation to rely on AI.
Xu concluded that AI use has already led researchers and journal editors to pay more attention to data quality. She did not think the problem was yet large enough to force corrections or retractions of past studies, but said that it should serve as a warning. Scientists must now think carefully about how they gather and check their data if they want to keep their findings trustworthy.
