When academics need participants for large-scale surveys, they often turn to platforms like Prolific or Amazon Mechanical Turk. These platforms promise access to real people, vetted to ensure they are who they claim to be, in exchange for payment. Yet a new study from Cornell University shows that even if the participants are genuine, some may be outsourcing their responses to artificial intelligence (AI).

The study surveyed around 800 Prolific users and found that nearly one-third had used AI tools like ChatGPT to answer survey questions. Most admitted to doing so occasionally, with less than 10% reporting frequent use. The most common reason? They needed help expressing their thoughts.

Others avoided AI out of ethical concerns. Many said they felt it would “cheat” the research or undermine the validity of their answers. As one researcher put it, “So many responses had a moral tone—participants felt that using AI would compromise the integrity of the data.”

Variable usage

Patterns of AI use varied. Men, Black participants, Republicans, college graduates, and newer users of Prolific were slightly more likely to use AI for writing assistance. The authors warn that such differences could bias results, as varying levels of AI involvement might shape how public opinion is reported.

To see how AI responses compare to human ones, the researchers analyzed older surveys conducted before the public release of ChatGPT in late 2022. Human answers were more emotional and often included language that was blunt or dehumanizing—particularly on sensitive topics like race and politics. AI-generated text, by contrast, was neutral and abstract, suggesting a detached approach to thorny issues.

While some published studies may already include AI-generated responses, the researchers don’t believe the practice is widespread enough to warrant retractions. Instead, it highlights the need for greater scrutiny in survey design and data analysis.

AI’s role isn’t inherently bad, they argue—it depends on how it’s used. If someone uses AI to articulate their own opinion more clearly, that can sharpen their response. But if they use it to generate a generic take on a topic they haven’t thought about, it flattens the diversity of responses. Over time, this could dilute the richness of human perspectives, leading to more homogenized data.

Widespread concerns

This concern extends beyond academia. In workplace surveys on diversity, for instance, widespread AI use could create a misleading picture of harmony. “If everyone uses AI to say nice things about groups historically discriminated against, the results might suggest there’s no problem at all,” the researchers warn.

What can be done? The study found that asking participants directly not to use AI can reduce its use. Other strategies include blocking copy-pasting or asking for voice recordings instead of written responses. But the simplest solution may be designing better surveys. Many participants said they turned to AI because instructions were unclear or questions felt overwhelming. Clear, concise questions could reduce the frustration and boredom that tempt users to rely on AI.

“Good survey design has always been important,” the authors conclude. “Now, it’s more important than ever.”