Suspected bot responses changed an intimate-partner violence survey’s findings

A suspect survey response is easy to frame as a cleaning problem: identify it, remove it, and proceed. That framing is too narrow when the responses in question can change both the reported outcome and the behavior of the instrument used to measure it. In an online study about intimate-partner violence, the stakes include not only an estimate of assault but confidence that the survey scale is measuring what researchers think it measures.

In a study published online on October 1, 2026, the Journal of the Society for Social Work and Research reported an analysis of 218 completed surveys about exposure to intimate-partner violence. The authors compared responses classified as bots with those classified as human. Their finding deserves attention from anyone using online surveys to inform research or program decisions: uncertain respondent authenticity may affect the conclusions a dataset appears to support.

The classification is part of the finding

The authors used manual review and an algorithmic method to classify responses, but the approaches did not produce identical results. The Journal of the Society for Social Work and Research reports that manual review classified 32.57% of responses as bots, while the algorithmic method classified 39.45%; consistency between the approaches was only fair to moderate. That disagreement matters. A label assigned during cleaning is not the same thing as independently verified knowledge of who supplied an answer.

Even with that qualification, the difference between the classified groups was consequential. Responses classified as bots scored more than 25 times higher than human responses on the Revised Conflict Tactics Scale physical-assault subscale, according to the study published in the Journal of the Society for Social Work and Research. The authors also found that bot-classified responses distorted the scale’s underlying statistical structure, whereas human responses showed the expected properties. This is a stronger warning than a shift in an average: it raises the possibility that a familiar measure can appear to behave differently because of who, or what, entered the data.

The responsible reading is not that a particular detection method has conclusively identified every suspect response. It is that the substantive result and the measurement check should both be examined under plausible classification decisions. If findings change materially depending on which review method is used, that uncertainty belongs in the interpretation, not just in an internal cleaning log.

Detection cannot carry the whole burden

A separate study points to the limits of relying on generated-text detection alone. In tests across three real-world surveys, researchers publishing on arXiv found that persona-grounded AI agents made existing machine-generated-text detectors perform near chance. That does not establish that the responses in the intimate-partner violence study were produced by such agents. It does mean that a detector’s apparent ability to recognize generic synthetic prose should not be treated as proof that it will recognize plausible survey answers.

The University of Michigan’s Survey Research Center, reporting on a survey-research forum, described NORC’s distinction between checking whether the right person supplied a response and whether that person supplied the right data. The distinction is useful here. Verifying a respondent’s identity would not establish that an answer is accurate, while inspecting answer text alone may not establish who supplied it. Research teams need to treat provenance and response quality as related but separate questions.

Make uncertainty visible in the analysis

For a careful team, the practical response begins before a final dataset is declared clean. Document recruitment and eligibility checks; define review criteria before examining outcomes where feasible; preserve the original records and classification decisions; and compare key estimates and scale diagnostics under alternative, defensible exclusions. When reviewers disagree, report that disagreement rather than presenting a single label as ground truth. Sensitive surveys also require proportionate checks that do not demand unnecessary disclosure from genuine participants.

The central lesson is methodological rather than technological. Authenticity checks can protect an analysis, but their own error and uncertainty must be measured. For researchers, monitoring-and-evaluation teams, and AI data teams, the question is not merely whether suspicious records were removed. It is whether readers can see how the conclusion changes when uncertainty about those records is taken seriously.