In May 2026, PNAS published a study with a curious result: in five-minute written conversations, participants chose GPT-4.5 as the human conversational partner more often than the person it was competing against. This happened in 73% of comparisons when the model received instructions to adopt a human persona. The research, first circulated in 2025, also added fifteen-minute tests with other models. Cameron Jones and Benjamin Bergen’s paper offers concrete evidence of progress on the Turing test.
For decades, that name has suggested a boundary: the day a conversation would no longer let us distinguish a machine from a person. Now that some systems cross that boundary under experimental conditions, a less comfortable question arises. How much does this result tell us about their intelligence?
Answering requires returning to the problem Turing was posing. It also requires distinguishing the historical value of his proposal, what a particular test can measure, and the consequences of living with systems that can be convincing.
In 1950, Turing changed the question
Alan Turing opened Computing Machinery and Intelligence by asking whether machines could think. He quickly proposed replacing that discussion with an observable game. His initial version involved a man, a woman and someone who had to distinguish them through written messages; he then asked what would happen if a machine took the man’s place.
This led to the best-known version: a judge converses in writing with a person and a machine, without knowing which is which. Text excludes appearance and voice. Questions can cover many subjects, and the judge can probe wherever they suspect a difference.
The judge knows there is a person and a machine. At the end, they must identify the person. Choosing randomly between them would be correct roughly half the time.
Our diagram of the usual three-party version. Turing's initial formulation started by distinguishing a man from a woman.
The move still has force: if we attribute intellectual abilities to other people through what they do and say, we need to justify what different evidence we would demand of a machine. Rejecting every answer because we know it comes from a computer would settle the discussion in advance.
An open conversation also lets us change the examination as we go. We can request an explanation, introduce a contradiction or check whether our partner follows a joke. That flexibility matters: the questioner can explore the previous answer instead of merely marking a list of expected solutions.
Turing also imagined machines learning through a kind of education. His paper was broader than the test that inherited his name. Reducing it to a contest to fool someone loses that ambition: building and studying capacities we had previously associated with the human mind.
ELIZA showed how much the conversational partner contributes
In 1966, Joseph Weizenbaum introduced ELIZA, a program that transformed sentences using rules. Its famous conversational script imitated some techniques of a psychotherapist: asking questions back, picking up the speaker’s words and encouraging them to continue.
A fragment of the published dialogue is enough to show the mechanism:
Person: “My mother takes care of me.”
ELIZA: “Who else in your family takes care of you?”
A family reference activates a rule that reconstructs the question. The relevance of the reply does not require a representation of the mother or the care she provides.
Two turns from Weizenbaum's original dialogue. This conversation was not generated for this article.
The situation makes the illusion easier. If we expect our partner to ask questions about what we just said, asking a question back can seem like a sign of attention. We supply the story, recognize a connection and fill in what the answer leaves open.
This helps explain the so-called ELIZA effect: attributing understanding or human qualities to a system based on its conversational behavior. It does not mean ELIZA passed a demanding general test. It shows something narrower and useful: part of the impression of intelligence is constructed by the person interpreting the answer.
Today’s language models work differently. They learn regularities from large amounts of data and generate answers conditioned on the conversation; they do not depend on a keyword script like that one. The technical distance is enormous. ELIZA’s question nevertheless remains open in every interaction: what is the system demonstrating, and what are we adding as we read?
The Chinese room objection and the problem of resemblance
In 1980, John Searle proposed the Chinese room. He imagined someone who does not understand Chinese, enclosed with instructions for manipulating symbols. They receive questions, follow the rules and return answers that seem written by a competent speaker. From outside, the conversation works; inside, the person still does not understand the language.
Searle wanted to show that running a program is not by itself sufficient for understanding. The systems reply argues that understanding might belong to the whole—person, rules and memory—even if one part does not understand. Searle disputed that reply; the disagreement depends on where we locate understanding and what we consider sufficient to produce it.
We should retain the thought experiment as a philosophical argument. It is not a measurement of a current model or experimental proof that a machine can never understand. It does require us to explain the leap from observing appropriate answers to attributing a particular mental life to them.
Another criticism bears directly on research. In Turing Test Considered Harmful (1995), Patrick Hayes and Kenneth Ford questioned whether imitating a person should be AI’s organizing objective. Their flight analogy distinguished reproducing an animal from studying the principles that enable a useful machine.
Consider a simple situation. We want a program to find calculation errors. Accurate answers make it useful. If it also learns to make plausible mistakes so we mistake it for a person, it may improve at an imitation game while becoming worse at the job we wanted it for.
Turing’s own imagined dialogue contained an incorrect sum:
Question: add 34,957 and 70,764.
Imagined answer: after about 30 seconds, 105,621.
The correct result is 105,721. In this game, slowness and error can fit the identity the machine is trying to portray. In an arithmetic test, they would count as a failure.
Adapted from the example in section 2 of Turing (1950). A hypothetical dialogue by the author, not a result from a machine of that period.
This does not invalidate conversation as a test. A rigorous interview can reveal abilities that a closed questionnaire misses. It calls into question the final criterion: choosing whoever seems human mixes competence, style, expectations and knowledge of social context.
What current models have actually passed
Jones and Bergen’s study observes this mixture with explicit controls. The first tests involved university students and people recruited on Prolific, a research platform. Each judge talked simultaneously with a human and a system through two separate chats.
The researchers compared two ways of instructing the models. One gave them the objective of convincing the judge they were human, along with contextual information. The other added a persona: someone young, introverted, familiar with internet culture and using informal language. “Without persona” therefore does not mean using the assistant as it appears when you open an application.
With two options, random choice would give roughly 50%. The 73% measures judges' choices: it does not mean “73% of human intelligence.”
Our chart based on “Win Rates” and table 1 of the published paper. Rounded percentages, two populations and conversations limited to five minutes. Aggregated estimates are shown without confidence intervals. The abstract reverses the order of the two no-persona figures; we follow the results section here.
The difference between the two instructions matters as much as the highest figure. The test responds to how the system presents itself to the judge. Adapting the conversation to that situation is part of the demonstrated ability; it also limits what we can infer about other tasks.
The 2026 publication went beyond the five minutes of the 2025 preprint. It added a replication with conversations lasting a full fifteen minutes. GPT-5, which replaced GPT-4.5 in that phase, was selected as human in 59% of comparisons; Llama 3.1 in 56%. Both received persona instructions. The authors interpret them as passing this version of the test.
That extension prevents dismissing the result by saying it all disappears once the conversation lasts a little longer. At the same time, replacing GPT-4.5 with GPT-5 prevents treating the two figures as a clean measurement of what the same model loses when the duration triples. The conclusion must retain the model, instructions, participants and duration that support it.
Nor can a percentage near 50% establish universal equivalence. A sample has uncertainty; failing to detect a difference in one experiment does not prove it cannot be found with other judges or conditions. These studies turn a cultural intuition into a comparison we can examine, repeat and debate.
Its place in AI is more precise than its fame
Historically, the test helped frame the discussion of artificial intelligence in terms of observable behavior. As a research tool, it remains valuable for studying conversational imitation and the signals we use to recognize other people. As a general measure of AI quality, it leaves too many questions open.
The difference becomes clear when comparing it with evaluations used to investigate other abilities. SWE-bench poses software project issues and evaluates whether proposed changes resolve them through tests. HELM combines scenarios with measures of accuracy, robustness, calibration, bias, toxicity and efficiency. Put simply, calibration checks whether the system’s confidence corresponds to its correct answers.
| Evaluation | What is observed | What it leaves out |
|---|---|---|
| Imitation game | Whether the judge identifies the human in a conversation. | The correctness of work the system has not been asked to do. |
| SWE-bench | Whether a code change resolves an issue and passes tests. | Whether the system can take on any engineering job. |
| HELM | How the model behaves across scenarios and dimensions. | A definitive answer about consciousness or understanding. |
In our article on how AI benchmarks changed, we traced the move from completing text to tackling professional assignments. The Turing test illuminates another part of the problem: taking the place of a human conversational partner without the judge noticing. This ability can coexist with errors in mathematics, code or verifiable facts.
Judgments of humanness do not automatically transfer from one medium to another. A paper presented at ICLR 2026 on voice conversation collected 2,968 human judgments of dialogues involving nine systems and 28 participants. None of the evaluated systems passed its test. The authors identified difficulties in emotional expressiveness and the vocal cues that accompany words.
The study has its own systems and protocol; it does not refute the written results. It shows why speaking of “the test” as a single examination can confuse. Removing voice, as Turing did, delimits a question. Reintroducing it adds different abilities and clues.
The strongest defense of Turing remains uncomfortable
It would be easy to end the debate by saying a machine merely imitates. But that leaves open what kind of evidence we would accept to revise our view. If a longer conversation, a more precise explanation or a response to a new situation could never count in its favor, we would have made the conclusion a starting condition.
The proposal’s strength lies in forcing us to specify our demands. Someone can understand something and explain it poorly. Another person can sound confident and be wrong. When judging an artificial system, we need to separate those possibilities just as carefully, neither settling its abilities through a convincing phrase nor dismissing them because of its origin.
We can design tests to find out whether it maintains an explanation when the data change, recognizes a contradiction or applies what it learned to a different problem. Those observations provide evidence of abilities. Whether there is also subjective experience—whether there is something it feels like to be that system—requires an additional argument. A vote on which partner seems human does not by itself supply that bridge.
Both ideas are worth keeping: passing a demanding conversational test is an achievement, and that achievement does not settle every question we associate with intelligence. Limits do not make a measure irrelevant; they require us to use it for what it lets us observe.
When recognizing a person no longer depends on how they write
Imagine a forum where someone responds patiently, remembers what we have told them and uses the same cultural references we do. We may come to trust that partner without asking about a theorem or administering a programming test. A conversation that fits our expectations of another person is enough.
If a system can produce that impression, there is a social consequence even while we debate whether it understands. An artificial participant could present itself as a satisfied customer, a neighbor or a fellow hobbyist. A convincing short interaction does not prove it can maintain that identity for months; it does give reasons to study where signals of humanness stop helping us recognize who is in front of us.
There is also the transparent case: someone knows they are talking to an AI and still finds the conversation useful or pleasant. The same linguistic ability can explain something tactfully or sustain a false identity. The difference depends on how the system is presented, what is promised to the user and who is responsible for its operation.
That is where we would locate an important part of the test’s relevance today. Turing’s history helps us think about the evidence we would accept from artificial intelligence. Its experimental versions also show which signals we use to attribute humanity and how easy or difficult they are to reproduce.
When a machine seems human, we have demonstrated something about its behavior and our judgment. To know what it understands, what it can do reliably and what relationship we want with it, we need to keep asking. And to know who we are talking to, we increasingly have reasons to ask for more than a good conversation.
Sources
- Alan Turing: Computing Machinery and Intelligence (1950). Imitation game, arithmetic example and learning machines.
- Joseph Weizenbaum: original ELIZA paper (1966). Rule-based operation and the reproduced dialogue.
- John Searle: Minds, Brains, and Programs (1980). Transcription of a prepublication draft: Chinese room and objections.
- Patrick Hayes and Kenneth Ford: Turing Test Considered Harmful (1995). Criticism of organizing research around human imitation.
- Cameron Jones and Benjamin Bergen: Large language models pass a standard three-party Turing test, PNAS (May 19, 2026), also available in PubMed Central. Results, instructions, controls and fifteen-minute replication. The March 2025 preprint contains the initial five-minute tests.
- Carlos E. Jimenez and colleagues: SWE-bench (2023). Evaluation of code changes to resolve real issues.
- Percy Liang and colleagues: Holistic Evaluation of Language Models (2022; published in TMLR, 2023). Evaluation across scenarios and multiple dimensions.
- Xiang Li and colleagues: Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction, ICLR 2026. Revised September 8, 2026; results limited to the evaluated systems and protocol.
