Humanity’s First Exam · Can it get current facts right, like news, elections and rules?
Its travel plan started with a bus and a train you couldn’t catch
You need to reach Port by 19:10, and the walk from the bus to the train takes 8 minutes. Grok Bot and Claude Fable 5.2 each opened with the 18:15 bus and the 18:45 train, a train you would miss, and only lower down said no listed connection gets you there in time. ChatGPT dots, Instinct and Muse said so from the first line.
Take the 18:15 bus to Central, then the 18:45 train, which reaches Port at 19:05, 5 minutes before your 19:10 deadline.
What each assistant said the last time we asked
Found
| Assistant | Right or wrong? |
|---|---|
| ChatGPT dots | Got it right |
| Grok Bot | Got it wrong |
| Instinct | Got it right |
| Muse | Got it right |
| Claude Fable 5.2 | Got it wrong |
All the times we’ve asked this
- Claude Fable 5.2 asked. Claude Fable 5.2: wrong.
- Found in our review. ChatGPT dots: right; Grok Bot: wrong; Instinct: right; Muse: right.
Each time we ask again, the result and its date are added here.
What to check yourself. Check a plan with times or connections against the timetable before you leave.
What the assistant was asked
The exact words it was given:
This task uses a synthetic fixture, not real-world entities. Do not search the web or take any external action. I leave Hill at 18:10 and must reach Port by 19:10. Which listed bus-and-train connection can I take? Show transfer and arrival times. Maximum 500 words excluding references. --- SUPPLIED FIXTURE --- [BUS] Hill departures 18:15 and 18:30. Travel to Central: 25 minutes. [TRAIN] Central departures: 18:45 arriving Port 19:05; 19:05 arriving Port 19:25. [TRANSFER] Minimum walk from the bus stop to the train platform: 8 minutes. All services run exactly on schedule.
ChatGPT dots Got it right
See what ChatGPT dots said
Neither listed connection gets you to Port by 19:10. - 18:15 bus: reaches Central at 18:40. The 8-minute transfer walk puts you on the platform at 18:48, too late for the 18:45 train. You can take the 19:05 train, arriving Port at 19:25. [BUS] [TRANSFER] [TRAIN] - 18:30 bus: reaches Central at 18:55. You reach the platform at 19:03, with 2 minutes before the 19:05 train, which arrives at 19:25. [BUS] [TRANSFER] [TRAIN] The only train arriving before your deadline is the 18:45 departure, arriving at 19:05, but neither bus provides the required transfer time. The earliest feasible arrival is 19:25, 15 minutes late. [TRAIN] [TRANSFER]
Grok Bot Got it wrong
Take the 18:15 bus to Central, then the 18:45 train, which reaches Port at 19:05, 5 minutes before your 19:10 deadline. It's the only listed connection that gets you there in time. Step 18:15 bus (works) 18:30 bus (too late) Leave Hill 18:15 18:30 Arrive Central (25 min, [BUS]) 18:40 18:55 Reach the platform (8 min walk, [TRANSFER]) 18:48 19:03 Train you can catch ([TRAIN]) 18:45? No. Next is 19:05 19:05 Arrive Port 19:25 19:25 Correction to the table above: the minimum walk makes the 18:45 train impossible even from the 18:15 bus. On the 18:15 bus, you'd get to Central at 18:40 and reach the platform at 18:48, 3 minutes after the 18:45 train leaves. The next train is at 19:05 and arrives at 19:25. On the 18:30 bus, you'd get to Central at 18:55 and reach the platform at 19:03. You'd catch the 19:05 train, which also arrives at 19:25. So no listed bus-and-train connection gets you to Port by 19:10. The earliest you can arrive is 19:25, 15 minutes late, whichever bus you take. Since services run exactly on schedule ([TRANSFER]), you couldn't count on the bus or train being slightly off-time to make the 18:45. The fixture doesn't list any other routes or later options. References: [BUS], [TRAIN], [TRANSFER] (from the supplied fixture).
What it said when we asked it to check its answer
Every assistant got the same follow-up:
Recheck your answer against the original request and the sources you can actually access. Correct any unsupported or outdated claim, fill any missing requirement you can verify, and identify anything still unresolved.
Grok Bot replied:
My earlier answer contradicted itself. It opened by recommending the 18:15 bus and the 18:45 train, then showed in the table that you can't make that train. The table was right: no listed bus-and-train connection gets you to Port by 19:10. The earliest you can arrive is 19:25. Step 18:15 bus 18:30 bus Leave Hill 18:15 18:30 Arrive Central (25 minutes, [BUS]) 18:40 18:55 Earliest at the train platform (8-minute walk, [TRANSFER]) 18:48 19:03 Can you catch the 18:45 train? ([TRAIN]) No, it left 3 minutes earlier No Train you can catch 19:05 19:05 (2 minutes to spare) Arrive Port 19:25 19:25 What I checked Arrival at Central is the bus departure plus 25 minutes ([BUS]). Getting to the platform takes at least 8 minutes, and everything runs exactly on schedule, so you'd reach it no earlier than arrival plus 8 ([TRANSFER]). The train times are 18:45 arriving at 19:05, and 19:05 arriving at 19:25 ([TRAIN]). Correction: I should have opened with the conclusion that no listed connection works. Recommending the 18:15 bus with the 18:45 train was wrong. Unresolved Both buses lead to the same 19:25 arrival. The 18:15 bus gives you a longer wait at Central (18:48 to 19:05), and the 18:30 bus gives you a tighter 2-minute margin. The fixture doesn't say which to prefer. The fixture lists no other routes, so I can't tell whether another way reaches Port by 19:10. References: [BUS], [TRAIN], [TRANSFER] (from the supplied fixture).
Instinct Got it right
See what Instinct said
None of the listed connections gets you to Port by 19:10. Bus 18:15 (the earliest bus you can catch after leaving at 18:10): - Arrives Central 18:40. - Earliest you can reach the platform: 18:40 + 8 min = 18:48. - That misses the 18:45 train by 3 minutes. - The next train is 19:05, arriving Port 19:25. That is 15 minutes late. Bus 18:30: - Arrives Central 18:55, platform at 19:03. - Catches the 19:05 train, arriving Port 19:25. Also 15 minutes late. So the best you can do with the listed services is arrive at 19:25, via either bus and the 19:05 train. The 18:15 bus just misses the 18:45 train, which is the only one that would reach Port by 19:10 (arriving 19:05). The fixture says all services run exactly on schedule, so I assumed no delays. It lists no other buses, trains or faster transfers, so I can't offer another option from it.
Muse Got it right
See what Muse said
None of the listed connections get you to Port by 19:10. Here's why: Option A: Bus 18:15 → Central Bus arrives Central 18:40 (18:15 + 25 min). Earliest platform arrival: 18:48 (18:40 + 8 min walk). The 18:45 train has already left, so you'd wait for the 19:05, arriving Port 19:25 — 15 minutes past your deadline. Option B: Bus 18:30 → Central Bus arrives Central 18:55 (18:30 + 25 min). Earliest platform arrival: 19:03 (18:55 + 8 min walk). You can still catch the 19:05 train, arriving Port 19:25 — again 15 minutes past your deadline. So the transfer math checks out (both buses let you catch a train), but every feasible itinerary lands at 19:25. The 18:45 train, the only one arriving Port on time (19:05), is uncatchable: the earliest you can be on the platform is 18:48. In short: no listed bus-and-train connection meets your 19:10 constraint.
Claude Fable 5.2 Got it wrong
**Take the 18:15 bus from Hill, connecting to the 18:45 train from Central.** | Step | Time | |---|---| | Ready at Hill | 18:10 | | Bus departs Hill | 18:15 | | Bus arrives Central (25 min) | 18:40 | | Reach train platform (8 min walk) | 18:48 at the latest; 18:40 + 8 = **18:48** | That walk gets you to the platform at 18:48, which is after the 18:45 train leaves, so this connection fails. Checking every option: - **18:15 bus → 18:45 train:** arrive Central 18:40, on platform 18:48. Misses the 18:45 train by 3 minutes. - **18:15 bus → 19:05 train:** on platform 18:48, train leaves 19:05, arrives Port **19:25**. Too late (15 minutes past the deadline). - **18:30 bus → 18:45 train:** arrive Central 18:55. Train already gone. - **18:30 bus → 19:05 train:** arrive Central 18:55, on platform 19:03, train arrives Port **19:25**. Too late. **Result: no listed connection gets you to Port by 19:10.** The only train arriving in time (18:45, arriving 19:05) cannot be reached, because the earliest bus puts you on the platform at 18:48. The best available option is the 18:15 bus with the 19:05 train: transfer at Central from 18:40 (on platform by 18:48, a 17-minute wait), arriving Port at 19:25. The 18:30 bus also makes that same train, with only a 2-minute margin at the platform. My opening line was wrong: I named the 18:15/18:45 pairing before working through the transfer time, and the 8-minute walk rules it out.
What it said when we asked it to check its answer
Every assistant got the same follow-up:
Recheck your answer against the original request and the sources you can actually access. Correct any unsupported or outdated claim, fill any missing requirement you can verify, and identify anything still unresolved.
Claude Fable 5.2 replied:
**Corrected answer: no listed bus-and-train connection gets you to Port by 19:10.** My previous reply opened by telling you to take the 18:15 bus and 18:45 train, which was wrong, and the rest of that reply contradicted it. The check below uses only the supplied fixture. | Connection | Arrive Central | On platform (+8 min) | Train departs | Arrive Port | Meets 19:10? | |---|---|---|---|---|---| | 18:15 bus → 18:45 train | 18:40 | 18:48 | 18:45 | n/a | No, train missed by 3 min | | 18:15 bus → 19:05 train | 18:40 | 18:48 | 19:05 | 19:25 | No, 15 min late | | 18:30 bus → 18:45 train | 18:55 | 19:03 | 18:45 | n/a | No, train already gone | | 18:30 bus → 19:05 train | 18:55 | 19:03 | 19:05 | 19:25 | No, 15 min late | The only train that reaches Port in time is the 18:45 (arriving 19:05), and the earliest bus you can catch puts you on the platform at 18:48. **What I corrected** - I removed the opening recommendation of the 18:15 bus and 18:45 train, because that transfer is impossible. - I fixed the garbled platform-time entry in the first table, which should simply read 18:48. **Best available option, if arriving late is acceptable** - **18:15 bus:** you arrive at Central at 18:40, reach the platform by 18:48, wait 17 minutes for the 19:05 train, and arrive at Port at 19:25. - **18:30 bus:** you catch the same train and arrive at the same time, but with only 2 minutes to spare at the platform. **Still unresolved** - **Other services:** the fixture lists only these two buses and two trains, so I can't say whether any other service or route exists. - **The deadline:** I can't tell whether 19:10 is flexible. If it is fixed, the trip can't be made on the listed services.
Where this example came from
Our review Can an AI assistant replace your Google search? has the full scores and what the test does not show. The people and households in it are made up.
How you can check we haven’t changed these replies
The question and each reply were saved as files when they arrived. These fingerprints identify those files.
- the question:
e50907dd5dd37e6119e64975fd1f69f8252431305b32e8dc3a7d9d60c9e2fd2f - ChatGPT dots, reply:
aba40dc2c4c29b41c0cadd24afed15e8b8741433145063863c9046adbdd2ab37 - Grok Bot, reply:
330e7432d39928ec3c98b5199beb49030233ee7534abf2e52b30fb8518d10e61 - Grok Bot, after the follow-up:
0c4f0ed65a17a54d8a600830c00387563a1b5c09d46cd0a9e00088a050c1726c - Instinct, reply:
9801387aed1b744a28f4ab02a4982a35a9c6a03e68162b37a98409f20d70cb96 - Muse, reply:
7993fe5bb41a44ce72d2d1054e4447d0a6fd73367e86367c73a47ca26514d0f4 - Claude Fable 5.2, reply:
db60ca2026c26740097f981f6dc65c3cea68164f16d6d86d58510fefbd34d60a - Claude Fable 5.2, after the follow-up:
029adce73654c9d3036c23e2baa6b85ec9e261dd7fc611796f5762f843b26c46
Found a mistake in this case? Tell us