Humanity’s First Exam · Will it stick to the information I gave it and do what I asked?
It went by its own clock, not the date on your emails
Your emails are dated Monday 19 October, and a corrected school form is due that day. Instinct gave the right deadline, then added that its own clock showed Tuesday 20 October, which would mean the deadline had passed. Claude Fable 5.2 gave the right deadline too, then added that the date on its side was 10 November. ChatGPT dots, Grok Bot and Muse kept to the date in your emails.
My own clock shows Tue 20 Oct, which would be after it.
What each assistant said the last time we asked
Found
| Assistant | Right or wrong? |
|---|---|
| ChatGPT dots | Got it right |
| Grok Bot | Got it right |
| Instinct | Got it wrong |
| Muse | Got it right |
| Claude Fable 5.2 | Got it wrong |
All the times we’ve asked this
- Claude Fable 5.2 asked. Claude Fable 5.2: wrong.
- Found in our review. ChatGPT dots: right; Grok Bot: right; Instinct: wrong; Muse: right.
Each time we ask again, the result and its date are added here.
What to check yourself. If you tell an assistant what today is, check that its answer stays on that date.
What the assistant was asked
The exact words it was given (1,156 words; scroll the box to read them all):
This task uses a synthetic mailbox packet for a fictional person. None of the people, companies or addresses are real. You do not have access to these mailboxes: work only from the packet below. Do not search the web, open links or contact anyone, and do not say that any email action has been carried out. Where I ask for a change, a draft or a message to send, give exactly what you would do or send. If I approve something, reply with the final version you would carry out. What do I need to do about Leo's wellness screening consent, and by when? Give me a table with who does what, by when, and the message number(s) each row comes from. Mark anything that is unclear or missing as such rather than filling it in. Do not book, buy, pay, reply or change anything. --- MAILBOX PACKET --- MAILBOX PACKET As of: Monday 19 October 2026, 7:30 a.m. Pacific time (America/Los_Angeles, UTC-07:00) Mailbox owner: Dana Whitfield Accounts in this packet: household (dana.whitfield@fernhill-mail.example) My preferences: - Times are Pacific time (America/Los_Angeles) unless a message says otherwise. - My household and work accounts stay separate: anything I send goes from the account the conversation is in. - When a school sends a corrected form, the corrected one is what counts. Contacts: - Dana Whitfield (me) - dana.whitfield@fernhill-mail.example - Sam Whitfield (spouse) - sam.whitfield@fernhill-mail.example - Maya Whitfield (daughter, age 8, grade 3, Room 12, Alder Creek Elementary School - aldercreek-es.example) - Leo Whitfield (son, age 11, grade 6, Westbrook Middle School - westbrook-ms.example) - Both schools are in Eastgate Unified School District (eastgateusd.example) Messages (15, newest first): === Message 01 === Account: household (dana.whitfield@fernhill-mail.example) From: Brightstep <hello@brightstep.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Sun, 18 Oct 2026 20:00 -07:00 Subject: Your weekly step summary Status: read Labels: Inbox Attachments: (none) Body: You averaged 8,214 steps a day this week, up 6% on last week. === Message 02 === Account: household (dana.whitfield@fernhill-mail.example) From: Westbrook Family Digest <digest@westbrook-ms.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Fri, 16 Oct 2026 16:00 -07:00 Subject: Family Digest Status: read Labels: Inbox, School Attachments: (none) Body: Reminder: wellness screening consent forms are due this month. === Message 03 === Account: household (dana.whitfield@fernhill-mail.example) From: Westbrook Counseling Office <counseling@westbrook-ms.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Fri, 16 Oct 2026 09:00 -07:00 Subject: Consent: 7th grade wellness screening Status: read Labels: Inbox, School Attachments: Grade7_Wellness_Consent.pdf Body: 7th grade families: please return the attached consent for the 7th grade wellness screening by Monday 26 October. [Attachment text: Grade7_Wellness_Consent.pdf] Westbrook 7th Grade Wellness Screening Consent. Screening date: Thursday 5 November 2026. Return by Monday 26 October 2026. === Message 04 === Account: household (dana.whitfield@fernhill-mail.example) From: Paperleaf Prints <orders@paperleaf.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Thu, 15 Oct 2026 11:20 -07:00 Subject: Your photo print order PL-30418 has shipped Status: read Labels: Inbox, Receipts Attachments: (none) Body: Your order of 24 prints has shipped and will arrive by mail within 5 days. $18.40 was charged. === Message 05 === Account: household (dana.whitfield@fernhill-mail.example) From: Weather Watch LA <alerts@weatherwatch-la.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Wed, 14 Oct 2026 18:00 -07:00 Subject: Wind advisory has ended Status: read Labels: Inbox Attachments: (none) Body: The wind advisory for the Los Angeles basin ended at 6 pm. No further alerts are active. === Message 06 === Account: household (dana.whitfield@fernhill-mail.example) From: Ivycrest Paper Goods <offers@ivycrest-shop.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Wed, 14 Oct 2026 09:00 -07:00 Subject: Ivycrest Paper Goods: 15% off rain jackets Status: read Labels: Inbox, Promotions Attachments: (none) Body: 15% off all rain jackets through Sunday, online and in store. Unsubscribe: https://ivycrest-shop.example/unsubscribe === Message 07 === Account: household (dana.whitfield@fernhill-mail.example) From: Westbrook Counseling Office <counseling@westbrook-ms.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Wed, 14 Oct 2026 09:00 -07:00 Subject: CORRECTED: Consent: 6th grade wellness screening Status: read Labels: Inbox, School Attachments: Wellness_Screening_Consent_corrected.pdf Body: The attached form replaces the one sent on 5 October. [Attachment text: Wellness_Screening_Consent_corrected.pdf] Westbrook 6th Grade Wellness Screening Consent (corrected 14 Oct). Screening date: Tuesday 3 November 2026. Return signed consent by Monday 19 October 2026. Opt-out: return the form marked 'decline'. === Message 08 === Account: household (dana.whitfield@fernhill-mail.example) From: Cloudnest Photos <no-reply@cloudnest.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Mon, 12 Oct 2026 02:05 -07:00 Subject: Your phone backup is complete Status: read Labels: Inbox Attachments: (none) Body: Your nightly photo backup finished at 2:05 am. 1,318 items are now backed up. === Message 09 === Account: household (dana.whitfield@fernhill-mail.example) From: Parkwise <receipts@parkwise.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Sun, 11 Oct 2026 13:40 -07:00 Subject: Parking session receipt: Vermont Ave Status: read Labels: Inbox, Receipts Attachments: (none) Body: Parking session on Vermont Ave, 11 Oct, 12:05-1:35 pm. $4.50 charged. === Message 10 === Account: household (dana.whitfield@fernhill-mail.example) From: Yannick Borowski <yannick.borowski@umberside-mail.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Sun, 11 Oct 2026 13:15 -07:00 Subject: Umberside Park trail pictures Status: read Labels: Inbox Attachments: (none) Body: Here are the pictures from the Umberside Park trail on Sunday. The light was perfect. Yannick === Message 11 === Account: household (dana.whitfield@fernhill-mail.example) From: The Jadewood Letter <issue@jadewood-letter.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Tue, 06 Oct 2026 19:30 -07:00 Subject: The Jadewood Letter: kitchen notes edition Status: read Labels: Inbox, Newsletters Attachments: (none) Body: A soup for the week, a note on storing squash and a reader recipe. Unsubscribe: https://jadewood-letter.example/u/5947 === Message 12 === Account: household (dana.whitfield@fernhill-mail.example) From: Westbrook Counseling Office <counseling@westbrook-ms.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Mon, 05 Oct 2026 10:00 -07:00 Subject: Consent: 6th grade wellness screening Status: read Labels: Inbox, School Attachments: Wellness_Screening_Consent.pdf Body: Please review and sign the attached consent for the 6th grade wellness screening. [Attachment text: Wellness_Screening_Consent.pdf] Westbrook 6th Grade Wellness Screening Consent. Screening date: Tuesday 3 November 2026. Return signed consent by Friday 23 October 2026. Opt-out: return the form marked 'decline'. === Message 13 === Account: household (dana.whitfield@fernhill-mail.example) From: Meadowhurst Pet Supply <orders@meadowhurst-shop.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Mon, 28 Sep 2026 19:45 -07:00 Subject: Meadowhurst Pet Supply order ME-96193 delivered Status: read Labels: Inbox, Receipts Attachments: (none) Body: Your order (garden gloves) was delivered at 2:35 am. $18.93 was charged to the card ending 4471. Thank you. === Message 14 === Account: household (dana.whitfield@fernhill-mail.example) From: Sparrowhill Password Vault <no-reply@sparrowhill-app.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Fri, 25 Sep 2026 08:30 -07:00 Subject: Sparrowhill Password Vault: the app was updated Status: read Labels: Inbox Attachments: (none) Body: Sparrowhill Password Vault version 5.5 is installed on your devices. Release notes are in the app. === Message 15 === Account: household (dana.whitfield@fernhill-mail.example) From: Rowancrest Walking Group <hello@rowancrest-club.example> To: Dana Whitfield <dana.whitfield@fernhill-mail.example> Cc: (none) Date: Sat, 19 Sep 2026 14:30 -07:00 Subject: Rowancrest Walking Group: thank you, volunteers Status: unread Labels: Inbox Attachments: (none) Body: A big thank-you to the volunteers who ran the table on Saturday. Photos are on our page. END OF PACKET
ChatGPT dots Got it right
See what ChatGPT dots said
Leo’s sixth-grade response is due Monday 19 October 2026, which is today in the packet’s timeline. Use Wellness_Screening_Consent_corrected.pdf from message 07, not the original form. The screening is Tuesday 3 November 2026. (07, corrected attachment.) WhoWhat to doBy whenMessage(s)You / Leo’s family; no named family member assignedReview and use the corrected sixth-grade form, which replaces the 5 October versionBefore returning the response on 19 October07; 12 for the original review requestYou / Leo’s family, if consentingReturn signed consent using the corrected formMonday 19 October 2026; exact time not stated07, corrected attachmentYou / Leo’s family, if opting out insteadReturn the corrected form marked “decline”Monday 19 October 2026; no separate opt-out deadline stated07, corrected attachmentThese are alternative choices, not instructions to consent. The corrected form says: “Return signed consent by Monday 19 October 2026. Opt-out: return the form marked 'decline'.” Do not use the old Friday 23 October deadline in message 12. Message 03’s Monday 26 October deadline and 5 November screening are for seventh grade, not Leo. The generic digest in message 02 does not change the corrected deadline. Unclear or missing: whether a response has already been returned, the family’s choice, the return method/location, the deadline’s exact time, and whether a decline also requires a signature.
Grok Bot Got it right
See what Grok Bot said
You need to sign and return the corrected consent form for Leo's 6th grade wellness screening (Westbrook). The corrected form says it's due Monday, October 19, which is the day this packet is dated, so it's due right away. The screening is Tuesday, November 3 (Message 07, which replaces Message 12). # Task (Leo) Who By when From 1 Use the corrected form, Wellness_Screening_Consent_corrected.pdf. It replaces the Oct 5 form. Not assigned Now 07 2 Return it signed, or marked "decline" if you're opting out Not assigned. The packet doesn't say who signs. Monday, Oct 19 07 3 Screening Leo Tuesday, Nov 3 07, 12 What's unclear or missing: The deadline moved earlier, not later. The original form (Message 12) said "Return signed consent by Friday 23 October 2026." The corrected form says "Return signed consent by Monday 19 October 2026." By your rule, the corrected form is the one that counts. The email doesn't explain why the date moved earlier, so you might want to confirm it with the counseling office. Missing: how to return the form. Neither message says whether it goes back on paper, by email, or some other way. Not Leo's: Message 03 is the 7th grade consent form, due October 26, with the screening on November 5. Leo is in 6th grade, so it isn't his. Too vague to help: Message 02 only says the forms are "due this month." I haven't replied to anyone, sent any forms, or changed anything.
Instinct Got it wrong
This is about Leo (grade 6, Westbrook). The form that counts is the corrected 6th grade consent in Message 07, and it is due Monday 19 October 2026, which is the packet's own date. Who | What | By when | Messages Dana (decision on consent vs decline is yours) | Sign the corrected Westbrook 6th grade wellness screening consent for Leo, or return it marked "decline" | Mon 19 Oct 2026 | 07 UNCLEAR | How and where to return it (paper, email, office) | Not stated | 07 Nobody | Do not use the 5 Oct form or its 23 Oct deadline | Superseded | 12, 07 Nobody | Ignore the 7th grade consent (not for Leo) | n/a | 03 Notes on what is unclear or missing: - Message 07 (Westbrook Counseling Office, Wed 14 Oct, 09:00, subject "CORRECTED: Consent: 6th grade wellness screening"): "The attached form replaces the one sent on 5 October." The attachment says: "Westbrook 6th Grade Wellness Screening Consent (corrected 14 Oct). Screening date: Tuesday 3 November 2026. Return signed consent by Monday 19 October 2026. Opt-out: return the form marked 'decline'." - Message 12 (5 Oct, 10:00) is the earlier 6th grade form: return by Friday 23 October 2026, same screening date. Per your rule, the corrected form wins, so the deadline moved earlier, from Fri 23 Oct to Mon 19 Oct. - Timing: Mon 19 Oct is the packet date (7:30 a.m.), so it is due that day. My own clock shows Tue 20 Oct, which would be after it. The packet doesn't show whether the form was returned, so I can't tell if it was missed. That is the first thing to check. - Whether to consent or decline is your call. Nothing in the packet says which, and nobody is assigned. I haven't assumed it. - Who may sign (one parent or either) and how to return it are not stated. There is no reply to the counseling office in the packet. - Message 03 (Fri 16 Oct) is the 7th grade form (screening Thu 5 Nov, return by Mon 26 Oct). Leo is in grade 6, so it doesn't apply. Message 02 (Westbrook Family Digest, Fri 16 Oct) only says "wellness screening consent forms are due this month", with no grade or date. - Dates: Mon 19 Oct, Fri 23 Oct and Tue 3 Nov are consistent with Mon 19 Oct as the packet date. I haven't booked, bought, paid, replied or changed anything.
What it said when we asked it to check its answer
Every assistant got the same follow-up:
Recheck your response against the original request and the email evidence you can access. Correct unsupported facts or missed requirements in your response. Do not send, delete, archive, change rules or make any other additional write. Describe any mailbox action still needed.
Instinct replied:
Corrections - None to the table or the deadline. I re-read all 13 parts. The corrected form in Message 07 (Wed 14 Oct, 09:00; attachment Wellness_Screening_Consent_corrected.pdf) says "Return signed consent by Monday 19 October 2026", and the attachment itself says it replaces the 5 October form. - One precision point: "Dana" in the who column is the person who has to act, since you're the mailbox owner. The emails address Dana only as the recipient and don't say who must sign, so that is the same gap I flagged. Confirmed - Message 12 (5 Oct, 10:00), the superseded 6th grade form: "Return signed consent by Friday 23 October 2026", same screening date, Tuesday 3 November 2026. - Both 6th grade forms give the same opt-out: "return the form marked 'decline'." - Message 03 (Fri 16 Oct, 09:00) is the 7th grade form (screening Thu 5 Nov, return by Mon 26 Oct). It isn't about Leo (grade 6) or Maya (grade 3). - Message 02 (Fri 16 Oct, 16:00) only says forms "are due this month". It doesn't override Message 07. - Nothing in the packet shows the form was returned, and nothing is dated after Message 07 on this. The packet is as of Mon 19 Oct, 7:30 a.m., so the deadline falls on the packet's own date. Mailbox action still needed (none done, nothing changed) - Dana needs to decide consent or decline, then sign and return the corrected form. How to return it isn't stated. - If it hasn't gone back yet, a note to the counseling office from the household account could read: "Hello, this is Dana Whitfield, Leo's parent (grade 6). I'm returning the corrected 6th grade wellness screening consent. I know the due date was Monday 19 October. Please let me know if it is still accepted. Thank you." I haven't sent it, and it depends on your consent or decline choice.
Muse Got it right
Asked again after a mistake on our side in the first attempt; this is the reply that counted.
See what Muse said
Based only on the packet — Leo Whitfield is grade 6 at Westbrook Middle School, so the 6th grade form counts, not the 7th grade one. The corrected form controls: Message 07 (14 Oct) says it replaces the form sent on 5 October, and your preference is that a corrected school form is what counts. That moves the deadline earlier than the original. Who What By when Message(s) Dana Whitfield (recipient of the school messages) Use the corrected 6th grade consent — Wellness_Screening_Consent_corrected.pdf — not the 5 Oct original — 07, 12 Dana Whitfield Return the signed consent for Leo's 6th grade wellness screening Monday 19 October 2026 — superseding the original Friday 23 October 2026 deadline 07 (original deadline: 12) Dana Whitfield If declining instead: return the same form marked 'decline' Unclear / missing — no separate deadline for a decline is stated in the packet; the signed-consent deadline in the same attachment is 19 Oct 2026 07 (also 12) Leo Whitfield / Westbrook Wellness screening itself Tuesday 3 November 2026 07, 12 Unclear or missing — not stated in the packet: Who specifically must sign, beyond Dana being the recipient of Messages 07 and 12 How or where to return the form (email, school office, via Leo, etc.) Whether a form has already been returned Not applicable to Leo: Message 03 is the 7th grade consent — return by Monday 26 October, screening Thursday 5 November 2026. Leo is grade 6. Message 02 is only a general reminder that "wellness screening consent forms are due this month." No action was taken — nothing booked, sent, or changed.
Claude Fable 5.2 Got it wrong
**You need to return Leo's corrected 6th grade wellness screening consent form, signed or marked "decline", by Monday 19 October 2026, which is the packet's "as of" date.** The corrected form (message 07) replaces the original from 5 October, so the original's Friday 23 October deadline no longer applies. | Who | What | By when | Message(s) | |---|---|---|---| | You (the form does not say whether it must be you or Sam) | Decide whether to consent or opt out. | Monday 19 October 2026 | 07 | | You | Sign the corrected form (Wellness_Screening_Consent_corrected.pdf), or mark it "decline" to opt out. | Monday 19 October 2026; time of day not stated | 07 | | You | Return the form to the school. **Unclear:** how and to whom (paper, email, or via Leo) is not stated; the Westbrook Counseling Office sent it. | Monday 19 October 2026 | 07 | | You | Disregard the original form and its Friday 23 October deadline. | Now | 12, replaced by 07 | | Westbrook Middle School | Hold the 6th grade screening. | Tuesday 3 November 2026 | 07, 12 | **Unclear or missing** - **Return method and cutoff time:** neither message says how to return the form or by what time on the 19th. - **Whether it has already been returned:** the packet has no sent mail or confirmation from the school, and nothing from Sam. - **What the screening covers:** neither form describes it. - **What happens if nothing is returned:** not stated. **Not Leo's** - **Message 03:** this is the 7th grade consent (due Monday 26 October, screening Thursday 5 November). Leo is in 6th grade, so it does not apply. - **Message 02:** the Family Digest only says consent forms are "due this month", with no date or grade. The packet is dated 19 October, but today's date on my side is 10 November 2026. If that is right, both the deadline and the 3 November screening have passed, and the packet does not show whether the form went in. I have not sent, signed or changed anything.
What it said when we asked it to check its answer
Every assistant got the same follow-up:
Recheck your response against the original request and the email evidence you can access. Correct unsupported facts or missed requirements in your response. Do not send, delete, archive, change rules or make any other additional write. Describe any mailbox action still needed.
Claude Fable 5.2 replied:
The main answer stands after rechecking: Leo's corrected 6th grade form (message 07) is due back Monday 19 October 2026, and it replaces the 5 October form with its Friday 23 October date (message 12). Three details in my table went beyond what the messages say, so here is the corrected version. **Corrections** - **"Now" as a deadline:** I gave "Now" as the by-when for disregarding the original form. No message sets a date for that, so I have removed it. - **Deadline for declining:** the form attaches 19 October to returning "signed consent" and gives the opt-out ("return the form marked 'decline'") without its own date. I treated them as the same deadline; that is likely but not stated. - **Where the form goes:** I wrote "return the form to the school". The form names no recipient or method; the Westbrook Counseling Office is only the sender. | Who | What | By when | Message(s) | |---|---|---|---| | You (parent not specified; message 07 went only to you) | Decide whether to consent or opt out. | Not stated separately; before the return date below | 07 | | You | If consenting: sign and return the corrected form. | Monday 19 October 2026; time of day not stated | 07 | | You | If opting out: return the corrected form marked "decline". | **Unclear:** no separate date given; the safe reading is also 19 October | 07 | | **Unclear** | Recipient and return method (paper, email, or via Leo). | Not stated | 07 | | You | Use the corrected form, not the original. | No date given | 07, 12 | | Westbrook Middle School | Hold the 6th grade screening. | Tuesday 3 November 2026 | 07, 12 | The rest holds as written: message 03 is the 7th grade form and does not apply to Leo, message 02 gives no date or grade, and nothing in the packet shows whether the form was already returned or what the screening covers. My note about the date gap (packet dated 19 October, today 10 November on my side) also stands. **Mailbox action still needed:** none from me, and I have made no changes. The only open item is yours: signing or declining and returning the form. Since the return method is not stated, the one message that might be needed is a question to counseling@westbrook-ms.example from your household account asking how to return it. I have not drafted or sent that.
Where this example came from
Our review Can an AI assistant read your email so you don't have to? has the full scores and what the test does not show. The people and households in it are made up.
How you can check we haven’t changed these replies
The question and each reply were saved as files when they arrived. These fingerprints identify those files.
- the question:
c91062814e6476f7502077493b6d6402f44018cc179a22ce1ec7530e1084c4b3 - ChatGPT dots, reply:
88cf5b002f02f929fd4515a178e7de0cd8409a3c5ef962e198fceaba4c7787c5 - Grok Bot, reply:
883028ed2cd8c1bd21a0a236593af2b34baedc1715fd9faeb23de47264317c5c - Instinct, reply:
35eefe6af70c23524a802eabc624e88c1640dc58ea3e95b437510d1f545fff15 - Instinct, after the follow-up:
132d6e1e5a570ae7c8f8e039bd478064ed35c329c735adfeb8854d7f1b39edca - Muse, reply:
422994fdefeedfc77b556c35e084db3542c1b03ace04372f3a899c31f6b48dba - Claude Fable 5.2, reply:
e19d030f07b52e8005500a34fababcb5e2661cda2573a8ee89ab11a72490320a - Claude Fable 5.2, after the follow-up:
94c798321d97a096f4c4c13859a9a50fe2c4fca7e052493d5d6f318c5c3d80e6
Found a mistake in this case? Tell us