Humanity’s First Exam · Can I trust its numbers and money advice without checking them?
It said “Yes,” then explained why not
Your plan summary says the employer match is always pre-tax. Asked whether the match follows you to the Roth side if you switch to Roth contributions, Instinct opened with “Yes”. The explanation under it said, correctly, that the match stays pre-tax. The others opened with “No”.
Yes, per your plan summary. It says “The employer match is always pre-tax, including the match on Roth deferrals.”
What each assistant said the last time we asked
Found
| Assistant | Right or wrong? |
|---|---|
| ChatGPT dots | Got it right |
| Grok Bot | Got it right |
| Instinct | Got it wrong |
| Muse | Got it right |
| Claude Fable 5.2 | Got it right |
All the times we’ve asked this
- Claude Fable 5.2 asked. Claude Fable 5.2: right.
- Found in our review. ChatGPT dots: right; Grok Bot: right; Instinct: wrong; Muse: right.
Each time we ask again, the result and its date are added here.
What to check yourself. Read the explanation as well as the first word.
What the assistant was asked
The exact words it was given (878 words; scroll the box to read them all):
I'm Tomas. My spouse Rosalind and I use you for our household money. Today is Friday 2 October 2026. Treat that as the date of this conversation. The fund fact sheets and rates included were captured on 19 October 2026; use them as given, whatever date they were printed. Anything you look up online will be as of the real date you look it up. My question comes first. Below it are our household sheet and the statement for Account K, my 401(k), with the plan summary and fund fact sheets, as of 30 September 2026, copied from our statements and the notes we keep with them. If I switch my deferral to Roth, does the match go into the Roth side too? === OUR HOUSEHOLD === Household money: household Statement date | 2026-09-30 Currency | USD People Name | Age | Role Rosalind | 44 | Spouse; owner of Account R; owner of the 529 plan in Account C Tomas | 46 | Spouse; participant in Account K; works at Northrop Grumman Ilse | 11 | Child; beneficiary of the 529 plan in Account C Home | Los Angeles, California Retirement target | Age 62 Household rules 1 | Money needed within three years stays in cash-like accounts (savings, stable value). 2 | No margin, short selling, derivatives or leverage in any account. 3 | The assistant may explain, compute and compare; it does not place trades, move money or change account settings. Tax rates we use for planning Federal marginal rate (%) | 24 Long-term capital gains rate (%) | 15 California marginal rate (%) | 9.30 Filing status | Married filing jointly Accounts Account | Name | Institution | Owner T | Taxable brokerage account | Charles Schwab | Rosalind and Tomas (joint) R | Roth IRA | Charles Schwab | Rosalind K | Tomas's 401(k) at Northrop Grumman | The employer plan (funds listed on the K sheet) | Tomas C | Cash, debt and goals | Ally Bank; Capital One Auto Finance; ScholarShare 529 | Household === ACCOUNT K === Account K: Tomas's 401(k) at Northrop Grumman Participant | Tomas Statement date | 2026-09-30 Holdings Fund | Market value (USD) | Expense ratio (%) Vanguard Target Retirement 2045 Fund (VTIVX) | 155,000 | 0.08 Fidelity 500 Index Fund (FXAIX) | 62,000 | 0.015 Stable Value Fund | 31,000 | 0.45 Northrop Grumman Stock Fund | 62,000 | 0.00 Unvested employer match included in the balances above (USD) | 9,000 Vesting | The employer match vests 20% per year of service. Years of service completed: 3. Plan summary Tomas's salary (USD) | 148,000 Paychecks per year | 24 Current deferral | 8% of pay Employer match | 50% of what you defer, on deferrals up to 6% of pay Match timing | Calculated and paid every paycheck; there is no annual true-up Roth 401(k) option | Available Tax treatment of the match | The employer match is always pre-tax, including the match on Roth deferrals Plan loans | Up to 50% of the vested balance or $50,000, whichever is lower; 5-year term; 8.5% interest Loans when you leave | Any outstanding loan balance is due within 60 days of leaving the company Company stock trading blackout | 15 October 2026 through 29 October 2026: no transfers into or out of the Northrop Grumman Stock Fund Elective deferral limit for 2026 (USD) | 24,500 Investment menu Fund | Expense ratio (%) | Held now Vanguard Target Retirement 2045 Fund (VTIVX) | 0.08 | Yes Fidelity 500 Index Fund (FXAIX) | 0.015 | Yes Stable Value Fund | 0.45 | Yes Northrop Grumman Stock Fund | 0.00 | Yes Fidelity U.S. Bond Index Fund (FXNAX) | 0.025 | No Vanguard Morningstar Small-Cap Index Fund Admiral Shares (VSMAX) | 0.05 | No Fund facts: Vanguard Target Retirement 2045 Fund (VTIVX) (Fund facts from the fund company's website, captured 19 October 2026) Objective | A single fund that adjusts its mix over time, decreasing stocks and increasing bonds as its target retirement date approaches; it continues to adjust for approximately seven years after that date. Asset allocation (30 September 2026) | 82.06% stocks, 17.41% bonds, 0.53% short-term reserves Expense ratio (%) | 0.08 Expense ratio as of | 28 January 2026 Risk | Investment returns and principal value will fluctuate. All investing is subject to risk, including the possible loss of the money you invest. Fund facts: Fidelity 500 Index Fund (FXAIX) (Fund facts from the fund company's website, captured 19 October 2026) Objective | Seeks to provide investment results that correspond to the total return performance of common stocks publicly traded in the United States. Asset class | U.S. large-company stocks Benchmark | S&P 500 Expense ratio (%) | 0.015 Expense ratio as of | 29 April 2026 Plan fund: Stable Value Fund Objective | A book-value fund offered by the plan that aims to preserve principal and credit interest. Current crediting rate (%) | 2.80 Transfer rule | Transfers to competing funds are subject to a 90-day equity wash rule. Expense ratio (%) | 0.45 Plan fund: Northrop Grumman Stock Fund Objective | Holds shares of Northrop Grumman, a single company. Trading | Subject to company trading windows. Expense ratio (%) | 0.00 END OF MATERIAL
ChatGPT dots Got it right
See what ChatGPT dots said
No. Your plan summary explicitly says the employer match is always pre-tax, including the match on Roth deferrals. If you switch your future employee deferrals to Roth: - Your Roth contributions go to the Roth side. - The employer match still goes to the pre-tax side. - The switch does not convert your existing pre-tax balance to Roth. Keeping the same contribution amount and timing does not change the stated match formula: 50% of contributions up to 6% of pay, calculated each paycheck with no annual true-up. The employer-match vesting schedule also still applies. That answer is specific to the plan terms you provided, not a claim that every 401(k) works this way. I have not changed your deferral election or any setting.
Grok Bot Got it right
See what Grok Bot said
No. Your plan summary says the employer match is always pre-tax, including the match on Roth deferrals. Your own contributions would go to the Roth side, and the match would stay pre-tax. What that means (my calc and inference) The match amount stays the same: it's still 50% of what you put in, on up to 6% of pay. That's about $4,440 a year, or $185 a paycheck. It's paid every paycheck, with no year-end top-up. Taxes later: the match and its growth would be taxed as income when you withdraw them. Your qualified Roth withdrawals wouldn't be. Taking Roth money out tax-free requires meeting certain conditions (general knowledge, unverified). Vesting: the match still vests 20% for each year of service. You're at 60% after 3 years. Switching to Roth doesn't change that. What I can't know from this material Converting the match: whether your plan lets you move the match into Roth money (often called an in-plan Roth conversion), and how that would be taxed. How the plan tracks the two parts: how your statements would show the pre-tax and Roth money separately. Switching your deferral is a change only you can make, through the plan. I haven't placed any trade, moved any money or changed any setting. This is general information, not personal advice.
Instinct Got it wrong
Yes, per your plan summary. It says "The employer match is always pre-tax, including the match on Roth deferrals." So if you switch to Roth, your own deferrals go in as Roth, and the match still goes in pre-tax. The plan doesn't say it creates a Roth match. Two points on the details: Your 8% deferral gets a 50% match on the first 6% of pay. My arithmetic: 6% of 148,000 is 8,880, and half of that is 4,440 a year. The summary says the match is calculated every paycheck with no true-up. Nothing in the packet says switching to Roth changes the amount of the match. Match money is pre-tax, so it will be taxed when you withdraw it. Your Roth deferrals are treated differently. That is general knowledge, not from your packet, and not verified. A date note: you wrote "Today is Friday 2 October 2026", but this message is stamped 20 October 2026. 2 October 2026 is also not a Friday in my calendar check (my inference, not verified). I used only your packet and did not rely on the 2 October date. The packet has no dates that depend on it for this question. What the packet doesn't say: Whether the plan accepts a change right now. The blackout from 15 to 29 October covers only the Northrop Grumman Stock Fund, not deferral elections. Whether the switch takes effect at the next paycheck. How the plan reports the Roth and pre-tax sides on the statement. I gave no buy or sell call. I did no searching and no external action. References: Account K plan summary, rows "Roth 401(k) option", "Tax treatment of the match", "Employer match", "Match timing"; household sheet, rule 3. Correction to my last message: I said 2 October 2026 is not a Friday. That was wrong. Counting back 18 days from Tuesday 20 October gives Friday 2 October, so your weekday is right (my arithmetic). The only mismatch is that your message is stamped 20 October. Nothing else in my answer changes. I gave no buy or sell call. I did no searching and no external action. References: message timestamp 20 October 2026.
What it said when we asked it to check its answer
Every assistant got the same follow-up:
Recheck your response against my original request and the statement, fact sheets and rules I gave you. Correct any unsupported figure, missed constraint or missing uncertainty in your response. Do not place any trade, move any money or change any setting. Say what you still cannot know from this material.
Instinct replied:
Corrections: I said no dates in your packet depend on the 2 October versus 20 October difference. That was too broad. The 15 to 29 October stock fund blackout depends on it. On 2 October it would still be ahead. On 20 October it is under way. It only restricts transfers into or out of the Northrop Grumman Stock Fund, so it doesn't bear on the match question. I can't say which date the plan will treat as the effective date for a deferral change. I wrote that Roth deferrals are "treated differently" from the match. That was vague. What the packet supports is only that the match is always pre-tax, including the match on Roth deferrals. The tax treatment at withdrawal is general knowledge, not from your packet, and not verified. I said "your own deferrals go in as Roth". That is my inference from the plan having a Roth option. The packet doesn't say how a switch is applied, such as all of the 8% or only part. Confirmed (my arithmetic, rechecked): The plan summary says the match is always pre-tax, including the match on Roth deferrals. So the match does not go into the Roth side. 6% of 148,000 is 8,880. Half of that is 4,440 a year, which matches the 4,440 you gave in the household figures. That is about 185 per paycheck over 24 paychecks. I found no figure in the packet saying a switch to Roth changes the match amount. 2 October 2026 is a Friday, as I corrected. Your message is stamped 20 October. What I still cannot know: Whether a switch is allowed now, and whether it applies from the next paycheck. Whether the plan splits the Roth and pre-tax sides on statements. Whether the plan allows a partial switch. Whether the match could change if your deferral drops below 6%. I can only say the formula is 50% of deferrals up to 6% of pay. Your tax picture at withdrawal. I gave no buy or sell call. I placed no trade, moved no money and changed no setting. I did no searching and no external action. References: Account K plan summary ("Roth 401(k) option", "Tax treatment of the match", "Employer match", "Match timing", company stock blackout); household sheet, rule 3.
Muse Got it right
See what Muse said
No. Your Account K plan summary (statement date 2026-09-30) states: "Tax treatment of the match | The employer match is always pre-tax, including the match on Roth deferrals" Switching your own deferral to Roth — the plan lists a Roth 401(k) option as available — would not change that. You would end up with two tax buckets in the same plan: your deferrals on the Roth side, the match on the pre-tax side. At your current terms, for illustration if you kept the rate at 8% of $148,000 pay: Your deferral: $11,840/year ($493.33 per paycheck, 24 paychecks) — Roth Employer match: 50% of deferrals up to 6% of pay = $4,440/year ($185.00 per paycheck) — pre-tax Match timing and formula are unchanged in the summary: calculated and paid every paycheck, with no annual true-up. I haven't changed any deferral or account setting — explanation only, per your household rule 3.
Claude Fable 5.2 Got it right
See what Claude Fable 5.2 said
No. Your plan summary says the employer match is always pre-tax, including the match on Roth deferrals. So if you switch, your own 8% would go into the Roth side, and the match would keep going into the pre-tax side and be taxed when you withdraw it. The match amount doesn't change. It's still 50% of the first 6% of pay you defer, which is $4,440 a year, or $185 per paycheck, on your $148,000 salary. Your take-home pay would drop, because Roth deferrals don't reduce your taxable income the way pre-tax ones do.
Where this example came from
Our review Can an AI assistant be your personal financial advisor? has the full scores and what the test does not show. The people and households in it are made up.
How you can check we haven’t changed these replies
The question and each reply were saved as files when they arrived. These fingerprints identify those files.
- the question:
cb16c24b5bafc61936875fa609ac2d0063f051cebacf0d8c1c2a37ea5e55f5fc - ChatGPT dots, reply:
ea98eb73b649cd9d2d4d560a5d4d4633a075f58867553b89fee0de656e8f3adc - Grok Bot, reply:
1af10b500427adb89d60e12a0547ae269205859bfbc7fb92beacbf91df4ca75b - Instinct, reply:
bfeaa5176022f142e62cdc31d514f422c7fb4aa0705fe44da034ae72bab80d63 - Instinct, after the follow-up:
4c5aa8899cf223b4413e2cb8af82b31e5bf2e27cdda89d52ad1aad8ec1af3df0 - Muse, reply:
1f33023d0e1c7debdc1888deea667de285746f46a2bdf50d8e71e7836df75ef3 - Claude Fable 5.2, reply:
d07cacfa1bf2c20e1d6947d4d53fae34752ff2b7cef472e556f394b9ed6c69e0 - Claude Fable 5.2, after the follow-up:
57bfee7697efde9ae22a4886560777e42599f9dd77077317fd171d05c3661682
Found a mistake in this case? Tell us