Skip to content

The result looks wrong

How to tell an invented claim from a checkable one without re-reading everything — the signals worth looking at, and what to do once you have found one.

Check the step list, not the summary. A run’s closing paragraph is written by the same thing that did the work, so it is the least independent part of the result. The list of steps — what it searched, what it opened, what it wrote — is a record of what actually happened, and comparing the two takes less time than re-reading the output.

Checking its work is the full method, including which safeguards are enforced by the system and which are only rules it follows. This page is the short version for when you already suspect something.

  • A claim with no step behind it. If the summary says it checked a source and no step in the list opened anything, the claim came from the model, not from the source. This is the single highest-yield check.
  • Specific numbers, names or dates that no step produced. Precision is not evidence. A figure that did not come out of a file it opened or a search it ran is a figure it wrote.
  • Links. A link it produced without a step that fetched it may or may not resolve. Open the ones the decision rests on.
  • A file it says it wrote. Deliverables are named for you — download the file and look at it rather than trusting the description of it. A summary can describe a file more optimistically than the file deserves.
  • The job it did versus the job you asked for. A common failure is not a false statement but a substituted task: you asked for an analysis of three options and got a summary of one. Re-read your own request first.
  • Anything it was never given access to. If it reports on the contents of an app you never added to the run, it did not read that app — see A connected app isn’t being used.

Some things are enforced rather than promised, and they are worth knowing so you check the right things:

  • The receipt. It is generated from what was actually charged, not written by the model.
  • Anything that leaves the app. Sending or publishing stops for your approval no matter what setting you chose, so a result cannot claim to have sent something you did not approve.

What it won’t do draws the line between the limits that are mechanical and the ones that are only rules.

  • Say what was wrong, in the same chat. A specific correction — “the second figure is wrong, check the file again” — gets further than “this is wrong”. It has the context; it does not have your judgement.
  • Give it the material. Most invented content is a gap being filled. Attaching the source it needed is usually a better fix than instructing it to be careful — see Giving it materials.
  • A follow-up is a fresh run with its own cost. There is no free correction, and a follow-up to a finished task does not carry the old result or its files with it.
  • If it is on the wrong track rather than merely inaccurate, stop and re-ask rather than correcting repeatedly. Three corrections cost more than one clear restatement.

Reviewing what comes back covers where deliverables live and how to tell finished from stalled, which is a different question from whether the content holds up.