Metacognition: a personal exercise that applies to AI
Checking, doubting, remembering, learning from mistakes: I do it for myself, and I have set it up around my AI with files and scripts.
I have just written an article about what happens on my side when Claude Code goes down. While describing my system, I found six gaps between what I thought I had set up and what was really in place. I fixed them the same day.
I wondered what to call that work. The closest word is metacognition.
What the word means
The psychologist John Flavell proposed the term in 1979: metacognition is what you know about your own way of thinking, and how you monitor it. A pupil who rereads their paper is doing it. So is someone who knows they are bad at mental arithmetic and writes the sum down.
In practice it covers four moves: checking what you have just done, doubting an answer that came too easily, remembering your past mistakes, and changing method when the same mistake comes back.
A model does it badly on its own
My agents announce “done” with the same confidence when the work is good and when it is not. When a model rereads its own work, it has the same blind spots as when it wrote it. And it forgets everything from one session to the next: yesterday’s mistake teaches it nothing today.
I paid for this once with 56 emails sent without my approval. The rule was written down, and the model did not stop to reread it.
Four moves, taken out of the model
In my harness, each of these moves is handled by a file or a script.
Check
When a session ends with modified files, a script runs the project’s fast tests. As long as a test fails, the agent cannot announce “done”. Its own impression of being finished does not count.
Doubt
As soon as a change gets large enough, a second model reviews it. It comes from another vendor and does not have the same blind spots. In a trial with a bug planted on purpose, it flagged it in ten seconds.
Remember
The memory is a folder of about fifty short notes. Every important decision is written there with its date and its reason, so that the next session does not undo it.
Learn from mistakes
A script rereads my conversations and lists the moments when I corrected an agent. When the same correction comes back, it becomes a written rule. And when a rule must never be broken, code locks it.
What it is not
The model does not reflect on itself. When I hand my system to a new model for review, it reads text files, like any reader. It has no access to its own workings.
The idea is an old one. In 1998, the philosophers Andy Clark and David Chalmers described a man with a failing memory who writes everything in a notebook. For them, that notebook is part of his memory. My harness plays that role for my agents: their checking and their memory are in files.
My share of the work
The files do not decide alone. I am the one who says what counts as a mistake, who writes the rule and who chooses what deserves a lock. I have ADHD, and keeping track of all this in my head day after day is exactly what I find hard. The harness carries it for me as it does for the agents.
Writing is part of it. The outage article forced me to check every sentence against the real configuration, and six of them were wrong in the first draft.
Three questions for your own setup
- When your AI says “done”, who checks?
- Where is the last mistake written down, and who will reread it tomorrow?
- Who reviews the work with different blind spots from whoever did it?
If the answer to all three is “the model itself”, this layer is missing. It fits in a few text files.
