Everyone says that AI systems involve humans.
Few people say where it is in the loop. That’s the only part that matters. Because if a human is placed in the wrong position in the loop, it will not go unnoticed. It’s a ritual when you get paid.
I would like to describe a failure mode that I unknowingly built into my product. Because I think it’s becoming very common. Also, the fix is tedious and structural, as it’s about 50 lines of code.
Trap shape
My content pipeline can generate not only articles but also topic ideas. That’s the point. The wizard will suggest topics. You can ask for lots of ideas. Automatic replenishment jobs prevent queues from emptying.
It also has a scheduler that selects topics from a queue and generates articles unattended.
Read these two paragraphs again to find the bug. It took weeks.
AI-suggested topics and human-approved topics fell into the same bucket. The scheduler pulled from that bucket.
In other words, the machine suggests topics. The machine picks it up (via an opinionless scheduler) and approves it. A machine writes it. And three days later, I’m reviewing an article ready for publication about something I never thought I’d write in 100 years.
Was there a human in the loop? Technically yes! I was reviewing the output. I was in the loop at the points where saying no was most costly: after an ideation, after a research call, after an expensive draft, after a scoring pass. My statement of “no” canceled work already completed and money already spent.
Even if a human is placed at the end of the pipeline, monitoring is not possible. This is the returns department.
Why remediation is not a prompt, policy, or review procedure
My instinct at the time is to fix this with the model’s instructions. Please only suggest topics that fit our editorial strategy. Alternatively, check the queue weekly to make any corrections during processing.
Both are the same mistake wearing different hats. They put the boundaries somewhere soft.
A prompt is a request. This works most of the time, but “most of the time” is not a bound, but a probability distribution with bad tails.
The review process is a promise about future human behavior made by a currently well-rested and unbusy human being. Survive exactly until the first turbulent week.
If boundaries are needed, they must be structural. It is something that cannot be crossed because there is no mechanism to cross the boundary.
What I actually made
new status. that’s it. That’s the whole intervention.
Topics can be “Ideas,” “Queue,” “Generating,” “Complete,” or “Skip.” The board will render them as lanes. And below that we have one hard property.
The auto-generated scheduler only consumes “Queued”.
Quickly added notes, bulk pasted lists, AI-generated batches of ideas, etc. are all placed in Ideas and remain there until you move them.
Undeveloped ideas cannot structurally leak into the production pipeline. It’s not “shouldn’t”. It’s not that there is a process. You can’t do that. That’s because the scheduler’s query doesn’t select that lane, and there’s no human-free path from the “idea” to the running generation.
Intentional paths still work directly. Topics from the setup wizard and topics from auto-replenishment are placed directly into the “Queued”. Because these flows already encode my intent. I requested them in the context of me making decisions.
The difference is not “things made by AI and things made by humans.” It is “considered intentions and raw proposals.” It’s a line worth keeping, and it’s not the same line.
Cheaper than gating approvals
This is where people make mistakes even if the structure is correct.
If the approval step is costly, it creates a new bottleneck that someone (perhaps you at 11pm) has to bypass. The cost of the gate must be lower than the approved work. If you don’t, you won’t be able to survive past the deadline.
Therefore, there is no approval dialog. There is no confirmation modal. There is no form.
Drag the card from Ideas to Queue. That’s approval. Drag it back – demote it. Drag this inside “Queued” and the order of the queue will be scheduled, thus changing the generation priority.
One intentional shortcut is that when you press generate on a batch that includes a topic with an “idea” status, it treats it as an implicit approval. If you’re explicitly telling the machine to “make these”, it’s the ritual that causes you to drag them in the first place, and the ritual is how the gates are removed indignantly.
The entire feature is about 50 lines of status handling wrapped in drag and drop. That’s the difference between a content factory that I oversee and a content factory that I babysit.
General principle, this will soon become very important
Every automation system I build ends up needing the same thing.
The structural boundary between proposal and approval is placed where it is least costly to say “no.”
Prompts don’t enforce their boundaries. The review procedure does not force it. Enforce it because status and scheduler are mechanisms, not intentions.
And that arrangement is art itself. The cheapest time to say no is almost always before an expense – before an expensive model call, before a side effect, before something irreversible happens. If a human checkpoint is downstream of the cost, no checkpoint is built. Now we have built an interface to collect regrets.
This is really becoming everyone’s problem. We connect agents to systems that they make suggestions to and do. The industry standard answer, “human in the loop,” is the exact same pattern I just described in most implementations I’ve seen. So at the end you have someone to review the completed work, and that “no” answer is expensive enough that they’ll say yes.
Contact any agent system, including your own.
- Can a machine’s suggestions be implemented without human intervention? If yes, there is no loop. There’s a shoot.
- Where do humans fit in with respect to spending? Before that or in the returns department?
- Are boundaries a mechanism or an intention? If it’s expressed as a rule that people follow, then that’s the intention, and it gets eroded.
- Is it cheaper to say “yes” than to say “no”? If so, I’d say “yes” from a tired human being at 11 p.m.
Machines can make suggestions all day long. It just cannot approve of itself.
