How we built an AI agent that creates videos while you sleep

AI Video & Visuals


A video example is at the end.

Somewhere in a Dutch railway data center, a small worker process is started on a cron schedule. In the next 2 minutes and 20 seconds, you come up with 5 video ideas, debate with yourself which one is best, write a script, match the script to your brand rules, generate a German voiceover and music bed, render a 30-second vertical video, and upload it to YouTube.

After that, I fall asleep again for 3 days.

Total cost for the past few days: $1.91. There are 5 videos. My involvement: 0 minutes. These videos are about Safebird, the agent operating system we’re building for insurance brokers.

In my previous article, I said that the founders who are winning right now are: systemnot just the product. This is what it actually looks like, a running process with a heartbeat. Here we show you exactly how it’s built, including failed versions, real prompts, and guardrails to stop you from running before you spend money on off-brand videos.

what the machine actually does

The agent is a seven-step pipeline divided into three distinct stages of predictability. There are only three steps to using LLM, GPT-5.6 Terra Pro via OpenRouter, and each is a separate little agent with its own persona, its own prompts, and its own temperature. Two further steps are generative but limited. ElevenLab synthesizes narration and music in one one-shot call, without feedback loops. And the rest of the steps, guardrail lint, remotion rendering, and uploading, are simple deterministic code, same input, same output every time.

imageimage

Notice its shape. Three creative steps, then a hard gate, then a production step. The creative part is cheap (about $0.23 for an LLM call). all rear Gates cost real money, including speech synthesis, music generation, and rendering computing. Therefore, the gate is exactly where the money begins.

That order is no coincidence. This is the entire design philosophy in one sentence.

Judgment before spending.

Which version failed (and why)

We didn’t start here. In version 1, we did what everyone does first. I left it to LLM. Create a visual.

LLM wrote the raw Remotion TSX code for all scenes, including layout, animation, and positioning. Next, since LLM-generated layouts are unreliable, we added a second LLM pass to examine and grade the rendered frames. This is a multimodal image QA loop that regenerates the scene until it passes.

Technically it worked. It was also a disaster:

  • Up to 6 euros per video: Scene-by-scene code generation and image QA loops burn your tokens like a furnace.
  • Contrast issues, text overflow, visual artifacts: A model that produced great German copy turned out to be a mediocre layout engine.
  • Hard API error: GPT-5.6 Terra Pro is an inference model and inference tokens count against the max_tokens budget. Generating approximately 500 rows of TSX in one shot results in 500 errors. A monolithic “one call generates everything” approach not only looks bad; Died at API level.

This fix came from accepting a simple truth about how good these models actually are.

LLM excels in picking and filling. They don’t like pixel-perfect creations.

So I removed pixels from the model. I hand-designed a library of 10 parameterized scene templates. Kinetic headlines, quote cards, chat mockups, stat callouts, closers, everything built once in code and built on brand. The LLM job has become just one thing to do with certainty. It’s all about choosing the right template for each beat and creating the copies it contains.

The cost per video has fallen from about 6 euros to less than $0.40. quality has declined aboveBecause the visuals became coherent by construction, not by chance.

imageimage

Three agents and their real prompts

Everyone asks about the prompt. So here are the authentic prompts that are in production today. Each agent is one focused call with a narrow job. Notice what they have in common. It’s a sharp personality, a narrow scope of work, and a temperature tailored to the job.

Agent 1: Brainstorming · Temperature 0.95

You are a sharp short-form video strategist for safebird.ai.
[guardrails injected here]

Your job: pitch FIVE genuinely different rough storyline ideas for a
30-second vertical YouTube Short. Each must be a distinct angle on how
AI assistants change a broker's daily work — capacity, speed of first
response, documentation, dormant Bestand potential, being present at 24/7, staying the trusted human while the assistant does the
Vorarbeit, etc. Vary the emotional register (aspirational, relief,
fear-of-missing-out, pride). Keep each thesis concrete and specific
to insurance broking.

One detail I like is that before brainstorming, agents are told to retrieve the titles of their recent uploads from the YouTube API and not repeat those angles. Machines have memory. It remembers what has already been said, rather than just generating it.

Agent 2: Rank/Temperature 0.3

You are a ruthless creative director for safebird.ai short-form video.
[guardrails injected here]

Score each of the 5 ideas 0–10 on: hook strength in the first 2
seconds, clarity of the single takeaway, emotional pull for a solo
broker, and how well it can be told VISUALLY (kinetic type +
charts/shapes, no live footage). Pick the single best as winner.

Agent 3: Write · Temperature 0.8 · Big One

You are a German short-form scriptwriter + art director for safebird.ai.
[guardrails + the full template catalog injected here]

Build a vertical Short (30fps) from the idea, ~25-35 seconds.
Structure: a 5-6 scene arc — hook → build → a calm "substance"
turn → CLOSER. Each scene = a VO line + a chosen visual TEMPLATE
with its copy fields. You DO NOT write any visual code —
you only pick a template and fill its text.

Rules:
- LANGUAGE QUALITY (critical): every VO line and every on-screen
  text must be flawless, natural, idiomatic German — exactly how a
  native speaker would say it out loud. Read each sentence in your
  head; if it sounds constructed, translated, or awkward, rewrite
  it simply. Short clear sentences beat clever-but-clunky ones.
- EVERY scene MUST have a spoken VO line — no silent scenes.
- The LAST scene MUST be the "closer" template.
- VO: calm deep voice (~2 words/sec). ~45-60 spoken words TOTAL.
  Never speak "safebird.ai".
- Also write YouTube metadata (title, Du-form description, tags).

There’s a real lesson in these three prompts, but it’s not “prompt engineering.” That’s scope engineering. There are no agents that perform multiple cognitively distinct jobs. Divergence (0.95) is separated from Judgment (0.3), which is separated from Technology (0.8). If you try to make one call creatively, critically, and accurately at the same time, you’ll end up with an average of all three, which will be mediocre on the scale.

How can we trust what we haven’t seen?

This is a question everyone has about autonomous agents, and it has a concrete answer. Translate your decisions into code and write that code. before money.

Between the writing agent and the first paid API call, there is a deterministic linter that hard-gates everything LLM writes, a simple regular expression that is not AI. Check audio narration, all on-screen text fields, and YouTube metadata against brand and compliance rules.

// real rules from guardrails.ts — a violation THROWS and aborts the run
GENDER_STAR  = /\b[A-Za-zÄÖÜäöü]+[:*·/][Ii]nn?en?\b/   // no "Makler:innen"
PRICE        = /(\d+\s?(€|eur|euro)|\b\d+\s?%|…)/i     // no price/% claims
FAKE_PROOF   = /(garantiert|marktführer|testsieger|…)/i // no invented proof
AVOID_PHRASES = [/in deiner marke/i, /unter deinem namen/i]

// and my favorite, learned the hard way:
if (/safebird\s*\.\s*ai/i.test(vo))
  → violation: "spoke raw domain safebird.ai"
  // because the text-to-speech voice mispronounces it.
  // On-screen text may show it. The voice may never say it.

The last rule is a detail that can only be learned by shipping. There is no “TTS dismantles domain names” framework. You notice it, wince, encode it, and the machine will never make that mistake again from then on. Every guardrail in that file is a scar from having failed once. This is what a production prompt actually looks like. It’s half creativity and half scar tissue.

And the failure modes are intentionally set to be cheap. If the linter is thrown, it will abort execution after incurring about $0.23 in LLM calls and before spending a cent on audio, music, or rendering. Skipped cycles cost pennies. Public off-brand videos take trust. The system is designed to skip cycles because a skipped cycle costs less than bad video.

Additionally, uploads are private, live, and linkable by default, but won’t be pushed to subscribers until you switch one environment variable to public. Full autonomy but gradual trust. We don’t hand over the keys on the first day. Observe the machine’s behavior and then extend its privileges.

two halves of the people never speak to each other

One more architectural choice worth stealing. Agents operate on railways. My admin dashboard is running on Vercel. They have no network connections to each other and share one thing: the database.

imageimage

When you press “Start Now” on the dashboard, a row is inserted into the trigger table. The worker polls that table every 30 seconds, retrieves the rows, runs the pipeline, and writes back the timing, status, and exact cost of each step. The dashboard displays all the contents of your database. If an employee dies, the dashboard will show the old heartbeat within 30 minutes. There are no webhooks, queues, service meshes, or tables.

This is the sleepy, boring kind of architecture. Any part can be stopped, redeployed, or rewritten without affecting other parts.

Numbers, numbers are the key

imageimage

The $0.37 breaks down to approximately $0.23 for OpenRouter LLM calls (3 text agents) and ~$0.14 for ElevenLab (voiceover + 1 music generation). Also, since OpenRouter returns the exact cost of every call in the response, these are not estimates, and the runtime collects real numbers from every step and writes them to the database. The dashboard screenshot at the top of this article is not a mockup.

For comparison: The last time I paid a freelancer to edit a short video, the cost of one 30-second clip was more than what this machine would spend. 1 year.

What does this actually mean?

If you zoom out, the video itself almost misses the point. A 30-second short video for a German insurance broker won’t break the internet. What matters is the shape of things.

  • Instead of one giant prompt with branching, judgment, and crafting, three smaller agents were separated and tempered.
  • LLM is limited to what it does best: select and embed pixels that the code owns.
  • Decisions encoded before spending and deterministic gates that default to cheap failure modes.
  • Gradual trust. Make your video private at first, then make it public when you’re confident.
  • Boring separation where the two platforms only share database tables.

None of this is an AI trick. These are engineering decisions. And that’s the real argument behind “Build systems, not just products.” In other words, the model has no influence. Models are a commodity and everyone has the same thing as me for 15 cents at a time. The power is in the machines you build around it Models: intentionally designed constraints, gates, memory, instrumentation, and failure modes.

We need you to market your product. The system sells itself. That means you’ll sell one video every three days while you sleep for less than a cup of coffee.

A four-hour work week does not mean reducing working hours. By spending time building the machine, the machine can spend time building the rest.

This took a weekend to build and one painful lesson at 6 euros per video to get it right. Now it has paid for itself in only one currency. Time you’ll never have to spend again.

Build a machine Build a machine.



Source link