● Last verified Jul 9, 2026
Most people try to make AI reliable by wording the prompt better. That's like fixing a flaky test by re-running it until it passes.
Large language models are the most non-deterministic system I've ever shipped against. Same input, different output, equally confident either way. If you've spent any time in QA, that sentence should make the hair on your neck stand up — because making non-deterministic systems trustworthy is the entire job.
So I stopped treating prompts as magic words and started treating AI work the way I treat anything under test: with contracts, grounding, examples, gates, and feedback loops. This isn't a tidy workflow I sat down and designed — it's what survived after hundreds of hours and more prompts than I've bothered counting: operating prompts, role prompts, planning prompts, review prompts, documentation prompts, handoff prompts, memory systems. Most of it didn't work the first time. Here's what did, built on real systems you can go look at — this blog and a Magic side project called Vox Mana, which forced me to solve source authority, structured knowledge, and long-lived project memory at a small scale before I trusted the pattern anywhere bigger.
A prompt is a system under test
QA doesn't eliminate variance. It contains it with structure — known inputs, expected behavior, and a review step that has to pass before anything ships. The mistake people make with AI is trying to win on wording, one prompt at a time. That doesn't scale and it doesn't repeat.
The reframe that changed everything for me: a prompt isn't a wish, it's a test case. It has inputs, a contract for the output, and a way to tell whether the result is acceptable. Once you see it that way, the whole QA toolkit applies.
I use AI like MacGyver, not like an oracle
Ask what I actually do and "prompt engineering" undersells it. I don't wait for the perfect tool or the perfect model. I grab what's already there, chain pieces together, prototype fast, and replace the weak part once I know which one it is. That's not a metaphor I'm reaching for — it's how I've built things long before AI showed up: find a working example, take it apart, understand it, swap out what doesn't hold, put it back together better than you found it.
QA gave me the discipline. This gave me the instinct. Together they're closer to what I actually do than either word alone — QA plus systems thinking plus MacGyver engineering, pointed at whatever system is in front of me that week.
The operating prompt that runs before anything else
Every project I run with AI gets a master operating prompt that sets the rules before a single task starts. On this blog it's an AGENTS.md plus a set of role prompts — pre-flight, planning, testing, writing — that each say what that role may and may not do.
It isn't clever wording. It's an operating model, the same thing I build for QA at work: standardize the intake, the boundaries, and the review, so a hundred small changes don't drift into chaos. Mine spells out the project's purpose, the hard boundaries (keep examples field-safe, never expose confidential detail), the workflow (one branch per task, don't commit without asking), and a checklist every change passes before it's "done." The agent reads it first, every time. Orchestration is just refusing to start work from blank context.
AGENTS.md didn't start as documentation and it didn't stay documentation for long. It became shared memory — the thing that answers "can I trust this," "where did it come from," "what changed," and "will someone else understand this six months from now" without needing me in the room to explain it. That's a bigger claim than "it's accurate." Accuracy is table stakes. Trust is what I'm actually optimizing for.
Grounding beats cleverness (RAG, the boring kind)
RAG doesn't have to mean a vector database. Mine is a curated, repo-local reference pack the agent must read before acting. For this blog it's a voice pack — a profile, a list of anti-patterns, a publish gate, and real annotated examples. For Vox Mana it's a source hierarchy: the repo and the evidence ledgers outrank generated output, which outranks whatever the model sounds sure about.
Most people hear "grounding" and think retrieval. Mine is broader: trusted sources, project history, past decisions, real examples, explicit boundaries, and documentation, all outranking whatever the model sounds sure about. Context is worth more than a clever prompt — which is why I've spent more hours building vaults, handoffs, and decision logs than I ever have tuning wording. Those aren't filing cabinets. They're externalized memory: the project remembering things so I don't have to re-explain them every session.
Persuasive output is not proof. A model can sound completely certain about something it invented. Ground the claim against a source you trust, or cut it. Grounding kills more non-determinism than any prompt tweak ever will.
Show it, don't tell it
The fastest way to raise output quality isn't a longer instruction — it's an example. My reference packs carry before/after pairs: a generic corporate sentence sitting next to the one I'd actually publish, a banned construction next to its fix. Models pattern-match; if you want a pattern, show the pattern.
It's the same reason a good bug report shows the failing input and the expected-versus-actual instead of a paragraph describing the vibe of the bug. Concrete examples remove ambiguity for a junior engineer and a language model for exactly the same reason.
This is also just how I've always learned, long before AI entered the picture. Show me one working example and I can reverse-engineer the whole system faster than any explanation gets me there. So I don't ask a model to invent from a blank page — I give it something that already works and ask it to extend or repair that, then I take the result apart the same way I'd take apart the example.
One real workflow per email
If this is your kind of thing, I send one practical writeup like it when it's worth your time — a pattern, a gotcha, or a piece of the field kit I actually use. No roundups, no filler.
Prompts that write prompts
The highest-leverage move is meta-prompting: a prompt whose only job is to produce the reference the rest of the workflow depends on. The voice pack this very post was checked against started as a prompt I wrote to codify my writing voice — then I had it critiqued and revised before it became the thing every future post gets measured against.
A prompt that builds a gate is worth more than a hundred one-off prompts that each re-explain the gate from scratch. You're not writing instructions anymore; you're building the tooling the instructions run inside.
Close the loop: every miss becomes a rule
In QA this is root-cause analysis. A defect escapes to production, you find out why, and you change the system so that class of defect can't escape again. The fix is never "be more careful next time." The fix is a guardrail.
I run AI work the same way. When the model invented data in Vox Mana, the repair wasn't "prompt harder" — it was a check that fails the build when generated data outruns its source. When a draft came back sounding like AI filler, that exact tell went into the anti-patterns file so it gets caught next time. The workflow gets less non-deterministic over time because every surprise is converted into a constraint.
The clearest time I got this wrong: I spent a long stretch trying to prompt a UI mockup into looking a specific way, one increasingly desperate instruction at a time, and it kept almost-but-not-quite landing. The fix wasn't a better prompt. It was breaking the mockup into a puzzle — smaller, surgical prompts for individual pieces, built separately, then reassembled by hand. A bigger prompt wasn't the answer. Smaller, well-scoped ones were.
That turned into a habit: three questions before I trust anything the model hands me. What evidence supports this? How would I know if it were wrong? How do I stop this exact mistake from happening again? The third one is the one that compounds — it's what turns a one-time correction into a permanent rule instead of a thing I have to catch again next month.
My answer to "how do I make AI more reliable" used to be "better prompts." Then it was "better systems." Now it's smaller than either: tighter tasks, tighter acceptance criteria, shorter feedback loops, validation at each step instead of at the end. A giant prompt asking for a giant result fails in ways that are hard to even diagnose. Six small prompts with six checkpoints fail in ways you can actually point at.
Vibe-prompting vs. an engineered workflow
| Vibe-prompting | Engineered workflow | |
|---|---|---|
| Reliability | re-roll until it looks right | grounded and gated, repeatable |
| When it fails | reword and hope | a rule gets added |
| Why you trust it | "it sounded confident" | evidence plus a review gate |
| Scaling | breaks past a few tasks | accumulates without drifting |
| QA parallel | re-running a flaky test | fixing the system that made it flaky |
Reducing non-determinism isn't about a better sentence. It's about structure the output has to survive.
Where this breaks
The discipline is a tax. For a one-off throwaway prompt, all of this is overkill — just ask the model and move on. The system earns its keep on high-context work that spans many sessions, where the real cost is drift and decisions you can't recover later.
The other failure mode is process theater — building the operating model becomes a way to feel productive while avoiding the actual work. And none of this removes the human gate: a model's fluency still isn't truth, so the last step is always me reading the output with the same suspicion I'd give a green dashboard that's a little too green.
My rule of thumb
Treat the model like a brilliant junior with infinite confidence and no memory. Give it the contract, the examples, and the boundaries up front; ground every claim; gate the output; and turn every mistake into a rule it can't repeat.
That's not really prompt engineering. It's not really about AI, either — AI is just the newest system that happened to need this treatment. The same pattern shows up everywhere I build: QA, automation, Vox Mana, even sorting through a Commander mana base. Take something fuzzy. Understand it. Create structure. Reduce ambiguity. Leave behind a system that's better than the one you started with. If nothing else from this sticks, that's the part worth keeping.
The AI system was never really the point. It's about whether the next person — a teammate, a contributor, or just me in six months — can trust what's there, verify where it came from, and pick up where I left off without a debrief. That's not a side effect of the process. That's the actual job.
I don't think of what I've built as notes. Notes are things you write down and forget. This is accumulated judgment — the project remembering, on my behalf, every mistake I've already made so I don't make it twice.
Reduce the fuzzy. Build the structure. Leave it better than you found it.
// take this with you
Make an AI workflow repeatable — the short checklist:
- Write the operating prompt first: purpose, hard boundaries, workflow, and a "done" gate.
- Ground it: a reference pack the model must read before acting. Source outranks fluency.
- Show, don't tell: keep before/after examples of good and bad output.
- Meta-prompt the tooling: spend your best prompt building the reference the others rely on.
- Keep tasks small and well-scoped: tighter acceptance criteria, shorter feedback loops.
- Close the loop: every escaped mistake becomes a new rule or guardrail.
- Keep the human gate: review the output like a dashboard you don't fully trust yet.
Spinning this into a printable Field Kit card next.