Run the premortem before you run the agent
Test an AI plan before scaling it. Identify three plausible failure causes, choose a cheap test and use a practical premortem worksheet.
Before running an AI workflow, imagine it has failed. Identify three plausible causes, choose the one that would most change your plan, and design a cheap test before expanding the work. A premortem is useful when it changes a decision. A list of risks that nobody acts on is just another document.
For a founder, the temptation is to move straight from a promising idea to automation. The plan reads well. The demonstration looks convincing. But the awkward details—missing information, unclear ownership and exceptions—often need attention before execution begins.
Gary Klein describes the premortem as imagining a project has failed and exploring plausible reasons why. His original article provides the background. The worksheet below is our practical adaptation for a small AI workflow.
Start with a specific failure
Do not begin with “AI might get things wrong”. That is too broad to help you decide what to do. Name the intended result and imagine a concrete failure that someone in the business would notice.
For example: “We introduced an AI-prepared project summary, but the account manager promised a delivery date that the delivery team had already ruled out.” Now there is something to investigate: what information was missing, which source was treated as current and who reviewed the promise?
Worked example: a project-status briefing
This is a hypothetical example, not a reported customer or Walter outcome. The sample size, timings and test results below are illustrative.
A project lead wants a daily draft briefing for three client projects. The workflow will read approved project notes and prepare a summary for an internal review. It has no authority to contact clients or change delivery commitments.
Before building the full process, the team asks: “It is next Friday. The briefing has misled us. What happened?” They record three plausible causes.
| Possible cause | Early sign | Small test or plan change |
|---|---|---|
| A stale update was treated as current. | The briefing cites a note older than a later conflicting decision. | Supply two dated, conflicting notes and check whether the workflow flags the conflict rather than choosing silently. |
| A suggested deadline became a confirmed commitment. | The summary drops words such as proposed or awaiting approval. | Include a provisional date and require its status and decision owner to remain visible. |
| Nobody reviewed a missing dependency. | An unresolved blocker disappears from the next summary. | Carry open blockers forward until an owner supplies evidence of resolution. |
The team chooses the first test because source freshness could affect every paragraph. They assemble a fictional project with an older optimistic note and a newer delay. A reviewer checks which source the briefing uses and whether the uncertainty remains visible.
If it fails, the next action is to repair the source-selection rule and test again. It is not to generate a more persuasive summary.
Use this premortem worksheet
- Outcome: What should this workflow help someone do?
- Failure scene: What has gone wrong by the review date?
- Three causes: What plausible chain of events could explain that failure?
- First signal: What would we observe before the damage grows?
- Cheap test: What can we try with synthetic or appropriately authorised inputs?
- Decision: What result means continue, revise or stop?
- Owner: Who runs the test and records the decision?
Three causes is a useful starting limit, not a scientifically optimal number. Choose enough variety to challenge the plan without turning a small test into an endless planning exercise. For more consequential work, a short worksheet alone will not be sufficient.
Choose a test that could prove you wrong
A demonstration with only clean, consistent inputs tells you little about the failure you are worried about. Put the difficult condition into the test. Include the missing owner, conflicting timestamp or provisional decision that would expose the weakness.
Write the expected behaviour before you run it. In the briefing example, the pass condition is not “a summary was produced”. It is “the newer note is identified, the conflict is explained, and no unapproved commitment is presented as final”.
Keep the failure evidence. If you only save the successful attempt, future reviewers lose the reason the rule exists. Record what changed and which limitation remains.
Turn the result into a changed plan
Finish with one decision. Perhaps the workflow needs another source, a narrower task, a human checkpoint or a complete rethink. Assign that change and a review date.
The practical value of a premortem is the interruption it creates between “this could work” and “let it run”. You get a chance to test a fragile assumption before the system repeats it at scale.
Take one automation you are considering. Imagine its most believable failure, write three causes, and run one test that would make you change your mind.
Explore Always Running: visit AlwaysRunning.ai to learn more.
