Why You Shouldn't Use LLMs to Write Your Team's Incident Post-Mortems (And the Ethical Way to Document Outages)

The Temptation of the Instant Incident Report

It is 3:00 AM. Production database CPU usage spiked to 100%, the API gateway started throwing 502 errors, and your primary customer database was locked out for forty-five minutes. After a frantic Slack call, three database restarts, and a rolled-back deployment, everything is stable.

Now comes the part every engineer dreads: writing the incident post-mortem.

You have a Slack channel full of panicked messages, a raw log from Grafana, and a template to fill out. The temptation to select all, paste it into GPT-4o, and prompt: "Generate an incident post-mortem from these logs and Slack messages" is almost overwhelming.

It seems harmless. After all, the incident is over, the bug is patched, and you just need to appease management with a formal report. But outsourcing the analysis of a system failure to an LLM is a profound mistake. It damages your engineering culture, bypasses critical team learning, and raises serious ethical questions about accountability.

Why LLMs are Horrible Post-Mortem Authors

Incident post-mortems are not just administrative paperwork; they are historical records of how your socio-technical systems fail. When you let an LLM write these documents, you introduce several structural failures into your team's engineering practice.

1. The Danger of "Sanitisation" LLMs are trained to be polite, helpful, and smooth. They excel at writing clean narratives with clear, linear progressions. But real incidents are messy, chaotic, and non-linear.

When an LLM parses an incident chat, it tends to sanitise the human confusion. It glosses over the fact that two senior developers were looking at entirely different dashboards for twenty minutes, or that the runbook was outdated. By smoothing out these rough edges, the LLM hides the exact friction points that your organisation needs to address to prevent the next outage.

2. Chronology and Attribution Hallucinations Parsing unstructured chat logs is incredibly difficult for transformer models. During a high-stress incident, developers speak in fragments, make assumptions, and pivot rapidly between hypotheses.

If you ask an LLM to build a timeline from these logs, it will frequently attribute actions to the wrong people, misinterpret the sequence of events, or invent technical links that do not exist. If you publish a report with a hallucinated timeline, you risk accidentally assigning blame to an engineer who was actually trying to fix the problem.

For advice on identifying and mitigating these kinds of model errors, check out our platform-specific troubleshooting guides at /platforms/openai/articles and /platforms/claude/articles.

3. The Death of Blameless Culture A healthy post-mortem culture relies on psychological safety. Teams must feel safe enough to admit: *"I ran this script because I thought the replica was passive, but it was actually active."*

If your developers know that their raw, honest Slack messages are going to be fed into a third-party LLM to generate a summary for executive leadership, they will stop talking openly in public channels during incidents. They will move to private calls and DMs to protect themselves, destroying the transparent communication required to debug complex distributed systems.

The Ethical Duty of Accountability

When a service fails, customers lose money, users are frustrated, and team members burn out. Writing a post-mortem is an act of taking responsibility.

If you present a post-mortem that was generated by an automated system, you are essentially saying, "We care so little about this failure that we didn't even want to spend the human cognitive effort to reflect on why it happened." It is an evasion of engineering duty.

Reflective thinking is where the learning happens. The physical act of writing down the timeline, grappling with why you made a specific decision, and explaining it to others is what prevents you from making the same mistake next quarter. You cannot outsource learning.

How to Ethically Use AI in the Post-Mortem Process

This does not mean you must ban AI from your post-incident workflow entirely. There are ways to leverage LLMs to assist you without outsourcing your critical thinking. You can use these tools to handle data processing, leaving the analysis and narrative to humans.

| Task | How to Handle It Ethically with AI | What to Avoid | | :--- | :--- | :--- | | Timeline Generation | Use an LLM to extract timestamped events from raw logs to build a draft chronological table, which you then manually verify. | Accepting the generated timeline without cross-referencing log files. | | Log Analysis | Ask an LLM to find anomalies or parsing patterns in a specific, anonymised error log. Refer to our /glossary for sanitisation best practices. | Uploading raw user data or PII to a public model during debugging. | | Action Item Refining | Take human-written action items and ask an LLM to format them as SMART goals. | Letting the LLM decide what the remediation steps should be. |

A Better Prompt for Your Post-Mortem Workflow

If you want to use Claude or GPT-4o to help you organize your thoughts, feed it your rough human-written notes and ask it to look for cognitive bias, rather than writing the narrative for you:

`markdown I am writing an incident post-mortem based on my notes below. Do not write the post-mortem for me. Instead, read my notes and identify: 1. Any instances where we seem to be blaming individuals rather than system designs. 2. Any logical leaps where we assumed something was true without verifying it in the logs. 3. Gaps in our timeline where we have missing context. `

Reflection Cannot Be Automated

Your post-mortems are a reflection of your team's values. A beautifully formatted, AI-generated PDF that sits in a Confluence folder might look professional, but it does nothing to make your systems more resilient. Keep the human in the loop, embrace the messy truth of your failures, and write your own post-mortems. Your systems—and your team's culture—will thank you for it.