Splunk SOAR · maintainer’s tool
The incidents nobody saw
When a Splunk SOAR playbook stops halfway, the cases it drops don’t error. They just never reach the queue. This design shows it first to the person who maintains the playbooks, inside the screens they already use.
Splunk SOAR’s playbook editor with a Runs tab added: today’s runs counted on every connection, the block where eleven stopped, and the steps they never reached.
What it is
Two surfaces Splunk SOAR already has, the case list and the playbook editor, each extended to cover what happens after a playbook ships. The structure is Splunk SOAR’s; the visual treatment and the additions are mine.
Where it comes from
The failure, the debugging loop and the constraints are from my own work maintaining these playbooks. The screens are designed.
Project
Self-initiated concept · Desktop web · Product design · Aug–Sep 2026
What’s at stake
Cases that silently miss the queue have no owner and no SLA, the clock on how fast a response is due, and nobody sees them until someone searches by hand.
In Splunk SOAR, a playbook is an automated workflow that runs on incoming events. Here it also assigned each event into the response queue, and that assignment is what made it work someone owned.
The automation that should have put them in the response queue had stopped. The first anyone heard was a colleague asking why they couldn’t work an incident.
I spent four and a half years building and maintaining the automation layer in a bank’s security operations. About seventy percent of that time went on fixing playbooks that already existed. The product is built almost entirely for the other thirty. None of this is specific to security. Any automation that reports on runs rather than results has the same gap, and AI agents are the newest example.
Tines lets you trace individual story runs and, like Torq, tracks case SLAs. Cortex XSOAR records playbook task errors. In the documentation I reviewed (Oct 2026), I didn’t find a built-in check of each case against a declared minimum of expected automation.
Fixing one playbook crosses six surfaces.
1Someone tells you
2The case view
As it ships · case view crop3The playbooks list
4The playbook editor
5A debug run
6Pre-production → QA → production
Someone tells you
A colleague asks why they can’t work an incident. You carry the case ID.
Dashed tiles are sketches, not screenshots.
Someone tells you
A colleague asks why they can’t work an incident. You carry the case ID.
The case view
As it ships · case view cropWhich playbooks ran, with a red cross on the one that errored. You carry the playbook.
The playbooks list
A lookup screen to find the thing to edit. You carry the block.
The playbook editor
It shows what a playbook is, never what it did. You carry the change.
A debug run
Switch the playbook off, fire real actions, run as yourself. You carry the fix.
Pre-production → QA → production
Mandatory approval, and a rollback needs the engineering team.
The case list and case view
The list notices. The case view keeps the consequence with the failure.
The playbook editor
The run evidence stays with the replay and the fix.
QA stays separate
Pre-production → QA → production. Approval is still mandatory.
The context stays with the case and the editor. QA still gates production.
The product treats them as unrelated screens, so the person doing the job carries the context between each one.
From a stalled case to a fix ready for QA.
The case list shows incidents, owners and ages. It does not show where automation stopped.
As Splunk SOAR ships it
The case list shows incidents, owners and ages. It does not show where automation stopped.
The failure and its consequence were on different screens, so nobody owned noticing.
One column added, Automation, and rows grouped by where it stopped. Created and Opened, aged in colour, separate a stalled case from a new one. From here on, an outline marks what I added.
The unit of failure is cases, not runs.
The error opens where it happened, and the consequence is stated across the top: enrich_artifact failed, so nothing after it ran, including the two steps this label declares as its minimum. The feed shows the step that failed and, below it, the two that never ran, which is where the harm is.
“It ran” and “it worked” are different questions.
The design reports against a declared minimum, assignment and workbooks, that every case under this label must get. So the case gets an Expected group: 1 of 4 expected, 0 of 2 expected, not started.
Where eleven stopped
The editor's graph carrying the last twelve hours: 251 events into enrich_artifact, 240 out, and the three steps the other eleven never reached.
Diagnosing the failure made more of it.
Debugging a live playbook meant switching it off, so investigating made more stopped cases. Here the fix is written in pre-production and replayed as an analyst, with the playbook still active.
Run as the analyst who hit it. Today the choice is you or the automation user.
Outbound actions: Will not fire
Before it goes to QA, the change shows what it would affect: three child playbooks and about 250 events a day.
Waiting on QA · lived.
The fix is written and tested, and eleven events are still stopped. Nothing ships without QA approval, and rollback needs another team.
QA still gates production.
What I rejected
A new dashboard. The responders’ dashboards are filled by the automation, so when it fails they show less work, not a fault. The check belongs on the case list, which every event reaches either way.
Writing playbooks from plain English. It would help with the smaller part of the job. The design spends its effort after a playbook ships.
Drawing the steps that never ran. With branches, what would have run depends on data that never arrived, so a minimum is declared instead.
The same screens on a normal day, and when the fix is blocked.
Nothing stopped · designed. The same list on a normal day. Three events that arrived minutes ago say so, and the completed work sits open underneath. A monitoring surface should be almost invisible most of the time.
Replay refused · lived, indirectly. The analyst’s role can’t reach the directory the enrichment calls. My admin account could, which is exactly why debugging as myself never reproduced it. The refusal is the finding, not a fault.
Switched off by someone else · lived, one extension. Eighteen events waiting and nothing running, because the playbook was deactivated forty-two minutes ago and nobody switched it back. I did this to myself while debugging; a colleague doing it is the extension.
Next case study