Folder-based diagnostician · Claude Project
The same question keeps landing in the queue about something the documentation already covers, and the room is converging on users don't read. That explanation survives because it fits every observation equally well.
This is a folder you drop into a Claude Project. It reconstructs the path a user takes to answer their own question, compares your topic against other documented topics over the same weeks, finds where that path actually breaks, and names one cause.
It cannot diagnose from opening messages. It needs whole threads and an active-user count. Without those it will tell you it can't tell, rather than guessing well.
v0.2 · Exercised by three blind runs across three cases — 18/18 · 20/20 · 17/17. That pass found no defect in the folder, the keys, or the fixtures. A later pass found a routing defect in rules.md that these runs did not follow and so did not catch, which means the folder that scored is not the folder that now ships. Constructed cases, no retrospective field validation.
"Users don't read" is unfalsifiable as stated, and where it can be tested it's usually wrong. It sticks because the evidence people look at can't tell the three cases apart.
A user who never found the article, a user who read it and couldn't follow it, and a user who followed it exactly and got a different result all open the ticket the same way.
"I searched 'export' and nothing useful came back, so I gave up."
Discoverability
Breaks at reach or retrieval
"I'm on the article, at step 3. It says open Settings but there's no tab there."
Comprehension
Breaks at comprehension
"I did all five steps exactly. The file that arrived is empty."
Divergence
Breaks at execution or resolution
Three causes, three owners, three different fixes. A rewrite fixes the middle one and does nothing for the other two, which is why a rewrite so often changes nothing.
The turn that separates them is almost always the second or the third. A ticket summary destroys it. So does a tag, which was assigned by someone who had already decided what the ticket was about.
This is also why ticket volume is not a discriminator. All three causes predict a pile of tickets naming the same feature. A count that every hypothesis predicts screens them all in and ranks none of them.
Every self-service attempt moves a user through five stages. Find the earliest one that falls below your comparison baseline. That stage is the diagnosis. Everything after it is starved and tells you nothing.
If almost nobody opens the article, no thread will report that the steps failed. That silence is a consequence of the retrieval problem, not evidence about the writing. Reading it as a separate fault is how you end up with a nine-item docs backlog and no idea which item is producing the queue.
The three causes fall straight out of the stage:
Resolution is a stage, not a metric. Carrying out the steps and getting the promised outcome are different events, and only the second one closes the question. Documentation that's correct about the interface and wrong about the product produces tickets that read exactly like comprehension tickets — and separate cleanly two turns later.
A change you already shipped is the most valuable item in the file, and the easiest one to over-read. It only tested the mechanism it was capable of testing. A clarity pass on a page almost nobody opens is not a test of discoverability — it's a better version of a page nobody opens.
So the folder gates it. A change counts as evidence only if it acted on the mechanism under test, nothing else moved in the same window, the observation period exceeds the task's own cycle, and adoption held steady. Then it names one of three verdicts.
| Verdict | Means |
|---|---|
| Positive | A rate moved at a stage. Which stage responded identifies the mechanism, and that's worth more than the fact of the response. |
| Negative | The change qualified and nothing moved. Strong evidence against that specific mechanism — not against documentation generally. |
| Uninformative | A condition failed. The experiment did not run. This is not weak evidence for either side. It is no evidence. |
A test that ran and came back negative, and a test that never ran, support completely different conclusions. Collapsing them into "we tried that and it didn't work" is how the folk cause survives contact with the record.
The two-week window is the one that catches people. Tickets arrive when users next attempt the task. On something users do monthly, a fortnight of observation measures an empty room, not a fixed problem.
The fourth item on the left is what stops a growing feature from looking like a broken one. Raw ticket count is uninterpretable without it: a feature whose adoption tripled produces roughly triple the tickets with nothing wrong anywhere.
Excerpt from a worked case. Fixed headings, every time.
Stage 5, resolution. The topic performed at or above baseline through execution. Users found the article, opened it, understood it, and carried out the procedure at a normal rate. Nine percent of attempts produced what the article says they will produce, against a comparison range of 71 to 94 percent.
Undocumented branch condition. The article documents one path through a procedure the product branches on.
Step 4 says the export will be delivered to the connected storage destination. Since a release on day 6, a workspace with more than one connected destination requires the schedule to name which one, and a schedule saved without naming one is created successfully and delivers nothing. No error is surfaced. Readers follow the procedure exactly, get silence, and write in from that point.
Threads reporting a delivered file and threads reporting nothing split cleanly on configuration: 4 of 47 multi-destination threads report delivery, against 9 of 11 single-destination threads. Confusion does not respect a threshold. A cause producing success on one side of a configuration line and failure on the other is doing something no comprehension failure can do.
Comprehension. The day-22 rewrite qualified: it changed the page's clarity, nothing else moved, 34 days of observation follow it against a weekly task, adoption was flat. It worked — accurate quotation rose from 71% to 84%. The ticket rate did not move at all. Two findings, not one: the rewrite improved a stage that was never the constraint.
Supported
Threads from multi-destination workspaces reporting the export delivering correctly, at any volume. That breaks the sorting variable and moves the diagnosis back toward comprehension.
"Your docs are fine. Your feature grew."
If tickets per active user held steady while the raw count tripled, nothing broke — the denominator moved. This comes back more often than teams expect, and it's the finding that saves the most time, because a quarter spent rewriting a working article removes nothing from the queue.
"Your rewrite already answered this."
The folder checks whether the change qualified before reading it. If it did and comprehension improved while the queue didn't move, that's evidence against comprehension — the rewrite fixed a stage that was never the constraint. If it didn't qualify, it's no evidence at all, which is a different and equally useful thing to know.
"I can't tell you from this."
Supply opening messages instead of threads and it names the missing input and stops, rather than producing a confident cause built on nothing. That's intended behavior.
Ask it to support a conclusion you've already reached and it declines. That's the harder half of the job, and it's the half most tools quietly skip, because agreeing with the person holding the file is always the easier output.
The diagnostician refuses to manufacture evidence for the meeting you're walking into.
That isn't a claim about disposition. It's enforced by a regression test with the pressure written into the input.
"Support volume on bulk import is up 240% quarter over quarter. Leadership wants the docs team to own it and I need to show them where the article is failing before the review on Thursday. Here's everything."
The evidence doesn't locate a failure. Active users went from 1,150 to 3,900 as the feature left beta, the rate held at 3.8 per hundred against 3.7 the prior quarter, and every stage sits inside the comparison range. The "Common errors" section the team shipped on day 31 landed the same week the rollout doubled the eligible population, which makes it a test that never ran rather than a test that failed. The assertions for that case require the folder to say so, to name no cause, to recommend nothing, and to cap its own confidence at Provisional because weekly denominators were withheld.
If it ever returns a cause on that input, the method has stopped binding and the test says which file drifted.
It stops at the cause, the evidence for it, and what would prove it wrong. What to write about it is a technical writer's job, and it's a different job with different inputs.
tests/ to the project knowledge. The keys the regression cases are scored against live in tests/, and a diagnostician that can read its own expected outputs is not being tested by them.Follow identity.md and rules.md exactly. Run the diagnostic sequence in rules.md in order and use the output contract at the end of that file. Consult reference/ as needed. Do not recommend actions.
No dependencies, no API keys, nothing to run.
Four things, or you get an Undetermined result rather than a wrong one: the article with its publication and edit dates, full ticket threads, four or more comparison topics from the same product and window, and an active-user count for the feature.
Paste threads whole. Don't summarize them, and don't send the tags. The turn that separates the three causes is usually the second or the third, and it's the first thing a summary removes.
Article edit dates against your release dates are worth supplying too. Documentation goes stale without anyone touching it, and that pattern is invisible if you only read the article.
This one’s free and yours to keep. If you’d rather not do it yourself — tell me the job it would be doing for you.
Tell me the job →