Anthropic’s ‘Dangerous’ Fable Is Back! How Does It Do?

Fable 5's Self-Reported Benchmark Gap Is the Real Story, Not the "Dangerous" Branding

Artificial Lawyer published a hands-on look at Anthropic's Fable 5 model on 2 July 2026, the latest release in the vendor's "Mythos" line, which Anthropic markets with a dual-use safety warning and routes certain sensitive capabilities to a fallback model, Opus 4.8. AL asked Fable 5 to grade itself against legal benchmarks and reported the model's own figures: about 90% of individual criteria answered correctly, but only around 11% of complete legal work products scored as fully correct. In the same session, AL asked the model to draft the "world's most concise" mutual NDA under English law within a 150-word limit; it returned roughly 120 words. AL was explicit that none of this was independently verified.

What does the 90%-versus-11% gap actually measure?

The gap measures the distance between getting most details right and producing a document a court or client can rely on end to end, per Fable 5's own scoring.

Ninety percent correctness on individual criteria sounds strong until you notice that only 11% of complete deliverables cleared the bar — meaning most outputs had at least one wrong element buried inside an otherwise fluent answer. For a research memo that's an inconvenience. For an issue-coded litigation record, a chronology entry, or a privilege call, a single wrong fact inside a fluent-looking answer is the failure mode that actually costs money: it's the error nobody catches because everything around it reads correctly.

Can you trust a model's benchmark of itself?

No — Artificial Lawyer said as much, noting the figures were AI-generated and unverified, and buyers should treat any vendor-reported accuracy number the same way.

Nobody outside Anthropic ran this benchmark, defined its criteria, or audited its scoring. That's not unique to Fable 5 — it's the default state of most model-quality claims circulating in legal AI right now. The question worth asking your own vendor isn't "what's your accuracy rate" but who defined the metric, who scored it, and whether you can see the failure cases, not just the headline number. A platform that codes and cites every document against your own record, rather than reporting a percentage, gives you something you can actually check.

Does this change the calculus on Claude for Legal displacing legal tech vendors?

Not for workflow-heavy tools — Fable 5 itself argued that thin prompt-and-UI wrappers are exposed, while integration and deployment depth are not easily replicated.

That's a notable admission from the model itself, discussing what it called "workflow depth" — DMS integration, matter management, precedent plumbing — as the layer a frontier model sitting in a chat window doesn't replace. For evidence platforms specifically, the moat isn't drafting fluency; it's defensibility infrastructure: chain of custody, Bates production, single-tenant deployment inside a firm's own cloud account so privileged material never trains a model. None of that shows up in a benchmark about criteria-matching.

Frequently asked questions

Is Fable 5 actually restricted for legal use?

Anthropic gates certain dual-use capabilities — described in the interview as cyber, bio and chemistry-related — and routes those queries to Opus 4.8 instead. General legal drafting and review sit outside that restriction.

Should e-discovery buyers care about the NDA drafting test?

Only as a party trick. Fitting a mutual NDA into roughly 120 words against a 150-word limit shows constraint-following, not that the clauses are correct or enforceable under English law — AL didn't verify that either.

What should I ask my own AI vendor after reading this?

Ask for the gap between per-criterion accuracy and complete-document accuracy on your own document types, who scored it, and whether every answer traces back to a source exhibit you can check yourself.

Source: Artificial Lawyer.