A Billion Tokens of Its Own Logs: Perplexity Open-Weights Decider v1.3 After It Fixed 14 Agent Security Rules

2026-11-13 AI Security Research by al-ice.ai Editorial

On 12 November 2026 Perplexity published research describing how its Decider models audit its own agents — and simultaneously open-weighted the two newest ones. In a one-day run, Decider read one billion tokens across 7,000 of Perplexity’s own agent sessions and identified flaws that led to 14 fixes to its agent security rules, six of which were approved and shipped within two days. Decider v1.3 and Decider Flash v1.3 are on Hugging Face under the Apache 2.0 license and served free on OpenRouter until 19 November, per OpenRouter chief executive Alex Atallah. It is the clearest production example yet of the pattern this site keeps documenting — defense scaling by reading agent telemetry at machine speed — except here the auditor itself is downloadable.

What a decision model is

A decision model does not write text. As Perplexity puts it, it is a general-purpose classifier: it takes an input plus a set of questions and returns a probability for each possible answer — yes or no, one of up to 255 options, or a level on a scored rubric with an expected score. Application code thresholds the numbers instead of parsing a chat reply, which is exactly what high-volume classification, routing, and moderation want. Version 1.3 adds locate, which pulls exact text spans out of the input with character positions, served on an Extractions route. The Hugging Face records we checked first-hand confirm the bases and license: perplexity-ai/pplx-decider-v1.3 is built on Qwen3.8-27B and pplx-decider-flash-v1.3 on Qwen3.5-4B, both tagged Apache 2.0 and created 9 November 2026. OpenRouter’s Perplexity page, also read first-hand, lists both v1.3 models as free variants at $0 per million input and output tokens, dated 12 November — with real usage already showing: about 2.17 billion tokens processed on V1.3 and 1.74 billion on Flash at the time we looked. Perplexity’s own pricing lists the hosted models at $0.02 and $0.01 per million input tokens with output tokens free.

The security loop: rules watch live, Decider audits offline, humans approve

The setup Perplexity describes is a two-speed monitoring architecture worth copying. Perplexity records agent activity and runs rule-based checks in real time; offline, Decider reads samples of those logs in both directions — where no rule fired, checking whether the rules missed something, and where an alarm was raised, checking whether it was false. Records where the rules failed either way go to an agent for triage, which proposes a fix to the rules, and a member of the security team approves each fix. Of the 14 proposed fixes from the one-day run, 6 shipped within two days. Replayed on a week of logs, the fixes cut the busiest rule’s volume by 52% while the widened rules caught about 40% more legitimate actions worth investigating — fewer noisy alarms, more real catches. At list price, the billion-token audit run costs on the order of ten to twenty dollars, which is the economic point chief executive Aravind Srinivas made on 13 November: classifiers can process billions of tokens of telemetry and surface fixes at a cost structure chat-scale judges cannot touch.

Benchmarks, and what they do not prove

Perplexity reports its own evaluations, carried here as stated: across 63 classification tasks, Decider v1.3 averaged 91.6% accuracy, 6.54 points above Perplexity’s production classifiers; across 72 retrieval tasks, mean nDCG@10 of 83.99, 1.81 points above Qwen3.8-reranker-8B; on one million Decision API requests from the API’s first 18 days, v1.3 agreed with frontier-model judges 93.98% of the time, against 93.68% for v1.2 and 93.35% for TypeSafe AI’s Jev 1.14; and when options were reordered or relabelled, v1.3 changed its answer on only 0.27% of questions versus 2.69–4.84% for Jev 1.14. Treat these as vendor-reported: agreement with a frontier judge measures agreement, not correctness, and the security-rule outcome — six shipped fixes from fourteen proposals, a 43% acceptance rate with a human in the loop — is the more honest metric. The human-approval step is load-bearing, not decorative: the classifier proposes, triage drafts, a person ships. That is the same lesson as Daybreak’s human-gated patching and the Comet audit’s hardening history on this site — Perplexity’s earlier Bumblebee scanner showed the company already thinks in tooling, and Decider moves that tooling from endpoints to the telemetry itself.

What to actually do

Split detection into live rules and offline audit. Cheap deterministic checks in real time, plus a classifier re-reading samples for misses and false alarms, beats either layer alone — and replay candidate fixes over historical logs before shipping.
Keep a human approval on every rule change. A 43% ship rate means the majority of proposals were wrong or unwanted; auto-merging auditor output into enforcement would launder the classifier’s errors into your controls.
Measure the loop, not the model. Track shipped-fix rate, alarm-volume change, and legitimate-catch change — the metrics Perplexity reported — rather than benchmark agreement alone.
Try the open weights while they are free to run. Apache-licensed classifiers with typed probabilistic outputs are directly reusable for telemetry triage; evaluate one against your own agent logs before the free OpenRouter window closes on 19 November.

Our verification was read-only. We confirmed first-hand via the Hugging Face model API that perplexity-ai/pplx-decider-v1.3 (base Qwen3.8-27B) and pplx-decider-flash-v1.3 (base Qwen3.5-4B) carry Apache 2.0 license tags and 9 November 2026 creation dates, and via OpenRouter’s Perplexity page that both v1.3 models are listed as free variants dated 12 November 2026 with typed yes/no, choice, and score outputs plus a locate Extractions route and live processed-token counters. Perplexity’s research post itself refused automated fetching (HTTP 403), so its internal claims — the 12 November date, the one-billion-token, 7,000-session, 14-fix, six-shipped-in-two-days run, the 52% volume cut and ~40% catch gain, the benchmark figures, and the locate feature description — are carried as reported via the 13 November 2026 FourWeekMBA briefing and the quoted 13 November X posts by Aravind Srinivas and Alex Atallah. Brave Search was used for discovery only. We ran no model, audited no logs, and contacted no vendor.

Sources: