Classifiers: speed and accuracy

One classifier, several questions

A request can need several answers before it is forwarded: is it a prompt injection, is it harmful, and what kind of work is it. With a decision model (TypeSafe Jev, Bespoke Nimble) all three can come from the same classifier. Janus collects the questions bound for each classifier and asks them in the way that classifier answers best.

All checks that apply to a request run at the same time, so a request waits for its slowest classifier, not for the sum of them.

Usage insights, in background or inline mode, adds its question to the guards' call when they use the same classifier about the same text and the classifier takes questions together (see Usage insights → Background or inline): the label then costs no extra call and no extra wait. Otherwise it is asked separately, before the request is forwarded (inline) or afterwards (background).

Together or one at a time

Hosted decision APIs (TypeSafe, OpenRouter) answer each question in a request independently and in parallel. Several questions in one request cost about the same time as one and give the same answers as asking separately, so Janus sends them together.

Self-hosted Nimble (llama.cpp or Ollama) reads every question in the request while answering each one, so related questions influence each other: a coding request asked alongside a language question can come back labeled as data analysis. Janus therefore sends each question in its own request, all at the same time. The server reads the prompt once per question, which costs some throughput but keeps every answer as accurate as asking alone.

Janus picks the style from the provider automatically. Administration → Usage insights → Performance lists every decision classifier with the style in use, and lets you override it; run a benchmark before and after so the change is measured, not guessed.

Which classifier for which job

Prompt injection: a decision model is a good alternative to Llama Prompt Guard 2. In our evaluation on public injection and jailbreak sets plus ordinary work requests, both Nimble and Jev caught more attacks than Prompt Guard 2 with fewer false alarms. Jev was the most accurate; Nimble is the one you can run yourself.

Harmful content: a decision model can replace Llama Guard 4. Both Jev and Nimble flagged more harmful requests than Llama Guard 4 while naming the right hazard category about as often. Llama Guard is more conservative: it rarely flags something harmless, but it misses more.

Security categories (the Categories method of content safety) need a decision model. Asking every category in one question is the fastest and works well on hosted Jev. Self-hosted Nimble and Clef judge categories more accurately when each is its own yes/no question, at one call per category, sent in parallel: pick One yes/no question per category on the check, and keep the category list short for those models. The larger Clef is more accurate than Clef-Flash and Nimble but slower.

Usage insights: both decision models label requests accurately. Write category descriptions that say what belongs and what does not; short descriptions and bare names were measurably worse.

Asking a classifier directly whether a request needs a stronger model was close to chance in our tests, so Janus does not ask it. Auto-routing instead routes by category and difficulty, which classifiers judge well.

Thresholds

A decision model answers "is this an injection?" with a probability. Its probabilities are calibrated differently from Prompt Guard's scores: at Prompt Guard's usual 0.9 a decision model lets most attacks through. Thresholds between about 0.1 and 0.3 catch most attacks with few false alarms; Janus uses 0.3 when you bind a decision model and leave the threshold empty, and suggests it in the policy editor.

Start in observe mode, review Security → Violations for a week of real traffic, then move the threshold: lower catches more and flags more ordinary requests; higher the reverse.

Content safety with a decision model counts a request as harmful when the probability of "unsafe" is at least one half; the hazard categories you select decide which of those block. On responses (direction egress or both) the decision model is asked the same question about the assistant's reply; like every response verdict it is observed and logged, never blocked.

With the Categories method each category has its own minimum confidence (0.5 unless you change it). Lower catches more and raises more false alarms. Run Test all examples on Security → Categories after changing wording or a threshold.

What a decision model reads

Prompt Guard scans every message in overlapping windows. A decision model is asked once per check, and by default reads the new turn (what the caller added since the last assistant reply; a trailing assistant prefill is skipped) and at most its last 12,000 characters. That keeps each check to one fast call, and earlier turns were judged when they were new, provided they came through Janus.

Two options on each prompt-injection and content-safety check bound to a decision model widen this. What to judge → Whole conversation also judges the earlier user, system and tool messages the caller sends, so made-up history cannot hide an attack. Long text → Whole text, in windows judges text longer than 12,000 characters in overlapping windows, one call per window in parallel, and takes the worst answer, so an attack placed before a long filler is still seen. Text longer than 16 windows (about 180,000 characters) counts as a classifier failure and the check's fail setting applies.

Both cost time and money in proportion to the text: more characters per call, or more calls per request. On per-token hosted classifiers that is a direct cost; on a self-hosted one it is GPU time, and the slowest window decides the wait. Turn them on for blocking checks that guard sensitive applications, and keep the timeout large enough for the longest text you expect. A mandatory policy's wider setting cannot be narrowed by a more specific one.

Questions from other features (Usage insights, Auto-routing) join a security check's call only when they are about the same text and may wait as long as the check does; a check never waits longer than its own timeout because another feature shares its call.

Hosting Nimble yourself

llama.cpp (a release with System One support) serves Nimble with the same accuracy as Ollama and without Ollama's limit of 26 choices per question, so a large category list is still asked in one question. Prefer it for new installations.

Cloudflare Clef and the smaller Clef-Flash (Apache 2.0) run on llama.cpp releases that include Clef support. Clef reads the whole request in one pass, so start llama-server with a physical batch at least as large as the context per slot (--batch-size and --ubatch-size); otherwise longer texts fail with "input is too large to process". In our tests Clef was as accurate as Jev on injection and work categories but is the slowest to self-host; Clef-Flash was about as accurate as Nimble and somewhat faster. Clef answers are far less affected by sibling questions than Nimble's, but Janus still asks them separately by default.

On consumer and workstation GPUs the Vulkan build of llama.cpp was faster than the ROCm build in our tests. A 4-bit quantisation was no faster than 8-bit: the work is reading the prompt, not generating, so keep the 8-bit model for its accuracy.

Give the server several parallel slots (llama.cpp --parallel, Ollama OLLAMA_NUM_PARALLEL). With a single slot, simultaneous requests wait in line and the wait grows with every concurrent user; with several, the GPU reads them side by side. Keep the model loaded permanently (Ollama keep-alive -1) so the first request after a quiet period does not pay for loading it.

Give each slot enough context for your longest classified text plus the questions; Usage insights sends only the most recent characters up to its limit, and the guards judge the newest turn of the conversation.

Benchmarking your choices

Administration → Usage insights → Performance → Benchmark measures what your classifier choices cost: pick the prompt-injection and content-safety classifiers and whether Usage insights runs inline or in the background, set the wait you will accept, and run it. Janus sends a built-in set of sample prompts through the real classifiers exactly as live traffic would, at increasing numbers of simultaneous requests.

The result shows the wait one request pays with nothing else running, the typical and 95th-percentile wait at each level, how many classifier calls each request needs, and the highest number of simultaneous requests whose 95th-percentile wait stays within your budget. Compare runs with different classifiers or grouping to see the trade-off before you change a policy.

Nothing is forwarded to a model, nothing is written to the request log, and the benchmark uses its own circuit breakers so overloading a classifier cannot trip the breaker live traffic depends on. It does share the classifier servers with live requests, so run it when traffic is light. One benchmark runs at a time.

Two decision formats

Decision models come in two wire formats. System One is spoken by TypeSafe (api.typesafe.ai), OpenRouter (/api/alpha/decisions), llama.cpp and Ollama. OpenAI Decisions (/v1/decisions) is spoken by OpenAI, for example gpt-6-luna. Janus uses either one everywhere a decision model can be used: security checks, Usage insights, Auto-routing, the Try box, benchmarks and troubleshooting captures.

Janus picks the format from the provider's address: OpenAI (api.openai.com) and Azure OpenAI speak OpenAI Decisions, everything else System One. Override it per model under Administration → Usage insights → Performance → Decision models, for a proxy or a self-hosted server that speaks the other format.

OpenAI lists its decision models as chat models. Discovery adds a separate catalog entry for each one, named with :decisions (gpt-6-luna:decisions), because the decision API is priced differently from chat. OpenRouter decision models are found automatically and marked as decision models, with their prices.

What decision calls cost

Each decision model has a rate card like any other model and, under Usage insights → Performance → Decision models, a Cost of each call setting and an optional Fee per call (USD). Automatic (the default) uses the charge the provider reports with each answer when it sends one (OpenRouter does), and the rate card otherwise. What the provider reports uses only the reported charge; The rate card always uses the card; Free records no cost, for a model you host yourself. The fee is added to every call, for example to recover hosting costs.

Typical prices: OpenAI charges for decision input only ($0.10 per million tokens for gpt-6-luna); TypeSafe Jev is $0.042 per million; OpenRouter passes on the provider price. The same figure is used whether Janus calls the model for itself (security checks, insights, routing) or a person calls it, so the two always agree: cached input is charged at the cached rate, and a charge the provider reports as zero is recorded as zero. A decision model with no rate card and no reported charge is shown as unpriced in reports, not as free.

Serving decisions to people and apps

People can call decision models through Janus with their own token, in either format, at /v1/decisions (OpenAI SDKs), /v1/systemone (TypeSafe SDK) or /api/alpha/decisions (OpenRouter SDK). Janus translates when the model speaks the other format and passes same-format requests through unchanged; the X-Janus-Decision-Translated header says when it translated. GET /v1/models lists decision models with janus.kind set to decision.

Calls are granted, rate-limited, checked by security policies, counted against quotas, priced and logged like any other request, with the kind of request shown as Decision. Security policies read everything the caller wrote: the evidence (input or state) and the question text (instructions, yes/no boundaries, choices and their descriptions, score levels, question names). A finding in text Janus can rewrite (instructions, descriptions, level labels) is redacted where the policy redacts; one in a question name or a choice value, which the answers refer to, blocks the request instead. Answers are probabilities, so egress checks do not apply to them. A block always answers HTTP 403 with the error object (policy.security_blocked), even when the policy is set to return a synthetic refusal.

Rate limits and blocking rules written for /v1/* or /v1/systemone also apply to /api/alpha/decisions, which takes the same requests. A managed model with a pool of decision models picks its member per request, like a chat pool.

A question the model's format cannot express is refused rather than changed: images to a text-only model, or System One criteria for a choice the OpenAI format has no place for. System One yes/no criteria are kept, written into the OpenAI question's instructions.

A decision model that blocks requests in a security policy cannot be granted: anyone who can call the judge can test what gets past it. That includes a model named in observe mode by a policy layered on a mandatory organization policy that blocks the same check, because the mandatory policy keeps the check blocking with that model as its judge. Existing grants stop working while it guards, the model is no longer listed in GET /v1/models, and callers are told only that it is reserved for security checks. An administrator can allow it per model.

If a provider's decision model list cannot be read during discovery, the decision models already known are kept as they are. Any model that drops out of a provider's list and comes back returns in the state it had (an enabled model stays enabled), decision and chat models alike.

A managed model (alias) can point at a decision model, with a fallback; for example jev falling back to gpt-6-luna:decisions. A chat request to a decision model, or a decision request to a chat model, is refused with a message that says which endpoint to use.

from openai import OpenAI

client = OpenAI(base_url="${JANUS_PUBLIC_URL}/v1", api_key="<your Janus token>")
result = client.decisions.create(
    model="typesafe/jev-router",  # a System One model: Janus translates
    input="I was charged twice for my subscription.",
    questions=[{"type": "predicate", "name": "urgent", "instructions": "Is this urgent?"}],
)
print(result.answers[0].probability)

Troubleshooting classifier decisions

While troubleshooting mode is capturing a request, every classifier call the request caused is captured with it: the feature that asked, the classifier, inline or background, timing, status, and, when the session captures request bodies, the exact questions sent and answers returned. A background label that finishes after the request is attached when it lands.

Open the request in Administration → Requests to read them under Classifier calls, or download the capture: classifiers.json sits next to the request and response. Captured classifier calls follow the session's encryption and retention and are deleted with their capture. Viewing them is written to the audit log.