Auto-routing

What it does

A router is a model name callers use, like auto/legal or auto/smart. For each request, a decision model answers which category of work it is (from your routing categories) and, when a route needs it, how difficult it is on a 1–5 scale. The router then tries its routes top to bottom and sends the request to the first one that fits. Anything else goes to the router’s default model.

Routing never fails a request. If the classifier is unsure, slow, busy or down, no route fits, or the chosen route cannot serve (its model was removed, or every server in its pool is out of service), the default serves it and the request log says why. For someone covered by Child Safety, a route whose model their age range blocks is skipped, like a route they aren’t granted. If the default is blocked, the first route they may use serves; if every model is blocked, they’re told the model isn’t available. Grants, quotas and security policies still apply to the model that serves, as below.

Typical setups: keep legal and other sensitive specialist work on a self-hosted or fine-tuned model and send everything else to a frontier model; send easy everyday work to a cheaper model; or use one frontier model for everything and let Janus raise or lower how hard it thinks.

New router offers four templates — Cost saver (auto/cost-saver), Keep specialist work on-prem (auto/legal), Effort scaler (auto/smart) and Coding by difficulty (auto/code) — and a Blank router. A template fills in categories, difficulty, thinking and instructions; you choose the models. New routers start as drafts.

Routers are managed models

A router is a managed model of kind router, so the rest of Janus treats it like an alias. Callers need a grant on the router (Grants). Security policies bound to the router apply to every request sent through it, and policies bound to the model a route picks apply too. A quota on the router counts everything sent to it, whichever model served; a quota on a route’s model (or on an alias the route serves through) still applies and counts the traffic the route sends it. By default a caller over that quota is refused; tick “Skip this route when the caller is over a quota on its model” and the next route or the default serves instead. A caller already over a quota on the router is refused without a classifier call.

A route can be limited to callers who are also granted its model (“only for callers also granted this model”), for example to keep an expensive model to one team while everyone else gets the default. When the router’s configuration cannot be read, the default serves and the policies bound to the default model and its upstream apply.

When the default or a route is an alias or a pool, the router uses that alias as it is at the moment of each request — its current target, its fallback and its pool members — so repointing the alias needs no change to the router. The router is shown with a problem, and leaves the model list and the Chat picker, only when the alias itself cannot serve.

The request log shows the router name the caller asked for, the model that served, and the reason (matched a route, same conversation, no route fit, classifier unsure, classifier too slow, …). With “Tell callers which route served” on, responses carry X-Janus-Route, X-Janus-Route-Reason and X-Janus-Route-Model (and X-Janus-Route-Instructions when instructions were added).

Draft, shadow and live

Draft: not callable. Use the builder’s Test this router panel to see where prompts would go.

Shadow: callable, but always served by the default. Janus classifies in the background (from the text the model receives, after any redaction) and records which route would have served, so the route mix and the classifier’s speed are visible before anything changes for callers. No added latency.

Live: requests are routed. The decision waits at most the router’s time budget (800 ms by default); past it the default serves.

Routing categories

Routing categories are what the classifier chooses between. Fifteen are built in, including Legal, which is kept apart from Specialist so legal work can go to its own model; you can add your own. Each has a name, what it includes, what it does not include, a minimum confidence, and test examples. The descriptions are what the classifier reads: say what belongs, and name the neighbors that do not and where they go. That is what stops misfiling; bare names were measurably worse in testing.

Every enabled category is offered on every request, plus “none of these”. Offering only the categories a router uses pulls borderline requests into them; offering the near neighbors lets the classifier say “this is planning advice, not research”.

At most 25 routing categories can be enabled at once. A category a router uses cannot be disabled or deleted until it is removed from that router’s routes. Built-in categories can be disabled or reset to their shipped wording, not deleted.

A routing category can follow a Usage insights category instead of having its own description, so one definition serves both.

Below a category’s minimum confidence, the answer does not count: the request is not routed by that category.

The classifier

Any decision model can classify (Administration → Models, role “decision”). In testing, hosted Jev answered in a few hundred milliseconds and was as accurate as the larger self-hosted Clef; Clef-Flash and Nimble are faster self-hosted choices that misfile more borderline requests. A hosted classifier reads every prompt sent to a router, including ones you route to self-hosted models: use a self-hosted classifier if that matters. The builder warns when this is the case.

When a security policy already calls the same decision model and it takes questions together, the routing questions join that call: no extra call, no extra wait. Otherwise they get a call of their own. That call starts in parallel with the Security Gateway checks only when no ingress check can block or redact the message and the caller is not covered by Child Safety; if not, the router asks after those checks pass, about the text that is actually forwarded, so a blocked message never reaches the routing classifier. A shadow router also decides from the forwarded (redacted) text.

Each routing decision call is recorded as a classification usage event with client janus-auto-routing and the caller’s request id; it counts as Auto-routing cost, not Security Gateway cost. When a routing question rides along on a Security Gateway call, that one call is recorded as Security Gateway cost. A request’s classifier calls and wait include the checks added by the model the router chose.

Classifier → Try routing classifies a prompt with any classifier (saved or not) and shows where every router would send it. The batch test runs every category’s test examples and lists the misfiles.

The routing classifier reads the latest message with the recent conversation as context, so a short follow-up is routed by the task it belongs to: “Keep going” on a hard coding task stays hard coding, and “What about Ohio?” after a legal question stays legal. The last six earlier messages are included (user messages cut to their first 1,000 characters and assistant replies to their first 300, at most 4,000 characters in all, newest kept) with the latest message marked as the one to judge, which keeps a real change of topic (“Unrelated quick one: …”) from being dragged to the earlier subject. A first message is classified on its own, exactly as before, and can share a Security Gateway guard’s call; a follow-up is asked in a call of its own, in parallel with the guards, because the guards still judge the new turn alone. Try routing takes an optional earlier conversation to test follow-ups.

On a live router, conversations stick to the route of their first decided turn for 30 minutes by default (1 minute to 24 hours; each turn starts the window again), so the model does not change mid-task: switching models makes the new one re-read the whole conversation, and the old one’s cached context is thrown away. Only the model sticks. When the route’s thinking scales with difficulty, each turn’s thinking still follows that turn: Janus asks the classifier about difficulty only (with the conversation as context) and sets the level, so an easy step of a hard task does not pay for high thinking, and a hard turn in an easy conversation gets it. If that question fails or times out, the level from the last judged turn is used. A sticky turn with fixed or passthrough thinking makes no classifier call. Either way the route must still fit the turn: a conversation that outgrows the route model’s context window, loses the grant a restricted route needs, or goes over a quota the route skips on is decided afresh. Revoking a grant therefore takes effect on the next turn.

Janus recognizes a conversation by an X-Janus-Session header (or X-Session-Id, X-Conversation-Id or X-Correlation-Id), a prompt_cache_key or session_id in the body, or otherwise by its opening (tools, system prompt and first user message), always per caller. A turn the default served because the classifier was too slow, failed or was busy is not remembered, so the next turn is decided afresh.

Difficulty and thinking

A route can require a difficulty band: Easy (1–2), Medium (3), Hard (4–5), Expert (5), or a custom range. The classifier’s rating is a weighted average, so 2.5 sits between easy and moderate.

Each route and the default set thinking: As requested (the caller’s setting passes through), a fixed level (none, low, medium, high, …), or Scale with difficulty (low, medium or high from the rating). Janus fits the level to what the target model supports, as it does for any request. OpenAI-compatible upstreams (including Vertex) get reasoning_effort; Anthropic gets the thinking form the Claude model takes (adaptive thinking with that effort on Claude 4.6 and later, a thinking budget before), see Thinking settings and provider limits. Bedrock’s Converse format has no thinking setting Janus can send, so on Bedrock a route’s thinking level is not applied and the request is sent as the caller made it.

Instructions

A router can add text to the system prompt of the requests it serves. Router instructions are added to every request the router serves, whichever route serves it; each route, and the default, can add its own instructions after them. For example, a router says “Answer in British English.” for everything, and its Legal route adds “End every answer with a short note that this is general information, not legal advice.” Each field is plain text of up to 8,000 characters; surrounding blank space is trimmed.

The router text and the route text are joined with a blank line and added before the caller’s own system prompt: on chat completions, at the start of the first system or developer message (or as a new first text part when that message is a list of parts, or as a new system message when there is none); on Anthropic messages, at the start of system (or as a first text block, or as system when there is none); on responses, at the start of instructions. Completions, embeddings and other endpoints have no system prompt, so instructions are not applied there, and the request log says so.

Instructions are sent to the model and counted in its input tokens, so they cost what any prompt text costs. Callers cannot see or remove them, and they are not returned in responses. They are added on every turn, sticky turns included, because callers resend the conversation without them.

The Security Gateway and the routing classifier read only the caller’s text: instructions are added after both have run, and they are not scanned (an administrator wrote them). The text Janus classifies is unchanged.

A shadow router always serves its default, so its requests get the router instructions and the default’s instructions, never a route’s: shadow mode is observably the default plus those instructions. When a route’s model cannot serve and the default takes over, the default’s instructions apply instead of the route’s; instructions are added once, to the request that is finally sent.

Evaluation compares like with like: the routed answer was produced with the route’s instructions, and the default’s replay carries the router instructions and the default’s.

The request log detail for a routed request shows “Router instructions added: router”, “route” or “router + route”. With “Tell callers which route served” on, responses also carry X-Janus-Route-Instructions with the same value.

Evaluating a route

Turn on “Evaluate against the default” on a route to find out whether it is as good as the default for the requests it gets. For a sample of the route’s requests (5% by default), after the caller has their answer, Janus asks the router’s default model the same question, exactly as the default would have been asked (the caller’s own request with the default’s thinking setting, not the route’s), then a judge model compares the two answers without knowing which is which, twice with the order swapped. If the two verdicts disagree, the comparison counts as a tie.

The copies sent to the default and to the judge go through the Security Gateway policies that would apply to the caller sending them there (policies on the router, the model, its provider, the caller and their groups). If a policy would block or redact a copy, the sample is skipped and counted as “not evaluated”; nothing is sent. A route that keeps work on a self-hosted model and is evaluated against an externally hosted default or judge still sends a sample of those prompts off site, so the builder and the Evaluations tab warn about it.

The Evaluations tab shows, per route: comparisons, better / tie / worse, “at least as good” (wins plus half of ties) with a 95% range, the judge’s consistency, and cost against the default. After at least 10 comparisons, a route is marked Losing when even the top of the range is below 50%, Keep when the bottom of the range is above 45% (a near-tie that saves money is worth keeping), and Needs more data otherwise.

A route’s tally covers only comparisons between the models it uses now: its model, the router’s default and the judge. Change any of them and the tally starts again; earlier comparisons stay listed as history under the card and never count towards the new verdict.

Only the verdict, scores, the judge’s one-line reason, cost and timing are stored, never the prompt or either answer. Evaluation covers chat completions without tools or images, up to 64 KB of request; a streamed answer too long to keep in full is not judged. It stops at the end date (14 days by default), at its daily count and daily budget, and pauses while the gateway is busy. The count and budget are reserved before anything is spent, at a deliberately high estimate, so a burst or several replicas cannot overshoot them. Evaluations at once per replica, the busy threshold and the default answer’s token cap are set on the Evaluations tab. Evaluation calls are platform overhead: they never count against the caller’s quota and are hidden from the request log. Requests from people covered by Child Safety are never sampled for evaluation.

Auto-routing → Classifier → Advanced → Keep decisions (days) (90 by default; 0 keeps them) deletes decisions and evaluation verdicts older than that in the daily retention job. Remembered conversations are deleted once they expire. A decision kept longer than usage events (JANUS_USAGE_RETENTION_DAYS) no longer names the caller: its user and service token are cleared when the matching usage events are purged.

Pick a judge that is neither the route’s model nor the default: judges favor their own answers. The judge must be an enabled model that is not a classifier; a disabled default or judge is not called.