Security gateway
Parallel classifier checks
Model-backed checks run at the same time, so a request waits for its slowest guard rather than all of them in turn.
What it is
Prompt-injection, content-safety and inline Usage insights checks that need a classifier now run at the same time. Questions bound for the same decision model are asked together when the model answers them independently, and one question per request in parallel when it does not.
Why you want it
Each guard adds latency before a request is forwarded. Running them side by side, and sharing one call where the model allows it, keeps the added wait close to that of a single check.
How it works
- Style chosen automatically per provider (combined for hosted decision APIs, separate for self-hosted Nimble), overridable per classifier
- Auto-routing and inline Usage insights questions join a security check's call when they can
- A check never waits past its own timeout because another feature shares its call
- The request log records classifier wait and the number of classifier calls per request
See it on your own network.
The Community edition is free for up to 25 people. The 30-day Business trial unlocks every Business feature.