Performance
Detection
Against JailbreakBench, SusFactor blocks roughly 88% of attacks at a 9% benign false-positive rate (threshold 0.5). This is the operating point for the current SusFactor model; tune the threshold per path to trade detection against your own false-positive tolerance. This false-positive rate is lower than comparable public detectors evaluated the same way: WildGuard sits at 45%, Mistral at 39%, and Llama Guard 3 at 23%, per our launch announcement. It also performs competitively against generative guardrail models 7-8B parameters in size, more than 10x SusFactor's own 560M.
These figures describe SusFactor's binary safe/suspicious classification; there is no sub-category breakdown (e.g. "role-play attack" vs. "encoding attack") in the score itself. See our blog for detailed evaluation results for SusFactor.
Latency
SusFactor is low-latency: self-hosted deployments score a prompt in roughly sub-50ms on recommended CPU hardware under specified conditions (actual latency varies by prompt length and hardware).
- Self-hosted (in-process): inference runs in the same process as your request handler, with no network call. This is the fastest path and the default for self-hosted/commercial deployments.
- Early-access beta: the hosted API currently runs inference on a remote backend rather than in-process, trading a small amount of latency for zero infrastructure to manage. Latency on the early-access beta is not a fixed SLA yet.
Treat these figures as directional; full evaluation methodology will follow in a companion research paper.
Choosing Self-Hosted vs. Hosted
| Self-hosted | Early-access beta | |
|---|---|---|
| Latency | Lowest (in-process) | Slightly higher (remote inference) |
| Infrastructure | You run it yourself | None; 0DIN runs it |
| Data locality | Prompts never leave your environment | Prompts are sent to 0DIN's hosted endpoint |
| Best for | Production, latency-sensitive paths | Evaluating fit before committing to self-hosting |