Using Jev for fast structured decisions in AI agents

Agents run in a loop: an LLM decides what to do, a tool executes, the result is evaluated, and the loop repeats until the task is complete. Each decision normally requires another call to a large language model, which adds latency and cost.

Jev is a System One model released by TypeSafe AI. It does not generate text. Instead, it takes a state and a set of structured questions, evaluates them in parallel, and returns typed answers with probabilities. This makes it suitable for the classification steps that appear repeatedly in an agent loop.

How Jev works

You invoke Jev by providing a JSON payload with a model, a state (the context), and a questions object. Each question can be one of three types:

• Choice – pick from a set of options; returns a probability for each option and a confidence score.
• Score – rate the input against ordered levels (e.g. low, medium, high); returns a continuous score, the underlying distribution, and a confidence value.
• Noul – answer a yes/no question; returns the probability that the statement is true.

All questions in a single request are evaluated in parallel. Adding questions barely changes response time; the extra cost is only the tokens needed to encode the extra questions.

Example: a single Noul question about urgency in a support ticket.

{
  "model": "jev-latest",
  "state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity"
    }
  }
}

The response contains:

{
  "is_urgent": {
    "type": "noul",
    "noul": 0.999
  }
}

A value of 0.999 indicates a 99.9 % probability that the message is urgent.

Using Jev in code

TypeSafe provides a small client library that wraps the HTTP API. Install it with:

pip install typesafe
export TYPESAFE_API_KEY=your_key_here

Basic call

from typesafe import JevClient, Noul

client = JevClient()  # reads TYPESAFE_API_KEY from environment

response = client.invoke(
    model="jev-latest",
    state=(
        "The deploy failed twice and customers are seeing 500s. "
        "Can someone look now?"
    ),
    questions={
        "urgent": Noul(
            instructions="Does this need attention right now?"
        ),
    },
)

urgency = response.nouls["urgent"].noul
print(urgency)  # e.g. 0.987

The state argument can be plain text, a JSON‑serialisable object, or a list of messages; the client will serialize it for you.

Multiple question types

from typesafe import JevClient, Noul, Choice, Score

client = JevClient()

response = client.invoke(
    model="jev-latest",
    state="User wants to export data as CSV and email it to themselves.",
    questions={
        "needs_export": Noul(
            instructions="The user is asking for an export of their data."
        ),
        "format": Choice(
            options=["CSV", "JSON", "Excel"],
            instructions="Which file format does the user prefer?"
        ),
        "complexity": Score(
            levels=["low", "medium", "high"],
            instructions="How complex is the requested operation?"
        ),
    },
)

print(response.nouls["needs_export"].noul)      # probability of export request
print(response.choices["format"].probs)        # dict of option → probability
print(response.scores["complexity"].value)     # continuous score 0‑1

All question types are evaluated in parallel; adding more questions adds almost no latency.

Use cases

Model routing

Not every request needs the most powerful (and expensive) LLM. You can use Jev to decide which model to route a request to.

from typesafe import JevClient, Noul

client = JevClient()

def choose_model(user_message: str) -> str:
    resp = client.invoke(
        model="jev-latest",
        state=user_message,
        questions={
            "is_simple": Noul(
                instructions="The request is a direct lookup or a minor edit."
            )
        },
    )
    if resp.nouls["is_simple"].noul > 0.8:
        return "fast-model"      # e.g. a smaller, cheaper LLM
    else:
        return "powerful-model"  # e.g. a larger, more capable LLM

# Example usage
selected = choose_model("What is the current price of Bitcoin?")
# selected -> "fast-model"

The function returns a model identifier that your agent can then use for the actual generation step. The probabilities and confidence scores are available if you need to log or adjust thresholds later.

Auto Mode (guardrail)

Agents can be persuaded to take unsafe actions. Jev can act as a cheap guardrail that checks tool calls before they execute.

from typesafe import JevClient, Noul

client = JevClient()
RISKY_TOOLS = {"bash", "sudo", "rm"}

def safe_tool_call(tool_name: str, tool_input: str) -> bool:
    """Return True if the call is allowed, False if it should be blocked."""
    if tool_name not in RISKY_TOOLS:
        return True

    resp = client.invoke(
        model="jev-latest",
        state=tool_input,
        questions={
            "is_safe": Noul(
                instructions="Does this input look safe to execute?"
            ),
        },
    )
    # Allow the call only if the model is confident it is safe (<10 % risk)
    return resp.nouls["is_safe"].noul < 0.1

# Example integration in an agent loop
def agent_step(user_input):
    # ... reasoning step that proposes a tool call ...
    proposed_tool = "bash"
    proposed_input = "rm -rf /tmp/some-dir"
    if safe_tool_call(proposed_tool, proposed_input):
        execute_tool(proposed_tool, proposed_input)
    else:
        log("Blocked risky tool call")
        return "I cannot run that command."

The guardrail adds only a single Jev invocation per tool proposal, keeping latency low while providing a programmable safety check.

Getting started

The TypeSafe team reports that Jev can reduce inference latency and cost dramatically for classification work (they cite up to 200× faster inference and 400× lower cost than comparable LLMs on classification tasks). Because Jev only returns structured decisions, you still need an LLM for open‑ended reasoning and generation, but you can offload the frequent classification steps to Jev to keep the agent loop responsive and inexpensive.

Early adopters have used Jev for browser‑based agents, live trading systems, and large‑scale email triage. To try it, install the typesafe client, obtain a TypeSafe API key, and follow the examples above.

We welcome feedback on the forum, on X (@typesafeai), or via the TypeSafe issue tracker.