Using Jev for fast structured decisions in AI agents

Agents run in a loop: an LLM decides what to do, a tool executes, the result is evaluated, and the loop repeats until the task is complete. Each decision normally requires another call to a large language model, which adds latency and cost.

Jev is a System One model released by TypeSafe AI. It does not generate text. Instead, it takes a state and a set of structured questions, evaluates them in parallel, and returns typed answers with probabilities. This makes it suitable for the classification steps that appear repeatedly in an agent loop.

How Jev works

You invoke Jev by providing a JSON payload with a model, a state (the context), and a questions object. Each question can be one of three types:

• Choice – pick from a set of options; returns a probability for each option and a confidence score.
• Score – rate the input against ordered levels (e.g. low, medium, high); returns a continuous score, the underlying distribution, and a confidence value.
• Noul – answer a yes/no question; returns the probability that the statement is true.

All questions in a single request are evaluated in parallel. Adding questions barely changes response time; the extra cost is only the tokens needed to encode the extra questions.

Example: a single Noul question about urgency in a support ticket.

{
  "model": "jev-latest",
  "state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity"
    }
  }
}

The response contains:

{
  "is_urgent": {
    "type": "noul",
    "noul": 0.999
  }
}

A value of 0.999 indicates a 99.9 % probability that the message is urgent.

Using Jev with LangChain

LangChain’s provider‑agnostic interface makes it easy to plug Jev in as a classifier. The langchain-typesafe package provides TypeSafeClassifier.

First install the package and set your API key:

pip install langchain-typesafe
export TYPESAFE_API_KEY=your_key_here

Then use it in Python:

from langchain_typesafe import Noul, TypeSafeClassifier

classifier = TypeSafeClassifier()

response = classifier.invoke({
    "state": (
        "The deploy failed twice and customers are seeing 500s. "
        "Can someone look now?"
    ),
    "questions": {
        "urgent": Noul(
            instructions="Does this need attention right now?"
        ),
    },
})

urgency = response.nouls["urgent"].noul
print(urgency)  # e.g. 0.987

The state can be plain text, structured data, or a list of LangChain messages, so you can call Jev from any point in your agent where you already have the relevant context.

Use cases

Model routing

Not every request needs the most powerful (and expensive) LLM. You can use Jev to decide which model to route a request to.

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import ModelChoice, ModelRouterMiddleware

router = ModelRouterMiddleware(
    choices={
        "fast": ModelChoice(
            model="openai:luna",
            criteria="Direct lookups, extraction, and localized changes.",
        ),
        "powerful": ModelChoice(
            model="openai:sol",
            criteria="Architecture and high‑stakes decisions.",
        ),
    },
    instructions="Choose the least costly model that can complete the task.",
)

agent = create_agent(
    model="openai:gpt-5.6-luna",
    middleware=[router],
)

The router inspects the latest user message, asks Jev which criteria apply, and selects the appropriate model for the whole run. The probabilities and confidence scores remain available in the agent state if you need them later.

Auto Mode (guardrail)

Agents can be persuaded to take unsafe actions. Jev can act as a cheap guardrail that checks tool calls before they execute.

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import AutoModeMiddleware

guardrail = AutoModeMiddleware(tools=["bash"])

agent = create_agent(
    model="openai:gpt-5.6-luna",
    middleware=[guardrail],
)

AutoModeMiddleware uses Jev to classify whether a proposed tool call is risky. If the probability exceeds a threshold, the call is blocked and the agent receives a refusal instead of executing the tool.

Getting started

The TypeSafe team notes that Jev can reduce inference latency and cost dramatically for classification work (they report up to 200× faster inference and 400× lower cost than comparable LLMs on classification tasks). Because Jev only returns structured decisions, you still need an LLM for open‑ended reasoning and generation, but you can offload the frequent classification steps to Jev to keep the agent loop responsive and inexpensive.

Early adopters have used Jev for browser‑based agents, live trading systems, and large‑scale email triage. If you want to try it, install langchain-typesafe, obtain a TypeSafe API key, and follow the examples above.

We welcome feedback on the forum, on X (@langchainai), or via LangChain issues.