Jev for AI Agents: Routing Between Agents

The hard part of a multi-agent system is rarely the agents. It is the traffic control: deciding which agent takes a given step, and knowing when the system should not decide at all. That layer is what Jev is built for, and it is where general LLMs quietly become the slowest and most expensive part of the stack.

Routing is a decision, not a generation

A router answers one question: given this request and these available agents, which one should handle it? The answer space is closed and known. There is nothing to generate — no prose, no explanation, no chain of thought. Just a choice, a distribution, and a confidence value.

Teams that use an LLM here pay for generation they do not need and inherit the failure modes that come with it: JSON mode drift, schema violations, the occasional hallucinated agent name, and no trustworthy confidence to gate on. Jev returns a typed choice with probabilities and confidence by contract, so your code can branch on it directly.

Writing the router as a Choice question

Agents become criteria, each with a short description of what it handles. Those descriptions do real work. They define the boundary between agents, so overlapping ones are the main source of misroutes.

const AGENTS = {
	kubernetes: "Cluster, pods, deployments, CrashLoopBackOff, resource limits",
	coding: "Writing or refactoring application code, tests, code review",
	research: "Finding and summarizing external information, docs, papers",
	general: "Anything that does not clearly fit another agent",
} as const;

const response = await fetch("https://api.typesafe.ai/v1/systemone", {
	method: "POST",
	headers: {
		Authorization: `Bearer ${process.env.TYPESAFE_API_KEY}`,
		"Content-Type": "application/json",
	},
	body: JSON.stringify({
		state: `User request: Debug my Kubernetes deployment. Pods are stuck in CrashLoopBackOff.`,
		model: "jev-latest",
		questions: {
			agent: {
				type: "choice",
				instructions: "Which agent should handle this request",
				criteria: AGENTS,
			},
		},
	}),
});

const { choice, confidence, probabilities } = (await response.json()).answers.agent;

A note on general as a catch-all: it keeps the distribution honest. Without an escape option, the model is forced to pick the least-bad specialist even when nothing fits, and you end up reading a flat distribution as uncertainty when it was really a forced choice.

Gating dispatch on confidence

TypeSafe's confidence documentation is explicit that confidence comes from the shape of the probability distribution: concentrated on one option means confident, spread out means unsure. It also recommends the pattern this router needs, with three ranges and thresholds that scale with risk.

async function route(state: string) {
	const { choice, confidence } = await classify(state);

	if (confidence < 0.5) {
		// Model is genuinely unsure — don't guess.
		return { action: "escalate" as const };
	}

	if (confidence >= 0.85) {
		return { action: "dispatch" as const, agent: choice };
	}

	// Moderate confidence: verify before committing.
	return { action: "confirm" as const, agent: choice };
}

The stakes decide the numbers. Dispatching a read-only research agent is cheap to get wrong. Dispatching one that can deploy infrastructure is not. In TypeSafe's own example, a low-stakes action proceeds at moderate confidence while a destructive one needs 0.9 or better and explicit confirmation. Encode that asymmetry in your thresholds instead of using one number everywhere.

Route many, run in parallel

Every question in a Jev request is evaluated against the same state in parallel and in isolation, so you can ask several routing questions at once without paying a latency penalty per question. TypeSafe documents this as speculative fan-out: one request asks for the primary agent, a fallback agent, and whether the request is urgent, and you use whichever answers you need.

{
	"questions": {
		"agent": {
			"type": "choice",
			"instructions": "Which agent should handle this",
			"criteria": { "...": "..." }
		},
		"fallback": {
			"type": "choice",
			"instructions": "If the primary agent is unavailable, which should handle this",
			"criteria": { "...": "..." }
		},
		"is_urgent": { "type": "noul", "instructions": "The request conveys urgency" }
	}
}

One request, three typed answers. That is how routing cost and latency stay flat as the system grows.

What breaks agent routers

Overlapping criteria. If coding mentions "debugging code" and kubernetes mentions "debugging pods", a CrashLoopBackOff request lands between them. Write criteria as boundaries, not as marketing copy for each agent.

A state that is too thin. The router only knows what you send. "It's broken" with no context produces a flat distribution, which is the model correctly telling you it lacks information. Send more context or route to a clarification step.

Accuracy drift as state grows. TypeSafe publishes model jaggedness notes describing how accuracy shifts with state size. The context budget is generous at 64k per request, but packing a whole agent transcript into every routing call is not automatically better than sending the relevant slice.

Treating confidence as a guarantee. Calibration is a group property. A 0.9 does not mean this particular decision is right 90% of the time. That is the whole reason the escalation branch exists.

Try it

The Jev Agent playground has an agent-router scenario where you can define agents, throw real requests at them, and watch the probabilities and confidence respond. It is the fastest way to calibrate thresholds before wiring anything into production. Raw primitives are in the official TypeSafe playground.