Benchmarks

Jev latency, cost, and consistency

Reproducible measurements of Jev decisions across state sizes: latency percentiles, input tokens, cost per decision, and how often the same choice comes back.

// Methodology

We send the same Choice and Noul question set to POST /v1/systemone at three state sizes, repeat each size, and record end-to-end latency, input tokens, computed cost, and how consistent the winning choice is. No prompt tuning, no retries. Anyone can reproduce it with the harness in this site's repository:

TYPESAFE_API_KEY=... pnpm --filter marketing benchmark

// Results

No measurements have been published yet.

We only publish numbers we measured ourselves. Run the harness with a TypeSafe API key and the results appear here.

// Caveats

Latency depends on your network, region, and current load; TypeSafe notes that rate limits are adjusting dynamically during the launch period. Cost is computed from input tokens only, because output tokens are free. Treat these numbers as a starting point and measure with your own states and questions.