THE RESEARCH RECORD
108 experiments on Jev
Jev is an AI model made by a company called TypeSafe. Unlike a chatbot, it does not write sentences, code, or explanations. You hand it some material, such as a support ticket, a paragraph, or a record, plus a fixed set of questions with fixed possible answers, and it tells you which answer it picked, in about a tenth of a second per question.
We handed it: Some text: a support ticket, a document, a record
We asked: Fixed questions with fixed answer options
It returned: The chosen answer, and a probability for every option
There are three kinds of question. A Choice question picks one option from a list you supply, up to 255 options. A Score question places the material on a numbered scale you define, from 2 up to 10 levels, and its answer can land between two levels rather than exactly on one. A Noul question answers a plain yes or no question and returns a single number: the probability that the answer is yes.
Alongside its pick, Jev returns a probability for every option it was given. For Choice and Score it also returns a confidence number from 0 to 1, how sure it is, kept separate from which option it chose. A Noul has no separate confidence, because its one number already carries that meaning: near 1 is a strong yes, near 0 a strong no, near 0.5 means it cannot tell.
Jev cannot generate text, write code, or explain its reasoning. It only reads text, not images, audio, or video. It cannot chain decisions together on its own, and it cannot use one question's answer as hidden context for another question asked in the same request.
The confidence number, tes…E17–E28 · 12 experimentsE17E18E19Choosing where a request g…E29–E46 · 18 experimentsE32E33E34E42E43Multi-step tasks and rechecking cla…E79–E92 · 14 experimentsE89E90E91E92Headlines, claims, and sour…E93–E108 · 16 experimentsE99E100E101E107
What we found
·
Ask Jev to rate something on more than 10 levels and it does not quietly cap the request: it returns an HTTP 400 error, not the 422 the docs describe (E01).
·
The same holds for picking one of many options: past 255 choices the request fails with a 400 error, and nothing is silently trimmed to fit (E02).
·
Confidence is not just another name for the top answer's probability: across 360 answers, confidence came in lower than the top probability 25.8% of the time, and the gap widened as the list of options grew (E18).
·
A written permission policy that correctly blocked 95% of actions it should refuse also blocked 11 of 120 actions it should have allowed, a 9.2% false positive rate (E47).
·
Asking Jev the exact same question twice does not reliably give the exact same answer: across 20 repeats of one request, only 2 of 12 questions came back byte-identical every time (E12).
·
A widely repeated claim that Jev could not navigate a map at all, succeeding 0 times in 128 tries, did not hold up on a rerun, where it succeeded 75 times out of 128 (E87).
108 experiments, 38,194 API responses logged, 45,861,280 input tokens, $1.9262 at $0.042 per million.
Compiled from dependency-docs-reference/jev/notes/12-experiments on 2026-09-19.