E93–E108 · DESIGN FILE, AS WRITTEN
Headlines, claims, and source stance
16 experiments, 4,688 records and 4,718,489 input tokens.
C
Confirmed
11
F
Falsified
1
M
Mixed
3
B
Blocked
1
Whether Jev can read a short headline or claim and correctly judge its stance, its packaging, its overstatement, and whether the evidence behind it actually holds up.
Reading the record…
Experiments in this group
Exp
Verdict
Question
E93
C
Confirmed
Does splitting 'unrelated' from 'neutral' change the model's answer?
E94
B
Blocked
Do two human readers agree with each other before the model gets graded?
E95
C
Confirmed
Does the order options are listed in change the model's answer?
E96
F
Falsified
Does the model spot which headline style actually won a real click test?
E97
C
Confirmed
Does checking several branches instead of one change a headline's category?
E98
C
Confirmed
Does naming the target inside the question work as well as pointing to it in data?
E99
C
Confirmed
Can the model tell when a headline claims more than the article says?
E100
C
Confirmed
Does the overstatement score change with how much article text the model sees?
E101
C
Confirmed
Are tone, urgency, and conflict-framing really three separate things?
E102
M
Mixed
Does a three-way support-or-contradict check work on real news and abstracts?
E103
M
Mixed
Does a 'mixed' option get used correctly, or as a hedge?
E104
C
Confirmed
Can the model name a study's design and judge if it can prove a cause?
E105
C
Confirmed
Does the model rate a randomized trial as stronger evidence than an observational one?
E106
M
Mixed
Can the model tell if a trial was superseded, or a paper was retracted?
E107
C
Confirmed
When the true source is missing, does the model say so instead of guessing?
E108
C
Confirmed
Does the model agree with a real medical review's own included-study list?
108 experiments, 38,194 API responses logged, 45,861,280 input tokens, $1.9262 at $0.042 per million.
Compiled from dependency-docs-reference/jev/notes/12-experiments on 2026-09-19.