Merlise achieves breakthrough results on SciFact-Open
Merlise
Legal · Failure mode

The case exists. It just does not say that.

March 20264 min read

A lookup tool that only checks existence will wave this failure through. The case is real, the volume and page match, the citation is topically close enough to seem right. What breaks is the bridge between the source and the proposition the brief hangs on it.

Found the source. Wrong question is what the source actually answers.

An analogy that travels beyond law

The same shape shows up outside legal writing. An article cites a real medical paper to support a claim that a treatment is effective, when the paper only studied whether the treatment was safe. The citation resolves. The claim does not follow from it. Whiting and LNU both document the legal version: a case that discusses sanctions, or fees, or a constitutional question, cited for a rule the opinion never addresses.

A reviewer who only confirms the case exists has done half the job and may not know it, because the reassurance of a working citation feels complete even when it is not.

Extracting the actual proposition

A citation carries a specific claim somewhere close by, in a quoted sentence, a parenthetical, or the clause right after "holding that." Merlise pulls that claim out and checks it against what the opinion actually says. Sometimes the mismatch is exact, the quoted words simply are not there. More often it is a quieter kind of drift: the case answers a different question, or lands on a different result than the one implied.

The output says which. A lawyer fixing "case exists but addresses a different issue" needs a different citation. A lawyer fixing "words not found" needs to drop the quotation marks or find the actual source of the phrase. Collapsing both into a single red flag would tell the reader less than they need.

The evidence ledger

The same claim by claim view the product shows on a live document, built from this case.

VERIFICATION LEDGERReal source, unsupported proposition
Cited case "holds that arbitration clauses survive termination of the contract."Citation
15%
Case addresses venue selection, not arbitration survival.Refuted
Case law index
The case was decided by the cited court in the cited year.Atomic fact
96%
Court and date both match.Supported
Case law index
The rule from this case applies to the facts at hand.Relational
24%
Underlying holding does not match the proposition claimed.Uncertain
Case law index

How it resolves

Same failure, different fields
Field checkedMissing citationReal case, wrong holding
Case exists in a reporterFailsPasses
Quoted words appear in the opinionFailsVaries
Opinion supports the stated propositionFailsFails

The resolving record

Both Whiting and the Ninth Circuit's LNU order document this exact pattern from the bench: citations that resolve to a real case and still fail because the proposition attached to them is unsupported.

Sixth Circuit and Ninth Circuit sanctions orders, 2026