Merlise achieves breakthrough results on SciFact-Open
Merlise
Scientific · Teardown

Five reviewers read the paper. None of them opened this reference.

January 20265 min read

NeurIPS accepted roughly a quarter of the papers submitted to its 2025 cycle, after review by several experts per paper. An independent analysis of the accepted set still found more than a hundred fabricated citations spread across fifty one papers. Peer review caught most problems it was built to catch. Reference integrity was not one of them.

One of the hardest conferences to get into still accepted papers citing work that does not exist.

A reference that looks correct at a glance

A fabricated citation in a scientific paper can carry a believable title and a venue that sounds right, author names nobody would think to question, an identifier that resolves to something else entirely. None of that is obvious to a reader focused on the argument instead of the bibliography, which describes most readers most of the time.

One frequently cited example lists fictional authors alongside a fabricated arXiv identifier. The paper's currently posted version has since replaced that reference with a corrected one, which is itself a useful detail: a fabricated citation and its later correction can both be shown in the same ledger, without pretending the earlier version never happened.

Checking a reference means checking several fields at once

No single index is complete, so a reference missing from one can still be sitting in another. That is why Merlise checks scientific references against more than one source at once: DOI lookups, arXiv, Crossref, Semantic Scholar, OpenAlex, and the author's own posted PDF. A fabricated reference often lives exactly in that gap, a real sounding identifier attached to a title it never belonged to, or an author list that does not match what the identifier actually resolves to.

The ledger reports which specific field failed, title, author, year, venue, or identifier, instead of a single pass or fail mark. That is what turns "this citation is bad" into something a researcher can act on.

The evidence ledger

The same claim by claim view the product shows on a live document, built from this case.

VERIFICATION LEDGERReference check, one flagged citation shown
Reference to a 2024 simulation paper by two named authors, arXiv identifier given.Citation
3%
No matching work found in Crossref, Semantic Scholar, or OpenAlex for this identifier.Refuted
Cross index reference check
Currently posted author PDF cites a different, resolvable source in the same position.Citation
90%
Corrected reference resolves cleanly across all indices checked.Supported
Author's current PDF

How it resolves

Checked against four independent indices
IndexCoverage for this reference
CrossrefNo matching record
Semantic ScholarNo matching record
OpenAlexNo matching record
Author's current posted PDFReference since corrected

The resolving record

An independent analysis of the full NeurIPS 2025 accepted paper set documents the fabricated citation count directly, cross checked against Crossref, Semantic Scholar, and the open proceedings.

GPTZero, NeurIPS 2025 citation analysis