Climate ErrataIndependent, reproducible checks of climate papers, datasets and code

How Climate Errata works

An independent register of errors and issues in climate science: published papers, gridded and station datasets, and the software used to analyse them.

What we check

Methods, statistics, data processing and code. Most entries come from reproducing a result with the published data and code, or from comparing a dataset with independent references and station records.

Contact first

Every finding is sent privately to the authors, data producers or developers before anything is posted. An entry appears here only after they reply or three weeks pass without a reply. Their response is recorded on the entry.

Fixes we propose

We don't just report problems. Where we can, we propose a fix and send it with the report:

These are recommendations. The authors, data producers or developers decide whether and how to fix their work, and each entry records what they did. The "What users can do" part of an entry says how to avoid the problem in the meantime.

Statuses

How the work is done

The analyses were carried out with an AI system (Claude, Anthropic) under the author's direction. The author checked the methods and results and is responsible for the content.

Our reports on Zenodo

Write-ups of single findings are linked from their entries. Two broader reports cover the work as a whole:

Significance

Each finding that claims a problem is scored 0–2 on three criteria. The class comes from the total (0–6): High 5–6, Medium 3–4, Low 0–2. Clean checks and already-known issues are not scored.

Criterion012
S · Size of the effect, relative to the signal users take from the output< 5% of the signal, or inside stated uncertainty5–25%, or moves a result across a significance threshold> 25%, reverses the sign or conclusion, or the output is invalid
R · Reach of the affected code, data or paperNiche: non-default option, under ~1,000 downloads a month, or one paper with few citationsModerate: standard tool in a sub-field, tens of citing papers, or a recent paper in a high-profile journalInfrastructure: CMIP/IPCC/ISIMIP or national datasets, the dominant tool of its kind, 100+ citing papers, or policy and press use
C · Consequence for published or operational resultsNone shownAffects results in principle (including bugs confirmed or fixed by developers)A specific published headline or production dataset is shown to change, by our rerun or the authors' own statement

Reliability of the scoring

Independent raters applied the written rubric blind to all 80 findings in the full audit (public and not yet public). Agreement with the first rater (weighted Cohen's kappa), rubric v1 → v2: Size 0.77 → 0.81; Reach 0.68 → 0.73; Consequence 0.62 → 0.73; Class 0.62 (65% exact) → 0.70 (79% exact). Between rounds one rule was clarified: a developer-confirmed or merged fix alone counts as C = 1, not C = 2. All raters so far are AI instances; a human rater on a random sample is planned. Reach scores marked "est." are estimates to be replaced by measured download or citation counts.

Rules

Policies

The full policies (author responses and corrections, moderation, code of conduct, funding and conflicts of interest, privacy, takedown requests) are on the Policies and privacy page.

Site updates

Changes to the website itself are listed on the Site updates page, with an RSS feed.

Contact

Write to us · GitHub klemmeri · Substack @markmorrissey3.

Discussion and questions

Climate Errata is also a place for researchers to ask about the datasets and papers they use. Every register entry has a discussion thread. Any paper or dataset can have one too: enter its DOI on the Discuss a DOI page, and everyone asking about the same work shares one thread. Ask other readers, the authors or the maintainer. Search covers both the register and the discussions.

Threads use GitHub Discussions, so commenting needs a free GitHub account. Without one, use Write to us.

Papers that use affected data

For a problem in a dataset, the entry may list papers whose abstract or text shows they use the affected data for the affected region and period. The list is a pointer for readers and authors, not a finding about those papers: the effect on each paper is not assessed, and many conclusions will hold. Papers whose authors we are in contact with are not listed until that exchange has had its three weeks. Authors can reply on the entry or write to us, and we correct the list on request.

Posting policy

Reuse

Entries are licensed CC BY 4.0 and the register is available as entries.json. Code is MIT-licensed.