Scientists sometimes take back a paper they published: a
retraction. Wikipedia articles that cite the paper often keep citing it for
years, as if nothing had happened. This page explains how the Tracker finds those passages, and
how it orders them so that the ones most worth fixing come first.
Why this exists
A paper is retracted: its journal says it should no longer be trusted.
The Wikipedia sentence that cites it stays as it was, footnote and all.
A scientific paper is a report of a study. When something turns out to be badly wrong with it,
the journal that published it retracts it: a notice says the paper should no
longer be trusted. The data may have been faked, an experiment may have gone wrong, or the text
may have been copied. About 6,000 papers a year are retracted.
A Wikipedia sentence that relies on a source ends in a small number in brackets, like this:
“The new pill halves recovery time.[12]” The number is a footnote.
It points to an entry in the reference list at the bottom of the Wikipedia article, which names
the source. When that source is retracted, the sentence usually stays exactly as it was. A bot
may add a “retracted” tag to the entry in the reference list, but few readers ever look there. A
2025 study from Northwestern University counted
1,181 citations of retracted papers on English Wikipedia, and found that the typical fix took more
than three and a half years.
The Tracker finds these passages every day and lists them for Wikipedia's volunteer editors,
the worst and most-read first. It never edits Wikipedia itself. People do that.
How a passage gets on the list
Six steps, run by a program every day. No person reads anything until the list is up.
1
Get the list of retracted papers
The Retraction Watch Database lists more than 60,000 retracted papers, each with the reasons
it was retracted. The Center for Scientific Integrity, which also runs the Retraction Watch blog,
adds new ones every working day, and Crossref publishes it. The Tracker downloads it every
day, and leaves out some withdrawn Cochrane reviews.
2
Find the Wikipedia articles that cite them
Every paper has an ID code, called a DOI, that works like a book's ISBN. (Medical papers have
a second one, a PubMed ID.) Wikipedia's citations carry those codes as links. Once a month
Wikipedia publishes a list of every link on every page, which the Tracker matches against the
retracted papers' codes. Every day it also asks Wikipedia which pages link to the papers retracted
since the day before.
3
Find the exact sentences
The Tracker fetches each Wikipedia article and finds every footnote that points to the paper,
with the sentence, list item or table row it ends. That takes care: one footnote is often reused
after many sentences, and some footnotes point to another footnote first.
4
A model reads the Wikipedia article
For each Wikipedia article and retracted paper it cites, an AI language model (the kind behind
chatbots) reads the whole wiki article, the paper's title and summary, and why it was retracted.
For each citing sentence it picks out the words that rest on the paper. It also lists every other
paragraph that repeats the paper's finding with no footnote to it, and every place the wiki
article warns readers that the paper was retracted or its claim was wrong, which a card lists as
where our Tracker found a warning. The Tracker checks every quote the model gives, word for word,
against the wiki article.
5
A model compares the passages
A model reads two passages side by side, each with its section of the wiki article, the wiki
article's opening, and the paper and why it was retracted. It says which one an editor should fix
first, or that they're about equal. It also gives each one a short phrase, which sets its step.
Each passage plays about 10 such comparisons.
6
A model explains each card
For each passage on a list, a model writes the opened card's middle paragraph: what our
Tracker believes about the claim, and the edit it suggests, if any. It explains the step the
comparisons chose, and never changes it. The Tracker checks every quote it gives, word for word,
and keeps words such as fraud or misconduct to where Retraction Watch's reasons or the wiki
article use them.
Each sentence, list item or table row that cites the paper, and each repetition the model
found, becomes one passage on the list (a “finding” in the data file). The
models never write Wikipedia text: they only read and explain.
The lists and their order
The model's comparisons put each passage on one of three lists, as explained
below. Within each list, every passage gets a priority, and
the list shows the highest first. A card ranks by its passage with the highest priority. The
priority is two numbers multiplied together:
How many people read the wiki article each day, on average, over the last 30 days.
Wikimedia publishes a daily count of views for every Wikipedia article, by people rather than
bots, and the Tracker fetches it each run. A passage that more people read matters more.
The count runs to the day before yesterday, since the newest day's count isn't
complete. When the same text appears in several wiki articles (one can show a section of
another), their readers are added together.
Severity
How bad the passage is in itself, however many people read it. It comes from the comparisons
the way a chess rating comes from games: a passage the model picks over others gets a higher
severity, and one it passes over gets a lower one. If one passage has severity 3 and another
has 1, the model would pick the first about three times in four. A tie counts as half a win.
Severity 1 is a fixed point near the middle of the list. It's set by one imaginary passage,
whose severity is always exactly 1. The math gives every real passage one extra comparison
against it, a tie, which the model never plays. On the list of 9 November 2026,
severities ran from 0.198 to 147, and half were below 0.695.
The tie also keeps every severity finite. Take a passage that beat ten passages of severity 1:
the higher its severity, the more likely that record becomes, so no severity explains it best.
With the imaginary tie, it has 10.5 wins in 11 comparisons. At severity 21 it would be expected
to win 21 in 22, which is 10.5 in 11, so 21 explains it best. The tie is an assumption, not
something the model said: that no passage is wildly far from the middle. It pulls hardest on the
passages furthest from 1, so the most severe ones get severities closer to the rest's than their
comparisons alone would give them.
A passage plays its comparisons mostly against passages close to its own severity, as in a
chess tournament, since a close match tells the most. A new passage, or one whose section or
wiki article's opening is edited, plays 10 new comparisons. Until it does, it is on no list,
and the counts at the top of each list say how many passages wait.
The lists and the steps
In each comparison, the model also gives each passage one of six phrases. Each phrase votes
for one of three lists, and a passage goes on the list most of its comparisons vote for. On a
tie, claims to fix come before sources to replace, and both come before the passages already
covered. Its card shows the step for the phrase most of those votes gave.
SuggestedFix the claim
The retraction puts the claim in doubt, and the passage states it as fact with no warning nearby.
SuggestedCheck the claim
It can't be told whether the claim stands, and the passage states it as fact.
SuggestedReplace the source
The claim stands without the paper, but the paper is its only source, and a reason it was retracted questions its findings or is plagiarism, or none is given.
SuggestedCheck the citation
The comparisons say another source is needed, but a program finds the footnote may not cite the retracted paper as Retraction Watch records it: its own PubMed ID and title name another paper, or Retraction Watch names only the publisher's error as what was wrong, so a version that wasn't retracted may exist, and the footnote may cite it.
SuggestedBacked by other sources
Other sources cited on the passage back the same words.
SuggestedNot affected by the retraction
The paper was retracted over how it was made or published, such as missing ethics approval or consent, not over its results, and nothing else covers the passage.
SuggestedDisclosed in the wiki article
A warning nearby tells readers, or the passage tells the story of the paper rather than stating its claim.
A filled dot is a list's main step, and a hollow one a lighter step worth a check first. A
dash marks a passage the retraction doesn't affect. A card leads with the passage that asks most
of an editor, in the order above. A wiki article with passages on several lists has a card on
each, and each card links the others.
Severity of the open passages on 9 November 2026, by list
Fix the claimCheck the claim
82 passages · median 21
Replace the sourceCheck the citation
188 passages · median 3.2
Disclosed in the wiki articleBacked by other sources
599 passages · median 0.58
0.20.5125102050100
Each dot is a passage, placed by its severity on a log scale: from 1 to 10 is as far as from 10 to 100. The dashed line is 1, the imaginary passage's severity. The thick line in each row is the row's median, with half its passages below it. The chart is kept as it was that day. The data file has today's severities.
Readers don't move a passage from one list to another. A claim to fix on a quiet wiki article
stays on the claims list, low down, and a source to replace on a busy one sits high on its
own.
Why multiply
Multiplying makes each number count in proportion. A passage on a wiki article read ten times
as often needs a tenth of the severity to draw level. So a quiet page's severe passage can
still rank below a busy page's ordinary one:
Readers a day
Severity
Priority
Quiet page, severe passage
5
10
50
Busy page, typical passage
200
1
200
Three examples
Made-up passages, so the numbers come out round.
A. A quiet page, a false claim
A Wikipedia article read 40 times a day says, as fact, that a new drug halves
recovery time. Its only source is a paper retracted because its data were fabricated. Nothing
nearby warns the reader.
40 readers a dayseverity 30SuggestedFix the claim
40 × 30 = 1,200
B. A busy page, a claim that can't be checked
A Wikipedia article read 600 times a day says, as fact, that a vitamin lowers
blood pressure. Its only source is a paper retracted because its results couldn't be reproduced,
and no other source settles the question.
600 readers a dayseverity 4SuggestedCheck the claim
600 × 4 = 2,400
C. A wiki article about the scandal
A Wikipedia article read 2,500 times a day, about a scientist's fraud, says
that the paper was retracted in 2014.
2,500 readers a dayseverity 0.2SuggestedDisclosed in the wiki article
2,500 × 0.2 = 500
A1,200
B2,400
A and B are both claims to fix, and B comes first. Whether its claim stands can't be told, and
600 readers a day see it. A's claim is worse: a reader there is told something false. But few
people read it. C has the most readers, but its passage tells the story of the retraction, so it
goes on another list, the passages already covered, and needs nothing.
Reading a card
Each card is one wiki article's passages on one list. A card ranks on its list by its
highest-priority passage. Here is the first card on today's claims to fix list, exactly as the list
shows it, with the arithmetic of the passage that ranks it. Tap anywhere on it to open or close
it.
1
Effects of climate change
SuggestedFix the claim145 readers/day
Regional median economic impacts predicted due to global warming by 2050 compared to present.
Footnote [267] on this image caption cites a paper that was retracted for concerns about its methods, errors in its analyses and errors in its data.
Our Tracker believes these regional projections are likely unreliable because they rest on the paper. The caption presents them without warning readers about that uncertainty. Suggested edit: remove the image and caption, or replace them with projections supported by a reliable source.
A person labelled this passage “real problem” on 25 October 2026, reading an earlier revision of the page.
Details
In its paragraph
Regional median economic impacts predicted due to global warming by 2050 compared to present.[267]
Concerns/Issues about Data; Concerns/Issues about Methods; Error in Analyses; Error in Data; Investigation by Journal/Publisher; Investigation by Third Party; Upgrade/Update of Prior Notice(s)
Wikipedia's retraction tag
Wikipedia marks this reference as retracted.
How it ranks
Severity 31: how serious 10 comparisons with other passages rate it, not how sure they are. 145 readers a day × 31 ≈ priority 4,421. How it works
1The number is the card's
place on its list. Its colour, and the colour of the card's left edge, is the list's.
SuggestedFix the claimThe step for the
card's first passage: what the model's comparisons suggest an editor do, as explained
above. It says what to do, not how.
145 readers/dayReaders a day of the wiki
article, and how many passages the card holds, when more than one.
Regional median economic impacts predicted due to global warming by 2050 compared to present.The words of the first
passage that rest on the paper, cut to two lines; then their section, and a link that opens
Wikipedia at them.
A tap anywhere on a card, or on its “Why”, opens or closes it; a tap on a link opens the link.
Opened, the card shows the passage's words in full, then explains them in three short
paragraphs:
What's on record: whether the passage cites the retracted paper, and why the
paper was retracted, from Retraction Watch's reasons, in plain words. These are fixed sentences in
our code.
What our Tracker believes: an AI model's explanation of the step: what it
believes about the claim, and the edit it suggests, if any. When the explanation fails our checks
twice, the card says the reading is missing, and until one is written for the passage as it is
now, that it isn't written yet.
What's still open, from our code: what nobody has checked yet, or the label a
person gave the passage, with its date.
“Details” holds the evidence: the passage in its paragraph, the paper, Cochrane's
withdrawal notice when the paper is a Cochrane review, Retraction Watch's categories as
recorded, where our Tracker found a warning, the other sources cited on it, Wikipedia's
retraction tag, how the passage ranks, and the revision it was read from. A card's other
passages follow, each with its own “Why”. A card also names the wiki article's
passages on other lists, with a link to each. The data file lists
every passage with its readers a day (views), its severity
(passage_severity), its priority, its place among all open passages
(rank), its phrase, step and list (phrase, step,
list), what our Tracker believes (why), and its comparisons won,
tied and lost (record).
A note about Cochrane reviews
A Cochrane review sums up all the medical studies on one question. Some that Wikipedia cites
were withdrawn, yet the Tracker leaves them off its lists. Here's why.
Withdrawn for being old, not wrong. Until August 2019, Cochrane withdrew a
review when it fell out of date or a newer one replaced it, though nothing in it was found wrong.
Since then, it withdraws one only for a real problem.
Upkeep, not bad science. Replacing one on Wikipedia keeps a source current.
It doesn't take bad science off Wikipedia, which is what this list is for.
Real problems stay listed. A review withdrawn for a problem with it, such as
unreliable data, is still on the list.
Old guidelines withdrawn as out of date are left out the same way. Their passages stay in the
data file, marked upkeep: 146 so far.
What the order can't tell you
It's an AI model's reading, not a person's. The steps, the severities, where
our Tracker found a warning, and what it believes all come from an AI model. No person
has checked them unless the card says so. A card's other sentences, and the check that sends a
footnote to be checked, are fixed rules in our code.
A severity is rough. Ten comparisons is a small sample, so a severity can be
off by a factor of about two. Passages far apart are in the right order; close ones may not be.
Older passages keep playing as opponents of new ones, so their severities firm up.
The model isn't perfectly consistent. Asked the same comparison twice, it
sometimes answers differently, and which passage it reads first can change its answer.
Whether a claim stands rests on what the model knows. It weighs each claim
against what it knows of the field, which can be out of date or wrong.
Readers are counted for the whole wiki article, not the passage. Most readers
never reach any given paragraph.
A wrong phrase can put a real problem on the wrong list, and a low severity
can sink it down its list. That is why no model's reading takes a passage off: every open passage
is on one of the lists.
Before you edit: read the Wikipedia article and
its sources yourself, and write any change in your own words. The Tracker never edits Wikipedia,
and no model writes Wikipedia article text; English Wikipedia's rules forbid it.