A systematic review that graded its own homework

A systematic review is supposed to be the boring, trustworthy end of medical evidence: not one study, but an accounting of many, with each one’s quality graded by reviewers with no stake in the outcome. Remove that independence and a systematic review is just an opinion with extra formatting.

In a detailed thread, bioethicist Brian D. Earp walks through a widely-cited systematic review concluding that circumcision has no adverse effect on sexual sensation or satisfaction.[1] His point isn’t that the conclusion is automatically wrong. It’s that the quality ratings behind it — which studies counted as strong evidence and which were downgraded — were assigned by the review’s own authors, rather than by an independent panel, which is what the standard methodology the review cites is supposed to require.

Earp’s thread goes claim by claim: which older, thinly-described studies were rated as high quality, which larger and more rigorous studies were rated as low quality, and how the pattern in those ratings tracks the desired conclusion more consistently than it tracks methodological rigor. Read in full, it’s a study in how a review can look procedurally sound while the one step that’s supposed to guarantee objectivity — grading done by someone without a stake in the answer — quietly isn’t there.