Yearly Archive

Yearly Archives: 2013

June 24, 2013 no comments

Positive publication bias IS harmful

DrugBaron was intrigued to see a paper published in PLOS One on June 20th entitled “Why selective publication of statistically significant results can be effective”.  After all, selective publication of positive results is one of the key phenomena that underpins the replicability crisis that afflicts science.  Could it really be that positive publication bias is a good thing?

The paper is based on a simulation, and nicely demonstrates that selectively publishing positive effects results in an increased risk of poorly replicatable findings (the so-called Proteus Phenomenon).  That much is uncontentious.

But they go on to show that, over time, the scientific record is more “content-rich” with a positive publication bias.  In this context, content-rich means that a smaller number of published studies are required to accurately estimate the “real” effect than would be the case if every study was published.  In other words a smaller scientific record is required to “know the truth” with positive publication bias than without.

This finding, it seems, justified their grand title.  “Effective”, according to the authors, therefore means that keeping the scientific record small and content-rich is the dominant goal for society.

That depends on your viewpoint.

For academic science, establishing truth eventually is all that matters.  Speed is not critical, and even if the first few papers on a topic turn out to be poorly replicatable, as long as the scientific records as a whole eventually yields the truth in some kind of grand meta-analysis then all is well with the world.

And certainly historically, when the only way to find anything was to stand for hours in dusty libraries consulting large, musty leather-bound tomes, having a small content-rich scientific record was important (and indeed inevitable given the marginal cost of publishing in a pre-internet age).

But the world has changed.  Today, content can be filtered easily, millions of records searched, and related content grouped in a myriad of different ways.  Small and content-rich doesn’t carry the currency it did even a decade ago.

The relative harm of the Proteus Phenomenon also depends on the use to which the data is being put.  For the grand academic goal of establish truth eventually, a little early “fluctuation” in the conclusion doesn’t matter.  But for those scouring the frontiers of science for the next big thing (which VCs and pharma companies do all the time) the lack of replicability of the latest findings are VERY harmful.

Almost every exciting new finding that you pick up and play with turns out to be less bright and shiny than it initially appeared to be.  For many years, keen …

June 17, 2013 no comments

Ichorcumab: the blood of the gods?

Index Ventures have announced the completion of an …

June 11, 2013 no comments

Soriot and AstraZeneca face increasing headwinds: can the acquisitions of Omthera and Pearl boost the engine?

The acquisition of Pearl Therapeutics for up to …

June 10, 2013 comments

Did I say asset-centric? I meant team-centric!

In many Buddhist traditions there is a dislike …

June 4, 2013 no comments

Better drugs cannot cure healthcare system woes

For those of us investing money or our …


Yearly Archive