Yearly Archive

Yearly Archives: 2014

February 11, 2014 no comments

The all new, good old-fashioned, solution to the “replication crisis” in science

As the number of reports highlighting the difficulty replicating academic studies proliferate, the clamour for better frameworks to ensure the repeatability of published science is becoming deafening.

 

But is replication the incomparable paragon it is held up to be? Should the scientific community be wasting resources doing the same things over and over again, rather than exploring new avenues?

 

At first sight, it seems obvious that shoring up important conclusions by independent replication is a good thing.  After all, if (for example) costly drug development activities are initiated on the basis of flawed experimental data, then the investment will be wasted.

 

But there is a problem with replication.  When the first two attempts at the same experiment yield different results, who was correct?  Without a large number of replicates, the true answer remains unknown.  And there is also an alternative approach: weight-of-evidence.  By looking at related questions, its possible to determine how likely the original experimental result is to be accurate.

 

Seeking consistency with neighbouring questions, rather than strict replication, has another important advantage too: it tests whether an observation is generalizable rather than simply true.  If a finding applies only under the strict conditions of the original experiment, it is very unlikely to be a useful conclusion at all.

 

More

More

Yearly Archive