Profiting from pilot studies

A Greybox transplant experiment

A Greybox transplant experiment

I graduated with a Masters of Philosophy at the University Melbourne earlier this year. And the first paper to come out of that work, to paraphrase Lil’ Wayne, has just “dropped like it’s hot”.

In our new paper (in early view at Basic & Applied Ecology) Pete, Mick and I illustrate the use of Bayesian informative priors to recover the inferential and predictive power of otherwise unusable pilot study data.

You can open pretty much any textbook on experimental design and one of the first things it’ll tell you is to do a pilot study. Pilot studies are standard practice and we do them so that we don’t waste resources doing an experiment only to find we had the wrong study design. The same textbooks will also tell you that if your pilot study indicates that you have to change the design of your study, forget about the data you’ve collected and get on with the new experiment. We argue that this default stance can be wrong and that otherwise unusable pilot study data can be used to construct a Bayesian prior for an analysis of a subsequent experiment.

We showed how data from a pilot study can be used in a case study on eucalypt seedling mortality during a transplant experiment conducted in Goulburn-Broken catchment in central Victoria, Australia. Using a pilot study to construct a prior prediction of mortality rate during a subsequent larger experiment, we found that including the informative prior effectively saved us thousands of dollars. Had we ignored the pilot study and included a flat prior in the final model, we would have needed to spend the extra money on monitoring hundreds more seedlings if we wanted the same amount of information as we did when including the informative prior.

Morris, W.K., Vesk, P.A., McCarthy, M.A., (early view, 2012) Profiting from pilot studies: analysing mortality using Bayesian models with informative priors Basic & Applied Ecology.

This entry was posted in Science and tagged , , , , . Bookmark the permalink.

One Response to Profiting from pilot studies

  1. Pingback: Recommended Reading | January 2013 | Cindy E Hauser

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s