I’m currently reading this book. But find my head swimming….. I keep wondering about all the misinterpreted stats that I’ve used in the past. And realising that I never had any concept of the Power of a test.(though guess I was always aware that more samples tended to give more accurate results). The power for any hypothesis test is the probability that it will yield a statistically significant outcome (defined in this example as p ( )

http://pro-libertate.net/20150920/278-read-statistics-done-wrong ( )

This book was a great dive into some of the gotchas that make statistical analysis of data challenging. If I were to try to narrow the common analysis mistakes to one theme, I would say that the common thread of much bad statistical analysis is trying to get more information out of the data than it can really yield. The answer isn't just to lower your p-values because, in addition to the problems with p-values themselves, requiring stricter tolerances often means that while the result measured is more likely to be a true one, the magnitude is likely to be exaggerated since you'll only accept the data sets which show the effect very strongly.

Better understanding of statistics and including those with formal statistical training as collaborators can help, but ultimately, the take away lesson from this book is that unless (and even when) you're looking at a result based on truly massive amounts of data, you should take any result as provisional until it's been replicated and replicated and replicated.

My main criticism of this book is that it was an easy enough read that, a few weeks later, I feel myself having forgot most of the details of the statistical methods discussed in the first part of the book. Retention takes a bit more struggle, and this book didn't force the reader into that struggle. ( )

If you're used to statistical analysis, you won't much that is new here: pay attention to statistical power, beware of multiple comparisons and repeated measurements without post-hoc tests and measure of effect size. However, the book is a good series of cautionary tales for new students in statistics and research methods. It is highly readable.
Towards the end, the book veers a bit off course and get more into the ethics of research and research publication. It is interesting (but not really new... especially in light of the whole recent Lacour fiasco) but it does not necessarily have to do with statistics done wrong.
Nevertheless, if you teach intro to statistics, the book is a good additional reading as it is not so much about computation, and more about statistical reasoning and understanding the strengths and weaknesses of different tests. ( )

Great book, learned a bunch. Doesn't really mention Bayesian methods though which is a shame ( )