10 Summary
Design experiments as randomized blocks where possible and plan statistical analysis before gathering data.
Visualise & explore your data by plotting graphs and eye-balling them. I cannot emphasise this enough!
Pay attention to the (probability density) distribution & characteristics of your data.
Statistical comparisons rely on data distributions, degrees of freedom (total number of samples) and extreme values. These decide the P value. The P value does not tell you whether the \(signal\) in your results is large or whether the experimental design is good. It does not tell you the likelihood of obtaining the same result if your entire study was repeated by someone else.
Linear models are easy to understand and t tests & ANOVAs can be solved using linear equations.
Block design, paired or matched observations reduce \(noise\) and increase the power of comparisons.
Remember to correct P values for multiple comparisons especially when making more than 3 comparisons.
Consider data transformations to make your observed data and/or model residuals closer to normal distribution, and use parametric tests. These tests are able to tolerate moderate departure from normality and are more powerful than non-parametric counterparts.
When normalising data into percentages or fold-changes ensure the SD of control group is not 0. (as it would violate assumptions of the normal distribution)
Be aware of pseudo-replicates, which are seemingly more data but are not statistically independent. Only biologically independent and representative data should be included for statistical comparisons.
Use preliminary data or literature to find the effect size, mean and SD you expect from experiments to decide how many sample number or experiments to perform. Ensure sufficient power (at least 80%) for experiments with animals to comply with principles of 3R.