zweifach, a.

Statistical tests can be used to help determine whether experimental manipulations produce effects. In tests of means, when more than two groups are compared the number of false positive results increases unless a correction is used. Tukey\'s test is thought to offer good control of the false positive rate and high statistical power when all pairwise comparisons are made. However, the number of replicates in laboratory experiments is often quite low, and small sample sizes can undermine assumptions underlying statistical methods. I used simulations to investigate how well Tukey\'s test controls the false positive rate when there are 3- 6 experimental groups and 2- 6 experimental replicates, conditions that span the range of typical values, and found that it generates too many false positives. I investigated 11 other approaches to controlling the false positive rate and found that none is as effective as the simple Bonferroni correction or offers much more power. I conclude that researchers should not make all pairwise comparisons using Tukey\'s test but instead use Bonferroni\'s correction on a limited number of pre-selected comparisons.