Hi DCR Scholars - Two follow up items from my Sample Size lecture this morning in DCR:
1) I just uploaded a fresh version of the Lecture 3 slides to the website with a couple of errors fixed.
1) My explanation of 1-sided vs. 2-sided hypotheses was somewhat tortured and unclear this morning (at least it felt that way to me). If you're confused, re-read "The Null and Alternative Hypotheses" on pages 45-46 of DCR-4. Or see below for another quick explanation of the bottom line on one-sided vs two-sided tests from a UCLA stats website (http://www.ats.ucla.edu/stat/mult_pkg/faq/general/tail_tests.htm).
See you next week.
When is a one-tailed test appropriate?
Because the one-tailed test provides more power to detect an effect, you may be tempted to use a one-tailed test whenever you have a hypothesis about the direction of an effect. Before doing so, consider the consequences of missing an effect in the other direction. Imagine you have developed a new drug that you believe is an improvement over an existing drug. You wish to maximize your ability to detect the improvement, so you opt for a one-tailed test. In doing so, you fail to test for the possibility that the new drug is less effective than the existing drug. The consequences in this example are extreme, but they illustrate a danger of inappropriate use of a one-tailed test.
So when is a one-tailed test appropriate? If you consider the consequences of missing an effect in the untested direction and conclude that they are negligible and in no way irresponsible or unethical, then you can proceed with a one-tailed test. For example, imagine again that you have developed a new drug. It is cheaper than the existing drug and, you believe, no less effective. In testing this drug, you are only interested in testing if it less effective than the existing drug. You do not care if it is significantly more effective. You only wish to show that it is not less effective. In this scenario, a one-tailed test would be appropriate.
When is a one-tailed test NOT appropriate?
Choosing a one-tailed test for the sole purpose of attaining significance is not appropriate. Choosing a one-tailed test after running a two-tailed test that failed to reject the null hypothesis is not appropriate, no matter how "close" to significant the two-tailed test was. Using statistical tests inappropriately can lead to invalid results that are not replicable and highly questionable--a steep price to pay for a significance star in your results table!