Hello - I am trying to decide whether to use Fisher's exact test or the chi squared test for analyzing the association between two dichotomous variables. I seem to recall that when we have cells with small frequencies, we may want to use Fisher's exact test (and the exact command). Can you please clarify when it is appropriate to use Fisher's exact test instead of the chi squared test? Thank you.
To test for independence between two categorical variables, you can use the chi2 test or fisher’s exact test.
Before computers were readily available, contingency tables were analyzed by hand or using calculators. The chi2 method works by computing expected values for each cell if the RR or OR = 1. It then combines discrepancies between observed and expected values into a X2 statistic from which a p-value is computed. This is an approximation, thus with small sample sizes, a X2 is not accurate. The Fisher’s exact test requires more complex calculations but is as its name suggests, exact. However, it does assume that row and column totals are fixed by the experiment, which may not always be the case depending on study design/experiment.
The general rule of thumb in chi2 testing is that no more than 20% of cells should have expected frequencies of <5. Again, the chi2 is an approximate method and the chi2 distribution is an idealized mathematical model, whereas fisher’s exact test calculates exact probabilities. Thus you are correct that the chi2 test works well for large sample sizes, but is not accurate when it comes to rare events.
In Stata, you can actually run both tests at once and compare p-values generated using both chi2 and fisher's exact methods. For example, "tab rowvar colvar, chi exact"
Hi, I have a similar question. In determining whether freqcaller is related to gender, I ran both chi2 and Fisher's exact tests and obtained p value of 0.005 for chi2 and p-value 0.008 for Fisher's. Since there are 5 females among freqcallers, does that meet the criteria for using chi2 testing?
Could you also clarify what "20% of cells with expected frequencies of <5" means? Thank you.
For 2x2 contingency tables, the "rule of thumb" for use of the Pearson's Chi Squared test is that:
1) All expected counts should be 5 or greater
2) If any expected counts are < 5, another test should be used, such as Fisher’s Exact Test.
In this case, you have a 2x2 with one cell that includes a value of 5, so Chi2 is an appropriate test to use. When you run the numbers, you see that both give statistically significant p values of <0.05.
For contingency tables larger than 2x2 the rule of thumb for using Pearson’s Chi Squared test is that “…no more than 20% of the expected counts are less than 5 and all individual expected counts are 1 or greater." (Yates, Moore & McCabe, 1999) So the "20% of cell" really applies to larger tables with more cells.