## How to Determine a p-Value When Testing a Null Hypothesis

### F Statistic / F Value: Definition and How to Run an F-Test

Most statistical tests culminate in a statement regarding the -value, without which reviewers or readers may feel shortchanged. The -value is commonly defined as the probability of obtaining a result (more formally a ) that is at least as extreme as the one observed, assuming that the is true. Here, the specific null hypothesis will depend on the nature of the experiment. In general, the null hypothesis is the statistical equivalent of the “innocent until proven guilty” convention of the judicial system. For example, we may be testing a mutant that we suspect changes the ratio of male-to-hermaphrodite cross-progeny following mating. In this case, the null hypothesis is that the mutant does not differ from wild type, where the sex ratio is established to be 1:1. More directly, the null hypothesis is that the sex ratio in mutants is 1:1. Furthermore, the complement of the null hypothesis, known as the or , would be that the sex ratio in mutants is different than that in wild type or is something other than 1:1. For this experiment, showing that the ratio in mutants is different than 1:1 would constitute a finding of interest. Here, use of the term “significantly” is short-hand for a particular technical meaning, namely that the result is , which in turn implies only that the observed difference appears to be real and is not due only to random chance in the sample(s). . Moreover, the term significant is not an ideal one, but because of long-standing convention, we are stuck with it. Statistically or statistically may in fact be better terms.

### Null and Alternative Hypothesis | Real Statistics Using …

The basis for many nonparametric tests involves discarding the actual numbers in the dataset and replacing them with numerical rankings from lowest to highest. Thus, the dataset 7, 12, 54, 103 would be replaced with 1, 2, 3, and 4, respectively. This may sound odd, but the general method, referred to as a , is well grounded. In the case of the Mann-Whitney test, which is used to compare two unpaired groups, data from both groups are combined and ranked numerically (1, 2, 3, … ). Then the rank numbers are sorted back into their respective starting groups, and a is tallied for each group. If both groups were sampled from populations with identical means (the null hypothesis), then there should be relatively little difference in their mean ranks, although chance sampling will lead to some differences. Put another way, high- and low-ranking values should be more or less evenly distributed between the two groups. Thus for the Mann-Whitney test, the -value will answer the following question: Based on the mean ranks of the two groups, what is the probability that they are derived from populations with identical means? As for parametric tests, a -value ≤ 0.05 is traditionally accepted as statistically significant.

It is also worth pointing out that there is another way in which the -test could be used for this analysis. Namely, we could take the ratios from the first three blots (3.33, 3.41, and 2.48), which average to 3.07, and carry out a one-sample two-tailed -test. Because the null hypothesis is that there is no difference in the expression of protein X between wild-type and backgrounds, we would use an expected ratio of 1 for the test. Thus, the -value will tell us the probability of obtaining a ratio of 3.07 if the expected ratio is really one. Using the above data points, we do in fact obtain = 0.02, which would pass our significance cutoff. In fact, this is a perfectly reasonable use of the -test, even though the test is now being carried out on ratios rather than the unprocessed data. Note, however, that changing the numbers only slightly to 3.33, 4.51, and 2.48, we would get a mean of 3.44 but with a corresponding -value of 0.054. This again points out the problem with -tests when one has very small sample sizes and moderate variation within samples.

## Significance Tests / Hypothesis Testing - Jerry Dallal

The Central Limit Theorem having come to our rescue, we can now set aside the caveat that the populations shown in are non-normal and proceed with our analysis. From we can see that the center of the theoretical distribution (black line) is 11.29, which is the actual difference we observed in our experiment. Furthermore, we can see that on either side of this center point, there is a decreasing likelihood that substantially higher or lower values will be observed. The vertical blue lines show the positions of one and two SDs from the apex of the curve, which in this case could also be referred to as SEDMs. As with other SDs, roughly 95% of the area under the curve is contained within two SDs. This means that in 95 out of 100 experiments, we would expect to obtain differences of means that were between “8.5” and “14.0” fluorescence units. In fact, this statement amounts to a 95% CI for the difference between the means, which is a useful measure and amenable to straightforward interpretation. Moreover, because the 95% CI of the difference in means does not include zero, this implies that the -value for the difference must be less than 0.05 (i.e., that the null hypothesis of no difference in means is not true). Conversely, had the 95% CI included zero, then we would already know that the -value will not support conclusions of a difference based on the conventional cutoff (assuming application of the two-tailed -test; see below).

## Analysis of Variance 3 -Hypothesis Test with F-Statistic

Again, the χ2 test of independence is used to test whether the distribution of the outcome variable is similar across the comparison groups. Here we rejected H0 and concluded that the distribution of exercise is not independent of living arrangement, or that there is a relationship between living arrangement and exercise. The test provides an overall assessment of statistical significance. When the null hypothesis is rejected, it is important to review the sample data to understand the nature of the relationship. Consider again the sample data.