How should States balance probabilities of Type I and II errors?
A second key policy issue in hypothesis testing is what significance level to use, in deciding whether to reject the null hypothesis. Picking a high level of significance for rejecting the null hypothesis means that great emphasis is being placed on avoiding a Type I error (rejecting the null hypothesis, when in fact, the null hypothesis is true). This means that if a 0.10 significance level is chosen, the State wants to keep the chance of making a Type I error at or below 10%. Hence, if the chosen null hypothesis is “water meeting WQS,” the State is trying to keep the chance of saying a water is impaired, based on available evidence – when in reality it is not – under 10%. Another key issue is the determination of Type II errors (not rejecting the null hypothesis, when it should have been). The probability of Type II errors depends on several factors. One key factor is which alternative hypothesis is chosen. Another key factor is the number of samples available. With a fixed number of
