Power is defined as the probability of correctly rejecting the null hypothesis (alternatively, the probability of detecting true difference/effect).
A smaller level of alpha will make it harder to reject the null hypothesis. For example, we need to see something more extreme to make us reject the null hypothesis using alpha = 0.01 (1% level) compared with alpha = 0.05 (5%) level. We're being more stringent with a smaller alpha; we need much more evidence to reject the null. This means it's harder to conclude a difference exists. Since this is true whether the null is true or false in reality (either case), we can say it's harder to reject a false null, meaning the probability of rejecting a false null hypothesis (detecting a real difference/effect) has decreased (power has gone down). This is also true for increasing alpha and increasing power. Both scenarios assume that every other factor is unchanged.
A smaller level of alpha will make it harder to reject the null hypothesis. For example, we need to see something more extreme to make us reject the null hypothesis using alpha = 0.01 (1% level) compared with alpha = 0.05 (5%) level. We're being more stringent with a smaller alpha; we need much more evidence to reject the null. This means it's harder to conclude a difference exists. Since this is true whether the null is true or false in reality (either case), we can say it's harder to reject a false null, meaning the probability of rejecting a false null hypothesis (detecting a real difference/effect) has decreased (power has gone down). This is also true for increasing alpha and increasing power. Both scenarios assume that every other factor is unchanged.
Last edited:
