What is the structure of a test?

A hypothesis test sets a null hypothesis H_0 and an alternative H_1, chooses a significance level α, defines a rejection region, and rejects H_0 if the test statistic falls in that region. The decision can be correct or can commit one of two errors.

What is the power of a test in terms of β? [1 mark]

Why does reducing α tend to increase β for a fixed sample size? [2 marks]

SEAB Original-style practice: A test of H_0: p = 0.5 against H_1: p > 0.5 uses n = 20 trials and rejects H_0 if the number of successes X ≥ 15. Find the probability of a Type I error, and the probability of a Type II error if the true value is p = 0.7.

Under H_0, X B(20, 0.5). The Type I error probability is rejecting H_0 when it is true: $α = P(X ≥ 15 p = 0.5) = 1 - P(X ≤ 14 p = 0.5) ≈ 1 - 0.9793 = 0.0207.$ A Type II error is failing to reject H_0 when H_1 holds. If the true p = 0.7, then X B(20, 0.7), and we fail to reject when X ≤ 14: $β = P(X ≤ 14 p = 0.7) ≈ 0.5836.$ So the Type I error probability is about 0.0207 and the Type II error probability (at p = 0.7) about 0.5836. Markers reward the binomial under each hypothesis, α = P(X ≥ 15 p = 0.5) ≈ 0.0207, and β = P(X ≤ 14 p = 0.7) ≈ 0.5836.

SingaporeFurther MathsSyllabus dot point

How do Type I and Type II errors and the power of a test describe what a hypothesis test can get wrong?

Carry out hypothesis tests and analyse Type I and Type II errors and the power of a test

A focused answer to the H2 Further Mathematics outcome on errors in hypothesis testing. Type I and Type II errors, the significance level as the Type I error probability, computing the probability of a Type II error, and the power of a test.

Generated by Claude Opus 4.811 min answerUpdated 2026-06-06

Reviewed by: AI editorial process; not yet individually human-reviewed

Have a quick question? Jump to the Q&A page

Jump to a section

What this dot point is asking
The answer
Examples in context
Try this

What this dot point is asking

SEAB wants you to carry out a hypothesis test and analyse the two ways it can be wrong: a Type I error (rejecting a true null hypothesis) and a Type II error (failing to reject a false one). You must relate the significance level to the Type I error, compute the probability of a Type II error for a specified alternative, and define the power of the test.

The answer

The structure of a test

A hypothesis test sets a null hypothesis $H_0$ and an alternative $H_1$ , chooses a significance level $\alpha$ , defines a rejection region, and rejects $H_0$ if the test statistic falls in that region. The decision can be correct or can commit one of two errors.

Type I and Type II errors

The four outcomes are summarised by truth versus decision:

Type I error: reject $H_0$ when $H_0$ is true (a false positive). Probability $\alpha$ .
Type II error: do not reject $H_0$ when $H_0$ is false (a false negative). Probability $\beta$ .

A correct decision is either rejecting a false $H_0$ or retaining a true $H_0$ .

The significance level is the Type I error probability

By construction, the significance level $\alpha$ is the probability of a Type I error, because the rejection region is chosen so that, when $H_0$ is true, the test statistic lands there with probability $\alpha$ . Choosing a smaller $\alpha$ reduces Type I errors but, for fixed sample size, increases Type II errors.

Computing the Type II error probability

$\beta$ depends on the specific true value assumed under $H_1$ . To find it, work out the distribution of the test statistic at that true value and compute the probability the statistic falls in the acceptance region (so $H_0$ is not rejected):

\beta = \mathrm{P}(\text{statistic in acceptance region} \mid H_1 \text{ true value}).

The power of a test

The power is the probability of correctly rejecting a false $H_0$ :

\text{power} = 1 - \beta.

A powerful test is good at detecting a real effect. Power rises with a larger sample size, a larger true effect, and a larger $\alpha$ .

Worked example

A test of $H_0: \mu = 50$ against $H_1: \mu < 50$ uses a sample of $n = 25$ with known $\sigma = 10$ , rejecting $H_0$ if $\bar{x} \leq 46.71$ . Find the significance level and the probability of a Type II error when the true mean is $\mu = 45$ .

Step 1: Distribution under H0

Under $H_0$ , $\bar{X} \sim \mathrm{N}\left(50, \dfrac{10^2}{25}\right)$ , so the standard error is $\dfrac{10}{5} = 2$ .

Step 2: Significance level (Type I error)

\alpha = \mathrm{P}(\bar{X} \leq 46.71 \mid \mu = 50) = \mathrm{P}\left(Z \leq \frac{46.71 - 50}{2}\right) = \mathrm{P}(Z \leq -1.645) = 0.05.

Step 3: Distribution under the alternative

If the true mean is $\mu = 45$ , then $\bar{X} \sim \mathrm{N}\left(45, 4\right)$ with the same standard error $2$ .

Step 4: Type II error probability

A Type II error is failing to reject $H_0$ , that is $\bar{X} > 46.71$ , when $\mu = 45$ :

\beta = \mathrm{P}(\bar{X} > 46.71 \mid \mu = 45) = \mathrm{P}\left(Z > \frac{46.71 - 45}{2}\right) = \mathrm{P}(Z > 0.855) \approx 0.196.

Step 5: State results and power

$\alpha = 0.05$ , $\beta \approx 0.196$ , so the power is $1 - \beta \approx 0.804$ : the test has about an $80\%$ chance of detecting a true mean of $45$ .

Examples in context

Example 1. Medical screening. A diagnostic test's Type I error is a false positive (alarming a healthy patient) and its Type II error a false negative (missing a real disease); the power is the test's sensitivity, which is why screening programmes are tuned to balance these two errors.

Example 2. Manufacturing acceptance sampling. Accepting a bad batch is one error and rejecting a good batch the other; the power curve of the sampling plan shows how reliably a genuinely defective batch is caught, the basis of quality-control standards.

Try this

Q1. Define a Type I error. [1 mark]

Cue. Rejecting the null hypothesis $H_0$ when it is actually true (a false positive); its probability is $\alpha$ .

Q2. What is the power of a test in terms of $\beta$ ? [1 mark]

Cue. Power $= 1 - \beta$ , the probability of correctly rejecting a false $H_0$ .

Q3. Why does reducing $\alpha$ tend to increase $\beta$ for a fixed sample size? [2 marks]

Cue. A smaller rejection region makes it harder to reject $H_0$ , so a false $H_0$ is more often retained, raising the Type II error rate.

Exam-style practice questions

Practice questions written in the style of SEAB exam questions on this dot point, with worked answer explainers. The year tag is the paper they imitate, not the source.

Original6 marksExplain what is meant by a Type I error and a Type II error in a hypothesis test, and state how the significance level relates to each.

Show worked answer →

A Type I error is rejecting the null hypothesis $H_0$ when it is in fact true (a false positive). Its probability equals the significance level $\alpha$ , because $\alpha$ is precisely the probability of obtaining a result in the rejection region when $H_0$ holds.

A Type II error is failing to reject $H_0$ when it is in fact false (a false negative). Its probability is denoted $\beta$ and depends on the true value of the parameter, the sample size, and $\alpha$ .

So the significance level $\alpha$ is the Type I error probability, set by the experimenter; $\beta$ (the Type II error probability) is not directly set and must be computed for a specific alternative.

Markers reward the definitions of both errors (rejecting a true $H_0$ ; not rejecting a false $H_0$ ), and stating that $\alpha$ is the Type I error probability while $\beta$ is the Type II error probability.

Original7 marksA test of

H_0: p = 0.5

against

H_1: p > 0.5

uses

n = 20

trials and rejects

H_0

if the number of successes

X \geq 15

. Find the probability of a Type I error, and the probability of a Type II error if the true value is

p = 0.7

Show worked answer →

Under $H_0$ , $X \sim \mathrm{B}(20, 0.5)$ . The Type I error probability is rejecting $H_0$ when it is true:

\alpha = \mathrm{P}(X \geq 15 \mid p = 0.5) = 1 - \mathrm{P}(X \leq 14 \mid p = 0.5) \approx 1 - 0.9793 = 0.0207.

A Type II error is failing to reject

H_0

when

H_1

holds. If the true

p = 0.7

, then

X \sim \mathrm{B}(20, 0.7)

, and we fail to reject when

X \leq 14

\beta = \mathrm{P}(X \leq 14 \mid p = 0.7) \approx 0.5836.

So the Type I error probability is about $0.0207$ and the Type II error probability (at $p = 0.7$ ) about $0.5836$ .

Markers reward the binomial under each hypothesis, $\alpha = \mathrm{P}(X \geq 15 \mid p = 0.5) \approx 0.0207$ , and $\beta = \mathrm{P}(X \leq 14 \mid p = 0.7) \approx 0.5836$ .