Lesson 12: Correlation and Regression (Intro)

Do taller parents have taller children? Does advertising spend drive sales? Do ice-cream sales cause drownings? Correlation is the statistics of relationships — and the gateway to data science. This introductory lesson (Higher-tier depth) covers describing correlation, lines of best fit, interpolation and extrapolation, and the first ideas of regression: fitting y = a + bx by least squares and measuring fit with the correlation coefficient r.

Describing correlation

Plot paired data on a scatter graph (Lesson 11) and describe the pattern: strong or weak, positive or negative. Shoe size vs height in a class of teenagers: strong positive. Hours of sunshine vs umbrella sales: negative. Hair colour vs maths score: no correlation. Strength is about how tightly the points hug an imaginary line; direction is about the slope of that line.

The crucial warning: correlation is not causation. Ice-cream sales and drowning incidents correlate strongly — but neither causes the other; hot weather (a lurking variable) drives both. Similarly, schools with smaller classes may get better results, but wealthier areas fund both smaller classes and tutoring. GCSE and A-level both test this distinction: describing a correlation is one mark; claiming a cause without evidence is an error.

Worked example: a study finds that villages with more storks have higher birth rates. The correlation is real but the 'cause' is nonsense — larger villages have more chimneys (stork nests) and more families. Always ask: what third factor could explain both?

Outliers matter: one point far from the pattern (a pupil who revised 0 hours but scored 95 thanks to prior knowledge) weakens the apparent correlation. State whether an outlier should be investigated (data error?) or kept (genuine). In exams, an anomalous point is often planted deliberately.

Lines of best fit — prediction and its limits

A line of best fit passes through the mean point (x̄, ȳ) and the middle of the scatter, with roughly equal numbers of points on each side. Use it to predict: a pupil who revised 6 hours — read up from 6 to the line, across to the score axis, say 68 marks. This is interpolation (inside the data range) and is reasonably safe.

Extrapolation — extending beyond the data — is risky: predicting a score for 20 hours' revision from data spanning 0–8 hours assumes the line continues; in reality, scores cap at 100 and exhaustion bites. GCSE loves asking 'why is this estimate unreliable?' — the answer is nearly always 'it is extrapolation; the trend may not continue'.

Worked example: a café plots temperature (°C) against iced-drink sales: at 15°C, 40 drinks; at 25°C, 90 drinks. The line through these suggests sales at 20°C ≈ (40 + 90)/2 = 65 drinks (midpoint of a straight trend). Predicting sales at 40°C from the same line (≈ 165) is extrapolation — no data exists beyond 27°C, and demand may level off. State the limitation whenever you extrapolate.

Estimating the gradient of a line of best fit gives a rate: 'each extra hour of revision is associated with about 5 more marks'. That slope interpretation — linking the graph back to the story — is what distinguishes a strong answer.

Least squares regression (intro)

The least squares regression line is the line that minimises the sum of the squared vertical distances (residuals) from the points. Its equation is y = a + bx with:

b = Sxy / Sxx,  where Sxy = Σxy − (Σx)(Σy)/n and Sxx = Σx² − (Σx)²/n
a = ȳ − b·x̄   (so the line always passes through (x̄, ȳ))

Worked example: a market stall records advertising spend x (£100s) and weekly sales y (£100s) for 5 weeks: (2, 10), (4, 14), (5, 20), (6, 22), (8, 29). Compute: Σx = 25, Σy = 95, so x̄ = 5, ȳ = 19. Σxy = 20 + 56 + 100 + 132 + 232 = 540; Σx² = 4 + 16 + 25 + 36 + 64 = 145. Then Sxy = 540 − (25 × 95)/5 = 540 − 475 = 65; Sxx = 145 − 625/5 = 145 − 125 = 20. So b = 65/20 = 3.25 and a = 19 − 3.25 × 5 = 2.75. The line is y = 2.75 + 3.25x: each extra £100 of advertising is associated with about £325 more sales. Prediction at x = 7: y = 2.75 + 22.75 = 25.5 → ≈ £2,550 of sales. (Full least-squares computation appears at A-level; at GCSE you meet the concepts through lines of best fit — this worked example is your bridge.)

Residuals — actual minus predicted — show where the model fails. For week 3: predicted = 2.75 + 3.25(5) = 19, actual 20, residual +1 (a slightly better week than the line suggests). Large residuals flag outliers worth investigating.

The correlation coefficient r

The product-moment correlation coefficient r measures linear fit on a scale from −1 to +1: r = Sxy / √(Sxx · Syy). For the stall data: Σy² = 100 + 196 + 400 + 484 + 841 = 2,021, so Syy = 2,021 − 95²/5 = 2,021 − 1,805 = 216. Then r = 65 / √(20 × 216) = 65 / √4,320 = 65 / 65.73 ≈ 0.99 — very strong positive correlation: the points hug the line.

Interpretation benchmarks: |r| above 0.8 ≈ strong; 0.4–0.8 ≈ moderate; below 0.4 ≈ weak. r says nothing about the slope's size (a steep line and a shallow line can both have r = 0.99) and nothing about causation. Note the consistency check: b and r always share the same sign, because b = r × (sy/sx).

Worked example: revision hours vs test marks for a class gives r = 0.87. Interpretation: a strong positive linear association — pupils who revised more tended to score higher. It does not prove revision caused the marks (motivated pupils may do both), and it does not mean 87% of marks are 'explained'. Precision in wording is the mark.

Worked example — regression by hand, small data

Four pupils record hours of sleep x and next-day test score y: (6, 52), (7, 61), (8, 66), (9, 77). Compute the line. Σx = 30, Σy = 256, so x̄ = 7.5, ȳ = 64. Σxy = 312 + 427 + 528 + 693 = 1,960; Σx² = 36 + 49 + 64 + 81 = 230. Sxy = 1,960 − (30 × 256)/4 = 1,960 − 1,920 = 40. Sxx = 230 − 900/4 = 230 − 225 = 5. So b = 40/5 = 8 and a = 64 − 8 × 7.5 = 4. The line is y = 4 + 8x: each extra hour of sleep is associated with 8 more marks. Predicted score after 7.5 hours: y = 4 + 60 = 64 = ȳ — as it must be, since the line passes through (x̄, ȳ). For r: Σy² = 2,704 + 3,721 + 4,356 + 5,929 = 16,710, so Syy = 16,710 − 256²/4 = 16,710 − 16,384 = 326. r = 40/√(5 × 326) = 40/√1,630 = 40/40.37 ≈ 0.99 — a very strong fit for these four tidy points (real classroom data is far messier).

Interpretation discipline: even r = 0.99 here does not prove sleep caused the marks — four pupils is a tiny sample, and organised pupils may both sleep and revise well. Regression quantifies association; causation needs controlled experiments. Hold both truths at once — that is the whole point of this topic.

Spearman's rank correlation — a first mention

When data is ranked rather than measured (judges' scores, league positions), Spearman's rank correlation replaces the values with their ranks and applies the same correlation idea; it detects monotonic relationships (consistently rising) even when they are not straight lines. You will meet it properly at A-level; for now, note the pattern: measurement → product-moment r; ranks → Spearman.

Common mistakes

• 'Correlation proves cause.' Never. Name a possible lurking variable when asked to critique a claim.
• Extrapolating silently. Predictions beyond the data range must be flagged as unreliable — the examiner is waiting for that sentence.
• Line of best fit through the origin or the first point. It should pass through (x̄, ȳ) and the middle of the scatter, not connect endpoints.
• Confusing r with the gradient. r = 0.99 does not mean 'y rises 0.99 per unit of x' — that is b. r is unit-free and bounded by ±1.
• Ignoring outliers. A single wild point drags the regression line; mention it, and consider whether it is an error.
• Describing correlation vaguely. Always give direction and strength: 'strong positive', not just 'positive'.

Exam Experience

Question 1. A scatter graph shows revision hours plotted against test marks for a class, with correlation coefficient r = 0.87. Describe the correlation, and explain what it does not prove. [2 marks]

Question 2. For a set of data, x̄ = 10, ȳ = 32, sx = 3, sy = 8 and r = 0.75. The regression line of y on x is y = a + bx. (a) Work out the values of a and b. (b) Use the line to predict y when x = 14. [3 marks]

Question 3. Data on hours of sunshine x and ice-cream sales y is collected for values of x between 2 and 9 hours. A line of best fit is used to predict sales when x = 40. Give two reasons why this prediction is unreliable. [3 marks; 1 + 2]

Mark scheme.

• Q1: B1 for 'strong positive correlation' (both strength and direction). B1 for the causation point: it does not prove that revision caused the higher marks — another factor (e.g. motivation or prior attainment) could explain the association.
• Q2: M1 for b = r × (sy/sx) = 0.75 × 8/3 = 2. M1 for a = ȳ − b·x̄ = 32 − 2 × 10 = 12, giving y = 12 + 2x. A1 for the prediction at x = 14: y = 12 + 28 = 40.
• Q3: B1 for extrapolation: x = 40 is far outside the data range 2–9. B1 for the trend not necessarily continuing (sales cannot rise for ever; the linear pattern may break down). B1 for the association itself being possibly driven by a third factor such as temperature/holidays rather than sunshine alone. (Any two distinct reasons; the first two are the expected pair.)