Lesson 13: Statistical Sampling

Statistics begins before any calculation: with the data itself. How was it collected? Who was asked? What was left out? A-level statistics demands critical thinking about sampling — because a biased sample makes every subsequent calculation meaningless, however precise. This lesson covers populations and samples, random and non-random sampling methods, bias, and the large data set: the real UK weather, transport, and demographic data that A-level exam questions are built on. Learn to interrogate data before you compute with it.

Populations, samples, and bias

A population is the complete set of interest; a sample is the subset actually observed. A census examines the whole population (the UK Census, every ten years — expensive, slow, but complete); a survey samples it. We sample because censuses are impractical: testing every light bulb to destruction leaves none to sell. But a sample represents the population only if it is unbiased — otherwise conclusions are fiction with decimal places.

Bias is systematic distortion. Selection bias: surveying shopping-centre visitors about online shopping misses homebound internet users. Non-response bias: postal surveys get ~30% response, and responders differ from non-responders. Leading questions bias answers ('Do you agree the council's excellent recycling scheme...?'). Survivorship bias: studying successful businesses ignores the failed ones with the same strategies. The 1936 Literary Digest US election poll sampled 2.4 million people — and predicted the wrong winner, because its sample (car owners, telephone subscribers) skewed wealthy. Size does not cure bias.

Worked context: a Manchester school wants mean travel time. Sampling only pupils arriving before 8:30 am misses latecomers with the longest journeys. A good design defines the population (all 1,400 pupils), chooses a random method, and chases non-response. A-level questions ask you to 'criticise' sampling designs — hunt for who is excluded and who refuses.

Random sampling methods

Simple random sampling: every member has an equal chance — use random number generators against a numbered list. Example: sample 30 from 1,400 pupils: number them 1–1400, generate 30 random integers. Unbiased but can miss small subgroups entirely (a sample of 30 might contain no Year 13s). Systematic sampling: pick every kth member (k = population/sample size) from a random start — easy, but vulnerable to hidden periodicity (sampling every 7th person from a weekly roster always hits the same weekday).

Worked example (systematic sampling). A quality manager needs 50 items from a production run of 1,200. k = 1200/50 = 24: choose a random start between 1 and 24 (say 11), then take items 11, 35, 59, ... every 24th item. One random choice generates the whole sample — cheap and spread evenly through the run. The danger: if the production cycle repeats every 24 items (e.g. a 24-cavity mould with one faulty cavity), the sample hits the same cavity every time. Periodicity in the list is systematic sampling's Achilles heel.

Stratified sampling: divide into strata (year groups, gender), then sample proportionally within each. For 30 pupils with year-group sizes 300, 280, 260, 240, 320: sample 30 × (300/1400) ≈ 6, 6, 6, 5, 7. Guarantees representation of every stratum — the gold standard when strata differ (travel times vary hugely by year group as older pupils travel further). Cluster sampling: randomly select whole clusters (5 tutor groups of 30) — cheap when lists are unavailable, but clusters may be unrepresentative.

Quota sampling (non-random): interviewers fill quotas (15 boys, 15 girls) by choosing whom to approach — fast, but interviewer choice introduces bias (approaching friendly-looking people). Opportunity/convenience sampling: whoever is available — the weakest method, biased toward the accessible. Exam questions ask you to compare methods for a scenario: name the method, give one advantage, one disadvantage, and match the method to the context (stratified when subgroups matter, systematic when a list flows naturally).

The large data set — real UK data

A-level statistics is examined through the large data set: real measurements from UK weather stations (Camborne, Heathrow, Hurn, Leeming, Leuchars), covering daily max temperature, rainfall, sunshine, wind speed, and more across 1987 and 2015. Exam questions give extracts — you must know the variables, units, and quirks (e.g. 'tr' means trace rainfall, less than 0.05 mm; 'n/a' means missing).

Working with extracts: a question might give daily max temperatures at Hurn for May 2015 and ask for the mean, or compare rainfall distributions between Camborne 1987 and 2015. Strategy: (1) read the extract carefully — note units and missing values; (2) exclude 'n/a' and handle 'tr' as 0 (or note the assumption); (3) compute with clear working; (4) interpret geographically (coastal vs inland, north vs south). The data set rewards students who have explored it — download it, sort it, plot it before the exam.

Limitations to criticise: only 5 stations (not representative of the whole UK — no Northern Ireland, no mountains, no city centres); only two years (1987 notably stormy — the Great Storm of October 1987! — so 1987 data may be atypical); daily summaries hide hourly extremes. 'The data may not be representative because...' — complete that sentence with specifics for full critical marks.

Sampling distributions and practical design

Different random samples give different means — the sampling distribution describes this variation. Larger samples vary less: the standard error of the mean shrinks as 1/√n (quadrupling the sample halves the error). This is why polls of 1,000 have ±3% margins: it is the mathematics of √n, not magic. A-level does not derive sampling distributions fully, but hypothesis testing (Lesson 17) rests on them: 'is this sample mean surprising under the null hypothesis?'

Designing a study — exam-style: 'A researcher investigates whether A-level students in Leeds exercise more than GCSE students. Describe a sampling method.' Strong answer: stratified by key stage (KS4/KS5) with proportional allocation from school rolls; simple random sampling within strata via random number generator; anonymous questionnaire to reduce response bias; target n = 200 for reasonable precision; acknowledge non-response and propose follow-ups. Marks for: named method, how implemented, why suitable, one limitation.

Ethics and practicality: get consent, anonymise data (GDPR applies — the UK data protection law), avoid sensitive questions without justification, and consider cost vs precision. Under-sampling wastes the study; over-sampling wastes money. The 1/√n rule guides the trade-off: beyond n ≈ 1,000, gains are marginal for most purposes — which is why national polls use ~1,000–2,000 respondents.

Critical evaluation — reading statistics

Every published statistic deserves interrogation: who funded it? (A crisp company reporting '90% prefer our crisps' sampled its own customers.) What is the exact question? ('Do you support the government's education reforms?' vs 'Do you support investing in schools?' — framing shifts answers by double digits.) What is the base? ('Cases doubled!' — from 2 to 4.) Is it mean or median? (Mean house prices are dragged up by mansions; the median describes the typical home.)

Worked critique: 'A survey of 500 people outside London Euston station found 68% support HS2.' Problems: location bias (Euston is the HS2 terminus — travellers there benefit); time bias (commuters, not residents); non-response (rush-hour refusals); leading context (surveyed at the station named for the project). A better design: stratified random sample of UK adults by region, neutral question wording, reported with margin of error. A-level 'criticise' questions want 2–3 specific, explained flaws — vague 'it might be biased' earns little.

Designing a questionnaire

Data quality starts with the questions. Principles: keep questions short and unambiguous ('How many hours did you exercise last week?' beats 'Do you exercise regularly?'); avoid leading wording ('Why do you love our canteen?' assumes love); offer exhaustive, exclusive response categories (age bands 11–13, 14–16, 17–19 — no gaps, no overlaps); put sensitive questions late (income, health) after trust is built; and pilot the questionnaire on a small group first to catch confusion.

Question types: closed questions (tick-boxes, scales) give analysable quantitative data — Likert scales ('strongly agree' to 'strongly disagree') are standard; open questions ('Suggest improvements') give rich qualitative data but are slow to code. A-level studies use closed questions almost exclusively — examiners may ask you to 'write a suitable question' to measure something: it must be closed, neutral, and precisely worded. Example — measuring phone use: 'Yesterday, roughly how many hours did you spend on your phone? Tick one: 0–1 / 1–2 / 2–4 / more than 4.'

Bias-proofing: randomise option order where order might matter; avoid double-barrelled questions ('Do you support longer school days and shorter holidays?' — which one?); ensure 'prefer not to say' / 'not applicable' options so respondents are not forced into false answers; and guarantee anonymity for honest answers on sensitive topics. A biased questionnaire corrupts even a perfect sample — the instrument matters as much as the sampling frame.

Common mistakes

• 'The sample is big, so it must be representative.' The Literary Digest polled 2.4 million and still failed. Size shrinks random error; it does nothing to systematic bias. Examiners reward naming the bias, not the size.
• Choosing quota or opportunity sampling without criticism. If you propose a non-random method, you must state its bias risk in the same breath — or choose a random method instead.
• Stratified sample sizes not proportional. Strata of 300 and 150 from a sample of 30 need 20 and 10 from a population of 450, not 15 and 15. Show the proportional calculation.
• Vague criticism. 'The sample may be biased' earns nothing. 'Pupils arriving before 8:30 are sampled, excluding latecomers whose journeys are longest, so the mean travel time is underestimated' earns everything.
• Double-barrelled or leading questionnaire items. One idea per question, neutrally worded, with exhaustive non-overlapping categories and a piloted draft.

Exam Experience

Questions in the style of the A-level statistics paper. Attempt them before reading the mark scheme.

A sixth-form college with 1,200 students wants to estimate the mean weekly time students spend on homework. Year-group sizes are: Year 7: 240, Year 8: 200, Year 9: 200, Year 10: 160, Year 11: 160, Year 12: 120, Year 13: 120. A sample of 60 students is to be taken.

1. Explain why a stratified sample is more appropriate than taking 60 students from a single year group. [2]

2. Calculate the number of students to be sampled from each year group. [3]

3. Describe how to select the students within each year group so that the sample is random. [2]

Mark scheme.

• 1. Homework time is likely to vary by year group (older students typically do more), so restricting to one year group would bias the estimate (B1). Stratification guarantees every year group is represented in proportion to its size (B1).
• 2. Sample fraction = 60/1200 = 1/20. Year 7: 240/20 = 12; Year 8: 200/20 = 10; Year 9: 10 (M1) for the proportional method. Year 10: 8; Year 11: 8; Year 12: 6; Year 13: 6 (A1) (total 12 + 10 + 10 + 8 + 8 + 6 + 6 = 60 (A1) — the total check is worth doing explicitly).
• 3. Number the students in each year group from an alphabetical list/register (B1). Use a random number generator to select the required number of distinct students in each group, ignoring repeats and numbers outside the list (B1).