Lesson 19: Modeling with Functions: Regression and Real Data

Real data rarely arrives as a formula. Modelling with functions is the discipline of choosing a function family that fits data, fitting it well, and knowing when the fit stops meaning anything. This lesson covers the three workhorse families (linear, quadratic, exponential), the transformation trick that turns exponential and power fits into linear ones, and the interpretation skills — residuals, correlation, extrapolation — that separate modelling from curve-copying.

Choosing the family: differences and ratios

For equally spaced x-values, interrogate the y-values:

• Constant first differences → linear (y = mx + b)
• Constant second differences → quadratic
• Constant ratios → exponential (y = A·r^x)

Worked example. y-values at x = 0, 1, 2, 3, 4: 3, 6, 11, 18, 27. First differences: 3, 5, 7, 9 — not constant. Second differences: 2, 2, 2 — constant, so the model is quadratic; indeed y = x^2 + 2x + 3 (check x = 3: 9 + 6 + 3 = 18 ✓). A table with ratios 2, 6, 18, 54, 162 (each ×3) is exponential, y = 2·3^x. This diagnostic is the first step of every modelling task: family first, parameters second.

Linear regression: the line of best fit

For scattered (x, y) data, the least-squares line minimises the sum of squared vertical residuals. With means x̄, ȳ: slope b = Σ(x − x̄)(y − ȳ)/Σ(x − x̄)^2, intercept a = ȳ − b·x̄, and the line always passes through (x̄, ȳ). Worked example — study hours vs test score: (1, 52), (2, 58), (3, 65), (4, 71), (5, 79). Here x̄ = 3, ȳ = 65; Σ(x − x̄)(y − ȳ) = 26 + 7 + 0 + 6 + 28 = 67 and Σ(x − x̄)^2 = 10, so b = 6.7 and a = 65 − 6.7 × 3 = 44.9: score ≈ 44.9 + 6.7 × hours. The correlation r = 67/sqrt(10 × 450) ≈ 0.999 — a very strong positive linear association (r ∈ [−1, 1]; |r| near 1 is strong). The residual for x = 4 is actual − predicted = 71 − 71.7 = −0.7: the student scored 0.7 below the line. Residuals scattered randomly around 0 validate the model; a pattern in the residuals indicts it.

Linearisation: fitting exponentials and powers with lines

Least squares fits lines, so curves must be straightened first. Exponential y = A·e^(kt): take ln of both sides — ln y = ln A + kt, linear in t with slope k. Power law y = A·x^b: ln y = ln A + b ln x, linear in ln x. Worked example — cooling coffee. A drink at 90°C cools in a 22°C room; the *excess* temperature E = T − 22 at t = 0, 5, 10, 15, 20 min is 68, 52, 41, 32, 25. Taking logs: ln E = 4.220, 3.951, 3.714, 3.466, 3.219 — successive differences ≈ −0.26, −0.24, −0.25, −0.25, nearly constant, confirming exponential decay. Regression on (t, ln E) gives slope ≈ −0.0497 and intercept ≈ ln 67.4, so T(t) ≈ 22 + 67.4·e^(−0.0497t). Predictions: 89.4, 74.6, 63.0, 54.0, 46.9 against actual 90, 74, 63, 54, 47 — residuals under a degree. The model predicts T(30) ≈ 37.2°C. Newton's law of cooling is this exponential — the data, transformed, confessed it.

Extrapolation, correlation and causation

Three professional cautions. Extrapolation: the study-hours line predicts 44.9 + 6.7 × 20 = 179 for 20 hours — on a 100-point test. Models are local; outside the data range they expire silently. Interpolation between data points is far safer. Correlation ≠ causation: ice-cream sales correlate with drowning deaths (both driven by summer heat — a lurking variable). A high |r| certifies association, never mechanism. Choosing between close models: compare residuals and respect the mechanism — populations and cooling are exponential *by process*, not just by fit; when two families fit similarly, prefer the one whose mechanism matches the phenomenon, and say so.

Modelling: depreciation as an exponential

A car bought for $28,000 loses 12% of its value each year: V(t) = 28000(0.88)^t. When is it worth under $10,000? Solve (0.88)^t = 10000/28000 ≈ 0.357: t = ln(0.357)/ln(0.88) ≈ (−1.0296)/(−0.1278) ≈ 8.05 years. Notice the workflow is Lesson 3's, now wearing data clothes: fit (the 12% from market data), model (the exponential), solve (the logarithm). The same pipeline — choose, fit, predict, caveat — is the whole lesson, and it is the daily workflow of every data scientist.

Common mistakes

• Declaring a family from a graph's vibe. "Curved, so exponential" confuses quadratics, powers and exponentials. Run the differences/ratios test on the numbers — it is decisive for clean data and indicative for noisy data.
• Forgetting to transform back. After regressing ln E on t, the slope −0.0497 is the decay constant k and the intercept must be exponentiated (e^4.21 ≈ 67.4) to give the amplitude. Reporting "the model is ln E = 4.21 − 0.0497t" as the final answer leaves the job half done.
• Extrapolating cheerfully. A model fitted on t ∈ [0, 20] minutes says nothing reliable about t = 200. Flag the range every time you predict.
• Claiming causation from r. r ≈ 0.999 in the study example does not prove hours cause scores (motivated students both study more and score higher). Association is the claim; mechanism needs design, not regression.
• Ignoring residual patterns. If residuals curve (all positive at the ends, negative in the middle), the family is wrong even when r is high. Plot or at least list the residuals before trusting a fit.

Exam Experience

1. Data at equally spaced x-values are 2, 6, 18, 54, 162. The best model family is

• (A) linear
• (B) quadratic
• (C) exponential
• (D) logarithmic

Answer: (C). Ratios are constantly 3 while differences (4, 12, 36, 108) grow — the signature of y = 2·3^x. (B) fails because second differences (8, 24, 72) are not constant.

2. A correlation coefficient of r = −0.92 indicates

• (A) a weak negative linear association
• (B) a strong negative linear association
• (C) that x causes y to decrease
• (D) no association

Answer: (B). |r| = 0.92 is close to 1 (strong) and the sign is negative. (C) overclaims: correlation never certifies causation, however strong.

3. Using T(t) = 22 + 67.4·e^(−0.0497t) for the cooling coffee, the predicted temperature at t = 10 min is

• (A) 47°C
• (B) 54°C
• (C) 63°C
• (D) 75°C

Answer: (C). e^(−0.497) ≈ 0.608, so T ≈ 22 + 67.4 × 0.608 ≈ 22 + 41.0 = 63.0°C — matching the measured 63°C. (B) is the t = 15 prediction; (A) is t = 20: index slips in the exponent are the intended trap.

Constructed response. Five students report study hours x = 1, 2, 3, 4, 5 and scores y = 52, 58, 65, 71, 79. (a) Compute the least-squares line, showing the sums you use. (b) Predict the score for 6 hours of study, and compute the residual for the student who studied 4 hours. (c) Interpret the slope in context. (d) Explain why the line should not be used to predict the score for 20 hours of study.

Scoring guide. (a) 2 points: x̄ = 3, ȳ = 65; Σ(x − x̄)(y − ȳ) = 67 and Σ(x − x̄)^2 = 10 — 1 point; slope 6.7, intercept 44.9, line ŷ = 44.9 + 6.7x — 1 point. (b) 2 points: ŷ(6) = 44.9 + 40.2 = 85.1 — 1 point; residual at x = 4: 71 − (44.9 + 26.8) = 71 − 71.7 = −0.7 — 1 point. (c) 1 point: each additional study hour is associated with about 6.7 more points on average (association, not proven causation). (d) 1 point: 20 hours is far outside the data range [1, 5] (extrapolation), and the line predicts 179 on a test capped at 100 — the linear pattern cannot persist.

Going deeper: r-squared and Simpson's paradox

r², the coefficient of determination, squares the correlation into a variance statement: r² is the fraction of the variation in y that the model accounts for. For the study-hours data, r ≈ 0.9988 gives r² ≈ 0.9976 — the line explains about 99.8% of the variation in scores. For the cooling model, comparing residuals (all under 1°C on values spanning 43°C) gives the same verdict qualitatively. Report r² alongside any fit: it converts "strong correlation" into "this much of the story told," and the missing fraction names what the model does not know.

Simpson's paradox is the deepest caution in data work: a trend present in every subgroup can reverse in the aggregate. Two schools report pass rates by gender. School X: boys 90% (90 of 100), girls 80% (40 of 50) — overall 130/150 ≈ 86.7%. School Y: boys 92% (46 of 50), girls 82% (82 of 100) — overall 128/150 ≈ 85.3%. School Y is better for boys *and* better for girls, yet worse overall — because Y's cohort is weighted towards the group with lower pass rates at both schools. Nothing is miscounted; the aggregate mixes different compositions. The moral for modelling: always ask whether a lurking grouping variable is driving the comparison, disaggregate before concluding, and never let a single r or a single rate carry a decision about people.

Workshop: fitting a quadratic exactly, and a power law

Three points determine a quadratic. Fit y = ax^2 + bx + c through (0, 3), (1, 6), (2, 11): c = 3 from the first point; a + b = 3 from the second; 4a + 2b = 8, i.e. 2a + b = 4, from the third. Subtracting: a = 1, then b = 2 — the model is y = x^2 + 2x + 3, the very function Lesson 19's difference table pointed to. Exact fitting (as many points as parameters) and least squares (more points than parameters) are the same linear-algebra act at different sizes: systems of Lesson 14 underneath, modelling on top.

Power laws by log-log. Data (1, 3), (4, 6), (9, 9), (16, 12) — y doubles-ish as x quadruples: plot ln y against ln x and the slope is (ln 6 − ln 3)/(ln 4 − ln 1) = ln 2/ln 4 = 1/2, so y = 3x^(1/2) = 3·sqrt(x) ✓. The log-log slope *is* the power. Engineers read exponents off log-log plots exactly this way; when a log-log plot is straight, the phenomenon is a power law, and its exponent is one subtraction away.