Introduction to Statistics

Statistics is the science of collecting, organizing, summarizing, and drawing conclusions from data. Every field that gathers evidence, from medicine to economics to sports, leans on it, because data almost never speaks for itself. Raw numbers are noisy and partial; statistics is the discipline that turns them into defensible claims. It splits into two branches.

Descriptive statistics summarizes what a dataset shows using tables, graphs, and numbers, without reaching beyond the data in hand. Inferential statistics uses a sample to make claims about the larger group it came from, always attaching a measure of uncertainty. This whole course is a journey from the first branch to the second: Modules 1 and 2 describe data, and Modules 3 through 5 use it to infer.

One dataset runs through this lesson. The Maple Street Clinic intake survey records four things about every patient who walks in: blood type, a pain rating from 1 to 10, body temperature in degrees Celsius, and annual income in dollars. The clinic has about 4,000 registered patients, and the survey covers 200 of them. Almost every distinction below can be demonstrated on those four columns, so keep them in mind as the definitions arrive.

Key idea: Statistics turns raw measurements into claims that can be defended, and the first move is naming exactly what kind of measurement you are holding.

In plain terms

Here is the whole lesson in ordinary language. Because you almost never get to measure everyone, you measure a few people and reason outward from them: the whole group is the population, and the few you actually measured are the sample. A number that describes the whole group is a parameter, and you will usually never learn its value, while a number computed from your few is a statistic that serves as your estimate of that parameter. Then you ask what kind of thing each column holds, since some columns merely sort people into groups such as blood type, while others hold genuine quantities that can be added and averaged, such as income. Getting these labels right is not a vocabulary exercise, because they decide which picture you may legitimately draw and which test you may legitimately run.

Population, sample, parameter, statistic

A population is the entire group we want to learn about, such as all registered voters in a country or every bottle a factory will ever fill. A sample is the smaller subset we actually measure, because reaching the whole population is usually impossible or wasteful. A number that describes the whole population is a parameter (almost always unknown), while a number computed from a sample is a statistic (which we use to estimate the parameter).

For example, the true average height of all adults is a parameter; the average height of 200 adults we measured is a statistic. A memory hook: parameter goes with population, and statistic goes with sample. Nearly every inference problem in this course is really the same shape: we see a statistic and want to say something trustworthy about the parameter we cannot see.

Worked example. The clinic's 4,000 registered patients have some true average annual income. That number is a parameter, and nobody will ever learn it, because the clinic cannot survey all 4,000 people. What the clinic does have is completed surveys. Suppose five of them report incomes of 30,000, 42,000, 38,000, 51,000, and 39,000 dollars. Their average is (30,000 + 42,000 + 38,000 + 51,000 + 39,000) / 5 = 200,000 / 5 = 40,000 dollars. That 40,000 is a statistic. It is a fact about those five patients, computed with certainty and open to checking. The parameter it estimates is almost certainly not exactly 40,000. Every method in the second half of this course exists to say how far from 40,000 the truth might reasonably sit.

Measuring everyone, which is called a census, sounds better than sampling, but it is usually so expensive and slow that the population has shifted before the count finishes, and in some settings it is impossible in principle, because a factory that tests whether a match lights cannot test every match without destroying its own product.

Key idea: A statistic is a number you can compute exactly; a parameter is a number you want but cannot see. Inference is the bridge between them.

Categorical versus quantitative variables

A variable is any characteristic that can differ from one individual to the next; the individuals (or cases) are the people or objects being measured. A categorical (qualitative) variable places each individual into a group, such as eye color, country, or whether a patient improved. A quantitative variable is a number you can meaningfully do arithmetic with, such as height, income, or reaction time.

Quantitative variables are further split into discrete (countable values with gaps between them, like the number of siblings or cars in a lot) and continuous (any value in a range, limited only by measuring precision, like weight or temperature). A quick test for "is this quantitative?" is to ask whether averaging the values makes sense. The average of 3 and 5 siblings (4) is meaningful; the "average" of ZIP codes 10001 and 90210 is nonsense even though both look like numbers.

Run that test on the clinic's four columns. Averaging blood types is meaningless, because A and O are names rather than amounts. Averaging incomes is clearly fine, and so is averaging temperatures. Averaging pain ratings is the interesting case, and the next section explains why it sits in a gray zone. Notice too that a variable's type belongs to the measurement rather than to the thing measured. Age recorded in years is quantitative, while the same age recorded as "child, adult, senior" is categorical. The recording decision, not the person, fixes the type.

Key idea: Ask whether averaging the values would mean anything. If it would not, the variable is categorical however numeric it looks.

The four levels of measurement

Beyond the categorical-quantitative split, statisticians recognize four levels of measurement, a ladder of increasing information first named by the psychologist Stanley Smith Stevens:

• Nominal: named categories with no order (blood type, favorite color, jersey number used only as a label). You can count how many fall in each group, but ordering them is arbitrary.
• Ordinal: ordered categories, but the gaps between them are not equal or not meaningful (small, medium, large; survey ratings from "strongly disagree" to "strongly agree"; race finishing places). You know first beat second, but not by how much.
• Interval: ordered numbers with equal gaps but no true zero (temperature in Celsius or Fahrenheit; calendar years). A ten-degree jump means the same amount anywhere on the scale, but zero degrees does not mean "no temperature," so ratios fail.
• Ratio: ordered numbers with equal gaps and a true zero, so ratios make sense (height, weight, income, time elapsed). Because zero means "none," 40 kg really is twice 20 kg.

The four levels form a ladder, and each rung adds one ability while keeping everything the rungs below allow. Nominal lets you count how many fall in each group. Ordinal adds a defensible ordering, which makes medians and percentiles available. Interval adds equal spacing, so differences and means become meaningful. Ratio adds a true zero, so ratios and percentage changes become legitimate. One practical rule follows directly: record data at the highest level you can afford, because a precise measurement can always be collapsed into coarse categories later, while detail you never collected can never be recovered.

Key idea: Each level licenses one more kind of arithmetic, so the level you record at sets a ceiling on the analysis you can ever do.

Worked example: sorting a survey

Imagine a health survey records, for each person: blood type, pain rating on a 1 to 10 scale, body temperature in Celsius, and annual income in dollars. Blood type is categorical, nominal. The pain rating is often treated as ordinal, since the distance from 3 to 4 need not equal the distance from 8 to 9. Temperature is quantitative, continuous, interval. Income is quantitative, continuous, ratio (someone earning 0 dollars truly earns nothing, and 80,000 dollars is twice 40,000). Naming each correctly is not busywork: it decides, for instance, that you may report a mean income but should report a median or mode for pain ratings.

Worked example: what the level lets you compute

Take ten pain ratings from the Maple Street survey: 2, 3, 3, 4, 4, 5, 6, 7, 8, 10. Their sum is 2 + 3 + 3 + 4 + 4 + 5 + 6 + 7 + 8 + 10 = 52, so the arithmetic mean is 52 / 10 = 5.2. The values are already sorted, and with ten of them the median is the average of the fifth and sixth, which is (4 + 5) / 2 = 4.5. Both numbers are easy to compute, but only one is easy to defend. The mean assumes that the step from 2 to 3 carries the same meaning as the step from 9 to 10, and on a subjective pain scale that assumption is doubtful. The median assumes only that the ratings can be placed in order, which is exactly what an ordinal scale guarantees. Reporting 4.5 therefore rests on weaker and safer assumptions than reporting 5.2.

Now contrast the other two numeric columns. If two patients report 40,000 and 80,000 dollars, the second earns exactly twice the first, and that claim is meaningful because zero dollars really does mean no income. Try the same move on temperature. A reading of 40 degrees Celsius is not twice as hot as 20 degrees, because 0 Celsius is the freezing point of water rather than the absence of heat. Differences still behave: 40 minus 20 is a genuine 20-degree gap. Only the ratio fails, and it fails for exactly one reason, the missing true zero.

Key idea: Interval data supports differences and means; ratio data supports those plus ratios. A missing true zero is the whole difference.

Why the type decides everything downstream

Getting the type right matters because it dictates which graphs and which summaries are legal, and which statistical test you will eventually run. You can find the average of a ratio variable like income, but averaging nominal codes for eye color is meaningless even though a spreadsheet will happily do it.

A histogram fits a quantitative variable; a bar chart fits a categorical one. A correlation needs two quantitative variables; a chi-square test handles two categorical ones. Throughout this course the very first question to ask about any dataset is: what kind of variable is this, and at what level is it measured? Answer that, and the right tool usually names itself.

Key idea: Naming the variable type is the first move in any analysis, because it narrows the legal tools down to a short list.

Where people get stuck

• Numerals versus numbers. ZIP codes, phone numbers, and jersey numbers are labels that happen to be written with digits. Software will average them without complaint, and the answer will be meaningless. The test is never what the values look like, only whether arithmetic on them means anything.
• Population versus sample. The population is whoever you want to describe, which is a decision you make rather than a fact about the data. The 200 clinic surveys are a sample when the question concerns all 4,000 patients, and a population when the question concerns only those 200 respondents.
• Parameter versus statistic. A parameter is fixed but unknown; a statistic is known but changes from sample to sample. Collapsing the two makes the logic of inference vanish, because inference exists precisely because they differ.
• Interval versus ratio. The only question that matters is whether zero means "none." Zero dollars means no income, so income is ratio. Zero degrees Celsius is just a temperature, so Celsius is interval, and the year 2000 is not twice the year 1000.

Recap

• Descriptive statistics summarizes the data in hand; inferential statistics uses a sample to make claims about a larger group, always with stated uncertainty.
• A population is the whole group of interest and a sample is the part you measure; a parameter describes the population and a statistic describes the sample.
• Categorical variables sort individuals into groups, while quantitative variables measure amounts and split into discrete counts and continuous measurements.
• The four levels of measurement are nominal, ordinal, interval, and ratio, and each licenses more arithmetic than the one before it.
• Variable type decides the legal graph, the legal summary, and the legal test, so it is the first question to ask about any dataset.

Sources

1. OpenStax. (2023). 1.1 Definitions of statistics, probability, and key terms. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 1.3 Frequency, frequency tables, and levels of measurement. In Introductory statistics 2e. Rice University. openstax.org
3. Pennsylvania State University. (n.d.). 1.1.1 Categorical and quantitative variables. In STAT 200: Elementary statistics. online.stat.psu.edu
4. Pennsylvania State University. (n.d.). 1.2 Samples and populations. In STAT 200: Elementary statistics. online.stat.psu.edu
5. NIST/SEMATECH. (n.d.). 1.1.1 What is EDA? In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
6. UCLA Office of Advanced Research Computing. (n.d.). Choosing the correct statistical test in SAS, Stata, SPSS and R. Statistical Methods and Data Analytics. stats.oarc.ucla.edu
7. Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103(2684), 677-680. find source ↗

Once you know a variable's type, the next step is to picture its distribution, which is the pattern of values it takes and how often each occurs. A good graph reveals in one glance what a column of numbers hides: where the data piles up, how far it spreads, and whether anything looks out of place. For a single quantitative variable, the workhorse graph is the histogram.

One dataset carries this lesson. The Riverside Commute Study asked 40 employees at a single office park how many minutes their one-way trip to work took that morning, and the answers ranged from a four-minute walk to a forty-seven-minute drive. Forty raw numbers in a spreadsheet column tell you almost nothing at a glance, but the same forty numbers grouped and drawn reveal their shape immediately, which is exactly the trade this lesson teaches you to make.

Key idea: A distribution is the pattern of values a variable takes together with how often each value occurs, and a graph is how you see that pattern rather than merely storing it.

In plain terms

The whole lesson is one habit. Take a column of numbers, chop the number line into equal slices, count how many values land in each slice, and draw one bar per slice with its height equal to the count. Because the horizontal axis is a genuine number line, the bars must touch, and the picture that results answers three questions at once: where the data piles up, how widely it spreads, and whether anything sits oddly far from the rest. Two decisions can change what you see, so both deserve conscious thought. The first is how wide to make the slices, since very wide slices flatten real structure while very narrow ones turn ordinary randomness into a jagged mess. The second is whether the variable deserves a histogram at all, because a variable made of named categories rather than measured amounts needs a bar chart with visible gaps instead.

Frequency and relative-frequency tables

To build a histogram you first group the data into equal-width intervals called bins (or classes) and count how many values fall in each. The count is the frequency; dividing by the total gives the relative frequency, a proportion between 0 and 1 that you can also read as a percent. Consider 40 values grouped into five bins:

Bin	Frequency	Relative frequency
0-9	5	0.125
10-19	8	0.200
20-29	12	0.300
30-39	10	0.250
40-49	5	0.125

The relative frequencies sum to 1.000, which is always a useful arithmetic check: if they do not, a value was miscounted or misfiled. Relative frequencies matter because they let you compare datasets of different sizes fairly. A bin holding 12 of 40 values (30%) and a bin holding 300 of 1000 values (also 30%) represent the same share even though the raw counts look nothing alike.

Worked example: building the table. The bins above are the Riverside commute times in minutes, and every relative frequency comes from the same division by the total of 40. The first bin holds 5 commuters, so its relative frequency is 5 / 40 = 0.125, which is 12.5 percent. The second holds 8, giving 8 / 40 = 0.200. The busiest bin holds 12, giving 12 / 40 = 0.300. The fourth holds 10, giving 10 / 40 = 0.250, and the last holds 5 again for another 0.125. Adding those five proportions gives 0.125 + 0.200 + 0.300 + 0.250 + 0.125 = 1.000 exactly, and adding the five counts gives 5 + 8 + 12 + 10 + 5 = 40, which is the number of commuters you started with. Run both checks every time, because a total that misses by one is almost always a value filed into the wrong bin.

A third column is often worth adding. The cumulative relative frequency accumulates the proportions from the bottom up, so here it runs 0.125, then 0.125 + 0.200 = 0.325, then 0.325 + 0.300 = 0.625, then 0.625 + 0.250 = 0.875, then 1.000. This column answers "at or below" questions directly, and reading it off tells you that about 62.5 percent of these commuters travel under thirty minutes, while the remaining 37.5 percent take longer than that.

Key idea: Frequencies count, relative frequencies express those counts as proportions of the whole, and the proportions must sum to 1, which gives you a free arithmetic check on your own bookkeeping.

From table to histogram

A histogram draws one bar per bin, with bar height equal to the frequency (or relative frequency). Because the variable is quantitative, the bars touch, showing that the horizontal axis is a continuous number line with no gaps between neighboring intervals.

The choice of bin width

Bin width is a real decision, not a detail. Too few, very wide bins smear the data into a featureless block and hide structure; too many, very narrow bins turn the histogram into a jagged comb where every bump is just noise. A common starting point is to aim for roughly 5 to 20 bins and adjust until the shape reads clearly. The same data can look quite different under different bin choices, so it is honest practice to try a couple of widths before trusting the picture.

Worked example: the same data, two bin widths. Suppose the Riverside times are regrouped into just two very wide bins, 0 to 24 and 25 to 49 minutes. The counts become 5 + 8 + 6 = 19 for the lower bin and 6 + 10 + 5 = 21 for the upper one, once the 20 to 29 bin is split evenly between them. Two bars of nearly equal height say almost nothing, because the mound in the middle has been smeared flat. Now swing to the other extreme and use ten narrow bins of width five, and the counts scatter into small numbers like 2, 3, 4, 4, and 5, where a bin holding three commuters and its neighbour holding five differ by an amount that random variation alone could easily produce. Neither picture is wrong, but the five-bin version at width ten shows the real structure with the least noise, which is why the usual advice is to aim for something like five to twenty bins and then look at the result critically.

Key idea: Bin width is an analyst's choice rather than a property of the data, so a responsible histogram is one whose shape survives a change in that choice.

Describing shape, center, and spread

When you describe a distribution, always mention three things: shape, center, and spread, plus any unusual points. Common shapes are symmetric (the left and right halves mirror each other, like the mound above), right-skewed (a long tail stretching toward high values, common for income, house prices, or wait times), and left-skewed (a long tail toward low values, as with exam scores on an easy test).

A distribution can also be uniform (roughly flat, every bin about equal) or have two clear peaks (bimodal), which often signals two different groups blended together, such as heights of a mixed-sex population. An outlier is a value sitting far from the rest and always deserves a second look: it may be a data-entry error, or the most interesting point in the set.

Worked example: describing the Riverside histogram in sentences. A complete description names all three features and then mentions anything unusual. Shape comes first: the bars rise from 5 to 8 to 12 and then fall back through 10 to 5, which is a single mound that is close to symmetric, with only a slight lean toward the higher values. Center comes second: the tallest bin covers 20 to 29 minutes, and the cumulative column showed that half the commuters sit below roughly the mid-twenties, so a typical trip is about twenty-five minutes. Spread comes third: the values run from under ten minutes to nearly fifty, so the full range is about forty minutes wide, and the middle bins hold most of the crowd. Finally, nothing sits detached from the rest, so this dataset reports no obvious outliers. Those four sentences say more about the commute than the forty raw numbers ever could.

Skew is worth one extra caution, because the name points the opposite way from most people's first instinct. The direction of skew is named for the tail, not for the pile. A right-skewed distribution has its bulk on the left and a thin tail stretching right, which is why incomes, house prices, hospital stays, and waiting times are almost always right-skewed: most values cluster low, and a small number of very large values trail off toward the high end.

Key idea: Describe every distribution with shape, center, and spread, then note outliers, and remember that skew is named for the direction of the tail rather than the direction of the pile.

Histogram versus bar chart

A histogram is for a quantitative variable, and its bars touch because the axis is a number line whose order cannot be changed. A bar chart is for a categorical variable: its bars have deliberate gaps, and the categories can be reordered freely (alphabetically, by size, however you like) without changing the meaning. Confusing the two is one of the most common early mistakes, and it is not merely cosmetic. Touching bars imply a continuous scale; if you draw pizza toppings with touching bars, you are falsely suggesting "pepperoni" and "mushroom" are adjacent points on a number line. Always check the variable type first, then pick the display.

Key idea: Touching bars claim a continuous number line underneath them, so a histogram is only honest when the variable really is quantitative.

Where people get stuck

• Reading the vertical axis as a value rather than a count. A tall bar does not mean large commute times; it means many commuters in that interval. The heights answer "how many," while the horizontal positions answer "how much."
• Naming skew backwards. Students see the tall bars on the left and call the distribution left-skewed. The label follows the tail, so bulk on the left with a long tail to the right is right-skewed, and the classic examples are income and waiting times.
• Trusting a single bin width. Any histogram is one of many possible pictures of the same numbers, so a bump that appears at one width and vanishes at another was never real structure, and redrawing at a second width costs seconds.
• Mixing up histograms and bar charts. Bar charts show categories, so their bars are separated and can be reordered freely. Histogram bins live on a number line, so their bars touch and their order is fixed by arithmetic rather than by preference.
• Treating an outlier as a mistake. A detached value deserves investigation rather than automatic deletion, since it may be a typing error, a genuinely unusual case, or the most informative observation in the entire dataset.

Recap

• A frequency table groups quantitative data into equal-width bins and counts them, while relative frequencies convert those counts to proportions that must sum to 1.
• A histogram draws one bar per bin with touching bars, because the horizontal axis is a continuous number line rather than a list of labels.
• Bin width is a judgment call: too wide hides structure, too narrow manufactures noise, and roughly five to twenty bins is a sensible starting range.
• Describe any distribution by shape, center, and spread, then note outliers, and name skew after the direction of the long tail.
• Histograms serve quantitative variables and bar charts serve categorical ones, so the variable type decides the display before any drawing begins.

Sources

1. OpenStax. (2023). 2.2 Histograms, frequency polygons, and time series graphs. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 2.6 Skewness and the mean, median, and mode. In Introductory statistics 2e. Rice University. openstax.org
3. NIST/SEMATECH. (n.d.). 1.3.3.14 Histogram. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
4. Pennsylvania State University. (n.d.). 2.2.1 Graphs: Dotplots and histograms. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Pennsylvania State University. (n.d.). 2.2.3 Shape. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Diez, D., Cetinkaya-Rundel, M., and Barr, C. D. (2019). OpenIntro statistics (4th ed.). OpenIntro. openintro.org
7. Khan Academy. (n.d.). Displaying and describing quantitative data. khanacademy.org

A boxplot (box-and-whisker plot) is a compact picture of a distribution built from five key numbers. Where a histogram shows the full shape of one variable, a boxplot strips the distribution down to its skeleton, which makes it especially good for two jobs: comparing several groups side by side and spotting outliers at a glance.

A single concrete dataset anchors the entire lesson. The Northside Clinic wait-time log records, for seven consecutive weekday afternoons, how many minutes elapsed between a patient's scheduled appointment and the moment the physician actually appeared. Sorted from smallest to largest, those seven observations are 3, 5, 7, 8, 12, 13, and 15 minutes. Every quantity introduced below is computed from exactly those numbers, so the arithmetic remains verifiable by hand throughout, and the resulting summary is compact enough to compare against another clinic's log later.

Key idea: A boxplot compresses an entire distribution into five carefully chosen positional summaries, which sacrifices detail in exchange for immediate comparability across groups.

In plain terms

The underlying procedure is entirely about position rather than magnitude. Arrange the observations in order, identify the value that splits them into two equal halves, and then identify the value that splits each of those halves again, which produces three cut points and therefore four quarters of equal size. Attaching the smallest and largest observations to those three cut points yields the five numbers that define the display. The rectangle drawn between the first and third cut points contains the middle half of the data, its width measures how tightly the central observations cluster, and the line inside it locates the typical case. Because the middle half is measured without reference to the extremes, a single unusually large observation can stretch the whiskers dramatically while leaving the box almost untouched, which is precisely the resistance that makes this summary valuable for skewed measurements such as waiting times, incomes, and hospital stays.

The five-number summary

The five-number summary is the minimum, first quartile, median, third quartile, and maximum. The median (Q2) is the middle value once the data is sorted. The first quartile Q1 is the median of the lower half of the data, and the third quartile Q3 is the median of the upper half. These three quartiles split the sorted data into four equal-sized quarters: a quarter of the data lies below Q1, half lies below the median, and three quarters lies below Q3. Together the five numbers tell you where the data starts, where its middle 50% sits, and where it ends.

Quartiles are a special case of percentiles, which describe the percentage of observations falling at or below a specified value, so the first quartile is simply the 25th percentile, the median is the 50th, and the third quartile is the 75th. Percentile language is worth internalising because it appears constantly in applied reporting, from paediatric growth charts to standardised admissions testing, and because it is inherently resistant to extreme observations. One warning belongs here: statistical software packages implement several slightly different quartile conventions, so a colleague's Q1 may differ from yours by a small amount on a small dataset. The convention used throughout this course, and in most introductory texts, excludes the median itself when the two halves are formed.

Key idea: Quartiles are positional summaries that report the proportion of observations lying below a value, which makes them insensitive to how extreme the extreme observations happen to be.

Worked example

Find the five-number summary of these 7 sorted values: 3, 5, 7, 8, 12, 13, 15.

• Minimum = 3, Maximum = 15.
• Median (Q2) = the 4th value = 8 (with 7 values, the middle one is position 4).
• Lower half (the values strictly below the median) is 3, 5, 7, so Q1 = 5.
• Upper half (the values strictly above the median) is 12, 13, 15, so Q3 = 13.

The interquartile range is IQR = Q3 - Q1 = 13 - 5 = 8. The IQR measures the spread of the middle 50% of the data, and because it ignores the extreme quarter on each end, it is resistant to outliers, unlike the full range. That resistance is exactly why the IQR pairs naturally with the median, another resistant measure, when a distribution is skewed.

Worked example: what resistance actually buys you. Suppose an eighth afternoon is recorded during which an emergency delayed the physician for 60 minutes, producing the sorted list 3, 5, 7, 8, 12, 13, 15, 60. The overall range immediately explodes from 15 - 3 = 12 minutes to 60 - 3 = 57 minutes, an almost fivefold increase produced by a single observation. Now recompute the quartiles under the same convention. With eight observations, the lower half consists of 3, 5, 7, and 8, whose median is (5 + 7) / 2 = 6, while the upper half consists of 12, 13, 15, and 60, whose median is (13 + 15) / 2 = 14. The interquartile range therefore becomes 14 - 6 = 8 minutes, which is identical to the original value. One catastrophic afternoon multiplied the range by nearly five and left the interquartile range completely unchanged, which is the entire practical argument for preferring resistant summaries whenever a distribution has a long tail.

Key idea: The range depends entirely on the two most extreme observations, whereas the interquartile range depends only on the central half, so the two respond very differently to a single unusual value.

The 1.5 x IQR rule for outliers

A value is flagged as a potential outlier if it falls below the lower fence Q1 - 1.5 x IQR or above the upper fence Q3 + 1.5 x IQR. Here the fences are 5 - 1.5(8) = 5 - 12 = -7 and 13 + 1.5(8) = 13 + 12 = 25. Since every value lies between -7 and 25, there are no outliers in this dataset. The 1.5 multiplier is a convention, not a law of nature: it is calibrated so that, for roughly bell-shaped data, only a tiny fraction of genuine values get flagged, so a flag is worth a second look rather than an automatic deletion.

Worked example: applying the fences to the eight-day log. Return to the extended dataset that includes the 60-minute afternoon, for which Q1 = 6, Q3 = 14, and IQR = 8. The lower fence is 6 - 1.5(8) = 6 - 12 = -6 minutes, and the upper fence is 14 + 1.5(8) = 14 + 12 = 26 minutes. Every observation from 3 through 15 lies comfortably inside that interval, but 60 exceeds the upper fence of 26 by a wide margin and is therefore flagged. Notice what the flag does and does not establish. It establishes that the observation is unusual relative to the central half of this particular distribution, and it establishes nothing whatever about whether the observation is erroneous. A clinic administrator reading this result should investigate the afternoon in question, because the correct response depends entirely on what caused the delay, and discarding a legitimate emergency would systematically understate how long patients actually wait.

Key idea: The 1.5 multiplier is a widely adopted convention rather than a mathematical necessity, and a flagged observation is an invitation to investigate rather than an instruction to delete.

Drawing and reading a boxplot

To draw the boxplot, mark Q1, the median, and Q3 to form the box, then draw whiskers out to the smallest and largest values that are not outliers. Any outliers are plotted as separate dots beyond the whiskers. A boxplot instantly shows center (the median line), spread (the box width for the middle half, and the whisker length for the rest), and skew: a longer whisker on one side, or a median pushed toward one end of the box, signals a tail in that direction.

Note what a boxplot hides, though. Because it reports only five numbers, it cannot show whether a distribution is bimodal; two very different shapes can produce identical boxplots, so for a single variable a histogram is often more revealing.

Comparing groups side by side

The real payoff of boxplots comes when you line several up on the same axis, one per group. Suppose you plot test scores for three class sections. In one glance you can compare their medians (which section is typically higher), their spreads (which section is more consistent, shown by a narrower box), and their outliers (which section has an unusually low or high student). This side-by-side comparison, clean and free of clutter, is a task histograms do poorly and boxplots do beautifully, which is why boxplots are a staple of exploratory data analysis.

Worked example: comparing two clinics. Suppose the Southside Clinic reports a five-number summary of 6, 9, 11, 13, and 17 minutes for its own seven afternoons, against Northside's 3, 5, 8, 13, and 15. Southside's median of 11 exceeds Northside's median of 8, so a typical Southside patient waits about three minutes longer. Southside's interquartile range is 13 - 9 = 4 minutes against Northside's 13 - 5 = 8 minutes, so Southside is substantially more consistent even though it is typically slower. That combination of observations supports a genuinely useful managerial conclusion: a patient who values predictability should prefer Southside, whereas a patient who simply wants the shortest expected wait should prefer Northside and accept greater variability. Two boxplots drawn on a shared axis communicate that entire comparison instantly.

Key idea: Comparing medians answers which group is typically higher, while comparing box widths answers which group is more consistent, and those two questions have genuinely independent answers.

Where people get stuck

• Assuming each section of the boxplot contains equally many observations. Every quarter contains 25 percent of the data by construction, so a narrow box section and a long whisker contain identical counts. Width communicates density rather than quantity, and a long whisker means those observations are spread out rather than numerous.
• Treating an outlier flag as a verdict. The fences implement an arbitrary convention calibrated so that genuinely bell-shaped data produces few flags. Deleting flagged observations without investigation biases the resulting summaries and is a documented source of misleading applied analyses.
• Expecting a boxplot to reveal a bimodal distribution. Because only five positional numbers are plotted, two profoundly different distributions can generate visually identical boxplots. When the underlying shape matters, a histogram or dotplot must accompany the boxplot.
• Confusing the interquartile range with the range. The range subtracts the minimum from the maximum and is destroyed by any single extreme observation, whereas the interquartile range subtracts Q1 from Q3 and ignores the extreme quarter at each end entirely.

Recap

• The five-number summary consists of the minimum, first quartile, median, third quartile, and maximum, and those three quartiles divide the sorted observations into four equally sized quarters.
• The interquartile range, computed as Q3 minus Q1, measures the spread of the central half and is resistant to extreme observations in a way the range is not.
• The conventional fences at Q1 minus 1.5 times the IQR and Q3 plus 1.5 times the IQR flag potential outliers for investigation rather than for deletion.
• Whiskers extend to the most extreme observations that are not flagged, and flagged observations are plotted individually beyond the whiskers.
• Boxplots aligned on a common axis compare centers, spreads, and outliers across groups efficiently, but they cannot reveal bimodality or exact frequencies.

Sources

1. OpenStax. (2023). 2.4 Box plots. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 2.3 Measures of the location of the data. In Introductory statistics 2e. Rice University. openstax.org
3. NIST/SEMATECH. (n.d.). 1.3.3.7 Box plot. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
4. NIST/SEMATECH. (n.d.). 1.3.5.17 Detection of outliers. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
5. Pennsylvania State University. (n.d.). 3.1 Single boxplot. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Pennsylvania State University. (n.d.). 3.2 Identifying outliers: IQR method. In STAT 200: Elementary statistics. online.stat.psu.edu
7. Tukey, J. W. (1977). Exploratory data analysis. Addison-Wesley. find source ↗

A measure of center reports a single typical value that stands in for a whole dataset. It answers the everyday question "what is a normal amount?" The three measures you need are the mean, the median, and the mode, and a surprising amount of statistical judgment comes down to knowing which one to report.

One small dataset threads through the lesson. The Bluebird Coffee order book records how many custom cakes were ordered on each of six consecutive shifts, and those counts were 4, 7, 7, 9, 10, and 3. Six numbers are few enough that every calculation below can be checked mentally, yet enough to demonstrate every distinction that matters, including what happens later when a single enormous wedding order arrives and distorts one summary while leaving another untouched.

Key idea: Center is not a single well-defined quantity but a question with three standard answers, and choosing among them is a substantive judgment rather than a formatting preference.

In plain terms

Three summaries compete for the title of typical value, and each answers a slightly different question. The mean answers what everyone would receive if the total were redistributed equally, which requires adding everything up and dividing by how many there are. The median answers which value sits exactly in the middle once the observations are placed in order, so it depends on position rather than on magnitude. The mode answers which value occurs most frequently, and it is the only one of the three that remains meaningful when the observations are names rather than numbers. The practical consequence is straightforward: because the mean incorporates every observation into a single sum, one extraordinary value can drag it a long way, whereas the median barely notices. That single fact explains why economists, epidemiologists, and property analysts habitually report medians for quantities such as income, hospital cost, and house price.

The mean

The mean (arithmetic average) is the sum of all values divided by how many there are. For a sample of n values, the sample mean is written x-bar. In symbols, x-bar = (sum of all x) / n. The mean is the balance point of the data: if you imagined the values as equal weights placed along a ruler, the mean is the spot where the ruler would balance. Every single value tugs on that balance point, which is both the mean's great strength (it uses all the information) and its weakness (one extreme value can drag it far).

Worked example. Find the mean of 4, 7, 7, 9, 10, 3. The sum is 4 + 7 + 7 + 9 + 10 + 3 = 40, and there are n = 6 values, so x-bar = 40 / 6 = 6.67 (rounded to two decimals).

Worked example: verifying the balance point. The claim that the mean balances the data can be checked arithmetically, because the deviations from the mean must sum to exactly zero. Subtract 6.67 from each Bluebird count in turn to obtain 4 - 6.67 = -2.67, 7 - 6.67 = 0.33, 7 - 6.67 = 0.33, 9 - 6.67 = 2.33, 10 - 6.67 = 3.33, and 3 - 6.67 = -3.67. The negative deviations total -2.67 + -3.67 = -6.34 and the positive deviations total 0.33 + 0.33 + 2.33 + 3.33 = 6.32, which agree to within two hundredths, the entire discrepancy coming from rounding 6.67. Working with the unrounded value of 40/6 would produce exactly zero. This property is not a curiosity; it is the reason the next lesson has to square the deviations before averaging them, since their plain average is guaranteed to be zero for every dataset in existence and therefore measures nothing.

Key idea: The mean is the unique point at which the positive and negative deviations exactly cancel, which is what makes it the balance point and also what makes it sensitive to every observation.

The median

The median is the middle value when the data is sorted. Sorting the same data gives 3, 4, 7, 7, 9, 10. With an even count of 6, the median is the average of the two middle values (positions 3 and 4), so (7 + 7) / 2 = 7. If the count is odd, the median is simply the single middle value, at position (n + 1) / 2. Half the data lies at or below the median and half at or above, which makes it the exact center of position rather than of magnitude.

Worked example: odd and even counts. Both cases are worth practising once. Drop the shift that produced 3 cakes, leaving the five sorted values 4, 7, 7, 9, 10. With n = 5 the median occupies position (5 + 1) / 2 = 3, so it is the third value, which is 7. Now restore the sixth shift, giving 3, 4, 7, 7, 9, 10. With n = 6 there is no single middle position, so the median averages positions 3 and 4, giving (7 + 7) / 2 = 7 once more. Notice that the median need not be a value that actually occurs in the dataset: had the two middle values been 7 and 9, the median would have been 8, a quantity nobody ever recorded.

Key idea: The median depends only on the ordering of the observations, so replacing the largest value with one ten times larger leaves it completely unchanged.

The mode

The mode is the value that occurs most often. In our data, 7 appears twice and every other value once, so the mode is 7. A dataset can have no mode (all values distinct), one mode (unimodal), or several (bimodal, trimodal, and so on). The mode is the only measure of center that also works for categorical data: the modal eye color or the most common blood type is a perfectly good "center" even though a mean or median of colors is meaningless.

Key idea: The mode is the only measure of center defined for nominal data, which is why survey reports of favourite brand, blood type, or transport mode always quote it.

Which one to trust

The median is resistant (robust): it barely moves when an extreme value is added, because it depends only on the order of the middle values, not their size. The mean is sensitive to outliers because every value enters the sum. Suppose one more value, 100, joined the data, giving seven values.

The sorted list becomes 3, 4, 7, 7, 9, 10, 100, so the median shifts only from 7 to 7 (the new middle value is still 7), while the mean leaps from 6.67 to 140 / 7 = 20. One point moved the mean by 13 units and the median not at all. This is why the median is preferred for skewed data such as incomes, home prices, or wait times, while the mean is ideal for roughly symmetric data with no wild outliers.

Key idea: Resistance is the technical name for insensitivity to extreme observations, and the median has it while the mean does not, which is the single most consequential difference between them.

Shape tells you which is larger

The relationship between the two measures is itself a clue to shape. In a right-skewed distribution the long high tail pulls the sensitive mean above the median. In a left-skewed distribution the mean is dragged below the median. In a symmetric distribution the mean and median roughly coincide. So if a report tells you the mean US household income is much higher than the median, you can infer without seeing the data that the income distribution is right-skewed, with a relatively small number of very high earners pulling the average up. This is exactly why economists usually quote the median income: it better reflects the typical household.

Worked example: reading a national income report. The United States Census Bureau publishes both figures every year, and the mean household income consistently exceeds the median by a wide margin. Suppose a report gives a median of about 80,000 dollars and a mean of about 114,000 dollars. Nothing has gone wrong, and neither number is an error. The gap is a direct consequence of shape: a comparatively small number of extremely high incomes enter the mean's sum at full weight while affecting the median's position hardly at all. A journalist writing that the typical household earns 114,000 dollars would be badly misleading readers, because far more than half of all households earn less than that. The general diagnostic is worth memorising in both directions: mean noticeably above median indicates right skew, mean noticeably below median indicates left skew, and rough equality indicates approximate symmetry.

Key idea: Comparing the mean with the median is a free diagnostic for skewness that requires no graph and no software, only the two numbers themselves.

Where people get stuck

• Forgetting to sort before finding the median. The raw Bluebird order 4, 7, 7, 9, 10, 3 has 7 and 9 in the middle positions, which would suggest 8. Sorting first gives 3, 4, 7, 7, 9, 10 and the correct median of 7. The median is defined on the ordered list, never on the recorded order.
• Reporting the mean for obviously skewed data. Incomes, house prices, hospital costs, insurance claims, and waiting times all have long right tails, so their means are pulled above the typical case. Reporting a mean there is not arithmetically wrong, but it answers a question most readers were not asking.
• Assuming the median ignores the extreme values entirely. It uses them to establish the ordering, but not their magnitudes. Replacing the largest Bluebird order of 10 with 10,000 leaves the median at 7 and moves the mean from 6.67 to roughly 1,671.
• Treating "no mode" as an error. When every value occurs exactly once the dataset simply has no mode, and when several values tie for the highest frequency it has several. Neither situation indicates a mistake in the data.
• Confusing the mode with the maximum. The mode is the most frequently occurring value, not the largest one. In the Bluebird data the mode is 7 while the maximum is 10.

Recap

• The mean is the sum of all values divided by the count, it acts as the balance point of the data, and the deviations from it always sum to zero.
• The median is the middle value of the sorted data, taken as the single central observation when the count is odd and as the average of the two central observations when it is even.
• The mode is the most frequently occurring value and is the only measure of center available for categorical data.
• The median is resistant to extreme observations while the mean is not, which is why skewed quantities such as income are normally summarised by the median.
• A mean well above the median signals right skew, a mean well below signals left skew, and near equality signals approximate symmetry.

Sources

1. OpenStax. (2023). 2.5 Measures of the center of the data. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 2.6 Skewness and the mean, median, and mode. In Introductory statistics 2e. Rice University. openstax.org
3. NIST/SEMATECH. (n.d.). 1.3.5.1 Measures of location. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
4. Pennsylvania State University. (n.d.). 2.2.4 Measures of central tendency. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Pennsylvania State University. (n.d.). 2.2.4.1 Skewness and central tendency. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Guzman, G., and Kollar, M. (2025). Income in the United States: 2024 (Report No. P60-286). U.S. Census Bureau. census.gov
7. Khan Academy. (n.d.). Summarizing quantitative data. khanacademy.org

Two datasets can share the same mean yet look completely different because their values are more or less spread out. Consider two work teams that both average 40 hours a week: one where everyone works close to 40, and one where half work 20 and half work 60. The center is identical; the experience of being on those teams is not. Measures of spread quantify that variability, and they matter just as much as center in describing data.

The anchor dataset is the Willow Park tutoring log, which records how many students attended on each of eight consecutive afternoons: 2, 4, 4, 4, 5, 5, 7, and 9. Every quantity computed below comes from those eight numbers, and every intermediate step is small enough to verify by hand, which matters because the standard deviation is the first formula in this course whose result cannot be guessed by inspection.

Key idea: Two distributions with identical centers can behave entirely differently, so a summary that reports center without reporting variability is incomplete and frequently misleading.

In plain terms

Variability means how far the observations typically sit from their own center, and the standard deviation is the standard way of expressing that distance. Computing it involves four mechanical steps that never change. First, find the mean. Second, subtract the mean from each observation to obtain that observation's deviation. Third, square every deviation, which serves two purposes simultaneously, since squaring eliminates the cancellation problem that would otherwise force the total to zero and additionally penalises large departures more heavily than small ones. Fourth, average the squared deviations and take the square root, which returns the answer to the units the data was measured in. The intermediate quantity before that final square root is called the variance, and it is indispensable mathematically but awkward to describe in words, because its units are the original units squared. Nobody has an intuition for a squared student or a squared dollar, which is why the standard deviation, not the variance, appears in reports.

Range

The range is the maximum minus the minimum. It is the simplest measure of spread and instantly interpretable, but it uses only the two most extreme values and ignores everything in between, so a single outlier can blow it up. For the two teams above, both might have a range of 40 (from 20 to 60), which shows how blunt the range can be. It is a useful quick summary but rarely the last word.

For the Willow Park log the range is 9 - 2 = 7 students, which is genuinely informative as far as it goes. Its weakness is that it discards the six observations in between entirely, so it cannot distinguish a dataset clustered tightly around 4 with two isolated extremes from a dataset spread evenly across the whole interval. A further and less obvious defect is that the range tends to grow with sample size, because collecting more observations gives more opportunities to encounter an unusually small or unusually large one. That dependence on n makes the range unsuitable for comparing the variability of samples of different sizes.

Key idea: The range uses only two observations and systematically increases with sample size, so it summarises spread crudely and cannot fairly compare groups of unequal size.

Variance and standard deviation

The standard deviation is the measure of spread that statistics is built on. It reports the typical distance of values from the mean. To find the sample variance (its square), follow four steps: (1) find the mean; (2) subtract the mean from each value to get deviations; (3) square each deviation; (4) average the squared deviations by dividing their sum by n - 1.

The standard deviation is then the square root of the variance, which returns the answer to the original units. Why square the deviations at all? Because the deviations always sum to exactly zero (the positives and negatives cancel at the balance point), so their plain average is useless. Squaring makes every term non-negative and additionally punishes large deviations more than small ones.

Why divide by n minus 1

Using n - 1 (the degrees of freedom) rather than n corrects a subtle bias. Because the deviations are taken from the sample mean rather than the unknown true mean, they are systematically a touch too small, and dividing by the smaller number n - 1 compensates, giving a sample variance that estimates the population variance without bias on average. This is called Bessel's correction. When you genuinely have the entire population, you divide by n instead, because there is no estimation happening.

Notation makes the distinction visible, and it is worth adopting immediately. A population variance is written sigma^2 and a population standard deviation sigma, both computed by dividing by N, and both are parameters that are ordinarily unknown. A sample variance is written s^2 and a sample standard deviation s, both computed by dividing by n - 1, and both are statistics calculated from data actually in hand. The Greek letter always signals the population and the Roman letter always signals the sample, a convention this course follows without exception.

Key idea: Dividing by n - 1 compensates for the fact that deviations measured from an estimated mean are systematically slightly too small, which is why the sample variance is an unbiased estimator of the population variance.

Worked example

Find the sample standard deviation of 2, 4, 4, 4, 5, 5, 7, 9.

Step 1, the mean: the sum is 2 + 4 + 4 + 4 + 5 + 5 + 7 + 9 = 40 and n = 8, so x-bar = 40 / 8 = 5.

Steps 2 and 3, deviations and their squares:

x	2	4	4	4	5	5	7	9
x - 5	-3	-1	-1	-1	0	0	2	4
(x - 5)^2	9	1	1	1	0	0	4	16

Step 4, sum the squared deviations: 9 + 1 + 1 + 1 + 0 + 0 + 4 + 16 = 32. The sample variance is s^2 = 32 / (8 - 1) = 32 / 7 = 4.57. The sample standard deviation is s = square root of 4.57 = 2.14 (to two decimals).

So values in this dataset sit about 2.14 units from the mean of 5, on average. (For reference, if these 8 numbers were an entire population you would divide by 8 instead of 7, giving a variance of 4.0 and a population standard deviation of exactly 2.0. The two answers are close, and they converge as n grows.)

Key idea: The Willow Park attendance typically departs from its mean of 5 by about 2.14 students, and that single number now describes the whole dataset's variability in the original units.

Standard deviation is not standard error

One confusion causes more damage in applied work than any other idea in this lesson, and it is worth confronting before it can take root. The standard deviation describes how much individual observations vary around their mean, and it is a property of the data itself. The standard error describes how much a computed statistic, most commonly the sample mean, would vary from sample to sample if the study were repeated, and it is a property of an estimation procedure rather than of any individual. The two answer entirely different questions, and they are not interchangeable.

The arithmetic makes the difference concrete. For Willow Park the sample standard deviation is 2.14 students with n = 8, so the standard error of the mean is 2.14 / sqrt(8) = 2.14 / 2.83 = 0.76 students. Those two numbers describe different things: 2.14 says that a randomly chosen afternoon typically differs from average by about two students, while 0.76 says that sample means built from repeated batches of eight afternoons would typically fall about three quarters of a student away from the true population mean. Because dividing by the square root of n always shrinks the number, the standard error is always the smaller of the two, and reporting it where the standard deviation belongs makes data look far more consistent than it is. Medical statisticians have complained about this substitution in published research for decades, precisely because it flatters results.

Key idea: Standard deviation describes the spread of observations; standard error describes the spread of an estimate. Confusing them systematically understates how variable the underlying data actually is.

Reading the standard deviation

A larger standard deviation means more spread. The standard deviation is always zero or positive, and it equals zero only when every value is identical (no spread at all). It carries the same units as the data (dollars, centimeters, hours), which is exactly why it is usually preferred over the variance for describing spread in plain language: a variance of 4 "square dollars" means nothing to a reader, but a standard deviation of 2 dollars does.

The standard deviation also feeds directly into the z-scores and the normal model of the next lesson, where "how many standard deviations from the mean" becomes the universal ruler for comparing any two values.

Where people get stuck

• Forgetting the final square root. The quantity 32 / 7 = 4.57 is the variance, measured in squared students. The standard deviation is its square root, 2.14. Reporting 4.57 as a standard deviation roughly doubles the apparent spread.
• Dividing by n when the data is a sample. Nearly all real datasets are samples, so n - 1 is the default. Dividing by 8 rather than 7 here gives 4.0 instead of 4.57, an understatement that grows more serious as the sample gets smaller.
• Averaging the raw deviations. Their sum is always exactly zero, so their average is always zero for every dataset. Squaring is what makes an average of distances possible at all.
• Confusing standard deviation with standard error. The first describes individual observations, the second describes an estimate. The standard error is always smaller because it divides by the square root of the sample size.
• Expecting the standard deviation to be resistant. It is built from squared distances to the mean, so a single extreme observation inflates it substantially. When a distribution is badly skewed, the interquartile range is the more honest companion to the median.

Recap

• The range is the maximum minus the minimum; it is instantly interpretable, uses only two observations, and tends to grow with sample size.
• The variance averages the squared deviations from the mean, and the standard deviation is its square root, which restores the original units.
• Sample calculations divide by n - 1 rather than n, a correction that removes the systematic downward bias created by measuring deviations from an estimated mean.
• Greek letters denote population parameters (sigma, sigma squared) and Roman letters denote sample statistics (s, s squared).
• Standard deviation measures the spread of individual observations, while standard error measures the sampling variability of a statistic, and the two must never be interchanged.

Sources

1. OpenStax. (2023). 2.7 Measures of the spread of the data. In Introductory statistics 2e. Rice University. openstax.org
2. NIST/SEMATECH. (n.d.). 1.3.5.6 Measures of scale. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
3. Pennsylvania State University. (n.d.). 2.2.5 Measures of spread. In STAT 200: Elementary statistics. online.stat.psu.edu
4. Pennsylvania State University. (n.d.). Lesson 1: Collecting and summarizing data. In STAT 500: Applied statistics. online.stat.psu.edu
5. Altman, D. G., and Bland, J. M. (2005). Standard deviations and standard errors. BMJ, 331(7521), 903. pmc.ncbi.nlm.nih.gov
6. Curran-Everett, D. (2008). Explorations in statistics: Standard deviations and standard errors. Advances in Physiology Education, 32(3), 203-208. pubmed.ncbi.nlm.nih.gov
7. Motulsky, H. J. (2015). Common misconceptions about data analysis and statistics. Pharmacology Research and Perspectives, 3(1), e00093. pmc.ncbi.nlm.nih.gov

A z-score restates a value as the number of standard deviations it sits above or below the mean. It is one of the most powerful ideas in the course because it puts values from completely different scales onto one common ruler, so a test score, a height, and a reaction time can all be compared fairly.

The anchor for this lesson is the Harbor High chemistry final, whose scores across the whole cohort have a mean of 70 points and a standard deviation of 8 points, and whose histogram is close enough to bell-shaped that the normal model is a reasonable approximation. Every calculation below refers to that examination, so the numbers accumulate into a single coherent picture rather than a sequence of disconnected exercises.

Key idea: Standardising converts a raw measurement into a position, expressed as a signed number of standard deviations from the mean, and positions from different scales are directly comparable in a way raw measurements never are.

In plain terms

The procedure has two steps and one important limitation. Subtract the mean from the value, which gives the distance from average in the original units, then divide that distance by the standard deviation, which re-expresses it in standard-deviation units and cancels the units entirely. The resulting number is positive above the mean, negative below it, and zero exactly at it. Because the units cancel, a chemistry score and a running time become comparable quantities, which is the whole point. The limitation concerns what happens next: a z-score can be computed for any distribution whatsoever, but translating that z-score into a percentile requires an additional assumption, namely that the distribution is approximately normal. When that assumption fails, the arithmetic still works and the percentile claim does not, which is one of the most common quiet errors in applied statistics.

The z-score formula

For a value x from a distribution with a given mean and standard deviation, z = (x - mean) / (standard deviation). Read it in two steps: the numerator is the deviation (how far x is from the mean, in original units), and dividing by the standard deviation rescales that distance into standard-deviation units. A positive z means the value is above the mean; a negative z means below; z = 0 means exactly at the mean. Z-scores are unitless, which is precisely what lets them cross between scales.

Worked example. Test scores have a mean of 70 and a standard deviation of 8. A student scores 86. Then z = (86 - 70) / 8 = 16 / 8 = 2.0. The score is 2 standard deviations above the mean. Another student scores 64, giving z = (64 - 70) / 8 = -6 / 8 = -0.75, three quarters of a standard deviation below the mean.

Worked example: running the formula backwards. The relationship rearranges to x = mean + z x (standard deviation), which converts a position back into a raw score. A student one standard deviation above average scored 70 + 1 x 8 = 78 points. A student at z = -2 scored 70 + (-2) x 8 = 70 - 16 = 54 points. A student at z = 1.5 scored 70 + 1.5 x 8 = 70 + 12 = 82 points. This reversed calculation is not an afterthought, because most practical questions arrive in exactly this form. Admissions offices, manufacturers, and clinicians all begin by choosing a percentile and then need the raw cutoff that corresponds to it, which means converting a position into a measurement rather than the other way round.

Key idea: Subtract the mean before dividing by the standard deviation, and remember the inverse formula, because half of all practical questions run from position back to raw value.

Comparing across scales

Z-scores shine when two things are measured differently. Suppose you earn 650 on an exam with mean 500 and standard deviation 100, so z = (650 - 500) / 100 = 1.5. A friend earns 86 on the test above (z = 2.0). Even though 650 is a much bigger raw number than 86, your friend's z-score of 2.0 beats your 1.5, so relative to each group, your friend performed better. Raw scores across different tests are apples and oranges; z-scores make them comparable.

Two cautions belong with that comparison. First, a negative z-score carries no stigma, because it reports a position rather than a grade. A golfer whose round has z = -2 is performing superbly, since low scores win golf. Second, a fair comparison requires that both groups be genuinely comparable reference populations. Comparing a z-score computed within a class of six against one computed within a national cohort of sixty thousand compares two very different things, and the arithmetic will not warn you.

Key idea: The sign of a z-score reports direction and its magnitude reports distance, but whether a large z is good news depends entirely on the variable and on which reference group produced the mean and standard deviation.

The normal distribution

The normal distribution is a symmetric, bell-shaped model that describes a remarkable range of natural measurements: heights, birth weights, measurement errors, and (thanks to the Central Limit Theorem in Module 3) the behavior of sample means. It is completely determined by two numbers, its mean (which locates the center) and its standard deviation (which sets the width). The special case with mean 0 and standard deviation 1 is the standard normal distribution, and its values are simply z-scores. When data is approximately normal, z-scores unlock precise statements about how common or rare a value is.

The word "normal" here is a technical name for a particular mathematical curve, and it carries no suggestion that other distributions are abnormal or defective. Normality is a modelling assumption about a dataset, and like every assumption it should be checked rather than presumed. Many measured quantities are approximately normal, particularly those assembled from many small independent influences, but a great many are emphatically not. Incomes, house prices, insurance claims, hospital lengths of stay, and waiting times are all strongly right-skewed, and applying bell-curve percentages to them produces confidently stated nonsense. A quick histogram is usually enough to decide, and formal procedures such as a normal probability plot exist when the decision is consequential.

Key idea: The normal distribution is a model chosen for a dataset rather than a property that data automatically possesses, so check the shape before importing any bell-curve percentage.

The 68-95-99.7 empirical rule

For a normal distribution, the empirical rule gives quick, memorable benchmarks:

• About 68% of values lie within 1 standard deviation of the mean (between z = -1 and z = +1).
• About 95% lie within 2 standard deviations.
• About 99.7% lie within 3 standard deviations.

Applying the rule. With mean 70 and standard deviation 8, about 68% of scores fall in 70 plus or minus 8, that is between 62 and 78. About 95% fall between 70 plus or minus 16, that is 54 to 86. A score of 86 (z = 2) therefore sits right at the edge of the middle 95%.

Worked example: slicing the rule into tails. Symmetry converts each of the three headline figures into several usable ones. If 68 percent of Harbor High scores fall between 62 and 78, then 32 percent fall outside that window, and symmetry splits those equally into 16 percent below 62 and 16 percent above 78. Likewise, since 95 percent fall between 54 and 86, the remaining 5 percent splits into 2.5 percent below 54 and 2.5 percent above 86. The same logic applied to the three-standard-deviation window leaves only about 0.15 percent in each tail beyond 46 and 94. So the probability that a randomly chosen student scored above 86 is roughly 0.025, and the more precise value from a standard normal table is 0.0228, which is close enough that the mental estimate is usually adequate.

Key idea: Each empirical-rule percentage becomes two tail percentages by subtracting from 100 and halving, and that one manoeuvre answers most bell-curve questions without any table at all.

From z-scores to percentiles

Because the normal curve is symmetric, the empirical rule also gives percentiles, the percentage of values at or below a point. Consider the score of 86 (z = 2). The middle 95% lies between z = -2 and z = +2, leaving 5% split evenly into the two tails, so about 2.5% sits above z = 2. That means about 97.5% of scores are at or below 86, placing it near the 97.5th percentile.

Similarly, a value exactly at the mean (z = 0) sits at the 50th percentile, and a value at z = -1 sits near the 16th percentile (since 68% is in the middle, 16% falls in each tail). For z-scores that are not whole numbers, statisticians read a standard normal table or use software, but the empirical rule handles the common benchmark cases in your head.

Worked example: setting a cutoff from a percentile. Suppose Harbor High wants to award distinction to the top 2.5 percent of the cohort. That percentile corresponds to z = 1.96, because the central 95 percent of a normal curve lies between -1.96 and +1.96, leaving 2.5 percent in the upper tail. Converting back to raw points gives x = 70 + 1.96 x 8 = 70 + 15.68 = 85.68, so the cutoff is a score of about 86. Three landmark values do most of this work in practice and are worth memorising: z = 1.28 marks the 90th percentile, z = 1.645 marks the 95th, and z = 1.96 marks the 97.5th. The last of those reappears constantly in the confidence intervals of Module 4.

Key idea: Forward questions run value to z-score to percentile, and reversed questions run percentile to z-score to value, with the z-scale serving as the shared middle step in both directions.

Where people get stuck

• Getting the order of operations wrong. Subtract the mean first, then divide the entire difference by the standard deviation. Dividing before subtracting, or dividing only part of the difference, produces a meaningless number.
• Dividing by the variance instead of the standard deviation. If a problem reports a variance of 64, take its square root to get a standard deviation of 8 before standardising. A z-score must be unitless, and points divided by points-squared is not.
• Applying the empirical rule to skewed data. The 68, 95, and 99.7 percentages are properties of the bell curve, not of data in general. For incomes or waiting times, the fraction within two standard deviations can be very different.
• Treating a negative z-score as a failure. The sign says which side of the mean a value falls on and nothing more. Whether below average is good depends entirely on the variable being measured.
• Reading the rule's percentages as exact. The true figures are 68.27, 95.45, and 99.73 percent. The rounded versions are excellent for estimates, while tables and software supply exact areas when precision matters.

Recap

• A z-score, computed as the value minus the mean divided by the standard deviation, expresses a measurement as a signed number of standard deviations from its own mean.
• Because z-scores are unitless, they permit fair comparison across different scales, provided the reference groups are genuinely comparable.
• A normal distribution is fully determined by its mean and standard deviation, and the standard normal has mean 0 and standard deviation 1.
• The empirical rule places about 68, 95, and 99.7 percent of a normal distribution within one, two, and three standard deviations of the mean, and symmetry converts those into tail percentages.
• Normality is an assumption to be checked, and percentile claims derived from z-scores are only valid when that assumption holds.

Sources

1. OpenStax. (2023). 6.1 The standard normal distribution. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 6.2 Using the normal distribution. In Introductory statistics 2e. Rice University. openstax.org
3. NIST/SEMATECH. (n.d.). 1.3.6.6.1 Normal distribution. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
4. NIST/SEMATECH. (n.d.). 6.5.1 What do we mean by "normal" data? In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
5. Pennsylvania State University. (n.d.). 2.2.7 The empirical rule. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Pennsylvania State University. (n.d.). 2.2.8 z-scores. In STAT 200: Elementary statistics. online.stat.psu.edu
7. Khan Academy. (n.d.). Modeling data distributions. khanacademy.org

Probability measures how likely an event is, on a scale from 0 (impossible) to 1 (certain). It is the mathematical language of uncertainty and the essential bridge from describing data to making inferences: every confidence interval and hypothesis test in the coming modules is ultimately a probability statement. Getting the basic rules right here pays off everywhere later.

The anchor for this lesson is game night at the Lakeside Community Center, where the equipment consists of one standard 52-card deck, one fair six-sided die, and one fair coin. Those three objects generate every calculation below, which keeps the arithmetic small enough to verify and makes the underlying structure visible rather than hidden behind an application.

Key idea: Probability assigns a number between 0 and 1 to an event, and the rules of this lesson describe how those numbers combine when events are joined by the words "not", "or", and "and".

In plain terms

Three connecting words drive almost every introductory probability question, and each has its own rule. The word "not" invokes the complement rule, which says that the probability an event fails to occur is one minus the probability that it occurs. The word "or" invokes the addition rule, which adds the individual probabilities and then subtracts the overlap, because outcomes belonging to both events would otherwise be counted twice. The word "and" invokes the multiplication rule, which multiplies the probabilities when the events do not influence one another, and multiplies by a conditional probability when they do. A fourth phrase deserves special attention because it appears constantly and looks harder than it is: whenever a question asks for the probability of "at least one" success, the fastest route is almost always to compute the probability of no successes at all and subtract that from one.

Basic definition and the sample space

The sample space is the set of all possible outcomes of a random process. An event is any subset of that space, some collection of outcomes we care about. When the outcomes are equally likely, the probability of an event A is P(A) = (number of outcomes in A) / (total number of outcomes).

Rolling a fair die, the sample space is {1, 2, 3, 4, 5, 6}, so P(rolling a 4) = 1 / 6. Drawing from a standard 52-card deck, P(a heart) = 13 / 52 = 1 / 4. Every probability obeys three basics: it lies between 0 and 1, the probability of the whole sample space is 1, and the probabilities of a complete set of non-overlapping outcomes add to 1.

Worked example: counting a sample space. Roll the Lakeside die twice and record the ordered pair of results. The first roll has 6 possible outcomes and the second has 6, so the sample space contains 6 x 6 = 36 equally likely ordered pairs. Now count the event "the two rolls sum to 7", which contains the pairs (1,6), (2,5), (3,4), (4,3), (5,2), and (6,1), a total of 6 outcomes. Therefore P(sum of 7) = 6 / 36 = 1 / 6, approximately 0.167. Compare that with the event "the two rolls sum to 12", which contains only the single pair (6,6), so P(sum of 12) = 1 / 36, approximately 0.028. A sum of 7 is six times as likely as a sum of 12, and the reason is visible in the counting rather than hidden in a formula.

Key idea: When outcomes are equally likely, every probability question reduces to two counts, and the difficulty lies entirely in counting the sample space correctly.

The complement rule

The complement of A, written "not A," is everything in the sample space in which A does not happen. Because A either happens or it does not, P(not A) = 1 - P(A). If the chance of rain is 0.30, the chance of no rain is 1 - 0.30 = 0.70. The complement rule is the go-to shortcut whenever a question contains the phrase "at least one," because "at least one" is the complement of the often much simpler event "none."

Worked example. Drawing one card from the Lakeside deck, the probability of a face card (jack, queen, or king) is 12 / 52 = 3 / 13, approximately 0.231. The probability of not drawing a face card is therefore 1 - 3/13 = 10/13, approximately 0.769. Verify it directly by counting: there are 52 - 12 = 40 non-face cards, and 40 / 52 = 10 / 13. The two routes agree, which is the point of the rule, since the subtraction is faster whenever the complement is easier to describe than the event itself.

Key idea: An event and its complement partition the sample space, so their probabilities always sum to exactly 1, which turns any hard "at least" question into an easier "none" question.

The addition rule

For the probability that A or B occurs (meaning at least one of them), P(A or B) = P(A) + P(B) - P(A and B). You subtract the overlap because outcomes counted in both A and B would otherwise be double-counted. Two events are mutually exclusive (disjoint) if they cannot both happen at once; then P(A and B) = 0 and the rule simplifies to P(A) + P(B).

Worked example. Draw one card. Find P(King or Heart). There are 4 kings and 13 hearts, but the king of hearts belongs to both, so it must not be counted twice: P(King or Heart) = 4/52 + 13/52 - 1/52 = 16/52 = 4/13, about 0.31.

Contrast that with a pair of events that genuinely cannot overlap. Drawing one card, the events "the card is a heart" and "the card is a spade" are mutually exclusive, because no card belongs to both suits. Here the overlap term is zero, so the rule collapses to P(heart or spade) = 13/52 + 13/52 = 26/52 = 1/2. The general lesson is that you should always check whether an overlap exists before deciding which version of the rule applies, since forgetting to subtract a real overlap inflates the answer, and subtracting an overlap that does not exist deflates it.

Key idea: Add the probabilities and subtract the overlap, remembering that the overlap term is zero only when the events genuinely cannot occur together.

The multiplication rule and independence

For the probability that A and B both occur, when the events are independent (one happening does not change the other's chance), P(A and B) = P(A) x P(B). Flipping a fair coin twice, P(two heads) = 0.5 x 0.5 = 0.25.

It is vital not to confuse independence with mutual exclusivity: mutually exclusive events cannot both happen (they are strongly dependent, in fact, since knowing one occurred tells you the other did not), while independent events simply do not influence each other's chances. When events are not independent, you need conditional probability, where P(A and B) = P(A) x P(B given A); drawing two cards without replacement is the classic case, because the first draw changes what remains.

Worked example: with and without replacement. Draw two cards from the Lakeside deck and ask for the probability that both are hearts. If the first card is returned and the deck reshuffled, the draws are independent and P(both hearts) = (13/52) x (13/52) = (1/4) x (1/4) = 1/16 = 0.0625. If the first card is kept, the second draw faces a 51-card deck containing only 12 hearts, so P(both hearts) = (13/52) x (12/51) = (1/4) x (12/51) = 12/204 = 1/17, approximately 0.0588. The second answer is smaller because removing a heart makes another heart slightly less likely. The gap is modest here, but in small populations it becomes decisive, and assuming independence when sampling without replacement from a small group is a genuine source of wrong answers.

Key idea: Independent events cannot influence each other, whereas mutually exclusive events influence each other absolutely, since knowing one occurred guarantees the other did not. The two terms describe opposite situations and are routinely confused.

Combining the rules

The real skill is recognizing which rule a problem needs, and many problems chain several together. "At least one head in two flips" is far easier by complement than by listing cases: P(at least one head) = 1 - P(no heads) = 1 - (0.5 x 0.5) = 1 - 0.25 = 0.75. The pattern generalizes: the probability of at least one success in n independent tries is 1 - (probability of failure)^n.

For example, the chance of rolling at least one six in four rolls of a fair die is 1 - (5/6)^4 = 1 - 0.482 = 0.518, a little better than even. Spotting that "at least one" invites the complement, and that "and" with independence invites multiplication, resolves the large majority of introductory probability questions.

Key idea: The probability of at least one success in n independent attempts equals 1 minus the failure probability raised to the power n, which is the single most reusable formula in this lesson.

Where people get stuck

• Confusing independent with mutually exclusive. Independent means one event carries no information about the other. Mutually exclusive means they cannot both happen, which is maximal dependence. Two mutually exclusive events with non-zero probabilities are never independent.
• Forgetting to subtract the overlap. Adding 4/52 and 13/52 without removing the king of hearts counts that single card twice and gives 17/52 instead of 16/52. Always ask whether any outcome satisfies both descriptions.
• Multiplying when sampling without replacement. If the first item is not returned, the second probability has a different denominator and often a different numerator. Two hearts in a row is 1/17, not 1/16.
• The gambler's fallacy. A fair coin that has landed heads five times running still has probability 0.5 of heads on the next flip. The coin has no memory, and independence means exactly that past outcomes carry no information about future ones.
• Enumerating unordered pairs when order matters. The 36-outcome sample space for two dice treats (2,5) and (5,2) as distinct, which is what makes the outcomes equally likely. Collapsing them to 21 unordered pairs destroys that equal likelihood and gives wrong answers.

Recap

• Probability runs from 0 to 1, and for equally likely outcomes it equals the count of favourable outcomes divided by the size of the sample space.
• The complement rule states that P(not A) equals 1 minus P(A), which is the standard shortcut for any question phrased as "at least one".
• The addition rule for "or" adds the two probabilities and subtracts the probability of both, and that final term vanishes only for mutually exclusive events.
• The multiplication rule for "and" multiplies the probabilities when the events are independent, and uses a conditional probability when they are not.
• Independence and mutual exclusivity are opposite conditions rather than synonyms, and distinguishing them correctly resolves most introductory errors.

Sources

1. OpenStax. (2023). 3.1 Terminology. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 3.2 Independent and mutually exclusive events. In Introductory statistics 2e. Rice University. openstax.org
3. OpenStax. (2023). 3.3 Two basic rules of probability. In Introductory statistics 2e. Rice University. openstax.org
4. Pennsylvania State University. (n.d.). 2.1.3.2.1 Disjoint and independent events. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Pennsylvania State University. (n.d.). 2.1.3.2.4 Complements. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Grinstead, C. M., and Snell, J. L. (1997). Introduction to probability (2nd rev. ed.). American Mathematical Society. math.dartmouth.edu
7. Khan Academy. (n.d.). Probability library. khanacademy.org

Inference works only if the sample fairly represents the population it comes from. This lesson covers two linked ideas: how to sample well so your data is trustworthy, and how a sample statistic such as the mean behaves when you imagine repeating the study many times. That second idea, the sampling distribution, is the conceptual heart of all inference.

The anchor is the Cedar County screen-time survey, a study attempting to estimate the average number of minutes county residents spend looking at a phone each day. Suppose that, unknown to anyone conducting the study, the true population standard deviation is 15 minutes. That figure is a parameter, so in reality nobody would know it, but pretending to know it here makes the behaviour of the sampling distribution visible in a way that would otherwise remain abstract.

Key idea: Inference depends on two separate guarantees, namely that the sample was selected without systematic favouritism and that the behaviour of the resulting statistic across hypothetical repeated samples is predictable.

In plain terms

Two distinct ideas share this lesson, and keeping them separate prevents most of the confusion that follows. The first idea concerns how you choose people, where the governing principle is that chance rather than convenience must do the choosing, because any selection procedure that systematically favours one kind of person produces an error that more data cannot repair. The second idea concerns what would happen if you repeated the entire study. Your sample produced one particular average, but a different random sample would have produced a slightly different one, and the pattern formed by all those hypothetical averages is called the sampling distribution. That pattern has three predictable properties: it centres on the true population value, it is narrower than the spread of individuals, and it becomes narrower still as the sample grows. Those three properties are what make it possible to attach a margin of error to a single study.

Sampling methods

A simple random sample gives every individual an equal chance of selection, and every group of the chosen size an equal chance too; it is the gold standard because it removes human choice from who gets picked.

Other probability sampling methods also use randomness in structured ways: stratified sampling divides the population into meaningful groups (strata) such as age brackets and randomly samples within each, guaranteeing representation of every group; cluster sampling randomly selects whole pre-existing groups (such as entire schools) and measures everyone in them, which is cheaper when the population is geographically spread; and systematic sampling takes every kth individual from a list after a random start. What all of these share is randomness, which is the only reliable protection against bias.

Stratified and cluster sampling are frequently confused because both begin by dividing the population into groups, yet their logic is opposite. Stratified sampling draws from every group and is chosen to improve precision, since guaranteeing representation of each stratum removes one whole source of sampling variability. Cluster sampling draws only some groups and is chosen to reduce cost, since sending interviewers to four Cedar County neighbourhoods is far cheaper than sending them to forty. That saving carries a penalty, because people within a single neighbourhood tend to resemble one another, so a cluster sample of a given size carries less information than a simple random sample of the same size. The memorable summary is that you stratify to buy precision and cluster to buy affordability.

Key idea: Stratified sampling takes some members from every group, cluster sampling takes all members from some groups, and only the first of those two improves precision.

Bias and its sources

Bias is a systematic tendency to over- or under-represent part of the population, and unlike random error it does not shrink as the sample grows. A convenience sample (just whoever is easy to reach) is the classic source: an online poll captures only people who visit that site and choose to respond.

Voluntary response bias, undercoverage (some groups left off the sampling list entirely), and nonresponse bias all push a sample away from the truth in ways more data cannot fix. The infamous 1936 Literary Digest poll predicted the wrong US presidential winner despite millions of responses, precisely because its list of car and telephone owners skewed wealthy. The lesson is stark: a small random sample beats a huge biased one.

Key idea: Bias is a systematic error in the selection procedure, so it does not shrink as the sample grows, and a self-selected sample of ten million estimates the self-selected group very precisely and the population not at all.

The idea of a sampling distribution

A single sample gives one value of a statistic, such as one sample mean x-bar. If you drew a fresh random sample, you would get a slightly different x-bar, and a third sample a different one again. The sampling distribution of the mean is the distribution of x-bar over all possible samples of a fixed size n from the population. It is a distribution not of raw data but of a statistic, and it has three properties that make inference possible:

• Its center equals the population mean. So x-bar is an unbiased estimator: across all possible samples, it neither systematically overshoots nor undershoots the target.
• Its spread is smaller than the spread of individual values, and it shrinks as n grows. Larger samples give more consistent estimates because averaging cancels out individual highs and lows.
• Its shape becomes more bell-shaped as n grows, whatever the shape of the population (the Central Limit Theorem, next lesson, makes this precise).

Worked example: three samples from the same county. Imagine Cedar County's true mean screen time is 200 minutes per day, a value no researcher will ever observe. A first random sample of 25 residents produces x-bar = 197 minutes. A second independent sample of 25 produces x-bar = 204. A third produces x-bar = 199. None of the three equals 200, none is wrong, and the variation among them is not a defect in the study but the raw material of inference. Averaging those three sample means gives (197 + 204 + 199) / 3 = 600 / 3 = 200, which in this deliberately tidy example lands on the population value and illustrates what unbiasedness means: across all possible repetitions the sample mean neither systematically overshoots nor systematically undershoots. Three samples do not prove that property, of course; they only make it visible. Continue that process for every possible sample of size 25 and the resulting collection of values is the sampling distribution of the mean.

Notice carefully what kind of object this is. A histogram of the 25 residents in one sample displays raw screen times, and its spread reflects how much individuals differ from one another. The sampling distribution displays sample means, and its spread reflects how much a study's answer would move if the study were repeated. Those are different distributions describing different things, and conflating them is the source of the standard deviation and standard error confusion introduced two lessons ago.

Key idea: A sampling distribution is a distribution of a statistic across hypothetical repeated samples, not a distribution of raw observations, and it is entirely theoretical since no researcher ever draws all possible samples.

Standard error of the mean

The standard deviation of the sampling distribution has a special name, the standard error, to distinguish it from the standard deviation of the raw data. For the sample mean it equals the population standard deviation divided by the square root of the sample size: standard error = (population standard deviation) / (square root of n).

Worked example. A population has standard deviation 15. For samples of size n = 25, the standard error of the mean is 15 / (square root of 25) = 15 / 5 = 3. Sample means will typically fall about 3 units from the true mean. Now quadruple the sample size to n = 100: the standard error becomes 15 / 10 = 1.5. Notice the pattern: quadrupling n only halved the standard error, because it is the square root of n in the denominator.

This square-root-of-n law is one of the most important facts in statistics. It explains why precision improves with bigger samples, but with diminishing returns: to cut your error in half you must quadruple your sample, and to cut it to a tenth you need a hundredfold sample. That trade-off drives the economics of every survey and experiment.

Worked example: sizing a study. Suppose Cedar County wants the standard error of its estimate to be no larger than 1.5 minutes, with the population standard deviation still taken as 15. Setting 15 / sqrt(n) = 1.5 and rearranging gives sqrt(n) = 15 / 1.5 = 10, so n = 100 residents. Tightening the requirement to 0.75 minutes gives sqrt(n) = 20 and therefore n = 400. Halving the target standard error quadrupled the required sample, exactly as the square-root law predicts. One further fact surprises most students: the population size never entered that calculation. Cedar County would need roughly the same 400 residents whether the county contains 50,000 people or five million, provided the population is large relative to the sample.

Key idea: The standard error of the mean equals the population standard deviation divided by the square root of n, so precision improves with sample size but only at a square-root rate, and the size of the population is essentially irrelevant.

Where people get stuck

• Believing a large sample fixes bias. The 1936 Literary Digest poll collected over two million responses and still called the election wrongly, because its list of car and telephone owners systematically over-represented wealthier households. Size amplifies precision, never accuracy.
• Confusing the sampling distribution with the data distribution. One displays individual observations and the other displays hypothetical sample means. The second is always narrower, and by a factor of the square root of the sample size.
• Expecting the sample mean to equal the population mean. An unbiased estimator is correct on average across all possible samples, not correct in any particular sample. Sampling variability is normal and expected.
• Mixing up stratified and cluster sampling. Stratified takes some members from every group and improves precision; cluster takes many members from a few groups and reduces cost at the expense of precision.
• Assuming population size drives sample size. The standard error formula contains n but not the population total, so a national survey and a county survey need comparable sample sizes for comparable precision.

Recap

• Probability sampling methods, including simple random, stratified, cluster, and systematic designs, all use chance to decide who is selected, which is the only reliable defence against bias.
• Bias is a systematic tendency to misrepresent the population, and unlike random error it does not diminish as the sample grows.
• The sampling distribution of the mean is the distribution of x-bar across all possible samples of a fixed size, and it is a distribution of a statistic rather than of raw data.
• That distribution centres on the population mean, is narrower than the population, and becomes more bell-shaped as n increases.
• The standard error of the mean equals sigma divided by sqrt(n), so quadrupling the sample size halves the standard error and the population total is essentially irrelevant.

Sources

1. OpenStax. (2023). 1.2 Data, sampling, and variation in data and sampling. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 7.1 The central limit theorem for sample means (averages). In Introductory statistics 2e. Rice University. openstax.org
3. Pennsylvania State University. (n.d.). 1.2.1 Sampling bias. In STAT 200: Elementary statistics. online.stat.psu.edu
4. Pennsylvania State University. (n.d.). 1.2.2 Sampling methods. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Pennsylvania State University. (n.d.). 4.1 Sampling distributions. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Pennsylvania State University. (n.d.). Lesson 4: Sampling distributions. In STAT 500: Applied statistics. online.stat.psu.edu
7. Squire, P. (1988). Why the 1936 Literary Digest poll failed. Public Opinion Quarterly, 52(1), 125-133. find source ↗

The Central Limit Theorem (CLT) is the reason so much of statistics leans on the normal curve, and it is arguably the single most important result in the entire subject. It describes the shape of the sampling distribution of the mean, and in doing so it licenses nearly every technique in the final two modules.

The anchor is the Mapleton Bakery loaf log. The bakery's sourdough loaves have a mean weight of 100 grams above the tin and a standard deviation of 15 grams, and individual loaf weights are noticeably right-skewed, because an occasional loaf rises far more than the rest while none can weigh less than nothing. That skewness is deliberate in this example, since the whole force of the theorem lies in what happens to averages drawn from a population that is not itself bell-shaped.

Key idea: The Central Limit Theorem describes the behaviour of sample means across hypothetical repeated samples, and it says nothing whatever about the shape of the population those samples came from.

In plain terms

Individual measurements can be distributed in any awkward way imaginable, but their averages behave far better than they do. Take a sample, compute its mean, and record that single number. Repeat the process conceptually for every possible sample of the same size, and the collection of recorded means forms a distribution whose shape approaches the familiar bell curve as the sample size grows, regardless of how lopsided the individual measurements were. Three separate facts are bundled into that statement, and separating them helps: the collection of means is centred on the population mean, its spread equals the population standard deviation divided by the square root of the sample size, and its shape tends toward normality. Only the third of those is genuinely surprising, and it is the reason a small number of standard techniques can be applied to an enormous variety of data.

Statement of the theorem

The CLT says: for a population with any mean and standard deviation, the sampling distribution of the sample mean x-bar becomes approximately normal as the sample size n grows, regardless of the shape of the population. Its center stays at the population mean, and its standard deviation is the standard error, (population standard deviation) divided by (square root of n). Three facts are bundled together here: where the distribution is centered, how wide it is, and, the new and remarkable part, what shape it takes.

The phrase "regardless of the shape" is what makes the theorem astonishing. Even if the population is heavily skewed, bimodal, or lumpy, the distribution of the average of a sample smooths out toward a symmetric bell as n increases. Individual values might be wildly non-normal, but their means behave themselves. A common rule of thumb is that n of at least 30 is usually enough for the approximation to be good; when the population is already fairly symmetric, much smaller samples suffice, and when it is extremely skewed, you may need more.

What the theorem does not say

This is the single most misremembered result in introductory statistics, and the misremembering takes a predictable form. The theorem does not say that a population becomes normal as you collect more data from it. Mapleton's loaf weights are right-skewed, and they remain exactly as right-skewed after 30 loaves, after 300, and after 30,000. Collecting more observations reveals the population's true shape more clearly; it never alters that shape. What becomes normal is a different object entirely, namely the distribution of the sample mean across repeated samples. A histogram of 1,000 individual loaves will look skewed, while a histogram of 1,000 averages of 25 loaves each will look strikingly bell-shaped, and both statements describe the same bakery.

Two further limits belong here. First, the theorem requires that the observations be independent and drawn from a distribution with a finite variance, so it does not rescue data with wildly heavy tails or observations that are strongly linked to one another. Second, the frequently quoted threshold of thirty observations is a working rule of thumb rather than any part of the mathematics. A symmetric population may need only a handful of observations for the approximation to be excellent, while a severely skewed one may need several hundred. Treat thirty as a starting presumption to be checked rather than as a guarantee.

Key idea: The population's shape never changes; only the sampling distribution of the mean becomes normal. Any sentence beginning "the data becomes normal as n increases" is a misstatement of the theorem.

Why it matters

The CLT lets us attach probabilities to sample means using the familiar normal model, which is the engine behind confidence intervals and hypothesis tests in the next module. Because x-bar is approximately normal, we can standardize it exactly as we standardized raw values, but using the standard error in place of the raw standard deviation: z = (x-bar - mean) / standard error. That one substitution is the whole trick that carries the normal machinery from single values to sample means.

Key idea: Standardising a sample mean uses the standard error in the denominator, whereas standardising an individual observation uses the standard deviation, and choosing the wrong one is the most consequential arithmetic error in this module.

Worked example

A population has mean 100 and standard deviation 15. You take a random sample of n = 25. What is the probability the sample mean exceeds 106?

First, the standard error is 15 / (square root of 25) = 15 / 5 = 3. By the CLT, x-bar is approximately normal with mean 100 and standard error 3. Standardize the value 106: z = (106 - 100) / 3 = 6 / 3 = 2.0.

From the empirical rule, only about 2.5% of a normal distribution lies beyond z = 2. So the probability that the sample mean exceeds 106 is about 0.025. Note carefully that we used the standard error (3), not the raw standard deviation (15), because we are asking about an average of 25 values, not a single individual. Had we asked about one individual exceeding 106, we would have used 15 and found z = 0.4, a completely different and much larger probability. Confusing the two is the most common CLT error.

Worked example: the same question asked two ways. Return to Mapleton, where loaves average 100 grams with a standard deviation of 15 grams. Question one asks for the probability that a single randomly chosen loaf exceeds 106 grams. Because this concerns one loaf, standardise with the standard deviation: z = (106 - 100) / 15 = 6 / 15 = 0.4. A z-score of 0.4 is entirely ordinary, and roughly 34 percent of loaves exceed it. Question two asks for the probability that a crate of 25 loaves averages more than 106 grams. Because this concerns an average, standardise with the standard error: 15 / sqrt(25) = 15 / 5 = 3, giving z = (106 - 100) / 3 = 2.0, which corresponds to roughly 2.5 percent. The same threshold of 106 grams is unremarkable for one loaf and distinctly unusual for an average of twenty-five, because averaging cancels out individual highs and lows. Ask yourself every time whether the question concerns one thing or the average of many things.

Key idea: Averages vary less than individuals, so a value that is ordinary for a single observation can be extraordinary as a sample mean, and the two questions require different denominators.

A second look at the role of n

It is worth separating the two things that n controls. As n grows, the sampling distribution gets narrower (the standard error shrinks like the square root of n) and more normal in shape. These are distinct effects. Even for a fixed, small n, a normal population already produces a perfectly normal sampling distribution; the CLT's contribution is specifically the shape improvement for non-normal populations. Understanding this keeps you from over- or under-trusting the normal approximation: with a symmetric population you can lean on it at modest n, while with a severely skewed population you should be cautious until n is comfortably large.

It is also worth separating the Central Limit Theorem from its frequent companion, the Law of Large Numbers. The Law of Large Numbers says that a single sample mean converges toward the population mean as that one sample grows, so it concerns accuracy. The Central Limit Theorem describes the shape and spread of the distribution formed by many hypothetical sample means, so it concerns the pattern of the fluctuations around that target. One promises that you will get closer; the other describes the geometry of how you miss.

Key idea: Growing n does two separable things, narrowing the sampling distribution at a square-root rate and improving its normality, and the second effect is the theorem's distinctive contribution.

Where people get stuck

• Believing the population becomes normal. It does not, ever. Mapleton's loaves stay right-skewed no matter how many are weighed. Only the distribution of the sample mean approaches normality.
• Using the standard deviation where the standard error belongs. A question about one loaf uses 15; a question about the average of 25 loaves uses 15 / 5 = 3. The two produce z-scores of 0.4 and 2.0 for the identical threshold.
• Treating n = 30 as a law. It is a rule of thumb. Symmetric populations often need far fewer observations, and severely skewed ones can need far more, so the threshold should be checked rather than assumed.
• Forgetting the finite-variance and independence conditions. The theorem applies to independent observations from a distribution with finite variance. Strongly dependent observations or extremely heavy-tailed data can violate it.
• Confusing the theorem with the Law of Large Numbers. One concerns a single mean homing in on the target, the other concerns the shape of the distribution of many hypothetical means.

Recap

• The Central Limit Theorem states that the sampling distribution of the sample mean approaches a normal shape as n grows, whatever the shape of the population.
• That sampling distribution is centred at the population mean and has spread equal to the standard error, sigma divided by sqrt(n).
• The theorem describes sample means, not raw data, so the population's own shape is unaffected by collecting more observations.
• Its conditions are independent observations from a distribution with finite variance, and the n of at least 30 guideline is a rule of thumb rather than part of the mathematics.
• Because x-bar is approximately normal, it can be standardised with z = (x-bar - mu) / standard error, which is the engine behind confidence intervals and hypothesis tests.

Sources

1. OpenStax. (2023). 7.1 The central limit theorem for sample means (averages). In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 7.2 The central limit theorem for sums. In Introductory statistics 2e. Rice University. openstax.org
3. OpenStax. (2023). 7.3 Using the central limit theorem. In Introductory statistics 2e. Rice University. openstax.org
4. Pennsylvania State University. (n.d.). 7.4 Central limit theorem. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Encyclopedia of Mathematics. (n.d.). Central limit theorem. European Mathematical Society. encyclopediaofmath.org
6. Weisstein, E. W. (n.d.). Central limit theorem. MathWorld: A Wolfram web resource. mathworld.wolfram.com
7. Kunin, D., Guo, J., Devlin, T., and Xiang, D. (n.d.). Seeing theory: Probability distributions. Brown University. seeing-theory.brown.edu

A single sample mean is a point estimate: our single best guess for the population mean. But a point estimate is almost never exactly right, and reporting it alone hides how uncertain it is. A confidence interval fixes this by reporting a whole range of plausible values, together with a stated level of confidence that the range captures the true parameter. It is the honest way to answer "what is the population mean?" because it reports both an estimate and its uncertainty in one statement.

The anchor is the Brookfield Library visit log. The library wants to know how long a typical visitor stays, so it times a random sample of 36 visitors and obtains a sample mean of 50 minutes. Suppose, for the sake of a clean first calculation, that the population standard deviation is known to be 12 minutes. That assumption is unrealistic and will be relaxed in the next lesson, but it isolates the logic of interval estimation from the additional complication of estimating variability.

Key idea: A confidence interval reports a range of parameter values consistent with the observed data, and the stated confidence level describes the reliability of the procedure that produced it.

In plain terms

Every confidence interval for a mean has the same three-part anatomy. Start at your best single guess, which is the sample mean. Measure how much that guess would bounce around if the study were repeated, which is the standard error. Then decide how cautious you wish to be, expressed as a multiplier applied to that bounce, and step out that far in both directions. Higher confidence demands a larger multiplier and therefore a wider interval, which is the fundamental trade-off of the whole method: you can be more certain of catching the truth, or you can be more precise about where it lies, but not both at once from the same data. The only escape from that trade-off is a larger sample, which shrinks the standard error and narrows the interval at every confidence level simultaneously.

Structure of a confidence interval

Every confidence interval for a mean has the same form: point estimate plus or minus margin of error, where margin of error = (critical value) x (standard error). The point estimate centers the interval, the standard error measures how much the estimate bounces from sample to sample (from Module 3), and the critical value scales that bounce up to the confidence level you want. When the population standard deviation is known and the sampling distribution is normal, the critical value is a z-value read from the standard normal distribution. The three most common levels are:

Confidence level	Critical z-value
90%	1.645
95%	1.96
99%	2.576

These critical values come straight from the empirical rule and the normal curve: 95% of a normal distribution lies within 1.96 standard errors of center, which is exactly why 1.96 builds a 95% interval.

Key idea: The critical value converts a desired confidence level into a number of standard errors, so 1.96 is not a magic constant but simply the point beyond which 2.5 percent of a normal curve lies in each tail.

Worked example

A sample of n = 36 has a mean of x-bar = 50, and the population standard deviation is 12. Build a 95% confidence interval for the population mean.

• Standard error = 12 / (square root of 36) = 12 / 6 = 2.
• Critical value for 95% confidence is z = 1.96.
• Margin of error = 1.96 x 2 = 3.92.
• Interval = 50 plus or minus 3.92 = (46.08, 53.92).

We are 95% confident the true population mean lies between 46.08 and 53.92.

Every step of that calculation deserves a sentence of interpretation, because the arithmetic is easy and the meaning is not. The standard error of 2 minutes says that if Brookfield repeated this study with a fresh sample of 36 visitors, the new sample mean would typically land within about 2 minutes of this one. The critical value of 1.96 says that the interval will stretch out 1.96 such typical distances in each direction. The margin of error of 3.92 minutes is the resulting half-width, and it is the number a newspaper would report as "plus or minus about four minutes". The interval itself, from 46.08 to 53.92 minutes, is the set of population means that would not be contradicted by this sample at the 95 percent level.

Key idea: Margin of error equals the critical value times the standard error, and the interval is always the point estimate plus or minus that margin.

What "95% confident" really means

This is the most misunderstood idea in the course, so read it slowly. The confidence level describes the procedure, not one particular interval. If we repeated the sampling many times and built a 95% interval from each fresh sample, about 95% of those many intervals would contain the true mean and about 5% would miss it.

For the one interval we actually computed, (46.08, 53.92), the true mean is either inside it or it is not; there is no probability left to assign, because nothing about that fixed interval or the fixed parameter is random anymore. The 95% lives in the long-run success rate of the method, not in any single result. The tempting but wrong reading, "there is a 95% probability the mean is between 46.08 and 53.92," misplaces the randomness, treating a fixed unknown constant as if it were random.

A concrete picture helps. Imagine Brookfield ran the study one hundred times, each time drawing a fresh random sample of 36 visitors and constructing a 95 percent interval. Roughly 95 of those hundred intervals would contain the true mean, and roughly 5 would miss it entirely, and no one examining any single interval could tell which kind it was. The confidence level is a property of that manufacturing process, in the same way that a claim about a factory's defect rate describes the factory rather than the particular item in your hand.

Because this is the most reliably misunderstood idea in the subject, it is worth listing the readings that are wrong, all of which have been documented in surveys of researchers and students alike. It is wrong to say there is a 95 percent probability that the population mean lies between 46.08 and 53.92, since neither the fixed interval nor the fixed parameter is random once computed. It is wrong to say 95 percent of library visitors stay between 46.08 and 53.92 minutes, since the interval estimates a mean and not the spread of individuals. It is wrong to say 95 percent of future sample means will fall inside this interval, which is a different and considerably narrower claim. And it is wrong to say a wider interval indicates a more reliable study, since width reflects uncertainty rather than quality.

Key idea: Confidence describes the long-run coverage of the procedure across repeated samples; it is not the probability that this particular interval contains the parameter.

Width, confidence, and sample size

Two levers change the margin of error, and they pull against each other. Raising the confidence level (say to 99%) uses a bigger critical value, which widens the interval: more confidence costs precision, because to be surer of catching the mean you must cast a wider net. Increasing the sample size shrinks the standard error, which narrows the interval at any confidence level: more data buys precision.

For the same data at 90% confidence, the margin would be 1.645 x 2 = 3.29, giving the narrower interval (46.71, 53.29); at 99% it would be 2.576 x 2 = 5.15, giving the wider (44.85, 55.15). There is always a trade-off between how confident and how precise you want to be, and because the standard error falls only with the square root of n, buying a much narrower interval at a fixed confidence level requires a great deal more data.

Worked example: how many visitors would Brookfield need? Suppose the library wants a 95 percent interval with a margin of error no larger than 2 minutes rather than 3.92. Set 1.96 x (12 / sqrt(n)) = 2. Dividing both sides by 1.96 gives 12 / sqrt(n) = 1.0204, so sqrt(n) = 12 / 1.0204 = 11.76 and n = 138.3, which rounds up to 139 visitors. Compare that with the original 36. Cutting the margin of error roughly in half required nearly quadrupling the sample, which is the square-root law reappearing in a practical budget decision. Sample sizes are always rounded upward, because rounding down would leave the achieved margin slightly larger than the target.

Key idea: Confidence level and precision pull against each other for fixed data, and only additional observations improve both at once, at a cost that grows with the square of the precision demanded.

Where people get stuck

• Assigning probability to a computed interval. Once the numbers are in, the interval either contains the parameter or it does not. The 95 percent lives in the procedure's long-run success rate, not in the single result.
• Reading the interval as a range for individuals. The Brookfield interval estimates the mean visit length, not the visits themselves. Individual visitors vary far more widely than the interval suggests.
• Believing a wider interval is better. Width measures uncertainty. A 99 percent interval is wider than a 95 percent interval computed from identical data, and it is not thereby a stronger finding.
• Forgetting that the interval assumes valid sampling. The formula quantifies random sampling error only. If the sample was biased, the interval is centred in the wrong place and its stated coverage is fiction.
• Concluding that two overlapping intervals mean no difference. Overlap is suggestive but not decisive, and the correct procedure is a direct comparison of the two groups rather than an eyeball check of their separate intervals.

Recap

• A confidence interval takes the form point estimate plus or minus margin of error, where the margin equals a critical value times the standard error.
• Common critical values for a normal model are 1.645, 1.96, and 2.576 for 90, 95, and 99 percent confidence respectively.
• Confidence describes the long-run coverage of the procedure across hypothetical repeated samples, not the probability that a specific computed interval contains the parameter.
• Raising the confidence level widens the interval, while raising the sample size narrows it, and only the second improves precision without sacrificing certainty.
• Because the standard error falls with the square root of n, halving the margin of error requires roughly quadrupling the sample.

Sources

1. OpenStax. (2023). 8.1 A single population mean using the normal distribution. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 8.2 A single population mean using the Student t distribution. In Introductory statistics 2e. Rice University. openstax.org
3. NIST/SEMATECH. (n.d.). 1.3.5.2 Confidence limits for the mean. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
4. Pennsylvania State University. (n.d.). 4.2.1 Interpreting confidence intervals. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Pennsylvania State University. (n.d.). Lesson 5: Confidence intervals. In STAT 500: Applied statistics. online.stat.psu.edu
6. Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., and Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350. pmc.ncbi.nlm.nih.gov
7. Hoekstra, R., Morey, R. D., Rouder, J. N., and Wagenmakers, E.-J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin and Review, 21(5), 1157-1164. pubmed.ncbi.nlm.nih.gov

A hypothesis test is a formal procedure for deciding whether sample data provides convincing evidence against a claim about a population. The logic mirrors a courtroom: we presume innocence (a claim of "no effect") and reject it only when the evidence is strong enough that innocence becomes implausible. Understanding this structure is what separates using statistics from merely computing it.

The anchor is the Northwind bottling line, which is supposed to fill each bottle to a mean of 100 millilitres. Quality control suspects the machine has drifted in one direction or the other, so it measures 25 bottles and records a sample mean of 105 millilitres. The population standard deviation of the fill process is known from long experience to be 15 millilitres. Those four numbers generate every calculation in this lesson.

Key idea: A hypothesis test asks how surprising the observed data would be if a specified claim about the population were true, and answers with a probability rather than a verdict about that claim.

In plain terms

The procedure has a fixed shape that never varies. Begin by writing down a default claim that you are prepared to abandon only under pressure, which is almost always a statement of no effect or no difference. Collect data, then compute how far the data sits from what that default claim predicted, measured in standard errors. Convert that distance into a probability, which answers the question: supposing the default claim were true, how often would data at least this extreme appear? A small probability means the data would be strange under the default claim, which counts as evidence against it. A large probability means the data is unremarkable under the default claim, which is not evidence for it, merely an absence of evidence against it. That asymmetry is the part students most often lose, and it is the part that matters most.

The two hypotheses

Every test pits two claims against each other. The null hypothesis (H0) is the default, skeptical position, usually "no difference" or "no effect," and it is always stated with an equals sign, for example H0: mean = 100. The alternative hypothesis (Ha) is what we suspect might be true instead, such as Ha: mean is not equal to 100 (a two-sided alternative), or a one-sided version like Ha: mean is greater than 100. A crucial asymmetry: we never try to prove H0. We either find enough evidence to reject it or we do not, just as a jury returns "guilty" or "not guilty," never "proven innocent."

The alternative is what you set out to demonstrate, so it should be chosen before seeing the data.

Test statistic and p-value

We compress the sample into a single test statistic that measures how far the data falls from what H0 predicts, expressed in standard-error units (so it is essentially a z-score for the sample statistic). From it we compute the p-value: the probability of getting a result at least as extreme as the one observed, assuming H0 is true.

Read that definition carefully, because every word matters. The p-value is computed in a hypothetical world where the null holds, and it asks how surprising our data would be there. A small p-value means the data would be very unlikely if H0 were true, which counts as evidence against H0. A large p-value means the data is unremarkable under H0, so H0 survives.

What a p-value is not

In 2016 the American Statistical Association took the unusual step of issuing a formal statement on p-values, because misinterpretation had become widespread enough to distort published research across many fields. Three specific errors are worth naming and rejecting explicitly, since intro students who absorb them tend to carry them for years.

First, a p-value is not the probability that the null hypothesis is true. It is computed by assuming the null hypothesis is true, so it cannot simultaneously be evidence about whether that assumption holds. A p-value of 0.03 does not mean there is a 3 percent chance the null is correct, and it does not mean there is a 97 percent chance the alternative is correct. Second, a p-value is not the probability that the results are due to chance. That phrase, which appears constantly in press coverage, again reverses the conditioning: the p-value is the probability of data like yours given a chance-only world, not the probability of a chance-only world given your data. Third, a p-value is not one minus the probability that the alternative hypothesis is true, which is simply the first error stated in a different arrangement.

What a p-value actually is remains narrow and useful: the probability, computed under the null hypothesis and the model's other assumptions, of obtaining a result at least as extreme as the one observed. The ASA's accompanying principles add that p-values do not measure the size or importance of an effect, that scientific conclusions should not rest on whether a p-value crosses a threshold, and that a p-value reported without the analysis choices that produced it is close to uninterpretable.

Key idea: A p-value is a statement about data given a hypothesis, never a statement about a hypothesis given data, and every common misreading consists of swapping those two around.

The decision rule

Before looking at data we fix a significance level alpha, most commonly 0.05, which is the threshold of surprise we require. Then:

• If the p-value is less than or equal to alpha, we reject H0; the result is called "statistically significant."
• If the p-value is greater than alpha, we fail to reject H0; there is not enough evidence.

We deliberately say "fail to reject," never "accept," because absence of evidence is not evidence of absence. Not convicting a defendant does not prove them innocent; it only means the case was not strong enough.

The practical consequence is worth stating bluntly, because it is routinely ignored in published work. A non-significant result is not evidence that the effect is zero. It is compatible with a genuinely zero effect, with a small effect the study lacked the power to detect, and with a substantial effect that this particular sample happened to miss. Reporting "no difference was found" and reporting "there is no difference" are different claims, and only the first is licensed by a large p-value. The standard corrective is to inspect the confidence interval alongside the test, because an interval running from a trivial decrease to a large increase makes the residual uncertainty visible in a way that a bare p-value conceals.

Key idea: Failing to reject the null hypothesis means the evidence was insufficient, not that the null is true, and absence of evidence is not evidence of absence.

Worked example

A machine is supposed to fill bottles to a mean of 100 ml, and we suspect it is off in either direction. Set H0: mean = 100 versus Ha: mean is not equal to 100. Suppose the population standard deviation is 15 ml, we sample n = 25 bottles, and the sample mean is x-bar = 105. The standard error is 15 / 5 = 3, so the test statistic is z = (105 - 100) / 3 = 5 / 3 = 1.67.

For a two-sided test at alpha = 0.05, the critical values are plus or minus 1.96, marking off the most extreme 5% split between the two tails. Since 1.67 falls inside that range (equivalently, its two-sided p-value is about 0.095, which exceeds 0.05), we fail to reject H0. The data does not give convincing evidence the machine is miscalibrated, even though 105 differs from 100, because a gap that size is not surprising for a sample of 25.

Trace where that p-value of about 0.095 came from, since the arithmetic is short. A z-statistic of 1.67 has roughly 4.75 percent of the standard normal curve beyond it in the upper tail. The alternative is two-sided, meaning a deviation of 5 millilitres in either direction would have been equally interesting, so the equally extreme result on the low side must be counted as well: 2 x 0.0475 = 0.095. Read that number carefully. It says that if the machine really were calibrated to 100 millilitres, samples of 25 bottles would produce a mean at least 5 millilitres away from target about 9.5 percent of the time. It does not say there is a 9.5 percent chance the machine is correctly calibrated.

Key idea: A two-sided p-value doubles the one-tail area because deviations in either direction count as equally extreme, and the sidedness must be chosen before the data is seen.

Two kinds of error, and the meaning of significance

Because we decide under uncertainty, two mistakes are possible, and no procedure can eliminate both. A Type I error is rejecting a true H0 (a false alarm, convicting the innocent); its probability is exactly alpha, which is why we set alpha small. A Type II error is failing to reject a false H0 (a missed effect, acquitting the guilty); its probability is called beta, and 1 - beta is the test's power.

The two trade off: lowering alpha guards against false alarms but makes misses more likely, unless you compensate with a larger sample. One final caution about the word "significant": it means only that an effect is detectable, not that it is large or important. With a huge sample, a trivially small, practically meaningless difference can be highly statistically significant. Statistical significance answers "is there an effect?"; it does not answer "does the effect matter?", which requires looking at the effect's actual size.

Worked example: significance without importance. Suppose Northwind installs a second line and tests 10,000 bottles, obtaining a sample mean of 100.4 millilitres with the same population standard deviation of 15. The standard error is 15 / sqrt(10000) = 15 / 100 = 0.15, so the test statistic is z = (100.4 - 100) / 0.15 = 0.4 / 0.15 = 2.67, giving a two-sided p-value of about 0.008. That result is comfortably significant at alpha = 0.05, and the effect it detects is a drift of four tenths of a millilitre, roughly eight drops of water, which no customer could perceive and no regulator would care about. The reverse case is equally instructive: a genuinely important effect measured on a sample of 8 bottles may produce a p-value well above 0.05 simply because the standard error is large. Significance and importance are separate questions, and only reporting the effect size and its confidence interval answers the second.

Key idea: Statistical significance depends on effect size and sample size together, so a large enough study will detect trivial effects and a small enough study will miss important ones.

Where people get stuck

• Reading the p-value as the probability the null is true. It is computed by assuming the null is true, so it cannot measure that assumption's credibility. A p-value of 0.03 says nothing about a 3 percent chance of anything concerning the hypothesis.
• Saying "the results are due to chance" with probability p. This reverses the conditioning. The p-value gives the probability of extreme data given a chance-only world, not the probability of a chance-only world given the data.
• Treating "fail to reject" as "accept". A large p-value is compatible with a zero effect, with a small effect the study could not detect, and with a real effect this sample happened to miss. Report the confidence interval to make that ambiguity visible.
• Treating 0.05 as a natural boundary. Nothing distinguishes 0.049 from 0.051 scientifically. The threshold is a convention, and the ASA explicitly warns against letting it substitute for reasoning.
• Equating significance with importance. With 10,000 bottles a drift of 0.4 millilitres is significant and irrelevant. Always ask how large the estimated effect is before asking whether it cleared a threshold.
• Choosing the alternative after seeing the data. Switching to a one-sided test because the result nearly reached significance doubles the real Type I error rate and invalidates the stated alpha.

Recap

• The null hypothesis states no effect or no difference and is written with an equals sign; the alternative states what you suspect instead and must be chosen before seeing the data.
• The test statistic measures how far the data falls from the null prediction in standard-error units, and the p-value converts that distance into a probability computed under the null.
• A p-value is not the probability the null is true, not the probability the result is due to chance, and not one minus the probability the alternative is true.
• Rejecting H0 when it is true is a Type I error with probability alpha; failing to reject a false H0 is a Type II error with probability beta, and power equals 1 - beta.
• Statistical significance answers whether an effect is detectable, while practical significance requires the effect size and its confidence interval.

Sources

1. Wasserstein, R. L., and Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129-133. American Statistical Association. amstat.org
2. OpenStax. (2023). 9.1 Null and alternative hypotheses. In Introductory statistics 2e. Rice University. openstax.org
3. OpenStax. (2023). 9.2 Outcomes and the Type I and Type II errors. In Introductory statistics 2e. Rice University. openstax.org
4. Pennsylvania State University. (n.d.). 5.4 p-values. In STAT 200: Elementary statistics. online.stat.psu.edu
5. Pennsylvania State University. (n.d.). 6.4 Practical significance. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Amrhein, V., Greenland, S., and McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567(7748), 305-307. pubmed.ncbi.nlm.nih.gov
7. Altman, D. G., and Bland, J. M. (1995). Absence of evidence is not evidence of absence. BMJ, 311(7003), 485. pmc.ncbi.nlm.nih.gov
8. Sullivan, G. M., and Feinn, R. (2012). Using effect size - or why the P value is not enough. Journal of Graduate Medical Education, 4(3), 279-282. pmc.ncbi.nlm.nih.gov

In real studies we almost never know the population standard deviation; if we do not even know the mean we are testing, we surely do not know its spread. We estimate the spread with the sample standard deviation s, and that extra layer of estimation means the z-distribution is no longer exactly right. The fix, one of the most practical tools in statistics, is the t-distribution.

The anchor is the Ridgeway typing course, which claims to raise a standardised typing score whose long-run benchmark is 50 points. A sample of 25 graduates produces a mean of 52 points with a sample standard deviation of 8 points. Nobody knows the population standard deviation of typing scores, which is exactly the situation this lesson exists to handle, and every calculation below refers to those figures.

Key idea: Replacing an unknown population standard deviation with a sample estimate introduces additional uncertainty, and the t-distribution is the reference curve that accounts for it honestly.

In plain terms

Everything you learned about hypothesis testing carries over unchanged, and only one component is swapped. Previously the standard error used the population standard deviation, a number you were told. Now it uses the sample standard deviation, a number you estimated from the same small sample that produced the mean, so it wobbles from study to study just as the mean does. Because two quantities are now uncertain rather than one, extreme test statistics arise a little more often than a normal curve would predict, and the reference distribution must therefore have slightly heavier tails. That heavier-tailed curve is the t-distribution, its exact shape is governed by a quantity called the degrees of freedom, and as the sample grows the estimate becomes reliable, the tails thin, and the curve converges on the familiar normal. The practical consequence is that small samples require a larger test statistic to reach the same verdict.

The t-distribution

Developed by William Gosset, a chemist at the Guinness brewery who published under the pen name "Student" because his employer forbade staff from publishing, the t-distribution is bell-shaped and symmetric like the normal curve but has heavier tails. Those fatter tails are exactly the mathematical expression of the extra uncertainty introduced by estimating the standard deviation from the sample: because s itself wobbles from sample to sample, extreme test statistics are a bit more likely than the normal would predict, so the critical values are pushed slightly farther out.

The precise shape depends on the degrees of freedom, which for a one-sample test equal n - 1. As the sample size grows, s becomes a reliable estimate of the true spread, the tails thin out, and the t-distribution converges to the normal. For n above about 30 the two are nearly indistinguishable, which is why large-sample z and t procedures agree.

The convergence is easy to see in the two-sided critical values at alpha = 0.05. With 4 degrees of freedom the critical value is about 2.776, with 9 it is about 2.262, with 24 it is about 2.064, with 100 it is about 1.984, and the limiting normal value is 1.960. Each step toward a larger sample pulls the cutoff closer to the familiar 1.96, and the penalty for a very small sample is substantial: a study with five observations needs a test statistic roughly 40 percent larger than a very large study needs, purely to compensate for uncertainty about the spread.

Key idea: Degrees of freedom, equal to n - 1 for a one-sample test, control how heavy the t-distribution's tails are, and heavier tails mean a larger critical value and a more demanding test.

The one-sample t statistic

To test H0: mean = mu0, the test statistic is t = (x-bar - mu0) / (s / square root of n). It has exactly the same form as a z-statistic (observed minus expected, divided by the standard error) but with the sample standard deviation s standing in for the unknown population value. The result is compared against a t-distribution with n - 1 degrees of freedom rather than the standard normal. Everything you learned about hypotheses, p-values, and the reject/fail-to-reject decision carries over unchanged; only the reference distribution swaps.

Worked example

A training program claims to change a mean score from 50. A sample of n = 25 participants has x-bar = 52 and s = 8. Test H0: mean = 50 against Ha: mean is not equal to 50 at alpha = 0.05.

• Standard error = s / square root of n = 8 / 5 = 1.6.
• Test statistic = t = (52 - 50) / 1.6 = 2 / 1.6 = 1.25.
• Degrees of freedom = n - 1 = 24.

For a two-sided test with 24 degrees of freedom at alpha = 0.05, the critical t-value is about 2.064 (slightly larger than the z-value of 1.96, reflecting the heavier tails). Since our t of 1.25 is smaller in magnitude than 2.064 (equivalently, the two-sided p-value exceeds 0.05), we fail to reject H0. The program has not shown a statistically significant change; a 2-point gain in a sample of 25, with this much variability, is well within what chance alone produces.

State that conclusion carefully, because the wording matters. The correct sentence is that this study did not find convincing evidence of a change, not that the course has no effect. A confidence interval makes the residual uncertainty explicit. Using the same critical value of 2.064, the 95 percent interval for the population mean is 52 plus or minus 2.064 x 1.6, which is 52 plus or minus 3.30, giving roughly 48.7 to 55.3 points. That interval contains 50, which is precisely why the test failed to reject, but it also contains 55, a gain that Ridgeway would consider a genuine success. The study is simply too small to distinguish those possibilities, and reporting the interval says so honestly while a bare p-value does not.

Key idea: A t-test and its matching confidence interval always agree, and the interval is more informative because it displays the whole range of parameter values the data leaves open.

Three flavors of t-test

The same t machinery adapts to three common study designs, and choosing correctly among them is the main practical skill:

• One-sample t-test: compares one sample mean to a fixed benchmark value (as above). Use it when you have one group and a target number.
• Two-sample (independent) t-test: compares the means of two separate, unrelated groups, such as a treatment group versus a control group of different people. Use it when two distinct groups are measured once each.
• Paired t-test: compares two measurements on the same individuals, such as each person's before-and-after score; you compute each person's difference and test whether the mean difference is zero. Use it when the same subjects are measured twice, or when subjects are naturally matched.

Worked example: a paired test on five typists. Suppose Ridgeway records each of five students before and after the course. The before scores are 40, 55, 62, 48, and 70, and the after scores are 44, 58, 67, 51, and 75. Work with the differences, taking after minus before: 4, 3, 5, 3, 5. Their sum is 4 + 3 + 5 + 3 + 5 = 20, so the mean difference is 20 / 5 = 4 points. The deviations from that mean are 0, -1, 1, -1, 1, whose squares are 0, 1, 1, 1, 1 and sum to 4, so the sample variance of the differences is 4 / (5 - 1) = 1 and the standard deviation is 1 point. The standard error is 1 / sqrt(5) = 0.447, giving t = (4 - 0) / 0.447 = 8.94 on 4 degrees of freedom. Against a critical value of 2.776 that is overwhelming evidence of improvement.

Now notice what an independent-samples analysis would have faced. The raw before scores range from 40 to 70, an enormous spread that has nothing to do with the course and everything to do with who the students already were. Pairing removes that person-to-person variation entirely, because each student serves as their own comparison, leaving only the consistent 3-to-5-point gain. Same data, same students, and a dramatically clearer signal.

Key idea: The paired test analyses one column of differences rather than two columns of scores, which is why its degrees of freedom equal the number of pairs minus one rather than the total number of measurements minus two.

Why pairing matters

The paired design is not just a bookkeeping choice; it is often far more powerful. By looking at within-person differences, a paired test cancels out the large person-to-person variation that would otherwise swamp a modest treatment effect. If everyone's blood pressure drops by roughly 5 points after a drug, that consistent shift is easy to detect in the differences even though people's baseline blood pressures vary enormously.

Feeding the same data into an independent two-sample test would bury the 5-point signal under that baseline noise. So when a design allows pairing, using it, and analyzing it as paired, can turn an undetectable effect into a clear one. Matching the test to the design is therefore not a technicality but a decision that can change the conclusion.

Conditions the t-test relies on

Three assumptions underlie every t procedure, and checking them is part of doing the test rather than an optional extra. First, the observations must be independent of one another, which is a consequence of how the data was collected and cannot be repaired by any calculation. Second, the sampling distribution of the mean must be approximately normal, which for small samples effectively requires the underlying data to be roughly symmetric and free of severe outliers, and which for larger samples follows from the Central Limit Theorem. Third, the sample must actually be drawn from the population the conclusion is about, since no formula can extend an inference beyond the group that was sampled. When these conditions fail badly, alternatives exist, including rank-based procedures that do not assume normality and resampling methods that build the reference distribution from the data itself.

Key idea: The t-test is fairly robust to mild departures from normality at moderate sample sizes, but it is never robust to dependent observations or to a biased sample.

Where people get stuck

• Using z when sigma is unknown. If the spread came from the sample, use t and n - 1 degrees of freedom. With 24 degrees of freedom the critical value is 2.064 rather than 1.96, and using the smaller value overstates significance.
• Choosing the test to get a smaller p-value. Paired or independent is dictated by the study design. Two measurements on the same people are paired whether or not pairing helps, and two separate groups are independent even if pairing would be more convenient.
• Getting the degrees of freedom wrong for paired data. Five students measured twice give five differences and 4 degrees of freedom, not 10 observations and 8 degrees of freedom.
• Reading a non-significant t-test as proof of no effect. The Ridgeway interval from 48.7 to 55.3 shows the study is compatible with no change and with a useful gain, which is a statement about insufficient precision rather than about absence of effect.
• Ignoring the conditions. A t-test on badly skewed data from a small sample, or on observations that are not independent, produces a p-value that looks respectable and means very little.

Recap

• The t-distribution replaces the normal whenever the population standard deviation is estimated from the sample, and its heavier tails reflect that extra uncertainty.
• Degrees of freedom equal n - 1 for a one-sample test, and larger degrees of freedom pull the critical value down toward the normal value of 1.96.
• The one-sample statistic is t = (x-bar - mu0) divided by s / sqrt(n), and every step of hypothesis-testing logic carries over unchanged from the z procedure.
• One-sample tests compare a mean to a benchmark, independent-samples tests compare two separate groups, and paired tests analyse the differences within matched units.
• Pairing removes person-to-person variability and can turn an undetectable effect into an obvious one, but the design, not the resulting p-value, determines which test is legitimate.

Sources

1. OpenStax. (2023). 10.1 Two population means with unknown standard deviations. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 10.4 Matched or paired samples. In Introductory statistics 2e. Rice University. openstax.org
3. Pennsylvania State University. (n.d.). 8.2.1 The t distribution. In STAT 200: Elementary statistics. online.stat.psu.edu
4. Pennsylvania State University. (n.d.). 8.3 Paired means. In STAT 200: Elementary statistics. online.stat.psu.edu
5. NIST/SEMATECH. (n.d.). 7.2.2 Are the data consistent with the assumed process mean? In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
6. Kim, T. K. (2015). T test as a parametric statistic. Korean Journal of Anesthesiology, 68(6), 540-546. pmc.ncbi.nlm.nih.gov
7. MacTutor History of Mathematics Archive. (n.d.). William Sealy Gosset. University of St Andrews. mathshistory.st-andrews.ac.uk
8. Student. (1908). The probable error of a mean. Biometrika, 6(1), 1-25. find source ↗

So far every technique has concerned a single variable. To study how two quantitative variables move together, such as height and weight or advertising and sales, we need new tools. We start with a scatterplot, where each point is one individual's (x, y) pair, and then summarize the pattern with a number (correlation) and a line (regression).

The anchor is the Bridgeway Tutoring log, which records for five students the number of hours each spent studying and the score each earned on a ten-point quiz. The hours are 1, 2, 3, 4, and 5, and the corresponding scores are 2, 4, 5, 4, and 5. Five points is a deliberately tiny dataset, chosen so that every sum, every coefficient, and every residual can be computed and checked by hand rather than accepted on faith from software.

Key idea: Correlation summarises the strength and direction of a straight-line relationship with a single number, while regression supplies the line itself and therefore the ability to predict.

In plain terms

Two quantitative variables measured on the same individuals can be plotted as a cloud of points, and two summaries describe that cloud. The first is a number between -1 and +1 that reports how tightly the points hug a straight line and whether that line rises or falls. Values close to either extreme mean the points sit almost exactly on a line, and values close to zero mean no straight-line pattern is present, although a strong curved pattern may still be hiding there. The second summary is the line that fits best, defined as the line making the total of the squared vertical misses as small as possible. That line has a slope, which says how much the outcome changes per one-unit increase in the predictor, and an intercept, which says what the line predicts when the predictor equals zero. Both summaries describe association only, and neither one, by itself, can establish that changing the first variable would change the second.

The correlation coefficient

The correlation coefficient r measures the strength and direction of a linear relationship between two quantitative variables. It always falls between -1 and +1. A positive r means y tends to rise as x rises (a positive association); a negative r means y tends to fall as x rises; r near 0 means little linear association.

Values near plus or minus 1 indicate points hugging a straight line, while values near 0.5 indicate a loose cloud with a visible trend. Correlation is unitless and symmetric: swapping x and y leaves r unchanged, and rescaling the variables (say, centimeters to inches) does not affect it either. The word "linear" is doing heavy lifting: r measures only straight-line association, and a strong curved relationship can have an r near zero.

Verbal labels for the size of r should be treated with caution, because what counts as a strong correlation depends heavily on the field. A correlation of 0.4 between a psychological measure and a behaviour is a substantial finding, while a correlation of 0.4 between two readings from the same laboratory instrument would indicate a serious calibration problem. Rough conventions do exist, with values around 0.1 to 0.3 often called weak, 0.4 to 0.6 moderate, and above 0.7 strong, but these labels describe magnitude rather than importance and should always be reported alongside the scatterplot that produced them.

Key idea: The correlation coefficient is unitless, symmetric in the two variables, unaffected by rescaling, and sensitive only to straight-line association.

Least-squares regression

The regression line is the single straight line that best fits the points by making the sum of squared vertical distances (the residuals) as small as possible, which is why it is called the least-squares line. We write it y-hat = b0 + b1 x, where b1 is the slope and b0 is the intercept, and the hat on y signals a predicted value rather than an observed one.

The slope tells how much y-hat changes for each one-unit increase in x, and it carries the units of y per unit of x (for example, dollars of sales per dollar of advertising). The intercept is the predicted y when x = 0, which is meaningful only when x = 0 is within or near the range of the data.

The vertical misses are called residuals, each one computed as the observed value minus the predicted value. A positive residual means the point lies above the line and the model under-predicted; a negative residual means the opposite. The line is chosen to minimise the sum of the squared residuals, and one consequence of that choice is that the residuals from a fitted least-squares line always sum to zero, which supplies a free arithmetic check on any hand calculation.

Key idea: Least squares minimises vertical distances specifically, which is why swapping the roles of x and y produces a different line even though it leaves the correlation unchanged.

Worked example

Five points: x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5. The needed sums are: sum x = 15, sum y = 20, sum xy = 66, sum x-squared = 55, with n = 5, so mean x = 3 and mean y = 4.

The slope is b1 = (n(sum xy) - (sum x)(sum y)) / (n(sum x-squared) - (sum x)^2) = (5(66) - 15(20)) / (5(55) - 15^2) = (330 - 300) / (275 - 225) = 30 / 50 = 0.6.

The intercept is b0 = mean y - b1(mean x) = 4 - 0.6(3) = 4 - 1.8 = 2.2. So the regression line is y-hat = 2.2 + 0.6 x. Notice the line always passes through the point of averages (mean x, mean y) = (3, 4), a fact you can use to check your work: 2.2 + 0.6(3) = 4.

To predict y when x = 6: y-hat = 2.2 + 0.6(6) = 2.2 + 3.6 = 5.8. For this data the correlation works out to r = 0.775, a moderately strong positive linear relationship.

Worked example: computing r from deviations. The correlation can be built from the same deviations used for the mean. With mean x = 3 and mean y = 4, the x-deviations are -2, -1, 0, 1, 2 and the y-deviations are -2, 0, 1, 0, 1. Multiply them pairwise to obtain 4, 0, 0, 0, 2, which sum to 6. Square the x-deviations to obtain 4, 1, 0, 1, 4, summing to 10, and square the y-deviations to obtain 4, 0, 1, 0, 1, summing to 6. Then r = 6 / sqrt(10 x 6) = 6 / sqrt(60) = 6 / 7.746 = 0.775. The slope follows immediately as 6 / 10 = 0.6, matching the earlier calculation and confirming both.

Worked example: residuals and the zero-sum check. Using y-hat = 2.2 + 0.6x, the five predictions are 2.8, 3.4, 4.0, 4.6, and 5.2. Subtracting each from its observed score gives residuals of 2 - 2.8 = -0.8, 4 - 3.4 = 0.6, 5 - 4.0 = 1.0, 4 - 4.6 = -0.6, and 5 - 5.2 = -0.2. Their sum is -0.8 + 0.6 + 1.0 - 0.6 - 0.2 = 0, exactly as least squares guarantees. Squaring and adding gives 0.64 + 0.36 + 1.00 + 0.36 + 0.04 = 2.40, and since the total squared variation in y was 6, the fraction left unexplained is 2.40 / 6 = 0.40, so the fraction explained is 1 - 0.40 = 0.60. That is exactly r-squared, computed by a completely independent route.

Key idea: Residuals always sum to zero for a least-squares line, and r-squared can be obtained either by squaring the correlation or by comparing unexplained variation with total variation.

R-squared and the danger of extrapolation

The square of the correlation, r-squared, is the proportion of the variation in y explained by the linear relationship with x. Here r-squared = 0.775^2 = 0.60, so about 60% of the variation in y is accounted for by x, and the remaining 40% is due to other factors and noise. R-squared runs from 0 to 1, and higher means a tighter fit.

One serious hazard is extrapolation: using the line to predict far outside the range of the observed x-values. Our data covers x from 1 to 5; predicting at x = 6 is a mild stretch, but predicting at x = 50 would be reckless, because nothing guarantees the linear pattern continues where we have no data.

Key idea: R-squared reports the proportion of variation in y accounted for by the linear relationship with x, and a high value indicates a tight fit rather than a correct causal model.

Correlation is not causation

The most important caution in this lesson: a strong r means two variables move together, but it does not establish that one causes the other. A hidden lurking variable can drive both and manufacture a correlation between things with no causal link. Ice cream sales and drowning deaths correlate strongly, but neither causes the other; hot weather (the lurking variable) drives both. Only a well-designed experiment with random assignment can establish cause and effect, because randomization is what breaks the influence of lurking variables. Observational correlations are valuable for prediction and for generating hypotheses, but on their own they can never prove causation.

It helps to name the four explanations that compete whenever a correlation appears, because dismissing them one by one is what causal reasoning actually consists of. The first is that x genuinely causes y. The second is reverse causation, where y causes x, as when researchers observe that people who visit doctors more often are less healthy. The third is confounding, where a third variable drives both, as with ice cream sales and drowning deaths sharing hot weather as their common cause. The fourth is coincidence, since with enough variables under examination some strong correlations will appear by chance alone. Bridgeway's tutoring data illustrates the difficulty concretely: students who chose to study five hours may simply have been the more motivated students all along, so motivation confounds the comparison and the observed slope of 0.6 points per hour cannot be read as the effect of assigning an extra hour of study.

Key idea: Random assignment in a designed experiment is what breaks the link between the predictor and any lurking variable, which is why experiments support causal claims and observational correlations do not.

Where people get stuck

• Reading a correlation near zero as no relationship. It means no straight-line relationship. A perfect U-shaped pattern can have r close to zero, which is why the scatterplot must always be inspected before r is trusted.
• Interpreting the intercept outside the data. The Bridgeway intercept of 2.2 predicts the score for zero hours of study, and no student in the sample studied zero hours, so that number is an extrapolation rather than an observation.
• Predicting far beyond the observed range. The model was fitted on 1 to 5 hours. Predicting at 6 hours is a mild stretch; predicting at 50 hours would produce a score of 32 on a ten-point quiz, which is arithmetically valid and physically impossible.
• Treating a high r-squared as proof of causation. R-squared measures how tightly the points cluster around the line and says nothing about why they do. A confounded relationship can fit beautifully.
• Assuming the regression of y on x is the same as x on y. Least squares minimises vertical distances, so reversing the roles of the variables produces a different slope, even though the correlation is unchanged.

Recap

• The correlation coefficient r lies between -1 and +1 and measures the strength and direction of a straight-line association between two quantitative variables.
• The least-squares regression line minimises the sum of squared vertical residuals, and those residuals always sum to zero.
• The slope reports the predicted change in y per one-unit increase in x, and the intercept is meaningful only when x = 0 lies within or near the observed range.
• R-squared equals the square of the correlation and gives the proportion of variation in y explained by the linear relationship with x.
• Correlation is consistent with direct causation, reverse causation, confounding, and coincidence, so only a randomised experiment can establish cause.

Sources

1. OpenStax. (2023). 12.2 Scatter plots. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 12.3 The regression equation. In Introductory statistics 2e. Rice University. openstax.org
3. Pennsylvania State University. (n.d.). 12.2 Correlation. In STAT 200: Elementary statistics. online.stat.psu.edu
4. Pennsylvania State University. (n.d.). 12.5 Cautions. In STAT 200: Elementary statistics. online.stat.psu.edu
5. NIST/SEMATECH. (n.d.). 4.4.3.1 Least squares. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
6. Schober, P., Boer, C., and Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia and Analgesia, 126(5), 1763-1768. pubmed.ncbi.nlm.nih.gov
7. Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1), 17-21. find source ↗

Correlation and regression handle two quantitative variables. When the variables are instead categorical, so our data is counts in categories rather than measured numbers, we turn to the chi-square test. Its core idea is beautifully simple: compare the counts we actually observed against the counts we would expect if a stated model were true, and see whether the gap is bigger than chance can explain.

Two anchors run through this lesson, both drawn from the Elmwood Community Center. The board-game club suspects that one of its dice is loaded and has rolled it 60 times to find out. The clinic that shares the building has run a small trial in which 100 patients received either a treatment or a control condition and were recorded as improved or not improved. The first situation calls for a goodness-of-fit test and the second for a test of independence, and both use exactly the same statistic.

Key idea: A chi-square test compares counts you observed against counts a stated model predicts, and asks whether the discrepancy is larger than random variation would comfortably produce.

In plain terms

The procedure is unusually intuitive for a formal test. Write down what the counts would look like if the claim being examined were true, which for a fair die means an equal number of each face and which for a clinical trial means the same improvement rate in both arms. Compare those expected counts with what actually happened, one cell at a time. Square each gap so that shortfalls and excesses both contribute positively, then divide each squared gap by the expected count for that cell, because a discrepancy of five is trivial where a thousand were expected and alarming where three were. Adding those adjusted contributions gives a single number that grows as the observed pattern departs from the model. A small total means the model survives, a large total counts as evidence against it, and how large is large enough depends on the degrees of freedom, which is essentially a count of how many cells were free to vary.

Two common chi-square tests

• Goodness-of-fit test: checks whether one categorical variable follows a claimed distribution. Example: is a six-sided die fair, meaning each face equally likely?
• Test of independence: checks whether two categorical variables are associated, using a two-way table of counts. Example: is recovery (improved or not) associated with treatment group?

The chi-square statistic

Both tests use the same statistic: chi-square = sum of (observed - expected)^2 / expected, added over every category or cell. Each term compares one observed count O to its expected count E: the squared gap (O - E)^2 makes every contribution positive and magnifies large discrepancies, and dividing by E scales the gap relative to how big a count we expected there (a gap of 5 is trivial when we expected 1000 but alarming when we expected 3).

Large chi-square values mean the observed counts sit far from what the model predicts, which is evidence against the null hypothesis (that the claimed distribution or independence holds). The chi-square distribution used to judge the statistic is right-skewed and, like the t, its exact shape depends on the degrees of freedom.

Two features of this statistic deserve emphasis before any arithmetic. First, it is always non-negative and always right-skewed, so the test is inherently one-tailed: only large values count as evidence against the model, and a chi-square of zero would mean the observed counts matched the model exactly. Second, the degrees of freedom count how many cells could vary freely once the totals are fixed. For a goodness-of-fit test with k categories that number is k - 1, because once the first k - 1 counts are known the last is determined by the total. For a two-way table it is (rows - 1)(columns - 1), for the same reason applied in two directions at once.

Key idea: Dividing each squared discrepancy by its expected count is what makes the contributions comparable, since the same absolute gap means very different things at different expected frequencies.

Worked example: goodness of fit

A die is rolled 60 times. If it is fair, each face is expected 60 / 6 = 10 times. Observed counts are 8, 12, 9, 11, 10, 10 for faces 1 through 6. The contributions to chi-square are:

Face	Observed	Expected	(O - E)^2 / E
1	8	10	0.4
2	12	10	0.4
3	9	10	0.1
4	11	10	0.1
5	10	10	0.0
6	10	10	0.0

Adding the last column, chi-square = 0.4 + 0.4 + 0.1 + 0.1 + 0 + 0 = 1.0. The degrees of freedom for goodness of fit are (number of categories - 1) = 6 - 1 = 5. A chi-square of only 1.0 with 5 degrees of freedom is small (its critical value at alpha = 0.05 is 11.07, far above 1.0, so the p-value is large), and we fail to reject the claim that the die is fair. The small wobbles from 10 are exactly what chance produces in 60 rolls.

Word that conclusion with care. The club has not shown the die is fair; it has failed to find evidence that the die is unfair, and with only 60 rolls a mild bias could easily hide. Contrast that outcome with a genuinely suspicious die. Suppose a second die rolled 60 times produces counts of 4, 6, 8, 10, 14, and 18 across the six faces, again totalling 60 against an expectation of 10 each. The contributions are (4-10)^2/10 = 3.6, (6-10)^2/10 = 1.6, (8-10)^2/10 = 0.4, (10-10)^2/10 = 0, (14-10)^2/10 = 1.6, and (18-10)^2/10 = 6.4. Adding them gives 3.6 + 1.6 + 0.4 + 0 + 1.6 + 6.4 = 13.6, which exceeds the critical value of 11.07 on 5 degrees of freedom, so this time the club rejects fairness. Notice that the largest single contribution, 6.4, comes from the face that appeared 18 times, which points straight at where the model failed.

Key idea: Individual cell contributions identify which categories drive a significant result, which turns a single yes-or-no verdict into a usable diagnosis.

Test of independence and expected counts

For a two-way table, the expected count in any cell, under the null of independence, is (row total x column total) / grand total. This formula is just the multiplication rule for independent events applied to counts. Consider 100 patients in a 2-by-2 table (treatment versus control, improved versus not):

	Improved	Not improved	Row total
Treatment	30	10	40
Control	20	40	60
Col total	50	50	100

The expected count for the treatment-improved cell is (40 x 50) / 100 = 20. Working out all four cells (the expected counts are 20, 20, 30, 30) and summing (O - E)^2 / E gives chi-square = 16.67. The degrees of freedom for an independence test are (rows - 1)(columns - 1) = (2 - 1)(2 - 1) = 1.

A chi-square of 16.67 with 1 degree of freedom is very large, well beyond the 0.05 critical value of 3.84, so we reject independence: improvement and treatment group are associated. The treatment group improved far more often (30 of 40, or 75%) than the control group (20 of 60, or 33%), and that gap is too large to be chance.

Worked example: every cell of the Elmwood trial. Set the four expected counts out explicitly, each one computed as row total times column total divided by 100. Treatment-improved is (40 x 50) / 100 = 20, treatment-not-improved is (40 x 50) / 100 = 20, control-improved is (60 x 50) / 100 = 30, and control-not-improved is (60 x 50) / 100 = 30. Check the bookkeeping: the expected counts reproduce the original row totals of 40 and 60 and the column totals of 50 and 50, which they must. Now compute the four contributions. Treatment-improved gives (30 - 20)^2 / 20 = 100 / 20 = 5.00, treatment-not-improved gives (10 - 20)^2 / 20 = 5.00, control-improved gives (20 - 30)^2 / 30 = 100 / 30 = 3.33, and control-not-improved gives (40 - 30)^2 / 30 = 3.33. The total is 5.00 + 5.00 + 3.33 + 3.33 = 16.67 on 1 degree of freedom, far beyond the critical value of 3.84.

Key idea: Expected counts under independence come from the row and column totals alone, and they must reproduce those totals exactly, which is a reliable check on the arithmetic.

Conditions and cautions

Two practical points. First, the chi-square test needs counts, not percentages or averages; if you are handed proportions you must convert back to raw counts using the sample size before testing.

Second, the approximation that the statistic follows a chi-square distribution requires the expected counts to be reasonably large, a common rule being that every expected count should be at least 5. When some expected counts are tiny (as in a sparse table), the approximation breaks down and an exact method such as Fisher's exact test is used instead. As always, rejecting independence tells you the variables are associated, not that one causes the other; the correlation-is-not-causation lesson applies with equal force to categorical data.

One further limitation is worth stating, because it explains why a chi-square result is rarely the last word. The test reports only that an association exists; it does not report how strong that association is, and with a large enough sample even a negligible association becomes significant. The Elmwood trial is convincing because the improvement rates differ enormously, 75 percent against roughly 33 percent, not merely because the p-value is small. Always accompany a significant chi-square with the actual percentages, or with a measure of association such as a risk ratio, so that readers can judge importance rather than merely detectability.

Key idea: A significant chi-square establishes that two categorical variables are associated, and the percentages in the table, not the statistic, tell you whether the association matters.

Where people get stuck

• Feeding percentages into the formula. The statistic requires raw counts, because the sample size is what determines how surprising a discrepancy is. If a table reports 75 percent and 33 percent, convert back to counts before testing.
• Applying the wrong degrees of freedom. Goodness of fit uses categories minus one, so six faces give 5. A two-way table uses (rows - 1) times (columns - 1), so a 2-by-2 table gives 1, not 3.
• Checking observed counts rather than expected counts against the rule of 5. The condition concerns expected counts. A cell that observed zero events is not automatically a problem if its expected count is comfortably above 5.
• Reading a non-significant result as proof of independence. The fair-die test failed to reject, which is not the same as demonstrating fairness. Sixty rolls simply cannot detect a mild bias.
• Concluding causation from a significant association. A two-way table shows that the variables move together. Only random assignment of patients to conditions, as in a properly designed trial, licenses a causal reading.

Recap

• The chi-square statistic sums (observed minus expected) squared divided by expected across all categories or cells, and it is always non-negative and right-skewed.
• A goodness-of-fit test checks one categorical variable against a claimed distribution, with degrees of freedom equal to the number of categories minus one.
• A test of independence checks two categorical variables for association, with expected counts equal to row total times column total divided by grand total and degrees of freedom equal to (rows - 1)(columns - 1).
• The test requires raw counts and expected counts that are reasonably large, conventionally at least 5 per cell, with Fisher's exact test available for sparse tables.
• Rejecting independence establishes association, not causation, and the percentages in the table are what communicate the size and importance of the effect.

Sources

1. OpenStax. (2023). 11.1 Facts about the chi-square distribution. In Introductory statistics 2e. Rice University. openstax.org
2. OpenStax. (2023). 11.2 Goodness-of-fit test. In Introductory statistics 2e. Rice University. openstax.org
3. OpenStax. (2023). 11.3 Test of independence. In Introductory statistics 2e. Rice University. openstax.org
4. NIST/SEMATECH. (n.d.). 1.3.5.15 Chi-square goodness-of-fit test. In e-Handbook of statistical methods. National Institute of Standards and Technology. itl.nist.gov
5. Pennsylvania State University. (n.d.). 11.2 Goodness of fit test. In STAT 200: Elementary statistics. online.stat.psu.edu
6. Pennsylvania State University. (n.d.). 11.3 Chi-square test of independence. In STAT 200: Elementary statistics. online.stat.psu.edu
7. McHugh, M. L. (2013). The chi-square test of independence. Biochemia Medica, 23(2), 143-149. pmc.ncbi.nlm.nih.gov