Introduction to Statistics

Statistics is the science of collecting, organizing, summarizing, and drawing conclusions from data. Every field that gathers evidence, from medicine to economics to sports, leans on it, because data almost never speaks for itself. Raw numbers are noisy and partial; statistics is the discipline that turns them into defensible claims. It splits into two branches. Descriptive statistics summarizes what a dataset shows using tables, graphs, and numbers, without reaching beyond the data in hand. Inferential statistics uses a sample to make claims about the larger group it came from, always attaching a measure of uncertainty. This whole course is a journey from the first branch to the second: Modules 1 and 2 describe data, and Modules 3 through 5 use it to infer.

Population, sample, parameter, statistic

A population is the entire group we want to learn about, such as all registered voters in a country or every bottle a factory will ever fill. A sample is the smaller subset we actually measure, because reaching the whole population is usually impossible or wasteful. A number that describes the whole population is a parameter (almost always unknown), while a number computed from a sample is a statistic (which we use to estimate the parameter). For example, the true average height of all adults is a parameter; the average height of 200 adults we measured is a statistic. A memory hook: parameter goes with population, and statistic goes with sample. Nearly every inference problem in this course is really the same shape: we see a statistic and want to say something trustworthy about the parameter we cannot see.

Categorical versus quantitative variables

A variable is any characteristic that can differ from one individual to the next; the individuals (or cases) are the people or objects being measured. A categorical (qualitative) variable places each individual into a group, such as eye color, country, or whether a patient improved. A quantitative variable is a number you can meaningfully do arithmetic with, such as height, income, or reaction time. Quantitative variables are further split into discrete (countable values with gaps between them, like the number of siblings or cars in a lot) and continuous (any value in a range, limited only by measuring precision, like weight or temperature). A quick test for "is this quantitative?" is to ask whether averaging the values makes sense. The average of 3 and 5 siblings (4) is meaningful; the "average" of ZIP codes 10001 and 90210 is nonsense even though both look like numbers.

The four levels of measurement

Beyond the categorical-quantitative split, statisticians recognize four levels of measurement, a ladder of increasing information first named by the psychologist Stanley Smith Stevens:

• Nominal: named categories with no order (blood type, favorite color, jersey number used only as a label). You can count how many fall in each group, but ordering them is arbitrary.
• Ordinal: ordered categories, but the gaps between them are not equal or not meaningful (small, medium, large; survey ratings from "strongly disagree" to "strongly agree"; race finishing places). You know first beat second, but not by how much.
• Interval: ordered numbers with equal gaps but no true zero (temperature in Celsius or Fahrenheit; calendar years). A ten-degree jump means the same amount anywhere on the scale, but zero degrees does not mean "no temperature," so ratios fail.
• Ratio: ordered numbers with equal gaps and a true zero, so ratios make sense (height, weight, income, time elapsed). Because zero means "none," 40 kg really is twice 20 kg.

Worked example: sorting a survey

Imagine a health survey records, for each person: blood type, pain rating on a 1 to 10 scale, body temperature in Celsius, and annual income in dollars. Blood type is categorical, nominal. The pain rating is often treated as ordinal, since the distance from 3 to 4 need not equal the distance from 8 to 9. Temperature is quantitative, continuous, interval. Income is quantitative, continuous, ratio (someone earning 0 dollars truly earns nothing, and 80,000 dollars is twice 40,000). Naming each correctly is not busywork: it decides, for instance, that you may report a mean income but should report a median or mode for pain ratings.

Why the type decides everything downstream

Getting the type right matters because it dictates which graphs and which summaries are legal, and which statistical test you will eventually run. You can find the average of a ratio variable like income, but averaging nominal codes for eye color is meaningless even though a spreadsheet will happily do it. A histogram fits a quantitative variable; a bar chart fits a categorical one. A correlation needs two quantitative variables; a chi-square test handles two categorical ones. Throughout this course the very first question to ask about any dataset is: what kind of variable is this, and at what level is it measured? Answer that, and the right tool usually names itself.

Once you know a variable's type, the next step is to picture its distribution, which is the pattern of values it takes and how often each occurs. A good graph reveals in one glance what a column of numbers hides: where the data piles up, how far it spreads, and whether anything looks out of place. For a single quantitative variable, the workhorse graph is the histogram.

Frequency and relative-frequency tables

To build a histogram you first group the data into equal-width intervals called bins (or classes) and count how many values fall in each. The count is the frequency; dividing by the total gives the relative frequency, a proportion between 0 and 1 that you can also read as a percent. Consider 40 values grouped into five bins:

The relative frequencies sum to 1.000, which is always a useful arithmetic check: if they do not, a value was miscounted or misfiled. Relative frequencies matter because they let you compare datasets of different sizes fairly. A bin holding 12 of 40 values (30%) and a bin holding 300 of 1000 values (also 30%) represent the same share even though the raw counts look nothing alike.

From table to histogram

A histogram draws one bar per bin, with bar height equal to the frequency (or relative frequency). Because the variable is quantitative, the bars touch, showing that the horizontal axis is a continuous number line with no gaps between neighboring intervals.

The choice of bin width

Bin width is a real decision, not a detail. Too few, very wide bins smear the data into a featureless block and hide structure; too many, very narrow bins turn the histogram into a jagged comb where every bump is just noise. A common starting point is to aim for roughly 5 to 20 bins and adjust until the shape reads clearly. The same data can look quite different under different bin choices, so it is honest practice to try a couple of widths before trusting the picture.

Describing shape, center, and spread

When you describe a distribution, always mention three things: shape, center, and spread, plus any unusual points. Common shapes are symmetric (the left and right halves mirror each other, like the mound above), right-skewed (a long tail stretching toward high values, common for income, house prices, or wait times), and left-skewed (a long tail toward low values, as with exam scores on an easy test). A distribution can also be uniform (roughly flat, every bin about equal) or have two clear peaks (bimodal), which often signals two different groups blended together, such as heights of a mixed-sex population. An outlier is a value sitting far from the rest and always deserves a second look: it may be a data-entry error, or the most interesting point in the set.

Histogram versus bar chart

A histogram is for a quantitative variable, and its bars touch because the axis is a number line whose order cannot be changed. A bar chart is for a categorical variable: its bars have deliberate gaps, and the categories can be reordered freely (alphabetically, by size, however you like) without changing the meaning. Confusing the two is one of the most common early mistakes, and it is not merely cosmetic. Touching bars imply a continuous scale; if you draw pizza toppings with touching bars, you are falsely suggesting "pepperoni" and "mushroom" are adjacent points on a number line. Always check the variable type first, then pick the display.

A boxplot (box-and-whisker plot) is a compact picture of a distribution built from five key numbers. Where a histogram shows the full shape of one variable, a boxplot strips the distribution down to its skeleton, which makes it especially good for two jobs: comparing several groups side by side and spotting outliers at a glance.

The five-number summary

The five-number summary is the minimum, first quartile, median, third quartile, and maximum. The median (Q2) is the middle value once the data is sorted. The first quartile Q1 is the median of the lower half of the data, and the third quartile Q3 is the median of the upper half. These three quartiles split the sorted data into four equal-sized quarters: a quarter of the data lies below Q1, half lies below the median, and three quarters lies below Q3. Together the five numbers tell you where the data starts, where its middle 50% sits, and where it ends.

Worked example

Find the five-number summary of these 7 sorted values: 3, 5, 7, 8, 12, 13, 15.

• Minimum = 3, Maximum = 15.
• Median (Q2) = the 4th value = 8 (with 7 values, the middle one is position 4).
• Lower half (the values strictly below the median) is 3, 5, 7, so Q1 = 5.
• Upper half (the values strictly above the median) is 12, 13, 15, so Q3 = 13.

The interquartile range is IQR = Q3 - Q1 = 13 - 5 = 8. The IQR measures the spread of the middle 50% of the data, and because it ignores the extreme quarter on each end, it is resistant to outliers, unlike the full range. That resistance is exactly why the IQR pairs naturally with the median, another resistant measure, when a distribution is skewed.

The 1.5 x IQR rule for outliers

A value is flagged as a potential outlier if it falls below the lower fence Q1 - 1.5 x IQR or above the upper fence Q3 + 1.5 x IQR. Here the fences are 5 - 1.5(8) = 5 - 12 = -7 and 13 + 1.5(8) = 13 + 12 = 25. Since every value lies between -7 and 25, there are no outliers in this dataset. The 1.5 multiplier is a convention, not a law of nature: it is calibrated so that, for roughly bell-shaped data, only a tiny fraction of genuine values get flagged, so a flag is worth a second look rather than an automatic deletion.

Drawing and reading a boxplot

To draw the boxplot, mark Q1, the median, and Q3 to form the box, then draw whiskers out to the smallest and largest values that are not outliers. Any outliers are plotted as separate dots beyond the whiskers. A boxplot instantly shows center (the median line), spread (the box width for the middle half, and the whisker length for the rest), and skew: a longer whisker on one side, or a median pushed toward one end of the box, signals a tail in that direction. Note what a boxplot hides, though. Because it reports only five numbers, it cannot show whether a distribution is bimodal; two very different shapes can produce identical boxplots, so for a single variable a histogram is often more revealing.

Comparing groups side by side

The real payoff of boxplots comes when you line several up on the same axis, one per group. Suppose you plot test scores for three class sections. In one glance you can compare their medians (which section is typically higher), their spreads (which section is more consistent, shown by a narrower box), and their outliers (which section has an unusually low or high student). This side-by-side comparison, clean and free of clutter, is a task histograms do poorly and boxplots do beautifully, which is why boxplots are a staple of exploratory data analysis.

A measure of center reports a single typical value that stands in for a whole dataset. It answers the everyday question "what is a normal amount?" The three measures you need are the mean, the median, and the mode, and a surprising amount of statistical judgment comes down to knowing which one to report.

The mean

The mean (arithmetic average) is the sum of all values divided by how many there are. For a sample of n values, the sample mean is written x-bar. In symbols, x-bar = (sum of all x) / n. The mean is the balance point of the data: if you imagined the values as equal weights placed along a ruler, the mean is the spot where the ruler would balance. Every single value tugs on that balance point, which is both the mean's great strength (it uses all the information) and its weakness (one extreme value can drag it far).

Worked example. Find the mean of 4, 7, 7, 9, 10, 3. The sum is 4 + 7 + 7 + 9 + 10 + 3 = 40, and there are n = 6 values, so x-bar = 40 / 6 = 6.67 (rounded to two decimals).

The median

The median is the middle value when the data is sorted. Sorting the same data gives 3, 4, 7, 7, 9, 10. With an even count of 6, the median is the average of the two middle values (positions 3 and 4), so (7 + 7) / 2 = 7. If the count is odd, the median is simply the single middle value, at position (n + 1) / 2. Half the data lies at or below the median and half at or above, which makes it the exact center of position rather than of magnitude.

The mode

The mode is the value that occurs most often. In our data, 7 appears twice and every other value once, so the mode is 7. A dataset can have no mode (all values distinct), one mode (unimodal), or several (bimodal, trimodal, and so on). The mode is the only measure of center that also works for categorical data: the modal eye color or the most common blood type is a perfectly good "center" even though a mean or median of colors is meaningless.

Which one to trust

The median is resistant (robust): it barely moves when an extreme value is added, because it depends only on the order of the middle values, not their size. The mean is sensitive to outliers because every value enters the sum. Suppose one more value, 100, joined the data, giving seven values. The sorted list becomes 3, 4, 7, 7, 9, 10, 100, so the median shifts only from 7 to 7 (the new middle value is still 7), while the mean leaps from 6.67 to 140 / 7 = 20. One point moved the mean by 13 units and the median not at all. This is why the median is preferred for skewed data such as incomes, home prices, or wait times, while the mean is ideal for roughly symmetric data with no wild outliers.

Shape tells you which is larger

The relationship between the two measures is itself a clue to shape. In a right-skewed distribution the long high tail pulls the sensitive mean above the median. In a left-skewed distribution the mean is dragged below the median. In a symmetric distribution the mean and median roughly coincide. So if a report tells you the mean US household income is much higher than the median, you can infer without seeing the data that the income distribution is right-skewed, with a relatively small number of very high earners pulling the average up. This is exactly why economists usually quote the median income: it better reflects the typical household.

Two datasets can share the same mean yet look completely different because their values are more or less spread out. Consider two work teams that both average 40 hours a week: one where everyone works close to 40, and one where half work 20 and half work 60. The center is identical; the experience of being on those teams is not. Measures of spread quantify that variability, and they matter just as much as center in describing data.

Range

The range is the maximum minus the minimum. It is the simplest measure of spread and instantly interpretable, but it uses only the two most extreme values and ignores everything in between, so a single outlier can blow it up. For the two teams above, both might have a range of 40 (from 20 to 60), which shows how blunt the range can be. It is a useful quick summary but rarely the last word.

Variance and standard deviation

The standard deviation is the measure of spread that statistics is built on. It reports the typical distance of values from the mean. To find the sample variance (its square), follow four steps: (1) find the mean; (2) subtract the mean from each value to get deviations; (3) square each deviation; (4) average the squared deviations by dividing their sum by n - 1. The standard deviation is then the square root of the variance, which returns the answer to the original units. Why square the deviations at all? Because the deviations always sum to exactly zero (the positives and negatives cancel at the balance point), so their plain average is useless. Squaring makes every term non-negative and additionally punishes large deviations more than small ones.

Why divide by n minus 1

Using n - 1 (the degrees of freedom) rather than n corrects a subtle bias. Because the deviations are taken from the sample mean rather than the unknown true mean, they are systematically a touch too small, and dividing by the smaller number n - 1 compensates, giving a sample variance that estimates the population variance without bias on average. This is called Bessel's correction. When you genuinely have the entire population, you divide by n instead, because there is no estimation happening.

Worked example

Find the sample standard deviation of 2, 4, 4, 4, 5, 5, 7, 9.

Step 1, the mean: the sum is 2 + 4 + 4 + 4 + 5 + 5 + 7 + 9 = 40 and n = 8, so x-bar = 40 / 8 = 5.

Steps 2 and 3, deviations and their squares:

Step 4, sum the squared deviations: 9 + 1 + 1 + 1 + 0 + 0 + 4 + 16 = 32. The sample variance is s^2 = 32 / (8 - 1) = 32 / 7 = 4.57. The sample standard deviation is s = square root of 4.57 = 2.14 (to two decimals).

So values in this dataset sit about 2.14 units from the mean of 5, on average. (For reference, if these 8 numbers were an entire population you would divide by 8 instead of 7, giving a variance of 4.0 and a population standard deviation of exactly 2.0. The two answers are close, and they converge as n grows.)

Reading the standard deviation

A larger standard deviation means more spread. The standard deviation is always zero or positive, and it equals zero only when every value is identical (no spread at all). It carries the same units as the data (dollars, centimeters, hours), which is exactly why it is usually preferred over the variance for describing spread in plain language: a variance of 4 "square dollars" means nothing to a reader, but a standard deviation of 2 dollars does. The standard deviation also feeds directly into the z-scores and the normal model of the next lesson, where "how many standard deviations from the mean" becomes the universal ruler for comparing any two values.

A z-score restates a value as the number of standard deviations it sits above or below the mean. It is one of the most powerful ideas in the course because it puts values from completely different scales onto one common ruler, so a test score, a height, and a reaction time can all be compared fairly.

The z-score formula

For a value x from a distribution with a given mean and standard deviation, z = (x - mean) / (standard deviation). Read it in two steps: the numerator is the deviation (how far x is from the mean, in original units), and dividing by the standard deviation rescales that distance into standard-deviation units. A positive z means the value is above the mean; a negative z means below; z = 0 means exactly at the mean. Z-scores are unitless, which is precisely what lets them cross between scales.

Worked example. Test scores have a mean of 70 and a standard deviation of 8. A student scores 86. Then z = (86 - 70) / 8 = 16 / 8 = 2.0. The score is 2 standard deviations above the mean. Another student scores 64, giving z = (64 - 70) / 8 = -6 / 8 = -0.75, three quarters of a standard deviation below the mean.

Comparing across scales

Z-scores shine when two things are measured differently. Suppose you earn 650 on an exam with mean 500 and standard deviation 100, so z = (650 - 500) / 100 = 1.5. A friend earns 86 on the test above (z = 2.0). Even though 650 is a much bigger raw number than 86, your friend's z-score of 2.0 beats your 1.5, so relative to each group, your friend performed better. Raw scores across different tests are apples and oranges; z-scores make them comparable.

The normal distribution

The normal distribution is a symmetric, bell-shaped model that describes a remarkable range of natural measurements: heights, birth weights, measurement errors, and (thanks to the Central Limit Theorem in Module 3) the behavior of sample means. It is completely determined by two numbers, its mean (which locates the center) and its standard deviation (which sets the width). The special case with mean 0 and standard deviation 1 is the standard normal distribution, and its values are simply z-scores. When data is approximately normal, z-scores unlock precise statements about how common or rare a value is.

The 68-95-99.7 empirical rule

For a normal distribution, the empirical rule gives quick, memorable benchmarks:

• About 68% of values lie within 1 standard deviation of the mean (between z = -1 and z = +1).
• About 95% lie within 2 standard deviations.
• About 99.7% lie within 3 standard deviations.

Applying the rule. With mean 70 and standard deviation 8, about 68% of scores fall in 70 plus or minus 8, that is between 62 and 78. About 95% fall between 70 plus or minus 16, that is 54 to 86. A score of 86 (z = 2) therefore sits right at the edge of the middle 95%.

From z-scores to percentiles

Because the normal curve is symmetric, the empirical rule also gives percentiles, the percentage of values at or below a point. Consider the score of 86 (z = 2). The middle 95% lies between z = -2 and z = +2, leaving 5% split evenly into the two tails, so about 2.5% sits above z = 2. That means about 97.5% of scores are at or below 86, placing it near the 97.5th percentile. Similarly, a value exactly at the mean (z = 0) sits at the 50th percentile, and a value at z = -1 sits near the 16th percentile (since 68% is in the middle, 16% falls in each tail). For z-scores that are not whole numbers, statisticians read a standard normal table or use software, but the empirical rule handles the common benchmark cases in your head.

Probability measures how likely an event is, on a scale from 0 (impossible) to 1 (certain). It is the mathematical language of uncertainty and the essential bridge from describing data to making inferences: every confidence interval and hypothesis test in the coming modules is ultimately a probability statement. Getting the basic rules right here pays off everywhere later.

Basic definition and the sample space

The sample space is the set of all possible outcomes of a random process. An event is any subset of that space, some collection of outcomes we care about. When the outcomes are equally likely, the probability of an event A is P(A) = (number of outcomes in A) / (total number of outcomes). Rolling a fair die, the sample space is {1, 2, 3, 4, 5, 6}, so P(rolling a 4) = 1 / 6. Drawing from a standard 52-card deck, P(a heart) = 13 / 52 = 1 / 4. Every probability obeys three basics: it lies between 0 and 1, the probability of the whole sample space is 1, and the probabilities of a complete set of non-overlapping outcomes add to 1.

The complement rule

The complement of A, written "not A," is everything in the sample space in which A does not happen. Because A either happens or it does not, P(not A) = 1 - P(A). If the chance of rain is 0.30, the chance of no rain is 1 - 0.30 = 0.70. The complement rule is the go-to shortcut whenever a question contains the phrase "at least one," because "at least one" is the complement of the often much simpler event "none."

The addition rule

For the probability that A or B occurs (meaning at least one of them), P(A or B) = P(A) + P(B) - P(A and B). You subtract the overlap because outcomes counted in both A and B would otherwise be double-counted. Two events are mutually exclusive (disjoint) if they cannot both happen at once; then P(A and B) = 0 and the rule simplifies to P(A) + P(B).

Worked example. Draw one card. Find P(King or Heart). There are 4 kings and 13 hearts, but the king of hearts belongs to both, so it must not be counted twice: P(King or Heart) = 4/52 + 13/52 - 1/52 = 16/52 = 4/13, about 0.31.

The multiplication rule and independence

For the probability that A and B both occur, when the events are independent (one happening does not change the other's chance), P(A and B) = P(A) x P(B). Flipping a fair coin twice, P(two heads) = 0.5 x 0.5 = 0.25. It is vital not to confuse independence with mutual exclusivity: mutually exclusive events cannot both happen (they are strongly dependent, in fact, since knowing one occurred tells you the other did not), while independent events simply do not influence each other's chances. When events are not independent, you need conditional probability, where P(A and B) = P(A) x P(B given A); drawing two cards without replacement is the classic case, because the first draw changes what remains.

Combining the rules

The real skill is recognizing which rule a problem needs, and many problems chain several together. "At least one head in two flips" is far easier by complement than by listing cases: P(at least one head) = 1 - P(no heads) = 1 - (0.5 x 0.5) = 1 - 0.25 = 0.75. The pattern generalizes: the probability of at least one success in n independent tries is 1 - (probability of failure)^n. For example, the chance of rolling at least one six in four rolls of a fair die is 1 - (5/6)^4 = 1 - 0.482 = 0.518, a little better than even. Spotting that "at least one" invites the complement, and that "and" with independence invites multiplication, resolves the large majority of introductory probability questions.

Inference works only if the sample fairly represents the population it comes from. This lesson covers two linked ideas: how to sample well so your data is trustworthy, and how a sample statistic such as the mean behaves when you imagine repeating the study many times. That second idea, the sampling distribution, is the conceptual heart of all inference.

Sampling methods

A simple random sample gives every individual an equal chance of selection, and every group of the chosen size an equal chance too; it is the gold standard because it removes human choice from who gets picked. Other probability sampling methods also use randomness in structured ways: stratified sampling divides the population into meaningful groups (strata) such as age brackets and randomly samples within each, guaranteeing representation of every group; cluster sampling randomly selects whole pre-existing groups (such as entire schools) and measures everyone in them, which is cheaper when the population is geographically spread; and systematic sampling takes every kth individual from a list after a random start. What all of these share is randomness, which is the only reliable protection against bias.

Bias and its sources

Bias is a systematic tendency to over- or under-represent part of the population, and unlike random error it does not shrink as the sample grows. A convenience sample (just whoever is easy to reach) is the classic source: an online poll captures only people who visit that site and choose to respond. Voluntary response bias, undercoverage (some groups left off the sampling list entirely), and nonresponse bias all push a sample away from the truth in ways more data cannot fix. The infamous 1936 Literary Digest poll predicted the wrong US presidential winner despite millions of responses, precisely because its list of car and telephone owners skewed wealthy. The lesson is stark: a small random sample beats a huge biased one.

The idea of a sampling distribution

A single sample gives one value of a statistic, such as one sample mean x-bar. If you drew a fresh random sample, you would get a slightly different x-bar, and a third sample a different one again. The sampling distribution of the mean is the distribution of x-bar over all possible samples of a fixed size n from the population. It is a distribution not of raw data but of a statistic, and it has three properties that make inference possible:

• Its center equals the population mean. So x-bar is an unbiased estimator: across all possible samples, it neither systematically overshoots nor undershoots the target.
• Its spread is smaller than the spread of individual values, and it shrinks as n grows. Larger samples give more consistent estimates because averaging cancels out individual highs and lows.
• Its shape becomes more bell-shaped as n grows, whatever the shape of the population (the Central Limit Theorem, next lesson, makes this precise).

Standard error of the mean

The standard deviation of the sampling distribution has a special name, the standard error, to distinguish it from the standard deviation of the raw data. For the sample mean it equals the population standard deviation divided by the square root of the sample size: standard error = (population standard deviation) / (square root of n).

Worked example. A population has standard deviation 15. For samples of size n = 25, the standard error of the mean is 15 / (square root of 25) = 15 / 5 = 3. Sample means will typically fall about 3 units from the true mean. Now quadruple the sample size to n = 100: the standard error becomes 15 / 10 = 1.5. Notice the pattern: quadrupling n only halved the standard error, because it is the square root of n in the denominator. This square-root-of-n law is one of the most important facts in statistics. It explains why precision improves with bigger samples, but with diminishing returns: to cut your error in half you must quadruple your sample, and to cut it to a tenth you need a hundredfold sample. That trade-off drives the economics of every survey and experiment.

The Central Limit Theorem (CLT) is the reason so much of statistics leans on the normal curve, and it is arguably the single most important result in the entire subject. It describes the shape of the sampling distribution of the mean, and in doing so it licenses nearly every technique in the final two modules.

Statement of the theorem

The CLT says: for a population with any mean and standard deviation, the sampling distribution of the sample mean x-bar becomes approximately normal as the sample size n grows, regardless of the shape of the population. Its center stays at the population mean, and its standard deviation is the standard error, (population standard deviation) divided by (square root of n). Three facts are bundled together here: where the distribution is centered, how wide it is, and, the new and remarkable part, what shape it takes.

The phrase "regardless of the shape" is what makes the theorem astonishing. Even if the population is heavily skewed, bimodal, or lumpy, the distribution of the average of a sample smooths out toward a symmetric bell as n increases. Individual values might be wildly non-normal, but their means behave themselves. A common rule of thumb is that n of at least 30 is usually enough for the approximation to be good; when the population is already fairly symmetric, much smaller samples suffice, and when it is extremely skewed, you may need more.

Why it matters

The CLT lets us attach probabilities to sample means using the familiar normal model, which is the engine behind confidence intervals and hypothesis tests in the next module. Because x-bar is approximately normal, we can standardize it exactly as we standardized raw values, but using the standard error in place of the raw standard deviation: z = (x-bar - mean) / standard error. That one substitution is the whole trick that carries the normal machinery from single values to sample means.

Worked example

A population has mean 100 and standard deviation 15. You take a random sample of n = 25. What is the probability the sample mean exceeds 106?

First, the standard error is 15 / (square root of 25) = 15 / 5 = 3. By the CLT, x-bar is approximately normal with mean 100 and standard error 3. Standardize the value 106: z = (106 - 100) / 3 = 6 / 3 = 2.0. From the empirical rule, only about 2.5% of a normal distribution lies beyond z = 2. So the probability that the sample mean exceeds 106 is about 0.025. Note carefully that we used the standard error (3), not the raw standard deviation (15), because we are asking about an average of 25 values, not a single individual. Had we asked about one individual exceeding 106, we would have used 15 and found z = 0.4, a completely different and much larger probability. Confusing the two is the most common CLT error.

A second look at the role of n

It is worth separating the two things that n controls. As n grows, the sampling distribution gets narrower (the standard error shrinks like the square root of n) and more normal in shape. These are distinct effects. Even for a fixed, small n, a normal population already produces a perfectly normal sampling distribution; the CLT's contribution is specifically the shape improvement for non-normal populations. Understanding this keeps you from over- or under-trusting the normal approximation: with a symmetric population you can lean on it at modest n, while with a severely skewed population you should be cautious until n is comfortably large.

A single sample mean is a point estimate: our single best guess for the population mean. But a point estimate is almost never exactly right, and reporting it alone hides how uncertain it is. A confidence interval fixes this by reporting a whole range of plausible values, together with a stated level of confidence that the range captures the true parameter. It is the honest way to answer "what is the population mean?" because it reports both an estimate and its uncertainty in one statement.

Structure of a confidence interval

Every confidence interval for a mean has the same form: point estimate plus or minus margin of error, where margin of error = (critical value) x (standard error). The point estimate centers the interval, the standard error measures how much the estimate bounces from sample to sample (from Module 3), and the critical value scales that bounce up to the confidence level you want. When the population standard deviation is known and the sampling distribution is normal, the critical value is a z-value read from the standard normal distribution. The three most common levels are:

These critical values come straight from the empirical rule and the normal curve: 95% of a normal distribution lies within 1.96 standard errors of center, which is exactly why 1.96 builds a 95% interval.

Worked example

A sample of n = 36 has a mean of x-bar = 50, and the population standard deviation is 12. Build a 95% confidence interval for the population mean.

• Standard error = 12 / (square root of 36) = 12 / 6 = 2.
• Critical value for 95% confidence is z = 1.96.
• Margin of error = 1.96 x 2 = 3.92.
• Interval = 50 plus or minus 3.92 = (46.08, 53.92).

We are 95% confident the true population mean lies between 46.08 and 53.92.

What "95% confident" really means

This is the most misunderstood idea in the course, so read it slowly. The confidence level describes the procedure, not one particular interval. If we repeated the sampling many times and built a 95% interval from each fresh sample, about 95% of those many intervals would contain the true mean and about 5% would miss it. For the one interval we actually computed, (46.08, 53.92), the true mean is either inside it or it is not; there is no probability left to assign, because nothing about that fixed interval or the fixed parameter is random anymore. The 95% lives in the long-run success rate of the method, not in any single result. The tempting but wrong reading, "there is a 95% probability the mean is between 46.08 and 53.92," misplaces the randomness, treating a fixed unknown constant as if it were random.

Width, confidence, and sample size

Two levers change the margin of error, and they pull against each other. Raising the confidence level (say to 99%) uses a bigger critical value, which widens the interval: more confidence costs precision, because to be surer of catching the mean you must cast a wider net. Increasing the sample size shrinks the standard error, which narrows the interval at any confidence level: more data buys precision. For the same data at 90% confidence, the margin would be 1.645 x 2 = 3.29, giving the narrower interval (46.71, 53.29); at 99% it would be 2.576 x 2 = 5.15, giving the wider (44.85, 55.15). There is always a trade-off between how confident and how precise you want to be, and because the standard error falls only with the square root of n, buying a much narrower interval at a fixed confidence level requires a great deal more data.

A hypothesis test is a formal procedure for deciding whether sample data provides convincing evidence against a claim about a population. The logic mirrors a courtroom: we presume innocence (a claim of "no effect") and reject it only when the evidence is strong enough that innocence becomes implausible. Understanding this structure is what separates using statistics from merely computing it.

The two hypotheses

Every test pits two claims against each other. The null hypothesis (H0) is the default, skeptical position, usually "no difference" or "no effect," and it is always stated with an equals sign, for example H0: mean = 100. The alternative hypothesis (Ha) is what we suspect might be true instead, such as Ha: mean is not equal to 100 (a two-sided alternative), or a one-sided version like Ha: mean is greater than 100. A crucial asymmetry: we never try to prove H0. We either find enough evidence to reject it or we do not, just as a jury returns "guilty" or "not guilty," never "proven innocent." The alternative is what you set out to demonstrate, so it should be chosen before seeing the data.

Test statistic and p-value

We compress the sample into a single test statistic that measures how far the data falls from what H0 predicts, expressed in standard-error units (so it is essentially a z-score for the sample statistic). From it we compute the p-value: the probability of getting a result at least as extreme as the one observed, assuming H0 is true. Read that definition carefully, because every word matters. The p-value is computed in a hypothetical world where the null holds, and it asks how surprising our data would be there. A small p-value means the data would be very unlikely if H0 were true, which counts as evidence against H0. A large p-value means the data is unremarkable under H0, so H0 survives.

The decision rule

Before looking at data we fix a significance level alpha, most commonly 0.05, which is the threshold of surprise we require. Then:

• If the p-value is less than or equal to alpha, we reject H0; the result is called "statistically significant."
• If the p-value is greater than alpha, we fail to reject H0; there is not enough evidence.

We deliberately say "fail to reject," never "accept," because absence of evidence is not evidence of absence. Not convicting a defendant does not prove them innocent; it only means the case was not strong enough.

Worked example

A machine is supposed to fill bottles to a mean of 100 ml, and we suspect it is off in either direction. Set H0: mean = 100 versus Ha: mean is not equal to 100. Suppose the population standard deviation is 15 ml, we sample n = 25 bottles, and the sample mean is x-bar = 105. The standard error is 15 / 5 = 3, so the test statistic is z = (105 - 100) / 3 = 5 / 3 = 1.67. For a two-sided test at alpha = 0.05, the critical values are plus or minus 1.96, marking off the most extreme 5% split between the two tails. Since 1.67 falls inside that range (equivalently, its two-sided p-value is about 0.095, which exceeds 0.05), we fail to reject H0. The data does not give convincing evidence the machine is miscalibrated, even though 105 differs from 100, because a gap that size is not surprising for a sample of 25.

Two kinds of error, and the meaning of significance

Because we decide under uncertainty, two mistakes are possible, and no procedure can eliminate both. A Type I error is rejecting a true H0 (a false alarm, convicting the innocent); its probability is exactly alpha, which is why we set alpha small. A Type II error is failing to reject a false H0 (a missed effect, acquitting the guilty); its probability is called beta, and 1 - beta is the test's power. The two trade off: lowering alpha guards against false alarms but makes misses more likely, unless you compensate with a larger sample. One final caution about the word "significant": it means only that an effect is detectable, not that it is large or important. With a huge sample, a trivially small, practically meaningless difference can be highly statistically significant. Statistical significance answers "is there an effect?"; it does not answer "does the effect matter?", which requires looking at the effect's actual size.

In real studies we almost never know the population standard deviation; if we do not even know the mean we are testing, we surely do not know its spread. We estimate the spread with the sample standard deviation s, and that extra layer of estimation means the z-distribution is no longer exactly right. The fix, one of the most practical tools in statistics, is the t-distribution.

The t-distribution

Developed by William Gosset, a chemist at the Guinness brewery who published under the pen name "Student" because his employer forbade staff from publishing, the t-distribution is bell-shaped and symmetric like the normal curve but has heavier tails. Those fatter tails are exactly the mathematical expression of the extra uncertainty introduced by estimating the standard deviation from the sample: because s itself wobbles from sample to sample, extreme test statistics are a bit more likely than the normal would predict, so the critical values are pushed slightly farther out. The precise shape depends on the degrees of freedom, which for a one-sample test equal n - 1. As the sample size grows, s becomes a reliable estimate of the true spread, the tails thin out, and the t-distribution converges to the normal. For n above about 30 the two are nearly indistinguishable, which is why large-sample z and t procedures agree.

The one-sample t statistic

To test H0: mean = mu0, the test statistic is t = (x-bar - mu0) / (s / square root of n). It has exactly the same form as a z-statistic (observed minus expected, divided by the standard error) but with the sample standard deviation s standing in for the unknown population value. The result is compared against a t-distribution with n - 1 degrees of freedom rather than the standard normal. Everything you learned about hypotheses, p-values, and the reject/fail-to-reject decision carries over unchanged; only the reference distribution swaps.

Worked example

A training program claims to change a mean score from 50. A sample of n = 25 participants has x-bar = 52 and s = 8. Test H0: mean = 50 against Ha: mean is not equal to 50 at alpha = 0.05.

• Standard error = s / square root of n = 8 / 5 = 1.6.
• Test statistic = t = (52 - 50) / 1.6 = 2 / 1.6 = 1.25.
• Degrees of freedom = n - 1 = 24.

For a two-sided test with 24 degrees of freedom at alpha = 0.05, the critical t-value is about 2.064 (slightly larger than the z-value of 1.96, reflecting the heavier tails). Since our t of 1.25 is smaller in magnitude than 2.064 (equivalently, the two-sided p-value exceeds 0.05), we fail to reject H0. The program has not shown a statistically significant change; a 2-point gain in a sample of 25, with this much variability, is well within what chance alone produces.

Three flavors of t-test

The same t machinery adapts to three common study designs, and choosing correctly among them is the main practical skill:

• One-sample t-test: compares one sample mean to a fixed benchmark value (as above). Use it when you have one group and a target number.
• Two-sample (independent) t-test: compares the means of two separate, unrelated groups, such as a treatment group versus a control group of different people. Use it when two distinct groups are measured once each.
• Paired t-test: compares two measurements on the same individuals, such as each person's before-and-after score; you compute each person's difference and test whether the mean difference is zero. Use it when the same subjects are measured twice, or when subjects are naturally matched.

Why pairing matters

The paired design is not just a bookkeeping choice; it is often far more powerful. By looking at within-person differences, a paired test cancels out the large person-to-person variation that would otherwise swamp a modest treatment effect. If everyone's blood pressure drops by roughly 5 points after a drug, that consistent shift is easy to detect in the differences even though people's baseline blood pressures vary enormously. Feeding the same data into an independent two-sample test would bury the 5-point signal under that baseline noise. So when a design allows pairing, using it, and analyzing it as paired, can turn an undetectable effect into a clear one. Matching the test to the design is therefore not a technicality but a decision that can change the conclusion.

So far every technique has concerned a single variable. To study how two quantitative variables move together, such as height and weight or advertising and sales, we need new tools. We start with a scatterplot, where each point is one individual's (x, y) pair, and then summarize the pattern with a number (correlation) and a line (regression).

The correlation coefficient

The correlation coefficient r measures the strength and direction of a linear relationship between two quantitative variables. It always falls between -1 and +1. A positive r means y tends to rise as x rises (a positive association); a negative r means y tends to fall as x rises; r near 0 means little linear association. Values near plus or minus 1 indicate points hugging a straight line, while values near 0.5 indicate a loose cloud with a visible trend. Correlation is unitless and symmetric: swapping x and y leaves r unchanged, and rescaling the variables (say, centimeters to inches) does not affect it either. The word "linear" is doing heavy lifting: r measures only straight-line association, and a strong curved relationship can have an r near zero.

Least-squares regression

The regression line is the single straight line that best fits the points by making the sum of squared vertical distances (the residuals) as small as possible, which is why it is called the least-squares line. We write it y-hat = b0 + b1 x, where b1 is the slope and b0 is the intercept, and the hat on y signals a predicted value rather than an observed one. The slope tells how much y-hat changes for each one-unit increase in x, and it carries the units of y per unit of x (for example, dollars of sales per dollar of advertising). The intercept is the predicted y when x = 0, which is meaningful only when x = 0 is within or near the range of the data.

Worked example

Five points: x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5. The needed sums are: sum x = 15, sum y = 20, sum xy = 66, sum x-squared = 55, with n = 5, so mean x = 3 and mean y = 4.

The slope is b1 = (n(sum xy) - (sum x)(sum y)) / (n(sum x-squared) - (sum x)^2) = (5(66) - 15(20)) / (5(55) - 15^2) = (330 - 300) / (275 - 225) = 30 / 50 = 0.6.

The intercept is b0 = mean y - b1(mean x) = 4 - 0.6(3) = 4 - 1.8 = 2.2. So the regression line is y-hat = 2.2 + 0.6 x. Notice the line always passes through the point of averages (mean x, mean y) = (3, 4), a fact you can use to check your work: 2.2 + 0.6(3) = 4.

To predict y when x = 6: y-hat = 2.2 + 0.6(6) = 2.2 + 3.6 = 5.8. For this data the correlation works out to r = 0.775, a moderately strong positive linear relationship.

R-squared and the danger of extrapolation

The square of the correlation, r-squared, is the proportion of the variation in y explained by the linear relationship with x. Here r-squared = 0.775^2 = 0.60, so about 60% of the variation in y is accounted for by x, and the remaining 40% is due to other factors and noise. R-squared runs from 0 to 1, and higher means a tighter fit. One serious hazard is extrapolation: using the line to predict far outside the range of the observed x-values. Our data covers x from 1 to 5; predicting at x = 6 is a mild stretch, but predicting at x = 50 would be reckless, because nothing guarantees the linear pattern continues where we have no data.

Correlation is not causation

The most important caution in this lesson: a strong r means two variables move together, but it does not establish that one causes the other. A hidden lurking variable can drive both and manufacture a correlation between things with no causal link. Ice cream sales and drowning deaths correlate strongly, but neither causes the other; hot weather (the lurking variable) drives both. Only a well-designed experiment with random assignment can establish cause and effect, because randomization is what breaks the influence of lurking variables. Observational correlations are valuable for prediction and for generating hypotheses, but on their own they can never prove causation.

Correlation and regression handle two quantitative variables. When the variables are instead categorical, so our data is counts in categories rather than measured numbers, we turn to the chi-square test. Its core idea is beautifully simple: compare the counts we actually observed against the counts we would expect if a stated model were true, and see whether the gap is bigger than chance can explain.

Two common chi-square tests

• Goodness-of-fit test: checks whether one categorical variable follows a claimed distribution. Example: is a six-sided die fair, meaning each face equally likely?
• Test of independence: checks whether two categorical variables are associated, using a two-way table of counts. Example: is recovery (improved or not) associated with treatment group?

The chi-square statistic

Both tests use the same statistic: chi-square = sum of (observed - expected)^2 / expected, added over every category or cell. Each term compares one observed count O to its expected count E: the squared gap (O - E)^2 makes every contribution positive and magnifies large discrepancies, and dividing by E scales the gap relative to how big a count we expected there (a gap of 5 is trivial when we expected 1000 but alarming when we expected 3). Large chi-square values mean the observed counts sit far from what the model predicts, which is evidence against the null hypothesis (that the claimed distribution or independence holds). The chi-square distribution used to judge the statistic is right-skewed and, like the t, its exact shape depends on the degrees of freedom.

Worked example: goodness of fit

A die is rolled 60 times. If it is fair, each face is expected 60 / 6 = 10 times. Observed counts are 8, 12, 9, 11, 10, 10 for faces 1 through 6. The contributions to chi-square are:

Adding the last column, chi-square = 0.4 + 0.4 + 0.1 + 0.1 + 0 + 0 = 1.0. The degrees of freedom for goodness of fit are (number of categories - 1) = 6 - 1 = 5. A chi-square of only 1.0 with 5 degrees of freedom is small (its critical value at alpha = 0.05 is 11.07, far above 1.0, so the p-value is large), and we fail to reject the claim that the die is fair. The small wobbles from 10 are exactly what chance produces in 60 rolls.

Test of independence and expected counts

For a two-way table, the expected count in any cell, under the null of independence, is (row total x column total) / grand total. This formula is just the multiplication rule for independent events applied to counts. Consider 100 patients in a 2-by-2 table (treatment versus control, improved versus not):

The expected count for the treatment-improved cell is (40 x 50) / 100 = 20. Working out all four cells (the expected counts are 20, 20, 30, 30) and summing (O - E)^2 / E gives chi-square = 16.67. The degrees of freedom for an independence test are (rows - 1)(columns - 1) = (2 - 1)(2 - 1) = 1. A chi-square of 16.67 with 1 degree of freedom is very large, well beyond the 0.05 critical value of 3.84, so we reject independence: improvement and treatment group are associated. The treatment group improved far more often (30 of 40, or 75%) than the control group (20 of 60, or 33%), and that gap is too large to be chance.

Conditions and cautions

Two practical points. First, the chi-square test needs counts, not percentages or averages; if you are handed proportions you must convert back to raw counts using the sample size before testing. Second, the approximation that the statistic follows a chi-square distribution requires the expected counts to be reasonably large, a common rule being that every expected count should be at least 5. When some expected counts are tiny (as in a sparse table), the approximation breaks down and an exact method such as Fisher's exact test is used instead. As always, rejecting independence tells you the variables are associated, not that one causes the other; the correlation-is-not-causation lesson applies with equal force to categorical data.