Advanced Quantitative Methods

Doctoral students often learn statistics as a bag of disconnected tests: a t-test here, an ANOVA there, a correlation somewhere else, chosen from a flowchart. This course begins by dismantling that view. Nearly every technique you will use is a special case of one framework, the general linear model (GLM). Learning the GLM once, deeply, is far more powerful than memorizing a dozen tests, because it tells you what the tests share and where they differ.

The equation

The GLM expresses an outcome as a weighted sum of predictors plus error. For a single case i:

Y_i = β₀ + β₁X_1i + β₂X_2i + ... + β_kX_ki + ε_i

The right-hand side has two parts. The systematic (deterministic) component, everything except the error, is the model's prediction: the part of Y explained by the predictors. The random component, the error term ε_i, is everything the model does not capture for case i. Statistical modeling is the disciplined splitting of an outcome into a part we can explain and a residual we treat as noise.

• β₀ is the intercept: the model's predicted value of Y when every predictor equals zero.
• Each β_j is a slope or partial regression coefficient: the predicted change in Y for a one-unit increase in X_j, holding the other predictors fixed.
• ε_i is the residual error for case i, the vertical distance from the observation to the model's prediction.

Why the classic tests are all the GLM

What makes a predictor "count" is not whether it is a number or a category, because categories can be represented numerically (a topic we treat in Module 3). Once you accept that, the familiar tests collapse into one family:

Classic test	As a GLM
Pearson correlation	Regression of Y on one continuous X (standardized)
Independent-samples t-test	Regression of Y on one two-level group indicator
One-way ANOVA	Regression of Y on a set of group indicators
Multiple regression	Regression of Y on several predictors of any type
ANCOVA	Regression of Y on group indicators plus a continuous covariate

Every entry in that table estimates coefficients by the same criterion and tests them the same way. A t-test comparing two group means is numerically identical to a regression with one dummy predictor: the slope equals the difference in means, and the test of the slope is the t-test. This is not an analogy; it is an identity.

The assumptions travel with the model

Because these tests are one model, they share one set of assumptions, which we develop across this course: the relationship is linear in the parameters, the errors are independent, have constant variance (homoscedasticity), and are approximately normal, and the predictors are measured without gross error. When you later read that a t-test "assumes normality," you are seeing one instance of the GLM's error assumption. Learn the assumptions once and you have learned them for the whole family.

Throughout the course, when a new technique appears, ask the orienting question: how does this extend the GLM? Logistic regression changes the outcome's distribution and adds a link function; multilevel models relax the independence-of-errors assumption; factor analysis models the covariance among many indicators. Each is a principled modification of the same skeleton, and seeing the skeleton keeps you from getting lost.

The GLM leaves one question open: given data, how do we choose the coefficients? The dominant answer is ordinary least squares (OLS), and understanding its logic clarifies both what regression delivers and where it can mislead.

The least-squares criterion

For each case the residual is the observed value minus the predicted value, e_i = Y_i minus Y-hat_i. OLS chooses the coefficients that minimize the sum of squared residuals, the quantity Σe_i². Why square them? Two reasons. First, residuals above and below the line have opposite signs; squaring prevents them from canceling, so the criterion measures total misfit rather than net misfit. Second, squaring penalizes large misses disproportionately, which yields a unique, smooth solution obtainable in closed form. A consequence worth remembering is that OLS is sensitive to outliers, because one far-off point contributes its squared distance to the total.

The simple-regression formulas

For one predictor, the least-squares slope and intercept are:

b₁ = Σ(X_i - X̄)(Y_i - Ȳ) / Σ(X_i - X̄)²    and    b₀ = Ȳ - b₁X̄

The slope numerator is the covariance of X and Y (up to a constant); the denominator is the variance of X (up to the same constant). So the slope is covariance divided by the variance of the predictor. The intercept formula guarantees the line passes through the point of means (X̄, Ȳ), a fact you can use to sanity-check any fitted line.

A worked example

Suppose we regress exam score (Y) on hours studied (X) for five students and, from the data, obtain Σ(X - X̄)(Y - Ȳ) = 40 and Σ(X - X̄)² = 10, with X̄ = 4 and Ȳ = 70. Then:

• b₁ = 40 / 10 = 4. Each additional hour of study predicts a 4-point higher exam score.
• b₀ = 70 minus (4)(4) = 70 minus 16 = 54. A student who studied zero hours is predicted to score 54.
• Prediction for 6 hours: Y-hat = 54 + 4(6) = 78.

Notice the interpretive caution the intercept demands: predicting Y at X = 0 is only meaningful if X = 0 is within, or near, the observed range. Predicting an exam score for negative study hours would be nonsense, and predicting far outside the data range is extrapolation, which the model cannot justify.

What OLS does and does not promise

OLS gives the line that minimizes squared vertical error in this sample. Under the GLM assumptions it also has attractive theoretical properties: the Gauss-Markov theorem shows OLS is the best linear unbiased estimator (BLUE), meaning that among all linear unbiased estimators it has the smallest variance. But "best linear unbiased" is conditional on the assumptions holding. When errors are heteroscedastic or correlated, OLS remains unbiased but is no longer most efficient and its standard errors are wrong, a theme we return to in Module 3. For now, hold two ideas together: OLS is a precise, well-defined recipe, and its guarantees are only as good as the assumptions behind them.

Once a line is fitted, we ask how well it fits. The answer rests on partitioning the outcome's variability into a part the model explains and a part it does not, an idea that recurs in ANOVA, multilevel models, and beyond.

The three sums of squares

Consider the deviation of each observation from the mean of Y. That total deviation splits into two pieces: the part captured by the fitted line and the part left over.

• Total sum of squares (SS_total) = Σ(Y_i - Ȳ)². The total variability in the outcome around its mean.
• Regression (model) sum of squares (SS_reg) = Σ(Y-hat_i - Ȳ)². The variability the model explains, the spread of the predictions around the mean.
• Residual (error) sum of squares (SS_res) = Σ(Y_i - Y-hat_i)². The variability the model fails to explain.

The identity SS_total = SS_reg + SS_res holds exactly for OLS with an intercept. This clean decomposition is the arithmetic heart of the GLM.

R-squared

The coefficient of determination is the fraction of total variance the model explains:

R² = SS_reg / SS_total = 1 - (SS_res / SS_total)

If SS_total = 200 and SS_res = 50, then R² = 1 minus (50/200) = 0.75: the model accounts for 75 percent of the variance in Y. In simple regression R² equals the squared Pearson correlation, which is why r = 0.5 corresponds to only 25 percent of variance explained, a useful antidote to overreading modest correlations.

The problem R-squared cannot solve, and adjusted R-squared

R² never decreases when you add a predictor, even a useless one, because OLS can always exploit chance associations to shave a little off SS_res. This makes raw R² a poor guide for comparing models with different numbers of predictors. Adjusted R² corrects for this by penalizing model complexity, using the degrees of freedom:

R̄²_adj = 1 - (1 - R²) · (n - 1) / (n - k - 1)

where n is the sample size and k the number of predictors. Adjusted R² can fall when a predictor adds nothing, which is exactly the behavior we want for model comparison.

The overall F-test

The F-test asks whether the model as a whole explains significantly more variance than an intercept-only (mean) model. It compares mean squares, each a sum of squares divided by its degrees of freedom:

F = (SS_reg / k) / (SS_res / (n - k - 1)) = MS_reg / MS_res

Under the null hypothesis that all slopes are zero, this ratio follows an F distribution. A large F, and correspondingly small p-value, indicates the predictors jointly carry real explanatory power. Note the division of labor: the F-test evaluates the model globally, whereas the t-tests on individual coefficients evaluate each predictor's unique contribution. It is possible, especially under collinearity (Module 3), for the F-test to be significant while no single coefficient is, because the predictors share explanatory work.

The move from one predictor to several is where regression becomes a workhorse of the social and biomedical sciences, because it lets us estimate the association of one variable with an outcome while statistically holding others constant. Reading these coefficients correctly, and knowing the limits of the control they provide, is the central interpretive skill of this module.

The meaning of a partial coefficient

In the model Y = b₀ + b₁X₁ + b₂X₂ + e, the coefficient b₁ is the predicted change in Y for a one-unit increase in X₁ with X₂ held constant. It is a partial coefficient: it isolates the part of X₁'s association with Y that is not shared with X₂. Equivalently, b₁ is what you would get by removing X₂'s linear influence from both Y and X₁ and then regressing the two residualized variables on each other. This is why adding a predictor can change, shrink, or even flip the sign of an existing coefficient: the coefficient answers a different question once the set of things held constant changes.

A concrete reading

Suppose a wage regression yields Wage = 20000 + 2500·(years of education) + 800·(years of experience). The education coefficient of 2500 says: among people with the same years of experience, each additional year of education is associated with 2500 more dollars of predicted annual wage. The phrase "among people with the same experience" is not optional decoration; it is the coefficient's definition. Strip it away and you have misstated the result.

Statistical control is not experimental control

It is tempting to treat "controlling for X₂" as if it recreated an experiment in which X₂ was held fixed by design. It does not. Statistical control adjusts only for the variables you include and measure, and only insofar as the linear form is correct. Three limits deserve emphasis:

• Only measured confounders are controlled. A variable you did not include cannot be adjusted for. An omitted variable correlated with both predictor and outcome biases the coefficients that remain (omitted-variable bias, treated in Module 6).
• Control is only as good as measurement. If X₂ is measured with error, adjusting for it is incomplete, and residual confounding survives.
• The functional form must be right. Adjusting linearly for a variable whose true relationship is curved leaves part of its influence unremoved.

These limits are why a regression coefficient, however many covariates accompany it, is not automatically a causal effect. Module 6 develops the conditions under which it can be. For now, the discipline is to state control claims exactly: "adjusting for the measured covariates in the model," never "holding all else equal" as though the model were omniscient.

Suppression: a cautionary pattern

Occasionally a predictor that correlates near zero with the outcome becomes a strong, significant predictor once another variable is added. This suppression effect is not a paradox; it happens when the added variable removes irrelevant variance from the first predictor, sharpening its relationship with Y. Suppression is a vivid reminder that a coefficient's value is a property of the whole model, not of the predictor in isolation. You cannot interpret any single coefficient without knowing what else is in the equation.

A coefficient's raw magnitude depends on the units of its variables, which makes comparing predictors measured on different scales impossible from the raw numbers alone. Standardization addresses this, but it introduces trade-offs that doctoral researchers must weigh deliberately.

Unstandardized coefficients (b)

The unstandardized coefficient b is expressed in the original units: dollars per year of education, millimeters of mercury per milligram of drug. Its great virtue is concrete interpretability, and it is the right choice when the units are meaningful and you want a real-world statement of effect. Its limitation is that you cannot tell from b alone whether education or experience "matters more," because a dollar and a year are not comparable.

Standardized coefficients (beta)

The standardized coefficient, often written β or "beta weight," is what you obtain after converting every variable to z-scores (mean 0, standard deviation 1). It expresses the change in Y in standard deviations per one-standard-deviation change in the predictor. Because all predictors are now on a common standard-deviation scale, their standardized coefficients are roughly comparable within a single model, offering a rough ranking of relative importance. In simple regression the standardized slope equals the Pearson correlation.

	Unstandardized (b)	Standardized (beta)
Units	Original units of X and Y	Standard deviations
Best for	Concrete, real-world statements	Comparing predictors within a model
Comparable across studies?	Yes, if same measures/units	Not safely, since SDs differ by sample

The pitfalls of standardization

Standardized coefficients are seductive but treacherous. First, a standard deviation is a sample-specific quantity: a restricted-range sample has a smaller SD, which inflates the standardized coefficient even when the true relationship is unchanged. This makes betas poorly suited to cross-study comparison, contrary to common belief. Second, standardizing a binary predictor (say, a 0/1 group indicator) produces a coefficient that is hard to interpret, because a "one standard deviation change" in a dichotomy is not a natural quantity. For categorical predictors, keep the unstandardized coefficient and interpret it as a group difference.

Effect size, not just significance

Recall from prior study that statistical significance answers only whether an effect is distinguishable from zero, not whether it is large. Doctoral reporting therefore always pairs tests with effect sizes. In regression the natural effect sizes are the coefficients themselves (with confidence intervals) and variance-explained measures. For a single predictor's unique contribution, partial eta-squared or the closely related squared semipartial correlation reports the share of variance it explains beyond the others. Cohen's rough benchmarks (small, medium, large) exist for these, but benchmarks are field-dependent conventions, not natural constants; a "small" standardized effect can be practically decisive in a high-stakes context, and a "large" one can be trivial if the outcome barely matters. Report the number, its interval, and its practical meaning, and let readers judge.

Every regression result carries an implicit warranty: the inferences are trustworthy if a set of assumptions holds. A skilled analyst does not recite the assumptions; they interrogate them, and they know which violations are fatal to the estimates themselves and which merely corrupt the standard errors. That distinction organizes this lesson.

The assumptions, and what each violation costs

• Linearity. The relationship between predictors and the mean of Y is linear in the parameters. Violation biases the coefficients: a straight line through a curved relationship misestimates the effect everywhere. Detect with a plot of residuals versus fitted values; a systematic curve signals nonlinearity.
• Independence of errors. Residuals are uncorrelated across cases. Violated by clustered data (students in classrooms) or time series (repeated measures). Non-independence usually does not bias the coefficients but badly understates standard errors, producing false precision and inflated significance. This assumption motivates the multilevel models of Module 6.
• Homoscedasticity. The error variance is constant across the range of predictions. Heteroscedasticity (a fan-shaped residual plot) leaves coefficients unbiased but makes standard errors wrong, so confidence intervals and p-values cannot be trusted. Detect with a residuals-versus-fitted plot; remedy with robust (heteroscedasticity-consistent) standard errors or a variance-stabilizing transformation.
• Normality of errors. The residuals are approximately normal. This matters mainly for small-sample inference; by the central limit theorem, coefficient sampling distributions become approximately normal in large samples even when errors are not. Check with a normal quantile-quantile (Q-Q) plot of residuals.
• No perfect multicollinearity. No predictor is an exact linear combination of others (treated in the next lesson).

Reading the residuals-versus-fitted plot

The single most informative diagnostic plots residuals on the vertical axis against fitted values on the horizontal. The ideal is a formless horizontal band centered on zero. A curve indicates nonlinearity; a funnel or fan (spread growing with the fitted value) indicates heteroscedasticity; isolated points far from the band flag potential outliers. One plot thus probes three assumptions at once, which is why it is the first thing an experienced analyst examines.

Influence and outliers

Not every unusual point is equally dangerous. A point can be an outlier in Y (large residual), a point of high leverage (extreme in the predictor space), or, most consequentially, an influential point whose removal would substantially change the coefficients. Cook's distance summarizes influence by combining residual size and leverage; conventionally, points with notably large Cook's distance deserve scrutiny. The correct response to an influential point is investigation, not automatic deletion: it may be a data-entry error, or it may be the most informative observation you have. Report what you found and what you did.

Two structural problems in the predictor set can undermine a regression even when every assumption about the errors holds: predictors that are too intertwined (multicollinearity), and a model chosen by fishing rather than theory. Both are matters of research judgment as much as statistics.

What multicollinearity is

Multicollinearity is high correlation among predictors. When two predictors carry nearly the same information, the model struggles to allocate credit between them. Perfect multicollinearity (one predictor an exact linear combination of others) makes the coefficients mathematically undefined and must be removed. Severe but imperfect multicollinearity has a characteristic signature: coefficients with large standard errors, hence wide confidence intervals and non-significant t-tests, even while the overall model fits well and the F-test is significant. The estimates also become unstable, swinging wildly with small changes in the data.

Detecting it: VIF and tolerance

The standard diagnostic is the variance inflation factor (VIF). For predictor X_j, regress it on all the other predictors and obtain that regression's R²_j. Then:

VIF_j = 1 / (1 - R²_j)    and    Tolerance_j = 1 - R²_j = 1 / VIF_j

A VIF of 1 means the predictor is uncorrelated with the others; larger values mean its standard error is inflated by that factor's square root. If X_j is 90 percent predictable from the others (R²_j = 0.90), then VIF = 1/(1 minus 0.90) = 10, a common rule-of-thumb threshold for concern. Remedies include dropping or combining redundant predictors, collecting more data, or, if the collinear set is theoretically a single construct, forming a composite or using the factor-analytic methods of Module 6. Crucially, multicollinearity does not bias the coefficients; it inflates their variance. If your interest is prediction rather than isolating individual effects, moderate collinearity may not matter at all.

The trouble with stepwise selection

Faced with many candidate predictors, software offers stepwise selection (forward, backward, or both), which adds or removes predictors according to significance thresholds. It is popular and, for confirmatory inference, largely indefensible. The problems are severe:

• The p-values and confidence intervals it reports are invalid, because the same data were used to choose the variables and to test them, and this selection is not acknowledged in the final inference.
• It capitalizes on chance, selecting predictors that fit sample-specific noise, so results replicate poorly.
• It substitutes an algorithm for theory, producing models that may fit yet make no substantive sense.

Principled alternatives

For explanatory research, build the model from theory: include predictors because a defensible argument says they belong, and pre-specify the model before seeing the outcomes where possible. When the goal is genuinely predictive and you must select among many variables, use methods designed for it, such as penalized regression (ridge or lasso, which shrink coefficients to control overfitting) evaluated by cross-validation on held-out data rather than in-sample p-values. The governing principle is the same one from Module 1 of the companion methods course: a model chosen by exploring the data cannot then be honestly "confirmed" on that same data.

The claim in Module 1 that ANOVA is just regression rests on a technique: representing categories as numbers. And the most substantively interesting regressions involve interactions, where the effect of one variable depends on another. Both live here.

Dummy coding a categorical predictor

A categorical variable with g levels is entered as g minus 1 dummy (indicator) variables, each coded 0 or 1. One level is omitted and becomes the reference category. Consider a predictor "treatment group" with levels Control, Drug A, Drug B. Choosing Control as reference, we create two dummies: D_A = 1 for Drug A else 0, and D_B = 1 for Drug B else 0. The model Y = b₀ + b₁D_A + b₂D_B yields:

• b₀ = the mean of the reference group (Control), since both dummies are 0 there.
• b₁ = the difference between Drug A's mean and Control's mean.
• b₂ = the difference between Drug B's mean and Control's mean.

So each dummy coefficient is a comparison against the reference, and the choice of reference category changes the coefficients' values though not the model's fit. Why g minus 1 dummies rather than g? Including one dummy per level plus an intercept creates perfect multicollinearity (the dummies would sum to a column of ones, identical to the intercept), the so-called dummy-variable trap. Omitting one level resolves it.

Interactions: when effects depend on other variables

A model with only additive terms assumes each predictor's effect is the same at every value of the others. That assumption often fails. An interaction (moderation) term relaxes it. For two predictors X and Z, we add their product:

Y = b₀ + b₁X + b₂Z + b₃(X·Z) + e

The coefficient b₃ on the product is the interaction: it tells you how the slope of X changes for each one-unit increase in Z. Rearranging shows this explicitly, since the effect of X is (b₁ + b₃Z), a conditional slope that depends on Z.

The interpretation trap you must avoid

When an interaction is in the model, b₁ and b₂ are no longer main effects. They are simple (conditional) effects at the value zero of the other variable. Specifically, b₁ is the effect of X when Z = 0, and b₂ is the effect of Z when X = 0. If zero is not a meaningful or observed value of Z, then b₁ describes a case that may not exist, and reading it as "the effect of X" is a serious error. This is why analysts center continuous predictors (subtract their mean) before forming an interaction: after centering, zero is the mean, so b₁ becomes the effect of X at the average level of Z, a far more interpretable quantity. Centering also reduces the nonessential collinearity between the product term and its components. To understand a significant interaction, compute and plot simple slopes: the slope of X at low, mean, and high values of Z (often the mean and one standard deviation on either side). A significant b₃ tells you the slopes differ; the simple-slopes analysis tells you how.

Analysis of variance is taught as a distinct method, but Module 1 promised it is regression with categorical predictors. Here we cash that promise and add a continuous covariate to reach ANCOVA, keeping the unifying GLM view throughout.

One-way ANOVA

One-way ANOVA compares the means of a continuous outcome across the levels of one categorical factor. It partitions variance exactly as regression does: total variability splits into between-groups variability (differences among the group means) and within-groups variability (spread inside each group). The test statistic is the ratio of the corresponding mean squares:

F = MS_between / MS_within

A large F says the group means are more spread out than within-group noise would predict. This is identical to the overall F-test of a regression whose predictors are the group dummies. The null hypothesis is that all group means are equal. Importantly, a significant omnibus F says only that the means are not all equal; it does not say which differ. That requires post-hoc comparisons (such as Tukey's HSD) that control the familywise error rate inflated by making many pairwise tests, or planned contrasts specified in advance.

Factorial ANOVA and interaction

With two or more factors, factorial ANOVA estimates each factor's main effect and, crucially, their interaction, the same product-term logic from Module 3. A significant interaction means the effect of one factor depends on the level of the other, and it takes interpretive priority: when an interaction is present, main effects can be misleading because they average over conditions in which the effect genuinely differs. Always interpret the interaction first, then the simple effects within it.

ANCOVA: adding a covariate

Analysis of covariance (ANCOVA) adds one or more continuous covariates to an ANOVA. In GLM terms it is simply a regression with both group dummies and a continuous predictor. It serves two purposes:

• Increased power. A covariate that predicts the outcome absorbs residual variance, shrinking MS_error and making group differences easier to detect.
• Adjustment. It compares adjusted means: the group means one would expect if every group had the same average value of the covariate. In a pretest-posttest design, adjusting the posttest for the pretest sharpens the comparison.

The assumption ANCOVA rests on

ANCOVA's validity depends on homogeneity of regression slopes: the relationship between the covariate and the outcome must be the same in every group, meaning the covariate does not interact with the grouping factor. If the slopes differ, the single adjusted comparison is meaningless because the group gap depends on where you are along the covariate, and you have, in fact, discovered an interaction that should be modeled rather than adjusted away. A further caution: in observational data, using ANCOVA to "equate" pre-existing groups on a covariate can mislead, because groups that differ naturally may differ on unmeasured variables too. Adjustment is not randomization, a theme Module 6 develops.

Many outcomes are not continuous: a patient recovers or does not, a student graduates or does not, a loan defaults or does not. Forcing a 0/1 outcome into ordinary regression fails badly, and repairing it introduces the wider framework of the generalized linear model, of which OLS is one member.

Why OLS breaks on a binary outcome

Regressing a 0/1 outcome directly (the "linear probability model") violates the framework in ways that matter: predictions can fall below 0 or above 1, which are impossible probabilities; the errors are necessarily heteroscedastic (their variance depends on the predicted probability); and the linear form poorly captures a relationship that must saturate near 0 and 1. We need a model that keeps predictions in [0, 1] and respects the binary nature of the data.

The logistic solution: link and odds

Logistic regression models not the outcome directly but the log-odds of the outcome, which are unbounded and so can be a linear function of the predictors. Define the odds of an event with probability p as p/(1 minus p). The natural log of the odds is the logit. The model is:

ln( p / (1 - p) ) = b₀ + b₁X₁ + ... + b_kX_k

The right side is the familiar linear predictor; the left side, the logit, is the link function that connects it to a probability. Solving back for p gives the S-shaped logistic (sigmoid) curve that stays neatly between 0 and 1. Because the outcome is not normal, coefficients are estimated by maximum likelihood rather than least squares: we choose the coefficients that make the observed pattern of successes and failures most probable.

Reading the coefficients: log-odds and odds ratios

A raw logistic coefficient b_j is the change in the log-odds of the outcome per one-unit increase in X_j. Log-odds are not intuitive, so we exponentiate: e^b_j is the odds ratio (OR), the multiplicative change in the odds per one-unit increase in the predictor. Reading it:

• OR = 1 (equivalently b = 0): the predictor does not change the odds.
• OR > 1 (b positive): the odds increase; OR = 1.5 means the odds are 1.5 times as large, a 50 percent increase per unit.
• OR < 1 (b negative): the odds decrease; OR = 0.8 means the odds are multiplied by 0.8, a 20 percent reduction per unit.

Two interpretive cautions are essential at the doctoral level. First, an odds ratio is not a risk ratio. Odds and probabilities coincide only when the outcome is rare; for common outcomes an odds ratio can be much larger than the corresponding relative risk, and reporting "1.5 times more likely" for an OR of 1.5 is often wrong. Second, the effect on the probability is not constant: because the curve is S-shaped, the same one-unit change moves probability a lot in the middle of the curve and little near 0 or 1. Overall model fit is judged not by R² but by likelihood-based measures such as the likelihood-ratio test (comparing nested models) and information criteria, with pseudo-R² measures reported cautiously as approximate analogs.

Module 3 introduced interactions mechanically. Here we treat them as a substantive research question: moderation asks whether the strength or direction of an X-to-Y relationship depends on a third variable, the moderator. The question "does it always work, or only for some people or in some conditions?" is a moderation question, and it is often more scientifically interesting than the average effect.

Moderation is an interaction

A moderator M changes the effect of X on Y. Statistically, moderation is the interaction term X·M from Module 3; "moderation" is the conceptual name, "interaction" the statistical mechanism. The regression is:

Y = b₀ + b₁X + b₂M + b₃(X·M) + e

The test of moderation is the test of b₃. If b₃ differs significantly from zero, the slope of X on Y changes across levels of M. If b₃ is effectively zero, the effect of X is the same regardless of M, and there is no moderation, whatever the theory hoped.

Direction of moderation

The sign of b₃ tells you the form. Recall the conditional slope of X is (b₁ + b₃M). A positive b₃ means the effect of X grows stronger (more positive) as M increases; a negative b₃ means it weakens or reverses as M increases. A moderator can even flip an effect's sign: a coping strategy that helps under low stress might hurt under high stress, which no single average effect could reveal.

Probing with simple slopes

A significant interaction says the slopes differ but not what they are, so you probe it. The standard approach, the pick-a-point or simple-slopes method, evaluates the slope of X at chosen values of M, conventionally the mean and one standard deviation above and below (labeled, loosely, low, average, and high M). Each simple slope has its own estimate and significance test. A common and informative pattern is a significant conditional effect at high M but not at low M, which you would report as "X predicted Y among high-M participants but not among low-M participants." A more complete alternative, the Johnson-Neyman technique, identifies the entire range of M values over which the effect of X is statistically significant, rather than testing only two or three arbitrary points.

Two disciplines that prevent errors

First, center (or standardize) X and M before forming the product, so the lower-order coefficients are effects at the mean rather than at a possibly meaningless zero, as Module 3 stressed. Second, always plot the interaction: draw the predicted line of Y on X separately for low and high M. The plot converts an abstract product coefficient into a picture a reader can grasp, and it guards against overinterpreting a statistically significant but substantively tiny interaction. Report b₃, its confidence interval, the simple slopes, and the plot together; any one alone can mislead.

Where moderation asks when or for whom an effect holds, mediation asks how or why: it posits a mechanism, an intervening variable through which X transmits its effect to Y. This is among the most used and most abused analyses in the social sciences, so precision here is essential.

The mediation model

A mediator M lies on the causal path X to M to Y. The classic decomposition uses three coefficients: the effect of X on M (path a), the effect of M on Y controlling for X (path b), and the effect of X on Y controlling for M (path c', the direct effect). The indirect effect, the part of X's influence that runs through M, is the product a × b. The total effect of X on Y is:

c = c' + (a × b)    (total = direct + indirect)

When the indirect effect accounts for the entire relationship (c' near zero), the mediation is called full; when a direct effect remains alongside a nonzero indirect effect, it is partial. Modern practice de-emphasizes this labeling and focuses on estimating the indirect effect and its uncertainty.

From causal steps to the indirect effect

An older approach (the Baron and Kenny causal-steps method) required a chain of separate significance tests, including a significant total effect before mediation could be claimed. This has fallen out of favor for good reasons: it has low power, and an indirect effect can exist even when the total effect is non-significant (for instance, if a mediated positive path and a direct negative path partly cancel). Contemporary practice tests the indirect effect a×b directly. Because the product of two normally distributed coefficients is not itself normal, the preferred method is bootstrapping: repeatedly resample the data, recompute a×b in each resample, and use the resulting distribution to form a bias-corrected confidence interval. If that interval excludes zero, the indirect effect is significant. Bootstrapping respects the skewed sampling distribution of the product and generally has more power than the older normal-theory (Sobel) test.

The assumptions people forget

Mediation is a causal claim dressed in regression clothing, and the statistics do not supply the causation. Three assumptions are routinely violated and rarely acknowledged:

• Correct causal order. The model assumes X causes M causes Y. If M and Y are measured at the same time, or if Y could influence M, the estimated "mediation" may be an artifact of a misspecified order. Establishing temporal precedence (X before M before Y) is a design task, not a statistical one.
• No unmeasured confounding of the M-to-Y relationship in particular. Even in an experiment that randomizes X, the mediator M is not randomized; any variable that affects both M and Y confounds path b and biases the indirect effect. Randomizing X does not rescue this.
• No measurement error in the mediator, which otherwise biases the paths, typically attenuating b.

The honest reporting standard is therefore to present mediation from cross-sectional data as a model-consistent account, not proof of mechanism, and to note that alternative causal orderings may fit the same data equally well. A significant bootstrapped indirect effect is necessary for the mediation story, but it is far from sufficient to establish it.

Module 3 flagged that clustered data violate the independence-of-errors assumption. This lesson develops the remedy: multilevel models (also called hierarchical linear models, mixed-effects models, or random-effects models), which are indispensable whenever observations are nested.

Nested data and why independence fails

Data are nested (hierarchical) when lower-level units cluster within higher-level units: students within classrooms within schools, patients within hospitals, repeated measurements within persons. Units in the same cluster tend to resemble one another, so their residuals are correlated, violating independence. Ignoring the nesting and running ordinary regression produces standard errors that are too small, and hence false confidence and inflated Type I error, because the analysis counts correlated observations as if each contributed full independent information. Treating the cluster mean as the unit instead (aggregating) discards within-cluster variation and can produce the ecological fallacy, inferring individual relationships from group-level associations.

The intraclass correlation

The degree of clustering is quantified by the intraclass correlation coefficient (ICC), the proportion of total variance that lies between clusters:

ICC = σ²_between / (σ²_between + σ²_within)

An ICC near zero means clusters barely differ and ordinary regression is roughly acceptable; a substantial ICC signals meaningful clustering that must be modeled. Even a modest ICC, combined with large clusters, can drastically inflate naive significance, so the ICC is not a formality but a decision input.

Random intercepts and random slopes

A multilevel model partitions the coefficients into fixed and random parts.

• A random-intercept model lets each cluster have its own baseline level of the outcome, drawn from a distribution of cluster intercepts. It says: schools differ in average achievement, and we model that spread rather than pretending it away.
• A random-slope model additionally lets the effect of a predictor vary across clusters. It says: the relationship between, say, study time and achievement may be stronger in some schools than others, and that variation is itself of interest.

The fixed effects are the average coefficients across clusters; the random effects are the cluster-specific departures, summarized by their variances (the variance components). A great strength of the approach is partial pooling: estimates for small or noisy clusters are pulled (shrunk) toward the overall mean, borrowing strength across clusters and yielding more stable, less overfit cluster estimates than analyzing each cluster separately.

Cross-level questions

Multilevel models also let predictors at different levels coexist: a student-level predictor (hours studied) and a school-level predictor (school funding) in one model. A cross-level interaction then asks whether a school-level variable moderates a student-level effect, for example whether funding strengthens the study-time-to-achievement slope. These questions are simply unaskable in single-level regression, which is why nested designs demand the multilevel framework rather than treating clustering as a nuisance to be patched over.

Many constructs in the human sciences, depression, motivation, political trust, cannot be measured by a single item without error. The response is to use several items and model the latent variable they share. Factor analysis is the family of methods for doing so, and it links the measurement concerns of the companion methods course to formal statistical modeling.

The latent-variable idea

A latent variable (factor) is an unobserved construct inferred from the pattern of correlations among observed indicators. The organizing assumption is that the latent factor causes the responses: people high in the trait tend to respond high across all its indicators, which is why those indicators correlate. Factor analysis reverses this logic, using the observed correlations to estimate the underlying factors. A crucial conceptual distinction is between reflective indicators, which are effects of the latent variable (the usual case in psychometrics), and formative indicators, which combine to define a composite; the modeling differs, and treating one as the other is a common error.

Exploratory versus confirmatory

• Exploratory factor analysis (EFA) is used when you do not have a firm hypothesis about how many factors underlie a set of items or which items load on which factor. It lets the data suggest a structure. Decisions include how many factors to retain (guided by eigenvalues, the scree plot, and parallel analysis) and how to rotate the solution to make it interpretable. Rotation can be orthogonal (factors kept uncorrelated, as in varimax) or oblique (factors allowed to correlate, usually more realistic for psychological constructs).
• Confirmatory factor analysis (CFA) is used when theory specifies the structure in advance. You fix which items load on which factors and test whether that hypothesized measurement model fits the data, using fit indices (such as the comparative fit index and the root mean square error of approximation). CFA is a special case of structural equation modeling and is the appropriate tool for validating an established scale.

Reading the output

A factor loading is the correlation (in the standardized, orthogonal case) between an indicator and the factor; loadings near plus or minus 1 mean the item strongly reflects the factor, and loadings near zero mean it does not. The communality of an item is the proportion of its variance explained by the retained factors, the sum of its squared loadings; its complement is the item's uniqueness (specific variance plus error). Items with low communality are poorly explained by the factor solution and are candidates for revision or removal. A clean solution shows simple structure: each item loads strongly on one factor and weakly on the others, so the factors are distinct and interpretable.

Why this matters for inference

Factor analysis is not a decorative preliminary; it is how you establish that your measured variables mean what you claim, the construct validity at the heart of measurement. Feeding a poorly validated composite into the regression and multilevel models of this course propagates measurement error into every downstream estimate, attenuating relationships and biasing conclusions. Establishing a sound measurement model first is therefore a prerequisite for trustworthy structural analysis, not an optional flourish.

This course has repeatedly warned that a regression coefficient is not automatically a causal effect. Now we make the conditions explicit. Modern causal inference gives a precise language, the directed acyclic graph, for deciding what to control, and a toolkit of quasi-experimental designs for approximating randomization when it is impossible.

The counterfactual core

A causal effect is a comparison of counterfactuals: the outcome for a unit if treated versus if untreated. Since we never observe both for the same unit (the "fundamental problem of causal inference"), we rely on a comparison group meant to stand in for the unobserved counterfactual. Randomization makes the treated and control groups exchangeable in expectation, which is why it licenses causal claims. Without it, we must argue that adjustment achieves the same balance, and DAGs tell us what adjustment is needed.

Directed acyclic graphs

A DAG encodes assumed causal relationships as arrows among variables. Three configurations govern what you should and should not control:

• A confounder is a common cause of both treatment and outcome (X ← C → Y). It opens a spurious "backdoor" path; you must adjust for it (or a variable that blocks the path) to obtain the causal effect. This is the classic reason to include control variables.
• A mediator lies on the causal path from treatment to outcome (X → M → Y). Adjusting for it removes part of the very effect you want to estimate, so for the total effect you should not control it. This is overcontrol bias.
• A collider is a common effect of two variables (X → K ← Y). It is naturally closed; adjusting for it (or selecting on it) opens a spurious association between its causes. Conditioning on a collider is collider bias, a subtle and underappreciated way that adding a "control" variable can create bias rather than remove it.

The lesson is stark: "controlling for more variables" is not automatically safer. The DAG tells you which variables to include (confounders, to block backdoor paths) and which to leave alone (mediators and colliders). The set of variables that blocks all backdoor paths without opening new ones is a valid adjustment set.

Quasi-experimental designs

When randomization is impossible, several designs exploit features of the world to approximate it:

• Difference-in-differences (DiD) compares the before-to-after change in a treated group with the change in an untreated comparison group. By differencing out fixed group differences and common time trends, it isolates the treatment effect, provided the groups would have followed parallel trends absent treatment, an assumption that must be argued, not assumed.
• Regression discontinuity (RDD) applies when treatment is assigned by whether a continuous running variable crosses a cutoff (a test score threshold for a scholarship, say). Units just above and just below the cutoff are nearly identical except for treatment, so comparing outcomes at the threshold estimates a local causal effect with a credibility approaching that of an experiment.
• Instrumental variables (IV) address unmeasured confounding by finding an instrument that affects the treatment but influences the outcome only through the treatment (the exclusion restriction) and is unrelated to the confounders. The instrument induces variation in treatment that is "as good as random," recovering a causal effect for those whose treatment responds to it.

None of these is a statistical trick that manufactures causation from nothing; each rests on an assumption (parallel trends, no manipulation of the cutoff, a valid instrument) that is itself untestable in full and must be defended substantively. That is the recurring moral of causal inference: the credibility of a causal claim rests on the credibility of its design and assumptions, not on the sophistication of the estimator.

A study too small to detect a real effect wastes resources and, worse, litters the literature with false negatives and unstable estimates. Power analysis is the discipline of planning a study large enough to answer its question, and it is expected in doctoral research and grant proposals alike.

The power quartet

Statistical power is the probability of correctly rejecting a false null, that is, of detecting an effect that truly exists. It equals 1 minus β, the complement of the Type II (false-negative) error rate. Power is determined by four interlocking quantities, any three of which fix the fourth:

• Significance level (α). The Type I error rate, conventionally 0.05. A stricter α (say 0.01) makes rejection harder and thereby lowers power, all else equal.
• Effect size. The magnitude of the true effect. Larger true effects are easier to detect, so power rises with effect size. This is usually the hardest input to specify and the one that most influences the answer.
• Sample size (n). More data means less sampling error and more power. Sample size is the quantity researchers most often solve for.
• Power (1 minus β). The target probability of detection, conventionally set to 0.80, meaning a 20 percent chance of missing a real effect. Higher targets (0.90, 0.95) demand larger samples.

Because these four are locked together, an a priori power analysis fixes α, the desired power, and a plausible effect size, and solves for the required sample size. This is the calculation that belongs in a preregistration or a grant methods section.

Where the effect size comes from

The credibility of a power analysis rests entirely on the effect-size estimate feeding it, and there is no purely statistical way to conjure one. Legitimate sources include meta-analytic averages for the phenomenon, the effect reported by a well-conducted prior study (discounted, since published effects are often inflated by publication bias), or the smallest effect size of interest, the smallest effect that would be practically or theoretically meaningful. The last is often the most defensible: rather than guessing the true effect, you power the study to detect anything large enough to matter, and treat smaller effects as acceptable to miss. Overoptimistic effect sizes are the commonest way power analyses go wrong, yielding underpowered studies dressed in the appearance of planning.

The post-hoc power fallacy

After a study returns a non-significant result, researchers are sometimes asked to compute "observed" or post-hoc power using the effect size actually estimated in that sample. This is uninformative and misleading. Observed power is a deterministic transformation of the p-value: a non-significant result always corresponds to low observed power, so reporting it adds nothing beyond the p-value and cannot explain the null. If you want to interpret a non-significant finding, the right tools are the confidence interval (does it exclude effects large enough to matter?) or an equivalence test (is the effect statistically smaller than a bound you care about?). Power analysis is a design tool to be used before data collection; used afterward on the observed effect, it answers no useful question.

The credibility of the entire quantitative enterprise rests on whether results can be trusted and reproduced. A decade of replication difficulties across fields has made reproducibility a central methodological concern, and doctoral researchers are now expected to build transparency into their workflow from the start rather than bolt it on at publication.

Two kinds of reproduction

Precision matters here. Computational reproducibility means that another analyst, given your data and code, obtains the identical results you reported. It is a minimum, and it is entirely under your control. Replicability is more demanding: a new study, with new data, finds a consistent result. A finding can be perfectly reproducible (your code runs and matches your tables) yet fail to replicate (the effect vanishes in fresh data). This course's models cannot deliver replicability by themselves; they can, and must, be reported reproducibly.

The confirmatory-exploratory distinction

The single most important discipline is to keep confirmatory and exploratory analyses distinct and honestly labeled. Confirmatory analysis tests hypotheses specified before seeing the data; its p-values mean what they claim. Exploratory analysis searches the data for patterns; it is valuable and generative, but its p-values are not valid confirmatory evidence, because searching inflates the chance of finding something. Two practices corrupt this line and inflate false positives:

• P-hacking: trying many analyses, subsets, covariates, or outcome definitions and reporting only those that cross the significance threshold. Each undisclosed choice is a researcher degree of freedom, and exploiting many of them all but guarantees a spurious "finding."
• HARKing (Hypothesizing After the Results are Known): presenting a hypothesis discovered by exploration as though it had been predicted in advance, converting an exploratory result into a fake confirmatory one.

Preregistration and registered reports

The corrective is to commit in advance. Preregistration time-stamps the hypotheses, design, sample-size rationale, and analysis plan in a public registry before data collection, so the confirmatory-exploratory boundary cannot be blurred after the fact. It does not forbid exploration; it requires that exploration be labeled as such. A registered report goes further: the introduction and methods are peer-reviewed and provisionally accepted before data collection, so publication depends on the quality of the question and design rather than on whether the results turn out significant, directly countering publication bias.

The transparent workflow

Concrete habits make an analysis reproducible and let others check it:

• Analyze with scripted code rather than point-and-click steps, so every transformation is recorded and re-runnable; avoid manual edits that leave no trace.
• Share data and code (or a documented synthetic dataset when data are sensitive) in a public repository, with enough documentation for a stranger to reproduce the tables and figures.
• Report all measures, conditions, exclusions, and analytic decisions, including the ones that did not work, so the reader sees the full garden of forking paths rather than the one path that reached significance.
• Report effect sizes and confidence intervals, not p-values alone, so results can be cumulated across studies in future meta-analyses.

These practices are not bureaucratic overhead; they are what separates a durable contribution from a result that evaporates on scrutiny. The sophistication of your models, everything in this course, buys credibility only if the analysis behind them is transparent enough to be believed and reproduced.