Artificial Intelligence

Artificial intelligence is the study of building machines that perceive their environment and take actions that advance their goals. That one sentence hides a deep ambiguity: intelligent compared to what, and measured how? The textbook by Russell and Norvig organizes historical definitions along two axes, giving four families of aims.

	Human-like	Rational (ideal)
Thinking	Systems that think like people (cognitive modeling)	Systems that think correctly (logic, "laws of thought")
Acting	Systems that act like people (the Turing Test)	Systems that act to achieve the best expected outcome (rational agents)

The Turing Test

In 1950 Alan Turing proposed replacing the vague question "can machines think?" with an operational one: the imitation game. A human interrogator converses by text with two hidden participants, one human and one machine, and tries to tell which is which. If the interrogator cannot reliably distinguish them, the machine is said to display intelligent behavior. The test is influential because it sidesteps metaphysics and grades behavior. It is also criticized: John Searle's Chinese Room argument contends that manipulating symbols convincingly is not the same as understanding them, so passing the test would not establish genuine comprehension.

Weak, strong, and narrow AI

Philosophers distinguish weak AI (machines can act as if intelligent) from strong AI (machines actually have minds and understanding). Engineers use a different split: narrow AI solves one specific task, such as recognizing faces or translating text, while artificial general intelligence (AGI) would match human flexibility across arbitrary tasks. Every system deployed today is narrow, however impressive; AGI remains a research aspiration, not a shipped product.

Rationality: the modern organizing idea

Rather than chase a slippery notion of "thinking," this course adopts the rational agent view, the bottom-right cell above. A rational agent selects actions expected to maximize a performance measure, given what it perceives and what it knows. This framing is precise, it lends itself to mathematics, and it does not require the machine to resemble a human. Crucially, rationality is not omniscience: a rational agent does the best it can with the information available, which may still lead to a bad outcome if the world was not fully observable. Almost every technique in the coming modules, whether search, logic, probability, or learning, is a way of building a more nearly rational agent.

Artificial intelligence has advanced in waves of optimism, disappointment, and renewal. Knowing the arc helps you judge current claims with perspective.

The founding and early optimism (1943 to 1969)

McCulloch and Pitts modeled an artificial neuron in 1943. In 1950 Turing framed machine intelligence. The field was named at the 1956 Dartmouth workshop, organized by John McCarthy, where the term "artificial intelligence" was coined. Early programs proved geometry theorems, played checkers, and solved algebra word problems, fueling bold predictions. Frank Rosenblatt's perceptron (1958) learned simple classifications from examples.

The first winter (1970s)

Reality intruded. Many methods that worked on toy problems failed to scale to real ones, partly due to combinatorial explosion. Minsky and Papert's 1969 book showed a single-layer perceptron could not even represent the exclusive-or function, chilling neural network research. Funding dried up in the first AI winter.

Expert systems and the second winter (1980s to early 1990s)

The 1980s brought expert systems, programs encoding human specialist knowledge as if-then rules (for example, MYCIN for medical diagnosis and XCON for configuring computers). Industry invested heavily, but the systems were brittle, hard to maintain, and could not learn. When they underdelivered, a second AI winter followed.

The statistical and learning turn (1990s to 2000s)

AI regrouped around probability and machine learning. Rather than hand-coding rules, systems learned patterns from data. Support vector machines, Bayesian networks, and ensemble methods delivered reliable results, and IBM's Deep Blue defeated world chess champion Garry Kasparov in 1997 through massive search.

The deep learning revolution (2012 to present)

Three forces converged: enormous labeled datasets, powerful parallel hardware (GPUs), and refined training methods. In 2012 a deep neural network called AlexNet won the ImageNet image-recognition contest by a wide margin, igniting the modern era. The 2017 Transformer architecture then enabled large language models. Progress since has been rapid, though the fundamental questions of the 1950s, what intelligence is and how to build it safely, remain open.

An agent is anything that perceives its environment through sensors and acts upon it through actuators. The agent's behavior is described abstractly by an agent function that maps any history of percepts to an action; concretely, that function is implemented by an agent program running on some architecture. Defining the task means specifying the PEAS: Performance measure, Environment, Actuators, and Sensors. For a self-driving taxi, for example, performance might weigh safety and speed, the environment is roads and traffic, actuators are steering and braking, and sensors are cameras and radar.

Classifying environments

How hard an agent's job is depends on the environment. The standard dimensions are:

• Fully vs. partially observable: can the sensors see the complete state, or only part of it?
• Deterministic vs. stochastic: does an action always produce the same next state, or is there randomness?
• Episodic vs. sequential: are decisions independent one-shots, or does the current choice affect future ones?
• Static vs. dynamic: does the world stay put while the agent deliberates, or keep changing?
• Discrete vs. continuous: are states, time, and actions countable or real-valued?
• Single vs. multi-agent: is the agent alone, or must it consider other agents (possibly adversarial)?

Chess is fully observable, deterministic, sequential, static, discrete, and multi-agent. Driving is partially observable, stochastic, sequential, dynamic, and continuous, which is exactly why it is so much harder.

Agent architectures

Agent programs form a rough ladder of sophistication:

1. Simple reflex agents act only on the current percept using condition-action rules. They fail when the right action depends on history they cannot see.
2. Model-based reflex agents keep an internal state that tracks the unobserved parts of the world, updated with a model of how it evolves.
3. Goal-based agents consider the future: they choose actions that lead toward an explicit goal, which is where search and planning enter.
4. Utility-based agents go further, using a utility function to weigh competing goals and handle trade-offs and uncertainty, choosing the action of highest expected utility.
5. Learning agents improve from experience: a learning element tunes the performance element based on feedback from a critic, so the agent gets better over time.

function SIMPLE-REFLEX-AGENT(percept):
    state = INTERPRET-INPUT(percept)
    rule  = RULE-MATCH(state, rules)     # first matching condition-action rule
    return rule.action

This ladder frames the whole course: search and logic build goal-based agents, probability and decision theory build utility-based agents, and the machine learning modules build learning agents.

Many goal-based problems reduce to search: finding a sequence of actions that leads from an initial state to a goal. A search problem is defined by five parts: an initial state, a set of actions available in each state, a transition model saying what each action does, a goal test, and a path cost function. The set of all states reachable from the start forms the state space, which we explore as a tree or graph of nodes.

The general strategy

Search algorithms keep a frontier of nodes waiting to be expanded. To expand a node is to generate its successors. The algorithms differ only in the order they take nodes off the frontier. On graphs we also keep an explored set to avoid revisiting states.

Breadth-first search (BFS)

BFS expands the shallowest unexpanded node, using a FIFO queue as the frontier. It explores the state space in layers of increasing depth. BFS is complete (it finds a solution if one exists) and, when every step costs the same, optimal (it finds the shallowest, fewest-step goal). Its cost is steep: with branching factor b and solution depth d, both time and space are O(b^d). The memory demand is usually the binding constraint.

function BFS(problem):
    node = problem.initial
    if problem.goal_test(node): return node
    frontier = FIFO_queue([node])
    explored = empty set
    while frontier not empty:
        node = frontier.pop_front()
        explored.add(node.state)
        for action in problem.actions(node.state):
            child = child_node(node, action)
            if child.state not in explored and child not in frontier:
                if problem.goal_test(child.state): return child   # shallowest goal
                frontier.push_back(child)
    return failure

Depth-first search (DFS)

DFS expands the deepest unexpanded node, using a LIFO stack (or recursion). Its great advantage is memory: it stores only the current path plus siblings, so space is O(b*m) where m is the maximum depth. But DFS is not optimal, and it is not complete in infinite or looping spaces, where it can descend forever. Time is O(b^m) in the worst case.

Uniform-cost search

When step costs differ, shallowest is not cheapest. Uniform-cost search expands the node with the lowest path cost g(n) using a priority queue, and it is optimal for any non-negative costs. It is the cost-sensitive generalization of BFS and the direct ancestor of the informed algorithms in the next lesson.

Strategy	Complete?	Optimal?	Time	Space
BFS	Yes	Yes (equal costs)	O(b^d)	O(b^d)
DFS	No (infinite spaces)	No	O(b^m)	O(b*m)
Uniform-cost	Yes	Yes	large	large

Uninformed search treats all unexplored directions alike. Informed (heuristic) search uses extra, problem-specific knowledge to head toward the goal, often exploring far fewer nodes. That knowledge is packaged in a heuristic function h(n), an estimate of the cost of the cheapest path from node n to a goal. A good heuristic can turn an intractable search into an easy one.

Greedy best-first search

Greedy best-first search expands the node that looks closest to the goal, that is, the node with the smallest h(n). It is often fast but shortsighted: because it ignores the cost already spent, it can charge toward a nearby-looking goal along an expensive route, and it is neither optimal nor complete in general.

A* search

A* (pronounced "A star") fixes the greedy flaw by adding back the path cost. It expands the node with the smallest value of

f(n) = g(n) + h(n)
    g(n) = actual cost from the start to n
    h(n) = estimated cost from n to the goal

So f(n) estimates the total cost of the cheapest solution that passes through n. A* balances what a path has cost so far against what it is likely to cost to finish. Implemented with a priority queue keyed by f, it is the workhorse of pathfinding.

function A_STAR(problem, h):
    start = problem.initial
    frontier = priority_queue keyed by f = g + h
    frontier.push(start, g=0)
    best_g = { start: 0 }
    while frontier not empty:
        node = frontier.pop_min()          # smallest f = g + h
        if problem.goal_test(node.state): return reconstruct(node)
        for action in problem.actions(node.state):
            child = child_node(node, action)
            new_g = best_g[node.state] + step_cost(node, action)
            if child.state not in best_g or new_g < best_g[child.state]:
                best_g[child.state] = new_g
                frontier.push(child, f = new_g + h(child.state))
    return failure

When is A* optimal?

A* is optimal exactly when its heuristic is well behaved:

• A heuristic is admissible if it never overestimates the true remaining cost, that is, 0 <= h(n) <= h*(n). With an admissible heuristic, tree-search A* returns an optimal solution.
• A heuristic is consistent (monotone) if for every edge, h(n) <= cost(n, n') + h(n'). Consistency implies admissibility and guarantees optimality for graph-search A* as well.

The straight-line distance to a destination is an admissible (and consistent) heuristic for road navigation, because a real road can only be as short as, or longer than, the straight line. If h(n) = 0 everywhere, A* degenerates into uniform-cost search: still optimal, but uninformed. The art of applied search is designing a heuristic that is admissible yet as close to the true cost as possible, so A* explores few nodes while still guaranteeing the best answer.

Search so far assumed a single agent in a cooperative world. In a game there is an opponent whose interests oppose yours. For a two-player, zero-sum, perfect-information game such as chess, tic-tac-toe, or checkers, adversarial search finds optimal play against an optimal opponent.

The game tree and minimax value

We model the game as a tree of states. The two players are called MAX (who tries to maximize the final score) and MIN (who tries to minimize it). Leaves are terminal states with a utility (say +1 win, 0 draw, -1 loss for MAX). The minimax value of a node is the utility MAX can guarantee assuming MIN always replies optimally: at MAX nodes take the maximum over children, at MIN nodes take the minimum.

function MINIMAX(state):
    if TERMINAL(state): return UTILITY(state)
    if PLAYER(state) == MAX:
        value = -infinity
        for a in ACTIONS(state):
            value = max(value, MINIMAX(RESULT(state, a)))
        return value
    else:                              # MIN to move
        value = +infinity
        for a in ACTIONS(state):
            value = min(value, MINIMAX(RESULT(state, a)))
        return value

Minimax returns the optimal value; MAX then plays the action leading to the child with that value. It performs a complete depth-first exploration of the game tree, so for a tree of branching factor b and depth m its time cost is O(b^m), which is infeasible for large games like chess (roughly 35 legal moves per position).

Alpha-beta pruning

Alpha-beta pruning computes the exact same minimax decision while skipping branches that cannot affect it. It carries two bounds down the tree: alpha, the best value MAX can already guarantee, and beta, the best value MIN can already guarantee. When a node's value provably falls outside the alpha-beta window, its remaining children are pruned because no rational player would ever let the game reach them.

function ALPHA_BETA(state, alpha, beta):
    if TERMINAL(state): return UTILITY(state)
    if PLAYER(state) == MAX:
        value = -infinity
        for a in ACTIONS(state):
            value = max(value, ALPHA_BETA(RESULT(state, a), alpha, beta))
            alpha = max(alpha, value)
            if alpha >= beta: break        # beta cutoff: prune the rest
        return value
    else:
        value = +infinity
        for a in ACTIONS(state):
            value = min(value, ALPHA_BETA(RESULT(state, a), alpha, beta))
            beta = min(beta, value)
            if beta <= alpha: break        # alpha cutoff: prune the rest
        return value

Pruning does not change the answer; it only avoids wasted work. With a good move ordering (trying likely-best moves first), alpha-beta can roughly halve the effective exponent, examining about O(b^(m/2)) nodes, which effectively doubles the search depth reachable in the same time. Real engines add a cutoff at a depth limit and replace true utility with a heuristic evaluation function at that frontier, since full games are too deep to search to the end.

A constraint satisfaction problem (CSP) is a special, structured kind of search. Instead of an opaque goal test, a CSP is defined by three things: a set of variables, a domain of possible values for each variable, and a set of constraints that specify allowable combinations. A solution is an assignment of a value to every variable that violates no constraint. Map coloring, scheduling, Sudoku, and cryptarithmetic all fit this mold.

Example: map coloring

Color a map so that no two adjacent regions share a color, using three colors. The variables are the regions, each domain is {red, green, blue}, and each constraint says two neighboring regions must differ. This is a binary CSP because every constraint involves two variables, and it maps directly onto a constraint graph whose edges are the constraints.

Backtracking search

Because assignments are commutative (order does not matter), we assign one variable at a time. Backtracking search is depth-first search specialized for CSPs: pick an unassigned variable, try a value consistent with the constraints so far, recurse, and if we get stuck, undo (backtrack) and try another value.

function BACKTRACK(assignment, csp):
    if assignment is complete: return assignment
    var = SELECT-UNASSIGNED-VARIABLE(csp, assignment)
    for value in ORDER-DOMAIN-VALUES(var, assignment, csp):
        if value is consistent with assignment:
            add {var = value} to assignment
            inferences = INFERENCE(csp, var, value)     # e.g. forward checking
            if inferences != failure:
                result = BACKTRACK(assignment, csp)
                if result != failure: return result
            remove {var = value} and inferences from assignment
    return failure

Making backtracking smart

Naive backtracking can be slow, but a few classic heuristics prune enormously:

• Minimum remaining values (MRV): choose the variable with the fewest legal values left, the "most constrained" one, to hit dead ends early.
• Degree heuristic: break ties by choosing the variable involved in the most constraints on other unassigned variables.
• Least constraining value: when ordering values to try, prefer the one that rules out the fewest options for neighbors.
• Forward checking: after each assignment, remove now-illegal values from neighbors' domains; if any domain empties, backtrack immediately.
• Arc consistency (AC-3): a stronger propagation that repeatedly enforces, for every constraint between two variables, that each value has some consistent partner, pruning further before and during search.

Together these turn a blind search into an efficient one. The lesson generalizes: exposing a problem's structure as variables and constraints lets specialized reasoning outperform brute force.

A knowledge-based agent keeps a knowledge base (KB), a set of sentences in a formal representation language, and reasons by inference: deriving new sentences that follow from what it already knows. This separates knowledge (declarative facts and rules) from the inference procedure that uses them, so the same reasoner can work over many domains.

Propositional logic

Propositional logic is the simplest useful formal language. Its atoms are propositions, statements that are true or false, such as P = "it is raining." Atoms combine with connectives:

Connective	Meaning	Read as
NOT P	negation	not P
P AND Q	conjunction	P and Q
P OR Q	disjunction	P or Q (inclusive)
P implies Q	implication	if P then Q
P iff Q	biconditional	P if and only if Q

The meaning of a sentence is fixed by a truth table: given a truth value for each atom (a model), the connectives determine the sentence's value. The one point that trips people up: P implies Q is false only when P is true and Q is false; it is true in every other case, including whenever P is false.

Three central concepts

• Satisfiable: a sentence is satisfiable if some model makes it true. Determining this for propositional logic is the famous SAT problem, the first problem proven NP-complete.
• Valid: a sentence is valid (a tautology) if every model makes it true, for example P OR NOT P.
• Entailment: a KB entails a sentence alpha, written KB entails alpha, if alpha is true in every model where the KB is true. Entailment is what "follows logically" means, and it is the target of correct inference.

Sound and complete inference

An inference procedure is sound if it derives only entailed sentences (it never lies) and complete if it can derive every entailed sentence (it misses nothing). One simple sound and complete method is model checking: enumerate all models and confirm alpha holds wherever the KB does. It is correct but costs O(2^n) for n atoms. Faster methods apply inference rules such as Modus Ponens (from P and "P implies Q," infer Q) and resolution, which underlie automated theorem provers and are the logical backbone of goal-based reasoning agents.

Propositional logic is coarse: it can only assert whole facts, so "Socrates is a man" and "Plato is a man" are unrelated atoms with no shared structure. First-order logic (FOL), also called predicate logic, is far more expressive because it can talk about objects, their properties, and the relations among them, and it can generalize over them.

The building blocks

• Constants name specific objects: Socrates, 3, TajMahal.
• Predicates name properties or relations that are true or false of objects: Man(Socrates), GreaterThan(5, 3), Brother(Cain, Abel).
• Functions map objects to objects: FatherOf(Isaac) refers to Abraham.
• Variables such as x and y stand for objects not yet specified.

Quantifiers

The real power comes from two quantifiers that let one sentence speak about many objects:

• The universal quantifier "for all x" asserts something of every object. "All men are mortal" becomes: for all x, Man(x) implies Mortal(x).
• The existential quantifier "there exists x" asserts something of at least one object. "Some student passed" becomes: there exists x such that Student(x) and Passed(x).

A subtle but critical pairing: universal quantifiers naturally combine with implication, while existential quantifiers naturally combine with conjunction. Writing "there exists x such that Man(x) implies Mortal(x)" is a common error; it is trivially satisfied by any non-man and does not say what you meant.

The classic syllogism

FOL captures Aristotle's syllogism cleanly. From the KB

1.  for all x:  Man(x) implies Mortal(x)     # all men are mortal
2.  Man(Socrates)                            # Socrates is a man

we infer Mortal(Socrates) by Universal Instantiation (substitute Socrates for x in sentence 1 to get Man(Socrates) implies Mortal(Socrates)) followed by Modus Ponens with sentence 2. Inference in full FOL uses unification (finding substitutions that make expressions match) and generalized resolution. FOL is semi-decidable: if a sentence is entailed, a complete procedure will eventually confirm it, but if it is not, the procedure may run forever. This greater expressiveness, at the cost of harder inference, is the standard trade-off in knowledge representation, and FOL is the foundation for ontologies, the semantic web, and much of automated reasoning.

Logic assumes the agent knows facts for certain, but real agents face noisy sensors, unpredictable environments, and incomplete knowledge. Trying to enumerate every exception in logic ("a bird flies unless it is a penguin, or injured, or a chick, or ...") becomes hopeless. Probability theory gives a principled calculus for degrees of belief, letting an agent reason and act sensibly when it cannot be sure.

The basic axioms

A probability P(A) is a number between 0 and 1 measuring how likely event A is. The axioms (due to Kolmogorov) are few:

• 0 <= P(A) <= 1 for any event A.
• P(true) = 1 and P(false) = 0.
• For mutually exclusive events, P(A or B) = P(A) + P(B). In general, P(A or B) = P(A) + P(B) - P(A and B).

From these, P(not A) = 1 - P(A).

Joint, marginal, and conditional

The joint probability P(A and B) is the chance both happen. The marginal P(A) is the chance of A regardless of B, recovered by summing the joint over B's values. The conditional probability P(A given B), the probability of A once we know B, is defined by

P(A given B) = P(A and B) / P(B),   provided P(B) > 0

Rearranged, this gives the product rule: P(A and B) = P(A given B) * P(B).

Independence

Two events are independent when knowing one tells you nothing about the other: P(A given B) = P(A), equivalently P(A and B) = P(A) * P(B). Independence is precious because it lets us factor a huge joint distribution into small pieces, which is the whole idea behind Bayesian networks and the naive Bayes classifier.

Worked example

Roll a fair six-sided die. Let A = "the result is even" and B = "the result is greater than 3." Then P(A) = 3/6 = 1/2 (the evens 2, 4, 6). The outcomes greater than 3 are 4, 5, 6, so P(B) = 3/6 = 1/2. The outcomes that are both even and greater than 3 are 4 and 6, so P(A and B) = 2/6 = 1/3. Therefore P(A given B) = P(A and B) / P(B) = (1/3) / (1/2) = 2/3. Learning that the die exceeds 3 raised the probability of "even" from 1/2 to 2/3, because among {4, 5, 6} two of the three are even. These events are not independent, since P(A given B) does not equal P(A).

Bayes' rule is the engine of rational belief update. It tells an agent exactly how to revise its beliefs when new evidence arrives, and it underlies spam filters, medical diagnosis, and much of machine learning. From the product rule written two ways, P(A and B) = P(A given B)P(B) = P(B given A)P(A), we solve for one conditional in terms of the other:

P(H given E) = P(E given H) * P(H) / P(E)

    P(H)        prior:       belief in hypothesis H before evidence
    P(E given H) likelihood: how well H predicts the evidence E
    P(H given E) posterior:  updated belief in H after seeing E
    P(E)        evidence:    total probability of E (a normalizer)

In words: the posterior is the likelihood times the prior, divided by the total probability of the evidence. When there are two hypotheses (H and not H), the denominator expands by the law of total probability: P(E) = P(E given H)P(H) + P(E given not H)P(not H).

The classic medical test

A disease affects 1 percent of a population. A test is 90 percent sensitive (it is positive in 90 percent of sick people) and has a 9 percent false positive rate (it is wrongly positive in 9 percent of healthy people). You test positive. What is the probability you actually have the disease?

Let D = disease, Pos = positive test. We want P(D given Pos).

Prior:        P(D) = 0.01,   P(not D) = 0.99
Likelihoods:  P(Pos given D) = 0.90,   P(Pos given not D) = 0.09
Evidence:     P(Pos) = 0.90*0.01 + 0.09*0.99
                     = 0.009 + 0.0891 = 0.0981
Posterior:    P(D given Pos) = (0.90 * 0.01) / 0.0981
                            = 0.009 / 0.0981  approximately  0.092

The answer is only about 9 percent, not 90 percent. Most people, and many professionals, guess far too high. The reason is the base-rate fallacy: because the disease is rare, the many healthy people generating false positives (about 89 per 10,000) swamp the few true positives (about 9 per 1,000). Bayes' rule forces you to weigh the evidence against the prior, which intuition tends to ignore.

Bayesian networks

Reasoning over many variables at once is intractable if you store the full joint distribution. A Bayesian network is a directed acyclic graph whose nodes are random variables and whose edges encode direct dependencies; each node carries a conditional probability table given its parents. By exploiting conditional independence, the network factors the full joint into a compact product, making otherwise impossible probabilistic inference practical. Bayesian networks are the probabilistic counterpart to the logical knowledge bases of the previous module.

Machine learning (ML) builds programs that improve at a task by learning patterns from data rather than following hand-written rules. Instead of a programmer specifying how to recognize a cat, we show the system many labeled images and let it infer a rule. Formally, a learner searches a space of possible models for one that fits the data and, crucially, generalizes to new, unseen examples.

Three paradigms

• Supervised learning: the data comes as input-output pairs (x, y), and the goal is to learn a function from x to y. If y is a category, the task is classification (spam or not, digit 0 to 9); if y is a number, it is regression (predict a house price).
• Unsupervised learning: the data has inputs x but no labels, and the goal is to find structure, such as clustering customers into groups or dimensionality reduction to compress features.
• Reinforcement learning: an agent learns by acting in an environment and receiving rewards or penalties, discovering a policy that maximizes long-run reward. This is how systems learn to play games and control robots.

Generalization, overfitting, and underfitting

The central challenge is generalization. A model that is too simple underfits: it misses real patterns and does poorly even on training data (high bias). A model that is too complex overfits: it memorizes noise and quirks of the training set and fails on new data (high variance). The bias-variance trade-off is the tension between these; good learning finds the sweet spot with low total error.

Underfit  (high bias):     poor on training AND test data
Good fit:                   good on training AND test data
Overfit   (high variance):  great on training, poor on test data

Splitting the data honestly

To measure generalization we never test on data we trained on. We split the data into three parts: the training set fits the model, the validation set tunes choices such as model complexity (hyperparameters), and the test set, touched only once at the very end, gives an honest estimate of real-world performance. Techniques such as regularization (penalizing complexity) and cross-validation (rotating which slice is held out) further guard against overfitting. This discipline, learning a rule that works on data you have never seen, is what separates machine learning from mere curve-fitting or memorization.

Artificial neural networks are the models behind modern AI's biggest successes. Loosely inspired by the brain, a network is built from simple units called neurons arranged in layers. Each neuron computes a weighted sum of its inputs, adds a bias, and passes the result through a nonlinear activation function:

output = activation( w1*x1 + w2*x2 + ... + wn*xn + b )

Common activations include the sigmoid (squashes to between 0 and 1) and, most used today, ReLU (rectified linear unit), which outputs the input if positive and 0 otherwise. The nonlinearity is essential: stacking purely linear layers would collapse to a single linear function, so without it depth would buy nothing.

Layers and forward propagation

Neurons are organized into an input layer, one or more hidden layers, and an output layer. Running data through the network from input to output, layer by layer, is forward propagation, and it produces the network's prediction.

Training: loss, gradient descent, and backpropagation

Learning means adjusting the weights so predictions improve. A loss function measures how wrong the current predictions are (for example, mean squared error for regression, cross-entropy for classification). Training minimizes this loss by gradient descent: compute the gradient (the direction of steepest increase in loss) and take a small step in the opposite direction, scaled by the learning rate.

repeat until converged:
    prediction = forward_propagate(inputs)          # forward pass
    loss       = loss_function(prediction, target)  # how wrong we are
    gradients  = backpropagate(loss)                # dLoss/dWeight per weight
    for each weight w:
        w = w - learning_rate * gradient_of_w       # step downhill

The key algorithm that makes this efficient is backpropagation: it uses the chain rule of calculus to compute, in one backward sweep, how much each weight contributed to the loss. Without backpropagation, training networks with millions of weights would be hopeless.

What makes it "deep," and the major architectures

Deep learning simply means neural networks with many hidden layers. Depth lets a network learn a hierarchy of features: early layers detect simple patterns (edges), later layers combine them into complex ones (shapes, then objects), all learned automatically rather than hand-engineered. A few families dominate:

• Convolutional neural networks (CNNs) exploit spatial structure and excel at images.
• Recurrent neural networks (RNNs) process sequences by maintaining a hidden state over time.
• Transformers, introduced in 2017, use an attention mechanism to weigh relationships across an entire sequence in parallel. They now power large language models and much of state-of-the-art AI.

Deep learning's power comes at the cost of needing large data, heavy computation, and care against overfitting, but its ability to learn representations directly from raw data is what unlocked the current era.

Natural language processing (NLP) is the subfield of AI concerned with getting computers to understand, interpret, and generate human language. It is hard because language is ambiguous ("I saw the man with the telescope" has two readings), deeply context-dependent, full of idiom, and endlessly creative. NLP powers translation, search, question answering, sentiment analysis, and conversational assistants.

Core tasks and the classic pipeline

Traditional NLP broke language into stages: tokenization (splitting text into words or subword tokens), part-of-speech tagging, parsing (recovering grammatical structure), named entity recognition (finding people, places, organizations), and resolving meaning. Early systems used hand-written grammar rules; later ones used statistics over large text corpora.

Representing meaning as vectors

A pivotal idea is the word embedding: represent each word as a dense vector of numbers such that words used in similar contexts have similar vectors. This follows the distributional hypothesis, that a word's meaning is characterized by the company it keeps. Embeddings famously capture analogies through vector arithmetic; the vector for "king" minus "man" plus "woman" lands near "queen." Turning words into vectors lets neural networks compute over meaning rather than raw symbols.

Large language models

Modern NLP is dominated by large language models (LLMs) built on the Transformer architecture. They are trained by self-supervised learning on vast text: the model repeatedly predicts a masked or next token and adjusts its weights when wrong. Doing this across billions of sentences forces the model to absorb grammar, facts, and patterns of reasoning as a side effect of getting good at prediction. The key mechanism, attention, lets the model weigh which earlier words are relevant to each word it processes, capturing long-range context that older models missed.

training objective (simplified):
    for each position in a huge text corpus:
        predict the next token given all previous tokens
        increase the probability the model assigns to the correct token

Strengths and honest limits

LLMs are remarkably fluent and general, but they have real limitations you must design around:

• They can hallucinate: produce fluent statements that are false, because they optimize for plausible continuations, not truth.
• They lack grounded understanding and a reliable model of the world, and can fail at multi-step logical or arithmetic reasoning.
• They inherit biases present in their training data.
• Their knowledge is fixed at training time unless connected to external tools or retrieval.

Understanding both the power and these failure modes is essential to using language models responsibly, which leads directly to the final lesson on ethics and safety.

As AI systems make or influence consequential decisions, in hiring, lending, medicine, policing, and beyond, their design becomes an ethical matter, not just a technical one. This lesson surveys the principal concerns and the emerging responses. The goal is sober judgment: neither hype nor panic, but engineering responsibility.

Fairness and bias

Machine learning models learn from historical data, and if that data reflects past discrimination, the model can perpetuate or amplify it, giving a veneer of objectivity to unfair outcomes. A hiring model trained on a company's past hires may disadvantage groups underrepresented in that history. Mitigations include auditing training data, measuring outcomes across groups with explicit fairness metrics, and testing for disparate impact. A hard truth from this literature is that several reasonable mathematical definitions of fairness can be mutually incompatible, so fairness requires value choices, not just formulas.

Privacy, transparency, and accountability

• Privacy: models trained on personal data can leak it or enable surveillance; techniques such as anonymization and differential privacy help, imperfectly.
• Transparency and explainability: many powerful models are black boxes whose reasoning is opaque. Explainable AI seeks methods to justify individual decisions, which matters when someone is denied a loan or parole.
• Accountability: when an autonomous system causes harm, who is responsible, the developer, the deployer, the user? Clear lines of responsibility and human oversight are essential.

The alignment problem

A distinct, forward-looking concern is alignment: ensuring an AI system actually pursues what its designers intend. The difficulty is value specification. We cannot fully enumerate our goals, so a system optimizing a proxy can pursue it in unintended, harmful ways, a failure called specification gaming or reward hacking (an agent told to maximize a game score may exploit a bug rather than play well). As systems grow more capable and autonomous, ensuring they remain controllable and aligned with human values becomes increasingly important.

Toward responsible AI

Practical responses are converging across research and policy:

1. Keep a human in the loop for high-stakes decisions rather than fully automating them.
2. Test rigorously for bias, robustness, and failure modes before deployment, and monitor after.
3. Build in transparency, documentation, and clear accountability from the start.
4. Invest in technical AI safety research on alignment, interpretability, and robustness.
5. Support thoughtful governance and regulation, such as risk-based frameworks that match oversight to the stakes of an application.

The recurring lesson of this course is that intelligence is the ability to act well toward goals. For artificial agents, deciding whose goals, and ensuring the agent pursues them safely and fairly, is the defining challenge of the field's future.