Algorithms

Step 3 of the machine learning workflow

NoteAlgorithms · at a glance
Workflow step Step 3 of 5 · Algorithms
What you’ll learn How to choose a model family for a task, and the core idea behind each one.
Algorithms covered Classification · Trees · Linear · Other models · Neural nets

Algorithms are the procedures that turn prepared data into a model — the engines of machine learning. Where earlier steps got your data ready, this step is where a model is actually chosen and its shape decided. This chapter surveys the main families of algorithms used in this course.

There is no single best algorithm. Each family makes different assumptions about what patterns in data look like, and each trades off differently between accuracy, interpretability, speed, and how much data it needs. Much of the craft of machine learning is matching an algorithm’s assumptions to the problem in front of you.

This chapter is organized as a reference: a short guide to choosing among the families, followed by a self-contained section for each algorithm. You can read it straight through, or jump to whichever algorithm you need from the list above.

Overview: choosing an algorithm

Before reaching for any specific model, it helps to narrow the field by asking a few questions about your problem and your data.

  • What kind of task is it? Classification, regression, or clustering? This alone rules out whole families.
  • How much do you value interpretability? A tree you can read end-to-end is very different from a neural network whose reasoning is opaque.
  • How much data do you have? Some families (neural nets) are data-hungry; others work well on small datasets.
  • How much compute and time? Simpler models train in seconds; others take much longer.
NoteKey idea: match the algorithm to the problem, not the hype

The most powerful model is not always the right one. A simple, interpretable model that is “good enough” often beats a complex one that no one can explain or maintain.

How this chapter is organized

Each algorithm below is a self-contained section with the same structure, so you always know where to look:

  • Readings — do before class to build intuition.
  • Slides — what we work through in class.
  • Homework — apply the algorithm yourself.
  • Lab — hands-on practice.
  • Standards — what you should be able to do (assessed on the quiz).

Classification (CLS)

NoteClassification · at a glance
Type Supervised · predicts discrete categories
Key idea Learn a decision boundary that separates classes.
Standards CLS
Key terms classifier · decision boundary · class · label
Materials readings · slides · homework · lab

(Overview of classification goes here — what it is, when to use it, the core idea.)

Readings

(Add readings for CLS.)

Slides

(Add slide deck links for CLS.)

Homework

(Add homework for CLS.)

Lab

(Add lab for CLS.)

Standards

(Add the CLS standards — “you should be able to…”.)

Trees (TREE)

NoteTrees · at a glance
Type Supervised · classification or regression
Key idea Split data with a sequence of yes/no questions.
Standards TREE
Key terms node · split · leaf · depth · overfitting
Materials readings · slides · homework · lab

(Overview of tree-based models goes here.)

Readings

(Add readings for TREE.)

Slides

(Add slide deck links for TREE.)

Homework

(Add homework for TREE.)

Lab

(Add lab for TREE.)

Standards

(Add the TREE standards.)

Linear models (LIN)

NoteLinear models · at a glance
Type Supervised · regression or classification
Key idea Fit a straight-line (linear) relationship between features and target.
Standards LIN
Key terms coefficient · weight · intercept · regularization
Materials readings · slides · homework · lab

(Overview of linear models goes here.)

Readings

(Add readings for LIN.)

Slides

(Add slide deck links for LIN.)

Homework

(Add homework for LIN.)

Lab

(Add lab for LIN.)

Standards

(Add the LIN standards.)

Other models (MOD)

NoteOther models · at a glance
Type Supervised · a grouping of additional approaches
Key idea Several distinct methods — kNN, SVM, naive Bayes — each with its own logic.
Standards MOD
Key terms k-nearest neighbors · support vector · margin · prior probability
Materials readings · slides · homework · lab

(Overview of the additional model types — kNN, SVM, naive Bayes, etc.)

Readings

(Add readings for MOD.)

Slides

(Add slide deck links for MOD.)

Homework

(Add homework for MOD.)

Lab

(Add lab for MOD.)

Standards

(Add the MOD standards.)

Neural networks (NN)

NoteNeural networks · at a glance
Type Supervised · flexible, high-capacity models
Key idea Stacked layers of simple units learn complex patterns from lots of data.
Standards NN
Key terms neuron · layer · weight · activation · training
Materials readings · slides · homework · lab

(Overview of neural networks goes here.)

Readings

(Add readings for NN.)

Slides

(Add slide deck links for NN.)

Homework

(Add homework for NN.)

Lab

(Add lab for NN.)

Standards

(Add the NN standards.)