Algorithms

Step 3 of the machine learning workflow

NoteAlgorithms · at a glance
Workflow step Step 3 of 5 · Algorithms
What you’ll learn How to choose a model family for a task, and the core idea behind each one.
Algorithms covered Clustering · Trees · Linear · Other models · Neural nets

Algorithms are the procedures that turn prepared data into a model. You can think of these like the engines of machine learning. Where earlier steps got your data ready, this step is where a model is actually chosen and its shape decided. This chapter surveys the main families of algorithms used in this course.

There is no single best algorithm. Each family makes different assumptions about what patterns in data look like, and each trades off differently between accuracy, interpretability, speed, and how much data it needs. Much of the craft of machine learning is matching an algorithm’s assumptions to the problem in front of you.

This chapter is organized as a reference: a short guide to choosing among the families, followed by a self-contained section for each algorithm. You can read it straight through, or jump to whichever algorithm you need from the list above.

Overview: choosing an algorithm

Before reaching for any specific model, it helps to narrow the field by asking a few questions about your problem and your data.

  • What kind of task is it? Classification, regression, or clustering?
  • How much do you value interpretability? Some stakeholders will only be on board with your recommendations if there is full transparency about the decisions being made.
  • How much data do you have? Some algorithms require immense amounts of data, while others work best on limited data.
  • How much compute and time? Sometimes a simple model is best because of limited resources and turn-around.
NoteKey idea: match the algorithm to the problem rather than how “flashy” it is

The most powerful model is not always the right one. A simple, interpretable model that is good enough often beats a complex one that no one can explain or maintain.

Clustering CLS · at a glance

Clustering algorithms are a form of unsupervised learning which focuses on finding patterns or groups among unlabeled data. We can use these findings either to support claims about groups, or go a step further and use it to label our data into groups. Since we do not have a target variable for “truth” we must rely on domain-level expertise and our evaluation step to quantify how “good” our model is performing.

Workflow step Step 3 of 5 · Algorithm Selection
What you’ll learn
Standards assessed ALG-CLS · EVL-CLS
Key terms
Materials readings · slides · homework · lab
Self-quiz standards & practice →

Readings

ISLP selected readings

Dive deeper in the content we’ll be covering. These readings can be quite math-heavy. If anything is unclear, don’t hesitate to reach out to your instructor. Aim to grasp the big-picture ideas so you can explain each topic to a non-technical audience.

Slides

  • CLS1: Overview of some unsupervised learning algorithms, with a deep dive on the k-means algorithm
    Open in new tab

  • CLS2: Evaluation of clustering algorithms and computing the sillhouette score.
    Open in new tab

  • CLS3: Walkthrough of how to do clustering end-to-end. Discussion on improvements.
    Open in new tab

Homework

This homework goes over calculating centroid assignments using the euclidean distance, and evaluating clusters using silhouette score. Click to view pdf.

Lab

In this lab, we will be working with image data and exploring how decisions made in the modeling workflow impact results for our clustering models. There are a selection of open-ended questions (give your best guess on these) along with a few questions that require you to implement your own clustering functions. Click to view Colab Notebook.

Self-quiz

Use the standards below as a checklist as a study guide for the upcoming quiz.

ALG-CLS

You should be able to:

  • Define the unsupervised learning approach and identify it’s associated algorithms
  • Explain these algorithms and their associated terms to a non-technical audience
  • Identify when to use various unsupervised algorithms
  • Define intra vs inter cluster distances and it’s relevance to the k-means clustering algorithm

EVL-CLS

You should be able to:

  • Interpret clustering results and identify which clusters are related to ground truths
  • Calculate and interpret silhouette coefficients
  • Identify good choices for cluster size \(k\)

Formulas given

\(\text{Silhouette coefficient} = \frac{b-a}{max(a,b)}\)
\(\text{Euclidean distance} = \sqrt{(x_2 - x_1)^2+ (y_2 - y_1)^2}\)

Trees (TREE)

Linear models (LIN)

Other models (MOD)

Neural networks (NN)