Modern Deep Learning Foundations
Learn by predicting, running mathematically grounded experiments, explaining evidence, and exporting an inspectable mastery artifact.
- Laboratories
- 6
- Guided time
- 4h 20m
- Mastery artifacts
- 6
Generalization is measured on data the model did not fit
Increasing polynomial degree gives the model more capacity. Too little capacity produces bias; too much capacity allows the model to follow sampling noise and increases variance.
L2 regularization penalizes large coefficients. The objective is not the smallest training error, but a stable model with low error on unseen observations.
Lesson 6 - Overfitting and Regularization
Printed pages 41-46; PDF pages 43-48.
- Loss Behavior as a Diagnostic Tool printed pp. 42-43 / PDF pp. 44-45 training-versus-validation paragraphs
- Why Overfitting Happens printed pp. 43-44 / PDF pp. 45-46 capacity and sampling-noise discussion
- Weight Decay (L2 Regularization) printed pp. 45-46 / PDF pp. 47-48 regularized objective; Equation 6.1
Evidence and provenance
Bias squared and variance are estimated from repeated fits on a fixed evaluation grid.
The current blue curve and train/held-out MSE use the selected degree and L2 value.
Noise variance is fixed by the known synthetic data-generating process.
In real projects the noise-free function is unknown and model selection relies on validation evidence.
Run, inspect, and compare
Follow the three guided moves above. Change one variable at a time so every visual change has a clear cause.
Compare runs instead of trusting one result
Explain what happened, then transfer it
Use the displayed Bias squared, Variance, noise, train MSE, and held-out MSE to explain the observed fit.
Use at least one value or visible change from the experiment.Describe how you would choose model capacity and regularization when the true data-generating function is unknown.
Name the new context and the design choice you would make.- Defines a validation or resampling procedure
- Separates bias, variance, and noise
- Chooses capacity and L2 using unseen data
- Explains how stability will be monitored
By the end of this lesson, you will be able to:
- Distinguish underfitting, appropriate model capacity, and overfitting from training and test behaviour.
- Explain the bias-variance trade-off and how polynomial degree changes model flexibility.
- Use L2 regularization to control model complexity and reduce the generalization gap.
In the early stages of training a neural network, everything may appear to be going well: the training loss decreases steadily, and the training accuracy improves epoch by epoch. However, when new, unseen data is introduced, the model's performance can suddenly collapse. This phenomenon is not a bug - it is a fundamental challenge in deep learning, known as overfitting. Overfitting occurs when the model fits the training data too closely, capturing not only the underlying patterns but also noise, anomalies, and incidental correlations. As a result, the model performs well on the data it has seen, but fails to generalize to new examples. In contrast, underfitting describes a scenario where the model fails to capture even the patterns in the training data, often due to insufficient capacity or inadequate training. Both overfitting and underfitting degrade the model's generalization ability - the fundamental goal.
Leave with inspectable evidence, not a completion click
Your artifact combines the prerequisite check, prediction, experiment configuration, displayed evidence, explanation, transfer rubric, and confidence change.
Retrieve the concept after time has passed
Complete the artifact to schedule a 24-hour retrieval prompt and a seven-day transfer revisit.
Help evaluate whether the laboratory teaches the concept
No name, email address, free text, or IP address is retained in the learning record.