Modern Deep Learning Foundations

Labs: 6

Lab 6 of 6 Transfer Learning and Fine-Tuning Advanced / 45 min / Deliverable: Fine-tuning strategy brief

Screen 1 of 3 - Read before you experiment

Fine-tuning is a choice about which parameters are allowed to change

Frozen stages preserve pretrained representations and require no gradient update. Unfreezing later stages increases adaptation capacity, memory use, and the risk of overwriting useful source knowledge.

The best strategy depends on target-data volume and domain similarity. Treat the displayed accuracy and forgetting values as comparative estimates, not measured training results.

Exact book map

Lesson 19 - Transfer Learning and Fine-Tuning

Printed pages 120-125; PDF pages 122-127.

Open the exact pages

Three Fine-Tuning Strategies printed pp. 121-123 / PDF pp. 123-125 feature extraction, partial, and full fine-tuning paragraphs
Mathematical Considerations printed pp. 123-124 / PDF pp. 125-126 parameter-update objective; Equation 19.1
When Not to Use / Best Practices printed pp. 124-125 / PDF pp. 126-127 domain-shift and validation guidance

Evidence levelArchitecture-accurate parameter counts + clearly labelled heuristic outcomes

What is real, computed, or illustrative

Evidence and provenance

Measured Architecture ResNet-50, EfficientNet-B0, and MobileNetV3 parameter structures

Stage depth and parameter counts follow the selected pretrained backbone.

Derived Trainability Exact frozen and trainable parameter totals

Moving the freeze boundary changes which stages receive gradients.

Illustrative Outcome estimates Expected accuracy, cost, and forgetting-risk heuristics

These values compare strategies; they are not results from a live training run.

Scope Claim boundary A strategy must be validated on target data

The simulator supports planning, while final selection requires measured validation and training traces.

60-second prerequisite check

Choose the support level from evidence

When only the classifier head is trainable, what happens inside the frozen backbone? It performs a forward pass but its weights are not updated It is removed from the model Its gradients update with a larger learning rate

Frozen layers still compute features in the forward pass. They are excluded from optimizer updates, while trainable stages receive gradients.

Guided task

Complete these three moves

1
Select
Choose a backbone and define the target-data volume and source-domain similarity.
2
Configure
Move the freeze boundary and learning rate while watching trainable parameters and updated stages.
3
Compare
Build two strategies and justify the one that best balances adaptation, cost, and forgetting risk.

Record your prediction before you run the lab

Predict which stages you would unfreeze for the selected data volume and domain similarity, and state the expected trade-off.

Confidence before experimenting 50% A useful prediction names the expected change and why it should happen.

Start the fine-tuning strategy lab

Screen 2 of 3 - Experiment

Run, inspect, and compare

Follow the three guided moves above. Change one variable at a time so every visual change has a clear cause.

Guided mode Predict first, advance one stage at a time, and explain the displayed values before changing another control.

Reproducible experiment record

Compare runs instead of trusting one result

Run the laboratory, then capture the controls, metrics, evidence status, seed, and timestamp.

The accessible visual summary will update when the laboratory renders its first result.

Evidence checkpoint

Explain what happened, then transfer it

0 / 3responses complete

Explain the evidence

Use trainable parameters, freeze boundary, learning rate, adaptation estimate, and forgetting risk to justify the strategy.

Use at least one value or visible change from the experiment.

Transfer the idea

Propose an empirical validation plan that could confirm or reject the simulator recommendation.

Name the new context and the design choice you would make.

Defines source and target domains
Uses data volume and domain similarity
Specifies frozen/trainable stages and learning rate
Defines validation, cost, and forgetting measurements

Continue to the strategy rationale

Screen 3 of 3 - Consolidate

By the end of this lesson, you will be able to:

Identify frozen and trainable blocks in a pretrained neural-network architecture.
Compare feature extraction, partial fine-tuning, and full fine-tuning strategies.
Select an adaptation strategy that balances target accuracy, data volume, and forgetting risk.

Covers pretrained models and adaptation strategies. Explains feature extraction, partial, and full fine-tuning. Highlights catastrophic forgetting and best practices.

Mastery artifact

Leave with inspectable evidence, not a completion click

Your artifact combines the prerequisite check, prediction, experiment configuration, displayed evidence, explanation, transfer rubric, and confidence change.

Complete the evidence cycle to unlock the artifact.

Import progress JSON

Retention plan

Retrieve the concept after time has passed

Complete the artifact to schedule a 24-hour retrieval prompt and a seven-day transfer revisit.

Learning-study instrument

Help evaluate whether the laboratory teaches the concept

Clarity 3 / 5 Mental effort 3 / 5 Transfer confidence 3 / 5

No name, email address, free text, or IP address is retained in the learning record.

INTERACTIVE DEEP-LEARNING GUIDE

Transfer Learning and Fine-Tuning

Use the experiment to compare frozen features, partial fine-tuning, and full fine-tuning while tracking trainable parameters, adaptation, and forgetting risk.

What to change

Change the pretrained backbone, target-sample count, source–target similarity, and fine-tuning boundary separately.

What to inspect

Track which layers receive gradients, the number of trainable parameters, adaptation capacity, cost, and forgetting risk.

What to conclude

Fine-tune only as much of the network as the target evidence supports; more trainable layers are not automatically better.

TRANSFER LEARNING FAQ

What is transfer learning?

Transfer learning reuses representations learned on a source task to improve learning on a related target task.

When should the backbone stay frozen?

Freezing is useful with limited target data, similar domains, or strict compute and stability constraints.

When is partial fine-tuning appropriate?

It is often effective when the target differs moderately and enough data exists to adapt higher-level features.

When should the full model be fine-tuned?

Full fine-tuning may help with a large target dataset or substantial domain shift, but it increases cost and forgetting risk.

What is catastrophic forgetting?

It is the loss of useful pretrained behavior when target-task updates overwrite previously learned representations.

How should learning rates be chosen?

Use smaller rates for pretrained layers and potentially larger rates for newly initialized task-specific layers.

Does a larger target dataset always justify full fine-tuning?

No. Domain similarity, label quality, compute, robustness, and validation evidence still determine the best strategy.

How should transfer learning be evaluated?

Compare against training from scratch and frozen baselines using held-out performance, calibration, robustness, and deployment cost.

Explore all interactive deep-learning laboratories View the Modern Deep Learning Foundations course