Complete self-study package

Modern Deep Learning Foundations

Learn by predicting, running mathematically grounded experiments, explaining evidence, and exporting an inspectable mastery artifact.

Laboratories: 6
Guided time: 4h 20m
Mastery artifacts: 6

0 of 6 laboratories complete Your progress is stored in this browser. Continue this laboratory

Lab 4 of 6 Self-Attention and the Transformer Principle Intermediate / 45 min / Deliverable: Attention computation trace

Screen 1 of 3 - Read before you experiment

Attention constructs a new representation for every token

Each token is projected into Query, Key, and Value vectors. Query-Key dot products create compatibility scores, scaling controls their magnitude, and softmax converts them into attention weights.

A causal mask removes future positions before softmax. The resulting weights combine Value vectors into a context vector that represents the selected token in its current sequence.

Exact book map

Lesson 11 - Self-Attention and the Transformer Principle

Printed pages 72-76; PDF pages 74-78.

Open the exact pages

Query, Key, and Value Projections printed pp. 72-73 / PDF pp. 74-75 projection paragraphs; Equations 11.1-11.2
Scaled Scores, Masking, and Softmax printed pp. 73-74 / PDF pp. 75-76 attention normalization; Equations 11.3-11.5
Context Vector and Transformer Block printed pp. 74-76 / PDF pp. 76-78 weighted sum and block composition; Equation 11.6

Evidence levelExact deterministic educational forward pass, not pretrained GPT weights

What is real, computed, or illustrative

Evidence and provenance

Derived Forward pass Deterministic educational Transformer calculation

Token IDs, embeddings, Q/K/V projections, scores, softmax, and context vectors are computed in the browser.

Derived Attention rule Scaled dot-product attention with optional causal masking

Masked future positions are removed before softmax and therefore receive zero probability.

Illustrative Parameters Small transparent matrices rather than pretrained GPT weights

The compact dimensions make every number inspectable; they are not claims about a production language model.

Scope Claim boundary Attention weights are computational coefficients

A high weight shows contribution in this forward pass, not human-like reasoning or causal importance.

60-second prerequisite check

Choose the support level from evidence

Why is a causal mask applied before softmax in autoregressive attention? To give future positions zero probability To make every token attend equally To remove the Value vectors

Masked logits are excluded before normalization, so future positions receive zero probability in an autoregressive forward pass.

Guided task

Complete these three moves

1
Run
Enter a short sentence, run the forward pass, and select one query token to follow.
2
Inspect
Move through Tokenize, Embedding, Q/K/V, Scaled Scores, Softmax, Context, and Output in order.
3
Explain
Identify the strongest allowed key and explain how masking and softmax produced the context vector.

Record your prediction before you run the lab

Select one query token and predict which earlier token will receive the strongest allowed attention weight. Explain the role of the mask.

Confidence before experimenting 50% A useful prediction names the expected change and why it should happen.

Start the transformer walkthrough

Screen 2 of 3 - Experiment

Run, inspect, and compare

Follow the three guided moves above. Change one variable at a time so every visual change has a clear cause.

Guided mode Predict first, advance one stage at a time, and explain the displayed values before changing another control.

Reproducible experiment record

Compare runs instead of trusting one result

Run the laboratory, then capture the controls, metrics, evidence status, seed, and timestamp.

The accessible visual summary will update when the laboratory renders its first result.

Evidence checkpoint

Explain what happened, then transfer it

0 / 3responses complete

Explain the evidence

Follow the selected token through Q/K/V, scaled scores, masking, softmax, and the context vector using at least one displayed number.

Use at least one value or visible change from the experiment.

Transfer the idea

Explain how the computation would differ for bidirectional attention or for a longer sequence with several heads.

Name the new context and the design choice you would make.

Distinguishes causal from bidirectional attention
Explains Q-K scoring and scaling
Explains softmax and the weighted Value sum
Addresses sequence length or multiple heads

Continue to the attention explanation

Screen 3 of 3 - Consolidate

By the end of this lesson, you will be able to:

Follow tokens through embedding and Query, Key, and Value projections.
Explain scaled dot-product attention and why a causal mask blocks future tokens.
Read attention weights and construct the weighted context vector for a selected token.

Explains shift from RNN to self-attention. Introduces Query, Key, Value, and weighted sums. Outlines Transformer encoder-decoder and multi-head attention.

Mastery artifact

Leave with inspectable evidence, not a completion click

Your artifact combines the prerequisite check, prediction, experiment configuration, displayed evidence, explanation, transfer rubric, and confidence change.

Complete the evidence cycle to unlock the artifact.

Import progress JSON

Retention plan

Retrieve the concept after time has passed

Complete the artifact to schedule a 24-hour retrieval prompt and a seven-day transfer revisit.

Learning-study instrument

Help evaluate whether the laboratory teaches the concept

Clarity 3 / 5 Mental effort 3 / 5 Transfer confidence 3 / 5

No name, email address, free text, or IP address is retained in the learning record.