Model Explainability
Use the experiment to compare image explanations, change the attribution method and opacity, and inspect whether highlighted regions support the model’s prediction.
What to change
Choose an image, compare the available explanation methods, and move overlay opacity from the original image to the attribution view.
What to inspect
Check whether highlighted evidence falls on the object, the background, or artifacts that should not drive the prediction.
What to conclude
An attribution map can reveal model sensitivity, but it does not by itself prove causality, fairness, or reliability.
EXPLAINABILITY FAQ
What does an attribution map show?
It visualizes which input regions most influenced a selected model output under a particular explanation method.
What is Grad-CAM?
Grad-CAM combines gradients with late convolutional feature maps to produce a class-specific spatial explanation.
Why compare more than one method?
Different methods make different assumptions and can produce different maps; agreement and stability are useful checks.
Does a clear heatmap prove the prediction is correct?
No. A persuasive-looking map can accompany an incorrect or brittle prediction.
What should opacity change?
Only the visibility of the overlay. The underlying prediction and attribution values should remain unchanged.
How can explanation stability be tested?
Apply small, meaning-preserving input changes and check whether the explanation remains reasonably consistent.
Can explanations detect shortcuts?
They can expose reliance on backgrounds, borders, watermarks, or other spurious cues, but confirmation requires controlled tests.
How should explanations be reported?
Name the model, class, method, parameters, input, and limitations, and keep the original image available for comparison.