Interpretowalność

Interpretability and explainability · id explainability

Jak zrozumieć, dlaczego model podjął daną decyzję: od modeli przejrzystych z natury, przez wyjaśnienia dokładane z zewnątrz (SHAP, mapy istotności), po zaglądanie do wnętrza sieci.

How to understand why a model made a given decision: from models that are transparent by design, through explanations added from outside (SHAP, saliency maps), to looking inside the network.

In one sentence

To badanie, które informacje i obliczenia wiążą się z odpowiedzią modelu oraz jak zmieniają jego zachowanie.

This investigates which information and computations relate to a model's answer and how they affect its behaviour.

Problem it solves

Model podaje wynik, ale trudno ocenić, na czym się opiera i czy wykorzystuje właściwe informacje.

A model gives a result, but its basis and whether it uses appropriate information can be hard to assess.

What it does

Odczytuje proste reguły, porównuje zmiany wejść lub bada wnętrze modelu, sprawdzając też ograniczenia i wierność uzyskanych wyjaśnień.

It reads simple rules, compares input changes or studies model internals, also checking the limits and faithfulness of resulting explanations.

Everyday example

Model przewiduje opóźnienie dostawy; sprawdzasz wpływ trasy i wagi paczki oraz to, czy korzysta z nieprzydatnej przypadkowej cechy.

A model predicts delivery delays; you examine route and parcel weight effects and whether it relies on an irrelevant accidental feature.

This concerns you if you…

chcesz zrozumieć podstawy wyniku modelu
musisz sprawdzać, jakie informacje wpływają na zachowanie
want to understand the basis of a model result
need to examine what information affects its behaviour

Examples

modele przejrzyste · wyjaśnienia lokalne i globalne · sondy · interpretowalność mechanistyczna
transparent models · local and global explanations · probes · mechanistic interpretability

Subtopics

Interpretowalność mechanistyczna (Mechanistic interpretability): Studying a model's internal components and computations and testing how they contribute to its behaviour.
- Obwody i grafy atrybucji (circuit tracing) (Circuits and attribution graphs): Building an approximate graph of features and connections that may mediate the computation behind a model answer. The graph requires intervention-based validation and does not guarantee a complete, faithful reconstruction of the mechanism.
- Podmiana aktywacji (activation patching) (Activation patching): An intervention experiment: activations of a chosen component are replaced with activations from another input and the output change is measured. This provides causal evidence whose interpretation depends on the experimental design and metric.
- Rzadkie autoenkodery (SAE) i słowniki cech (Sparse autoencoders and dictionary learning): Decomposing model activations into many sparsely active features, some of which can be associated with understandable concepts. Feature interpretability and practical downstream usefulness are not guaranteed.
Mapy istotności dla sieci neuronowych (Saliency maps and gradient attribution): Visualisations assigning importance to pixels or tokens for a network output using gradients or activations. High importance is not by itself evidence of causal influence.
Modele interpretowalne z natury (Inherently interpretable models): Models simple enough that their decision can be read directly — from weights, rules or a tree path — without extra explanation tools.
Ogólne zachowanie modelu: ważność cech i wykresy zależności (Global model explanations: feature importance and dependence plots): A description of model behaviour across a dataset: which features matter most to predictions and how model output varies as a feature changes. Such plots do not by themselves establish causality.
Sondy klasyfikacyjne (probing) (Probing classifiers): A classifier trained on a model's internal representations to test whether a property can be decoded from them. Strong probe performance alone does not prove that the model uses that information in its behaviour.
Sterowanie aktywacjami (wektory sterujące) (Activation steering and representation engineering): Finding directions or representations in activations associated with a concept and amplifying or suppressing them at run time to study or change model behaviour without retraining.
Wyjaśnianie pojedynczej decyzji (SHAP, LIME) (Local feature attribution (SHAP, LIME)): Methods that explain an individual prediction by assigning contributions to input features. They may treat the model as a black box or exploit its structure.
Wyjaśnienia „co by było, gdyby” (kontrfaktyczne) (Counterfactual explanations): The smallest change to the input that would flip the model's decision — e.g. what would have to change in an application for the loan to be approved.

Sources

Full Stack Deep Learning (2022, archiwum) · reference: Pełny cykl systemu ML: dane, testy, wdrażanie, monitoring, zespoły i etyka.
Hugging Face · Agents Course · reference: Agenci, narzędzia, frameworki, agentic RAG i ewaluacja.
Stanford CS229 · Machine Learning (2018, archiwum) · reference: Klasyczne ML, statystyka i podstawy uczenia. Użyte jako stabilny szkielet, nie przegląd nowości.
Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (3rd edition) · Christoph Molnar