Ewaluacja i interpretowalność

Evaluation and interpretability · id evaluation

Jak sprawdzać jakość i rozumieć ograniczenia modeli oraz systemów.

How to check quality and understand the limits of models and systems.

In one sentence

To sprawdzanie, jak dobrze model działa, gdzie się myli i co można ustalić o jego zachowaniu.

This checks how well a model works, where it makes mistakes and what can be learned about its behaviour.

Problem it solves

Przekonująca odpowiedź lub udany pokaz nie mówi, czy system będzie działał poprawnie w różnych rzeczywistych sytuacjach.

A convincing answer or successful demonstration does not show whether a system will work correctly across real situations.

What it does

Łączy testy, analizę błędów, oceny ludzi, obserwację działania i badanie modelu, dobierając miary do zadania oraz jego ryzyka.

It combines tests, error analysis, human assessments, operational monitoring and model studies, choosing measures suited to the task and its risks.

Everyday example

Przed użyciem asystenta sprawdzasz typowe i trudne sprawy, a po wdrożeniu oglądasz pomyłki, koszty i zmiany jakości.

Before using an assistant, you test ordinary and difficult cases; after deployment, you examine mistakes, costs and changes in quality.

This concerns you if you…

• chcesz sprawdzić działanie modelu poza udanym pokazem
• musisz znać pomyłki, ograniczenia i jakość systemu
• want to assess a model beyond a successful demonstration
• need to understand errors, limits and system quality

Subtopics

• Analiza błędów (Error analysis): Reading real traces of a system and turning the notes into a list of the ways it fails. Only that list tells you what is worth measuring.
  • Kodowanie otwarte i taksonomia błędów (Open and axial coding to build a failure taxonomy): Qualitative trace analysis: open coding of failure notes, then axial coding into a taxonomy of failure modes.
  • Przegląd i próbkowanie śladów (Trace review and sampling): Which session records to look at, and in what tool, so a person can review them quickly and carefully.
• Ewaluacja RAG i agentów (RAG and agent evaluation): Checking sources, trajectories and the final result.
  • Miary jakości wyszukiwania (recall@k, MRR, nDCG) (Retrieval quality metrics (recall@k, MRR, nDCG)): Ranking metrics that grade the retrieval stage against labelled relevant documents.
  • Ocena po stanie końcowym zamiast po przebiegu (Outcome-based vs transcript-based grading of agents): Grading agents by verifying environment state after a trial, complemented by checks on the transcript.
  • Powtarzalność agenta: pass@k i pass^k (Agent reliability metrics: pass@k and pass^k): Metrics from repeated trials: pass@k (at least one success in k) and pass^k (all k succeed).
  • Symulowany użytkownik w testach rozmów wieloetapowych (Simulated users for multi-turn agent evaluation): Evaluating conversational agents with an LLM that role-plays the user from a hidden task instruction.
  • Wierność odpowiedzi wobec znalezionych źródeł (RAG) (RAG faithfulness and answer relevance metrics): RAG metrics scoring how grounded an answer is in its context, how relevant it is to the question and how relevant the context is.
• Ewaluatory kodowe (Code-based evaluators): A deterministic check in code when a failure can be described by a rule. The cheapest rung of evaluation — before an LLM judge.
  • Asercje deterministyczne: format, schemat, dopasowanie tekstu (Deterministic assertions: format, schema and string checks): Code graders that check outputs with rules: exact or regex match, schema validation, length and keyword presence.
  • Ocena przez uruchomienie testów (poprawność funkcjonalna) (Execution-based grading with unit tests): Grading generated code by executing it in a sandbox against unit or integration tests.
• Interpretowalność (Interpretability and explainability): How to understand why a model made a given decision: from models that are transparent by design, through explanations added from outside (SHAP, saliency maps), to looking inside the network.
  • Interpretowalność mechanistyczna (Mechanistic interpretability): Studying a model's internal components and computations and testing how they contribute to its behaviour.
    • Obwody i grafy atrybucji (circuit tracing) (Circuits and attribution graphs): Building an approximate graph of features and connections that may mediate the computation behind a model answer. The graph requires intervention-based validation and does not guarantee a complete, faithful reconstruction of the mechanism.
    • Podmiana aktywacji (activation patching) (Activation patching): An intervention experiment: activations of a chosen component are replaced with activations from another input and the output change is measured. This provides causal evidence whose interpretation depends on the experimental design and metric.
    • Rzadkie autoenkodery (SAE) i słowniki cech (Sparse autoencoders and dictionary learning): Decomposing model activations into many sparsely active features, some of which can be associated with understandable concepts. Feature interpretability and practical downstream usefulness are not guaranteed.
  • Mapy istotności dla sieci neuronowych (Saliency maps and gradient attribution): Visualisations assigning importance to pixels or tokens for a network output using gradients or activations. High importance is not by itself evidence of causal influence.
  • Modele interpretowalne z natury (Inherently interpretable models): Models simple enough that their decision can be read directly — from weights, rules or a tree path — without extra explanation tools.
  • Ogólne zachowanie modelu: ważność cech i wykresy zależności (Global model explanations: feature importance and dependence plots): A description of model behaviour across a dataset: which features matter most to predictions and how model output varies as a feature changes. Such plots do not by themselves establish causality.
  • Sondy klasyfikacyjne (probing) (Probing classifiers): A classifier trained on a model's internal representations to test whether a property can be decoded from them. Strong probe performance alone does not prove that the model uses that information in its behaviour.
  • Sterowanie aktywacjami (wektory sterujące) (Activation steering and representation engineering): Finding directions or representations in activations associated with a concept and amplifying or suppressing them at run time to study or change model behaviour without retraining.
  • Wyjaśnianie pojedynczej decyzji (SHAP, LIME) (Local feature attribution (SHAP, LIME)): Methods that explain an individual prediction by assigning contributions to input features. They may treat the model as a black box or exploit its structure.
  • Wyjaśnienia „co by było, gdyby” (kontrfaktyczne) (Counterfactual explanations): The smallest change to the input that would flip the model's decision — e.g. what would have to change in an application for the loan to be approved.
• Metryki i benchmarki (Metrics and benchmarks): Choosing a measure that fits the task and comparing models reliably: classic metrics, standardised benchmarks, leaderboards and statistical uncertainty in results.
  • Benchmarki modeli językowych (LLM benchmarks): Standardised task sets used to compare language models. They are grouped by the skill measured, such as knowledge and reasoning, coding or agent work.
    • Benchmarki agentowe (Agent benchmarks): Environments where a model carries out multi-step tasks with tools — in a browser, an operating system or a conversation with a simulated customer — and is scored on whether the goal was achieved.
    • Benchmarki długiego kontekstu (igła w stogu siana i dalej) (Long-context benchmarks (needle-in-a-haystack and beyond)): Synthetic and realistic tests of effective context length: retrieving planted facts, multi-hop tracing and aggregation.
    • Benchmarki programistyczne (Coding benchmarks): Tasks in which a model writes or fixes code and tests check the result — from single functions to real issues in repositories.
    • Benchmarki wiedzy i rozumowania (Knowledge and reasoning benchmarks): Closed-ended questions across many fields, from school level to expert level, graded automatically. Older sets saturate quickly, so ever harder ones are created.
    • Przeciek danych i nasycenie benchmarków (Benchmark contamination and saturation): Why a benchmark score can mislead: test questions may have leaked into training data, and popular tests stop separating top models once all score near 100%.
  • Metryki klasyfikacji i regresji (Classification and regression metrics): Classic measures for models that predict a class or a number. The choice depends on the cost of errors and whether classes are balanced.
  • Niepewność statystyczna wyników ewaluacji (Statistical uncertainty in evaluations): An evaluation is an experiment on a sample of tasks, so its score is uncertain. Without estimating that uncertainty, a small gap between models may be indistinguishable from sampling variation.
  • Porównanie tekstu z wzorcem (BLEU, ROUGE, BERTScore) (Reference-based text generation metrics): Automatic scoring of generated text by comparing it to a reference answer — from counting shared words to meaning similarity in embeddings.
  • Rankingi z głosami ludzi (areny) (Human-preference leaderboards (arenas)): People anonymously compare two model answers and choose the better one; many such comparisons produce a ranking. The result nevertheless depends on vote collection, models' access to the arena and comparison sampling.
• Monitoring produkcyjny (Monitoring and observability): Watching a system after deployment: traces, cost, regressions.
  • Ewaluacja online na ruchu produkcyjnym (Online evaluation on production traffic): Continuous grading of sampled production traffic with code graders and LLM judges, complementing offline evals.
  • Wykrywanie dryfu danych i pojęć (Data and concept drift detection): Detecting that the production-data distribution or the relationship between inputs and correct outputs has changed relative to training data, which may require diagnosis or model adaptation.
  • Śledzenie wywołań modeli i agentów (tracing) (LLM and agent tracing): Instrumenting LLM applications to record model, tool and agent calls as a tree of spans with attributes.
• Ocena ludzi i modeli (Human evaluation and model judges): Judging answers, with quality control on the judging itself.
  • Model językowy jako sędzia (LLM-as-a-judge) (LLM-as-a-judge): Using a language model as an automatic evaluator in single-answer, pairwise or reference-guided mode.
    • Uprzedzenia sędziów LLM: kolejność, długość, własny styl (LLM judge biases and mitigation): Systematic errors of LLM judges (position, verbosity, self-preference) and techniques to reduce them.
  • Ocena według rubryki i listy kontrolnej (Rubric-based and checklist grading): Grading answers against explicit criteria with level descriptions or binary checklist items, done by people or an LLM judge.
  • Walidacja ewaluatorów (Evaluator validation against human labels): Testing an LLM judge or classifier against human labels, on a set that played no part in tuning it.
• Odporność i niepewność (Robustness and calibration): Behaviour on unusual data, and how trustworthy the confidence is.
  • Kalibracja pewności modelu (Confidence calibration): Whether a predicted probability or other confidence score matches how often the model is right — e.g. whether 80%-confidence predictions are correct about 80% of the time — and how to correct a mismatch.
  • Predykcja konformalna (Conformal prediction): Methods that equip any model with prediction sets or intervals having controlled marginal coverage under exchangeable data, without parametric distribution assumptions.
  • Wykrywanie halucynacji (Hallucination detection): Checking whether a model is inventing facts: against the provided source, against outside knowledge, or by how much its answers vary when the question is repeated.
  • Wykrywanie przykładów spoza rozkładu (OOD) (Out-of-distribution detection): Recognising inputs that do not fit the training-data distribution, so the system can raise an alert, abstain or hand the decision to a person.
• Zbiory testowe i regresje (Evaluation datasets and regression tests): Repeatable checks on behaviour after changes.
  • Ewaluacja agentów zgodna z przepływem zadań (Task-flow aligned agent evaluation): Method turns issue-derived tasks into multi-turn interactions matching measured user task flows.
  • Frameworki do uruchamiania ewaluacji (Evaluation harnesses and frameworks): Tools that run test sets against a model or agent repeatably — with the same prompts, settings, sandbox and logs — so results can be compared and reproduced.
  • Syntetyczne przypadki testowe z wymiarów zapytań (Synthetic test case generation for evals): Building eval sets from query variation dimensions, value tuples and a separate conversion step into natural language.

Sources

• Full Stack Deep Learning (2022, archiwum) · reference: Pełny cykl systemu ML: dane, testy, wdrażanie, monitoring, zespoły i etyka.
• Hugging Face · Agents Course · reference: Agenci, narzędzia, frameworki, agentic RAG i ewaluacja.
• Stanford CS229 · Machine Learning (2018, archiwum) · reference: Klasyczne ML, statystyka i podstawy uczenia. Użyte jako stabilny szkielet, nie przegląd nowości.
• DeepLearning.AI — course catalogue · course catalogue: 42 topic labels with counts, about 130 courses.