# Algorytmy RL

EN: **Reinforcement learning algorithms** · id `rlalgorithms`

Path: [AI Atlas](https://aiatlasgraph.com/llms.txt) › [Uczenie ze wzmocnieniem i decyzje](https://aiatlasgraph.com/t/decisions.md) › Algorytmy RL

## Description

- PL: Metody uczenia wartości i polityki.
- EN: Methods for learning values and policies.

## In one sentence

- PL: To metody poprawiania wyborów programu na podstawie ocen dotychczasowych działań.
- EN: These methods improve a program's choices using assessments of its previous actions.

## Problem it solves

- PL: Program zbiera doświadczenia, ale potrzebuje sposobu przełożenia ich na lepsze decyzje w kolejnych próbach.
- EN: Software collects experience but needs a way to turn it into better decisions on subsequent trials.

## What it does

- PL: Uczy ocen możliwych działań, bezpośrednio zmienia sposób ich wybierania albo łączy oba podejścia na podstawie rezultatów prób.
- EN: It learns action values, directly changes how actions are chosen or combines both approaches using trial outcomes.

## Everyday example

- PL: Program sterujący postacią wykorzystuje wyniki rozegranych prób, żeby częściej wybierać ruchy prowadzące do ukończenia poziomu.
- EN: Software controlling a character uses previous trial outcomes to choose moves leading to level completion more often.

## This concerns you if you…

- PL: masz środowisko prób i sposób oceny wyniku
- PL: chcesz porównać metody uczenia kolejnych decyzji
- EN: have a trial environment and a way to score outcomes
- EN: want to compare methods for learning sequential decisions

## Keywords

DQN · Double DQN · REINFORCE · A2C · A3C · PPO · SAC · TD3 · on-policy learning · off-policy learning

## How it works

- PL: Algorytmy aktualizują wartości przez cele oparte na zwrocie lub równaniu Bellmana, a polityki przez gradient lub wybór najlepiej ocenionych akcji. Bufory doświadczeń i harmonogram zbierania danych określają sposób wykorzystania interakcji.
- EN: Algorithms update values using return-based or Bellman targets, and policies through gradients or choosing highly valued actions. Experience buffers and collection schedules determine how interactions are used.

## Limits

- PL: Dane z wcześniejszej polityki nie zawsze można ponownie wykorzystać bez korekty. Eksploracja, wariancja estymatorów i niestabilna aproksymacja wartości ograniczają efektywność.
- EN: Earlier-policy data cannot always be reused without correction. Exploration, estimator variance and unstable value approximation limit efficiency.

## Key terms

- PL: **on-policy**: Uczenie wykorzystujące doświadczenia odpowiadające aktualizowanej polityce.
- PL: **off-policy**: Uczenie polityki z doświadczeń zebranych także przez inne strategie.
- EN: **on-policy**: Learning from experience corresponding to the policy being updated.
- EN: **off-policy**: Learning a policy from experience also collected by other strategies.

## Know first

- [Funkcje wartości i równanie Bellmana (Value functions and the Bellman equation)](https://aiatlasgraph.com/t/bellman.md)
- [Proces uczenia (Training optimization)](https://aiatlasgraph.com/t/optimization.md)

## Examples

- PL: Q-learning · policy gradient · actor–critic

## Subtopics (10 direct, 10 in total)

- [Bezpieczne RL i RL z ograniczeniami (Safe and constrained reinforcement learning)](https://aiatlasgraph.com/t/saferl.md): Learning policies that maximise reward while aiming to keep cost or risk within set limits during training or deployment, e.g. through a separate cap on expected cost instead of folding penalties into the reward. Guarantees usually hold in expectation, not for every run.
- [Bufor doświadczeń i priorytetowe odtwarzanie (Experience replay and prioritised replay)](https://aiatlasgraph.com/t/experiencereplay.md): A mechanism that stores transitions for repeated off-policy learning; prioritised replay samples more informative experiences more often.
- [Deterministyczne gradienty polityki (Deterministic policy-gradient methods)](https://aiatlasgraph.com/t/ddpgtd3.md): Off-policy actor–critic methods for continuous actions with a deterministic actor; TD3 adds fixes that curb DDPG's value overestimation.
- [Gradient polityki (REINFORCE) (Policy gradient methods (REINFORCE))](https://aiatlasgraph.com/t/policygradient.md): Directly adjusting policy parameters towards actions that led to higher reward. Simple and general, but sample-hungry.
- [Hierarchiczne RL i opcje (Hierarchical reinforcement learning and options)](https://aiatlasgraph.com/t/hierarchicalrl.md): Splitting behaviour into levels: a higher-level policy picks multi-step skills (options), which in turn choose low-level moves and when to stop.
- [Metody aktor–krytyk (A2C, SAC) (Actor–critic methods (A2C, SAC))](https://aiatlasgraph.com/t/actorcritic.md): Combines a learned policy (actor) with a value or Q function (critic) that guides policy updates and can reduce gradient-estimator variance. A widely used family for continuous control.
- [Proximal Policy Optimization (PPO) (Proximal policy optimization (PPO))](https://aiatlasgraph.com/t/ppo.md): A policy-gradient method using a surrogate objective designed to constrain overly large policy changes during updates. Widely used in simulation, robotics and some RLHF systems.
- [Q-learning i głębokie sieci Q (DQN) (Q-learning and deep Q-networks (DQN))](https://aiatlasgraph.com/t/qlearning.md): Off-policy learning of action values from the Bellman equation; in DQN a neural network replaces the Q-table. Methods in this family reuse data from past interactions, but training with function approximation needs stabilising mechanisms.
- [Rozkładowe uczenie ze wzmocnieniem (Distributional reinforcement learning)](https://aiatlasgraph.com/t/distributionalrl.md): A family of methods learning the distribution of returns (value distribution) instead of its expectation, via a distributional Bellman equation.
- [Uczenie z celami i odtwarzanie z perspektywy czasu (HER) (Goal-conditioned RL and hindsight experience replay)](https://aiatlasgraph.com/t/goalconditioned.md): One policy trained for many goals at once, with the goal given as an extra input. Hindsight replay relabels failed attempts as if the goal had been whatever was actually reached, enabling learning from success-only rewards.

## Related topics

Untyped, symmetric "see also" links.

- [Preferencje i RL dla LLM (Preference optimization and RL for LLMs)](https://aiatlasgraph.com/t/preferences.md)
- [Uczenie wieloagentowe (MARL) (Multi-agent reinforcement learning (MARL))](https://aiatlasgraph.com/t/marl.md)

## Sources

- [Berkeley CS185/285 · Deep Reinforcement Learning (2026)](<https://rail.eecs.berkeley.edu/deeprlcourse/>) `rl` · reference: Uczenie przez wzmacnianie, imitacja, sterowanie i RL dla LLM.
- [Stanford CS221 · AI: Principles and Techniques (2025)](<https://stanford-cs221.github.io/autumn2025/>) `cs221` · reference: Rozumowanie, przeszukiwanie, logika i modele probabilistyczne.

---

AI Atlas revision 122 (2026-10-30) · HTML: https://aiatlasgraph.com/t/rlalgorithms · index: https://aiatlasgraph.com/llms.txt · everything: https://aiatlasgraph.com/llms-full.txt · updated 2026-10-30