Mixture of Experts (MoE) — mieszanka ekspertów (Mixture of Experts (MoE)): Layers containing many expert subnetworks, where a router sends each token to only a selected subset. Sparse parameter activation can reduce compute per token compared with a dense model of similar total parameter count, while weight storage, communication and load balancing remain important costs.
Modele przestrzeni stanów (Mamba) (State-space models (Mamba)): A family of sequence models that update a fixed-size state instead of storing a full attention matrix or KV cache. Mamba introduces a selective, input-dependent state mechanism; its authors report linear scaling with sequence length and competitive results across several modalities.
Uwaga lokalna z przesuwnym oknem (Sliding-window (local) attention): Local attention in which a token interacts only with a bounded window of neighbouring tokens rather than the entire sequence. With a fixed window, total compute grows linearly with sequence length, and the KV memory used for generation can be bounded by the window size. Some architectures combine it with global or full attention.
Funkcje aktywacji (ReLU, GELU, SwiGLU) (Activation functions (ReLU, GELU, SwiGLU)): Non-linear functions applied to neuron outputs — without them, a composition of linear layers remains a linear transformation. ReLU is common in vision networks; transformers often use GELU and gated variants such as SwiGLU in feed-forward blocks.
Funkcje straty (entropia krzyżowa, MSE) (Loss functions (cross-entropy, MSE)): The number training minimises: how far predictions are from the target. Cross-entropy is used for classification and next-token prediction; mean squared error (MSE) for regression and, among others, noise prediction in diffusion models.
Perceptron wielowarstwowy (MLP) (Multilayer perceptron (MLP)): A network of fully connected layers that alternate linear transformations with non-linear activation functions. The MLP is a fundamental neural model and a standard component of feed-forward blocks in transformer architectures.
Połączenia resztkowe (skip connections) (Residual (skip) connections): Shortcut connections that add a block's input to its output, allowing the layer to learn a correction relative to the input representation. They enabled effective training of very deep ResNets and are a common component of many modern deep architectures, including transformers.
Propagacja wsteczna (backpropagation) (Backpropagation): An algorithm for computing the gradient of the loss with respect to network parameters: the forward pass computes the output, while the backward pass propagates derivatives through the computation graph using the chain rule. Gradient-based optimisers use these gradients; libraries usually obtain them through automatic differentiation.
Embeddingi zdań i dokumentów (modele embeddingowe) (Sentence and document embeddings (embedding models)): Models that turn a whole sentence or document chunk into a vector compared, for example, with cosine similarity. They are commonly used in semantic retrieval and RAG pipelines, as well as clustering and deduplication; embedding models are compared on benchmarks such as MTEB. Cross-encoders for reranking and multi-vector representations complement them.
Osadzenia słów (word2vec, GloVe) (Word embeddings (word2vec, GloVe)): Static vectors assigned to words and learned from regularities in their contexts, so that similar uses receive nearby representations. word2vec and GloVe were important steps in the development of text representations; a single vector per word does not distinguish meanings that depend on the particular context.
Autoenkodery wariacyjne (VAE) (Variational autoencoders (VAE)): A model with an encoder that compresses data into a latent distribution and a decoder that reconstructs it from samples; trained by maximising the evidence lower bound (ELBO) with the reparameterisation trick. Today it often works behind the scenes — as the image compressor in latent diffusion.
Generatywne sieci przeciwstawne (GAN) (Generative adversarial networks (GAN)): Two networks trained in an adversarial game: a generator creates samples and a discriminator distinguishes them from real data; the discriminator's signal trains the generator to better match the data distribution. Training can be unstable and suffer from mode collapse.
Dyfuzja w przestrzeni ukrytej (Stable Diffusion) (Latent diffusion (Stable Diffusion)): Diffusion performed in an autoencoder's compressed latent space rather than directly in pixel space, reducing computation; text conditioning can be implemented through cross-attention. This design is used by Latent Diffusion models, including Stable Diffusion.
Sterowanie promptem bez klasyfikatora (classifier-free guidance) (Classifier-free guidance): A technique to strengthen adherence to the prompt: one model learns to generate with and without the condition (dropped randomly in training), and at sampling time the output is pushed toward the conditional version. The guidance scale is the familiar “prompt fidelity vs diversity” knob.
Zaszumianie i odszumianie krok po kroku (DDPM) (Forward noising and learned denoising (DDPM)): The basic construction of denoising diffusion probabilistic models: a forward process gradually adds noise to data, while a model learns to reverse the corruption, often by predicting the added noise. Sampling-step count affects cost and quality, and separate fast-sampling or distillation techniques can reduce it.
Przepływy normalizujące i flow matching (Normalising flows and flow matching): Models that turn simple noise into data with an invertible transform (normalising flows) or a learned velocity field along a path from noise to data (flow matching, rectified flow). Flow matching includes diffusion as a special case and underpins many new image models, e.g. Stable Diffusion 3.
Rozumowanie w modelach językowych (Reasoning in language models): Techniques and model families for tasks that require multi-step reasoning: from chain-of-thought prompting and larger inference-time compute budgets to training reasoning models with RL, SFT and distillation.
Skracanie łańcucha myśli (Reasoning length reduction): Training or prompting aimed at reducing reasoning length while preserving answer quality as much as possible — for example via a length-cost signal, fine-tuning on concise traces, or brevity instructions.
Monitorowanie łańcucha myśli (Chain-of-thought monitoring): Analysing a model's written chain of thought (automatically or by a human) to detect signals of undesirable behaviour. Its usefulness depends in part on the monitorability and faithfulness of the reasoning trace, which are not guaranteed.
Obliczenia w czasie wnioskowania (Test-time compute scaling): Giving a model a bigger compute budget at answer time — without changing weights: many attempts in parallel, successive revisions, verifier-guided search, or simply thinking longer.
RL z weryfikowalnymi nagrodami (RLVR) (Reinforcement learning with verifiable rewards): The model receives reward from an automatically verifiable signal — such as answer correctness, passing code tests, or satisfying a checkable condition — rather than relying only on a learned preference model. RLVR is an important post-training approach for reasoning models, especially in domains with unambiguous verification.
Weryfikatory rozumowania (Reasoning verifiers): Models or programs that judge whether reasoning is correct. Used both as a training reward and to pick the best answer at inference time.
Sprawdzanie końcowego wyniku (Outcome verification): Only the final answer is judged: by comparing with a reference, running tests, or an outcome reward model (ORM). Simpler and cheaper than step scoring.
Drzewo myśli (Tree of Thoughts) (Tree of Thoughts): The model expands several alternative reasoning paths, evaluates which look promising and can backtrack — tree search instead of one line of thought.
Sieci grafowe (GNN) (Graph neural networks (GNN)): Neural networks operating on graphs (nodes, edges, features), invariant to node ordering; the basis of learning on molecules, social networks and meshes.
Transformery grafowe (Graph transformers): Architectures combining a transformer's global attention with graph positional and structural encodings, often alongside local message passing.
LSTM i GRU — sieci z bramkami pamięci (LSTM and GRU (gated recurrent networks)): Recurrent networks with gating mechanisms that control how information flows into and remains in hidden state. LSTM uses a separate cell state with input, forget and output gates; GRU combines some of these mechanisms in a simpler design with reset and update gates.
Sieci rekurencyjne (RNN) (Recurrent neural networks (RNN)): Networks that process a sequence step by step while carrying a hidden state from the previous step. Classical RNNs can struggle to learn long dependencies because gradients vanish or explode; LSTMs and GRUs mitigate the problem, and transformers have replaced RNNs in many text tasks.
Splot, filtry i pooling (Convolution, filters and pooling): The basic operations of a convolutional network: small filters slide over the image and detect local patterns (edges, textures), and pooling reduces feature-map resolution. Filter size, stride and padding determine the output shape.
FlashAttention: Attention computed with memory in mind: less data movement, the same result.
Kodowanie pozycji (sinusoidalne, RoPE) (Positional encoding (sinusoidal, RoPE)): A way to provide token-order information that the attention operation alone does not encode. The original Transformer added sinusoidal vectors; many contemporary LLMs use RoPE, which rotates queries and keys according to position. For context extension RoPE is often additionally scaled or interpolated; some architectures leave part of their layers without explicit positional encoding.
Mechanizm self-attention (zapytania, klucze, wartości) (Self-attention (queries, keys, values)): The heart of the transformer: each token produces a query (Q), key (K) and value (V), and the output is a weighted average of values, weighted by query–key similarity across all tokens. This lets the model mix information from the whole sequence at once; cost grows quadratically with length. Text-generating models use a causal mask (a token sees only earlier ones).