Layers, loss functions and representation learning.
In one sentence
To podstawowe elementy sieci neuronowej i zasady, dzięki którym uczy się ona na przykładach.
These are the basic parts of a neural network and the principles that let it learn from examples.
Problem it solves
Sieć wygląda jak czarna skrzynka, więc trudno zrozumieć, skąd biorą się jej wyniki i błędy.
A network looks like a black box, making its results and errors difficult to understand.
What it does
Warstwy przekształcają liczby, miara błędu ocenia wynik, a trening wykorzystuje ją do poprawiania połączeń.
Layers transform numbers, an error measure evaluates the result, and training uses it to adjust connections.
Everyday example
Ucząc model rozpoznawać rodzaje kwiatów, obserwujesz, jak kolejne poprawki zmniejszają błędy na przykładach treningowych.
While training a model to recognise flower types, you watch successive adjustments reduce errors on training examples.
This concerns you if you…
zaczynasz poznawać budowę modeli
chcesz rozumieć, co zmienia się podczas uczenia
are starting to learn how models are built
want to understand what changes during training
Examples
MLP · backpropagation
Subtopics
Funkcje aktywacji (ReLU, GELU, SwiGLU) (Activation functions (ReLU, GELU, SwiGLU)): Non-linear functions applied to neuron outputs — without them, a composition of linear layers remains a linear transformation. ReLU is common in vision networks; transformers often use GELU and gated variants such as SwiGLU in feed-forward blocks.
Funkcje straty (entropia krzyżowa, MSE) (Loss functions (cross-entropy, MSE)): The number training minimises: how far predictions are from the target. Cross-entropy is used for classification and next-token prediction; mean squared error (MSE) for regression and, among others, noise prediction in diffusion models.
Perceptron wielowarstwowy (MLP) (Multilayer perceptron (MLP)): A network of fully connected layers that alternate linear transformations with non-linear activation functions. The MLP is a fundamental neural model and a standard component of feed-forward blocks in transformer architectures.
Połączenia resztkowe (skip connections) (Residual (skip) connections): Shortcut connections that add a block's input to its output, allowing the layer to learn a correction relative to the input representation. They enabled effective training of very deep ResNets and are a common component of many modern deep architectures, including transformers.
Propagacja wsteczna (backpropagation) (Backpropagation): An algorithm for computing the gradient of the loss with respect to network parameters: the forward pass computes the output, while the backward pass propagates derivatives through the computation graph using the chain rule. Gradient-based optimisers use these gradients; libraries usually obtain them through automatic differentiation.