Akceleratory

AI hardware · id hardware

Sprzęt pod inferencję i trening.

Hardware for inference and training.

In one sentence

To sprzęt przyspieszający uczenie modeli i przygotowanie ich odpowiedzi.

This hardware accelerates model training and response generation.

Problem it solves

Zwykły procesor może wykonywać obliczenia modelu zbyt wolno, a sam wzrost mocy nie rozwiązuje ograniczeń pamięci.

A general processor may run model calculations too slowly, while more computing power alone cannot resolve memory limits.

What it does

Specjalizowane układy wykonują wiele obliczeń naraz, a pamięć i połączenia dostarczają im dane; wszystkie te elementy wpływają na wydajność.

Specialised chips perform many calculations at once, while memory and connections supply data; all these components affect performance.

Everyday example

Porównując sprzęt do asystenta, sprawdzasz nie tylko szybkość obliczeń, ale też pamięć potrzebną na model i rozmowy.

When comparing hardware for an assistant, you check computing speed and the memory needed for the model and conversations.

This concerns you if you…

• wybierasz sprzęt do modelu
• chcesz rozumieć ograniczenia pamięci i obliczeń
• choose hardware for a model
• want to understand memory and computing limits

Examples

• GPU · układy dedykowane
• GPU · dedicated chips

Subtopics

• Model roofline: obliczenia czy przepustowość pamięci (Roofline model and arithmetic intensity): A simple performance model: attainable throughput is bounded by the minimum of peak compute and memory bandwidth multiplied by arithmetic intensity (operations per byte). Within the model's assumptions, it helps determine whether an operation is compute-bound or memory-bandwidth-bound.
• Pamięć HBM i ściana pamięci (High-bandwidth memory and the memory wall): Accelerator compute grows faster than memory bandwidth, so moving weights and KV cache from HBM can become a bottleneck. This is especially important during LLM token generation, particularly at small batch sizes.
• Połączenia między układami i sieć klastra (Accelerator interconnects and cluster networking): Ways accelerators exchange data: fast links within a server or pod (scale-up) and networking between servers (scale-out), using topologies such as torus or fat-tree. Their bandwidth and latency strongly affect how training and inference scale across many chips.
• Wyspecjalizowane akceleratory AI (TPU, NPU) (Specialised AI accelerators (ASICs and NPUs)): Specialised chips or compute blocks optimised for neural-network operations, often using multiply-accumulate arrays and low-precision arithmetic. They include datacenter accelerators such as TPUs and NPUs integrated into phones and laptops.

Courses

• Fast LLM Inference with Cerebras

Sources

• DeepLearning.AI — course catalogue · course catalogue: 42 topic labels with counts, about 130 courses.
• Performance Engineering for AI Infra — learning guide · GitHub repository · tiered curriculum (Tier 1–3) · MIT licence · Wafer (wafer-ai): Company material with a hiring note, but the evidence is mostly primary sources. Table of contents sieved: only topics became nodes; tiers kept as link labels.