# Język, obraz, audio

EN: **Language, vision and multimodality** · id `modalities`

Path: [AI Atlas](https://aiatlasgraph.com/llms.txt) › Język, obraz, audio

## Description

- PL: Obszary według rodzaju sygnału i zadania. Łączą się z trenowaniem, architekturami i aplikacjami.
- EN: Areas by signal type and task. They connect back to training, architectures and applications.

## In one sentence

- PL: To sposoby, dzięki którym programy pracują z językiem, obrazem, dźwiękiem i przestrzenią.
- EN: These are ways for software to work with language, images, sound and space.

## Problem it solves

- PL: Ważne informacje masz w zdjęciach, rozmowach i dokumentach, a ręczne przeglądanie wszystkiego zajmuje dużo czasu.
- EN: Important information sits in photos, conversations and documents, and checking everything by hand takes time.

## What it does

- PL: Modele uczą się wzorów właściwych różnym rodzajom danych, żeby je opisywać, przekształcać, tworzyć lub łączyć.
- EN: Models learn patterns in different kinds of data so they can describe, transform, create or combine them.

## Everyday example

- PL: Pokazujesz asystentowi zdjęcie przepisu, prosisz głosem o przeliczenie porcji i słyszysz odczytaną listę składników.
- EN: You show an assistant a photographed recipe, ask aloud to adjust the portions and hear the ingredient list read back.

## This concerns you if you…

- PL: pracujesz ze zdjęciami, tekstem lub nagraniami
- PL: chcesz połączyć różne rodzaje informacji
- EN: work with photos, text or recordings
- EN: want to combine different kinds of information

## Keywords

BPE · CLIP · Vision Transformer · Whisper · Stable Diffusion · LLaVA · Qwen3-Omni · NeRF · 3D Gaussian Splatting · AudioSet · ImageNet · COCO · cross-modal alignment

## How it works

- PL: Sygnały przekształca się w tokeny, fragmenty obrazu lub reprezentacje czasowe, a modele uczą ich zależności. Enkodery, moduły fuzji i generatory odwzorowują wejścia na rozpoznane struktury albo nową treść w wybranej modalności.
- EN: Signals become tokens, image patches or temporal representations, and models learn their dependencies. Encoders, fusion modules and generators map inputs to recognised structures or new content in a chosen modality.

## Limits

- PL: Kodowanie może tracić szczegóły, czas lub relacje przestrzenne. Dopasowanie między modalnościami wymaga odpowiednich danych; wiarygodny tekst, obraz czy dźwięk nie potwierdza poprawności treści.
- EN: Encoding can lose detail, timing or spatial relationships. Cross-modal alignment needs suitable data; plausible text, images or audio do not establish content correctness.

## Key terms

- PL: **modalność**: Rodzaj sygnału, taki jak tekst, obraz lub dźwięk.
- PL: **enkoder**: Moduł przekształcający wejście w reprezentację dla dalszych obliczeń.
- PL: **fuzja**: Łączenie informacji z różnych reprezentacji lub rodzajów sygnału.
- EN: **modality**: A signal type, such as text, images or audio.
- EN: **encoder**: A module converting inputs into representations for subsequent computation.
- EN: **fusion**: Combining information from different representations or signal types.

## Know first

- [Reprezentacje i embeddingi (Representation learning)](https://aiatlasgraph.com/t/representations.md)
- [Uczenie nadzorowane (Supervised learning)](https://aiatlasgraph.com/t/supervised.md)

## Subtopics (6 direct, 107 in total)

- [Dane przestrzenne i 3D (Spatial and 3D perception)](https://aiatlasgraph.com/t/spatial.md): Representations of the world and objects in space.
  - [Chmury punktów i dane z LiDAR-u (Point clouds and LiDAR data)](https://aiatlasgraph.com/t/pointclouds.md): Machine-learning methods for sets of 3D points from LiDAR or scanners. PointNet processes points in a permutation-invariant way, while other approaches group them into voxels or vertical pillars; these representations support classification, segmentation and 3D object detection.
  - [Gaussian splatting: szybkie sceny 3D ze zdjęć (3D Gaussian splatting)](https://aiatlasgraph.com/t/gaussiansplatting.md): A scene representation built from anisotropic 3D Gaussians optimised from images and rendered by splatting. It enables high-quality real-time novel-view synthesis and provides an alternative to radiance-field representations.
  - [Generowanie obiektów 3D z tekstu lub zdjęcia (Text-to-3D and image-to-3D generation)](https://aiatlasgraph.com/t/gen3d.md): Creating 3D objects or scenes — such as meshes, Gaussians or radiance fields — from text or images. Methods include distilling guidance from 2D generators, generating consistent multiple views, and models trained directly on 3D asset datasets; outputs often require further editing.
  - [Jednoczesna lokalizacja i budowa mapy (SLAM) (Simultaneous localisation and mapping (SLAM))](https://aiatlasgraph.com/t/slam.md): Simultaneously estimating the pose of a camera or robot while building or updating a map of an unknown environment. Classical methods combine observations with geometric optimisation, while newer systems also use learned modules, recurrent networks and scene representations based on radiance fields or Gaussians.
  - [Neuronowe pola promieniowania (NeRF) (Neural radiance fields (NeRF))](https://aiatlasgraph.com/t/nerf.md): Storing a 3D scene in a neural network that returns colour and density for a point in space and a viewing direction; trained from photos with known camera poses, it renders new, unphotographed views.
  - [Szacowanie głębi z jednego zdjęcia (Monocular depth estimation)](https://aiatlasgraph.com/t/depthestimation.md): Predicting a depth map — each pixel's distance from the camera — from a single image. The output may represent only relative depth ordering or metric distance; training on large, diverse datasets has improved generalisation to new scenes.
- [Generowanie obrazu i wideo (Image and video generation)](https://aiatlasgraph.com/t/imagegen.md): Synthesising and editing visual content.
  - [Edycja obrazu poleceniem i podmiana fragmentów (inpainting) (Instruction-based image editing and inpainting)](https://aiatlasgraph.com/t/imageediting.md): Changing an existing image according to a text instruction, regenerating a masked region (inpainting), or extending the image beyond its original frame (outpainting). Contemporary models can combine generation and editing, but preserving object identity across repeated edits still depends on the model and task.
  - [Generowanie obrazów z opisu tekstowego (Text-to-image generation)](https://aiatlasgraph.com/t/texttoimage.md): Creating an image from a text description. Many contemporary systems use latent diffusion or transformers trained with flow-based objectives. Evaluation includes prompt adherence and image quality; some tools expose settings such as guidance scale, negative prompts or a random seed.
  - [Generowanie wideo z tekstu i obrazu (Text-to-video and image-to-video generation)](https://aiatlasgraph.com/t/videogen.md): Creating clips from text descriptions or animating an image while trying to preserve motion and appearance across frames. Many contemporary systems use latent diffusion models and transformers that process space and time; some also generate synchronised audio.
  - [Personalizacja: nauczenie modelu konkretnej osoby, rzeczy lub stylu (Personalisation: teaching a model a specific subject or style)](https://aiatlasgraph.com/t/subjectpersonalization.md): Teaching a pretrained image model a new person, object or style from a few photos so it can be invoked with a special word in prompts. Methods: full fine-tuning (DreamBooth), a new word embedding (Textual Inversion), lightweight LoRA weights.
  - [Sterowanie kompozycją: poza, kontury, głębia, obraz wzorcowy (Controllable generation: pose, edges, depth, reference image)](https://aiatlasgraph.com/t/controllablegen.md): Guiding generation with signals beyond text: edge maps, depth maps, pose skeletons, segmentation or a reference image. A common approach uses additional adapters or conditioning networks attached to a frozen base model; some conditions can be combined.
- [Język naturalny (Natural language processing)](https://aiatlasgraph.com/t/nlp.md): Understanding and producing language.
  - [Automatyczna poprawa gramatyki i pisowni (Grammatical error correction)](https://aiatlasgraph.com/t/gec.md): Detecting and correcting grammatical, spelling and word-choice errors in text with minimal changes to content.
  - [Klasyfikacja tekstu i analiza sentymentu (Text classification and sentiment analysis)](https://aiatlasgraph.com/t/textclassification.md): Assigning one or more labels to a text: sentiment, topic, spam, intent. Done today with a fine-tuned encoder or without task training — NLI-based zero-shot classification or prompting an LLM.
    - [Klasyfikacja tekstu bez przykładów (zero-shot) (Zero-shot text classification)](https://aiatlasgraph.com/t/zeroshotclassification.md): Assigning texts labels unseen in training, usually by recasting classes as NLI hypotheses or as an LLM prompt.
    - [Opinie o konkretnych cechach (sentyment aspektowy) (Aspect-based sentiment analysis)](https://aiatlasgraph.com/t/aspectsentiment.md): Extracting aspect-level opinions from text: aspect term, aspect category, opinion term and polarity.
    - [Wnioskowanie z tekstu: czy zdanie wynika z innego (NLI) (Natural language inference (textual entailment))](https://aiatlasgraph.com/t/nli.md): Classifying a premise–hypothesis pair as entailment, contradiction or neutral.
  - [Odpowiadanie na pytania (QA) (Question answering (QA))](https://aiatlasgraph.com/t/questionanswering.md): Answering natural-language questions. Extractive QA selects an answer span from a supplied text; open-domain QA first retrieves relevant sources and then derives an answer. This retriever-reader pattern is closely related to later RAG systems.
    - [Odpowiadanie na pytania o dane w tabelach (Table question answering)](https://aiatlasgraph.com/t/tableqa.md): Answering natural-language questions from a table: selecting cells, aggregating, or generating a query such as SQL.
    - [Odpowiadanie na pytania z dużego zbioru tekstów (open-domain QA) (Open-domain question answering (retriever–reader))](https://aiatlasgraph.com/t/opendomainqa.md): Answering questions with no given context by retrieving passages from a large corpus and reading the answer from them.
    - [Wskazywanie odpowiedzi w podanym tekście (extractive QA) (Extractive question answering (reading comprehension))](https://aiatlasgraph.com/t/extractiveqa.md): Answering a question by selecting an answer span in a given context, or stating that the context has no answer.
  - [Streszczanie tekstu (Text summarisation)](https://aiatlasgraph.com/t/summarization.md): Shortening a text while preserving its key information. Extractive summarisation selects source passages, while abstractive summarisation creates new wording; modern language models are widely used for abstractive summaries, where quality evaluation and faithfulness to the source remain important challenges.
    - [Streszczanie przez wybór najważniejszych zdań (Extractive summarisation)](https://aiatlasgraph.com/t/extractivesumm.md): Building a summary from selected source sentences or spans, without generating new text.
    - [Streszczanie własnymi słowami modelu (Abstractive summarisation)](https://aiatlasgraph.com/t/abstractivesumm.md): Generating a summary as new text with a sequence-to-sequence model or LLM, allowing paraphrase and information fusion.
  - [Tokenizacja tekstu (BPE, WordPiece, SentencePiece) (Text tokenisation (BPE, WordPiece, SentencePiece))](https://aiatlasgraph.com/t/tokenization.md): Turning text into a sequence of tokens — sub-word pieces from a fixed vocabulary — before the model sees it. Sub-word methods dominate: BPE (also byte-level, as in GPT), WordPiece (BERT) and Unigram via SentencePiece (T5). Tokenisation affects cost, context length and how well a model handles non-English languages.
  - [Tłumaczenie maszynowe (Machine translation)](https://aiatlasgraph.com/t/machinetranslation.md): Automatic translation of text between languages. It evolved from statistical systems to neural encoder–decoders with attention and Transformers, then to massively multilingual models and prompted LLMs.
    - [Ocena jakości tłumaczenia bez wzorca (quality estimation) (Machine translation quality estimation)](https://aiatlasgraph.com/t/mtqe.md): Predicting machine-translation quality from the source and translation alone, without a reference translation.
    - [Tłumaczenie mowy na inny język (Speech translation)](https://aiatlasgraph.com/t/speechtranslation.md): Translating source speech into target-language text or speech, either cascaded (ASR→MT→TTS) or directly with one model.
  - [Wielojęzyczność i języki o małych zasobach (Multilingual and low-resource language processing)](https://aiatlasgraph.com/t/multilingualnlp.md): Processing many languages at once and languages with little data. It relies on cross-lingual transfer in shared multilingual models; quality for low-resource languages still lags clearly behind English.
  - [Wydobywanie informacji z tekstu (nazwy własne, relacje) (Information extraction (named entities and relations))](https://aiatlasgraph.com/t/infoextraction.md): Turning free text into structured facts: named entities (NER — people, places, organisations, dates), relations and events. Classically done as token labelling; now also by LLMs emitting schema-constrained JSON or by compact zero-shot extractors.
    - [Rozpoznawanie nazw własnych (NER) (Named entity recognition (NER))](https://aiatlasgraph.com/t/ner.md): Labelling text spans with entity types such as person, organisation, location or date; classically framed as token classification with a BIO scheme.
    - [Rozwiązywanie koreferencji (kto to „on”?) (Coreference resolution)](https://aiatlasgraph.com/t/coreference.md): Clustering mentions (names, descriptions, pronouns) that refer to the same entity within a document.
    - [Wydobywanie relacji między encjami (Relation extraction)](https://aiatlasgraph.com/t/relationextraction.md): Identifying typed relations between entity pairs in text and recording them as (subject, relation, object) triples.
    - [Łączenie wzmianek z bazą wiedzy (entity linking) (Entity linking)](https://aiatlasgraph.com/t/entitylinking.md): Assigning a text mention the identifier of a knowledge-base entity, disambiguated by context.
- [Modele multimodalne (Multimodal models)](https://aiatlasgraph.com/t/multimodal.md): Processing several kinds of data together.
  - [Modele audio–język (rozumienie mowy, dźwięków i muzyki) (Audio-language models)](https://aiatlasgraph.com/t/audiolanguage.md): Language models connected to audio representations that can jointly handle speech, environmental sounds and music: transcription, translation, questions about recordings, audio captioning or speaker-related analysis. The supported task range depends on each model's data and design.
  - [Modele wizja–język (opisywanie obrazów i pytania o nie) (Vision-language models (VLMs))](https://aiatlasgraph.com/t/vlm.md): Models that combine images or video with text to generate textual answers: they describe scenes, answer questions about visual content, read documents or identify objects. They vary in scale and design, from on-device models to large systems used for interface analysis and visual reasoning.
    - [Automatyczne opisywanie zdjęć (image captioning) (Image captioning)](https://aiatlasgraph.com/t/imagecaptioning.md): Generating a natural-language description of an image with a model joining an image encoder and a text decoder.
    - [Modele językowe rozumiejące wideo (Video-LLM) (Video-language models (Video-LLMs))](https://aiatlasgraph.com/t/videollm.md): Models joining a video-frame encoder with an LLM for open-ended video understanding: description, questions, temporal localisation and reasoning.
    - [Pytania i odpowiedzi o treść obrazu (VQA) (Visual question answering (VQA))](https://aiatlasgraph.com/t/vqa.md): Answering a natural-language question about an image's content, open-ended or as multiple choice.
    - [Rozumienie dokumentów, wykresów i zrzutów ekranu (Document, chart and screenshot understanding)](https://aiatlasgraph.com/t/docvision.md): Reading and interpreting images rich in text and spatial layout — scans, forms, invoices, charts, infographics and screenshots. Some models work without a separate OCR stage, answer questions or extract data directly from pixels; these capabilities are also used for interface analysis.
    - [Wskazywanie na obrazie rzeczy opisanej słowami (visual grounding) (Visual grounding (referring expression comprehension))](https://aiatlasgraph.com/t/visualgrounding.md): Localising the image region (box or mask) matching a language expression, including expressions matching several objects or none.
    - [Łączenie enkodera obrazu z modelem językowym (Connecting an image encoder to a language model)](https://aiatlasgraph.com/t/vlmconnector.md): A common way to build a VLM: a pretrained image encoder produces visual features, while a projector, adapter or cross-attention layers expose them to a language model. The system is then tuned on image-instruction data; the modules trained and number of stages depend on the architecture.
  - [Ujednolicone modele wielomodalne (wejście i wyjście) (Unified multimodal models with multimodal input and output)](https://aiatlasgraph.com/t/omnimodels.md): Models that handle several input modalities — such as text, images, audio and video — and at least several output forms, such as text or speech. They combine these signals in one system; architectures differ in tokenisation, modality fusion and output generation.
- [Mowa i audio (Speech and audio)](https://aiatlasgraph.com/t/audio.md): Analysing and synthesising sound.
  - [Agenci głosowi (Real-time voice agents)](https://aiatlasgraph.com/t/voiceagents.md): A system that conducts spoken conversations, understands utterances, makes decisions and completes tasks using models and tools.
    - [Architektury agentów głosowych (Voice agent architectures)](https://aiatlasgraph.com/t/voicearch.md): Ways to combine speech understanding, agent decisions and voice generation: modular, integrated or with delegated reasoning.
      - [Głosowy front i osobny agent zadaniowy (Voice frontend with delegated reasoning)](https://aiatlasgraph.com/t/delegatedvoice.md): A voice model conducts the live conversation while delegating reasoning and tools to a separate text agent; the result returns to the spoken dialogue.
      - [Modele mowa–mowa (Speech-to-speech models)](https://aiatlasgraph.com/t/speech2speech.md): A model takes speech and generates speech without separate STT and TTS services in the pipeline. Its internal representation may still include text or transcription; tool use and context control depend on the model.
      - [Potok STT → agent tekstowy → TTS (Cascaded STT–text agent–TTS pipeline)](https://aiatlasgraph.com/t/cascadedpipeline.md): Separate speech recognition, a text agent and speech synthesis. Stages can be swapped, inspected and connected to an existing workflow.
    - [Ewaluacja agentów głosowych (Voice agent evaluation)](https://aiatlasgraph.com/t/voiceevals.md): Evaluating task success and conversational quality on real audio: tools, turns, interruptions, audible latency and robustness across speaking conditions.
      - [Ocena tur, przerwań i słyszalnego czasu reakcji (Turn-taking, interruption and audible-latency evaluation)](https://aiatlasgraph.com/t/turnquality.md): Tests measure premature takeovers, unwanted silence, overlap, post-interruption behavior and time to a useful audible response.
      - [Odporność na akcent, hałas i cudzą mowę (Robustness to accents, noise and third-party speech)](https://aiatlasgraph.com/t/audiorobustness.md): The same tasks are tested with varied accents, noise and other speakers, tracking understanding, turns and task outcomes separately.
      - [Symulacje wieloturowych rozmów audio (Multi-turn audio conversation simulation)](https://aiatlasgraph.com/t/audiosim.md): A simulated caller speaks with the agent through a full scenario, including pauses, corrections and interruptions; both dialogue and task state are evaluated.
      - [Weryfikacja wyniku zadania i narzędzi (Voice-agent task and tool outcome verification)](https://aiatlasgraph.com/t/taskoutcome.md): Evaluation compares spoken user intent, tool calls, final system state and the agent's spoken confirmation.
    - [Generowanie odpowiedzi w czasie rzeczywistym (Real-time spoken response generation)](https://aiatlasgraph.com/t/voicelatency.md): Preparing agent speech so a useful response begins promptly while the model remains responsive to further user speech.
      - [Strumieniowe przekazywanie odpowiedzi do mowy (Streaming model output into speech)](https://aiatlasgraph.com/t/streamingaggregation.md): Model output is passed to speech generation before the whole reply is complete; chunk boundaries affect latency and naturalness.
      - [Wyprzedzające generowanie odpowiedzi (Preemptive response generation)](https://aiatlasgraph.com/t/preemptivegen.md): The agent starts model work before the user's turn is confirmed complete; continued user speech may require discarding or revising the response.
    - [Prowadzenie tur rozmowy (Turn-taking in voice agents)](https://aiatlasgraph.com/t/turndetection.md): Decisions about when to listen, start responding and allow simultaneous listening and speaking. Silence does not necessarily mean a thought is complete.
      - [Detekcja aktywności głosowej (VAD) (Voice activity detection)](https://aiatlasgraph.com/t/vad.md): Detecting speech segments in an audio stream. VAD can trigger downstream stages but does not itself determine whether the user has finished a turn.
      - [Dialog pełnodupleksowy (Full-duplex spoken dialogue)](https://aiatlasgraph.com/t/fullduplex.md): A system can receive speech and generate a response at the same time, handling overlap and listener backchannels. This can be achieved with an integrated model or by synchronizing modules.
      - [Kontekstowe wykrywanie tury (Context-aware turn detection)](https://aiatlasgraph.com/t/semanticturn.md): Estimating whether the user has completed a thought from semantic and/or acoustic cues. Implementations may use audio, a transcript or both.
    - [Przerwania i powrót do rozmowy (Interruption handling and recovery)](https://aiatlasgraph.com/t/bargein.md): Determining whether the user is interrupting, stopping the response and resuming the dialogue and task in line with what was already heard.
      - [Rozróżnianie przerwania od potakiwania i tła (Distinguishing interruptions from backchannels and background speech)](https://aiatlasgraph.com/t/spuriousinterrupt.md): After detecting audio, deciding whether the user wants to take the turn, is backchanneling or coughing, or another person spoke in the background.
      - [Usuwanie niesłyszanej części odpowiedzi z kontekstu (Aligning conversation context with played speech)](https://aiatlasgraph.com/t/heardcontext.md): After an interruption, remove generated speech the user did not hear from conversation history. The boundary depends on playback tracking and the API.
      - [Wznowienie zadania po przerwaniu (Task recovery after a voice interruption)](https://aiatlasgraph.com/t/interruptionrecovery.md): After the user interrupts, the agent handles the new request and returns to the right task step without repeating speech already heard.
    - [Rozumienie wypowiedzi w dialogu (Understanding spoken input in voice agents)](https://aiatlasgraph.com/t/voiceaudio.md): Interpreting who is speaking and what they said, including uncertainty and exact data capture. ASR itself is a separate topic in the map.
      - [Dopytywanie przy niepewnej interpretacji mowy (Clarification under spoken-input uncertainty)](https://aiatlasgraph.com/t/inputclarification.md): When speech admits multiple interpretations, the agent asks a short clarifying question rather than guessing or taking action.
      - [Przechwytywanie i potwierdzanie dokładnych danych (Exact entity capture from speech)](https://aiatlasgraph.com/t/entitycapture.md): The agent captures spoken names, numbers and codes, clarifies ambiguous characters and reads back exact values for confirmation before using a tool.
      - [Rozpoznawanie właściwego rozmówcy (Target-speaker attribution in voice agents)](https://aiatlasgraph.com/t/speakerisolation.md): Distinguishing the user's speech from other speakers so background voices do not become commands or interruptions.
    - [Zadania i narzędzia w rozmowie (Task and tool orchestration in spoken dialogue)](https://aiatlasgraph.com/t/voicetools.md): Using tools and completing multi-step tasks while keeping the conversation coherent. Calls may block or run in the background, depending on the API and agent design.
      - [Komunikowanie pracy narzędzia (Spoken progress cues during tool use)](https://aiatlasgraph.com/t/toollatencymask.md): The agent indicates it is working or waiting for a result while keeping its speech aligned with the actual operation state.
      - [Narzędzia działające w tle rozmowy (Non-blocking tools in spoken dialogue)](https://aiatlasgraph.com/t/asyncfunctions.md): The agent can keep conversing while a long-running task executes, then incorporate its result. Semantics and support vary across APIs.
      - [Potwierdzanie działań wykonywanych głosem (Spoken confirmation before consequential actions)](https://aiatlasgraph.com/t/spokenconfirm.md): Before an action that changes data, the agent speaks the intended action and consequence, waits for explicit confirmation and reports success only after the tool succeeds.
      - [Scenariusze i stan rozmowy (Stateful spoken workflows)](https://aiatlasgraph.com/t/voiceflows.md): The model conducts a multi-step conversation; changing the step affects instructions, available tools and transferred context.
  - [Automatyczny zapis nut z nagrania (Automatic music transcription)](https://aiatlasgraph.com/t/musictranscription.md): This identifies the pitch and duration of notes heard in a recording.
  - [Diaryzacja: kto kiedy mówi (Speaker diarization)](https://aiatlasgraph.com/t/speakerdiarization.md): This labels which parts of a conversation are spoken by the same person.
  - [Dopasowanie tekstu do czasu nagrania (Forced alignment of speech and text)](https://aiatlasgraph.com/t/forcedalignment.md): This determines when words from a supplied text occur in a recording.
  - [Generowanie muzyki i efektów dźwiękowych (Music and sound generation)](https://aiatlasgraph.com/t/audiogen.md): Models that create music, sound effects or ambience from a text prompt (sometimes with a melody or lyrics), usually as language models over discrete audio tokens or as latent diffusion.
  - [Oczyszczanie i poprawa jakości mowy (Speech enhancement)](https://aiatlasgraph.com/t/speechenhancement.md): This reduces interference to make a recorded voice clearer.
  - [Rozdzielanie wokalu i instrumentów (Music source separation)](https://aiatlasgraph.com/t/musicseparation.md): This splits a finished music recording into separate sound parts.
  - [Rozpoznawanie dźwięków i klasyfikacja nagrań (Audio and sound event classification)](https://aiatlasgraph.com/t/audioclassification.md): Assigning one or more labels that describe a recording, such as sound type, music genre, spoken command or language. Models often analyse a spectrogram or a learned audio representation; shared audio-text spaces also enable zero-shot classification.
  - [Rozpoznawanie mowy / ASR (Automatic speech recognition)](https://aiatlasgraph.com/t/asr.md): Turning speech into text.
  - [Synteza mowy / TTS (Text-to-speech)](https://aiatlasgraph.com/t/tts.md): Producing speech from text.
- [Widzenie komputerowe (Computer vision)](https://aiatlasgraph.com/t/visionnode.md): Recognising content and structure in images and video: what is in the picture, where it is, what shape it has and how it moves. Image generation, 3D and vision–language models have their own sibling nodes.
  - [Detekcja obiektów (Object detection)](https://aiatlasgraph.com/t/objectdetection.md): Finding objects in an image: a bounding box plus a class for each object. Usually evaluated with IoU and mAP.
    - [Detekcja obiektów opisanych tekstem (Open-vocabulary object detection)](https://aiatlasgraph.com/t/openvocabdetection.md): Detecting objects queried by names or natural-language descriptions rather than one fixed closed label set; includes zero-shot generalisation to new categories.
    - [Detekcja z transformerem (DETR) (Transformer-based object detection (DETR))](https://aiatlasgraph.com/t/detrdetectors.md): Detection as direct set prediction by a transformer, without hand-designed anchors or NMS post-processing.
    - [Detektory jednoetapowe (YOLO) (Single-stage object detectors)](https://aiatlasgraph.com/t/onestagedetectors.md): The model predicts boxes and classes without a separate region-proposal stage. The YOLO family shows that such detectors can run in real time and on edge devices.
  - [Estymacja pozy i punktów kluczowych (Pose and keypoint estimation)](https://aiatlasgraph.com/t/poseestimation.md): Detecting keypoints (joints, hands, face) and body skeletons in 2D or 3D — the basis of motion analysis, sports and gesture control.
  - [Klasyfikacja obrazów (Image classification)](https://aiatlasgraph.com/t/imageclassification.md): Assigning one or more labels to a whole image (e.g. “cat”, “defective product”). It is one of the foundational computer-vision tasks.
  - [Modele bazowe dla obrazu (Vision backbones and foundation models)](https://aiatlasgraph.com/t/visionbackbones.md): Pretrained vision models that extract image features reusable for classification, detection and segmentation.
    - [Dostrajanie modeli wizyjnych do własnych danych (Fine-tuning vision models on custom data)](https://aiatlasgraph.com/t/visiontransfer.md): Adapting a pretrained model to a custom dataset by fine-tuning the whole network, selected layers, or a new task head.
    - [Samonadzorowane uczenie z obrazów (Self-supervised visual pretraining)](https://aiatlasgraph.com/t/visionssl.md): Training a vision model on unlabelled images — e.g. by reconstructing masked patches or comparing different views of the same image.
    - [Vision Transformer (ViT)](https://aiatlasgraph.com/t/vit.md): A transformer applied to an image split into patches treated as tokens; a major family of vision architectures alongside convolutional networks.
    - [Wspólna przestrzeń obraz–tekst (CLIP) (Contrastive image–text pretraining (CLIP))](https://aiatlasgraph.com/t/clipcontrastive.md): Learning joint embeddings of images and captions from image–text pairs; enables zero-shot classification and image search by text description.
  - [Powiększanie i naprawa zdjęć (super-rozdzielczość) (Image upscaling and restoration (super-resolution))](https://aiatlasgraph.com/t/imagerestoration.md): Recovering a clean, higher-resolution image from a degraded one (low resolution, noise, blur, compression). Approaches range from GAN models trained on synthetic degradations to diffusion-based restoration with a generative prior, which can add plausible but not necessarily true detail.
  - [Rozpoznawanie tekstu na obrazach (OCR) (Optical character recognition (OCR))](https://aiatlasgraph.com/t/ocr.md): Converting images of print, handwriting or scene text into machine text; usually line or word detection plus recognition.
  - [Rozumienie wideo (Video understanding)](https://aiatlasgraph.com/t/videounderstanding.md): Analysing frame sequences: adds time, motion and far more data than a single image.
    - [Rozpoznawanie akcji w wideo (Action recognition)](https://aiatlasgraph.com/t/actionrecognition.md): Recognising what happens in a clip (running, falling, passing a ball) from appearance and motion over time.
    - [Śledzenie obiektów w wideo (Multi-object tracking)](https://aiatlasgraph.com/t/objecttracking.md): Keeping the identity of the same objects across frames, usually by linking detector outputs frame to frame (tracking-by-detection).
  - [Segmentacja obrazu (Image segmentation)](https://aiatlasgraph.com/t/segmentation.md): Dividing an image into regions at pixel level — masks instead of boxes.
    - [Segmentacja instancji i panoptyczna (Instance and panoptic segmentation)](https://aiatlasgraph.com/t/instanceseg.md): A separate mask for each object (instances); the panoptic variant unifies objects and background in one consistent map.
    - [Segmentacja na wskazanie (Segment Anything) (Promptable segmentation)](https://aiatlasgraph.com/t/promptseg.md): A general model returns a mask for an object indicated by a click, a box or a short text phrase — no class-specific training; newer versions also work on video.
    - [Segmentacja semantyczna (Semantic segmentation)](https://aiatlasgraph.com/t/semanticseg.md): Every pixel gets a class label (road, sky, person) without separating individual instances.

## Sources

- [Hugging Face · Audio Course](<https://huggingface.co/learn/audio-course/en/chapter0/introduction>) `audio` · reference: Dane audio, klasyfikacja, ASR, TTS i modele mowy.
- [Stanford CS231n · Computer Vision (2026)](<https://cs231n.stanford.edu/>) `vision` · reference: Sieci neuronowe i rozumienie obrazów.
- [Stanford CS336 · Language Modeling from Scratch (2025)](<https://cs336.stanford.edu/spring2025/>) `cs336` · reference: Budowa modeli, dane, trening, systemy, inferencja i post-training.

---

AI Atlas revision 122 (2026-10-30) · HTML: https://aiatlasgraph.com/t/modalities · index: https://aiatlasgraph.com/llms.txt · everything: https://aiatlasgraph.com/llms-full.txt · updated 2026-10-30