# Agenci głosowi EN: **Real-time voice agents** · id `voiceagents` Path: [AI Atlas](https://aiatlasgraph.com/llms.txt) › [Język, obraz, audio](https://aiatlasgraph.com/t/modalities.md) › [Mowa i audio](https://aiatlasgraph.com/t/audio.md) › Agenci głosowi ## Description - PL: System prowadzący rozmowę głosową, który rozumie wypowiedzi, podejmuje decyzje i wykonuje zadania z pomocą modeli oraz narzędzi. - EN: A system that conducts spoken conversations, understands utterances, makes decisions and completes tasks using models and tools. ## In one sentence - PL: To programy, z którymi rozmawiasz na głos, żeby uzyskać pomoc lub wykonać zadanie. - EN: These programs hold spoken conversations to help you or complete a task. ## Problem it solves - PL: Pisanie i klikanie przeszkadza, gdy masz zajęte ręce albo łatwiej wyjaśnisz sprawę w rozmowie. - EN: Typing and clicking get in the way when your hands are busy or conversation makes the issue easier to explain. ## What it does - PL: System łączy rozumienie mowy, pamięć rozmowy, wybór działań i odpowiedzi głosowe, pilnując przebiegu zadania. - EN: A system combines speech understanding, conversation memory, action selection and spoken responses while tracking the task. ## Everyday example - PL: Dzwonisz do punktu napraw, opisujesz problem i z pomocą głosowego asystenta umawiasz odbiór sprzętu. - EN: You call a repair shop, describe the problem and arrange equipment collection with a voice assistant. ## This concerns you if you… - PL: chcesz załatwiać sprawy przez rozmowę - PL: dodajesz głos do istniejącego narzędzia - EN: want to get things done through conversation - EN: add voice interaction to an existing tool ## Keywords LiveKit Agents · Pipecat · WebRTC · streaming ASR · streaming TTS · barge-in · turn detection · dialogue state tracking · tool calling · audible response latency ## How it works - PL: Strumień audio zasila interpretację wypowiedzi, a stan dialogu łączy ją z decyzjami i wynikami narzędzi. Harmonogram tur oraz generowanie mowy synchronizują odpowiedź z dalszym wejściem użytkownika i rzeczywistym stanem zadania. - EN: An audio stream feeds utterance interpretation, while dialogue state connects it with decisions and tool results. Turn scheduling and speech generation synchronise responses with further user input and actual task state. ## Limits - PL: Błędy słyszenia, czasu tur i narzędzi mogą się kumulować. Krótki czas reakcji nie zapewnia poprawnego zadania, a przerwania wymagają uzgodnienia historii z wykonanymi działaniami. - EN: Hearing, turn-timing and tool errors can compound. Fast responses do not ensure correct task completion, while interruptions require reconciling history with executed actions. ## Key terms - PL: **stan dialogu**: Zapis informacji, oczekujących działań i etapu bieżącej rozmowy. - PL: **koordynator rozmowy**: Komponent koordynujący audio, generowanie, narzędzia i zdarzenia dialogu. - EN: **dialogue state**: A record of information, pending actions and the current conversation stage. - EN: **conversation orchestrator**: A component coordinating audio, generation, tools and dialogue events. ## Know first - [Rozpoznawanie mowy / ASR (Automatic speech recognition)](https://aiatlasgraph.com/t/asr.md) - [Narzędzia i integracje (Tool use and integration)](https://aiatlasgraph.com/t/tools.md) ## Examples - PL: telefon · nakładka głosowa · głos jako narzędzie - EN: phone · voice layer · voice as a tool ## Subtopics (7 direct, 29 in total) - [Architektury agentów głosowych (Voice agent architectures)](https://aiatlasgraph.com/t/voicearch.md): Ways to combine speech understanding, agent decisions and voice generation: modular, integrated or with delegated reasoning. - [Głosowy front i osobny agent zadaniowy (Voice frontend with delegated reasoning)](https://aiatlasgraph.com/t/delegatedvoice.md): A voice model conducts the live conversation while delegating reasoning and tools to a separate text agent; the result returns to the spoken dialogue. - [Modele mowa–mowa (Speech-to-speech models)](https://aiatlasgraph.com/t/speech2speech.md): A model takes speech and generates speech without separate STT and TTS services in the pipeline. Its internal representation may still include text or transcription; tool use and context control depend on the model. - [Potok STT → agent tekstowy → TTS (Cascaded STT–text agent–TTS pipeline)](https://aiatlasgraph.com/t/cascadedpipeline.md): Separate speech recognition, a text agent and speech synthesis. Stages can be swapped, inspected and connected to an existing workflow. - [Ewaluacja agentów głosowych (Voice agent evaluation)](https://aiatlasgraph.com/t/voiceevals.md): Evaluating task success and conversational quality on real audio: tools, turns, interruptions, audible latency and robustness across speaking conditions. - [Ocena tur, przerwań i słyszalnego czasu reakcji (Turn-taking, interruption and audible-latency evaluation)](https://aiatlasgraph.com/t/turnquality.md): Tests measure premature takeovers, unwanted silence, overlap, post-interruption behavior and time to a useful audible response. - [Odporność na akcent, hałas i cudzą mowę (Robustness to accents, noise and third-party speech)](https://aiatlasgraph.com/t/audiorobustness.md): The same tasks are tested with varied accents, noise and other speakers, tracking understanding, turns and task outcomes separately. - [Symulacje wieloturowych rozmów audio (Multi-turn audio conversation simulation)](https://aiatlasgraph.com/t/audiosim.md): A simulated caller speaks with the agent through a full scenario, including pauses, corrections and interruptions; both dialogue and task state are evaluated. - [Weryfikacja wyniku zadania i narzędzi (Voice-agent task and tool outcome verification)](https://aiatlasgraph.com/t/taskoutcome.md): Evaluation compares spoken user intent, tool calls, final system state and the agent's spoken confirmation. - [Generowanie odpowiedzi w czasie rzeczywistym (Real-time spoken response generation)](https://aiatlasgraph.com/t/voicelatency.md): Preparing agent speech so a useful response begins promptly while the model remains responsive to further user speech. - [Strumieniowe przekazywanie odpowiedzi do mowy (Streaming model output into speech)](https://aiatlasgraph.com/t/streamingaggregation.md): Model output is passed to speech generation before the whole reply is complete; chunk boundaries affect latency and naturalness. - [Wyprzedzające generowanie odpowiedzi (Preemptive response generation)](https://aiatlasgraph.com/t/preemptivegen.md): The agent starts model work before the user's turn is confirmed complete; continued user speech may require discarding or revising the response. - [Prowadzenie tur rozmowy (Turn-taking in voice agents)](https://aiatlasgraph.com/t/turndetection.md): Decisions about when to listen, start responding and allow simultaneous listening and speaking. Silence does not necessarily mean a thought is complete. - [Detekcja aktywności głosowej (VAD) (Voice activity detection)](https://aiatlasgraph.com/t/vad.md): Detecting speech segments in an audio stream. VAD can trigger downstream stages but does not itself determine whether the user has finished a turn. - [Dialog pełnodupleksowy (Full-duplex spoken dialogue)](https://aiatlasgraph.com/t/fullduplex.md): A system can receive speech and generate a response at the same time, handling overlap and listener backchannels. This can be achieved with an integrated model or by synchronizing modules. - [Kontekstowe wykrywanie tury (Context-aware turn detection)](https://aiatlasgraph.com/t/semanticturn.md): Estimating whether the user has completed a thought from semantic and/or acoustic cues. Implementations may use audio, a transcript or both. - [Przerwania i powrót do rozmowy (Interruption handling and recovery)](https://aiatlasgraph.com/t/bargein.md): Determining whether the user is interrupting, stopping the response and resuming the dialogue and task in line with what was already heard. - [Rozróżnianie przerwania od potakiwania i tła (Distinguishing interruptions from backchannels and background speech)](https://aiatlasgraph.com/t/spuriousinterrupt.md): After detecting audio, deciding whether the user wants to take the turn, is backchanneling or coughing, or another person spoke in the background. - [Usuwanie niesłyszanej części odpowiedzi z kontekstu (Aligning conversation context with played speech)](https://aiatlasgraph.com/t/heardcontext.md): After an interruption, remove generated speech the user did not hear from conversation history. The boundary depends on playback tracking and the API. - [Wznowienie zadania po przerwaniu (Task recovery after a voice interruption)](https://aiatlasgraph.com/t/interruptionrecovery.md): After the user interrupts, the agent handles the new request and returns to the right task step without repeating speech already heard. - [Rozumienie wypowiedzi w dialogu (Understanding spoken input in voice agents)](https://aiatlasgraph.com/t/voiceaudio.md): Interpreting who is speaking and what they said, including uncertainty and exact data capture. ASR itself is a separate topic in the map. - [Dopytywanie przy niepewnej interpretacji mowy (Clarification under spoken-input uncertainty)](https://aiatlasgraph.com/t/inputclarification.md): When speech admits multiple interpretations, the agent asks a short clarifying question rather than guessing or taking action. - [Przechwytywanie i potwierdzanie dokładnych danych (Exact entity capture from speech)](https://aiatlasgraph.com/t/entitycapture.md): The agent captures spoken names, numbers and codes, clarifies ambiguous characters and reads back exact values for confirmation before using a tool. - [Rozpoznawanie właściwego rozmówcy (Target-speaker attribution in voice agents)](https://aiatlasgraph.com/t/speakerisolation.md): Distinguishing the user's speech from other speakers so background voices do not become commands or interruptions. - [Zadania i narzędzia w rozmowie (Task and tool orchestration in spoken dialogue)](https://aiatlasgraph.com/t/voicetools.md): Using tools and completing multi-step tasks while keeping the conversation coherent. Calls may block or run in the background, depending on the API and agent design. - [Komunikowanie pracy narzędzia (Spoken progress cues during tool use)](https://aiatlasgraph.com/t/toollatencymask.md): The agent indicates it is working or waiting for a result while keeping its speech aligned with the actual operation state. - [Narzędzia działające w tle rozmowy (Non-blocking tools in spoken dialogue)](https://aiatlasgraph.com/t/asyncfunctions.md): The agent can keep conversing while a long-running task executes, then incorporate its result. Semantics and support vary across APIs. - [Potwierdzanie działań wykonywanych głosem (Spoken confirmation before consequential actions)](https://aiatlasgraph.com/t/spokenconfirm.md): Before an action that changes data, the agent speaks the intended action and consequence, waits for explicit confirmation and reports success only after the tool succeeds. - [Scenariusze i stan rozmowy (Stateful spoken workflows)](https://aiatlasgraph.com/t/voiceflows.md): The model conducts a multi-step conversation; changing the step affects instructions, available tools and transferred context. ## Related topics Untyped, symmetric "see also" links. - [Agenci i orkiestracja (Agents and orchestration)](https://aiatlasgraph.com/t/agentsnode.md) ## Courses - [Voice for AI Agents and Applications]() ## Sources - [DeepLearning.AI — course catalogue]() `dlai` · course catalogue: 42 topic labels with counts, about 130 courses. - [Voice AI & Voice Agents: An Illustrated Primer]() `voiceprimer` · by Kwindla Hultman Kramer i współautorzy (Daily / Pipecat) - [OpenAI API: Voice agents]() `openai-voice-guide` · by OpenAI --- AI Atlas revision 122 (2026-10-30) · HTML: https://aiatlasgraph.com/t/voiceagents · index: https://aiatlasgraph.com/llms.txt · everything: https://aiatlasgraph.com/llms-full.txt · updated 2026-10-30