Bezpieczeństwo i governance

AI safety, security and governance · id safety

Ryzyko, zabezpieczenia i odpowiedzialność za system AI.

Risk, safeguards and responsibility for an AI system.

In one sentence

To ograniczanie szkód powodowanych przez modele oraz ustalanie, kto odpowiada za ich działanie.

This limits harm from models and establishes who is responsible for their operation.

Problem it solves

Model może ujawnić dane, potraktować ludzi nierówno lub wykonać szkodliwe działanie, mimo że zwykle odpowiada poprawnie.

A model can reveal data, treat people unequally or perform harmful actions even when its responses are usually correct.

What it does

Łączy ocenę ryzyka, trening, ochronę danych, ograniczenia działania i nadzór; skuteczność sprawdza się również po udostępnieniu systemu.

It combines risk assessment, training, data protection, action limits and oversight; effectiveness is also checked after a system is released.

Everyday example

Przed uruchomieniem asystenta sprawdzasz ujawnianie danych, dostęp do narzędzi i sposób zgłaszania błędów przez użytkowników.

Before launching an assistant, you check data exposure, tool access and how users can report mistakes.

This concerns you if you…

udostępniasz model innym osobom
jego odpowiedzi lub działania mogą komuś zaszkodzić
make a model available to other people
use outputs or actions that could harm someone

Examples

Alignment · red teaming · prompt injection · prywatność · AI Act
Alignment · red teaming · prompt injection · privacy · AI Act

Subtopics

Alignment i safety (Alignment and safety): Aligning model behaviour with intended goals and limits: how to train it, how to oversee models stronger than humans, and how to detect a model gaming its objective or hiding its intent.
- Obchodzenie celu: reward hacking i służalczość (Reward hacking, specification gaming and sycophancy): The model satisfies the literal training objective but not its intent: it exploits loopholes in the reward (e.g. gaming tests) or tells users what they want to hear instead of the truth. Detecting and preventing this is a practical problem in RL training today.
  - Lokalizacja niepożądanych zachowań przez routowanie gradientów (Gradient-routed localization of unintended behavior): Localizes reward hacking in a removable adapter by routing gradients.
- Postępujące rozminięcie z celami (Progressive misalignment): An observed phenomenon in which a model increasingly answers harmful queries across rounds of recursive self-distillation despite benign extra information for the teacher.
- Skalowalny nadzór (debata, weak-to-strong) (Scalable oversight): Methods for supervising models that may outperform humans on a task: humans assisted by an AI, a debate between two models judged by a human, a weaker model supervising a stronger one.
- Trening według zasad (Constitutional AI, RLAIF) (Constitutional AI and RL from AI feedback): Training a model to be harmless from a written list of principles (a “constitution”): the model critiques and revises its own answers, and preference labels come from an AI instead of humans (RLAIF).
- Udawane dopasowanie i ukryte cele (alignment faking, scheming) (Alignment faking and scheming): Studying whether a model can strategically hide its goals: behaving well only when it believes it is being trained or tested, deliberately underperforming (sandbagging) or evading oversight. An evaluation research area since 2024.
Bezpieczeństwo aplikacji AI (AI application security): Protecting data, tools and execution.
- Ataki przez prompt (Prompt attacks): Hijacking model behaviour through input content: injected instructions, leaking the prompt or data, bypassing safeguards.
  - Jailbreaki — obchodzenie zabezpieczeń modelu (Jailbreaking): Prompts that get a model to break its own safety rules: role-play, obfuscated requests, low-resource languages, automatically searched adversarial suffixes. They work because the goals “be helpful” and “be safe” compete.
  - Pośrednie wstrzykiwanie promptu (przez dokumenty i narzędzia) (Indirect prompt injection): The attacker never talks to the model — they hide instructions in a web page, email, file or tool result that the LLM application will fetch itself. Especially dangerous for RAG and agents, as it can lead to data theft or unwanted actions.
- Bezpieczeństwo agentów: uprawnienia i izolacja (AI agent security): Designing agents so that a compromised model cannot cause harm: least privilege, sandboxed code execution, separating trusted instructions from untrusted data, confirmations before risky actions.
  - Mechanistyczne sterowanie bezpieczeństwem agentów CLI (Mechanistic subspace safety steering for multi-turn agents): An experimental white-box method detects harmful execution states of multi-turn CLI agents from hidden representations and intervenes in ten selected dimensions of a single layer.
  - Monitor interwencji krokowej dla agentów (Step-level timely intervention monitors for agents): A step-level reasoning guardrail that chooses CONTINUE, ASK or STOP before each agent action and is trained with Adaptive Stage-Aligned Policy Optimization (ASPO).
- Bezpieczeństwo łańcucha dostaw modeli (AI model supply chain security): Risks of pulling in third-party weights, adapters, datasets and libraries: malicious code in model files, hidden backdoors, unknown provenance. Practice: safe weight formats, source verification, scanning.
- Zatruwanie danych i backdoory (Data poisoning and backdoors): Attacking the training data: false information or a hidden trigger that changes the model's behaviour.
  - Podgląd aktualizacji dla bezpiecznego dostrajania (Update-preview defense for safe fine-tuning): Previews and penalizes fine-tuning updates that weaken safe behaviour from a buffer.
Ewaluacje bezpieczeństwa modeli (AI safety evaluations): Deliberately probing a model for harmful behaviour and dangerous capabilities before and after deployment — by people, other models and standardised tests.
- Ewaluacje niebezpiecznych zdolności (cyber, bio, autonomia) (Dangerous capability evaluations): Tests of whether a model can meaningfully help with dangerous activities — cyberattacks, biological or chemical weapons, persuasion, self-proliferation or automating AI research. Results trigger thresholds in companies' safety policies.
- Red teaming modeli (ręczny i automatyczny) (Red teaming language models): Attacking your own model to find harmful outputs before users do: manually by tester teams, or automatically with one model generating attacks against another.
- Samocena rozmowy pod kątem nadużyć (Conversation self-assessment for misuse detection): After responding, the model generates a short safety assessment of its own conversation, reusing the already computed context.
- Sondy aktywacji do wykrywania niepokojących wzorców (Activation probes for detecting concerning model behavior): Lightweight classifiers analysing the model's internal activations for patterns associated with undesirable behaviour.
- Wielowarstwowe monitorowanie agentów (Multi-layer monitoring for agent misalignment): A cascade of monitors from cheap activation probes, through model reviewers, to investigative agents analysing full transcripts and logs.
Governance i zarządzanie ryzykiem (Governance and risk management): Accountability, policies, documentation and incidents.
- Dokumentacja modeli i danych (karty modeli) (Model and dataset documentation): Short, standardised documents shipped with a model or dataset: intended use, limitations, evaluation procedure, results across groups, data origin and composition.
- Polityki bezpieczeństwa modeli granicznych (Frontier AI safety policies): Public commitments by companies building the most capable models: which dangerous-capability thresholds they test for, which safeguards then apply, and when they would pause deployment or development.
- Ramy zarządzania ryzykiem AI (NIST AI RMF, ISO/IEC 42001) (AI risk management frameworks): Voluntary frameworks and standards for organising AI risk management: who is accountable, how to map and measure risks, how to respond. ISO/IEC 42001 specifies an organisational AI management system.
- Regulacje AI i klasy ryzyka (AI Act) (AI regulation and risk tiers (EU AI Act)): Law that sorts AI systems by risk level — prohibited practices, high risk, transparency obligations, minimal risk — with separate duties for general-purpose model providers. In the EU the rules phase in over 2025–2028.
Guardrails (Guardrails and output control): Blocks and filters on the model's input and output.
- Klasyfikatory bezpieczeństwa wejścia i wyjścia (Input and output safety classifiers): Separate, usually smaller models that score the prompt and the response against a harm taxonomy and block what is not allowed. They can be adapted to a custom category list.
- Programowalne reguły przepływu rozmowy (rails) (Programmable dialogue rails): Rules plugged into different stages of an LLM application — input, dialogue, retrieval, tool execution and output — that keep the conversation on topic and on script.
Prywatność i prawa do danych (Privacy and data rights): Rules for using data and models.
- Oduczanie modeli (machine unlearning) (Machine unlearning): Removing the influence of selected data or capabilities (e.g. personal data, copyrighted content) from an already-trained model without retraining from scratch. Still an open problem: it is hard to verify the model truly “forgot”.
- Prywatność różnicowa (DP-SGD) (Differential privacy in machine learning): A mathematical guarantee that the training result barely depends on whether any single person is in the data. In practice: gradient clipping and added noise (DP-SGD) under a controlled “privacy budget” — at a cost to model quality.
- Uczenie federacyjne (Federated learning): Collaborative model training without centralising raw data: data stays on participants’ devices or servers, which send updates for aggregation. Federated learning alone does not guarantee privacy — updates can leak information, so secure aggregation and differential privacy are among the additional protections used.
- Wykrywanie i anonimizacja danych osobowych (PII) (PII detection and anonymisation): Automatically finding personal data (names, numbers, addresses) in text, images and tables and masking or replacing it — before training, in logs and in prompts sent to models. No tool catches everything.
Stronniczość i sprawiedliwość (Bias and fairness): Effects of a system behaving differently toward different groups.
- Metody ograniczania stronniczości (Bias mitigation techniques): Reducing disparities at three stages: before training (transforming data), during training (fairness-constrained optimisation) and after training (e.g. group-specific decision thresholds).
- Metryki sprawiedliwości grupowej (Group fairness metrics): Comparing model behaviour across groups defined by sensitive features: are positive decisions and errors distributed similarly? Different metrics conflict with each other, and a near-zero gap does not by itself mean the result is fair.
Znaki wodne i pochodzenie treści AI (AI content watermarking and provenance): Techniques that signal content origin: invisible watermarks embedded during generation and cryptographically signed metadata manifests. Their limitations differ: editing and paraphrasing can weaken watermarks, while C2PA metadata can be detached from the file.

Sources

NIST · AI Risk Management Framework · reference: Uzupełnienie kursów o ryzyko i zarządzanie AI; nie porada prawna.
Full Stack Deep Learning (2022, archiwum) · reference: Pełny cykl systemu ML: dane, testy, wdrażanie, monitoring, zespoły i etyka.
International AI Safety Report 2026 · Panel ponad 100 ekspertów pod przewodnictwem Yoshuy Bengio; Expert Advisory Panel z przedstawicielami 30+ państw i organizacji