Traditional software security has a reasonably well-understood threat model. Attackers try to exploit bugs in code, gain unauthorized access to systems, or intercept data in transit. The defenses are mature: input validation, access controls, encryption, patching. These still apply to AI systems. But AI introduces an entirely different set of attack surfaces that conventional security frameworks weren't designed to address.
The most fundamental difference is that the model itself is an attack surface. In traditional software, the application logic is fixed code. In an AI system, the "logic" is encoded in billions of learned parameters — and those parameters can be manipulated. Data Poisoning attacks corrupt the training data before the model ever sees it, causing the model to learn incorrect associations or behaviors that persist into production. Backdoor Attacks are a specific variant: an attacker embeds a hidden trigger in the training data so that the model behaves normally in all cases except when it encounters a specific input pattern, at which point it does something the attacker intended.
At inference time, the attack surface shifts to the interface. Adversarial Attacks craft inputs that are imperceptible to humans but cause the model to produce wrong outputs with high confidence. Prompt injection — tested through Prompt Injection Testing — exploits the fact that instructions and data share the same channel in language models: malicious content in a document or data source can override the model's original instructions. Jailbreak Testing probes for ways to bypass safety guardrails through carefully constructed prompts.
The model itself can also be a target for theft or reverse engineering. Model Extraction Attacks reconstruct a functional copy of a proprietary model by querying it repeatedly and learning from its outputs. Model Inversion Attacks go further: they attempt to recover training data from the model's parameters, which is a serious concern when that training data includes sensitive information.
Red Teaming is the practice of systematically probing AI systems for all of these vulnerabilities before attackers find them. It borrows the adversarial mindset from traditional security and applies it to the AI-specific threat landscape.
The articles in this section cover each of these attack categories in detail. Together they build the threat vocabulary that effective AI security requires.
