// MODEL OPTIMIZATION AND PROMPT SYNTAX TERM
Evasion Attack
An attacker tries to make an AI model misclassify new, unseen data by subtly changing it.
TECHNICAL DEFINITION
An adversarial attack occurring at inference time, where an adversary crafts input samples (adversarial examples) to bypass or mislead a deployed AI model, causing it to make incorrect predictions without altering the model itself.
BACKGROUND
Adversarial machine learning is the study of the attacks on machine learning algorithms, and of the defenses against such attacks.
READ MORE ON WIKIPEDIASYNONYMS & ALIASES
- Inference-time attack
- adversarial evasion
- bypassing detection
USAGE NOTE
Evasion attacks are a major concern for AI systems deployed in security-critical applications like spam filters or malware detection.
DEVELOPERS
Organizations developing technology related to Evasion Attack.
Develops an AI Firewall and a continuous validation platform to protect AI models from adversarial attacks, including real-time detection and blocking of evasion attempts designed to cause model misclassification or bypass safety controls.
Provides a security platform specifically for machine learning models that detects and responds to adversarial attacks. Their technology monitors model inputs and outputs to identify evasion attacks that aim to fool the model at inference time.
A non-profit research organization that created MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems), a knowledge base of adversarial tactics and techniques. It explicitly documents various evasion attack methods used against machine learning systems.
Offers AI security and model validation solutions designed to test and protect large language models (LLMs) and other AI systems. Their platform identifies vulnerabilities to evasion attacks like prompt injection and jailbreaking before models are deployed.
As a leading developer of large language models, OpenAI conducts extensive internal and external red teaming to discover and mitigate evasion attacks, often termed 'jailbreaking.' Their safety and alignment research focuses on making models more robust against adversarial prompts.
An AI safety and research company that develops techniques like 'Constitutional AI' to make models less susceptible to evasion attacks that attempt to bypass their safety and ethical guidelines. Their research is heavily focused on preventing models from generating harmful outputs in response to adversarial inputs.
A company specializing in adversarial AI research and solutions. They provide services to test the security of AI applications, including penetration testing that uses sophisticated evasion techniques to assess the robustness of machine learning models.
Provides an AI security platform to evaluate, monitor, and secure large language models against risks. Their tools test for and mitigate various evasion attacks, including jailbreaking, data leakage, and prompt injection, ensuring models adhere to policy.