// MODEL OPTIMIZATION AND PROMPT SYNTAX TERM
X-Risk
A shorthand for "existential risk," referring to a potential event that could destroy or permanently cripple human civilization.
TECHNICAL DEFINITION
X-Risk is an abbreviation for existential risk, denoting a hypothetical event or scenario, such as unaligned superintelligent AI, that could cause the extinction of humanity or permanently and drastically reduce its potential for future development, representing the most severe category of global catastrophic risks.
BACKGROUND
Prompt injection is a cybersecurity exploit and an attack vector in which innocuous-looking inputs are designed to cause unintended behavior in machine learning models, particularly large language models (LLMs). The attack takes advantage of the model's inability to distinguish between developer-defined prompts and user inputs to bypass safeguards and influence model behaviour. While LLMs are designed to follow trusted instructions, they can be manipulated into carrying out unintended responses through carefully crafted inputs.
READ MORE ON WIKIPEDIASYNONYMS & ALIASES
- Existential Risk
- Global Catastrophe
- Species-Level Threat
- Catastrophic Risk
USAGE NOTE
Discussions about X-risk often involve scenarios where advanced AI systems lose control or act against human interests.
DEVELOPERS
Organizations developing technology related to X-Risk.
Developing advanced AI systems while actively researching and implementing methods for AI safety and alignment, including dedicated 'Superalignment' efforts to address potential catastrophic risks from future AI.
Conducts extensive research into AI safety, ethics, and responsible AI, aiming to develop robust and aligned AI systems and mitigate potential large-scale societal or existential risks.
Focused on large language model safety and alignment, pioneered 'Constitutional AI' to train models to be helpful, harmless, and honest, specifically addressing potential misuses and risks of powerful AI.
Dedicated to theoretical and mathematical research on AI alignment and safety to prevent potentially catastrophic outcomes from advanced artificial general intelligence (AGI).
A government-backed research and evaluation organization focused on ensuring the safe and responsible development of advanced AI, including testing frontier models for extreme risks.
A private research organization working on fundamental technical problems in AI alignment and safety, aiming to ensure that highly capable AI systems remain beneficial and controllable.
Focused on developing and deploying standards, tools, and tests for advanced AI models, with a key objective of identifying and mitigating severe risks, including those that could be existential.