// MODEL OPTIMIZATION AND PROMPT SYNTAX TERM
Misinformation
False or inaccurate information that is spread, regardless of intent to deceive.
TECHNICAL DEFINITION
Misinformation is factually incorrect or misleading information disseminated without malicious intent, often due to error or misunderstanding, which can be amplified by AI systems and pose risks to public discourse and decision-making.
BACKGROUND
Prompt injection is a cybersecurity exploit and an attack vector in which innocuous-looking inputs are designed to cause unintended behavior in machine learning models, particularly large language models (LLMs). The attack takes advantage of the model's inability to distinguish between developer-defined prompts and user inputs to bypass safeguards and influence model behaviour. While LLMs are designed to follow trusted instructions, they can be manipulated into carrying out unintended responses through carefully crafted inputs.
READ MORE ON WIKIPEDIASYNONYMS & ALIASES
- Falsehoods
- Inaccuracies
- Incorrect information
- Erroneous data
USAGE NOTE
AI models can inadvertently generate or spread misinformation if not properly trained and fact-checked.
DEVELOPERS
Organizations developing technology related to Misinformation.
Develops large language models and actively researches alignment and safety techniques to make AI systems more truthful and less likely to generate or spread misinformation. Their work includes training models to refuse inappropriate requests and improving factuality.
Jigsaw, a unit within Google, builds technology to address global security threats, including disinformation and online toxicity. They develop AI tools like the Perspective API to help platforms identify and moderate harmful content at scale.
Invests heavily in AI research and engineering to detect and mitigate the spread of misinformation across its platforms, including Facebook, Instagram, and Threads. Their systems use AI to identify fake accounts, manipulated media, and coordinated inauthentic behavior.
Through its research labs and AI for Good initiatives, Microsoft develops technology for media integrity. This includes tools for detecting deepfakes and manipulated content, as well as promoting standards for content provenance to help track the origin of digital media.
An AI-driven platform specializing in narrative and risk intelligence. Their technology is designed to detect and analyze disinformation campaigns and harmful online narratives in real-time, helping organizations understand and counter manipulation.
A technology company that combines advanced AI with one of the world's largest dedicated fact-checking teams to help governments and businesses identify and counter harmful misinformation and disinformation.
As a central hub for the machine learning community, Hugging Face hosts numerous open-source models and datasets specifically designed for tasks like fact-checking, stance detection, and misinformation classification, enabling widespread research and development in the field.
Develops technology that integrates human-powered credibility ratings of news sources directly into user workflows. Their browser extensions and data feeds use these ratings to warn users about unreliable sites known for publishing misinformation.
While not a tech company, the Poynter Institute's International Fact-Checking Network (IFCN) is a key organization that helps develop, standardize, and promote the use of technology, including AI tools, among fact-checking organizations globally to combat misinformation.