// MODEL OPTIMIZATION AND PROMPT SYNTAX TERM
In-Context Learning
The ability of large language models to learn a new task or adapt their behavior by simply reading examples provided within the prompt, without any model updates.
TECHNICAL DEFINITION
In-context learning (ICL) is the phenomenon where large language models (LLMs) acquire new skills or adapt to tasks by processing examples presented directly within the input prompt, demonstrating rapid task acquisition without gradient updates.
BACKGROUND
Prompt engineering is the process of structuring natural language inputs to produce specified outputs from a generative artificial intelligence (GenAI) model. Context engineering is the related area of software engineering that focuses on the management of non-prompt contexts supplied to the GenAI model, such as metadata, API tools, and tokens.
READ MORE ON WIKIPEDIASYNONYMS & ALIASES
- Prompt-based learning
- Few-shot learning (broadly)
- Example-driven learning
USAGE NOTE
In-context learning is a core capability that makes few-shot and one-shot prompting effective.
DEVELOPERS
Organizations developing technology related to In-Context Learning.
OpenAI develops advanced large language models (LLMs) like GPT-3.5 and GPT-4, which are foundational to in-context learning. Their research and development efforts directly advance the capabilities and understanding of how models learn from examples and instructions provided within the prompt, a core aspect of AI engineering and prompt design.
Google's AI divisions, including Google AI and DeepMind, are at the forefront of developing large language models such as PaLM and Gemini. Their extensive research explores and enhances in-context learning abilities, few-shot prompting, and the broader field of prompt engineering to improve model performance and generalization.
Anthropic develops AI models like Claude, with a strong focus on helpful, harmless, and honest AI. Their work on large language models inherently involves deep research into in-context learning, prompt techniques, and constitutional AI principles to guide model behavior and reasoning within given contexts.
Meta AI conducts significant research in large language models, including the Llama family of models. Their work contributes to the advancement of in-context learning mechanisms, understanding model behavior with various prompting strategies, and developing techniques for more effective AI engineering.
Through its Azure AI services and extensive partnership with OpenAI, Microsoft enables and develops tooling for deploying and optimizing large language models. This includes robust support for prompt engineering, finetuning, and RAG (Retrieval Augmented Generation) architectures, all of which leverage and enhance in-context learning for enterprise applications.
Cohere specializes in enterprise-grade large language models and provides powerful tools for prompt engineering and RAG. Their platform is designed to help businesses leverage in-context learning effectively, allowing users to customize and ground LLMs with domain-specific information without extensive fine-tuning.
Hugging Face is a leading platform and community for machine learning, hosting a vast array of pre-trained models, including many large language models. They actively support and promote research into prompt engineering, in-context learning, and the development of tools (like Transformers library) that enable practitioners to experiment with and deploy models exhibiting these capabilities.
Adept AI is focused on building universal AI assistants that can perform actions by learning from human interactions and contextual information. Their core mission directly involves advancing in-context learning and reasoning to enable models to understand and execute complex tasks across various software applications.