// MODEL OPTIMIZATION AND PROMPT SYNTAX TERM
Edge Deployment
Edge deployment involves running AI models directly on devices closer to where the data is generated, like smartphones or IoT devices, instead of sending all data to a central cloud.
TECHNICAL DEFINITION
Edge deployment positions AI models and inference engines directly on local devices (e.g., IoT sensors, mobile phones, embedded systems) at the "edge" of the network, minimizing data transfer latency, enhancing privacy, and enabling offline operation by reducing reliance on centralized cloud infrastructure.
BACKGROUND
Generative artificial intelligence (GenAI) is a subfield of artificial intelligence (AI) that uses generative models to generate text, images, videos, audio, software code or other forms of data. These models learn the underlying patterns and structures of their training data, and use them to generate new data in response to input, which often takes the form of natural language prompts.
READ MORE ON WIKIPEDIASYNONYMS & ALIASES
- On-device AI
- local AI
- distributed AI (at the edge)
USAGE NOTE
Edge deployment is vital for applications requiring low latency, privacy, or operation in environments with limited connectivity.
DEVELOPERS
Organizations developing technology related to Edge Deployment.
Develops the Jetson platform, a series of embedded computing boards, and the TensorRT SDK for optimizing and deploying high-performance AI models on edge devices.
Designs Snapdragon processors with dedicated AI Engines. They provide the Qualcomm AI Stack for developers to optimize and deploy machine learning models directly on devices like smartphones and IoT hardware.
Offers the Coral platform, featuring Edge TPU hardware accelerators for running fast ML inference on-device. They also develop TensorFlow Lite, a framework for deploying models on mobile and embedded devices.
Provides hardware like Movidius Vision Processing Units (VPUs) and the OpenVINO toolkit, a software solution for optimizing and deploying deep learning models on Intel hardware at the edge.
Designs the architecture for most edge processors and provides specific IP like Ethos Neural Processing Units (NPUs) and software libraries (Arm NN) to accelerate machine learning workloads on low-power devices.
A leading development platform for machine learning on edge devices, enabling developers to create and deploy models on microcontrollers and other resource-constrained hardware.
Develops custom silicon (A-series, M-series) with an integrated Neural Engine for fast on-device ML processing. They provide the Core ML framework for deploying models within the Apple ecosystem.
Offers Azure IoT Edge, a service for deploying and managing cloud workloads, including AI models packaged as containers, to run directly on IoT devices.
Provides services like AWS IoT Greengrass and Amazon SageMaker Neo, which help developers to build, train, optimize, and deploy machine learning models on edge devices.