// MODEL OPTIMIZATION AND PROMPT SYNTAX TERM

Edge Deployment

Edge deployment involves running AI models directly on devices closer to where the data is generated, like smartphones or IoT devices, instead of sending all data to a central cloud.

Edge Deployment — illustration from Wikipedia
Image via Wikipedia

TECHNICAL DEFINITION

Edge deployment positions AI models and inference engines directly on local devices (e.g., IoT sensors, mobile phones, embedded systems) at the "edge" of the network, minimizing data transfer latency, enhancing privacy, and enabling offline operation by reducing reliance on centralized cloud infrastructure.

BACKGROUND

Grok is a generative artificial intelligence chatbot developed by xAI. It was launched in November 2023 by Elon Musk as an initiative based on the large language model (LLM) of the same name. Grok has apps for iOS and Android and is integrated with the X social network and Tesla's Optimus robot. The chatbot is named after the verb to grok, created by the American science fiction author Robert A. Heinlein to convey a form of deep, intuitive understanding.

READ MORE ON WIKIPEDIA

SYNONYMS & ALIASES

  • On-device AI
  • local AI
  • distributed AI (at the edge)

USAGE NOTE

Edge deployment is vital for applications requiring low latency, privacy, or operation in environments with limited connectivity.

DEVELOPERS

Organizations developing technology related to Edge Deployment.

  • NVIDIA

    Develops the Jetson platform, a series of embedded computing boards, and the TensorRT SDK for optimizing and deploying high-performance AI models on edge devices.

  • Qualcomm

    Designs Snapdragon processors with dedicated AI Engines. They provide the Qualcomm AI Stack for developers to optimize and deploy machine learning models directly on devices like smartphones and IoT hardware.

  • Google

    Offers the Coral platform, featuring Edge TPU hardware accelerators for running fast ML inference on-device. They also develop TensorFlow Lite, a framework for deploying models on mobile and embedded devices.

  • Intel

    Provides hardware like Movidius Vision Processing Units (VPUs) and the OpenVINO toolkit, a software solution for optimizing and deploying deep learning models on Intel hardware at the edge.

  • Arm

    Designs the architecture for most edge processors and provides specific IP like Ethos Neural Processing Units (NPUs) and software libraries (Arm NN) to accelerate machine learning workloads on low-power devices.

  • Edge Impulse

    A leading development platform for machine learning on edge devices, enabling developers to create and deploy models on microcontrollers and other resource-constrained hardware.

  • Apple

    Develops custom silicon (A-series, M-series) with an integrated Neural Engine for fast on-device ML processing. They provide the Core ML framework for deploying models within the Apple ecosystem.

  • Microsoft

    Offers Azure IoT Edge, a service for deploying and managing cloud workloads, including AI models packaged as containers, to run directly on IoT devices.

  • Amazon Web Services (AWS)

    Provides services like AWS IoT Greengrass and Amazon SageMaker Neo, which help developers to build, train, optimize, and deploy machine learning models on edge devices.

RELATED TERMS IN MLOPS & DEPLOYMENT