Skip to main content
Hosting AI models on a Livepeer orchestrator means running inference workloads on GPU-backed nodes and making those models available for consumption via Livepeer Gateways. This is not training and not a generic GPU cloud - it is decentralised, real-time AI inference optimised for video and streaming workloads.
Consumers never talk to orchestrators directly. All traffic flows through gateways, which handle routing, auth, pricing, and QoS.
App / User
     ↓
Gateway (routing, auth, pricing, QoS)
     ↓
Orchestrator(s) (GPU + model + runtime)

​
Models best suited for orchestrators

Livepeer orchestrators are best for low-latency, real-time inference - especially:
  • Diffusion models (Stable Diffusion, StreamDiffusion)
  • Image-to-image and video-to-video models
  • ControlNet pipelines
  • Vision models (depth, pose, segmentation)
  • ComfyUI-based DAG workflows
  • Real-time video AI effects
Not ideal: large batch LLM inference, long-running training jobs, stateful fine-tuning loops. For a full compatibility matrix by model family, see Model Support.

​
What you do and don’t need to build

As an orchestrator operator, you focus on running excellent inference infrastructure. You do not need to build:
  • A marketplace
  • Authentication systems
  • Billing infrastructure
  • Service discovery
  • Brand or user trust
These are handled by Gateways and the protocol layer. Orchestrators compete on performance - lower latency, better GPUs, higher uptime, better-optimised pipelines, specialised or niche models.

​
Setup steps

​
Hosting multiple models on one orchestrator

A single GPU node can host multiple models, but VRAM is finite. Common patterns: Model multiplexing (cold swap): Load one model at a time, swap on request. Higher latency per swap, lower VRAM pressure. Suitable for less frequently requested models. Warm multi-model (concurrent): Keep multiple small models resident simultaneously. Works well for depth + segmentation + ControlNet combinations where each is lightweight. Dedicated GPU per model: For high-throughput production, assign one GPU per capability. Eliminates swap overhead entirely.
Cold starts significantly reduce job assignment. Gateways track orchestrator latency and avoid nodes with slow first-inference times. Keep your most-requested models warm.

​
How developers consume models you host

From a developer’s perspective, consumption looks like:
  • HTTP API calls (image/video in → output out)
  • WebRTC streams for real-time video
Examples already running on Livepeer that consume orchestrator-hosted models:
  • Daydream - generative AI video platform
  • StreamDiffusionTD - real-time diffusion via TouchDesigner
  • ComfyStream - browser-based ComfyUI pipelines
  • OBS plugins - live stream AI effects
Developers target capability types, not your specific node. If you advertise image-to-image with competitive latency and pricing, you’ll receive their jobs.

​
Partnering with gateways

Instead of competing with gateways, orchestrators can partner with them for guaranteed routing and exclusive pipeline access. This is a commercial arrangement outside the protocol. Relevant contacts: Cloud SPE operators, Daydream gateway team. See Gateway Providers.

​
See also

BYOC

Developer guide to building BYOC containers - how to structure your inference server.

Model Support

Full model family compatibility matrix - which models work on Livepeer and why.

ComfyStream

How ComfyStream works as the AI pipeline runtime for orchestrator nodes.

Orchestrator quickstart

Start here if you haven’t run a Livepeer orchestrator node yet.
Last modified on March 9, 2026