What a gateway pipeline does
- Accepts AI requests from apps and clients
- Matches request requirements to advertised orchestrator capabilities
- Applies policy: price ceilings, latency expectations, retries, failover
- Dispatches work to orchestrators and returns outputs
Gateway vs orchestrator responsibilities
| Layer | Primary responsibility |
|---|
Pipeline types
- Real-time frame pipelines (video-to-video, overlays, style transfer)
- Streaming audio pipelines (ASR and translation)
- Capability-composed pipelines (for example, depth -> conditioning -> generation)
BYOC in gateway pipelines
Gateway operators can route to orchestrators that expose BYOC capabilities. BYOC extends what workloads a gateway can serve by allowing operators to run custom inference containers behind orchestrator capabilities.Gateway BYOC Guide
Gateway-side operating guidance for BYOC-capable routing.
Developer BYOC Guide
Implementation guide for teams building BYOC services consumed by gateways.