} ; }; A Dual Mode Orchestrator serves video transcoding and AI inference jobs from a single go-livepeer process. Video transcoding routes through NVENC and NVDEC, the fixed-function hardware blocks built into NVIDIA GPUs. AI inference runs on CUDA compute. Because these two execution paths use separate hardware resources, adding AI capabilities to a running video node requires no changes to your on-chain registration, staking, or existing transcoding configuration. Dual Mode requires Linux. The AI Runner container supports Linux only. ## What Changes A video-only Orchestrator and a Dual Mode Orchestrator run the same go-livepeer binary. The difference is configuration: Dual Mode adds three flags and one `aiModels.json` file to your existing setup. Video only AI only Dual Mode **Revenue streams** ETH transcoding fees + LPT rewards ETH AI inference fees Both streams simultaneously **Pricing config** `-pricePerUnit` (wei per pixel) `aiModels.json` per pipeline Both `-pricePerUnit` and `aiModels.json` **GPU execution path** NVENC/NVDEC hardware blocks CUDA compute cores Both paths, independent queues **VRAM requirement** Minimal (frame buffers only) 8-24 GB depending on model 16 GB recommended; 8 GB viable for LLM-only pipelines **Active Set** Required for video job eligibility AI job eligibility routes through capability advertisement Required for video; AI routes via capability advertisement **OS support** Linux, Windows Linux only Linux only ## Before You Start Confirm these prerequisites before starting: * **Hardware:** NVIDIA GPU with 16 GB VRAM or more. 8 GB is sufficient for LLM pipelines via the Ollama runner. Most diffusion model pipelines require 16 GB or more. See for per-GPU capability guidance. * **OS:** Linux. The AI Runner container is Linux-only. * **Docker and NVIDIA Container Toolkit:** Docker installed and the NVIDIA Container Toolkit configured so that `nvidia-smi` runs successfully inside a container. * **For video transcoding:** Your Arbitrum wallet is funded with ETH for gas, your node is activated via `livepeer_cli`, and you are staked with enough LPT to be eligible for the Active Set. * **AI model weights:** Model weights downloaded to your host machine before starting the AI Runner. See for download instructions. ## Setup Select the path that matches your current state. These steps set up a Dual Mode Orchestrator from scratch on a machine with no existing Livepeer configuration. Complete all four steps in order. Model weights must be present on the host machine before go-livepeer starts the AI Runner containers. Use the download script from the official AI Runner image to pull your chosen model. ```bash icon="terminal" title="Run the dual-mode container" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run --rm \ -v ~/.lpData/models:/models \ --gpus all \ livepeer/ai-runner:latest \ bash -c "PIPELINE=text-to-image MODEL_ID=ByteDance/SDXL-Lightning bash /app/dl_checkpoints.sh" ``` Replace `PIPELINE` and `MODEL_ID` with your chosen values. For all supported pipelines, model IDs, and VRAM requirements, see . Create `~/.lpData/aiModels.json` with at least one pipeline entry. Each entry declares a pipeline, model, price, and whether the model stays warm in VRAM between jobs. ```json icon="code" title="aiModels.json" theme={"theme":{"light":"github-light","dark":"dark-plus"}} [ { "pipeline": "text-to-image", "model_id": "ByteDance/SDXL-Lightning", "price_per_unit": 4768371, "warm": true } ] ``` `price_per_unit` is denominated in wei. The value above is illustrative. Check current per-pipeline market rates on before setting your own. To run multiple AI pipelines, add one entry per pipeline. To assign different pipelines to different GPUs, add separate entries referencing each GPU ID. The full `aiModels.json` schema is documented in . The command below starts a Dual Mode Orchestrator. It combines the standard video transcoding flags with the three AI-specific additions. ```bash icon="terminal" title="Run the dual-mode container" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run \ --name livepeer_dual_orchestrator \ -v ~/.lpData/:/root/.lpData/ \ -v /var/run/docker.sock:/var/run/docker.sock \ --network host \ --gpus all \ livepeer/go-livepeer:master \ -network arbitrum-one-mainnet \ -ethUrl \ -ethAcctAddr \ -orchestrator \ -transcoder \ -nvidia 0 \ -pricePerUnit \ -serviceAddr :8935 \ -aiWorker \ -aiModels /root/.lpData/aiModels.json \ -aiModelsDir ~/.lpData/models ``` The three AI-specific flags are: Flag Purpose `-aiWorker` Enables AI inference mode alongside transcoding `-aiModels` Path to `aiModels.json` inside the container (`/root/.lpData/`) `-aiModelsDir` Path to model weights on the host machine, required by Docker-out-of-Docker `-nvidia 0` assigns GPU 0 to the node. Use a comma-separated list (`0,1`) for multiple GPUs. `all` assigns every GPU. Mixed-VRAM hosts still need each container to fit on the GPU it lands on. Run `livepeer_cli` to register as an Orchestrator, set the Reward Cut and Fee Cut, and stake LPT. This step is required to receive video transcoding jobs. AI inference jobs route through capability advertisement and start reaching a node before it enters the Active Set for video work. ```bash icon="terminal" title="Run livepeer_cli" theme={"theme":{"light":"github-light","dark":"dark-plus"}} livepeer_cli ``` Select the "Invoke multi-step become an Orchestrator" option. After activation, your node must reach the top 100 Orchestrators by total stake to enter the Active Set and begin receiving video jobs. Check your standing on . A running video transcoding Orchestrator adds AI inference through three changes only: a model download, a new `aiModels.json`, and three additional flags in the start command. On-chain registration, staking, and transcoding configuration stay unchanged. Pull model weights to the host machine before restarting the node. The AI Runner containers mount this directory at startup, so the weights need to be present already. ```bash icon="terminal" title="Download model weights" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run --rm \ -v ~/.lpData/models:/models \ --gpus all \ livepeer/ai-runner:latest \ bash -c "PIPELINE=text-to-image MODEL_ID=ByteDance/SDXL-Lightning bash /app/dl_checkpoints.sh" ``` Replace `PIPELINE` and `MODEL_ID` with your chosen values. Supported pipelines and model IDs are listed in . Create `~/.lpData/aiModels.json` with the pipeline configuration. The same format applies to single-pipeline and multi-pipeline setups. ```json icon="code" title="aiModels.json" theme={"theme":{"light":"github-light","dark":"dark-plus"}} [ { "pipeline": "text-to-image", "model_id": "ByteDance/SDXL-Lightning", "price_per_unit": 4768371, "warm": true } ] ``` Check current per-pipeline market rates on before setting `price_per_unit`. Prices above the current Gateway buy-side range stop AI jobs from routing to that pipeline regardless of hardware capability. Add the three AI flags to your existing go-livepeer command. Your existing flags stay in place. ```bash icon="terminal" title="Add AI flags to the start command" theme={"theme":{"light":"github-light","dark":"dark-plus"}} # Add to your existing start command: -aiWorker \ -aiModels /root/.lpData/aiModels.json \ -aiModelsDir ~/.lpData/models ``` Also mount the Docker socket when it is missing. Go-livepeer uses Docker-out-of-Docker to manage AI Runner containers: ```bash icon="terminal" title="Add the Docker socket mount" theme={"theme":{"light":"github-light","dark":"dark-plus"}} # Include this volume mount: -v /var/run/docker.sock:/var/run/docker.sock \ ``` Restart the container with these additions. Your transcoding configuration, service address, and on-chain registration carry over unchanged. ## Verify After starting your node, confirm that both workload types are advertised to the network. **Check local capability registration:** ```bash icon="terminal" title="Check local capabilities" theme={"theme":{"light":"github-light","dark":"dark-plus"}} curl http://localhost:7935/getNetworkCapabilities | jq ``` The response includes a `pipelines` array listing your registered AI capabilities alongside your transcoding capability profile. A missing pipeline indicates a configuration error in `aiModels.json` or a container startup failure. **Check external network visibility:** Your node's capabilities, including warm models, are visible externally at [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities). Search by service address or ETH account address to confirm the network sees the AI pipelines alongside transcoding registration. AI pipelines that remain absent externally for two to three minutes after startup usually point to a container startup or registration issue. Inspect the container logs: ```bash icon="terminal" title="Inspect AI container logs" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker logs livepeer_dual_orchestrator 2>&1 | grep -i "ai-runner\|container\|pipeline\|error" ``` For a full error reference including AI Runner startup failures and capability registration issues, see . ## Resource Management Understanding how your GPU handles both workloads prevents VRAM-related failures. ### Why contention is lower than expected NVIDIA GPUs contain dedicated NVENC and NVDEC silicon for video encoding and decoding. These hardware blocks are separate from the CUDA execution units used for AI inference. Video transcoding in go-livepeer routes through NVENC and leaves CUDA cores for AI work. The VRAM constraint for a Dual Mode node is set by AI model weights. ### GPU class and viable AI pipelines The table below maps common GPU classes to AI pipeline combinations that are viable for Dual Mode. Transcoding capacity is available on all cards listed regardless of which AI pipelines are running. GPU class VRAM AI pipelines viable alongside transcoding RTX 2060 / 3060 8-12 GB LLM via Ollama runner, `audio-to-text`, `image-to-text`, and light diffusion models with SFAST optimisation enabled. RTX 3080 / 4070 10-12 GB Most batch pipelines. `text-to-image` with SDXL-Lightning. `image-to-video` pipelines require more VRAM than this class provides. RTX 3090 / 4090 / A5000 24 GB All batch pipelines including `image-to-video`, multiple warm models simultaneously, and live-video-to-video pipelines with VRAM headroom. ### Earnings Dual Mode operators earn from two revenue streams with independent demand patterns. Transcoding fees track network video demand. AI inference fees track per-capability demand from application developers. Both are paid in ETH via Probabilistic Micropayments. Current per-pipeline AI pricing and per-Orchestrator earnings are live on and . Market rates shift as network supply and demand change, so third-party figures age quickly. ## Related Pages Full `aiModels.json` schema, per-pipeline VRAM requirements, model download commands, and warm model management. How Gateways discover your capabilities and select your node for both video and AI jobs. Setting competitive prices for video transcoding and AI inference to maximise job volume from both revenue streams. Common errors and fixes: AI Runner startup failures, VRAM out-of-memory, and capability registration issues.