Attach Remote AI Workers

Introduction

The AI Worker is a crucial component of the Livepeer AI network, responsible for performing AI inference tasks. It can be run as a separate process on compute machines distinct from the Orchestrator or combined with the Orchestrator on the same machine.

Key Setup Considerations

Startup Configuration: If you decide to use separate AI Workers, this must be selected at the Orchestrator’s startup. A combined Orchestrator cannot simultaneously support remote AI Workers.
Shared Configuration File: Both the Orchestrator and AI Workers use the aiModels.json file (see Configuring AI Models).
- The Orchestrator uses aiModels.json to set model pricing.
- The AI Worker uses it to manage the runner containers for each model.

Remote AI Worker Setup

When using experimental external runner containers, ensure they connect to the AI Worker and not directly to the Orchestrator.

In a split configuration, the Orchestrator manages multiple AI Workers and allocates tasks based on the connected workers’ capacity. Worker capacity is determined by the following formula:

runner container count per pipeline/model_id = capacity per pipeline/model_id

The Orchestrator’s capacity is the sum of the capacities of all connected AI Workers. This setup enables flexible scaling of compute resources by adding or removing AI Workers as needed.

Launch Commands for Remote AI Worker

Below are the launch commands for both the Orchestrator and AI Worker nodes.

For the full Orchestrator launch command, see Start Your AI Orchestrator.

Show Launch Commands

Orchestrator Command

docker run \
    --name livepeer_ai_orchestrator \
    ...
    livepeer/go-livepeer:master \
    -orchestrator \
    ...
    -orchSecret <orchestrator secret>
    -serviceAddr <orchestrator service address>

AI Worker Command

docker run \
    --name livepeer_ai_worker \
    ...
    livepeer/go-livepeer:master \
    -aiWorker \
    ...
    -orchSecret <orchestrator secret> \
    -orchAddr <orchestrator service address>

Pricing: Prices in this example may vary and should be adjusted based on market research and the operational costs of providing compute.

Configuration Files (`aiModels.json`)

The aiModels.json file configures AI model parameters separately for the Orchestrator and the AI Worker, with each configuration tailored to the specific needs of that node.

For detailed guidance on configuring aiModels.json with advanced model settings, see Configuring AI Models.

Show Configuration Examples

Orchestrator Configuration

[
  {
    "pipeline": "text-to-image",
    "model_id": "SG161222/RealVisXL_V4.0_Lightning",
    "price_per_unit": 4768371,
    "pixels_per_unit": 1
  }
]

AI Worker Configuration

[
  {
    "pipeline": "text-to-image",
    "model_id": "SG161222/RealVisXL_V4.0_Lightning",
    "warm": true,
    "optimization_flags": {
      "SFAST": true
    }
  }
]

Verifying Remote AI Worker Operation

After starting your remote AI Worker node, you can verify it is operational by following the same inference test instructions used for the Orchestrator, as described in the Orchestrator Confirmation Section.

When accessing the AI Runner from a separate machine, replace localhost with the Worker Node’s IP address in the inference test instructions.

AI Video

​Introduction

​Key Setup Considerations

​Remote AI Worker Setup

​Launch Commands for Remote AI Worker

​Orchestrator Command

​AI Worker Command

​Configuration Files (aiModels.json)

​Orchestrator Configuration

​AI Worker Configuration

​Verifying Remote AI Worker Operation

Introduction

Key Setup Considerations

Remote AI Worker Setup

Launch Commands for Remote AI Worker

Orchestrator Command

AI Worker Command

Configuration Files (`aiModels.json`)

Orchestrator Configuration

AI Worker Configuration

Verifying Remote AI Worker Operation