Before deploying your AI Orchestrator node on the AI Subnet, you must choose the AI models for which you want to perform AI inference tasks. This guide assists in configuring these models. The following page, Download AI Models, provides instructions for their download. For details on supported pipelines and models, consult Pipelines.

Configuration File Format

Orchestrators specify supported AI models in an aiModels.json file, typically located in the ~/.lpData directory. Below is an example configuration showing currently recommended models and their respective prices.

[
  {
    "pipeline": "text-to-image",
    "model_id": "ByteDance/SDXL-Lightning",
    "price_per_unit": 4768371
  },
  {
    "pipeline": "image-to-image",
    "model_id": "ByteDance/SDXL-Lightning",
    "price_per_unit": 4768371
  },
  {
    "pipeline": "upscale",
    "model_id": "stabilityai/stable-diffusion-x4-upscaler",
    "price_per_unit": 4768371,
  }
  {
    "pipeline": "image-to-video",
    "model_id": "stabilityai/stable-video-diffusion-img2vid-xt-1-1",
    "price_per_unit": 3390842,
    "warm": true,
    "optimization_flags": {
      "SFAST": true,
      "DEEPCACHE": false
    }
  }
]

Key Configuration Fields

During the Alpha phase, only one “warm” model per GPU is supported.

pipeline
string
required

The inference pipeline to which the model belongs (e.g., text-to-image).

model_id
string
required
price_per_unit
integer
required

The price in Wei per unit, which varies based on the pipeline (e.g., per pixel for image-to-video).

warm
boolean

If true, the model is preloaded on the GPU to decrease runtime.

optimization_flags
object

Optional flags to enhance performance (details below).

Optimization Flags

These flags are still experimental and may not always perform as expected. If you encounter any issues, please report them to the go-livepeer repository.

At this time, these flags are only compatible with warm models.

Our AI Subnet pipelines offer a suite of optimization flags. These are designed to enhance the performance of warm models by either increasing inference speed or reducing VRAM usage. Currently, the following flags are available:

Image-to-video Pipeline Optimization

SFAST
boolean

The SFAST flag enables the Stable Fast optimization framework, potentially boosting inference speed by up to 25% with no quality loss. Can not be used in conjunction with DEEPCACHE.

Text-to-image pipeline Optimization

DO NOT enable DEEPCACHE for Lightning/Turbo models since they’re already optimized. Due to known limitations, it does not provide speed benefits and may significantly lower image quality.
DEEPCACHE
boolean

The DEEPCACHE flag enables the DeepCache optimization framework, which can enhance inference speed by up to 50% with minimal quality loss. The speedup becomes more pronounced as the number of inference steps increases. Cannot be used simultaneously with SFAST.