Configuring AI Models
Before deploying your AI Orchestrator node on the AI Subnet, you must choose the AI models for which you want to perform AI inference tasks. This guide assists in configuring these models. The following page, Download AI Models, provides instructions for their download. For details on supported pipelines and models, consult Pipelines.
Configuration File Format
Orchestrators specify supported AI models in an aiModels.json file, typically
located in the ~/.lpData directory. Below is an example configuration showing
currently recommended models and their respective prices.
[
{
"pipeline": "text-to-image",
"model_id": "ByteDance/SDXL-Lightning",
"price_per_unit": 4768371
},
{
"pipeline": "image-to-image",
"model_id": "ByteDance/SDXL-Lightning",
"price_per_unit": 4768371
},
{
"pipeline": "upscale",
"model_id": "stabilityai/stable-diffusion-x4-upscaler",
"price_per_unit": 4768371
},
{
"pipeline": "audio-to-text",
"model_id": "openai/whisper-large-v3",
"price_per_unit": 12882811
},
{
"pipeline": "image-to-video",
"model_id": "stabilityai/stable-video-diffusion-img2vid-xt-1-1",
"price_per_unit": 3390842,
"warm": true,
"optimization_flags": {
"SFAST": true,
"DEEPCACHE": false
}
}
]
Key Configuration Fields
During the Alpha phase, only one “warm” model per GPU is supported.
The inference pipeline to which the model belongs (e.g., text-to-image).
The price in Wei per unit, which
varies based on the pipeline (e.g., per pixel for image-to-video).
If true, the model is preloaded on the GPU to decrease runtime.
Optional flags to enhance performance (details below).
Optimization Flags
These flags are still experimental and may not always perform as expected. If you encounter any issues, please report them to the go-livepeer repository.
Our AI Subnet pipelines offer a suite of optimization flags. These are designed to enhance the performance of warm models by either increasing inference speed or reducing VRAM usage. Currently, the following flags are available:
Image-to-video Pipeline Optimization
The SFAST flag enables the Stable
Fast optimization framework,
potentially boosting inference speed by up to 25% with no quality
loss. Can not be used in conjunction with DEEPCACHE.
Text-to-image pipeline Optimization
DEEPCACHE for Lightning/Turbo models since they’re already optimized. Due to known limitations, it does not provide speed benefits and may significantly lower image quality.The DEEPCACHE flag enables the
DeepCache optimization framework,
which can enhance inference speed by up to 50% with minimal quality
loss. The speedup becomes more pronounced as the number of inference steps
increases. Cannot be used simultaneously with SFAST.
Was this page helpful?