runner container count per pipeline/model_id = capacity per pipeline/model_id
.
For expanded scalability, operators can deploy dedicated (remote) worker nodes
that connect to the orchestrator, increasing overall compute capacity.
Instructions for setting up remote workers are available on the
next page.
Retrieve the Livepeer AI Docker Image
Fetch the Latest AI Runner Docker Image
Pull Pipeline-Specific Images (optional)
Verify the AI Models are Available
Configure your AI Orchestrator
aiModels.json
file in the ~/.lpData/
directory. For guidance on configuring the aiModels.json
file, refer to the AI Models Configuration page. The configuration file should resemble:Launch an (off-chain) AI Orchestrator
-aiWorker
: This flag enables the AI Worker functionality.-aiModels
: This flag sets the path to the JSON file that contains the AI models.-aiModelsDir
: This flag indicates the directory where the AI models are stored on the host machine.-aiRunnerImage
: This optional flag specifies which version of the ai-runner image is used. Example: livepeer/ai-runner:0.0.2
--network host
flag facilitates communication between the AI Orchestrator and the AI Runner container.Lastly, the -nvidia
can be configured in a few ways. Use a comma seperated list of GPUs ie. 0,1
to activate specific GPU slots, each GPU will need it’s own config item in aiModels.json
. Alternativly we can use "all"
to activate all GPUs on the machine with a single model loaded in aiModels.json
(Warning: If different RAM size GPUs are installed it may cause containers to fail if they have less than the required RAM).aiModelsDir
path should be defined as being on the host machine.Confirm Successful Startup of the AI Orchestrator
Check Port Availability
8936
is unblocked on your machine. Additionally, consider setting up port forwarding on your router, allowing the Gateway node to be reachable from the internet.curl
command for this check.
Access the Swagger UI
http://localhost:8000/docs
to access the Swagger UI interface.Initiate an Inference Request
POST /text-to-image
endpoint and click the Try it out
button. Use the following example JSON payload:prompt
field.Inspect the Inference Response
url
field contains the base64 encoded image generated by the AI model. To convert this image to PNG, use a base64 decoder such as Base64.guru.