Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.livepeer.org/llms.txt

Use this file to discover all available pages before exploring further.

PyTrickle’s processing surface is the FrameProcessor base class. Subclass it, implement the async methods below, and pass an instance to StreamServer. The SDK manages connection lifecycle, decoding, encoding, and transport.

​
FrameProcessor

​
Initialize Method

async def initialize(self) -> None
Called once before the first frame is delivered. Load AI models, allocate CUDA tensors, or warm up any resources that must be ready before processing begins. Raises are propagated and halt startup.

​
Process Video Async Method

async def process_video_async(self, frame: VideoFrame) -> Optional[VideoFrame]
Called for each decoded video frame. Return a VideoFrame to include it in the output stream, or None to drop the frame. Blocking operations (model inference on CPU, disk I/O) stall the processing loop and increase output latency. Run them in a thread pool: 
import asyncio

async def process_video_async(self, frame: VideoFrame) -> Optional[VideoFrame]:
    tensor = frame.tensor.clone()
    result_tensor = await asyncio.to_thread(self._run_model, tensor)
    return frame.replace_tensor(result_tensor)

def _run_model(self, tensor):
    # Blocking model inference here
    return self.model(tensor)

​
Process Audio Async Method

async def process_audio_async(self, frame: AudioFrame) -> Optional[List[AudioFrame]]
Called for each decoded audio frame. Return a list of AudioFrame objects (pass-through returns [frame]), or None to drop the frame. The SDK automatically detects and converts between mono, stereo, and multi-channel formats.

​
Update Params Method

def update_params(self, params: dict) -> None
Called when the Livepeer orchestrator forwards a parameter update from the gateway. params is a plain dict; the schema is defined by your service. Updates are applied immediately to the running processor without interrupting the stream. Example: 
def update_params(self, params: dict):
    if "prompt" in params:
        self.prompt = params["prompt"]
    if "strength" in params:
        self.strength = float(params["strength"])

​
VideoFrame

Represents a single decoded video frame.

​
AudioFrame

Represents a decoded audio buffer.

​
StreamServer

StreamServer(
    frame_processor: FrameProcessor,
    port: int = 8000,
    capability_name: str = "custom-processor",
)

​
HTTP endpoints

​
Framerate and Throughput

The max_framerate parameter in the /api/stream/start request body controls the maximum number of frames passed to process_video_async per second. Frames arriving faster than max_framerate are dropped before the processor sees them. This prevents backpressure from accumulating when model inference is slower than the input frame rate. Set max_framerate to the throughput your model can sustain. For StreamDiffusion workflows on an RTX 4090, this is typically 15-30 FPS depending on resolution. For lighter processing, match the input stream frame rate. The PyTrickle quickstart walks through a complete FrameProcessor from scratch. The PyTrickle reference covers the full API surface.

PyTrickle Quickstart

Working example of a FrameProcessor running against a local test stream.

ComfyStream as BYOC

How to register a trickle service with a Livepeer orchestrator.
Last modified on May 19, 2026