> ## Documentation Index
> Fetch the complete documentation index at: https://docs.livepeer.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Cascade Setup

> Deploy the live-video-to-video pipeline on a Livepeer orchestrator – Cascade architecture, ComfyStream setup, live-video frame processing, WebRTC streaming, hardware requirements, and custom pipeline development.

export const CustomDivider = ({color = "var(--lp-color-border-default)", middleText = "", spacing = "default", style = {}, className = "", ...rest}) => {
  const spacingPresets = {
    default: {
      margin: "24px 0"
    },
    overlap: {
      margin: "-1rem 0 -1rem 0"
    },
    tight: {
      margin: "0 0 -1rem 0"
    },
    section: {
      margin: "0 0 -2rem 0"
    },
    sectionOverlap: {
      margin: "-1rem 0 -2rem 0"
    },
    deepOverlap: {
      margin: "-1rem 0 -1.5rem 0"
    }
  };
  const spacingStyle = spacingPresets[spacing] || spacingPresets.default;
  return <div role="separator" aria-orientation="horizontal" className={className} style={{
    display: "flex",
    alignItems: "center",
    ...spacingStyle,
    fontSize: style?.fontSize || "16px",
    height: "fit-content",
    ...style
  }} {...rest}>
      <span style={{
    marginRight: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
      </span>
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      {middleText && <>
          <Icon icon="circle" size={2} />
          <span style={{
    margin: "0 8px",
    fontWeight: "bold",
    color: color,
    opacity: 0.7
  }}>
            {middleText}
          </span>
          <Icon icon="circle" size={2} />
        </>}
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      <span style={{
    marginLeft: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <span style={{
    display: "inline-block",
    transform: "scaleX(-1)"
  }}>
          <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
        </span>
      </span>
    </div>;
};

export const TableCell = ({children, align = "left", header = false, style = {}, className = "", ...rest}) => {
  const Component = header ? "th" : "td";
  return <Component className={className} style={{
    padding: "0.75rem 1rem",
    textAlign: align,
    border: header ? "none" : "1px solid var(--lp-color-border-default)",
    ...style
  }} {...rest}>
      {children}
    </Component>;
};

export const TableRow = ({children, header = false, hover = false, style = {}, className = "", ...rest}) => {
  const rowId = `table-row-${Math.random().toString(36).substr(2, 9)}`;
  return <>
      {hover && <style>{`
          #${rowId}:hover {
            background-color: var(--lp-color-bg-card);
          }
        `}</style>}
      <tr id={rowId} className={className} style={{
    ...header && ({
      backgroundColor: "var(--lp-color-accent-strong)",
      color: "var(--lp-color-on-accent)",
      fontWeight: "bold"
    }),
    ...style
  }} {...rest}>
        {children}
      </tr>
    </>;
};

export const StyledTable = ({children, variant = "default", style = {}, className = "", ...rest}) => {
  const wrapperVariants = {
    default: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "var(--lp-color-bg-card)",
      overflow: "hidden"
    },
    bordered: {
      border: "2px solid var(--lp-color-accent)",
      backgroundColor: "var(--lp-color-bg-page)",
      overflow: "hidden"
    },
    minimal: {
      border: "none",
      backgroundColor: "transparent",
      overflow: "visible"
    }
  };
  return <div data-docs-styled-table-shell className={className} style={{
    width: "100%",
    padding: 0,
    margin: 0,
    ...wrapperVariants[variant],
    ...style
  }} {...rest}>
      <table data-docs-styled-table style={{
    width: "100%",
    borderCollapse: "collapse",
    borderSpacing: 0,
    margin: 0,
    backgroundColor: "transparent"
  }}>
        {children}
      </table>
    </div>;
};

export const StyledStep = ({title, icon, titleSize = 'h3', iconColor = null, titleColor = null, children, className = '', style = {}, ...rest}) => {
  const styledTitle = titleColor ? <span style={{
    color: titleColor
  }}>{title}</span> : title;
  return <Step title={styledTitle} icon={icon} iconColor={iconColor || undefined} titleSize={titleSize} className={className} style={style} {...rest}>
      {children}
    </Step>;
};

export const StyledSteps = ({children, iconColor, titleColor, lineColor, iconSize = '24px', className = '', style = {}, ...rest}) => {
  const resolvedIconColor = iconColor || 'var(--accent-dark, #18794E)';
  const resolvedTitleColor = titleColor || 'var(--lp-color-accent)';
  const resolvedLineColor = lineColor || 'var(--lp-color-accent)';
  return <div className={['docs-styled-steps', className].filter(Boolean).join(' ')} style={style} {...rest}>
      <style>{`
        .docs-styled-steps .steps > div > div.absolute > div {
          background-color: ${resolvedIconColor};
        }
        .docs-styled-steps .steps > div > div.w-full > p {
          color: ${resolvedTitleColor};
        }
        .docs-styled-steps .steps > div > div.absolute.w-px {
          background-color: ${resolvedLineColor};
        }
        .docs-styled-steps .steps > div:last-child > div.absolute.w-px::after {
          content: '';
          position: absolute;
          bottom: 0;
          left: 50%;
          transform: translateX(-50%);
          width: 6px;
          height: 6px;
          background-color: ${resolvedLineColor};
          transform: translateX(-50%) rotate(45deg);
        }
      `}</style>
      <div>
        <Steps>{children}</Steps>
      </div>
    </div>;
};

Cascade inference is fundamentally different from batch inference. Instead of processing a discrete request and returning a result, your node continuously transforms live video – receiving frames from a WebRTC stream, running inference on each frame, and streaming processed frames back with sub-100ms latency.

This is the **Cascade** architecture: the `live-video-to-video` pipeline that powers live AI video effects, live style transfer, and streaming AI agents on the Livepeer Network.

<CustomDivider />

## What Cascade is

Cascade is Livepeer's live-video AI processing pipeline. The name refers to the architecture – video streams Cascade through AI transformation nodes in the network, enabling live applications that previously required centralised infrastructure.

Example applications on Cascade:

* **Daydream** – generative AI video platform with live style application
* **StreamDiffusionTD** – live diffusion via TouchDesigner
* **ComfyStream** – browser-based ComfyUI pipelines with live video input
* **OBS plugins** – live AI effects applied to streaming content

**Source:** [Livepeer AI Subnet Announcement – Mirror.xyz](https://mirror.xyz/livepeer.eth) · [ComfyStream on GitHub](https://github.com/livepeer/comfystream) · [Daydream](https://daydream.live)

<CustomDivider />

## How it differs from batch AI

<StyledTable variant="bordered">
  <TableRow header>
    <TableCell header />

    <TableCell header>Batch AI</TableCell>
    <TableCell header>Cascade live-video AI</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Input**</TableCell>
    <TableCell>Discrete file or prompt</TableCell>
    <TableCell>Continuous WebRTC stream</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Output**</TableCell>
    <TableCell>Result returned</TableCell>
    <TableCell>Processed WebRTC stream</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Latency target**</TableCell>
    <TableCell>Seconds per request</TableCell>
    <TableCell>\<100ms per frame</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Inference pattern**</TableCell>
    <TableCell>Request → process → respond</TableCell>
    <TableCell>Continuous frame loop</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Runtime**</TableCell>
    <TableCell>`livepeer/ai-runner`</TableCell>
    <TableCell>`livepeer/ai-runner:live-base`</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Pipeline interface**</TableCell>
    <TableCell>REST API</TableCell>
    <TableCell>`Pipeline` interface (async frame queue)</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Models**</TableCell>
    <TableCell>Standard diffusion, Whisper, BLIP</TableCell>
    <TableCell>StreamDiffusion, ComfyUI DAGs, ControlNet</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>**Min VRAM**</TableCell>
    <TableCell>4–24 GB</TableCell>
    <TableCell>24 GB recommended</TableCell>
  </TableRow>
</StyledTable>

The key difference is the **continuous frame loop**. Your pipeline receives frames as they arrive from the upstream WebRTC stream and must process and emit them quickly enough to avoid buffering. At 30 fps, you have 33 ms per frame budget. At 24 fps, 42 ms.

<CustomDivider />

## Prerequisites

Cascade has stricter hardware requirements than batch inference:

* **GPU:** RTX 4090 (24 GB) strongly recommended. RTX 3090 (24 GB) is functional but with less headroom. A100/H100 for production multi-stream setups.
* **CPU:** 8+ cores recommended. Frame decoding/encoding is CPU-bound.
* **Network:** Low-latency connection. WebRTC streams are sensitive to packet loss and jitter.
* **CUDA:** 12.0+
* **Docker** with NVIDIA Container Toolkit
* **go-livepeer** running with `-aiWorker` enabled

<Warning>
  GPUs below 24 GB VRAM (e.g. RTX 3080 10 GB, RTX 3060 12 GB) are typically insufficient for Cascade inference at acceptable quality and frame rates. The combination of model weights, StreamDiffusion/ComfyUI overhead, and frame buffers exhausts available VRAM.
</Warning>

<CustomDivider />

## Architecture overview

```icon="terminal" title="Cascade architecture overview" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Live video source (OBS, browser, camera)
         ↓
    WebRTC ingest
         ↓
   Livepeer Gateway
         ↓ routes stream to capable orchestrator
  go-livepeer AI Worker
         ↓ dispatches to live-video-to-video runner
  ai-runner:live-base container
         ↓ ComfyStream / custom pipeline
  Frame processing loop:
    receive frame → inference → emit frame
         ↓
  Processed WebRTC stream
         ↓
   Application / viewer
```

The Orchestrator receives the WebRTC stream, passes it to the AI Runner container, and streams the processed output back through the Gateway to the application. From the application's perspective, it's an input stream in and a transformed stream out.

<CustomDivider />

## ComfyStream – the live-video pipeline runtime

[ComfyStream](https://github.com/livepeer/comfystream) is the primary runtime for live-video AI inference on Livepeer. It wraps ComfyUI's node-based workflow system and adapts it for continuous frame processing.

**What ComfyStream adds over standard ComfyUI:**

* WebRTC frame ingestion and emission
* Async frame queue for continuous processing
* Warm model management to avoid per-frame load latency
* Livepeer AI worker integration via the `Pipeline` interface

**Compatible model types:**

* StreamDiffusion (optimised for live diffusion at 30+ fps)
* Standard SDXL / SD 1.5 (lower fps, higher quality)
* ControlNet variants (depth, pose, sketch, canny)
* IP-Adapter (style reference)
* DepthAnything / MiDaS (depth estimation)
* SAM2 (live segmentation)
* Any ComfyUI-compatible model loaded as a DAG node

**Source:** [Livepeer/ComfyStream on GitHub](https://github.com/livepeer/comfystream) · [ComfyUI-Stream-Pack](https://github.com/livepeer/ComfyUI-Stream-Pack)

<CustomDivider />

## Setup

<StyledSteps>
  <StyledStep title="Verify GPU and driver setup">
    ```bash icon="terminal" title="Verify GPU access on host and in Docker" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    nvidia-smi
    # Should show your GPU and driver version

    docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi
    # Docker must also see your GPU
    ```
  </StyledStep>

  <StyledStep title="Pull the live-base AI runner image">
    ```bash icon="terminal" title="Pull the live-base AI runner" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    docker pull livepeer/ai-runner:live-base
    ```

    The `live-base` tag is separate from the standard `livepeer/ai-runner` image used for batch pipelines. It includes ComfyStream and its dependencies.
  </StyledStep>

  <StyledStep title="Configure aiModels.json for the live pipeline">
    ```json icon="terminal" title="Minimal live-video-to-video aiModels.json entry" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    [
      {
        "pipeline": "live-video-to-video",
        "model_id": "streamdiffusion",
        "price_per_unit": 500,
        "warm": true
      }
    ]
    ```

    `model_id` for live-video pipelines refers to the ComfyUI workflow or pipeline name, not a HuggingFace model ID in the traditional sense. The models themselves are loaded inside the ComfyStream container.

    **Pricing and Gateway compatibility:** Your `price_per_unit` must stay at or below the Gateway's configured `-maxPricePerCapability` for `live-video-to-video`. This is a hard filter – Gateways skip nodes priced above that cap, regardless of hardware quality or latency.

    For transcoding, the equivalent flag is `-maxPricePerUnit` on the Gateway side. Before going live, verify that your transcoding price (set via `-pricePerUnit` or `livepeer_cli`) is below what major Gateways have configured as their maximum. See [Gateway Pricing Configuration](/v2/Gateways/guides/payments-and-pricing/pricing-configuration) for the full Gateway pricing flag reference and the JSON structure used for AI capability pricing.
  </StyledStep>

  <StyledStep title="Download the pipeline model weights">
    ComfyStream requires model weights to be present before the container starts. Download via the ComfyStream setup script:

    ```bash icon="terminal" title="Download ComfyStream model weights" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    git clone https://github.com/livepeer/comfystream
    cd comfystream
    pip install -r requirements.txt

    # Download StreamDiffusion and base models
    python scripts/download_models.py
    ```

    Models are downloaded to the volume mounted by the AI Runner container. Ensure the path matches what go-livepeer expects via the `-aiModelsDir` flag.
  </StyledStep>

  <StyledStep title="Start go-livepeer with live AI flags">
    ```bash icon="terminal" title="Start go-livepeer with live-video AI flags" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    docker run -d \
      --gpus all \
      --name livepeer-orchestrator \
      --network livepeer-ai \
      livepeer/go-livepeer:master \
      -orchestrator \
      -network arbitrum-one-mainnet \
      -aiWorker \
      -aiModels /root/.lpData/cfg/aiModels.json \
      -aiModelsDir /root/.lpData/models \
      -v 6
    ```
  </StyledStep>

  <StyledStep title="Verify the pipeline is registered">
    Check [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities). Your Orchestrator should appear under `live-video-to-video` with **Warm** status.

    Test the pipeline locally by sending a test WebRTC stream to your node. For setup scripts and test utilities, see the [ComfyStream repository](https://github.com/livepeer/comfystream).
  </StyledStep>
</StyledSteps>

<CustomDivider />

## StreamDiffusion – live-video diffusion models

[StreamDiffusion](https://github.com/cumulo-autumn/StreamDiffusion) is the primary model architecture for live-video AI on Livepeer. It was designed specifically for continuous frame processing and achieves 30+ fps on an RTX 4090.

**How StreamDiffusion achieves live performance:**

1. **Stream Batch** – processes multiple frames simultaneously as a batch, amortising model overhead across frames
2. **Residual CFG** – approximates classifier-free guidance with fewer forward passes
3. **Stochastic Similarity Filter** – skips inference on frames that are sufficiently similar to the previous frame
4. **TinyVAE acceleration** – uses a compressed VAE encoder/decoder for faster latency

**Source:** [cumulo-autumn/StreamDiffusion on GitHub](https://github.com/cumulo-autumn/StreamDiffusion) · [StreamDiffusion paper on arXiv](https://arxiv.org/abs/2312.12491)

### ComfyUI workflow for StreamDiffusion

A typical ComfyStream workflow for live style application:

```json icon="terminal" title="ComfyStream workflow example" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
{
  "nodes": [
    { "id": 1, "type": "LoadVideoInput", "outputs": ["frame"] },
    { "id": 2, "type": "VAEEncode", "inputs": ["frame"] },
    { "id": 3, "type": "StreamDiffusionSampler",
      "inputs": ["latent", "prompt", "model"],
      "config": { "num_inference_steps": 2, "guidance_scale": 1.2 }
    },
    { "id": 4, "type": "VAEDecode", "inputs": ["sampled_latent"] },
    { "id": 5, "type": "VideoOutput", "inputs": ["frame"] }
  ]
}
```

Steps as low as `2` support live performance. Quality vs latency is tunable through the workflow.

<CustomDivider />

## The Pipeline interface (custom pipelines)

For operators who want to build custom live-video AI processing beyond ComfyUI, the AI Runner exposes a Python `Pipeline` interface. Custom pipelines extend this interface and are packaged as Docker images extending `livepeer/ai-runner:live-base`.

```python icon="terminal" title="Custom live-video Pipeline interface example" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
from runner.live.pipelines import Pipeline
from runner.live.trickle import VideoFrame, VideoOutput

class MyLiveVideoPipeline(Pipeline):
    async def initialize(self, **params):
        # Load your model here
        # Use asyncio.to_thread() for blocking model load operations
        self.model = await asyncio.to_thread(load_my_model, params)

    async def put_video_frame(self, frame: VideoFrame, request_id: str):
        # Process one frame - this is called continuously
        result = await asyncio.to_thread(self.model.predict, frame.tensor)
        await self.frame_queue.put(VideoOutput(result, request_id))

    async def get_processed_video_frame(self) -> VideoOutput:
        return await self.frame_queue.get()

    async def update_params(self, **params):
        # Called when pipeline parameters are updated mid-stream
        pass

    async def stop(self):
        # Clean shutdown
        pass

    @classmethod
    def prepare_models(cls):
        # Download/compile models — called once during operator setup
        snapshot_download("my-org/my-model", ...)
```

**Source:** [ai-runner Pipeline interface](https://github.com/livepeer/ai-runner/blob/main/runner/src/runner/live/pipelines/interface.py) · [scope-runner reference implementation](https://github.com/daydreamlive/scope-runner)

### Integration requirements for custom pipelines

Custom live-video pipelines require two code changes in the upstream repositories:

1. **`ai-worker/runner/dl_checkpoints.sh`** – add your pipeline to the model download script
2. **`go-livepeer/ai/worker/docker.go`** – add your pipeline to the container image map (`livePipelineToImage`)

These changes are required because the current pipeline registry still uses a static mapping. See the full custom pipeline development guide in the [ai-runner.md](https://github.com/livepeer/ai-runner) documentation.

<Note>
  The Livepeer team is working toward a fully dynamic plugin architecture that eliminates these manual upstream changes. Track progress on the [ai-worker GitHub repository](https://github.com/livepeer/ai-worker).
</Note>

<CustomDivider />

## Model types for live-video inference

### StreamDiffusion (primary)

Best for: Continuous style application, generative video effects, live prompt-to-video

* `lcm-lora` variants for fastest inference
* SD 1.5 base with Lightning LoRA
* SDXL Turbo at reduced resolution

### ControlNet variants

ControlNet conditioning allows style transfer guided by structure maps extracted from the input frame:

<StyledTable variant="bordered">
  <TableRow header>
    <TableCell header>ControlNet</TableCell>
    <TableCell header>Input signal</TableCell>
    <TableCell header>Use case</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>Depth (MiDaS / DepthAnything)</TableCell>
    <TableCell>Depth map</TableCell>
    <TableCell>Preserve 3D structure while changing appearance</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>Canny / Sketch</TableCell>
    <TableCell>Edge map</TableCell>
    <TableCell>Structural outline preservation</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>Pose (DWPose)</TableCell>
    <TableCell>Pose keypoints</TableCell>
    <TableCell>Character animation, avatar driving</TableCell>
  </TableRow>

  <TableRow>
    <TableCell>Normal</TableCell>
    <TableCell>Surface normals</TableCell>
    <TableCell>Lighting-aware transformation</TableCell>
  </TableRow>
</StyledTable>

**Source:** [DepthAnything on HuggingFace](https://huggingface.co/LiheYoung/depth-anything-large-hf) · [DWPose](https://github.com/IDEA-Research/DWPose) · [ControlNet paper](https://arxiv.org/abs/2302.05543)

### IP-Adapter (style reference)

IP-Adapter conditions generation on a reference image, enabling consistent style application across frames. Effective for brand-consistent visual transformation.

**Source:** [tencent-ailab/IP-Adapter on GitHub](https://github.com/tencent-ailab/IP-Adapter)

<CustomDivider />

## Performance tuning

### Maximising fps

Cascade performance is dominated by inference latency per frame. Key levers:

**Model selection:** Use 1–2 step LCM or Lightning models instead of 20-step DDIM. The quality difference for streaming is acceptable; the latency difference is not.

**Resolution:** Lower resolution dramatically increases fps. 512×512 at 30 fps is achievable on an RTX 4090 with StreamDiffusion. 768×768 drops to \~20 fps. 1024×1024 drops to \~12 fps.

**TensorRT compilation:** For production deployments, compile the model to TensorRT engine format. One-time compilation overhead; 2–4× runtime speedup.

```python icon="terminal" title="TensorRT compilation example" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
# In your Pipeline.prepare_models():
from torch2trt import torch2trt

model_trt = torch2trt(model, [example_input], fp16_mode=True)
torch.save(model_trt.state_dict(), 'model_trt.pth')
```

**Stochastic Similarity Filter:** Enable in StreamDiffusion to skip inference on static or slow-moving frames. This usually lifts fps with no visible quality loss for typical video content.

### VRAM management

Unlike batch AI, live-video pipelines hold models in VRAM continuously for the duration of the stream. VRAM must be reserved for:

* Model weights (\~8–18 GB for SDXL-class)
* Frame buffers (input + output, \~500 MB–1 GB per resolution)
* ControlNet/LoRA adapters (\~1–3 GB each)
* Stream batch buffer (StreamDiffusion's continuous frame queue)

On a 24 GB GPU, keep total loaded model + adapter weight below 20 GB to leave headroom for frame buffers and batch operations.

<CustomDivider />

## Multi-stream capacity

A single RTX 4090 usually handles **1–2 concurrent live-video streams** depending on resolution and model complexity. For multi-stream capacity:

* **Multiple GPUs:** The AI worker dispatches streams across multiple physical GPUs
* **Model parallelism:** ComfyStream uses one GPU per stream
* **Scale-out:** Run multiple Orchestrator instances, each handling 1–2 streams, behind a Gateway load balancer

Enterprise operators running 4090/A100 fleets monetise live-video AI at scale by advertising higher `capacity` values and maintaining multiple warm pipeline instances.

<CustomDivider />

## Watch: Cascade and live-video AI

<Frame>
  <iframe width="100%" height="400" src="https://www.youtube.com/embed/7a6kBrL0RYg" title="ComfyStream - Live Video AI with Livepeer" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen />
</Frame>

<CardGroup cols={2}>
  <Card title="Encode Club Live Video AI Bootcamp" icon="graduation-cap" href="https://www.youtube.com/watch?v=UKWdvJBqrNw">
    Full Q1 2025 bootcamp session. Covers ComfyStream, Cascade architecture, and Orchestrator setup for live-video inference.
  </Card>

  <Card title="StreamDiffusion Demo" icon="play" href="https://github.com/cumulo-autumn/StreamDiffusion">
    StreamDiffusion GitHub repository includes benchmark videos showing 30+ fps live style transfer.
  </Card>
</CardGroup>

<CustomDivider />

## Troubleshooting

<AccordionGroup>
  <Accordion title="Frames dropping or high latency" icon="circle-question">
    1. **Model is too slow for target fps** – try a lower-step model (LCM, Lightning) or reduce output resolution
    2. **VRAM OOM on frame buffer** – reduce `stream_batch_size` in StreamDiffusion config
    3. **CPU bottleneck on encode/decode** – WebRTC frame codec operations are CPU-bound; monitor CPU usage during streaming
    4. **Network jitter** – WebRTC is sensitive to packet loss; check your upstream network quality
  </Accordion>

  <Accordion title="Restore live-video job flow" icon="microchip">
    1. Confirm `live-video-to-video` appears on [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities) under your Orchestrator
    2. Verify the `live-base` container is running and healthy: `docker ps --filter name=livepeer-ai-runner-live`
    3. Check that your Orchestrator's `serviceAddr` is reachable from Gateways – WebRTC ICE negotiation requires bidirectional reachability
    4. Confirm your node has WebRTC port access (typically UDP 8935 or your configured port)
  </Accordion>

  <Accordion title="ComfyStream container failing to start" icon="microchip">
    1. Check model weights are present at the expected path (`-aiModelsDir` location)
    2. Check CUDA/driver compatibility – ComfyStream requires CUDA 12.0+
    3. Run the container manually to see startup output:

    ```bash icon="terminal" title="Check CUDA access inside the live-base container" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    docker run --gpus all --rm livepeer/ai-runner:live-base python -c "import torch; print(torch.cuda.is_available())"
    ```

    This confirms CUDA is accessible inside the container.
  </Accordion>

  <Accordion title="Custom pipeline registration issues" icon="triangle-exclamation">
    1. Verify `livePipelineToImage` in `go-livepeer/ai/worker/docker.go` includes your pipeline name
    2. Confirm `dl_checkpoints.sh` in `ai-runner` includes your pipeline's model preparation step
    3. The `model_id` in `aiModels.json` must match the `name` field in your `PipelineSpec` exactly
    4. After rebuilding and redeploying, re-register capabilities with the network
  </Accordion>
</AccordionGroup>

<CustomDivider />

## Related

<CardGroup cols={2}>
  <Card title="Batch AI Setup" icon="gears" href="/v2/orchestrators/guides/ai-and-job-workloads/diffusion-pipeline-setup">
    Configure text-to-image, audio-to-text, LLM, and other batch pipelines.
  </Card>

  <Card title="Model Hosting and VRAM" icon="microchip" href="/v2/orchestrators/guides/ai-and-job-workloads/model-demand-reference">
    VRAM planning, warm model strategy, and aiModels.json reference.
  </Card>

  <Card title="ComfyStream on GitHub" icon="github" href="https://github.com/livepeer/comfystream">
    Source repository for ComfyStream, including setup scripts, example workflows, and community contributions.
  </Card>

  <Card title="AI Workloads Overview" icon="diagram-project" href="/v2/orchestrators/guides/ai-and-job-workloads/ai-inference-operations">
    Batch vs live-video AI, pipeline types, and network routing.
  </Card>
</CardGroup>