> ## Documentation Index
> Fetch the complete documentation index at: https://docs.livepeer.org/llms.txt
> Use this file to discover all available pages before exploring further.

# ComfyStream Overview

> ComfyStream turns any ComfyUI workflow into a real-time video pipeline. Phase 4 hardened it for production: multimodal output, dynamic warm-up, BYOC packaging.

export const CenteredContainer = ({children, maxWidth = "800px", padding = "0", preset = "default", width = "", minWidth = "", marginRight = "", marginBottom = "", textAlign = "", style = {}, className = "", ...rest}) => {
  const presets = {
    default: {},
    fitContent: {
      width: "fit-content",
      minWidth: "fit-content"
    },
    readable70: {
      width: "70%",
      minWidth: "fit-content"
    },
    readable80: {
      width: "80%",
      minWidth: "fit-content"
    },
    readable90: {
      width: "90%"
    },
    wide900: {
      maxWidth: "900px"
    }
  };
  const presetStyle = presets[preset] || presets.default;
  return <div className={className} style={{
    maxWidth: presetStyle.maxWidth || maxWidth,
    margin: "0 auto",
    padding: padding,
    ...presetStyle.width ? {
      width: presetStyle.width
    } : {},
    ...presetStyle.minWidth ? {
      minWidth: presetStyle.minWidth
    } : {},
    ...width ? {
      width
    } : {},
    ...minWidth ? {
      minWidth
    } : {},
    ...marginRight ? {
      marginRight
    } : {},
    ...marginBottom ? {
      marginBottom
    } : {},
    ...textAlign ? {
      textAlign
    } : {},
    ...style
  }} {...rest}>
      {children}
    </div>;
};

export const CustomDivider = ({color = "var(--lp-color-border-default)", middleText = "", spacing = "default", style = {}, className = "", ...rest}) => {
  const spacingPresets = {
    default: {
      margin: "24px 0"
    },
    overlap: {
      margin: "-1rem 0 -1rem 0"
    },
    tight: {
      margin: "0 0 -1rem 0"
    },
    section: {
      margin: "0 0 -2rem 0"
    },
    sectionOverlap: {
      margin: "-1rem 0 -2rem 0"
    },
    deepOverlap: {
      margin: "-1rem 0 -1.5rem 0"
    }
  };
  const spacingStyle = spacingPresets[spacing] || spacingPresets.default;
  return <div role="separator" aria-orientation="horizontal" className={className} style={{
    display: "flex",
    alignItems: "center",
    ...spacingStyle,
    fontSize: style?.fontSize || "16px",
    height: "fit-content",
    ...style
  }} {...rest}>
      <span style={{
    marginRight: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
      </span>
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      {middleText && <>
          <Icon icon="circle" size={2} />
          <span style={{
    margin: "0 8px",
    fontWeight: "bold",
    color: color,
    opacity: 0.7
  }}>
            {middleText}
          </span>
          <Icon icon="circle" size={2} />
        </>}
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      <span style={{
    marginLeft: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <span style={{
    display: "inline-block",
    transform: "scaleX(-1)"
  }}>
          <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
        </span>
      </span>
    </div>;
};

export const LinkArrow = ({href, label, description, newline = true, borderColor, className = '', style = {}, ...rest}) => {
  const linkArrowStyle = {
    display: 'inline-flex',
    alignItems: 'center',
    justifyContent: 'center',
    gap: "var(--lp-spacing-1)",
    width: 'fit-content',
    ...borderColor && ({
      borderColor
    })
  };
  return <span className={className} style={style} {...rest}>
      {newline && <br />}
      <span style={linkArrowStyle}>
        <a href={href} target="_blank" rel="noopener noreferrer">
          {label}
        </a>
        <Icon icon="arrow-up-right" size={14} color="var(--lp-color-accent)" />
      </span>
      {description && description}
      {description && <div style={{
    height: "var(--lp-spacing-3)"
  }} />}
    </span>;
};

<CenteredContainer preset="readable90">
  <Tip>ComfyStream turns ComfyUI workflows into real-time video pipelines. One Docker image, one workflow JSON, sub-second latency.</Tip>
</CenteredContainer>

***

ComfyStream is a Livepeer-maintained extension of ComfyUI that swaps batch image generation for a real-time video loop. A ComfyUI workflow that takes an image and returns an image becomes a live-video-to-video pipeline when run through ComfyStream: video frames flow in over WebRTC, the workflow processes each frame, transformed frames flow back out at sub-second latency.

Phase 4 (January 2026) hardened ComfyStream for production. The runtime added audio processing, data-channel output, dynamic workflow warm-up, and PyTrickle-based BYOC packaging. Daydream and Embody both run on ComfyStream infrastructure.

The canonical install reference is `docs.comfystream.org`. The repository is `livepeer/comfystream`. The current Docker image is `livepeer/comfystream`, with `livepeer/comfyui-base:stable` as the BYOC base.

<CustomDivider />

## Pipeline Modes

ComfyStream workflows produce one of four output types. Every workflow declares its output mode through the nodes it composes.

| Mode                  | Input                   | Output                              | Representative node                     |
| --------------------- | ----------------------- | ----------------------------------- | --------------------------------------- |
| Image-to-image (live) | Live video frames       | Transformed video frames            | `StreamDiffusionSampler`                |
| Video-to-video        | Video segment           | Processed video                     | StreamDiffusion V2                      |
| Audio processing      | Audio track from stream | Audio (pass-through or transformed) | `LoadAudioTensor`                       |
| Data-channel output   | Audio or video frames   | Structured text alongside video     | `AudioTranscription` + data output node |

A single ComfyStream container can host multiple pipelines (Phase 4 addition). Dynamic warm-up loads new workflows mid-stream without restarting the server, which lets one Orchestrator advertise multiple capabilities from one image.

<CustomDivider />

## Node Ecosystem

ComfyStream uses standard ComfyUI custom nodes. Any node that executes per-frame without maintaining incompatible state runs in a real-time workflow.

### Core I/O Nodes

Required for every ComfyStream workflow. They handle the real-time tensor handoff between the stream and the ComfyUI graph.

| Node              | Source                 | Purpose                                               |
| ----------------- | ---------------------- | ----------------------------------------------------- |
| `LoadTensor`      | `livepeer/comfystream` | Load a video frame tensor from the live stream        |
| `LoadAudioTensor` | `livepeer/comfystream` | Load an audio frame tensor for audio-aware processing |

### Real-Time Control Nodes

These nodes update their output on every workflow execution, which makes them suitable for animating parameters across a continuous stream.

| Node                   | Source                  | Purpose                                                       |
| ---------------------- | ----------------------- | ------------------------------------------------------------- |
| `FloatControl`         | `ComfyUI_RealtimeNodes` | Outputs a float that changes over time (sine, bounce, random) |
| `IntControl`           | `ComfyUI_RealtimeNodes` | Same as `FloatControl` for integer values                     |
| `StringControl`        | `ComfyUI_RealtimeNodes` | Cycles through a list of strings per frame                    |
| `FloatSequence`        | `ComfyUI_RealtimeNodes` | Cycles through comma-separated float values                   |
| `IntSequence`          | `ComfyUI_RealtimeNodes` | Cycles through comma-separated integer values                 |
| Motion detection nodes | `ComfyUI_RealtimeNodes` | Detect motion between frames; can trigger parameter changes   |

### StreamDiffusion Nodes (Phase 4)

The primary generative video nodes. Ported from Daydream's StreamDiffusion pipeline.

| Node                                  | Purpose                                                                   |
| ------------------------------------- | ------------------------------------------------------------------------- |
| `StreamDiffusionCheckpoint`           | Loads a StreamDiffusion checkpoint model. Use with SD1.5 or SDXL          |
| `StreamDiffusionConfig`               | Configures CFG, t-index, acceleration mode                                |
| `StreamDiffusionSampler`              | Runs StreamDiffusion inference per frame                                  |
| `StreamDiffusionLPCheckpointLoader`   | Alternative checkpoint loader for Livepeer-hosted models                  |
| `StreamDiffusionTensorRTEngineLoader` | Loads a pre-compiled TensorRT engine. Not compatible with all ControlNets |

StreamDiffusion V2 adds video-to-video mode and stable diffusion V2 base models.

### Phase 4 Additions

* **SuperResolution.** Real-time video upscaling. Input: standard-resolution frame. Output: upscaled frame.
* **AudioTranscription.** Whisper-based real-time speech transcription. Two output modes: SRT subtitles burned into video, or text delivered to the application via WebRTC data channel.

<CustomDivider />

## Workflow Format

ComfyStream requires workflows in ComfyUI **API format**, not the default save format. The default ComfyUI export includes layout metadata that ComfyStream does not parse.

To export a workflow in API format:

1. Enable **Developer Mode** in ComfyUI settings.
2. Use **Save (API Format)** to produce the JSON file.

Workflows saved in the default format will not load correctly. API format is the only supported input.

Place the workflow file in your ComfyStream workspace's `workflows/` directory. For Docker deployments, mount this directory as a volume. The canonical workspace layout is in `docs.comfystream.org`.

When the workflow loads in the ComfyStream UI, the server compiles TensorRT engines for the relevant nodes. First run takes between two and ten minutes depending on the model and the GPU. Subsequent loads skip compilation.

<CustomDivider />

## Data-Channel Output

Phase 4 added a structured-text output path that runs alongside video. The ComfyStream WebRTC connection extends with a data channel; workflows containing a data output node emit text to the browser or application that connects to the server.

Use cases:

* Real-time audio transcription delivered as text to a downstream application
* Frame-level metadata (object labels, confidence scores) delivered to an overlay UI
* Any workflow where the output is data, not video

To receive data-channel output from a browser client, use <LinkArrow href="/v2/developers/build/ai-and-agents/realtime-ai/pytrickle/overview" label="PyTrickle" newline={false} />, which handles WebRTC video streaming and the data channel from the same connection.

<CustomDivider />

## Performance Characteristics

ComfyStream compiles TensorRT engines and runs `torch.compile` on model components at first run. This is a one-time cost per workflow on each machine.

| Operation                         | Duration       | Frequency                           |
| --------------------------------- | -------------- | ----------------------------------- |
| TensorRT compilation              | 2-10 minutes   | First run per machine, per workflow |
| `torch.compile` (ControlNet, VAE) | On first frame | First frame per session             |
| Subsequent workflow loads         | Immediate      | All later runs                      |

Achievable frame rate depends on model complexity, GPU, and image resolution. Reference figures from community testing on an RTX 4090:

* SD1.5 + DMD one-step + DepthControlNet workflow: 14-15 fps at 640x360 input
* StreamDiffusion with TensorRT: higher throughput at the same resolution

Frame rates vary substantially with LoRA stack and ControlNet load. Test under expected concurrency before production launch.

<CustomDivider />

## Hardware Requirements

ComfyStream requires an NVIDIA GPU. The server component runs on Linux only; Windows and macOS are not supported for the server, though the browser client runs anywhere.

| Workload                   | Minimum VRAM | Recommended |
| -------------------------- | ------------ | ----------- |
| Real-time AI (ComfyStream) | 12 GB        | 16 GB+      |

VRAM headroom matters for stability. A workflow that runs at 12 GB may stutter under load that 16 GB absorbs cleanly. Source: the <LinkArrow href="/v2/developers/build/ai-and-agents/model-support" label="model support" newline={false} />.

CUDA 12.0+ is required. Current ComfyStream releases target CUDA 12.8 with NVIDIA driver 570.124.06 or later.

<CustomDivider />

## Relationship to BYOC

ComfyStream is itself BYOC-compatible. Phase 4 integrated ComfyStream with PyTrickle, which means the `livepeer/comfystream` Docker image can register directly as a BYOC capability on an Orchestrator without rewriting the workflow as a custom container.

| If you want to                                           | Use                                                                                          |
| -------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| Run a ComfyUI workflow as a real-time pipeline           | ComfyStream directly                                                                         |
| Run a custom Python model that isn't a ComfyUI workflow  | <LinkArrow href="/v2/developers/build/compute/byoc/overview" label="BYOC" newline={false} /> |
| Run multiple ComfyStream workflows from one Orchestrator | ComfyStream's multi-pipeline mode (Phase 4)                                                  |
| Earn fees from your ComfyStream instance                 | Register as a BYOC capability on go-livepeer                                                 |

<CustomDivider />

The [ComfyStream quickstart](/v2/developers/build/ai-and-agents/realtime-ai/ComfyStream/ComfyStream-quickstart) gets you to a working pipeline in under 30 minutes. Start there.

## Next Steps

<CardGroup cols={2}>
  <Card title="ComfyStream Quickstart" icon="rocket" href="/v2/developers/build/ai-and-agents/realtime-ai/comfystream/comfystream-quickstart">
    Docker, RunPod, or local install. First real-time AI effect on a webcam in fifteen minutes.
  </Card>

  <Card title="Workflow Authoring" icon="diagram-project" href="/v2/developers/build/ai-and-agents/realtime-ai/comfystream/workflow-authoring">
    Build a custom workflow, configure StreamDiffusion, tune for latency.
  </Card>

  <Card title="ComfyStream as BYOC" icon="cube" href="/v2/developers/build/ai-and-agents/realtime-ai/comfystream/comfystream-as-byoc">
    Register a ComfyStream instance as a BYOC capability and earn fees.
  </Card>

  <Card title="docs.comfystream.org" icon="book" href="https://docs.comfystream.org">
    Canonical install reference, hardware deep-dive, troubleshooting.
  </Card>
</CardGroup>