Skip to main content
This page covers all available ComfyStream pipeline modes, the node ecosystem, how to build and load custom workflows, and how to configure output types including video, audio, and data-channel. If you have not yet run ComfyStream, start with the ComfyStream quickstart.

​
Pipeline Modes

ComfyStream supports four output modalities. Every ComfyStream workflow produces one of these output types.
ModeInputOutputRepresentative nodeNotes
Image-to-image (live)Live video frames (webcam or stream)Transformed video framesStreamDiffusion samplerPrimary mode for style transfer and generative overlays
Video-to-videoVideo segmentProcessed videoStreamDiffusion V2Temporal consistency across frames; suited to V2V tasks
Audio processingAudio track from streamAudio (pass-through or transformed)LoadAudioTensorProcesses audio alongside video in the same workflow
Data-channel outputAudio (for transcription) or video framesStructured text data alongside videoAudioTranscription + data output nodePhase 4 addition; Whisper-based; output via WebRTC data channel
ComfyStream can serve multiple pipelines in a single container (Phase 4 BYOC addition). Dynamic warm-up allows new pipelines to load mid-stream without restarting the server. //: # (REVIEW: Confirm “multiple pipelines in single container” framing from docs.comfystream.org or Phase 4 BYOC implementation details. Phase 4 retrospective says “hosting multiple models and disparate workflow/pipelines on one orchestrator in a single container.”)

​
Node Ecosystem

ComfyStream uses standard ComfyUI custom nodes. Any node that executes per-frame without maintaining incompatible state can be used in a real-time workflow.

​
Core I/O nodes

These nodes handle real-time tensor input and output. They are required for ComfyStream to read from and write to the video stream.
NodeSourcePurpose
LoadTensorlivepeer/comfystreamLoads a video frame tensor from the live stream for processing
LoadAudioTensorlivepeer/comfystreamLoads an audio frame tensor for audio-aware processing

​
Real-time control nodes

These nodes update their output on every workflow execution — designed specifically for real-time video loops.
NodeSourcePurpose
FloatControlComfyUI_RealtimeNodesOutputs a float that changes over time (sine, bounce, random) — use to animate parameters
IntControlComfyUI_RealtimeNodesSame as FloatControl for integer values
StringControlComfyUI_RealtimeNodesCycles through a list of strings per-frame
FloatSequenceComfyUI_RealtimeNodesCycles through comma-separated float values
IntSequenceComfyUI_RealtimeNodesCycles through comma-separated integer values
Motion detection nodesComfyUI_RealtimeNodesDetects motion between frames; can trigger parameter changes

​
StreamDiffusion nodes (Phase 4)

The primary generative video nodes, ported from Livepeer Inc’s Daydream StreamDiffusion pipeline.
NodePurposeNotes
StreamDiffusionCheckpointLoads a StreamDiffusion checkpoint modelUse with SD1.5 or SDXL models
StreamDiffusionConfigConfigures StreamDiffusion pipeline parametersControls CFG, t-index, acceleration mode
StreamDiffusionSamplerRuns StreamDiffusion inference per framePrimary inference node
StreamDiffusionLPCheckpointLoaderAlternative checkpoint loaderUse for Livepeer-hosted models
StreamDiffusionTensorRTEngineLoaderLoads a TensorRT-compiled engineRequires pre-compiled TRT engine; not compatible with all ControlNets
StreamDiffusion V2 adds support for video-to-video mode and stable diffusion V2 base models.

​
SuperResolution node (Phase 4)

Real-time video upscaling. Input: standard-resolution frame; output: upscaled frame. Suitable for adding resolution to low-quality input streams.

​
AudioTranscription nodes (Phase 4)

Whisper-based real-time speech transcription. Two output modes:
  • Video output with SRT subtitles — captions are burned into the video segments
  • Data-channel text output — transcript text delivered to the application separately via WebRTC data channel; no visual overlay

​
Custom Workflows

Any ComfyUI workflow can run in ComfyStream, provided it:
  • Accepts a LoadTensor input (for video) or LoadAudioTensor (for audio)
  • Produces output compatible with the stream output node
  • Does not require UI-format-only features (e.g., layout groups that are not API-compatible)

​
Workflow format

ComfyStream requires workflows in ComfyUI API format. This is not the same as the default ComfyUI save format, which includes layout information ComfyStream does not parse. To export a workflow in API format from ComfyUI:
  1. Enable Developer Mode in ComfyUI settings
  2. Use Save (API Format) — this produces the JSON file ComfyStream accepts
Workflows saved from ComfyUI in the default format (with UI layout data) will not load correctly in ComfyStream. Always use API format.

​
Loading a workflow

1

Export your workflow from ComfyUI in API format

In ComfyUI, go to Settings → Enable Dev mode. Then save your workflow using “Save (API Format)” to produce a .json file.
2

Place the workflow file

Copy the workflow JSON into the workflows/ directory inside your ComfyStream workspace. For Docker deployments, mount this directory as a volume. //: # (REVIEW: Confirm exact path convention from docs.comfystream.org. The workflows/ dir is confirmed from the ComfyStream repo but the precise expected path may differ per deployment mode.)
3

Load the workflow in the ComfyStream UI

Open the ComfyStream UI (default: http://localhost:8889). In the workflow selector, choose your file. The server will load the workflow and warm up the required models.First run triggers any TensorRT compilation required by the workflow. Subsequent loads skip compilation.

​
Custom node dependencies

If your workflow uses custom nodes beyond the core ComfyStream nodes, install those nodes’ dependencies inside the ComfyStream conda environment (or Docker container) before starting the server: 
# Inside the comfystream conda env
cd ComfyUI/custom_nodes/<your-custom-node>
pip install -r requirements.txt
For ComfyStream Docker deployments, Phase 4 added a config-based method to specify which custom node subsets are included in the container build: //: # (REVIEW: Confirm the exact config mechanism from docs.comfystream.org. Phase 4 retrospective says “a simple, config based method to allow for developing and deploying workflows using custom nodes which have different underlying python package requirements.”)

​
Data-Channel Output

The data-channel output type (Phase 4) allows ComfyStream to produce structured text data alongside video — without requiring it to be embedded in the video frames. Use cases:
  • Real-time audio transcription delivered as text to a downstream application
  • Frame-level metadata (e.g., object labels, confidence scores) delivered to an overlay UI
  • Any workflow where the output is data, not video
How it works: ComfyStream extends the WebRTC connection with a data channel. When the workflow contains a data output node, the text output is sent over the data channel to the browser or application that has connected to the ComfyStream server. To receive data-channel output from the client side, use @muxionlabs/byoc-sdk, which provides data-channel support alongside WebRTC video streaming.

​
Performance Tuning

​
First-run compilation

ComfyStream compiles TensorRT engines and runs torch.compile on model components at first run. This is a one-time cost per workflow on each machine.
  • TensorRT compilation: 2–10 minutes depending on model and GPU
  • torch.compile (ControlNet, VAE): compiles on first frame, subsequent frames are fast
  • Subsequent workflow loads on the same machine skip recompilation

​
Frame rate and throughput

Achievable frame rate depends on model complexity, GPU, and image resolution. Reference figures (from community testing, RTX 4090):
  • SD1.5 + DMD one-step + DepthControlNet workflow: ~14–15 fps at 640×360 input
  • StreamDiffusion with TensorRT: higher throughput at same resolution (exact figures vary by LoRA and ControlNet load)

​
Dynamic warm-up (Phase 4)

ComfyStream now supports dynamic warm-up, allowing new workflows to load mid-stream without restarting the server. This enables:
  • Multi-model hosting on a single orchestrator container
  • Hot-swap between workflows on demand

​
Configuration parameters

ParameterHow to setEffectDefault
--workspaceCLI flag to server/app.pyPath to ComfyUI workspace directoryRequired
--media-portsCLI flagComma-delimited UDP port range for WebRTC1024–65535
Portdocker run -p or --portServer port//: # (REVIEW: Confirm default port)

​
Next Steps

Bring Your Own Container

Deploy ComfyStream or any custom AI model as a Livepeer BYOC worker to earn network fees.

ComfyStream documentation

Full install reference, hardware requirements, and troubleshooting at the canonical ComfyStream docs.

ComfyStream quickstart

Back to getting started — if you need to revisit installation or first-run setup.
Last modified on March 16, 2026