;
};
the **request pipeline** - the layer between clients and the Gateway.
The **payment pipeline** (how the Gateway settles with Orchestrators) is covered in .
The `go-livepeer` binary is a routing and payment engine. It does not authenticate users, meter usage, or route premium customers to different Orchestrators.
Those concerns belong to a middleware layer sitting between clients and the Gateway process.
## Responsibilities
```mermaid theme={"theme":{"light":"github-light","dark":"dark-plus"}}
%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#18794E', 'primaryTextColor': '#fff', 'primaryBorderColor': '#3CB540', 'lineColor': '#3CB540', 'mainBkg': '#18794E', 'nodeBorder': '#3CB540', 'clusterBkg': 'transparent', 'clusterBorder': '#3CB540', 'titleColor': '#3CB540', 'edgeLabelBackground': 'transparent', 'textColor': '#3CB540', 'nodeTextColor': '#fff'}}}%%
flowchart TB
C[Client App] --> MW
subgraph MW["Middleware Layer"]
A[Auth / API keys]
RL[Rate limiting]
M[Usage metering]
R[Request routing]
end
MW --> GW
subgraph GW["go-livepeer Gateway"]
O[Orchestrator selection]
P[Payment tickets]
end
GW --> ORCH[Orchestrator Network]
```
The middleware handles the customer-facing interface. The Gateway handles the network interface.
ConcernMiddlewarego-livepeerAPI key validationYesNoJWT / OAuth verificationYesNoPer-customer rate limitsYesNoUsage tracking and billingYesNoCustomer-to-tier routingYesNoOrchestrator selectionNoYesProbabilistic MicropaymentsNoYesOn-chain settlementNoYes
## Architecture
Three deployment patterns cover most middleware use cases.
The simplest and most maintainable pattern: an existing reverse proxy handles TLS, auth, and rate limiting before passing requests to the Gateway.
**Caddy with API key auth** (simplest):
```caddyfile icon="terminal" Caddy File theme={"theme":{"light":"github-light","dark":"dark-plus"}}
ai-gateway.yourdomain.com {
@authenticated {
header Authorization "Bearer {$VALID_API_KEY}"
}
handle @authenticated {
reverse_proxy localhost:8935
}
respond 401
}
```
**Nginx with external auth** (more flexible - delegates auth to a backend):
```nginx icon="terminal" NGINX theme={"theme":{"light":"github-light","dark":"dark-plus"}}
server {
listen 443 ssl;
server_name ai-gateway.yourdomain.com;
location / {
auth_request /auth;
proxy_pass http://127.0.0.1:8935;
}
location = /auth {
proxy_pass http://127.0.0.1:3000/verify-token;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
}
}
```
The `/verify-token` endpoint returns `200` to allow or `401` to deny. It can also return headers that nginx forwards to the Gateway, such as customer ID or tier.
For multi-tenant deployments where per-customer rate limits, quota enforcement, and usage analytics are needed, a dedicated API gateway layer fits well.
**Kong** (open source, self-hosted):
```yaml icon="terminal" Kong theme={"theme":{"light":"github-light","dark":"dark-plus"}}
# kong.yml - declarative config
services:
- name: livepeer-gateway
url: http://livepeer-gateway:8935
routes:
- name: ai-inference
paths:
- /
plugins:
- name: key-auth
- name: rate-limiting
config:
minute: 60
policy: local
- name: request-termination
config:
status_code: 429
message: "Rate limit exceeded"
```
**Traefik** (Docker-native, automatic service discovery):
```yaml icon="terminal" Traefik theme={"theme":{"light":"github-light","dark":"dark-plus"}}
# docker-compose.yml
services:
traefik:
image: traefik:v3
command:
- "--providers.docker=true"
- "--entrypoints.websecure.address=:443"
livepeer-gateway:
image: livepeer/go-livepeer:master
labels:
- "traefik.http.routers.livepeer.rule=Host(`ai-gateway.yourdomain.com`)"
- "traefik.http.routers.livepeer.middlewares=auth@docker"
- "traefik.http.middlewares.auth.basicauth.users=user:$$apr1$$..."
```
For operators who need full control over routing logic, a lightweight custom proxy in Node.js, Go, or Python sits in front of the Gateway.
**Node.js example** (Express + http-proxy-middleware):
```javascript icon="code" Custom Proxy theme={"theme":{"light":"github-light","dark":"dark-plus"}}
const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');
const app = express();
// Auth middleware
app.use(async (req, res, next) => {
const apiKey = req.headers['x-api-key'];
const customer = await validateApiKey(apiKey);
if (!customer) return res.status(401).json({ error: 'Invalid API key' });
req.customer = customer;
next();
});
// Usage metering
app.use((req, res, next) => {
const start = Date.now();
res.on('finish', () => {
recordUsage(req.customer.id, req.path, Date.now() - start);
});
next();
});
// Route to Gateway
app.use('/', createProxyMiddleware({
target: 'http://localhost:8935',
changeOrigin: true,
}));
app.listen(3000);
```
## Authentication
The simplest approach. Issue each customer a unique API key, validate it on every request, and store it hashed in a database.
```icon="terminal" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Request: X-Api-Key: sk_live_abc123...
Middleware: hash(sk_live_abc123...) -> look up in DB -> get customer record
```
API keys are easy to rotate without requiring customers to re-authenticate. They are appropriate for server-to-server integrations where the key is stored securely on the client side.
For consumer-facing applications where users authenticate via OAuth (Google, GitHub, etc.), issue short-lived JWT access tokens after authentication. Validate the JWT signature on every request without a database lookup.
```javascript icon="terminal" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
const jwt = require('jsonwebtoken');
function verifyToken(req, res, next) {
const token = req.headers.authorization?.replace('Bearer ', '');
try {
const payload = jwt.verify(token, process.env.JWT_SECRET);
req.customer = { id: payload.sub, tier: payload.tier };
next();
} catch {
res.status(401).json({ error: 'Invalid token' });
}
}
```
JWTs are appropriate for Gateways that serve end-users directly through a web or mobile application.
For Live AI (video-to-video) workloads, go-livepeer supports webhook-based RTMP authentication. The Gateway calls the webhook when an RTMP stream connects, and the webhook returns allow or deny.
```bash icon="terminal" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
-liveAIAuthWebhookUrl https://auth-service.example.com/rtmp-auth
-liveAIAuthApiKey
```
The webhook receives the stream key and can validate it against a customer database before allowing the stream through.
## Settings
### Rate Limiting
Rate limiting protects the Gateway from abuse and enforces fair usage across customers. Implement it in the middleware layer, not in go-livepeer.
The appropriate limits depend on the Orchestrator pool capacity and customer agreements. A reasonable starting point for an AI inference Gateway:
TierRequests per minuteConcurrent sessionsFree101Standard605Premium30020
Track limits in Redis for shared-memory rate limiting across multiple middleware instances:
```javascript icon="code" Redis theme={"theme":{"light":"github-light","dark":"dark-plus"}}
const rateLimit = require('express-rate-limit');
const RedisStore = require('rate-limit-redis');
const limiter = rateLimit({
windowMs: 60 * 1000,
max: (req) => req.customer.tier === 'premium' ? 300 : 60,
store: new RedisStore({ client: redisClient }),
keyGenerator: (req) => req.customer.id,
});
```
### Custom Routing
The most powerful use of middleware is routing different customers or job types to different Gateway instances, each configured with a distinct Orchestrator pool.
```javascript icon="terminal" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
// Route premium customers to high-performance Orchestrators
app.use('/process', async (req, res) => {
const target = req.customer.tier === 'premium'
? 'http://localhost:8935' // Gateway A: premium orch list, high price cap
: 'http://localhost:8936'; // Gateway B: standard orch list, lower price cap
proxyRequest(req, res, target);
});
```
This pattern, combined with , allows delivering different SLA levels from one middleware deployment.
### Usage Metering
go-livepeer does not track how much each customer has consumed - it only tracks what the Gateway owes Orchestrators. Customer usage must be metered in the middleware.
What to track per request:
* Customer ID
* Pipeline type (text-to-image, video transcoding, etc.)
* Input parameters (resolution, number of outputs, duration)
* Response time
* Success or failure
Store events in a time-series database or event queue, then aggregate for invoicing.
```javascript icon="code" Usage Metering theme={"theme":{"light":"github-light","dark":"dark-plus"}}
res.on('finish', () => {
usageQueue.push({
customerId: req.customer.id,
pipeline: extractPipeline(req.path),
durationMs: Date.now() - startTime,
statusCode: res.statusCode,
timestamp: new Date().toISOString(),
});
});
```
## Middleware vs Clearinghouse
A clearinghouse combines middleware concerns with payment pipeline concerns into a single managed service:
ConcernCustom middlewareClearinghouseCustomer authOperator managesManagedUsage billingOperator managesManagedETH key custodyNot in scopeManagedOrchestrator paymentsgo-livepeer handlesManagedMulti-tenant isolationOperator buildsBuilt inFiat settlementOperator buildsBuilt in
To avoid building the payment side, see for delegating ETH custody and payment accounting while keeping custom middleware for auth and routing.
## Related Pages
Scale middleware and Gateway instances together.
Make the Gateway discoverable to external developers.
Delegate payment handling to a clearinghouse service.
Tiering, scoring, and failover for Orchestrator selection.