POST
/
llm
import { Livepeer } from "@livepeer/ai";

const livepeer = new Livepeer({
  httpBearer: "<YOUR_BEARER_TOKEN_HERE>",
});

async function run() {
  const result = await livepeer.generate.llm({
    messages: [

    ],
    model: "",
    temperature: 0.7,
    maxTokens: 256,
    topP: 1,
    topK: -1,
    stream: false,
  });

  // Handle the result
  console.log(result);
}

run();
{
  "id": "<string>",
  "model": "<string>",
  "created": 123,
  "usage": {
    "prompt_tokens": 123,
    "completion_tokens": 123,
    "total_tokens": 123
  },
  "choices": [
    {
      "index": 123,
      "finish_reason": "",
      "delta": {
        "role": "<string>",
        "content": "<string>"
      },
      "message": {
        "role": "<string>",
        "content": "<string>"
      }
    }
  ]
}

The LLM pipeline is OpenAI API-compatible but does not implement all features of the OpenAI API.

The default Gateway used in this guide is the public Livepeer.cloud Gateway. It is free to use but not intended for production-ready applications. For production-ready applications, consider using the Livepeer Studio Gateway, which requires an API token. Alternatively, you can set up your own Gateway node or partner with one via the ai-video channel on Discord.

​
Streaming Responses

Ensure your client supports SSE and processes each data: line as it arrives.

By default, the /llm endpoint returns a single JSON response in the OpenAI chat/completions format, as shown in the sidebar.

To receive responses token-by-token, set "stream": true in the request body. The server will then use Server-Sent Events (SSE) to stream output in real time.

Each streamed chunk will look like:

data: {
  "choices": [
    {
      "delta": {
        "content": "...token...",
        "role": "assistant"
      },
      "finish_reason": null
    }
  ]
}

The final chunk will have empty content and "finish_reason": "stop":

data: {
  "choices": [
    {
      "delta": {
        "content": "",
        "role": "assistant"
      },
      "finish_reason": "stop"
    }
  ]
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
​
messages
object[]
required
​
model
string
default:
​
temperature
number
default:0.7
​
max_tokens
integer
default:256
​
top_p
number
default:1
​
top_k
integer
default:-1
​
stream
boolean
default:false

Response

200
application/json
Successful Response
​
id
string
required
​
model
string
required
​
created
integer
required
​
usage
object
required
​
choices
object[]
required

Was this page helpful?

Suggest editsRaise issue