} ; }; export const CustomDivider = ({color = "var(--lp-color-border-default)", middleText = "", spacing = "default", style = {}, className = "", ...rest}) => { const spacingPresets = { default: { margin: "24px 0" }, overlap: { margin: "-1rem 0 -1rem 0" }, tight: { margin: "0 0 -1rem 0" }, section: { margin: "0 0 -2rem 0" }, sectionOverlap: { margin: "-1rem 0 -2rem 0" }, deepOverlap: { margin: "-1rem 0 -1.5rem 0" } }; const spacingStyle = spacingPresets[spacing] || spacingPresets.default; return

{middleText && <> {middleText} }

; }; AI inference routing follows capability and price. A node with a 24 GB GPU and a warm model enters the routing pool as soon as registration propagates, without waiting for active-set entry or a large LPT bond. *** This tutorial gets an Orchestrator earning from AI inference in under two hours. One GPU, one warm diffusion model, no Active Set membership needed. Estimated time: **1.5 to 2.5 hours** (most of this is model download time - approximately 6 GB). **You will verify:** * go-livepeer starts with the AI worker enabled * The warm model registers at `tools.livepeer.cloud/ai/network-capabilities` * A local test inference returns a result * The node is live on the Livepeer AI network ## Prerequisites Requirement Minimum Verify with Linux server Ubuntu 22.04+ `lsb_release -a` NVIDIA GPU 24 GB VRAM for diffusion models (8 GB for LLM or audio pipelines) `nvidia-smi` - shows GPU and VRAM NVIDIA drivers CUDA 12.0 compatible (`525.60.13`+) `nvidia-smi` shows driver version Docker + NVIDIA Container Toolkit Docker Engine 24+ `docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi` Arbitrum One RPC endpoint Free tier (Alchemy / Infura) Must return `"result":"0xa4b1"` for `eth_chainId` ETH on Arbitrum One \~0.005 ETH for AI registration gas Check your wallet on [Arbiscan.io](https://arbiscan.io) Disk space 20 GB free on the model directory partition `df -h ~/.lpData/` ## Step 1: Install go-livepeer and pull the AI Runner ```bash icon="terminal" filename="pull-livepeer" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker pull livepeer/go-livepeer:latest ``` Verify: ```bash icon="terminal" filename="verify-livepeer" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run --rm livepeer/go-livepeer:latest livepeer --version ``` go-livepeer uses Docker-out-of-Docker to manage AI Runner containers. Verify the container runtime can see the GPU: ```bash icon="terminal" filename="verify-gpu" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi ``` Expected: the `nvidia-smi` table showing your GPU, driver version, and VRAM. Re-install the NVIDIA Container Toolkit until this command succeeds. ```bash icon="terminal" filename="setup-dirs" theme={"theme":{"light":"github-light","dark":"dark-plus"}} mkdir -p ~/.lpData/models docker volume create livepeer-ai-data ``` ## Step 2: Write aiModels.json `aiModels.json` declares which AI pipelines and models the node serves, and at what price. For a 24 GB GPU, `ByteDance/SDXL-Lightning` is the recommended starting model for `text-to-image`. It is fast (4 inference steps), in high demand, and approximately 6 GB on disk. Check current network demand at [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities) before deciding. The models with the most Gateway registrations receive the most routing traffic. ```bash icon="terminal" filename="create-aimodels" theme={"theme":{"light":"github-light","dark":"dark-plus"}} cat > ~/.lpData/aiModels.json << 'EOF' [ { "pipeline": "text-to-image", "model_id": "ByteDance/SDXL-Lightning", "price_per_unit": 4768371, "warm": true } ] EOF ``` Field reference: * `pipeline` - the pipeline type (must match exactly) * `model_id` - the HuggingFace model ID (case-sensitive, including the org prefix) * `price_per_unit` - price in wei per output pixel (`4768371` ≈ \$0.0005 per megapixel at late-2025 ETH rates) * `warm: true` - load the model into VRAM at startup for instant first-response Check current competitive pricing at [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities) before finalising `price_per_unit`. Gateways skip Orchestrators priced above their cap. ## Step 3: Pre-download the model Download model weights before starting the node. The warm model must be available at startup or the AI worker will attempt to download it on-demand, which delays the first inference. ```bash icon="terminal" filename="download-model" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run --rm \ -v ~/.lpData/models:/models \ --gpus all \ livepeer/ai-runner:latest \ bash -c "PIPELINE=text-to-image MODEL_ID=ByteDance/SDXL-Lightning bash /app/dl_checkpoints.sh" ``` This downloads approximately 6 GB. On a 100 Mbps connection, expect 8 to 12 minutes. Watch for the download progress output. Verify the model files are present: ```bash icon="terminal" filename="verify-model" theme={"theme":{"light":"github-light","dark":"dark-plus"}} ls -lh ~/.lpData/models/ ``` A non-empty directory confirms the download completed. ## Step 4: Start go-livepeer with AI worker ```bash icon="terminal" filename="start-ai-node" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker run -d \ --name livepeer-ai-orchestrator \ -v ~/.lpData/:/root/.lpData/ \ -v /var/run/docker.sock:/var/run/docker.sock \ --network host \ --gpus all \ livepeer/go-livepeer:latest \ -network arbitrum-one-mainnet \ -ethUrl https://arb-mainnet.g.alchemy.com/v2/YOUR_API_KEY \ -orchestrator \ -transcoder \ -nvidia 0 \ -pricePerUnit 1000 \ -serviceAddr YOUR_PUBLIC_IP:8935 \ -aiWorker \ -aiModels /root/.lpData/aiModels.json \ -aiModelsDir /root/.lpData/models ``` Replace `YOUR_API_KEY` and `YOUR_PUBLIC_IP`. The three AI-specific flags: * `-aiWorker` - enables the AI inference worker * `-aiModels` - path to `aiModels.json` inside the container * `-aiModelsDir` - path to model weights on the **host machine** (Docker mounts this into AI Runner containers) `-aiModelsDir` must be a host path, not a path inside the container. Docker uses this path to mount model files into AI Runner containers via Docker-out-of-Docker. A container-local path will not resolve correctly. ```bash icon="terminal" filename="watch-startup" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker logs -f livepeer-ai-orchestrator 2>&1 | grep -i "ai-runner\|pipeline\|warm\|model\|container" ``` Expected lines (may take 2 to 5 minutes while the model loads into VRAM): ```text icon="terminal" title="Expected startup lines" theme={"theme":{"light":"github-light","dark":"dark-plus"}} Starting AI worker Pulling livepeer/ai-runner container Pipeline text-to-image started Warm model loaded: ByteDance/SDXL-Lightning ``` A missing `Warm model loaded` line after 10 minutes points to a startup problem. Check for errors: ```bash icon="terminal" filename="check-errors" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker logs livepeer-ai-orchestrator 2>&1 | grep -i "error\|fail\|oom" | tail -20 ``` ## Step 5: Register AI capabilities Existing video Orchestrators can keep their current on-chain registration. New AI-only nodes should run `livepeer_cli` to register: ```bash icon="terminal" filename="activate" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker exec -it livepeer-ai-orchestrator livepeer_cli ``` Select **"Invoke multi-step become an Orchestrator"**. Minimal LPT self-delegation is sufficient for AI-only operation - the Active Set threshold applies to video transcoding routing, not AI routing. ```bash icon="terminal" filename="check-capabilities" theme={"theme":{"light":"github-light","dark":"dark-plus"}} curl http://localhost:7935/getNetworkCapabilities | python3 -m json.tool ``` Expected: a JSON response with a `pipelines` array that includes `text-to-image` with your model listed. Open [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities). Search by your Ethereum address or service URI. Your node should appear under `text-to-image` with status **Warm** within 2 to 5 minutes of the AI worker reporting the model loaded. A missing entry after 10 minutes points to a registration or container problem. Inspect container status: ```bash icon="terminal" filename="check-containers" theme={"theme":{"light":"github-light","dark":"dark-plus"}} docker ps --filter name=livepeer ``` All containers (`livepeer-ai-orchestrator` and the AI Runner container) should show `Up`. A restarting AI Runner container indicates a model load failure. ## Step 6: Test a local inference Send a test inference request directly to the Orchestrator to confirm the pipeline is serving: ```bash icon="terminal" filename="test-inference" theme={"theme":{"light":"github-light","dark":"dark-plus"}} curl -X POST http://localhost:8935/text-to-image \ -H "Content-Type: application/json" \ -d '{ "model_id": "ByteDance/SDXL-Lightning", "prompt": "a blue mountain lake at sunrise", "width": 512, "height": 512, "num_inference_steps": 4 }' \ -o test-output.png ``` Verify the response: ```bash icon="terminal" filename="check-output" theme={"theme":{"light":"github-light","dark":"dark-plus"}} file test-output.png ls -lh test-output.png ``` Expected: `test-output.png: PNG image data` with a non-zero file size. The PNG is the generated image. The first inference after startup is slower than later requests because the CUDA kernels warm up. That is normal. From the second request onward, SDXL-Lightning at 512×512 should complete in under 5 seconds. ## What happened The node completed the AI inference path: 1. **go-livepeer started with `-aiWorker`** - it read `aiModels.json`, pulled the `livepeer/ai-runner` container, mounted the model weights from the host, and loaded `ByteDance/SDXL-Lightning` into GPU VRAM as a warm model. 2. **Capability advertisement** - go-livepeer registered the `text-to-image` pipeline and warm model status on-chain (via the AI Service Registry) and announced it to the network. Gateways that query for `text-to-image` capability now see this node in their routing pool. 3. **The test inference** travelled from the local HTTP call to the AI worker, to the AI Runner container, through SDXL-Lightning's inference pipeline, and back as a PNG. In production, this same path is triggered by a Gateway routing an inference request. 4. **Earning begins** when a Gateway routes a job to this node. Each completed job sends a PM ticket worth approximately `price_per_unit × pixels_in_output` wei. See for how winning tickets translate to ETH. **AI routing ignores active-set rank.** It checks capability advertisement and price instead. A new node with a warm model and a competitive price starts receiving AI jobs on the same day it registers. ## Related pages Add AI to an existing video Orchestrator without touching the video configuration. Which models are receiving the most Gateway requests and at what price tiers. Set AI pricing in aiModels.json: wei vs USD notation and competitive positioning. Warm vs cold strategy, VRAM allocation, and optimisation flags.