> ## Documentation Index
> Fetch the complete documentation index at: https://docs.livepeer.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Add AI to a Video Node

> Add AI inference to an existing video orchestrator. Covers the exact configuration delta, VRAM headroom check, and dual earnings verification.

export const BorderedBox = ({children, variant = "default", padding = "var(--lp-spacing-4)", borderRadius = "var(--lp-spacing-px-8)", margin = "", accentBar = "", style = {}, className = "", ...rest}) => {
  const variants = {
    default: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "var(--lp-color-bg-card)"
    },
    accent: {
      border: "1px solid var(--lp-color-accent)",
      backgroundColor: "var(--lp-color-bg-card)"
    },
    muted: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "transparent"
    }
  };
  const accentBarColors = {
    accent: "var(--lp-color-accent)",
    positive: "var(--green-9)"
  };
  return <div data-docs-bordered-box="" data-accent-bar={accentBarColors[accentBar] ? "" : undefined} className={className} style={{
    ...variants[variant],
    padding: padding,
    borderRadius: borderRadius,
    ...margin ? {
      margin
    } : {},
    ...accentBarColors[accentBar] ? {
      position: "relative",
      '--accent-bar-color': accentBarColors[accentBar]
    } : {},
    ...style
  }} {...rest}>
      {children}
    </div>;
};

export const StyledStep = ({title, icon, titleSize = 'h3', iconColor = null, titleColor = null, children, className = '', style = {}, ...rest}) => {
  const styledTitle = titleColor ? <span style={{
    color: titleColor
  }}>{title}</span> : title;
  return <Step title={styledTitle} icon={icon} iconColor={iconColor || undefined} titleSize={titleSize} className={className} style={style} {...rest}>
      {children}
    </Step>;
};

export const StyledSteps = ({children, iconColor, titleColor, lineColor, iconSize = '24px', className = '', style = {}, ...rest}) => {
  const resolvedIconColor = iconColor || 'var(--accent-dark, #18794E)';
  const resolvedTitleColor = titleColor || 'var(--lp-color-accent)';
  const resolvedLineColor = lineColor || 'var(--lp-color-accent)';
  return <div className={['docs-styled-steps', className].filter(Boolean).join(' ')} style={style} {...rest}>
      <style>{`
        .docs-styled-steps .steps > div > div.absolute > div {
          background-color: ${resolvedIconColor};
        }
        .docs-styled-steps .steps > div > div.w-full > p {
          color: ${resolvedTitleColor};
        }
        .docs-styled-steps .steps > div > div.absolute.w-px {
          background-color: ${resolvedLineColor};
        }
        .docs-styled-steps .steps > div:last-child > div.absolute.w-px::after {
          content: '';
          position: absolute;
          bottom: 0;
          left: 50%;
          transform: translateX(-50%);
          width: 6px;
          height: 6px;
          background-color: ${resolvedLineColor};
          transform: translateX(-50%) rotate(45deg);
        }
      `}</style>
      <div>
        <Steps>{children}</Steps>
      </div>
    </div>;
};

export const TableCell = ({children, align = "left", header = false, style = {}, className = "", ...rest}) => {
  const Component = header ? "th" : "td";
  return <Component className={className} style={{
    padding: "0.75rem 1rem",
    textAlign: align,
    border: header ? "none" : "1px solid var(--lp-color-border-default)",
    ...style
  }} {...rest}>
      {children}
    </Component>;
};

export const TableRow = ({children, header = false, hover = false, style = {}, className = "", ...rest}) => {
  const rowId = `table-row-${Math.random().toString(36).substr(2, 9)}`;
  return <>
      {hover && <style>{`
          #${rowId}:hover {
            background-color: var(--lp-color-bg-card);
          }
        `}</style>}
      <tr id={rowId} className={className} style={{
    ...header && ({
      backgroundColor: "var(--lp-color-accent-strong)",
      color: "var(--lp-color-on-accent)",
      fontWeight: "bold"
    }),
    ...style
  }} {...rest}>
        {children}
      </tr>
    </>;
};

export const StyledTable = ({children, variant = "default", style = {}, className = "", ...rest}) => {
  const wrapperVariants = {
    default: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "var(--lp-color-bg-card)",
      overflow: "hidden"
    },
    bordered: {
      border: "2px solid var(--lp-color-accent)",
      backgroundColor: "var(--lp-color-bg-page)",
      overflow: "hidden"
    },
    minimal: {
      border: "none",
      backgroundColor: "transparent",
      overflow: "visible"
    }
  };
  return <div data-docs-styled-table-shell className={className} style={{
    width: "100%",
    padding: 0,
    margin: 0,
    ...wrapperVariants[variant],
    ...style
  }} {...rest}>
      <table data-docs-styled-table style={{
    width: "100%",
    borderCollapse: "collapse",
    borderSpacing: 0,
    margin: 0,
    backgroundColor: "transparent"
  }}>
        {children}
      </table>
    </div>;
};

export const LinkArrow = ({href, label, description, newline = true, borderColor, className = '', style = {}, ...rest}) => {
  const linkArrowStyle = {
    display: 'inline-flex',
    alignItems: 'center',
    justifyContent: 'center',
    gap: "var(--lp-spacing-1)",
    width: 'fit-content',
    ...borderColor && ({
      borderColor
    })
  };
  return <span className={className} style={style} {...rest}>
      {newline && <br />}
      <span style={linkArrowStyle}>
        <a href={href} target="_blank" rel="noopener noreferrer">
          {label}
        </a>
        <Icon icon="arrow-up-right" size={14} color="var(--lp-color-accent)" />
      </span>
      {description && description}
      {description && <div style={{
    height: "var(--lp-spacing-3)"
  }} />}
    </span>;
};

export const CustomDivider = ({color = "var(--lp-color-border-default)", middleText = "", spacing = "default", style = {}, className = "", ...rest}) => {
  const spacingPresets = {
    default: {
      margin: "24px 0"
    },
    overlap: {
      margin: "-1rem 0 -1rem 0"
    },
    tight: {
      margin: "0 0 -1rem 0"
    },
    section: {
      margin: "0 0 -2rem 0"
    },
    sectionOverlap: {
      margin: "-1rem 0 -2rem 0"
    },
    deepOverlap: {
      margin: "-1rem 0 -1.5rem 0"
    }
  };
  const spacingStyle = spacingPresets[spacing] || spacingPresets.default;
  return <div role="separator" aria-orientation="horizontal" className={className} style={{
    display: "flex",
    alignItems: "center",
    ...spacingStyle,
    fontSize: style?.fontSize || "16px",
    height: "fit-content",
    ...style
  }} {...rest}>
      <span style={{
    marginRight: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
      </span>
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      {middleText && <>
          <Icon icon="circle" size={2} />
          <span style={{
    margin: "0 8px",
    fontWeight: "bold",
    color: color,
    opacity: 0.7
  }}>
            {middleText}
          </span>
          <Icon icon="circle" size={2} />
        </>}
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      <span style={{
    marginLeft: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <span style={{
    display: "inline-block",
    transform: "scaleX(-1)"
  }}>
          <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
        </span>
      </span>
    </div>;
};

<Tip>
  Adding AI inference to a running video Orchestrator is an additive change. On-chain registration, staking, reward calling, and all transcoding flags stay exactly as they are. Three new flags and one new file are all that change.
</Tip>

***

This tutorial adds AI inference capability to an existing go-livepeer video Orchestrator. Estimated time: **1 hour** plus model download time (6 to 10 GB depending on the model chosen).

**Prerequisites:** A working video Orchestrator already running on Arbitrum One mainnet. Fresh nodes should start with <LinkArrow href="/v2/orchestrators/guides/tutorials/zero-to-first-reward" label="Zero to First Reward" newline={false} />.

<CustomDivider />

## What changes and what stays the same

Video transcoding uses NVENC and NVDEC, the fixed-function hardware blocks built into NVIDIA GPUs. AI inference uses CUDA compute cores. These are separate hardware resources on the same die. Adding AI workloads leaves video transcoding capacity intact and preserves available NVENC sessions.

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>Configuration item</TableCell>
      <TableCell header>Status when adding AI</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>On-chain registration</TableCell>
      <TableCell>**Unchanged** - no re-registration needed</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Staking and LPT bond</TableCell>
      <TableCell>**Unchanged** - existing stake carries over</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Reward Cut and Fee Cut</TableCell>
      <TableCell>**Unchanged** - commission settings stay the same</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`-pricePerUnit` (video)</TableCell>
      <TableCell>**Unchanged** - video pricing is separate from AI pricing</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`-serviceAddr`</TableCell>
      <TableCell>**Unchanged** - same port (8935)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`-maxSessions` (video)</TableCell>
      <TableCell>**Unchanged** - applies to video sessions only</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`-aiWorker` flag</TableCell>
      <TableCell>**NEW** - enables the AI inference worker</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`-aiModels` flag</TableCell>
      <TableCell>**NEW** - path to `aiModels.json`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`-aiModelsDir` flag</TableCell>
      <TableCell>**NEW** - path to model weights on host</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>`aiModels.json` file</TableCell>
      <TableCell>**NEW** - declares pipelines, models, and AI pricing</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Docker socket mount</TableCell>
      <TableCell>**NEW on many video-only nodes** - required for AI Runner containers</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

<CustomDivider />

## VRAM headroom check

AI models require VRAM. Video transcoding uses NVENC/NVDEC silicon and consumes negligible VRAM. The available VRAM for AI is the total GPU VRAM minus a small system overhead.

Check current VRAM state while the video node is running:

```bash icon="terminal" filename="check-vram" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
nvidia-smi --query-gpu=name,memory.total,memory.free,memory.used \
  --format=csv,noheader,nounits
```

Example output:

```text icon="terminal" title="Example VRAM output" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
NVIDIA GeForce RTX 4090, 24564, 22100, 2464
```

Values are in MB. In this example: 24 GB total, 22 GB free, 2.4 GB used by the running video node.

Choose a model that fits within the free VRAM, with at least 2 GB headroom:

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>GPU VRAM</TableCell>
      <TableCell header>Viable AI pipelines alongside video</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>24 GB (RTX 4090, A10G)</TableCell>
      <TableCell>`text-to-image`, `image-to-image`, `image-to-video`, `audio-to-text` (warm)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>16 GB (RTX 4080, A10)</TableCell>
      <TableCell>`text-to-image` (SDXL-Lightning, warm), `audio-to-text` (warm)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>12 GB (RTX 4070 Ti, RTX 3080)</TableCell>
      <TableCell>`audio-to-text` (warm), `image-to-text` (warm), `llm` via Ollama</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>8 GB (RTX 3070, RTX 2070)</TableCell>
      <TableCell>`audio-to-text` (warm), `image-to-text` (warm), `llm` via Ollama (8B quantised)</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

<CustomDivider />

## Step 1: Download model weights

Download the model before restarting the node. AI Runner containers mount the model directory at startup - weights must already be present.

```bash icon="terminal" filename="download-model" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker run --rm \
  -v ~/.lpData/models:/models \
  --gpus all \
  livepeer/ai-runner:latest \
  bash -c "PIPELINE=text-to-image MODEL_ID=ByteDance/SDXL-Lightning bash /app/dl_checkpoints.sh"
```

Replace `text-to-image` and `ByteDance/SDXL-Lightning` with your chosen pipeline and model. See the VRAM table above and check [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities) for current demand before choosing.

Download is approximately 6 GB for SDXL-Lightning. Verify:

```bash icon="terminal" filename="verify-download" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
ls -lh ~/.lpData/models/
```

<CustomDivider />

## Step 2: Create aiModels.json

```bash icon="terminal" filename="create-aimodels" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
cat > ~/.lpData/aiModels.json << 'EOF'
[
  {
    "pipeline": "text-to-image",
    "model_id": "ByteDance/SDXL-Lightning",
    "price_per_unit": 4768371,
    "warm": true
  }
]
EOF
```

Set `price_per_unit` from current market rates at [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities). Keep it at or below current Gateway caps so the node remains routable.

<CustomDivider />

## Step 3: Add AI flags to the start command

Stop the running video node:

```bash icon="terminal" filename="stop-node" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker stop livepeer-orchestrator
docker rm livepeer-orchestrator
```

Restart with the three AI flags added. The existing video flags are unchanged:

```bash icon="terminal" filename="restart-with-ai" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker run -d \
  --name livepeer-orchestrator \
  -v ~/.lpData/:/root/.lpData/ \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --network host \
  --gpus all \
  livepeer/go-livepeer:latest \
  -network arbitrum-one-mainnet \
  -ethUrl https://arb-mainnet.g.alchemy.com/v2/YOUR_API_KEY \
  -orchestrator \
  -transcoder \
  -nvidia 0 \
  -maxSessions 10 \
  -pricePerUnit 1000 \
  -serviceAddr YOUR_PUBLIC_IP:8935 \
  -aiWorker \
  -aiModels /root/.lpData/aiModels.json \
  -aiModelsDir /root/.lpData/models
```

The restart command includes the `-v /var/run/docker.sock:/var/run/docker.sock` mount required for Docker-out-of-Docker. Go-livepeer uses the Docker daemon to start AI Runner containers through this socket.

<Warning>
  Use the host path for `-aiModelsDir` (`~/.lpData/models`). Docker mounts that directory into each AI Runner container it creates.
</Warning>

<CustomDivider />

## Step 4: Verify both workloads

<StyledSteps>
  <StyledStep title="Verify the node started cleanly">
    ```bash icon="terminal" filename="check-startup" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    docker logs livepeer-orchestrator 2>&1 | grep -E "GPU|pipeline|warm|Transcoding|error" | head -20
    ```

    Expected lines:

    ```text icon="terminal" title="Expected dual-workload startup log" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    Transcoding on Nvidia GPU 0
    Starting AI worker
    Pipeline text-to-image started
    Warm model loaded: ByteDance/SDXL-Lightning
    ```
  </StyledStep>

  <StyledStep title="Verify video transcoding still works">
    Check the node is still registered as a video Orchestrator:

    ```bash icon="terminal" filename="check-video" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    curl -k https://YOUR_PUBLIC_IP:8935/status
    ```

    Expected: a JSON response including transcoding capabilities. A timeout means port 8935 is not reachable (firewall or serviceAddr misconfiguration).

    Also confirm on [Livepeer Explorer](https://explorer.livepeer.org/orchestrators) that the node still appears active and the service URI matches.
  </StyledStep>

  <StyledStep title="Verify AI capabilities registered locally">
    ```bash icon="terminal" filename="check-ai-local" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    curl http://localhost:7935/getNetworkCapabilities | python3 -m json.tool
    ```

    Expected: a JSON response with a `pipelines` array including `text-to-image` with the model listed.
  </StyledStep>

  <StyledStep title="Verify AI capabilities registered on the network">
    Open [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities) and search for the node's Ethereum address. The `text-to-image` pipeline should appear with status **Warm** within 2 to 5 minutes.

    If the pipeline does not appear, inspect AI Runner containers:

    ```bash icon="terminal" filename="check-containers" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    docker ps --filter name=livepeer
    docker logs <ai-runner-container-name> --tail 30
    ```
  </StyledStep>

  <StyledStep title="Test AI inference">
    Send a local test inference to confirm the AI pipeline is serving:

    ```bash icon="terminal" filename="test-inference" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    curl -X POST http://localhost:8935/text-to-image \
      -H "Content-Type: application/json" \
      -d '{"model_id":"ByteDance/SDXL-Lightning","prompt":"a lighthouse at sunset","width":512,"height":512,"num_inference_steps":4}' \
      -o test-ai.png
    ```

    ```bash icon="terminal" filename="check-inference" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    file test-ai.png
    ```

    Expected: `test-ai.png: PNG image data` with a non-zero file size.
  </StyledStep>
</StyledSteps>

<CustomDivider />

## What happened

The node now operates in **dual-workload configuration**: video transcoding and AI inference run simultaneously on the same go-livepeer process.

Video transcoding routes through NVENC/NVDEC, dedicated silicon with negligible VRAM demand. AI inference routes through CUDA compute cores with model weights loaded in VRAM. The two workloads use separate hardware paths.

**Income now flows from two sources:**

* **ETH from video transcoding** - probabilistic micropayment tickets per transcoded segment
* **ETH from AI inference** - probabilistic micropayment tickets per completed inference job

Both streams use the same wallet, the same `Reward()` call for LPT inflation, and the same on-chain registration. The AI income stream uses the existing operator identity.

<CustomDivider />

## Related pages

<CardGroup cols={2}>
  <Card title="Capacity Planning" icon="gauge" href="/v2/orchestrators/guides/config-and-optimisation/capacity-planning" arrow horizontal>
    Detailed VRAM budgeting and the benchmarking process for setting maxSessions correctly.
  </Card>

  <Card title="AI Model Management" icon="sliders" href="/v2/orchestrators/guides/config-and-optimisation/ai-model-management" arrow horizontal>
    Warm vs cold strategy, model rotation by demand, and optimisation flags.
  </Card>

  <Card title="Dual Mode Configuration" icon="layer-group" href="/v2/orchestrators/guides/deployment-details/dual-mode-configuration" arrow horizontal>
    Full reference for dual-workload configuration including multi-GPU assignments.
  </Card>

  <Card title="Pricing Strategy" icon="tag" href="/v2/orchestrators/guides/config-and-optimisation/pricing-strategy" arrow horizontal>
    Set AI pipeline pricing in aiModels.json: per-pipeline, per-model, and USD notation.
  </Card>
</CardGroup>