> ## Documentation Index
> Fetch the complete documentation index at: https://docs.livepeer.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Full AI Pipeline Tutorial

> End-to-end tutorial: gateway routes an inference request to an orchestrator, the AI runner processes it, and the result returns through the full pipeline. Covers off-chain local setup with a HuggingFace model.

export const StyledStep = ({title, icon, titleSize = 'h3', iconColor = null, titleColor = null, children, className = '', style = {}, ...rest}) => {
  const styledTitle = titleColor ? <span style={{
    color: titleColor
  }}>{title}</span> : title;
  return <Step title={styledTitle} icon={icon} iconColor={iconColor || undefined} titleSize={titleSize} className={className} style={style} {...rest}>
      {children}
    </Step>;
};

export const StyledSteps = ({children, iconColor, titleColor, lineColor, iconSize = '24px', className = '', style = {}, ...rest}) => {
  const resolvedIconColor = iconColor || 'var(--accent-dark, #18794E)';
  const resolvedTitleColor = titleColor || 'var(--lp-color-accent)';
  const resolvedLineColor = lineColor || 'var(--lp-color-accent)';
  return <div className={['docs-styled-steps', className].filter(Boolean).join(' ')} style={style} {...rest}>
      <style>{`
        .docs-styled-steps .steps > div > div.absolute > div {
          background-color: ${resolvedIconColor};
        }
        .docs-styled-steps .steps > div > div.w-full > p {
          color: ${resolvedTitleColor};
        }
        .docs-styled-steps .steps > div > div.absolute.w-px {
          background-color: ${resolvedLineColor};
        }
        .docs-styled-steps .steps > div:last-child > div.absolute.w-px::after {
          content: '';
          position: absolute;
          bottom: 0;
          left: 50%;
          transform: translateX(-50%);
          width: 6px;
          height: 6px;
          background-color: ${resolvedLineColor};
          transform: translateX(-50%) rotate(45deg);
        }
      `}</style>
      <div>
        <Steps>{children}</Steps>
      </div>
    </div>;
};

export const TableCell = ({children, align = "left", header = false, style = {}, className = "", ...rest}) => {
  const Component = header ? "th" : "td";
  return <Component className={className} style={{
    padding: "0.75rem 1rem",
    textAlign: align,
    border: header ? "none" : "1px solid var(--lp-color-border-default)",
    ...style
  }} {...rest}>
      {children}
    </Component>;
};

export const TableRow = ({children, header = false, hover = false, style = {}, className = "", ...rest}) => {
  const rowId = `table-row-${Math.random().toString(36).substr(2, 9)}`;
  return <>
      {hover && <style>{`
          #${rowId}:hover {
            background-color: var(--lp-color-bg-card);
          }
        `}</style>}
      <tr id={rowId} className={className} style={{
    ...header && ({
      backgroundColor: "var(--lp-color-accent-strong)",
      color: "var(--lp-color-on-accent)",
      fontWeight: "bold"
    }),
    ...style
  }} {...rest}>
        {children}
      </tr>
    </>;
};

export const StyledTable = ({children, variant = "default", style = {}, className = "", ...rest}) => {
  const wrapperVariants = {
    default: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "var(--lp-color-bg-card)",
      overflow: "hidden"
    },
    bordered: {
      border: "2px solid var(--lp-color-accent)",
      backgroundColor: "var(--lp-color-bg-page)",
      overflow: "hidden"
    },
    minimal: {
      border: "none",
      backgroundColor: "transparent",
      overflow: "visible"
    }
  };
  return <div data-docs-styled-table-shell className={className} style={{
    width: "100%",
    padding: 0,
    margin: 0,
    ...wrapperVariants[variant],
    ...style
  }} {...rest}>
      <table data-docs-styled-table style={{
    width: "100%",
    borderCollapse: "collapse",
    borderSpacing: 0,
    margin: 0,
    backgroundColor: "transparent"
  }}>
        {children}
      </table>
    </div>;
};

export const LinkArrow = ({href, label, description, newline = true, borderColor, className = '', style = {}, ...rest}) => {
  const linkArrowStyle = {
    display: 'inline-flex',
    alignItems: 'center',
    justifyContent: 'center',
    gap: "var(--lp-spacing-1)",
    width: 'fit-content',
    ...borderColor && ({
      borderColor
    })
  };
  return <span className={className} style={style} {...rest}>
      {newline && <br />}
      <span style={linkArrowStyle}>
        <a href={href} target="_blank" rel="noopener noreferrer">
          {label}
        </a>
        <Icon icon="arrow-up-right" size={14} color="var(--lp-color-accent)" />
      </span>
      {description && description}
      {description && <div style={{
    height: "var(--lp-spacing-3)"
  }} />}
    </span>;
};

export const CustomDivider = ({color = "var(--lp-color-border-default)", middleText = "", spacing = "default", style = {}, className = "", ...rest}) => {
  const spacingPresets = {
    default: {
      margin: "24px 0"
    },
    overlap: {
      margin: "-1rem 0 -1rem 0"
    },
    tight: {
      margin: "0 0 -1rem 0"
    },
    section: {
      margin: "0 0 -2rem 0"
    },
    sectionOverlap: {
      margin: "-1rem 0 -2rem 0"
    },
    deepOverlap: {
      margin: "-1rem 0 -1.5rem 0"
    }
  };
  const spacingStyle = spacingPresets[spacing] || spacingPresets.default;
  return <div role="separator" aria-orientation="horizontal" className={className} style={{
    display: "flex",
    alignItems: "center",
    ...spacingStyle,
    fontSize: style?.fontSize || "16px",
    height: "fit-content",
    ...style
  }} {...rest}>
      <span style={{
    marginRight: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
      </span>
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      {middleText && <>
          <Icon icon="circle" size={2} />
          <span style={{
    margin: "0 8px",
    fontWeight: "bold",
    color: color,
    opacity: 0.7
  }}>
            {middleText}
          </span>
          <Icon icon="circle" size={2} />
        </>}
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      <span style={{
    marginLeft: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <span style={{
    display: "inline-block",
    transform: "scaleX(-1)"
  }}>
          <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
        </span>
      </span>
    </div>;
};

<Tip>
  The Gateway handles routing and payment negotiation. The Orchestrator handles compute. Run both on one machine, off-chain, and watch a full inference request travel through both sides and return a result without a wallet or on-chain registration.
</Tip>

***

This tutorial runs a complete local AI inference pipeline: a Gateway receives a client request, routes it to a local Orchestrator, the Orchestrator processes it through an AI Runner container, and the result returns to the caller. Estimated time: **2 to 3 hours** (most of this is model download time).

**What you will verify:**

* The Gateway routes an inference request to the Orchestrator
* The Orchestrator processes it through the AI Runner
* The response returns through the Gateway to the caller
* Each step is visible in the respective logs

<CustomDivider />

## Pipeline architecture

```text icon="terminal" filename="architecture" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Client (curl)
      ↓ POST /text-to-image
Gateway (port 8936)
      ↓ routes job + PM ticket
Orchestrator (port 8935)
      ↓ dispatches to AI runner
AI runner container
      ↓ SDXL-Lightning inference on GPU
Orchestrator
      ↓ result + ticket evaluation
Gateway
      ↓ PNG response
Client
```

The Gateway and Orchestrator run as separate processes. In production, they run on separate machines. This tutorial runs both locally to make the log trace visible end-to-end.

<CustomDivider />

## Prerequisites

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>Requirement</TableCell>
      <TableCell header>Notes</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>Linux with NVIDIA GPU (24 GB VRAM)</TableCell>
      <TableCell>24 GB needed for SDXL-Lightning. See AI Earning Quickstart for 8 GB alternatives.</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Docker + NVIDIA Container Toolkit</TableCell>
      <TableCell>Verify: `docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>go-livepeer Docker image</TableCell>
      <TableCell>`docker pull livepeer/go-livepeer:latest`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Ports 7935, 7936, 8935, 8936 free</TableCell>
      <TableCell>`ss -tlnp | grep -E ':793[56]|:893[56]'` (no output = free)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>20 GB free disk space</TableCell>
      <TableCell>For model weights</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

No ETH wallet, Arbitrum RPC, or on-chain registration required. This tutorial runs off-chain.

<CustomDivider />

## Step 1: Download the model

Download the model weights before starting either process. Both the Orchestrator and AI Runner need the weights present at startup.

```bash icon="terminal" filename="create-dirs" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
mkdir -p ~/.lpData/models ~/.lpData-gateway
```

```bash icon="terminal" filename="download-model" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker run --rm \
  -v ~/.lpData/models:/models \
  --gpus all \
  livepeer/ai-runner:latest \
  bash -c "PIPELINE=text-to-image MODEL_ID=ByteDance/SDXL-Lightning bash /app/dl_checkpoints.sh"
```

This downloads approximately 6 GB. Watch the download output and wait for completion.

Verify:

```bash icon="terminal" filename="verify-model" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
ls -lh ~/.lpData/models/
```

<CustomDivider />

## Step 2: Write aiModels.json

```bash icon="terminal" filename="write-aimodels" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
cat > ~/.lpData/aiModels.json << 'EOF'
[
  {
    "pipeline": "text-to-image",
    "model_id": "ByteDance/SDXL-Lightning",
    "price_per_unit": 4768371,
    "warm": true
  }
]
EOF
```

`price_per_unit` sets the Orchestrator's sell-side price. The Gateway's buy-side cap must be at or above this value for the job to route. In Step 4 the Gateway is started with no explicit price cap, so it accepts any price.

<CustomDivider />

## Step 3: Start the Orchestrator

In a terminal, start the Orchestrator in off-chain mode with the AI worker:

```bash icon="terminal" filename="start-orchestrator" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker run -d \
  --name livepeer-orchestrator \
  -v ~/.lpData/:/root/.lpData/ \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --network host \
  --gpus all \
  livepeer/go-livepeer:latest \
  -orchestrator \
  -transcoder \
  -nvidia 0 \
  -pricePerUnit 1000 \
  -serviceAddr 127.0.0.1:8935 \
  -cliAddr 127.0.0.1:7935 \
  -network offchain \
  -aiWorker \
  -aiModels /root/.lpData/aiModels.json \
  -aiModelsDir /root/.lpData/models
```

Wait for the warm model to load - this takes 2 to 5 minutes:

```bash icon="terminal" filename="watch-orchestrator" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker logs -f livepeer-orchestrator 2>&1 | grep -i "warm\|pipeline\|ai-runner\|error"
```

Expected:

```text icon="terminal" title="Expected warm-model startup log" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Starting AI worker
Pipeline text-to-image started
Warm model loaded: ByteDance/SDXL-Lightning
```

Verify the Orchestrator is accepting connections locally:

```bash icon="terminal" filename="verify-orchestrator" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
curl http://localhost:7935/registeredOrchestrators
```

Expected: a JSON array with your Orchestrator at `127.0.0.1:8935`.

<CustomDivider />

## Step 4: Start the Gateway

In a new terminal, start an off-chain AI Gateway pointing at the local Orchestrator. The community remote signer handles payment operations:

```bash icon="terminal" filename="start-gateway" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker run -d \
  --name livepeer-gateway \
  -v ~/.lpData-gateway/:/root/.lpData/ \
  --network host \
  livepeer/go-livepeer:latest \
  -gateway \
  -cliAddr 127.0.0.1:7936 \
  -httpAddr 0.0.0.0:8936 \
  -orchAddr http://127.0.0.1:8935 \
  -httpIngest \
  -remoteSignerAddr https://signer.eliteencoder.net \
  -network offchain
```

Key flags:

* `-orchAddr http://127.0.0.1:8935` - points directly at the local Orchestrator (off-chain mode bypasses on-chain discovery)
* `-httpIngest` - enables the AI inference HTTP endpoints
* `-remoteSignerAddr` - community remote signer for payment ticket signing (no wallet needed)
* Separate `-cliAddr` and `-httpAddr` from the Orchestrator's ports (7936 and 8936 vs 7935 and 8935)

<Note>
  The remote signer at `signer.eliteencoder.net` is a community-hosted service for testing. Check availability in [#local-Gateways on Discord](https://discord.gg/livepeer) before you start.
</Note>

Verify the Gateway started:

```bash icon="terminal" filename="verify-gateway" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker logs livepeer-gateway 2>&1 | grep -i "started\|gateway\|signer\|orchestrator" | head -10
```

Expected:

```text icon="terminal" title="Expected gateway startup log" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Gateway started on :8936
Connected to remote signer at https://signer.eliteencoder.net
Registered orchestrator: 127.0.0.1:8935
```

Verify the Gateway API is responding:

```bash icon="terminal" filename="check-gateway-health" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
curl http://localhost:8936/health
```

Expected: `{"status":"ok"}`

<CustomDivider />

## Step 5: Send an inference request through the Gateway

Send a `text-to-image` request through the Gateway on port 8936. Keep port 8935 for the Gateway-to-Orchestrator hop:

```bash icon="terminal" filename="send-via-gateway" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
curl -X POST http://localhost:8936/text-to-image \
  -H "Content-Type: application/json" \
  -d '{
    "model_id": "ByteDance/SDXL-Lightning",
    "prompt": "a coastal town in evening light, photorealistic",
    "width": 512,
    "height": 512,
    "num_inference_steps": 4
  }' \
  -o pipeline-output.png \
  --max-time 60
```

This request travels the full pipeline. A typical first inference takes 5 to 15 seconds (VRAM kernel warm-up on the first job). Subsequent requests take 2 to 4 seconds.

Verify the output:

```bash icon="terminal" filename="check-output" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
file pipeline-output.png
ls -lh pipeline-output.png
```

Expected: `pipeline-output.png: PNG image data` with a non-zero file size.

<CustomDivider />

## Step 6: Trace the request through logs

The request left footprints in each component. Read the logs to understand what happened at each hop:

**Gateway log** - shows routing decision and payment signing:

```bash icon="terminal" filename="gateway-log" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker logs livepeer-gateway 2>&1 | grep -i "route\|signer\|ticket\|orchestrat" | tail -10
```

Expected entries:

```text icon="terminal" title="Expected gateway log entries" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Routing job to orchestrator: 127.0.0.1:8935
Calling remote signer: getOrchInfoSig
Calling remote signer: signTicket
```

**Orchestrator log** - shows job receipt, dispatch to AI Runner, and result:

```bash icon="terminal" filename="orchestrator-log" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker logs livepeer-orchestrator 2>&1 | grep -i "job\|ai-runner\|inference\|ticket" | tail -10
```

Expected entries:

```text icon="terminal" title="Expected orchestrator log entries" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Received AI job: text-to-image
Dispatching to AI runner container
Inference complete
Ticket received
```

**AI Runner container log** - shows inference execution:

```bash icon="terminal" filename="airunner-log" theme={"theme":{"light":"github-light","dark":"dark-plus"}}
docker ps --filter name=livepeer | grep -v "livepeer-orchestrator\|livepeer-gateway"
# Find the ai-runner container name, then:
docker logs <ai-runner-container> 2>&1 | tail -20
```

Expected entries include model inference steps and output dimensions.

<CustomDivider />

## What happened

The request completed the full Livepeer AI pipeline:

1. **The curl request** hit the Gateway at `:8936` on the `/text-to-image` endpoint.
2. **The Gateway** selected the local Orchestrator at `:8935` (the only option via `-orchAddr`), signed a payment ticket using the community remote signer, and forwarded the job request.
3. **The Orchestrator** received the job, forwarded it to the AI Runner container via Docker-out-of-Docker, and waited for the result.
4. **The AI Runner** loaded the SDXL-Lightning model from VRAM (it was pre-warmed), ran 4 diffusion steps, and returned a PNG.
5. **The Orchestrator** returned the result to the Gateway and evaluated the payment ticket (in off-chain mode, settlement is handled by the remote signer instead of the Arbitrum TicketBroker).
6. **The Gateway** returned the PNG to the curl client.

**In production**, the Orchestrator is registered on-chain and the Gateway discovers it via the Livepeer Protocol. Payment tickets settle on Arbitrum through the TicketBroker contract. The inference mechanics are identical.

<CustomDivider />

## Related pages

<CardGroup cols={2}>
  <Card title="Gateway-Orchestrator Interface" icon="handshake" href="/v2/orchestrators/guides/advanced-operations/gateway-orchestrator-interface" arrow horizontal>
    Production combined deployment: port allocation, self-routing, and pricing alignment.
  </Card>

  <Card title="AI Inference Operations" icon="microchip" href="/v2/orchestrators/guides/ai-and-job-workloads/ai-inference-operations" arrow horizontal>
    Full aiModels.json reference and pipeline architecture concepts.
  </Card>

  <Card title="Gateway Relationships" icon="network-wired" href="/v2/orchestrators/guides/advanced-operations/gateway-relationships" arrow horizontal>
    How Gateways discover and select Orchestrators in production.
  </Card>

  <Card title="Realtime AI Tutorial" icon="video" href="/v2/orchestrators/guides/tutorials/realtime-ai-tutorial" arrow horizontal>
    The live video-to-video pipeline - streaming AI instead of batch inference.
  </Card>
</CardGroup>