> ## Documentation Index
> Fetch the complete documentation index at: https://docs.livepeer.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Orchestrator Capabilities

> What Orchestrators can execute, the workload types they support across video and AI, how Gateways select them, and the boundaries of the Orchestrator role.

export const BorderedBox = ({children, variant = "default", padding = "var(--lp-spacing-4)", borderRadius = "var(--lp-spacing-px-8)", margin = "", accentBar = "", style = {}, className = "", ...rest}) => {
  const variants = {
    default: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "var(--lp-color-bg-card)"
    },
    accent: {
      border: "1px solid var(--lp-color-accent)",
      backgroundColor: "var(--lp-color-bg-card)"
    },
    muted: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "transparent"
    }
  };
  const accentBarColors = {
    accent: "var(--lp-color-accent)",
    positive: "var(--green-9)"
  };
  return <div data-docs-bordered-box="" data-accent-bar={accentBarColors[accentBar] ? "" : undefined} className={className} style={{
    ...variants[variant],
    padding: padding,
    borderRadius: borderRadius,
    ...margin ? {
      margin
    } : {},
    ...accentBarColors[accentBar] ? {
      position: "relative",
      '--accent-bar-color': accentBarColors[accentBar]
    } : {},
    ...style
  }} {...rest}>
      {children}
    </div>;
};

export const CenteredContainer = ({children, maxWidth = "800px", padding = "0", preset = "default", width = "", minWidth = "", marginRight = "", marginBottom = "", textAlign = "", style = {}, className = "", ...rest}) => {
  const presets = {
    default: {},
    fitContent: {
      width: "fit-content",
      minWidth: "fit-content"
    },
    readable70: {
      width: "70%",
      minWidth: "fit-content"
    },
    readable80: {
      width: "80%",
      minWidth: "fit-content"
    },
    readable90: {
      width: "90%"
    },
    wide900: {
      maxWidth: "900px"
    }
  };
  const presetStyle = presets[preset] || presets.default;
  return <div className={className} style={{
    maxWidth: presetStyle.maxWidth || maxWidth,
    margin: "0 auto",
    padding: padding,
    ...presetStyle.width ? {
      width: presetStyle.width
    } : {},
    ...presetStyle.minWidth ? {
      minWidth: presetStyle.minWidth
    } : {},
    ...width ? {
      width
    } : {},
    ...minWidth ? {
      minWidth
    } : {},
    ...marginRight ? {
      marginRight
    } : {},
    ...marginBottom ? {
      marginBottom
    } : {},
    ...textAlign ? {
      textAlign
    } : {},
    ...style
  }} {...rest}>
      {children}
    </div>;
};

export const ScrollableDiagram = ({children, title = '', maxHeight = '500px', minWidth = '100%', showControls = false, className = '', style = {}, ...rest}) => {
  const buildDiagramKey = (currentTitle = '', currentClassName = '') => {
    const source = `${currentTitle}|${currentClassName}|scrollable-diagram`;
    let hash = 0;
    for (let index = 0; index < source.length; index += 1) {
      hash = hash * 31 + source.charCodeAt(index) >>> 0;
    }
    return `docs-diagram-${hash.toString(36)}`;
  };
  const diagramKey = buildDiagramKey(title, className);
  const zoomName = `${diagramKey}-zoom`;
  const zoomLevels = [{
    label: '75%',
    value: 0.75
  }, {
    label: '100%',
    value: 1
  }, {
    label: '125%',
    value: 1.25
  }, {
    label: '150%',
    value: 1.5
  }];
  const containerStyle = {
    overflow: 'auto',
    maxHeight,
    border: '1px solid var(--lp-color-border-default)',
    borderRadius: "8px",
    padding: "var(--lp-spacing-4)",
    background: 'var(--lp-color-bg-card)',
    position: 'relative'
  };
  return <div className={className} style={{
    position: 'relative',
    marginBottom: "var(--lp-spacing-4)",
    ...style
  }} {...rest}>
      {title && <p style={{
    textAlign: 'center',
    fontStyle: 'italic',
    color: 'var(--lp-color-text-secondary)',
    marginBottom: "var(--lp-spacing-2)",
    fontSize: '0.875rem'
  }}>
          {title}
        </p>}

      {showControls ? <style>{`
          [data-docs-diagram-key="${diagramKey}"] [data-docs-diagram-content] {
            transform: scale(1);
            transform-origin: top left;
            width: max-content;
          }
          ${zoomLevels.map(zoomLevel => `
          #${diagramKey}-${zoomLevel.label.replace('%', '')}:checked ~ [data-docs-diagram-shell] [data-docs-diagram-content] {
            transform: scale(${zoomLevel.value});
          }
          #${diagramKey}-${zoomLevel.label.replace('%', '')}:checked ~ [data-docs-diagram-controls] label[for="${diagramKey}-${zoomLevel.label.replace('%', '')}"] {
            background: var(--lp-color-accent);
            color: var(--lp-color-on-accent);
            border-color: var(--lp-color-accent);
          }`).join('\n')}
        `}</style> : null}

      {showControls ? zoomLevels.map(zoomLevel => {
    const inputId = `${diagramKey}-${zoomLevel.label.replace('%', '')}`;
    return <input key={inputId} id={inputId} type="radio" name={zoomName} defaultChecked={zoomLevel.value === 1} style={{
      position: 'absolute',
      opacity: 0,
      pointerEvents: 'none'
    }} />;
  }) : null}

      <div data-docs-diagram-key={diagramKey} data-docs-diagram-shell style={containerStyle}>
        <div data-docs-diagram-content style={{
    minWidth,
    transformOrigin: 'top left',
    width: 'max-content'
  }}>
          {children}
        </div>
      </div>

      {showControls ? <div data-docs-diagram-controls style={{
    display: 'flex',
    justifyContent: 'flex-end',
    alignItems: 'center',
    gap: "var(--lp-spacing-2)",
    marginTop: "var(--lp-spacing-2)",
    flexWrap: 'wrap'
  }}>
          <span style={{
    fontSize: "0.75rem",
    color: 'var(--lp-color-text-muted)',
    marginRight: 'auto'
  }}>
            Scroll to pan
          </span>
          {zoomLevels.map(zoomLevel => {
    const inputId = `${diagramKey}-${zoomLevel.label.replace('%', '')}`;
    return <label key={inputId} htmlFor={inputId} style={{
      background: 'transparent',
      color: 'var(--lp-color-text-secondary)',
      border: '1px solid var(--lp-color-border-default)',
      borderRadius: "4px",
      padding: '4px 10px',
      cursor: 'pointer',
      fontSize: "0.75rem",
      fontWeight: '600'
    }}>
                {zoomLevel.label}
              </label>;
  })}
        </div> : null}
    </div>;
};

export const CustomDivider = ({color = "var(--lp-color-border-default)", middleText = "", spacing = "default", style = {}, className = "", ...rest}) => {
  const spacingPresets = {
    default: {
      margin: "24px 0"
    },
    overlap: {
      margin: "-1rem 0 -1rem 0"
    },
    tight: {
      margin: "0 0 -1rem 0"
    },
    section: {
      margin: "0 0 -2rem 0"
    },
    sectionOverlap: {
      margin: "-1rem 0 -2rem 0"
    },
    deepOverlap: {
      margin: "-1rem 0 -1.5rem 0"
    }
  };
  const spacingStyle = spacingPresets[spacing] || spacingPresets.default;
  return <div role="separator" aria-orientation="horizontal" className={className} style={{
    display: "flex",
    alignItems: "center",
    ...spacingStyle,
    fontSize: style?.fontSize || "16px",
    height: "fit-content",
    ...style
  }} {...rest}>
      <span style={{
    marginRight: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
      </span>
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      {middleText && <>
          <Icon icon="circle" size={2} />
          <span style={{
    margin: "0 8px",
    fontWeight: "bold",
    color: color,
    opacity: 0.7
  }}>
            {middleText}
          </span>
          <Icon icon="circle" size={2} />
        </>}
      <div style={{
    flex: 1,
    height: "1px",
    background: "var(--lp-color-border-default)",
    opacity: 0.4
  }}></div>
      <span style={{
    marginLeft: "var(--lp-spacing-px-8)",
    opacity: 0.2
  }}>
        <span style={{
    display: "inline-block",
    transform: "scaleX(-1)"
  }}>
          <Icon icon="/snippets/assets/logos/Livepeer-Logo-Symbol-Theme.svg" />
        </span>
      </span>
    </div>;
};

export const TableCell = ({children, align = "left", header = false, style = {}, className = "", ...rest}) => {
  const Component = header ? "th" : "td";
  return <Component className={className} style={{
    padding: "0.75rem 1rem",
    textAlign: align,
    border: header ? "none" : "1px solid var(--lp-color-border-default)",
    ...style
  }} {...rest}>
      {children}
    </Component>;
};

export const TableRow = ({children, header = false, hover = false, style = {}, className = "", ...rest}) => {
  const rowId = `table-row-${Math.random().toString(36).substr(2, 9)}`;
  return <>
      {hover && <style>{`
          #${rowId}:hover {
            background-color: var(--lp-color-bg-card);
          }
        `}</style>}
      <tr id={rowId} className={className} style={{
    ...header && ({
      backgroundColor: "var(--lp-color-accent-strong)",
      color: "var(--lp-color-on-accent)",
      fontWeight: "bold"
    }),
    ...style
  }} {...rest}>
        {children}
      </tr>
    </>;
};

export const StyledTable = ({children, variant = "default", style = {}, className = "", ...rest}) => {
  const wrapperVariants = {
    default: {
      border: "1px solid var(--lp-color-border-default)",
      backgroundColor: "var(--lp-color-bg-card)",
      overflow: "hidden"
    },
    bordered: {
      border: "2px solid var(--lp-color-accent)",
      backgroundColor: "var(--lp-color-bg-page)",
      overflow: "hidden"
    },
    minimal: {
      border: "none",
      backgroundColor: "transparent",
      overflow: "visible"
    }
  };
  return <div data-docs-styled-table-shell className={className} style={{
    width: "100%",
    padding: 0,
    margin: 0,
    ...wrapperVariants[variant],
    ...style
  }} {...rest}>
      <table data-docs-styled-table style={{
    width: "100%",
    borderCollapse: "collapse",
    borderSpacing: 0,
    margin: 0,
    backgroundColor: "transparent"
  }}>
        {children}
      </table>
    </div>;
};

export const LinkArrow = ({href, label, description, newline = true, borderColor, className = '', style = {}, ...rest}) => {
  const linkArrowStyle = {
    display: 'inline-flex',
    alignItems: 'center',
    justifyContent: 'center',
    gap: "var(--lp-spacing-1)",
    width: 'fit-content',
    ...borderColor && ({
      borderColor
    })
  };
  return <span className={className} style={style} {...rest}>
      {newline && <br />}
      <span style={linkArrowStyle}>
        <a href={href} target="_blank" rel="noopener noreferrer">
          {label}
        </a>
        <Icon icon="arrow-up-right" size={14} color="var(--lp-color-accent)" />
      </span>
      {description && description}
      {description && <div style={{
    height: "var(--lp-spacing-3)"
  }} />}
    </span>;
};

<CenteredContainer width="90%">
  <Tip>Orchestrators perform compute - they do not route or discover work. Gateways send jobs; Orchestrators execute them on GPU hardware and return results. Every capability an Orchestrator exposes must be declared on registration so Gateways can discover and select it.</Tip>
</CenteredContainer>

<CustomDivider style={{margin: "0 0 -1rem 0"}} />

## Workload Types

Orchestrators can execute four categories of compute workload. Which workloads a node accepts depends
on its hardware, the pipelines and models it has loaded, and how it is configured.

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>Workload</TableCell>
      <TableCell header>What the Orchestrator does</TableCell>
      <TableCell header>Unit of work</TableCell>
      <TableCell header>GPU required</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>**Video transcoding**</TableCell>
      <TableCell>Re-encode live video segments from RTMP ingest into multiple output profiles (resolution, bitrate, format)</TableCell>
      <TableCell>Per segment (typically 2 seconds of video)</TableCell>
      <TableCell>Yes (NVENC preferred; CPU possible)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**Batch AI inference**</TableCell>
      <TableCell>Run single-request AI jobs: text-to-image, image-to-image, image-to-video, audio-to-text, upscale, segmentation</TableCell>
      <TableCell>Per request (pixel or millisecond-based)</TableCell>
      <TableCell>Yes (VRAM requirements vary by model)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**Real-time AI (Cascade)**</TableCell>
      <TableCell>Apply continuous AI processing to a live video stream - style transfer, avatar rendering, real-time generative overlays</TableCell>
      <TableCell>Per interval during stream duration</TableCell>
      <TableCell>Yes (high VRAM; dedicated GPU strongly recommended)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**BYOC containers**</TableCell>
      <TableCell>Execute application-defined Docker containers on Orchestrator hardware for custom ML workflows or novel compute tasks</TableCell>
      <TableCell>Per request or per interval (defined by pipeline)</TableCell>
      <TableCell>Depends on container</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

<Note>
  An Orchestrator running both video transcoding and AI inference is described as operating in a
  **dual-workload configuration**. This is not a separate mode - it is the same Orchestrator process
  with both pipelines enabled. See <LinkArrow href="/v2/orchestrators/concepts/architecture" label="Orchestrator Architecture" newline={false} /> for the pipeline internals.
</Note>

<CustomDivider style={{margin: "0 0 -1rem 0"}} />

## Supported AI Pipelines

Livepeer defines a standard set of AI pipelines that Orchestrators can advertise. Each pipeline maps
to a category of inference task and a compatible set of models.

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>Pipeline</TableCell>
      <TableCell header>Input</TableCell>
      <TableCell header>Output</TableCell>
      <TableCell header>Example models</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>**text-to-image**</TableCell>
      <TableCell>Text prompt</TableCell>
      <TableCell>Image</TableCell>
      <TableCell>`stabilityai/stable-diffusion-3-medium-diffusers`, `ByteDance/SDXL-Lightning`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**image-to-image**</TableCell>
      <TableCell>Image + prompt</TableCell>
      <TableCell>Image</TableCell>
      <TableCell>`timbrooks/instruct-pix2pix`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**image-to-video**</TableCell>
      <TableCell>Image + motion prompt</TableCell>
      <TableCell>Video clip</TableCell>
      <TableCell>`stabilityai/stable-video-diffusion-img2vid-xt-1-1`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**audio-to-text**</TableCell>
      <TableCell>Audio file</TableCell>
      <TableCell>Transcript (JSON)</TableCell>
      <TableCell>`openai/whisper-large-v3`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**upscale**</TableCell>
      <TableCell>Low-resolution image</TableCell>
      <TableCell>High-resolution image</TableCell>
      <TableCell>`stabilityai/stable-diffusion-x4-upscaler`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**segment-anything-2**</TableCell>
      <TableCell>Image + prompt</TableCell>
      <TableCell>Segmentation mask</TableCell>
      <TableCell>`facebook/sam2-hiera-large`</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**live-video-to-video**</TableCell>
      <TableCell>Live video stream</TableCell>
      <TableCell>Transformed live stream</TableCell>
      <TableCell>Cascade pipeline models</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

An Orchestrator can support any subset of pipelines and models. Each combination of pipeline and model
is independently priced and advertised. Gateways discover these via the AIServiceRegistry contract or
from the Orchestrator's capability response during session negotiation.

<CustomDivider style={{margin: "0 0 -1rem 0"}} />

## How Capabilities Are Advertised

When a Gateway wants to route a job, it must find an Orchestrator that can handle it. Orchestrators
make themselves discoverable through two mechanisms:

```mermaid theme={"theme":{"light":"github-light","dark":"dark-plus"}}
%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#18794E', 'primaryTextColor': '#fff', 'primaryBorderColor': '#3CB540', 'lineColor': '#3CB540', 'mainBkg': '#18794E', 'nodeBorder': '#3CB540', 'clusterBkg': 'transparent', 'clusterBorder': '#3CB540', 'titleColor': '#3CB540', 'edgeLabelBackground': 'transparent', 'textColor': '#3CB540', 'nodeTextColor': '#fff'}}}%%
flowchart TD
    A["Orchestrator starts"] --> B["Register on-chain\n(ServiceRegistry / AIServiceRegistry)"]
    B --> C["Advertise service URI\n(HTTPS endpoint)"]
    C --> D["Gateway queries registry"]
    D --> E["Gateway contacts Orchestrator\ndirectly (capability negotiation)"]
    E --> F["Orchestrator returns\ncapabilities + prices"]
    F --> G["Gateway selects, dispatches job"]

    classDef default fill:#1a1a1a,color:#fff,stroke:#2d9a67,stroke-width:2px
```

### On-chain registration

Orchestrators register their service URI in the **ServiceRegistry** contract on Arbitrum. AI-capable
Orchestrators additionally register with the **AIServiceRegistry** contract (or use the `-aiServiceRegistry`
flag to connect to the AI subnet). This makes the Orchestrator discoverable to all Gateways that query
the registry.

### Capability negotiation

When a Gateway establishes a session with an Orchestrator, the Orchestrator returns a **capability
manifest** - the full list of pipelines it supports, the models it has loaded, and its price per unit
for each. The Gateway uses this to decide whether to proceed with the session.

Capabilities that are advertised but not actually available (e.g. Models not yet loaded into VRAM) will
result in job failures. Keep declared capabilities in sync with loaded models.

<CustomDivider style={{margin: "0 0 -1rem 0"}} />

## How Gateways Select Orchestrators

Understanding Gateway selection is essential for Orchestrators that want to attract work. Gateways do
not randomly assign jobs - they apply a multi-factor ranking algorithm to every session.

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>Factor</TableCell>
      <TableCell header>What the Gateway looks at</TableCell>
      <TableCell header>Operator implication</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>**Capability match**</TableCell>
      <TableCell>Does the Orchestrator support the requested pipeline and model?</TableCell>
      <TableCell>Declare only what you can actually run</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**Price**</TableCell>
      <TableCell>Is your `-pricePerUnit` at or below the Gateway's `-maxPricePerUnit`?</TableCell>
      <TableCell>Price competitively; too high means no jobs</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**Latency**</TableCell>
      <TableCell>How fast did you respond to recent segments?</TableCell>
      <TableCell>Low-latency GPU access and reliable network connectivity are critical</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**Performance history**</TableCell>
      <TableCell>What is your success rate and error rate over recent sessions?</TableCell>
      <TableCell>Uptime and job reliability directly affect how much work you receive</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>**Stake weight**</TableCell>
      <TableCell>How much LPT is bonded to your Orchestrator (self-stake + delegated)?</TableCell>
      <TableCell>Higher stake increases selection probability when other factors are equal</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

A Gateway that sends a job and receives an error or timeout will deprioritise your Orchestrator for
subsequent sessions. Sustained availability and accurate capability declaration are the strongest
signals for consistent job flow.

See <LinkArrow href="/v2/orchestrators/guides/payments-and-pricing/pricing-strategy" label="Pricing Strategy" newline={false} /> for how to configure competitive prices.

<CustomDivider style={{margin: "0 0 -1rem 0"}} />

## Capability Boundaries

Orchestrators handle compute and payment receipt. They do not handle job routing, application
integration, or business-layer concerns.

<StyledTable variant="bordered">
  <thead>
    <TableRow header>
      <TableCell header>Not part of the Orchestrator role</TableCell>
      <TableCell header>Where it happens instead</TableCell>
    </TableRow>
  </thead>

  <tbody>
    <TableRow>
      <TableCell>Accepting RTMP streams from applications</TableCell>
      <TableCell>Gateway (RTMP ingest, port 1935)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Routing jobs between multiple Orchestrators</TableCell>
      <TableCell>Gateway (Orchestrator selection algorithm)</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Managing API keys or billing for end users</TableCell>
      <TableCell>Gateway or application layer</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Sending payment tickets to anyone</TableCell>
      <TableCell>Gateways send tickets; Orchestrators receive and redeem them</TableCell>
    </TableRow>

    <TableRow>
      <TableCell>Staking LPT for Delegators</TableCell>
      <TableCell>BondingManager contract (Delegators do this themselves)</TableCell>
    </TableRow>
  </tbody>
</StyledTable>

If you want to aggregate application demand and route work across multiple Orchestrators, that is
the Gateway role. See <LinkArrow href="/v2/gateways/concepts/role" label="Gateway Role" newline={false} /> for that path.

<CustomDivider style={{margin: "0 0 -1rem 0"}} />

## Related Pages

<CardGroup cols={2}>
  <Card title="Orchestrator Role" icon="user-gear" href="/v2/orchestrators/concepts/role" arrow horizontal>
    What Orchestrators are and how the role has evolved.
  </Card>

  <Card title="Orchestrator Architecture" icon="diagram-project" href="/v2/orchestrators/concepts/architecture" arrow horizontal>
    Internal components, request flow, and system interactions.
  </Card>

  <Card title="Incentive Model" icon="coins" href="/v2/orchestrators/concepts/incentive-model" arrow horizontal>
    Revenue streams, cost structure, and earnings potential.
  </Card>

  <Card title="Workloads and AI" icon="microchip" href="/v2/orchestrators/guides/workloads-and-ai/job-types" arrow horizontal>
    Detailed setup guides for video, AI, and BYOC workloads.
  </Card>
</CardGroup>