OpenRelay: AI Inference and GPU Compute

Faster autoscaling than other providers

Tokens processed every month

Lower cost at high throughput

Backed by

Y Combinator

Built by engineers previously at

Built for teams that outgrew the hyperscaler bill.

Buy inference per token, a month or a quarter at a time with an SLA, or in batches at half price.

Compare inference options

On demand · per token

Pay for the tokens you use.

One API for every hosted model, billed per token with no minimum. When traffic stops, the bill stops with it.

MonthlyQuarterly

SLA held

Renews each quarter

Dedicated · SLA

Month to month, or quarter to quarter.

Your own endpoint with a throughput and latency SLA, on a monthly or quarterly term. No annual contract.

Batch · 50% off

Half price for work that can wait.

Submit a JSONL file. It runs as capacity frees up, at half the on-demand rate.

Three ways onto one network.

Call models, rent GPUs, or get paid to plug your own hardware in. All three run on the same network.

Call the endpoint.

OpenAI- and Anthropic-compatible, billed per token with no minimum.

base_url="https://inference.openrelay.inc/v1"

Per token · input and output

Browse models

Rent a GPU.

Root over SSH, your image and a persistent volume.

$ orl deploy trainer \
    --ssh-key <key-id> \
    --gpu-model h100-sxm \
    --gpu-count 1 --wait
VM <vm-id> is ready

Hourly rate · accrues while it runs

Compare GPUs

Supply the network.

Enroll your GPU or CPU nodes, and we handle scheduling, billing and payouts.

$ orl provider apply \
    --note "8x H100, one node"

Monthly payout on verified usage

Become a provider

Your terminal is the console. So is your agent.

Everything the dashboard does ships in the orl CLI, and one command serves the same API to Claude and any MCP client.

Install orl

Deploy from the command line.

orl wraps the full OpenRelay API: launch a GPU VM, drop into SSH, and script your fleet from a shell or CI.

CLI quickstart

orl cli

$ brew install openrelayinc/tap/orl
$ orl deploy trainer \
    --ssh-key <key-id> \
    --gpu-model "h100-sxm" --gpu-count 1 --connect
✓ training-01 running
root@training-01:~$ nvidia-smi
NVIDIA H100 80GB HBM3, 81559 MiB
root@training-01:~$

Hand the keys to your agent.

orl mcp serve exposes the platform as MCP tools. Claude deploys VMs, reads usage, and manages resources with your access, or inspect-only with --read-only.

For AI agents

mcp

$ claude mcp add openrelay \
    -- orl mcp serve
✓ Added stdio MCP server openrelay
• vms_create { "name": "training-01",
    "gpuModelId": "h100-sxm",
    "gpuCount": 1,
    "sshKeyIds": ["<key-id>"] }
⎿  training-01 → running

A CDN, for inference.

A CDN put the web's content close to every user. An inference delivery network puts every request on a GPU that can serve it now.

How the network works

Now

Spot capacity joins in SEA and LAX and takes nearby traffic.

Routed: 167
In flight: 4
Dropped: 0

SFO Serving
IAD Serving
DFW Serving
ORD Serving
SEA spotServing
LAX spotServing

Sites shown are illustrative.

One endpoint: Apps, agents and batch jobs anywhere call one OpenAI- and Anthropic-compatible API.
Routed to the best site: Each request goes to a site that is near, healthy and has headroom now.
Failover: A degraded site leaves rotation. Your code does not change.
Spot capacity: Spot sites join when GPUs free up and drain before they are reclaimed.

OpenRelay API

Control OpenRelay from your own stack.

Explore docs

Inference API

Point any OpenAI SDK at OpenRelay and change nothing else.

OpenAI-compatible: Drop-in for existing clients
Automatic routing: Latency, cost, and health aware

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.openrelay.inc/v1",
  apiKey: process.env.OPENRELAY_API_KEY,
});

await client.chat.completions.create({
  model: "openai/gpt-oss-120b",
  messages: [{ role: "user", content: "Hello" }],
});

Compute API

Create GPU VMs from code: pick a GPU, an image, and a count, then SSH in when it boots.

RTX 4090 to H100: Same call, any tier
Persistent volumes: Disks survive restarts

import os
import requests

org = os.environ["OPENRELAY_ORG_ID"]
key = os.environ["OPENRELAY_API_KEY"]

resp = requests.post(
    f"https://api.openrelay.inc/v1/orgs/{org}/vms/create",
    headers={"Authorization": f"Bearer {key}"},
    json={"name": "vllm-h100", "gpuModelId": "h100", "gpuCount": 1, "sshKeyIds": ["<ssh-key-id>"]},
)

Frequently asked questions.

Talk to the team

Provisioning time depends on the hardware and image you choose. The dashboard reports each stage until your VM is ready. Hosted models need no deployment.

The inferencedelivery network.