LLM inference APIs for open models
Open-weight models are good enough to ship. Serving them fast and cheap is the hard part. Every API here hosts the models for you: send an OpenAI-style request, pay per token, and move to dedicated hardware when traffic justifies it. Groq and Cerebras run a short list of models on their own chips. Together AI, Fireworks, Baseten and DeepInfra add dedicated GPUs, and the first three add fine-tuning or training too. Modal Endpoints suits teams already running code on Modal, and RunInfra is built around agents that resend the same context every turn. Token prices move often, so check the date on the table. Not here: routers like OpenRouter that sit in front of many providers. Those are on AI gateways. 8 tools, in the order they appear in the directory. This is not a ranking.
Grouped by job: AI Agent Infrastructure
RunInfra
Open-model APIs built for agents: fast responses, warm replicas with up to 98% cache hits for repeated context, and one API key that works with the OpenAI and Anthropic SDKs. Backed by Y Combinator.
Together AI
Serverless API for open models such as gpt-oss, GLM and Kimi, OpenAI-compatible and billed per token, with batch, provisioned throughput, dedicated endpoints, fine-tuning and GPU clusters when you outgrow shared capacity.
Fireworks AI
Inference platform for open models: per-token serverless with Standard, Priority and Fast modes, batch at half the serverless price, on-demand GPUs billed per second, and LoRA fine-tuning with tuned models served at the base price. OpenAI-compatible.
Groq
GroqCloud runs a short list of open models, including gpt-oss and Whisper, on Groq's own LPU hardware behind an OpenAI-compatible API, with a free tier and a pay-as-you-go Developer plan.
Cerebras Inference
Open models such as GPT OSS 120B and Qwen 3.8 27B served on Cerebras hardware through an OpenAI-compatible API: a pay-as-you-go Developer tier for building and testing, Enterprise plans for production and fine-tuning.
Baseten
Per-token Model APIs for open models such as GLM, DeepSeek and Kimi, with OpenAI and Anthropic-compatible endpoints, plus dedicated deployments of your own or fine-tuned models packaged with the open-source Truss, billed by the minute.
DeepInfra
OpenAI-compatible API for open models such as Llama, DeepSeek and Qwen, billed per token with Standard, Priority and Flex tiers, plus dedicated GPUs for your own LLMs billed by the minute.
Modal Endpoints
Modal's inference product: OpenAI-compatible Shared Endpoints for open models billed per token, or Dedicated Endpoints that run open models or your own weights, billed per second of compute.
What it is, where it runs, what a million tokens costs.
Straight from each vendor's own pages. Plans move often, so the check date is below.
| Tool | What it is | Runs on | Open source | Price |
|---|---|---|---|---|
| RunInfra | Open-model APIs built for agents: session-aware routing keeps repeat calls on a warm replica with the context cached. OpenAI and Anthropic-compatible. | RunInfra's cloud. Dedicated infrastructure on Enterprise. | No. | Pay as you go from a $5 top-up, $1 free to test. Per 1M tokens: GLM 5.3 $1.30 in, $0.13 cached, $4.20 out. Qwen3.8 27B $0.10 in, $0.40 out. Coding plans from $10/mo. Enterprise custom. |
| Together AI | Serverless API for open models, plus batch, provisioned throughput, dedicated endpoints, fine-tuning and GPU clusters. | Together's cloud, with a separate base URL that keeps processing in the US. | No. | Per 1M tokens: gpt-oss-120B $0.15 in, $0.60 out. GLM-5.3 $1.40 in, $4.40 out. Kimi K3 $3.00 in, $15.00 out. Dedicated H100 $5.49 per GPU-hour on demand, B200 $8.99. |
| Fireworks AI | Serverless open models in Standard, Priority and Fast modes, batch, on-demand GPUs and fine-tuning. | Fireworks' cloud. SOC 2 Type 2 and ISO 27001. | No. | Prepaid credits. Per 1M tokens: gpt-oss-120B $0.15 in, $0.60 out. GLM 5.3 $1.40 in, $4.40 out. Kimi K3 $3.00 in, $15.00 out. Batch 50% of serverless. On-demand H100 $8.00/hr, B200 $13.00/hr, billed per second. |
| Groq | GroqCloud: an OpenAI-compatible API for a short list of open models on Groq's LPU hardware. | Groq's cloud. | No. | Free tier with lower rate limits. Developer, pay as you go, per 1M tokens: gpt-oss-120b $0.15 in, $0.60 out. gpt-oss-20b $0.075 in, $0.30 out. Whisper large v3 $0.111 per hour of audio. Llama 3.1 8B and 3.3 70B: contact sales. |
| Cerebras Inference | OpenAI-compatible API for open models on Cerebras hardware. The self-serve Developer tier is for building and testing, not production. | Cerebras' cloud, also sold through AWS Marketplace and reachable via OpenRouter, Hugging Face and Vercel. | No. | Developer, pay as you go, per 1M tokens: GPT OSS 120B $0.35 in, $0.75 out. Qwen 3.8 27B $0.99 in, $1.49 out. One-time $5 credit. Enterprise, with all models and fine-tuning: contact for a quote. |
| Baseten | Per-token Model APIs for open models, dedicated deployments for open, custom or fine-tuned models, and training. | Baseten's cloud. Your VPC or hybrid on Enterprise. | Truss, its model packaging tool, is MIT. The platform is not. | Basic $0/mo, pay as you go. Per 1M tokens: GPT OSS 120B $0.10 in, $0.50 out. GLM-5.3 $1.40 in, $4.40 out. Kimi K3 $3.00 in, $15.00 out. Dedicated H100 $0.10833 a minute. Pro and Enterprise: get a quote. |
| DeepInfra | OpenAI-compatible, per-token API for open models with Standard, Priority and Flex tiers, plus dedicated GPUs for custom LLMs. | DeepInfra's cloud. | No. | Per 1M tokens: Llama 3.1 8B Turbo $0.02 in, $0.04 out. Llama 3.3 70B Turbo $0.10 in, $0.32 out. DeepSeek-V4-Flash $0.09 in, $0.18 out. Priority 1.5x, Flex 0.8x. Dedicated H100 $2.90 per GPU-hour, billed per minute. |
| Modal Endpoints | Shared Endpoints serve open models from a Modal-managed pool. Dedicated Endpoints run open models or your own weights. | Modal's cloud. Shared Endpoints route through us-west. | No. | Shared, per 1M tokens: GLM 5.3 $1.40 in, $4.40 out. Kimi K3 $3.00 in, $15.00 out. DeepSeek V4.1 Flash $0.30 in, $1.20 out. Plan credits don't apply. Dedicated billed per second of compute, H100 $0.001097/sec. Starter $0/mo with $30 of compute. |
Checked 2026-11-01. Sources: RunInfra pricing, Together AI pricing, Together AI OpenAI compatibility, Fireworks pricing, Fireworks serverless pricing docs, Groq models and prices, Cerebras pricing, Cerebras OpenAI compatibility, Baseten pricing, Baseten Model APIs pricing and limits, Truss on GitHub, DeepInfra pricing, DeepInfra OpenAI API, Modal inference, Modal pricing.
Pick it if.
- RunInfraYour agent resends the same long context every turn and you want cached input billed at the cached rate.
- Together AIYou want serverless now and batch, fine-tuning or a GPU cluster from the same vendor later.
- Fireworks AIYou plan to fine-tune with LoRA and want the tuned model served at the base model's price.
- Groqgpt-oss or Whisper covers what you need and you want to start on a free tier.
- Cerebras InferenceYou want to try GPT OSS 120B or Qwen on $5 of credit before talking to sales about production.
- BasetenYou expect to move from shared endpoints to your own models on dedicated GPUs, or into your own cloud.
- DeepInfraPrice per token matters most and some of your traffic can wait for the cheaper Flex tier.
- Modal EndpointsYour code already runs on Modal and you want inference next to it, shared or on your own weights.
Each listing.
The directory page, alternatives in the same category, and a head-to-head where we have one.
- RunInfraAlternatives
- Together AIAlternatives
- Fireworks AIAlternatives
- GroqAlternatives
- Cerebras InferenceAlternatives
- BasetenAlternatives
- DeepInfraAlternatives
- Modal EndpointsAlternatives
Category pages list every tool in that category, including ones left off this page. Built one of these? Get it featured in its category.
The 2 biggest AI stories, every morning.
One short email: the two stories that matter, and why.
You're in.
The Daily AI Brief will land in your inbox soon. If you don't see it, check Promotions or Spam and move it to your main inbox.