Together AI

What Is Together AI?

Together AI sells compute for open models at every level of abstraction. At the top you send a prompt and pay per token. At the bottom you rent H200s by the hour and do what you like with them. In between sit dedicated endpoints, fine-tuning and a code sandbox — and the point is that moving between those levels does not mean changing supplier.

That range is unusual. Most companies in this space pick a layer: token APIs, or GPU rental, or training. Together covers the whole column, which matters because a successful AI product tends to walk down it. You start serverless because it is easy, discover your traffic is steady enough that dedicated capacity is cheaper, then fine-tune a model on your own data and need somewhere to serve it.

It is also unusually broad on modality. Alongside chat models sit image generation, video generation and audio transcription, all priced on the same account — Flux, Imagen, Veo, Sora and Seedance appear next to Llama and DeepSeek.

How It Works

Serverless Inference

The starting point is a per-token API across open models. Rates span a wide range: around $0.14 in and $0.28 out per million tokens for a fast DeepSeek variant, roughly $1.04 both ways for Llama 3.3 70B, and $3.00 in with $15.00 out for Kimi K3.

A more than twenty-fold spread on output means the model choice dominates the bill, and it is worth checking whether the cheap model actually fails at your task before assuming it does. Most teams discover that a good deal of their traffic never needed the expensive option.

Dedicated Endpoints and Raw Clusters

Dedicated inference gives you a model on hardware that is yours for the hour — an HGX H100 at $5.49, an HGX B200 at $8.99. The arithmetic against per-token pricing is straightforward: steady high volume favours the hourly rate, spiky traffic favours per token, and the crossover is worth calculating rather than guessing.

Below that sit GPU clusters. On demand an H100 is $3.99 per GPU per hour, an H200 $5.99, a B200 $8.19. Reserving for 181 days or more brings those down substantially — H200 to $3.99 and B200 to $6.79 — which is the usual shape of this market: capacity is cheap if you can promise you will use it.

Fine-Tuning as a Priced Product

Fine-tuning is billed per million training tokens and the numbers are published rather than quoted on request. LoRA supervised tuning on a model up to 16B is $0.48; DPO is $0.54; full-parameter tuning runs $1.20 to $1.35. Larger and specialised models cost considerably more — a LoRA on GLM-5.2 is $40.00 per million tokens.

Publishing this matters. Fine-tuning is usually where costs turn opaque, and being able to estimate a training run from your dataset size before committing is the difference between an experiment and a budget request.

What It Costs

There is no subscription. You pay for what you use at whichever layer you use it, and the table below gives the reference points that anchor the rest.

The reserved-versus-on-demand gap on clusters is the largest lever here. An H200 falls from $5.99 to $3.99 for a long commitment — a third off, in exchange for certainty that most early-stage teams cannot honestly provide.

Who Gets the Most From It

Products That Will Outgrow a Token API

If you can foresee dedicated capacity or your own fine-tuned model in the next year, starting somewhere that offers both removes a migration you would otherwise have to schedule.

Teams Fine-Tuning on Their Own Data

Published per-token training prices, a range of base model sizes, and somewhere to serve the result afterwards. That is a complete path rather than three separate procurement conversations.

Multimodal Applications

Text, images, video and transcription on one account and one bill. For anything that generates media alongside text, consolidating that is worth more than a few percent on any individual rate.

Anyone Renting GPUs Anyway

If you already lease compute for training, having inference from the same supplier — with reserved pricing available — is a simpler arrangement than splitting across a cloud provider and an API vendor.

What to Watch Out For

• Open models only. There are no frontier commercial models here, so the strongest available reasoning is not on the menu.
• The breadth is also a complexity cost. Four ways to buy compute means four ways to pick the wrong one and pay more than you needed.
• Reserved cluster pricing needs a 181-day commitment. The discount is real and so is the obligation.
• Output token prices vary more than twentyfold across models. Defaulting to the strongest option is the fastest way to a surprising invoice.
• Fine-tuning costs scale sharply with model size. A LoRA at $0.48 per million tokens and one at $40.00 are the same operation on different bases.
• Dedicated endpoints bill by the hour whether or not traffic arrives. Idle capacity is the classic way this becomes more expensive than serverless.

Frequently Asked Questions

Is there a free tier?

There is a free start rather than an ongoing free tier — the site invites you to begin at no cost without publishing a specific credit amount. Treat it as enough to evaluate, not enough to run on.

When should I move from serverless to dedicated?

When your traffic is steady enough that an hourly rate beats your token spend. Take your monthly token bill, divide by the hours you are actually serving traffic, and compare with $5.49 for an H100. Bursty workloads almost always stay serverless.

Can I fine-tune and then serve the result?

Yes — that path is the reason to choose a vendor covering several layers. Training is priced per million tokens and the resulting model can run on dedicated infrastructure on the same account.

Does it handle images and video?

Yes. Image generation is priced per image, from under a fifth of a cent for SD XL to six cents for Imagen 4.0 Ultra, and video per clip, with Sora 2 at $0.80 and Veo 3.0 at $1.60.

How does it compare to Groq or Fireworks?

Groq optimises for raw speed on a narrow catalogue. Fireworks focuses on production inference and fine-tuning. Together is the broadest of the three, reaching down to raw GPU rental — choose it when you expect to need more than a token API.

Is my data used for training?

Serving open models is the business here rather than improving proprietary ones, which puts the incentives in a reasonable place. As always, read the current terms for your specific plan rather than relying on a general statement.

The Bottom Line

Together AI is the option for teams who suspect a token API will not be the end of the story. Serverless inference to start, dedicated endpoints when volume justifies them, fine-tuning with published prices, and raw clusters underneath — all on one account, which removes the migrations that usually punctuate a growing AI product.

The catalogue is open models, so the strongest commercial reasoning is elsewhere. But if your work is open weights and you expect to grow into infrastructure, the breadth here is the reason to choose it — and worth the discipline of checking, at each layer, that you are buying the one you actually need.