Investment Memo: Together AI
Company: Together AI — open-model cloud (inference, fine-tuning, GPU clusters) Stage under consideration: Hypothetical Series A Author: [Partner], [Fund]
---
Thesis
If open-weight models remain within striking distance of frontier closed models — and Llama, Mistral, Qwen, and DeepSeek suggest they will — then a large share of enterprise AI workloads will run on open models for cost, control, and data-privacy reasons. Those workloads need a serving layer. Together AI's bet is to be the "AWS of open models": a full-stack cloud spanning serverless inference, fine-tuning, and dedicated GPU clusters. The fund-returner case: inference is a compounding, usage-based revenue stream tied to the fastest-growing compute category in history; if Together captures even a few percent of open-model inference spend at scale, this is a multi-billion-dollar revenue company. Crucially, the team (Tri Dao, author of FlashAttention, is Chief Scientist; founders include Vipul Ved Prakash and Stanford's Chris Ré and Percy Liang) has genuine systems-research depth — the one durable edge in a business where speed and cost-per-token are the product.
Product & wedge
Three layers: (1) serverless inference APIs for 100+ open models with strong price/performance claims (FlashAttention, speculative decoding, custom kernels); (2) fine-tuning as a service, letting customers own their weights — a real differentiator versus OpenAI; (3) reserved GPU clusters (H100/H200/B200) for training. The wedge is developer-first inference: drop-in OpenAI-compatible endpoints at lower cost, converting API experiments into fine-tuning and cluster commitments. The research pedigree isn't marketing — kernel-level optimization directly determines gross margin and pricing power in a business where the product is literally tokens-per-dollar.
Market & competition
AI inference spend is plausibly a $50B+ market by late decade. But this is the most crowded lane in infrastructure:
- Direct open-model inference rivals: Fireworks AI, Baseten, Replicate, Anyscale, Groq (custom silicon), and hyper-aggressive price players like DeepInfra.
- GPU clouds moving up-stack: CoreWeave, Lambda, Nebius, Crusoe.
- Hyperscalers: AWS Bedrock, Azure AI, Google Vertex — distribution monsters bundling open models into existing enterprise contracts.
- Model providers themselves: Mistral's own API; Meta partnering directly with clouds.
- Existential adjacency: Nvidia (NIM microservices) could commoditize the serving layer.
Differentiation rests on research-driven performance and the full-stack (inference → fine-tune → cluster) motion. That's real but fragile: kernel optimizations diffuse (FlashAttention is open source; vLLM and SGLang are excellent and free), and the floor price of a token trends toward raw GPU cost.
Traction & business signal (public only)
- Raised a $102.5M Series A (Nov 2023, Kleiner Perkins), $106M (Salesforce Ventures, Mar 2024), a $305M Series B at ~$3.3B valuation (Feb 2025, General Catalyst/Prosperity7), and reports of a subsequent round at higher valuation.
- Reported (press) ~$100M+ annualized revenue run-rate as of early 2025 and hundreds of thousands of registered developers. Not audited; gross margin unknown — critical, since GPU resale revenue can carry thin or negative margins.
- Named customers/partners publicly referenced include Salesforce, Zoom, Washington Post, and various AI startups.
- Unknown: revenue mix between serverless inference (high multiple) vs. reserved cluster resale (low multiple), net revenue retention, customer concentration, GPU lease liabilities and their terms.
Risks — the three that kill the deal
- Margin structure is a mirage. If most revenue is GPU cluster resale, this is a capital-intensive middleman marked like a software company. CoreWeave trades on infrastructure multiples for a reason. A $3B+ valuation on $100M of ~20%-gross-margin revenue is not venture math. The serving-layer software premium only exists if inference is the majority of revenue and defensibly priced.
- Price war to zero. Open-model inference is nearly a commodity: identical weights, OpenAI-compatible APIs, switching cost measured in one config line. DeepInfra, hyperscalers, and eventually Nvidia can price at or below cost. Together's performance edge must stay ahead of open-source serving stacks forever to sustain margin. History of infrastructure commoditization says that's a hard treadmill.
- Open-model dependency. The entire business assumes frontier-competitive open weights keep shipping. If Meta deprioritizes open Llama releases (signals in 2025 were mixed) and Chinese models face enterprise procurement friction in the US, the quality gap versus closed models widens and workloads flow back to OpenAI/Anthropic. Together doesn't control its own supply of product.
Verdict
Invest at Series A — at Series A prices and terms, i.e., the 2023 ~$500M–$1B entry, not today's marks. The team is top-decile for exactly this problem, the market is enormous and early, and full-stack open-model serving is a legitimate shot at a generational infrastructure company. The bear cases are real but are execution/market-structure risks, not "this shouldn't exist" risks — acceptable at Series A pricing with 30–50x upside if inference margins hold.
The one question I need answered first: What is the revenue split and gross margin between serverless inference and GPU cluster resale — and what is net revenue retention on the inference cohort? That single disclosure tells you whether this is a software company riding compute growth or a leveraged GPU reseller wearing a software multiple.