Design an internal LLM serving platform

Twelve product teams want to use LLMs. Some need a hosted API model, some need a fine-tuned open-weights model, one needs a model running where no data leaves the VPC. Usage is bursty and unpredictable: one team may generate 200 requests per second during a batch job and nothing for a week.

You are asked to build one platform for all of it. The constraints given: per-team cost attribution, no credential sharing, an audit trail of every prompt and response for compliance, and a hard requirement that no team's traffic can starve another's.

Budget is real but not stated. You will be asked to justify spend.

Design the platform. Cover routing, isolation, cost attribution, the self-hosted path, and what you would deliberately refuse to build. Justify the refusals as carefully as the inclusions.