Case study · Private engagement
Building an AI platform that teams actually adopt.
An ongoing platform engineering engagement for a product organization shipping AI features. This page documents the architecture, the boundaries the platform respects, and the trade-offs behind them — client identity, data, and implementation details are covered by confidentiality agreements and are not published.
Confidentiality boundaries. What is public on this page: architectural patterns, decision rationale, and trade-offs. What is not: the client's name, their data, model providers under contract, traffic figures, and anything derived from their codebase.
The problem
Product teams wanted to ship AI-assisted features. Each team solved the same problems independently: provider credentials scattered across services, no shared usage visibility, agent behavior that could not be tested, and incident response that meant grepping logs across a dozen repositories. The organization did not need an "AI strategy" — it needed shared infrastructure with clear ownership.
The engagement started with a scoped architecture review, then continued into building the platform. That sequencing mattered: the review established trust and a written decision record before any code was written.
Architecture overview
The platform owns four planes: a model gateway, an agent runtime, an evaluation harness, and observability. Product teams keep their own release cadence and consume the platform through explicit contracts — CLI, SDK surface, and structured events.
Platform planes and their contracts. Providers are reached only through the gateway; agents reach tools only through the runtime's policy layer.
Decisions and alternatives
One gateway instead of direct provider SDKs
Credential handling, budget enforcement, and routing belong in one place. The rejected alternative — each service keeping its own provider integration — was the status quo, and it was the reason the review was commissioned. The cost of the gateway is a dependency teams did not have before; it is paid back the first time a provider outage requires a routing change in one place instead of twelve.
Agents as platform citizens, not scripts
The runtime gives agents sessions, tool allowlists, and policy checks. The alternative — agents as ad-hoc scripts with blanket credentials — is faster to start and impossible to audit. Given that agents act on production systems, auditability won.
Evaluation as a gate, not a dashboard
Eval suites run in CI against every change to prompts, tools, and models. A regression gate that can block a merge is worth more than an offline score nobody reads. The trade-off: maintaining eval datasets is real work, owned by the platform team.
Trade-offs
- Control vs. flexibility. The platform constrains how teams call models and tools. Teams that need to move outside the contract can — but the exception path is explicit and reviewed, not silently available.
- Latency vs. observability. Structured events on every agent step add overhead. Batching and sampling keep the p99 cost bounded; the visibility is not negotiable.
- Central platform vs. team autonomy. A platform team can become a bottleneck. Contract-first interfaces and self-service usage keep the platform an accelerator, not a queue.
What adoption looked like
Adoption followed the written decision record, not a mandate: teams migrated when the gateway solved a problem they had hit themselves — a rotated credential, an unexplained spend spike, a prompt regression caught by the eval gate before release. Qualitatively, incident response went from log archaeology to reading one trace; quantitatively, that is the client's information to share, not mine.
Related writing
The extended version of this architecture story, including the failures and the parts that got rebuilt, is in the writing section: Building an AI platform: architecture, boundaries, and trade-offs.