SlimLLM

The tools couldn't reach the AI

Pfizer already had a way to access AI models, but it wasn't compatible with OpenAI's API. That matters more than it sounds. OpenAI's API has become the common language of AI tooling: coding assistants, chat apps, SDKs and agent frameworks all expect to speak it. Without it, most of the tools people wanted to use simply couldn't connect, and every team had to write its own workarounds.

The obvious fix was an off-the-shelf proxy, and LiteLLM was the one in use. In practice it proved unreliable and too slow for something that every AI request has to pass through. A gateway is infrastructure: when it's working, nobody should notice it's there.

What I built

I built SlimLLM from scratch in Rust, with two priorities above everything else: speed and stability.

One OpenAI-compatible front door

The gateway speaks OpenAI's API on one side and Pfizer's existing AI access on the other. For a team adopting it, connecting a tool was usually a matter of changing a base URL, rather than writing and maintaining an integration of their own.

Fast enough to forget about

Every request to every model goes through the gateway, so any delay it adds is felt everywhere. Rust has no garbage collector pausing at awkward moments and gives tight control over memory, so the gateway adds as little as possible to each request.

Stable enough to build on

Teams will only build on infrastructure they trust. Rust's compiler rules out whole classes of crashes and concurrency bugs before code ever runs, which is exactly what you want from a service that sits in front of every AI tool in the company.

A clear view of how AI is used

Because every request passed through one place, SlimLLM could provide analytics on how AI was actually being used across the company. For an organisation working out where AI delivers value and what it costs, that visibility is as valuable as the access itself.

The outcome

Thousands of people at Pfizer used SlimLLM and connected their tools to it. Once it had proved itself, it was handed over to a dedicated team to own and run. Building it alone and then passing it on meant it had to be simple to understand, operate and extend, not just fast.

What this means for your organisation

Most large organisations are somewhere on this path: AI access exists, but it's fragmented, hard to connect to, invisible to the people paying for it, or not trusted enough to build on. I can help you decide whether to buy or build, design a gateway that your teams and their tools can actually use, and get it to the point where your own team can run it. Get in touch to talk it through.