What is AI gateway security, and why did LiteLLM’s 2026 flaws matter?

An AI gateway holds, on one host, every provider key, tenant credential and tool connection its products use to reach a model. AI gateway security is less about the traffic it inspects than about what a breach of that host hands over: keys, the proxy’s database, connected tools, its cloud identity. The design that limits the loss stores as little there as possible.

Between May and September 2026, CISA added three LiteLLM flaws to its Known Exploited Vulnerabilities catalogue, and on 2026-09-02 a Starlette flaw its entry says chains with one of them. On 2026-10-27, Red Hat published a different kind of gateway flaw, in an OpenShift AI component: a request naming another tenant’s model got that tenant’s keys injected. I have not audited or run LiteLLM; what follows stays with the records.

When (catalogue date) What failed What the caller needed What was exposed
2026-05-08 SQL injection in the API key check (CVE-2026-42208) No key: a crafted Authorization header on any LLM route The database “and the credentials it manages”
2026-06-08 Command injection in the MCP test endpoints (CVE-2026-42271) Any valid proxy key, even low-privilege Arbitrary commands on the host, as the proxy process
2026-09-02 Starlette Host header handling (CVE-2026-48710) No key: a malformed Host header Whatever a path-based check protected
2026-09-02 MCP authentication fallback (CVE-2026-59822) No key: any Bearer value An MCP session: configured tools and connected services
2026-10-27, Red Hat, not in the catalogue Credential lookup keyed by a client-supplied model identifier (CVE-2026-117574) One tenant’s credential Use of another tenant’s provider keys, and its quota

Five records. In each, the thing worth taking was already sitting in the gateway.

What does an attacker get from a gateway?

Provider keys first. The Cloud Security Alliance’s note on the command injection lists what a command run as the proxy can read: environment variables holding provider keys, mounted credential files, the configuration, the database. Its advice to rotate every key the proxy held is also an admission.

The proxy’s database is the second prize. The catalogue entry for the SQL injection ends on “unauthorized access to the proxy and the credentials it manages”, and the advisory adds how: a crafted header on any LLM route reached the query through the error-handling path. The CSA note adds the access configuration a deployment holds, which teams and services can access which models.

An MCP session comes next, and whatever stands behind it: the advisory for the authentication bypass says an attacker could list and call configured MCP tools and reach the connected services, and Wiz shows Authorization: Bearer a opening one. Wiz goes a step further with a pass-through feature that forwards requests, caller headers included, to any URL an admin configures, with no check against the metadata address; pointed at 169.254.169.254, it returned the host’s role credentials, and a gateway breach became a cloud breach.

The Red Hat record adds the multi-tenant form of the same mistake. A shared credential store was queried with the model identifier the caller sent, with no authorisation check, so the service used another tenant’s keys for the caller’s request. It affects one named component, is not in the catalogue and has not been reported exploited.

Should the gateway hold provider keys at all?

Not where the provider lets you avoid it, and Amazon Bedrock does. In the Bedrock gateway design I published, the gateway calls Bedrock with an IAM role and holds no provider key. A role has no long-term credentials; when the gateway assumes it, STS issues temporary credentials for the session, which expire.

The role’s policy allows invocation only on approved EU inference profiles, through bedrock:InferenceProfileArn, and a service control policy caps what the role can do whatever its policy says: AWS describes SCPs as the maximum available permissions in a member account, granting nothing by themselves. Both are in the Bedrock gateway design guide, layer 1 of the enterprise AI platform architecture, with the same caveat: a design to test on one organisational unit first.

What the role does and does not change

What is a stolen session worth? The calls the gateway could make, on the same EU profiles, for the rest of the session, each in CloudTrail under the role’s name, since Bedrock records Converse and InvokeModel as management events. Still worth something, your models on your bill, the LLMjacking Wiz describes. Darktrace described a real setting in July 2026, though not a Bedrock theft: an EC2 instance named LiteLLM-Proxy, instance profile able to reach Bedrock, SSH open, used for cryptomining. Notice in time, and AWS lets you deny all permissions to sessions issued before a chosen moment.

What the role does not change: a session lifted from the process (on EKS, the web-identity token file the pod exchanges for role credentials) or from the metadata service works until it expires. So the pass-through endpoint stays off, egress is restricted, and metadata access is too, with IMDSv2 required and --http-put-response-hop-limit 1, per the EKS best-practices guide; AWS’s IRSA page says pod credential isolation holds only then.

When the provider leaves you no choice but a key

A provider with no workload identity means a key, which needs more care. Read it from a secrets manager at call time, not from the environment at start, because the command injection read the environment. Keep one per tenant or product, under the provider’s spending cap, rotated through Secrets Manager, by a Lambda function where there is no managed rotation. Never use a shared store queried by a name the caller sends. None of it hides the key from code inside the process: encryption at rest, which WSO2 documents with AES-GCM and a Kubernetes Secret, protects the stored copy; the process decrypts it to use it.

Which gateway features would I switch off?

The features behind these flaws run something the caller supplied, or forward a credential. A gateway that authenticates, routes, limits and logs needs none of them.

The advisory for CVE-2026-42271 describes two preview endpoints that took command, args and env from the request body and spawned the command on the host. I would not expose one, admin role or not; the cost is testing MCP configurations in CI rather than from the console.

Custom-code guardrails. Wiz describes a registration endpoint that passed submitted Python to exec, fixed since; a guardrail that is code is remote execution with a friendlier name. On Bedrock I would let the account apply Guardrails to every invocation. The cost: guardrails become configuration, or a separate service with its own identity.

Pass-through endpoints to arbitrary URLs are the feature teams will miss most, because every upstream route then gets declared, reviewed and allow-listed, and “just proxy this URL” waits for a change. Wiz’s metadata example settles it.

CVE-2026-59822 was a fallback: per the advisory, a failed key check was replaced by an empty auth object so an upstream OAuth token could pass through. Any authentication fallback goes. WorkOS’s write-up of three MCP auth bugs states the rule I would adopt: “Never substitute a default identity or an empty auth object.” Upstream OAuth, if needed, is its own route with its own validation.

I would also keep MCP tool execution out of the model gateway: tools that write belong behind the action layer, with approval bound to exact parameters and the user’s rights, as the MCP write-access checklist spells out.

Keep the admin plane and the data plane apart

A flaw on the data path should not reach the management API, and where both run on one listener it can, through a shared process and a path-based check: the SQL injection arrived on any LLM route, and the Starlette flaw defeats path-based checks.

WSO2’s hardening page is a fair picture of the default. The controller and the policy engine each run an admin server that dumps configuration; the Helm chart ships allowed_ips: ["*"] for both, too wide for a cluster by its own account; the management API “can read and rewrite every LLM provider, proxy, and policy on the gateway”. Its advice is to narrow the ranges and keep the admin ports off the Services. I would go further: a separate listener in a network the data plane has no route to, authenticated by your identity provider, configuration changed through a reviewed pipeline, not a console. The cost: a second ingress, a break-glass procedure, and administrators who cannot fix production from the UI at midnight.

How do you test that a gateway fails closed?

Five checks, an afternoon’s work:

  1. Send a made-up Authorization: Bearer value to every route, MCP and pass-through routes included, and expect a refusal, not a session.
  2. Start the gateway with no master key, then with the documented default, and send a request. Wiz reports that before the fix no master key meant admin for every request, and that sk-1234 remained the default in Docker Compose and pip installs. The only acceptable answer is a refusal.
  3. From the data-plane network, open a connection to the management listener and expect no route, not a login page.
  4. From inside the gateway’s process, request 169.254.169.254/latest/api/token, then the role credentials. On an EKS pod with metadata restricted, expect no answer; with an EC2 instance role, credentials for the approved Bedrock profiles and nothing else.
  5. With one tenant’s credential, send another tenant’s model identifier in the body and in a header, and expect a refusal.

A gateway that passes all five can still be breached. It has less to hand over.

In practice

On the ENSO platform, I designed a central AI gateway as the single access path to Amazon Bedrock, with EU-only inference profiles enforced organisation-wide via SCP, Guardrails PII filtering, and model routing that isolates products from the model lifecycle. The gateway is one part of a shared, multi-tenant platform that powers four product capabilities; the full architecture is in the ENSO case study, and gateways of this kind are part of the enterprise LLM platform service.

Trade-offs and limits

An IAM-role design removes the long-lived key, not the session. A stolen session works until it expires, within the role’s policy and the SCP, and it is taken from the process or from the metadata service, so the IMDS and egress rules above are not extras. It does nothing for a provider with no workload identity, and the thin gateway is less convenient; its users will say so.

Patching still matters. Each catalogue entry carried a due date within two weeks of listing, and each of the four was fixed before it was listed. A thin gateway buys a smaller loss while you patch, not permission to patch later.

The catalogue shows that exploitation happened, not how often or to whom. I have not audited or run LiteLLM; this is not a verdict on it or on any vendor, but on what a self-run or shared gateway holds, and what it need not.

Common questions

Is LiteLLM safe to use in production?

Three LiteLLM flaws were listed as exploited by CISA in 2026, each with a fixed version before listing, so the question is your patch speed and exposure, not the project. I have not audited it. If you run it: patch, replace the default key, block unused MCP and test routes, restrict metadata and egress.

Should an AI gateway store API keys?

Only when the provider offers no workload identity. On Amazon Bedrock a gateway can call with an IAM role: no provider key to steal, only a session that expires and that the account’s policies cap. Where a key is unavoidable, read it at call time from a secrets manager, one per tenant, capped and rotated.

How do I secure an LLM gateway?

Hold less: a cloud role instead of provider keys where you can. Switch off features that run caller-supplied code or forward requests to arbitrary URLs. Put the management API on its own listener and network. Restrict instance metadata and egress. Then test that fabricated tokens, a missing master key and a cross-tenant model name are refused.

What is CISA’s Known Exploited Vulnerabilities catalogue, and why watch it for AI tools?

The list of flaws the US cybersecurity agency has evidence are being exploited, each with a due date for federal agencies. It ranks nothing and counts no attacks. Three LiteLLM entries between May and September 2026 make it a cheap early signal for AI infrastructure.

Sources