What is an AI gateway on Amazon Bedrock?

An AI gateway on Amazon Bedrock is the one internal service every product calls to reach a foundation model. It decides which model answers, in which Regions, after which filters and against whose budget, and it records the call. It is not a model, not a product you buy, and not a replacement for account controls.

Each rule below is enforced twice where Bedrock allows it, never in a prompt: in the gateway, where products see it, and in the AWS account, where a team that skips the gateway still meets it. The gateway is layer 1 of the enterprise AI platform architecture; Bedrock facts here are as documented in October 2026.

What the gateway enforces, and where

Concern Where it is enforced How to test it
Data residency Gateway: EU profiles, or application profiles built on them. Account: SCPs denying non-EU Regions and global profiles GetInferenceProfile from each EU source Region: only EU destinations. A global profile or non-EU Region: denied
The only path to models An SCP reserving model invocation to the gateway’s role, which only the gateway can assume From a role allowed bedrock:* and bedrock-mantle:*, call Converse and bedrock-mantle: denied by the SCP
PII filtering A guardrail on every text-generation call, applied by the account As the gateway’s role, Converse with no guardrail and a fake email address: still blocked or masked
Routing and the model lifecycle A versioned capability-to-profile table, released through the golden set Swap the model on a branch: the gate names the cases that dropped
Attribution and quotas per product Application profiles or session tags per product; token budgets in the gateway Exhaust one product’s budget: only its calls get the quota error
Logging A gateway record per call; requestMetadata in invocation logs From a product’s request ID, find model, prompt version, tokens and outcome

EU data residency on Bedrock: geographic, global or a single Region

Bedrock inference profiles decide where a prompt may be processed, so the gateway allows only EU profiles and the account refuses the rest. As of October 2026, a geographic profile routes within its geography; a global one, to any supported commercial Region. For a single Region, AWS points to direct invocation there, but some models are served only through a cross-Region profile.

Invocation logs stay in your Region; prompts and outputs that Bedrock keeps for abuse detection are stored in the destination Region. A Standard-tier guardrail is a second cross-Region path: give it the EU guardrail profile and treat its destinations like a model profile’s.

The Region-deny SCP and the gateway’s IAM policy must be tested together. Bedrock authorises the profile and the model in every destination Region, so IAM must allow each one, and a request fails if your SCPs block any one of those Regions. The Region-deny SCP must therefore allow every destination of your EU profiles, or exempt those exact profiles with bedrock:InferenceProfileArn. The second SCP turns global profiles off.

To test, call GetInferenceProfile from each EU source Region and read every destination: the prefix alone is not proof. On the Claude Sonnet 4.5 model card, the eu. profile called from London also routes to London, and from Zurich to Zurich. On real traffic, CloudTrail’s additionalEventData.inferenceRegion must name an EU Region. The cost is upkeep: each new profile means an SCP review.

Making the gateway the only path to models

A gateway that a team can skip is a convention. I would make its role the only principal allowed to invoke a model, with one more SCP, whose deny statement is:

{
  "Sid": "OnlyTheGatewayInvokesModels",
  "Effect": "Deny",
  "Action": [
    "bedrock:InvokeModel*",
    "bedrock:CreateModelInvocationJob",
    "bedrock-mantle:CreateInference"
  ],
  "Resource": "*",
  "Condition": {
    "ArnNotLike": {
      "aws:PrincipalArn": "arn:aws:iam::111122223333:role/ai-gateway"
    }
  }
}

The statement lists three actions because Converse is authorised as bedrock:InvokeModel, batch jobs have their own, and bedrock-mantle, with no Guardrails and no invocation logging, has a third. The condition uses ArnNotLike on aws:PrincipalArn, which for a role holds the role’s ARN, whoever assumed it, so the role’s trust policy is part of the control and should admit only the gateway’s workload identity.

The gateway’s own IAM policy allows invocation only on the approved EU profiles, and on foundation models only when bedrock:InferenceProfileArn names one of them. With application profiles, allow their ARNs too, and test which ARN the condition sees.

I have not run this policy: it is a design from the pages linked here, to test on one OU first. Test it from a role whose IAM policy allows bedrock:*, bedrock-mantle:* and sts:AssumeRole on the gateway’s role: Converse, a batch job and bedrock-mantle are denied, the error should name a service control policy, and the trust policy refuses the assumption.

The cost: SCPs skip the management account, so no workload runs there, and one role holds every invocation right. In my reading it also denies console users and Bedrock features that invoke models for someone: a knowledge base’s ingestion role, an agent, an evaluation job, a CI judge calling Bedrock directly. Every role added to the exception is a second path to models.

PII filtering with Bedrock Guardrails at the gateway

Every text-generation call carries a guardrail; embedding models cannot take one. Bedrock Guardrails’ sensitive information filters detect personal data in prompts and responses, probabilistically or by your regular expressions, and block or mask it. The built-in types include US, Canadian and UK identifiers but none from an EU member state, so a French social security number needs a regex.

In the account, an IAM condition on bedrock:GuardrailIdentifier refuses requests without the named guardrail, scoped to text models, though a caller can still skip input evaluation with input tags. Guardrails enforcement instead applies one to every invocation in an account or organisation, ignores caller tags by default and can exclude embedding models. I would use enforcement. It needs a guardrail version in every enforced Region, and a guardrail the gateway also attaches runs and is billed too, so let the enforced one carry the PII policy. Either way, forward no product’s guard tags: in Converse, one guardContent block limits evaluation to guarded blocks.

Tool use is the gap: personal data in tool-call arguments, tool results and tool definitions is neither blocked nor masked. I would run tool results through ApplyGuardrail, which checks any text without calling a model, at one more billed evaluation per result.

A block is not an error. Converse returns stopReason: guardrail_intervened and your configured message as the output; unless the gateway turns that into a typed policy refusal, the product shows the canned text as an answer.

Test: as the gateway’s role, call Converse with no guardrail and a fake email address. Under enforcement it is still blocked or masked, as the response’s guardrail assessment shows; under the IAM route, it is denied.

Routing and the contract: capabilities and a typed error model

Products, the retrieval API included, ask for a capability, never a model, and the gateway maps it to a profile in a versioned table. Models move on AWS’s schedule: Active, Legacy, then end-of-life, after which requests fail and nothing migrates for you; you can lose access to a Legacy model after 15 days without calls. A model change is a table change, released when the platform’s golden set passes. Within one conversation, a Claude Fable 5.2 model change is no longer invisible to the product: see what Fable 5.2 preserved thinking asks of a gateway. Routing also fits the model to the task: small and fast for classification, larger for reasoning.

The contract is capability names, input and output schemas, a request ID and three error kinds:

  • Policy refusal: a guardrail intervened, or the capability is not allowed. Never retry.
  • Quota or throttling: the product’s budget, or Bedrock’s ThrottlingException. Retry later, with backoff.
  • Provider outage: ServiceUnavailableException, InternalServerException, or ModelNotReadyException, which the SDK already retries. Retry, then fall back.

Map by exception name: ThrottlingException and ModelNotReadyException both return HTTP 429.

A fallback stays inside the residency policy. AWS suggests another Region or cross-Region inference for persistent 503s; here that means another EU profile or model. A global profile or a non-EU Region would be a violation, not resilience, and the SCPs refuse it.

Test: inject each error on a branch; check what the product receives.

Attribution and quotas per product

Bedrock’s limits belong to the account, so per-product limits belong in the gateway. Token quotas are set per model and Region for the account; in my reading nothing there is per product, so the gateway keeps a token budget per product, or one product’s burst throttles the others.

Set maxTokens per capability. Bedrock deducts input tokens plus max_tokens when a request starts, and Converse defaults it to the model’s maximum: an unset value reserves the most quota.

For cost, AWS documents two routes. An application inference profile per product, tagged and built on an EU profile, keeps its Regions but is model-specific: one per product and model, more at each model change. Or the gateway assumes its role per product with session tags and a session name, caching the credentials, since assuming it per request can exceed STS quotas; the trust policy must allow sts:TagSession, and in my reading the only-path SCP still matches the role’s ARN. Either way, tags appear up to 24 hours after activation, are not retroactive, and give daily totals.

Test: exhaust one product’s budget; reconcile a day’s Cost Explorer total per tag with the gateway’s count.

Logging: one record per call

The gateway writes one record per call: request ID, product, capability, profile, prompt version, guardrail outcome, tokens, estimated cost, error kind and latency, and it feeds the evaluation loop (layer 4). Prompt text stays out of it, and so does the raw guardrail trace, whose match field holds the original personal data.

Bedrock’s model invocation logging is the backstop. It is off by default, writes full request and response bodies to CloudWatch Logs or S3 in the same account and Region, and in CloudWatch Logs its input field keeps the original request even when a guardrail masked it. Treat those logs as personal data, erasure included.

Every call comes from the gateway’s role, so products are told apart by a per-product session name in the log’s identity field, or by requestMetadata, which Bedrock records only when invocation logging is on and never enforces. No personal data goes in either.

Test: trace one product request ID through the gateway record, the invocation log and CloudTrail.

In practice

On the ENSO platform, I designed a central AI gateway as the single access path to Amazon Bedrock, with EU-only inference profiles enforced organisation-wide via SCP, Guardrails PII filtering, and model routing that isolates products from the model lifecycle. The gateway is one part of a shared, multi-tenant platform that powers four product capabilities; the full architecture is in the ENSO case study, and gateways of this kind are part of the enterprise LLM platform service.

Trade-offs and limits

Centralising has a cost. The gateway is one more component to run, and because every model call passes through it, it has to be highly available and fast. It can also become a bottleneck for change if its owners cannot keep up with product needs, which is why the contract stays small and stable.

A gateway governs only the traffic that goes through it, and the account-level controls cover the rest. They have their own cost: one blocked destination Region fails the whole call, and when several policy types deny a request, the error names only one.

The gateway is itself a target, and what it holds is a design question in its own right; Don’t give your AI gateway the keys takes it up.

Inference profiles do not support Provisioned Throughput.

A single product with a single model may not need a gateway yet. The account-level residency rule does not depend on one.

A gateway does not make an answer right, nor stop an instruction injected through a retrieved document or a tool result: it filters only what it reads.

Common questions

How do you keep Amazon Bedrock inference in the EU?

Use only EU geographic inference profiles, backed by two service control policies: one denying every Region outside the EU, one turning global profiles off. Check every destination with GetInferenceProfile from each source Region, and confirm in CloudTrail that inferenceRegion is always an EU Region.

Do you need an AI gateway if you use Amazon Bedrock?

Not necessarily. A single product with a single model may not need one yet, and the account-level residency rule works without it. From the second product, residency, PII filtering, model choice and per-product cost recur. A gateway settles them once.

How do you track Amazon Bedrock costs per team?

Two documented routes: a tagged application inference profile per team and model, or a gateway assuming its Bedrock role per team with session tags. Activated as cost allocation tags, either gives daily totals in Cost Explorer. For per-request detail, set requestMetadata and read token counts from invocation logs.

Do Bedrock Guardrails filter personal data in prompts?

Yes, in prompts and responses: sensitive information filters detect personal data and block or mask it. They evaluate text only: tool-call arguments and tool results pass untouched. A block arrives as a normal response with the stop reason guardrail_intervened, which the gateway must turn into a refusal.

Sources