Red Hat AI Inference 3.5 MaaS governance on AKS

Running AI models on public cloud Kubernetes clusters offers great flexibility, but it creates an immediate governance challenge: how do you secure endpoints, authenticate developers, and prevent runaway GPU costs across teams? For platform engineers, Red Hat AI Inference addresses this by bringing multi-tenant governance to any Kubernetes distribution.

With Red Hat AI Inference 3.5, the inference stack (vLLM, llm-d, KServe) and now the governance layer (MaaS) can run on non-OpenShift Kubernetes distributions. This article focuses on what that looks like on Azure Kubernetes Service (AKS).

Alexa Griffith's article, Red Hat AI Inference on Amazon EKS: Exploring the Kubernetes resources, explored Red Hat AI's inference layer on non-OpenShift, managed Kubernetes distributions—often referred to as xKS (such as EKS, AKS, or GKE). The inference layer includes the platform components that deploy and serve AI models: cert-manager, Istio, KServe, llm-d, LeaderWorkerSet, the Red Hat AI Inference operator, and the cloud manager. This article builds on that foundation by focusing on the top governance layer: Model-as-a-Service (MaaS). MaaS is the multi-tenant API governance layer that controls who can access your models, how much they can use them, and how access is authenticated.

I deployed Red Hat AI Inference 3.5 with MaaS enabled on AKS to see exactly what Kubernetes resources the governance layer creates and how they connect to the inference platform Alexa described.

What changes on xKS: OpenShift vs. Kubernetes

OpenShift comes with built-in infrastructure to simplify how you install, secure, and authenticate workloads. This includes its own Operator Lifecycle Manager (OLM), a default certificate authority for Transport Layer Security (TLS), and a built-in OAuth server for identity. When you deploy MaaS on OpenShift, you can rely on all of those features.

A vanilla Kubernetes distribution lacks these integrated services. You need to supply exgternal tools: Helm charts for installation, cert-manager for TLS certificates, and an OIDC-compliant identity provider (such as Azure Active Directory on AKS).

What MaaS adds to the platform

Alexa's post covered the inference platform—everything from cert-manager through the cloud manager. MaaS adds 2 layers on top:

[Inference platform—covered by Alexa]
cert-manager → Istio / Gateway API → KServe / llm-d → RHAII Operator → Cloud Manager

[Governance layer—this article]
RHCL (Kuadrant)    → Authentication and rate limiting engine
MaaS               → Subscriptions, API keys, model access control

The request flow through the governance layer:

Client
  → MaaS Gateway
    → Authorino (RHCL)—validates API key or JWT
      → Limitador (RHCL)—checks token rate limit
        → Model backend (vLLM via KServe)
          → Response

Failure cases:

Scenario	Response
No API key or invalid key	401 Unauthorized
Valid key but wrong subscription tier	403 Forbidden
Valid key but quota exceeded	429 Too Many Requests
Everything valid	200 + model output

My cluster

Azure Kubernetes Service (AKS), East US region
Kubernetes 1.34.8
4 nodes (3 CPU + 1 GPU with NVIDIA T4)
Red Hat AI Inference 3.5 with MaaS enabled

$ kubectl get nodes
NAME                                STATUS   ROLES    AGE   VERSION
aks-gpupoolt4-28259789-vmss000002   Ready    <none>   10d   v1.34.8
aks-nodepool1-11805091-vmss00000q   Ready    <none>   10d   v1.34.8
aks-nodepool1-11805091-vmss00000r   Ready    <none>   10d   v1.34.8
aks-nodepool1-11805091-vmss00000s   Ready    <none>   10d   v1.34.8

Note: MaaS runs on CPU; only model serving requires GPU.

Deploying a model before the MaaS state

Before we explore the governance layer, let's deploy a model. This is the starting point: a running model with no authentication, no rate limiting, and no API keys. MaaS changes that later.

The LLMInferenceService Custom Resource Definition (CRD) is the single object you apply to deploy a model. Here's the YAML for a Qwen3-0.6B model:

apiVersion: serving.kserve.io/v1alpha2
kind: LLMInferenceService
metadata:
  name: single-gpu
  namespace: llm
spec:
  model:
    uri: hf://Qwen/Qwen3-0.6B
    name: Qwen/Qwen3-0.6B
  replicas: 1
  router:
    scheduler:
      template:
        imagePullSecrets:
        - name: rhai-pull-secret
        containers:
        - name: main
    route: {}
    gateway: {}
  template:
    imagePullSecrets:
    - name: rhai-pull-secret
    containers:
      - name: main
        resources:
          limits:
            cpu: '4'
            memory: 32Gi
            nvidia.com/gpu: "1"
          requests:
            cpu: '2'
            memory: 16Gi
            nvidia.com/gpu: "1"

After you apply the single YAML file, the llmisvc-controller-manager handles everything from there. It creates a Deployment for the vLLM server, a router-scheduler Deployment for intelligent request routing, an InferencePool for load balancing, an HTTPRoute connecting the gateway to the model, a Service for internal networking, and a self-signed TLS certificate. You specify the model and resources, and the controller builds the rest.

$ kubectl get llminferenceservice -n llm
NAME         URL                                    READY   REASON   AGE
single-gpu   http://<INFERENCE_GATEWAY_IP>/llm/single-gpu   True             15m

The LLMInferenceService shows Ready: True with an externally routable URL. The model is accessible at http://<INFERENCE_GATEWAY_IP>/llm/single-gpu.

$ kubectl get pods -n llm
NAME                                                 READY   STATUS    RESTARTS   AGE
single-gpu-kserve-747b944f87-xshkv                   1/1     Running   0          14m
single-gpu-kserve-router-scheduler-7b9bbf8cd-nh4cj   1/1     Running   0          14m

There are 2 pods running:

single-gpu-kserve: The vLLM server running Qwen3-0.6B. Inside one of our AKS pods, we have our model ready to produce tokens. It started with an init container (storage-initializer) downloading the model weights from Hugging Face, then launched vLLM v0.24.0 with the CUDA runtime image (vllm-cuda-rhel9, 9.3 GB). Kubernetes automatically scheduled this pod onto the T4 GPU node because the pod requested nvidia.com/gpu: 1.
single-gpu-kserve-router-scheduler: The intelligent request router distributing inference traffic across model replicas (in this case, only 1).

This is what the inference platform provides out of the box. The model is live, reachable, and serving completions. Here's a direct call to the model through the inference gateway to see it in action:

$ curl -s http://<INFERENCE_GATEWAY_IP>/llm/single-gpu/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"Qwen/Qwen3-0.6B",
         "messages":[{"role":"user","content":"What is Red Hat?"}],
         "max_tokens":100,
         "chat_template_kwargs":{"enable_thinking":false}}' \
    | jq -r '.choices[0].message.content'

Output:

Red Hat is an American enterprise software company that provides open source solutions,
including Red Hat Enterprise Linux, OpenShift, and Ansible, to organizations worldwide.

It returns 200, with no API key, no authentication, and no rate limiting. It's a raw HTTP request to a public IP address. The endpoint is OpenAI-compatible and supports all vLLM serving arguments (here we pass enable_thinking: false to disable Qwen3's chain-of-thought reasoning for a cleaner response).

The same request through the MaaS gateway (<MAAS_GATEWAY_IP>) also returns 200. So far, both gateways are wide open because no MaaSSubscription or MaaSAuthPolicy resources exist yet.

The problem is that anyone with the IP address can call any model, consume unlimited GPU resources, and there's no way to track usage. That's where MaaS comes in.

Inside the MaaS deployment

The governance layer installs across several namespaces. Here's the full picture, organized by component:

MaaS Governance Layer
├── RHCL / Kuadrant (Authentication + Rate Limiting)
│   ├── Namespace: kuadrant-operators
│   │   ├── kuadrant-operator-controller-manager
│   │   ├── authorino-operator
│   │   ├── limitador-operator-controller-manager
│   │   └── dns-operator-controller-manager
│   ├── Namespace: kuadrant-system
│   │   ├── authorino (validates API keys + JWTs)
│   │   └── limitador-limitador (enforces rate limits)
│   └── CRDs: authpolicies, ratelimitpolicies, tokenratelimitpolicies, ...
│
├── MaaS Platform (API Governance)
│   ├── Namespace: redhat-ods-applications
│   │   ├── ai-gateway-operator (deploys MaaS components)
│   │   ├── maas-controller (reconciles MaaS CRDs → Kuadrant policies)
│   │   ├── maas-default-gateway-istio (policy-enforced gateway)
│   │   ├── payload-processing (request body routing)
│   │   └── payload-pre-processing
│   ├── Namespace: redhat-ai-gateway-infra
│   │   ├── maas-api (REST API for subscriptions + API keys)
│   │   └── maas-postgres (PostgreSQL — API key hash storage)
│   └── CRDs: tenants, aitenants, maassubscriptions, maasauthpolicies,
│             maasmodelrefs, externalmodels, configs, maastenantconfigs
│
└── Namespace: models-as-a-service
    └── (MaaS subscription and auth policy resources go here)

All of this is additive. The inference platform from Alexa's article remains untouched. MaaS layers on top of it.

Red Hat Connectivity Link / Kuadrant (policy enforcement engine)

Red Hat Connectivity Link provides the core policy enforcement mechanisms for authentication and rate limiting in the Model-as-a-Service architecture.

The problem it solves

Red Hat Connectivity Link is the Red Hat productization of Kuadrant. Without it, the gateway routes traffic straight to models with no checks. Connectivity Link adds 2 enforcement engines:

Authorino: Validates API keys and JSON Web Tokens (JWTs) on every request (returns 401 on failure)
Limitador: Tracks token counters per subscription (returns 429 when exceeded)

Deployed resources

$ kubectl get deployment -n kuadrant-system
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
authorino             1/1     1            1           4h57m
limitador-limitador   1/1     1            1           4h57m

These 2 processes handle every request at the gateway. Authorino checks API keys against the MaaS API and validates JWT tokens against the configured OIDC provider (on AKS, it's Azure Active Directory). Limitador checks token counters against the rate limits defined in each subscription.

4 operator controllers in the kuadrant-operators namespace manage the lifecycle of these engines, but MaaS users don't interact with them directly.

Why it matters

Red Hat Connectivity Link installs 16 CRDs, but only 2 matter for MaaS: authpolicies.kuadrant.io and tokenratelimitpolicies.kuadrant.io. Users don't create these directly. When you create a MaaSSubscription, the maas-controller automatically translates it into a Kuadrant TokenRateLimitPolicy. Limitador then enforces that policy at the gateway on every request. Without Connectivity Link, subscriptions and auth policies would be inert YAML with nothing to execute them.

AI Gateway Operator (MaaS deployment chain)

Deploying MaaS on AKS requires several operators working in sequence. The AI Gateway Operator coordinates that chain.

The problem it solves

In Red Hat AI Inference 3.5, you deploy MaaS through a chain of operators rather than directly. The main opendatahub-operator provisions the ai-gateway-operator, which in turn deploys the MaaS components. This architecture allows all gateway-related services (MaaS, batch gateway, future components) to be managed by a single module operator.

Deployed resources

The AI Gateway Operator runs in the main applications namespace:

$ kubectl get deployment ai-gateway-operator -n redhat-ods-applications
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
ai-gateway-operator   1/1     1            1           4h53m

The deployment chain works as follows:

The Helm chart installs the underlying platform manager (opendatahub-operator) and creates a Platform CR with MaaS enabled.
The opendatahub-operator detects the platform type (AKS) and deploys ai-gateway-operator.
The ai-gateway-operator reads the AIGateway CR, selects xKS-specific manifests, and deploys maas-controller.
The maas-controller sets up the MaaS API, gateway, auth policies, and rate limit policies.

The Helm values that enable this chain:

components:
  aigateway:
    managementState: Managed
    modelsAsAService:
      managementState: Managed

Why it matters

This operator chain means customers interact only with the Helm chart. They set modelsAsAService.managementState: Managed in their values, and the operators handle everything downstream—selecting the right manifests for AKS (cert-manager TLS instead of OpenShift service-CA, no monitoring CRDs), deploying the controller, and reconciling the gateway.

MaaS controller (policy reconciliation)

The MaaS controller automates access control by translating custom governance resources into enforcement rules.

The problem it solves

Kubernetes administrators shouldn't need to understand Kuadrant's AuthPolicy and TokenRateLimitPolicy schemas to manage model access. The maas-controller solves this. You create a MaaSSubscription (an object defining who can access which models and at what rate limit), and the controller generates the corresponding Kuadrant policies automatically.

Deployed resources

$ kubectl get deployment maas-controller -n redhat-ods-applications
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
maas-controller   1/1     1            1           4h53m

The controller performs 2 roles:

Reconciler: Watches MaaSSubscription and MaaSAuthPolicy resources. When one is created or modified, the controller creates or updates the corresponding Kuadrant AuthPolicy and TokenRateLimitPolicy objects.
Webhook server: Validates MaaS resources before they're persisted. It rejects invalid configurations (such as a subscription referencing a model that doesn't exist) before they can cause problems downstream.

Why it matters

Without the controller, every subscription change would require manually editing Kuadrant policies. You'd need to understand the AuthPolicy schema, the Limitador counter format, and the correct gateway references. The controller eliminates that. Create a MaaSSubscription, and the system enforces rate limiting within seconds.

MaaS API (key management and subscription service)

Managing API key lifecycles by hand is error-prone. The MaaS API gives platform teams a secure, automated service to issue, validate, and revoke keys without requiring direct cluster admin rights.

The problem it solves

API key lifecycle management requires a stateful service. Keys need to be generated, hashed, stored, validated on every request, and revoked. Static Kubernetes resources alone can't handle this. It requires a running server with database access.

Deployed resources

The MaaS API and its PostgreSQL backend run in a dedicated infrastructure namespace:

$ kubectl get deployment -n redhat-ai-gateway-infra
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
maas-api        1/1     1            1           4h48m
maas-postgres   1/1     1            1           4h56m

The maas-api deployment handles subscription operations and API key management. It exposes HTTPS on port 8443 as a ClusterIP service, using cert-manager for TLS on AKS. It isn't directly reachable from outside the cluster; the MaaS gateway fronts it.
The maas-postgres deployment stores API key hashes, subscription metadata, and tenant configuration.

How API keys work

When a user creates an API key, the MaaS API:

Generates a random sk-oai-* key.
Hashes it with SHA-256.
Stores only the hash (plus metadata: username, groups, subscription binding, and expiry) in PostgreSQL.
Returns the plaintext key once (afterward, it's irretrievable).

On every inference request, Authorino calls the MaaS API's internal validation endpoint. The API hashes the incoming key, looks up the hash in Postgres, and returns the associated identity and subscription. Otherwise, it rejects the request.

Why it matters

Without the MaaS API, there's no programmatic way to issue, validate, or revoke API keys. Consumers would need direct cluster access (kubectl) to authenticate, which defeats the purpose of a multi-tenant inference service.

MaaS gateway (policy-enforced entry point)

The MaaS gateway inspects and enforces policies on incoming requests before forwarding them to backend model engines.

The problem it solves

The inference gateway (deployed by KServe, covered in Alexa's article) provides raw model access with no authentication or rate limiting. MaaS deploys a second gateway (the maas-default-gateway) that applies Kuadrant policies to every request before it reaches the model backend.

Deployed resources

$ kubectl get gateway -A
NAMESPACE                 NAME                   CLASS   ADDRESS          PROGRAMMED   AGE
redhat-ods-applications   inference-gateway      istio   <INFERENCE_GATEWAY_IP>   True         4h50m
redhat-ods-applications   maas-default-gateway   istio   <MAAS_GATEWAY_IP>    True         4h49m

Both show PROGRAMMED: True with Azure Load Balancer public IPs. The MaaS gateway's service confirms it's externally reachable:

$ kubectl get svc maas-default-gateway-istio -n redhat-ods-applications
NAME                         TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                                      AGE
maas-default-gateway-istio   LoadBalancer   10.0.223.143   <MAAS_GATEWAY_IP>   15021:31791/TCP,80:32628/TCP,443:31374/TCP   4h53m

Why it matters

MaaS provides an alternative, policy-enforced gateway. It doesn't replace the inference gateway. You can access models directly or through governed access.

MaaS CRDs (the governance API surface)

The MaaS controller installs 8 CRDs under the maas.opendatahub.io API group:

$ kubectl get crd | grep maas
aitenants.maas.opendatahub.io                          2026-08-04T15:43:43Z
configs.maas.opendatahub.io                            2026-08-04T15:43:43Z
externalmodels.maas.opendatahub.io                     2026-08-04T15:43:44Z
maasauthpolicies.maas.opendatahub.io                   2026-08-04T15:43:44Z
maasmodelrefs.maas.opendatahub.io                      2026-08-04T15:43:44Z
maassubscriptions.maas.opendatahub.io                  2026-08-04T15:43:44Z
maastenantconfigs.maas.opendatahub.io                  2026-08-04T15:43:44Z
tenants.maas.opendatahub.io                            2026-08-04T15:43:44Z

These define the API that administrators use to manage model governance.

Tenant

The Tenant object is the singleton configuration object for the MaaS platform. It tells the controller which gateway to attach policies to, how long API keys can live, and (on AKS) which OIDC provider to use for identity.

$ kubectl get tenant -A
NAMESPACE             NAME             READY   REASON   AGE
models-as-a-service   default-tenant                    3m

On AKS, the Tenant is where Azure AD OIDC is wired in:

spec:
  externalOIDC:
    issuerUrl: "https://login.microsoftonline.com/<tenant-id>/v2.0"
    clientId: "<app-registration-client-id>"
  gatewayRef:
    namespace: redhat-ods-applications
    name: maas-default-gateway

MaaSSubscription

A MaaSSubscription defines a subscription plan with per-model token rate limits. When you create one, the maas-controller automatically generates the corresponding Kuadrant TokenRateLimitPolicy.

apiVersion: maas.opendatahub.io/v1alpha1
kind: MaaSSubscription
metadata:
  name: demo-subscription
  namespace: models-as-a-service
spec:
  owner:
    groups:
      - name: premium-users
  modelRefs:
    - name: granite-7b
      namespace: llm
      tokenRateLimits:
        - limit: 10000
          window: "1h"

MaaSAuthPolicy

A MaaSAuthPolicy defines who can access which models. When you create one, the controller generates a Kuadrant AuthPolicy connecting Authorino to validate API keys through the MaaS API callback and verify subscription access.

MaaSModelRef

A MaaSModelRef is a pointer to a model—either an on-cluster LLMInferenceService or an external provider (ExternalModel). MaaS uses this to build its model catalog and determine where to route governed requests.

Additional CRDs

ExternalModel defines AI models hosted outside the cluster, such as OpenAI, Anthropic, or Azure OpenAI. These models go through the same API key authentication, rate limiting, and usage tracking as locally served models. AITenant creates isolated tenant namespaces with independent gateways and Role-Based Access Control (RBAC) for multi-tenant deployments.

Activating governance

We've explored every component MaaS deploys. Now let's see it in action.

Before governance: The model responds to anyone

Right now, the MaaS gateway is still wide open. This is because no MaaSSubscription or MaaSAuthPolicy resources exist yet:

$ curl -s http://<MAAS_GATEWAY_IP>/llm/single-gpu/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"Qwen/Qwen3-0.6B",
         "messages":[{"role":"user","content":"Write a haiku about why interns are cool"}],
         "max_tokens":100,
         "chat_template_kwargs":{"enable_thinking":false}}' | python3 -m json.tool
{
    "id": "chatcmpl-2bb808e1-65b2-4ec3-9859-90826b6373c8",
    "object": "chat.completion",
    "created": 1785940212,
    "model": "Qwen/Qwen3-0.6B",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "Why interns are cool —\nThey're curious, they're kind,\nThey spark the world with their spark."
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 22,
        "total_tokens": 46,
        "completion_tokens": 24
    }
}

No API key, no authentication header, yet the model still answers anyone who asks. Applying a YAML file will change that.

MaaSSubscription (defining a rate-limited plan)

To configure a rate-limited plan, define a MaaSSubscription resource specifying token quotas and user permissions:

apiVersion: maas.opendatahub.io/v1alpha1
kind: MaaSSubscription
metadata:
  name: demo-subscription
  namespace: models-as-a-service
spec:
  owner:
    users:
      - matthew
  modelRefs:
    - name: single-gpu
      namespace: llm
      tokenRateLimits:
        - limit: 1000
          window: "1m"

This creates a subscription that grants user matthew access to the single-gpu model with a rate limit of 1,000 tokens per minute.

$ kubectl apply -f maas-demo/03-maassubscription.yaml
maassubscription.maas.opendatahub.io/demo-subscription created

MaaSAuthPolicy: Granting access

Apply a MaaSAuthPolicy manifest to enforce access controls and specify authorized users for the model:

apiVersion: maas.opendatahub.io/v1alpha1
kind: MaaSAuthPolicy
metadata:
  name: demo-access
  namespace: models-as-a-service
spec:
  modelRefs:
    - name: single-gpu
      namespace: llm
  subjects:
    users:
      - matthew

This tells the maas-controller to create a Kuadrant AuthPolicy connecting Authorino to validate API keys for this model. Only users listed in subjects can access it.

$ kubectl apply -f maas-demo/04-maasauthpolicy.yaml
maasauthpolicy.maas.opendatahub.io/demo-access created

Verifying the governance pairing

Within seconds, the maas-controller reconciles both resources and creates the corresponding Kuadrant policies. The subscription, auth policy, and model ref all show Active / Ready:

$ kubectl get maassubscription -n models-as-a-service
NAME                PHASE    PRIORITY   AGE
demo-subscription   Active   0          7m
$ kubectl get maasauthpolicy -n models-as-a-service
NAME          PHASE    AGE
demo-access   Active   7m
$ kubectl get maasmodelref -n llm
NAME         PHASE   ENDPOINT                 HTTPROUTE                 GATEWAY                AGE
single-gpu   Ready   https://<MAAS_GATEWAY_IP>/   single-gpu-kserve-route   maas-default-gateway   13h

After governance: 401

The exact same request, now that governance is active:

$ curl -s -o /dev/null -w "%{http_code}" http://<MAAS_GATEWAY_IP>/llm/single-gpu/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"Qwen/Qwen3-0.6B",
         "messages":[{"role":"user","content":"Write a haiku about why interns are cool"}],
         "max_tokens":100,
         "chat_template_kwargs":{"enable_thinking":false}}'
401

401 Unauthorized. Same request, same gateway, same model, no haiku. Authorino intercepted the request, found no API key, and rejected it before it ever reached vLLM. It never reached the inference layer.

Minting an API key

To get through, we'll need an API key. The MaaS API mints keys for authenticated users. After authenticating with a Kubernetes service account token, the API returns a one-time sk-oai-* key:

$ TOKEN=$(kubectl create token default -n models-as-a-service \
    --duration=1h --audience=https://kubernetes.default.svc)
$ curl -s http://<MAAS_GATEWAY_IP>/v1/api-keys \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '{"name":"blog-demo-key","expirationDays":7}'
{
    "key": "sk-oai-****...****",
    "name": "blog-demo-key",
    "subscription": "single-gpu-subscription",
    "expiresAt": "2026-08-12T15:41:03Z"
}

Now we can make this request.

Authenticated request: 200

The same request, this time with a valid API key. Note the path changes: the MaaS gateway uses /v1/chat/completions with a namespaced model identifier (publishers/<namespace>/models/<model>) instead of the namespace-prefixed path used by the inference gateway:

$ curl -s http://<MAAS_GATEWAY_IP>/v1/chat/completions \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"publishers/llm/models/Qwen/Qwen3-0.6B",
         "messages":[{"role":"user","content":"Write a haiku about why interns are cool"}],
         "max_tokens":100,
         "chat_template_kwargs":{"enable_thinking":false}}'
{
    "model": "Qwen/Qwen3-0.6B",
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "Why interns are cool —\nFresh eyes see clearly, Questions no one thought to ask,\nnew growth in fresh directions"
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 21,
        "completion_tokens": 28
    }
}

The haiku is back. Authorino validated the API key against the MaaS API, confirmed the user's subscription covers this model, and let the request through.

Rate limiting: 429

The subscription allows 1,000 tokens per minute. What happens when we exceed it?

$ for i in $(seq 1 5); do
    curl -s -o /dev/null -w "Request $i: HTTP %{http_code}\n" \
      http://<MAAS_GATEWAY_IP>/v1/chat/completions \
      -H "Authorization: Bearer $API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"model":"publishers/llm/models/Qwen/Qwen3-0.6B",
           "messages":[{"role":"user","content":"Write a long essay about Kubernetes"}],
           "max_tokens":100}'
  done
Request 1: HTTP 200
Request 2: HTTP 429
Request 3: HTTP 429
Request 4: HTTP 429
Request 5: HTTP 429

The first request consumes the token budget. Limitador rejects every subsequent request with 429 Too Many Requests until the rate limit window resets. No model compute is wasted on rejected requests; Limitador blocks them at the gateway before they reach vLLM.

Bringing it together

This article walked through the full MaaS governance stack on AKS.

Deployed a model with a single LLMInferenceService YAML file and got a live endpoint that responded to anyone.
Explored the governance infrastructure: Red Hat Connectivity Link/Kuadrant for policy enforcement, the MaaS controller for reconciliation, the MaaS API for key management, and the MaaS gateway as the policy-enforced entry point.
Activated governance: With 2 short YAML files, a MaaSSubscription and a MaaSAuthPolicy are now applied with each request.
Proved all 3 enforcement cases live:
- 401: Unauthenticated request rejected before reaching the model.
- 200: Authenticated request with a valid API key returns a model response.
- 429: Rate limit exceeded, request blocked at the gateway.

The inference platform Alexa described gets models running. MaaS determines who gets to use them, how much, and tracks it all. With MaaS running on AKS, platform engineers can now deliver governed, multi-tenant LLM endpoints across public cloud Kubernetes wihtout sacrificing access control or rate limiting as we bring Red Hat AI capabilities to any Kubernetes distribution.

To set up multi-tenant AI governance on your cluster, follow the Red Hat AI Inference on xKS documentation for step-by-step installation instructions, or explore Red Hat OpenShift AI to see the full platform in action.

Red Hat AI Inference 3.5 MaaS governance on AKS

Adding multi-tenant API governance to Red Hat AI Inference on Azure Kubernetes Service (AKS)

What changes on xKS: OpenShift vs. Kubernetes

What MaaS adds to the platform

My cluster

Deploying a model before the MaaS state

Inside the MaaS deployment

Red Hat Connectivity Link / Kuadrant (policy enforcement engine)

The problem it solves

Deployed resources

Why it matters

AI Gateway Operator (MaaS deployment chain)

The problem it solves

Deployed resources

Why it matters

MaaS controller (policy reconciliation)

The problem it solves

Deployed resources

Why it matters

MaaS API (key management and subscription service)

The problem it solves

Deployed resources

How API keys work

Why it matters

MaaS gateway (policy-enforced entry point)

The problem it solves

Deployed resources

Why it matters

MaaS CRDs (the governance API surface)

Tenant

MaaSSubscription

MaaSAuthPolicy

MaaSModelRef

Additional CRDs

Activating governance

Before governance: The model responds to anyone

MaaSSubscription (defining a rate-limited plan)

MaaSAuthPolicy: Granting access

Verifying the governance pairing

After governance: 401

Minting an API key

Authenticated request: 200

Rate limiting: 429

Bringing it together

Platforms

Build

Quicklinks

Communicate

RED HAT DEVELOPER

Red Hat legal and privacy links

Red Hat legal and privacy links