A deep dive into Stakater's Multi-Tenant Operator

Welcome back to our in-depth exploration of Multi Tenant Operator (MTO) by Stakater. In our previous blog, we covered the fundamental aspects of MTO and its role in achieving efficient and secure multi-tenancy in Red Hat OpenShift. This time, we're diving deeper into MTO’s core features.

We will navigate through the three core features of MTO: Quotas, Tenants, and Console.

Installation

MTO is available on both Red Hat Marketplace and OperatorHub. Installation instructions are available on the official MTO Docs on installation.

Core features

In the realm of multi-tenancy, the concept of a "Tenant" is akin to having distinct football clubs like Arsenal, Chelsea, and PSG within a single league. Each club (Tenant) operates independently, with its own players (users) and resources (quotas, labels, and integrations).

Quotas

Quota is essential for the creation of a Tenant. Quotas are like the financial budgets assigned to each football club. They define the maximum amount of resources (like CPU, memory, storage) that each Tenant (club) can use within the OpenShift cluster (league).

Custom Resource (YAML)

A Quota in MTO is defined as a custom resource (CR). Here's an example where Bill, a cluster administrator, creates a medium quota:

apiVersion: tenantoperator.stakater.com/v1beta1
kind: Quota
metadata:
 name: medium
spec:
 limitrange:
   limits:
     - max:
         cpu: '2'
         memory: 1Gi
       min:
         cpu: 200m
         memory: 100Mi
       type: Pod
 resourcequota:
   hard:
     configmaps: '10'
     requests.cpu: '5'
     requests.memory: 5Gi
     secrets: '10'
     services: '10'
     services.loadbalancers: '2'

• Resource quotas: The resourcequota field in the Quota CR specifies the maximum amount of resources a Tenant can use. It is like setting a budget cap for each club.
• Limit ranges: The limitrange field defines minimum and maximum usage limits for resources like CPU and memory per Pod, similar to setting limits on player salaries.
• Linking Quotas to Tenants: The quota is then linked to a Tenant, ensuring that the Tenant’s resource usage stays within the defined limits.
• Limiting PersistentVolume: To restrict storage usage, the quota can include a requests.storage field. This is akin to setting a maximum limit on the stadium size.
• StorageClass restrictions: For more granular control, storage limits can be set per StorageClass, much like having specific budgets for different categories like player acquisitions, medical facilities, or youth academies.

Bill then links this quota to a Tenant named devops, managed by Anna:

apiVersion: tenantoperator.stakater.com/v1beta2
kind: Tenant
metadata:
 name: devops
spec:
 onDelete:
   cleanAppProject: true
   cleanNamespaces: false
 owners:
   users:
     - anna@stakater.com
 quota: medium

When Anna, managing the devops Tenant, tries to deploy resources beyond her quota, she receives an error, ensuring that her usage stays within the club's (Tenant's) budget:

Error creating: pods "myapp-devops-thd98" is forbidden: exceeded quota: medium, requested: requests.cpu=200Mi, used: requests.cpu=6, limited: requests.cpu=16

In a multi-tenant environment like a shared OpenShift cluster, Quotas ensure that resources are fairly distributed and that one department’s (Tenant’s) excessive usage doesn’t impact others.

More information about Quotas can be found on the official MTO Docs on Quotas.

Tenants

MTO's Tenant feature ensures that clubs (Tenants) coexist harmoniously, with clearly defined boundaries and operational rules. It's about maximizing the league's (cluster's) potential while maintaining each club's (Tenant’s) unique identity and needs.

Custom resource (YAML)

A Tenant in MTO is defined using a custom resource (CR). The smallest valid Tenant definition is straightforward, as shown below:

apiVersion: tenantoperator.stakater.com/v1beta2
kind: Tenant
metadata:
 name: arsenal
spec:
 quota: medium

Let’s focus on some key aspects of the Tenant CR:

• Owners, Editors, Viewers (roles and permissions): Like assigning roles in a club—managers (owners), coaches (editors), and scouts (viewers)—these fields define the permissions for users within the Tenant.
• Quota (resource limits): Just as a club has a budget cap, the quota specifies resource limits for the Tenant, promoting efficient usage.
• Argo CD integration (tactical tools): Integrating with Argo CD is akin to providing the club with tactical tools and resources, dictated by the Tenant’s requirements.
• Template instances (playbooks): Like deploying specific playbooks to different squads, this feature applies certain templates to namespaces under the Tenant.

The complete set of options available can be found in the official MTO Docs on Tenants.

E-commerce example

Now that we understand the basic structure of Tenants, let's see how this works in a technical scenario at the e-commerce company NordMart. NordMart faced challenges in managing their single OpenShift cluster used by various departments like sales, marketing, and design. To promote efficient management, security, and resource optimization, NordMart adopted MTO’s Tenants feature. This enabled the creation of distinct Tenants for each department.

Each department was set up as a Tenant with customized resource quotas, role assignments, and specific namespaces—sales-tenant for sales with high resources and sandbox environments, marketing-tenant for marketing with moderate quotas and isolated testing namespaces, and design-tenant for design with flexible resources and hibernation configurations. This setup not only streamlined resource allocation and operational efficiency but also ensured departmental autonomy and security within the shared cluster.

1. Sales Tenant: sales-tenant

apiVersion: tenantoperator.stakater.com/v1beta2
kind: Tenant
metadata:
  name: sales-tenant
spec:
  owners:
    users:
      - seniorsales@nordmart.com
      - salesmanager@nordmart.com
  quota: medium
  namespaces:
    withTenantPrefix:
      - customer-engagement
      - sales-analytics

2. Marketing Tenant: marketing-tenant

apiVersion: tenantoperator.stakater.com/v1beta2
kind: Tenant
metadata:
  name: marketing-tenant
spec:
  owners:
    users:
      - marketinglead@nordmart.com
  quota: medium
  namespaces:
    withTenantPrefix:
      - campaign-planning
      - market-research

3. Design Tenant: design-tenant

apiVersion: tenantoperator.stakater.com/v1beta2
kind: Tenant
metadata:
  name: design-tenant
spec:
  owners:
    users:
      - designdirector@nordmart.com
      - leaddesigner@nordmart.com
  quota: medium
  namespaces:
    withTenantPrefix:
      - ui-ux
      - graphic-design

MTO Console

Now that we have installed all the resources, let’s see how they are shown on the MTO Console when logged in as an MTO Admin.

The dashboard is shown below in Figure 1.

Figure 2 shows the Quotas.

Lastly, Figure 3 shows the Tenants.

Conclusion: Mastering multitenancy with MTO

For Kubernetes multi-tenancy, Stakater's Multi Tenant Operator (MTO) plays a role akin to a skilled coach in a football league, orchestrating harmony and peak performance among diverse teams. MTO ensures each Tenant operates optimally, with the right resources and strategies in place.

Throughout this article, we've seen how MTO's Tenants and Quotas features contribute to a well-organized, efficient environment. In essence, MTO empowers administrators to enforce security, ensure operational efficiency, and maintain uniformity across a multi-tenant landscape.