Overview
Regulated estates keep clusters and data inside their own data centre, often with no outbound network access and change control measured in weeks. Upgrades, certificate rotation and image updates all have to work through a private registry and a formal change process. Tier 3 managed services fit this model because AceMQ works inside the customer's boundary.
Challenge
The customer runs OpenShift and a separate air-gapped Kubernetes cluster on virtual machines. Upgrades wait for a registry mirror that nobody owns, brokers and Kafka run on the clusters through operators with their own version matrices, and the on-call rota depends on two people. Backups exist, but a full restore has never been timed.
Environment
On-premises OpenShift and an air-gapped kubeadm cluster, a private image registry, CSI-backed storage, the RabbitMQ Cluster Operator and Strimzi for Kafka, Velero to on-premises object storage, and the customer's change management process.
Approach
AceMQ agrees access through the customer's own controls, assesses the estate and fixes the supply chain first: the registry mirror, image promotion and the order in which platform, operators and workloads can be updated. Upgrades are rehearsed on a non-production cluster and run in change windows the customer approves. Cluster platform work is delivered by AceMQ with partner Kubernetes specialists; broker and Kafka operations are AceMQ's own.
Solution
- Mapped every component's version, support window and upgrade prerequisites, including operators, CSI drivers and the registry mirror
- Documented the image mirroring and promotion path so a patch release reaches the cluster without outbound access
- Wrote upgrade plans per cluster that respect version skew and put stateful workloads last, rehearsed on non-production
- Set PodDisruptionBudgets and readiness checks that protect broker quorum and rehearsed drains with the customer's operations team
- Configured Velero against on-premises object storage and timed a full restore into a scratch environment
- Took over monitoring and 24/7 on-call inside the customer's access and change controls
Outcome
The customer receives an upgrade calendar and runbooks that work without internet access, a documented image supply chain, a measured restore time, and PodDisruptionBudgets and drain procedures for the brokers. AceMQ holds the pager inside the customer's boundary, and nothing leaves the data centre.