Back to all use cases

AssessmentCross-Industry

CloudStack ships a separate upgrade guide for every release, so the plan starts with which guides apply

Private Cloud Operations Team

A typical engagement to plan an Apache CloudStack upgrade onto the current LTS line: read every upgrade guide between the running version and the target, check Java 17 on the management servers and KVM agents, and write the order of management servers, hosts and system VMs.

CloudStack consulting

Overview

This is a typical engagement for an operations team that runs a CloudStack cloud for internal tenants and has been told to move to the current long-term support line. The estate has two management servers, a MySQL database, KVM hosts in several clusters and a few system VMs per zone. AceMQ and a partner CloudStack specialist read the upgrade guides, inspect the estate and write the plan before anything is changed.

Challenge

CloudStack documents each upgrade in its own guide, from one release to the next, so an estate several releases behind has to read every guide on the path and note the steps that differ. The 4.22 guide asks for Java 17 as the default Java on the management server and on KVM hosts, for the new system VM template to be registered and READY before the upgrade, and for the database to be dumped first. From 4.20.3 and 4.22.1 a management server that finds a database upgrade is needed aborts if other management servers are still running. Any of these, missed on the night, turns a planned window into an outage.

Environment

Two CloudStack management servers behind a load balancer, a MySQL database for the cloud and cloud_usage schemas, KVM hosts on a mix of Ubuntu and RHEL-family releases, per-zone secondary storage, and a set of virtual routers and system VMs.

Approach

The engagement starts read-only. AceMQ lists the running CloudStack version, the Java version on every management server and KVM host, the operating system of each host, the system VM template in each zone and the state of the mshost table. It then reads the upgrade guide for each hop to the target and records what applies to this estate. Nothing in production is changed during the assessment.

Solution

  1. Records the running version, the Java version and the operating system of every management server and KVM host
  2. Reads the upgrade guide for every release between the running version and the LTS target and lists the steps that apply
  3. Checks that Java 17 is installable and can be the default on each host before the window
  4. Plans the system VM template registration and the wait for its READY state in each zone
  5. Writes the dump of the cloud and cloud_usage databases into the runbook, with a restore rehearsal on a spare server
  6. Orders the work: first management server, hosts and agents, remaining management servers, then system VMs and virtual routers, with a decision point and a rollback boundary after each

Outcome

The engagement delivers an inventory of versions and Java levels, the list of upgrade steps that apply to the estate, a window plan in the order above, a rehearsed database restore and a runbook the operations team can run. The team learns what must be true on every host before the first command is typed.

Technologies

Apache CloudStackKVMMySQLJava

Ready for an Apache CloudStack Health Check?

AceMQ's senior Apache CloudStack engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.