Overview
This is a typical engagement for a DevOps platform team that runs one self-managed GitLab instance for several hundred developers and has fallen several minor releases behind. GitLab ships monthly minor releases, backports security fixes to the current release and the two before it, and defines required upgrade stops. AceMQ and a partner Git specialist plan the route to a maintained release.
Challenge
GitLab's upgrade guide says each required stop must be installed and its background migrations finished before moving to the next, and that the latest patch of a release should be used, not its first. An instance that skipped releases may need several stops, each with its own maintenance window. Runners, integrations and the Git storage layer have to keep working at every stop.
Environment
A single self-managed GitLab instance installed from the Linux package, an external PostgreSQL database, Git repositories on local or networked storage, a pool of CI runners at mixed versions, and integrations with a ticket tracker, a chat tool and a registry.
Approach
AceMQ records the running version, the install method and the database version, then calculates the required stops to the target with GitLab's upgrade path tool and reads the release notes for each. The route is rehearsed on a restored copy of the instance in an isolated network, with background migrations timed.
Solution
- Records the version, install method, database version and runner versions
- Calculates the required upgrade stops to the target with GitLab's upgrade path tool and adds the environment-specific stops
- Restores a backup onto a spare server and rehearses each stop, timing the background migrations
- Checks runner versions, integrations and the registry against each stop
- Plans the maintenance windows, the backup before each stop and the point after which a rollback means restoring from backup
- Sets a routine for the monthly minor and twice-monthly patch releases so the instance stays inside the maintained releases
Outcome
The engagement delivers the stop-by-stop route with rehearsal timings, a runner and integration compatibility list, a window plan with a backup and a rollback decision at each stop, and an update routine for the maintained releases. The platform team gets a calendar instead of one risky weekend.