Root containers, writable filesystems, and credentials in layer history
Images run as non-root with read-only filesystems across the estate, exposed credentials were rotated, and the accreditation findings tied to container configuration were closed.
Overview
Container hardening has a short list of high-value items: do not run as root, do not allow privilege escalation, make the root filesystem read-only, drop capabilities, and keep secrets out of image layers. The last one matters most because a deleted file in a later layer is still recoverable from the earlier one.
Challenge
Nearly every image ran as root by default, none set a read-only root filesystem, and several ran with added capabilities that were copied from an old troubleshooting session and never removed. Layer history analysis found credentials in build-time ARG values and a config file that had been added and then deleted in a later layer — still fully recoverable from the image.
Environment
Containerized applications on Kubernetes under security accreditation requirements, built from internally maintained base images.
Approach
AceMQ analyzed the image layer history for every service alongside its runtime security context, prioritizing findings by exploitability rather than by count. Secrets found in layers were treated as compromised and rotated, not merely removed from future builds.
Solution
- 1Scanned image layer history for credentials, keys, and config files removed in later layers
- 2Rotated every credential found in image history rather than only stripping it from future builds
- 3Converted images to run as a non-root user with an explicit UID and matching filesystem ownership
- 4Set read-only root filesystems with explicit writable mounts only where genuinely required
- 5Dropped all capabilities by default and re-added only those a service demonstrably needs
- 6Moved build-time secrets to BuildKit secret mounts so they never enter a layer
Outcome
Images run as non-root with read-only filesystems across the estate, exposed credentials were rotated, and the accreditation findings tied to container configuration were closed.
Technologies
Related Use Cases
Docker Image Build and Layer Caching Optimization
Restructuring Dockerfiles and CI caching so builds reuse layers properly, cutting pipeline time and image size across a large service estate.
Docker OOMKilled Container Remediation
Resolving containers repeatedly OOMKilled because the JVM and Node runtimes inside them were sizing heap against host memory rather than the cgroup limit.
Ready for a Docker Health Check?
AceMQ's senior Docker engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.