Cyber security
Air-Gapped Delivery for a Leading Cyber Security Company
A cyber security vendor with a global customer base needed faster, traceable releases—from internal lab testing through to remote customer deployments.
Client context
- Profile
- Cyber security vendor, ~180 engineers, customers across multiple regions. The product is validated in internal labs before it ships. Customer deployments happen over controlled remote sessions from the company network. The estate mixes Kubernetes for newer services with VMware VMs for legacy workloads and Jenkins agents.
- Engagement
- Scoped project with milestone gates, then paired handoff
- Duration
- Multi-month
- Team
- 2 FOO engineers + client release engineering squad (5 people)
Stack
- Jenkins
- Ansible
- Kubernetes
- VMware vSphere
- Nexus
- HashiCorp Vault
- Trivy
- SonarQube
- Prometheus
Challenge
Getting a release out took several days of manual steps. Production clusters were already hardened, but dev had not kept up—so builds often passed locally and failed later in the pipeline. Internal labs were supposed to catch that, but promotion between environments was still hand-driven. When something did reach a customer site, there was no clean trail from the lab build to the remote deployment. Security reviewers spent days assembling evidence for each release.
Milestones
1.We mapped how releases actually move
A workshop with the security team traced the path a build takes: dev, internal labs, production, then out to a customer over a remote session. We agreed what has to match at each hop and what evidence security needs before a customer deployment goes ahead.
Next step: Shared security baseline across environments
2.Dev caught up to production
Production was strict; dev was not. We aligned CIS baselines, network policies, and scan gates so engineers build in an environment that looks like what they are promoting into—not a relaxed sandbox that hides problems until the lab.
Next step: Offline artifact pipeline
3.Artifacts flow offline, zone by zone
Nexus mirrors and approved base images let each environment pull dependencies without reaching outside its boundary. We tested promotion with deliberate failures to make sure nothing drifted between zones quietly.
Next step: Automated pipelines in the labs
4.Labs run the same rules as production
Jenkins builds and scans on agents inside each zone. Ansible provisions VM and K8s workers from playbooks. A build only moves forward if it cleared the zone above it—lab clusters exercise the same promotion logic production will use.
Next step: Production promotion with evidence
5.Production and customer delivery on one track
Helm deploys to production with a full evidence bundle attached. That same validated artifact is what goes out on a remote customer deployment from the company network. Prometheus watches for sync drift and failed hooks across the board.
Next step: Client team ownership
6.Client team took it from here
Release engineering extended the pattern to two more services on their own. Runbooks covered the full loop—lab, production, customer deploy. A few pairing sessions and we stepped off the critical path.
Next step: Project closed
Solution summary
We traced the real release path—from dev through internal labs and production, out to customer sites—and automated it zone by zone. Dev and production clusters were brought to the same security baseline. Nexus mirrors and CIS-hardened images fed Jenkins pipelines in each environment; Ansible handled VM and K8s provisioning from versioned playbooks. Every promoted build carried scan results, an SBOM, and approval metadata, so what ran in the labs was exactly what security could sign off on before a remote customer deployment.
Results
| Metric | Before | After |
|---|---|---|
| Release lead time | Several days (manual) | Same day (automated) |
| Security evidence prep | Days of manual work | Attached to each build |
| Scan gate coverage | Partial | 100% of promoted artifacts |
Outcomes
- Same-day releases through labs and production, down from several days of manual promotion
- Dev and production now share the same hardening bar—fewer surprises when a build hits the lab
- Remote customer deployments trace back to a lab-validated build with evidence already attached
- Security reviewers query evidence bundles instead of assembling spreadsheets per release