Jenkins & CI/CD
Delivery end to end — declarative Jenkins pipelines, shared libraries and ephemeral agents, then Google Cloud Build and Cloud Deploy, deployment strategies, supply-chain signing, and a pipeline you can defend commit by commit.
Stage 1 — Foundations
3 lessonsThe three terms people use interchangeably, why the pipeline is a contract rather than a script, and the four numbers that tell you whether yours is working.
Controller and agents, where all the state actually lives, and why the first thing to configure on a new Jenkins is zero executors on the controller.
Why a job configured in a web form cannot be reviewed, what a Jenkinsfile changes, and how multibranch turns branches and pull requests into builds automatically.
Stage 2 — Pipelines
4 lessonsThe seven blocks every pipeline is made of, what agent placement costs you, and the post conditions that make a pipeline report the truth.
Running a stage only when it should run, shortening the critical path with parallel branches, and generating a build matrix instead of copying stages.
How a credential reaches a build step, the five ways log masking is defeated, and why OIDC federation removes the problem instead of managing it.
Static agents versus ephemeral pods, why a pinned container per stage beats installing tools on a VM, and building images without handing out the Docker socket.
Stage 3 — Scaling Jenkins
3 lessonsTurning forty lines of copied Jenkinsfile into four, the three directories a library needs, and why pinning it to main is how you break fifty repositories at once.
Rebuilding a controller from a file in Git instead of a backup, keeping the plugin surface patched, and proving the restore works before you need it.
A triage order for a failing build, why replay is the fastest loop you have, and what to do about a test suite nobody believes any more.
Stage 4 — Cloud-Native Delivery
7 lessonsEvery field of a build step, substitutions and secrets done safely, the artifacts block, and the four caching strategies — with the measurement that tells you which one helped.
What starts a build and what it is allowed to do, reaching private networks from a managed builder, and the identity, notification and cost model you inherit.
The four objects, why rendering once at release time is the whole point, deploy parameters instead of a values file per environment, and the execution identity behind each target.
Rollout job phases and where they fail, verify jobs that actually gate, canary phases without writing orchestration, automation rules, and a timed rollback.
Recreate, rolling, blue-green and canary compared on blast radius and cost — plus the database constraint that caps how progressive your delivery can be.
SBOMs, signing and provenance in the stages that produce them — and the admission control that makes any of it enforcement rather than a report.
Jenkins, Cloud Build, GitHub Actions and GitLab CI on what each actually costs you — plus push versus pull delivery, and how to migrate without a rewrite.
Stage 5 — Capstone
1 lessonOne commit to production through build, scan, sign, staged deploy and a gated promotion — then nine drills that prove every gate actually closes.
🗺️ Beginner → Expert Roadmap
5 stages with prerequisites and a concrete mastery check at each.
🎯 What You'll Learn
- • Separate continuous integration, delivery and deployment by where the human gate sits — and say which one you actually have.
- • Measure a delivery system with four DORA numbers instead of describing it.
- • Build an artifact once and promote the same digest, so what ships is what was tested.
- • Explain why a Jenkins controller runs zero executors, and rebuild one from Git rather than a backup.
- • Convert a freestyle job to a multibranch pipeline and get PR builds for free.
- • Write declarative pipelines whose agent placement does not waste executors on approvals.
- • Use when, parallel and matrix deliberately — including beforeAgent, which saves real money.
- • Bind credentials narrowly, name the five ways log masking is defeated, and replace stored keys with OIDC.
- • Run builds on ephemeral pod agents and build images without ever mounting the Docker socket.
- • Ship a shared library that is versioned, tested and pinned to a tag rather than to main.
- • Operate Jenkins as code: JCasC, pinned plugins, and a restore drill with a measured time.
- • Triage a failing build in a fixed order, and quarantine a flaky test instead of retrying it.
- • Write a Cloud Build config using the full step schema, with substitutions and secretEnv done safely.
- • Choose between four caching strategies and prove which one helped with measured build durations.
- • Give every trigger its own service account, so a fork pull request cannot reach a deploy credential.
- • Reach private networks from a managed builder with a private worker pool and a fixed egress IP.
- • Model delivery in Cloud Deploy: pipelines, targets, releases and rollouts, with manifests rendered once.
- • Use deploy parameters instead of a values file per environment, and split render from deploy identity.
- • Gate promotion on a verify job that compares the canary against stable, and automate what needs no human.
- • Roll back to any prior release without a rebuild, and state what the rollback did not undo.
- • Choose between rolling, blue-green and canary on blast radius, and know what the database caps.
- • Make supply-chain claims enforceable: SBOM, signing, deploy by digest, and admission control.
- • Choose a CI/CD platform from a constraint rather than a feature list, and migrate without a rewrite.
- • Ship a pipeline proven by nine drills, including the four that fail on pipelines that deploy perfectly.
🛡️ Best Practices in Production
The short version of this path. Every lesson also ends with the specific mistake it exists to prevent.
- ✓ Build once and promote the same digest; never rebuild per environment.
- ✓ Keep the pipeline definition in the repository it builds, so build changes are reviewed with code changes.
- ✓ Run zero executors on the Jenkins controller, and use ephemeral agents.
- ✓ Set
timeouton every pipeline andbuildDiscarderon every job. - ✓ Bind credentials narrowly with
withCredentials, use single quotes insh, and prefer OIDC over stored keys. - ✓ Every gate blocks or is deleted — a scan that warns and proceeds is a dashboard.
- ✓ Rebuild the controller from JCasC plus
plugins.txt, and run a restore drill with a measured time. - ✓ Measure the four DORA numbers, and measure rollback from the decision rather than the command.
- ✗ A fork pull request that can reach a deploy credential. That is a shell on your infrastructure.
- ✗
retry(3)around a flaky test — it hides the defect and triples the bad-day duration. - ✗ Mounting the Docker socket in a build agent to build images. Use kaniko or buildkit.
- ✗ Plain
--forcepush, or a pipelineinputstep holding an executor for hours. - ✗ Deploying a mutable tag instead of the digest you scanned and signed.
- ✗ Configuring anything important in the Jenkins UI, where it has no diff and no history.
💼 Interview Readiness
Once you reach the end of this path, test your engineering knowledge against real questions asked by top technical teams:
A junior engineer on your team says a system issue "just fixed itself" during a live troubleshooting session, and they can no longer reproduce a bug you were actively diagnosing together. What questions would you ask before concluding the issue is actually resolved?
Structured DebuggingA production incident shows a mix of failing and succeeding network-layer tests (e.g., ARP fails, but a TCP connection to a specific port succeeds). How would you characterize this kind of failure mode, and what should your next diagnostic steps be?
Linux & NetworkingA client insists on continuing to run both a self-hosted Prometheus/Grafana stack and a cloud-native monitoring tool in parallel, citing team preference. How would you approach cost optimization given this constraint, rather than pushing for consolidation?
Cloud Cost Optimization