Continuous Integration and Delivery¶
Continuous integration (CI) gives every change fast, repeatable evidence that it integrates. Continuous delivery keeps the main branch releasable through an automated path. Continuous deployment additionally releases every qualifying change without a manual business decision.
The goal is small, safe changes and short feedback—not a large YAML program.
The Delivery Path¶
change → review → fast checks → build once → deeper checks → deploy progressively → verify
One immutable artifact should move through environments. Rebuilding per environment makes the tested artifact different from the released artifact.
Continuous Integration¶
A healthy CI workflow has:
- short-lived branches or direct integration to a protected trunk;
- small changes merged at least daily;
- automated builds and deterministic dependency resolution;
- fast tests and static checks on every change;
- an immediately visible broken-main state with clear ownership;
- reproducible local commands for pipeline failures.
Order checks by expected information value and cost: formatting and focused tests first, then build, integration, security, and broad system checks. Run independent jobs in parallel. Cache dependencies only when keys include all inputs that affect them.
Avoid long-lived branches, environment-specific code paths, ignored flaky tests, pipelines that can pass after skipping work, and queues longer than the checks themselves.
Pipeline Shape¶
A minimal pipeline usually needs:
- validate source and configuration;
- run focused tests and static analysis;
- build a versioned artifact;
- test the artifact at real boundaries;
- produce provenance and dependency metadata when required;
- publish to an immutable registry;
- deploy the same artifact;
- verify technical and user outcomes.
Do not duplicate orchestration across repositories. Reuse a small versioned workflow only after several pipelines have the same proven need; keep application-specific commands in the repository.
Artifacts and Versioning¶
Tie every artifact to its source revision, build definition, dependency inputs, and builder identity. Human-friendly versions and immutable digests serve different purposes: a release may be called 2.4.1, while deployment pins its content digest.
Store test results, checksums, software bills of materials, and provenance beside the artifact according to risk and policy. The living SLSA specification describes supply-chain provenance and build integrity levels.
Never promote an artifact whose evidence cannot be traced back to the exact bytes being deployed.
Delivery and Deployment¶
- rolling replacement gradually swaps instances;
- canary delivery exposes a small share of traffic or users first;
- blue–green maintains old and new environments for a traffic switch;
- feature flags separate code deployment from feature exposure;
- shadow traffic duplicates requests without using the candidate response.
Choose the simplest method matching the blast radius. Progressive delivery needs representative traffic, reliable comparison signals, explicit promotion criteria, and an abort path.
Feature flags require owners, safe defaults, telemetry, expiry dates, and deletion. They are temporary operational controls, not a permanent configuration system.
Database and Contract Changes¶
Deployments commonly run mixed versions. Preserve compatibility with expand–migrate–contract:
- add backward-compatible schema or contract support;
- deploy code that handles old and new forms;
- migrate or backfill gradually;
- verify use and correctness;
- remove the old form later.
A code rollback cannot undo a destructive migration or external side effect. Prefer roll-forward when state has advanced, and rehearse recovery for critical changes.
Environments and Configuration¶
Minimize material differences between test and production while keeping credentials, capacity, and external endpoints separate. Configuration must be explicit, validated, versioned where safe, and injected without rebuilding.
Ephemeral environments help test isolation but cost time and money and rarely reproduce all production behavior. Create them only for tests that benefit from the fidelity, and delete them reliably.
Pipeline Security¶
CI/CD runners execute repository-controlled code and often hold valuable authority. Treat pipeline changes as privileged code.
- use short-lived workload identity instead of static cloud keys;
- grant each job only the permissions and environment it needs;
- isolate untrusted pull-request code from secrets and release runners;
- pin third-party actions, images, and tools to reviewed immutable versions;
- protect release branches, tags, approvals, and artifact registries;
- prevent secrets from reaching arguments, logs, caches, or artifacts;
- sign or attest releases in a controlled builder;
- audit deployments and emergency bypasses.
Security scans provide evidence, not immunity. Triage findings by exploitability and maintain an owned exception with expiry when risk is accepted.
Approval and Separation of Duties¶
Automate evidence collection. Place human approval only where a legal, safety, financial, or business decision requires it. An approval screen without useful evidence is ceremony.
High-risk releases may require independent review, protected environments, and two-person control. Emergency paths should be faster but still authenticated, logged, reversible, and reviewed afterward.
Verification and Rollback¶
Before release, define:
- expected service and business signals;
- acceptable error, latency, and saturation changes;
- observation window;
- automatic and manual stop conditions;
- rollback or roll-forward owner.
Verify from outside the deployed process where possible. A successful deployment command only proves the command completed.
Pipeline Reliability¶
Pipelines are production systems. Monitor job queue time, execution time, failure causes, flaky checks, runner capacity, artifact publication, deployment status, and credential expiry.
Make jobs idempotent where retries are allowed. Set timeouts, cancel superseded runs, retain enough evidence for diagnosis, and test the release path regularly rather than only during emergencies.
Use DORA's current delivery metrics to evaluate the delivery system over time, not to rank people or compare unrelated services.
Checklist¶
- Does every change get fast, trustworthy feedback?
- Is one immutable artifact built and promoted?
- Can old and new versions coexist during deployment?
- Are credentials short-lived and permissions minimal?
- Is untrusted code isolated from release authority?
- Are promotion and abort criteria based on user-visible signals?
- Can stateful changes recover safely?
- Are flaky tests and slow queues owned?
- Can every release be traced to source and build evidence?
The smallest useful pipeline is the one that repeatedly turns a reviewed change into a verifiable, recoverable release.