Upstream Drift

What Is Upstream Drift?

Upstream drift is the growing distance between the version of a component a maintained image ships and the version its upstream project has since released. It is a property of the artifact as built, not of anything running. Every image carries some of it, because no rebuild happens the moment an upstream release lands.

The gap compounds quietly. Nothing in a tag states how old the contents are, so an image that looks current can carry a dependency untouched for a year. A tag is a name, not a statement about age.

Distance alone is not a verdict. Some of it is deliberate, held in place so a stable interface does not change under consumers. The question is whether a given gap is a maintained decision or an abandoned one, and a version number alone cannot answer it.

How Upstream Drift Differs From Container Drift

The two names sound alike and describe opposite sides of the same artifact.

• Container drift is a runtime condition. A running container diverges from the image it started from, through packages installed or files edited in place.
• Drift from upstream is supply-side. The image lags its own sources, and the divergence exists before anything is scheduled.
• They are detected differently. One compares a live filesystem against its image; the other compares shipped versions against release feeds.
• They are fixed differently. One is fixed by redeploying immutably; the other by rebuilding against newer sources.

Both are measured from a reference; what differs is the reference. A container is compared against its own image, a maintained image against the world outside it.

That also decides who can act: runtime divergence is yours to remove, while the supply-side gap belongs to whoever builds the image.

Why the Gap Accumulates, and What Version Numbers Hide

Rebuild cadence sets the floor: an image rebuilt weekly is at most a week behind on what it pulls fresh. Pinned dependencies set the ceiling, because a pin is excluded from the rebuild by design.

Several forces push the distance up:

• Patch burden. A maintainer carrying local patches must rebase them on every upstream release, so a large patch set slows adoption.
• Interface stability. Holding a major version avoids breaking consumers, and that choice is often correct for years.
• Transitive pinning. Lockfiles and vendored directories freeze versions the top-level manifest never names.
• Abandonment. Nothing in a version string separates a deliberately held release from an unmaintained one.

Backporting makes the last point hard. A fix backported into an older release changes its content without changing its version, so a scanner matching version numbers against advisories reports the old number rather than the applied fix. The opacity runs both ways.

How to Measure Drift in Production Environments

Measurement is per component rather than per image, and it needs an age as well as a difference. The component list comes from the image SBOM, in a standard format such as CycloneDX.

• Enumerate shipped components from the SBOM rather than inferring them from the tag.
• Compare each one against its upstream release feed, recording the shipped version, the current version and its publication date.
• Express the result as an age in days behind, because a count of versions says nothing about elapsed exposure.
• Classify every gap as deliberate, pending or unexplained, and require a written reason for anything deliberate.
• Set a budget per class: tight for runtimes and TLS libraries, looser for stable system utilities, none past end of life.

A budget is a threshold with an owner: ninety days behind on a logging utility is not ninety days behind on a TLS library.

How Hardened Container Images Change the Picture

A hardened image provider takes on the rebuild cadence and patch burden, where most of the gap originates. Cadence becomes a contract, not an internal habit.

Echo's images are continuously rebuilt and updated, and workloads can stay on the versions that fit their needs without forced migrations (echo.ai/product). That separates staying current from being moved.

It does not remove the measurement. Buyers should still ask which components are held deliberately, and how quickly a held version receives a fix.

FAQs

Is drift from upstream the same as container drift?

No. Container drift is a runtime condition: a live container diverges from the image it was started from. The supply-side gap describes an image that lags the sources it was built from, and exists before deployment. One is fixed by redeploying immutably, the other by rebuilding against newer sources or recording why not.

Can a scanner measure how far behind an image is?

Not reliably. Scanners match package versions against advisory databases, so they report known flaws rather than distance from upstream. A backported fix leaves the version string unchanged, which produces a finding where no risk remains. Measuring the gap needs the shipped version compared against the upstream release feed, with publication dates.

Is every gap a security problem?

No. A version held deliberately, with security fixes backported into it, is a maintained decision and often the right one for a stable interface. The failure case is a gap nobody owns: no rebuild, no backports, no stated reason. The version number looks identical in both cases, which is why reasons must be recorded.

What does a drift budget look like?

It is a per-component threshold with an owner. Runtimes, TLS libraries and anything parsing untrusted input get the tightest numbers; stable system utilities get looser ones. Anything past end of life gets none at all. The budget is useful only if exceeding it triggers a decision rather than a dashboard colour.

How often should a base image be rebuilt?

Often enough that cadence is not the dominant source of the gap. A weekly rebuild caps the distance on freshly pulled packages at a week; a rebuild only on release leaves the image as old as that release. Cadence does nothing for pinned dependencies, which need their own review schedule.