kserve

The official KServe image — a Kubernetes-native model serving platform for deploying machine learning and generative AI models with autoscaling, canary rollouts, and standard inference protocols.

kserve, model serving, inference, kubernetes, mlops, cncf

What is KServe?

The KServe images package KServe, a Kubernetes-native platform for serving machine learning and generative AI models behind standard inference APIs. KServe defines an InferenceService custom resource and provides a controller manager, the storage initializer that fetches models from object storage, a model server SDK, and the agent and router components. It supports autoscaling including scale-to-zero, canary rollouts, request batching, and the open inference protocol, and it works with serving runtimes for frameworks such as scikit-learn, XGBoost, PyTorch, TensorFlow, and large language model runtimes. It is used by ML platform teams that run production inference on Kubernetes with reproducible, version-pinned images.

What is Echo's KServe image?

Echo's KServe image is a hardened build of KServe on Echo's hardened base. Echo images are designed to be a drop-in replacement: swap the image reference in your Dockerfile or deployment manifest and CVEs go to zero without disrupting your InferenceService definitions, serving runtimes, storage URIs, and autoscaling settings. Every image is tested across clouds, image use cases, and deployment targets. Echo ships every image in two variants:

  • Distroless variant — optimized for runtime use, with the smallest possible attack surface
  • Default variant — includes essential build tools, package managers, and shells for teams that need operational access

For production KServe deployments, the distroless variant keeps all runtime operations — model loading, request routing, autoscaling, canary rollouts, and inference request handling — fully intact while minimizing exposure; the default variant suits ML platform and MLOps teams that need shell access for debugging predictor pods, testing model endpoints, or inspecting runtime logs.

What is the difference between Echo's KServe image and the public KServe image?

Public KServe images ship on bases that include OS-level tooling convenient for development but which contribute CVEs that accumulate on a model serving platform used in production. KServe images run continuously with access to trained models, inference traffic, and object storage credentials. OS-level CVEs in the base image — particularly in libraries such as openssl, zlib, and libc, along with the Go and Python runtime dependencies — create exploitable paths that bypass application-layer controls entirely, allowing attackers to move from a network-reachable service to host-level access. For teams running KServe across many environments, the image is also a steady source of scanner noise in the CI pipeline — every package in the base becomes a finding your security team must triage, even when none of it is reachable from the service. Echo's build retains everything KServe needs for model loading, request routing, autoscaling, canary rollouts, and inference request handling while removing the packages that don't belong in a production KServe model serving platform. As we covered in our post on how to protect your company from software supply chain attacks, long-running services that hold sensitive data and listen on the network are among the widest attack surfaces in the container stack. Echo commits to a 7-day SLA for critical and high severity vulnerabilities, and 10 days for medium, low, and unknown — with vulnerabilities triaged within 24 hours. Echo images are recognized by all major scanners and mirrored to all major registries, so they fit into existing pipelines without changing your registry, scanner, or runtime tooling.

FAQ

Can I replace my KServe image with Echo's KServe image?

Yes. Echo's KServe image is a drop-in replacement. Update the FROM line in your Dockerfile and your workloads keep running. InferenceService resources, serving runtime references, storage initializer credentials, and autoscaling and canary settings all continue to work without changes to your models or manifests.

Is Echo's KServe image FIPS-validated?

Yes. Echo's FIPS-validated images use cryptographic modules with an active FIPS 140-3 CMVP certificate, making them fit for federal use — unlike FIPS-compliant images that haven't been validated. This matters for teams running KServe workloads inside FedRAMP boundaries, where a model serving platform that loads trained models and serves inference requests from production applications must meet cryptographic requirements across the full stack — including the OpenSSL and TLS libraries bundled with the KServe image.

What is Echo's vulnerability management SLA on the KServe image?

Echo commits to a 7-day SLA for critical and high severity vulnerabilities, and 10 days for medium, low, and unknown — with vulnerabilities triaged within 24 hours. Patches are mirrored automatically into your private registry so you're always running a clean version — critical for a long-running service that holds trained models, inference traffic, and object storage credentials and is reachable over the network.

Is Echo's KServe image distroless?

Echo ships every image in two variants: a distroless variant optimized for runtime use, and a default variant that includes essential build tools, package managers, and shells. For production KServe deployments, the distroless variant is the leaner, more secure choice; for ML platform and MLOps teams that need shell access for debugging predictor pods, testing model endpoints, or inspecting runtime logs, the default variant is the right fit.

How does Echo achieve such a drastic CVE reduction in KServe?

Echo's KServe image is built from source with only the absolute essentials needed to run the KServe model serving platform, which significantly shrinks the attack surface. Echo also patches aggressively over time, with backports available so you can stay on the KServe version your applications and their dependencies are certified against — avoiding a disruptive upgrade forced purely by OS-level vulnerabilities in the base image rather than by your own requirements.

Will Echo's KServe image help us achieve FedRAMP?

Yes. The hard parts of FedRAMP — managing vulnerabilities, applying fixes, and using FIPS-validated cryptography — are baked into Echo images, including STIG-hardened configuration and ConMon/POA&M-ready reporting. For ML platform and MLOps teams running KServe under an ATO — whether that's a production model serving platform, a generative AI inference service, or a regulated analytics workload — Echo's hardened KServe image keeps the model serving platform layer in-boundary and compliant without requiring custom hardening scripts or manual base image patching between compliance cycles.

‍

Interested in base images that start and stay clean?