litellm-fips
Echo's FIPS-validated LiteLLM image — the LLM gateway that unifies access to model providers, built on FIPS 140-3 validated cryptography for FedRAMP and US government workloads.
What is LiteLLM FIPS?
LiteLLM is an open-source gateway and Python SDK that exposes 100+ LLM providers through a single OpenAI-compatible API. Teams run its proxy server to centralize routing, virtual keys, budgets, and logging.
The LiteLLM FIPS image is Echo's FIPS variant of LiteLLM: the same LLM gateway, built to run on FIPS 140-3 validated cryptography. It is designed for organizations that sell to or operate for the US government — FedRAMP-authorized services, federal contractors, and regulated industries — where an LLM gateway that carries sensitive prompts and provider credentials over TLS must use cryptographic modules with an active NIST CMVP certificate, not merely FIPS-enabled libraries.
What is Echo's LiteLLM FIPS image?
Echo's LiteLLM FIPS image is a hardened, FIPS-validated build of LiteLLM on Echo's hardened base. Cryptography on this image is routed through the Echo FIPS Provider for OpenSSL, a software module validated under FIPS 140-3 (NIST CMVP certificate #5560, Level 1, active). Echo images are designed to be a drop-in replacement: swap the image reference in your Dockerfile or deployment manifest for the litellm-fips image and CVEs go to zero without disrupting your configuration, data volumes, and the clients that connect to it. Every image is tested across clouds, image use cases, and deployment targets. Echo ships every image in two variants:
- Distroless variant — optimized for runtime use, with the smallest possible attack surface
- Default variant — includes essential build tools, package managers, and shells for teams that need operational access
For production LiteLLM deployments, the distroless variant keeps all runtime operations — model routing, key management, and request logging — fully intact while minimizing exposure; the default variant suits operators and platform teams that need shell access for inspecting configuration, diagnosing issues, or running administrative commands.
What is the difference between Echo's LiteLLM FIPS image and the public LiteLLM image?
Public LiteLLM images ship on general-purpose bases whose cryptographic libraries have not been validated under FIPS 140-3, and FIPS-enabled is not the same as FIPS-validated — auditors for FedRAMP and other federal frameworks flag the gap. Those bases also include OS-level tooling convenient for development but which contribute CVEs that accumulate on an LLM gateway used in production. LiteLLM images run continuously with access to provider API keys, virtual keys, and prompts and completions. OS-level CVEs in the base image — particularly in libraries such as openssl, zlib, and libc — create exploitable paths that bypass application-layer controls entirely, allowing attackers to move from a network-reachable service to host-level access. For teams running LiteLLM across many environments, the image is also a steady source of scanner noise in the CI pipeline — every package in the base becomes a finding your security team must triage, even when none of it is reachable from the service. Echo's FIPS build retains everything LiteLLM needs for model routing, key management, and request logging while routing cryptography through a validated module and removing the packages that don't belong in a production LiteLLM LLM gateway. As we covered in our post on how to protect your company from software supply chain attacks, long-running services that hold sensitive data and listen on the network are among the widest attack surfaces in the container stack. Echo commits to a 7-day SLA for critical and high severity vulnerabilities, and 10 days for medium, low, and unknown — with vulnerabilities triaged within 24 hours. Echo FIPS images are STIG-hardened, ship with SBOMs in SPDX and CycloneDX formats and signed provenance, and are recognized by all major scanners and mirrored to all major registries, so they fit into existing pipelines without changing your registry, scanner, or runtime tooling.
FAQ
Can I replace my LiteLLM image with Echo's LiteLLM FIPS image?
Yes. Echo's LiteLLM FIPS image is a drop-in replacement. Update the FROM line in your Dockerfile to the litellm-fips image and your workloads keep running. Configuration, data volumes, mounted secrets, and existing client connections all continue to work without changes. Because FIPS mode disallows non-approved algorithms, anything that depends on legacy ciphers or hashes such as MD5-based signatures needs to move to approved algorithms.
Is Echo's LiteLLM FIPS image FIPS-validated?
Yes. Echo's FIPS images use CMVP-validated cryptographic modules, operating in a configuration consistent with the module's FIPS 140-3 security policy. Cryptography on this image is routed through the Echo FIPS Provider for OpenSSL, a software module validated under FIPS 140-3 (NIST CMVP certificate #5560, Level 1, active). That makes the image fit for federal use — unlike FIPS-enabled or FIPS-compliant images whose cryptographic libraries have never been validated. This matters for teams running LiteLLM inside FedRAMP boundaries, where an LLM gateway that carries sensitive prompts and provider credentials over TLS must meet cryptographic requirements across the full stack. Echo also provides a FIPS runtime tester that executes approved and unapproved algorithms to show how the image behaves at runtime.
What is Echo's vulnerability management SLA on the LiteLLM FIPS image?
Echo commits to a 7-day SLA for critical and high severity vulnerabilities, and 10 days for medium, low, and unknown — with vulnerabilities triaged within 24 hours. Patches are mirrored automatically into your private registry so you're always running a clean version — critical for a long-running service that holds provider API keys, virtual keys, and prompts and completions and is reachable over the network.
Is Echo's LiteLLM FIPS image distroless?
Echo ships every image in two variants: a distroless variant optimized for runtime use, and a default variant that includes essential build tools, package managers, and shells. For production LiteLLM deployments, the distroless variant is the leaner, more secure choice; for operators and platform teams that need shell access for inspecting configuration, diagnosing issues, or running administrative commands, the default variant is the right fit.
How does Echo achieve such a drastic CVE reduction in LiteLLM?
Echo's LiteLLM FIPS image is built from source with only the absolute essentials needed to run the LiteLLM LLM gateway, which significantly shrinks the attack surface. Echo also patches aggressively over time, with backports available so you can stay on the LiteLLM version your applications and their dependencies are certified against — avoiding a disruptive upgrade forced purely by OS-level vulnerabilities in the base image rather than by your own requirements.
Will Echo's LiteLLM FIPS image help us achieve FedRAMP?
Yes. The hard parts of FedRAMP — managing vulnerabilities, applying fixes, and using FIPS-validated cryptography — are baked into Echo images, including STIG-hardened configuration and ConMon/POA&M-ready reporting. For federal agencies, contractors, and FedRAMP-authorized providers running LiteLLM under an ATO — whether that's a FedRAMP-authorized SaaS boundary, a DoD enclave, or an agency-operated platform — Echo's FIPS-validated LiteLLM image keeps the LLM gateway layer in-boundary and compliant without requiring custom hardening scripts or manual base image patching between compliance cycles.
.avif)