filebeat

What is filebeat?

The filebeat image provides Filebeat, Elastic's lightweight log shipping agent and part of the Elastic (ELK) Stack. It monitors log files and container log streams, applies processors to enrich or filter events, and forwards them to a configured output — most commonly Elasticsearch directly, or Logstash for further transformation before indexing.

Unlike Logstash, which runs on the JVM and is designed for heavy-duty parsing and enrichment, Filebeat is written in Go and optimised to run with a minimal CPU and memory footprint as a sidecar or DaemonSet agent on every node. It implements backpressure-aware protocols when sending to Logstash or Elasticsearch, slowing its read rate when the downstream is busy rather than dropping events or buffering unboundedly. Filebeat remembers its position in each file it tails, so it resumes correctly after restarts or network interruptions without re-shipping already-processed lines.

In Kubernetes, Filebeat is deployed as a DaemonSet, mounts the host's container log directory, and uses the add_kubernetes_metadata processor to automatically enrich each log event with the pod name, namespace, container name, image, and labels of the container that produced it — without requiring any changes to application logging configuration.

How to use this image

In Kubernetes, Filebeat is deployed as a DaemonSet with the host log directory mounted read-only, and its configuration supplied via a ConfigMap.

Deploy Filebeat as a DaemonSet:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: logging
spec:
  selector:
    matchLabels:
      app: filebeat
  template:
    metadata:
      labels:
        app: filebeat
    spec:
      serviceAccountName: filebeat
      containers:
        - name: filebeat
          image: registry.echo.ai/filebeat:8.12.0
          args: ["-c", "/etc/filebeat/filebeat.yml", "-e"]
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: config
              mountPath: /etc/filebeat
            - name: varlog
              mountPath: /var/log
              readOnly: true
            - name: containers
              mountPath: /var/lib/docker/containers
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: filebeat-config
        - name: varlog
          hostPath:
            path: /var/log
        - name: containers
          hostPath:
            path: /var/lib/docker/containers

With a corresponding ConfigMap that collects container logs and enriches them with Kubernetes metadata before forwarding to Elasticsearch:

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: logging
data:
  filebeat.yml: |
    filebeat.inputs:
      - type: container
        paths:
          - /var/lib/docker/containers/*/*.log
        processors:
          - add_kubernetes_metadata:
              host: ${NODE_NAME}
              matchers:
                - logs_path:
                    logs_path: /var/lib/docker/containers/
    output.elasticsearch:
      hosts: ["https://elasticsearch:9200"]
      username: "${ES_USERNAME}"
      password: "${ES_PASSWORD}"

Once the DaemonSet rolls out, Filebeat begins tailing all container logs on each node and forwarding enriched events to Elasticsearch. Position state is persisted to /usr/share/filebeat/data so events are not re-shipped after pod restarts.

Image variants

Published under docker.elastic.co/beats/filebeat and mirrored at registry.echo.ai/filebeat, the image is versioned in lockstep with the Elastic Stack:

• filebeat:<version> — Version-pinned tags (e.g., 8.12.0) aligned with Elastic Stack releases. Filebeat, Elasticsearch, Logstash, and Kibana should all run the same major and minor version to ensure index template and API compatibility. This is the recommended tag for production.
• filebeat:latest — Tracks the most recent Elastic release. Not recommended for production given the strict version alignment requirement across the Elastic Stack.
• filebeat:<version>-oss — Open-source licensed build of Filebeat, without Elastic's proprietary features. Used in environments where the Elastic licence terms are a constraint.

Filebeat ships with built-in modules for common log formats — nginx, system, Kubernetes audit logs, and more — that bundle pre-built Elasticsearch ingest pipelines and Kibana dashboards. These are enabled per-module rather than being separate image variants.