Skip to content

jmnicolescu/create-fluentd-k8s-windows-image

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

Custom Fluentd Docker Images for Kubernetes on Windows Server

Author: Julius Nicolescu Date: March 2026


Overview

This repository builds custom Fluentd Docker images for Kubernetes log collection on Windows Server. Two variants are provided — one for Windows Server Core LTSC2019 and one for Windows Server Core LTSC2022.

Variant Directory README
Fluentd 1.19.2 on Windows Server Core LTSC2019 fluentd-image-2019/ README.md
Fluentd 1.19.2 on Windows Server Core LTSC2022 fluentd-image-2022/ README.md

Why Custom Images

The official Fluentd Windows images published at fluent/fluentd-docker-image ship with only the Fluentd core gem. They include no plugins. A production Kubernetes log collection pipeline requires additional plugins for three distinct functions:

  1. Metadata enrichment — raw container logs carry no Kubernetes context. Without a metadata filter, there is no way to identify which pod, namespace, or workload a log record originated from.
  2. Log parsing — containers in a Kubernetes cluster emit logs in multiple formats: JSON application logs, CRI-formatted runtime logs, and multiline output from Java or Python stack traces. Each format requires a dedicated parser.
  3. Output routing — the core Fluentd gem can write to local files, but forwarding to Elasticsearch or VMware Aria Operations for Logs requires output plugins that are not bundled in the base image.

These images extend the official Windows base with all required plugins installed and the gem cache removed to keep the image size as small as possible.


Image Summary

Property LTSC2019 LTSC2022
Base image mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2019 mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2022
Fluentd version 1.19.2 1.19.2
Ruby version 3.4.5.1 3.4.5.1
Image size (approx.) 8.8 GB 7.1 GB
Docker Hub jmnicolescu/fluentd-win2019-k8s jmnicolescu/fluentd-win2022-k8s

Exposed ports: 24224/tcp (Fluentd forward protocol), 5140/tcp (syslog).


Installed Plugins

All plugins are installed via gem install --no-document during the Docker build. The gem cache is cleared in the same layer to avoid persisting downloaded .gem files in the image.

Kubernetes Metadata Enrichment

fluent-plugin-kubernetes_metadata_filter

Filter plugin that enriches each log record with Kubernetes metadata by querying the Kubernetes API server at runtime. Adds namespace_name, pod_name, pod_id, pod_ip, container_name, and all pod labels and annotations to every record. Without this plugin, log records have no Kubernetes context and cannot be correlated to a workload. Reference: https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter

fluent-plugin-kubernetes

Shared Kubernetes client library used by other Fluentd plugins. Handles kubeconfig loading, in-cluster service account authentication, and API server connectivity. Acts as a dependency for the metadata filter and other Kubernetes-aware plugins. Reference: https://github.com/cowabanga/fluent-plugin-kubernetes


Log Parsing and Processing

fluent-plugin-concat

Filter plugin that reassembles multiline log records into a single Fluentd event. Buffers incoming lines and flushes on a configurable start/end regex pattern or a timeout. Required for runtimes that emit a single logical message across multiple lines, such as Java exception stack traces or Python tracebacks. Reference: https://github.com/fluent-plugins-nursery/fluent-plugin-concat

fluent-plugin-rewrite-tag-filter

Filter plugin that rewrites Fluentd event tags using regex rules applied to record field values. Rules are evaluated in order and the first match wins. Used to implement dynamic log routing — for example, sending records from a specific Kubernetes namespace to a different output, or separating application logs from system logs by tag. Reference: https://github.com/fluent/fluent-plugin-rewrite-tag-filter

fluent-plugin-multi-format-parser

Parser plugin that tries multiple format parsers in sequence and returns the result of the first successful match. Handles heterogeneous log sources where the format is not known at configuration time, such as a DaemonSet collecting from containers that emit both JSON and plain-text logs. Reference: https://github.com/repeatedly/fluent-plugin-multi-format-parser

fluent-plugin-parser-cri

Parser plugin for the CRI (Container Runtime Interface) log format emitted by containerd and CRI-O. Each CRI log line consists of a RFC 3339 timestamp, stream (stdout/stderr), partial-log flag, and the message body. This plugin parses that wire format into a structured Fluentd record. Required on any Kubernetes node that uses a CRI-compatible container runtime instead of the legacy Docker shim. Reference: https://github.com/fluent/fluent-plugin-parser-cri CRI logging spec: https://github.com/kubernetes/design-proposals-archive/blob/main/node/kubelet-cri-logging.md


Output Integrations

fluent-plugin-elasticsearch

Output plugin that writes log records to an Elasticsearch cluster using the bulk indexing API. Supports dynamic index naming by date or record field value, TLS, HTTP basic authentication, and index lifecycle management (ILM) policy assignment. The bulk API minimises round-trips and is the recommended ingestion path for high-throughput log pipelines. Reference: https://github.com/uken/fluent-plugin-elasticsearch Elasticsearch bulk API: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

fluent-plugin-vmware-loginsight

Output plugin that forwards log records to VMware Aria Operations for Logs (formerly VMware Log Insight) via its ingestion REST API. Supports custom field mapping, TLS, and agent-based authentication tokens. Used in VMware Cloud Foundation environments where Aria Operations for Logs is the centralised log aggregation platform. Reference: https://github.com/vmware/fluent-plugin-vmware-loginsight Aria Operations for Logs ingestion API: https://docs.vmware.com/en/VMware-Aria-Operations-for-Logs/index.html


Repository Structure

create-fluentd-k8s-windows-images/
├── README.md                        # This file
├── fluentd-image-2019/
│   ├── README.md                    # Build and usage guide for the LTSC2019 image
│   ├── Dockerfile                   # Image definition
│   ├── fluent.conf                  # Default Fluentd configuration
│   └── entrypoint.sh                # Container entrypoint
└── fluentd-image-2022/
    ├── README.md                    # Build and usage guide for the LTSC2022 image
    ├── Dockerfile                   # Image definition
    ├── fluent.conf                  # Default Fluentd configuration
    └── entrypoint.sh                # Container entrypoint

The two variants are functionally identical. The only difference between them is the base image (ltsc2019 vs ltsc2022). Changes to plugins, configuration, or the entrypoint should be applied to both.

About

Custom Fluentd Docker images for Windows Server 2019 and 2022 that extend the official Fluentd base images with Kubernetes metadata enrichment, multiline log parsing, and output plugins for Elasticsearch and VMware Aria Operations for Logs.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors