From Docker Compose to Kubernetes: Adding Helm Charts to EventLens
Docker Compose is great until it isn't. Here's how I packaged the entire EventLens stack into a Helm chart — and what Kubernetes forced me to think about that Compose never did.
🐳 The Docker Compose Comfort Zone
Docker Compose is a wonderful lie.
It makes distributed systems feel simple. One file, one command, everything starts. Services talk to each other by name. Volumes just work. You never think about scheduling, health checking, storage classes, or what happens when a container crashes and takes a volume with it.
That's great for development. It's not how production works.
I'd been running EventLens on Docker Compose — API, worker, Kafka, PostgreSQL, Redis, Prometheus, Grafana, Jaeger — all living in one docker-compose.yml. Getting it running locally took seconds. But the whole thing lived on one machine. No redundancy. No self-healing. No way to scale individual components without pulling everything apart and rewriting the config.
Kubernetes is the answer to that. Helm is the answer to Kubernetes being extremely verbose.
📦 What Helm Actually Does
Raw Kubernetes manifests are just YAML files. Lots of them. One per resource, and a non-trivial system has dozens of resources. If you want to deploy the same stack to dev, staging, and production with slightly different values — different replica counts, different image tags, different storage sizes — you either copy-paste everything and maintain three diverging copies, or you template it.
Helm is the templating layer. You write your manifests once with placeholder values like {{ .Values.api.replicas }}, put your actual values in values.yaml, and run helm install. One command deploys the entire stack. Want to bump the API to 3 replicas for production? Change one line in values.yaml. Done.
The chart for EventLens ended up with templates for every component: api.deployment.yml — API serverworker.deployment.yml — Kafka consumer workerpostgres.statefulset.yml — PostgreSQLredis.statefulset.yml — Redisgrafana.deployment.yml — Grafanaprometheus.deployment.yml — Prometheusjaeger.deployment.yml — Jaegeringress.yml — NGINX ingress routingconfigmap.yml — Shared environment config
That's the whole system. One helm install brings all of it up.
🗄️ StatefulSet vs Deployment
This was the first real Kubernetes concept I had to actually understand, not just copy from documentation.
Most workloads in Kubernetes use a Deployment. Deployments are stateless — pods can be killed and replaced freely, and the new pod is identical to the old one. That works perfectly for the API server and the worker. If a pod crashes, Kubernetes spins up a new one. No problem.
PostgreSQL and Redis are different. They have data. If Kubernetes kills a PostgreSQL pod and starts a new one, the new pod needs to find the same data the old one had. That requires two things: a stable network identity (so other services can always find it) and persistent storage that follows the pod.
That's what a StatefulSet is for.
PostgreSQL and Redis both run as StatefulSets with PersistentVolumeClaims attached. The PVC provisions actual disk storage on the cluster. When the pod restarts, it mounts the same volume. The data survives.
volumeClaimTemplates:
- metadata:
name: postgres-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
In Docker Compose, this is invisible. You mount a volume, it works. In Kubernetes, you explicitly declare the storage class, access mode, and size. That explicitness is annoying until you realize it's exactly that explicitness that makes it predictable in production.
💓 Health Probes
Docker Compose has no concept of "is this container actually ready to serve traffic?" It starts the container and assumes everything is fine.
Kubernetes has two probes: liveness and readiness.
Liveness answers: is this pod still alive? If the liveness probe fails repeatedly, Kubernetes restarts the pod.
Readiness answers: is this pod ready to receive traffic? If the readiness probe fails, Kubernetes stops sending requests to that pod — but doesn't restart it.
The distinction matters. An API server might be alive (process is running) but not ready (still connecting to Kafka, still running migrations). Without a readiness probe, Kubernetes would route traffic to it immediately and users would get errors during startup.
The EventLens API already had /healthz and /readyz endpoints from the observability work. Wiring them up in Kubernetes was just pointing the probes at them:
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
httpGet:
path: /readyz
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
/readyz already checks whether the Kafka producer is connected before returning 200. So Kubernetes won't route traffic to the API until Kafka is actually ready. That's the health check paying for itself.
🚦 Ingress Routing
In Docker Compose, NGINX was a separate container configured with a config file. In Kubernetes, the equivalent is an Ingress resource — a set of routing rules that tells the cluster's ingress controller how to direct external traffic.
The EventLens ingress routes everything to three backends:
- path: /()(.*) → api-service:8080
- path: /grafana(/|$)(.*) → grafana-service:3000
- path: /jaeger(/|$)(.*) → jaeger:16686
The API gets the root. Grafana and Jaeger are accessible at sub-paths. The rewrite-target: /$2 annotation strips the path prefix before forwarding, so Grafana receives requests at / even though they came in at /grafana/.
One ingress. Three services. Clean.
⚙️ ConfigMap for Environment Config
All environment variables are centralized in a single ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: eventlens-configmap
data:
DB_HOST: "postgres-service"
KAFKA_BROKERS: "kafka:9092"
REDIS_HOST: "redis-service"
...
# Every deployment pulls from it:
env:
- name: DB_HOST
valueFrom:
configMapKeyRef:
name: eventlens-configmap
key: DB_HOST
...
One thing worth noting: passwords are in the ConfigMap right now. That's fine for local Kubernetes (minikube, kind) but wrong for production. Passwords should live in Secrets — base64-encoded, access-controlled, separate from config. It's the next thing to fix.
🚀 Deploying
helm install eventlens ./helm
That's it. The entire stack — API, worker, Kafka, PostgreSQL, Redis, Grafana, Prometheus, Jaeger, Ingress — comes up in one command.
To change values: helm upgrade eventlens ./helm --set api.replicas=3
To tear everything down: helm uninstall eventlens
The jump from Docker Compose to Kubernetes felt large at first. StatefulSets, PVCs, probes, ingress controllers — there's a lot of surface area. But Helm absorbs most of the complexity. Once the chart is written, operating it feels almost as simple as Compose.
The difference is that what you're operating is actually production-grade.
🏁 Closing Remarks
Helm made the whole thing manageable. Writing raw Kubernetes YAML for a system this size would have produced hundreds of lines of nearly identical manifests. Helm collapses that into a single values file and a set of templates you write once.
If you've been running everything in Docker Compose and it's starting to feel fragile — one machine, no self-healing, no scaling controls — Kubernetes is the next step. It asks more of you upfront. It gives back a lot more in return.
The chart is in the repo at https://github.com/ydv-ankit/eventlens-server, if you want to look at the full setup.
