Environment Variables, ConfigMaps & Secrets in Kubernetes

Every real application needs settings: a database host, a username, a password, an API URL, a log level, a feature flag. You could bake them into the Docker image, but then you need a new image for dev, test and production, and every password ends up inside an image anyone can pull.

The better rule: build one image, and give it its configuration at runtime. Kubernetes has three tools for that:

We use a MariaDB database for the demo, because it refuses to start until you give it configuration. That makes it easy to see whether our settings arrived.

Step 1: A pod that needs configuration

You can create a Deployment straight from the command line, without a YAML file:

kubectl create deploy mydb --image=mariadb
kubectl get pods

NAME                    READY   STATUS              RESTARTS   AGE
mydb-5fcf4f6fbb-bjvpg   0/1     ContainerCreating   0          9s
...
mydb-5fcf4f6fbb-bjvpg   0/1     Error               1 (6s ago) 16s

It starts and then fails. The logs tell you why:

kubectl logs mydb-5fcf4f6fbb-bjvpg

[ERROR] [Entrypoint]: Database is uninitialized and password option is not specified
    You need to specify one of MARIADB_ROOT_PASSWORD, MARIADB_ROOT_PASSWORD_HASH, MARIADB_ALLOW_EMPTY_ROOT_PASSWORD and MARIADB_RANDOM_ROOT_PASSWORD

The image needs an environment variable. After a few restarts the status becomes CrashLoopBackOff, which you met in Kubernetes Pod Statuses.

Step 2: Plain environment variables

kubectl set env deploy/mydb MARIADB_ROOT_PASSWORD=root123
kubectl get pods

NAME                    READY   STATUS    RESTARTS   AGE
mydb-7c9d8f6b5d-q2wxz   1/1     Running   0          5s

We did not redeploy anything by hand. kubectl set env changed the Deployment's pod template, and any change to the template triggers a rolling update: a new pod with the variable was created and the broken one removed.

This is quick, but it does not scale. Ten settings means ten flags, and the password is now in your shell history and in the Deployment spec in plain text. Configuration also belongs in files you can review, not in commands you have to remember.

Step 3: ConfigMaps for many settings

A ConfigMap stores configuration as key-value pairs. Pods read them when they start. It is meant for non-sensitive data: hostnames, ports, application names, log levels, feature flags.

Create a file with the settings, one KEY=value per line:

vi vars

MARIADB_ROOT_PASSWORD=root123
MARIADB_USER=admin

The lecture puts the root password into a ConfigMap only to show how ConfigMaps work. Do not do that in a real project. Passwords go into Secrets (Step 4).

Create a ConfigMap from it:

kubectl create configmap dbvars --from-env-file=vars
kubectl get cm
kubectl describe cm dbvars

NAME               DATA   AGE
dbvars             2      5s
kube-root-ca.crt   1      51m

cm is the short name for configmap. kube-root-ca.crt exists in every namespace: Kubernetes puts the cluster's CA certificate there so pods can trust the API server. describe shows both keys with their values in plain text.

Now use it. Delete the old Deployment and create a fresh one that fails the same way:

kubectl delete deploy mydb
kubectl create deploy newdb --image=mariadb
kubectl set env deploy/newdb --from=configmap/dbvars
kubectl get pods

--from=configmap/dbvars copies every key of the ConfigMap into the container as an environment variable. The pod restarts with both variables and runs.

If you see deployments.apps "mydb" not found, check the name: the second Deployment is newdb.

Step 4: Secrets for sensitive data

A Secret also stores key-value pairs, but it is meant for confidential data: passwords, tokens, API keys, TLS certificates, private keys. Kubernetes treats Secrets more carefully: describe hides the values, they can be limited with RBAC separately from ConfigMaps, and when they are mounted into pods they are kept in memory (tmpfs), not written to the node's disk.

Clean the namespace and create a Secret from the same file:

kubectl delete deploy newdb
kubectl create secret generic my-secret --from-env-file=vars
kubectl get secret

NAME        TYPE     DATA   AGE
my-secret   Opaque   2      6s

generic creates a Secret of type Opaque: arbitrary user data that Kubernetes does not interpret. Other types exist for specific uses, such as kubernetes.io/tls (kubectl create secret tls) and kubernetes.io/dockerconfigjson (kubectl create secret docker-registry) for pulling private images.

Use it the same way:

kubectl create deploy newdb --image=mariadb
kubectl set env deploy/newdb --from=secret/my-secret
kubectl get pods

Base64 is not encryption

kubectl describe secret my-secret

Data
====
MARIADB_ROOT_PASSWORD:  7 bytes
MARIADB_USER:           5 bytes

describe shows only sizes. But look at the object itself:

kubectl get secret my-secret -o yaml

apiVersion: v1
data:
  MARIADB_ROOT_PASSWORD: cm9vdDEyMw==
  MARIADB_USER: YWRtaW4=
kind: Secret
metadata:
  name: my-secret
  namespace: default
type: Opaque

Those values look scrambled, but they are only base64 encoded. Anyone can reverse it:

echo -n "cm9vdDEyMw==" | base64 --decode

root123

The YAML way (what you will use in real projects)

The kubectl set env commands are good for learning. In real projects the Deployment YAML says where each value comes from. The ConfigMap and the Secret can also be YAML:

apiVersion: v1
kind: ConfigMap
metadata:
  name: dbvars
data:
  MARIADB_USER: admin
  MARIADB_DATABASE: telusko
---
apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
stringData:
  MARIADB_ROOT_PASSWORD: root123      # demo only, never commit real values
  MARIADB_PASSWORD: admin123

stringData lets you write plain values. The API server base64 encodes them into data for you.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: newdb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: newdb
  template:
    metadata:
      labels:
        app: newdb
    spec:
      containers:
        - name: mariadb
          image: mariadb:11.4
          envFrom:
            - configMapRef:
                name: dbvars          # every key becomes an env variable
          env:
            - name: MARIADB_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: my-secret
                  key: MARIADB_ROOT_PASSWORD
            - name: MARIADB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: my-secret
                  key: MARIADB_PASSWORD

You can also mount a ConfigMap or Secret as files, which suits whole config files like application.properties or TLS certificates:

          volumeMounts:
            - name: config
              mountPath: /etc/config
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: dbvars          # or: secret: { secretName: my-secret }

Each key becomes a file under /etc/config.

A few more things worth knowing:

• ConfigMaps and Secrets are namespaced. A pod can only use those in its own namespace.
• Each one is limited to 1 MiB.
• If a pod refers to a ConfigMap or Secret that does not exist, it stays in CreateContainerConfigError until you create it.
• Mark config as immutable: true if it should never change. It protects against accidental edits and reduces load on the API server.

Clean up

The lecture uses kubectl delete all --all. That removes Deployments and pods, but not ConfigMaps or Secrets, which are not part of all. Delete them explicitly:

kubectl delete deploy newdb
kubectl delete configmap dbvars
kubectl delete secret my-secret

Try It Yourself

1. Create a ConfigMap with --from-literal=APP_COLOR=blue, inject it with envFrom, and check the value with kubectl exec <pod> -- env | grep APP_COLOR. Change the ConfigMap and check again. What do you need to do to see the new value?
2. Mount the same ConfigMap as a volume, change it, and watch the file with kubectl exec <pod> -- cat /etc/config/APP_COLOR for a minute.
3. Create a Secret with stringData, then read it back with kubectl get secret -o jsonpath='{.data.MARIADB_PASSWORD}' | base64 --decode.
4. Reference a Secret that does not exist in a Deployment. Which pod status do you see, and what does kubectl describe pod say?
5. Rewrite the MySQL StatefulSet from the StatefulSet lesson so that MYSQL_ROOT_PASSWORD comes from a Secret.

Official references

• Kubernetes: ConfigMaps
• Kubernetes: Secrets
• Kubernetes: Good practices for Kubernetes Secrets
• Amazon EKS: Default envelope encryption for all Kubernetes API data

Written By: Shramik Masti