Container and Kubernetes Security Hardening Guide

Executive Summary

Kubernetes has become the de facto platform for running containerised workloads at scale. Its adoption has outpaced security understanding: many clusters are deployed with default configurations, misconfigured RBAC, over-permissioned service accounts, and no runtime security controls.

This guide provides hardening guidance for container images and Kubernetes clusters, aligned with the CIS Kubernetes Benchmark v1.9 and NSA/CISA Kubernetes Hardening Guide. It is written for DevOps engineers, cloud architects, and security teams responsible for securing containerised infrastructure.

Chapter 1: Container Image Security

The Principle of Minimal Images

The security attack surface of a container is determined by what is in the image. Every tool, library, and process in a container that is not required for its function is a potential vulnerability.

Image hierarchy (most to least secure):

CyberneticsPlus recommendation: Use distroless or Alpine for production containers. Never use latest as a base tag — pin to a specific digest or version tag for reproducibility and auditability.

# DON'T — large attack surface, mutable tag
FROM python:latest

# DO — minimal, pinned
FROM python:3.12.3-slim-bookworm@sha256:abc123...

Multi-Stage Builds

Multi-stage builds separate build tools from the runtime image:

# Stage 1: Build
FROM golang:1.22 AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /app/server

# Stage 2: Runtime — only the binary
FROM gcr.io/distroless/static-debian12
COPY --from=builder /app/server /server
EXPOSE 8080
USER nonroot:nonroot
ENTRYPOINT ["/server"]

The runtime image contains only the compiled binary — no Go toolchain, no package manager, no shell.

Run as Non-Root

Running containers as root is the most common container security misconfiguration. If an attacker achieves container breakout, running as root gives them host root access.

# Create a non-root user
RUN addgroup --gid 1001 appgroup && \
    adduser --uid 1001 --gid 1001 --disabled-password appuser

USER appuser:appgroup

Or with distroless images (which include a nonroot user):

USER nonroot:nonroot

Container Image Scanning

Scan images for known CVEs before deployment:

# GitHub Actions: Trivy image scan
- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: '${{ env.IMAGE_NAME }}:${{ github.sha }}'
    format: 'sarif'
    output: 'trivy-results.sarif'
    exit-code: '1'
    severity: 'CRITICAL,HIGH'
    ignore-unfixed: true

- name: Upload scan results to GitHub Security
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: 'trivy-results.sarif'

Scanning cadence:

• On every build (new images)
• Continuous scanning of deployed images (images contain new CVEs as they are disclosed after initial build)

Tools that support continuous scanning of running images: AWS ECR enhanced scanning (Inspector), Azure Defender for Containers, GCP Artifact Analysis, Anchore, Snyk Container.

Software Bill of Materials (SBOM)

Generate an SBOM for every production image. An SBOM is a complete list of all software components in the image — enabling rapid impact assessment when a new CVE is disclosed.

# Generate SBOM with Syft
syft myimage:v1.0 -o cyclonedx-json > sbom.json

# Scan SBOM with Grype
grype sbom:sbom.json

Chapter 2: Kubernetes RBAC

Understanding RBAC Objects

Kubernetes RBAC has four object types:

Principle of Least Privilege in RBAC

The default cluster-admin ClusterRole grants unrestricted access to all resources. Never bind cluster-admin to service accounts, users, or groups unless absolutely necessary.

Over-permissioned example (bad):

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: my-service-account-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin  # NEVER DO THIS for application service accounts
subjects:
  - kind: ServiceAccount
    name: my-service-account
    namespace: production

Least privilege example (correct):

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: pod-reader
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: production
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pod-reader
subjects:
  - kind: ServiceAccount
    name: my-service-account
    namespace: production

Service Account Hardening

Disable automounting of service account tokens for pods that don’t need API access:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: no-api-access
automountServiceAccountToken: false

Or at the pod level:

spec:
  automountServiceAccountToken: false

One service account per application: Do not share service accounts across applications. Each application should have its own service account with only its required permissions.

RBAC Audit

Regularly audit RBAC configuration:

# List all ClusterRoleBindings to cluster-admin
kubectl get clusterrolebindings -o json | \
  jq '.items[] | select(.roleRef.name == "cluster-admin") | .metadata.name'

# List all service accounts with their roles
kubectl get rolebindings,clusterrolebindings -A -o json | jq '.items[] |
  select(.subjects[]?.kind == "ServiceAccount") |
  {name: .metadata.name, namespace: .metadata.namespace, role: .roleRef.name}'

Tool: rbac-lookup (open source) provides a clear view of what subjects have access to in a cluster.

Chapter 3: Network Policies

Default-Deny Network Policy

By default, Kubernetes allows all pod-to-pod communication within a cluster. This means a compromised pod can freely communicate with any other pod — enabling lateral movement.

Implement default-deny policies and explicit allow rules:

# Default deny all ingress and egress in a namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}  # Applies to all pods in namespace
  policyTypes:
    - Ingress
    - Egress

Then explicitly allow required communication:

# Allow frontend to communicate with backend on port 8080
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

CNI requirement: Network Policies require a CNI plugin that supports them. Calico, Cilium, and Weave Net support Network Policies. The default AWS VPC CNI and Azure CNI require additional configuration or an alternative CNI.

Service Mesh for mTLS

For environments requiring encrypted pod-to-pod communication and fine-grained traffic control, a service mesh (Istio, Linkerd) provides:

• Mutual TLS (mTLS) between all services
• Traffic authorisation policies (beyond what Network Policies can express)
• Observability (traffic metrics, distributed tracing)

This is more complex to operate but provides stronger security for sensitive workloads.

Chapter 4: Pod Security Standards

Pod Security Admission

Kubernetes v1.25+ ships with Pod Security Admission (PSA) as a stable feature, replacing the deprecated PodSecurityPolicy.

PSA enforces three Pod Security Standards:

Apply PSS at the namespace level:

# Enforce restricted policy on a namespace
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: v1.29
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: v1.29

Recommendation: Use restricted for all production namespaces. Use baseline for namespaces with workloads that cannot meet restricted requirements (some legacy applications). Audit which namespaces use privileged — this should be minimal (system namespaces only).

Security Context Best Practices

spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1001
    runAsGroup: 1001
    fsGroup: 1001
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop:
            - ALL
        readOnlyRootFilesystem: true

Key settings:

• allowPrivilegeEscalation: false — prevents sudo and setuid binaries from escalating privileges
• capabilities: drop: ALL — removes all Linux capabilities (containers inherit many by default)
• readOnlyRootFilesystem: true — prevents writes to the container filesystem (forces use of mounted volumes for any required writes)
• runAsNonRoot: true — enforces non-root execution

Chapter 5: Secrets Management in Kubernetes

The Problem with Kubernetes Secrets

Kubernetes Secrets are, by default:

• Base64-encoded (not encrypted) — anyone with access to the etcd database can read them in plaintext
• Accessible to any pod in the same namespace with the right service account permissions
• Not encrypted at rest in etcd unless explicitly configured

Encryption at Rest

Enable encryption at rest for etcd:

# /etc/kubernetes/encryption-config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: <base64-encoded-32-byte-key>
      - identity: {}

For managed Kubernetes (EKS, AKS, GKE), enable envelope encryption using cloud KMS:

• EKS: Enable envelope encryption with KMS key on cluster creation
• AKS: Enable etcd encryption with Azure Key Vault integration
• GKE: Application-level encryption is enabled by default; CMEK available

External Secrets Operator

For the most secure secrets management, use External Secrets Operator (ESO) to pull secrets from external secret stores (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, HashiCorp Vault) at pod startup:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: database-credentials
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: database-credentials
    creationPolicy: Owner
  data:
    - secretKey: DB_PASSWORD
      remoteRef:
        key: production/database
        property: password

ESO creates a standard Kubernetes Secret, but the source of truth is your cloud secrets manager. Secrets are automatically refreshed on rotation.

Chapter 6: Runtime Security and Threat Detection

Falco — Runtime Threat Detection

Falco is the CNCF standard for Kubernetes runtime security. It monitors system calls and Kubernetes API events to detect anomalous behaviour:

# Example Falco rule: detect shell spawned in container
- rule: Terminal shell in container
  desc: A shell was used as the entrypoint/exec point into a container
  condition: >
    spawned_process and container
    and shell_procs and proc.tty != 0
    and container_entrypoint
  output: >
    A shell was spawned in a container with an attached terminal
    (user=%user.name container_id=%container.id image=%container.image.repository
    shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)
  priority: NOTICE

Falco default rules detect: shell execution in containers, file writes to sensitive paths, network connections to unusual destinations, privilege escalation, credential file reads.

Integration: Falco → Falcosidekick → Slack/PagerDuty/SIEM for real-time alerting.

Kubernetes Audit Logging

Enable Kubernetes audit logging to capture all API server requests:

# Audit policy
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  # Log all requests to the audit log
  - level: Metadata
    resources:
      - group: ""
        resources: ["secrets", "configmaps"]
  # Log pod exec and port-forward
  - level: Request
    verbs: ["create"]
    resources:
      - group: ""
        resources: ["pods/exec", "pods/portforward"]
  # Log all other requests at Metadata level
  - level: Metadata

Ship audit logs to your SIEM for analysis. Key events to alert on:

• kubectl exec on production pods
• Secret creation or modification
• RBAC changes (new RoleBindings, ClusterRoleBindings)
• Service account token creation
• Pod with privileged: true security context admitted

Chapter 7: CIS Kubernetes Benchmark

The CIS Kubernetes Benchmark provides scored hardening recommendations across:

• 1.x — Control Plane Components: API server, controller manager, scheduler configuration
• 2.x — etcd: etcd security configuration, TLS, client cert auth
• 3.x — Control Plane Configuration: Authentication, authorisation modes
• 4.x — Worker Nodes: Kubelet configuration
• 5.x — Policies: RBAC, Pod Security, CNI, secrets, general policies

Running the CIS Benchmark

Use kube-bench to assess your cluster against the CIS Benchmark:

# Run against current cluster
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml

# Check results
kubectl logs job.batch/kube-bench

For managed Kubernetes services (EKS, AKS, GKE), many control plane benchmarks are managed by the cloud provider — focus on the node and policy sections.

Key CIS Benchmark Checks

Conclusion: Kubernetes Security Hardening Checklist

Image Security

• Minimal base images (distroless or Alpine)
• Multi-stage builds
• Non-root user in Dockerfile
• All images scanned for CVEs before deployment (Trivy/Snyk)
• Image scanning in CI/CD — Critical CVEs block deployment
• SBOM generated for all production images
• Image tags pinned (no latest)

Kubernetes Configuration

• RBAC: no cluster-admin bindings for application service accounts
• RBAC: one service account per application, least privilege
• Service account token automounting disabled where not needed
• Network Policies: default-deny in all production namespaces
• Pod Security Standards: restricted enforced in production namespaces
• Security contexts configured on all pods (non-root, no privilege escalation, drop capabilities)
• etcd encryption at rest enabled
• Kubernetes audit logging enabled and shipped to SIEM

Secrets Management

• No credentials in ConfigMaps or environment variables
• Kubernetes Secrets encrypted at rest
• External Secrets Operator for secrets from cloud KMS (recommended)
• Secret rotation supported and tested

Runtime Security

• Falco (or equivalent) deployed for runtime threat detection
• Alerts configured for critical Falco rules (shell in container, privilege escalation)
• Kubernetes API audit log alerts for sensitive operations (exec, RBAC changes, secret access)

Continuous Assessment

• kube-bench run quarterly against CIS Benchmark
• RBAC audit (rbac-lookup) quarterly
• Container image re-scan of deployed images (continuous via registry scanning)
• Annual penetration test of Kubernetes environment

CyberneticsPlus provides Kubernetes security assessments, CIS Benchmark hardening, and container security programme implementation. Contact us to secure your container workloads.