Introduction
Modern infrastructure demands sysadmins to go beyond basic container orchestration. Docker and Kubernetes (K8s) are powerful, but misconfigurations can lead to security breaches, wasted resources, or downtime. This guide dives into advanced techniques for optimizing images, hardening clusters, debugging at scale, and cutting costs without sacrificing performance.
1. Docker Deep Dive: Beyond the Basics
Optimizing Images for Production
Problem: Bloated images slow deployments and increase attack surfaces.
Solutions:
- Multi-stage builds: Remove build dependencies in the final image.
dockerfile
Copy
Download
# Stage 1: Build
FROM golang:1.21 as builder
WORKDIR /app
COPY . .
RUN go build -o myapp .
# Stage 2: Runtime
FROM alpine:latest
COPY –from=builder /app/myapp /usr/local/bin/
CMD [“myapp”]
- Minimal base images: Prefer distroless or alpine over ubuntu.
- Layer caching: Order Dockerfile commands from least to most frequently changed.
Pro Tip: Use dive to analyze image layers:
bash
Copy
Download
dive my-image:latest
Advanced Storage & Networking
- Overlay2 Driver: Default for Docker, but tune dm.basesize for disk space limits.
- Custom Bridge Networks: Isolate containers and control traffic.
bash
Copy
Download
docker network create –driver=bridge –subnet=192.168.100.0/24 my_network
- Volume Optimization: Use tmpfs for ephemeral data to reduce I/O overhead.
yaml
Copy
Download
# docker-compose.yml
services:
redis:
volumes:
– type: tmpfs
target: /data
Security Hardening
- Rootless Containers: Run Docker daemon as a non-root user (requires uidmap).
bash
Copy
Download
dockerd-rootless-setuptool.sh install
- Seccomp Profiles: Restrict syscalls. Use the default profile or customize:
json
Copy
Download
{
“defaultAction”: “SCMP_ACT_ERRNO”,
“syscalls”: [{“name”: “mkdir”, “action”: “SCMP_ACT_ALLOW”}]
}
- Image Signing: Verify integrity with Docker Content Trust (DCT):
bash
Copy
Download
export DOCKER_CONTENT_TRUST=1
docker push my-repo/image:signed
2. Kubernetes Mastery for Sysadmins
Advanced Scheduling
- Taints & Tolerations: Dedicate nodes for specific workloads (e.g., GPU nodes).
yaml
Copy
Download
# Taint a node
kubectl taint nodes node1 gpu=true:NoSchedule
# Pod toleration
tolerations:
– key: “gpu”
operator: “Equal”
value: “true”
effect: “NoSchedule”
- Topology Spread Constraints: Prevent zone outages by spreading pods evenly.
yaml
Copy
Download
topologySpreadConstraints:
– maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
Resource Management
- LimitRanges: Enforce default CPU/memory requests per namespace.
yaml
Copy
Download
apiVersion: v1
kind: LimitRange
metadata:
name: cpu-limiter
spec:
limits:
– defaultRequest:
cpu: “500m”
type: Container
- Autoscaling: Combine HPA (Horizontal Pod Autoscaler) and VPA (Vertical Pod Autoscaler).
bash
Copy
Download
# Install VPA
kubectl apply -f https://github.com/kubernetes/autoscaler/releases/download/vertical-pod-autoscaler-0.14.0/vertical-pod-autoscaler.yaml
Debugging Tricks
- Ephemeral Containers: Inspect running pods without restarting.
bash
Copy
Download
kubectl debug -it my-pod –image=busybox –target=my-pod
- Logging Pipelines: Send logs to Elasticsearch via Fluentd.
yaml
Copy
Download
# fluentd-configmap.yaml
data:
fluent.conf: |
<source>
@type tail
path /var/log/containers/*.log
read_from_head true
<parse>
@type json
</parse>
</source>
<match **>
@type elasticsearch
host elasticsearch.default.svc.cluster.local
</match>
3. Security at Scale
Pod Security Policies (Replacement)
K8s deprecated PSPs in 1.25. Use Pod Security Admission (PSA) or OPA Gatekeeper:
yaml
Copy
Download
# PSA example (enforce baseline policy)
apiVersion: v1
kind: Namespace
metadata:
name: secure-ns
labels:
pod-security.kubernetes.io/enforce: baseline
Network Policies
- Zero-Trust with Calico: Deny all traffic by default, then allow specific flows.
yaml
Copy
Download
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-all
spec:
podSelector: {}
policyTypes:
– Ingress
– Egress
Secrets Management
- HashiCorp Vault + CSI Driver: Inject secrets dynamically.
yaml
Copy
Download
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: vault-db-creds
spec:
provider: vault
parameters:
vaultAddress: “http://vault:8200”
roleName: “k8s-role”
objects: |
– objectPath: “secret/database”
secretKey: “password”
4. Performance & Cost Optimization
Cluster Sizing
- Spot Instances: Save 90% with intelligent node pools (AWS Spot, GCP Preemptible).
yaml
Copy
Download
# AWS EKS node group with spot
apiVersion: eksctl.io/v1alpha5
nodeGroups:
– name: ng-spot
instanceTypes: [“t3.large”, “t3a.large”]
spot: true
Monitoring
- Prometheus Alerts: Detect memory leaks early.
yaml
Copy
Download
– alert: HighMemoryUsage
expr: container_memory_usage_bytes{pod=”my-pod”} > 1GB
for: 10m
Waste Reduction
- Kube-cost: Track spending by namespace/deployment.
bash
Copy
Download
kubectl cost –window 7d –show-all-resources
Conclusion
Mastering advanced Docker and Kubernetes requires a balance of security, efficiency, and observability. Key takeaways:
- Docker: Optimize images, harden runtime, and use rootless mode.
- Kubernetes: Leverage advanced scheduling, autoscaling, and OPA for governance.
- Costs: Right-size clusters and automate scaling policies.
Next Steps:
- Experiment in a sandbox (e.g., Kind or Minikube).
- Join the K8s Slack (#security, #autoscaling channels).
- Read the CIS Benchmarks for compliance.
