Daily micro-tips for C#, SQL, performance, and scalable backend engineering.
In a large cluster, you don’t want ‘general’ workloads running on your expensive GPU nodes or high-security master nodes. The Logic: – Taint: Applied to a Node (“Only special pods allowed here”). – Toleration: Applied to a Pod (“I am allowed to run on that special node”). This is the foundation of multi-tenant cluster management […]
Misconfiguring these two can cause your app to go into a restart loop or serve 500 errors to users. Readiness Probe: Tells K8s when the app is ready to receive traffic (e.g., after DB migrations). Liveness Probe: Tells K8s if the app is alive. If this fails, K8s kills and restarts the container. Pro Tip: […]
One of the most common K8s failures is the Out of Memory (OOM) Killer. This happens when your container exceeds its allocated limit. Type Definition Requests What the pod is guaranteed to have. Limits The absolute maximum the pod can consume. Pro Strategy: Always set requests equal to limits for memory to avoid unstable rescheduling […]
🚨 The 3 AM Wake-Up Call Your Kubernetes cluster is down. Users are angry. You have no idea what happened. This is why Fortune 500 companies spend $500K/year on observability. The Complete Observability Stack đź“Š The Three Pillars Metrics – What’s happening right now (Prometheus) Logs – What happened in the past (Loki) Traces – […]
Maintaining separate YAML files for dev/staging/prod is messy. Kustomize manages variations without duplication. Base Configuration: # base/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: replicas: 2 template: spec: containers: – name: app image: myapp:latest Production Override: # overlays/production/kustomization.yaml resources: – ../../base patchesStrategicMerge: – replicas.yaml # Sets replicas: 10 – image.yaml # Sets image: myapp:1.5.3 […]
In a professional DevOps environment, the Git repository is the Single Source of Truth. Manual kubectl apply commands are replaced by ArgoCD. Git Push → CI Build → Manifest Update → ArgoCD Sync → K8s Cluster This ensures that your cluster state always matches your code, enabling instant rollbacks and better security audits.
In a production cluster, you often want to keep ‘special’ nodes (like GPU-powered ones) only for specific pods. Concept: • Taint: The Node says “I only allow special pods.” • Toleration: The Pod says “I am allowed on that special node.” # Add taint to node kubectl taint nodes node1 key1=value1:NoSchedule
In a multi-tenant cluster, one runaway pod can consume all resources and crash other services. The Solution: Define a ResourceQuota per namespace. This limits the total CPU and Memory that a specific team or environment (dev/prod) can use. apiVersion: v1 kind: ResourceQuota metadata: name: mem-cpu-demo spec: hard: requests.cpu: “1” requests.memory: 1Gi limits.cpu: “2” limits.memory: 2Gi
If your pod is ‘Alive’ but not ‘Ready’ to serve traffic (e.g., still loading cache), K8s might send traffic to a black hole. Use Readiness Probes. readinessProbe: httpGet: path: /health/ready port: 8080 initialDelaySeconds: 5
Not setting resources leads to unpredictable behavior. Requests guarantee minimum, Limits cap maximum. Understanding the Difference: – Requests: Guaranteed resources (used for scheduling) – Limits: Maximum resources (hard cap) Example Configuration: apiVersion: v1 kind: Pod metadata: name: myapp spec: containers: – name: app image: myapp:1.0 resources: requests: memory: “256Mi” cpu: “250m” # 0.25 CPU core […]
Rolling updates can accidentally take down too many pods. PDB guarantees minimum availability. Create PDB: apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: myapp-pdb spec: minAvailable: 2 # Always keep at least 2 pods running selector: matchLabels: app: myapp Or Use maxUnavailable: spec: maxUnavailable: 1 # At most 1 pod can be down at a time selector: […]
Storing passwords in ConfigMaps exposes them in plain text. Secrets encode data and have better access control. Create Secret: kubectl create secret generic db-secret \ –from-literal=username=admin \ –from-literal=password=secretpass123 Use in Pod: apiVersion: v1 kind: Pod metadata: name: myapp spec: containers: – name: app image: myapp:1.0 env: – name: DB_USER valueFrom: secretKeyRef: name: db-secret key: username […]
Manual scaling is reactive and slow. HPA automatically scales pods based on resource usage. Create HPA: kubectl autoscale deployment myapp \ –cpu-percent=70 \ –min=2 \ –max=10 # When CPU > 70%, adds pods (up to 10) # When CPU < 70%, removes pods (down to 2) YAML Version: apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: myapp-hpa […]
Hardcoding config in containers requires rebuilding images. ConfigMaps separate config from code. Create ConfigMap: kubectl create configmap app-config \ –from-literal=API_URL=https://api.example.com \ –from-literal=MAX_RETRIES=3 Use in Pod: apiVersion: v1 kind: Pod metadata: name: myapp spec: containers: – name: app image: myapp:1.0 envFrom: – configMapRef: name: app-config Change config without rebuilding image – just update ConfigMap! Update ConfigMap: […]
Want to see CPU/Memory usage without installing monitoring tools? kubectl top shows real-time stats. Node Resource Usage: kubectl top nodes # Output: # NAME CPU MEMORY # node-1 45% 2.5Gi # node-2 23% 1.8Gi Pod Resource Usage: kubectl top pods # Or specific namespace: kubectl top pods -n production # Sort by CPU: kubectl top […]
Applying YAML changes blindly is risky. Preview exactly what will change first. Before Applying: # See what kubectl apply will change kubectl diff -f deployment.yaml # Output shows: # – Lines being removed # + Lines being added # ~ Lines being modified Safe Workflow: # 1. Check diff kubectl diff -f deployment.yaml # 2. […]
Tired of running kubectl logs for each pod separately? Stern streams logs from multiple pods simultaneously. Install Stern: # Mac brew install stern # Linux wget https://github.com/stern/stern/releases/download/v1.28.0/stern_linux_amd64 Usage: # Tail all pods matching pattern stern my-app # All pods in namespace stern . -n production # Color-coded by pod, auto-follows new pods Why Better: Kubectl […]
Paying for multiple cloud load balancers? Ingress routes traffic to different services based on hostname or path. # ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: main-ingress annotations: # NGINX Ingress Controller nginx.ingress.kubernetes.io/rewrite-target: / nginx.ingress.kubernetes.io/ssl-redirect: “true” nginx.ingress.kubernetes.io/proxy-body-size: “10m” # Cert-Manager for SSL cert-manager.io/cluster-issuer: “letsencrypt-prod” spec: tls: – hosts: – api.example.com – app.example.com – admin.example.com secretName: tls-secret […]
Pod stuck in “Terminating” state for hours? Kubernetes is waiting for graceful shutdown that will never complete. Force delete it properly. The Problem: # Pod stuck forever kubectl get pods NAME STATUS AGE stuck-pod-abc123 Terminating 3h # Normal delete doesn’t work kubectl delete pod stuck-pod-abc123 # Still shows Terminating… Why Pods Get Stuck: When deleting […]
Your pod crashes in 2 seconds, and logs vanish before you can read them? Here’s how to catch the output before Kubernetes deletes the container. The Problem: When a pod CrashLoops, the container exits so fast that ‘kubectl logs’ shows nothing useful or “container not found” errors. By the time you run the command, Kubernetes […]
This is a classic production trap. Root cause readinessProbe ≠livenessProbe Most people use only one. Correct pattern livenessProbe: httpGet: path: /health/live port: 80 readinessProbe: httpGet: path: /health/ready port: 80 Why this matters Liveness = should I restart? Readiness = should I receive traffic? If your app is warming caches or reconnecting DBs, traffic arrives […]
Everything green… pods still restart. Hidden killer Liveness probes too aggressive Short timeouts during GC pauses Cold starts under load Fix Separate readiness & liveness Increase initialDelaySeconds Avoid HTTP probes on heavy endpoints livenessProbe: initialDelaySeconds: 30 timeoutSeconds: 5
Pods restart, logs look clean. Hidden reason Memory limit hit → kernel kills container → no app log. Check kubectl describe pod <pod-name> Look for: Reason: OOMKilled Fix Increase memory or fix memory leak — don’t just scale blindly.
livenessProbe: httpGet: path: /health port: 80 Why it mattersWrong probes cause infinite restarts or traffic to broken pods.
resources: requests: cpu: “250m” memory: “256Mi” Why it mattersWithout requests, Kubernetes can’t schedule intelligently → random slowdowns.
livenessProbe: httpGet: path: /health readinessProbe: httpGet: path: /ready Why it mattersLiveness restarts pods, readiness controls traffic. Mixing them causes outages.
Readiness vs Liveness — One Mistake Takes Down Clusters livenessProbe: httpGet: path: /health/live readinessProbe: httpGet: path: /health/ready Why this mattersLiveness restarts pods. Readiness controls traffic. Mixing them causes cascading failures.
Most outages happen because traffic hits half-ready pods. readinessProbe: httpGet: path: /health/ready port: 80 Key idea: Liveness = “restart me” Readiness = “send traffic or not” If you only use liveness → Kubernetes will happily route traffic to chaos.
Most people stare at logs too long. Kubernetes already tells you why your pod died. Fastest diagnostic command: kubectl describe pod <pod-name> Look for: Last State: Terminated Reason: OOMKilled Why this matters:Logs may be empty if the container never fully started. Real fix:Increase memory requests, not limits: resources: requests: memory: “512Mi” limits: memory: “1Gi” Result:Stable […]
Metrics look clean, pods restart anyway. Why it happensProcess exits with code 0 (app logic exit). Why it mattersKubernetes assumes failure. Vital fix Keep main process alive or use proper controllers.