- App dies at startup — bad config, missing env, failed migration.
- Liveness probe too aggressive: killing a slow-starting pod.
- Backoff resets after 10 minutes of stability.
diagnose
kubectl describe pod <pod> | sed -n '/Events/,$p'
kubectl logs <pod> --previous --tail=100
fixes
kubectl set env deploy/<name> KEY=value # supply missing config
# add initialDelaySeconds / startupProbe for slow boots
- Image/tag doesn’t exist or repo is private.
- Missing or wrong imagePullSecrets on the ServiceAccount.
- Registry rate limit (Docker Hub anonymous pulls).
diagnose
kubectl describe pod <pod> | grep -A5 "Failed"
kubectl get secret regcred -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d | jq .
fixes
kubectl create secret docker-registry regcred --docker-server=<r> --docker-username=<u> --docker-password=<p>
- Insufficient CPU/memory on every node in the scheduler’s view.
- No node matches nodeSelector / affinity / taints.
- PVC unbound (no StorageClass or wrong accessMode).
diagnose
kubectl describe pod <pod> | sed -n '/Events/,$p' # scheduler messages end here
kubectl get nodes -o custom-columns=NAME:.metadata.name,CPU:.status.allocatable.cpu,MEM:.status.allocatable.memory
- Working set exceeds the memory limit — kernel kills PID 1.
- JVM/heap sized above the cgroup limit.
- Memory spike from traffic burst without HPA headroom.
diagnose
kubectl top pod <pod> --containers
kubectl describe pod <pod> | grep -i -B2 oom
# raise limits OR fix the leak; check exit reason 137
- selector doesn’t match pod labels (case-sensitive).
- TargetPort mismatch with containerPort.
- Endpoints empty — ready probes failing, so pods removed.
diagnose
kubectl get endpointslices -l kubernetes.io/service-name=<svc>
kubectl run tmp --rm -it --image=busybox --restart=Never -- wget -qO- http://<svc>.<ns>.svc.cluster.local
- ServiceAccount lacks RoleBinding in this namespace.
- ClusterRole exists but nobody binds it.
- API group or subresource spelled differently (deployments/apps, logs/pods).
diagnose
kubectl auth can-i --list --as=system:serviceaccount:<ns>:<sa>
kubectl auth can-i delete deployments --as=system:serviceaccount:<ns>:<sa>