Status columns are summaries, and the useful detail sits one command deeper in the events and in the dead container's logs. This walks each failure state in the order you meet it, with the exit codes and event messages that narrow a diagnosis to one cause, and separates the failures your manifest caused from the ones the cluster did.
A pod that will not run tells you what is wrong, in a place most people do not look. The status column gives a symptom; the events and the previous container's logs give a cause. Almost every diagnosis below is three commands, and the discipline is running them in order rather than guessing.
The Three Commands
kubectl get pod my-app-7d9f8b6c5-x2klm -o wide
kubectl describe pod my-app-7d9f8b6c5-x2klm
kubectl logs my-app-7d9f8b6c5-x2klm --previousdescribe is the one that matters. Its Events section at the bottom carries the scheduler's and the kubelet's own explanation: why it could not place the pod, why the image pull failed, why the container was killed. Read it before forming a theory.
--previous is the flag people miss. A container in a restart loop has already died, and its logs are gone from the current instance. The previous instance's logs are what contain the stack trace.
Pending Means Nowhere to Put It
Pending is a scheduling failure, not an application failure. The container has not started because no node was found. The events name the reason precisely.
| Event message | Meaning | Fix |
|---|---|---|
| Insufficient cpu / memory | No node has the requested capacity free | Lower the request, or add capacity |
| node(s) had untolerated taint | Nodes are reserved and the pod has no toleration | Add the toleration, or target other nodes |
| pod has unbound immediate PersistentVolumeClaims | The claim is not bound to a volume | Describe the PVC, check the StorageClass |
| didn't match Pod's node affinity/selector | Affinity rules exclude every node | Relax the rule or label a node |
| node(s) didn't have free ports | A hostPort is already taken | Drop the hostPort or use a Service |
The capacity case deserves care, because it is about requests, not usage. A cluster with idle CPU can still refuse a pod if existing pods have reserved everything through generous requests. Compare what is requested against what is allocatable:
kubectl describe node worker-2 | sed -n '/Allocated resources/,/Events/p'CrashLoopBackOff Is a Symptom, Not a Cause
The container starts, exits, and Kubernetes restarts it with a growing delay. All the status tells you is that this has happened more than once. The exit code narrows it down considerably.
kubectl get pod my-app -o jsonpath='{.status.containerStatuses[0].lastState.terminated}'Exit code 1 or 2. The application itself failed. Missing configuration, an unreachable dependency, a bad connection string. Read the previous logs; the answer is nearly always printed there.
Exit code 137. Killed by SIGKILL, which almost always means the memory limit. Covered below.
Exit code 143. SIGTERM, a graceful shutdown that Kubernetes asked for. Usually a failing liveness probe, not the application crashing.
Exit code 126 or 127. The command could not be executed or was not found. A wrong entrypoint, a missing binary, or a shell script without the interpreter it expects.
The liveness case is worth its own note because it is self-inflicted and looks like an application bug. A probe with a short initialDelaySeconds against an application that takes forty seconds to warm up will kill it during startup, forever. Use a startupProbe for slow starts and keep the liveness probe for genuine hangs.
OOMKilled and the Limit You Guessed
OOMKilled means the container exceeded its memory limit and the kernel killed it. It is not the node running out of memory, which is a different and noisier event.
Two causes, and they need opposite responses. Either the limit is too low for the workload's real working set, in which case measure and raise it. Or the application has a leak, in which case raising the limit buys time and postpones the same failure.
kubectl top pod -n prod --sort-by=memory
kubectl get events -n prod --field-selector reason=OOMKillingRuntimes with their own heap management need explicit alignment between the runtime's limit and the container's, or the runtime will happily grow past what the container is allowed and get killed while believing it has headroom. A JVM without a heap setting derived from the container limit is the classic instance.
ImagePullBackOff and ErrImagePull
The kubelet cannot fetch the image. Four causes cover essentially all of them, and describe distinguishes them.
Wrong name or tag. A typo, or a tag that no longer exists because someone overwrote it. The message says not found.
Missing credentials. A private registry with no imagePullSecrets on the pod or its service account. The message mentions authorization or an anonymous pull.
Registry rate limit. Anonymous pulls from public registries are throttled per source address, so a cluster behind one NAT address hits the limit collectively. Authenticate, or mirror the images you depend on.
No egress. A cluster on a private network with no route to the registry. Test from a node, not from your laptop.
Running but Unreachable
The pod is Running and traffic does not arrive. This is a Service or a network problem, and it is a different investigation.
kubectl get endpoints my-svc # empty means no pod matched
kubectl get pod my-app -o jsonpath='{.metadata.labels}'
kubectl get svc my-svc -o jsonpath='{.spec.selector}'Empty endpoints has two causes: the Service selector does not match the pod's labels, or the pods are not Ready because a readiness probe is failing. Both are visible in those three lines. If endpoints are populated and traffic still fails, the problem is above the Service, in ingress or DNS, and our comparison of ingress controllers covers debugging that layer.
A NetworkPolicy is the other quiet culprit. Once any policy selects a pod, everything not explicitly allowed is denied, including responses to traffic you thought was permitted.
When the Cluster Is the Problem
Some failures are not the pod's. A node under memory pressure evicts pods, which appear as Failed with an Evicted reason rather than as a crash. Disk pressure does the same and also stops image pulls. Sudden mass restarts across unrelated workloads usually mean a node went away, not that every application broke at once.
kubectl get nodes
kubectl describe node worker-2 | grep -A6 Conditions
kubectl get events -A --sort-by=.lastTimestamp | tail -30That last command is the one to run first during an incident. A cluster-wide event stream in timestamp order shows the sequence, and the sequence is usually the diagnosis.
Removing a Class of Failure
Two of the categories above are infrastructure properties rather than application bugs. Evictions from node pressure and mass restarts from node loss both come down to how much headroom the cluster has and what happens when a machine disappears.
On MassiveGRID, managed Kubernetes bills in cloudlets of 128 MiB RAM and 400 MHz CPU from $0.03474 per hour, roughly $25.37 a month, so headroom can be added in small increments instead of by buying another whole instance, which is what tempts teams into limits that are too tight. Underneath, Proxmox high-availability clustering restarts a lost node's workloads automatically and Ceph storage replicates every block three times across independent NVMe drives, so a rescheduled pod finds its volume rather than staying Pending on an unbound claim. Where nobody is watching the event stream at three in the morning, NOC services provide monitoring and response.
Clusters can be ordered across a partner footprint of more than 700 datacenters in 85 metros, 30 countries and six continents, with auto-provisioning in New York, London, Frankfurt and Singapore.