Scheduling, probes, workloads and the failure modes that come with them.
109 items · all topics
How an autoscaler actually decides, and why it always lags
A Horizontal Pod Autoscaler runs one formula on a loop: desired = ceil(replicas × current ÷ target). Everything people find surprising about autoscaling (the lag, the overshoot, the slow scale-down) falls out of that formula and the timers around it.
Liveness vs readiness: one restarts, one just stops traffic
A readiness probe decides whether a pod gets traffic. A liveness probe decides whether it gets killed. Point them at the same slow dependency and readiness protects you while liveness takes the whole deployment down.
Explain the architectural difference between AWS ECS Fargate and Amazon EKS (Elastic Kubernetes Service).
ECS and EKS are both orchestrators. Fargate isn't: it's a compute mode that either one can run on, so all four combinations exist. The real comparison is ECS vs EKS on who operates what, and Fargate vs EC2 on whether you want to own nodes at all.
A pod is stuck in Pending for 10 minutes in production. Walk me through how you diagnose it.
Pending means the scheduler hasn't placed the pod on a node. The cause is almost always resources, node selection constraints, or an unbound volume, and `kubectl describe pod` tells you which within seconds if you read the Events section rather than guessing.
What happens if an Auto Scaling instance never becomes healthy?
Auto Scaling terminates and replaces an instance that never passes health checks, which is fine once and expensive in a loop. How long you burn depends on the grace period, the check thresholds, and which health check type the group is actually using. The fix is working out why it's unhealthy: usually startup config, a wrong health endpoint, or a grace period shorter than boot time.
You use GitOps and someone changes Kubernetes manually. Git still has the old config. What happens?
You get configuration drift: Git says one thing, the cluster says another. What happens next depends on whether the controller self-heals or just reports it. Either way the manual change is temporary, and the fix is to put the intended change into Git, not to argue with the controller.
Argo CD keeps reverting your emergency production change. Why, and what should you do?
Self-heal is doing exactly what it was configured to do: pulling the cluster back to what Git says. During an incident, mitigate through something the controller doesn't manage, then get the real change into Git fast rather than fighting reconciliation head on.
Git is the source of truth, but production differs from Git. How do you prove where the drift happened?
Compare state at each stage, Git, rendered manifests, what was applied, and the live cluster, and use Kubernetes' own metadata to name the culprit. managedFields records which controller last wrote each field, which usually answers the question outright without any guessing.
A service depends on a database that goes down for 60 seconds. Which probe configuration behaves correctly?
Readiness controls whether a pod receives traffic; liveness controls whether it gets killed and restarted. Checking a downstream dependency in a liveness probe turns a brief dependency blip into a cluster-wide restart storm.
You run kubectl get pods and the request succeeds, but new Pods created afterwards stay in Pending forever with no events about nodes. Which control plane component is the first one to suspect?
The API server is clearly working, since kubectl gets answers. Deciding which node a Pod runs on is the kube-scheduler's only job, so a Pod that is accepted but never placed points at the scheduler.
A colleague runs kubectl get pv -n team-a and is surprised the namespace makes no difference to the output. Why does the flag change nothing?
PersistentVolume is a cluster-scoped resource, so it belongs to no namespace and -n is ignored. kubectl api-resources --namespaced=false lists everything that behaves this way.
You need to change a kube-apiserver flag on a control plane node where the API server runs as a static Pod. What actually makes the change take effect?
A static Pod is defined by a file on the node, not by an object in the API. The kubelet watches its staticPodPath, so saving the edited manifest is the whole procedure.
You have one kubeconfig holding three clusters. Every kubectl command keeps hitting the wrong one. Which command changes the target for subsequent commands?
A context ties a cluster, a user and a default namespace together. kubectl config use-context switches which one is current, and kubectl config get-contexts shows the choices.
A Pod create request is rejected with a message from a LimitRanger. At which stage of the API server request path did that happen, and what had already succeeded?
Every request runs authentication, then authorization, then admission control, then validation and storage. An admission rejection means the caller was already known and already permitted.
You delete a Pod that a Deployment created. Seconds later a Pod with a similar name is running again. What is doing that, and how do you actually stop it?
A ReplicaSet controller compares desired replicas with what exists and creates a replacement. To stop it you change the desired state, by scaling to zero or deleting the Deployment.
A team wants to record a build URL and a git commit on every Deployment, and also to select those Deployments by team name. Which field takes which piece of data?
Labels are identifying data that selectors query, with a restricted value format. Annotations hold arbitrary non-identifying metadata that nothing selects on.
A manifest is already applied to the cluster. Someone edits one field in the file and reruns the command. Which behaviour separates kubectl apply from kubectl create here?
create is imperative and fails when the object exists. apply is declarative: it creates the object the first time and merges changes into it on every run afterwards.
You are asked to back up the cluster state of a cluster where etcd runs as a static Pod on the control plane node. Which approach produces a restorable backup?
Take an etcdctl snapshot save against the etcd endpoint, passing the CA certificate, client certificate and key that etcd is configured with. That one file is the cluster state.
A cluster must be rolled back to an etcd snapshot taken this morning. The control plane is still running. What does the documented restore procedure require?
Stop every API server first, restore the snapshot into a fresh data directory, then start the API servers again. Restoring under a live API server is explicitly warned against.
A team is planning a highly available control plane and must choose between the stacked etcd topology and the external etcd topology. Which statement correctly describes the trade-off?
Stacked etcd runs an etcd member on each control plane node, so losing a node loses both an API server and an etcd member. External etcd separates the two at the cost of twice the hosts.
You need to grant a user permission to list nodes. Which object must the permission be defined in, and why?
Nodes are cluster-scoped, so permission on them can only be expressed in a ClusterRole, granted through a ClusterRoleBinding. A Role covers namespaced resources only.
True or false: you can stop a user who has been granted cluster-wide pod read access from reading Pods in one sensitive namespace by adding a Role in that namespace that denies it.
False. RBAC permissions are purely additive and there are no deny rules. To take access away you have to change or remove the binding that granted it.
A ClusterRole named pod-reader grants get, list and watch on pods. You create a RoleBinding in the dev namespace that binds this ClusterRole to user alice. What can alice do?
A RoleBinding that references a ClusterRole grants those permissions only inside the RoleBinding's own namespace. The same ClusterRole reused this way is how one definition serves many namespaces.
A configuration has both a required_providers entry for aws and a provider "aws" block setting a region. A reviewer asks whether one of them is redundant. What does each one do?
required_providers declares which providers the module needs, with their source address and version constraint. A provider block configures one of them, with settings such as region.
An application Pod needs to call the Kubernetes API using a ServiceAccount named deployer. How does the Pod get a token in a current cluster?
Setting serviceAccountName on the Pod is enough. The kubelet requests a short-lived, automatically rotated token through the TokenRequest API and mounts it as a projected volume.
A team wants to add a new resource type, backups.example.com, that users manage with kubectl. They have no appetite for running another API server. Which extension mechanism fits, and what do they still need?
A CustomResourceDefinition adds the type and gets storage, validation and kubectl support for free. It does nothing on its own, so a controller is still needed to act on the objects.
You must deploy one cluster component to three environments where only a few values differ, using tooling already available with kubectl and with no templating language to learn. Which approach fits best?
Kustomize is built into kubectl and patches plain YAML through overlays, so no templating is involved. Helm is the right tool when you want packaging, release history and rollbacks.
Pods on a new cluster stay in ContainerCreating and the kubelet logs mention that the network plugin is not ready. Which extension interface is involved?
CNI is the container network interface, and Kubernetes ships no default implementation. Until a CNI plugin is installed and healthy, Pods cannot be given network namespaces and stay in ContainerCreating.
A worker node needs a kernel patch and reboot. You run kubectl drain node-3 and it fails with an error about DaemonSet-managed Pods. Which command completes the drain as documented?
kubectl drain refuses to proceed while DaemonSet Pods are present, so pass --ignore-daemonsets. It cannot evict them usefully anyway, since the DaemonSet controller recreates them at once.
A Deployment with 4 replicas and no rollingUpdate settings is updated to a new image. What does Kubernetes guarantee about Pod counts during the rollout?
maxUnavailable and maxSurge both default to 25%. With 4 replicas that allows one Pod unavailable and one extra Pod, so the count stays between 3 and 5.
kubectl get all in a namespace shows one Deployment, two ReplicaSets and three Pods. Only one ReplicaSet has replicas. What does the second one represent?
A Deployment creates a new ReplicaSet for each revision of its Pod template. Old ReplicaSets are scaled to zero and kept, which is what makes a rollback possible.
A bad image was rolled out an hour ago and several deployments have happened since. kubectl rollout history shows revisions but the CHANGE-CAUSE column is empty. Which statements are true? Choose TWO.
Rollback works because old ReplicaSets are kept, ten by default. CHANGE-CAUSE comes from the kubernetes.io/change-cause annotation, which nothing sets for you any more.
A Job must process exactly 12 work items, running at most 4 Pods at a time. Which fields express that, and what does the Job controller do as Pods finish?
completions is how many Pods must succeed, and parallelism is how many may run at once. Set 12 and 4, and the controller keeps starting Pods until 12 have succeeded.
A ConfigMap is consumed by a Deployment in two ways: one key as an environment variable, another mounted as a file. You edit the ConfigMap. What happens inside the running Pods?
A mounted ConfigMap key is eventually updated by the kubelet. An environment variable is fixed when the container starts and needs a restart to change.
A Deployment exists and someone tries to change spec.selector.matchLabels to a new value. The apply is rejected. What is the rule, and what does it force you to do?
A Deployment's label selector is immutable after creation in apps/v1. Changing it means deleting and recreating the Deployment, optionally with --cascade=orphan to keep the Pods up.
A developer says Secrets are safe to commit to Git because Kubernetes encrypts them. What is the accurate correction?
Secret data is base64 encoded, not encrypted. Anyone with API or etcd access can read it, so protecting Secrets means RBAC plus encryption at rest, which is off by default.
A CronJob runs every minute, but the job sometimes takes three minutes. Runs are piling up on top of each other. Which field stops that, and what are the choices?
concurrencyPolicy decides what happens when a run is due while the previous one is still going. Allow is the default, Forbid skips the new run, and Replace kills the old one and starts fresh.
A Pod has one container with memory request 256Mi, memory limit 512Mi, and a CPU request but no CPU limit. Which QoS class does it get, and what does that mean under node memory pressure?
Requests that do not equal limits make the Pod Burstable. Under node pressure the kubelet evicts BestEffort Pods first, then Burstable, and Guaranteed last.
A Job manifest sets restartPolicy: Always in its Pod template and is rejected. What are the valid values, and why is that one refused here?
restartPolicy is Always, OnFailure or Never, and it applies to containers inside a Pod. A Job allows only OnFailure or Never, because Always would mean the Pod could never complete.
A HorizontalPodAutoscaler targeting 70% average CPU never scales, and kubectl describe hpa shows unknown for the current metric. Which TWO conditions would cause this?
CPU-based autoscaling needs a metrics source serving the Metrics API, normally metrics-server, and it needs CPU requests on the Pods' containers, since utilisation is a percentage of the request.
Six replicas keep landing unevenly across three zones, and losing one zone takes most of them out. Which mechanism enforces an even spread, and what does maxSkew mean?
topologySpreadConstraints spread Pods across a topology key such as zone. maxSkew is the largest allowed difference between domains, and whenUnsatisfiable decides whether that is a hard rule.
GPU nodes carry the taint gpu=true:NoSchedule and the label gpu=true. You add a matching toleration to a training Pod. Where can that Pod now be scheduled?
A toleration only removes an objection. It does not attract a Pod to the tainted nodes, so a nodeSelector or node affinity is still needed to keep the Pod on the GPU nodes.
kubectl drain is taking a node out of service and appears to hang, evicting nothing further. A PodDisruptionBudget on the affected app sets minAvailable: 3 and exactly 3 Pods are ready. What is happening?
drain uses the Eviction API, which respects PodDisruptionBudgets. With minAvailable already at the limit, no further eviction is allowed until a replacement Pod becomes ready somewhere else.
A namespace has a ResourceQuota with requests.cpu and limits.memory set. A developer's Pod is rejected with a 403 saying it must specify resource limits. What is the cleanest fix for every future Pod in that namespace?
When a quota covers a compute resource, every Pod must specify that request or limit. A LimitRange in the namespace supplies defaults, so Pods that omit them are still admitted.
A log-shipping container must start before the application container and keep running for the life of the Pod. How is that expressed in the Pod spec?
A sidecar is an entry in initContainers with its own restartPolicy set to Always. It starts in init order, then keeps running while the application containers start.
A monitoring agent must run exactly once on every node, including nodes added to the cluster next week. Which workload object fits, and why not the alternative?
A DaemonSet places one Pod per eligible node and covers new nodes automatically. A Deployment counts replicas without caring which nodes they land on.
A configuration repeats the expression "${var.project}-${var.env}" in eleven resource names. The team wants to write it once. Should that be a variable or a local value?
A local value names an expression for reuse inside one module and cannot be set from outside. An input variable is a value the caller supplies, so it cannot be computed from other variables.
A subnet resource needs the id of a VPC declared in the same configuration as aws_vpc.main. Which expression provides it, and what side effect does writing it have?
Write aws_vpc.main.id to read the attribute. The reference also creates an implicit dependency, so Terraform creates the VPC before the subnet without any depends_on.
An input variable for an environment name must only ever be dev, staging or prod, and a wrong value should fail immediately with a clear message. Which feature does this?
A validation block inside the variable declaration checks the value with a condition and reports error_message when it fails. It runs before planning, so a bad value never reaches a provider.
An internal API must be reachable by other Pods in the cluster but must not be exposed outside it. Which Service type meets that, and how do the other types relate to it?
ClusterIP is the default and is reachable only from inside the cluster. NodePort and LoadBalancer build on top of it by adding external entry points.
A container listens on 8080. Clients inside the cluster must reach the Service on 80. Which combination of port and targetPort is correct?
port is the port clients use on the Service. targetPort is the port on the Pod that traffic is forwarded to. Here that means port 80 and targetPort 8080.
A Pod in the frontend namespace must reach a Service named api in the backend namespace. Which name resolves, and what is the fully qualified form?
A Service is reachable as api.backend from another namespace, and the fully qualified name is api.backend.svc.cluster.local. A bare api only resolves inside the Service's own namespace.
Two Pods on different nodes talk to each other directly by Pod IP with no port mapping anywhere. Which rule of the Kubernetes network model makes that work?
Every Pod gets its own cluster-wide IP, and Pods reach each other on that IP without NAT. The network plugin is what has to deliver that, not Kubernetes itself.
A StatefulSet's Pods must each be addressable individually so peers can form a cluster. Which Service configuration provides that, and what does DNS return?
A headless Service, clusterIP: None, allocates no virtual IP. DNS returns the Pod IPs directly, and each StatefulSet Pod also gets its own name under the Service.
A cluster must expose two applications on one external hostname, routed by URL path. Which object does the path-based routing, and what is still required underneath it?
An Ingress routes HTTP by host and path, which a Service cannot do. It still forwards to ClusterIP Services, and it needs an Ingress controller to be running.
Requests to a ClusterIP Service time out. kubectl get endpointslices shows no endpoints for it, though the Pods are Running. What are the TWO most likely causes?
A Service with no endpoints means nothing matched or nothing is ready. Check that the selector matches the Pod labels, and that the Pods pass their readiness probes.
An Ingress rule is written with path /api and no pathType field. The manifest is rejected. What does pathType control, and which value matches /api and /api/v1 alike?
pathType is required on every path. Prefix matches element by element, so /api also matches /api/v1. Exact matches the whole path only, and ImplementationSpecific leaves it to the controller.
A Service of type NodePort is created without specifying a node port. Which port is assigned, and what is the constraint if you want to pick one yourself?
Kubernetes allocates a port from the range set by the API server's --service-node-port-range flag, 30000-32767 by default. A port you choose by hand must be inside that range and free.
After applying a default-deny egress NetworkPolicy, Pods in the namespace cannot reach anything by name, though they can still reach IP addresses you allowed. Why?
A default-deny egress policy also blocks the UDP and TCP port 53 traffic that Pods send to CoreDNS, so every name lookup fails. DNS has to be allowed back explicitly.
An application needs the real client IP address. The Service is type LoadBalancer and currently reports the node IP as the source. Setting externalTrafficPolicy: Local fixes it. What is the cost?
Local preserves the client source IP by refusing to forward between nodes. A node with no local ready endpoint stops serving that Service, so traffic can be dropped and load can spread unevenly.
A Service must send cluster traffic to a database running on a VM outside the cluster, at a fixed IP. Which approach keeps the in-cluster name and works with a raw IP address?
A Service with no selector gets no endpoints automatically, so you create an EndpointSlice yourself pointing at the external address. ExternalName cannot be used with a bare IP.
You are asked what actually makes a ClusterIP reachable from a Pod, given that the cluster IP is not assigned to any network interface. What is the accurate explanation?
A cluster IP is a virtual address. kube-proxy programs packet forwarding rules on every node, in iptables, IPVS or nftables mode, which rewrite traffic for that IP to a chosen endpoint.
A child module declares variable "instance_type" with no default. The caller's module block omits it. What happens, and how should the caller supply the value?
A variable with no default is required, so Terraform errors out naming the missing argument. The caller sets it as an argument inside the module block, alongside source.
You apply a NetworkPolicy that should block all ingress to a namespace, but every Pod stays reachable. kubectl get networkpolicy shows the object exists. What is the most likely explanation?
NetworkPolicies are enforced by the network plugin. If the cluster's CNI plugin does not implement them, the objects are accepted by the API server and have no effect.
Write the smallest NetworkPolicy that denies all ingress traffic to every Pod in a namespace. Which spec achieves it?
An empty podSelector selects every Pod in the namespace, and policyTypes: Ingress with no ingress rules allows nothing. That combination is the documented default-deny policy.
A NetworkPolicy ingress rule has one from entry containing both a namespaceSelector (user=alice) and a podSelector (role=client). Which traffic is allowed?
Selectors inside one from entry are combined with AND. Splitting them into two entries in the from list makes it OR, and that single dash is the whole difference.
An Ingress resource for app.example.com has been applied and the object exists, but nothing responds and the ADDRESS column stays empty. What is missing?
An Ingress is only configuration. An Ingress controller has to be running to satisfy it, and the Ingress usually needs an ingressClassName telling that controller to pick it up.
A platform team owns the cluster's shared entry point and application teams own their own routes. Which Gateway API objects match that split?
GatewayClass describes an implementation, a Gateway is the entry point the platform team runs, and HTTPRoute is the routing an application team attaches to it. The kinds are modelled on those roles.
A shared Gateway lives in the infra namespace. A team applies an HTTPRoute in their own namespace referencing that Gateway, and it is not accepted. What is required?
A Gateway only accepts routes from its own namespace by default. Attaching a route from elsewhere requires the Gateway's listener to permit it through allowedRoutes.
Three Pods on three different nodes must all write to the same volume. The available storage class provisions block volumes that support ReadWriteOnce only. What does that mean?
ReadWriteOnce means read-write by a single node, so Pods on other nodes cannot mount it. Shared writes across nodes need ReadWriteMany, which usually means file storage rather than block.
An application team needs 20Gi of storage. A cluster with dynamic provisioning is available. Which object does the team write, and where does the other one come from?
The team writes a PersistentVolumeClaim asking for size and access mode. With dynamic provisioning the StorageClass creates the matching PersistentVolume automatically.
A PVC created from the default StorageClass is deleted. The team expected the data to survive and it is gone. What explains this, and how is it prevented next time?
A dynamically provisioned PV inherits its StorageClass's reclaim policy, which is Delete unless set otherwise. Retain keeps the PV and the backing storage after the claim goes away.
A Deployment with three replicas mounts a hostPath volume at /data. Replicas land on three different nodes and each sees different files. Why, and what is the deeper risk?
hostPath mounts a directory from whichever node the Pod runs on, so each node has its own copy. It also exposes the host filesystem, which is why it is discouraged outside single-node use.
A PVC with no storageClassName field is applied to a cluster and stays Pending. Which check explains it fastest?
A PVC that omits storageClassName uses the default StorageClass, marked by the storageclass.kubernetes.io/is-default-class annotation. With no default and no matching PV, the claim waits.
A team mounts a ConfigMap as a volume and asks whether they should set a reclaim policy and access mode on it. What is the accurate answer?
ConfigMap and Secret volumes are projections of API objects, not persistent storage. They have no PV, no claim, no reclaim policy, and they are always mounted read-only.
In a cluster spread over three availability zones, Pods regularly fail to start because their volume was created in a zone the Pod cannot be scheduled into. Which StorageClass setting fixes this?
volumeBindingMode: WaitForFirstConsumer delays provisioning until a Pod using the claim is scheduled, so the volume is created where the Pod actually landed.
A ConfigMap key is mounted with subPath so it lands beside files the container already has. The ConfigMap is edited, and unlike other mounts this file never changes. Why?
A container using a ConfigMap or Secret as a subPath mount does not receive updates. Only a whole-volume mount is refreshed by the kubelet, so a subPath mount needs a Pod restart.
A database PVC is nearly full and must grow from 20Gi to 100Gi with no data loss. What does Kubernetes require for that to work?
Edit the PVC's requested size. It only works if the claim's StorageClass has allowVolumeExpansion: true, and PVCs can be grown but never shrunk.
You delete a PVC and it sits in Terminating. A Pod is still using it. What is holding the deletion, and what happens when the Pod goes away?
Storage object in use protection adds a kubernetes.io/pvc-protection finalizer. The PVC stays in Terminating while a Pod uses it, and deletion completes once no Pod references it.
A StatefulSet with volumeClaimTemplates is deleted. Its PVCs are still present afterwards. Is that a bug, and how is the behaviour controlled?
Keeping the PVCs is the default and deliberate, so data survives a recreated StatefulSet. persistentVolumeClaimRetentionPolicy changes what happens on delete and on scale-down.
An admin creates a 50Gi PV with accessModes ReadWriteOnce and no storage class. A PVC asking for 10Gi ReadWriteOnce stays Pending. Which mismatch most likely explains it?
Binding matches on capacity, access mode and storage class. A claim that omits storageClassName asks for the default class, which a PV with no class cannot satisfy.
A container writes a cache to /scratch using an emptyDir volume. Which statement describes when that data is lost?
An emptyDir lives as long as the Pod on that node. It survives a container restart but is gone when the Pod is deleted or rescheduled elsewhere.
A Pod has been Pending for ten minutes. Which single command gives you the reason, and what should you expect to read in it?
kubectl describe pod shows the scheduler's FailedScheduling event, which names the reason node by node: insufficient CPU or memory, an untolerated taint, or no node matching the selector.
kubectl get pods shows a Pod with STATUS Running and READY 1/2, and requests to its Service are failing. What does that pair of columns tell you?
READY counts ready containers out of total containers in the Pod. At 1/2 the Pod is not ready, so it is left out of Service endpoints even though STATUS says Running.
A Pod is in ImagePullBackOff. Which TWO causes are consistent with that status?
ImagePullBackOff means the kubelet cannot fetch the image. A wrong name or tag and a missing registry credential are the two usual reasons, and the Pod's events name which one.
A Pod will not start and you need both the scheduling events and the exact resource requests as submitted. Which two commands give you those, and what does each leave out?
describe gives a human summary plus recent Events, which the YAML never contains. get -o yaml gives the exact stored object including defaults, which describe abbreviates.
A container is in CrashLoopBackOff and kubectl logs returns nothing useful because the container has just restarted. Which command shows the output from the failed run?
kubectl logs --previous returns the logs of the previous instantiation of the container, which is where the reason for the crash actually is.
A Pod is scheduled to a node but its status reads CreateContainerConfigError. The image pulled fine. What class of problem is this, and where do you look?
The kubelet could not assemble the container's configuration, usually because a referenced ConfigMap, Secret or key does not exist. The Pod events name the missing object.
kubectl describe pod shows Last State: Terminated, Reason: OOMKilled, Exit Code: 137, and a restart count climbing. What does that tell you, and what does not follow from it?
The container was killed for exceeding its memory limit, or the node ran out of memory. It is a container-level kill, so the Pod stays and the container restarts in place.
You need a shell alongside a running Pod to test connectivity, but the image is distroless and kubectl exec fails because there is no shell in it. What is the intended approach?
kubectl debug adds an ephemeral container to the running Pod, sharing its network namespace. That gives you a shell and tooling without rebuilding the image or restarting the Pod.
An application takes about 90 seconds to warm up. Its liveness probe uses the defaults with no initialDelaySeconds, and the Pod restarts continuously. What is the best fix?
The liveness probe is failing during startup and killing the container before it is ready. A startupProbe handles slow starts properly, holding the liveness probe off until startup succeeds.
A namespace holds forty Pods with status Evicted and no containers running. Replacement Pods are healthy on other nodes. What do these objects represent?
Evicted Pods are terminated Pod objects the kubelet left behind as a record of node pressure. They consume no resources, and deleting them is safe once you have read why they were evicted.
During a rolling update, some requests fail with connection errors even though every Pod eventually becomes healthy. Which mechanism prevents this, and how?
A readiness probe keeps a Pod out of a Service's endpoints until it can serve. Without one, a Pod receives traffic as soon as its container starts.
You need to check whether the API server is healthy and which of its internal checks is failing. Which approach reflects current practice?
The API server exposes livez and readyz, and healthz is deprecated. Adding verbose lists every individual check, which is how you find the one that is failing.
A worker node shows NotReady and its Pods are being replaced elsewhere. kubectl describe node reports the kubelet has stopped posting status. What do you check on the node itself?
The kubelet is a systemd service on the node. Check systemctl status kubelet and its logs with journalctl -u kubelet, which name the real failure.
After editing the kube-apiserver static Pod manifest, kubectl fails with a connection refused error. Which approach diagnoses this?
With the API server down, kubectl is useless. Use crictl on the control plane node to find the container and read its logs, and check the kubelet journal for manifest errors.
A Pod writes a large volume of logs. kubectl logs returns only recent output, and the earlier lines you need are missing. Why, and what does that imply?
The kubelet rotates container logs and kubectl logs only reads the latest file. Keeping history means shipping logs off the node to a cluster-level logging system.
You are asked to find out what happened in a namespace over the last few minutes. Which command gives the most useful ordered picture, and what limitation should you expect?
kubectl get events --sort-by=.lastTimestamp gives a namespace timeline. Events are namespaced and short-lived, retained for one hour by default, so older history is simply gone.
kubectl top nodes fails with an error saying the metrics API is not available, though every node is Ready and workloads are healthy. What does that indicate?
kubectl top reads the Metrics API, which is served by metrics-server rather than by the API server itself. Without it installed there is nothing to answer, and resource usage has to come from elsewhere.
Pods on one node show status Evicted and the node reports the DiskPressure condition. Which explanation is correct?
The kubelet evicts Pods when a node-level resource crosses an eviction threshold. DiskPressure comes from node or image filesystem thresholds, whose defaults are 10% and 15% available.
A Pod cannot resolve any Service name. Other Pods in the cluster resolve names normally. Which check comes first?
With other Pods resolving fine, CoreDNS is healthy, so look at this Pod. Its /etc/resolv.conf and its dnsPolicy decide which resolver it uses at all.
A Pod has been Terminating for fifteen minutes after a delete. What is the correct sequence of things to consider?
Check whether the process is ignoring SIGTERM within its grace period, whether the node is unreachable, and whether a finalizer is holding the object. Force deletion is a last resort.
On a node using containerd, you need to list containers and read one container's logs without going through the API server. Which tool is intended for this?
crictl is the CRI-compatible command line interface for inspecting containers on a node. It talks to the runtime directly, so it works when the control plane does not.
An application using the ServiceAccount ci in the build namespace gets a 403 listing Pods. Which command confirms the permission gap without deploying anything?
kubectl auth can-i with --as impersonates the identity and answers yes or no against the real authorization layer, so you can test a ServiceAccount's access from your own session.
A Pod that was Running is suddenly gone, and events show the scheduler removed it to make room for a higher-priority Pod. Which mechanism is this, and what protects against it?
This is preemption, driven by PriorityClass. The scheduler removes lower priority Pods so a pending higher priority Pod can be scheduled, and preemptionPolicy: Never opts a Pod out of preempting others.
On a fresh install, kubectl get nodes fails with a message about the connection to the server localhost:8080 being refused. What does that specific message mean?
localhost:8080 is kubectl's fallback when it finds no kubeconfig. The fix is to point it at a real one, usually by copying admin.conf into ~/.kube/config.
A Pod's status is Init:0/1 and it has stayed that way for several minutes. Where is the problem, and how do you read its output?
An init container has not completed, and the app containers cannot start until it does. Read its logs by naming it with kubectl logs -c.