Pipelines, deployment strategies and release safety.
33 items · all topics
Compare blue-green, canary, and rolling deployments. When would you choose each?
Rolling replaces instances gradually, blue-green switches all traffic between two full environments at once, and canary sends a small slice of real traffic to the new version while watching metrics. They differ mainly in rollback speed, cost, and how much confidence you gain before full exposure.
How do you identify the exact deployment that introduced an issue?
Put the deployment log and the metrics timeline side by side. A regression that started fifteen minutes ago was almost certainly caused by something that shipped in that window. Confirm which commit is actually running from the image digest, then roll back and watch whether the metrics recover, that is your proof.
Code works on the developer's machine but fails in Jenkins. How could Git be involved?
Stop guessing about the code and compare commit SHAs first. CI often builds a different commit than the developer tested: a merge commit, a stale workspace, a shallow clone, missing submodules, or a file that's gitignored locally but needed at build time. Only once the SHAs match is it worth looking at the code.
A developer started a feature branch from an outdated main. How would you fix it?
Fetch, then rebase the branch onto origin/main so the work replays on current code. Rebase is right while the branch is private; if it's been pushed and others use it, merge main in instead so you don't rewrite shared SHAs.
A Git repository has become huge and cloning takes 20 minutes. What would you investigate?
Measure before you guess. Find the biggest objects in history, because size usually comes from binaries and build artifacts committed long ago. Deleting them today does nothing: the old blobs stay in history until you rewrite it or route developers around downloading them in the first place.
The deploy says it shipped main, but production doesn't have the latest commit. How do you debug it?
Walk the chain, commit, CI checkout, build, artifact, deploy, running pod, and compare the SHA at each step. main is a moving pointer, so the usual cause is that something in the chain resolved it at a different moment, or shipped a cached artifact instead of a fresh one.
Your CI uses git clone --depth=1 and a deploy script that needs history suddenly fails. Why?
A shallow clone downloads the current tree and exactly one commit, no parents, usually no tags. Anything that reads history breaks: `git describe`, changelogs, `git diff HEAD~10`, commit counts, `merge-base`. Fetch the depth you actually need instead of defaulting to depth 1 everywhere.
A developer wants to git reset --hard and force-push a shared branch to undo a bad commit. Do you allow it?
On a shared branch, no. Use git revert, which undoes the change with a new commit and leaves history intact. Reset plus force-push rewrites history that other people, CI and deployment records already depend on.
A bad feature was merged into production. How would you undo it?
Roll back the running deployment first, that's faster than any Git fix, then fix Git properly. Reverting a merge needs git revert -m 1 <merge-commit>, and the catch nobody mentions upfront is that you have to revert the revert later or the feature will never merge back in.
You need one bug fix from a branch that contains 20 other commits. What do you do?
Cherry-pick copies just that one commit onto your branch. It's the right tool for hotfixes and release backports, but it duplicates the change, so use it deliberately and keep fixes isolated in their own commits in the first place so they're easy to lift out later.
Your CI pipeline runs twice for every pull request. How do you investigate?
Almost always two triggers firing on one action, usually push and pull_request both matching the same branch. Read the event that started each run, then make the trigger config deliberate instead of deleting jobs until the noise stops.
Your deployment system needs to know exactly which Git commit is running in production. How do you design that?
Stamp the commit SHA into the artifact at build time and expose it at runtime. Build once per commit, tag and deploy by digest, and serve a /version endpoint, so the answer to "what's live?" comes from the running process itself, not from a pipeline log someone has to go dig up.
A deploy of commit A is still running when commit B lands on main. What can go wrong?
If the pipeline resolves `main` at each step instead of pinning one commit early, later stages can pick up B while earlier ones tested A. You get mixed versions, out-of-order deploys, and a rollback target that no longer means anything. The fix is pinning the SHA once plus serializing production deploys.
Your 5 GB monorepo has code, binaries, Terraform and Helm charts, and clones are slow. What do you do?
Measure what the 5 GB actually is: big blobs, deep history, and a wide tree are three different problems with three different fixes. Work through them in order of cost, artifacts out of Git, binaries into LFS, partial clone and sparse checkout for developers, and treat splitting the repo as the last resort rather than the first idea.
Your organization wants signed commits for production code. How would you implement it?
git config user.email is free text, not identity, so signing is what turns authorship into a cryptographic claim. Roll out SSH signing, enforce it with branch protection and a trusted-key list, and plan for the parts that actually break rollouts: bots, squash merges, and key rotation.
Two CI pipelines try to create the same Git release tag at the same time. What happens?
The remote accepts one push and rejects the other, because ref updates are atomic. The danger isn't the collision, it's a pipeline that "fixes" the rejection with --force, which silently moves an existing release tag onto a different commit. Serialize releases and protect tags server-side.
A critical production bug needs an emergency fix, but your PR process takes two hours. What do you do?
Mitigate first: rollback or a feature flag beats writing code under pressure. If code is genuinely needed, use a documented hotfix lane: branch from the production tag, minimal fix, fast tests, one reviewer, deploy, then merge back. Emergency means a faster controlled process, not no process.
You use GitOps and someone changes Kubernetes manually. Git still has the old config. What happens?
You get configuration drift: Git says one thing, the cluster says another. What happens next depends on whether the controller self-heals or just reports it. Either way the manual change is temporary, and the fix is to put the intended change into Git, not to argue with the controller.
Someone moved a production Git tag to a different commit. Why is that dangerous?
A release tag is a promise that a version name means one exact, unchanging set of code. Moving it breaks that promise everywhere at once: rollbacks, audits, incident timelines, and any pipeline that deploys by tag now point somewhere different from what people believe, and some clones won't even notice the change.
Your repository has hundreds of branches, many already merged into main. What would you change?
Deleting merged branches is easy and worth automating, but it's the symptom. The real disease is branches living long enough to diverge, which shows up as painful merges and untested integration risk. Fix PR size and review speed first, and watch for squash-merged branches that `--merged` can't even see.
Your organization wants production deployments to be reproducible six months later. How does Git help?
Git pins the source exactly and that's all it pins. Real reproducibility also needs locked dependencies, versioned infrastructure, immutable artifacts, and a stored record linking a deployment to its digest, plus keeping the actual artifact, because rebuilding it later is rarely byte-identical.
Deployment says SUCCESS but production is running an older commit. You have 10 minutes. What do you check?
Ask the running process what it actually is, then walk backwards through the chain until the SHA stops matching. A green pipeline only proves each step exited zero, not that anything actually changed in production, and "unchanged" is a success message that means nothing happened at all.
An attacker steals a developer's Git credentials and pushes malicious code. How do you defend against this?
Assume one credential will eventually be stolen and design so that alone isn't enough to ship code. Layer identity, branch protection, review, signing, pipeline isolation and detection, and remember the attacker's real target is usually the CI workflow, not the application code itself.
Nobody knows which Git branch or tag corresponds to production. How would you fix the release process?
Establish one unbroken chain of identity from commit to running process, then make the pipeline the only way anything reaches production. Start by discovering what's actually deployed today, you can't design a release process around a system you can't describe.
A Deployment with 4 replicas and no rollingUpdate settings is updated to a new image. What does Kubernetes guarantee about Pod counts during the rollout?
maxUnavailable and maxSurge both default to 25%. With 4 replicas that allows one Pod unavailable and one extra Pod, so the count stays between 3 and 5.
A bad image was rolled out an hour ago and several deployments have happened since. kubectl rollout history shows revisions but the CHANGE-CAUSE column is empty. Which statements are true? Choose TWO.
Rollback works because old ReplicaSets are kept, ten by default. CHANGE-CAUSE comes from the kubernetes.io/change-cause annotation, which nothing sets for you any more.
A monitoring agent must run exactly once on every node, including nodes added to the cluster next week. Which workload object fits, and why not the alternative?
A DaemonSet places one Pod per eligible node and covers new nodes automatically. A Deployment counts replicas without caring which nodes they land on.