What infrastructure actually costs and how to bring it down.
3 items at advanced level · all topics
When would you scale vertically instead of horizontally?
Go vertical when the bottleneck doesn't split: database buffer cache, cache memory, network bandwidth on one node, a single-threaded lock. Go horizontal when the work partitions cleanly. The real question isn't which is better, it's whether the thing that's saturated can be divided across machines at all.
Your 5 GB monorepo has code, binaries, Terraform and Helm charts, and clones are slow. What do you do?
Measure what the 5 GB actually is: big blobs, deep history, and a wide tree are three different problems with three different fixes. Work through them in order of cost, artifacts out of Git, binaries into LFS, partial clone and sparse checkout for developers, and treat splitting the repo as the last resort rather than the first idea.
Your LLM feature costs $40k/month and is growing 30% monthly. Leadership wants it cut by 70% without hurting quality. What do you do?
Most LLM spend goes on tokens that never needed generating, requests that never needed a frontier model, and identical prefixes reprocessed on every call. Measure where the money actually goes first, then attack caching, routing and prompt size before anything as drastic as self-hosting.