Field notes from altitude.

Every team I audit has retry logic. Almost none of it is correct. Naive retries without backoff, jitter, and circuit breakers turn a small degradation into a cascading failure. I have watched a single slow dependency take down an entire platform because every service retried at the same time. Here is what your retry logic actually does, the four patterns I install in every codebase, and why most retry libraries give you a false sense of safety.

I have sat in on enough due diligence calls to know that investors ask about infrastructure before they ask about the product roadmap. Most founders fumble it. Not because their infrastructure is bad, but because they do not know what it actually is. Here are the questions that come up every time, the answers that build confidence, and the answers that kill the deal.

Every team I audit has a SIGTERM handler in their application. Almost none of them work correctly under real termination conditions. I have watched pods get killed mid-request, load balancers send traffic to pods that are already shutting down, and databases lose in-flight transactions because the application stopped listening before the connections drained. The gap between 'we handle SIGTERM' and 'we shut down gracefully' is where production incidents live. Here is what your shutdown sequence actually does, what Kubernetes does behind it, and the configuration I install for every service I put in production.

Helm solved a real problem in 2016. In 2026 it creates more problems than it solves for most teams. After four years of maintaining Helm charts for startups across Europe, I ripped it out and replaced it with Kustomize overlays and plain Kubernetes manifests. Here is why, what changed, and the specific failure modes that made me stop trusting Helm as the default.

Every startup with a Kubernetes cluster and ambition eventually wants canary deployments. They read about progressive delivery, install Argo Rollouts or Flagger, and spend weeks wiring up metric analysis and traffic shifting. Then the first real incident happens and they discover they cannot roll back cleanly, the canary never would have caught it, and the problem was never about traffic percentage. It was about the fact that their database migration was irreversible. Here is why canary deployments are the last 10% of deployment maturity, not the first, and what to build instead.

A Series A startup in Munich deployed to production by SSHing into a server and running git pull. It worked for eighteen months. Then it did not. This is the postmortem of the incident that took their platform down for six hours, what manual deployments actually cost you, and why the transition to CI/CD is not about speed but about predictability.

Every few months I get called into a startup that ran their production database on a Kubernetes StatefulSet and lost data. Not hypothetically. Actually lost data, or lost availability for hours during an incident that a managed database would have survived automatically. Kubernetes is the best orchestration platform we have for stateless workloads. It is the worst place I know to run a stateful one. Here is why, with the specific failure modes I have watched happen in production, what StatefulSets actually guarantee versus what people think they guarantee, and where your database should run instead.

Most startups do not have a cloud cost problem. They have a cloud waste problem. The difference matters. Here is what I find when I audit a startup's AWS or GCP bill in the first 90 days, what it actually costs, and what to fix before you hire anyone or switch providers.

Every Kubernetes cluster I audit has CPU limits set on every pod. Almost none of them need them. CPU limits do not protect the node. They throttle your application the moment it needs a burst, and CFS bandwidth throttling does not care that your latency just doubled. Here is what CPU limits actually do, why the default advice to set them is wrong, and what I install instead.

Founders tell me they will set up CI/CD after they get traction. They never do. The first time a bad deploy takes the MVP offline on launch day, the investors watching the demo stop returning emails. I have watched this happen three times this year. Here is why CI/CD is not a scaling concern, it is a launch prerequisite, and what the minimum pipeline for a pre-seed MVP actually looks like.

Every team I audit has liveness and readiness probes configured. Almost none of them are configured correctly. I have seen probes that return 200 while the application is fully broken, probes that cause cascading restarts during traffic spikes, and probes that make outages worse by masking the real problem. Here is what your health checks actually do, what they should do, and the specific configurations I install for every team I work with.

Most teams run terraform plan before a deploy and trust the output. They never run it between deploys, which means they never notice when someone changed something in the console at 3am and forgot to tell anyone. Infrastructure drift is the silent failure that does not trigger an alert, does not page anyone, and does not show up until a plan fails in production at the worst possible moment. I have watched it take down a payment gateway for six hours. Here is what drift looks like, how it accumulates, and how to detect it before it detects you.

Every team has a staging environment they trust until production breaks in a way staging could never predict. The gap between staging and production is not a config difference. It is a structural lie, and I have watched it burn startups at the worst possible moment. Here is what the gap looks like, why it exists, and how to close it without doubling your cloud bill.

I have done security audits for fourteen startups in their first year of life. The same five mistakes show up every time. None of them require a security team to fix. They require an engineer who knows what the baseline looks like and has the discipline to enforce it before the first customer asks for a SOC 2.

Kubernetes Secrets are base64-encoded strings in etcd. Anyone with cluster read access can decode them. I have audited fifteen clusters and found database passwords in Slack, in CI logs, and in plain text ConfigMaps. Here is what 'secret' actually means in Kubernetes, why the default setup is worse than you think, and the three patterns that fix it without standing up a Vault cluster you cannot maintain.

Every startup I have worked with that hit a wall at scale had the same root cause: they could not see what was happening inside their own system. Not because they lacked tools. Because they had dashboards full of green lines and no idea what any of them meant when things broke. Here is what observability actually costs to build, what it costs to skip, and the specific things I instrument first when I walk into a team that is flying blind.

Most teams deploy database migrations the same way they deploy code: run the migration, hope nothing breaks, fix it at 2am when it does. There is a better way. I have run schema migrations on production databases for fifteen companies, and the pattern that works is not clever. It is patient, it is boring, and it takes three deploys instead of one. Here is exactly how it works, with the SQL, the failure modes, and the moment you should never use it.

Founders ask me this question every week. The honest answer is not a number. It is three numbers: what it costs if you do it yourself, what it costs if you hire someone, and what it costs if you do not build it at all. I have built enough of these to know which number matters.

Most teams have backups. Most teams cannot recover from them. I have run disaster recovery exercises for fifteen companies, and the gap between having a backup and having a recovery is the most expensive illusion in infrastructure. Here is what a real DR plan looks like, with the configs, the timings, and the failures I have seen.

Every funded startup reaches the moment where someone says 'we need to hire a DevOps engineer.' Most of them hire too early, hire the wrong person, or hire when they do not need to hire at all. Here is how I tell founders to think about it.

You raised seed money. Now someone expects you to ship reliably, answer investor questions about uptime, and not lose customer data at 2am. Here is the actual stack I install for seed-stage startups, what each piece costs, and what I deliberately leave out.

Teams adopt ArgoCD or Flux, call it GitOps, and keep deploying the old way. The pull request is the contract. The tool is just the enforcement mechanism. What changes when you take the contract seriously.

Postmortems are supposed to be blameless, honest, and actionable. Most of them are none of those things. What I find when I audit postmortem culture, and how to fix it without making it feel like homework.

SOC 2, ISO 27001, GDPR readiness — startups collect certifications like badges, spending months and tens of thousands on audits that do not make them more secure. Here is what actually does.

Not a hype list. Not a vendor pitch. These are the tools I wire together for clients when they want agents in their pipelines, their incident response, and their code review. What each one does, where it breaks, and when I would not use it.

Every engineering team with a Kubernetes cluster and ambition eventually builds an internal developer platform. Most of them build the wrong thing. Here is how to tell if you are building a platform or a monument to yourself.

The tools for on-call are solved. The culture around them is not. What I learned fixing incident response for teams that had PagerDuty, runbooks, and still burned out their best engineers.

Investors do technical diligence. They look at your deploy process before they look at your product. I have sat on both sides of that call. Here is what they actually ask, what they hear when you answer, and the deploy process that passes diligence without you having to lie about it.

After running Coolify in production for months, here is what it actually does well, where it breaks, and why I recommend it to startups who are not ready for Kubernetes but have outgrown Docker Compose.

When your office is in Leh, Ladakh, and your clusters are in Frankfurt and Singapore, every architectural decision is a latency decision. What I learned about infrastructure from operating at altitude.

Certifications open doors. They don't fix clusters at 3am. The gap between the exam and production is wider than most candidates realise, and the industry is quietly hiring around it.

Pre-product-market-fit, a €20 server with Docker Compose beats a managed container platform on cost, speed and sleep. The real numbers — and the honest moment to switch.

The fastest-adopted agent framework in open source runs on hardware you own, speaks through your messaging apps, and holds your credentials. What its rise teaches operators about running agents safely.

On-call, code review, runbooks, postmortems — every ritual of DevOps is being renegotiated as AI agents take their place in the loop. What survives, what inverts, and what becomes more important than ever.

A CNCF sandbox project treats agents as cluster citizens — declared in CRDs, reconciled by a controller, governed by the RBAC and GitOps you already trust. Why that pattern matters more than the project.

Terraform taught us to declare machines instead of building them by hand. The next step is infrastructure that is operated by agents — and agents that are themselves declared, reviewed and promoted like any other artifact.

Why the best deploy systems feel slower than they are — and faster than they need to be.

What happens when you stop treating SLOs as numbers and start treating them as agreements.