GitOps is not a tool choice, it is a contract

A team I worked with had ArgoCD installed, a Git repository with Kubernetes manifests, and a CI pipeline that built and pushed images. They called it GitOps. On paper, they were doing everything right. In practice, the cluster was full of manual changes that nobody had reviewed, nobody could trace, and nobody could reproduce.

The engineer who had made the manual changes was not being reckless. There was an incident, the deployment pipeline was slow, and kubectl apply seemed faster than waiting for ArgoCD to sync. The fix worked. The incident ended. And the cluster state drifted from what was in Git. The next time ArgoCD synced, it reverted the manual fix and the incident came back.

This is not a tool problem. It is a contract problem.

What GitOps actually is

GitOps is not ArgoCD. GitOps is not Flux. GitOps is not having your Kubernetes manifests in a Git repository. These are implementations of GitOps, not GitOps itself.

GitOps is a contract between the engineering team and the production system. The contract has three clauses:

1. Git is the source of truth. If it is not in Git, it does not exist in production.
2. Changes go through pull requests. No direct pushes, no manual kubectl, no console clicks.
3. The system reconciles to Git automatically. If production drifts from Git, it is corrected. If Git changes, production changes.

Every GitOps failure I have audited traces back to a violation of one of these three clauses. The tool was installed but the contract was not enforced.

Clause one: Git is the source of truth

The most common violation. Someone runs kubectl apply during an incident. Someone changes a ConfigMap via the Kubernetes dashboard. Someone scales a deployment manually. Each of these creates drift between Git and the cluster. The drift is invisible until ArgoCD syncs and either reverts the change or fails because of a conflict.

The fix is cultural, not technical. The rule is: if you touched the cluster directly, you broke the contract. Even if it was an emergency. Even if it was faster. The post-incident process includes a PR that captures whatever change was made manually, so Git catches up to the cluster.

Some teams add a technical safeguard: ArgoCD in “sync only” mode, which means it will not auto-correct drift but will alert on it. This is a mistake. It teaches the team that drift is acceptable as long as someone notices. The right setting is “auto-sync with self-heal,” which means any manual change is immediately reverted. Engineers learn quickly not to make manual changes when the system undoes them in seconds.

Clause two: changes go through pull requests

The second most common violation. The CI pipeline pushes directly to the main branch. A script runs kubectl apply after a build. A developer merges a PR without review because it is “just a ConfigMap change.”

The pull request is the review gate. It is where a second pair of eyes catches the typo in the replica count, the missing resource limit, the wrong namespace. Without review, Git is just a version control system. With review, it is a quality gate.

The teams that take this seriously require:

• At least one approval on every PR that changes production state.
• CI checks that validate manifests before merge (kubeval, kubeconform, or kyverno).
• No force pushes to main. No direct commits to main. Branch protection enforced by the Git provider, not by convention.

Clause three: the system reconciles automatically

The third violation is subtler. The team has ArgoCD installed, but it is in manual sync mode. Every deployment requires someone to click “Sync” in the ArgoCD UI. This is not GitOps. This is Git-plus-a-button.

Manual sync mode exists for good reasons: risk-averse environments, regulated industries, teams that want a human in the loop for every production change. But it undermines the core value of GitOps, which is that the system converges to the desired state without human intervention.

The teams I see getting the most value from GitOps run auto-sync with progressive delivery. ArgoCD syncs automatically, but the deployment is gated by Argo Rollouts or Flagger. The new version starts at 10% traffic. If the error rate stays below the threshold, it goes to 50%, then 100%. If the error rate spikes, it rolls back automatically. No human clicks a button. The system reconciles to Git, and the safety net is the progressive delivery controller, not a human watching a dashboard.

What changes when you take the contract seriously

The teams that enforce all three clauses experience a shift that is hard to quantify but easy to feel:

Audits become trivial. “Show me every change to production in the last 30 days” is a git log command. “Who approved this change?” is a PR review. “When did this ConfigMap change?” is a commit timestamp.

Rollbacks become boring. git revert + ArgoCD sync = rollback. No 2am kubectl sessions. No scrambling to find the previous image tag. The commit history is the rollback path.

Drift becomes visible. When Git is the source of truth and the system auto-syncs, any manual change is immediately reverted. The team learns that manual changes do not stick. The only way to change production is through Git.

Onboarding becomes faster. A new engineer reads the Git repository and sees the entire production state. They do not need access to the cluster to understand what is running. They can make changes via PR and learn the system by reading diffs.

The honest tradeoff

GitOps is slower than kubectl. A PR takes minutes. A kubectl command takes seconds. For teams that are used to the speed of manual changes, GitOps feels like bureaucracy.

It is bureaucracy. It is the bureaucracy that prevents the 3am incident, the untraceable change, the drift that nobody notices until it breaks. The speed you lose in deployment, you gain in reliability, auditability, and sleep.

The teams that adopt GitOps and keep doing manual changes have a tool but not a contract. The teams that adopt the contract find that the tool becomes invisible. Nobody thinks about ArgoCD. They think about the PR, the review, the merge. The tool does the rest.