The desired state of GitOps: “everybody does GitOps”.
The actual State of GitOps: “it depends…”.
Prehistoric GitOps
8 years ago I started a lightning talk on the then burning hot topic of immutable infrastructure by declaring the configuration management tools of that epoch (Puppet, Chef, Salt and Ansible) doomed.
I was pretty much on the money - out of that quartet only Ansible somewhat survived the test of time. Mainly because the late majority together with the laggards still constitute 50% of the whole technology adoption lifecycle. Some folks evidently still prefer that cuddly feeling of grooming and hand-feeding a bunch of tamagotchi servers.
BTW - even though modern GitOps proponents claim it has defined totally new principles, I believe we were doing GitOps even back then. All the state of my servers was stored in Git (or SVN for that matter) and Puppet or Chef agents were pulling it from the central state store. Syncing the actual state with the desired state. And yes, we were deploying apps the same way. Hitting the exact same bumps we’re hitting today. It’s just that the scale was smaller.
So, for a short moment there we had the hope of being done with the clumsiness of configuration languages. They all start with a promise of a simple DSL and then mutate by growing horns and tails in the form of loops, iterators and all kinds of wrappers that make the result even harder to decipher than if it were written in a regular programming language. (I’m looking at you Terragrunt and Helmfile).
The DSC (Desired State Config) Reconciled
Then Kubernetes came and revived the idea of desired state configuration. You declare the state, you run the reconciliation loop, you converge. Everything is predictable and deterministic. A dream come true. The practitioners knew it was never going to be that easy, but the ability to help the system when it rolls on its back and wiggles legs in the air gives us the sense of power. In general if we can ride two thirds of the distance - it’s still better than walking the whole way.
Kubernetes gave us much more than container orchestration! It gave us a declarative API that can potentially manage anything in the world! “How wonderful!” - we said.”Let’s put the specs for that API in Git, version them and create agents that will reconcile the state of the spec to the state stored in etcd so that other controllers then reconcile that to the state of whatever it is they are managing.”
GitOps was born. Or rather re-invented, if you follow my line of thought.
The Actual State of GitOps
Weaveworks (RIP) coined the term in parallel with announcing FluxCD - their OSS GitOps controller in 2017. ArgoCD was announced a year after that. The two projects have been competing for Ops love since then with ArgoCD taking a clear lead thanks to its slick UI. Lately, though, FluxCD has seen a surge in popularity with Weaveworks going out of business and a bunch of big brands (most notably Gitlab) clearly targeting the project as their OSS marketing vehicle.
6 years have passed since. In a recent overview of hundreds of Kubernetes clusters we only observed ArgoCD in a very small percentage of them. And we barely saw any FluxCD instances. On the other hand, if we believe the buzz - the force seems to be strong with GitOps. There’s even an OpenGitops initiative now defining the standards for how it should be done. But even though GitOps adopters swear it’s the best thing since sliced bread, we all know there are some actual issues holding us back.
To name just a couple:
- the whole concept of sync in GitOps is broken. GitOps approach seemingly assumes static configuration and then - life happens (mutating webhooks, anyone?) leading to endless sync options and diff customizations
- and of course GitOps controllers have a lot of trouble dealing with elastic resources (such as those created by autoscalers) and ephemeral entities such as jobs. Again - life isn’t static and, believe it or not - our clusters are alive.
The Question
Which leads me to the question I posed in the title of this post. With our compute resources becoming increasingly ephemeral and elastic, with automation slowly replacing humans in introducing change both to infrastructure and application logic - is GitOps really the path forward? What do you think? Are you doing GitOps? Are you content or are there pains? And if not GitOps-ing - then what is holding you back?
.png)
