When I am on the side that’s interviewing this is actually my favorite question to ask about Kubernetes. It might be the one question to ask someone just to understand how much Kubernetes they know. It includes pods, replica sets, controllers, doing updates – roll out and recreate, rolling back, coordinating the replica sets …
Let’s set up the post with the following diagram:
On worker A we have just the pod, as you see there is no higher level order object wrapped around him. This means that if someone or something deletes our pod it’s gone. Let’s say that some imaginary blue guy does it.
Ok, we lost the workload on Worker Node A. But Kubernetes is providing a lot of abstractions that we take for granted, this include having a ReplicaSet. Now it’s worth to also mention that one of the Kubernetes control plane parts include a set of controllers, for example a Replica Set controller which will react on different type of events. Deleting the pod on node B will trigger a DELETE POD event which the Kubernetes REST API will make sure that it gets sended to the Replica Set controller. From there on it’s the controller job to decide what to do with it.
In our case if we delete the pod on Worker Node B, the Replica Set controller will be notified and will make sure to spin up enough replicas to cover the number in the manifest. Let’s say it’s just one.
We do not have etcd on the picture just to simplify it and also the controllers usually register the events that they care about initially.
Now if you notice this is a native high availability mechanism in Kubernetes that you get by default :) Now I just need to make the difference that if your set of worker nodes is deployed in a single place and that single place goes down for whatever reason the internal Kubernetes high availability does not help you out too much, just so that we are on the same page.
Now if we delete the pod on worker node C again the Replica Set Controller will be the one to bring back the number of replicas. But then why do we wrap the Replica Set inside a Deployment ?
The deployment is responsible to coordinate the process that happens internally when you wish to deploy a new version of your application. In Kubernetes you have 2 native deployment strategies:
- a roll out deployment, that make sure that the new pods are deployed one by one, as for each new healthy one it destroys an old one.
- a recreate strategy, where all of the pods in v1 are being deleted at once and replaced by v2. Here you have some small downtime, but if you application can’t tolerate running different versions at the same time this is the way to go.
- There are other custom deployment strategies, like Canary, Blue-Green and Progressive Delivery, but these ones are coming with some extensions, CRDs and their own Controllers. (here a good example is Argo Rollouts ).
When you are changing the version of you app, let’s say from v1 to v2 there is a new Replica Set that will be created, which will be co-existing at least for some time with the current Replica Set.
The main responsibility of the Deployment is to make sure that it gradually scales up the pods from the new Replica Set and scales down the pods from the old Replica Set ( with the assumption that this is the default rollout update process ), keep an eye on that process and if it does not work out for some time go back with the old v1.
Ok, we did not get just a new version. We also get a number of replica set revisions in case we have to roll back. By default, Kubernetes keeps the last 10 ReplicaSets (controlled by spec.revisionHistoryLimit) with 0 replicas just sitting there waiting, so you can perform an instant rollout if needed.
Pretty great for a free ride, if you ask me :)
srejournals 
Leave a comment