Kubernetes’ Node Allocatable feature allows the cluster to reserve the resources of node for system daemons of OS and Kubernetes itself. For example, when I ran
kubectl describe node for a node in my AKS cluster, I got the following capacity related output. The size of this node was Standard DS2 v2 which had 2 CPU cores and 7GB memory.
Capacity: attachable-volumes-azure-disk: 8 cpu: 2 ephemeral-storage: 101445900Ki hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 7113660Ki pods: 110 Allocatable: attachable-volumes-azure-disk: 8 cpu: 1900m ephemeral-storage: 93492541286 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 4668348Ki pods: 110
From the output, out of 2 cores and 7GB memory, there were 1900 millicores and 66% memory (4668348/7113660) allocatable to pods. The details about how these numbers were calculated are described in this document. In short, the configuration of node allocatable in AKS is as follows:
|CPU cores on host||1||2||4||8||16||32||64|
- eviction-hard: 750Mi, which is the default configuration of the upstream aks-engine.
- kube-reserved: regressive rate
- 25% of the first 4 GB of memory
- 20% of the next 4 GB of memory (up to 8 GB)
- 10% of the next 8 GB of memory (up to 16 GB)
- 6% of the next 112 GB of memory (up to 128 GB)
- 2% of any memory above 128 GB
However, the above memory reservation doesn’t seem to applicable to Windows nodes. The following was the output when I ran the
kubectl describe node for a Windows node. More memory was reserved for Windows nodes.
Capacity: attachable-volumes-azure-disk: 8 cpu: 2 ephemeral-storage: 133703676Ki memory: 7339572Ki pods: 30 Allocatable: attachable-volumes-azure-disk: 8 cpu: 1900m ephemeral-storage: 133703676Ki memory: 3565108Ki pods: 30
The document doesn’t have any information regarding Windows node. GKE reserves approximately 1.5 times more resources on Windows Server nodes. Not sure if it’s the same for AKS. I’ve opened an issue to ask for further information.
This post provides a good comparison of reserved resources for 3 major cloud offerings: AKS, GKE and EKS.