kubernetes调度亲和性

发表于： 2019-03-28 2021-03-25
分类： SRE
标签： affinity, I/O, kubernetes, nginx, node

官方文档：https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
通常情况下不需要人为限制，kubernetes会自动调度到空闲的node上；若有特殊需求则需要人为干预。

nodeName: 最直接的限定

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
  nodeName: node01

nodeSelector：通过node label来限制部署的目标节点

如：
kubectl label nodes node01 disktype=ssd
使用：

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    env: test
spec:
  containers:
  - name: nginx
    image: nginx
    imagePullPolicy: IfNotPresent
  nodeSelector:
    disktype: ssd

内置的node labels有：

beta.kubernetes.io/arch: amd64
beta.kubernetes.io/os: linux
kubernetes.io/arch: amd64
kubernetes.io/hostname: node01
kubernetes.io/os: linux
node-role.kubernetes.io/master 等

Affinity and anti-affinity

扩展了表达式的约束细节，分为node affinity和inter-pod affinity/anti-affinity两类。

node affinity： nodeSelector的增强版

如果同时设置了nodeSelector和nodeAffinity，则需要同时满足才能成为候选者node
node affinity语法支持以下集中操作符：In， NotIn， Exists， DoesNotExist，Gt，Lt。

requiredDuringSchedulingIgnoredDuringExecution

强亲和，它保证所选的Node必须满足Pod对Node的所有要求,条件必须满足从而调度到一个节点。

preferredDuringSchedulingIgnoredDuringExecution

弱亲和，它对kube-scheduler提出需求，kube-scheduler会尽量但不保证满足NodeSelector的要求,条件不满足则调度到不满足条件的节点。
例如：

apiVersion: v1
kind: Pod
metadata:
  name: miner
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/gpu
            operator: In
            values:
            - gtx2080
            - gtx1080
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: kubernetes.io/cpu
            operator: In
            values:
            - 9900k
  containers:
  - name: miner
    image: my.harbor/miner

表示要运行这个pod，节点必须有2080或者1080的kubernetes.io/gpu,最好还要有9900k的kubernetes.io/cpu

inter-pod affinity/anti-affinity

根据节点上运行的pod的标签而不是node自己标签来调度目标node。
pod affinity 合法的操作符有In, NotIn, Exists, DoesNotExist, Gt, Lt。
例如：

apiVersion: v1
kind: Pod
metadata:
  name: miner
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: cointype
            operator: NotIn
            values:
            - btc
        topologyKey: failure-domain.beta.kubernetes.io/zone
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: logger
              operator: In
              values:
              - fluented
          topologyKey: kubernetes.io/hostname
  containers:
  - name: miner
    image: my.harbor/miner

表示要运行这个pod，节点上一定不能存在标签为cointype=btc的pod，最好节点上还存在标签logger=fluented为的pod

joker

174