tjtjtjのメモ

自分のためのメモです

kubernetes 学習 taints tolerations

coredns が起動しないのを taints tolerations のせいかと勘違いしちょっといじったが違った。今回はちゃんと taints tolerations 試す。

ありがたい資料

Taints and Tolerations https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/

ポッドが不適切なノードにスケジュールされないようにする。 ノードが汚染を容認しないポッドを受け入れない。

Kubernetesのtaintsとtolerationsについて https://qiita.com/sheepland/items/8fedae15e157c102757f

最初の確認

$ kubectl describe node kb1
Name:               kb1
Roles:              master
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/hostname=kb1
                    node-role.kubernetes.io/master=
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 192.168.0.101/24
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 13 Feb 2019 20:28:37 +0900
Taints:             node-role.kubernetes.io/master:NoSchedule
Unschedulable:      false
$ kubectl describe node kb2
Name:               kb2
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/hostname=kb2
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 192.168.0.102/24
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 13 Feb 2019 20:51:00 +0900
Taints:             <none>
Unschedulable:      false
$ kubectl describe node kb3
Name:               kb3
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/hostname=kb3
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 192.168.0.103/24
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 13 Feb 2019 20:51:51 +0900
Taints:             <none>
Unschedulable:      false

taint, toleration なし

pod-nginx.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
  replicas: 1

てきとーなpod適用。kb2で動作中。

$ kubectl create -f pod-nginx.yaml
pod/nginx created
$ kubectl get pod -o wide
NAME    READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
nginx   1/1     Running   0          55s   10.244.1.39   kb2    <none>           <none>

$ kubectl delete pod nginx pod "nginx" deleted $ kubectl get pod -o wide No resources found.

taint, toleration 設定 許容nodeあり

node:kb2 に taint:key1=test を設定。 node:kb3 に taint:key1=prod を設定。 kb2 はテスト用、kb3 は本番用みたいな感じ。

$ kubectl taint node kb2 key1=test:NoSchedule
node/kb2 tainted
$ kubectl describe node kb2
:
Taints:             key1=test:NoSchedule
:
$ kubectl taint node kb3 key1=prod:NoSchedule
node/kb3 tainted
$ kubectl describe node kb3
:
Taints:             key1=prod:NoSchedule
:

pod:nginx は実は本番用だったってことにする。 toleration:prod:NoSchedule を設定。

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
  replicas: 1
  tolerations:
  - key: key1
    value: prod
    effect: NoSchedule

node:kb3 で pod:nginx がrunning

$ kubectl create -f pod-nginx.yaml
pod/nginx created
$ kubectl describe pod nginx 
:
Tolerations:     key1=prod:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
:
$ kubectl get pod -o wide
NAME    READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
nginx   1/1     Running   0          13s   10.244.2.30   kb3    <none>           <none>

許容nodeをなくし追い出す。

node:kb3 の taint 変えてみる。kb3 から追い出された。

$ kubectl taint node kb3 key1=test:NoExecute
node/kb3 tainted
$ kubectl get pod -o wide
NAME    READY   STATUS        RESTARTS   AGE     IP            NODE   NOMINATED NODE   READINESS GATES
nginx   0/1     Terminating   0          4m43s   10.244.2.31   kb3    <none>           <none>
$ kubectl get pod -o wide
No resources found.
$ kubectl describe pod nginx
Error from server (NotFound): pods "nginx" not found

お掃除

$ kubectl taint node kb2 key1-
$ kubectl taint node kb3 key1-

うーん

pod でなくdeployment で試すんだった。テスト/本番が同じクラスタってのもオカシイ。