tjtjtjのメモ

自分のためのメモです

Kubernetes 学習 Performing a Rolling Update を自前クラスタで復習

Performing a Rolling Update

https://kubernetes.io/docs/tutorials/kubernetes-basics/update/update-intro/

前回の続きを確認

$ kubectl get nodes
NAME   STATUS   ROLES    AGE   VERSION
kb1    Ready    master   10d   v1.13.2
kb2    Ready    <none>   10d   v1.13.2
kb3    Ready    <none>   10d   v1.13.2

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   2/2     2            2           18m

$ kubectl get pods -o wide
NAME                                   READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
kubernetes-bootcamp-6bf84cb898-6k94z   1/1     Running   0          16m   10.244.2.10   kb3    <none>           <none>
kubernetes-bootcamp-6bf84cb898-qswcf   1/1     Running   0          18m   10.244.2.9    kb3    <none>           <none>

$ kubectl get services
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
bootcamp-service   LoadBalancer   10.108.87.251   <pending>     8080:31729/TCP   10m
kubernetes         ClusterIP      10.96.0.1       <none>        443/TCP          10d

チュートリアルは replicas:4 でスタートするので スケールさせる

$ kubectl scale deployments/kubernetes-bootcamp --replicas=4
deployment.extensions/kubernetes-bootcamp scaled

$ kubectl get pods -o wide
NAME                                   READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
kubernetes-bootcamp-6bf84cb898-6k94z   1/1     Running   0          18m   10.244.2.10   kb3    <none>           <none>
kubernetes-bootcamp-6bf84cb898-gqms4   1/1     Running   0          4s    10.244.1.14   kb2    <none>           <none>
kubernetes-bootcamp-6bf84cb898-hnl46   1/1     Running   0          4s    10.244.1.13   kb2    <none>           <none>
kubernetes-bootcamp-6bf84cb898-qswcf   1/1     Running   0          21m   10.244.2.9    kb3    <none>           <none>

Step 1: Update the version of the app

確認

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   4/4     4            4           22m

$ kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-6bf84cb898-6k94z   1/1     Running   0          20m
kubernetes-bootcamp-6bf84cb898-gqms4   1/1     Running   0          79s
kubernetes-bootcamp-6bf84cb898-hnl46   1/1     Running   0          79s
kubernetes-bootcamp-6bf84cb898-qswcf   1/1     Running   0          22m

pods 詳細確認。長いので2コ目以降は省略

$ kubectl describe pods
Name:               kubernetes-bootcamp-6bf84cb898-6k94z
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               kb3/192.168.0.103
Start Time:         Tue, 05 Feb 2019 20:05:30 +0900
Labels:             pod-template-hash=6bf84cb898
                    run=kubernetes-bootcamp
Annotations:        cni.projectcalico.org/podIP: 10.244.2.10/32
Status:             Running
IP:                 10.244.2.10
Controlled By:      ReplicaSet/kubernetes-bootcamp-6bf84cb898
Containers:
  kubernetes-bootcamp:
    Container ID:   docker://81839f7a06215d94260e38103f735cd51625b555ec8242d24eecc5bf52ad6d05
    Image:          gcr.io/google-samples/kubernetes-bootcamp:v1
    Image ID:       docker-pullable://gcr.io/google-samples/kubernetes-bootcamp@sha256:0d6b8ee63bb57c5f5b6156f446b3bc3b3c143d233037f3a2f00e279c8fcc64af
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 05 Feb 2019 20:05:32 +0900
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-88mmv (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  default-token-88mmv:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-88mmv
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  20m   default-scheduler  Successfully assigned default/kubernetes-bootcamp-6bf84cb898-6k94z to kb3
  Normal  Pulled     20m   kubelet, kb3       Container image "gcr.io/google-samples/kubernetes-bootcamp:v1" already present on machine
  Normal  Created    20m   kubelet, kb3       Created container
  Normal  Started    20m   kubelet, kb3       Started container

~略~

kubernetes-bootcamp のイメージを jocatalin/kubernetes-bootcamp:v2 にする。

$ kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v2
deployment.extensions/kubernetes-bootcamp image updated

Step 2: Verify an update

イメージ更新を確認。v1 のpod がterminateして、v2のpodがrun してきた。

$ kubectl get pods
NAME                                   READY   STATUS        RESTARTS   AGE
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running       0          36s
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running       0          31s
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running       0          36s
kubernetes-bootcamp-5bf4d5689b-kpbvc   1/1     Running       0          31s
kubernetes-bootcamp-6bf84cb898-6k94z   1/1     Terminating   0          22m
kubernetes-bootcamp-6bf84cb898-gqms4   1/1     Terminating   0          3m40s
kubernetes-bootcamp-6bf84cb898-hnl46   0/1     Terminating   0          3m40s
kubernetes-bootcamp-6bf84cb898-qswcf   1/1     Terminating   0          25m

$ kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running   0          5m47s
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running   0          5m42s
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running   0          5m47s
kubernetes-bootcamp-5bf4d5689b-kpbvc   1/1     Running   0          5m42s

deployments/kubernetes-bootcamp の詳細確認 image: jocatalin/kubernetes-bootcamp:v2 になっている。

$ kubectl describe deployments/kubernetes-bootcamp
Name:                   kubernetes-bootcamp
Namespace:              default
CreationTimestamp:      Tue, 05 Feb 2019 20:02:55 +0900
Labels:                 run=kubernetes-bootcamp
Annotations:            deployment.kubernetes.io/revision: 2
Selector:               run=kubernetes-bootcamp
Replicas:               4 desired | 4 updated | 4 total | 4 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  run=kubernetes-bootcamp
  Containers:
   kubernetes-bootcamp:
    Image:        jocatalin/kubernetes-bootcamp:v2
    Port:         8080/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   kubernetes-bootcamp-5bf4d5689b (4/4 replicas created)
Events:
  Type    Reason             Age                  From                   Message
  ----    ------             ----                 ----                   -------
  Normal  ScalingReplicaSet  28m                  deployment-controller  Scaled up replica set kubernetes-bootcamp-6bf84cb898 to 1
  Normal  ScalingReplicaSet  6m51s (x2 over 25m)  deployment-controller  Scaled up replica set kubernetes-bootcamp-6bf84cb898 to 4
  Normal  ScalingReplicaSet  3m47s                deployment-controller  Scaled up replica set kubernetes-bootcamp-5bf4d5689b to 1
  Normal  ScalingReplicaSet  3m47s                deployment-controller  Scaled down replica set kubernetes-bootcamp-6bf84cb898 to 3
  Normal  ScalingReplicaSet  3m47s                deployment-controller  Scaled up replica set kubernetes-bootcamp-5bf4d5689b to 2
  Normal  ScalingReplicaSet  3m42s (x2 over 15m)  deployment-controller  Scaled down replica set kubernetes-bootcamp-6bf84cb898 to 2
  Normal  ScalingReplicaSet  3m42s                deployment-controller  Scaled up replica set kubernetes-bootcamp-5bf4d5689b to 3
  Normal  ScalingReplicaSet  3m42s                deployment-controller  Scaled down replica set kubernetes-bootcamp-6bf84cb898 to 1
  Normal  ScalingReplicaSet  3m42s                deployment-controller  Scaled up replica set kubernetes-bootcamp-5bf4d5689b to 4
  Normal  ScalingReplicaSet  3m40s                deployment-controller  Scaled down replica set kubernetes-bootcamp-6bf84cb898 to 0

pods の詳細確認。ながいので略

$ kubectl describe pods
Name:               kubernetes-bootcamp-5bf4d5689b-4gd7q
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               kb2/192.168.0.102
Start Time:         Tue, 05 Feb 2019 20:27:29 +0900
Labels:             pod-template-hash=5bf4d5689b
                    run=kubernetes-bootcamp
Annotations:        cni.projectcalico.org/podIP: 10.244.1.15/32
Status:             Running
IP:                 10.244.1.15
Controlled By:      ReplicaSet/kubernetes-bootcamp-5bf4d5689b
Containers:
  kubernetes-bootcamp:
    Container ID:   docker://1912a6a10c45f44ef1267ed7d585abed509fdc27ffcfcd8c8b85d22e087d87b6
    Image:          jocatalin/kubernetes-bootcamp:v2
    Image ID:       docker-pullable://jocatalin/kubernetes-bootcamp@sha256:fb1a3ced00cecfc1f83f18ab5cd14199e30adc1b49aa4244f5d65ad3f5feb2a5
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 05 Feb 2019 20:27:33 +0900
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-88mmv (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  default-token-88mmv:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-88mmv
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  5m1s   default-scheduler  Successfully assigned default/kubernetes-bootcamp-5bf4d5689b-4gd7q to kb2
  Normal  Pulling    5m     kubelet, kb2       pulling image "jocatalin/kubernetes-bootcamp:v2"
  Normal  Pulled     4m57s  kubelet, kb2       Successfully pulled image "jocatalin/kubernetes-bootcamp:v2"
  Normal  Created    4m57s  kubelet, kb2       Created container
  Normal  Started    4m57s  kubelet, kb2       Started container


~略~

curl 連打。各ポッドがお仕事しています。

$ curl http://10.108.87.251:8080
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-5bf4d5689b-dsh7s | v=2

$ curl http://10.108.87.251:8080
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-5bf4d5689b-4gd7q | v=2

$ curl http://10.108.87.251:8080
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-5bf4d5689b-kpbvc | v=2

$ curl http://10.108.87.251:8080
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-5bf4d5689b-4xzxq | v=2

ロールアウトステータスコマンドを実行しても、アップデートを確認できるそうだ。 successfully rolled out

$ kubectl rollout status deployments/kubernetes-bootcamp
deployment "kubernetes-bootcamp" successfully rolled out

pods の詳細は略

Step 3: Rollback an update

チュートリアルやったときここ意味が分からなかった。

もう一度アップデートを実行して、v10としてタグ付けされたイメージをデプロイしましょう。

だそうです。

$ kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=gcr.io/google-samples/kubernetes-bootcamp:v10
deployment.extensions/kubernetes-bootcamp image updated

status:ImagePullBackOff <-> ErrImagePull を繰り返す。これは v10 のイメージがないから。

$ kubectl get pods
NAME                                   READY   STATUS             RESTARTS   AGE
kubernetes-bootcamp-597cfc5b76-flxjt   0/1     ImagePullBackOff   0          28s
kubernetes-bootcamp-597cfc5b76-zrshs   0/1     ImagePullBackOff   0          28s
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running            0          17m
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running            0          17m
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running            0          17m
kubernetes-bootcamp-5bf4d5689b-kpbvc   1/1     Terminating        0          17m

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   3/4     2            3           42m

@kb1 ~ $ kubectl get pods
NAME                                   READY   STATUS         RESTARTS   AGE
kubernetes-bootcamp-597cfc5b76-flxjt   0/1     ErrImagePull   0          39s
kubernetes-bootcamp-597cfc5b76-zrshs   0/1     ErrImagePull   0          39s
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running        0          17m
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running        0          17m
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running        0          17m

$ kubectl get pods
NAME                                   READY   STATUS             RESTARTS   AGE
kubernetes-bootcamp-597cfc5b76-flxjt   0/1     ImagePullBackOff   0          44s
kubernetes-bootcamp-597cfc5b76-zrshs   0/1     ImagePullBackOff   0          44s
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running            0          17m
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running            0          17m
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running            0          17m

$ kubectl get pods
NAME                                   READY   STATUS         RESTARTS   AGE
kubernetes-bootcamp-597cfc5b76-flxjt   0/1     ErrImagePull   0          64s
kubernetes-bootcamp-597cfc5b76-zrshs   0/1     ErrImagePull   0          64s
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running        0          17m
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running        0          17m
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running        0          17m

rollout を undo 。なるほど rollout がここで生きてくると。

$ kubectl rollout undo deployments/kubernetes-bootcamp
deployment.extensions/kubernetes-bootcamp rolled back
$ kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Running   0          21m
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Running   0          21m
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Running   0          21m
kubernetes-bootcamp-5bf4d5689b-rtxj8   1/1     Running   0          5s

お掃除

$ kubectl delete services bootcamp-service
service "bootcamp-service" deleted

$ kubectl get services
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   10d

$ kubectl delete deployment kubernetes-bootcamp
deployment.extensions "kubernetes-bootcamp" deleted

$ kubectl get deployment 
No resources found.

$ kubectl get pods
NAME                                   READY   STATUS        RESTARTS   AGE
kubernetes-bootcamp-5bf4d5689b-4gd7q   1/1     Terminating   0          32m
kubernetes-bootcamp-5bf4d5689b-4xzxq   1/1     Terminating   0          32m
kubernetes-bootcamp-5bf4d5689b-dsh7s   1/1     Terminating   0          32m
kubernetes-bootcamp-5bf4d5689b-rtxj8   1/1     Terminating   0          10m

$ kubectl get pods
No resources found.