Kubernetes集群部署之五node节点部署

Node节点是Kubernetes集群中的工作负载节点.每个node都会被master分配一些工作负载,每个node节点都运行以下关键服务进程.
Kubelet :负责pod对应的容器的创建、启停等任务，同时与master节点密切协作，实现集群管理的基本功能.
Kube-proxy: 实现kubernetes service的通信与负载均衡机制的重要组件.

Docker Engine(docker): Docker引擎，负责本机的容器创建和管理工作.

部署kubelet:

1.二进制包准备 将软件包可执行文件从k8s-master复制到node节点中去。

1[root@k8s-master ~]# cd /usr/local/src/kubernetes/server/bin
2[root@k8s-master bin]# scp kubelet kube-proxy 10.200.3.106:/opt/kubernetes/bin/
3[root@k8s-master bin]# scp kubelet kube-proxy 10.200.3.107:/opt/kubernetes/bin/

2.创建角色绑定

1[root@k8s-master ~]# cd /usr/local/src/ssl/
2[root@k8s-master ssl]#  kubectl create clusterrolebinding kubelet-bootstrap \
3  --clusterrole=system:node-bootstrapper \
4  --user=kubelet-bootstrap
5
6clusterrolebinding "kubelet-bootstrap" createdclusterrolebinding.rbac.authorization.k8s.io "kubelet-bootstrap" created

3.创建 kubelet bootstrap.kubeconfig 文件 设置集群参数

1[root@k8s-master ssl]#  kubectl config set-cluster kubernetes \
2   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
3   --embed-certs=true \
4   --server=https://10.200.3.105:6443 \
5   --kubeconfig=bootstrap.kubeconfig
6
7Cluster "kubernetes" set.

4.设置客户端认证参数,token值为之前生成的.

1[root@k8s-master ssl]# kubectl config set-credentials kubelet-bootstrap \
2   --token=10a89e49bc403bce8fb134e5a2ae82f1 \
3   --kubeconfig=bootstrap.kubeconfig
4
5User "kubelet-bootstrap" set.

5.设置上下文参数

1[root@k8s-master ssl]# kubectl config set-context default \
2   --cluster=kubernetes \
3   --user=kubelet-bootstrap \
4   --kubeconfig=bootstrap.kubeconfig
5      
6Context "default" created.

6.选择默认上下文并向node节点分发在master端生成的bootstrap.kubeconfig文件.

1[root@k8s-master ssl]# kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
2Switched to context "default".
3
4[root@k8s-master ssl]# cp bootstrap.kubeconfig /opt/kubernetes/cfg
5[root@k8s-master ssl]# scp bootstrap.kubeconfig 10.200.3.106:/opt/kubernetes/cfg
6[root@k8s-master ssl]# scp bootstrap.kubeconfig 10.200.3.107:/opt/kubernetes/cfg

部署kubelet(node节点操作).

1.设置CNI支持.

1[root@k8s-node-1 ~]# mkdir -p /etc/cni/net.d
2[root@k8s-node-1 ~]# cat > /etc/cni/net.d/10-default.conf <<EOF
3{
4        "name": "flannel",
5        "type": "flannel",
6        "delegate": {
7            "bridge": "docker0",
8            "isDefaultGateway": true,
9            "mtu": 1400
10        }
11}
12
13EOF

2.创建kubelet目录

[root@k8s-node-1 ~]# mkdir /var/lib/kubelet

3.创建kubelet服务配置

1[root@k8s-node-1 ~]# cat > /usr/lib/systemd/system/kubelet.service <<EOF
2[Unit]
3Description=Kubernetes Kubelet
4Documentation=https://github.com/GoogleCloudPlatform/kubernetes
5After=docker.service
6Requires=docker.service
7
8[Service]
9WorkingDirectory=/var/lib/kubelet
10ExecStart=/opt/kubernetes/bin/kubelet \
11  --address=10.200.3.106 \
12  --hostname-override=10.200.3.106 \
13  --pod-infra-container-image=mirrorgooglecontainers/pause-amd64:3.0 \
14  --experimental-bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \
15  --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \
16  --cert-dir=/opt/kubernetes/ssl \
17  --network-plugin=cni \
18  --cni-conf-dir=/etc/cni/net.d \
19  --cni-bin-dir=/opt/kubernetes/bin/cni \
20  --cluster-dns=10.1.0.2 \
21  --cluster-domain=cluster.local. \
22  --hairpin-mode hairpin-veth \
23  --allow-privileged=true \
24  --fail-swap-on=false \
25  --logtostderr=true \
26  --v=2 \
27  --logtostderr=false \
28  --log-dir=/opt/kubernetes/log
29Restart=on-failure
30RestartSec=5
31
32EOF

4.启动Kubelet

1[root@k8s-node-1 ~]# systemctl daemon-reload
2[root@k8s-node-1 ~]# systemctl enable kubelet
3[root@k8s-node-1 ~]# systemctl start kubelet
4[root@k8s-node-1 ~]# systemctl status kubelet

5.查看csr请求 注意是在k8s-maste上执行

1[root@k8s-master ssl]# kubectl get csr
2NAME                                                   AGE       REQUESTOR           CONDITION
3node-csr-exRV4L4JbdyoO62eNEcTnSK7ehYji_47xhkU7e5-L5c   43s       kubelet-bootstrap   Pending

6.批准kubelet 的 TLS 证书请求

1[root@k8s-master ssl]# kubectl get csr|grep 'Pending' | awk 'NR>0{print $1}'| xargs kubectl certificate approve
2certificatesigningrequest.certificates.k8s.io "node-csr-exRV4L4JbdyoO62eNEcTnSK7ehYji_47xhkU7e5-L5c" approved

执行完毕后，查看节点状态如果是Ready的状态就说明一切正常.

1[root@k8s-master ~]# kubectl get node
2NAME           STATUS    ROLES     AGE       VERSION
310.200.3.106   Ready     <none>    1d        v1.10.1

部署Kubernetes Proxy

1.配置kube-proxy使用LVS

[root@v0-k8s-node-1]# yum install -y ipvsadm ipset conntrack

2.创建 kube-proxy 证书请求

1[root@k8s-master ~]# cd /usr/local/src/ssl/
2[root@k8s-master ssl]# cat > kube-proxy-csr.json <<EOF
3
4{
5  "CN": "system:kube-proxy",
6  "hosts": [],
7  "key": {
8    "algo": "rsa",
9    "size": 2048
10  },
11  "names": [
12    {
13      "C": "CN",
14      "ST": "BeiJing",
15      "L": "BeiJing",
16      "O": "k8s",
17      "OU": "System"
18    }
19  ]
20}
21
22EOF

3.生成证书，并分发至node节点.

1[root@k8s-master ssl]#cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
2   -ca-key=/opt/kubernetes/ssl/ca-key.pem \
3   -config=/opt/kubernetes/ssl/ca-config.json \
4   -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy
5   
6[root@k8s-master ssl]# cp kube-proxy*.pem /opt/kubernetes/ssl/
7[root@k8s-master ssl]# scp kube-proxy*.pem 10.200.3.106:/opt/kubernetes/ssl/
8[root@k8s-master ssl]# scp kube-proxy*.pem 10.200.3.107:/opt/kubernetes/ssl/

4.创建kube-proxy配置文件

1[root@k8s-master ssl]# kubectl config set-cluster kubernetes \
2   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
3   --embed-certs=true \
4   --server=https://10.200.3.105:6443 \
5   --kubeconfig=kube-proxy.kubeconfig
6   
7Cluster "kubernetes" set.
8
9[root@k8s-master ssl]#kubectl config set-credentials kube-proxy \
10   --client-certificate=/opt/kubernetes/ssl/kube-proxy.pem \
11   --client-key=/opt/kubernetes/ssl/kube-proxy-key.pem \
12   --embed-certs=true \
13   --kubeconfig=kube-proxy.kubeconfig
14
15User "kube-proxy" set.
16
17[root@k8s-master ssl]#kubectl config set-context default \
18   --cluster=kubernetes \
19   --user=kube-proxy \
20   --kubeconfig=kube-proxy.kubeconfig
21Context "default" created.
22
23[root@k8s-master ssl]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
24Switched to context "default".

6.分发kubeconfig配置文件到各节点.

1[root@k8s-master ssl]# cp kube-proxy.kubeconfig /opt/kubernetes/cfg/
2[root@k8s-master ssl]# scp kube-proxy.kubeconfig 10.200.3.106:/opt/kubernetes/cfg/
3[root@k8s-master ssl]# scp kube-proxy.kubeconfig 10.200.3.107:/opt/kubernetes/cfg/

7.创建kube-proxy服务配置

1[root@k8s-node-1 ~]# mkdir /var/lib/kube-proxy
2[root@k8s-node-1 ~]# cat > /usr/lib/systemd/system/kube-proxy.service <<EOF
3
4[Unit]
5Description=Kubernetes Kube-Proxy Server
6Documentation=https://github.com/GoogleCloudPlatform/kubernetes
7After=network.target
8
9[Service]
10WorkingDirectory=/var/lib/kube-proxy
11ExecStart=/opt/kubernetes/bin/kube-proxy \
12  --bind-address=10.200.3.106 \
13  --hostname-override=10.200.3.106 \
14  --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig \
15--masquerade-all \
16  --feature-gates=SupportIPVSProxyMode=true \
17  --proxy-mode=ipvs \
18  --ipvs-min-sync-period=5s \
19  --ipvs-sync-period=5s \
20  --ipvs-scheduler=rr \
21  --logtostderr=true \
22  --v=2 \
23  --logtostderr=false \
24  --log-dir=/opt/kubernetes/log
25
26Restart=on-failure
27RestartSec=5
28LimitNOFILE=65536
29
30[Install]
31WantedBy=multi-user.target
32
33EOF

8.启动Kubernetes Proxy

1[root@k8s-node-1 ~]# systemctl daemon-reload
2[root@k8s-node-1 ~]# systemctl enable kube-proxy
3[root@k8s-node-1 ~]# systemctl start kube-proxy
4[root@k8s-node-1 ~]# systemctl status kube-proxy

9.检查LVS状态

1[root@k8s-node-1 ~]# ipvsadm -L -n
2IP Virtual Server version 1.2.1 (size=4096)
3Prot LocalAddress:Port Scheduler Flags
4  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
5TCP  10.1.0.1:443 rr persistent 10800
6  -> 10.200.3.105:6443            Masq    1      0          0         
7[root@k8s-node-1 ~]#

如果你在两台实验机器都安装了kubelet和proxy服务，使用下面的命令可以检查状态：

1[root@k8s-master ~]# kubectl get nodes
2
3NAME           STATUS    ROLES     AGE       VERSION
410.200.3.106   Ready     <none>    3h        v1.10.1
510.200.3.107   Ready     <none>    29m       v1.10.1

参考文档：https://github.com/unixhot/salt-kubernetes/blob/master/docs/node.md