kubernetes集群安装指南：kubelet组件部署

kubelet是kubernetes中一个重要的组件。对pod容器的管理 ，执行交互式命令(如 exec、run、logs 等)都离不开它，kubelet 运行在每个 worker 节点上，负责接收 kube-apiserver 发送的请求，kubelet 在启动时会自动向 kube-apiserver 发送注册信息，内置的 cadvisor 统计和监控节点的资源使用情况。

1. 准备工作

特别说明：这里所有的操作都是在devops这台机器上通过ansible工具执行；kubelet在需要使用kubeconfig文件来认证访问kube-apiserver，因此需要为其开启证书轮转

为确保安全，部署时关闭了 kubelet 的非安全 http 端口，对请求进行认证和授权，拒绝未授权的访问(如 apiserver、heapster 的请求)。

环境变量定义

#################### Variable parameter setting ######################
KUBE_NAME=kubelet
K8S_INSTALL_PATH=/data/apps/k8s/kubernetes
K8S_BIN_PATH=${K8S_INSTALL_PATH}/sbin
K8S_LOG_DIR=${K8S_INSTALL_PATH}/logs
K8S_CONF_PATH=/etc/k8s/kubernetes
KUBE_CONFIG_PATH=/etc/k8s/kubeconfig
CA_DIR=/etc/k8s/ssl
SOFTWARE=/root/software
HOSTNAME=`hostname`
VERSION=v1.14.2
DOWNLOAD_URL=https://github.com/devops-apps/download/raw/master/kubernetes/kubernetes-server-${VERSION}-linux-amd64.tar.gz
ETH_INTERFACE=eth2
LISTEN_IP=$(ifconfig | grep -A 1 ${ETH_INTERFACE} |grep inet |awk '{print $2}')
USER=k8s
CLUSTER_DNS_DOMAIN=k8s.mo9.com
CLUSTER_DNS_IP=10.254.0.2
CLUSTER_PODS_CIDR=172.16.0.0/20

2 部署kubelet组件

2.1 安装kubelet二进制文件

### 1.Check if the install directory exists.
if [ ! -d "$K8S_BIN_PATH" ]; then
     mkdir -p $K8S_BIN_PATH
fi

if [ ! -d "$K8S_LOG_DIR/$KUBE_NAME" ]; then
     mkdir -p $K8S_LOG_DIR/$KUBE_NAME
fi

if [ ! -d "$K8S_CONF_PATH" ]; then
     mkdir -p $K8S_CONF_PATH
fi

if [ ! -d "$KUBE_CONFIG_PATH" ]; then
     mkdir -p $KUBE_CONFIG_PATH
fi

### 2.Install kubelet binary of kubernetes.
if [ ! -f "$SOFTWARE/kubernetes-server-${VERSION}-linux-amd64.tar.gz" ]; then
     wget $DOWNLOAD_URL -P $SOFTWARE >>/tmp/install.log  2>&1
fi
cd $SOFTWARE && tar -xzf kubernetes-server-${VERSION}-linux-amd64.tar.gz -C ./
cp -fp kubernetes/server/bin/$KUBE_NAME $K8S_BIN_PATH
ln -sf  $K8S_BIN_PATH/${KUBE_NAME} /usr/local/bin
chmod -R 755 $K8S_INSTALL_PATH

2.3 创建kubelet配置文件

```# configure default system config
cat >${K8S_CONF_PATH}/kubelet-config.yaml <<EOF
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: "${LISTEN_IP}"
staticPodPath: ""
syncFrequency: 1m
fileCheckFrequency: 20s
httpCheckFrequency: 20s
staticPodURL: ""
port: 10250
readOnlyPort: 0
rotateCertificates: true
serverTLSBootstrap: true
authentication:
anonymous:
enabled: false
webhook:
enabled: true
cacheTTL: 2m0s
x509:
clientCAFile: "${CA_DIR}/ca.pem"
authorization:
mode: Webhook
registryPullQPS: 0
registryBurst: 20
eventRecordQPS: 0
eventBurst: 20
enableDebuggingHandlers: true
enableContentionProfiling: true
healthzPort: 10248
healthzBindAddress: "${LISTEN_IP}"
clusterDomain: "${CLUSTER_DNS_DOMAIN}"
clusterDNS:

• "${CLUSTER_DNS_IP}"
  nodeStatusUpdateFrequency: 10s
  nodeStatusReportFrequency: 1m
  imageMinimumGCAge: 2m
  imageGCHighThresholdPercent: 85
  imageGCLowThresholdPercent: 80
  volumeStatsAggPeriod: 1m
  kubeletCgroups: ""
  systemCgroups: ""
  cgroupRoot: ""
  cgroupsPerQOS: true
  cgroupDriver: cgroupfs
  runtimeRequestTimeout: 10m
  hairpinMode: promiscuous-bridge
  maxPods: 220
  podCIDR: "${CLUSTER_PODS_CIDR}"
  podPidsLimit: -1
  resolvConf: /etc/resolv.conf
  maxOpenFiles: 1000000
  kubeAPIQPS: 1000
  kubeAPIBurst: 2000
  serializeImagePulls: false
  evictionHard:
  memory.available: "100Mi"
  nodefs.available: "10%"
  nodefs.inodesFree: "5%"
  imagefs.available: "15%"
  evictionSoft: {}
  enableControllerAttachDetach: true
  failSwapOn: true
  containerLogMaxSize: 20Mi
  containerLogMaxFiles: 10
  systemReserved: {}
  kubeReserved: {}
  systemReservedCgroup: ""
  kubeReservedCgroup: ""
  enforceNodeAllocatable: ["pods"]
  EOF

  
  * address：kubelet 安全端口（https，10250）监听的地址，不能为 127.0.0.1，否则 kube-apiserver、heapster 等不能调用 kubelet 的 API；
  * readOnlyPort=0：关闭只读端口(默认 10255)，等效为未指定；
  * authentication.anonymous.enabled：设置为 false，不允许匿名�访问 10250 端口；
  * authentication.x509.clientCAFile：指定签名客户端证书的 CA 证书，开启 HTTP 证书认证；
  * authentication.webhook.enabled=true：开启 HTTPs bearer token 认证；
  * 对于未通过 x509 证书和 webhook 认证的请求(kube-apiserver 或其他客户端)，将被拒绝，提示 Unauthorized；
  * authroization.mode=Webhook：kubelet 使用 SubjectAcce***eview API 查询 kube-apiserver 某 user、group 是否具有操作资源的权限(RBAC)；
  * featureGates.RotateKubeletClientCertificate、featureGates.RotateKubeletServerCertificate：自动 rotate 证书，证书的有效期取决于 kube-controller-manager 的 --experimental-cluster-signing-duration 参数；
  * 需要 root 账户运行；

2.4 创建kubelet 启动服务

cat >/usr/lib/systemd/system/${KUBE_NAME}.service<<EOF
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service
[Service] 
WorkingDirectory=${K8S_INSTALL_PATH}
ExecStart=${K8S_BIN_PATH}/${KUBE_NAME} \\
  --bootstrap-kubeconfig=${KUBE_CONFIG_PATH}/kubelet-bootstrap.kubeconfig \\
  --kubeconfig=${KUBE_CONFIG_PATH}/kubelet.kubeconfig \\
  --config=${K8S_CONF_PATH}/kubelet-config.yaml \\
  --cert-dir=${CA_DIR} \\
  --hostname-override=${HOSTNAME} \\
  --pod-infra-container-image=registry.cn-beijing.aliyuncs.com/k8s_images/pause-amd64:3.1 \\
  --image-pull-progress-deadline=15m \\
  --cni-conf-dir=/etc/cni/net.d \\
  --container-runtime=docker \\
  --container-runtime-endpoint=unix:///var/run/dockershim.sock \\
  --root-dir=${K8S_INSTALL_PATH}/${KUBE_NAME} \\
  --volume-plugin-dir=${K8S_INSTALL_PATH}/${KUBE_NAME}/plugins \\
  --log-dir=${K8S_LOG_DIR}/${KUBE_NAME} \\
  --alsologtostderr=true \\
  --logtostderr=false \\
  --v=2
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF