kubeadm使用外部etcd集群tls部署kubernetes-创新互联

环境：ubuntu 16.04.2

专注于为中小企业提供成都做网站、成都网站建设服务,电脑端+手机端+微信端的三站合一,更高效的管理,为中小企业内黄免费做网站提供优质的服务。我们立足成都，凝聚了一批互联网行业人才，有力地推动了上千余家企业的稳健成长，帮助中小企业通过网站建设实现规模扩充和转变。

cpu 4  内存 8G

内核4.4.0-119

ip地址：192.168.0.62

192.168.0.63

192.168.0.64

etcd版本： 3.2.12

kubernetes版本：1.11.5

一、部署etcd集群(需要sudo或者root权限)

1生成证书及etcd的二进制文件包，工具下载地址

wget -O /bin/cfssl  https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 

wget -O /bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 

chmod +x /bin/cfssl*

wget https://github.com/etcd-io/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz

tar xf etcd-v3.2.18-linux-amd64.tar.gz

cp etcd-v3.2.18-linux-amd64/etcd* /usr/bin/

2.ca-config.json配置文件，修改过期时间为10年（红色部分）

内容如下：

{

"signing": {

"default": {

"expiry": "87600h"

},

"profiles": {

"etcd": {

"expiry": "87600h",

"usages": [

"signing",

"key encipherment",

"client auth",

"server auth"

]

}

}

}

}

3.ca-csr.json配置文件如下：

{

"CN": "etcd",

"key": {

"algo": "rsa",

"size": 2048

},

"names": [

{

"C": "CN",

"ST": "shanghai",

"L": "shanghai",

"O": "etcd",

"OU": "System"

}

]

}

4.etcd集群的etcd-csr.json

{

"CN": "etcd",

"hosts": [

"127.0.0.1",

"192.168.0.62",

"192.168.0.63",

"192.168.0.64"

],

"key": {

"algo": "rsa",

"size": 2048

},

"names": [

{

"C": "CN",

"ST": "shanghai",

"L": "shanghai",

"O": "etcd",

"OU": "System"

}

]

}

4.生成证书并自签名

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd

复制pem文件到你指定的目录，3台主机都要复制的

不建议使用目录/etc/kubernetes/pki/etcd

mkdir -p /etc/etcdCA

cp *.pem /etc/etcdCA

5.etcd的配置文件如下，红色部分自行更改。复制配置文件为/etc/default/etcd

ETCD_NAME=test-node62

ETCD_DATA_DIR="/var/lib/etcd/"

ETCD_LISTEN_PEER_URLS="https://192.168.0.62:2380"

ETCD_LISTEN_CLIENT_URLS="https://192.168.0.62:2379,https://127.0.0.1:4001"

ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.62:2380"

ETCD_INITIAL_CLUSTER="test-node62=https://192.168.0.62:2380,test-node63=https://192.168.0.63:2380,test-node64=https://192.168.0.64:2380"

ETCD_INITIAL_CLUSTER_STATE="new"

ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-sdn"

ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.62:2379"

CLIENT_CERT_AUTH="true"

ETCD_CA_FILE="/etc/etcdCA/ca.pem"

ETCD_CERT_FILE="/etc/etcdCA/etcd.pem"

ETCD_KEY_FILE="/etc/etcdCA/etcd-key.pem"

PEER_CLIENT_CERT_AUTH="true"

ETCD_PEER_CA_FILE="/etc/etcdCA/ca.pem"

ETCD_PEER_CERT_FILE="/etc/etcdCA/etcd.pem"

ETCD_PEER_KEY_FILE="/etc/etcdCA/etcd-key.pem"

6.创建用户和服务并授权

useradd etcd

chmod 755 /etc/etcdCA/*

echo '[Unit]

Description=etcd - highly-available key value store

Documentation=https://github.com/coreos/etcd

Documentation=man:etcd

After=network.target

Wants=network-online.target

[Service]

Environment=DAEMON_ARGS=

Environment=ETCD_NAME=%H

Environment=ETCD_DATA_DIR=/var/lib/etcd/default

EnvironmentFile=-/etc/default/%p

Type=notify

User=etcd

PermissionsStartOnly=true

#ExecStart=/bin/sh -c "GOMAXPROCS=$(nproc) /usr/bin/etcd $DAEMON_ARGS"

ExecStart=/usr/bin/etcd $DAEMON_ARGS

Restart=on-abnormal

#RestartSec=10s

#LimitNOFILE=65536

[Install]

WantedBy=multi-user.target

Alias=etcd3.service'   >/lib/systemd/system/etcd.service

7.启动服务

systemctl start etcd

8.检查集群状态

export ETCDCTL_API=3 etcdctl  \   --cacert=/etc/etcdCA/ca.pem \   --cert=/etc/etcdCA/etcd.pem \   --key=/etc/etcdCA/etcd-key.pem \   --endpoints=192.168.0.62:2379,192.168.0.63:2379,192.168.0.64:2379 \   endpoint health

看到下图就ok了

二、部署kubernetes

1. 安装docker-ce （18.06.3）

sudo apt-get update sudo apt-get install \     apt-transport-https \     ca-certificates \     curl \     software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \    $(lsb_release -cs) \    stable" sudo apt-get update sudo apt-get install docker-ce=18.06.3~ce~3-0~ubuntu

2.安装kubernetes包

apt-get update && apt-get install -y apt-transport-https curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -  cat </etc/apt/sources.list.d/kubernetes.list deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main EOF apt-get install -y kubelet=1.11.5-00 kubeadm=1.11.5-00 kubectl=1.11.5-00

3.使用配置文件进行安装，配置文件（kubeadm-config.yaml）如下

apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration networking:   podSubnet: 172.16.0.0/16   serviceSubnet: 10.96.0.0/12 etcd:   endpoints:   - https://192.168.0.62:2379   - https://192.168.0.63:2379   - https://192.168.0.64:2379   caFile: /etc/etcdCA/ca.pem   certFile: /etc/etcdCA/etcd.pem   keyFile: /etc/etcdCA/etcd-key.pem kubernetesVersion: v1.11.5 kubeProxy:   config:     mode: "ipvs"

4.加载需要的kubernetes镜像

A="kube-proxy-amd64:v1.11.5 kube-apiserver-amd64:v1.11.5 kube-controller-manager-amd64:v1.11.5 kube-scheduler-amd64::v1.11.5 pause:3.1" for i in $A;do docker pull mirrorgooglecontainers/$i docker tag mirrorgooglecontainers/$i k8s.gcr.io/$i done docker pull coredns/coredns:1.1.3 docker tag coredns/coredns:1.1.3 k8s.gcr.io/coredns:1.1.3

5.安装master,出现下图就master的安装好了

kubeadm init --config /path/kubeadm-config.yaml

6.授权客户端访问

mkdir -p $HOME/.kube

sudo cp -f /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

7.安装客户端（请先执行1.安装docker-ce 2.kubernetes包和4.加载需要的kubernetes镜像）

执行master生成后的kubeadm jion ，需要root或者sudo权限

如上图是：

kubeadm join 192.168.0.62:6443 --token 4msj6v.plj3rcsq89c4y4mn --discovery-token-ca-cert-hash sha256:7fb655510bc0af2dda7e401a45932709c473b0f33acef0794924b54715512bbc

三、安装calico插件

wget https://github.com/projectcalico/calico/releases/download/v2.6.12/release-v2.6.12.tgz tar xf release-v2.6.12.tgz cd release-v2.6.12/k8s-manifests/hosted sed -i 's?http://127.0.0.1:2379?https://192.168.0.62:2379,https://192.168.0.63:2379,https://192.168.0.64:2379?g' calico.yaml cat /etc/etcdCA/etcd-key.pem|base64 -w 0 > ETCD-KEY cat /etc/etcdCA/ca.pem|base64 -w 0 > ETCD-CA cat /etc/etcdCA/etcd.pem|base64 -w 0 > ETCD-CERT sed -i "s?# etcd-key: null?etcd-key: $(cat ETCD-KEY)?g" calico.yaml sed -i "s?# etcd-ca: null?etcd-ca: $(cat ETCD-CA)?g" calico.yaml sed -i "s?# etcd-cert: null?etcd-cert: $(cat ETCD-CERT)?g" calico.yaml sed -i 's?etcd_ca: ""?etcd_ca: "/calico-secrets/etcd-ca"?g' calico.yaml sed -i 's?etcd_cert: ""?etcd_cert: "/calico-secrets/etcd-cert"?g' calico.yaml sed -i 's?etcd_key: ""?etcd_key: "/calico-secrets/etcd-key"?g' calico.yaml kubectl apply -f calico.yaml kubectl apply -f rbac-kdd.yaml

四、查看状态

至此k8s的基础部分完成

补充calico 3.10部分

wget https://github.com/projectcalico/calico/releases/download/v3.10.2/release-v3.10.2.tgz tar xf release-v3.10.2.tgz cd release-v3.10.2/k8s-manifests sed -i 's?http://:?https://192.168.0.62:2379,https://192.168.0.63:2379,https://192.168.0.64:2379?g' calico-etcd.yaml cat /etc/etcdCA/etcd-key.pem|base64 -w 0 > ETCD-KEY cat /etc/etcdCA/ca.pem|base64 -w 0 > ETCD-CA cat /etc/etcdCA/etcd.pem|base64 -w 0 > ETCD-CERT sed -i "s?# etcd-key: null?etcd-key: $(cat ETCD-KEY)?g" calico-etcd.yaml sed -i "s?# etcd-ca: null?etcd-ca: $(cat ETCD-CA)?g" calico-etcd.yaml sed -i "s?# etcd-cert: null?etcd-cert: $(cat ETCD-CERT)?g" calico-etcd.yaml sed -i 's?etcd_ca: ""?etcd_ca: "/calico-secrets/etcd-ca"?g' calico-etcd.yaml sed -i 's?etcd_cert: ""?etcd_cert: "/calico-secrets/etcd-cert"?g' calico-etcd.yaml sed -i 's?etcd_key: ""?etcd_key: "/calico-secrets/etcd-key"?g' calico-etcd.yaml sed -i 's?192.168.0.0/16?172.16.0.0/16?g' calico-etcd.yaml kubectl apply -f calico-etcd.yaml

注意：加密的etcd集群和明文的etcd集群不能通用

另外有需要云服务器可以了解下创新互联cdcxhl.cn，海内外云服务器15元起步，三天无理由+7*72小时售后在线，公司持有idc许可证，提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案，具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势，专为企业上云打造定制，能够满足用户丰富、多元化的应用场景需求。

当前文章：kubeadm使用外部etcd集群tls部署kubernetes-创新互联
文章网址：http://cdkjz.cn/article/sppjh.html