Flannel 是CNI阵营里标准的网络插件,在没有严格的网络的要求下,它通常作为kubernetes集群里网络互联比较常用的方案,当然还有其他网络方案,比如calico,weave,mxvlan等等,这里主要使用到flannel作为kubernetes的网络方案;
创新互联公司专注于企业网络营销推广、网站重做改版、老城网站定制设计、自适应品牌网站建设、成都h5网站建设、商城系统网站开发、集团公司官网建设、外贸网站建设、高端网站制作、响应式网页设计等建站业务,价格优惠性价比高,为老城等各大城市提供网站开发制作服务。
kubernetes 要求集群内各节点(包括 master 节点)能通过 Pod 网段互联互通。flannel 使用 vxlan 技术为各节点创建一个可以互通的 Pod 网络,使用的端口为 UDP 8472(需要开放该端口,如公有云 AWS 等)。
flanneld 第一次启动时,从 etcd 获取配置的 Pod 网段信息,为本节点分配一个未使用的地址段,然后创建 flannedl.1 网络接口(也可能是其它名称,如 flannel1 等)。
flannel 将分配给自己的 Pod 网段信息写入 /run/flannel/docker 文件,docker 后续使用这个文件中的环境变量设置 docker0 网桥,从而从这个地址段为本节点的所有 Pod 容器分配 IP。
#################### Variable parameter setting ######################
FLANNEL_INSTALL_PATH=/data/apps/k8s/networks/flannel
SOFTWARE=/root/software
VERSION=v0.11.0
PACKAGE=flannel-${VERSION}-linux-amd64.tar.gz
DOWNLOAD_URL=https://github.com/devops-apps/download/raw/master/network/$PACKAGE
ETCD_ENPOINTS=https://10.10.10.22:2379,https://10.10.10.23:2379,https://10.10.10.24:2379
FLANNEL_ETCD_PREFIX=/k8s/network
CA_DIR=/etc/k8s/ssl
NETWORK_SUBNET=172.16.0.0/20
IFACE=eth0
登陆devops机器,访问flannel github 官方地址下载稳定的 realease 包至本机;
wget $DOWNLOAD_URL -P $SOFTWARE
将flannel 软件包分发到集群所有节点服务器;
sudo ansible master_k8s_vgs -m copy -a "src=${SOFTWARE}/$PACKAGE dest=${SOFTWARE}/" -b
sudo ansible worker_k8s_vgs -m copy -a "src=${SOFTWARE}/$PACKAGE dest=${SOFTWARE}/" -b
### 1.Check if the install directory exists.
if [ ! -d $FLANNEL_INSTALL_PATH/bin ]; then
mkdir -p $FLANNEL_INSTALL_PATH/bin
fi
### 2.Install binary of flannel.
if [ ! -f "$SOFTWARE/flannel-${VERSION}-linux-amd64.tar.gz" ]; then
wget $DOWNLOAD_URL -P $SOFTWARE >>/tmp/install.log 2>&1
fi
cd $SOFTWARE && tar -xzf flannel-${VERSION}-linux-amd64.tar.gz -C ./
cp -fp ${SOFTWARE}/{flanneld,mk-docker-opts.sh} ${FLANNEL_INSTALL_PATH}/bin
ln -sf ${FLANNEL_INSTALL_PATH}/bin/{flanneld,mk-docker-opts.sh} /usr/local/bin
chmod -R 755 $FLANNEL_INSTALL_PATH
cd ${CA_DIR}
sudo ansible master_k8s_vgs -m copy -a "src=ca.pem dest=${CA_DIR}/" -b
sudo ansible master_k8s_vgs -m copy -a "src=flannel.pem dest=${CA_DIR}/" -b
sudo ansible master_k8s_vgs -m copy -a "src=flannel-key.pem dest=${CA_DIR}/" -b
### Create network subnet of flannel .
etcdctl --endpoint=$ETCD_ENPOINTS \
--ca-file=${CA_DIR}/ca.pem \
--cert-file=${CA_DIR}/etcd.pem \
--key-file=${CA_DIR}/etcd-key.pem \
mkdir $FLANNEL_ETCD_PREFIX
etcdctl --endpoints=$ETCD_ENPOINTS \
--ca-file=${CA_DIR}/ca.pem \
--cert-file=${CA_DIR}/etcd.pem \
--key-file=${CA_DIR}/etcd-key.pem \
mk ${FLANNEL_ETCD_PREFIX}/config \
'{"Network":"'${NETWORK_SUBNET}'","SubnetLen":24,"Backend":{"Type":"$TYPE"}}'
cat >/usr/lib/systemd/system/flanneld.service<
启动flannel服务并检查
sudo systemctl start flanneld
sudo systemctl status flanneld |grep 'Active:'
确保状态为 active (running),否则查看日志,确认原因:
sudo journalctl -u flanneld
### Create network subnet of flannel .
etcdctl --endpoint=$ETCD_ENPOINTS \
--ca-file=${CA_DIR}/ca.pem \
--cert-file=${CA_DIR}/etcd.pem \
--key-file=${CA_DIR}/etcd-key.pem \
get ${FLANNEL_ETCD_PREFIX}/config
输出:
{"Network":"172.20.0.0/20", "SubnetLen": 21, "Backend": {"Type": "vxlan"}}
查看已分配的 Pod 子网段列表(/24):
etcdctl --endpoints=$ETCD_ENPOINTS \
--ca-file=${CA_DIR}/ca.pem \
--cert-file=${CA_DIR}/etcd.pem \
--key-file=${CA_DIR}/etcd-key.pem \
ls ${FLANNEL_ETCD_PREFIX}/subnets
输出(结果视部署情况而定):
/k8s/network/subnets/172.16.1.0-24
/k8s/network/subnets/172.16.3.0-24
/k8s/network/subnets/172.16.13.0-24
/k8s/network/subnets/172.16.15.0-24
/k8s/network/subnets/172.16.8.0-24
查看某一 Pod 网段对应的节点 IP 和 flannel 接口地址:
etcdctl --endpoints=$ETCD_ENPOINTS \
--ca-file=${CA_DIR}/ca.pem \
--cert-file=${CA_DIR}/etcd.pem \
--key-file=${CA_DIR}/etcd-key.pem \
ls ${FLANNEL_ETCD_PREFIX}/subnets
输出(结果视部署情况而定):
{"PublicIP":"192.168.20.24","BackendType":"vxlan","BackendData":{"VtepMAC":"a6:92:04:07"}}
随机登陆一台master节点,这里以master-ks8-n03为例
$ ifconfig
eth0: flags=4163 mtu 1500
inet 192.168.20.24 netmask 255.255.255.0 broadcast 192.168.20.255
ether 00:50:56:b9:9c:b1 txqueuelen 1000 (Ethernet)
RX packets 305607 bytes 52151972 (49.7 MiB)
RX errors 0 dropped 1176 overruns 0 frame 0
TX packets 355321 bytes 58794327 (56.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth2: flags=4163 mtu 1500
inet 10.10.10.24 netmask 255.255.255.0 broadcast 10.10.10.255
ether 00:50:56:b9:2c:1e txqueuelen 1000 (Ethernet)
RX packets 216915716 bytes 36273314852 (33.7 GiB)
RX errors 0 dropped 1173 overruns 0 frame 0
TX packets 207708912 bytes 33334846046 (31.0 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
flannel.1: flags=4163 mtu 1450
inet 172.16.1.0 netmask 255.255.255.255 broadcast 0.0.0.0
ether a6:92:cc:85:04:07 txqueuelen 0 (Ethernet)
RX packets 260862 bytes 27280155 (26.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 309429 bytes 36792200 (35.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
查看路由
$ ip route show |grep flannel.1
172.16.3.0/24 via 172.16.3.0 dev flannel.1 onlink
172.16.8.0/24 via 172.16.8.0 dev flannel.1 onlink
172.16.13.0/24 via 172.16.13.0 dev flannel.1 onlink
172.16.15.0/24 via 172.16.15.0 dev flannel.1 onlink
在各节点上部署 flannel 后,检查是否创建了 flannel 接口(名称可能为 flannel0、flannel.0、flannel.1 等):这里为flannel.1
ansible master_k8s_vgs -m shell -a ''ping 172.16.1.1 -c 3 && ping 172.16.3.1 -c 3 \
&& ping 172.16.8.1 -c 3 && ping 172.16.13.1 -c 3 && ping 172.16.15.1 -c 3"
上述命令在devops机器上执行。正常情况下,到集群内每个节点都是互通的,如果有不通情况下,检查防火墙设置;
安装完flannel网络插件后,整个集群到这里算大功告成,后面还需要安装相关插件,如dashboard可视化,DNS解析,以及监控等等;关于flannel脚本可以从此处获取