Flannel安装配置

安装Flannel

flannel下载地址：https://github.com/coreos/flannel/releases

sudo yum install -y flannel

添加flannel网络配置信息到etcd

curl http://127.0.0.1:2379/v2/keys/coreos.com/network/config
{"errorCode":100,"message":"Key not found","cause":"/coreos.com","index":12}

curl -X PUT http://127.0.0.1:2379/v2/keys/coreos.com/network/config -d value='{ "Network": "172.17.0.0/16" }'

curl http://127.0.0.1:2379/v2/keys/coreos.com/network/config
{"action":"get","node":{"key":"/coreos.com/network/config","value":"{ \"Network\": \"172.17.0.0/16\" }","modifiedIndex":13,"createdIndex":13}}

有两种方法可以设置，但是flannel 0.7.1版默认使用的是etcd V2的http接口，所以如果etcd是V3版本而且用第一种方法设置那么会导致fannel读取不到配置，运行flannel会报错failed to retrieve network config: 100: Key not found (/coreos.com)；如果ectd是V2版本，那么两种方法设置都可以。

同时，如果出现类似的报错，可以使用下面的命令测试

curl http://172.0.0.1:2379/v2/keys/coreos.com/network/config

创建日志目录

sudo mkdir -p /var/log/flannel

修改flanneld配置

查看flanne服务

cat /usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS
ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure

[Install]
WantedBy=multi-user.target
WantedBy=docker.service

修改flannel配置文件

vim /etc/sysconfig/flanneld
修改为

FLANNEL_ETCD_ENDPOINTS="http://etcd-cluser-ip:2379,http://etcd-cluser-ip:2379,http://etcd-cluser-ip:2379"
FLANNEL_ETCD_PREFIX="/coreos.com/network"
FLANNEL_OPTIONS="--logtostderr=false --log_dir=/var/log/flannel/ --iface=你的网卡名字"

启动

sudo systemctl daemon-reload
sudo systemctl start flanneld
sudo systemctl status flanneld
sudo systemctl enable flanneld

配置Docker

Flanneld服务需要先于Docker启动，此时应该停掉Docker，待配置完之后重启。

配置好Flanneld之后它会从etcd中获取network的配置信息，然后划分subnet并在etcd中进行注册，然后将子网信息记录到/run/flannel/subnet.env中，最后将subnet.env转写成一个docker的环境变量文件/run/flannel/docker。

可以通过以下命令查看相关文件信息：

cat /run/flannel/subnet.env
FLANNEL_NETWORK=172.17.0.0/16
FLANNEL_SUBNET=172.17.75.1/24
FLANNEL_MTU=1472
FLANNEL_IPMASQ=false
[root@localhost ~]# cat /run/flannel/docker
DOCKER_OPT_BIP="--bip=172.17.75.1/24"
DOCKER_OPT_IPMASQ="--ip-masq=true"
DOCKER_OPT_MTU="--mtu=1472"
DOCKER_NETWORK_OPTIONS=" --bip=172.17.75.1/24 --ip-masq=true --mtu=1472"

此外，Flanneld还会自动修改Docker的配置

systemctl show docker
...
DropInPaths=/usr/lib/systemd/system/docker.service.d/flannel.conf
...

需要修改/usr/lib/systemd/system/docker.service，并应用/run/flannel/docker当中的DOCKER_NETWORK_OPTIONS配置：

vi /usr/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock $DOCKER_NETWORK_OPTIONS

重启Docker:

sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl status docker

测试安装结果

ip addr show
3: flannel0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1472 qdisc pfifo_fast state UNKNOWN group default qlen 500
   link/none
   inet 172.17.75.0/16 scope global flannel0
      valid_lft forever preferred_lft forever

测试容器互通

两台机器启动docker 容器互ping

docker run -it busybox
/ # ip a

13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1472 qdisc noqueue
   link/ether 02:42:ac:11:3f:03 brd ff:ff:ff:ff:ff:ff
   inet 172.17.63.3/24 brd 172.17.63.255 scope global eth0

docker run -it busybox
/ # ip a

5: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1472 qdisc noqueue
   link/ether 02:42:ac:11:4b:02 brd ff:ff:ff:ff:ff:ff
   inet 172.17.75.2/24 brd 172.17.75.255 scope global eth0

/ # ping 172.17.63.3
PING 172.17.63.3 (172.17.63.3): 56 data bytes
64 bytes from 172.17.63.3: seq=0 ttl=60 time=0.689 ms

容器不能互通的问题 =

遇到的问题是目标端flannel0上有包发过来，但docker0网段没有任何包。

所以定位是目标段的flannel0->docker0的转发出了问题。

通过iptables -nvL 查看现有的iptables规则，发现

chain FORWARD链路 policy是DROP,以下命令修改

iptables -P FORWARD ACCEPT