lvs中涉及到的arp_ignore与arp_announce理解

摘要： 对于这两个参数的英文文字说明，arp_ignore相对好理解，但arp_announce一直没有找到很好的解释和翻译，于是乎自己试验了一番：arp_announce(向局域网中发送...

对于这两个参数的英文文字说明，arp_ignore相对好理解，但arp_announce一直没有找到很好的解释和翻译，于是乎自己试验了一番：
arp_announce(向局域网中发送arp请求时,如何来选择源IP)

 Define different restriction levels for announcing the local source IP address from IP packets in ARP requests sent on interface:

默认为0

• 0 - (default) Use any local address, configured on any interface.

• 1 - Try to avoid local addresses that are not in the target's subnet for this interface. This mode is useful when target hosts reachable via this interface require the source IP address in ARP requests to be part of their logical network configured on the receiving interface. When we generate the request we will check all our subnets that include the target IP and will preserve the source address if it is from such subnet. If there is no such subnet we select source address according to the rules for level 2.

• 2 - Always use the best local address for this target. In this mode we ignore the source address in the IP packet and try to select local address that we prefer for talks with the target host. Such local address is selected by looking for primary IP addresses on all our subnets on the outgoing interface that include the target IP address. If no suitable local address is found we select the first local address we have on the outgoing interface or on all other interfaces, with the hope we will receive reply for our request and even sometimes no matter the source IP address we announce.

The max value from conf/{all,interface}/arp_announce is used.

Increasing the restriction level gives more chance for receiving answer from the resolved target while decreasing the level announces more valid sender's information.

===========================================
试验：
[root@vm57-120 keepalived]# ip a
1: lo: mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:6a:8d:5d brd ff:ff:ff:ff:ff:ff
    inet 192.168.57.120/24 brd 192.168.57.255 scope global eth0
    inet 192.168.57.126/24 brd 192.168.57.255 scope global secondary eth0:126
    inet6 fe80::20c:29ff:fe6a:8d5d/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:6a:8d:67 brd ff:ff:ff:ff:ff:ff
    inet 10.10.57.120/24 brd 10.10.57.255 scope global eth1
    inet6 fe80::20c:29ff:fe6a:8d67/64 scope link 
       valid_lft forever preferred_lft forever
4: sit0: mtu 1480 qdisc noop 
    link/sit 0.0.0.0 brd 0.0.0.0

[root@vm57-123 ~]#ip addr add 192.168.57.123/24 brd 192.168.57.255 dev eth1 label eth1:1
#如果不加这个ip，无论如何都是ping不通192.168.57.126的。加了，才“有可能”ping通.这个ip设置在lo接口亦可。
[root@vm57-123 ~]# ip a
1: lo: mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:58:34:c1 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe58:34c1/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:58:34:cb brd ff:ff:ff:ff:ff:ff
    inet 10.10.57.123/24 brd 10.10.57.255 scope global eth1
    inet 192.168.57.123/24 brd 192.168.57.255 scope global eth1:1
    inet6 fe80::20c:29ff:fe58:34cb/64 scope link 
       valid_lft forever preferred_lft forever
4: sit0: mtu 1480 qdisc noop 
    link/sit 0.0.0.0 brd 0.0.0.0

[root@vm57-123 ~]# ip nei
10.10.57.1 dev eth1 lladdr 00:50:56:c0:00:01 REACHABLE
10.10.57.120 dev eth1 lladdr 00:0c:29:6a:8d:67 REACHABLE

[root@vm57-120 keepalived]# ip nei
192.168.57.135 dev eth0  FAILED
10.10.57.122 dev eth1 lladdr 00:0c:29:b1:97:8c REACHABLE
10.10.57.123 dev eth1 lladdr 00:0c:29:58:34:cb REACHABLE
192.168.57.1 dev eth0 lladdr 00:0f:e2:8f:dc:cd REACHABLE

[root@vm57-123 ~]# ping 192.168.57.126 -I 10.10.57.123
PING 192.168.57.126 (192.168.57.126) from 10.10.57.123 : 56(84) bytes of data.
64 bytes from 192.168.57.126: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 192.168.57.126: icmp_seq=2 ttl=64 time=0.018 ms

[root@vm57-120 keepalived]# tcpdump -vnnN -e icmp -i eth1
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
17:44:15.703435 00:0c:29:58:34:cb > 00:0c:29:6a:8d:67, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 10.10.57.123 > 192.168.57.126: ICMP echo request, id 13419, seq 148, length 64
17:44:15.703625 00:0c:29:6a:8d:67 > 00:0c:29:58:34:cb, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 6784, offset 0, flags [none], proto: ICMP (1), length: 84) 192.168.57.126 > 10.10.57.123: ICMP echo reply, id 13419, seq 148, length 64
17:44:16.703715 00:0c:29:58:34:cb > 00:0c:29:6a:8d:67, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 10.10.57.123 > 192.168.57.126: ICMP echo request, id 13419, seq 149, length 64

192.168.57.120机器将通过eth1端口将icmp响应包返回给10.10.57.123. 本来首选应该是从57.120的eth0端口返回arp响应包的（可以抓到eth0发出的询问192.168.57.123ip的arp请求包），但是因为57.123机器的eth0端口没有这个ip，所以只能通过eth1端口。

如果123的eth0和lo都设置了192.168.57网段的ip，那么会优先选择eth0出口上的ip作为arp里的源ip。


-------------------
[root@vm57-123 ~]# ip a
1: lo: mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.57.127/32 scope global lo:0
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:58:34:c1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.57.128/24 brd 192.168.57.255 scope global eth0:1
    inet6 fe80::20c:29ff:fe58:34c1/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:58:34:cb brd ff:ff:ff:ff:ff:ff
    inet 10.10.57.123/24 brd 10.10.57.255 scope global eth1
    inet 192.168.57.123/24 brd 192.168.57.255 scope global eth1:1
    inet6 fe80::20c:29ff:fe58:34cb/64 scope link 
       valid_lft forever preferred_lft forever
4: sit0: mtu 1480 qdisc noop 
    link/sit 0.0.0.0 brd 0.0.0.0
[root@vm57-123 ~]# ping 192.168.57.126
在本地抓包可以看到是这样的情况：
tcpdump -vnnN -e icmp -i eth0 
tcpdump -vnnN -e icmp -i eth1

[root@vm57-123 ~]# tcpdump -vnnN -e icmp -i eth0
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
11:26:41.556489 00:0c:29:6a:8d:5d > 00:0c:29:58:34:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 17076, offset 0, flags [none], proto: ICMP (1), length: 84) 192.168.57.126 > 192.168.57.123: ICMP echo reply, id 42533, seq 1, length 64

[root@vm57-123 ~]# tcpdump -vnnN -e icmp -i eth1
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
11:26:57.561846 00:0c:29:58:34:cb > 00:0c:29:6a:8d:67, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 192.168.57.123 > 192.168.57.126: ICMP echo request, id 42533, seq 17, length 64

可以看到，从123发出的arp数据帧的源ip是192.168.57.123，而不是192.168.57.128. 并且出去的arp request帧是从本地的eth1接口，返回的arp reply则是从eth0接口接收。
120机器的arp缓存是以123机器的eth0的mac地址：
192.168.57.123 dev eth0 lladdr 00:0c:29:58:34:c1 REACHABLE
目前对于为什么会选择192.168.57.123作为arp帧中的源ip的原因不清楚。继续做实验吧。


如果指定接口或者源ip，则都会从eth0接口进出：
[root@vm57-123 ~]# ping 192.168.57.126 -I eth0
[root@vm57-123 ~]# ping 192.168.57.126 -I 192.168.57.128

[root@vm57-123 ~]# tcpdump -vnnN -e icmp -i eth0
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
13:37:38.513715 00:0c:29:58:34:c1 > 00:0c:29:6a:8d:5d, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 192.168.57.128 > 192.168.57.126: ICMP echo request, id 1846, seq 13, length 64
13:37:38.514509 00:0c:29:6a:8d:5d > 00:0c:29:58:34:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 27249, offset 0, flags [none], proto: ICMP (1), length: 84) 192.168.57.126 > 192.168.57.128: ICMP echo reply, id 1846, seq 13, length 64

2）现在将192.168.57.123、192.168.57.128ip地址的位置调换一下，如下：
[root@vm57-123 ~]# ip a
1: lo: mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.57.127/32 scope global lo:0
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:58:34:c1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.57.123/24 scope global eth0
    inet6 fe80::20c:29ff:fe58:34c1/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:58:34:cb brd ff:ff:ff:ff:ff:ff
    inet 10.10.57.123/24 brd 10.10.57.255 scope global eth1
    inet 192.168.57.128/24 scope global eth1:1
    inet6 fe80::20c:29ff:fe58:34cb/64 scope link 
       valid_lft forever preferred_lft forever
此时再去ping 192.168.57.126，则是属于正常情况了。
[root@vm57-123 ~]# ping 192.168.57.126

[root@vm57-123 ~]# tcpdump -vnnN -e icmp -i eth0
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
14:48:39.928214 00:0c:29:58:34:c1 > 00:0c:29:6a:8d:5d, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 192.168.57.123 > 192.168.57.126: ICMP echo request, id 319, seq 29, length 64
14:48:39.928410 00:0c:29:6a:8d:5d > 00:0c:29:58:34:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl  64, id 4614, offset 0, flags [none], proto: ICMP (1), length: 84) 192.168.57.126 > 192.168.57.123: ICMP echo reply, id 319, seq 29, length 64

此次arp帧中的源ip依旧选择了192.168.57.123

下面在eth1接口再添加192.168.57.102/24，结果跟上面的一样。