LVS集群，直接路由（VS-DR）工作模式。

LVS集群，直接路由（VS-DR）工作模式。最近因为工作需要，一直在从事这方面的研究。把找到的资料拿来与大家分享。

VS-DR is based on IBM's NetDispatcher. The NetDispatcher sits in front of a set of webservers, which appear as one webserver to the clients. The NetDispatcher served http for the Atlanta and the Sydney Olympic games and for the chess match between Kasparov and Deep Blue.

Here's an example set of IPs for a VS-DR setup. Note that in this example the RIPs are on the same network as the VIP. If the RIPs were on a different network (eg 192.168.2.0/24) to the VIP && the router (here the client) was not allowed to route to RIP network (ie the real-servers do not receive the arp requests from the router asking "who has VIP tell router"), then the arp problem is solved without requiring a patch on the director's kernel. In this example, for (my) convenience, the servers are on the same network as the client and you have to handle the arp problem (I used the arp -f /etc/ethers approach).

Host IP client CIP=192.168.1.254 director DIP=192.168.1.1 virtual IP (VIP) VIP=192.168.1.110 (arpable, IP clients connect to) real-server1 RIP1=192.168.1.2, VIP=192.168.1.110 (lo:0, not arpable) real-server2 RIP2=192.168.1.3, VIP=192.168.1.110 (lo:0, not arpable) real-server3 RIP3=192.168.1.4, VIP=192.168.1.110 (lo:0, not arpable) . . real-server-n 192.168.1.n+1 

#lvs_dr.conf LVS_TYPE=VS_DR INITIAL_STATE=on VIP=eth0:110 lvs 255.255.255.255 192.168.1.110 DIRECTOR_INSIDEIP=eth0 directorinside 192.168.1.0 255.255.255.0 192.168.1.255 DIRECTOR_DEFAULT_GW=client SERVICE=t telnet rr real-server1 real-server2 real-server3 SERVER_VIP_DEVICE=lo:0 SERVER_NET_DEVICE=eth0 SERVER_DEFAULT_GW=client #----------end lvs_dr.conf------------------------------------ 

 ________ | | | client | |________| CIP=192.168.1.254 | (router) | __________ | | | | VIP=192.168.1.110 (eth0:1, arps) | director |--- DIP=192.168.1.1 (eth0) |__________| | | | ------------------------------------- | | | | | | RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=192.168.1.4 (eth0) VIP=192.168.1.110 VIP=192.168.1.110 VIP=192.168.1.110 (all lo:0, non-arping) _____________ _____________ _____________ | | | | | | | real-server | | real-server | | real-server | |_____________| |_____________| |_____________| | | | (router) (router) (router) | | | ----------------------------------------------> to client (or router in front of director) 

Here's the lvs_dr.conf file

#--------------------lvs_dr.conf LVS_TYPE=VS_DR INITIAL_STATE=on #director setup VIP=eth0:12 192.168.1.110 255.255.255.255 192.168.1.110 DIRECTOR_INSIDEIP=eth0 192.168.1.10 192.168.1.0 255.255.255.0 192.168.1.255 #service setup, one service at a time SERVICE=t telnet rr 192.168.1.1 192.168.1.8 127.0.0.1 #real-server setup SERVER_LVS_DEVICE=lo0:1 SERVER_NET_DEVICE=eth0 #----------end lvs_dr.conf------------------------------------ 

Direct routing setup and testing is the same as VS-Tun except that all machines within the VS-DR LVS (ie the director and real-servers) must be able to arp each other. This means that they have to be on the same network without any forwarding devices between them. This means that they are using the same piece of transport layer hardware ("wire"), eg RJ-45, coax, fibre. There can be hub(s) or switch(es) in this mix. Communication within the LVS is by link-layer, using MAC addresses rather than IP's. All machines in the LVS have the VIP, only the VIP on the director replies to arp requests, the VIP on the real-servers must be on a non-arping device (eg lo:0, dummy).

The restrictions for VS-DR are

• The client must be able to connect to the VIP on the director
• Real-servers and the director must be on the same piece of wire (they must be able to arp each other) as packets are sent by link-layer from the director to the real-servers.
• The route from the real-servers to the client _cannot_ go through the director, i.e. the director cannot be the default gw for the real-servers. (Note: the client does not need to connect to the the real-servers for the LVS to function. The real-servers could be behind a firewall, but the real-servers must be able to send packets to the client). The return packets, from the real-servers to the client, go directly from the real-servers to the client and _do_not_ go back through the director. For high throughput, each real-server can have its own router/connection to the client/internet and return packets need not go through the router feeding the director. (for more info see e-mail postings about VS-DR topologies in the section More on the arp problem and topologies of VS-DR and VS-Tun LVS's). To allow the director to be the default gw for the real-servers (e.g. when the director is the firewall), see Julian's martian modification.

Note for VS-DR (and VS-Tun), the services on the real-servers are listening to the VIP. You can have the service listening to the RIP as well, but the LVS needs the service to be listening to the VIP. This is not an issue with services like telnet which listen to all local IPs (ie 0.0.0.0), but httpd is set up to listen to only the IPs that you tell it.

Normally for VS-DR, the client is on a different network to the director/server(s), and each real-server has its own route to the outside world. In the simple test case below, where all machines are on the 192.168.1.0 network, no routers are required, and the return packets, instead of going out (the router(s)) at the bottom of the diagram, would return to the client via the network device on 192.168.1.0 (presumably eth0).