apache+mod_jk+tomcat_cluster
时间:2008-07-21 来源:gyce
Apache + Mod_JK + Tomcat_cluster
objective:
load_balancing the jsp and servlet , session replication
Servers:
Suppose there are three servers, one is for apache and mod_JK, two of left run tomcat instance.
Apache server + mod_JK 192.168.0.1/24
Tomcat server: 192.168.0.2/3 255.255.255.0
Install procedure:
1. Install apache on 192.168.0.1 as usual. Do not forget to enable DSO support. Please refer to http://httpd.apache.org
2. Install Tomcat cluster on 192.168.0.2/3. Keep in mind that you have already set up JDK properly. Installation of tomcat is pretty easy,please refer to http://tomcat.apache.org. After installation completed successfully, modify configuration relating cluster and Engine as following to server.xml – main conf file of tomcat.
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker73">
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="192.168.0.2"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
Note that JvmRoute should be unique in the tomcat cluster,which identify the tomcat instance by mod_jk load balancing the requests , the address attribute of receiver in channel is the ip address of local network interface, which receiver the info of nodes.
Install the other tomcat server as above.
3. Enable Multicast to the NIC on which the receiver is configured.
a) check whether the multicast is enabled on the given NIC of two Tomcat server
shell>ifconfig –a
eth1 Link encap:Ethernet HWaddr 00:0F:20:32:4F:24
inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::20f:20ff:fe32:4f24/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1655417 errors:0 dropped:0 overruns:0 frame:0
TX packets:2487796 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1190134639 (1.1 GiB) TX bytes:236306869 (225.3 MiB)
Interrupt:201
If there is no such word “MULTICAST”, unfortunately, you have to update you OS kernel to support it, fortunately, most of OS support multicast by default.
b) Adding an multicast route info to the given NIC
Shell> route add -net 224.0.0.0 netmask 240.0.0.0.dev eth1
c) Here is java routine to test whether multicast works well:
Shell> cat MulticastNode.java
import java.io.*;
import java.net.*;
public class MulticastNode {
InetAddress group = null;
MulticastSocket s = null;
public static void main(String[] args){
if (args.length > 0){
System.out.println("sending message: " + args[0]);
MulticastNode node = new MulticastNode();
node.send(args[0]);
node.receive();
}
else{
System.out.println("Need an argument string to send.");
System.exit(1);
}
}
public MulticastNode() {
try{
group = InetAddress.getByName("224.0.0.1");
s = new MulticastSocket(12345);
s.joinGroup(group);
}
catch (Exception e) {
e.printStackTrace();
}
}
public void send(String msg) {
try{
DatagramPacket hi = new DatagramPacket(msg.getBytes(),msg.length(),group,12345);
s.send(hi);
}
catch (Exception e){
e.printStackTrace();
}
}
public void receive() {
byte[] buf;
while (true) {
try{
buf = new byte[1000];
DatagramPacket recv = new DatagramPacket(buf,buf.length);
s.receive(recv);
System.out.println("Received: " + new String(buf));
}
catch (Exception e){
e.printStackTrace();
}
}
}
}
After compile it with javac, running it on the first server as following:
Shell> java MulticastNode node1
then start it on the another server as below:
Shell> java MulticastNode node2
Back to the first server, if you see : Received node2
Then congratulations to you
4. Install Mod_JK
Install it according to the manual. Then configure it as following:
Shell> cat mod_jk.conf
JkWorkersFile /usr/local/apache/conf/workers.properties
# Where to put jk logs
JkShmFile /usr/local/apache/jklogs/live/mod_jk.shm
JkLogFile /usr/local/apache/jklogs/live/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel info
# Select the log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"
# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
# JkRequestLogFormat set the request format
JkRequestLogFormat "%w %V %T"
#JkMount /* loadbalancer
#JkMount /servlet/* loadbalancer
JkMount /*.jsp loadbalancer
JkMount /*.action loadbalancer
JkMount /*.do loadbalancer
JkMount /manager/* loadbalancer
JkMount /jkmanager/* jkstatus
Shell> cat workers.properties
# Defining a worker named worker72 and of type ajp13
worker.list=loadbalancer
# Set properties for worker73
worker.worker73.type=ajp13
worker.worker73.host=192.168.0.2
worker.worker73.port=8009
worker.worker73.lbfactor=50
# Set properties for worker74
worker.worker74.type=ajp13
worker.worker74.host=192.168.0.3
worker.worker74.port=8009
worker.worker74.lbfactor=50
worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=worker73,worker74
worker.loadbalancer.sticky_session=True
#
# Define a status worker
#
worker.list=jkstatus
worker.jkstatus.type=status
worker.jkstatus.read_only=true
5. start tomcat first , then start apache.