nginx配合puppet实现大负载下的负载均衡

时间：2011-01-05 来源：serwei

当client数量越来越多的情况下，但puppetmaster是无法满足需求的，会导致较高的失败率，类似如time out之类的，这时候就可以做成master群集。步骤如下： 1、相信到了这一步，基本的puppetmaster之类的包肯定都安装完毕了，做成集群还需要一个特殊的包：rubygem-mongrel，安装yum install rubygem-mongrel 2、编辑/etc/sysconfig/puppetmaster，添加下面2行 PUPPETMASTER_PORTS=( 18141 18142 18143 18144 ) PUPPETMASTER_EXTRA_OPTS="--servertype=mongrel --ssl_client_header=HTTP_X_SSL_SUBJECT" 3、下载安装nginx wget http://nginx.org/download/nginx-0.8.47.tar.gz tar -zxf nginx-0.8.47.tar.gz&&cd nginx-0.8.47 ./configure --with-pcre --with-http_realip_module --with-http_sub_module --with-http_stub_status_module --with-http_ssl_module --prefix=/usr/local/nginx && make && make install 4、修改/usr/local/nginx/conf/nginx.conf如下：

user www;
worker_processes 5;

error_log /home/logs/nginx/error.log info;
pid /home/logs/nginx/nginx.pid;

events {
    use epoll;
    worker_connections 65535;
}

http {

# include mime.types;

        log_format main '$remote_addr - $remote_user [$time_local] "$request" $request_length $request_time $time_local'
                        '$status $body_bytes_sent $bytes_sent $connection $msec "$http_referer" '
                        '"$http_user_agent" $http_x_forwarded_for $upstream_response_time $upstream_addr $upstream_status ';

        sendfile on;
        tcp_nopush on;

        keepalive_timeout 60;
        tcp_nodelay on;

        upstream puppet {
             server 127.0.0.1:18141;
             server 127.0.0.1:18142;
             server 127.0.0.1:18143;
             server 127.0.0.1:18144;
         }

        server {
                listen 8140;
                root /etc/puppet;
                ssl on;
                ssl_session_timeout 5m;
                ssl_certificate /var/lib/puppet/ssl/certs/sys.puppetmaster.com.pem;
                ssl_certificate_key /var/lib/puppet/ssl/private_keys/sys.puppetmaster.com.pem;
                ssl_client_certificate /var/lib/puppet/ssl/ca/ca_crt.pem;
                ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;
                ssl_verify_client optional;

                # File sections

                location /production/file_content/files/ {
                    types { }
                    default_type application/x-raw;
                    alias /etc/puppet/manifests/files/;
                }

                # Modules files sections

                location ~ /production/file_content/modules/.+/ {
                    root /etc/puppet/modules;
                    types { }
                    default_type application/x-raw;
                    rewrite ^/production/file_content/modules/(.+)/(.+)$ /$1/files/$2 break;
                }

                # Ask the puppetmaster for everything else

                location / {
                             proxy_pass http://puppet;
                             proxy_redirect off;
                             proxy_set_header Host $host;
                             proxy_set_header X-Real-IP $remote_addr;
                             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                             proxy_set_header X-Client-Verify $ssl_client_verify;
                             proxy_set_header X-SSL-Subject $ssl_client_s_dn;
                             proxy_set_header X-SSL-Issuer $ssl_client_i_dn;
                             proxy_buffer_size 10m;
                             proxy_buffers 1024 10m;
                             proxy_busy_buffers_size 10m;
                             proxy_temp_file_write_size 10m;
                             proxy_read_timeout 120;
                }
                access_log /home/logs/nginx/puppet.access.log main;

        }
}

5、启动nginx sh /etc/init.d/nginx start

6、启动puppetmaster service puppetmaster start

如果你的client还是很多，还可以做到把puppetmaster和fileserver分开，很简单，再找台server搭个master，需要注意的是/var/lib/puppet的内容要和master一致，我是做了个rsync的定时同步的。但是fileserver貌似不能做成nginx+master的集群，会报认证失败，有待继续研究。