Perl Parallel::ForkManager模块体验

时间：2010-10-19 来源：jayamge

Perl的Parallel::ForkManager是用来管理进程创建的模块，据说它可以做到根据各个CPU当前的负载决定在哪颗CPU上fork进程。

#!/usr/bin/perl
use strict;
use warnings;
use Parallel::ForkManager;
use LWP::Simple;

my @array = (
          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Cluster_Administration-as-IN-5.2-1.el5.centos.noarch.rpm',
          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Cluster_Administration-hi-IN-5.2-1.el5.centos.noarch.rpm',
          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Global_File_System-or-IN-5.2-1.el5.centos.noarch.rpm',
          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Virtualization-it-IT-5.2-11.el5.centos.noarch.rpm',
          'http://mirrors.163.com/centos/5.5/os/x86_64/CentOS/Global_File_System-zh-CN-5.2-1.el5.centos.noarch.rpm',
          'http://121.207.250.132/my/mybt_5520_client.exe',
          'http://dl.91.com/my/my_5520_client.exe',
          'ftp://download1.91.com/my/my_5520_client.exe',
          'ftp://download2.91.com/jz/jz_3560_client.exe',
          'http://dl.91.com/ty/TY_1328_client.exe',
          'ftp://download2.91.com/kx/kx_3500_client.exe',
          'http://www.sina.com/',
          'http://www.qq.com/'
);
my $max_process = 5;
my $pm = new Parallel::ForkManager( $max_process );
foreach my $link (@array){
        $pm->start and next;
        my ($fn) = $link =~ /^.*\/(.*?)$/;
        if (!$fn) {
          warn "Can not determine filename from $fn\n";
        }else{
          $0 .= " ".$fn;
          print "Geting $fn from $link\n";
          my $cs = getstore($link,$fn);
          print "$link downloaded.reponse code:$cs\n";
        }
        $pm->finish;
}

程序中提供了一个URL文件列表，使用Parallel::ForkManager创建多个进程同时下载，最大进程数为5。用ps命令查看系统进程如下所示：
/usr/sbin/sshd 4718 0.0 0.2 10084 2860 ? Ss 02:35 0:00 \_ sshd: @ 4720 0.0 0.1 4916 1484 Ss 02:35 0:00 | \_ -bash 3301 1.8 0.4 6668 4784 S+ 03:57 0:00 | \_ perl 1.pl 3308 8.0 0.7 9632 7484 S+ 03:57 0:00 | \_ 1.pl my_5520_client.exe 3309 5.6 0.6 9168 7088 S+ 03:57 0:00 | \_ 1.pl my_5520_client.exe 3310 3.3 0.6 9168 7088 S+ 03:57 0:00 | \_ 1.pl jz_3560_client.exe 3311 7.3 0.7 9632 7484 S+ 03:57 0:00 | \_ 1.pl TY_1328_client.exe 3312 8.0 0.6 9172 7088 S+ 03:57 0:00 | \_ 1.pl kx_3500_client.exe
另外，介绍三个函数 run_on_finish 子进程finish之后调用的函数 run_on_start 子进程start之后调用的函数 run_on_wait 当子进程需要等待时自动调用的函数例程：

#!/usr/bin/perl -w

use lib '.';
use strict;
use Parallel::ForkManager;

my $max_procs = 5;
my @names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine Rico Sara );
# hash to resolve PID's back to child specific information

my $pm = new Parallel::ForkManager($max_procs);

# Setup a callback for when a child finishes up so we can

# get it's exit code

$pm->run_on_finish(
  sub { my ($pid, $exit_code, $ident) = @_;
    print "** $ident just got out of the pool ".
      "with PID $pid and exit code: $exit_code\n";
  }
);

$pm->run_on_start(
  sub { my ($pid,$ident)=@_;
    print "** $ident started, pid: $pid\n";
  }
);

$pm->run_on_wait(
  sub {
    print "** Have to wait for one children ...\n"
  },
  0.5,
);

foreach my $child ( 0 .. $#names ) {

  my $pid = $pm->start($names[$child]) and next;

  # This code is the child process

  print "This is $names[$child], Child number $child\n";
  sleep ( 2 * $child );
  print "$names[$child], Child $child is about to get out...\n";
  sleep 1;
  $pm->finish($child); # pass an exit code to finish

}

print "Waiting for Children...\n";
$pm->wait_all_children;
print "Everybody is out of the pool!\n";