NAME
File::Find::Parallel - Traverse a number of similar directories in
parallel
VERSION
This document describes File::Find::Parallel version 0.52
SYNOPSIS
use File::Find::Parallel;
my $ffp = File::Find::Parallel->new( qw( /foo /bar ) );
print "Union:\n";
my $union = $ffp->any_iterator
print " $_\n" while $_ = $union->();
print "Intersection:\n";
my $inter = $ffp->all_iterator
print " $_\n" while $_ = $inter->();
DESCRIPTION
File::Find is the ideal tool for quickly scanning a single directory.
But sometimes it's nice to be able to perform operations on multiple
similar directories in parallel. Perhaps you need to compare the
contents of two directories or convert files that are shared in more
than one directory into hard links.
This module manufactures iterators that visit each file and directory in
either the union or the intersection of a number of directories. Hmm.
What does that mean?
Given two directory trees like this
foo
foo/a
foo/b/c
foo/d
bar
bar/a
bar/b
bar/e
you can choose to work with the intersection of the two directory
structures:
.
./a
./b
That is the subdirectories and files that the foo and bar share.
Alternately you can work with the union of the two directory structures:
.
./a
./b
./b/c
./d
./e
Still not clear? Well, if you wanted to do a recursive diff on the two
directories you'd iterate their union so you could report files that
were present in foo but missing from bar and vice-versa.
If, on the other hand you wanted to scan the directories and find all
the files that are common to all of them you'd iterate their
intersection and receive only files and directories that were present in
all the directories being scanned.
The "any_iterator" and "all_iterator" are built on a more general
purpose method: "want_iterator". If, for example, you want to make links
between files that are found in more than one directory you might get
your iterator like this:
my $iter = $ffp->want_iterator( 2 );
The apparently magic '2' reflects the fact that if you're going to be
making links you need at least two files. No matter how many directories
you are iterating over in parallel you will only see files and
directories that appear in at least two of those directories.
File::Find::Parallel can scan any number of directories at the same
time. Here's an example (on Unix systems) that returns the list of all
files and directories that are contained in all home directories.
use File::Glob ':glob';
use File::Find::Parallel;
my $find = File::Find::Parallel->new( bsd_glob( '/home/*' ) );
my @common = ( );
my $iter = $find->all_iterator;
while ( defined my $obj = $iter->() ) {
push @common, $obj;
}
print "The following files are common to ",
"all directories below /home :\n";
print " $_\n" for @common;
For a complete concrete example of its use see lncopies in the "bin"
subdirectory of this distribution.
Iterators
The iterator returned by "any_iterator", "all_iterator" or
"want_iterator" is a code reference. Call it to get the next file or
directory. When all files and directories have been returned the
iterator will return "undef".
Once created an iterator is independent of the File::Find::Parallel
object that created it. If the object goes out of scope and is destroyed
during the life of the iterator it will still function normally.
You may have many active iterators for a single File::Find::Parallel
object at any time.
INTERFACE
"new"
Create a new File::Find::Parallel. You may optionally pass a list of
directories to scan.
"set_dirs( @dirs )"
Set the list of directories to be scanned. Any number of directories
may be scanned. If you are scanning just a single directory consider
using File::Find instead.
"get_dirs"
Get the list of directories to be scanned.
my @dirs_to_scan = $ffp->get_dirs;
"add_dirs"
Add to the list of directories to be scanned.
$ffp->add_dirs( 'a' );
$ffp->add_dirs( 'b', 'c' );
"any_iterator"
Get an iterator that will return the names of all the files and
directories that are in the union of the directories to be scanned.
The returned iterator is a code reference that returns a new name
each time it is called. It returns "undef" when all names have been
returned.
The returned names are relative to the base directories. Given
directories like this
foo bar
foo/a bar/a
foo/b/c bar/d/e
the iterator would return
.
a
b
d
b/c
d/e
That is it returns the list of names that would result if foo was
copied over bar and then bar scanned. Note that the starting
directory '.' is returned.
Directories are searched in breadth first order.
"all_iterator"
Get an iterator that will return the names of all the files and
directories that are in the intersection of the directories to be
scanned.
Given directories like this
foo bar
foo/a bar/a
foo/b/c bar/d/e
the iterator would return
.
a
That is it returns the names of those files and directories that can
be found in both foo and bar.
"want_iterator( $threshold )"
Returns an iterator that returns all files and directories for which
there are at least the specified number of instances across all
directories being scanned. For example if you are scanning three
directories and you need to perform some operation whenever a
particular file is found in two or more of them:
my $ffp = File::Find::Parallel->new( $dir1, $dir2, $dir3 );
my $iter = $ffp->want_iterator( 2 );
while ( my $obj = $iter->() ) {
print "We have at least two copies of $obj\n";
}
This is the primitive on which "all_iterator" and "any_iterator" are
built.
DEPENDENCIES
The tests require File::Temp and optionally Test::Pod::Coverage and
Test::Pod.
The lncopies script requires Getopt::Long and Pod::Usage.
BUGS AND LIMITATIONS
I haven't checked but it must be slower than File::Find. Use that
instead if you only want to scan a single directory at a time.
Please report any bugs or feature requests to
"bug-file-find-parallel@rt.cpan.org", or through the web interface at
<http://rt.cpan.org>.
AUTHOR
Andy Armstrong "<andy@hexten.net>"
LICENCE AND COPYRIGHT
Copyright (c) 2007-2008, Andy Armstrong "<andy@hexten.net>". All rights
reserved.
This module is free software; you can redistribute it and/or modify it
under the same terms as Perl itself. See perlartistic.
DISCLAIMER OF WARRANTY
BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH
YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
NECESSARY SERVICING, REPAIR, OR CORRECTION.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE
TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.