Project Plan for Perl 5.005 Upgrade

In addition to the changes between Perl 5.004_04 and 5.005, this upgrade will mark the removal of the AT&T Sfio package from the Perl environment. Because of this, the upgrade will have to be handled carefully. However, there will likely be an improvement in performance associated with the upgrade as a result.

[ Overview ] [ Considerations ] [ Planning ] [ Timeline ] [ Conclusion ]

1. Overview

The version of the Perl interpreter currently in use on all ATG machines is 5.004, maintenance patch 4 (hereafter referred to as "5.004_04") with the AT&T Sfio package linked in to support FastCGI. However, the most current version of Perl considered to be stable is 5.005, maintenance patch 3 ("5.005_03") which addresses a large number of bugs and offers improvements in several areas (including, but not limited to: documentation, thread support, regression test suite enhancements). On some platforms, progress is being hindered by the lack of some of these features and/or fixes (such as a need for threads on HPUX 11 for the sake of packages such as DBD::Oracle).

2. Necessary Steps and Considerations

The upgrade cannot simply be done across all machines without care and attention to several existing issues related to Perl and its deployment within ATG.

2.1. Stabilizing for future upgrade efforts

First and foremost, this upgrade effort should be a vehicle for stabilizing the process of installing, maintaining and upgrading Perl within ATG. Currently, each machine within the sphere of ATG control is updated separately. Despite progress made in the area of centralizing the effort of initial Perl deployment (in the form of ready-to-deploy tar-style archive files), once a machine is set up there is no further synchronization of upgrades to either the core Perl distribution or the additional extensions and modules deployed and used by ATG.

To begin with, it is necessary to identify the variables involved in defining machine architectures as they impact the Perl upgrade. These are:

Variable Possible Values
Chip set PA-RISC 1.1, PA-RISC 2.0
OS Version HPUX 10.X, HPUX 11.X

Given this, there are only 4 possible combinations. Besides that, not all combinations may be in use. The only HPUX 11.X machine currently in the support pool is a PA2.0 machine, thus there is no current need for support for PA1.1 on 11.X. Indeed, it is possible to support HPUX 10.X without regard to chip set issues due to backwards-compatibility. It is better however to provide the Perl environment as optimized as possible for the target machine.

The proposal, then, is that a specific machine be designated as the build/test host for each chip/OS combination that requires support. Said machines will not need to be set aside exclusively for this purpose; at least two of the combinations can be supported using the existing hosts i2496ds4 (HPUX 10.X on PA2.0) and i2496ds6 (HPUX 11.X on PA2.0). Using the designated machines, complete distributions of Perl and the requisite modules can be installed and then archived with the tar command. Updates to either the Perl core or to extra modules should be restricted to these build machines, and updates made to other machines (in particular the servers outside the corporate firewall) made only when enough has changed from the existing deployment version to warrant it and then to all machines at the same time. Updates for such issues as addressing critical bugs in modules would also be done across the boards, hopefully lessening the chance of a critical update being overlooked on a less-visible server.

If this is implemented, it will provide the following additional benefits:

2.2. Removal of the sfio package

The other significant window of opportunity brought on by the upgrade of Perl is the phasing-out of the AT&T Sfio (Safe/Fast IO) package. This library had been in use in order to support the FastCGI plug-in for Netscape. Recent versions of FastCGI no longer require the sfio library, so this will be dropped from the Perl configuration effective with 5.005_03.

The immediate benefits to be had from this step include:

Because of the scale of Perl installation across ATG-supported hosts, backing out Sfio was not a viable option prior to the upgrade to 5.005_03.

3. Steps and Approach to Deployment

The effect of upgrading from 5.004_04 to 5.005_03 should be very minimal, if felt at all. Changes to the core language were primarily the fixing of bugs and the addition of native threads on those systems whose kernel supports threading. Still, it is not advisable to simply make the upgrade without extensive tests and a clear recovery plan.

At present, the installation of Perl is already engineered towards the goal of easing installation (and de-installation) of new versions. While developers reference the Perl interpreter as /opt/ims/perl5/bin/perl, the currently-installed base is actually housed under the directory /opt/ims/perl5.004.sfio. /opt/ims/perl5 is a symbolic link to this. Many of the current installations still have /opt/ims/perl5 compiled in as the root of the hierarchy. This will need to be addressed, as well. When the 5.005_03 package is built, it will use /opt/ims/perl5.005 as a root (note lack of "sfio" in the path). This will not only ease installation around issues of in-memory binaries, but will allow the old Perl installation to be left in place until ATG is satisfied that the transition is complete. Merely changing the value of the symbolic link /opt/ims/perl5 selects between the two.

With the assumption that the packages for 5.005_03 (and retro-fit packages for 5.004_04 configured to use a more explicit root path) can be easily built and bundled, ATG need only identify a build host for each hardware/OS permutation that will require support.

Step 1: Identify the hosts to be used to build the different Perl distribution bundles.
Step 2: Select a central storage repository for the installable bundles, including creating necessary directories and designing a hierarchy for separating the different hardware/OS permutations.
Step 3: Itemize the list of all modules that will be included in a distribution bundle (including local modules). This also acts as a useful reference if a project developer has questions about what components are available to them.
Step 4: Build distribution bundles for the various permutations and move them to the repository. These bundles must include all components listed above, even those such as DBD::Oracle that may not actually be used on the target machine. This is more maintainable than having bundles with different sets of modules.

Once this is done, it will be necessary to enter a period of testing (length to be determined) for the evaluation of individual project's Perl programs and any possible effects from the upgrade. Because of the portable nature of Perl, it should be sufficient to select just one machine for this (possibly a second machine from the stage group of servers, as well). The selected machine will have the symbolic link switched over from 5.004_04 to 5.005_03. Individual projects will then be responsible for testing their Perl application under the new interpreter.

Step 5: Identify a testing host. Possibly choose more than one, if there are other issues such as architecture of stage hosts versus development, or for HPUX 11 which will have to have thread support enabled to allow for the DBD::Oracle extension.
Step 6: Announce to project developers the starting time and duration of the testing period on the chosen host(s).
Step 7: On the chosen date, set Perl on that machine to the new version, and allow projects a reasonable amount of time to test their applications.

Once the testing has been deemed satisfactory, all that will remain will be deployment to the full range of ATG-supported machines. Once done, it is recommended that the machines retain the 5.004_04 installation for at least one month after deployment of 5.005_03, in case of critical failure.

(The following steps are iterated over all supported hosts)
Step 8: Deploy the appropriate package to the host being upgraded.
Step 9: Adjust the symbolic link in /opt/ims.
Step 10: Restart any resident processes that use the Perl interpreter (sysmonitord, rlsmgrd etc.). Do not use HUP signals in this case, but actually terminate and re-start all relevant processes.

4. Proposed Timeline for Implementation


5. Conclusions

This plan only provides for the deployment of 5.005 and the phasing out of 5.004. It is highly recommended that an ongoing maintenance plan also be developed to manage the eventual upgrades of the various CPAN and in-house modules. As well, Perl itself will inevitably mature to a version 5.006, at which time this plan should serve as effectively for that migration.