This file contains a few notes on building and installing PBS and 
Release notes for each release.


*         Portable Batch System (PBS) Software License
* 
* Copyright (c) 1999, MRJ Technology Solutions.
* All rights reserved.
* 
* Acknowledgment: The Portable Batch System Software was originally developed
* as a joint project between the Numerical Aerospace Simulation (NAS) Systems
* Division of NASA Ames Research Center and the National Energy Research
* Supercomputer Center (NERSC) of Lawrence Livermore National Laboratory.
* 
* Redistribution of the Portable Batch System Software and use in source
* and binary forms, with or without modification, are permitted provided
* that the following conditions are met:
* 
* - Redistributions of source code must retain the above copyright and
*   acknowledgment notices, this list of conditions and the following
*   disclaimer.
* 
* - Redistributions in binary form must reproduce the above copyright and 
*   acknowledgment notices, this list of conditions and the following
*   disclaimer in the documentation and/or other materials provided with the
*   distribution.
* 
* - All advertising materials mentioning features or use of this software must
*   display the following acknowledgment:
* 
*   This product includes software developed by NASA Ames Research Center,
*   Lawrence Livermore National Laboratory, and MRJ Technology Solutions.
* 
*         DISCLAIMER OF WARRANTY
* 
* THIS SOFTWARE IS PROVIDED BY MRJ TECHNOLOGY SOLUTIONS ("MRJ") "AS IS" WITHOUT 
* WARRANTY OF ANY KIND, AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,
* BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT ARE EXPRESSLY DISCLAIMED.
* 
* IN NO EVENT, UNLESS REQUIRED BY APPLICABLE LAW, SHALL MRJ, NASA, NOR
* THE U.S. GOVERNMENT BE LIABLE FOR ANY DIRECT DAMAGES WHATSOEVER,
* NOR ANY INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
* USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
* 
* This license will be governed by the laws of the Commonwealth of Virginia,
* without reference to its choice of law rules.

===========================================
BUILD AND INSTALLATION INSTRUCTIONS FOR PBS
===========================================


CONTENTS
--------

PBS is provided as a single tar file.  The tar file contains:
     -	All source code, header files, and make files required to build
	and install PBS.

     -	Full set of documentation sources.  These are troff input files.

     -	A subset of the ISODE package, as a tar file within the PBS tar file,
	required to generate code from the ASN.1 data description currently
	used for the client to server and server to server data exchange.
	This subset is based on ISODE release 8.0.  All code except that
	required for the code generation has been removed.  

     -	A copy of this memo.

     -	A postscript and plain text copy of the PBS Administrators Guide:
	pbs_admin_guide.ps and pbs_admin_guide.txt


ADDITIONAL REQUIREMENTS
-----------------------

An ANSI C compiler is required.   The native C compiler is recommended if it
is ANSI, otherwise use gcc.

Tcl/Tk version 8 or higher is required if you plan to build the GUI portion
of PBS or use a Tcl based scheduler.

A roff document formater, such as GNU's groff is required to format the
supplied documentation from the source.


BUILD AND INSTALLATION DIRECTION
--------------------------------

These directions are found the PBS Administrators Guide.  A postscript and
plain text copy are found in this directory.  Please print a copy, read and
following the directions CAREFULLY.



=============
RELEASE NOTES
=============

The following section contains known deficiencies and warnings about
situations which might appear.  It also lists major bug fixes and
new features added in the release.  Not every fix or change appears here,
but if it was important or major, it will likely be listed.


-------------------
Release 2.1 Changes
-------------------

Support for IBM SPPS 3.1 software on the SP has been added to PBS.
However, at this point, only one job and one window into the SP High
Speed Switch is allocated to a node.

PBS now supports checkpointing of jobs under irix6array, as well as irix6.
MPI jobs may be checkpointed provided either the -miser or the -cpr flag is
used on the mpirun command.  (The new support is mainly a result of improved
functionality in the SGI checkpoint code.  The MPI option requirement is a
requirement imposed by SGI checkpoint implementation.)

If the user's home file system on a cluster is NFS mounted amoung all the
nodes, Mom may be configured to deliver the job's output and error files via
/bin/cp rather than scp.   See the $usecp directive in Mom's config file.

When fifo is load-balancing jobs on a group of time-shared hosts, a specific
host may now be requested by name.

The Administrator Guide has been restructured and heavily revised to make
installation simplier.

A number of minor bugs have been fixed in the default fifo job scheduler.

A number of bugs have been fixed in the other parts of PBS.
A list of the Problem Reports closed and those still open is available at
	http://pbs.mrj.com/UserArea/bugs.html

----------------
2.1 Deficiencies
----------------

Under Irix 6.x, PBS calculates a cpu usage percentage.   It was intended that
the calculation result could be used to "enforce" the "ncpus" job limit.
The intent was a job that used X00% of a cpu in a sample time where X is greater
than the number of cpus requested would be aborted.    However, SGI clears the
Session ID and CPU Time from a zombie process table entry.   The CPU time used
is added into the parent proc entry only when the zombie has been "reaped" when
the parent calls wait().    This creates discontcontinuities in calculating
the  "delta-cput / delta-walltime" and can lead to jobs being aborted when
they should not be.   

A bug report has been filled with SGI.   Until, this is resolved, the 
apparent excessive CPU usage will be logged by Mom, but will NOT result
in the job being aborted.

--------------
 
-------------------
Release 2.2 Changes
-------------------
 
A new option has been added to pbs_mom.   If Mom is killed when there are
running jobs and she is restarted with the -p option, she will allow the
jobs to continue to run and poll to determine when they have terminated.
See that Admin Guide section on starting Mom and the pbs_mom man page for
more information.


PBS 2.2 supports SMP cluster nodes (cluster nodes with multiple processors).
In the server's node file, you may declare the number of "virtual" processors
per node by adding the special property "np=#", where # is the number of
virtual processors (VPs).   The term virtual is used because the number of VPs
does not have to match the number of physical processors.  Basically,  setting
np=# says that this node may be allocated #-times.  

A corresponding "properity" of ppn=# (for processor per node) is allowed on
the -lnodes= resource specification used by a user when submitting a job.
If not specified, the requested number of processors per node defaults to 1.
Therefore a request of -lnodes=4 asks for 1 VP per node on each of 4 nodes.
Where as a request of -lnodes=2:ppn=2  asks for 2 VPs on each of 2 nodes.
The two allocated nodes will each appear twice in the allocated nodes file
given to the job ($PBS_NODEFILE).

A new server attribute, node_pack, has been added.  This attribute provides 
a measure of control over the order of which nodes VPs are allocated from.
If this attribute is not set (unset), then the the jobs are assigned to nodes
in the order that the nodes are declared to the server (the order of in the
nodes file).   If set to "true", jobs are assigned to the nodes with the fewest
(non-zero) free VPs.  This packs jobs into the fewest possible nodes.  If set
to "false", jobs are assigned to the node with the most free VPs, scattering
the jobs across the node pool.
