NULIB(1)

NAME
nulib - NuFX file archiver

SYNOPSIS
nulib [-]option[subopts] archive_name [ filespec ]

DESCRIPTION
nulib is a shell-based NuFX archive utility, based loosely on ARC for the IBM PC and ar(1) under UNIX. It allows you to perform certain operations on the same archives used by ShrinkIt, including view archive contents, add to archive, extract from archive, and delete from archive. In addition, it will list and unpack files from Binary II archives.

This program is primarily targeted at users of non-Apple II computer systems, from IBM PC compatibles to workstations to mainframes. It will also be of use to people who use a GS/OS shell on the Apple IIgs, such as APW, ORCA, or ECP-16 (although it is considerably slower than ShrinkIt). may require more than 256k of free RAM to function properly. Apple //gs users can use yankit(1) instead, which is much smaller and many times faster (though considerably less powerful).

archive_name is the name of the NuFX archive. It doesn't need to exist prior to the add, create, or move options.

filespec is the name of a file, either on disk or in the archive. More than one file may be named on the command line. While wildcard expansion is done by the shell on UNIX systems (including GNO), nulib is required to do wildcard expansions when running under APW or the ORCA/Shell. The names of files in the actual archive do not undergo wildcard expansion on any system.

Nulib will only work with Binary II and NuFX archives. If you try to view some other kind of file, you will get an error message and possibly an indication of what kind of file it is. Nulib can recognize files processed with compress(1), Zip, Zoo, StuffIt, ar(1), shar(1), GIF, and many others.

OPTIONS
The following options are recognised, each of which is a single character, and may be followed by the specified suboptions. The option/suboption string may be entered in upper or lower case, and may be preceeded by a hyphen. Suboptions may be entered in any order. There must be no whitespace in the option-suboption string.

a[vurd][c<cval>][s<sval>][f<type>[/<aux>]]

    Quick Append

    Quickly appends the specified files to the end of the archive. This option does not scan the archive to see if the files already exist, and will create a new archive file if one is not found. This option corresponds roughly to N)ew archive or A)dd files on the ShrinkIt menu.

    If you add files from the current directory or a subdirectory of the current directory, then the entire partial pathname is stored. For example,
    	nulib av fubar.shk foo/bar/filename
    will store foo/bar/filename in the archive fubar.shk. If you add files from directory that is not a subdirectory of the current directory, only the filename will be stored. The example
    	nulib av fubar.shk ../zip/bang/filename
    will store filename in the archive. This does not work under UNIX, since filenames are expanded by the shell instead of by nulib.

    Wildcards are allowed, and subdirectories are recursively descended (the depth is limited only by the maximum length of a pathname and the available memory on the system), unless the r suboption is used. The r suboption will prevent subdirectories from being processed, so that for example
    	nulib avr fubar.shk *
    (under UNIX) will add all files in the current directory, but will not descend into any subdirectories. The v suboption displays the filenames as they are added.

    The u suboption adds a file without compressing it. The c suboption allows you to insert a file in the archive with a specific compression algorithm (specified by the argument cval, which must immediately follow the suboption with no intervening spaces. The example
    	nulib ac5v fubar.shk file1
    would store file1 in fubar.shk using 16-bit UNIX-style LZW compression. Value corresponds to the appopriate thread_format as specified in the section on compression methods.

    The s suboption does not compress the file, but will store it as if it were by setting the thread_format field to sval. One use for this is adding compressed files without having to uncompress them first. For example,
    	nulib as1 foo.shk file1.qq
    adds a squeezed file. This suboption could conceivably cause a NuFX extractor to crash; please don't use it unless you are sure of what you are doing. Note also that some compression methods (like ShrinkIt LZW) require both the "compressed length" and "uncompressed length" values to be correct; use of this suboption will (incorrectly) set them both to the current file length.

    Only one of the supoptions u, c, or s may be specified.

    The f suboption allows specification of the filetype attribute. It should be three characters long, and will be matched against a list of standard ProDOS file types. Those abbreviations currently recognised are
    	NON	no type (default)
    	BIN	binary
    	TXT	text
    	SRC	source code
    	OBJ	object code
    	EXE	executable shell program
    Both upper and lower case letters will match.

    If the filetype is followed by a "/", then the auxtype field will also be set. Nulib expects the auxtype to be a four-byte hexadecimal number; entering less than four digits could cause the auxtype to be misinterpreted. The following example stores APW C source files:
    	nulib cvfSRC/000a nulib.shk file1.c file2.c ...

    If the d suboption is given, the entry will be tweaked to make it appear that the file is actually a disk image. This may be useful somewhere. Don't use it if you don't know what you're doing.

    Adding comments is not currently supported.

b[xvi][tvalue]

    Binary II Archive Operations

    This option is included for compatibility with the older Binary II standard, since it is still in use by many communications programs.

    With no suboptions, this will list the information on the archived files specified by filespec. The x suboption extracts the named files from the archive, unsqueezing them if necessary. If the filespec parameter is omitted, then all files will be listed or extracted. Note that matches are case-independent.

    When the v suboption is used in conjunction with the x suboption, the names of the archived files are printed as they are extracted. The v suboption has no effect otherwise.

    The t suboption will perform a text substitution as it works; see the NEWLINE CONVERSION section for more information.

    If the filename of an extracted file would conflict with an existing file, the file will be overwritten and a message will be printed. If the i suboption is used, then nulib will give an "overwrite (y/n)?" prompt. An affirmative answer overwrites the file; a negative answer will skip the file and continue processing.

    Attempting to perform NuFX operations on a Binary II archive will fail. If the file appears to be Binary II format, then a message indicating this will be printed. Providing transparent support for Binary II archives is not impossible, but isn't needed often enough to be worth doing.

c[vurd][c<cval>][s<sval>][f<type>[/<aux>]]

    Create Archive

    This is identical to the a option, but the "creating archive" message is suppressed.

d[v+]

    Delete Files

    Deletes the named files from the archive. Nulib scans the archive, marking all records that match the names in the file specification (case-independent). If all files are marked for deletion, then the archive file itself is deleted. Otherwise, a new archive is created, the unmarked records are transferred, and the old archive is deleted. An error during this process will leave the original archive unmodified.

    Note that this does not require an exact match if the + suboption is used; the command
    	nulib d+ fubar.shk foo
    will delete "foo", "Foozle", "food/recipies" and "food/stock." The v suboption prints the list of marked records as it works.

u[vurd][c<cval>][s<sval>][f<type>[/<aux>]]

    Freshen Files

    Updates files in the archive, but doesn't add any new files. Creates a new archive file, and either transfers the old record or adds the file depending on which is more recent. Only exact filename matches are allowed (case-independent), including partial paths. The archive being updated must already exist.

    Wildcards are allowed, and subdirectories are automatically expanded unless the r suboption is used. The v suboption displays the filenames as they are added. The u, c, s, and f suboptions, explained under the description for the a option, only apply to files being added to the archive or being updated (files that aren't updated are left unaltered).

    Files that are updated will retain their previous filetype. New files will get either the default filetype, the filetype specified by the f suboption, or the actual filetype (on the IIgs only), in that order.

h[snw]

    Command Help

    Displays a help screen. Three screens are available.

    The command
    	nulib h
    alone displays help on the options. The n suboption gives help with numbers; it lists the known compression methods and text translation types. The w suboption gives a brief listing of contributors, and how to contact me. The s suboption gives help on the suboptions.

i[v]

    Verify Integrity

    Verifies that the archive is intact. Does not modify the archive in any way. The v suboption prints a list of CRCs for each entire record. This is different from those listed by the TZ option, which are only for the record headers and (sometimes) threads; this includes not only the headers but all data as well.

    Please note that this doesn't do much more than read the file, unless the record_version is $0002 or greater (which means that the data has a checksum stored; currently these records are only generated by GS/ShrinkIt), and no compression was used. This merely scans the records and verifies the header CRCs, not the data CRCs. The main purpose of the v suboption is to make a list of CRCs that can be sent along with the archive.

m[vurd][c<cval>][s<sval>][f<type>[/<aux>]]

    Move Files to Archive

    This is identical to the a option, but the files are deleted after they are added to the archive. Note that the actual directory files are not deleted, unless they were given distinct record entries.

    Care should be taken to avoid trying to move an archive into itself. The act of adding may (depending on the OS and the archive) go into an infinite loop creating a huge file, and the coup de grace is when NuLib then deletes the archive to which you were adding.

p[v+][t<val>]

    Print Contents

    Print the contents of an archived file without extracting it. Useful for viewing archived text files without having to actually unpack them. Note this only allows viewing of data_threads; resource forks and disk images will not be displayed, and comments are not shown. I take no responsibility for pagination or filtering of funky control characters ...

    The v suboption will print the file's name right before it's contents are printed. The + suboption allows you to specify the first part of a pathname; see the d or x options for details.

    The t suboption will perform a text substitution as it works; see the NEWLINE CONVERSION section for more information.

t[vaz]

    Table of Contents

    With no suboptions, this lists only the filenames of the archived files. Not only does this make it easier to view the archive contents (the ShrinkIt format filename field is about 20 characters wide; this is as wide as it has to be), but the output is suitable for transmission via a pipe to other utilities.

    Using the v suboption will make it use ShrinkIt v2.0 output format (same as using the v option); it is included as a suboption mainly for people used to ar(1). Using the a suboption will produce a list similar to the output of ARC or ZOO.

    Using the z suboption will dump everything known about the archive, including all information in headers, CRCs, relative file position, sizes of individual threads, etc.

u[vurd][c<cval>][s<sval>][f<type>[/<aux>]]

    Update Files

    Updates files in the archive, keeping the archived file or the file listed on the command line, whichever is most recent (or exists). Unlike freshen, this will add new files as necessary. Creates a new archive file, and either transfers the old record or adds the file. Only exact filename matches are allowed (case-independent), including partial pathnames. The archive being updated must already exist.

    Wildcards are allowed, and subdirectories are automatically expanded unless the r suboption is used. The v suboption displays the filenames as they are added. The u, c, s, and f suboptions, explained under the description for the a option, only apply to files being added to the archive (files that aren't updated are left unaltered). Note that the order of files in the archive will be preserved.

    Files that are updated will retain their previous filetype. New files will get either the default filetype, the filetype specified by the f suboption, or the actual filetype (on the IIgs only), in that order.

v

    Verbose Listing

    Lists the archive contents in a format indentical to that used by the ProDOS 8 ShrinkIt v2.0 L)ist archive contents option. This is equivalent to invoking nulib with the options -tv.

x|e[vumi+][t<type>]

    Extract From Archive

    The x and e options are synonymous; they extract the specified files from the archive. If the file already exists, you are asked if you want to overwrite it. If part of a partial pathname does not exist, the subdirectory will be created. Omitting filespec will cause the entire archive to be unpacked.

    When files are archived, a pathname separator is stored ("/" for ProDOS and UNIX, " the pathnames are broken down into component file names, converted to names legal under the current operating system, and then recombined using the pathname separator for the current OS. This facilitates extraction of files archived under any OS, but can lead to filename conflicts that didn't exist when the files were added (such as a UNIX file that contained a backslash is unpacked under MS-DOS).

    If the filename of an extracted file would conflict with an existing file, the file will be overwritten and a message will be printed. If the i suboption is used, then nulib will give an "overwrite (y/n)?" prompt. An affirmative answer overwrites the file; a negative answer will skip the file and continue processing.

    Note that extraction does not require an exact match if the + suboption is specified; the command
    	nulib x+ fubar.shk foo
    will extract "FOO", "Foozle", "food/recipies" and "food/stock." This makes it possible to extract entire subdirectories at a time. The v suboption prints the list of marked records.

    If comments are present, then the v suboption also prints those as it extracts. Using the m suboption will cause the comments to be printed, but the files will not be extracted. NOTE: The exact use of the m suboption has not been entirely settled. Since it may be desirable to extract comments into a file, future versions of nulib may use x (extract) to put them in a file and p (print) to view them. Comments welcome, caveat emptor, have a nice day.

    The t suboption will perform a text substitution as it extracts; see the NEWLINE CONVERSION section for more information.

    The u suboption will extract the files without uncompressing them. This is rarely useful, but is easy to implement and is present for the curious.

SUBOPTION SUMMARY
The full meaning of the suboptions are explained in the OPTIONS section, above. The following list is intended as a quick reference. The usual meaning of the suboptions are:

a   ARC/ZOO style format (table of contents listing only)

c   compression type, followed by a number. The table below shows compression numbers:
    #  Name                        Abbr  Pack?  Unpack?
    0: Uncompressed                unc     Y      Y
    1: SQueezed (sq/usq)           squ     N      Y
    2: Dynamic LZW-I (ShrinkIt)    shk     Y      Y
    3: Dynamic LZW-II (ShrinkIt)   sh2     N      Y
    4: UNIX 12-bit compress        u12     Y      Y
    5: UNIX 16-bit compress        u16     Y      Y

f   file/aux type to add, followed by file/aux type spec

d   store the files, but mark them as disk images

i   interactive; prompt before overwriting file

m   messages (add/extract comments instead of data)

r   don't recursively descend subdirectories

s   storage type (store as compressed w/o compressing), followed by number

t   text translation (CR<->LF), followed by conversion mode. The legal conversion modes (during extraction) are:
        0: Convert from CR to this system (Apple files)
        1: Convert from LF to this system (UNIX files)
        2: Convert from CRLF to this system (MS-DOS files)

u   store as uncompressed (same as c0)

v   verbose mode

x   extract during Binary II operations

+   match partial pathnames for extract and delete

z   full output, prints all aspects of archive

NEWLINE CONVERSION
Different operating systems use different line terminators in text files. The table below shows them for some popular systems:
  Operating System         Line Terminator
  Apple ProDOS             CR ($0d)
  UNIX                     LF ($0a)
  MS-DOS                   CRLF ($0d0a)
While nulib will know what kind of terminators the operating system it is running under uses, it cannot reliably determine what kind an archived file uses. Thus, the terminator used on the system where the file was created must be specified.

Note that text translation should not be performed on non-ASCII files (non-ASCII means anything other than pure text; word processing files and executables are two examples of files that should never have text translation performed on them). Doing so will probably ruin the file, causing strange errors. Because of the wide range of files that nulib must handle, it is impractical to automatically determine which files are binaries and which are text.

In order to tell nulib what format to expect, you have to specify one of the following values as an argument to the t suboption:
    0 -- Convert from CR (Apple to current system).
    1 -- Convert from LF (UNIX to current system).
    2 -- Convert from CRLF (MS-DOS to current system).

You may wish to use a more suitable and robust program for converting text files after extraction; udl(1) is recommended.

ENVIRONMENT

NULIBOPT
    If set, this environment variable will set certain default behavior options. Any number of the following options may be specified, provided that they are separated by commas (no whitespace).

    An example, suitable for the csh(1) shell, would be:
        setenv NULIBOPT=verbose,type=SRC,aux=000a

    verbose
        This sets verbose operation. The default is non-verbose.

    interactive
        This sets interactive operation. The defalut is non-interactive.

    type=typeid
        This sets the ProDOS file type associated with each file. typeid may be a hexadecimal number or one of NON, or SRC. The default type is NON.

    aux=auxid
        This sets the ProDOS auxillary file type associated with each file in the archive. auxid must be a hexadecimal number.

    compress=compressid
        This is the compression method to be used on the files in the archive. compressid must be a numerical id from the table for the c suboption shown in the SUBOPTION SUMMARY section above. A value of 2 (shk) is the default.

Note that the type and aux settings of this environment variable do not apply when running on the Apple IIgs. Files will be stored with their actual filetypes, regardless of the variable setting. Also, the f suboption will override these settings.

DIAGNOSTICS
Many errors simply cause nulib to exit, leaving the archive in an uncertain state (which sounds fairly evil). If you were extracting, deleting, viewing, or updating when the error occurred, the worst that can happen is you will be left with a bogus temporary file in the current directory (something like nulib.tmp5610).

If you were adding to an existing archive, the files that were there will be unharmed, but additional files will not appear, and the archive will be oversized. This is because the master header block (which keeps a count of the number of records in the archive) is written last.

If you were creating a new archive, the file will be guano. This is because, as mentioned before, the master header block is not written until the very end. Since nulib identifies NuFX archives by looking at certain bytes in the MHB, the file will not be identifiable as NuFX. Note that the move option is safer than it looks, because files on disk are not deleted until the archive is safely closed.

KNOWN BUGS
nulib will not archive extended files. This is currently only of concern for versions compiled for the Apple IIgs.

Under some systems, using UNIX compress on a file which does not compress will cause the archive's EOF to be larger than it should be. This slack space will be used up if you add more files to the archive (with nulib anyway; no guarantees about ShrinkIt or gshk [GS/ShrinkIt], but there's no reason why they shouldn't work).

To be more precise: If the file being compressed doesn't get any smaller, the compression halts and the file is simply stored uncompressed. Because of the way compress works, on some systems the space that would have been occupied by more compressed data is left attached to the file. The ShrinkIt compression does not suffer from this problem. If you add more stuff to the file, nulib will fill the slack space first, not just append to the end of the file.

See also the Nulib Auxiliary Documentation for more information on known bugs.

LEGAL
This program is Freeware. Please distribute as widely as possible, but don't sell it. Source code is available via e-mail upon request.

Users without Usenet/Internet access may send mail to me at the address below.

AUTHOR
Andy McFadden
1474 Saskatchewan Drive
Sunnyvale, CA 94087
<fadden@netcom.com>

NOTE:
    Nulib is currently maintained by Devin Reade, <gdr@eddore.myrias.com>.

HISTORY
The ShrinkIt and NuFX standards were written by Andy Nicholas.

ShrinkIt LZW compression was written by Kent Dickey. LZW-II (a modified version of Kent's algorithm) was designed by Andy Nicholas. The C LZW-II decoder was by Kent Dickey and Frank Petroski (independently and simultaneously).

The UNIX compress code adapted from COMPRESS v4.3.

The Binary II unpack and unsqueeze C code adapted from unblu.c and usq.c by Marcel J.E. Mol (usq.c based on usq2/sq3 by Don Elton).

MS-DOS port was by Robert B. Hess and Bruce Kahn.

SEE ALSO
gshk(1), yankit(1), shrinkit(1), udl(1), and the NuLib Auxiliary Documentation.