fetchnews(8)

NAME
fetchnews - fetch news via NNTP

SYNOPSIS
fetchnews [-v] ... [-x number] [-l] [-n] [-f] [-P]

DESCRIPTION
Leafnode is a USENET package intended for small sites, where there are few users and little disk space, but where a large number of groups is desired.

The design of leafnode is intended to self-repair after problems, and to require no manual maintenance.

Fetchnews (formerly called "fetch") is the program which submits and retrieves new articles to or from the upstream NNTP server.

OPTIONS

-v  Be verbose. A maximum of four -v can be used (the more v, the more verbose):

-v  Warn about non-critical errors.

-vv Be verbose whenever a group command occurs.

-vvv
    Be verbose whenever an article command occurs.

-vvvv
    For debugging purposes. Don't expect the additional output to make sense.

-x number
    Go number articles back and try to re-fetch what's missing. Chiefly used for debugging purposes.

-l  Don't use supplementary servers (-l is supposed to mean "local"). This does speed up the operation quite a bit.

-n  Don't unlink files in the interesting.groups directory. This avoids automatic unsubscription of newsgroups.

-f  Erase the active.read file, thus force redownloading the whole active file from all upstream servers. This is usually occurring only very infrequently because it takes a long time. See timeout_active in leafnode(8).

-P  Only post articles that have accumulated in the out.going directory but do not fetch articles from upstream servers.

EXIT STATUS
fetchnews will exit with code 0 if it could connect to all servers and encountered no other errors. It exits with code 2 when it was unable to connect to at least one server, and it exits with code 1 on other errors.

FILES AND CONFIGURATION

/var/spool/news/leaf.node/active.read
    is a file to track when the active files have been successfully downloaded from the upstream servers the last time. If it is missing, this has the same effect as giving the -f option. It is only created after the active file of each upstream servers has been downloaded successfully.

See leafnode(8) for the list of other files affected, and for information on configuring fetchnews.

ALGORITHM
Here is a brief description of what fetchnews does.

First, after connecting to the upstream NNTP server, all articles that have the u+x permission (0100) flag set in /var/spool/news/out.going are posted using POST. If a posting fails, that article is moved to /var/spool/news/failed.postings which an administrator has to clean up once in a while. Moving a file back to /var/spool/news/out.going is sufficient to resubmit it. The u+x flag is checked to prevent posting an (incomplete) article that leafnode(8) is still receiving at the same time.

Second, the upstream server's list of newsgroups (obtained using LIST and LIST NEWSGROUPS) is merged into leafnode's (if timeout_active has expired, the active.read file is not present or the active file is empty) or updated (with NEWGROUPS).

Third, fetchnews iterates over the list of newsgroups, performing a GROUP, an XOVER (if supported) or XHDR Message-ID and a number of HEAD and BODY commands for each group which has been read recently. It assumes that for the particular group it is looking at, "maxage" was set to the groupexpire or global expire. As a consequence, it makes only sense to set maxage lower than expire.

NOTES
Fetchnews does not use NEWNEWS. NEWNEWS is not used because it is often very expensive and therefore also disabled at many sites.

After some experiments, I decided that it was better to fetch all the articles in a group than to optimize away some; users would complain too loudly when the optmizer guessed wrongly.

The only way to control what newsgroups are accessible to local users is to control what the upstream server lets fetchnews read, and that will not keep out all crossposts (endemic in e.g. the alt.sex groups). If your site is big enough that you need to control newsgroup access, consider using a news server which is designed for a big site.

Fetchnews sets its real and effective uid to "news" when started as root and refuses to run if it cannot get this user ID.

BUGS
Fetchnews ignores the "Supersedes" and "Also-Cancel" headers. This needs to be fixed, or news.answers will suffer.

HISTORY
Older versions of fetchnews posted any articles found in out.going. Since v1.9.23, fetchnews will only post article files that have their user executable bit set. Leafnode sets this bit after it has successfully received a post. This behaviour avoids posting incomplete articles when leafnode receives a post at the time fetchnews is posting.

AUTHOR
Written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995-96 Troll Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47 22646949.

Modified by Cornelius Krasel <krasel@wpxx02.toxi.uni-wuerzburg.de>, Markus Enzenberger <enz@cip.physik.uni-muenchen.de> and Randolf Skerka <Randolf.Skerka@gmx.de>. Copyright of the modifications 1997-1999. Modified by Matthias Andree <matthias.andree@gmx.de>, copyright of the modifications 2001-2002.

SEE ALSO
leafnode(8), texpire(8), newsq(1).