This is the first paragraph of Fetching.
FETCHING AND UPDATING:
A convention about using the "update" and "fetch" targets makes it easy for
users to know how to use a recipe. The main recipe for a project should be
able to be used in three ways:
1. Without specifying a target.
This should build the program in the usual way. Files with an
"fetch" attribute are obtained when they are missing.
2. With the "fetch" target.
This should obtain the latest version of all the files for the
program, without building the program.
3. With the "update" target.
This should fetch all the files for the program and then build it.
It's like the previous two ways combined.
Here is an example of a recipe that works this way:
STATUS = status.txt
SOURCE = main.c version.c
INCLUDE = common.h
TARGET = myprog
$TARGET : $SOURCE
:cat $STATUS
:sys $CC $CFLAGS $LDFLAGS -o $target $source
$TARGET : $STATUS
# specify where to fetch the files from
:attr {fetch = cvs://:pserver:anonymous@cvs.myproject.sf.net:/cvsroot/myproject} $SOURCE $INCLUDE
:attr {fetch = ftp://ftp.myproject.org/pub/%file%} $STATUS
When the "fetch" target is not specified in the recipe or its children, it
is automatically generated. Its build commands will fetch all nodes with
the "fetch" attribute, except ones with a "constant" attribute set
(non-empty non-zero). To do the same manually:
fetch:
:fetch $SOURCE $INCLUDE $STATUS
NOTE: When any child recipe defines a "fetch" target no automatic fetching
is done for any of the recipes. This may not be what you expect.
When there is no "update" target it is automatically generated. It will
invoke the "fetch" target and the first target in the recipe. To do
something similar manually:
update: fetch $TARGET
Although the automatically generated "update" target uses one of $TARGET,
"all" or the first encountered dependency.
:fetchall
:fetchall {attr = val}
Fetch all the files in the recipe (and child recipes) that
have the "fetch" attribute.
Extra attributes for fetching can be specified here, they
overrule the attributes of the file itself.
aap fetch Does a ":fetchall".
THE FETCH ATTRIBUTE
The "fetch" attribute is used to specify a list of locations where the file
can be fetched from. The word at the start defines the method used to
fetch the file:
ftp from ftp server
http from http (www) server
scp secure copy
file local file system
cvs from CVS repository
For a module that was already checked out the part
after "cvs://" may be empty, CVS will then use the
same server (CVSROOT) as when the checkout was done.
other user defined
These kinds of locations can be used:
ftp://ftp.server.name/full/path/file
http://www.server.name/path/file
scp://host.name/path:path/file
cvs://:METHOD:[[USER][:PASSWORD]@]HOSTNAME[:[PORT]]/path/to/repository
file:~user/dir/file
file:///etc/fstab
For a local file there are two possibilities: using "file://" or "file:".
They both have the same meaning. Note that for an absolute path, relative to
the root of the file system, you use either one or three slashes, but not two.
Thus "file:/etc/fstab" and "file:///etc/fstab" are the file "/etc/fstab". A
relative path has two or no slashes, but keep in mind that moving the recipe
will make it invalid. You can also use "file:~/file" or "file://~/file" for a
file in your own home directory, and "file:~jan/file" or "file://~jan/file"
for a file in the home directory of user "jan".
To add a new method, define a Python function with the name "fetch_method",
where "method" is the word at the start. The function will be called with two
arguments:
name the url with "method://" removed
dict a dictionary with all attributes used of the URL,
dict["name"] is the full URL
node a node object. Useful items:
node.name short name
node.absname full name of the file
node.recipe_dir directory in which node.name
is valid
node.attributes dictionary with attributes
The function should return a non-zero number for success, zero for failure.
Here is an example:
:python
def fetch_foo(from_name, dict, node):
to_name = node.absname
try:
foo_the_file(from_name, to_name, dict["foo_option"])
except:
return 0
return 1
In the "fetch" attribute the string "%file%" can be used where the path of
the local target is to be inserted. This is useful when several files have a
common directory. Similarly "%basename%" can be used when the last item in the
path is to be used. This removes the path from the local file name, thus can
be used when the remote directory is called differently and only the file name
is the same. Examples:
:attr {fetch = ftp://ftp.foo.org/pub/foo/%file%} src/include/bar.h
Gets the file "src/include/bar.h" from
"ftp://ftp.foo.org/pub/foo/src/include/bar.h".
:attr {fetch = ftp://ftp.foo.org/pub/foo/src-2.0/include%basename%}
src/include/bar.h
Gets the file "src/include/bar.h" from
"ftp://ftp.foo.org/pub/foo/src-2.0/include/bar.h".
CACHING:
Remote files are downloaded when used. This can take quite a bit of time.
Therefore downloaded files are cached and only downloaded again when outdated.
The cache can be spread over several directories. The list is specified
with the $CACHE variable.
NOTE: Using a global, writable directory makes it possible to share the cache
with other users, but only do this when you trust everybody who can login to
the system!
A cached file becomes outdated as specified with the "cache_update" attribute
or $cache_update. The value is a number and a name. Possible values for the
name:
day number specifies days
hour number specifies hours
min number specifies minutes
sec number specifies seconds
The default is "12 hour".
When a file becomes outdated, its timestamp is obtained. When it differs
from when the file was last downloaded, the file is downloaded again. When
the file changes but doesn't get a new timestamp this will not be noticed.
When fetching files the cached files are not used (but may be updated).
|