Gnut Manual: Advanced Usage at MROB
Author: Robert Munafo

Gnut Manual: Advanced Usage

Next Previous Contents

4. Advanced Usage

4.1 Invocation

Gnut has a variety of command line arguments that can be used in special circumstances. They are listed below, with a short description of each.

> - -l range Sets the initial logging level to the value of . See the Debugging# portion of section 4.3 below.

> - -c file Specify a script to run at startup. This will bypass any ~/.gnutrc you may have.

> - -p num Start gnut listening on TCP port num.

> - -i [IP/interface] IP (or interface) to broadcast as belonging to us.

> - -d Daemon mode, never ask for input at the console.

> - -x Exit after having run any scripts.

> - -v Output version information and exit.

> - -h Display a usage message and exit.

4.2 Run-Time Commands

Overview

Gnut is a command line interface. That means you have to give it commands. Commands in gnut are a single word followed by optional arguments. All commands can be abbreviated to the point that they are still unique.

Notation

Brackets are used in the regular expression sense. Thus [abc] means use either a, b, or c. Textual arguments are indicated with italics, for example arg. The # sign is used to indicate a scalar number. "range" is used to indicate a list and/or range of numbers. These ranges can be specified in several ways. For example:

7 | Just a single number
4-6 | Range of numbers
2,6,5 | Comma-delimited list
1-4,17,7-10 | Any combination thereof

List of Commands

> info [hqnsctud] - Display various information.
> h - host totals
> q - queries received and replies sent
> n - network traffic totals
> s - shared files on this machine
> c - connections (GnutellaNet only)
> t - transfers in progress
> u - upload transfers in progress
> d - download transfers in progress

> open host:port - Open outgoing connection to a host.

> find string - Search the Gnutella network for string. The string you supply is is sent as-is to the network. Nearly every Gnutella node (servent) specifies this as a boolean AND search with all non-alphanumeric characters ignored and ignoring upper/lower case. If you want to force all results to be boolean AND matches, set the strict_search configuration variable. If you want to force non-alphanumeric characters to be ignored, simply leave all non-alphanumeric characters out of your search string.

> If you have set the multi_enable variable, you can perform more than one find at the same time; this also enables the list and lclear commands.

> search string - Synonym for find.

> find or search with no argument displays results.

> list - Shows all searches currently going on (if multi_enable is set). You can remove searches and their results from the list with the lclear command.

> lclear id - Clear a search and its associated results. The id is a number shown in the output of the list command.

> lclear string - You can also clear a search by giving the text that was used to issue the search.

> monitor keywords - Monitor incoming search queries. If arguments are supplied, they are used for a boolean AND match (ignoring case) before displaying search queries. So, for example, monitor fred jpg will only display search queries that contain both fred and jpg.

> mreply keywords - Monitor replies to search queries that others have issued. While monitor lets you see what people are searching for, mreply lets you find out what's actually available on the network. As with monitor, arguments can be given to specify a boolean AND match (ignoring case) before displaying filenames.

> mpush keywords - Monitor push requests issued by others. While mreply lets you see what files are available on the network, mpush lets you find out which of those files are actually in demand. NOTE: Push requests are kind of rare, and gnut cannot always identify the file being requested. So, you won't see much when you use mpush. Try it sometime when you can let it run for an hour or two.

> update - Send out ping packets to all connected hosts. If configuration variable update_clear is set, then the current host list is discarded as well.

> response regexp - Show the current query responses which match the given regular expression. The results are sorted according to the sort_order configuration variable.

> get range - Start downloading files referenced by range. If more downloads are requested than allowed by max_uploads, then the downloads are queued.

> push range - Same as get, however only a push connection is attempted.

> stop range - Stop the transfers referenced by range. The numbers you give the stop command come from the info or info t command.

> clear range - Removes finished transfers from the transfer list. If no range is given, all completed transfers will be removed. You can also set the auto_clear variable to make this automatic. Range numbers work the same way as for the 'stop' command.

> kill range - Terminate the gnutella connections referenced by range.

> kill w - Terminate the gnutella connection with the worst dropped-duplicate packets statistic.

> blacklist args - Add one or more hosts, port numbers, subnets, or search reply strings to the blacklist. All Gnutella nodes on the blacklist, and all query replies that match the blacklist will be ignored. Used properly, this can eliminate lots of spam. (See also the search_extensions and strict_search variables)

> Syntax of the arguments is:

> blacklist [ host hostname [subnet bits]] [search string] [ all | ping | search | incoming | outgoing | http ]

> The many forms of the blacklist command are best explained by example:

> blacklist host 1.2.3.4 all | Blacklists one host: All queries, replies, connection attempts, etc. from that host will be ignored.
> blacklist host 1.2.3.4 incoming | Prevents incoming connections from one host, but other activity (searches, etc) involving that host is still allowed.
> blacklist host 1.2.3.1 subnet 8 | Blacklists everyone from 1.2.3.1 to 1.2.3.255 (the "8" refers to the last 8 bits of the IP address).
> blacklist search hitler search | Stops any search that includes "hitler" from passing through this server.

> hosts - Displays the current host catcher, this can result in a lot of information!

> hosts file - Reads in a gnutella-hosts file.

> #limit num range - Limit the transfers referenced by range to num bytes per second.

> log num - Set the current log level to num. See description of log_level in the Debugging section below.

> share paths - Takes a ":" delimited list of paths to share. (";" on Win32)

> scan - Rescans the files in the share paths.

> play range - Runs the program set in play_prog with the URL of queries range as arguments. If play_prog is configured and installed properly, it will connect to, tranfer (if possible), and stream the data.

> player mime_type mimetype extention extension player program - Attach a player program to the given filename extension and MIME type. This command allows you to specify different programs for each type of file that you might want to play with the play command. There are six arguments, three are always the same. Two examples are:

> `player mime_type image/jpeg extension jpg player ee`
> `player mime_type image/gif89a extension gif player xv`

> set key val - Sets a configuration value. See Section 4.3.

> set key - Displays a configuration value. See Section 4.3.

> sleep num - Do nothing for num seconds.

> cls - Clears the screen (UNIX users only)

> ! cmd args - Shell escape: runs a shell and sends a command to it (UNIX users only) Examples:

> ! ls | List your current directory
> ! cd dir1 | This will not have any lasting effect
> ! ls dir1 | List files in directory 'dir1'
> ! cd dir1 ; ls | Multiple shell commands work
> ! mpg123 gdead.mp3 & | Play an MP3 file in the background

> eval cmd args - Evaluate shell escape: runs a shell and sends a command to it, then executes its output as gnut commands. (UNIX users only) Here is an example (this should all be typed as one line):

            ` perl -e "$bl=\"gnut_settings\";
                system \"wget -q http://10.1.2.3:8080/$bl\";
                open FILE, \"$bl\" or die \"Couldn't open $bl\";
                print \"load $bl\";"

> This example uses wget (type 'man wget') to try to get a file called "gnut_settings" from an HTTP server called "10.2.3.4:8080", and if the file is successfully loaded, runs the gnut command "load gnut_settings". This command could be used inside an organization like a school or company to load a centralized gnut settings file, or perhaps an up-to-date host list.

> The eval_echo configuration vaue determines if the commands executed by eval are printed. Of course, it can be set and cleared within an eval. Here is a script that uses this. It issues an update command and then checks the host count every 5 seconds for 10 minutes. Note the use of shell escape and echo at the end to print a message.

            #!/usr/bin/perl
            print "set eval_echo 0\n";
            print "set update_clear 1\n";
            print "update\n";
            $i = 0; $sl = 5;
            while($i < 10 * 60) {
              print "info h\n";
              print "sleep $sl\n";
              $i += $sl;
            }
            print "info h\n";
            print "! echo ---- test done ----\n";

> `cmd args` - A shell command can be enclosed in back-quotes and included as part of any other gnut command. This works like eval, except that only the last line of output from the shell command is used as part of a gnut command. For example, try:

           set prompt `date` >

> quit - Quit gnut.

> exit - Synonym for quit.

4.3 Configuration Variables

Nearly every aspect of gnut can be configured using its built in configuration system. All configuration variables can be considered to either be numeric, or alphanumeric. If you enter non-numeric characters into a numeric variable, it will just be treated as a zero.

To set a value, type a command like `set variable_name 1`. For convenience, "-" characters are treated like "_" for variable names. To display a variable's current value, type the command without a value: `set variable_name`.

Follows is a list of all the available configuration variables, along with a short description of how it affects gnut's operation. In most cases, each variable only controls one small aspect of gnut, allowing for a wide variety of possible configurations.

User Interface

> prompt - String, the text displayed each time gnut is waiting for a command. By default, this is "`gnut>` ". If any of the following special strings is included in prompt, they will be replaced with values as described here:

> ${HOSTS} | The number of hosts in the host list.
> ${FILES} | The number of files on active Gnutella servers on the network.
> ${SIZE} | The total size (in megabytes) of files on active Gnutella servers on the network.
> ${MSGS_R} | The total number of packets received over all connections since gnut started.
> ${MSGS_S} | The total number of packets sent out over all connections since gnut started.
> ${BYTES_R} | The total number of bytes received over all connections (not including downloads) since gnut started.
> ${BYTES_S} | The total number of bytes sent out over all connections (not including uploads) since gnut started.
> ${QUERIES} | The total number of query packets received over all connections since gnut started.
> ${QUERY_RESP} | The total number of query replies sent since gnut started.
> ${NUM_SHARED} | The number of files you're sharing.
> ${SIZE_SHARED} | The total size (in kilobytes) of the files you're sharing.
> ${RESPONSES} | The number of responses to your latest (current) find or search.
> ${BLACKLIST_S} | The number of hosts that have been blacklisted.

> auto_clear - Boolean, if set to 1, transfers are removed from the transfer list after completion without user intervention.

> beep_on_first - Boolean, if set to 1, an audible tone is emitted after the first search result of a new query is received (but only if wait_after_find is also set).

> beep_on_all - Boolean, if set to 1, an audible tone is emitted whenever the number of search results goes up (but only if wait_after_find is also set).

> download_path - String, directory where downloads should be stored.

> munge - Boolean, if set to 1, large numbers are displayed with unit postfixes.

> stats_format - Boolean, defaults to 0. If set to 1, the connection stats will be in the same format as Windows Gnutella 0.56 (one line per connection). The three numbers (e.g. "2407,4260,850") give packets sent, received and dropped. "Dropped" packets are duplicates that have been seen before; a large number means that two of your connections (this one and one other) are connected to each other by another, short route. See the autokill_thres option.

> paginate - Boolean, if set to 1, gnut will stop for user input after a screen-full of information has been displayed. This allows for easy viewing of long lists.

> play_prog - String, the program to be used to stream media files.

> sort_order - String, series of characters indicating what fields should be sorted on when using the response command. A sort string has the following format:

          "?[+-]?[+-]..."

          ? is a single character from the following set:
          s = size
          p = speed
          e = extension

          + indicates asecending order, - indicates descending order

          For Ex: "e+s-" sorts by extension, followed by size in descending order.

> verbose - Numeric, defaults to 3. Controls messages that notify when uploads and/or downloads start and end. The settings are:

> 0 | No messages
> 1 | Print messages related to downloads.
> 2 | Print messages related to uploads.
> 3 | Print messages related to both downloads and uploads.

> wait_after_find - Boolean, if set to 1, the find command displays a pretty little indicator showing how many results have come in. Otherwise they just come in the background.

> strict_search - Boolean, if set to 1, results of find or search will be filtered against the original search string, using a boolean AND match of all words in the string, ignoring case but not ignoring punctuation. This is useful because many other nodes on the Gnutella network do not use a boolean AND match, and almost all ignore punctuation. If strict_search is turned off, searches for "Mike" and "I'm Mike" will return the same matches, because most servents will ignore the ', and because the letters I and m are part of the word Mike. But with strict_search turned on, the "I'm Mike" search will only return filenames that actually contain both I'm and Mike. This option also helps filter out some types of spam. It defaults to 1.

> search_extensions - String, contains list of filename extensions to use as a filter against search results. For example, if set to "jpg txt", only files ending in ".jpg" or ".txt" will appear in search results. If search_extensions starts with "*", it will match all search results. Putting a "*" anywhere else in search_extensions has no useful effect. This setting defaults to "*".

> multi_enable - Boolean, if set to 1, allows multiple simultaneous searches. This involves a slightly more complicated interface because the results list has to be manually cleared to get rid of old search results. (That's why this setting is not enabled by default).

Paths

> download_path - String, the directory where downloaded files are written.

> share_paths - String, directories where scanning for uploadable files occurs. Setting this variable has the same effect as the share command.

Limits

Numerical limits, as well as bandwidth limits are supported for transfers and connections. The following variables control this operation.

> default_download_cap - Numeric, all downloads are limited to this many bytes per second, unless specifically set with the limit command. Any downloads that do not currently have a limit command applying to them will immediately respond to changes in default_download_cap, unless their speed is limited by a bottleneck somewhere out in the network.

> default_upload_cap - Same as default_download_cap, but for uploads.

> global_bandwidth_cap - Numeric, the sum of all upload and download bandwidths is limited to this. Changes take effect immediately.

> max_downloads - Numeric, maximum number of simultaneous downloads to allow. Any more than this are queued.

> download_timeout - Numeric, defaults to 3600. This is the number of seconds a download connection will wait before "giving up". Some Gnutella hosts make you wait before connecting if they have reached their max_downloads limit (Others allow the connection but wait before sending the data). Prior to version 0.4.7, the timeout was 10 but you couldn't adjust it.

> max_incoming - Numeric, the gnutella protocol requires that clients connect to other clients. This allows you to limit the number of other clients connected to yourself. max_incoming does not affect the number of file uploads (use max_uploads for that) and does not interfere with receiving files via the push command.

> max_uploads - Numeric, the maximum number of outgoing file transfers allowed at any one time. New requests are denied after this number is reached. max_uploads does not affect the number of incoming GnutellaNet connectiond (use max_incoming for that) and does not interfere with receiving files via the push command.

Password Protection

Gnut can form password protected mini-networks. The password is negotiated with each other gnutella servant that is connected to.

> password - String, phrase to use for authentication with remote hosts.

> password_required - Boolean, if set all incoming gnutella connections will be verified against the password.

Network and Performance Settings

These pertain to the gnutella network and protocol, and most users will have no need to mess with them.

> autokill_thres - Numeric. Defaults to 10. This is a percentage from 1 to 100 (set to 0 to disable autokill). With autokill enabled, gnut will periodically scan the connection list for connections that have at least this percentage of duplicate packets. It only kills one at a time, and the frequency of checking is 16 minutes divided by the number of connections. If you have 4 connections, gnut will check once every 4 minutes; if you have 8 it checks once every 2 minutes, and if you have 16 it checks once per minute.

> The effect of autokill is to gradually move the connections around until they're connected to different points as "far" from each other as possible. When your connections are "far" from each other you get a greater coverage of the Gnutellanet as a whole, which improves search results; you also get fewer duplicate packets, which improves efficiency.

> hide_pathname - Boolean, defaults to 0 (false). When set, the files you share will be shown to others as filenames, not full pathnames. This setting does not affect searches -- if they search for "abc" and the files are in a directory called "abc", the search will match. This is the way the original Windows Gnutella client worked, and most clients only show filenames.

> max_results - Numeric, defaults to 64. The maximum number of files that gnut will return to others doing queries. If a query matches more than this number of files, only the first matches will be returned.

> hidden - Boolean, if set, the client will not respond to pings, and thus will not end up in any other client's host catcher.

> local_port - Numeric, read-only, the port to which gnut is bound. Changing it after gnut is running has no effect, but you can put "set local_port 1234" in your .gnutrc file and it will work, or use the -p num command-line option.

> redirect - Boolean, when set, behavior when the maximum number of incoming connections is reached changes. Instead of ignoring all new connections, the oldest one is dropped and new connections are never refused. This can be used to set up a high availability entry point into the gnutella network.

> speed - Numeric, speed to broadcast us as over the network. This is mostly meaningless.

> ttl - Numeric, the TTL to use for new packets, and to limit routed packets to. See the section below Choosing the TTL value.

HTML interface

If you point your web browser to a gnut servant on the correct port, it has the ability to show a list of the shared files. You can also search the gnutella network through this interface. That way you could set up a gnut server that you can use from any web browser, and not necessarily be at your computer. See section 4.5 for instructions on how to use the HTML interface.

> html_enable - Boolean, if set to 1, the html interface is enabled.

> html_template - String, path to an optional html template file. See section 4.5.

Caching

As of gnut 0.3.27, caching of frequently requested files is supported. Gnut will watch all gnutella connections for query responses, and randomly download files into a local cache. This is intended to help distribute the load of highly requested files across many computers.

> cache_path - String, set to the directory where you want cached files to be stored.

> cache_refresh - Numeric, number of seconds between updates to the cache.

> cache_size - Numeric, maximum number of kilobytes to allow the cache to grow.

> min_cache_size - Numeric, minimum size of a file (in kilobytes) required for it to be stored in the cache. Any files smaller than this size will not be cached.

Debugging

> packet_stats - Boolean, if set to 1, a large amount of interesting statistics are shown in the connection information list.

> log_level - Numeric, defaults to 0. Sets the current log level. This is also set by the -l command-line option and the log command. Any nonzero value causes debugging information to be printed out. Larger numbers give more output. Anything higher than 2 is probably more output than you want to see. (However, you can type between the lines.)

> eval_echo - Boolean, defaults to 1. If set, the eval command prints the gnut commands it is going to execute before executing them.

4.4 Configuration File

Gnut is capable of executing files consisting of gnut commands. Upon startup it defaults to loading the file ~/.gnutrc (Win32 users can put this file in the current working directory). The actual name of the file can be set as a command-line option. All the commands are executed before any user input is requested.

4.5 Web Access

You can access a list of all files shared by a Gnut servant by creating a connection to them from within a web browser. To test it for yourself, just enter something like `http://127.0.0.1:5634` (if 5634 is your Gnut port) in your browser's address bar (127.0.0.1 is a special address that means "connect to my own computer"). A list of all your shared files should come up. By giving your IP address to others, you can let others access your files over the web. They can also search the gnutellaNet through your servant by clicking on "Search the GnutellaNet". Nifty, eh?

Gnut can use a template file for this interface. It is specified in the html_template configuration variable. This file is a standard HTML file, except that you embed the comment `<!-- gnut -->` where you want the gnut specific stuff to go.

4.6 Choosing a TTL Value

There is a lot of confusion about the TTL setting in Gnutella programs, what it does for the user and how it affects the performance of the Gnutella network.

The direct definition of TTL is very simple: it is a number that specifies the maximum number of "hops" from one computer to the next will be taken by queries sent out from your computer to other computers on the Gnutella network. With a TTL of 1, your queries only reach the hosts (usually 4) that you are connected to, with a value of 2 it will reach more (perhaps 16, if each of your 4 are each connected to another 3 and all of those are different), etc.

Typical default values for TTL are 7 or 9. These numbers were chosen based on the following formula, based on the "six degrees principle", the principle which states that any two people can be connected to each other by no more than six degrees of separation:

> number of hosts ~= 4TTL

(where ~= means "approximately equal") The idea is that, each time you increase the TTL by 1, you can get to 4 times as many hosts. This is based on the premise that each host connects to 4 other hosts. So, for a network of 16,000 nodes (about the typical size of the Gnutella network at a busy time of day in summer 2000), a TTL of 7 works well (47 is about 16,000). Well, that's not quite right, because actually each host connects to 3 more hosts (since one connection was already used to get to that host). So maybe the 4 just needs to be a 3. No, that's not right, either. Look at this table:

tree height | new nodes | total
0 | 1 | 1
1 | 4 | 5
2 | 12 | 17
3 | 36 | 53
4 | 108 | 161
5 | 324 | 485

The total is what matters, and clearly it's 4 times as great each time. So maybe we just need to add a constant and make the formula something like:

> number of hosts ~= K × 4TTL

In fact, the formula is completely wrong!

To see why, consider a TTL of 3. The common wisdom states that about 64 nodes can be reached with a TTL of 3, and the following diagram is shown or described as "proof":

                      (imagine 36 more nodes here)
                       ||||||||||||||||||||||||||
                       a b c  d e f  g h i  j k l
                        \|/    \|/    \|/    \|/
                         A      B      C      D
                          \      \    /      /
                           `------you-------'

This is a "tree" with your node at the bottom. You're connected to 4 nodes (A B C and D). Each of those is connected to 3 more (the nodes a b c, ... through l), which is 12 nodes. Each of those 12 has another 3 above it, which is 36 more. That's a total of 53 nodes, and that is pretty close to 43, which is 64. So what's the problem?

The problem is that, even though you can reach each of these nodes with a TTL of 3, most of them cannot reach each other! Consider a and g, for example. The only way a can get to g is through A, you, and C, and it takes a total of 4 steps to make the trip. The 36 nodes in the top row are even worse off -- they need a TTL as high as 6!

You could have the nodes in the top row connect to each other, since they each have three unused connections. But this doesn't solve the problem either. No matter how you use those extra connections at the top, there will still be nodes that cannot reach each other in less than 4 steps.

Of course, real Gnutella networks do not set themselves up as a tree like this -- they're a random spaghetti-like mess that is constantly rearranging itself.

Considering "ideal" or "optimally connected" networks, (still with 4 links per node), it is easy to see that for 5 nodes you can get a TTL of 1 (connect each to the other 4), and with 9 nodes you can get a TTL of 2 (connect them in a 3x3 torus grid). For bigger TTLs it's hard to see what the limits are. It is pretty clear however that you can't get 4 times as many each time you increase the TTL by 1.

Upper limits are easier to define. It appears that if the number of nodes is N and N is sufficiently large (in practice, 64 or more), a network can be constructed with a TTL of log2(N). This is accomplished by numbering the nodes with binary numbers and connecting each node n to the nodes n XOR 1, n XOR 2, AnotB(n ROL 2, n) and AnotB3(n ROR 2, n). AnotB(A, B) is defined as A (if A is not equal to B) or NOT(B) (if A is equal to B). AnotB3(A,B) is the same as AnotB except when A is equal to A ROR 2, when its value is A XOR floor(N/3). Note the difference between big N (size of network) and small n (node number).

So, for a network of 16,000 nodes, the TTL needs to be at least 14 for everyone to be able to reach everyone else.

It is also widely believed that Gnutella programs that use a fairly big TTL (like 20 or 30) cause huge amounts of traffic on the Gnutella network, because their data is getting sent around and around in circles.

This belief is also completely wrong!

All Gnutella client programs, in order to operate efficiently, have to keep track of all the query packets (and other types of packets) they have seen for the last 5 or 10 minutes, so they can "drop" (ignore) any duplicates they see. This takes a lot of memory, but is absolutely essential for the program to be even barely functional. If a Gnutella program did not do this, it would write every incoming packet to out to all other connections (aside from the one it came in on) and so it would be writing much more out than it's reading in. As a result, its output buffers on all active connections would very quickly fill up, and the Gnutella program would either fall way behind on reading input data (causing noticable delays in searching) or would have to indiscriminately throw away packets or close connections (causing searches and/or push requests to produce little or no results).

Next Previous Contents

WWW: http://www.mrob.com/
EMail: mrob at mrob com (If you aren't a spambot you can rewrite this yourself)
Send junk mail to: rpm@mrob.com
© 1996-2000 Robert P. Munafo. 3