AWStats logfile analyzer 5.3 Documentation

Benchmarks

AWStats update process must be ran frequently, so it's important to know what is AWStats speed to choose an optimum delay between each update process according to AWStats speed and the refresh rate you need to have.
AWStats speed depends on AWStats version and options/setup you use in configuration file.

This is benchmark results with AWStats version 5.1 and a common configuration:

HARDWARE: Athlon 1 GHz / 128Mb
SOFTWARE: Windows 2000 / Perl 5.8
CONFIG OPTIONS: LogFormat=1, DNSLookup=0, URLWithQuery=0, URLReferrerWithQuery=0, No plugins
AVERAGE SPEED: 4500 lines by seconds
Other times for different kind of web sites sizes are shown later in this page...

This is other important information to know:

- A log file size is about 150 (NCSA common/CLF log files) to 300 times (NCSA extended/XLF/ELF log files) its number of lines,
- A 30Mb file = 100,000 lines = 20,000 pages (with 5 hits/page) = 2,500 visits (with 8 pages/visits) => 75 seconds (Athlon 1GHz)
- History files (resuming the log analyze) has the following size (one file a month) : 15000+160*x+100*y bytes (where x is number of unique visitors a month and y is number of different pages on web sites)
WARNING ! All those data are average values for a common public site. Calculation rule can be seriously changed according to web server or AWStats configuration and web site content.

Don't forget that benchmarks of log analyzers are made without reverse DNS lookup because this depends on your system, networks and Internet and not on the log analyzer you use. And the reverse DNS lookup can take more than 95% of time of a log analysis !
So, have a look at the following chart to:
- Get more real ideas on benchmarks results
- Get more information and advice on a good setup for your site.

This is examples of frequency/parameters you should use to have a good use of AWStats:

Your Web site size	Recommended values for parameters			Recommended update frequency (Rotate log delay)	Memory required**	Duration***
Your Web site size	DNSLookup*	URLWithQuery	URLReferrerWithQuery	Recommended update frequency (Rotate log delay)	Memory required**	Duration***
0 - 1,000 visits/month	1	0 or 1	0 or 1	Once a week Log files are 0-3 MB 10000 lines to process	<4 MB	<2mn
0 - 1,000 visits/month	0 (or 2)	0 or 1	0 or 1	Once a month Log files are 0-12 MB 40000 lines to process	<4 MB	<20s.
1,000 - 10,000 visits/month	1	0 or 1	0 or 1	Every 12 hours Log files are 1-2 MB 660-6600 lines	4-8 MB	1-2mn
1,000 - 10,000 visits/month	0 (or 2)	0 or 1	0 or 1	Once a week Log files are 3-30 MB 10000-100000 lines	4-8 MB	5-50s
10,000 - 50,000 visits/month	1	0 or 1	0 or 1	Every 4 hours Log files are 1-4 MB 2200-11000 lines	8-16 MB	1-4mn
10,000 - 50,000 visits/month	0 (or 2)	0 or 1	0 or 1	Once a day Log Files are 4-20 MB 13000-65000 lines	8-16 MB	10-40s
50,000 - 250,000 visits/month	1	0	0 or 1	Every hour Log Files are 1-4 MB 2700-14000 lines	16-64 MB	2-4mn
50,000 - 250,000 visits/month	0 (or 2)	0	0 or 1	Every 6 hours Log Files are 5-25 MB 17000-80000 lines	16-64 MB	20-80s
250,000 - 1,000,000 visits/month	0 (or 2)	0	0	Every hour Log Files are 4-16 MB 14000-56000 lines	64-256 MB	30-120s
1,000,000 - 5,000,000 visits/month	AWStats is not a good choice for such web sites. Try a tool with less features but faster like Analog or Webalizer
+5,000,000 visits/month	AWStats is not a good choice for such web sites. Try a tool with less features but faster like Analog

* You should set DNSLookup parameter to 0 (or 2) if
- reverse DNS lookup is already done in your log file,
- or if your web site has more than 250,000 visits a month.
Note: Country report can works without reverse DNS lookup if plugin 'geoip' is enabled (faster and more accurate than reverse DNS lookup).
** Free memory required for update process (in MB).
Warning: If you use the URLWithQuery or URLReferrerWithQuery option, or forget to complete correctly URLQuerySeparators for some sites, this value can be dramatically increased.
*** Duration is with Athlon 1GHz/128Mb and LogFormat=1.

SOME IMPORTANT ADVICES FOR A GOOD USE OF AWSTATS:

- You can disable DNSLookup in AWStats (set DNSLookup=0 or DNSlookup=2 to only use DNS file resolver). Note that with DNSLookup disabled, if you still want to have the 'Country' report, you must enable the 'geoip' plugin. Without this plugin, 'Country' report will works only if hosts addresses in your log file are already resolved (need to setup your web server to do so, your web server will be slowed). Without DNSLookup, log analyze speed can be increased by 20 to 100 times.

- Use carefully parameters URLWithQuery and URLReferrerWithQuery (Let them set to 0 if you don't know what they means) and check your web site URLs' syntax to know if you need to complete parameter URLQuerySeparators. If you really need to use URLWithQuery=1, check that URLWithQueryWithoutFollowingParameters is set properly.

- Be sure that your HostAliases parameter list is complete.

- Rotate your log (See FAQ-SET500) and launch AWStats more often (from crontab or a scheduler, See FAQ-SET550). The more often you launch AWStats, the less AWStats has new lines in log to process. This solve also the ActiveState memory problem (see next advice).

- Use standard Perl distribution instead of ActiveState (It seems that ActiveState Perl 5.006 and may be other version) have an important memory problem making speed of analysis slower and slower when analysing more and more lines.

- Use last AWStats version.

- Use last Perl version. For example Perl 5.8 is 5% faster than Perl 5.6

- If you use Apache and don't rotate your logs (not the best way of working), set PurgeLogFile to 1 (By default, to avoid bad surprise, PurgeLogFile is 0 in configure file, but you can set it to 1 to ask AWStats to purge the log file after processing it, this increase speed for next run).