Server Log Forensics: Why You Need AWStats When Analytics Fail

There is a massive discrepancy between what your marketing department thinks is happening on your server and what /var/log/httpd/access_log actually says. Marketing loves Google Analytics. It’s pretty, it has charts, and it’s easy to read. But as any sysadmin who has stared down the barrel of a midnight DDoS knows, client-side Javascript trackers lie.

They don't count users with NoScript. They don't track search engine spiders. And they certainly don't tell you why your bandwidth usage spiked by 400% while your "Visitors" count remained flat. To survive in the hosting trenches, you need raw, server-side truth. You need AWStats.

The Illusion of Traffic vs. The Reality of I/O

I recently audited a client's server hosted in a budget datacenter. They claimed their site was slow despite low visitor counts. A quick look at their Javascript analytics showed 500 daily visitors. However, top showed the Apache processes were hammering the CPU, and the disk I/O wait was through the roof.

I installed AWStats 7.0 and ran a parse against the archived logs. The result? 500 humans, but 40,000 hits from a rogue scraper bot based in Eastern Europe. The Javascript tracker never fired because the bot didn't execute JS. The server was melting down serving HTML to a script.

Deploying AWStats on CentOS 6

If you are running a serious rig, you are likely on CentOS 5 or 6 (or perhaps Debian Squeeze). AWStats is a Perl script that parses Apache log files. It’s old school, but it’s battle-tested.

Here is the no-nonsense setup for a standard Apache environment:

The critical configuration line you must verify is the LogFormat. If you are using the standard NCSA combined log format (which you should be), ensure this is set:

Next, check your LogFile path. If you are using virtual hosts, point it directly to the specific domain's log, not the global server log:

The I/O Bottleneck: Parsing Gigabytes of Text

Here is where the hardware matters. AWStats works by reading every single line of your text logs. If you have a busy site generating 2GB of logs a day, running the update script (awstats.pl -update) puts a tremendous strain on your storage subsystem. On a standard SATA drive, this read operation can cause I/O wait times that make your actual website sluggish for users.

Pro Tip: Never run the AWStats update process as a CGI script triggered by a browser visit. It will time out on large logs. Set it up as a nightly cron job instead.

This is where CoolVDS architecture distinguishes itself. We utilize enterprise-grade RAID-10 SAS arrays and are currently rolling out high-performance SSD caching. When you parse a 5GB log file on a CoolVDS instance, the high read throughput ensures the job finishes in minutes, not hours, without starving your web server processes of disk access.

Data Sovereignty and The Norwegian Advantage

Beyond performance, there is the legal reality. Under the Norwegian Personal Data Act (Personopplysningsloven), you have a responsibility to handle IP addresses securely. When you rely solely on third-party US-based trackers, you are shipping user data out of the jurisdiction.

AWStats keeps the data on your server. If you host with CoolVDS, that data stays in our Oslo datacenter, directly connected to NIX (Norwegian Internet Exchange). This ensures low latency for your Norwegian users and stricter adherence to local data privacy standards than sending everything to a server in California.

Optimizing for Performance

To avoid security risks, don't expose the cgi-bin directory to the world if you don't have to. I prefer to generate static HTML reports that I can view securely. Here is a simple script I use to build static reports:

Put this in your /etc/cron.daily/ and you will have fresh reports every morning with zero CGI overhead.

Conclusion

Tools come and go, but logs are forever. If you aren't analyzing your server logs, you are flying blind. AWStats gives you the visibility to block bad bots, optimize bandwidth, and understand your true traffic.

However, log analysis is heavy lifting. Don't let your monitoring tools kill your production performance. Deploy your next project on CoolVDS, where we prioritize raw disk I/O and dedicated CPU resources to handle both your traffic and your analytics.

Ready to see what's really hitting your server? Spin up a CoolVDS instance in Oslo today.