Squid log analysis using sarg

From Notes_Wiki

Home > CentOS > CentOS 6.x > Squid proxy server configuration > squid log analysis using sarg

Manual installation of sarg

To install sarg manually use following steps:

  1. Install following packages from base, updates repositories:
    yum -y install gcc gd gd-devel make perl-GD wget httpd pcre-devel
  2. Download latest sarg code from http://sourceforge.net/projects/sarg/files/latest/download
    At time of this writing 2.3.9 was found to work and 2.3.10 was failing on CentOS-6.x
  3. Extract code and use ./configure; make; make install
  4. Edit /usr/local/etc/sarg.conf and set following values:
    access_log /var/log/squid/access.log
    output_dir /var/www/html/sarg-reports
    date_format e
    overwrite_report yes
  5. Generate one time report using sarg -x
  6. Run sarg over cron using:
    15 1 * * * /usr/local/bin/sarg -x >/dev/null 2>&1
  7. Restrict access to sarg by creating '/etc/httpd/conf.d/sarg.conf with
    <Location /sarg-reports>
    Options All
    AllowOverride All
    Order deny,allow
    Allow from 10.3.1.2
    Deny from all
    </Location>
    Here replace 10.3.1.2 with admin networksg

Some of the steps are contributed by Kiran Kollipara.


Sarg daily, weekly, monthly reports

By default sarg generates one report for each day or for a particular log file. It is more practical to look at weekly or monthly usage to understand Internet usage pattern of users. Thus, it might be desirable to have weekly and monthly reports along with daily reports. To setup sarg for multiple interval reports use:

    /usr/local/bin/sarg -x -d month-0 -o /var/www/html/monthly-reports -l /var/log/squid/access.log*
    /usr/local/bin/sarg -x -d week-0 -o /var/www/html/weekly-reports -l /var/log/squid/access.log*
    /usr/local/bin/sarg -x -d day-0 -o /var/www/html/daily-reports -l /var/log/squid/access.log*

for monthly, weekly and daily reports. These are one-time commands. To run them periodically using cron use following cron settings:

   15 22 * * * /usr/local/bin/sarg -x >/dev/null 2>&1
   15 23 * * * /usr/local/bin/sarg -x -d month-0 -o /var/www/html/monthly-reports -l /var/log/squid/access.log*
   15 0 * * * /usr/local/bin/sarg -x -d week-0 -o /var/www/html/weekly-reports -l /var/log/squid/access.log*
   15 1 * * * /usr/local/bin/sarg -x -d day-0 -o /var/www/html/daily-reports -l /var/log/squid/access.log*

where more details about '-d' option can be learned from sarg man page.

Further these different reports can be linked together by one top level HTML file such as :

   <html>
     <head>
       <title>Sarg reports</title>
     <head>
     <body>
       Different types of reports:
       <ul>
         <li> <a href="daily-reports" target="_blank">Daily reports</a> </li>
         <li> <a href="weekly-reports" target="_blank">Weekly reports</a> </li>
         <li> <a href="monthly-reports" target="_blank">Monthly reports</a> </li>
         <li> <a href="sarg-reports" target="_blank">Default reports</a> </li>
       </ul>
     </body>
   </html>


Refer http://www.linuxquestions.org/questions/linux-server-73/sarg-monthly-report-on-squid-server-927079/


Useful sarg configuration options

resolve_ip

Sarg resolve_ip option can be used to generate sarg reports using desired names instead of IP addresses. To use a custom script following configuration can be used:

resolve_ip exec
resolve_ip_exec /opt/script/name_to_ip.escript %IP

Example custom script could look like:

#!/usr/bin/escript

main(Args) ->
        case Args of
                [] -> ok;
                [IP1] ->
                        case IP1 of
                                "10.20.0.8" -> io:format("8 in 960");
                                "10.20.0.4" -> io:format("4 in 960");
                                "10.20.0.2" -> io:format("2 in 960");
                                IP1 -> ok
                        end
        end,
        ok.

Please note that this is an erlang script and for this erlang package must be installed. Further "chmod +x" on script is necessary to make it executable.


useralias

The name of a text file containing the user names one per line and the optional alias to use in the report instead of that user name. User names may contain wildcards denoted by a *. Example:

ext_* outstaffers

All accounts, containing ext_ in the beginning, will be represented by oustaffers alias. All their traffic will be summed up.

This option also supports regular expressions. For more details refer https://sourceforge.net/p/sarg/wiki/USER%20and%20IP%20options/


usertab

If resolve_ip is used then that is used before usertab is referred. It accepts none, filename or ldap as possible values. Example:

SirIsaac Isaac Newton
vinci Leonardo da Vinci
192.168.10.1 Karol Wojtyla

For more details refer https://sourceforge.net/p/sarg/wiki/USER%20and%20IP%20options/



Home > CentOS > CentOS 6.x > Squid proxy server configuration > squid log analysis using sarg