Mail traffic accounting

[ispcomm]

While working on another issue, I stumbled upon some (possible) bug in the mail traffic accounting script.

The problem is that it depends tightly on the server name (in /etc/hosts) and whether the logs contain "localhost", "127.0.0.1" or "hostname" for the localhost.

This is an issue on etchfor example because the /etc/hosts file contains:

127.0.0.1 <servername.local> localhost

On sarge the same file is
127.0.0.1 <servername.local> localhost localhost.localdomain

There's another interaction with the postfix configuration file. If "localhost" or "127.0.0.1" is used in main.cf (for the hooks of antivirus/antispam etc) the resulting logs can contain either the "server name" or "127.0.0.1" or "localhost".

All these variations don't play game with the way mail logs are analyzed in ispcp-vrl-traff. On line 343 This code is present:

Code:

        if(exists  $main::cfg{'MAIL_LOG_INC_AMAVIS'} && $main::cfg{'MAIL_LOG_INC_AMAVIS'}) {
            # Logs including AMAVIS/SPAMASSASSIN/CLAMAV entries
            $cmd = "$cat $smtp_delivery_log | $awk '{if (\$5 != \"localhost\" && \$6 != \"virtual\" && \$6 != \"localhost\" && \$10 != \"?\") print substr(\$3, index(\$3, \"@\")+1, length(\$3)),substr(\$4, index(\$4, \"@\")+1, length(\$4)),\$10;}' 1>$smtp_log";

        } else {
            # Logs not including AMAVIS/SPAMASSASSIN/CLAMAV entries
            $cmd = "$cat $smtp_delivery_log | $awk '{if (\$5 != \"localhost\" && \$6 != \"localhost\" && \$10 != \"?\") print substr(\$3, index(\$3, \"@\")+1, length(\$3)),substr(\$4, index(\$4, \"@\")+1, length(\$4)),\$10;}' 1>$smtp_log";

An the awk script will cause problems depending on the above variables.

A patch to alleviate the problem is this

Code:

=== traffic/ispcp-vrl-traff
==================================================================
--- traffic/ispcp-vrl-traff     (revision 31)
+++ traffic/ispcp-vrl-traff     (local)
@@ -340,11 +340,11 @@

                if(exists  $main::cfg{'MAIL_LOG_INC_AMAVIS'} && $main::cfg{'MAIL_LOG_INC_AMAVIS'}) {
                        # Logs including AMAVIS/SPAMASSASSIN/CLAMAV entries
-               $cmd = "$cat $smtp_delivery_log | $awk '{if (\$5 != \"localhost\" && \$6 != \"virtual\" && \$6 != \"localhost\" && \$10 != \"?\") print substr(\$3, index(\$3, \"@\")+1, length(\$3)),substr(\$4, index(\$4, \"@\")+1, length(\$4)),\$10;}' 1>$smtp_log";
+               $cmd = "$cat $smtp_delivery_log | $awk '{if ( \$10 != \"?\" && !(\$5 ~ /localhost|127.0.0.1/ && \$6 ~ /localhost|127.0.0.1/)) print substr(\$3, index(\$3, \"@\")+1, length(\$3)),substr(\$4, index(\$4, \"@\")+1, length(\$4)),\$10;}' 1>$smtp_log";

                } else {
                        # Logs not including AMAVIS/SPAMASSASSIN/CLAMAV entries
-           $cmd = "$cat $smtp_delivery_log | $awk '{if (\$5 != \"localhost\" && \$6 != \"localhost\" && \$10 != \"?\") print substr(\$3, index(\$3, \"@\")+1, length(\$3)),substr(\$4, index(\$4, \"@\")+1, length(\$4)),\$10;}' 1>$smtp_log";
+           $cmd = "$cat $smtp_delivery_log | $awk '{if ( \$10 != \"?\" && \$6 !~ /virtual/ && !(\$5 ~ /localhost|127.0.0.1/ && \$6 ~ /localhost|127.0.0.1/)) print substr(\$3, index(\$3, \"@\")+1, length(\$3)),substr(\$4, index(\$4, \"@\")+1, length(\$4)),\$10;}' 1>$smtp_log";
                }

         $rs = sys_command($cmd);

But it won't fix the issue when "hostname.local" is reported in the logs.

The patch has a few "side effects" that I'm not sure are present in the original version:

1. The smtp traffic arriving and leaving via smtp (mail relaying) is counted twice. This is correct as mail arrives at the server and leaves to another server.

2. When MAIL_LOG_INC_AMAVIS is false, traffic arriving to a virtual user is counted once (this is correct), as this traffic should not be counted.

3. When relaying mail from the domain to an MX end server (using the MX patch) the traffic is counted twice, which is correct behaviour (mail arrives and leaves).

I have created a ticket for this issue here: http://www.isp-control.net/ispcp/ticket/1345

ispcomm.