Mail traffic accounting

[kilburn]

I've made some tests regarding mail traffic accounting and got some interesting results. From now on me@external.dmn is my main mail account, hosted outside the test server and mx.external.dmn it's primary MX. The server uses amavis as spam filter.

"after-awk" logs are the result of ispcp's awk processing command:

Code:

DEBUG: push_el() sub_name: sys_command('/bin/cat /var/log/ispcp/smtp_delivery.log | /usr/bin/awk '{if ( $10 != "?" && $6 !~ /virtual/ && !($5 ~ /localhost|127.0.0.1/ && $6 ~ /localhost|127.0.0.1/)) print substr($3, index($3, "@")+1, length($3)),substr($4, index($4, "@")+1, length($4)),$10;}' 1>/var/log/ispcp/smtp.log'), msg: Ending...

"final count" is the real current count for this test.

Test 1: user@dmn.tld is a domain mailbox

Code:

2008-06-17 23:23:47 me@external.dmn user@dmn.tld test.server.tld 127.0.0.1 SMTP - 1 778291
2008-06-17 23:23:47 me@external.dmn user@dmn.tld test.server.tld virtual SMTP - 1 778662

after-awk:

Code:

external.dmn dmn.tld 778291

Final count: 778291

Test 2: user@sub.dmn.tld is a subdomain mailbox

Code:

2008-06-17 23:23:58 me@external.dmn user@sub.dmn.tld test.server.tld 127.0.0.1 SMTP - 1 222773
2008-06-17 23:23:58 me@external.dmn user@sub.dmn.tld test.server.tld virtual SMTP - 1 223152

after-awk:

Code:

external.dmn sub.dmn.tld 222773

Final count: 222773

Test 3: user@als.tld is an alias mailbox

Code:

2008-06-17 23:24:09 me@external.dmn user@als.tld test.server.tld 127.0.0.1 SMTP - 1 225375
2008-06-17 23:24:09 me@external.dmn user@als.tld test.server.tld virtual SMTP - 1 225746

after-awk

Code:

external.dmn als.tld 225375

Final count: 0
Comment: als.tld traffic should be counted as dmn.tld traffic, but it isn't! We need to check the post-awk processing to account for this.

Test 4: redir@dmn.tld is a domain redirection to me@external.dmn

Code:

2008-06-17 23:24:20 me@external.dmn redir@dmn.tld test.server.tld 127.0.0.1 SMTP - 1 778299
2008-06-17 23:24:20 me@external.dmn me@external.dmn test.server.tld mx.external.dmn SMTP - 1 778670

after-awk

Code:

external.dmn dmn.tld 778299
external.dmn external.dmn 778670

Final count: 778299
Comment: Missing outgoing traffic (2nd line) because the logs don't show that the originating user is "redir@dmn.tld"

Test 5: redir@sub.dmn.tld is a subdomain redirection to me@external.dmn

Code:

2008-06-17 23:24:35 me@external.dmn redir@sub.dmn.tld test.server.tld 127.0.0.1 SMTP - 1 222781
2008-06-17 23:24:35 me@external.dmn me@external.dmn test.server.tld mx.external.dmn SMTP - 1 223152

after-awk

Code:

external.dmn sub.dmn.tld 222781
external.dmn external.dmn 223152

Final count: 222781
Comment: Missing outgoing traffic (2nd line) because the logs don't show that the originating user is "redir@sub.dmn.tld"

Test 6: redir@als.tld is an alias redirection to me@external.dmn

Code:

2008-06-17 23:24:44 me@external.dmn redir@als.tld test.server.tld 127.0.0.1 SMTP - 1 225383
2008-06-17 23:24:45 me@external.dmn me@external.dmn test.server.tld mx.external.dmn SMTP - 1 225754

after-awk:

Code:

external.dmn als.tld 225383
external.dmn external.dmn 225754

Final count: 0
Comment: Missing incoming traffic for the same reason as test 3 and outgoing traffic (2nd line) because the logs don't show that the originating user is "redir@als.tld"

Final comments
The missing alias counts should be an easy fix and the missing info on logs is much harder to track as it would involve fixing the awstats maillogconvert script.

PS to ispcomm: I'll rerun the tests tomorrow with a realyed domain (mx patch) to get some more results.