Mail traffic accounting

[kilburn]

After some brainstorming I've come up with this possible solution to the unaccounted forwarded messages, based on the fact that any incoming mail to a forwarded address will generate mail_size * number of forwarders bytes of outgoing mail:

1) build an in-memory table of (forwarded_address, n_recipients) at the beggning of traffic accounting.
2) For every incoming smtp check if the address is in the in-memory table. If it is, add size * n_recipients as traffic for this mail account.
3) Ignore any non-localdomain to non-localdomain delivery

Example with test case 4:
In-memory built table has a record (redir@dmn.tld, 1) because "redir@dmn.tld" is forwarded to 1 address.

Code:

2008-06-17 23:24:20 me@external.dmn redir@dmn.tld test.server.tld 127.0.0.1 SMTP - 1 778299
2008-06-17 23:24:20 me@external.dmn me@external.dmn test.server.tld mx.external.dmn SMTP - 1 778670

after-"modified awk script" (as after-awk we don't have the address anymore)

Code:

external.dmn dmn.tld 1556598
external.dmn external.dmn 778670 <- Ignored

Final count: 1556598 (778299 + 778299*1)
REAL count: 1556969 (778299 + 778670)
Comment: The missing bytes are those of the headers added by our mail server + mail filter.

This solution is far easier to implement than tracking queue-id's (have you ever tried to do this? there are times when logs are not in real processing order so you have references to not-yet-seen queue-id's and such...).

The overhead is also minimal, as there's only 1 extra query, a really short in-memory record for each forwarded address and a lookup for each incoming record.

Comments?