ispCP - Board - Support
Mail relay to customer server - Printable Version

+- ispCP - Board - Support (http://www.isp-control.net/forum)
+-- Forum: ispCP Omega Development Area (/forum-1.html)
+--- Forum: General discussion (/forum-11.html)
+--- Thread: Mail relay to customer server (/thread-3331.html)

Pages: 1 2 3


RE: Mail relay to customer server - ispcomm - 05-20-2008 10:48 PM

We're moving OT now... but are you saying you have 10K accesses that pass blacklists??? I have the restrictions in this order:
- formal checks
- blacklist
- greylist
- final auth

This takes the burden on my shoulders but 100K-500K greylisted addresses are OK. I expire them quickly if no delivery is tried for 6 hours after greylisting them.

But 10K is within the possibilities on any exchange server. That is 1 check every 8.64 seconds. Easy as a cake...

ispcomm


RE: Mail relay to customer server - simple - 05-21-2008 12:51 AM

kilburn Wrote:As I'm only offering this service to customers who have their own Exchange server, right now I'm using a quite different approach. I install them a script that exports the valid mail addresses from AD, uploads them to the server (scp) and runs a sudoed script that fills the relay_recipients table and reloads postfix. If anyone is interested in the actual scripts, etc. I could write a how-to....

That would indeed be a nice howto, do you already have some pieces somewhere or would that be completely from scratch?


RE: Mail relay to customer server - kilburn - 05-21-2008 07:07 AM

I've to write it from scratch, so I'll probably release a first draft for more experienced admins (those who already know how to use putty, set ssh certificate auth, etc.) and extend it later to provide detailed explanations of this topics.


RE: Mail relay to customer server - ispcomm - 06-04-2008 07:59 PM

Just a quick update to let you know that I'm starting development on this issue, so I can do it properly and will attach patches in this ticket: http://www.isp-control.net/ispcp/ticket/1332

ispcomm


RE: Mail relay to customer server - ispcomm - 06-10-2008 02:14 AM

I'm attaching here a working patch for the MX relay feature.

To use it update the database (via gui). In the "edit domain" page of the reseller you'll find a new field on the bottom called "MX Relay". That is the FQDN or IP address of the server that will receive the mail coming from outside.

I have not provided (yet) an option to enable this feature from the "add user" page.

The patch is against trunk (r1204, pre-rc5). It's not big deal, but touches several places (dbase, gui, templates, engine), so I'm explaining it here:

1. A new database update: I have provided the necessary functions to upgrade the database to version 5 from the gui (just click dbase update and you're set).
2. Change in the engine to account for the new feature from dbase
3. Change in the ispcp-functions (gui/include)
4. Change in the reseller edit domain (the new option is on the bottom).
5. New translation is necessary for the new fields (currently english only).

Inner workings:

I'm using the postfix transport table (which was currently unused) to place the mx relay server. When the "mx relay" variable is set to a value different to the default ("_no_") the domain is removed from the "domain" table and a transport entry is placed in the "transport" table.

The transport table is used for unknown things in "ispcp-mbox-mngr". My patch concentrates on the ispcp-dmn-mngr. There're no conflicts between the tables and the patch is much lighter this way (as opposted to adding an extra table and backup to ALL configurations of ALL distributions).

The functionality of aliases, pop3 accounts etc is left untouched (but not guaranteed to work when MX is used). It's better to delete all accounts and aliases before using the MX relay.

The patch has been tested on debian etch but should have no problems on other platforms as well.

feedback is welcome.

ispcomm.


RE: Mail relay to customer server - kilburn - 06-10-2008 04:09 AM

First things first: thanks and congratz ispcomm!

Now the annoying part: the patch doesn't apply succesfully against trunk (r1204):
Code:
v2:/usr/local/src/ispcp# svn up
At revision 1204.
v2:/usr/local/src/ispcp# svn st
v2:/usr/local/src/ispcp# patch -p0 < ../mxpatch.txt
patching file engine/ispcp-dmn-mngr
Hunk #1 FAILED at 2079.
Hunk #2 FAILED at 2109.
Hunk #3 FAILED at 2132.
Hunk #4 FAILED at 2202.
Hunk #5 FAILED at 2239.
Hunk #6 succeeded at 3637 with fuzz 2 (offset -24 lines).
5 out of 6 hunks FAILED -- saving rejects to file engine/ispcp-dmn-mngr.rej
patching file gui/include/database-update-functions.php
patching file gui/include/ispcp-functions.php
patching file gui/reseller/edit_domain.php
patching file gui/themes/omega_original/reseller/edit_domain.tpl

The patch itself looks great but I've a few comments:

-) Relay is always set to smtp:[relay_value]. I would suggest leaving the brackets out so the user may set an MX relay (no brackets) or A relay (with brackets).

ispcomm Wrote:The functionality of aliases, pop3 accounts etc is left untouched (but not guaranteed to work when MX is used). It's better to delete all accounts and aliases before using the MX relay.

-) Just don't allow to set relaying if there are any mail accounts and/or aliases for this domain (this is a gui-only check that shouldn't take much effort).

-) Shouldn't we add "reject_unverified_recipients" to the smtpd_recipient_restrictions in the default postfix config?

Edit:

-) When HOSTING_PLANS_LEVEL is set to 'admin' domain edition is made through 'gui/admin/edit_domain.php' (and it's template 'gui/templates/omega_original/admin/edit_domain.tpl') so this files should also be modified.


RE: Mail relay to customer server - ispcomm - 06-10-2008 07:29 AM

Would you attach or copy/paste the "engine/ispcp-dmn-mngr.rej" file so I can have a look at it?

The patch rejects are probably due to white space or the fact that my ispcp-dmn-mngr has other things boiling in it and I'm currently working on a merged svn source (or I can switch to pure trunk and try to patch myself). Can you try to apply with a bigger fuzz like -F 3 or 4 ?

On the other points you raise I mostly agree. MX vs A relay might be usefull (but I've never used MX yet).

The aliases mappings might be usefull too (I'm not sure they're used but I guess so) to tweak some incoming addresses on the server.

I prefer to add reject_unverified_recipients by hand. It would be nice to be able to do it on a per-domain basis (is this possible in postfix)?

The admin templates can be changed too. Not done yet as it's another feature I never use. I normally make a reseller for myself too.

Happy to see that the patch looks OK as I tried to keep is as clean as possible.

Added: I can confirm that my ispcp-dmn-mngr has quite a few white-space issues compared to the trunk (remainings from the beautifying patch). I'll have to fix them before resubmitting a new patch Sad


RE: Mail relay to customer server - kilburn - 06-10-2008 05:23 PM

Still no luck with an increased fuzz (you can get the current file directly from the svn browser):

Code:
v2:/usr/local/src/ispcp# patch -F4 -p0 < ../mxpatch.txt
patching file engine/ispcp-dmn-mngr
Hunk #1 FAILED at 2079.
Hunk #2 succeeded at 2088 with fuzz 3 (offset -21 lines).
Hunk #3 FAILED at 2111.
Hunk #4 succeeded at 2181 with fuzz 3 (offset -21 lines).
Hunk #5 succeeded at 2218 with fuzz 3 (offset -21 lines).
Hunk #6 succeeded at 3637 with fuzz 2 (offset -24 lines).
2 out of 6 hunks FAILED -- saving rejects to file engine/ispcp-dmn-mngr.rej
patching file gui/include/database-update-functions.php
patching file gui/include/ispcp-functions.php
patching file gui/reseller/edit_domain.php
patching file gui/themes/omega_original/reseller/edit_domain.tpl

I've (a pair of) customers that use backup inet providers so when their main connection goes down mail should be served through another IP. Using an MX relay this is easy as cake Wink

ispcomm Wrote:I prefer to add reject_unverified_recipients by hand. It would be nice to be able to do it on a per-domain basis (is this possible in postfix)?

This is untested so it may not work, but you'll get the idea:

main.cf:
Code:
...
smtpd_recipient_restrictions = reject_non_fqdn_recipient,
                               reject_unknown_recipient_domain,
                               permit_mynetworks,
                               permit_sasl_authenticated,
                               reject_unauth_destination,
+                              check_recipient_access hash:/etc/postfix/ispcp/relayed_domains
                               reject_unlisted_recipient,
                               check_policy_service inet:127.0.0.1:12525,
                               check_policy_service inet:127.0.0.1:60000,
                               permit
+ smtpd_restriction_classes  = verify_relayed
+ verify_relayed             = reject_unverified_recipients,
+                              check_policy_service inet:127.0.0.1:12525,
+                              check_policy_service inet:127.0.0.1:60000,
+                              permit
...

/etc/postfix/ispcp/relayed_domains:
Code:
relayed-domain1.tld      verify_relayed
relayed-domain2.tld      verify_relayed
....

Hope this helps, keep up the good work ispcomm! Thanks again!


RE: Mail relay to customer server - ispcomm - 06-10-2008 06:28 PM

I can confirm that the failures in the dmn-mngr are due to some extra beautifying on my own code which is not in trunk (yet). I'll push it there via a ticket, but in the mean time here's a patch agains trunk that works so you can give back some feedback (dmn-mngr only)
Code:
--- ispcp-dmn-mngr    (/mirror/ispcp/trunk/engine)    (revision 1339)
+++ ispcp-dmn-mngr    (/local/ispcp/trunk/engine)    (local)
@@ -2059,6 +2080,7 @@
    }

    my $dmn_name        = @$dmn_data[1];
+    my $dmn_relay             = @$dmn_data[22];
    my $conf_dir        = $main::cfg{'CONF_DIR'};
    my $cmd_postmap     = $main::cfg{'CMD_POSTMAP'};
    my $tpl_dir         = "$conf_dir/postfix/parts";
@@ -2068,6 +2090,9 @@
    my $working_cfg     = "$working_dir/domains";
    my $timestamp       = time;
    my $backup_cfg      = "$backup_dir/domains.$timestamp";
+    my $transport_cfg         = $main::cfg{'MTA_TRANSPORT_HASH'};
+    my $transport_working_cfg = "$working_dir/transport";
+    my $transport_backup_cfg  = "$backup_dir/transport.$timestamp";
    my ($sys, $working) = (undef, undef);

    #
@@ -2084,8 +2109,8 @@
    #
    $working =~ s/^$dmn_name\t\t\t[^\n]+\n//gim;

-    # Add domain entry only if mail accounts are activated
-    if (@$dmn_data[8] >= 0) {
+    # Add domain entry only if mail accounts are activated and domain relay is off (i.e. deliver mail locally)
+    if (@$dmn_data[8] >= 0 && $dmn_relay eq "_no_" ) {
        $working .= "$dmn_name\t\t\tvdmn_entry\n";
    }

@@ -2108,6 +2133,48 @@
    $rs = sys_command("$cmd_postmap $sys_cfg");
    return $rs if ($rs != 0);

+    #
+    # Transport Table handling (for domain mx relay handling)
+    #
+    push_el(\@main::el, 'dmn_add_mta_cfg_data()', 'Transport Table...');
+    
+    ($rs, $sys) = get_file($transport_cfg);
+    return $rs if ($rs != 0);
+    
+    ($rs, $working) = get_file($transport_working_cfg);
+    return $rs if ($rs != 0);
+    
+    #
+    # Adding to the transport table in case mxrelay <> '_no_'
+    #
+    $working =~ s/^$dmn_name[\s]+[^\n]+\n//gim;
+    
+    if ( $dmn_relay ne "_no_" ) {
+        $working .= "$dmn_name\t\t\tsmtp:[$dmn_relay]\n";
+    }
+
+    #
+    # Let's do some backup first;
+    #
+    $rs = store_file($transport_backup_cfg, $sys, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
+    return $rs if ($rs != 0);
+    
+    #
+    # Let's write configs;
+    #
+    
+    $rs = store_file($transport_working_cfg, $working, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
+    return $rs if ($rs != 0);
+    
+    $rs = store_file($transport_cfg, $working, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
+    return $rs if ($rs != 0);
+    
+    $rs = sys_command("$cmd_postmap $transport_cfg");
+    return $rs if ($rs != 0);
+    
+    #
+    # Finishing off.
+    #
    push_el(\@main::el, 'dmn_add_mta_cfg_data()', 'Ending...');
    return 0;
}
@@ -2135,6 +2202,9 @@
    my $working_cfg     = "$working_dir/domains";
    my $timestamp       = time;
    my $backup_cfg      = "$backup_dir/domains.$timestamp";
+    my $transport_cfg         = $main::cfg{'MTA_TRANSPORT_HASH'};
+    my $transport_working_cfg = "$working_dir/transport";
+    my $transport_backup_cfg  = "$backup_dir/transport.$timestamp";
    my ($sys, $working) = (undef, undef);

    #
@@ -2169,6 +2239,38 @@
    $rs = sys_command("$cmd_postmap $sys_cfg");
    return $rs if ($rs != 0);

+    #
+    # Transport Table handling (for domain mx relay handling)
+    #
+    push_el(\@main::el, 'dmn_del_mta_cfg_data()', 'Transport Table...');
+    
+    ($rs, $sys) = get_file($transport_cfg);
+    return $rs if ($rs != 0);
+    
+    ($rs, $working) = get_file($transport_working_cfg);
+    return $rs if ($rs != 0);
+    
+    $working =~ s/^$dmn_name[\s]+[^\n]+\n//gim;
+
+    #
+    # Let's do some backup first;
+    #
+    $rs = store_file($transport_backup_cfg, $sys, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
+    return $rs if ($rs != 0);
+    
+    #
+    # Let's write configs;
+    #
+    
+    $rs = store_file($transport_working_cfg, $working, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
+    return $rs if ($rs != 0);
+    
+    $rs = store_file($transport_cfg, $working, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
+    return $rs if ($rs != 0);
+    
+    $rs = sys_command("$cmd_postmap $transport_cfg");
+    return $rs if ($rs != 0);
+
    push_el(\@main::el, 'dmn_del_mta_cfg_data()', 'Ending...');
    return 0;
}
@@ -3556,7 +3661,7 @@
        t1.domain_created_id, t1.domain_created, t1.domain_last_modified, t1.domain_mailacc_limit, t1.domain_ftpacc_limit,
        t1.domain_traffic_limit, t1.domain_sqld_limit, t1.domain_sqlu_limit, t1.domain_status, t1.domain_alias_limit,
        t1.domain_subd_limit, t1.domain_ip_id, t1.domain_disk_limit, t1.domain_disk_usage, t1.domain_php,
-        t1.domain_cgi, t2.ip_number FROM domain AS t1, server_ips AS t2 WHERE t1.domain_ip_id = t2.ip_id
+        t1.domain_cgi, t2.ip_number, t1.domain_mxrelay FROM domain AS t1, server_ips AS t2 WHERE t1.domain_ip_id = t2.ip_id
        AND t1.domain_id = $main::dmn_task_id";

    ($rs, $rows) = doSQL($sql);
ADDED: For some reason attachments didn't work so I had to quote the whole patch. sorry.

ispcomm.


RE: Mail relay to customer server - kilburn - 06-10-2008 11:35 PM

Your last patch wasn't working either so I just "manually" patched the failed chunks and created my own one (attached) that should work agains trunk.

While testing it, I've seen that when assigning a relay IP to a domain it gets removed from the "domains" table but not added to the "transport" one! :S I've tried to look at the source but the only "maybe" relevant part is:

Code:
($rs, $sys) = get_file($transport_cfg);
    return $rs if ($rs != 0);
    
    ($rs, $working) = get_file($transport_working_cfg);
    return $rs if ($rs != 0);
    
    $working =~ s/^$dmn_name[\s]+[^\n]+\n//gim;

    #
    # Let's do some backup first;
    #
    $rs = store_file($transport_backup_cfg, $sys, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
    return $rs if ($rs != 0);
    
    #
    # Let's write configs;
    #
    
    $rs = store_file($transport_working_cfg, $working, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
    return $rs if ($rs != 0);
    
    $rs = store_file($transport_cfg, $working, $main::cfg{'ROOT_USER'}, $main::cfg{'ROOT_GROUP'}, 0644);
    return $rs if ($rs != 0);

But I don't see how is this adding the domain to the transport table... maybe I misspatched something?