Number of connections per IP / POP3

[rbtux]

(02-25-2010 07:48 PM)kilburn Wrote:  

Quote:well I would never choose a provider with quota below 1gig per mb...

Amazing. I'm having a hard time trying to offer such high quotas for mailboxes, mainly because:

(1) Your mileage may vary, but my customers tend to retain each and every mail they've ever received, including multi-megabyte stupid powerpoint/video jokes and the like.

yeah so what ;-)) I do too... As long as the pay for it...

(02-25-2010 07:48 PM)kilburn Wrote:  (2) Backing up *and* being able to quickly emergency-recover multiple (100+) gigabytes of data is not an easy thing to do given my server's limitations (100Mb/s connections). Backup problem: as the maildirs grow, the backup process changed files scan increases quickly (additional stats for each mail). SATA disks really hate processing millions of stat ops for individual files, so the thing turns into a slow beast unless I/O throttled which solves the issue but makes backups slower. Recovery problem: a simple calculation shows it: 100Mb/s ~ 44GB/hour (assuming an ideal network entirely dedicated to the restore process).

How do you handle these situations? Any ideas about them without throwing more/different hardware at it?

we do backups for our Maildir with lvm snapshots (Maildir is on a separate LVM Partition...) and rsync. We have our own Racks and therefore gigabit network between the servers and the backbone, as well as a separate gigabit link for backup purposes only, so no big issues for us there... Even with 100Mbps the backup process should not make big pain (a full restore is an other story though...)

(02-25-2010 07:48 PM)kilburn Wrote:  Additionally, IMAP clients usually keep open processes in the server. Obviously, people just leaves their mail clients open the whole day while they are working. This means that the number of open imap processes can easily grow to the hundreds, even thousands in busy hours. Do you want this happening to your servers?

Well thats what the mailservers are for...
But I must say dovecot has a really small memory footprint each connection takes between 150k and 500k (swapable memory) on a decent server you can handle 500-1500 concurrent connections... (of course main limitation is i/o here..., but the idlers use almost nothing but memory)
[/quote]

(02-25-2010 07:48 PM)kilburn Wrote:  Now you might tell me that I'm overloading my servers, but I have to strongly disagree. IMHO, the hardware requirements should grow mainly according to the numbers/bytes of mails received/sent, not to the number of mail accounts.

yes you're right.


By the way I did attend an interessting talk with Timo Sirainen (the guy behind dovecot) last july and he has some really interessting plans for dovecot. He is working on storage separation of current and old mails.

That means you could serve the 50 most current mails for each user from SAS disk and the rest (archive) from a slower resource like sata, nas or even some sort of cloud filesystem (like amazon s3 or the like)

Also dovecot can act as imap proxy, so if you really have a lot of mailaccounts you can still provide a single point of access for your users, with multiple mailhosts in the backend...

---

here is the video and pdf of the talk I mentiond above:
http://www.heinlein-support.de/web/akade...ap-server/