ispcp needs backup engine overhaul...

[pgentoo]

I'm having some issues with backups my boxes. Mostly around performance when backups are running, pretty much bringing my websites to a halt due to high i/o wait %'s.

First, it seems that backups settings (full, db only, etc) can only be configured when adding a user, and not after the fact?

More importantly, currently backups are handled to the same logical volume as the websites (/var/www/virtual, for example) just under a backups folder, which means that backups thrash heavily reading and writing from the same volume. In a setup that isn't backed by very fast spindles (and a lot of them), this causes a lot of load on the system, and can slow down sites to the point where they stop responding. Further, it would be better use of resources to say put backups on a not so fault tolerant set of spindles (say RAID0 instead of RAID10) to further speed up the backup operations. Ideally we would natively have support for putting backups off on another server via rsync over ssh or something to get them off the box should something really bad happen.

I also have some sites that write data into large files (think sqlite, application specific log files, etc) and if files are modified while the tar operation is in process, the backup breaks entirely...

I'd like to propose the idea of moving to an (optional?) mirrored type backup, where files can be rsync'd to an identical copy on a specified folder (or even backup server in the future). Once rsync'd over, they could be tar'd, and optionally zipped (lzma, bzip2, gzip, etc) for a daily copy. I used this approach on a server/control panel that i developed myself back in the day, and it worked out very well, removing the i/o wait issues from the "production" volumes, and of to other spindles where load would not affect runtime performance (aside from cpu, but that can be mitigated with 'nice').

Backups could be off on the other volume, and ispcp could create symlinks out to the users folder under the backups volume, so that retrieval via ftp and recovery could still work the same.

I think backups are a big weakness in ispcp right now once you start adding more than few sites, especially if the sites have a lot of files (think large photogalleries, etc). I know that personally backups are not running (completely) for me on several boxes due to the issues I mentioned, which means that i have to run my own rsync's and such to get data to a safe place.

I'm really looking for feedback on this, to see what people think, and to see if others are having the same issues. If its a wide spread thing, I really think some effort should be focused in this area to make ispcp a more robust product.

Thoughts?