[HowTo] Rsync backup&restore + remote backup

[Kika]

UPDATED for ispCP 1.0.3-1

I did the rsync backup modification in the backup script, because i had too many data for packing everything day by day The old method use lot of resources for a long time (and maybe decrease the lifetime of the HDDs). With this modification the first backup was 2 hours on my sytem (the gzip backup was much longer), but the second was only 7 minutes, because this script backup only the databases and the modificated files.

This script create a folder in the {$DOMAIN_DIR}/backups and syncronizing every file to it without the log and the phptmp.

For database backup this script will use gzip (just because i like this, not the bzip2), but you can change this.

BACKUP

open /etc/ispcp/ispcp.conf

search for (about line 146)

Code:

ZIP =

replace this line with

Code:

ZIP = rsync

open /var/www/ispcp/engine/backup/ispcp-backup-all

search for (about line 161)

Code:

} elsif ($zip eq "gzip") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";

replace with

Code:

} elsif ($zip eq "gzip"  || $zip eq "rsync") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";

search for (about line 267)

Code:

if( $zip eq 'bzip2' || $zip eq 'gzip' || $zip eq 'lzma') {
                                $backup_filename = "$dmn_name-backup-$date.tar.".($zip eq 'bzip2' ? 'bz2' : ($zip eq 'gzip' ? 'gz' : $zip));
                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
                                        $backup_cmd = "$cmd_tar --create --directory=$dmn_dir --$zip --file=$dmn_dir/$backup_filename --exclude=logs --exclude=phptmp --exclude=backups --exclude=$backup_filen
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }

insert after

Code:

} elsif( $zip eq 'rsync') {
            $backup_filename = "";

                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
            $backup_cmd = "rsync -azv --delete-excluded  --exclude=logs --exclude=phptmp --exclude=backups $dmn_dir $dmn_dir/backups";
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }

searching for (about line 292)

Code:

        if ($rs == 0) { # everything ok

insert before

Code:

        if ($zip ne "rsync") {

searching for (about line 319)

Code:

send_error_mail('backup_all_engine()', "Domain $dmn_name: Error while removing file $dmn_dir/$backup_filename!");

                                        }

                                }

insert after

Code:

}

open /var/www/ispcp/engine/backup/ispcp-backup-ispcp

search for (about line 211)

Code:

my ($db_filename, $db_compresscmd) = (undef, undef);

insert after

Code:

if ($zip eq "rsync") {
            $zip = "gzip";
        }

search for (about line 274)

Code:

my $zip = $main::cfg{'ZIP'};

insert after

Code:

if ($zip eq "rsync") {
            $zip = "gzip";
        }

open /var/www/ispcp/engine/quota/ispcp-dsk-quota

search for

Code:

my $s = `$cmd_du --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;

replace with

Code:

            my $s = `$cmd_du --exclude="backups/$domain_name" --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;

RESTORE

open /var/www/ispcp/engine/ispcp-dmn-mngr

search for (about line 3567)

Code:

################################################################################​
##
## Restore Backups Data
##
sub dmn_restore_data {
  my ($dmn_data) = @_;

  push_el(\@main::el, 'dmn_restore_data()', 'Starting...');
  if (!defined($dmn_data) || $dmn_data eq '') {
    push_el(\@main::el, 'dmn_restore_data()', 'ERROR: Undefined Input Data...');
    return -1;
  }

  my ($rs, $rdata) = ('', '');
  my ($rs0, $rdata0) = ('','');
  my $dmn_id       = @$dmn_data[0];
  my $dmn_name     = @$dmn_data[1];
  my $dmn_uid<---> = @$dmn_data[3];
  my $www_dir      = $main::cfg{'APACHE_WWW_DIR'};
  my $dmn_dir      = "$www_dir/$dmn_name";
  my $dmn_bk_dir   = "$www_dir/$dmn_name/backups";
  my $cmd_tar      = $main::cfg{'CMD_TAR'};
  my $cmd_lzma     = $main::cfg{'CMD_LZMA'};
  my $cmd_bzcat    = $main::cfg{'CMD_BZCAT'};
  my $cmd_gzcat    = $main::cfg{'CMD_GZCAT'};
  my $cmd_mysql    = $main::cfg{'CMD_MYSQL'};
  my $cmd_chown<-> = $main::cfg{'CMD_CHOWN'};

insert after

Code:

my $zip = $main::cfg{ZIP};
  my $restore_cmd = undef;

  # Restore rsync
  if ($zip eq "rsync") {
    $rs = opendir(DIR, "$dmn_bk_dir/$dmn_name");
    if (!$rs) {
      push_el(\@main::el, 'dmn_restore_data()', "ERROR: Can't open '$dmn_bk_dir/$dmn_name' directory.");
      return -1;
    }
    my @bk_files = readdir(DIR);
    closedir(DIR);
    return 0 if (scalar(@bk_files) == 0);
    foreach(@bk_files) {
      my ($fname) = ($_);
      if ($fname ne "." && $fname ne "..") {
        if ( -d "$dmn_bk_dir/$dmn_name/$fname" ) {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname/ $dmn_dir/$fname/";
          $rs = sys_command($restore_cmd);
        } else {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname $dmn_dir/$fname";
          $rs = sys_command($restore_cmd);
        }
      }
    }
  }

REMOTE BACKUP

If you want to create a remote backup you can do that with this simple steps below. (This is not need for the local rsync backup)

1, Generate the rsa key

Code:

ispCP server:
ssh-keygen -N '' -C backup1 -t rsa -f ~/.ssh/backup
scp ~/.ssh/backup.pub server:.ssh

Remote backup server:
cd ~/.ssh
cat backup.pub >> authorized_keys
rm backup.pub

2, Create backup.sh file on your ispCP server:

Code:

#!/bin/sh
for i in /var/www/virtual/*; do
  domain=${i:17:${#i}};
  rsync -v --delete --delete-excluded --timeout=999 -az -e 'ssh -c blowfish -i /root/.ssh/backup -ax -o ClearAllForwardings=yes' /var/www/virtual/$domain/backups server.tld:backup_dir/$domain/
done

Don't forget to replace the "server.tld:backup_dir" to your backup server.

3, Add this script to your crontab after the ispCP backup

4, Run backup.sh from command line at the first time (because the new key)

[Mouv]

Excellent... I test this during day...

Thank you

[mydebians]

Yes very nice, i'll test too today or tomorrow
bests regards

[Quemeros]

Hi Kika,
...It's for RC7, right?
Very good mod ... in this and next week, i will try it and think new ideas.

[Kika]

Yes, this version was tested with RC7, but works with RC6 too (maybe the line numbers not equal).

[itanium]

Thanks for this mod!

but now, how does the restore backup script work?

[Kika]

The users could restore that files what he need via FTP, because if somebody want to restore something than usualy he need only some little file what he can download and upload to the good place.

If i will have more time than i will create a restore script to. (i never used the restore script, but i will try soon)

[itanium]

Thank you for your answer. I will look to change the restoration script

[Arturo]

someone continue using this tutorial?

[Kika]

I using this with the 1.0 too, but i had't time to see the automatic restore script. If somebody want to restore anything then he could use ftp