[HowTo] Rsync backup&restore + remote backup

[Kika]

UPDATED for ispCP 1.0.3-1

I did the rsync backup modification in the backup script, because i had too many data for packing everything day by day The old method use lot of resources for a long time (and maybe decrease the lifetime of the HDDs). With this modification the first backup was 2 hours on my sytem (the gzip backup was much longer), but the second was only 7 minutes, because this script backup only the databases and the modificated files.

This script create a folder in the {$DOMAIN_DIR}/backups and syncronizing every file to it without the log and the phptmp.

For database backup this script will use gzip (just because i like this, not the bzip2), but you can change this.

BACKUP

open /etc/ispcp/ispcp.conf

search for (about line 146)

Code:

ZIP =

replace this line with

Code:

ZIP = rsync

open /var/www/ispcp/engine/backup/ispcp-backup-all

search for (about line 161)

Code:

} elsif ($zip eq "gzip") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";

replace with

Code:

} elsif ($zip eq "gzip"  || $zip eq "rsync") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";

search for (about line 267)

Code:

if( $zip eq 'bzip2' || $zip eq 'gzip' || $zip eq 'lzma') {
                                $backup_filename = "$dmn_name-backup-$date.tar.".($zip eq 'bzip2' ? 'bz2' : ($zip eq 'gzip' ? 'gz' : $zip));
                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
                                        $backup_cmd = "$cmd_tar --create --directory=$dmn_dir --$zip --file=$dmn_dir/$backup_filename --exclude=logs --exclude=phptmp --exclude=backups --exclude=$backup_filen
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }

insert after

Code:

} elsif( $zip eq 'rsync') {
            $backup_filename = "";

                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
            $backup_cmd = "rsync -azv --delete-excluded  --exclude=logs --exclude=phptmp --exclude=backups $dmn_dir $dmn_dir/backups";
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }

searching for (about line 292)

Code:

        if ($rs == 0) { # everything ok

insert before

Code:

        if ($zip ne "rsync") {

searching for (about line 319)

Code:

send_error_mail('backup_all_engine()', "Domain $dmn_name: Error while removing file $dmn_dir/$backup_filename!");

                                        }

                                }

insert after

Code:

}

open /var/www/ispcp/engine/backup/ispcp-backup-ispcp

search for (about line 211)

Code:

my ($db_filename, $db_compresscmd) = (undef, undef);

insert after

Code:

if ($zip eq "rsync") {
            $zip = "gzip";
        }

search for (about line 274)

Code:

my $zip = $main::cfg{'ZIP'};

insert after

Code:

if ($zip eq "rsync") {
            $zip = "gzip";
        }

open /var/www/ispcp/engine/quota/ispcp-dsk-quota

search for

Code:

my $s = `$cmd_du --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;

replace with

Code:

            my $s = `$cmd_du --exclude="backups/$domain_name" --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;

RESTORE

open /var/www/ispcp/engine/ispcp-dmn-mngr

search for (about line 3567)

Code:

################################################################################​
##
## Restore Backups Data
##
sub dmn_restore_data {
  my ($dmn_data) = @_;

  push_el(\@main::el, 'dmn_restore_data()', 'Starting...');
  if (!defined($dmn_data) || $dmn_data eq '') {
    push_el(\@main::el, 'dmn_restore_data()', 'ERROR: Undefined Input Data...');
    return -1;
  }

  my ($rs, $rdata) = ('', '');
  my ($rs0, $rdata0) = ('','');
  my $dmn_id       = @$dmn_data[0];
  my $dmn_name     = @$dmn_data[1];
  my $dmn_uid<---> = @$dmn_data[3];
  my $www_dir      = $main::cfg{'APACHE_WWW_DIR'};
  my $dmn_dir      = "$www_dir/$dmn_name";
  my $dmn_bk_dir   = "$www_dir/$dmn_name/backups";
  my $cmd_tar      = $main::cfg{'CMD_TAR'};
  my $cmd_lzma     = $main::cfg{'CMD_LZMA'};
  my $cmd_bzcat    = $main::cfg{'CMD_BZCAT'};
  my $cmd_gzcat    = $main::cfg{'CMD_GZCAT'};
  my $cmd_mysql    = $main::cfg{'CMD_MYSQL'};
  my $cmd_chown<-> = $main::cfg{'CMD_CHOWN'};

insert after

Code:

my $zip = $main::cfg{ZIP};
  my $restore_cmd = undef;

  # Restore rsync
  if ($zip eq "rsync") {
    $rs = opendir(DIR, "$dmn_bk_dir/$dmn_name");
    if (!$rs) {
      push_el(\@main::el, 'dmn_restore_data()', "ERROR: Can't open '$dmn_bk_dir/$dmn_name' directory.");
      return -1;
    }
    my @bk_files = readdir(DIR);
    closedir(DIR);
    return 0 if (scalar(@bk_files) == 0);
    foreach(@bk_files) {
      my ($fname) = ($_);
      if ($fname ne "." && $fname ne "..") {
        if ( -d "$dmn_bk_dir/$dmn_name/$fname" ) {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname/ $dmn_dir/$fname/";
          $rs = sys_command($restore_cmd);
        } else {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname $dmn_dir/$fname";
          $rs = sys_command($restore_cmd);
        }
      }
    }
  }

REMOTE BACKUP

If you want to create a remote backup you can do that with this simple steps below. (This is not need for the local rsync backup)

1, Generate the rsa key

Code:

ispCP server:
ssh-keygen -N '' -C backup1 -t rsa -f ~/.ssh/backup
scp ~/.ssh/backup.pub server:.ssh

Remote backup server:
cd ~/.ssh
cat backup.pub >> authorized_keys
rm backup.pub

2, Create backup.sh file on your ispCP server:

Code:

#!/bin/sh
for i in /var/www/virtual/*; do
  domain=${i:17:${#i}};
  rsync -v --delete --delete-excluded --timeout=999 -az -e 'ssh -c blowfish -i /root/.ssh/backup -ax -o ClearAllForwardings=yes' /var/www/virtual/$domain/backups server.tld:backup_dir/$domain/
done

Don't forget to replace the "server.tld:backup_dir" to your backup server.

3, Add this script to your crontab after the ispCP backup

4, Run backup.sh from command line at the first time (because the new key)