Current time: 07-03-2020, 08:28 AM Hello There, Guest! (LoginRegister)


Post Reply 
[HowTo] Rsync backup&restore + remote backup
Author Message
Kika Offline
Member
***

Posts: 293
Joined: Feb 2007
Reputation: 8
Post: #1
[HowTo] Rsync backup&restore + remote backup
UPDATED for ispCP 1.0.3-1

I did the rsync backup modification in the backup script, because i had too many data for packing everything day by day The old method use lot of resources for a long time (and maybe decrease the lifetime of the HDDs). With this modification the first backup was 2 hours on my sytem (the gzip backup was much longer), but the second was only 7 minutes, because this script backup only the databases and the modificated files.

This script create a folder in the {$DOMAIN_DIR}/backups and syncronizing every file to it without the log and the phptmp.

For database backup this script will use gzip (just because i like this, not the bzip2), but you can change this.

BACKUP

open /etc/ispcp/ispcp.conf

search for (about line 146)
Code:
ZIP =

replace this line with
Code:
ZIP = rsync



open /var/www/ispcp/engine/backup/ispcp-backup-all

search for (about line 161)
Code:
} elsif ($zip eq "gzip") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";

replace with
Code:
} elsif ($zip eq "gzip"  || $zip eq "rsync") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";


search for (about line 267)
Code:
if( $zip eq 'bzip2' || $zip eq 'gzip' || $zip eq 'lzma') {
                                $backup_filename = "$dmn_name-backup-$date.tar.".($zip eq 'bzip2' ? 'bz2' : ($zip eq 'gzip' ? 'gz' : $zip));
                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
                                        $backup_cmd = "$cmd_tar --create --directory=$dmn_dir --$zip --file=$dmn_dir/$backup_filename --exclude=logs --exclude=phptmp --exclude=backups --exclude=$backup_filen
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }

insert after
Code:
} elsif( $zip eq 'rsync') {
            $backup_filename = "";

                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
            $backup_cmd = "rsync -azv --delete-excluded  --exclude=logs --exclude=phptmp --exclude=backups $dmn_dir $dmn_dir/backups";
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }


searching for (about line 292)
Code:
        if ($rs == 0) { # everything ok

insert before
Code:
        if ($zip ne "rsync") {

searching for (about line 319)
Code:
send_error_mail('backup_all_engine()', "Domain $dmn_name: Error while removing file $dmn_dir/$backup_filename!");

                                        }

                                }

insert after
Code:
}



open /var/www/ispcp/engine/backup/ispcp-backup-ispcp

search for (about line 211)
Code:
my ($db_filename, $db_compresscmd) = (undef, undef);
insert after
Code:
if ($zip eq "rsync") {
            $zip = "gzip";
        }


search for (about line 274)
Code:
my $zip = $main::cfg{'ZIP'};

insert after
Code:
if ($zip eq "rsync") {
            $zip = "gzip";
        }



open /var/www/ispcp/engine/quota/ispcp-dsk-quota

search for
Code:
my $s = `$cmd_du --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;

replace with
Code:
            my $s = `$cmd_du --exclude="backups/$domain_name" --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;


RESTORE

open /var/www/ispcp/engine/ispcp-dmn-mngr

search for (about line 3567)
Code:
################################################################################​
##
## Restore Backups Data
##
sub dmn_restore_data {
  my ($dmn_data) = @_;

  push_el(\@main::el, 'dmn_restore_data()', 'Starting...');
  if (!defined($dmn_data) || $dmn_data eq '') {
    push_el(\@main::el, 'dmn_restore_data()', 'ERROR: Undefined Input Data...');
    return -1;
  }

  my ($rs, $rdata) = ('', '');
  my ($rs0, $rdata0) = ('','');
  my $dmn_id       = @$dmn_data[0];
  my $dmn_name     = @$dmn_data[1];
  my $dmn_uid<---> = @$dmn_data[3];
  my $www_dir      = $main::cfg{'APACHE_WWW_DIR'};
  my $dmn_dir      = "$www_dir/$dmn_name";
  my $dmn_bk_dir   = "$www_dir/$dmn_name/backups";
  my $cmd_tar      = $main::cfg{'CMD_TAR'};
  my $cmd_lzma     = $main::cfg{'CMD_LZMA'};
  my $cmd_bzcat    = $main::cfg{'CMD_BZCAT'};
  my $cmd_gzcat    = $main::cfg{'CMD_GZCAT'};
  my $cmd_mysql    = $main::cfg{'CMD_MYSQL'};
  my $cmd_chown<-> = $main::cfg{'CMD_CHOWN'};

insert after
Code:
my $zip = $main::cfg{ZIP};
  my $restore_cmd = undef;

  # Restore rsync
  if ($zip eq "rsync") {
    $rs = opendir(DIR, "$dmn_bk_dir/$dmn_name");
    if (!$rs) {
      push_el(\@main::el, 'dmn_restore_data()', "ERROR: Can't open '$dmn_bk_dir/$dmn_name' directory.");
      return -1;
    }
    my @bk_files = readdir(DIR);
    closedir(DIR);
    return 0 if (scalar(@bk_files) == 0);
    foreach(@bk_files) {
      my ($fname) = ($_);
      if ($fname ne "." && $fname ne "..") {
        if ( -d "$dmn_bk_dir/$dmn_name/$fname" ) {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname/ $dmn_dir/$fname/";
          $rs = sys_command($restore_cmd);
        } else {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname $dmn_dir/$fname";
          $rs = sys_command($restore_cmd);
        }
      }
    }
  }


REMOTE BACKUP

If you want to create a remote backup you can do that with this simple steps below. (This is not need for the local rsync backup)

1, Generate the rsa key
Code:
ispCP server:
ssh-keygen -N '' -C backup1 -t rsa -f ~/.ssh/backup
scp ~/.ssh/backup.pub server:.ssh

Remote backup server:
cd ~/.ssh
cat backup.pub >> authorized_keys
rm backup.pub

2, Create backup.sh file on your ispCP server:
Code:
#!/bin/sh
for i in /var/www/virtual/*; do
  domain=${i:17:${#i}};
  rsync -v --delete --delete-excluded --timeout=999 -az -e 'ssh -c blowfish -i /root/.ssh/backup -ax -o ClearAllForwardings=yes' /var/www/virtual/$domain/backups server.tld:backup_dir/$domain/
done
Don't forget to replace the "server.tld:backup_dir" to your backup server.

3, Add this script to your crontab after the ispCP backup

4, Run backup.sh from command line at the first time (because the new key)
(This post was last modified: 01-03-2010 01:44 AM by Kika.)
12-16-2008 09:14 AM
Find all posts by this user Quote this message in a reply
Post Reply 


Messages In This Thread
[HowTo] Rsync backup&restore + remote backup - Kika - 12-16-2008 09:14 AM

Forum Jump:


User(s) browsing this thread: 1 Guest(s)