Current time: 07-03-2020, 04:42 AM Hello There, Guest! (LoginRegister)


Post Reply 
[HowTo] Rsync backup&restore + remote backup
Author Message
Kika Offline
Member
***

Posts: 293
Joined: Feb 2007
Reputation: 8
Post: #1
[HowTo] Rsync backup&restore + remote backup
UPDATED for ispCP 1.0.3-1

I did the rsync backup modification in the backup script, because i had too many data for packing everything day by day The old method use lot of resources for a long time (and maybe decrease the lifetime of the HDDs). With this modification the first backup was 2 hours on my sytem (the gzip backup was much longer), but the second was only 7 minutes, because this script backup only the databases and the modificated files.

This script create a folder in the {$DOMAIN_DIR}/backups and syncronizing every file to it without the log and the phptmp.

For database backup this script will use gzip (just because i like this, not the bzip2), but you can change this.

BACKUP

open /etc/ispcp/ispcp.conf

search for (about line 146)
Code:
ZIP =

replace this line with
Code:
ZIP = rsync



open /var/www/ispcp/engine/backup/ispcp-backup-all

search for (about line 161)
Code:
} elsif ($zip eq "gzip") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";

replace with
Code:
} elsif ($zip eq "gzip"  || $zip eq "rsync") {
                        $db_filename = "$db_backup_file.gz";
                        $db_compresscmd = "$main::cfg{'CMD_GZIP'} --force '$db_backup_file'";


search for (about line 267)
Code:
if( $zip eq 'bzip2' || $zip eq 'gzip' || $zip eq 'lzma') {
                                $backup_filename = "$dmn_name-backup-$date.tar.".($zip eq 'bzip2' ? 'bz2' : ($zip eq 'gzip' ? 'gz' : $zip));
                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
                                        $backup_cmd = "$cmd_tar --create --directory=$dmn_dir --$zip --file=$dmn_dir/$backup_filename --exclude=logs --exclude=phptmp --exclude=backups --exclude=$backup_filen
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }

insert after
Code:
} elsif( $zip eq 'rsync') {
            $backup_filename = "";

                                if ($allowbackup eq "full" || $allowbackup eq "dmn") {
            $backup_cmd = "rsync -azv --delete-excluded  --exclude=logs --exclude=phptmp --exclude=backups $dmn_dir $dmn_dir/backups";
                                }
                                if ( $allowbackup eq "full" || $allowbackup eq "sql" ) {
                                        $rs = backup_sql($dmn_id, $dmn_name, $domain_uid, $domain_gid);
                                }


searching for (about line 292)
Code:
        if ($rs == 0) { # everything ok

insert before
Code:
        if ($zip ne "rsync") {

searching for (about line 319)
Code:
send_error_mail('backup_all_engine()', "Domain $dmn_name: Error while removing file $dmn_dir/$backup_filename!");

                                        }

                                }

insert after
Code:
}



open /var/www/ispcp/engine/backup/ispcp-backup-ispcp

search for (about line 211)
Code:
my ($db_filename, $db_compresscmd) = (undef, undef);
insert after
Code:
if ($zip eq "rsync") {
            $zip = "gzip";
        }


search for (about line 274)
Code:
my $zip = $main::cfg{'ZIP'};

insert after
Code:
if ($zip eq "rsync") {
            $zip = "gzip";
        }



open /var/www/ispcp/engine/quota/ispcp-dsk-quota

search for
Code:
my $s = `$cmd_du --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;

replace with
Code:
            my $s = `$cmd_du --exclude="backups/$domain_name" --exclude="backups/$domain_name-*.lzma" --exclude="backups/$domain_name-*.bz2" --exclude="backups/$domain_name-*.gz" --exclude="backups/*.sql.lzma" --exclude="backups/*.sql.bz2" --exclude="backups/*.sql.gz" --exclude="logs/*.log" -s -B1 $main::cfg{APACHE_WWW_DIR}/$domain_name`;


RESTORE

open /var/www/ispcp/engine/ispcp-dmn-mngr

search for (about line 3567)
Code:
################################################################################​
##
## Restore Backups Data
##
sub dmn_restore_data {
  my ($dmn_data) = @_;

  push_el(\@main::el, 'dmn_restore_data()', 'Starting...');
  if (!defined($dmn_data) || $dmn_data eq '') {
    push_el(\@main::el, 'dmn_restore_data()', 'ERROR: Undefined Input Data...');
    return -1;
  }

  my ($rs, $rdata) = ('', '');
  my ($rs0, $rdata0) = ('','');
  my $dmn_id       = @$dmn_data[0];
  my $dmn_name     = @$dmn_data[1];
  my $dmn_uid<---> = @$dmn_data[3];
  my $www_dir      = $main::cfg{'APACHE_WWW_DIR'};
  my $dmn_dir      = "$www_dir/$dmn_name";
  my $dmn_bk_dir   = "$www_dir/$dmn_name/backups";
  my $cmd_tar      = $main::cfg{'CMD_TAR'};
  my $cmd_lzma     = $main::cfg{'CMD_LZMA'};
  my $cmd_bzcat    = $main::cfg{'CMD_BZCAT'};
  my $cmd_gzcat    = $main::cfg{'CMD_GZCAT'};
  my $cmd_mysql    = $main::cfg{'CMD_MYSQL'};
  my $cmd_chown<-> = $main::cfg{'CMD_CHOWN'};

insert after
Code:
my $zip = $main::cfg{ZIP};
  my $restore_cmd = undef;

  # Restore rsync
  if ($zip eq "rsync") {
    $rs = opendir(DIR, "$dmn_bk_dir/$dmn_name");
    if (!$rs) {
      push_el(\@main::el, 'dmn_restore_data()', "ERROR: Can't open '$dmn_bk_dir/$dmn_name' directory.");
      return -1;
    }
    my @bk_files = readdir(DIR);
    closedir(DIR);
    return 0 if (scalar(@bk_files) == 0);
    foreach(@bk_files) {
      my ($fname) = ($_);
      if ($fname ne "." && $fname ne "..") {
        if ( -d "$dmn_bk_dir/$dmn_name/$fname" ) {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname/ $dmn_dir/$fname/";
          $rs = sys_command($restore_cmd);
        } else {
          $restore_cmd = "rsync -azv --delete-excluded $dmn_bk_dir/$dmn_name/$fname $dmn_dir/$fname";
          $rs = sys_command($restore_cmd);
        }
      }
    }
  }


REMOTE BACKUP

If you want to create a remote backup you can do that with this simple steps below. (This is not need for the local rsync backup)

1, Generate the rsa key
Code:
ispCP server:
ssh-keygen -N '' -C backup1 -t rsa -f ~/.ssh/backup
scp ~/.ssh/backup.pub server:.ssh

Remote backup server:
cd ~/.ssh
cat backup.pub >> authorized_keys
rm backup.pub

2, Create backup.sh file on your ispCP server:
Code:
#!/bin/sh
for i in /var/www/virtual/*; do
  domain=${i:17:${#i}};
  rsync -v --delete --delete-excluded --timeout=999 -az -e 'ssh -c blowfish -i /root/.ssh/backup -ax -o ClearAllForwardings=yes' /var/www/virtual/$domain/backups server.tld:backup_dir/$domain/
done
Don't forget to replace the "server.tld:backup_dir" to your backup server.

3, Add this script to your crontab after the ispCP backup

4, Run backup.sh from command line at the first time (because the new key)
(This post was last modified: 01-03-2010 01:44 AM by Kika.)
12-16-2008 09:14 AM
Find all posts by this user Quote this message in a reply
Mouv Offline
Newbie
*

Posts: 8
Joined: Nov 2008
Reputation: 0
Post: #2
RE: [HowTo] Rsync backup + remote backup
Excellent... I test this during day...

Thank you
12-16-2008 04:57 PM
Find all posts by this user Quote this message in a reply
mydebians Offline
Junior Member
*

Posts: 27
Joined: Apr 2007
Reputation: 0
Post: #3
RE: [HowTo] Rsync backup + remote backup
Yes very nice, i'll test too today or tomorrow
bests regards
12-16-2008 06:58 PM
Find all posts by this user Quote this message in a reply
Quemeros Offline
Junior Member
*

Posts: 86
Joined: Nov 2007
Reputation: 0
Post: #4
RE: [HowTo] Rsync backup + remote backup
Hi Kika,
...It's for RC7, right?
Very good mod Smile... in this and next week, i will try it and think new ideas.
12-16-2008 10:38 PM
Find all posts by this user Quote this message in a reply
Kika Offline
Member
***

Posts: 293
Joined: Feb 2007
Reputation: 8
Post: #5
RE: [HowTo] Rsync backup + remote backup
Yes, this version was tested with RC7, but works with RC6 too (maybe the line numbers not equal).
12-16-2008 10:43 PM
Find all posts by this user Quote this message in a reply
itanium Offline
Junior Member
*

Posts: 35
Joined: Apr 2007
Reputation: 0
Post: #6
RE: [HowTo] Rsync backup + remote backup
Thanks for this mod! Smile

but now, how does the restore backup script work? Sad
12-24-2008 07:46 AM
Find all posts by this user Quote this message in a reply
Kika Offline
Member
***

Posts: 293
Joined: Feb 2007
Reputation: 8
Post: #7
RE: [HowTo] Rsync backup + remote backup
The users could restore that files what he need via FTP, because if somebody want to restore something than usualy he need only some little file what he can download and upload to the good place.

If i will have more time than i will create a restore script to. (i never used the restore script, but i will try soon)
(This post was last modified: 12-24-2008 08:58 AM by Kika.)
12-24-2008 08:55 AM
Find all posts by this user Quote this message in a reply
itanium Offline
Junior Member
*

Posts: 35
Joined: Apr 2007
Reputation: 0
Post: #8
RE: [HowTo] Rsync backup + remote backup
Thank you for your answer. I will look to change the restoration script
12-25-2008 12:03 PM
Find all posts by this user Quote this message in a reply
Arturo Offline
Junior Member
*

Posts: 77
Joined: Dec 2008
Reputation: 0
Post: #9
RE: [HowTo] Rsync backup + remote backup
someone continue using this tutorial?
05-03-2009 12:20 AM
Find all posts by this user Quote this message in a reply
Kika Offline
Member
***

Posts: 293
Joined: Feb 2007
Reputation: 8
Post: #10
RE: [HowTo] Rsync backup + remote backup
I using this with the 1.0 too, but i had't time to see the automatic restore script. If somebody want to restore anything then he could use ftp Wink
05-03-2009 01:01 AM
Find all posts by this user Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)