/bin/tar: .: file changed as we read it

[gandyman]

Hello.

first I stil use VHS2.
Why?
Wel I am very precise and my server runs a way I want and that take time.
So before I go over to ispCP I wil do this from the ground up.
New install Debian and then ispCP.
But need time for that, and from everything I have is time hard to find.

All the way glad to see ppl take the effort to go on with VHCS2, becouse all togheter its a shame to see such good isp software die.

BTW I am Dutch, so can write a bit weird Englisch.....

Owk, what is the problem.

Sinds a few monds now the backup dont work no more.
So the daily cronjob get stuck and leave me with a lock-file.

Read some around and seen a lot talk becouse the old gzip/tar version had probs.
( why it was working in the beginnen tough ?? )
Then some things from new kernels ??
Wel I uninstall gezip and install the newest version without sccs.

Stil looking ad the internet for a solution I saw you new script voor backups.
http://isp-control.net/ispcp/browser/tru...ll?rev=440

I see also that you use there "bzip2" instead "gzip" ad line 250.
And I see lot more improvements.......


Now my first question:
1. Can I use this vhcs2-backup-all script instead of the old one from VHCS2 ???
##################
Now my prob....
The backups get stuck as I say before with an error.

i give the command;

Code:

server:~# /var/www/vhcs2/engine/backup/vhcs2-backup-all yes

And it ends in:

Quote:/bin/tar: ./htdocs/modules/Postcalendar/pntemplates/cache: file changed as we read it
/bin/tar: .: file changed as we read it
ERROR: External command '/bin/tar --create --directory /var/www/virtual/www.mysite.com --gzip --file /var/www/virtual/www.mysite.com/www.mysite.com-backup-2008.01.11-153410.tar.gz --exclude=logs --exclude=backups --exclude=www.mysite.com-backup-2008.01.11-153410.tar.gz .' returned '1' status !

My second question:
2. wil this be solfed wen I use your vhcs2-backup-all ??
Becouse u use another program "bzip2" instead "gzip" ??

Or am I messed up and looking for the wrong thing ???
If yes .
Can someone help me out, becouse I cannot find much usefull on the internet......

Thx already.

[joximu]

Hi gandyman

please use the name ispcp if you refer to this project. There were complaints from the old vhcs name owner...

to the points:
- the tar command throws an error if the files are changing while they are to be saved, AFAIK this is still the case but this has to be solved (at least the following backups have to be done...)

/Joximu

[gandyman]

Lo Joximu,

Quote:please use the name ispcp if you refer to this project. There were complaints from the old vhcs name owner...

I wil, no worry.
Shame tough, while he is proud that ppl take the effort to continui his project.....
O-wel to complicated, he wil have his reasons. sigh, ??? ppl ???

Quote:to the points:
- the tar command throws an error if the files are changing while they are to be saved, AFAIK this is still the case but this has to be solved (at least the following backups have to be done...)

you lost me a bit. ??
hmz you say it is solved ??

And the following backups cant start becouse there is a backup_lock_file file, with I first must remove.

enyway:
Becouse I see a lot things with are inproved I stil have this question:
1. Can I use this ispcp-backup-all script in my old vhcs2 installation? , instead of the old one from VHCS2 ???

Forinstead I see this in my script:

Code:

my $backup_cmd = "$cmd_tar --create --directory $dmn_dir --gzip --file $dmn_dir/$backup_filename --exclude=logs --exclude=backups --exclude=$backup_filename .";
            
        if ($dmn_status eq 'ok') {

            if( -z $dmn_backup_dir)
            {
                $rs = del_file($dmn_backup_dir);
            
                return $rs if ($rs != 0);        
            }

            if(! -d $dmn_backup_dir)
            {
#                $rs = make_dir($dmn_backup_dir);
#                $rs = make_dir($dmn_backup_dir, $domain_uid, $httpd_gid, 0770);
                $rs = make_dir($dmn_backup_dir, $domain_uid, $domain_gid, 0770);
            
                return $rs if ($rs != 0);        

#                $rs = setfmode($dmn_backup_dir, "root", "root", 0644);

And I see this in your? script:

Code:

my $backup_cmd = "$cmd_tar --create --directory=$dmn_dir --bzip2 --file=$dmn_dir/$backup_filename --exclude=logs --exclude=phptmp --exclude=backups --exclude=$backup_filename . 2> $dmn_dir/$backup_filename.log";

        #
        # User Database Backup
        #

        if(! -d $dmn_backup_dir) {
            $rs = make_dir($dmn_backup_dir, $domain_uid, $domain_gid, 0770);

            return $rs if ($rs != 0);
        }

        #

BTW just pickt up things what look the same and or what look like belong togheter.

I know from what I read -left-and-wright- that the old vhcs2 script is somewhat buggy.
I change some things what ppl say, and must say it workt, until several months ago.

sigh

[joximu]

Hi

- AFAIK the problem still exists: when tar fails (e.g. because files changed during backup process) then the whole backup is stopped and the .lock-file is not removed.
I have to check this on an actual system and then open a ticket and find a solution.

- I don't know if you can use the ispcp-backup script for your VHCS installation. Just try it! make a copy of your files and try the backup.

/J

[gandyman]

Hmmz, owk ???

So it is stil a prob ???
What I dont understand is that in the script the file's get locked.
So they cannot change if all went wel ??

grrrr this dizzle me.... I cant stand I cant find a solution....

I wil backup the files from the www and try the script.

Edit:
I get stuck now rofl....
You dont wanna believe me but, Iget the same error wen I Tar the www directory ?????

I get the impression that it only happens with big files ???

owel then a backup to my other server without tar....
Only this wil take awhile. sigh.


note:
Remember something I read a while ago.
Maybe this wil help for the prob ???????

It had something to do with the status what the program give aboud the tar file.
(( that was why I uninstall and reinstall it))
The old program give the status 0 wen everything went well, and a 1 wen something went wrong.

The new one give status 1 went everything went well, and a 0 went something went wrong.

Becouse of this it is interpretated wrong and the backup stopt.
they had also a long talk aboud the kernel and changing things.....

Darn, I wil try to find the url from this posting for you.......

maybe this is nothing, maybe it can help ????

[joximu]

Hi

the problem is: if (mostly big) files are changes just in the moment they are being treated by tar then tar ends with status '1' (at least on my debian etch).
this leads to abort of the whole backups script.
-> the abort also happens if you run tar manually - in your case mybe you have to stop the webserver (or the process that changes the file...)

the .lock file just makes sure that there are not 2 backups script running at the same moment - and also makes sure: if an error happened then you have to fix this and then delete the .lock file manually....

/J

[joximu]

http://www.isp-control.net/ispcp/ticket/947

(it's after 1 o'clock - isp-control.net is very slow again...)

[gandyman]

K understand that.
and sounds logic.

What dazzle me is that it has work in the past, and without changing things this autobackup stop working ???
Hmz......

Thx for the ticket.
Maybe there is a workaround for this prob.....

meanwhile I get buzzy with it, becouse I wanna know why it stop working.
yeah I know I am nuts
But cant stand it wen a computer beats me.

BTW the site is slow indeed, pffffff

[joximu]

Hi

you have to find out which file changes during backup - it may be a problem in the website itself (since the logfiles are not in this tar)...

/J

[gandyman]

I wil run it a few time's so you can see what happens.

(( site problem with this script <old vhcs2 script> is that the host need to take away the old backup files.
Becouse the backup stops wen there is a old backup with a wrong date ))

######
1e attempt: with the old -outdated- backup in place, and I know for sure nothing change on this website.

Quote:server:~# /var/www/vhcs2/engine/backup/vhcs2-backup-all yes
/bin/tar: .: file changed as we read it
ERROR: External command '/bin/tar --create --directory=/var/www/virtual/website1.nl --gzip --file=/var/www/virtual/website1.nl /website1.nl -backup-2008.01.12-113153.tar.gz --exclude=logs --exclude=backups --exclude=website1.nl -backup-2008.01.12-113153.tar.gz .' returned '1' status !

After this the new backup is in the website-folder and he dit not copy it into the -backups- folder.

Now I trow away all the backups from all the website's and let the folder -backups- excist.
##########
2e attempt:
On this time the backup from website1 is compleet and he started with website2.

Quote:server:~# /var/www/vhcs2/engine/backup/vhcs2-backup-all yes
/bin/tar: .: file changed as we read it
ERROR: External command '/bin/tar --create --directory=/var/www/virtual/website2.nl --gzip --file=/var/www/virtual/website2.nl/website2.nl-backup-2008.01.12-120232.tar.gz --exclude=logs --exclude=backups --exclude=website2.nl-backup-2008.01.12-120232.tar.gz .' returned '1' status !

#############
Note:
<<You ,must know also that there is no cms in this website, it are only passive html files with can be read.
So there is nothing what can or will write in this website folder.
Also in my first post here, he succesfully made backups from website1 and website2 and got stuck ad website3>>
################

3e attempt:
He runs stuck on the same website.

#######################


Now I trow away also the folders with the name -backups-
So everything is clean, no tar-files and no Folders

4e attempt:

Quote:server:~# /var/www/vhcs2/engine/backup/vhcs2-backup-all yes
/bin/tar: ./htdocs/modules/Postcalendar/pntemplates/cache: file changed as we read it
ERROR: External command '/bin/tar --create --directory=/var/www/virtual/www.website3.com --gzip --file=/var/www/virtual/www.website3.com/www.website3.com-backup-2008.01.12-125246.tar.gz --exclude=logs --exclude=backups --exclude=www.website3.com-backup-2008.01.12-125246.tar.gz .' returned '1' status !

Interesthing to see that he come to website3 now, he only run stuck ad that Postcalendar cache.
I am 100% sure that noboby change something in Postcalendar ad the moment I made the backup.
So why the cache file change atm Tar is making a backup riddels me.

Maybe TAR trigger something in Postcalendar ???

Never the les.
He runs stuck on several things.
*Wen there is an old outdated backup.
*Wen there are no backup's, but de folder "backups" excist.
*Wen a file change......

Altough this last one would be normal.
But change nothing on the fact that it has work well in the past ????

Hope you have something on this input.

BTW the forum is fast atm....