high memory usage after 0:00u

[prale]

Hello I noticed that my server uses about 480mb ram after 0:00u.
I think ispcp is making backups and statistics at that time.
But the memory usage stays high, even after a few days.

I have configured apache with prefork just like the ispcp tutorial "running on low resources" discribes.

It doesn't seems to work.
What can I do to solve this problem?

[BeNe]

Take a look at your Cron (ispcp & root) and see which Applications runs at this time. Use top or htop to check which Process use so much memory.

Greez BeNe

[prale]

I attached a screenshot with the output from #top.
(I masked my hostname and username in the image)

I think it's php5-cgi, I already updated the latest releases from dotdeb, but it doesn't seems to help.

This is my setup:

ispcp rc4
monit
munin+vhosts module
eaccelerator

apache is compiled with prefork (default config)
also mysql has the default config

I know about the running on low resources tutorial.
Changing the fcgi config results in http connections from 5-10 seconds, so I don't want to use it.
Also the mysql and apache config mods are not really making any difference.

My cron is just the same as the default of ispcp.
I haven't modified it.

I hope someone can help me.

[joximu]

[sorry - I translate it...]
this look very strange
some vhosts are running with fastcgi (that's why there are php-cgi processes) and others are runing with apache mod-php.
If you really use apache prefork, then you better switch all vhosts to mod-php.

There are no php processes on midnight by ispcp - so either you have some additional crons (external?) or it's not the php... (maybe the mysql dumps?)

Have you checked all cron jobs starting at midnight...

/J

[prale]

English please or dutch, sorry.

CPU model Intel® Celeron® CPU 2.40GHz
CPU MHz 2399.85
CPU cache 128 KB
CPU bogomips 4803.73

I have installed prefork because I only have one CPU.
When you google, they say worker is only better if you have multiple cpu's.

Code:

/etc/cron.d/

awstats:
0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null

ispcp:
# delayed tasks START.

# Quota
@daily root /var/www/ispcp/engine/quota/ispcp-dsk-quota &>/var/log/ispcp/ispcp-dsk-quota.log

# Traffic
0,30    *       *       *       *  root /var/www/ispcp/engine/traffic/ispcp-srv-traff &>/var/log/ispcp/ispcp-srv-traff.log
0,30    *       *       *       *  root /var/www/ispcp/engine/traffic/ispcp-vrl-traff &>/var/log/ispcp/ispcp-vrl-traff.log
15,45   *       *       *       *  root /var/www/ispcp/engine/traffic/ispcp-vrl-traff-correction &>/var/log/ispcp/ispcp-vrl-traff-correction.log

# customer logs
@daily root /var/www/ispcp/engine/tools/ispcp-httpd-logs-mngr &>/var/log/ispcp/ispcp-httpd-logs-mngr.log

# Backup
@daily root /var/www/ispcp/engine/backup/ispcp-backup-all yes &>/var/log/ispcp/ispcp-backup-all-mngr.log
@daily root /var/www/ispcp/engine/backup/ispcp-backup-ispcp noreport &>/var/log/ispcp/ispcp-backup-ispcp-mngr.log

# Remove config backups older than seven days
@daily root find /etc/ispcp/*/backup/* -maxdepth 0 -type f -mtime +7 -print | egrep '.*\.[0-9]+$' | xargs -r /bin/rm

# Remove Daemon Logs older than 14 days (except .gz files)
@daily root find /var/log/ispcp/* -maxdepth 1 -type f -mtime +14 -print | egrep '.*\.gz$' | xargs -r /bin/rm

# AWStats
15      */6     *       *       *       root /var/www/ispcp/engine/awstats/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl >/dev/null 2>&1

# Rootkit Hunter
0       */12    *       *       *       root /usr/bin/rkhunter --scan-knownbad-files --check-deleted --createlogfile --cronjob --createlogfile /var/log/rkhunter.log >/dev/null 2>&1

# Chkrootkit
0       */12    *       *       *       root /usr/sbin/chkrootkit &> /var/log/chkrootkit.log

# Look for and purge old sessions every 30 minutes
0,30 *     * * *     root   /var/www/ispcp/engine/tools/ispcpphptemp.sh >/dev/null 2>&1

# [{DMN_NAME}] backup task START.
# [{DMN_NAME}] backup task END.

# [{DMN_NAME}]:{CRONJOB_ID} custom task START.
# [{DMN_NAME}]:{CRONJOB_ID} custom task END.

# [{DMN_NAME}] AWStats static tasks START.
# [{DMN_NAME}] AWStats static tasks END.

# delayed tasks END.

mdadm:
#
# cron.d/mdadm -- schedules periodic redundancy checks of MD devices
#
# Copyright © martin f. krafft <madduck@madduck.net>
# distributed under the terms of the Artistic Licence 2.0
#
# $Id: mdadm.cron.d 147 2006-08-30 09:26:11Z madduck $
#

# By default, run at 01:06 on every Sunday, but do nothing unless the day of
# the month is less than or equal to 7. Thus, only run on the first Sunday of
# each month. crontab(5) sucks, unfortunately, in this regard; therefore this
# hack (see #380425).
6 1 * * 0 root [ -x /usr/share/mdadm/checkarray ] && [ $(date +\%d) -le 7 ] && /usr/share/mdadm/checkarray --cron --all --quiet

munin:
#
# cron-jobs for munin
#

MAILTO=root

@reboot         root  if [ ! -d /var/run/munin ]; then /bin/bash -c 'perms=(`/usr/sbin/dpkg-statoverride --list /var/run/munin`); mkdir /var/run/munin; chown ${perms[0]:-munin}:${perms[1]:-root} /var/run/munin; chmod ${perms[2]:-0755} /var/run/munin'; fi
*/5 * * * *     munin if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi
14 10 * * *     munin if [ -x /usr/share/munin/munin-limits ]; then /usr/share/munin/munin-limits --force --contact nagios --contact old-nagios; fi

munin-node:
#
# cron-jobs for munin-node
#

MAILTO=root

# If the APT plugin is enabled, update packages databases approx. once
# an hour (12 invokations an hour, 1 in 12 chance that the update will
# happen), but ensure that there will never be more than two hour (7200
# seconds) interval between updates..
*/5 * * * *     root if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi

php4:
# /etc/cron.d/php4: crontab fragment for php4
#  This purges session files older than X, where X is defined in seconds
#  as the largest value of session.gc_maxlifetime from all your php.ini
#  files, or 24 minutes if not defined.  See /usr/lib/php4/maxlifetime

# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -d /var/lib/php4 ] && find /var/lib/php4/ -type f -cmin +$(/usr/lib/php4/maxlifetime) -print0 | xargs -r -0 rm
~


php5:
# /etc/cron.d/php5: crontab fragment for php5
#  This purges session files older than X, where X is defined in seconds
#  as the largest value of session.gc_maxlifetime from all your php.ini
#  files, or 24 minutes if not defined.  See /usr/lib/php5/maxlifetime

# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -r -0 rm

[prale]

Hmmz I googled again on "worker vs prefork"
The combination fastcgi/worker seems to result in more performance I've read.

I already changed to worker again:
#aptitude install apache2-mpm-worker

# apache2 -l
Compiled in modules:
core.c
mod_log_config.c
mod_logio.c
worker.c
http_core.c
mod_so.c

# apache2 -t -D DUMP_MODULES
Loaded Modules:
core_module (static)
log_config_module (static)
logio_module (static)
mpm_worker_module (static)
http_module (static)
so_module (static)
actions_module (shared)
alias_module (shared)
auth_basic_module (shared)
authn_file_module (shared)
authz_default_module (shared)
authz_groupfile_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
cband_module (shared)
cgid_module (shared)
dir_module (shared)
env_module (shared)
fastcgi_module (shared)
info_module (shared)
mime_module (shared)
negotiation_module (shared)
proxy_module (shared)
proxy_http_module (shared)
rewrite_module (shared)
setenvif_module (shared)
status_module (shared)
suexec_module (shared)
evasive20_module (shared)
watch_module (shared)

[joximu]

cron:
I'd put the backups to some later ("0 1 * * *" instead of "@daily" - or something similar)

worker/prefork:
worker is better in performance (with threaded processes or something like this :-) - but mod-php cannot deal with that, so with worker you need fastcgi, prefork can deal with both fastcgi and mod-php - but since fastcgi is more secure but less performant you have to decide...

/J

[prale]

well I see that I have fastcgi_module (shared) running so I guess I'm fine with worker.
php5 works fine as far as I can see.

I will try the cron tomorrow, I go to sleep now work goes on tomorrow morning :S

Many thanx for your help so far!

[joximu]

Just think about turning *all* vhosts to fastcgi...
(regenerate config files...)

I think you'll continue *today*

/J

[prale]

Well I just regenerated the config files.
Is there something else I must do? or is it all automatic?

I'll try the cron change tonight, so at 0:00u we will see...