Memory Usage Problem - Printable Version +- ispCP - Board - Support (http://www.isp-control.net/forum) +-- Forum: ispCP Omega Support Area (/forum-30.html) +--- Forum: Usage (/forum-34.html) +--- Thread: Memory Usage Problem (/thread-5653.html) |
Memory Usage Problem - Keirnan - 02-08-2009 01:05 PM I am new fairly new to LAMP. My problem is that my server will continue to consume memory, and never seems to release it. If I leave it alone for several days, it will have consumed all the physical RAM and started swapping. It does this pretty slowly - maybe 50MB an hour? I can't seem to figure out where the memory goes. I don't see any big consumers, but I notice that the amount of used memory never goes down, only up. If I restart apache (apache2ctl restart), it will only free up a small amount of memory. Likewise for MySQL. The only thing that frees up the memory counter is if I reboot the box. This is a website running on a dedicated Ubuntu 8.1 server. 3.0Ghz P4, 1GB RAM, hardware mirrored 500GB hard drives. It is running ISPCP and everything that comes with it: -Bind -MySQL -PHP -ProFTPD -Postfix -Courier As well as Fail2Ban. Everything has its default config, except for MySQL, where I changed the key_buffer_pool to 48MB and increased table cache size to 128. Everything else in the MySQL config is unchanged. I have not changed any PHP, mail, or Apache settings. The server is running 5 virtual hosts. 4 of those virtual hosts are basically unused. The remaining one gets approximately 4000 page views per day. There are never more than 25 concurrent sessions. CPU utilization is extremely low, typically 95+% idle. Here is top, sorted by memory: top - 15:53:19 up 10:50, 1 user, load average: 0.00, 0.00, 0.00 Tasks: 101 total, 1 running, 100 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.2%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1018156k total, 748576k used, 269580k free, 81292k buffers Swap: 3036276k total, 0k used, 3036276k free, 444640k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4469 mysql 20 0 161m 39m 5332 S 0 4.0 3:27.42 mysqld 26057 vu2002 20 0 29804 15m 3264 S 0 1.6 0:54.48 php5-cgi 26055 vu2002 20 0 29832 15m 3264 S 0 1.6 0:23.51 php5-cgi 26058 vu2002 20 0 27784 13m 3264 S 0 1.4 0:23.53 php5-cgi 5144 vu2001 20 0 25584 11m 3280 S 0 1.2 0:02.79 php5-cgi 26059 vu2002 20 0 25188 11m 3264 S 0 1.1 0:26.48 php5-cgi 4778 www-data 20 0 234m 10m 1644 S 0 1.1 0:04.78 apache2 5143 vu2001 20 0 24844 10m 3252 S 0 1.1 0:03.33 php5-cgi 4775 www-data 20 0 233m 10m 1640 S 0 1.0 0:04.85 apache2 4763 root 20 0 13480 9868 2040 S 0 1.0 0:03.45 ispcp-apache-lo 4762 root 20 0 13340 9800 2040 S 0 1.0 0:00.42 ispcp-apache-lo 4348 bind 20 0 44620 8836 2032 S 0 0.9 0:00.79 named 31403 www-data 20 0 232m 8420 1648 S 0 0.8 0:01.77 apache2 4533 postgrey 20 0 11136 8260 2680 S 0 0.8 0:00.18 postgrey I was thinking it was just a lot of apache php5-cgi or apache processes, but they don't seem to be using that much. MySQL is the #1 consumer right now, but sometimes it will be a php5 process. Here is ps aux at almost 11 hours of up time. In another 24 hours or so, it will be swapping: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.1 2844 1692 ? Ss 05:03 0:01 /sbin/init root 2 0.0 0.0 0 0 ? S< 05:03 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S< 05:03 0:00 [migration/0] root 4 0.0 0.0 0 0 ? S< 05:03 0:00 [ksoftirqd/0] root 5 0.0 0.0 0 0 ? S< 05:03 0:00 [watchdog/0] root 6 0.0 0.0 0 0 ? S< 05:03 0:00 [migration/1] root 7 0.0 0.0 0 0 ? S< 05:03 0:00 [ksoftirqd/1] root 8 0.0 0.0 0 0 ? S< 05:03 0:00 [watchdog/1] root 9 0.0 0.0 0 0 ? S< 05:03 0:00 [events/0] root 10 0.0 0.0 0 0 ? S< 05:03 0:00 [events/1] root 11 0.0 0.0 0 0 ? S< 05:03 0:00 [khelper] root 46 0.0 0.0 0 0 ? S< 05:03 0:00 [kblockd/0] root 47 0.0 0.0 0 0 ? S< 05:03 0:00 [kblockd/1] root 50 0.0 0.0 0 0 ? S< 05:03 0:00 [kacpid] root 51 0.0 0.0 0 0 ? S< 05:03 0:00 [kacpi_notify] root 123 0.0 0.0 0 0 ? S< 05:03 0:00 [kseriod] root 163 0.0 0.0 0 0 ? S 05:03 0:00 [pdflush] root 164 0.0 0.0 0 0 ? S 05:03 0:00 [pdflush] root 165 0.0 0.0 0 0 ? S< 05:03 0:00 [kswapd0] root 207 0.0 0.0 0 0 ? S< 05:03 0:00 [aio/0] root 208 0.0 0.0 0 0 ? S< 05:03 0:00 [aio/1] root 1275 0.0 0.0 0 0 ? S< 05:03 0:00 [ksuspend_usbd] root 1282 0.0 0.0 0 0 ? S< 05:03 0:00 [khubd] root 1434 0.0 0.0 0 0 ? S< 05:03 0:00 [scsi_eh_0] root 1468 0.0 0.0 0 0 ? S< 05:03 0:00 [ata/0] root 1469 0.0 0.0 0 0 ? S< 05:03 0:00 [ata/1] root 1471 0.0 0.0 0 0 ? S< 05:03 0:00 [ata_aux] root 2322 0.0 0.0 0 0 ? S< 05:03 0:00 [scsi_eh_1] root 2324 0.0 0.0 0 0 ? S< 05:03 0:00 [scsi_eh_2] root 2381 0.0 0.0 0 0 ? S< 05:03 0:00 [scsi_eh_3] root 2382 0.0 0.0 0 0 ? S< 05:03 0:00 [scsi_eh_4] root 2483 0.0 0.0 0 0 ? S< 05:03 0:00 [kjournald] root 2640 0.0 0.0 2224 672 ? S<s 05:03 0:00 /sbin/udevd --daemon root 2967 0.0 0.0 0 0 ? S< 05:03 0:00 [kpsmoused] root 4255 0.0 0.0 1716 512 tty4 Ss+ 05:03 0:00 /sbin/getty 38400 tty4 root 4256 0.0 0.0 1716 516 tty5 Ss+ 05:03 0:00 /sbin/getty 38400 tty5 root 4258 0.0 0.0 1716 516 tty2 Ss+ 05:03 0:00 /sbin/getty 38400 tty2 root 4260 0.0 0.0 1716 512 tty3 Ss+ 05:03 0:00 /sbin/getty 38400 tty3 root 4262 0.0 0.0 1716 512 tty6 Ss+ 05:03 0:00 /sbin/getty 38400 tty6 syslog 4304 0.0 0.0 1936 684 ? Ss 05:03 0:00 /sbin/syslogd -u syslog root 4323 0.0 0.0 1872 544 ? S 05:03 0:00 /bin/dd bs 1 if /proc/kmsg of /var/run/klogd/kmsg klog 4325 0.0 0.1 3152 2008 ? Ss 05:03 0:00 /sbin/klogd -P /var/run/klogd/kmsg bind 4348 0.0 0.8 44620 8836 ? Ssl 05:03 0:00 /usr/sbin/named -u bind root 4371 0.0 0.0 5316 1012 ? Ss 05:03 0:00 /usr/sbin/sshd root 4427 0.0 0.0 1772 520 ? S 05:03 0:00 /bin/sh /usr/bin/mysqld_safe mysql 4469 0.5 4.0 164892 40840 ? Sl 05:03 3:28 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql - root 4470 0.0 0.0 1700 556 ? S 05:03 0:00 logger -p daemon.err -t mysqld_safe -i -t mysqld polw 4529 0.0 0.5 8140 5948 ? Ss 05:03 0:00 policyd-weight (master) polw 4530 0.0 0.5 8272 5584 ? Ss 05:03 0:00 policyd-weight (cache) postgrey 4533 0.0 0.8 11136 8260 ? Ss 05:03 0:00 /usr/sbin/postgrey --pidfile=/var/run/postgrey.pid --daemo root 4557 0.0 0.0 1904 452 ? S 05:03 0:00 /usr/sbin/courierlogger -pid=/var/run/courier/authdaemon/p root 4558 0.0 0.0 2112 672 ? S 05:03 0:00 /usr/lib/courier/courier-authlib/authdaemond root 4573 0.0 0.0 1904 364 ? S 05:03 0:00 /usr/sbin/courierlogger -pid=/var/run/courier/imapd.pid -s root 4574 0.0 0.0 2008 600 ? S 05:03 0:00 /usr/sbin/couriertcpd -address=0 -maxprocs=40 -maxperip=20 root 4587 0.0 0.0 1904 452 ? S 05:03 0:00 /usr/sbin/courierlogger -pid=/var/run/courier/pop3d.pid -s root 4588 0.0 0.0 2008 612 ? S 05:03 0:00 /usr/sbin/couriertcpd -maxprocs=40 -maxperip=4 -nodnslooku root 4591 0.0 0.1 2912 1284 ? S 05:03 0:00 /usr/lib/courier/courier-authlib/authdaemond root 4592 0.0 0.0 2116 588 ? S 05:03 0:00 /usr/lib/courier/courier-authlib/authdaemond root 4593 0.0 0.0 2116 588 ? S 05:03 0:00 /usr/lib/courier/courier-authlib/authdaemond root 4594 0.0 0.0 2116 588 ? S 05:03 0:00 /usr/lib/courier/courier-authlib/authdaemond root 4595 0.0 0.0 2116 588 ? S 05:03 0:00 /usr/lib/courier/courier-authlib/authdaemond root 4660 0.0 0.1 5396 1784 ? Ss 05:03 0:00 /usr/lib/postfix/master postfix 4664 0.0 0.1 5444 1832 ? S 05:03 0:00 qmgr -l -t fifo -u nobody 4708 0.0 0.1 5992 1920 ? Ss 05:03 0:00 proftpd: (accepting connections) daemon 4722 0.0 0.0 1984 424 ? Ss 05:03 0:00 /usr/sbin/atd root 4733 0.0 0.0 2104 888 ? Ss 05:03 0:00 /usr/sbin/cron root 4759 0.0 0.3 11476 3340 ? Ss 05:03 0:01 /usr/sbin/apache2 -k start root 4760 0.0 0.0 1772 488 ? S 05:03 0:00 /bin/sh -c /var/www/ispcp/engine/ispcp-apache-logger -e root 4761 0.0 0.0 1772 484 ? S 05:03 0:00 /bin/sh -c /var/www/ispcp/engine/ispcp-apache-logger root 4762 0.0 0.9 13340 9800 ? S 05:03 0:00 /usr/bin/perl /var/www/ispcp/engine/ispcp-apache-logger -e root 4763 0.0 0.9 13480 9868 ? S 05:03 0:03 /usr/bin/perl /var/www/ispcp/engine/ispcp-apache-logger www-data 4765 0.0 0.2 10928 2148 ? S 05:03 0:00 /usr/sbin/apache2 -k start www-data 4772 0.0 0.2 11248 2240 ? S 05:03 0:00 /usr/sbin/apache2 -k start www-data 4775 0.0 1.0 239028 10272 ? Sl 05:03 0:04 /usr/sbin/apache2 -k start www-data 4778 0.0 1.0 240052 11036 ? Sl 05:03 0:04 /usr/sbin/apache2 -k start root 4837 0.0 0.4 40228 4616 ? Sl 05:03 0:12 /usr/bin/python /usr/bin/fail2ban-server -b -s /var/run/fa root 4860 0.0 0.0 1700 448 ? S 05:03 0:00 /var/www/ispcp/daemon/ispcp_daemon -p /var/run/ispcp_daemo root 4933 0.0 0.0 1716 512 tty1 Ss+ 05:03 0:00 /sbin/getty 38400 tty1 polw 4978 0.0 0.6 8400 6264 ? S 05:09 0:00 policyd-weight (child) vu2001 5142 0.0 0.5 19240 5484 ? Ss 05:39 0:00 /usr/bin/php5-cgi vu2001 5143 0.0 1.0 24844 11028 ? S 05:39 0:03 /usr/bin/php5-cgi vu2001 5144 0.0 1.1 25584 11820 ? S 05:39 0:02 /usr/bin/php5-cgi postfix 25912 0.0 0.1 5404 1672 ? S 14:36 0:00 pickup -l -t fifo -u -c vu2002 26054 0.0 0.5 19240 5488 ? Ss 15:00 0:00 /usr/bin/php5-cgi vu2002 26055 0.7 1.5 29832 15964 ? S 15:00 0:24 /usr/bin/php5-cgi vu2002 26056 0.0 0.5 19240 5488 ? Ss 15:00 0:00 /usr/bin/php5-cgi vu2002 26057 1.6 1.5 29804 15992 ? S 15:00 0:55 /usr/bin/php5-cgi vu2002 26058 0.7 1.3 27784 13872 ? S 15:00 0:24 /usr/bin/php5-cgi vu2002 26059 0.8 1.1 25444 11664 ? S 15:00 0:27 /usr/bin/php5-cgi root 26241 0.0 0.2 8056 2520 ? Ss 15:42 0:00 sshd: pragupia6060 [priv] 1000 26243 0.0 0.1 8056 1552 ? S 15:42 0:00 sshd: pragupia6060@pts/0 1000 26244 0.0 0.2 5560 2980 pts/0 Ss 15:42 0:00 -bash postfix 26266 0.0 0.3 6612 3188 ? S 15:46 0:00 smtpd -n smtp -t inet -u -c -o stress postfix 26267 0.0 0.1 5408 1732 ? S 15:46 0:00 anvil -l -t unix -u -c 1000 26294 0.0 0.0 2644 1008 pts/0 R+ 15:57 0:00 ps aux polw 31044 0.0 0.6 8404 6300 ? S 09:38 0:00 policyd-weight (child) www-data 31403 0.0 0.8 237696 8420 ? Sl 10:51 0:01 /usr/sbin/apache2 -k start Now, here is top again, about 2.5 hours later. Notice that I've used another 100MB of memory. Code: top - 18:19:28 up 13:16, 1 user, load average: 0.00, 0.02, 0.00 And here is vmstat -s run at the same time as this top. I wish I had it for when I grabbed the stats 2 hours ago, but I don't. Code: 1018156 K total memory |