The VM for Linux prefers system cache over application memory. What does this mean? The best way I can explain is by example.
Imagine you have 32 GB of RAM
MySQL is set to take 20 GB of RAM for a process based buffer and up to 6M for the various thread buffers.
Over a period of time the box swaps. The only thing that is running is mysql and its memory size is around 21GB for resident memory. Why does swap grow when there is plenty of memory? The reason is when a memory alloc is needed (thread based buffer is tickled) the VM will choose to use swap over allocating from the system cache, when there is not enough free memory.
DO NOT TURN OFF SWAP to prevent this. Your box will crawl, kswapd will chew up a lot of the processor, Linux needs swap enabled, lets just hope its not used.
So how do you stop Nagios pages because of swap usage? Well if you have a few choices.
reboot the box
or
stop mysql && swapoff -a;swapon -a;
or just
swapoff -a;swapon -a;
Doing the latter command is rather scary and fun at the same time. Because you can either crash mysql or not. I just did the swap* commands live, I was very certain nothing was using swap and it worked. YAY no more pages and I didn't have to shut down the service!
7 comments:
Hey Dathan,
I hate this stupid problem, but I have a slightly different approach: http://blogs.smugmug.com/don/2008/05/01/mysql-and-the-linux-swap-problem/
Hope that helps!
Which are the best among the three choices you gave? It seems the third is the best simple in operation and is influence at least.
reboot as a solution is evil
Hi Dathan,
do you use swappiness? I usually set it to 5 on my boxes, and then allocate about 80% of memory to the innodb buffer pool. Works fine, I eventually get some swapped in pages but they are never swapped out, so it doesn't really matter.
Don's method is pretty good. Messing with swppiness doesn't really help. Its the fact that the VM cannot nor will not pull from system cache over swap.
Doing the last step in the post is prob the best and fastest method to get rid of swap - when you don't want to make ram-disks for swap.
Actually turning off swap is the best thing to do. I've never heard of kswap problems ... ??
how would you define swap with respect to linux and what is the typical size of the swap space?
Post a Comment