LxCenter HyperVM & Kloxo Support

Forum



Members   Search      Help    Register    Login    Home
Home » Development Forums » Kloxo Development » apache-optimize should be removed until better thought out.
apache-optimize should be removed until better thought out. [message #93919] Tue, 10 January 2012 11:09 Go to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
I thought I had explained in good detail how the directives for httpd work. Here is a snippet from the apache-optimize script:

<IfModule prefork.c>
StartServers 2
MinSpareServers {$minpar_p}
MaxSpareServers {$maxpar_p}
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
MaxMemFree 2
</IfModule>

http://project.lxcenter.org/projects/kloxo/repository/revisi ons/master/entry/kloxo/bin/fix/apache-optimize.php

ServerLimit is always set at 256 and MaxClients is set to 256.

If $minpar_p is set to 10 and $maxpar_p is set to 25 then here is the result:

When httpd is started you will have 10 - 11 httpd processes to start with minimally. As the day goes on you will have 25 to 26 running. As the day goes further it can peak at 256 to 257 processes running. How does this optimize the memory? The min and max spare servers do not dictate what the peak httpd processes will be and therefore will not dictate peak memory usage.


https://www.mercuryvps.com
HyperVM Xen VPS hosting

[Updated on: Tue, 10 January 2012 11:11]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #93920 is a reply to message #93919] Tue, 10 January 2012 11:18 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
I mainly want to get a consensus on who agrees with what I am saying. Once we agree, I will put in a bug report.

Also another point is this script doesn't take into account what the httpd processes even use for memory. This can vary greatly and in a shared environment, can be much higher than the 30MB that is assumed in this script. *As far as I can tell it looks like it is assuming 30 MB anyway*


https://www.mercuryvps.com
HyperVM Xen VPS hosting
Re: apache-optimize should be removed until better thought out. [message #93922 is a reply to message #93920] Tue, 10 January 2012 11:40 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
Moved to Kloxo Development for more feedback.

https://www.mercuryvps.com
HyperVM Xen VPS hosting
Re: apache-optimize should be removed until better thought out. [message #93944 is a reply to message #93922] Wed, 11 January 2012 01:16 Go to previous messageGo to next message
mustafaramadhan is currently offline mustafaramadhan  Indonesia
Messages: 5669
Registered: December 2010
Location: Yogyakarta
Super Grandmaster
Forum Moderator

In apache 2.2.20/2.2.21, when ServerLimit=MaxClients=MaxSpareServers (say it 25) we have a warning when running 'service httpd restart' and apache set to 256 for MaxSpareServers.

No warning when using httpd 2.2.17/2.2.19.


http://download.lxcenter.org/kdev.png
..:: MRatWork ::..
Server/Web-integrator - perfect not always more useful

--- Need KVM/OpenVZ VPS? - click here (Kloxo-MR READY!) ---

For bug/feature/security - Member rank status

http://download.lxcenter.org/hdev.png

[Updated on: Wed, 11 January 2012 01:17]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #93972 is a reply to message #93944] Wed, 11 January 2012 09:38 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
That is another issue and has nothing to do with the shortcomings of this apache-optimize script that I am talking about here.

https://www.mercuryvps.com
HyperVM Xen VPS hosting

[Updated on: Wed, 11 January 2012 09:44]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #93979 is a reply to message #93972] Wed, 11 January 2012 11:32 Go to previous messageGo to next message
mustafaramadhan is currently offline mustafaramadhan  Indonesia
Messages: 5669
Registered: December 2010
Location: Yogyakarta
Super Grandmaster
Forum Moderator

I hope you know what you talking and found solution by yourself.

For system with memory less then 1GB that will be use memory bigger than system with running optimize. That it.


http://download.lxcenter.org/kdev.png
..:: MRatWork ::..
Server/Web-integrator - perfect not always more useful

--- Need KVM/OpenVZ VPS? - click here (Kloxo-MR READY!) ---

For bug/feature/security - Member rank status

http://download.lxcenter.org/hdev.png

[Updated on: Wed, 11 January 2012 11:36]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #93981 is a reply to message #93979] Wed, 11 January 2012 11:39 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
mustafa - I don't think you understand the directives here. Please explain what you think each of these directives mean. You are not understanding what the spare servers mean and this apache-optimize script keeps a server at 256 processes. All the script does is change the spare which is quite useless.

StartServers -
MinSpareServers -
MaxSpareServers -
ServerLimit -
MaxClients -
MaxRequestsPerChild -


https://www.mercuryvps.com
HyperVM Xen VPS hosting

[Updated on: Wed, 11 January 2012 11:40]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #93986 is a reply to message #93981] Wed, 11 January 2012 12:48 Go to previous messageGo to next message
mustafaramadhan is currently offline mustafaramadhan  Indonesia
Messages: 5669
Registered: December 2010
Location: Yogyakarta
Super Grandmaster
Forum Moderator

How about http://forum.lxcenter.org/index.php?t=msg&goto=93983& ;#msg_93983 (lupetalo) issue:
Quote:
When you have machine with 16gb RAM and 16 Gb SWAP and in 20hrs All memory is full and you have 900 httpd processes that are doing nothing, I call it BUG. Same thing on two other servers...


http://download.lxcenter.org/kdev.png
..:: MRatWork ::..
Server/Web-integrator - perfect not always more useful

--- Need KVM/OpenVZ VPS? - click here (Kloxo-MR READY!) ---

For bug/feature/security - Member rank status

http://download.lxcenter.org/hdev.png
Re: apache-optimize should be removed until better thought out. [message #93987 is a reply to message #93986] Wed, 11 January 2012 13:09 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
When he replies I will tell him how to properly configure his server so it doesn't run into out of memory issues. The explanation of each directive is very clear on the Apache website and here is how I would explain each:

StartServers - This is the number of initial forked processes.
MinSpareServers - This is the minimum amount of spare IDLE forked processes that it keeps running.
MaxSpareServers - This is the maximum amount of spare IDLE forked processes.
ServerLimit - This essentially mimics MaxClients
MaxClients - This is the maximum amount of forked process that will run simultaneously. This is the key directive to limiting the amount of memory that Apache uses as a whole.
MaxRequestsPerChild - This is the number of requests a forked process will handle before the process ends.

http://httpd.apache.org/docs/2.2/mod/prefork.html

Let me know if you fully understand this and then I can give you an example.


https://www.mercuryvps.com
HyperVM Xen VPS hosting
Re: apache-optimize should be removed until better thought out. [message #93988 is a reply to message #93987] Wed, 11 January 2012 13:13 Go to previous messageGo to next message
mustafaramadhan is currently offline mustafaramadhan  Indonesia
Messages: 5669
Registered: December 2010
Location: Yogyakarta
Super Grandmaster
Forum Moderator

try with:


<IfModule prefork.c>
StartServers 2
MinSpareServers 10
MaxSpareServers 20
ServerLimit 20
MaxClients 20
MaxRequestsPerChild 4000
MaxMemFree 2
</IfModule>

and restart apache from ssh.


http://download.lxcenter.org/kdev.png
..:: MRatWork ::..
Server/Web-integrator - perfect not always more useful

--- Need KVM/OpenVZ VPS? - click here (Kloxo-MR READY!) ---

For bug/feature/security - Member rank status

http://download.lxcenter.org/hdev.png
Re: apache-optimize should be removed until better thought out. [message #93990 is a reply to message #93988] Wed, 11 January 2012 13:28 Go to previous messageGo to next message
mustafaramadhan is currently offline mustafaramadhan  Indonesia
Messages: 5669
Registered: December 2010
Location: Yogyakarta
Super Grandmaster
Forum Moderator

This is default setting by httpd (inside httpd.conf):
<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>


and this is after optimize (inside ~lxcenter.conf; will be overrite value inside httpd.conf):
<IfModule prefork.c>
	StartServers 2
	MinSpareServers 10
	MaxSpareServers 21
	ServerLimit 256
	MaxClients 256
	MaxRequestsPerChild 4000
	MaxMemFree 2
</IfModule>


http://download.lxcenter.org/kdev.png
..:: MRatWork ::..
Server/Web-integrator - perfect not always more useful

--- Need KVM/OpenVZ VPS? - click here (Kloxo-MR READY!) ---

For bug/feature/security - Member rank status

http://download.lxcenter.org/hdev.png

[Updated on: Wed, 11 January 2012 13:29]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #93991 is a reply to message #93990] Wed, 11 January 2012 13:31 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
And notice they both will max out with 256 processes. Do you see this? How does that change the amount of memory that Apache uses as a maximum? It doesn't. Both configs there will allow 256 forked processes to run, do you not see that?

https://www.mercuryvps.com
HyperVM Xen VPS hosting
Re: apache-optimize should be removed until better thought out. [message #94011 is a reply to message #93919] Wed, 11 January 2012 16:29 Go to previous messageGo to next message
com2 is currently offline com2  Spain
Messages: 1119
Registered: August 2007
Location: Spain
Senior Master
Developer
I contribute to this discussion a spreadsheet I once made that should help me to find sensible figures for this dilemma. May be it helps to find a better formula.

I agree with the critique of this thread that to limit memory use you should be finding right values for Serverlimit and Maxclients. In the Apache documentation I quote:
Apache MPM prefork
Sites which need to serve more than 256 simultaneous requests may need to increase MaxClients, while sites with limited memory may need to decrease MaxClients to keep the server from thrashing (swapping memory to disk and back).


In my understanding there is little difference between Serverlimit and Maxclients. The only difference as far as I can tell is that you can vary the limit of Maxclients and graceful restart will read it. To make changes in Serverlimit will need a full restart. Serverlimit needs to be higher or equal to Maxclients.

I quote from Apache documentation again:
Apache MPM Common Directives - ServerLimit Directive
Any attempts to change this directive during a restart will be ignored, but MaxClients can be modified during a restart.

Special care must be taken when using this directive. If ServerLimit is set to a value much higher than necessary, extra, unused shared memory will be allocated. If both ServerLimit and MaxClients are set to values higher than the system can handle, Apache may not start or the system may become unstable.

With the prefork MPM, use this directive only if you need to set MaxClients higher than 256 (default). Do not set the value of this directive any higher than what you might want to set MaxClients to.

[Updated on: Wed, 11 January 2012 16:36]

Report message to a moderator

Re: apache-optimize should be removed until better thought out. [message #94050 is a reply to message #94011] Thu, 12 January 2012 14:32 Go to previous messageGo to next message
shazar is currently offline shazar  United States
Messages: 1856
Registered: May 2011
Grandmaster
LxCenter Core Team Member
LxCenter Representative
So is there an agreement that this is a bug? If there is I can put a bug issue and I can revamp it and you test it.

https://www.mercuryvps.com
HyperVM Xen VPS hosting
Re: apache-optimize should be removed until better thought out. [message #105266 is a reply to message #94050] Sat, 19 January 2013 11:48 Go to previous messageGo to next message
superjohn5566 is currently offline superjohn5566  Pakistan
Messages: 8
Registered: January 2013
Location: New York
Member
When he replies I will tell him how to properly configure his server so it doesn't run into out of memory issues. The explanation of each directive is very clear on the Apache website and here is how I would explain each:

allprizebond dot com is unique website accessible on internet and wireless communication which will help you to get any information you need regarding to prize bond scheme offer by Govt.
Prize Bond Schedule || Prize Bond Schedule || Free Movies Online Without Downloading
Re: apache-optimize should be removed until better thought out. [message #105924 is a reply to message #105266] Wed, 20 March 2013 03:28 Go to previous messageGo to next message
huaruyue is currently offline huaruyue
Messages: 3
Registered: March 2013
Location: florida
Member
As the day goes further it can peak at 256 to 257 processes running. How does this optimize the memory? 。。。





_____________________________________
love me little, love me long..
Diablo 3 Gold
Runescape Gold
Rs Gold
Re: apache-optimize should be removed until better thought out. [message #106032 is a reply to message #105924] Fri, 29 March 2013 14:37 Go to previous message
vRozenSch00n is currently offline vRozenSch00n  Australia
Messages: 15
Registered: January 2011
Location: Netherlands Indies
Member
This is the auto Optimized settings for my Centos 5.9 php53 on a 1GB RAM L5420 4x2.50GHz VPS, Apache mod_ruid2, mod_pagespeed, php-xcache.
Timeout 150
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

<IfModule prefork.c>
        StartServers 2
        MinSpareServers 13
        MaxSpareServers 26
        ServerLimit 256
        MaxClients 256
        MaxRequestsPerChild 4000
        MaxMemFree 2
</IfModule>

<IfModule itk.c>
        StartServers 2
        MinSpareServers 13
        MaxSpareServers 26
        ServerLimit 256
        MaxClients 256
        MaxRequestsPerChild 4000
        MaxMemFree 2
</IfModule>

<IfModule worker.c>
        StartServers 2
        MaxClients 150
        MinSpareThreads 11
        MaxSpareThreads 22
        ThreadsPerChild 25
        MaxRequestsPerChild 0
        ThreadStackSize 8196
        MaxMemFree 2
</IfModule>

<IfModule event.c>
        StartServers 2
        MaxClients 150
        MinSpareThreads 11
        MaxSpareThreads 22
        ThreadsPerChild 25
        MaxRequestsPerChild 0
        ThreadStackSize 8196
        MaxMemFree 2
</IfModule>

Include /home/apache/conf/exclusive/*.conf
Include /home/apache/conf/defaults/*.conf
Include /home/apache/conf/domains/*.conf
Include /home/apache/conf/redirects/*.conf
Include /home/apache/conf/webmails/*.conf
Include /home/apache/conf/wildcards/*.conf

###version0-7###


When Mr. GoogleBot, BingBot, Baidu, MsnBot and other seoBots reach my websites in that server at the same time, in just 15 minutes I ran out of memory and the server died.

Then I chaged the settings to
Timeout 30
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 3

<IfModule prefork.c>
        StartServers 2
        MinSpareServers 2
        MaxSpareServers 7
        ServerLimit 64
        MaxClients 64
        MaxRequestsPerChild 1500
        MaxMemFree 2
</IfModule>

<IfModule itk.c>
        StartServers 2
        MinSpareServers 2
        MaxSpareServers 7
        ServerLimit 64
        MaxClients 64
        MaxRequestsPerChild 1500
        MaxMemFree 2
</IfModule>

<IfModule worker.c>
        StartServers 2
        MaxClients 64
        MinSpareThreads 5
        MaxSpareThreads 11
        ThreadsPerChild 25
        MaxRequestsPerChild 0
        ThreadStackSize 8196
        MaxMemFree 2
</IfModule>

<IfModule event.c>
        StartServers 2
        MaxClients 64
        MinSpareThreads 5
        MaxSpareThreads 11
        ThreadsPerChild 25
        MaxRequestsPerChild 0
        ThreadStackSize 8196
        MaxMemFree 2
</IfModule>

Include /home/apache/conf/exclusive/*.conf
Include /home/apache/conf/defaults/*.conf
Include /home/apache/conf/domains/*.conf
Include /home/apache/conf/redirects/*.conf
Include /home/apache/conf/webmails/*.conf
Include /home/apache/conf/wildcards/*.conf

###version0-7###


Then it can survive the incoming rush. In my opinion, the mod_pagespeed takes quite a lot of resource, and I guess the auto optimized feature is only the base calculation, as in practice it depends on many things, such as your server configuration, xcache configuration, mod_pagespeed configuration and more.

Previous Topic:[INFO] different between Apache 2.2.22 and 2.4.1
Next Topic:PHP 5.3 for Kloxo
Goto Forum:
  


Current Time: Sun May 19 13:38:50 EDT 2013

Total time taken to generate the page: 0.01386 seconds
.:: Contact :: Home :: Privacy ::.

Click here to lend your support to: LxCenter and make a donation at www.pledgie.com !

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software