| [PRE] Nginx/Lighttpd proxy and template-based web config [message #95160] |
Sat, 18 February 2012 01:37  |
|
Want try?. Go to http://forum.lxcenter.org/index.php?t=msg&th=18106
According to 'Enhancement #865 - Webserver config template', 'Feature #862 - Nginx-proxy for Kloxo' and 'Feature #898 - Lighttpd-proxy for Kloxo', this features near release.
Need participant for testing.
How to install:
# update system
yum update
# update kloxo - repeat until version 6.1.10 when running 'sh /script/version'
sh /script/upcp
# download the code
mkdir /tmp/kloxo-test
cd /tmp/kloxo-test
rm -rf index*
### wget 'http://forum.lxcenter.org/index.php?t=getfile&id=1587'
### wget 'http://forum.lxcenter.org/index.php?t=getfile&id=1588'
### wget 'http://forum.lxcenter.org/index.php?t=getfile&id=1591'
### wget 'http://forum.lxcenter.org/index.php?t=getfile&id=1592'
### wget 'http://forum.lxcenter.org/index.php?t=getfile&id=1593'
wget 'http://forum.lxcenter.org/index.php?t=getfile&id=1596'
mv index* kloxo.zip
unzip kloxo.zip
# backup original kloxo
yes | cp -rf /usr/local/lxlabs/kloxo /usr/local/lxlabs/kloxo.bck
# overrite the code
yes | cp -rf ./kloxo /usr/local/lxlabs
# fix the code
sh /script/cleanup
# create custom repo (based-on centalt) because nginx and rpaf taken from here
echo '[centalt]' > /etc/yum.repos.d/kloxo-centalt.repo
echo 'name=centalt - $basearch' >> /etc/yum.repos.d/kloxo-centalt.repo
echo 'baseurl=http://centos.alt.ru/repository/centos/5/$basearch/' >> /etc/yum.repos.d/kloxo-centalt.repo
echo 'enabled=1' >> /etc/yum.repos.d/kloxo-centalt.repo
echo 'gpgcheck=0' >> /etc/yum.repos.d/kloxo-centalt.repo
echo 'includepkgs=nginx mod_rpaf' >> /etc/yum.repos.d/kloxo-centalt.repo
sh /script/fix-chownchmod
reboot
Unfinish job:
1. Swith program not automaticly fix config. Need run 'sh /script/fixweb' when switch (say it) apache to nginx-proxy- fixed
2. select fastcgi for lighttpd/nginx and suphp for apache not work. Need more times for fixed this issues - fixed
3. Add domain will be appear options for fastcgi. This is wrong. Need more times for fixed this issues
4. fixed wildcards subdomains on lighttpd
Advices:
1. don't use on production server
2. select 'mod_php_ruid2' for apache when testing nginx/lighttpd-proxy
3. select 'optimize' on 'webserver config' when select apache and nginx/lighttpd-proxy. Different setting for 'pure' apache and proxy.
----------------
Update 1:
1. Add php-fpm for Nginx and Lighttpd - NEW!
2. change from cache-based to local-based for static files
3. change how to djbdns read ip - NEW!
Update 2:
1. switch program automaticly running fix config -- fix old issue
2. php-fpm on lighttpd and nginx based on pool (more secure) - NEW!
3. add php-fpm on apache but still with 1 pool only (fastcgi finalty for setting with more than 1 pool) -- waiting for Apache 2.4.x and than change mod_fastcgi to mod_proxy_fcgi
4. including trick for install php 'old' version (PHP 5.2.17; not including php-fpm patch) when user(s) select suphp - NEW!
Update 3:
1. fixed wrong code inside serverweb__lib.php
2. Testing with upcoming lighttpd 1.4.30-1 and look like running well.
Update 4:
1. fix permalink on lighttpd (old issue) also use new template
Update 5:
1. fixed log_cleanup so call fix scripts from inside kloxo automaticly use '--nolog' -- mininize grow up cleanup log file
2. Only use 2 dirs (defaults and domains) instead 5 dirs (defaults, domains, redirects, webmails and wildcards) - NEW!
3. fixed awstarts and webalizer config for every domains
4. prepare to httpd 2.4.x
5. fixed php 5.2.17-13 issue for suphp - for fastcgi on lighttpd automaticly fixed when using php-fpm
---------------------------
Next step - http://forum.lxcenter.org/index.php?t=msg&goto=95665
---------------------------
[Updated on: Tue, 06 March 2012 23:33] Report message to a moderator |
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95171 is a reply to message #95166] |
Sat, 18 February 2012 08:11  |
|
Because nginx-proxy have cache system and enable by default, need disable it for testing step:
1. Change 'proxy_cache cache;' to '# proxy_cache cache;' inside '/home/nginx/conf/globals/proxy.conf', and or
2. delete content of '/home/nginx/conf/cache and '/home/nginx/conf/tmp'
Also change for '/usr/local/lxlabs/kloxo/file/nginx/globals/proxy.conf'.
If you want customize you can make custom.proxy.conf and place inside '/usr/local/lxlabs/kloxo/file/nginx/globals/'.
[Updated on: Sat, 18 February 2012 08:42] Report message to a moderator |
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95190 is a reply to message #95186] |
Sat, 18 February 2012 17:43 |
|
this my benchmark
memory 1GB swap 2GB vps xen
benchmark test with unixbench
========================================================================
[root@ks365910 ~]# ab -n 10 -c 2 http://www.somesite.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.somesite.com (be patient).....done
Server Software: nginx
Server Hostname: www.somesite.com
Server Port: 80
Document Path: /
Document Length: 123872 bytes
Concurrency Level: 2
Time taken for tests: 6.862 seconds
Complete requests: 10
Failed requests: 0
Write errors: 0
Total transferred: 1244958 bytes
HTML transferred: 1238720 bytes
Requests per second: 1.46 [#/sec] (mean)
Time per request: 1372.338 [ms] (mean)
Time per request: 686.169 [ms] (mean, across all concurrent requests)
Transfer rate: 177.18 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 136 138 0.8 138 139
Processing: 1138 1211 92.9 1161 1387
Waiting: 445 514 94.3 464 694
Total: 1276 1348 92.7 1300 1524
Percentage of the requests served within a certain time (ms)
50% 1300
66% 1311
75% 1444
80% 1469
90% 1524
95% 1524
98% 1524
99% 1524
100% 1524 (longest request)
[root@ks365910 ~]#
[root@ks365910 ~]# ab -n 1000 -c 5 http://www.somesite.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.somesite.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: nginx
Server Hostname: www.somesite.com
Server Port: 80
Document Path: /
Document Length: 123872 bytes
Concurrency Level: 5
Time taken for tests: 265.978 seconds
Complete requests: 1000
Failed requests: 246
(Connect: 0, Receive: 0, Length: 246, Exceptions: 0)
Write errors: 0
Total transferred: 124566092 bytes
HTML transferred: 123957484 bytes
Requests per second: 3.76 [#/sec] (mean)
Time per request: 1329.890 [ms] (mean)
Time per request: 265.978 [ms] (mean, across all concurrent requests)
Transfer rate: 457.36 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 136 138 2.3 138 169
Processing: 1003 1188 249.2 1160 6221
Waiting: 441 493 246.7 463 5524
Total: 1140 1326 249.3 1298 6360
Percentage of the requests served within a certain time (ms)
50% 1298
66% 1308
75% 1316
80% 1323
90% 1348
95% 1413
98% 1558
99% 1806
100% 6360 (longest request)
[root@ks365910 ~]#
server for vBulletin 4.1.10 + vBSEO
with 5000 - 8000 Unique User Online
ready
[Updated on: Sun, 19 February 2012 02:50] Report message to a moderator |
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95197 is a reply to message #95190] |
Sat, 18 February 2012 22:09 |
|
prandah - Is this benchmark run locally? The reason I ask is because a latency of 0ms and a real world latency of 40 - 200 ms for a real world user is a huge difference. You will find the limiting factor to be the processor locally where you won't use near the processor in a real world test. Also I am not certain with this test, but it may just load the first page you tell it and not all the children of that page.
As an example I ran a test against your blog and was quickly getting full page loads surpassing 5 and 7 seconds approaching just 50 concurrent connections. Just curious what your test against your blog gives to get a better idea here.
https://www.mercuryvps.com
HyperVM Xen VPS hosting
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95202 is a reply to message #95199] |
Sun, 19 February 2012 04:50 |
|
This is my test:
-------------------------------
Apache 10 concurrents
-------------------------------
[root@mustafa1 ~]# ab -n 1000 -c 10 http://vps30ovz.hostspectra.com:30080/wp/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking vps30ovz.hostspectra.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.22
Server Hostname: vps30ovz.hostspectra.com
Server Port: 30080
Document Path: /wp/
Document Length: 294 bytes
Concurrency Level: 10
Time taken for tests: 8.539 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Non-2xx responses: 1000
Total transferred: 474000 bytes
HTML transferred: 294000 bytes
Requests per second: 117.11 [#/sec] (mean)
Time per request: 85.388 [ms] (mean)
Time per request: 8.539 [ms] (mean, across all concurrent requests)
Transfer rate: 54.21 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 41 41 0.3 41 44
Processing: 41 44 14.6 42 244
Waiting: 41 44 14.6 42 244
Total: 82 85 14.6 83 285
Percentage of the requests served within a certain time (ms)
50% 83
66% 83
75% 83
80% 84
90% 85
95% 86
98% 87
99% 208
100% 285 (longest request)
[root@mustafa1 ~]#
-------------------------------
Apache 100 concurrents
-------------------------------
[root@mustafa1 ~]# ab -n 1000 -c 100 http://vps30ovz.hostspectra.com:30080/wp/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking vps30ovz.hostspectra.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.22
Server Hostname: vps30ovz.hostspectra.com
Server Port: 30080
Document Path: /wp/
Document Length: 294 bytes
Concurrency Level: 100
Time taken for tests: 4.232 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Non-2xx responses: 1000
Total transferred: 474000 bytes
HTML transferred: 294000 bytes
Requests per second: 236.30 [#/sec] (mean)
Time per request: 423.183 [ms] (mean)
Time per request: 4.232 [ms] (mean, across all concurrent requests)
Transfer rate: 109.38 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 41 41 0.2 41 44
Processing: 42 362 58.3 377 419
Waiting: 42 362 58.3 377 419
Total: 83 404 58.3 418 460
Percentage of the requests served within a certain time (ms)
50% 418
66% 419
75% 419
80% 420
90% 421
95% 421
98% 422
99% 456
100% 460 (longest request)
[root@mustafa1 ~]#
-------------------------------
Nginx-proxy 10 concurrents
-------------------------------
[root@mustafa1 ~]# ab -n 1000 -c 10 http://vps30ovz.hostspectra.com:80/wp/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking vps30ovz.hostspectra.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: nginx
Server Hostname: vps30ovz.hostspectra.com
Server Port: 80
Document Path: /wp/
Document Length: 291 bytes
Concurrency Level: 10
Time taken for tests: 8.588 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Non-2xx responses: 1000
Total transferred: 454000 bytes
HTML transferred: 291000 bytes
Requests per second: 116.44 [#/sec] (mean)
Time per request: 85.879 [ms] (mean)
Time per request: 8.588 [ms] (mean, across all concurrent requests)
Transfer rate: 51.63 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 41 41 0.2 41 42
Processing: 41 45 21.2 41 237
Waiting: 41 45 21.2 41 237
Total: 82 86 21.2 83 278
Percentage of the requests served within a certain time (ms)
50% 83
66% 83
75% 83
80% 83
90% 83
95% 85
98% 162
99% 277
100% 278 (longest request)
[root@mustafa1 ~]#
-------------------------------
Nginx-proxy 100 concurrents
-------------------------------
[root@mustafa1 ~]# ab -n 1000 -c 100 http://vps30ovz.hostspectra.com:80/wp/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking vps30ovz.hostspectra.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: nginx
Server Hostname: vps30ovz.hostspectra.com
Server Port: 80
Document Path: /wp/
Document Length: 291 bytes
Concurrency Level: 100
Time taken for tests: 0.879 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Non-2xx responses: 1000
Total transferred: 454000 bytes
HTML transferred: 291000 bytes
Requests per second: 1137.08 [#/sec] (mean)
Time per request: 87.944 [ms] (mean)
Time per request: 0.879 [ms] (mean, across all concurrent requests)
Transfer rate: 504.14 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 41 41 0.5 41 46
Processing: 41 45 8.8 42 82
Waiting: 41 44 8.3 42 81
Total: 82 86 8.8 83 123
Percentage of the requests served within a certain time (ms)
50% 83
66% 84
75% 85
80% 85
90% 86
95% 118
98% 119
99% 123
100% 123 (longest request)
[root@mustafa1 ~]#
Notes:
1. setting inside /etc/httpd/conf.d/~lxcenter.conf
...
<IfModule prefork.c>
StartServers 2
MinSpareServers 5
MaxSpareServers 10
ServerLimit 10
MaxClients 10
MaxRequestsPerChild 4000
MaxMemFree 5
SendBufferSize 65536
ReceiveBufferSize 65536
</IfModule>
...
2. In nginx/lighttpd-proxy, access to port 80 mean access to nginx/lightpd and access to port 30080 mean access to apache directly
3. Summary:
----------------------------------------------------------------------
Concurrencies
----------------------------------------------
Apache Nginx-proxy
----------------------------------------------
10 100 10 100
----------------------------------------------------------------------
1. Requests/second 117.11 236.30 116.44 1137.08
2. Transfer rate 54.21 109.38 51.63 504.14
----------------------------------------------------------------------
[Updated on: Sun, 19 February 2012 04:52] Report message to a moderator |
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95226 is a reply to message #95224] |
Sun, 19 February 2012 09:46 |
|
lupetalo wrote on Sun, 19 February 2012 09:40Actualy i cant make it run on test server with non static data... No errors apeared and i folowed procedure... i get nginx error 502 or "The page you are looking for is temporarily unavailable.
Please try again later."
Worked after reboot... Will post sone results...
|
|
|
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95253 is a reply to message #95249] |
Mon, 20 February 2012 01:20 |
|
nginx vs apache hmmmm
i think so many people will choose apache for default webserver ( simple and support .htaccess, and many site have tutorial about it )
but, if them get high user, high unique visitor, have black hat for sites ( in one server ). i think them will think twice to use it
---This Is My Experience---
apache when get many user, using blackhat seo ( like AGC : Automated generated Content ), it allways get high load, so need more resource ( i.e memory, cpu core, hardware ). so lets talk with budget  . "How Much Money just For Server"
Lighty ( Lighttpd ), It's fast but sometimes Lighty eat more more memory ussage.
for two reasons in above, i will choose nginx .
in a fact, nginx can handle my forum with 1GB memory and 2Gb and just less then 700MB ussage( 3000 - 8000 unique visitor per day, and 1xxx thread/post per day ).when i'm using apache, i allways get high load
this is the answer http://forum.lxcenter.org/index.php?t=msg&th=17931&s tart=0& 7 of 10 member need nginx
i hope my english not make you confuse 
ready
|
|
|
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95307 is a reply to message #95306] |
Tue, 21 February 2012 09:15 |
|
shazar wrote on Tue, 21 February 2012 21:04I was working with prandah and have a few suggestions for this config. Thought this may just be for the general apache, I don't know for certain. I think you should not need to run any scripts when switching between the drivers and the switch should do more to remove itself so that when switching back, you are guaranteed a fresh version of the config.
Yes, still have a problem when switch program need run 'sh /script/fixweb'.
It's possible to including 'fixweb' process on 'switch program' but I don't want complain from users that have many domains (say it more than 1000 domains).
Maybe add option 'fix configuration' on 'switch program'.
|
|
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95334 is a reply to message #95332] |
Tue, 21 February 2012 20:52 |
|
I don't know your config well enough but I would recommend first to remove anything that you deal with during the installme phase:
//Remove any previous httpd
system("rm -rf /etc/httpd");
system("rm -rf /home/httpd/conf");
system("rm -rf /home/apache/conf");
system("rm -f /etc/init.d/httpd");
system("rm -f /etc/sysconfig/httpd");
Any other directories that you add conf files to as well as they are included as *.conf, meaning anything left there from something else may interfere.
also probably better to use lx functions for removing files and folders...
https://www.mercuryvps.com
HyperVM Xen VPS hosting
|
|
|
|
|
|
|
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95435 is a reply to message #95433] |
Fri, 24 February 2012 10:02 |
|
mustafa I still think you are missing the point. Let me present it like this.
1. A user installs Kloxo for the first time. Apache is the default and is installed so he has a fresh install and configuration of it. The server goes out of memory because the apache drivers set bad defaults. The user add custom.conf to the conf.d directory but makes it works and just leaves it there.
2. The user decides to switch to lighttpd and it works fine.
3. There is an update to the apache driver that sets better defaults. It lowers the MaxClients from 256 to a more suitable number when installed and doesn't change if the user sets it to something else.
4. The user switches back to apache hoping this solves the problem, but his conf file is still in conf.d and he forgot about it.
Do you see what I am suggesting now? You may FIX the conf files that are made by Kloxo, but you leave conf files behind still.
https://www.mercuryvps.com
HyperVM Xen VPS hosting
|
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95453 is a reply to message #95449] |
Fri, 24 February 2012 20:56 |
|
@AviationAtom,
Yes, switch to lighttpd and then select fastcgi will automaticly use php-fpm. Nginx automaticly use php-fpm. For apache, must be select php-type as php-fpm_event or php-fpm_worker.
@shazar,
As your suggest, I change maxclients and serverlimit as the same as maxspareservers/maxsparethreads when running apache-optimize. In nginx/lighttpd-proxy, values only half.
I found memory leaks (buffers and cached always growing) found on xen and dedi.
[Updated on: Fri, 24 February 2012 20:56] Report message to a moderator |
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95457 is a reply to message #95453] |
Fri, 24 February 2012 23:18 |
|
Again you never understand what I am saying or completely miss my point. This is just a scenario. I am sorry, but I don't know how to get you to better understand.
And no there is no memory leak, you must ignore buffers and cache and stop trying to clear them. Buffers and cache are good for your overall performance. If the system needs to allocate more memory, the cache will be flushed on its own. I have said this quite a few times here. Again this is something that you just are not understanding.
https://www.mercuryvps.com
HyperVM Xen VPS hosting
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95458 is a reply to message #95457] |
Fri, 24 February 2012 23:22 |
|
shazar wrote on Sat, 25 February 2012 11:18Again you never understand what I am saying or completely miss my point. This is just a scenario. I am sorry, but I don't know how to get you to better understand.
And no there is no memory leak, you must ignore buffers and cache and stop trying to clear them. Buffers and cache are good for your overall performance. If the system needs to allocate more memory, the cache will be flushed on its own. I have said this quite a few times here. Again this is something that you just are not understanding.
Do you test on openvz also?. Buffers and cached issue only found on xen and dedicated; not on openvz.
|
|
|
| Re: [PRE] Nginx/Lighttpd proxy and template-based web config [message #95459 is a reply to message #95458] |
Fri, 24 February 2012 23:56 |
|
Not an issue. a Xen domU (guest) sees its dedicated memory the same as s dedicated installation of Linux. When you do file intensive operations writes are buffered in memory so when you read the file you read from the memory instead of having to do disk operations. When you do a copy of a file, it is much slower on a system with low memory as on a system with more memory the copy is complete after it is buffered into memory. In the background it is still writing to disk, If you try to read the new file you still read it fine as it is reading from that buffered/cached memory. It works out great. There is no problem at all with the buffering and caching.
https://www.mercuryvps.com
HyperVM Xen VPS hosting
|
|
|
HyperVM & Kloxo Support
Members
Search
Help
Register
Login
Home
