Benchmark 1 - Dynamic pages vs. static pages vs. Memcached
Published on 2010-12-14.
This is a benchmark test of the loading speed of dynamic pages vs. static pages vs. pages stored in Memchached using PHP. The test was performed on an Intel P4 1.7 Ghz with 1036028 kB of memory over a privat LAN. The server runs MySQL 5.0.51a and PHP 5.3.3 on a standard Debian installation. Between each change of settings all related processes was restarted.
The first test is run using time curl
. The first time Memcached is run isn't displayed in any of the tests below (that's when the data get stored in the cache).
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row
Loading dynamic data 3 times.
real 0m0.030s real 0m0.033s real 0m0.031s
Loading static data 3 times (134 KB).
real 0m0.019s real 0m0.019s real 0m0.020s
Loading dynamic data from Memcached 3 times.
real 0m0.031s real 0m0.038s real 0m0.048s
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 10 times
Loading dynamic data 3 times.
real 0m0.247s real 0m0.249s real 0m0.247s
Loading static data 3 times (1.3 MB).
real 0m0.124s real 0m0.124s real 0m0.123s
Loading dynamic data from Memcached 3 times.
real 0m0.213s real 0m0.214s real 0m0.208s
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 20 times
Loading dynamic data 3 times.
real 0m0.557s real 0m0.577s real 0m0.562s
Loading static data 3 times (2.6 MB).
real 0m0.325s real 0m0.323s real 0m0.325s
Loading dynamic data from Memcached 3 times.
real 0m0.398s real 0m0.410s real 0m0.401s
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 30 times
Loading dynamic data 3 times.
real 0m0.830s real 0m0.831s real 0m0.836s
Loading static data 3 times (3.9 MB).
real 0m0.471s real 0m0.467s real 0m0.475s
Loading dynamic data from Memcached 3 times.
real 0m0.578s real 0m0.569s real 0m0.585s
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 40 times
Loading dynamic data 3 times.
real 0m1.099s real 0m1.100s real 0m1.106s
Loading static data 3 times (5.2 MB).
real 0m0.615s real 0m0.638s real 0m0.621s
Loading dynamic data from Memcached 3 times.
real 0m0.766s real 0m0.764s real 0m0.773s
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 50 times
Loading dynamic data 3 times.
real 0m1.374s real 0m1.362s real 0m1.372s
Loading static data 3 times (6.5 MB).
real 0m0.772s real 0m0.769s real 0m0.763s
Loading dynamic data from Memcached 3 times.
real 0m0.933s real 0m0.953s real 0m0.945s
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 100 times
Loading dynamic data 3 times.
real 0m2.734s real 0m2.728s real 0m2.732s
Loading static data 3 times (12.9 MB).
real 0m1.502s real 0m1.508s real 0m1.507s
Loading dynamic data from Memcached 3 times.
real 0m1.870s real 0m1.868s real 0m1.867s
Using the Apache AB tool
In the next test I will try re-run some of the above tests, but this time I will test using the Apache HTTP server benchmarking tool:
$ ab -kc 10 -t 30 http://webserver/test.php
This will open 10 connections, using Keep-Alive on them and hammering the webserver for 30 seconds through those connections.
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row
Loading dynamic data.
Finished 1517 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 135916 bytes Concurrency Level: 10 Time taken for tests: 30.007 seconds Complete requests: 1517 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 206795814 bytes HTML transferred: 206440256 bytes Requests per second: 50.55 [#/sec] (mean) Time per request: 197.808 [ms] (mean) Time per request: 19.781 [ms] (mean, across all concurrent requests) Transfer rate: 6729.95 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 6 6.2 5 34 Processing: 52 191 56.1 185 524 Waiting: 12 102 46.3 100 479 Total: 52 197 56.5 191 535 Percentage of the requests served within a certain time (ms) 50% 190 66% 213 75% 227 80% 237 90% 261 95% 288 98% 347 99% 375 100% 535 (longest request)
Loading static data (134 KB).
Finished 2580 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /static1.html Document Length: 136438 bytes Concurrency Level: 10 Time taken for tests: 30.011 seconds Complete requests: 2580 Failed requests: 0 Write errors: 0 Keep-Alive requests: 2560 Total transferred: 352966330 bytes HTML transferred: 352010040 bytes Requests per second: 85.97 [#/sec] (mean) Time per request: 116.323 [ms] (mean) Time per request: 11.632 [ms] (mean, across all concurrent requests) Transfer rate: 11485.41 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 5.5 0 105 Processing: 37 116 10.7 116 267 Waiting: 2 99 16.9 105 126 Total: 38 116 13.3 116 290 Percentage of the requests served within a certain time (ms) 50% 116 66% 116 75% 116 80% 116 90% 116 95% 118 98% 129 99% 175 100% 290 (longest request)
Loading dynamic data from Memcached.
Finished 2295 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 135898 bytes Concurrency Level: 10 Time taken for tests: 30.010 seconds Complete requests: 2295 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 312996949 bytes HTML transferred: 312460583 bytes Requests per second: 76.48 [#/sec] (mean) Time per request: 130.762 [ms] (mean) Time per request: 13.076 [ms] (mean, across all concurrent requests) Transfer rate: 10185.38 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 11 6.5 10 35 Processing: 25 119 46.9 112 435 Waiting: 7 33 23.9 27 305 Total: 29 130 47.8 121 436 Percentage of the requests served within a certain time (ms) 50% 121 66% 138 75% 151 80% 161 90% 191 95% 223 98% 265 99% 296 100% 436 (longest request)
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 10 times
Loading dynamic data.
Finished 180 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 1356685 bytes Concurrency Level: 10 Time taken for tests: 30.257 seconds Complete requests: 180 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 244327776 bytes HTML transferred: 244285370 bytes Requests per second: 5.95 [#/sec] (mean) Time per request: 1680.941 [ms] (mean) Time per request: 168.094 [ms] (mean, across all concurrent requests) Transfer rate: 7885.84 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 12 17.1 3 88 Processing: 1035 1623 187.2 1617 2255 Waiting: 738 1349 165.9 1367 1955 Total: 1084 1634 191.2 1628 2309 Percentage of the requests served within a certain time (ms) 50% 1628 66% 1696 75% 1730 80% 1761 90% 1854 95% 1942 98% 2112 99% 2189 100% 2309 (longest request)
Loading static data (1.3 MB).
Finished 255 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /static2.html Document Length: 1361896 bytes Concurrency Level: 10 Time taken for tests: 30.018 seconds Complete requests: 255 Failed requests: 0 Write errors: 0 Keep-Alive requests: 255 Total transferred: 353160469 bytes HTML transferred: 353061987 bytes Requests per second: 8.49 [#/sec] (mean) Time per request: 1177.181 [ms] (mean) Time per request: 117.718 [ms] (mean, across all concurrent requests) Transfer rate: 11489.17 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 395 1153 463.6 1041 3199 Waiting: 2 110 22.7 115 142 Total: 395 1153 463.6 1041 3199 Percentage of the requests served within a certain time (ms) 50% 1039 66% 1224 75% 1307 80% 1400 90% 1753 95% 2093 98% 2384 99% 2956 100% 3199 (longest request)
Loading dynamic data from Memcached.
Finished 248 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 1356505 bytes Concurrency Level: 10 Time taken for tests: 30.085 seconds Complete requests: 248 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 342312256 bytes HTML transferred: 342252608 bytes Requests per second: 8.24 [#/sec] (mean) Time per request: 1213.088 [ms] (mean) Time per request: 121.309 [ms] (mean, across all concurrent requests) Transfer rate: 11111.65 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 35 24.8 30 112 Processing: 444 1144 480.5 1076 2737 Waiting: 83 246 126.2 216 874 Total: 484 1180 480.4 1113 2825 Percentage of the requests served within a certain time (ms) 50% 1113 66% 1366 75% 1484 80% 1571 90% 1824 95% 2013 98% 2492 99% 2672 100% 2825 (longest request)
From the above it is very clear that static pages significantly improve page loading speed. Memcached improves page loading speed significantly too, but it comes with a price, mainly memory.
Before I end the testing I would like to try to use the PHP include()
function to include static content and see how much such an approach slows down the page loading speed.
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row
Loading static data using include()
.
Finished 2543 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 136438 bytes Concurrency Level: 10 Time taken for tests: 30.003 seconds Complete requests: 2543 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 347874361 bytes HTML transferred: 347280444 bytes Requests per second: 84.76 [#/sec] (mean) Time per request: 117.984 [ms] (mean) Time per request: 11.798 [ms] (mean, across all concurrent requests) Transfer rate: 11322.76 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 13 5.3 13 34 Processing: 24 105 24.5 103 211 Waiting: 4 23 9.4 22 74 Total: 34 118 25.1 115 220 Percentage of the requests served within a certain time (ms) 50% 115 66% 123 75% 130 80% 135 90% 151 95% 162 98% 178 99% 186 100% 220 (longest request)
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 10 times
Loading static data using include()
Finished 256 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 1361896 bytes Concurrency Level: 10 Time taken for tests: 30.021 seconds Complete requests: 256 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 351585096 bytes HTML transferred: 351523817 bytes Requests per second: 8.53 [#/sec] (mean) Time per request: 1172.713 [ms] (mean) Time per request: 117.271 [ms] (mean, across all concurrent requests) Transfer rate: 11436.65 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 57 22.4 57 118 Processing: 236 1097 143.2 1093 1470 Waiting: 55 124 39.3 116 265 Total: 236 1154 139.6 1141 1522 Percentage of the requests served within a certain time (ms) 50% 1141 66% 1180 75% 1206 80% 1236 90% 1335 95% 1409 98% 1427 99% 1473 100% 1522 (longest request)
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 20 times
Loading static data using include()
Finished 126 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /test.php Document Length: 2723516 bytes Concurrency Level: 10 Time taken for tests: 30.471 seconds Complete requests: 126 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 356600854 bytes HTML transferred: 356569399 bytes Requests per second: 4.14 [#/sec] (mean) Time per request: 2418.332 [ms] (mean) Time per request: 241.833 [ms] (mean, across all concurrent requests) Transfer rate: 11428.68 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 72 29.8 78 119 Processing: 848 2227 397.5 2144 3576 Waiting: 50 220 89.8 198 539 Total: 849 2299 398.5 2235 3689 Percentage of the requests served within a certain time (ms) 50% 2235 66% 2333 75% 2424 80% 2466 90% 2623 95% 3384 98% 3566 99% 3589 100% 3689 (longest request)
I'm going to try loading the same page directly as HTML.
Displaying a table from MySQL containing 1.080 posts with only 1 column in each row - looped 20 times
Loading static data (2.6 MB).
Finished 127 requests Server Software: Apache/2.2.9 Server Hostname: webserver Server Port: 83 Document Path: /static3.html Document Length: 2723516 bytes Concurrency Level: 10 Time taken for tests: 30.160 seconds Complete requests: 127 Failed requests: 0 Write errors: 0 Keep-Alive requests: 127 Total transferred: 354821737 bytes HTML transferred: 354770999 bytes Requests per second: 4.21 [#/sec] (mean) Time per request: 2374.788 [ms] (mean) Time per request: 237.479 [ms] (mean, across all concurrent requests) Transfer rate: 11488.98 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 1039 2301 638.1 2156 5480 Waiting: 2 109 29.7 120 133 Total: 1039 2301 638.1 2156 5480 Percentage of the requests served within a certain time (ms) 50% 2151 66% 2313 75% 2597 80% 2727 90% 3128 95% 3384 98% 4037 99% 4503 100% 5480 (longest request)
Conclusion
Using the include()
function to display static pages doesn't slow down the loading time, but there is a major difference in the loading time between serving pages dynamically, serving static pages, and using Memcached as a memory cache.
Caching can be performed at many levels, but the most important level is HTTP. A static HTML document that uses HTTP caching Last-Modified header tag will give a much better performance than a dynamically generated document.
The only bottleneck with static pages will be the maximum number of child processes that the webserver can create, but this is another issue all together.