Let's have a quick look at the memory consumption in both the cases (mysqlnd and libmysql) with an example before delving into the internals. For easy comparison, the sample PHP script does not modify any of the arrays returned from the fetch method. The following example uses DTrace on Sun Solaris to monitor the calls to
malloc() and prints the requested bytes of memory on the standard output.Source code for the script: PHPmysqliClient.php. MySQL table structure and the sample data are shown in the other blog post: Demonstrating the Features of MySQL Native Driver for PHP, mysqlnd.
bash# cat monitormalloc.d
#!/usr/sbin/dtrace -s
pid$1:libc:malloc:entry
{
printf("\t\tSize : %d Bytes", arg0);
ustack();
@malloc[probefunc] = quantize(arg0);
}
CASE 1: ext/mysqli with libmysql
In one terminal window:
bash# /opt/coolstack/php5/bin/php PHPmysqliClient.php
In another terminal window, run monitormalloc.d `pgrep php` and press any key to continue ..
Retrieved 3 row(s).
CityName
--------
Hyderabad, India
San Francisco, USA
Sydney, Australia
In another terminal window:
bash# ./monitormalloc.d `pgrep php`
dtrace: script './monitormalloc.d' matched 1 probe
CPU ID FUNCTION:NAME
0 80920 malloc:entry Size : 964 Bytes
libc.so.1`malloc
libmysqlclient.so.16.0.0`0xce2173c1
libmysqlclient.so.16.0.0`0xce23bb47
mysqli.so`0xce11d292
0 80920 malloc:entry Size : 20 Bytes
libc.so.1`malloc
libmysqlclient.so.16.0.0`0xce2173c1
libmysqlclient.so.16.0.0`0xce23da60
mysqli.so`0xce11dc72
0 80920 malloc:entry Size : 20 Bytes
libc.so.1`malloc
libmysqlclient.so.16.0.0`0xce2173c1
libmysqlclient.so.16.0.0`0xce21d991
libmysqlclient.so.16.0.0`0xce21d9ce
libmysqlclient.so.16.0.0`0xce23da72
mysqli.so`0xce11dc72
0 80920 malloc:entry Size : 17 Bytes
... elided stack traces for brevity ...
0 80920 malloc:entry Size : 152 Bytes
0 80920 malloc:entry Size : 16384 Bytes
0 80920 malloc:entry Size : 8199 Bytes
0 80920 malloc:entry Size : 7 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 18261 Bytes
0 80920 malloc:entry Size : 58 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 4088 Bytes
0 80920 malloc:entry Size : 120 Bytes
0 80920 malloc:entry Size : 5 Bytes
0 80920 malloc:entry Size : 6 Bytes
0 80920 malloc:entry Size : 5 Bytes
0 80920 malloc:entry Size : 56 Bytes
0 80920 malloc:entry Size : 8164 Bytes
0 80920 malloc:entry Size : 8164 Bytes
0 80920 malloc:entry Size : 92 Bytes
0 80920 malloc:entry Size : 56 Bytes
0 80920 malloc:entry Size : 8164 Bytes
libc.so.1`malloc
libmysqlclient.so.16.0.0`0xce2173c1
libmysqlclient.so.16.0.0`0xce21a27b
libmysqlclient.so.16.0.0`0xce23b8a4
libmysqlclient.so.16.0.0`0xce23d4fa
mysqli.so`0xce11fe56
0 80920 malloc:entry Size : 262144 Bytes
libc.so.1`malloc
php`0x856fb98
^C
malloc
value ------------- Distribution ------------- count
2 | 0
4 |@@@@@ 4
8 | 0
16 |@@@@ 3
32 |@@@@ 3
64 |@@@ 2
128 |@ 1
256 | 0
512 |@ 1
1024 | 0
2048 |@@@@@@@@@@@@@@ 11
4096 |@@@@ 3
8192 |@ 1
16384 |@@@ 2
32768 | 0
65536 | 0
131072 | 0
262144 |@ 1
524288 | 0
CASE 2: ext/mysqli with mysqlnd
In one terminal window:
bash# /export/home/php53/bin/php PHPmysqliClient.php
In another terminal window, run monitormalloc.d `pgrep php` and press any key to continue ..
Retrieved 3 row(s).
CityName
--------
Hyderabad, India
San Francisco, USA
Sydney, Australia
In another terminal window:
bash# ./monitormalloc.d `pgrep php`
dtrace: script './monitormalloc.d' matched 1 probe
CPU ID FUNCTION:NAME
0 80920 malloc:entry Size : 262144 Bytes
libc.so.1`malloc
php`0x82f702b
php`0x82f80ab
php`0x82f841f
php`0x82f98c4
php`0x82c7668
php`0x83c30ae
php`0x80c059c
^C
malloc
value ------------- Distribution ------------- count
131072 | 0
262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1
524288 | 0
In the case of ext/mysqli with libmysql, there are more than 25 calls to
malloc() accounting to a total size around 367KB, where as in the case of ext/mysqli with mysqlnd, there is only one call to malloc() with a size of 256KB. In other words, mysqlnd is consuming 30% less memory relative to libmysql to do similar database operations (in reality, it is incorrect to treat every byte allocated as the memory consumed unless there exists a corresponding memory mapping -- however for the sake of this discussion, let's just assume that all the allocated bytes are eventually consumed).The memory savings in the above example are the result of mysqlnd's ability to hold the results only once in the memory. On the other hand, as libmysql is not a part of PHP, some of the results fetched by libmysql will be copied into memory twice. When libmysql fetches the data from the MySQL Server, it puts the data into its own buffers. Then the data gets copied from the libmysql buffers into respective ext/mysqli data structures (often referred as zvals) before it is made available to the PHP clients to consume. So with ext/mysqli and libmysql, there might be two copies of the data in the main memory - one copy inside the libmysql buffers and the other inside zvals. With mysqlnd, there might be only one copy of the data in the memory. mysqlnd also uses buffers but links the zval structures directly to the read buffers, wherever possible. Therefore in majority of the instances, mysqlnd consumes less memory relative to libmysql. In the worst case, it may consume as much memory as that of libmysql. The total memory savings depend on the size of the buffered result set.
Shown below is the simplified behind-the-scenes actions of ext/mysqli with libmysql and ext/mysqli with mysqlnd when mysqli sends a query:
ext/mysqli with libmysql
- mysqli sends a query
- result set gets fetched into libmysql buffers
- mysqli allocates zvals, then new buffers
- mysqli copies data from libmysql to its own buffers
- mysqli calls
mysql_free_result()and deallocates libmysql buffers
ext/mysqli with mysqlnd
- mysqli sends a query
- result set gets fetched row by row -- every row is a different buffer
- mysqlnd creates a result set of zvals pointing to the buffers
- mysqli calls
mysqlnd_free_result()and deallocates the row buffers
In short, ext/mysqli with libmysql does:
- one extra allocation for mysqli buffers
- one extra data copy
- one extra zval allocation (which can be saved with the zval cache)
Related Blog Posts:
- MySQL Native Driver for PHP, mysqlnd
- Demonstrating the Features of MySQL Native Driver for PHP, mysqlnd
Acknowledgments
Andrey Hristov & Ulf Wendel, Sun-MySQL AB
(Originally posted on blogs.sun.com at:
http://blogs.sun.com/mandalika/entry/php_memory_savings_with_mysqlnd)
_______________
Technorati Tags:
PHP | 
MySQL | 
mysqlnd
0 comments:
Post a Comment