Benchmarking Zend Framework loader

One of the things I am doing in course of my work is performance benchmarks for various stuff - PHP, Zend products, applications, etc. Performance in PHP space is currently like alchemy - there are a lot of rumors floating around about various properties of various stuff, but much less reliable data that can be verified and used. PHP has standard bench.php script, but it covers only a small part of what real-life applications do. I wish there were more established tests and methods for benchmarking PHP engine and applications. But benchmarking is a complicated subject, and variety of PHP platforms and applications makes it harder to create useful general-purpose benchmarks.

But more to the point. On Zend Framework lists there was a topic raised about performance impact of Zend_Loader component, which is used for - no surprise here! - loading classes, including autoloading, etc. Some folks thought that since Zend_Loader is executing some code before actual loading the required file, it must cost something. And it makes sense. However, how much does it cost?
Well, the best way to know the price of something is to ask - and in this case, to run the test. So that’s what I did - I made a list of 725 Framework classes (ZF now has more than 1000 but I composed the list some time ago and had also to drop some to avoid some tricky dependencies). And I wrote two scripts - one that would load these classes with require_once and one that would load them using Zend_Loader::loadClass. Both the data file and the scripts are available for download for those that would like to play with it. I tested them with and without Zend’s bytecode cache, to see how much one can save using bytecode caching technology.

So, the results were as follows:

Without bytecode cache:

          require_once Zend_Loader
php5.2        4.42      4.42
php5.3        4.96      4.97

With bytecode cache:

           require_once Zend_Loader
php5.2        63.04     56.62
php5.3        61.28     55.52

The numbers are requests per second, so more is better. Test run on Linux dual 2GHz AMD.

What we can conclude from these?

1. It is very important to understand that it is a narrow-point benchmark that tests only one function in one specific way. Please do not draw conclusions on behavior of whole applications based only on this benchmark.
2. You do want to use bytecode caching. You won’t get 15x performance on any real application, but it does speed up loading very significantly.
3. Without bytecode caching, it doesn’t matter if you use require_once or Loader - both are equally slow
4. With bytecode caching, Loader has some overhead - explanation for this is that with file accesses eliminated, require_once of course has little left, while Loader still does a couple of function calls. But on real-life apps it’d probably be very small, provided that it’s about 10% even on loading-only huge-class-list benchmark, and your application probably does something useful instead of loading 700+ framework classes :)) Meaning, fears of using the class loader vs. doing require_once are seriously overstated.
5. 5.3 is still a moving target, to don’t put too much stake in current benchmark results for 5.3, they probably will be different by the time 5.3 is in release cycle (hopefully, better :))

P.S. This post does not talk about other things like “what if I stuff all classes I use into single file”, etc. Maybe next time.

---

Possibly related posts: (automatically generated)

• Zend Framework for developing Web Application using PHP
• Php: Php Developers!!! Zend Framework Ready For Web Development
• OSS Camp - Delhi
• Keynote: Flying KITE in the Cloud