In addition to HowToImprovePerformance technical note, we can discuss here ideas and facts about performance.

Indexing

This is a reminder about the future performance improvement.

1. Using TALES is much slower than specifying portal types explicitly in filtering. If we can assume that whether an object should be filtered depends only on the portal type, it would be a good idea to cache the result of a TALES expression based on a portal type + a method id.

2. z_catalog_movement_list and z_catalog_stock_list are the bottlenecks. The time required for indexing is 10 times longer than z_catalog_object_list. This is not due to the number of rows to be inserted, because the bottleneck exists in making lists of arguments, but neither in Z SQL Method nor in MySQL. It is necessary to understand why they are so slow. My guess is that retrieving objects from ZODB by resolveCategory is slow. This must be confirmed.

Seb : This was already looked by Yoshinori. What is slow is getAcquiredCategoryList. This was already a lot improved (with a transaction cache), I don't know if it would be easy to improve it again.

3. It is necessary to remove immediateReindexObject whenever possible, in particular in DeliveryBuilder. This makes the system very slow, because the same objects tend to being indexing multiple times, and much fewer objects can be indexed at a time (so MySQL becomes a bottleneck). Using after_method_id is a good solution for most cases.

Seb : Don't forget that immediateReindexObject is a coding crime because this is unusable when we have more than one user.

Memory usage

Kazuhiko : We have so many Products.ERP5Type.*.*Getter or Products.ERP5Type.*.*Setter instances and they use much memory totally. In the current implementation, dict is used for attribute storage. But if we use slots instead of dict, the memory usage should be much smaller (about 1/10 in my simple test).

wiki organization

There are two pages in this wiki ( GuidelinesForPerformanceCrimes and HowToImprovePerformance ) that can probably be merged ?

Discussion/Performance (last edited 2008-01-03 13:48:06 by localhost)