Friday, March 8, 2024
1 change · 17.0
Enhancements to existing features
This update significantly improves the speed of tax mapping calculations in the accounting system. The change optimizes how taxes are processed during financial operations, making the system 6 times faster for these calculations. This is especially important during system upgrades and when processing large volumes of financial data, reducing processing time from hours to minutes.
Original PR description
Context: upgrade script calling `recompute_fields(cr, "sale.subscription.line", ["price_subtotal"])`. It was noticed that this update step runs a long time and is not limited by the DB, but…
Context: upgrade script calling `recompute_fields(cr, "sale.subscription.line", ["price_subtotal"])`. It was noticed that this update step runs a long time and is not limited by the DB, but CPU-limited in python. Profiling the code showed that 94% of the execution time is spent in the `filtered()` call in `map_tax()` of model `account.fiscal.position` during this upgrade. The profile also shows that this whole code path effectively creates deeply nested loops, leading to 3.6 **billion** calls to the `lambda` passed to `filtered()` with only ~4k rows on model `sale.subscription.line` (upg-1218044). The idea of this improvement is to - by the grace of `read_group()` - build a dict of ids that maps tax_src_id to dest_ids outside of the loop over `taxes` and then build the result only through lookups from that dict, reducing the loop nesting by two levels. Using cProfile again, it shows that the average runtime of `map_tax()` is ~6x faster with the patch (16.7ms vs. 2.8ms). For above cited upgrade, the process is no longer CPU limited and the runtime of this step is reduced from ~8h to ~2.5h, which is significant enough to be relevant for upgrades. Considering this code is unchanged on branches up to `master`, this should also reduce response times in normal operation, when recomputations involving multiple calls to `map_tax()` are triggered by user action. Forward-Port-Of: odoo/odoo#151455