Odoo Changelog — 2024-02-02

Friday, February 2, 2024

1 change

Improvements(1)

Enhancements to existing features

[FW][FIX] account_consolidation: speed up action_generate_journals

Enterprise17.0account_consolidationHIGH

This update significantly speeds up the process of generating consolidation journals in Odoo, which can be slow when dealing with large databases containing multiple companies and currencies. By implementing a smart caching mechanism for exchange rates, the system reduces unnecessary database queries. Real-world testing shows dramatic improvements—for example, processing 134,000 account entries that previously took 21.6 minutes now completes in just 3.3 minutes.

Original PR description

In databases with multiple companies, currencies and lots of `account_move_lines`, generating consolidations journals can take a lot of time. There are two bottlenecks for that. The first one comes from `_apply_historical_rates`. The second one is the creation of the `consolidation_journal_lines` records. Nothing can be done about the latter as there are just a lot of records to create. This commit focuses on the former.

To speed up `_apply_historical_rates`, a rate_cache is passed through the context to the `get_rate_for` method of consolidation rates. This method is called a lot of times (once by move_line) with mostly the same values (company_id and chart_id are fixed, only the date changes). Adding a small cache vastly reduces the number of queries on `consolidation.rate`.

#### speedup
Customer database with 2.5M account.move.lines, 2M account.moves, 202 consolidation_periods, 243 consolidation accounts, 0 consolidation_rate, 17 companies, 8512 accounts.

Timing to generate consolidation journals.

| #account_move_lines, #res_currencies | Before PR | After PR |
|:----------------------------------------------------:|:--------------:|:-----------:|
| 1036 amls, 2 currencies | 7s | 1.89s |
| 4329 amls, 7 currencies | 2.2min | 19.3s |
| 15301 amls, 7 currencies | 1.4min | 1.4min|
| 43749 amls, 6 currencies | 22.1min | 2min|
| 134055 amls, 7 currencies |21.6min | 3.3min|

The third entry only contains non-historical consolidation accounts so no visible speedup is expected.

Forward-Port-Of: odoo/enterprise#54030

#performance#consolidation#optimization#database efficiency

In databases with multiple companies, currencies and lots of `account_move_lines`, generating consolidations journals can take a lot of time. There are two bottlenecks for that. The first one comes from `_apply_historical_rates`. The second one is the creation of the `consolidation_journal_lines` records. Nothing can be done about the latter as there are just a lot of records to create. This commit focuses on the former. To speed up `_apply_historical_rates`, a rate_cache is passed through the context to the `get_rate_for` method of consolidation rates. This method is called a lot of times (once by move_line) with mostly the same values (company_id and chart_id are fixed, only the date changes). Adding a small cache vastly reduces the number of queries on `consolidation.rate`. #### speedup Customer database with 2.5M account.move.lines, 2M account.moves, 202 consolidation_periods, 243 consolidation accounts, 0 consolidation_rate, 17 companies, 8512 accounts. Timing to generate consolidation journals. | #account_move_lines, #res_currencies | Before PR | After PR | |:----------------------------------------------------:|:--------------:|:-----------:| | 1036 amls, 2 currencies | 7s | 1.89s | | 4329 amls, 7 currencies | 2.2min | 19.3s | | 15301 amls, 7 currencies | 1.4min | 1.4min| | 43749 amls, 6 currencies | 22.1min | 2min| | 134055 amls, 7 currencies |21.6min | 3.3min| The third entry only contains non-historical consolidation accounts so no visible speedup is expected. Forward-Port-Of: odoo/enterprise#54030