Saturday, February 10, 2024
3 changes · 17.0
Enhancements to existing features
This update improves the speed and efficiency of the bank reconciliation process by reducing unnecessary database queries and optimizing how data is processed. The changes make reconciliation operations faster, especially when handling multiple transactions or currency exchanges, resulting in a better user experience when working with bank accounts.
Original PR description
Simply prefetch all the values because we are analyzing a list of recordsets without any prefetch ids.
This update significantly improves the performance of bank reconciliation operations by reducing the number of database queries required. The changes optimize how the system processes payment batches and reconciliation data, resulting in up to 18x faster processing for large batch payments while maintaining the same functionality.
Original PR description
Summary: ======== 1. Handle the inverse fields of `account.full.reconcile` manually 2. Batch the update of matching numbers 3. Batch the balance consistency test 4. Batch the construction of…
Summary: ======== 1. Handle the inverse fields of `account.full.reconcile` manually 2. Batch the update of matching numbers 3. Batch the balance consistency test 4. Batch the construction of reconciliation batches 5. Precompute fields on partials 1: Handle the inverse fields of `account.full.reconcile` manually ----------------------------------------------------------------- In order to update the inverse fields of `reconciled_line_ids` and `partial_reconcile_ids`, the ORM is triggering one query per full reconcile record. In the case of a big batch payment, that is a lot of round trips. This also bypasses the computation for possible tracking, which is expensive to do for 2 records at a time. That field is not tracked anyway. 2: Batch the update of matching numbers --------------------------------------- Basically the same as 1. without the tracking. 3: Batch the balance consistency test ------------------------------------- Use the various context managers around `__reconcile_plan` to check for the balance consistency only at the end in one single query. This avoids checking it at every single intermediate write done on the lines. Also doing it for the sync of dynamic lines. It should be useless but it doesn't cost anything, and allows to remove the context keys in enterprise without a cost. (they shouldn't be used explicitly, the context managers should always be used instead) Using these context managers also allows to remove all the `skip_` context keys, which should not be used: using them will mean that as soon as the journal entry is modified for any reason, some business fields will change because of the synchronisation since the context won't be the same anymore. This commit therefore also makes `_sync_dynamic_line` more resilient and avoids even more useless syncs. 4: Batch the construction of reconciliation batches --------------------------------------------------- Introduce `_reconciled_by_number`, which is basically a `_read_group`. In conjuction with `_filter_reconciled_by_number`, we can find all the reconciled lines in one single query, for as many batches as needed; as opposed to one query per batch before. 5: Precompute fields on partials -------------------------------- Simply avoid doing one batched `INSERT INTO` followed by one `UPDATE` for each record. Benchmark: ========== We are testing 2 use cases: * a lot of small matchings that are producing a full 2 by 2 * one big matching (also producing a full) To reconcile a batch payment of 500 (identical) lines in the bank reco: | | Queries | SQL | Other | |--------|---------|-------|--------| | Before | 4191 | 4.649 | 16.253 | | After | 226 | 0.989 | 3.581 | To reconcile one big batch of 1000 lines with the generic reco widget (spaghetti): | | Queries | SQL | Other | |--------|---------|-------|--------| | Before | 82 | 0.478 | 1.131 | | After | 66 | 0.634 | 0.974 | All the times are in seconds. We can see the the use case of the batch payment is improved a lot (the time is expected to be linear with regards to the number of payments), while the time for big batches is not impacted.
This update brings the spreadsheet component to version 17.0.11, including multiple performance improvements that make formula processing faster, along with bug fixes for data filtering, formatting, and export functionality. Users will experience improved responsiveness when working with spreadsheets, and several issues with data filters, formula composition, and chart panels have been resolved.
Original PR description
### Contains the following commits: https://github.com/odoo/o-spreadsheet/commit/bb51d6315 [REL] 17.0.11 https://github.com/odoo/o-spreadsheet/commit/4a419ace3 [PERF] tokenizer: faster tokenize Task:…
### Contains the following commits: https://github.com/odoo/o-spreadsheet/commit/bb51d6315 [REL] 17.0.11 https://github.com/odoo/o-spreadsheet/commit/4a419ace3 [PERF] tokenizer: faster tokenize Task: 3735948 https://github.com/odoo/o-spreadsheet/commit/c0ee19701 [PERF] tokenizer: faster parenthesis tokenize Task: 3735948 https://github.com/odoo/o-spreadsheet/commit/c795196e7 [PERF] tokenizer: faster number tokenize Task: 3735948 https://github.com/odoo/o-spreadsheet/commit/bc2fa0472 [PERF] tokenizer: faster symbol tokenize Task: 3735948 https://github.com/odoo/o-spreadsheet/commit/31338a1ec [FIX] Composer: Persistent composition when starting the edition Task: 3685891 https://github.com/odoo/o-spreadsheet/commit/8f138849b [FIX] DataFilter: Fix overlapping filters Task: 3728009 https://github.com/odoo/o-spreadsheet/commit/201aea7ea [FIX] export: unbound formula stays unbound in snapshots https://github.com/odoo/o-spreadsheet/commit/bf4f9bc37 [PERF] cell: remove useless binded function Task: 3721226 https://github.com/odoo/o-spreadsheet/commit/73d8eae56 [PERF] cell: remove range string closure Task: 3721226 https://github.com/odoo/o-spreadsheet/commit/168aff7ea [PERF] cells: transform dependencies without closure Task: 3721226 https://github.com/odoo/o-spreadsheet/commit/55a4d4c2b [FIX] formatting: do not show escape character Task: 3698283 https://github.com/odoo/o-spreadsheet/commit/04be128c8 [REM] composer: remove CTRL+Space shortcut Task: 3504025 https://github.com/odoo/o-spreadsheet/commit/be06aa9e2 [REL] 17.0.10 https://github.com/odoo/o-spreadsheet/commit/7e6810474 [FIX] chart: fix incoherent side_pannel state Task: 3380568 https://github.com/odoo/o-spreadsheet/commit/67a98ca2a [REF] model: preserve default currency format https://github.com/odoo/o-spreadsheet/commit/6acfb3b07 [FIX] export: transform unbound references Task: 3618912 https://github.com/odoo/o-spreadsheet/commit/1ad87cd5a [FIX] export: transform unbound references Task: 3618912 https://github.com/odoo/o-spreadsheet/commit/da02417c7 [FIX] export: remove line returns in formula Task: 3618912 https://github.com/odoo/o-spreadsheet/commit/d8eef54dc [FIX] Grid: Support Mac shortcuts in `Grid` Task: 3707416