Friday, July 31, 2026
1 change · saas-19.2
Enhancements to existing features
The bank reconciliation setup now handles long payment references much more efficiently when creating automatic reconciliation rules. This prevents memory errors and reduces delays for customers processing bank statement lines with lengthy transaction descriptions.
Original PR description
When setting an account to an account.bank.statement there is a step to automatically create a reconciliation rule/model if one does not exist already. To do so, we retrieve 5…
When setting an account to an account.bank.statement there is a step to automatically create a reconciliation rule/model if one does not exist already. To do so, we retrieve 5 account.bank.statement.lines and use them to define the reconciliation model config. A matching is done on the payment_ref of the account.bank.statement.lines by finding the longest common substring in the reference. ### Current Implementation The current algorithm does so by first generating all the possible substrings for all the payment_ref before doing the intersection between these sets and returning the max if `len(substring) >=10`. This is reasonable when the payment_ref follows either a SEPA communication national standard like the Belgian one or the Creditor Reference standard (ISO 11649). For transactions with large, unstructured communication with more than 100 chars, the method `_get_common_substrings` quickly overfill the memory, sometimes raising a MemoryErorr, and takes a significant amount of time. That's because the nested function `_generate_all_substrings` generates n*(n+1)/2 substrings, with n being the lenght of a payment_ref, called `label` in `generate_all_substrings`. ### Proposed Fix This commit introduces another algorithm to find the largest common substring. It starts by taking the two smallest labels to find their substrings intersection. We know that for an arbitrary collection of labels, the intersection of their substrings sets A ∩ B ∩...∩ Z is included in the intersection of any two substrings sets. The underlying assumption of the first step is that for an arbitrary collection of labels the intersection of the substrings sets of the two smallest labels will be the smallest intersection of any given pair of substrings sets. This won't hold true everytime and using a metric such as label similarity instead of shortest string might be better. But on average this should be good enough and it's easier to implement + it removes the need of preprocessing the labels to compute the similarity. The point of the new nested function `common_substrings` is to discard common substrings as we build them. Using the current `generate_all_substsrings` on either the smallest label or both smallest labels would still generate and store a lot of substrings, especially for large labels. By yielding the common substrings as we find them, the memory footprint is vastly reduced. Lastly, the next substring in the common_substrings iterable is only checked against the remaining labels if it's longer than the current match. This speeds up the whole process ### speedup In a customer database with some account.bank.statement.line with payment_ref > 500 chars, setting a specific account (code 4970) on transactions goes from MemoryError to < 1Mb memory consumption. Because of the memory consumption it was not possible to gather timing value on the current version. Testing the new algorithm in a shell and using as labels the 5 longest payment_ref in the customer database (831, 831, 1117, 1178, 1300 chars), averaging to 2000 chars once normalised, the average time to execute `_get_common_substrings` is 900 ms ± 10.3 ms. Forward-Port-Of: odoo/enterprise#118824