Thursday, June 19, 2025
1 change · 18.0
Enhancements to existing features
Archiving or moving folders with many large documents now avoids loading full file contents when they are not needed. This significantly reduces memory use and helps prevent failures for users managing heavy document collections.
Original PR description
## Issue When there are heavy (or a lot of) documents in a folder, the user cannot archive this folder as it would trigger a `MemoryError`. ### Analysis The `write` override could unnecessarily fetch…
## Issue
When there are heavy (or a lot of) documents in a folder, the user cannot archive this folder as it would trigger a `MemoryError`.
### Analysis
The `write` override could unnecessarily fetch the `datas` field (binary file content).
This happened because a condition checking `record.datas` was evaluated before checking if `datas` or `url` was actually present in the values being written. This could lead to slowness when e.g. archiving/unarchiving documents, as the ORM would fetch the binary content even when it was not needed.
## Solution
We leverage Python's boolean short-circuiting to ensure that this behavior is avoided.
### Benchmarks
Profiling peak memory consumption when moving 8 documents (66 Mb each) to the trash:
| Before | After | % |
|------------|------------| --- |
| 1.3 GiB | 96 Mb | - ~93% |
Query count when unlinking account.move (using Runbot data):
| Before | After |
|------------|------------|
| 66 | 72 |
(we may avoid these additional queries by prefetching the fields beforehand, let me know your thoughts)
```
[0.564 ms] query: SELECT "ir_attachment"."id", "ir_attachment"."name", "ir_attachment"."description", "ir_attachment"."res_model", "ir_attachment"."res_field", "ir_attachment"."res_id", "ir_attachment"."company_id", "ir_attachment"."type", "ir_attachment"."url", "ir_attachment"."public", "ir_attachment"."access_token", "ir_attachment"."store_fname", "ir_attachment"."file_size", "ir_attachment"."checksum", "ir_attachment"."mimetype", "ir_attachment"."create_uid", "ir_attachment"."create_date", "ir_attachment"."write_uid", "ir_attachment"."write_date", "ir_attachment"."original_id", "ir_attachment"."key", "ir_attachment"."website_id", "ir_attachment"."theme_template_id" FROM "ir_attachment" WHERE "ir_attachment"."id" IN (3398)
[0.713 ms] query: UPDATE "ir_attachment"
SET "res_id" = "__tmp"."res_id"::int4, "res_model" = "__tmp"."res_model"::VARCHAR, "write_date" = "__tmp"."write_date"::timestamp, "write_uid" = "__tmp"."write_uid"::int4
FROM (VALUES (3398, 117, 'documents.document', '2025-06-17T15:24:12.867699'::timestamp, 22)) AS "__tmp"("id", "res_id", "res_model", "write_date", "write_uid")
WHERE "ir_attachment"."id" = "__tmp"."id"
[0.305 ms] query: SELECT "ir_attachment"."id", "ir_attachment"."res_model", "ir_attachment"."res_id", "ir_attachment"."res_field", "ir_attachment"."public", "ir_attachment"."create_uid" FROM "ir_attachment" WHERE ("ir_attachment"."id" IN (3397))
[0.666 ms] query: SELECT "account_move"."id" FROM "account_move" WHERE ("account_move"."id" IN (255)) AND ("account_move"."company_id" IN (10)) ORDER BY "account_move"."date" DESC , "account_move"."name" DESC , "account_move"."invoice_date" DESC , "account_move"."id" DESC
[0.436 ms] query: SELECT "ir_attachment"."id", "ir_attachment"."name", "ir_attachment"."description", "ir_attachment"."res_model", "ir_attachment"."res_field", "ir_attachment"."res_id", "ir_attachment"."company_id", "ir_attachment"."type", "ir_attachment"."url", "ir_attachment"."public", "ir_attachment"."access_token", "ir_attachment"."store_fname", "ir_attachment"."file_size", "ir_attachment"."checksum", "ir_attachment"."mimetype", "ir_attachment"."create_uid", "ir_attachment"."create_date", "ir_attachment"."write_uid", "ir_attachment"."write_date", "ir_attachment"."original_id", "ir_attachment"."key", "ir_attachment"."website_id", "ir_attachment"."theme_template_id" FROM "ir_attachment" WHERE "ir_attachment"."id" IN (3397)
[0.436 ms] query: SELECT "documents_document"."id", "documents_document"."attachment_id", "documents_document"."active" FROM "documents_document" WHERE ("documents_document"."attachment_id" IN (3397)) ORDER BY "documents_document"."id" DESC
```
### References
opw-4552436
Memory flamegraphs available on the ticket