[19.0][MIG] stock_account_valuation_report

[AaronHForgeFlow]

The stock value info is taken from the stock move instead of the layers

[gdgellatly]

Static review + Functional test on live migrated db with 80k products and 10s of millions of moves. Essentially functionally equivelant despite move from svl's It is still really slow but works as expected, maybe marginally better than before.

Update: OK really slow is understating this, before it was 2 or 3 minutes, now still loading after 15 mins when filter set to Valuation Discrepancy.

Alright effectively unusable on large db's although I can see it works.

Final update: It loaded 80 records using Valuation discrepancy after 1 hour. I scrolled to the next 80 in the hope it had them all. It did not. I feel like the idea of this module has value, but we need to rethink how it is done.

[gdgellatly]

I am trying to make this more performant, but honestly, so far not much luck, just what I've found so far.

Also I think we canhandle the zero cases better. If there are no aml's and/or no moves we can skip many computations.

[gdgellatly]

How does this differ from total_value on product_product? Also why not fields.Monetary

[gdgellatly]

I have an alternative implementation on top of this PR that addresses significant performance issues and a bug we encountered:

Branch: gdgellatly:19.0-gdg-valuation-perf

Bug fix

The existing accounting quantity calculation uses a CASE WHEN aml.balance < 0 THEN -aml.quantity ELSE aml.quantity END pattern. This was correct in older Odoo versions where aml.quantity was always unsigned (positive), but in v19 aml.quantity is already signed (negative for credit/outgoing lines). The CASE double-negates outgoing quantities, making them positive — so outgoing goods are added to the accounting quantity instead of subtracted.

Fix: replaced with sum(aml.quantity).

Performance changes

• Use core's total_value and qty_available instead of iterating stock.move records and summing remaining_qty/remaining_value — leverages Odoo's batch-optimised computation
• Pre-filter products via SQL — only storable products with done stock moves are considered, skipping thousands of irrelevant products
• Filter AML query by valuation accounts only — there are typically very few valuation accounts per company, so adding account_id IN (...) dramatically reduces the rows scanned
• Approximate stock value in discrepancy search for AVCO/Standard — uses qty_available * standard_price for Standard and AVCO products instead of triggering the expensive per-move replay during the search filter. FIFO products still use the exact total_value. The accurate total_value from core is always displayed in the list view for the rendered page.
• Removed pre-computed Many2many fields (stock_fifo_real_time_aml_ids, stock_move_valuated_ids) — button drill-downs now build their domains on click, avoiding bulk computation
• fields.Monetary with company_currency_id for value fields, digits="Product Unit" for quantity fields
• Aligned wizard context key from at_date to core's to_date
• Updated view: replaced stock_value with core total_value, fixed is_storable domain, removed redundant widget="monetary" attributes

Performance

On a database with ~2000 storable products and several million stock moves, the valuation discrepancy filter went from over 60 minutes (original implementation timed out) to ~30 seconds. The remaining ~30s is dominated by core's _compute_value rendering total_value for the displayed page (FIFO/AVCO replay), which is outside the scope of this module.

Caveat

The valuation discrepancy search uses qty_available * standard_price as a proxy for stock value for Standard and AVCO cost methods only. This is:

• Exact for Standard Cost products
• Very close for AVCO (standard_price is updated to weighted average on each move)
• FIFO products use the exact total_value — no approximation

This means the search filter results should be equivalent for Standard and FIFO, and near-equivalent for AVCO. The accurate value is always shown in the list view itself.

[gdgellatly]

OK I give up, this is almost impossible, the quantity field on aml is used differently in different operations, I just can't get it to make any sense. whether I use the current algorithm (definitely wrong), quantity, or -quantity. Anyway hopefully some of this helps

[AaronHForgeFlow]

Hi @gdgellatly, thank you, I am afraid I have not had the time to review your branch.

What's the overall performance improvement for your large database? I'll review it and likely merge it to this branch.

[gdgellatly]

Hi @gdgellatly, thank you, I am afraid I have not had the time to review your branch.

What's the overall performance improvement for your large database? I'll review it and likely merge it to this branch.

@AaronHForgeFlow It is just a bunch of experiments, and not very well working ones at that. I would not merge it. I actually feel like we might need to go back to the old v7 days where we used to maintain historical aggregates to lower the search window. I will relook later this week to see if we can more easily integrate that approach because it won't be just this module, just this module because of it's non stored computes is super badly affected

[AaronHForgeFlow]

Thank you @gdgellatly , I applied the code fixes suggestions. Also partial of the performance improvements suggestions. In a small size database (>10000 products, around (1000 stock moves and journal items) the performance improvement is significant (from 5 seconds to almost immediate). Likely not sufficient for large databases, but I think they are useful for small companies.

For large companies, we can store the report and run it overnight with an scheduled action, but I prefer to wait until I have a real use case in v19.

[rrebollo]

I did some suggestions for your consideration

[rrebollo]

@AaronHForgeFlow would you be so kind to review my #479 in return?

[AaronHForgeFlow]

@rrebollo Thank you for the suggestions, I applied them all.

[rrebollo]

Code Review. LGTM!

[BhaveshHeliconia]

Functional review LGTM!

[gdgellatly]

Thank you @gdgellatly , I applied the code fixes suggestions. Also partial of the performance improvements suggestions. In a small size database (>10000 products, around (1000 stock moves and journal items) the performance improvement is significant (from 5 seconds to almost immediate). Likely not sufficient for large databases, but I think they are useful for small companies.

For large companies, we can store the report and run it overnight with an scheduled action, but I prefer to wait until I have a real use case in v19.

Hi @AaronHForgeFlow I have not got back to this. But my current thinking is that we create markers to reduce the moves under consideration. You will notice that in certain operation such as changing standard price odoo creates a stock value record. When computing stock value it limits the moves search to the earliest of these records for AVCO/FIFO replay. I have not looked at how to implement yet.

We have also traced back most of the performance issues to lot valuated products, so the scope of issues may be limited and I am working with Odoo to see what we can do.