fix(data): backfill ingredient concern tiers#1169
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 358d776c2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| WHERE p.is_deprecated IS NOT TRUE | ||
| ) | ||
| UPDATE products p | ||
| SET ingredient_concern_score = cs.score |
There was a problem hiding this comment.
When this migration changes products.ingredient_concern_score, the stored unhealthiness_score is left at the value computed from the old concern score. That concern value is an input to compute_unhealthiness_v32 in score_category, and the CI post-enrichment script explicitly re-scores after the same update, but the production migration commits immediately after line 133. Any product whose tier is backfilled here will expose stale overall scores until some later full re-score happens.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
This PR backfills ingredient_ref.concern_tier using an additive/E-code classifier, recomputes products.ingredient_concern_score from tier weights (15/40/100, capped at 100), and adds CI post-enrichment + QA guards to keep concern-tier population and scoring consistent going forward.
Changes:
- Adds a Supabase migration to classify concern tiers (without downgrading) and recompute
ingredient_concern_scorefor all non-deprecated products. - Extends
db/ci_post_enrichment.sqlto populate concern tiers before recomputingingredient_concern_score(and sets score to 0 when no tiered ingredients exist). - Adds QA checks for concern-tier population and a formula-consistency test for
ingredient_concern_score.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| supabase/migrations/20260605223000_backfill_ingredient_concern_tiers.sql | New backfill migration for concern tiers + product concern-score recomputation. |
| db/ci_post_enrichment.sql | Adds post-enrichment concern-tier classification and refactors concern-score recompute to cover all products. |
| db/qa/QA__scoring_formula_tests.sql | Adds a QA test asserting ingredient_concern_score matches the tier-weight formula. |
| db/qa/QA__ingredient_quality.sql | Adds QA guards to ensure concern tiers and concern scores are populated consistently. |
| OR n LIKE '%potassium nitrate%' | ||
| THEN 3 | ||
|
|
||
| WHEN e_code IN ('e133','e150d','e211','e220','e223','e319','e385','e407','e407a','e621','e950','e951','e955') |
| WHEN e_code IN ( | ||
| 'e150','e150a','e150b','e150c','e172','e200','e202', | ||
| 'e281','e282','e338','e339','e340','e341', | ||
| 'e420','e421','e422','e440','e450','e451','e452', |
| OR n LIKE '%potassium nitrate%' | ||
| THEN 'Strict additive/E-code classifier: nitrite/nitrate preservative concern' | ||
|
|
||
| WHEN e_code IN ('e133','e150d','e211','e220','e223','e319','e385','e407','e407a','e621','e950','e951','e955') |
| WHEN e_code IN ( | ||
| 'e150','e150a','e150b','e150c','e172','e200','e202', | ||
| 'e281','e282','e338','e339','e340','e341', | ||
| 'e420','e421','e422','e440','e450','e451','e452', |
| OR n LIKE '%potassium nitrate%' | ||
| THEN 3 | ||
|
|
||
| WHEN e_code IN ('e133','e150d','e211','e220','e223','e319','e385','e407','e407a','e621','e950','e951','e955') |
| WHEN e_code IN ( | ||
| 'e150','e150a','e150b','e150c','e172','e200','e202', | ||
| 'e281','e282','e338','e339','e340','e341', | ||
| 'e420','e421','e422','e440','e450','e451','e452', |
| OR n LIKE '%potassium nitrate%' | ||
| THEN 'Strict additive/E-code classifier: nitrite/nitrate preservative concern' | ||
|
|
||
| WHEN e_code IN ('e133','e150d','e211','e220','e223','e319','e385','e407','e407a','e621','e950','e951','e955') |
| WHEN e_code IN ( | ||
| 'e150','e150a','e150b','e150c','e172','e200','e202', | ||
| 'e281','e282','e338','e339','e340','e341', | ||
| 'e420','e421','e422','e440','e450','e451','e452', |
|
|
||
|
|
||
| -- ═══════════════════════════════════════════════════════════════════════════ | ||
| -- Test 6: ingredient_concern_score matches concern-tier formula |
| -- ═══════════════════════════════════════════════════════════════════════════ | ||
| -- 18. concern_tier should not remain all-zero once product_ingredient exists | ||
| -- ═══════════════════════════════════════════════════════════════════════════ | ||
| SELECT '18. concern_tier populated when product ingredients exist' AS check_name, | ||
| CASE | ||
| WHEN NOT EXISTS (SELECT 1 FROM product_ingredient) THEN 0 | ||
| WHEN EXISTS ( | ||
| SELECT 1 | ||
| FROM product_ingredient pi | ||
| JOIN ingredient_ref ir ON ir.ingredient_id = pi.ingredient_id | ||
| WHERE COALESCE(ir.concern_tier, 0) > 0 | ||
| ) THEN 0 | ||
| ELSE 1 | ||
| END AS violations; |
| computed_scores AS ( | ||
| SELECT | ||
| p.product_id, | ||
| COALESCE(ps.score, 0)::int AS score | ||
| FROM products p | ||
| LEFT JOIN product_scores ps ON ps.product_id = p.product_id | ||
| WHERE p.is_deprecated IS NOT TRUE | ||
| ) |
Summary
ingredient_ref.concern_tierusing strict additive/E-code classification.products.ingredient_concern_scoreusing tier weights:Validation
UPDATE 24UPDATE 204UPDATE 0UPDATE 0Notes