Repository navigation
BUG: Fix defects in V&V'd filters and align their V&V records with the template - #1841
Draft
imikejackson wants to merge 91 commits into
Draft
imikejackson wants to merge 91 commits into
imikejackson wants to merge 91 commits into
Conversation
imikejackson
force-pushed
the
topic/vv_report_cleanup
branch
4 times, most recently
from
October 22, 2026 13:28
a825ad6 to
1d991d7
Compare
An oracle shows that a filter agrees with the fixtures that the engineer chose. It does not show that those fixtures can find a defect. Issue BlueQuartzSoftware#1758 (AlignGeometries Centroid mode uses the X offset on all axes) shows the gap: no test ran that mode, and a fixture with equal per-axis offsets would also pass. * Add defect_hunt.md: bug-pattern review, mode coverage, symmetry-breaking fixtures, mutation check, and metamorphic relations * vv_policy.md: require the defect hunt after code-path enumeration * report_gates.md: add the Defect hunt section and row, the symmetry gate for oracle fixtures, and the per-mode value-assertion gate * report_template.md: add the Defect hunt section and row * oracle_classes.md: add metamorphic relations to Class 4 * Reports that reached READY FOR REVIEW before 2026-10-02 are not required to add the section until their next V&V change Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add three Part 1 bug patterns: component shape not declared on the selection parameter, parent Attribute Matrix not checked, and feature-level bounds * State where each kind of check belongs: structure in the parameter, relationships between inputs in preflight, value-dependent bounds in execute * Add a "SIMPLNX vocabulary for findings" section with the Attribute Matrix kinds and an example of SIMPLNX wording Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
V&V documents and test comments named DREAM3D 6.5.172, which is a private proof branch and was never released. Each reference now names DREAM3D 6.5.171 or describes the locally patched legacy build; no sentence claims that 6.5.171 contains a fix that only the patched build had. The FillBadData summary no longer cites the withdrawn three-way A/B run as evidence. The DBSCAN files are left for BlueQuartzSoftware#1766. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* deviation_template.md: split the file into Part 1 (differences from DREAM3D 6.5.171) and Part 2 (SIMPLNX defects and SIMPLNX behavior changes). Part 2 entries record the affected released versions and the PR and release that fix the defect. * report_template.md: include defects that an earlier PR fixed in "Bugs found and fixed", give the lead sentence for those defects, and list only Part 1 entries in the deviations table. * report_gates.md: add the gates for "Bugs found and fixed". Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
* DBSCANFilter.md: keep only the template sections and fields. Remove the Phase 2 to Phase 13 working notes. Correct the Algorithm Relationship evidence: PR BlueQuartzSoftware#1421 did not change the filter UUID. * Record Michael Jackson as the second engineer (2026-08-28). * Give stable IDs to the Random-seed defect (DBSCAN-D7) and the all-false-mask defect (DBSCAN-D8), and state the affected releases for D5 to D8. * Record the comparison against DREAM3D 6.5.171 (2026-10-20). It gives the same label for every point as the earlier local proof version. Put the three-way table in DBSCAN-D1 as its evidence. * deviations/DBSCANFilter.md: split into Part 1 (D1 to D3) and Part 2 (D4 to D8), and remove the incorrect note about a UUID change. * provenance/dbscan_test.md: cite the DREAM3D 6.5.171 run and record the second-engineer review. * The record describes tests and DBSCAN-D6 from PR BlueQuartzSoftware#1766, so merge this change after that PR. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
* DREAM3D-NX 7.4.2 was released on 2026-08-28, not 2026-08-31. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
* Write the legacy application as "DREAM.3D v6.5.171", or "DREAM.3D" for short, in all V&V templates. "DREAM3D-NX" stays for the new application. * report_template.md: add the SIMPLNX Human Name header row, list the allowed Status values, and add a list of content that does not go in a V&V report. * report_template.md: call a run on a non-stock legacy build "a local proof version", and use the DBSCAN report as the worked instance. * report_gates.md: add the Human Name row and the header-row rules to the header gates. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
* Add the SIMPLNX Human Name header row. * Write the legacy application as "DREAM.3D v6.5.171" or "DREAM.3D". Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…late * Keep only the template sections and fields, and describe the current rolling-slice algorithm. * Map 26 code paths to the 13 current test cases. * State the release ranges of D1 and D2 from the release branches and the DREAM.3D v6.5.171 source. * Record the 28 of 28 match against DREAM.3D v6.5.171 (2026-08-18). * Make the provenance sidecar follow the provenance template. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…template * Keep only the template sections and fields. * List the 16 current test cases and map 44 code paths to them. * State the release ranges of D3 to D5 from the release branches and the DREAM.3D v6.5.171 source. D3 (2D flat spacing) is still open. * Remove repeated template blocks from the provenance sidecar. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
… the template * Keep only the template sections and fields. Fold the sub-sections into the sidecars. * Add stable IDs D3 to D5 for the tolerance, Laue-index and phase-index fixes, and state the release ranges from the release branches and the DREAM.3D v6.5.171 source. * Record the DREAM.3D v6.5.171 comparison (2026-09-17): 12 of 28 cases match; a local proof version matches all 28. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
- Sign-off uses "<Name>, YYYY-MM-DD; second engineer: <Name>, YYYY-MM-DD". The second engineer is the reviewer who approved the PR with the V&V work. - A Legacy comparison must run on a current nxrunner build. Its artifacts go to vv_work:<FilterName>/ on OneDrive. - When no record shows who generated an archive, use the PR author and the dates the PR was open, marked as inferred. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
- Use one sign-off form in DBSCAN, ErodeDilateMask, ComputeFeatureSizes, and BadDataNeighborOrientationCheck. - ErodeDilateMask: Nathan Young approved PR BlueQuartzSoftware#1717, so the record is COMPLETE. Remove stale "this branch" text and an outdated review note. - Fill in the inferred archive provenance for all three pilot archives. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…he filter class Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
A reader of a V&V report runs develop or a released DREAM3D-NX. Branch names, commit hashes, and PR numbers do not tell that reader whether their version has a defect. - Rename the "Resolution in this branch" column to "Resolution". - State fixes with DREAM3D-NX release versions only. - Add Git terms to the list of content that a report must not have. The Verified commit row and the provenance sidecar keep commit and PR evidence. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…vidence paragraph Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…roval Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
… the report template Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
… template Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…late Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…rt template Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…t template Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…rt template Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…s V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…umNeighbors V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…rison Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…deviations Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…ty V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…orCAxisMisalignments V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…ors V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…ids V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
… V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…uireMinNumNeighbors V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…Axis Orientations - Check at execute time that every Feature Id indexes the Average C-Axes array (error -76405). Before, an out-of-range Feature Id wrote past the end of the feature cache. - Accumulate the c-axis sums in double and convert to float32 once. Before, the float32 sums lost precision for large features. - Add tests for out-of-range Feature Ids and an analytical large-feature case. Both fail on the old code. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…e C-Axis Misalignments - Check at execute time that every Feature Id indexes the Average C-Axes array (error -9806). Before, an out-of-range or negative Feature Id read and wrote past the end of the feature arrays. - Apply the same cell inclusion rule (positive Feature Id, positive hexagonal phase) in the standard deviation pass as in the mean pass. Before, excluded cells added zeros and inflated the standard deviation. - Accumulate the feature means in double and convert to float32 once. - Add tests for each defect. Each test fails on the old code. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
… Centroids from Triangle Geometry - Preflight rejects an empty Face Labels array (-12903) and a Face Labels array whose tuple count is not the number of faces (-12902). Before, these inputs read out of bounds or requested a huge resize. - Resize the feature matrix only for positive labels, with 64-bit comparison. - Sum vertex coordinates in double and convert to float32 once. - The exemplar comparison tolerance changes from 0.0001 to 0.05. The archive holds float32-accumulated centroids; a float64 reference computed from the archive triangles differs from them by at most 0.0415 (feature 518). The archive is unchanged. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…ple) The connected-component path marked every voxel of a 2D image as a boundary voxel because it tested the unit axis, so no hole was filled. The BFS path already ignored unit axes. The boundary test now ignores an axis of size 1. Add bool and uint8 tests for all three 2D orientations; they fail on the old code. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Describe when V&V work happens after a filter has a V&V record, so that engineers and their tools apply one approach: - Proposed trigger: a change that alters output for valid input needs a scoped V&V update in the same PR; other changes need only unchanged V&V tests; a release gate runs the legacy comparisons again. - Agreed: all V&V work stays on one branch until the policy and the report template are stable, with code fixes in separate commits. - Agreed: the three-way legacy comparison packages (stock DREAM.3D v6.5.171, a local proof version, DREAM3D-NX) and their archive rules. Link the new file from vv_policy.md. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
imikejackson
force-pushed
the
topic/vv_report_cleanup
branch
from
October 24, 2026 13:31
40268f0 to
79962a8
Compare
…Check V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…isMisorientations V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…tions V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…tion V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…cord Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…thNeighborValues V&V record Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR holds all V&V work for the filters that have a V&V record: the V&V policy and templates, the V&V records, and the code fixes for defects found in V&V'd filters. It is a draft. Do not review it yet.
The work stays on one branch until the V&V policy and the report template are stable. This prevents a series of merges that later template changes would make out of date.
V&V policy and templates
docs/vv_templates/report_template.mdexactly.<Name>, YYYY-MM-DD; second engineer: <Name>, YYYY-MM-DD. The second engineer is the reviewer who approved the PR with the V&V work. When no approval exists, the sign-off says "second engineer: pending" and the Status is READY FOR REVIEW.docs/vv_templates/vv_recertification.mdsays when V&V work happens after a filter has a V&V record. The trigger rule (does the change alter output for valid input?) is proposed for team review. The single-branch rule and the legacy comparison package rules are agreed.V&V records
Fixes in V&V'd filters
Each filter fix is a separate commit with tests that fail on the old code. The fixes are separate from the documentation commits, so they can move to their own PR if necessary.
-76405) and accumulate in double.-9806), use the same cell rule for the mean and the standard deviation, and accumulate in double.-12903,-12902) and accumulate in double. The exemplar tolerance changes from 0.0001 to 0.05 because the archive holds float32-accumulated centroids; a float64 reference from the archive triangles differs from them by at most 0.0415.Status
Test Plan
PY::tests excluded).