Add five validated bug-fix dataset entries

[alexei-dobriansky]

Summary

This PR adds five new bug-fix tasks that passed the complete BC-Bench execution workflow, along with their problem statements and screenshots.

It also updates environment setup so BC 29+ bug-fix and test-generation entries use insider artifacts, and NAV sparse checkouts include internal application projects.

The benchmark version is bumped from 0.13.0 to 0.14.0 because dataset changes affect evaluation results.

Added entries

• microsoft__BCApps-7070 — customer cannot be changed when closed subscription lines exist
• microsoft__BCApps-10345 — service-contract invoicing state is not reset after deleting an invoice
• microsoft__BCApps-10808 — exchange-rate adjustment amount is calculated incorrectly
• microsoft__BCApps-11274 — drop-shipment reversal fails at bin-mandatory locations
• microsoftInternal__NAV-246904 — mileage expense posting fails because of inconsistent rounding

Candidate selection criteria

Candidates were discovered from merged BCApps and internal NAV pull requests and were retained only when they met the following structural criteria:

• the pull request was completed into the primary development branch (main or master);
• all changed files were AL files;
• the production fix changed at least two distinct production files;
• localization copies of the same W1 object did not count as additional fix files;
• the production fix was no larger than 70 non-blank, non-comment lines;
• the change added exactly one or two automated [Test] procedures;
• the production patch added no event publishers or event subscribers;
• the production patch introduced no new AL objects, table fields, or page fields;
• every linked Azure DevOps work item was type Bug, not Slice, Feature, Task, or Deliverable;
• the change represented one focused defect rather than a feature, broad refactor, rename, port, revert, or unrelated collection of fixes;
• Sales/Purchase, Sales/Service, Sales/Projects, and similar cross-area scenarios were preferred where available.

Execution verification

Static screening was followed by full execution validation. For an entry to remain in this PR, BC-Bench had to prove that:

1. the historical repository commit can be checked out;
2. the application and test projects build against the selected BC environment;
3. the selected regression tests fail before applying the gold patch;
4. the gold production and test patches apply cleanly;
5. the application and tests rebuild successfully;
6. the same regression tests pass after applying the gold patch.

A larger initial candidate set was removed after this stage when tests already passed before the fix, still failed afterward, or historical source could not be built and published against currently available artifact snapshots.

Final successful validation: Dataset Validation and Verification run 35894324791.

Environment setup changes

• BC 29 and newer bug-fix/test-generation entries use bcinsider artifacts.
• Released versions such as BC 28 continue using public artifacts.
• NAV sparse checkout now includes App/Internal/Apps for internal application tasks.
• Missing artifact URLs now fail with an explicit diagnostic instead of an empty ArtifactUrl binding error.
• Symbol downloads use the same version-aware artifact selection as container setup.

Validation

• python -m uv run pytest tests\test_artifact_config.py tests\test_dataset_integrity.py tests\test_collect_gh.py
• python -m uv run pre-commit run --all-files
• python -m uv lock --check
• Dataset Validation and Verification — all five dataset entries passed end to end

[Copilot]

Note

Copilot was unable to run its full agentic suite in this review.

Copilot review overview

Review effort: Lite
Findings: None

What changed in this PR

This PR updates how Business Central artifact URLs are resolved so that insider versions use the insider artifact feed, and expands repo sparse-checkout paths to include internal apps while adding new dataset problem statements and bumping the package version.

Changes:

• Pass resolved BC version into Get-BCBenchArtifactConfig and add version-based insider-feed selection logic.
• Improve robustness by throwing when Get-BCArtifactUrl fails to resolve an URL.
• Add new dataset problem statements and bump project version to 0.14.0.

File	Description
tests/​test_artifact_config.py	Adds regression tests validating artifact-config resolution and workflow wiring.
scripts/​Setup-ContainerAndRepository.ps1	Passes version into artifact-config and errors clearly when URL resolution fails.
scripts/​Download-BCSymbols.ps1	Aligns symbol download artifact selection with category/version config.
scripts/​BCBenchUtils.psm1	Extends sparse-checkout paths and adds -Version support + logic to Get-BCBenchArtifactConfig.
pyproject.toml	Bumps package version to 0.14.0.
.github/​actions/​setup-bc-container-repo/​action.yml	Passes resolved version into artifact-config inside the action.
dataset/​problemstatement/​**/​README.md	Adds new problem statements for additional dataset instances.
dataset/​bcbench.jsonl	Appends new dataset instances and patches.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

BC 30 setup currently fails, and 20 additional unvalidated entries contradict the documented five-entry scope.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• BC 30.0 entries fail setup due to missing pinned artifact URL · New
• Remove unvalidated extra candidate rows or rerun validation for all additions · New

[Sun Haoran (haoranpb)]

There are more than 5 tasks added, did we make a mistake?

Maybe consider splitting one task per PR, so it's easier to review and merge

---

Most importantly, have you manually reviewed each task? We have to make sure they each contain enough context for them to be resolved, for example, not in the attachment that are not captured here

[alexei-dobriansky]

Addressed the review feedback by replacing this monolithic PR with a five-PR stack:

1. Add NAV 246904 mileage rounding dataset task #899 — NAV 246904 plus shared artifact/sparse-checkout infrastructure
2. Add BCApps 7070 subscription customer dataset task #900 — BCApps 7070
3. Add BCApps 10345 service contract dataset task #901 — BCApps 10345
4. Add BCApps 10808 exchange rate dataset task #902 — BCApps 10808
5. Add BCApps 11274 drop shipment dataset task #903 — BCApps 11274 plus the 0.14.0 version bump

Changes made in the replacement stack:

• one dataset task per PR;
• only the five entries that passed end-to-end validation are included;
• BC 29 and BC 30 use pinned validated artifact URLs;
• every problem statement was manually rewritten from a validated ADO [AI-REPRO] comment;
• screenshot-only values were transcribed into concise text and all image dependencies removed;
• implementation hints and unrelated troubleshooting details were removed;
• NAV 242051 and the other non-reproducible candidates are not included.

ADO [AI-REPRO] comments:

• 624594: https://dev.azure.com/dynamicssmb2/_apis/wit/workItems/624594/comments/10039826
• 647165: https://dev.azure.com/dynamicssmb2/_apis/wit/workItems/647165/comments/10039829
• 648313: https://dev.azure.com/dynamicssmb2/_apis/wit/workItems/648313/comments/10039827
• 649618: https://dev.azure.com/dynamicssmb2/_apis/wit/workItems/649618/comments/10039828
• 634224: https://dev.azure.com/dynamicssmb2/_apis/wit/workItems/634224/comments/10039825

Cumulative validation is running on the top stack branch: https://github.com/microsoft/BC-Bench/actions/runs/36153236382

Closing this PR as superseded by the stack above.