You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[experiment] Add MSBuild binary log detector as replacement for DotNet and NuGet detectors (#1710)
* Create BinLogDetector
* Implement dev dependency overrides
* Add more tests, add selfcontained detection
* Fix environment variable detection
* Refactor dotnet/nuget/binlog detectors to share as much as possible
* Fix testing of mocks against internal interfaces
* Fix a couple failing tests
* Address feedback
* Reduce state to just assets to project info map
* Rebase binlog paths
* Add tests for PathRebasingUtility
* Don't run Integration tests in local build
* Try to fix linux tests
* Add some test diagnostics
* More logging
* More logging
* Save log file
* Add exception handling to binlog reading
* Workaround binlogger issue with older binlogs
Add diagnostics as well.
* Address feedback
* Address feedback
* Address feedback
* Support cancellation of binlog processing
* Make default off and document
* Don't catch OperationCanceledException
* Update docs, refine tests
* Fix merge conflict
* Address PR feedback: fix edge cases and improve robustness
- Make integration test filter conditional via ExcludeIntegrationTests property
- Update CI and CONTRIBUTING.md to use the new property
- Guard GetRebaseRoot against '.' and '..' relative paths
- Skip NuGet registration when library version is null instead of using 0.0.0
- Lazy-create MSBuildProjectInfo in ProjectStarted handler
- Treat empty strings as unset in MergeWith for path/identity properties
- Fix fallback recorder key to use assets file location consistently
- Fix test to find NuGet graph by component ID
---------
Co-authored-by: Greg Villicana <58237075+grvillic@users.noreply.github.com>
Co-authored-by: Greg Villicana <grvillic@microsoft.com>
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+16-2Lines changed: 16 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -61,6 +61,20 @@ Analysis rulesets are defined in [analyzers.ruleset](analyzers.ruleset) and vali
61
61
62
62
### Testing
63
63
64
-
L0s are defined in `Microsoft.ComponentDetection.*.Tests`.
64
+
**Unit tests (L0s)** are defined in `Microsoft.ComponentDetection.*.Tests` and should be fast, isolated, and free of external process calls.
65
65
66
-
Verification tests are run on the sample projects defined in [microsoft/componentdetection-verification](https://github.com/microsoft/componentdetection-verification).
66
+
**Integration tests** spawn real processes (e.g. `dotnet build`, `dotnet restore`) and are therefore slower and environment-dependent. Tag these with `[TestCategory("Integration")]` so they can be filtered during local development.
67
+
68
+
By default, `dotnet test`**excludes** Integration tests (configured in `test/Directory.Build.props`). CI runs all tests including Integration (see `.github/workflows/build.yml`).
69
+
70
+
To run only Integration tests locally:
71
+
```bash
72
+
dotnet test -- --filter "TestCategory=Integration"
73
+
```
74
+
75
+
To run everything (same as CI):
76
+
```bash
77
+
dotnet test -p:ExcludeIntegrationTests=false
78
+
```
79
+
80
+
**Verification tests** are run on the sample projects defined in [microsoft/componentdetection-verification](https://github.com/microsoft/componentdetection-verification).
The `MSBuildBinaryLog` detector is a **DefaultOff** detector intended to eventually replace both the `NuGetProjectCentric` and `DotNet` detectors. It combines MSBuild binary log (binlog) information with `project.assets.json` to provide enhanced component detection with project-level classifications.
42
+
43
+
It looks for `project.assets.json` files and separately discovers `*.binlog` files. The binlog provides build-time context that isn't available from `project.assets.json` alone.
44
+
45
+
### MSBuild Properties
46
+
47
+
The detector extracts the following MSBuild properties from binlog data:
48
+
49
+
| Property | Usage |
50
+
| --- | --- |
51
+
|`NETCoreSdkVersion`| Registered as the SDK version for the DotNet component. More accurate than `dotnet --version`, which can differ due to `global.json` rollforward. |
52
+
|`OutputType`| Classifies projects as "application" (`Exe`, `WinExe`, `AppContainerExe`) or "library" (`Library`, `Module`). The `DotNet` detector uses PE header inspection, which requires compiled output. |
53
+
|`ProjectAssetsFile`| Maps binlog project info to the corresponding `project.assets.json` on disk. |
54
+
|`TargetFramework` / `TargetFrameworks`| Identifies inner builds for multi-targeted projects and determines per-TFM properties. |
55
+
|`IsTestProject`| When `true`, all dependencies of the project are marked as development dependencies. |
56
+
|`IsShipping`| When `false`, all dependencies are marked as development dependencies. |
57
+
|`IsDevelopment`| When `true`, all dependencies are marked as development dependencies. |
58
+
|`IsPackable`| Indicates whether the project produces a NuGet package. |
59
+
|`SelfContained`| Detects self-contained deployment. Combined with lock file heuristics. |
60
+
|`PublishAot`| When `true`, project is treated as self-contained. Typically only set during publish pass. |
61
+
62
+
### PackageReference and PackageDownload Metadata
63
+
64
+
The detector reads `IsDevelopmentDependency` metadata from `PackageReference` and `PackageDownload` items in the binlog:
65
+
66
+
-**PackageReference**: When `IsDevelopmentDependency` is `true`, the package and **all of its transitive dependencies** are marked as development dependencies. This allows annotating a single top-level dependency to classify its entire closure as dev-only. If a transitive dependency should remain non-dev, reference it directly (or transitively via a non-overridden dependency) so that it is also registered through a non-dev path.
67
+
-**PackageReference**: When `IsDevelopmentDependency` is `false`, the package and all of its transitive dependencies are classified using the explicit override rather than the default heuristics.
68
+
-**PackageDownload**: Packages are registered as development dependencies by default unless explicitly overridden via `IsDevelopmentDependency` metadata.
69
+
70
+
### Multi-Targeting and Multi-Pass Merging
71
+
72
+
For multi-targeted projects, the detector understands the MSBuild outer/inner build structure. Properties from inner builds (per-TFM) are tracked separately, and when a project appears in multiple binlogs (e.g., a build pass and a publish pass), their properties are merged so that values like `PublishAot` (typically only set during publish) are available when processing the shared `project.assets.json`.
73
+
74
+
### Fallback Mode
75
+
76
+
When no binlog is available for a project, the detector falls back to standard NuGet detection behavior (equivalent to the `NuGetProjectCentric` detector).
77
+
78
+
### Enabling the Detector
79
+
80
+
Pass `--DetectorArgs MSBuildBinaryLog=EnableIfDefaultOff` and ensure a `*.binlog` file is present in the scan directory (e.g., by building with `dotnet build -bl`).
81
+
39
82
## Known Limitations
40
83
41
84
- Any components that are only found in `*.nuspec` or `*.nupkg` files will not be detected with the latest NuGet Detector approach, because the NuGet detector that scans `*.nuspec` or `*.nupkg` files overreports. This is due to of NuGet's [restore behaviour][8] which downloads all possible dependencies before [resolving the final dependency graph][9].
Copy file name to clipboardExpand all lines: docs/feature-overview.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@
11
11
| NPM | <ul><li>package.json</li><li>package-lock.json</li><li>npm-shrinkwrap.json</li><li>lerna.json</li></ul> | - | ✔ (dev-dependencies in package.json, dev flag in package-lock.json) | ✔ |
0 commit comments