Skip to content

Add offline provider metadata extraction and unify provider loading through a shared assembler#619

Open
jschick04 wants to merge 33 commits into
mainfrom
jschick/offline-provider-extraction
Open

Add offline provider metadata extraction and unify provider loading through a shared assembler#619
jschick04 wants to merge 33 commits into
mainfrom
jschick/offline-provider-extraction

Conversation

@jschick04

Copy link
Copy Markdown
Collaborator

Summary

Adds offline provider-metadata extraction and routes provider loading through one shared assembler so the online (native) and offline paths produce byte-identical content.

The new offline reader builds provider details purely from a provider DLL's WEVT_TEMPLATE and RT_MESSAGETABLE resources, with no EvtOpenPublisherMetadata call. Both the native path and the offline path feed the same ProviderDetailsFactory, so a provider resolved either way produces the same events, messages, value-maps, parameters, keywords/opcodes/tasks - and therefore the same content hash.

What's included

  • A shared ProviderDetailsFactory assembler; the native EventMessageProvider now delegates to it instead of carrying its own copy of the map-injection and value-map logic.
  • An OfflineWevtProviderReader that parses WEVT_TEMPLATE (events, templates, value-maps, channels, keywords/opcodes/tasks) and the legacy message tables, with bounds-checked, fail-closed resource reads.
  • Render-equivalent template hashing so identical providers collapse to one row and genuinely different versions coexist.
  • Correctness refinements that match native EvtFormatMessage output byte-for-byte: numbered-insert format-spec handling, classic-provider event-id qualification, apostrophe escaping in template attribute values, and parameter-reference resolution.
  • Integer-overflow hardening of the resource read-helper bounds checks.

Validation

  • Offline-vs-native content-hash parity rises from roughly 81% to roughly 98.25% across 972 comparable installed providers; the remaining 17 are an inherent floor (resource casing the native path drops, providers the offline path parses more completely, and one niche format).
  • Full unit and integration suites pass; the online resolver path is unchanged.

Scope

This lands the validated extraction engine and the shared-assembler consolidation. Two follow-ups are tracked separately:

  • Wiring the offline reader into the create-database UI and CLI as an opt-in mode (a hybrid that also preserves legacy, message-table-only providers).
  • Making the legacy-table population fully resource-only (no host registry) for foreign-image extraction.

Copilot AI review requested due to automatic review settings June 27, 2026 14:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an offline provider-metadata extraction path (reading WEVT_TEMPLATE + message tables directly) and consolidates both offline and native provider loading through a shared ProviderDetailsFactory, aiming for byte-identical provider content and matching content hashes.

Changes:

  • Introduces an offline WEVT reader/writer pipeline (WevtTemplateReader/WevtTemplateWriter, OfflineWevtProviderReader, message formatter + message-table session).
  • Unifies native/offline assembly via RawProviderContent + ProviderDetailsFactory, and aligns hashing/merging with render-equivalent template signatures (TemplateSignature).
  • Adds extensive unit + integration parity coverage for template parsing/writing, message formatting, and native-vs-offline equivalence.

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/Unit/EventLogExpert.Provider.Tests/Resolution/TemplateSignatureTests.cs Adds unit coverage for template signature equivalence behavior.
tests/Unit/EventLogExpert.Provider.Database.Tests/Maintenance/ProviderDetailsMergerTests.cs Ensures merge conflicts use render-equivalent template comparison rules.
tests/Unit/EventLogExpert.Provider.Database.Tests/Hashing/VersionKeyCalculatorTests.cs Adds hash/key tests covering template-signature changes and null/empty equivalence.
tests/Unit/EventLogExpert.Eventing.Tests/PublisherMetadata/Wevt/WevtTemplateReaderTests.cs Updates namespace/imports for new WEVT reader location.
tests/Unit/EventLogExpert.Eventing.Tests/PublisherMetadata/Wevt/WevtMessageFormatterTests.cs Adds unit tests for offline message formatting parity rules.
tests/Unit/EventLogExpert.Eventing.Tests/PublisherMetadata/Wevt/OfflineWevtProviderReaderTests.cs Adds comprehensive crafted-byte tests for offline WEVT parsing/writing and fail-closed behavior.
tests/Unit/EventLogExpert.Eventing.Tests/PublisherMetadata/ProviderDetailsFactoryTests.cs Adds tests for shared assembler behavior and map-attribute injection edge cases.
tests/Unit/EventLogExpert.Eventing.Tests/PublisherMetadata/EventMessageProviderTests.cs Removes tests for old map-injection location (now owned by factory).
tests/Shared/EventLogExpert.Eventing.TestUtils/Constants/Constants.Provider.cs Adds more provider names used by new parity tests.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/PublisherMetadata/Wevt/OfflineWevtProviderParityTests.cs Adds integration parity suite comparing offline vs native providers on host.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/PublisherMetadata/ProviderMetadataTests.cs Updates tests to validate raw-content extraction rather than cached properties.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/PublisherMetadata/EventMessageProviderIntegrationTests.cs Adds end-to-end validation that named values resolve via the new shared path.
src/EventLogExpert.Provider/Resolution/TemplateSignature.cs Introduces canonical byte encoding for render-relevant template fields.
src/EventLogExpert.Provider/Resolution/TemplateFieldReader.cs Adds allocation-free <data> scanner/canonicalizer for templates.
src/EventLogExpert.Provider/Resolution/TemplateField.cs Adds span-based representation of parsed-or-raw template nodes.
src/EventLogExpert.Provider/Resolution/ProviderContentMerge.cs Switches event equivalence to use TemplateSignature instead of raw string equality.
src/EventLogExpert.Provider.Database/Hashing/VersionKeyCalculator.cs Updates version key to use the renamed ProviderContentEncoder.
src/EventLogExpert.Provider.Database/Hashing/ProviderContentEncoder.cs Encodes templates via TemplateSignature for render-equivalent hashing stability.
src/EventLogExpert.Eventing/Resolvers/TemplateAnalyzer.cs Reuses shared TemplateFieldReader for consistent template scanning.
src/EventLogExpert.Eventing/PublisherMetadata/WevtTemplateReader.cs Removes old WEVT reader implementation (replaced by Wevt/* pipeline).
src/EventLogExpert.Eventing/PublisherMetadata/Wevt/WevtTypeNames.cs Adds mapping from WEVT inType/outType bytes to manifest strings (including default outType behavior).
src/EventLogExpert.Eventing/PublisherMetadata/Wevt/WevtTemplateWriter.cs Adds fail-closed writer producing canonical template XML from parsed descriptors/nodes.
src/EventLogExpert.Eventing/PublisherMetadata/Wevt/WevtTemplateReader.cs Adds full-table WEVT resource parser (events/channels/keywords/opcodes/tasks + maps/templates).
src/EventLogExpert.Eventing/PublisherMetadata/Wevt/WevtMessageFormatter.cs Adds offline formatting rules to match native EvtFormatMessage quirks.
src/EventLogExpert.Eventing/PublisherMetadata/Wevt/OfflineWevtProviderReader.cs Implements offline provider details build using WEVT + message tables and shared factory.
src/EventLogExpert.Eventing/PublisherMetadata/Wevt/MessageTableSession.cs Adds MUI-aware message-table session for offline message resolution.
src/EventLogExpert.Eventing/PublisherMetadata/RawProviderContent.cs Introduces raw intermediate model for unified provider assembly.
src/EventLogExpert.Eventing/PublisherMetadata/ProviderMetadata.cs Reworks native metadata reading to produce RawProviderContent (removes cached dictionaries and locks).
src/EventLogExpert.Eventing/PublisherMetadata/ProviderDetailsFactory.cs Adds shared assembler for events/named values/maps; includes map injection and trimming behavior.
src/EventLogExpert.Eventing/PublisherMetadata/LegacyMessageFileSource.cs Adds TryCreate and refactors message-table iteration to reduce duplication.
src/EventLogExpert.Eventing/PublisherMetadata/EventMetadata.cs Removes now-unused legacy wrapper type (replaced by raw content + factory).
src/EventLogExpert.Eventing/PublisherMetadata/EventMessageProvider.cs Delegates provider assembly to ProviderDetailsFactory and updates lazy-source creation.
src/EventLogExpert.DatabaseTools/CreateDatabase/CreateDatabaseOperation.cs Adds DEBUG-only tripwire for hash/merge drift on VersionKey collisions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/EventLogExpert.Provider/Resolution/TemplateFieldReader.cs Outdated
Comment thread src/EventLogExpert.Provider/Resolution/TemplateFieldReader.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 35 out of 35 changed files in this pull request and generated 2 comments.

Comment thread src/EventLogExpert.Provider/Resolution/TemplateFieldReader.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 35 out of 35 changed files in this pull request and generated 1 comment.

Comment thread src/EventLogExpert.Provider/Resolution/TemplateSignature.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 35 out of 35 changed files in this pull request and generated no new comments.

@jschick04 jschick04 marked this pull request as ready for review June 27, 2026 17:30
@jschick04 jschick04 requested a review from a team as a code owner June 27, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants