[Task] Add Dependency Ordering, Plugin Disable, Discovery Caching and Logging

[rian-be]

Summary

Resolve inter plugin dependencies by topological ordering, honor configuration/metadata disable flag so the compatibility gate skips plugins, cache discovery results for faster startup, and log every plugin outcome observably.

Goal

Guarantee deterministic, observable loading: dependencies load first, operators can disable plugins without deleting files, startup avoids full rediscovery when nothing changed, and every plugin's outcome is visible in logs.

Background

Plugins may require another plugin to be present/loaded first an undefined order causes missing dependency errors or non deterministic startup. There is no supported way to disable plugin at deploy time. Rediscovering and revalidating every plugin on each start is slow, especially with AssemblyLoadContext isolation and validator runs. Operators cannot see which plugins loaded, in what order, or why one was skipped. G7–G10 are applied by the host pipeline around the loader using the manifest and the LoadedPlugin runtime model the loader itself only loads what it is given in sorted order.

Scope

G7. Resolving DependsOn

The host pipeline builds dependency graph from DependsOn and topologically sorts it before invoking the loader.

G8. Disable via compatibility gate

The compatibility gate checks the effective disable flag (EffectiveIsEnabled) and skips disabled plugins this runs before ordering and before the loader, not inside it.

G9. Cache discovered plugins

A host owned IPluginDiscoveryCache sits before discovery and reuses cached DiscoveredPlugin[] on hit.

G10. Log discovered plugins

The host pipeline logs each outcome at two moments: discovery and final outcome (skipped/rejected/loaded/validated/activated) with reason.

Architecture

flowchart TD
    Cache["Host Pipeline<br/>IPluginDiscoveryCache G9"] --> Check{"Cache valid?"}
    Check -- yes --> Reuse["HIT → DiscoveredPlugin[]<br/>no discoverer call"]
    Check -- no --> Disc["IPluginDiscoverer G1<br/>source → DiscoveredPlugin[]"]
    Disc --> Store["StoreAsync<br/>PluginManifest + Location<br/>+ SourceFingerprint<br/>+ SchemaVersion"]
    Store --> Merge
    Reuse --> Merge["DiscoveredPlugin[]"]
    Merge --> PreVal["Pre-load validation G2<br/>structural + duplicate Id"]
    PreVal -- "invalid graph → startup error" --> Err0["startup failure<br/>missing/self/duplicate<br/>invalid Id / cycle"]
    PreVal -- ok --> Gate{"Compatibility Gate G3/G8<br/>EffectiveIsEnabled?<br/>MinHostVersion?"}
    Gate -- "disabled → SKIP" --> LogSkip["LOG Warning skipped"]
    Gate -- "incompatible → REJECT" --> LogRej["LOG Error rejected"]
    Gate -- accepted --> Graph["Dependency Graph G7<br/>validate DependsOn Id"]
    Graph -- "invalid graph → startup error" --> Err1["startup failure"]
    Graph -- valid --> Sort["Topo sort Kahn<br/>Priority asc → RegistrationOrder<br/>only among ready set"]
    Sort -- "dependency unavailable<br/>disabled/incompatible<br/>validation/load failed<br/>→ dependent REJECTED" --> Err2["REJECT<br/>dependency-unavailable"]
    Sort -- ok --> Loader["IPluginLoader G1/G4<br/>receives sorted accepted list<br/>knows nothing about cache"]
    Loader --> Loaded["LoadedPlugin[]"]
    Loaded --> Val["Validator G5/G6<br/>+ manifest consistency"]
    Val -- fail --> LogInv["LOG Error invalid"]
    Val -- pass --> Act["ACTIVATION"]
    Act --> LogOk["LOG Information loaded"]
    classDef gate fill:#fef3c7,stroke:#f59e0b,color:#92400e
    classDef err fill:#fee2e2,stroke:#ef4444,color:#991b1b
    classDef ok fill:#dcfce7,stroke:#22c55e,color:#14532d
    classDef cache fill:#dbeafe,stroke:#3b82f6,color:#1e40af
    class Cache,Check,Store,Reuse cache
    class Gate,Graph,Sort,PreVal gate
    class LogSkip,LogRej,Err0,Err1,Err2,LogInv err
    class Act,LogOk ok

Loading

Component	Responsibility
IPluginDiscoverer	find + read manifests
IPluginDiscoveryCache	cache discovery results (host owned, before discoverer)
Host pipeline	gate + graph + ordering
IPluginLoader	load + construct (sorted list, no cache knowledge)
Validator	validate loaded plugin
Activation	start plugin

flowchart TD
    V0{"Graph validation"}
    V0 -- "invalid → startup failure<br/>missing/self/duplicate<br/>invalid Id / cycle" --> Fail["startup error"]
    V0 -- valid --> V1{"Dependency resolution"}
    V1 -- "unavailable → dependent REJECTED<br/>disabled/incompatible<br/>validation/load failed" --> Rej["REJECT<br/>dependency-unavailable"]
    V1 -- available --> Ok["load"]
    classDef gate fill:#fef3c7,stroke:#f59e0b,color:#92400e
    classDef err fill:#fee2e2,stroke:#ef4444,color:#991b1b
    classDef ok fill:#dcfce7,stroke:#22c55e,color:#14532d
    class V0,V1 gate
    class Fail,Rej err
    class Ok ok

Loading

flowchart TD
    Ex["Example Priority tie-break<br/>A p-100 → B p-200 DependsOn A<br/>C p0"]
    Ex --> S1["ready {A, C}<br/>lower Priority first<br/>pick A -100 before C 0"]
    S1 --> S2["A placed → B ready<br/>ready {C, B}<br/>pick B -200 before C 0"]
    S2 --> Ord["order: A, B, C<br/>deps before dependents<br/>never full re-sort"]
    Alt["Alt values A p100 C p0<br/>would give C, A, B<br/>not A, C, B"] -. note .-> Ex

Loading

flowchart TD
    Eff["EffectiveIsEnabled<br/>EffectiveIsEnabled manifest config"]
    Eff --> R1["manifest true + absent → true"]
    Eff --> R2["manifest true + true → true"]
    Eff --> R3["manifest true + false → false"]
    Eff --> R4["manifest false + * → false"]
    R4 --> Rule["host can disable true→false<br/>never enable false→true<br/>does not mutate PluginManifest"]
    Rule --> Gate2{"Compatibility Gate<br/>checks EffectiveIsEnabled"}
    Rule -. consistency .-> Cons["manifest.IsEnabled == plugin.IsEnabled<br/>effective only host-side"]
    classDef gate fill:#fef3c7,stroke:#f59e0b,color:#92400e
    class Gate2 gate

Loading

flowchart TD
    Entry["PluginDiscoveryCacheEntry<br/>internal to cache impl"]
    Entry --> F1["Location"]
    Entry --> F2["SourceFingerprint<br/>assembly path/hash"]
    Entry --> F3["SchemaVersion"]
    Entry --> F4["PluginManifest"]
    Entry -. not stored .-> No["never: ALC / Type<br/>IAuthKitPlugin / LoadedPlugin"]
    classDef cache fill:#dbeafe,stroke:#3b82f6,color:#1e40af
    classDef err fill:#fee2e2,stroke:#ef4444,color:#991b1b
    class Entry,F1,F2,F3,F4 cache
    class No err

Loading

flowchart TD
    D1["DISCOVERY<br/>Information<br/>discovered id version"] --> D2["GATE<br/>Warning skipped / Error rejected<br/>reason: disabled / minHostVersion"]
    D2 --> D3["LOAD<br/>Information loaded<br/>or Error load failed"]
    D3 --> D4["VALIDATION<br/>Error invalid<br/>reason: mismatch / contract"]
    D4 --> D5["ACTIVATED<br/>Information"]

Loading

Proposed Contract

• G7: the host pipeline (not the loader) builds dependency graph from DependsOn and topologically sorts it. A cycle is startup error. A plugin whose required dependency is skipped, rejected, invalid, or otherwise unavailable is itself rejected with dependency unavailableerror. The resulting order feeds isolation and activation.IPluginLoader` receives the already sorted, already accepted list and only loads/constructs.
• G8: the compatibility gate checks EffectiveIsEnabled(manifest, config) (host side function, does not mutate PluginManifest truth table: true/absent -> true, true/true -> true, true/false -> false, false/*->false) before ordering and before the loader runs. A disabled plugin is not ordered, not isolated, not validated for activation, and not loaded. Consistency check remains manifest.IsEnabled == plugin.IsEnabled (declared), not effective. Composes with IsMiddlewareEnabled (C).
• G9: a host owned IPluginDiscoveryCache sits before IPluginDiscoverer (host pipeline responsibility, not loader):

public interface IPluginDiscoveryCache
{
    Task<IReadOnlyList<DiscoveredPlugin>?> TryGetAsync(CancellationToken ct = default);
    Task StoreAsync(IReadOnlyList<DiscoveredPlugin> plugins, CancellationToken ct = default);
}
// internal impl may use PluginDiscoveryCacheEntry { Location, SourceFingerprint, SchemaVersion, PluginManifest }
// fingerprinting is impl internal; public contract stays IReadOnlyList<DiscoveredPlugin>

Host pipeline: TryGetAsync -> HIT reuses DiscoveredPlugin[] without calling discoverer MISS calls IPluginDiscoverer then StoreAsync. It stores only discovery results and source fingerprints. It never stores LoadedPlugin, IAuthKitPlugin, Type, AssemblyLoadContext, or process bound state. Loader receives IReadOnlyList<DiscoveredPlugin> and knows nothing about cache.

• G10: the host pipeline logs each plugin at two moments: (1) discovery (Information: discovered id version) and (2) final outcome (Information: loaded, Warning: skipped reason=disabled, Error: rejected reason=min-host-version|mismatch|dependency-unavailable|validation). Status reasons from G3/G5/G7/G8 are included.

Requirements

G7. Ordering

• A dependency cycle is startup error.
• Dependencies are loaded before the plugins that declare them.
• Load order is deterministic for given dependency set.
• A plugin whose required dependency is skipped, rejected, invalid, or otherwise unavailable is rejected with dependency unavailable error (covers disabled, incompatible, invalid manifest, failed load/validation).
• Algorithm: perform topological sort at each step, among the set of currently dependency ready plugins, pick the next by Priority ascending (A7, lower = earlier), then stable RegistrationOrder ascending. Do not sort the entire topological result by Priority afterward that would break dependency edges.
  • Example: A (p-100) → B (p-200, DependsOn A), and C (p0). Initially ready = {A, C}; Priority picks A (-100) then B (-200) becomes ready → ready {B, C} picks B (-200) then C → A, B, C.
  • With values A(p100), C(p0) the correct ascending result would be C, A, B not A, C, B.
• DependsOn entries are plugin Id values (never display names), resolved from discovered plugins.
• Dependency validation (reject on violation): each entry must be a valid id, must exist among discovered plugins (missing → startup error), must not be a self-dependency, and must not be duplicated within DependsOn; a cycle is a startup error.
• No version-range resolution between dependent plugins.

G8. Disable

• A disabled plugin is not ordered, not isolated, not validated for activation, and not loaded.
• Effective value is EffectiveIsEnabled = manifest.IsEnabled && hostConfigEnabled != false host may disable but never reenable manifest disabled plugin.
• Disable state composes with the IsEnabled flag (A8) and IsMiddlewareEnabled (C).
• Status is reported in logs with reason. Manifest <-> instance IsEnabled consistency check compares declared values effective override is host side only.

G9. Cache

• Cache sits before discovery (TryGetAsync -> hit reuses DiscoveredPlugin[], miss runs IPluginDiscoverer).
• On cache hit, startup is faster than full rediscovery.
• Cache invalidation occurs when plugin inputs change (assembly paths/hashes), so stale entries are not used.
• The cache is host local no distributed cache.
• Cached data is discovery metadata only (PluginManifest + Location + fingerprints), never LoadedPlugin/IAuthKitPlugin/Type/AssemblyLoadContext.
• Interface IPluginDiscoveryCache as above (or host internal equivalent).

G10. Logging

• Each discovered plugin is logged at discovery (Information: discovered).
• Final outcome is logged separately: Information for loaded, Warning for skipped/disabled, Error for rejected/invalid with reason.
• No structured telemetry or metrics here (that belongs to Section H observability).

Backward Compatibility

Plugins without DependsOn keep their existing order. Plugins that are not disabled are unaffected. Caching is an optimization of the existing discovery path a cache miss falls back to full discovery. Logging is additive and does not change loading behavior.

Validation

G7. Ordering

• No dependency cycles are accepted (startup error on cycle).
• Dependencies are loaded before the plugins that declare them.
• Load order is deterministic for given dependency set.
• Algorithm: topological sort among dependency ready plugins pick by Priority ascending then RegistrationOrder ascending (not full resort of the result). Example A(p-100->B(p-200,DependsOn A), C(p0) ⇒ A, B, C.
• DependsOn entries are plugin Id values missing/self/duplicate/invalid-id dependencies and cycles are rejected (startup error).
• A plugin whose dependency is skipped/rejected/invalid is rejected with dependency unavailable.

G8. Disable

• A disabled plugin is skipped during loading and is not activated.
• EffectiveIsEnabled precedence: host may disable true -> false, never enable false -> true.
• A disabled plugin is reported in logs with reason.

G9. Cache

• Cache checked before discovery hit avoids full rediscovery.
• Cache stores only discovery results + fingerprints, never ALC/runtime objects.
• Cache invalidation occurs when plugin inputs change.
• A cache miss triggers full discovery.

G10. Logging

• Each discovered plugin appears in logs at discovery.
• Final outcome logged separately with name, version, status and reason.
• DevTokens appears in logs after the contract change.

Acceptance Criteria

• No dependency cycles are accepted (startup error on cycle).
• Dependencies are loaded before the plugins that declare them.
• Load order is deterministic for given dependency set.
• Ordering algorithm: topological sort among dependency ready plugins by Priority ascending then stable RegistrationOrder ascending (never full resort of the topological result).
• DependsOn entries are plugin Id values missing/self/duplicate/invalid-id dependencies and cycles are rejected.
• A plugin whose required dependency is skipped/rejected/invalid is rejected with dependency unavailable.
• A disabled plugin is skipped during loading and is not activated (not ordered, not isolated).
• EffectiveIsEnabled: host may disable but never reenable.
• On cache hit, startup is faster than full rediscovery cache never stores ALC/runtime objects.
• Cache invalidation occurs when plugin inputs change (so stale entries are not used).
• Each discovered plugin appears in logs at discovery; final outcome logged separately with reason.

Non Goals

• G7: no semantic version range resolution between dependent plugins (only MinHostVersion from G3 applies). No runtime dependency injection across plugins.
• G8: no runtime enable/disable toggle disable state is static at startup. No partial disable of individual plugin features beyond the existing flags.
• G9: no distributed cache caching is host local. No caching of activated plugin instances across process restarts.
• G10: no structured telemetry or metrics here (that belongs to Section H observability). No change to log levels beyond Information for normal loads and Warning/Error for skips/failures.

[rian-be]

Challenges & Risks G7–G10

Problem: #29 combines three mechanisms with different orderings: DependsOn (graph), Priority (tie breaker in ready set, Kahn) and discovery cache. Without strict separation:

• cycle A->B->C->A missed before ALC -> TypeLoadException at runtime instead of startup failure,
• EffectiveIsEnabled=false propagating implicitly -> dependency unavailable without clear log,
• cache storing ALC/Type -> context leak across restarts.

Impact: Nondeterministic startup, hard to reproduce in integration tests.

What needs attention:

1. Graph = host pipeline, not loader. Loader receives already sorted IReadOnlyList<DiscoveredPlugin> knows nothing about cache or priorities.
2. Two level validation: missing/self/duplicate/cycle/invalidId -> startup failure (hard) disabled/incompatible/validation-failed -> dependent REJECT with reason.
3. Cache fingerprint only: Manifest + Location + SourceFingerprint + SchemaVersion, never LoadedPlugin/ALC.
4. G10 two moment log: discovered (Information) + outcome (Warning/Error with reason: G3/G7/G8/G5).
5. Test: topo A(p-100) -> B(p-200), C(p0) ⇒ A,B,C + cycle + disabled dependency in PluginContractValidator.