Rebind captured component, conditional, and tagged loggers on initLogSettings(true)

[kriszyp]

⊙ Problem

When Harper starts without an existing install, harper_logger falls back to stdout before config exists ([level] [init]) while modules finish loading. After installer.ts writes harperdb-config.yaml and calls initLogSettings(true), mainLogger was replaced with a new HarperLogger instance. References from forComponent(...), loggerWithTag(...), and utility/logging/logger.ts captured at import time stayed bound to the dead fallback logger.

🤔 Is rebinding captured component and tagged loggers across initLogSettings(true) warranted?

Yes, as specified. Modules imported before first-boot install completes (mqtt, security/user, certificateVerification, derivedIndexes) cache their logger handles at module scope. Replacing mainLogger without updating those handles strands their logs on stdout for the life of the process.

💡 Solution

• Carry the existing mainLogger.components map and mainLogger.conditional proxy into the replacement mainLogger in initLogSettings and reconfigure every tracked component logger in place via updateLogger.
• Route loggerWithTag and HarperLogger.prototype.withTag plus utility/logging/logger.ts through live level/method lookups and live logger.logToStdstreams checks so captured handles follow both initLogSettings(true) and runtime applyLogSettings reloads.
• Normalize string logging: <level> shorthand to { level } via normalizeLogOptions and isolate per-component updateLogger errors in initLogSettings so a bad component path falls back to main's settings without taking file logging down.

🤔 Should conditional tagged loggers return null when the log level is raised at runtime?

Yes. logger.withTag(tag) and loggerWithTag(tag, true) check the level through getters on each access. They return null when a level is disabled, matching logger.conditional. Callers use optional chaining (?.). Keeping methods callable and filtering inside the function would still evaluate call arguments when disabled.

🤔 Should string logging: <level> values normalize to { level }?

Yes. normalizeLogOptions converts string values (logging: debug) to { level: 'debug' } for root, logging.external, and component sections. It ignores non-string scalars. Boot validation still requires an object at root. Normalizing string shorthand in component sections and hot reload prevents silent no-ops, and is easy to tighten if needed.

⚖️ Alternatives

• Mutate the existing mainLogger in place inside initLogSettings(true) instead of constructing a new mainLogger and handing off components/conditional — viable. But handing components and conditional into createLogger keeps mainLogger construction uniform and still reconfigures all captured component and conditional handles in place. Tagged loggers need getter dispatch either way.
• Store every withTag object in a registry and mutate its properties on config change — rejected because callers invoke withTag per request or per event, which would retain those closures for the life of the process.

🔧 Changes

• utility/logging/harper_logger.ts: ⚠️ Look hardest: initLogSettings carries mainLogger.components and mainLogger.conditional into createLogger and reconfigures external plus tracked component loggers in place with try/catch recovery (and resets tracked components on fallback); updateLogger clears logger.path and inherits logToStdstreams; normalizeLogOptions normalizes string logging blocks in applyLogSettings and initLogSettings; loggerWithTag and withTag dispatch dynamically; and createLogger reads live logger.logToStdstreams, rebinds conditional, and reconciles path-cleared rotation.
• utility/logging/logger.ts: re-exports harperLogger.conditional directly instead of snapshotting fallback methods at module load.
• server/mqtt.ts, resources/derivedIndexes.ts, resources/derivedIndexRuntime.ts, resources/indexes/hnswDerivedIndex.ts: add optional chaining (?.) on conditional mqttLogger.withTag(...) in server/mqtt.ts, derivedIndexLogger.error in resources/derivedIndexes.ts, and logger.error in resources/derivedIndexRuntime.ts and resources/indexes/hnswDerivedIndex.ts so disabled levels are skipped after runtime level raises.
• unitTests/logConfigFixture.js: removes the refreshExternalLogger() workaround now that initLogSettings(true) updates externalLogger in place.
• unitTests/utility/logging/fixtures/firstBootComponentLogging.cjs: adds the subprocess fixture exercising pre-install forComponent capture followed by initLogSettings(true).
• unitTests/utility/logging/harper_logger.test.js: adds first-boot rebind and subprocess tests plus scalar logging coverage in applyLogSettings.

✅ Verification

• Ran npx mocha unitTests/utility/logging/harper_logger.test.js (149 passing, fails on base where captured component/tagged loggers in harper_logger.test.js and firstBootComponentLogging.cjs stay on [init] stdout).
• Ran npx mocha unitTests/server/mqtt.test.js (16 passing) and npm run lint && npm run build.

_{🤖 Generated by Google Gemini; posted via @kriszyp.}

Release target: main — fixes first-boot logger rebinding across initLogSettings(true) with no backport milestone specified
Related PRs: #372 overlaps (touches mainLogger.components and tagged logger methods in harper_logger.ts), #3173 overlaps (tests updateLogger rotation inheritance in harper_logger.test.js)
Complexity: medium

Deferred findings

• 88a15e8c031d36a5 nit reliability — utility/logging/harper_logger.ts — applyLogSettings per-component loop not isolated; a non-string component path/root still throws from the async change listener and skips lat — NOT FILED
• ee81f05a12384ecc nit correctness — utility/logging/harper_logger.ts — conditional tagged loggers return null after a runtime level raise; unguarded logger.withTag() calls now throw — NOT FILED
• 0842d100046ffe2f nit performance — utility/logging/harper_logger.ts — tagged conditional loggers run a level-check getter on every method access — NOT FILED

Origin — the dispatch brief this PR was written from

Component loggers captured at module load do not survive the install-time logger re-init, so in a process that installs AND runs Harper (first-boot harper run on a fresh root, e.g. a container entrypoint: bin/run.ts ~101-104 calls install() inside run) their messages reach no sink. Example: security/keys.ts:25 const logger = forComponent('tls').conditional runs at import under the no-config fallback logger; installer.ts ~211 initLogSettings(true) builds a NEW mainLogger with a new components Map (utility/logging/harper_logger.ts ~561, components map ~821-841) and nothing rebinds already handed-out component loggers. Result: Reloading certificate, Waiting for matching TLS certificate and private key, Error loading private key and similar lines vanish on first boot, and appear after one restart.

Acceptance

Test: in one process, capture a component logger before the configured initLogSettings(true), then log through it after; the line lands in the configured log file. Integration or unit coverage that a first-boot install+run process logs a keys.ts tls message. Shown failing on origin/main. No change to log formats.

_{Dispatch: task harper-logger-captured-component-rebind · queued by finding-triage/wave-35 · ran by gemini/gemini-4-argon-high/medium · worker kzyp-xps-1}

_{Review-Coverage: authored=gemini; ran=claude; adjudicated=domain; declined=cursor-grok,cursor-composer,cursor-kimi,cursor-muse,muse; rounds=4; full=2 @ 0b037b7}

_{Review-Attention: study ~20m (critical: derivedIndexRuntime.ts, derivedIndexes.ts +1; hot: mqtt.ts; decisions: tagged-conditional-null-on-raise, scalar-logging-shorthand, component-error-recovery, live-stdstreams, do-less-alternative) @ 0b037b7}

[gemini-code-assist]

Code Review

This pull request refactors the logging system to support dynamic rebinding of component and tagged loggers, introduces a conditional property on loggers to dynamically resolve log levels via getters, and adds optional chaining to logger calls across various modules. Feedback on these changes identifies three key issues: first, reassigning externalLogger directly breaks CommonJS imports, which can be resolved using a Proxy; second, updating the main logger in stdout mode prints an unnecessary 'No path for logger' error to the console; and third, normalizeLogOptions fails to handle boolean values (like logging: false), causing disabled logging configurations to fall back to enabled states.

[gemini-code-assist]

Reassigning externalLogger directly breaks references for any modules that imported externalLogger at load time, because CommonJS module.exports snapshots the reference to the initial wrapper object. Any subsequent property accesses (like path, level, etc.) or updates on the imported externalLogger will be lost or fail silently because they are performed on the stale wrapper object rather than the newly assigned component logger.

To solve this robustly, consider defining externalLogger as a const Proxy that dynamically forwards all operations (get, set, etc.) to an internal activeExternalLogger variable. Then, instead of reassigning externalLogger, you can simply update activeExternalLogger to point to the new component logger.

[kriszyp]

mainLogger inherits previousMainLogger?.components (line 583 / 644), which already contains the 'external' entry created at module load. mainLogger.forComponent('external') returns that exact existing HarperLogger instance (externalLogger === mainLogger.forComponent('external')), and the loop at lines 589–600 / 649–651 reconfigures it in place via updateLogger. Callers that imported harperLogger.externalLogger at load time continue to reference the same object.

[gemini-code-assist]

When applyLogSettings is called to disable file logging or when running in stdout/no-file mode, path is undefined. Since logger === mainLoggerRef (i.e., we are updating the main logger), the else if condition is skipped, and it prints No path for logger to console.error. This pollutes the console with error messages during normal operation.

Simplifying this to just clear the path when it is undefined avoids the noisy error.

	if (path) logger.path = path;
	else logger.path = undefined;

[kriszyp]

When mainLoggerRef has a path, path defaults to mainLoggerRef.path at line 140, so else logger.path = undefined would not clear mainLogger.path anyway and would remove the existing No path for logger diagnostic completely. The else if (!mainLoggerRef?.path && logger !== mainLoggerRef) branch specifically handles component loggers resetting back to the no-file fallback when mainLogger itself has no path.

[gemini-code-assist]

If a component or external logger is configured with logging: false (or external: false), normalizeLogOptions will return undefined because typeof false is 'boolean'. This causes the configuration to fall back to the main logOptions or {} (default options), effectively enabling logging instead of disabling it.

Consider explicitly handling boolean values (or false specifically) in normalizeLogOptions to prevent them from falling back to enabled states.

[kriszyp]

updateLogger expects an options object ({ level, path, root, stdStreams, rotation, tag }) and has no concept of boolean false disabling a logger — on main, a boolean logging: false threw TypeError: Cannot create property 'root' on boolean 'false' at logOptions.root = .... Returning undefined for non-string scalars falls back to inherited defaults instead of crashing.