Add structured telemetry logging and observability support for ingest pipelines

[khanahmedm]

This PR introduces structured telemetry and observability capabilities to the Data Lakehouse Ingest framework.

The changes add JSON-formatted logging, pipeline execution context enrichment, telemetry lifecycle management, and supporting unit test updates. These enhancements provide a foundation for centralized monitoring, troubleshooting, operational analytics, and future integration with VictoriaLogs.

Key Changes

Telemetry and Logging

• Added structured JSON logging support.
• Added pipeline execution context enrichment to log records.
• Added pipeline run identifier support.
• Added telemetry upload and finalization workflow.
  • Temporarily disable telemetry uploads
• Improved logger initialization and configuration handling.
• Enhanced log metadata consistency across ingest workflows.

Updated unit tests for:

• config_loader
• core
• logger
• init_utils
• io_utils
• table_processor

[codecov]

Codecov Report

❌ Patch coverage is 99.03846% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.82%. Comparing base (b0a7e0a) to head (94d97b1).

Files with missing lines	Patch %	Lines
src/data_lakehouse_ingest/logger.py	98.51%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #74      +/-   ##
===========================================
+ Coverage    97.71%   98.82%   +1.10%     
===========================================
  Files           18       18              
  Lines          962     1103     +141     
===========================================
+ Hits           940     1090     +150     
+ Misses          22       13       -9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.