Skip to content

debezium/dbz#2128 feat: add Milvus adapter and integration tests#12

Open
KMohnishM wants to merge 4 commits into
gsoc-week-5-pgvectorfrom
gsoc-week-5-milvus
Open

debezium/dbz#2128 feat: add Milvus adapter and integration tests#12
KMohnishM wants to merge 4 commits into
gsoc-week-5-pgvectorfrom
gsoc-week-5-milvus

Conversation

@KMohnishM

Copy link
Copy Markdown
Contributor

Description

This Pull Request implements the second half of GSoC Week 5 targets under tracking issue debezium/dbz#2128. It introduces support for the Milvus vector database backend and provides integration tests leveraging milvus-lite.

Key Changes

  1. Milvus Adapter:
    • Implemented MilvusAdapter in pydebeziumai/adapters/milvus.py extending the core VectorStoreAdapter contract.
    • Utilizes langchain-milvus underneath to perform idempotent upsert, delete, and retriever generation.
    • Handled automatic registration of the PyMilvus connection pool handler via connections.connect dynamically to avoid ORM-layer initialization errors (ConnectionNotExistException).
    • Registered MilvusAdapter in the public adapters interface (pydebeziumai/adapters/__init__.py).
  2. Integration Tests:
    • Created a comprehensive integration test suite in tests/integration/test_milvus_adapter.py validating initialization, import errors, upserts, deletes, and retrievals.
    • Configured tests to use isolated, unique temporary database paths (/tmp/m_{uuid}.db) limited to 36 characters to prevent SQLite concurrent file locks and Milvus-Lite database name length limits.

Verification Results

All code validation checks have been successfully run and passed within WSL:

  • Ruff Quality Check: Passed (All checks passed!).
  • MyPy Type Safety: Passed (Success: no issues found in 36 source files).
  • PyTest Execution: 83/83 unit and integration tests passed successfully (including 4 new integration tests for MilvusAdapter).

Signed-off-by: KMohnishM <kmohnishm@gmail.com>
… are missing

Signed-off-by: KMohnishM <kmohnishm@gmail.com>
…ies and dynamic mypy target

Signed-off-by: KMohnishM <kmohnishm@gmail.com>
…oid connection alias lookup bug

Signed-off-by: KMohnishM <kmohnishm@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant