Skip to content

feat: define Milvus metadata cataLog boundary#1

Open
shaoting-huang wants to merge 4 commits into
masterfrom
pr/readme-v2
Open

feat: define Milvus metadata cataLog boundary#1
shaoting-huang wants to merge 4 commits into
masterfrom
pr/readme-v2

Conversation

@shaoting-huang

@shaoting-huang shaoting-huang commented Apr 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add a stable metadata catalog boundary for Milvus and wire the existing five coordinator catalogs through it.

This keeps the current etcd/TiKV-backed behavior unchanged while making the catalog backend replaceable later.

Changes

  • Expose Milvus catalog interfaces and model types from pkg/metastore.
  • Add milvus-catalog domain interfaces and compatibility adapters.
  • Wire RootCoord, DataCoord, QueryCoord, StreamingCoord, and StreamingNode through the catalog boundary.
  • Add mapping tests to ensure all methods in the five existing catalog interfaces are covered.

Current path:

internal/metastore/kv/*
  -> milvuscompat.Wrap(...)
  -> catalog.Catalog
  -> milvuscompat.New(...)
  -> existing component catalog interface

@shaoting-huang shaoting-huang changed the title doc: add README for milvus-catalog feat: Milvus Catalog Service — unified metadata entry for external engines Apr 24, 2026
@shaoting-huang shaoting-huang force-pushed the pr/readme-v2 branch 5 times, most recently from e5d7d14 to 079e5ca Compare May 11, 2026 11:00
1. Define pkg/catalog as the canonical domain-first Catalog API
with Metadata, AccessControl, InternalState, and Migration domains.

2. Add pkg/milvuscompat adapters that expose Milvus RootCoord,
DataCoord, QueryCoord, StreamingCoord, and StreamingNode catalog
interfaces on top of catalog.Catalog.

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
@shaoting-huang shaoting-huang changed the title feat: Milvus Catalog Service — unified metadata entry for external engines feat: define Milvus metadata cataLog boundary May 11, 2026
shaoting-huang and others added 3 commits May 12, 2026 20:06
- Remove MoveDatabase from CollectionCatalog; Alter handles DB changes
  with full-struct passthrough to prevent field loss when collections
  move between databases (regression test in full_struct_test.go)
- Add PartitionID filtering to wrappedSegments.List
- Add ErrNotWired sentinel separating wiring errors from spec gaps
- Surface ctx.Err() in 4 bool wrap methods so context cancellation
  doesn't silently become false
- Document SnapshotCatalog Get/ListManifests as compat-layer reserved
- Mark pkg/catalog as EXPERIMENTAL until v1.0
- CI: enable race detector
- Drop unused pkg/api/ deprecated alias
- Switch module path from milvus-io/milvus-catalog to zilliztech/milvus-catalog
  to match the actual repository URL

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
Follow-up to milvus rename: streamingcoord/node interface name typo
(CataLog -> Catalog) was fixed upstream in pkg/v3/metastore. Update
milvuscompat references accordingly and bump the milvus pkg/v3
dependency to the rename commit.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
Summary:
- Run gofmt on the four milvuscompat files reported by CI.
- Reorder the local catalog import according to gofmt output.

Rationale:
- PR #1 was failing the CI gofmt check because these files were not
  in canonical Go formatting.
- The change is formatting-only and does not alter business logic.

Tests:
- gofmt -l .
- go vet ./...
- go test ./...
- go build ./...
- go test -race ./...

Co-authored-by: Codex <codex@openai.com>
@shaoting-huang

Copy link
Copy Markdown
Collaborator Author

Codex update for ENG-1:

  • Fixed the failing gofmt check by running gofmt on the four reported pkg/milvuscompat files.
  • Diff is formatting-only: local pkg/catalog imports were moved into canonical gofmt order.
  • Pushed commit 05aa195 fix(milvuscompat): format imports to pr/readme-v2.

Local validation:

  • gofmt -l . returned empty output
  • go vet ./...
  • go test ./...
  • go build ./...
  • CI exact gofmt check: gofmt -l $(find . -name '*.go')
  • CI go mod tidy check: go mod tidy && git diff --exit-code -- go.mod go.sum
  • CI local replace check
  • go test -race ./...

Note: make is not available in this repo because there is no Makefile; CI uses go build ./..., which passes locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant