CASSANALYTICS-202: Opening one keyspace instance rebuilds every keysp…

[mansikhara]

…ace instance in the Cassandra 6.0 bridge

AbstractSchemaBuilder created a missing keyspace instance by calling SchemaUpdater.openKeyspaceInstances(), which diffs the current cluster metadata against DistributedSchema.empty(). Every keyspace in the process therefore looks newly created and all of them are rebuilt, so each keyspace that already has an instance constructs its KeyspaceMetrics again, and every metric it re-registers makes Dropwizard's MetricRegistry.register throw IllegalArgumentException. Filling in those stack traces is the real cost, and it scales with the number of keyspaces the process has seen.

This does not affect production, where a bulk job uses a single keyspace and builds its schema once. It is visible in tests: DataTypeTests opens a fresh keyspace per generated case, so the 6.0 core test job slowed down through the class until the CI step exceeded its deadline part way through. Two thread dumps taken fifteen seconds apart were both in Throwable.fillInStackTrace under MetricRegistry.register, and jstat ruled out memory pressure with zero full collections and seven seconds of GC across twelve minutes.

openKeyspaceInstances() is replaced by openKeyspaceInstance(String), which diffs against the current keyspaces minus the one being opened, leaving exactly one keyspace in Keyspaces.diff().created so a single instance is built and no metric is registered twice. That also keeps the keyspace out of diff().altered, which is the property removeTables relies on to avoid Keyspace.dropCf initializing CompactionManager in client mode, so the CDC caller passes keyspaceMetadata.name and the blunt method is removed altogether.

DataTypeTests on 6.0 goes from not completing to about ten minutes.

[sarankk]

+1 Thanks Mansi for this find and fix!

[bbotella]

+1

Thanks!