Skip to content

release-26.4: roachtest: replicate/wide failed #175656

Description

@cockroach-teamcity

roachtest.replicate/wide failed with artifacts on release-26.4 @ b4884d371d74d172ab184bc4ef0924ccb758f3c2:

Failed with:

(assertions.go:388).Fail: 
	Error Trace:	pkg/cmd/roachtest/tests/allocator.go:458
	            				pkg/cmd/roachtest/tests/allocator.go:464
	            				pkg/cmd/roachtest/test_runner.go:1992
	            				src/runtime/asm_amd64.s:1771
	Error:      	Received unexpected error:
	            	roachtest node 9 is not a CRDB node
	            	(1) attached stack trace
	            	  -- stack trace:
	            	  | main.validateCockroachPhysicalNode
	            	  | 	pkg/cmd/roachtest/cluster_node_identities.go:112
	            	  | main.cockroachNode
	            	  | 	pkg/cmd/roachtest/cluster_node_identities.go:198
	            	  | main.(*roachprodCluster).CockroachNode
	            	  | 	pkg/cmd/roachtest/cluster_node_identities.go:229
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runWideReplication.func8
	            	  | 	pkg/cmd/roachtest/tests/allocator.go:457
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runWideReplication
	            	  | 	pkg/cmd/roachtest/tests/allocator.go:464
	            	  | main.(*testRunner).runTest.func2
	            	  | 	pkg/cmd/roachtest/test_runner.go:1992
	            	  | runtime.goexit
	            	  | 	src/runtime/asm_amd64.s:1771
	            	Wraps: (2) roachtest node 9 is not a CRDB node
	            	Error types: (1) *withstack.withStack (2) *errutil.leafError
	Test:       	replicate/wide
(require.go:1376).NoError: FailNow called
test artifacts and logs in: /artifacts/replicate/wide/run_1

Cluster Node to Ip Mapping:

Node Public IP Private IP
teamcity-21523842-1789803587-55-n9cpu1-0001 10.91.98.75
teamcity-21523842-1789803587-55-n9cpu1-0002 10.91.99.235
teamcity-21523842-1789803587-55-n9cpu1-0003 10.91.99.36
teamcity-21523842-1789803587-55-n9cpu1-0004 10.91.98.98
teamcity-21523842-1789803587-55-n9cpu1-0005 10.91.98.76
teamcity-21523842-1789803587-55-n9cpu1-0006 10.91.97.46
teamcity-21523842-1789803587-55-n9cpu1-0007 10.91.98.62
teamcity-21523842-1789803587-55-n9cpu1-0008 10.91.98.110
teamcity-21523842-1789803587-55-n9cpu1-0009 10.91.98.24

Parameters:

  • arch=amd64
  • cloud=gce
  • coverageBuild=false
  • cpu=1
  • diskCount=0
  • encrypted=false
  • fs=ext4
  • localSSD=true
  • metamorphicLeases=default
  • runtimeAssertionsBuild=false
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

See: Datadog Logs

Same failure on other branches

/cc @cockroachlabs/kv-triage

Improve this report!

Jira issue: CRDB-68556

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-testingTesting tools and infrastructureC-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.C-test-failureBroken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.T-kvKV Teambranch-release-26.4Used to mark GA and release blockers, technical advisories, and bugs for 26.4target-release-26.4.0target-release-27.1.0

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions