Add Gnosis

[kaladinlight]

• No geth support, the execution node is nethermind which is written in C#. This resulted in the need for hand rolling the Header type and associated marshal/rlp handlers. I did start to think about if it is even necessary to use the Hash() function on the Header to get the block hash instead of just getting hash directly off the Block. Was this an intentional decision for some edge case that I am not thinking about?
• Start the consensus node first so it creates the jwt and then start the execution node.
• Nethermind is super quick to sync a full node to start testing (~10hrs), archive nodes still takes the better part of a week I believe

[kaladinlight]

@martinboehm ready for review whenever you have free cycles :)

[matyushkins]

@kaladinlight Hi. Archive node.
websocket: read limit exceeded

E0717 10:45:03.174752       1 ethrpc.go:652] debug_traceBlockByHash block 0xdc31a7341d59c76c89e2a20bccfa726b15ef165cf2cb57cf67af92cd46417133, error websocket: read limit exceeded
E0717 10:45:03.174769       1 ethrpc.go:652] debug_traceBlockByHash block 0xcdc64212c941d29f0a9b9905a858222de498b0377524cc42a90b76836e0910dc, error websocket: read limit exceeded
E0717 10:45:03.174774       1 ethrpc.go:652] debug_traceBlockByHash block 0x5dfc70726803b8b6a2db378c9e1d1686e8fb2f2a07a7f10dac398478b4f6b5b6, error websocket: read limit exceeded
E0717 10:45:03.174759       1 ethrpc.go:652] debug_traceBlockByHash block 0xc64f390b6556bf77708b5d11a8161c5ce15f3dbe2c066a02dde1f4e0b0490d95, error websocket: read limit exceeded

[kaladinlight]

@kaladinlight Hi. Archive node. websocket: read limit exceeded

E0717 10:45:03.174752       1 ethrpc.go:652] debug_traceBlockByHash block 0xdc31a7341d59c76c89e2a20bccfa726b15ef165cf2cb57cf67af92cd46417133, error websocket: read limit exceeded
E0717 10:45:03.174769       1 ethrpc.go:652] debug_traceBlockByHash block 0xcdc64212c941d29f0a9b9905a858222de498b0377524cc42a90b76836e0910dc, error websocket: read limit exceeded
E0717 10:45:03.174774       1 ethrpc.go:652] debug_traceBlockByHash block 0x5dfc70726803b8b6a2db378c9e1d1686e8fb2f2a07a7f10dac398478b4f6b5b6, error websocket: read limit exceeded
E0717 10:45:03.174759       1 ethrpc.go:652] debug_traceBlockByHash block 0xc64f390b6556bf77708b5d11a8161c5ce15f3dbe2c066a02dde1f4e0b0490d95, error websocket: read limit exceeded

Did you build blockbook using the Makefile? There is a hack in place to manually increase the websocket message size limit that should fix this issue unless the gnosis block traces are actually larger than the current 80mb setting.

prepare-sources:
	@ [ -n "`ls /src 2> /dev/null`" ] || (echo "/src doesn't exist or is empty" 1>&2 && exit 1)
	rm -rf $(BLOCKBOOK_SRC)
	mkdir -p $(BLOCKBOOK_BASE)
	cp -r /src $(BLOCKBOOK_SRC)
	cd $(BLOCKBOOK_SRC) && go mod download
	sed -i 's/wsMessageSizeLimit\ =\ 15\ \*\ 1024\ \*\ 1024/wsMessageSizeLimit = 80 * 1024 * 1024/g' $(GOPATH)/pkg/mod/github.com/ethereum/go-ethereum*/rpc/websocket.go
	sed -i 's/wsMessageSizeLimit\ =\ 15\ \*\ 1024\ \*\ 1024/wsMessageSizeLimit = 80 * 1024 * 1024/g' $(GOPATH)/pkg/mod/github.com/ava-labs/coreth*/rpc/websocket.go

[matyushkins]

@kaladinlight Hi. Archive node. websocket: read limit exceeded

E0717 10:45:03.174752       1 ethrpc.go:652] debug_traceBlockByHash block 0xdc31a7341d59c76c89e2a20bccfa726b15ef165cf2cb57cf67af92cd46417133, error websocket: read limit exceeded
E0717 10:45:03.174769       1 ethrpc.go:652] debug_traceBlockByHash block 0xcdc64212c941d29f0a9b9905a858222de498b0377524cc42a90b76836e0910dc, error websocket: read limit exceeded
E0717 10:45:03.174774       1 ethrpc.go:652] debug_traceBlockByHash block 0x5dfc70726803b8b6a2db378c9e1d1686e8fb2f2a07a7f10dac398478b4f6b5b6, error websocket: read limit exceeded
E0717 10:45:03.174759       1 ethrpc.go:652] debug_traceBlockByHash block 0xc64f390b6556bf77708b5d11a8161c5ce15f3dbe2c066a02dde1f4e0b0490d95, error websocket: read limit exceeded

Did you build blockbook using the Makefile? There is a hack in place to manually increase the websocket message size limit that should fix this issue unless the gnosis block traces are actually larger than the current 80mb setting.

prepare-sources:
	@ [ -n "`ls /src 2> /dev/null`" ] || (echo "/src doesn't exist or is empty" 1>&2 && exit 1)
	rm -rf $(BLOCKBOOK_SRC)
	mkdir -p $(BLOCKBOOK_BASE)
	cp -r /src $(BLOCKBOOK_SRC)
	cd $(BLOCKBOOK_SRC) && go mod download
	sed -i 's/wsMessageSizeLimit\ =\ 15\ \*\ 1024\ \*\ 1024/wsMessageSizeLimit = 80 * 1024 * 1024/g' $(GOPATH)/pkg/mod/github.com/ethereum/go-ethereum*/rpc/websocket.go
	sed -i 's/wsMessageSizeLimit\ =\ 15\ \*\ 1024\ \*\ 1024/wsMessageSizeLimit = 80 * 1024 * 1024/g' $(GOPATH)/pkg/mod/github.com/ava-labs/coreth*/rpc/websocket.go

Yes. I run make command with BASE_IMAGE=ubuntu:22.04.
make BASE_IMAGE=ubuntu:22.04

[kaladinlight]

I can push a change to bump that limit if you want to rebuild the image to test. I don't have an archive node to point at currently to test locally...

[matyushkins]

{
       "jsonrpc": "2.0",
       "method": "debug_traceBlockByHash",
       "params": ["0x9bccfe5644bc698235f904c76375e3ed882d037451b5dc8bd66a09c592b3902d", {"tracer": "callTracer"}],
       "id": 1
}

Size json file 205 MB

[kaladinlight]

{
       "jsonrpc": "2.0",
       "method": "debug_traceBlockByHash",
       "params": ["0x9bccfe5644bc698235f904c76375e3ed882d037451b5dc8bd66a09c592b3902d", {"tracer": "callTracer"}],
       "id": 1
}

Size json file 205 MB

Good lord... I will bump to 256 for temp fix. I would like to look into adding a logic fork to leverage the parity trace module instead of geth debug to reduce these payload sizes and improve perf long term.

[matyushkins]

@kaladinlight I raised the limit to 2048 because the page size is 1800MB. And rpc timeout to 300 sec.

{
       "jsonrpc": "2.0",
       "method": "debug_traceBlockByHash",
       "params": ["0xcd3cd8cf7540380c376d61799adc5990a707ac02c34fa9f4c802bf891de4679b", {"tracer": "callTracer"}],
       "id": 1
}

5 million blocks. The flight is normal.

[matyushkins]

~2200 MB

{
       "jsonrpc": "2.0",
       "method": "debug_traceBlockByHash",
       "params": ["0x7eb96747d3b5356abfef48cd3729e69394140315e6945254e3256e902ec1db72", {"tracer": "callTracer"}],
       "id": 1
}

[kaladinlight]

~2200 MB

{
       "jsonrpc": "2.0",
       "method": "debug_traceBlockByHash",
       "params": ["0x7eb96747d3b5356abfef48cd3729e69394140315e6945254e3256e902ec1db72", {"tracer": "callTracer"}],
       "id": 1
}

that is crazy... at this point I would say raise it as you need to for your purposes and when I have time I will work on supporting the trace module for gnosis which will hopefully help things

[kaladinlight]

@matyushkins not sure if you are still running this, but I finally got around to adding the parity trace debug for gnosis if you wanted to give that a shot. it should be significantly lighter to run with this change. let me know!

[kaladinlight]

support overridable GetInternalDataForBlock to support the use of the parity trace module for debug. This is only used by gnosis at this time, but I suppose I could also add that logic directly to ethrpc.go and leverage a DebugType flag if you would like that better.

[kaladinlight]

@martinboehm another one that would be nice to get upstreamed if you have any spare cycles. thanks!

[kaladinlight]

@martinboehm happy to talk through this guy as well, but quick tldr:

• due to massive amounts of nft on gnosis, the sync speed crawled to a halt
• after profiling, the bottleneck was pack/unpack -> read/write to db for address contracts
• I went ahead and upgraded the address contracts to protobufs for more efficient encode/decode along with a configurable address cache size in an effort to improve performance
• there did seem to be improvements, but fresh sync is slowing down drastically again resulting in my perf improvement attempts not being enough

Would love to brainstorm with you if you have the time and are up for it, but at this point gnosis is not really production ready in the current state of blockbook :(