XRouter Go fails to find peers

[synechist]

When requests are made on Go-XRouter (e.g. getBlockCount), its response is Failed to find enough peers supporting xrGetBlockCount even when there are several EXR service nodes supporting the request.

Ex:ample: requesting getBlockCount for LTC on Blocknet Core's console prints the following:

xrGetBlockCount LTC 20

{
  "code": 1032,
  "error": "Failed to find 20 service node(s) supporting LTC::xrGetBlockCount with config limits, found 12",
  "uuid": "9E17E2B3-2C1F-472B-AE5A-66BD3EE0130C"
}

xrGetBlockCount LTC 12

{
  "reply": 2120414,
  "uuid": "FF6241A4-91C3-4C9E-A538-CA21347DBCF6"
}

However, making the same request on Go-XRouter with a queryCount of 1 returns:

2021/09/10 16:57:05 No replies found. Failed to find enough peers supporting xrGetBlockCount for xr::LTC whose fees fall within the limits set in your config file. You requested responses from 1 nodes, but only got 0. Please try to connect to more peers before retrying the request.
2021/09/10 16:57:05 Server shutting down
2021/09/10 16:57:05 Shutdown peer 95.111.229.38:41412 (outbound)

Using Blocknet Core console to check the number of EXR service nodes supporting LTC reveals that there is at least one:

"snodekey": "022d58ff9223bccdcd778f9346d744b59ed8bacd4a16db42272a8708344adbdcd0",
 "tier": "SPV",
 "address": "BpX1n4k6c9QVkgpq53T3zgQw9u82uB7zZM",
 "timelastseen": 1631286330,
 "timelastseenstr": "2021-09-10T15:05:30.000Z",
 "exr": true,
 "status": "running",
 "score": 0,
 "services": [
   "BLOCK",
   "DGB",
   "LTC",
   "xr",
   "xr::BLOCK ",
   "xr::BTC",
   "xr::DGB",
   "xr::LTC"
 ]

Hence, there is an issue with Go-XRouter either learning of, or connecting to, EXR snodes.

[synechist]

Update: awaiting QA upon yesterday's fix.
Details:

1. Prior to the latest fix, I confirmed the existence of several LTC-supporting EXR snodes, but couldn't get any responses on Go-XRouter. This was tested on two machines in different regions, one Ubuntu machine and one Windows machine running WSL2 Ubuntu.
2. Required test: confirm that Go-XRouter can get responses from every LTC-supporting EXR service node on the network, comparing responses from the service node list returned in servicenodestatus on Blocknet Core.

@Akado2009 can you confirm which branch and pull request the fix was made in?

[synechist]

Update:
After our working through the code systematically, the issue appears to stem from the snodelist. Both Blocknet Core and go-xrouter do not always get a complete list of service nodes to connect to, which leads to the problem described above.

To investigate further, analysis is needed on the C++ code for snodelist. It should equip service nodes (and other peers running Blocknet Core) with a complete snodelist that is then gossiped to peers, updated, and re-gossiped, etc. but newly launched go-xrouter and Blocknet Core instances do not get a complete list immediately, and sometimes do not get it at all.

[atcsecure]

This could be potentially solved by exposing the xrshowconfigs RPC method via EXR

[atcsecure]

blocknetdx/exrproxy@dcd2d15