export functionality: start peers and wait for peer network to stabilize before exporting#162
Open
inqrphl wants to merge 3 commits into
Open
export functionality: start peers and wait for peer network to stabilize before exporting#162inqrphl wants to merge 3 commits into
inqrphl wants to merge 3 commits into
Conversation
…ize before exporting exporting used to only contact peers in the config and only call initAllTables. it did not start the peer at all, and onlt exported the initial state of the tables. this led to sub-peers in the lower tiers not being discovered. starting the peer would call peer.tryUpdate -> peer.updateTick -> datastoreset.updateDelta -> ds.sync.UpdateDelta this would then get sites table for LMD peers , and /thruk/r/v1/sites API endpoint for thruk peers. if a new site was discovered, it would then call peer.addSubPeer the sub peer would be added to lmd.peerMap , and started independently to possibly discover all sub peers. in the exporter function, add a network state stabilization function. this starts all peers in configuration, and lets them discover their network independently. after that it waits for stabilization in rounds. each round checks the peer map, seeing if the total count of peers went up and every peer has data. if the peer map looks stable for three consecutive rounds, it exits. maximum of 30 rounds with 10 second sleep time in between. after the stabilization rounds, the peer tables are exported. exported peer count is passed upwards in the call chain, up until to the main.go. this is done to stay true to the exported peer count. after the maximum rounds more peers could be discovered and added to peerMap, so saying that len(peerMap) of peers are exported could be wrong.
Author
|
Tested with the lmd_federation_multitier_e2e scenario, which spwans three tiers of OMD instances. |
…erAndDiscoverSubpeers this function calls peer.initAllTables and peer.updateTick once and does not do anything else with the peer use this function for initializing peers defined in config, and any other new peers while waiting for peer network to stabilize
Author
|
starting a peer would have it run in the background indefinitely while the peer network was stabilizing. instead of starting, wrote a new function instead called initializePeerAndDiscoverSubpeers. It calls peer.initAllTables and does a manual peer.updateTick, which will discover the subpeers. call this function for peers in lmd config first, and while waiting for stabilization iterate through peer list, and then call it for any new peers as well. the function calls waitGroup.Done() at the end, so concurrent instances for different peers are possible using goroutines. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
exporting used to only contact peers in the config and only call initAllTables. it did not start the peer at all, and only exported the initial state of the tables. this led to sub-peers in the lower tiers not being discovered.
starting the peer would call peer.tryUpdate -> peer.updateTick -> datastoreset.updateDelta -> ds.sync.UpdateDelta
this would then get sites table for LMD peers , and /thruk/r/v1/sites API endpoint for thruk peers. if a new site was discovered, it would then call peer.addSubPeer
the sub peer would be added to lmd.peerMap , and started independently to possibly discover all sub peers.
in the exporter function, add a network state stabilization function. this starts all peers in configuration, and lets them discover their network independently.
after that it waits for stabilization in rounds. each round checks the peer map, seeing if the total count of peers went up and every peer has data. if the peer map looks stable for three consecutive rounds, it exits. maximum of 30 rounds with 10 second sleep time in between.
after the stabilization rounds, the peer tables are exported. exported peer count is passed upwards in the call chain, up until to the main.go. this is done to stay true to the exported peer count. after the maximum rounds more peers could be discovered and added to peerMap, so saying that len(peerMap) of peers are exported could be wrong.