[Epic] Native asset and USD values calculations for XGraphs

[sh-rn]

Overview

The overall goal is to populate all the token price fields in terms of the native asset (e.g., UNI price in terms of ETH, JOE price in terms of AVAX, etc.), while integrating the new functionality into the existing solution and maintaining the performance and efficiency of the system.

1. Extend the existing data model to store token prices in terms of the native asset for each block in a new TokenPrice table.
2. Update the EventProcessorStageExchange_Bundle class to:
   • Calculate token prices in terms of the native asset using the bundle table data, with additional calculations for each token done in parallel using multiple worker threads.
   • Store the calculated token prices in the new TokenPrice table.
   • Use the Redis cache to efficiently store and retrieve token prices.
   • Accept a token_pairs parameter, allowing the class to track multiple token pairs without the need for separate derived classes.
   • Update the _process() method to handle calculations for multiple token pairs.
3. Modify the main processing loop to include instances of the updated EventProcessorStageExchange_Bundle class with the desired token pairs to track.

ref code: https://github.com/blocknetdx/XQuery/blob/master/xquery/event/processor_exchange_bundle.py

Tasks:

#4
#5
#6
#7

[sh-rn]

For review:

2. 1. Extend the existing data model:
   • Add a new table TokenPrice to the database schema. This table will store token prices in terms of the native asset for each block. It should have columns like block_id, token_address, native_asset_address, and price.
3. Update the EventProcessorStageExchange_Bundle class:
   • Modify the _process() method to calculate token prices in terms of the native asset using the bundle table data. For each sync event, after calculating the native asset price in USD (already implemented), you will also need to calculate token prices in terms of the native asset (e.g., UNI/ETH, JOE/AVAX, etc.). To do this, you can use the same weighted average approach as for native assets.
   • Store the calculated token prices in the new TokenPrice table. This can be done by creating new dictionary objects similar to the bundle dictionary and appending them to the objects list. Make sure to include the block_id, token_address, native_asset_address, and the calculated price in the dictionary.
   • Use the Redis cache to store and retrieve the token prices. When storing a new price in the TokenPrice table, also store it in the Redis cache with a suitable key (e.g., f"tokenprice:{block_id}:{token_address}:{native_asset_address}"). When retrieving a price, first check if it is available in the Redis cache before querying the database.
4. Implement a new derived class for each token pair you want to track:
   • Create a new class (e.g., EventProcessorStageExchangeUNI_ETH) that inherits from EventProcessorStageExchange_Bundle. Initialize it with the appropriate pair information (UNI and ETH in this case) and a default price.
   • Override the process() method if necessary, to handle any special cases or optimizations specific to the token pair.
5. Update the main processing loop:
   • In the main processing loop where the EventProcessorStageExchangePangolin_Bundle and EventProcessorStageExchangePegasys_Bundle classes are used, add instances of the new derived classes (e.g., EventProcessorStageExchangeUNI_ETH) and process them in the same way.