Feat/unify backend layer

[cardosofede]

Features

Unified User Preferences (TG + Web)
User preferences are now shared across Telegram and the web dashboard via config.yml. Every preference mutation in Telegram (active server, portfolio days, CLOB account, DEX slippage,
etc.) syncs to ConfigManager, and the web dashboard can read/write them through new GET/PATCH /auth/preferences endpoints. On first access, preferences are hydrated from config.yml into
the Telegram session.

Singleton Candle Store
New framework-agnostic candle-store.ts singleton that manages all candle data outside of React Query. Features reference-counted WS subscriptions with 5-minute deferred teardown, LRU
eviction (max 20 collections), timestamp dedup/merge, and O(1) upserts. A companion useCandleStore React hook bridges it to components with built-in staleness detection. The TradingView
datafeed now uses the candle store directly instead of managing its own WS subscriptions or REST polling fallback.

WebSocket onConnect Callback
CondorWebSocket now exposes an onConnect() handler so components can react immediately to (re)connections instead of relying on polling intervals.

Positions Notional Value Column
Added notional_value (amount × entry_price) to the positions table in both the MCP formatter and the frontend ConsolidatedPosition interface.

---

Improvements

Extract Fetchers into condor/fetchers/
All data-fetch functions scattered across handler _shared.py files were extracted into a dedicated condor/fetchers/ package with one module per data type (portfolio, prices, positions,
orders, executors, bots, connectors, trading_rules, market_data, server_status). This breaks circular dependencies between the data layer and handler layer and makes fetch logic
reusable without importing handler code.

Simplify SDS Registration
register_default_fetches() now imports directly from condor.fetchers instead of defining inline wrappers that imported from handlers. The listener notification passes ServerDataType
directly instead of reverse-mapping to the legacy DataType enum. The redundant DataManager.register_default_fetches() call was removed from main.py.

Backtest Store: Per-File Storage
Migrated from a single data/backtests.json to individual files under data/backtests/ with a lightweight _index.json for fast listing. Includes automatic legacy migration on first run.

Controller Config Full-Replace
When updating a controller config with YAML content, it now does a full replace (preserving identity fields id, controller_name, controller_type) instead of a shallow merge that left
stale keys behind.

---

Fixes

Executor Data Lost on Page Refresh
The executors query was enabled: false (WS-only), meaning a hard refresh showed no executors until the next WS push. Changed to a real REST fetch on mount with staleTime: 30s and
keepPreviousData to prevent flicker.

Candle Data Managed in React Query Caused Re-renders and Memory Growth
Candle updates flowing through React Query's setQueryData caused unnecessary re-renders across unrelated components and unbounded cache growth. Moved candle data to the dedicated
singleton store which handles its own lifecycle.

WS Reconnect Not Detected Immediately
Components relied on a polling interval to detect WS reconnections, causing a delay before data resumed. The new onConnect callback triggers React state updates instantly on reconnect.

---

QA Test Plan

1. Fetchers — No Regressions

• Open Telegram, run /portfolio — portfolio data loads correctly
• Run /bots — bot status list renders
• Run /executors — executor list loads
• Run /trade on a CEX pair — prices, order book, and trading rules load
• Edit any handler file to trigger hot-reload — data keeps flowing after reload

2. User Preferences Sync (TG ↔ Web)

• In Telegram, change portfolio days (e.g. 7d → 30d) or switch active server
• Open web dashboard → GET /auth/preferences — confirm the change is reflected
• In the web dashboard, call PATCH /auth/preferences with {"updates": {"portfolio": {"days": 14}}} — confirm it persists
• Back in Telegram, run /portfolio — verify it now uses 14 days
• Restart the bot — verify preferences survive restart (check config.yml has user_preferences section)

3. Candle Store (Charts)

• Open the Trade page — verify candlestick chart loads with data
• Switch trading pair — chart updates to new pair, old data doesn't leak
• Switch interval (1m → 5m → 1h) — candles reload correctly
• Wait 30+ seconds on a sub-1h interval — candles keep updating, no staleness indicator
• Open a second chart tab with a different pair, close the first — second tab keeps receiving data
• Disconnect network briefly, reconnect — chart resumes updating
• Hard refresh the page (F5) — chart loads from REST and WS resumes streaming

4. WebSocket Reconnection

• Open the web dashboard, verify WS connects (check browser console)
• Kill and restart the backend — frontend auto-reconnects and data resumes
• Check no duplicate subscriptions in WS after reconnect (inspect backend logs)

5. Executor Data Resilience

• Open the Trade page — executors load immediately (no blank state)
• Hard refresh the page — executors visible right away
• Create or close an executor — it appears/disappears in real-time via WS

6. Backtest Store Migration

• If data/backtests.json exists, start the bot — verify migration to data/backtests/*.json + _index.json
• Run a new backtest — result saved as individual file
• List backtests — both migrated and new results appear
• Restart the bot — all backtests still accessible

7. Positions Notional Value

• Open a connector with active perpetual positions
• In the web dashboard, check the positions table — notional_value column shows correctly
• Via MCP, run get_portfolio_overview — positions table includes notional column

8. Controller Config Full-Replace

• In the web dashboard, open a controller config
• Edit the full YAML and save — config is fully replaced, no stale keys from previous config
• Verify id, controller_name, controller_type are preserved after save
• Edit via the JSON field editor (non-YAML path) — still does a merge as before

9. Grid Chart

• Open a Grid executor detail page — chart loads with candles and grid lines
• Verify executor overlays render on the chart
• Resize the browser window — chart resizes properly
• Switch to a different grid executor — chart updates

10. Smoke Test — Full Flow

• Start fresh (no pickle, clean config) → register → approve → set server → /portfolio, /bots, /trade
• Open web dashboard → login → navigate Portfolio, Bots, Executors, Trade pages
• No errors in backend logs
• No console errors in browser dev tools

[nikspz]

commit 3a24d44

• Cloned and installed PR80
  • make install ✅
  • make run ✅
• there’s no /servers after /start command ❌(in the menu)
• 
• update server ip with /servers
• cannot reach servers (resolved)
  • There’s no info on condor site on how to set up hummingbot-api and it is not set up for 0.0.0.0 server also https://condor.hummingbot.org/getting-started/installing
    • set up with and update docker stuff

      make install              # Create conda environment
      conda activate hummingbot-api
      make run                  # Run with hot-reload
      

• Fetchers — No Regressions
  • Open Telegram, run /portfolio — portfolio data loads correctly ✅
  • Run /bots — bot status list renders ✅
  • Run /executors — executor list loads ✅
  • Run /trade on a CEX pair — prices, order book, and trading rules load ❌ shows chain and connector
    • used through dashboard: created order successfully ✅
    • cancelled order successfully (when stopped executor)✅
  • Edit any handler file to trigger hot-reload — data keeps flowing after reload
• User Preferences Sync (TG ↔ Web)
  • / web ✅
  • In Telegram, change portfolio days (e.g. 7d → 30d) or switch active server ✅
    • could change but not showing previous value (7d or 30 days ago since it just started)
• Candle Store (Charts)
  • Open the Trade page — verify candlestick chart loads with data ✅
  • Switch trading pair — chart updates to new pair, old data doesn't leak ✅
  • Switch interval (1m → 5m → 1h) — candles reload correctly ✅
  • Wait 30+ seconds on a sub-1h interval — candles keep updating, no staleness indicator ✅
  • Open a second chart tab with a different pair, close the first — second tab keeps receiving data ✅
  • Disconnect network briefly, reconnect — chart resumes updating ✅
  • Hard refresh the page (F5) — chart loads from REST and WS resumes streaming ✅
• Executor Data Resilience
  • Open the Trade page — executors load immediately (no blank state) ✅
  • Hard refresh the page — executors visible right away ✅
  • Create or close an executor — it appears/disappears in real-time via WS ✅
• Using web interface:
  • Portfolio: ok ✅
  • Trade: ok ✅
  • Bots: ok ✅
    • Note: logs is not available in Dashboard ❌ (But available to check from Condor telegram menu of Bots)

[rapcmia]

Commit de42796

• Check files changes and dev notes 👀
• Setup this PR (source, make run) with fresh installation of hummingbot-api (latest, make deploy) using docker on localhost/remote server ✅
• Fetchers — No Regressions ✅
  • Added spot and perpetual connectors ok
  • Bots, executors and trade (CEX) ok
  • Check portfolio before and after trades ok
• Candle Store (Charts) ✅
  • Chart data loaded correctly, pair/interval switching behave as expected, and no stale/leaked candles were observed
  • It also recovered correctly after internet reconnection, page refresh, and when one browser tab was closed while another stayed open
• WebSocket Reconnection ✅
  • Web dashboard connection recovered correctly after restarting hummingbot-api
  • After reload, live data resumed and the page worked as expected e.g trade page
  • Logs also looked clean with normal reconnect and resubscribed behavior
• Executor Data Resilience ✅
  • The executor list appeared correctly after reconnect/refresh, and status changes updated live
  • When one executor was stopped (early_stop), it was removed from the active list
• Backtest Store Migration ✅
  • Create simple generic.grid_strike ok
  • After adding the legacy backtests.json, startup correctly moved data into the new per-file format with _index.json
    
    
  • Both migrated and new backtests results remained visible, including after restart
• Positions Notional Value ✅
  • In the web dashboard, check the positions table — notional_value column shows correctly
    
    • Opened a long position using order executor, the positions tab shows (1) along with the notional value
    • However MCP's get_portfolio_overview returns position field but does not include notional column (related to hummingbot-mcp not this PR) ❗
• Controller Config Full-Replace ✅
  • On Bot page, editor tab full YAML edit/save correctly performs a full replace with no stale keys left behind
  • YAML save works as full replace, and no old keys remain
    • controller_name and controller_type are editable, so they are saved as whatever value user set
    • if invalid or typo, save button is disabled until user fixes it
    • Fixed Bot - Editing a config does not remove deleted custom parameters, so stale keys keep accumulating #78
  • JSON field editor still works the old way, merges changes instead of replacing the whole config
  • Deployment works along with improvements e.g logs and stop command ✅
    • This fixed Bot - Active bots appears in Bot menu but has no Stop button #79
• Grid Chart ✅
  • From executor page, all grtid chart check passed. Candles and grid lines loaded, overlays appear correctly and the chart resizes properly with the browser window
  • Switching to a different grid executor updates the chart to the selected executor correctly
• Smoke test
  • Fresh installation of hummingbot-api (latest) and this PR, server already at localhost
  • Successfully navigated portfolio, bots, executors and trade pages
  • Found no errors on backend logs when switching or refresh pages

---

Test User Preferences Sync (TG ↔ Web) ❗

• Open web dashboard → GET /auth/preferences using swagger ✅
• In the web dashboard, call PATCH /auth/preferences with {"updates": {"portfolio": {"days": 14}}} — confirm it persists ❌
  • Im not able to verify this on web dashboard but seem to work with API tests ❗

    curl -sS -X PATCH 'http://100.124.88.27:8088/api/v1/auth/preferences' \
        -H 'accept: application/json' \
        -H 'Content-Type: application/json' \
        -H 'Authorization: Bearer <TOKEN>' \
        -d '{"updates":{"portfolio":{"days":14}}}' | jq '.'
        
    Response:
    {
      "additionalProp1": {},
      "portfolio": {
        "days": 14
      }
    }
    
    
    curl -sS -X GET 'http://100.124.88.27:8088/api/v1/auth/preferences' \
      -H 'accept: application/json' \
      -H 'Authorization: Bearer <TOKEN>' | jq '.'
    
    Response (GET):
    {
      "additionalProp1": {},
      "portfolio": {
        "days": 14
      }
    }
    

  • Using dev tools ❌
    • Im not seeing any /api/v1/auth/me related when i check devTools ❌
      
    • However when I explicitly access the http://localhost:8088/api/v1/auth/me it returns not authrized
      
• However, I can still access web dashboard ✅
• Back in Telegram, run /portfolio to verify if now uses 12 days, no it does not display any related to it
  
• Restart the bot — verify preferences survive restart (check config.yml has user_preferences section) ❌
  • No user_preferences section on config.yml

---

Other issues found:

• Check config, observed formatting seems different like usual causing backtesting to fail due to invalid parameters (sample from gs_02)❌
  
  
  • I have to manually change them in order to have the backtest running as expected (sample from gs_01)