Skip to content

Commit 69e27ad

Browse files
keitajclaude
andcommitted
feat: add post-only rejection log downgrade and 5-min aggregation
Routine post-only rejections (``Post only order would have immediately matched``) are an expected retry signal under maker-only quoting, but were historically logged at ERROR. As order size scales up they dominate the ERROR stream, drowning out genuine errors and making any ERROR-rate-based monitoring noisy. This commit introduces ``OrderRejectionTracker``, an opt-in classifier that: * Pattern-matches each rejection's API error text against a static routine-pattern map (currently post-only only; reduce-only and other patterns can be added as one-line entries). * Logs routine matches at the level configured via ``--rejection-log-level`` (default ``error`` — preserves legacy behaviour). Unknown patterns always log at ERROR so format changes / new reject reasons remain visible. * Aggregates per-coin counts and emits a single ``[reject-summary]`` INFO line every ``--rejection-summary-interval`` seconds (default 300, 0 disables). Summary includes top-8 coins by count, sample bbo, and unknown count. Wiring is non-invasive: ``OrderManager.set_rejection_tracker`` is the only new public surface; with no tracker registered, both the single and bulk order paths emit the historical ``logger.error`` line verbatim. The MM strategy registers a tracker in ``__init__`` and calls ``log_summary_if_due`` from the existing periodic-summary block in the main loop. The default ``rejection_log_level=error`` means deployments see no behaviour change beyond the new (purely additive) summary line. Once the summary is trusted, operators can flip to ``warning`` to clean up the ERROR stream. Tests cover: * Pattern classification (post-only / unknown / empty / variant text) * Log-level downgrade and unknown-level fallback to ERROR * Counter increments, first/last timestamp tracking * Periodic summary: not-due / due / interval=0 / unknown counter inclusion / no-event suppression / top-8 truncation * Concurrent ``record`` from 10 threads × 100 calls — no lost increments * End-to-end via ``OrderManager`` (single + bulk paths) for both registered and unregistered tracker, ensuring back-compat is exact Design doc: hip-3-farming-agent-team/docs/design-doc/20260505_post_only_reject_log_aggregation.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent e47d2ae commit 69e27ad

12 files changed

Lines changed: 927 additions & 7 deletions

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -363,6 +363,8 @@ The `market_making` strategy uses **progressive close pricing**: as a position a
363363

364364
**Adverse selection logging** (`--enable-adverse-selection-log`): Measures mid-price movement 5s/30s/60s after each fill, logging per-coin summaries every 300s. Observation only — no trading impact.
365365

366+
**Order rejection log aggregation** (`--rejection-log-level`, `--rejection-summary-interval`): Routine post-only rejections (`Post only order would have immediately matched`) are an expected retry signal under maker-only quoting, but they were historically logged at ERROR — drowning out genuine errors as MM size grows. The strategy now classifies each rejection by API error text and routes routine matches through a small aggregator that logs a single `[reject-summary]` INFO line every `--rejection-summary-interval` seconds (default 300, set to 0 to disable). The default per-rejection log level stays `error` to preserve historical behaviour; flip `--rejection-log-level warning` (or `info`) once the summary line is trusted. Unknown rejection text always falls through to ERROR so format changes / new reject reasons stay visible.
367+
366368
**Dynamic offset** (`--dynamic-offset`): Auto-adjusts per-coin BBO offset based on adverse selection severity from the tracker. Coins with higher adverse selection get wider offsets; favorable coins get tighter offsets. Requires `--enable-ws` and `--enable-adverse-selection-log`. Manual `--coin-offset-overrides` serve as the baseline; dynamic adjustment adds/subtracts from it.
367369

368370
**Dynamic position age** (`--dynamic-age`): Adjusts `MAX_POSITION_AGE` per coin based on recent volatility. High-volatility coins get shorter holding times (reducing adverse selection risk), while low-volatility coins get longer times (improving maker fill probability). Uses the same mid-price history as `--vol-adjust`. Configure `--dynamic-age-baseline-vol` (bps) as the "normal" volatility reference, and `--dynamic-age-min` / `--dynamic-age-max` (seconds) for clamping bounds. Falls back to the fixed `--max-position-age` when data is insufficient.
@@ -686,6 +688,8 @@ ws_guards: # All require --enable-ws
686688
velocity_min_move_bps: 1.0 # --velocity-min-move-bps (min cumulative move in bps to trigger)
687689
enable_adverse_selection_log: false # --enable-adverse-selection-log (post-fill mid tracking)
688690
adverse_selection_log_interval: 300 # --adverse-selection-log-interval (summary log interval in seconds)
691+
rejection_log_level: "error" # --rejection-log-level (level for routine post-only rejections: error/warning/info/debug; default error preserves legacy behaviour)
692+
rejection_summary_interval: 300 # --rejection-summary-interval (aggregate rejection summary cadence in seconds; 0 disables)
689693

690694
config_merge_order: "default_configs[strategy] ← CLI overrides (only non-null)"
691695
priority: "CLI flag > env var > default_configs > strategy constructor fallback"

bot.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,8 @@
163163
'forager_weight_activity',
164164
'forager_weight_quality',
165165
'forager_weight_cost',
166+
'rejection_log_level',
167+
'rejection_summary_interval',
166168
'drain_flag_file',
167169
],
168170
}
@@ -1253,6 +1255,15 @@ def stop(self):
12531255
help='Enable post-fill adverse selection measurement logging')
12541256
parser.add_argument('--adverse-selection-log-interval', type=float,
12551257
help='Adverse selection summary log interval in seconds (default: 300)')
1258+
parser.add_argument('--rejection-log-level', type=str,
1259+
choices=['error', 'warning', 'info', 'debug'],
1260+
help='Log level for routine post-only order rejections '
1261+
'(default: error — preserves legacy behaviour). '
1262+
'Set to "warning" to reduce ERROR noise once the '
1263+
'5min summary line is trusted.')
1264+
parser.add_argument('--rejection-summary-interval', type=float,
1265+
help='Order rejection aggregate summary interval in seconds; '
1266+
'0 disables the summary (default: 300)')
12561267
parser.add_argument('--coin-offset-overrides', type=str, default='',
12571268
help='Per-coin BBO offset overrides in bps (e.g. "SP500:0.5,MSFT:3")')
12581269
parser.add_argument('--coin-spread-overrides', type=str, default='',

order_manager.py

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import logging
22
from collections import defaultdict
3-
from typing import Dict, List, Optional
3+
from typing import TYPE_CHECKING, Dict, List, Optional
44
from dataclasses import dataclass
55
from datetime import datetime
66
from enum import Enum
@@ -12,6 +12,9 @@
1212
from coin_utils import is_hip3, parse_coin
1313
from ttl_cache import TTLCacheEntry, TTLCacheMap
1414

15+
if TYPE_CHECKING:
16+
from order_rejection_tracker import OrderRejectionTracker
17+
1518
logger = logging.getLogger(__name__)
1619

1720

@@ -90,6 +93,30 @@ def __init__(self, exchange: Exchange, info: Info, account_address: str,
9093
# Not thread-safe; bot runs single-threaded.
9194
self._open_orders_cache: TTLCacheEntry[List[Dict]] = TTLCacheEntry(user_state_cache_ttl)
9295

96+
# Optional rejection tracker: when set, routine post-only rejections
97+
# are routed through it for log-level downgrade and aggregation.
98+
# When ``None`` the legacy ERROR-level path is used (unchanged).
99+
self._rejection_tracker: Optional["OrderRejectionTracker"] = None
100+
101+
def set_rejection_tracker(self, tracker: "OrderRejectionTracker") -> None:
102+
"""Wire a tracker so future rejections are classified and aggregated.
103+
104+
When unset, the legacy ERROR-level path is preserved exactly.
105+
"""
106+
self._rejection_tracker = tracker
107+
108+
def _record_rejection(self, coin: str, error_msg: str, prefix: str = "Order rejected") -> None:
109+
"""Route a rejection through the tracker if registered, else log ERROR.
110+
111+
The tracker handles its own logging at the configured level for
112+
routine matches; for unknown patterns it forwards to ERROR. With
113+
no tracker we preserve the historical ``logger.error`` line.
114+
"""
115+
if self._rejection_tracker is not None:
116+
self._rejection_tracker.record(coin, error_msg)
117+
else:
118+
logger.error(f"{prefix}: {error_msg}")
119+
93120
def _invalidate_open_orders_cache(self) -> None:
94121
"""Clear cached open orders after a write operation (place/cancel)."""
95122
self._open_orders_cache.invalidate()
@@ -392,9 +419,7 @@ def _place_order(self, order: Order) -> Optional[Order]:
392419
return order
393420

394421
if 'error' in status_info:
395-
logger.error(
396-
"Order rejected: %s", status_info['error']
397-
)
422+
self._record_rejection(order.coin, status_info['error'])
398423
order.status = OrderStatus.REJECTED
399424
return None
400425

@@ -495,9 +520,10 @@ def _bulk_place_orders_with_builder(
495520
break
496521

497522
if 'error' in status_info:
498-
logger.error(
499-
"Bulk order [%d] rejected: %s",
500-
i, status_info['error'],
523+
self._record_rejection(
524+
orders[i].coin,
525+
status_info['error'],
526+
prefix=f"Bulk order [{i}] rejected",
501527
)
502528
orders[i].status = OrderStatus.REJECTED
503529
continue

order_rejection_tracker.py

Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
"""Order rejection classification and aggregation.
2+
3+
The bot's order placement path (``order_manager.py``) historically emits
4+
an ERROR-level log for *every* rejection returned by the Hyperliquid
5+
exchange, regardless of whether the rejection signals a routine retry
6+
condition (e.g. post-only orders that would have crossed the BBO) or a
7+
genuine error condition. Under live MM load the routine post-only
8+
rejection accounts for nearly all ERROR records, drowning out real
9+
anomalies in monitoring.
10+
11+
This module reclassifies each rejection by matching the API error text
12+
against a static set of known "routine" patterns. Matched rejections can
13+
be downgraded to WARNING / INFO via config, and the tracker accumulates
14+
per-coin counts that are flushed as a single summary line every
15+
``rejection_summary_interval`` seconds.
16+
17+
The tracker is *opt-in*: ``order_manager.py`` only invokes it when one
18+
has been registered via ``set_rejection_tracker``. With no tracker the
19+
legacy ERROR-level path is preserved exactly.
20+
21+
Public surface:
22+
23+
* :class:`OrderRejectionTracker` — main tracker.
24+
* :func:`classify_rejection` — pure helper used by tests.
25+
26+
Both are intentionally cheap on the order-placement hot path: ``record``
27+
is O(1) under a short-lived lock, and the periodic summary is driven
28+
externally by the strategy's main loop (no background thread).
29+
"""
30+
31+
import logging
32+
import re
33+
import threading
34+
import time
35+
from collections import defaultdict
36+
from dataclasses import dataclass
37+
from typing import Dict, Optional, Tuple
38+
39+
logger = logging.getLogger(__name__)
40+
41+
42+
# Pattern matcher → tag. Patterns are evaluated in order; current entries
43+
# are mutually exclusive on the API's error text format. Adding a new
44+
# routine pattern (e.g. ``Reduce only``) is a one-line edit here plus a
45+
# regression test in ``tests/test_order_rejection_tracker.py``.
46+
_ROUTINE_PATTERNS: Tuple[Tuple["re.Pattern[str]", str], ...] = (
47+
(re.compile(r"Post only order would have immediately matched"),
48+
"post_only_match"),
49+
)
50+
51+
# Mapping from log-level config string to logging level constant. Keys
52+
# are normalised to lowercase before lookup so config values like
53+
# ``"WARNING"`` continue to work.
54+
_LEVEL_MAP: Dict[str, int] = {
55+
"error": logging.ERROR,
56+
"warning": logging.WARNING,
57+
"info": logging.INFO,
58+
"debug": logging.DEBUG,
59+
}
60+
61+
# Allowed config string values, exposed for validators.
62+
ALLOWED_LOG_LEVELS = tuple(_LEVEL_MAP.keys())
63+
64+
# Tag used for unmatched rejection text. Always logged at ERROR so a
65+
# format change on the exchange surfaces immediately.
66+
UNKNOWN_TAG = "unknown"
67+
68+
69+
def classify_rejection(msg: str) -> str:
70+
"""Return the routine tag for *msg*, or :data:`UNKNOWN_TAG` if none match.
71+
72+
Pure function exposed for tests so the pattern map can be exercised
73+
without instantiating a tracker.
74+
"""
75+
for pat, tag in _ROUTINE_PATTERNS:
76+
if pat.search(msg):
77+
return tag
78+
return UNKNOWN_TAG
79+
80+
81+
def _extract_bbo(msg: str) -> str:
82+
"""Best-effort extraction of the ``"bid@ask"`` snippet from a reject text.
83+
84+
Returns an empty string when the pattern is absent (e.g. older HL
85+
error messages or future format changes).
86+
"""
87+
m = re.search(r"bbo was ([\d.]+@[\d.]+)", msg)
88+
return m.group(1) if m else ""
89+
90+
91+
@dataclass
92+
class _CoinStats:
93+
count: int = 0
94+
first_ts: float = 0.0
95+
last_ts: float = 0.0
96+
last_bbo: str = ""
97+
98+
99+
class OrderRejectionTracker:
100+
"""Classify and aggregate order-rejection events.
101+
102+
Public methods are thread-safe. ``record`` is on the order-placement
103+
hot path and intentionally O(1).
104+
"""
105+
106+
def __init__(
107+
self,
108+
routine_log_level: str = "error",
109+
summary_interval: float = 300.0,
110+
) -> None:
111+
self._level: int = _LEVEL_MAP.get(
112+
(routine_log_level or "error").lower(), logging.ERROR
113+
)
114+
self._summary_interval: float = float(summary_interval)
115+
116+
# tag -> coin -> _CoinStats. Reset after each summary flush.
117+
self._stats: Dict[str, Dict[str, _CoinStats]] = defaultdict(
118+
lambda: defaultdict(_CoinStats)
119+
)
120+
self._unknown_count: int = 0
121+
self._lock = threading.Lock()
122+
self._last_summary_ts: float = time.monotonic()
123+
124+
# ------------------------------------------------------------------ #
125+
# Recording
126+
# ------------------------------------------------------------------ #
127+
def record(self, coin: str, raw_msg: str) -> str:
128+
"""Record a rejection. Returns the matched tag, or :data:`UNKNOWN_TAG`.
129+
130+
Emits the per-rejection log at the configured level for routine
131+
matches; unknown patterns are forwarded to ERROR so format
132+
changes / new reject reasons remain visible.
133+
"""
134+
tag = classify_rejection(raw_msg)
135+
bbo = _extract_bbo(raw_msg)
136+
now_wall = time.time()
137+
138+
if tag == UNKNOWN_TAG:
139+
with self._lock:
140+
self._unknown_count += 1
141+
logger.error(f"Order rejected ({coin}): {raw_msg}")
142+
return tag
143+
144+
with self._lock:
145+
stats = self._stats[tag][coin]
146+
if stats.count == 0:
147+
stats.first_ts = now_wall
148+
stats.count += 1
149+
stats.last_ts = now_wall
150+
stats.last_bbo = bbo
151+
152+
# Logger call deliberately outside the lock: the lock guards
153+
# only the in-memory counters.
154+
logger.log(self._level, f"[reject:{tag}] {coin}{raw_msg}")
155+
return tag
156+
157+
# ------------------------------------------------------------------ #
158+
# Periodic summary
159+
# ------------------------------------------------------------------ #
160+
def log_summary_if_due(self, now_monotonic: Optional[float] = None) -> bool:
161+
"""Emit a summary line if the configured interval has elapsed.
162+
163+
Returns ``True`` iff a summary was emitted.
164+
``summary_interval <= 0`` disables the summary entirely.
165+
"""
166+
if self._summary_interval <= 0:
167+
return False
168+
now = now_monotonic if now_monotonic is not None else time.monotonic()
169+
if now - self._last_summary_ts < self._summary_interval:
170+
return False
171+
172+
with self._lock:
173+
snapshot = self._stats
174+
unknown = self._unknown_count
175+
self._stats = defaultdict(lambda: defaultdict(_CoinStats))
176+
self._unknown_count = 0
177+
self._last_summary_ts = now
178+
179+
self._emit_summary(snapshot, unknown)
180+
return True
181+
182+
# ------------------------------------------------------------------ #
183+
# Internal helpers
184+
# ------------------------------------------------------------------ #
185+
def _emit_summary(
186+
self,
187+
snapshot: Dict[str, Dict[str, _CoinStats]],
188+
unknown: int,
189+
) -> None:
190+
any_emitted = False
191+
for tag, by_coin in snapshot.items():
192+
if not by_coin:
193+
continue
194+
total = sum(s.count for s in by_coin.values())
195+
ranked = sorted(
196+
by_coin.items(), key=lambda kv: kv[1].count, reverse=True
197+
)
198+
top = ", ".join(f"{coin}={s.count}" for coin, s in ranked[:8])
199+
sample_bbo = ranked[0][1].last_bbo if ranked else ""
200+
logger.info(
201+
f"[reject-summary] tag={tag} window={self._summary_interval:.0f}s "
202+
f"total={total} coins={len(by_coin)} top=[{top}] "
203+
f"sample_bbo={sample_bbo}"
204+
)
205+
any_emitted = True
206+
if unknown > 0:
207+
logger.warning(
208+
f"[reject-summary] tag=unknown window={self._summary_interval:.0f}s "
209+
f"count={unknown} (unknown patterns logged at ERROR individually)"
210+
)
211+
any_emitted = True
212+
# No-op when both snapshot and unknown are empty: keeps the log
213+
# quiet during idle periods rather than emitting a meaningless
214+
# "total=0" line every interval.
215+
return None if any_emitted else None
216+
217+
# ------------------------------------------------------------------ #
218+
# Operational helpers (read-only)
219+
# ------------------------------------------------------------------ #
220+
def get_stats_snapshot(self) -> Dict[str, Dict[str, int]]:
221+
"""Read-only counter snapshot for tests / external observers.
222+
223+
Does not reset internal state (use :meth:`log_summary_if_due`
224+
for flush semantics).
225+
"""
226+
with self._lock:
227+
return {
228+
tag: {coin: stats.count for coin, stats in by_coin.items()}
229+
for tag, by_coin in self._stats.items()
230+
}
231+
232+
def get_unknown_count(self) -> int:
233+
with self._lock:
234+
return self._unknown_count

0 commit comments

Comments
 (0)