Skip to content

Commit 10ba89c

Browse files
committed
feat(ai): add shadow benchmark support for alternate Mammouth models
1 parent 98493cb commit 10ba89c

3 files changed

Lines changed: 213 additions & 56 deletions

File tree

.env.example

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,18 @@ AI_COPILOT_REENTRY_COOLDOWN_MIN=300
150150
AI_COPILOT_REENTRY_COOLDOWN_MAX=1200
151151
AI_COPILOT_TABULAR_BUY_CONF_MIN=0.50
152152
AI_COPILOT_TABULAR_BUY_CONF_MAX=0.65
153+
AI_COPILOT_BENCHMARK_ENABLED=false
154+
AI_COPILOT_BENCHMARK_MODEL=
155+
AI_COPILOT_BENCHMARK_INTERVAL_MINUTES=62
156+
AI_COPILOT_BENCHMARK_STATE_FILE=ai_copilot_benchmark_state.json
157+
AI_COPILOT_BENCHMARK_MAX_CALLS_PER_DAY=24
158+
AI_COPILOT_BENCHMARK_MAX_CALLS_PER_MONTH=720
159+
AI_COPILOT_BENCHMARK_MAX_BUDGET_USD_PER_MONTH=1.0
160+
AI_COPILOT_BENCHMARK_MAX_OUTPUT_TOKENS=300
161+
AI_COPILOT_BENCHMARK_TEMPERATURE=0.1
162+
AI_COPILOT_BENCHMARK_MAX_CONSECUTIVE_ERRORS=3
163+
AI_COPILOT_BENCHMARK_COST_INPUT_PER_MTOK=0.2
164+
AI_COPILOT_BENCHMARK_COST_OUTPUT_PER_MTOK=1.25
153165

154166
# AutoResearch bridge – Fear & Greed Index fetcher (no API key required)
155167
# The fetch_research_signal.py script is run by the research-signal.timer on

.env.live.example

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,18 @@ AI_COPILOT_REENTRY_COOLDOWN_MIN=600
137137
AI_COPILOT_REENTRY_COOLDOWN_MAX=1800
138138
AI_COPILOT_TABULAR_BUY_CONF_MIN=0.55
139139
AI_COPILOT_TABULAR_BUY_CONF_MAX=0.65
140+
AI_COPILOT_BENCHMARK_ENABLED=false
141+
AI_COPILOT_BENCHMARK_MODEL=
142+
AI_COPILOT_BENCHMARK_INTERVAL_MINUTES=62
143+
AI_COPILOT_BENCHMARK_STATE_FILE=ai_copilot_benchmark_state.json
144+
AI_COPILOT_BENCHMARK_MAX_CALLS_PER_DAY=24
145+
AI_COPILOT_BENCHMARK_MAX_CALLS_PER_MONTH=720
146+
AI_COPILOT_BENCHMARK_MAX_BUDGET_USD_PER_MONTH=1.0
147+
AI_COPILOT_BENCHMARK_MAX_OUTPUT_TOKENS=250
148+
AI_COPILOT_BENCHMARK_TEMPERATURE=0.1
149+
AI_COPILOT_BENCHMARK_MAX_CONSECUTIVE_ERRORS=3
150+
AI_COPILOT_BENCHMARK_COST_INPUT_PER_MTOK=0.2
151+
AI_COPILOT_BENCHMARK_COST_OUTPUT_PER_MTOK=1.25
140152

141153
# AutoResearch bridge (conservative fail-open with neutral fallback)
142154
TABULAR_RESEARCH_SIGNAL_PATH=./data/research_signal_latest.json

0 commit comments

Comments
 (0)