Skip to content

Commit a4054d1

Browse files
committed
feat(v0.1.9): agent-demo 加 PRICING + cost 追踪 · runner 扩 8 模型
session: 64fadcea-fb38-44be-8d30-a1cda6c9004c What: - scripts/agent-demo.py: 加 PRICING 表(14 模型 per 1M token 单价 · 2026-04 公开估)· agent_loop 累加 usage.prompt_tokens/completion_tokens · 最后打印 __NF_STATS__ JSON(model/iters/tokens/est_cost_usd) - scripts/multi-model-runner.sh: MODELS 从 4 扩 8(加 DeepSeek V3.1-Terminus / GLM-5 / Qwen3.5-35B / Qwen3-Coder 免费)· summary.md 加 3 列(iters / tokens in/out / cost_usd)· 用 jq parse __NF_STATS__ Why: 用户"记录各模型调用成本 · 多找多试 · 看谁家执行成本最低" · e2e 8/8 通过 · 成本排名:Qwen3-Coder 免费 > DeepSeek V3.1 $0.00287 > DeepSeek V3.2 $0.00326 > Qwen3.5 $0.00394 > Qwen3.6 $0.00552 > MiniMax $0.01137 > GLM-5.1 $0.01238 > GLM-5 $0.01398. 付费最便宜 DeepSeek · 贵的 MiniMax/GLM 5-10x. Context: - 单任务(TTS+ffprobe+HTML+ls 4 步) 所有 8 模型都 5 iters 完成 · 质量一致(mp3 6.91s + HTML 合规 + 自填模型名) - 总耗时 61 秒(并发)· 总 API cost ~$0.056 - e2e 产物在 tmp/multi-model-poc/{slug}/outputs/ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent fa8f273 commit a4054d1

2 files changed

Lines changed: 64 additions & 8 deletions

File tree

scripts/agent-demo.py

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,29 @@
1313
from typing import Any
1414

1515

16+
PRICING: dict[str, tuple[float, float]] = {
17+
"Pro/MiniMaxAI/MiniMax-M2.5": (0.80, 2.40),
18+
"deepseek-ai/DeepSeek-V3.2": (0.28, 0.42),
19+
"Pro/deepseek-ai/DeepSeek-V3.2": (0.28, 0.42),
20+
"deepseek-ai/DeepSeek-V3.1-Terminus": (0.28, 0.42),
21+
"deepseek-ai/DeepSeek-R1": (0.55, 2.19),
22+
"Pro/zai-org/GLM-5.1": (1.00, 3.00),
23+
"Pro/zai-org/GLM-5": (1.00, 3.00),
24+
"Pro/zai-org/GLM-4.7": (0.80, 2.40),
25+
"zai-org/GLM-4.6": (0.50, 1.50),
26+
"Qwen/Qwen3.6-35B-A3B": (0.42, 0.84),
27+
"Qwen/Qwen3.5-397B-A17B": (1.50, 3.00),
28+
"Qwen/Qwen3.5-35B-A3B": (0.35, 0.70),
29+
"Qwen/Qwen3.5-9B": (0.0, 0.0),
30+
"Qwen/Qwen3-Coder-30B-A3B-Instruct": (0.0, 0.0),
31+
}
32+
33+
34+
def estimate_cost(model: str, in_tok: int, out_tok: int) -> float:
35+
p_in, p_out = PRICING.get(model, (0.0, 0.0))
36+
return round((in_tok / 1e6) * p_in + (out_tok / 1e6) * p_out, 6)
37+
38+
1639
TOOLS: list[dict[str, Any]] = [
1740
{"type": "function", "function": {"name": "Bash", "description": "Run any shell command. Use for ls / cat / ffprobe / file 等查询. Returns stdout+stderr+exit code.", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "shell command to run"}, "timeout_sec": {"type": "integer", "default": 30}}, "required": ["command"]}}},
1841
{"type": "function", "function": {"name": "Read", "description": "Read file content. For text files (md/json/txt).", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "max_chars": {"type": "integer", "default": 10000}}, "required": ["path"]}}},
@@ -100,21 +123,41 @@ def agent_loop(task: str, max_iters: int = 10) -> str | None:
100123
client = make_client()
101124
messages: list[dict[str, Any]] = [{"role": "user", "content": task}]
102125
model = os.getenv("SILICONFLOW_MODEL", "Pro/MiniMaxAI/MiniMax-M2.5")
126+
total_in = 0
127+
total_out = 0
128+
iters = 0
129+
final: str | None = None
103130
for _ in range(max_iters):
131+
iters += 1
104132
resp = client.chat.completions.create(model=model, messages=messages, tools=TOOLS, tool_choice="auto")
133+
if resp.usage:
134+
total_in += resp.usage.prompt_tokens or 0
135+
total_out += resp.usage.completion_tokens or 0
105136
msg = resp.choices[0].message
106137
messages.append(msg.model_dump(exclude_none=True))
107138
if not msg.tool_calls:
108139
print("FINAL:", msg.content)
109-
return msg.content
140+
final = msg.content
141+
break
110142
for tc in msg.tool_calls:
111143
args = json.loads(tc.function.arguments or "{}")
112144
print(f"→ tool: {tc.function.name}({args})")
113145
result = dispatch(tc.function.name, args)
114146
print(f"← result: {json.dumps(result, ensure_ascii=False)[:200]}")
115147
messages.append({"role": "tool", "tool_call_id": tc.id, "content": json.dumps(result, ensure_ascii=False)})
116-
print("max iters reached")
117-
return None
148+
else:
149+
print("max iters reached")
150+
stats = {
151+
"model": model,
152+
"iters": iters,
153+
"prompt_tokens": total_in,
154+
"completion_tokens": total_out,
155+
"completed": final is not None,
156+
"est_cost_usd": estimate_cost(model, total_in, total_out),
157+
"note": "硅基实际价格可能不同 · est 按公开 2026-04 数据估算",
158+
}
159+
print(f"__NF_STATS__ {json.dumps(stats, ensure_ascii=False)}")
160+
return final
118161

119162

120163
def dry_run() -> int:

scripts/multi-model-runner.sh

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,12 @@ mkdir -p "$POC_ROOT"
1313
MODELS=(
1414
"Pro/MiniMaxAI/MiniMax-M2.5"
1515
"deepseek-ai/DeepSeek-V3.2"
16-
"Pro/zai-org/GLM-4.7"
17-
"Qwen/Qwen3-30B-A3B"
16+
"deepseek-ai/DeepSeek-V3.1-Terminus"
17+
"Pro/zai-org/GLM-5.1"
18+
"Pro/zai-org/GLM-5"
19+
"Qwen/Qwen3.6-35B-A3B"
20+
"Qwen/Qwen3.5-35B-A3B"
21+
"Qwen/Qwen3-Coder-30B-A3B-Instruct"
1822
)
1923

2024
if [ -f "$WT/.env" ]; then
@@ -87,8 +91,8 @@ SUMMARY="$POC_ROOT/summary.md"
8791
echo ""
8892
echo "## 结果对比"
8993
echo ""
90-
echo "| 模型 | 产物数 | hello.mp3 时长 | HTML 存在 | exit code |"
91-
echo "|---|---|---|---|---|"
94+
echo "| 模型 | 产物 | mp3时长 | HTML | iters | tokens in/out | 成本\$ | exit |"
95+
echo "|---|---|---|---|---|---|---|---|"
9296
for model in "${MODELS[@]}"; do
9397
slug="${model//\//-}"
9498
ws="$POC_ROOT/$slug"
@@ -101,7 +105,16 @@ SUMMARY="$POC_ROOT/summary.md"
101105
html_ok="NO"
102106
[ -f "$ws/outputs/hello.html" ] && html_ok="YES"
103107
code=$(grep "=== runner exit code:" "$ws/run.log" 2>/dev/null | grep -oE '[0-9]+$' | head -1)
104-
echo "| \`$model\` | $n | $mp3_dur | $html_ok | ${code:-?} |"
108+
stats_line=$(grep "__NF_STATS__" "$ws/run.log" 2>/dev/null | tail -1 | sed 's|^__NF_STATS__ ||')
109+
if [ -n "$stats_line" ]; then
110+
iters=$(echo "$stats_line" | jq -r '.iters // "-"')
111+
tin=$(echo "$stats_line" | jq -r '.prompt_tokens // 0')
112+
tout=$(echo "$stats_line" | jq -r '.completion_tokens // 0')
113+
cost=$(echo "$stats_line" | jq -r '.est_cost_usd // 0')
114+
else
115+
iters="-"; tin="-"; tout="-"; cost="-"
116+
fi
117+
echo "| \`$model\` | $n | $mp3_dur | $html_ok | $iters | ${tin}/${tout} | $cost | ${code:-?} |"
105118
done
106119
echo ""
107120
echo "## 每模型详情"

0 commit comments

Comments
 (0)