Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 53 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

### 本插件实际表现出来的能力取决于配置的模型。

### 本插件工具调用、对话模型调用模型配置已兼容OpenAI与Anthropic 格式,详情请查看下方 模型服务配置 部分。
### 本插件所有 AI 模型配置(除 embeddingAiConfig 外)均已兼容 OpenAI 与 Anthropic 格式,详情请查看下方 模型服务配置 部分。
---

> [!TIP]
Expand Down Expand Up @@ -862,13 +862,22 @@ const result = await pluginBridge.instance?.enqueueProactiveTask(
## 模型服务配置

### 聊天跟踪判断模型配置 (`trackAiConfig`)

**OpenAI 兼容格式**:
```yaml
trackAiUrl: "https://api.openai.com/v1/chat/completions"
trackAiModel: "gpt-4o-mini"
trackAiApikey: "sk-xxxxx"
```

> 此模型在 `strict` 模式下用作"是否在跟机器人对话"的批量判官;在 `smart` 模式下用作 Timing Gate 子代理(输出 continue/no_action/wait 三选一)。两种用途都只输出极少 token,推荐用 gpt-4o-mini / gemini-2.0-flash / claude haiku 等小模型。
**Anthropic 格式**(自动识别):
```yaml
trackAiUrl: "https://api.anthropic.com/v1/messages"
trackAiModel: "claude-3-5-haiku-20241022"
trackAiApikey: "sk-ant-xxxxx"
```

> **说明**:此模型在 `strict` 模式下用作"是否在跟机器人对话"的批量判官;在 `smart` 模式下用作 Timing Gate 子代理(输出 continue/no_action/wait 三选一)。两种用途都只输出极少 token,推荐用 gpt-4o-mini / gemini-2.0-flash / claude-3-5-haiku 等小模型。插件会根据 URL 自动识别 API 格式。

### 工具调用模型配置 (`toolsAiConfig`)

Expand Down Expand Up @@ -907,37 +916,78 @@ chatApiKey: "sk-ant-xxxxx"
> **说明**:插件会根据 URL 自动识别 API 格式。如果 URL 包含 `/v1/messages`,自动切换为 Anthropic 格式(`max_tokens` 为 16000,并默认开启自适应思考 `thinking: adaptive`;模型不支持思考时会自动去掉该参数重试,不影响使用);否则使用 OpenAI 格式。

### 图像编辑模型配置 (`imageEditAiConfig`)

**OpenAI 兼容格式**:
```yaml
imageEditApiUrl: "https://api.openai.com/v1/chat/completions"
imageEditApiModel: "gemini-3-pro-image-preview"
imageEditApiKey: "sk-xxxxx"
```

**Anthropic 格式**(自动识别):
```yaml
imageEditApiUrl: "https://api.anthropic.com/v1/messages"
imageEditApiModel: "claude-3-5-sonnet-20241022"
imageEditApiKey: "sk-ant-xxxxx"
```

> **说明**:用于图片编辑和文生图功能。插件会根据 URL 自动识别 API 格式,自动处理多模态消息转换(OpenAI `image_url` ↔ Anthropic `image` 格式)。

### 图像识别模型配置 (`analysisAiConfig`)

**OpenAI 兼容格式**:
```yaml
analysisApiUrl: "https://api.openai.com/v1/chat/completions"
analysisApiModel: "gemini-3-pro-preview"
analysisApiKey: "sk-xxxxx"
```

**Anthropic 格式**(自动识别):
```yaml
analysisApiUrl: "https://api.anthropic.com/v1/messages"
analysisApiModel: "claude-3-5-sonnet-20241022"
analysisApiKey: "sk-ant-xxxxx"
```

> **说明**:用于图片识别和分析功能。插件会根据 URL 自动识别 API 格式,自动处理多模态消息转换。

### 联网搜索模型配置 (`searchAiConfig`)

**OpenAI 兼容格式**:
```yaml
searchApiUrl: "https://api.openai.com/v1/chat/completions"
searchApiModel: "deepseek-r1-search"
searchApiKey: "sk-xxxxx"
```

**Anthropic 格式**(自动识别):
```yaml
searchApiUrl: "https://api.anthropic.com/v1/messages"
searchApiModel: "claude-3-5-sonnet-20241022"
searchApiKey: "sk-ant-xxxxx"
```

> **说明**:用于联网搜索工具。插件会根据 URL 自动识别 API 格式。

### 记忆提取模型配置 (`memoryAiConfig`)

> ⚠️ **仅在开启 `memorySystem.enabled: true` 时需要配置**

**OpenAI 兼容格式**:
```yaml
memoryAiUrl: "https://api.openai.com/v1/chat/completions"
memoryAiModel: "gpt-4o-mini" # 推荐使用小模型,省钱且响应快
memoryAiApikey: "sk-xxxxx"
```

**说明**:用户记忆会在对话结束后立即异步调用此模型;群记忆会批量整理后再调用此模型,提取值得长期保存的事实。推荐使用 `gpt-4o-mini`、`gemini-2.0-flash` 等小模型。
**Anthropic 格式**(自动识别):
```yaml
memoryAiUrl: "https://api.anthropic.com/v1/messages"
memoryAiModel: "claude-3-5-haiku-20241022"
memoryAiApikey: "sk-ant-xxxxx"
```

> **说明**:用户记忆会在对话结束后立即异步调用此模型;群记忆会批量整理后再调用此模型,提取值得长期保存的事实。推荐使用 `gpt-4o-mini`、`gemini-2.0-flash`、`claude-3-5-haiku` 等小模型。插件会根据 URL 自动识别 API 格式。

### Embedding 模型配置 (`embeddingAiConfig`)

Expand Down
117 changes: 53 additions & 64 deletions core/conversationTracker.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
// 禁言缓存、回复 debounce、批量"是否在和 bot 说话"判断
// 以 mixin 形式挂到插件原型上,this 指向插件实例。
import { extractChatKeywords, isQuestionMessage, isFeedbackMessage } from "./chatHeuristics.js"
import { callAI } from "../utils/apiClient.js"

// 会话追踪: key: `${groupId}_${userId}`, value: { lastActiveTime, chatHistory: [], timer: null }
// (handleRandomReply / handleTextResponse 也会读写,故导出)
Expand Down Expand Up @@ -844,25 +845,21 @@ ${specialSignalsBlock}
const controller = new AbortController()
const timeoutId = setTimeout(() => controller.abort(), 15000)
try {
const response = await fetch(useCfg.url, {
method: 'POST',
headers: {
'Authorization': `Bearer ${useCfg.apikey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
model: useCfg.model,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt }
],
temperature: 0.3
}),
signal: controller.signal
})
if (!response.ok) return { decision: 'no_action', reason: `http_${response.status}` }
const data = await response.json()
const raw = data?.choices?.[0]?.message?.content?.trim() || ''
const result = await callAI(
useCfg,
[
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt }
],
{
temperature: 0.3,
signal: controller.signal
}
)

if (result.error) return { decision: 'no_action', reason: `api_error:${result.error}` }

const raw = result?.choices?.[0]?.message?.content?.trim() || ''
const jsonMatch = raw.match(/\{[\s\S]*\}/)
if (!jsonMatch) return { decision: 'no_action', reason: 'no_json' }
const parsed = JSON.parse(jsonMatch[0])
Expand Down Expand Up @@ -1029,18 +1026,16 @@ ${specialSignalsBlock}
? chatHistory.map(h => `[${h.role === 'bot' ? '机器人' : '用户'}] ${h.content}`).join('\n')
: '(无历史记录)'

const response = await fetch(this.config.trackAiConfig.trackAiUrl, {
method: 'POST',
headers: {
'Authorization': `Bearer ${this.config.trackAiConfig.trackAiApikey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
const result = await callAI(
{
url: this.config.trackAiConfig.trackAiUrl,
model: this.config.trackAiConfig.trackAiModel,
messages: [
{
role: "system",
content: `你是QQ群聊对话判断助手。机器人名字叫"${botName}",QQ号${Bot.uin}。
apikey: this.config.trackAiConfig.trackAiApikey
},
[
{
role: "system",
content: `你是QQ群聊对话判断助手。机器人名字叫"${botName}",QQ号${Bot.uin}。

根据对话历史,判断用户新消息是否在继续跟机器人对话。

Expand All @@ -1057,19 +1052,17 @@ ${specialSignalsBlock}

你只回复 true 或 false,不要输出其他内容。
`
},
{
role: "user",
content: `【近期对话记录】\n${historyText}\n\n【用户新消息】\n${userMessage}\n\n这条新消息是在跟机器人说话吗?`
}
]
})
})
},
{
role: "user",
content: `【近期对话记录】\n${historyText}\n\n【用户新消息】\n${userMessage}\n\n这条新消息是在跟机器人说话吗?`
}
]
)

if (!response.ok) return false // 请求失败时默认不触发
if (result.error) return false // 请求失败时默认不触发

const data = await response.json()
const answer = data?.choices?.[0]?.message?.content?.toLowerCase()?.trim()
const answer = result?.choices?.[0]?.message?.content?.toLowerCase()?.trim()
// logger.error(answer, historyText, userMessage, 8888)
return answer === 'true' || answer?.includes('true')
} catch (error) {
Expand Down Expand Up @@ -1139,18 +1132,16 @@ ${recentHistory || '(无)'}
---`
}).join('\n\n')

const response = await fetch(this.config.trackAiConfig.trackAiUrl, {
method: 'POST',
headers: {
'Authorization': `Bearer ${this.config.trackAiConfig.trackAiApikey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
const result = await callAI(
{
url: this.config.trackAiConfig.trackAiUrl,
model: this.config.trackAiConfig.trackAiModel,
messages: [
{
role: "system",
content: `你是QQ群聊对话判断助手。机器人名字叫"${botName}"。
apikey: this.config.trackAiConfig.trackAiApikey
},
[
{
role: "system",
content: `你是QQ群聊对话判断助手。机器人名字叫"${botName}"。

每条消息来自不同用户,有独立的对话历史,请分别独立判断。

Expand All @@ -1169,22 +1160,20 @@ ${recentHistory || '(无)'}
返回JSON对象,key为消息ID,value为判断结果。
示例: {"MSG_1_12345": true, "MSG_2_67890": false}
只返回JSON对象,不要其他内容。`
},
{
role: "user",
content: `分别判断以下${batchWithIds.length}条来自不同用户的消息:\n\n${messagesText}\n\n返回JSON对象:`
}
]
})
})
},
{
role: "user",
content: `分别判断以下${batchWithIds.length}条来自不同用户的消息:\n\n${messagesText}\n\n返回JSON对象:`
}
]
)

if (!response.ok) {
logger.error('[批量判断] API请求失败')
if (result.error) {
logger.error('[批量判断] API请求失败:', result.error)
return this.fallbackToSingleJudgment(batch)
}

const data = await response.json()
let content = data?.choices?.[0]?.message?.content?.trim() || '{}'
let content = result?.choices?.[0]?.message?.content?.trim() || '{}'

// 提取JSON对象
const jsonMatch = content.match(/\{[\s\S]*\}/)
Expand Down
26 changes: 14 additions & 12 deletions functions/functions_tools/BananaTool.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { AbstractTool } from './AbstractTool.js';
import { getBase64Image, normalizeImageUrls } from '../../utils/fileUtils.js';
import { dependencies } from "../../dependence/dependencies.js";
import { callAI } from "../../utils/apiClient.js";
import fs from "fs";
import YAML from "yaml";
import path from "path";
Expand Down Expand Up @@ -46,20 +47,21 @@ export class BananaTool extends AbstractTool {
const { imageEditApiUrl: apiUrl, imageEditApiKey: apiKey, imageEditApiModel: model } =
config.imageEditAiConfig || {};

const response = await fetch(apiUrl || 'https://api.openai.com/v1/chat/completions', {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey || 'sk-xxxxxx'}`,
},
body: JSON.stringify({
const result = await callAI(
{
url: apiUrl || 'https://api.openai.com/v1/chat/completions',
model: model || "gemini-3-pro-image-preview",
messages: [{ role: "user", content: imgurls }],
stream: false,
}),
});
apikey: apiKey || 'sk-xxxxxx'
},
[{ role: "user", content: imgurls }],
{ stream: false }
)

if (result.error) {
return { error: `图片生成失败: ${result.error}` }
}

const imageUrl = await this.parseResponse(response);
const imageUrl = result?.choices?.[0]?.message?.content || ''
const processedUrl = this.extractImageUrl(imageUrl);

if (processedUrl) {
Expand Down
30 changes: 15 additions & 15 deletions functions/functions_tools/GoogleAnalysisTool.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { AbstractTool } from './AbstractTool.js';
import { getBase64Image, normalizeImageUrls } from '../../utils/fileUtils.js';
import { dependencies } from "../../dependence/dependencies.js";
import { callAI } from "../../utils/apiClient.js";
const { mimeTypes, axios } = dependencies;
import fs from "fs";
import YAML from "yaml";
Expand Down Expand Up @@ -208,26 +209,25 @@ export class GoogleImageAnalysisTool extends AbstractTool {
const history = [{ role: "user", content: imgurls }];
try {


const apiUrl = config.analysisAiConfig?.analysisApiUrl || 'https://api.openai.com/v1/chat/completions'
const apiKey = config.analysisAiConfig?.analysisApiKey || 'sk-xxxxxx'
const apiModel = config.analysisAiConfig?.analysisApiModel || "gemini-3-pro-image-preview"

const requestData = {
model: config.analysisAiConfig?.analysisApiModel || "gemini-3-pro-image-preview",
messages: history,
stream: false
}

const response = await fetch(apiUrl, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
const result = await callAI(
{
url: apiUrl,
model: apiModel,
apikey: apiKey
},
body: JSON.stringify(requestData),
})
history,
{ stream: false }
)

if (result.error) {
return { error: `图片分析失败: ${result.error}` }
}

const content = await this.parseResponse(response)
const content = result?.choices?.[0]?.message?.content || '无法分析图片'
return {
analysis: content
};
Expand Down
Loading
Loading