feat: 升级至 v0.3.15 — MiMo 多模态 + 工作空间数据库继承 + 引导窗口上下文配置 + TS 错误修复
- feat(mimo): 适配器支持图片输入,将 images 转为 OpenAI 兼容 content parts 数组 - feat(workspace): 切换工作空间时支持继承数据库 agent.db(SQLite backup API 原子导出,自动 checkpoint WAL) - feat(onboarding): 引导弹窗 LLM 配置加上下文长度字段(DeepSeek/Agnes=1000000、MiMo=131072、Ollama=null) - fix(types): 修复 16 个 TypeScript 编译错误(setConfigLoaded 接口声明、10 个 IPC 返回类型加 error 字段、MUI TextField readOnly slot 迁移) - chore(workspace): 移除死代码 traces 目录(trace 数据实际存储于数据库 sessions.metadata 字段,traces 目录从未被任何代码读写) - docs(mimo): API 文档补充多模态输入章节和 curl/Python 代码示例 - chore: 版本号 0.3.14 → 0.3.15
This commit is contained in:
@@ -132,6 +132,7 @@
|
||||
<a href="#tools">🔧 工具调用 (Function Calling)</a>
|
||||
<a href="#websearch">🌐 联网搜索</a>
|
||||
<a href="#tts">🎙️ 语音合成 (TTS)</a>
|
||||
<a href="#multimodal">🖼️ 多模态输入</a>
|
||||
<a href="#examples">💻 代码示例</a>
|
||||
<a href="#errors">⚠️ 错误码说明</a>
|
||||
</div>
|
||||
@@ -331,6 +332,56 @@ Content-Type: application/json</pre>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ===== MULTIMODAL ===== -->
|
||||
<section id="multimodal">
|
||||
<h2>🖼️ 多模态输入</h2>
|
||||
<p>当 <code>messages[].content</code> 为数组时,可混合传入文本、图像内容。采用 OpenAI 兼容的 content parts 数组格式。</p>
|
||||
|
||||
<h3>content parts 字段</h3>
|
||||
<table>
|
||||
<tr><th width="200">字段名</th><th width="90">类型</th><th>描述</th></tr>
|
||||
<tr><td><code>content</code></td><td><span class="type-tag">string | array</span></td><td>纯文本字符串,或 content parts 数组</td></tr>
|
||||
<tr><td><code>content[].type</code></td><td><span class="type-tag">string</span></td><td>part 类型:<code>"text"</code> / <code>"image_url"</code></td></tr>
|
||||
<tr><td><code>content[].text</code></td><td><span class="type-tag">string</span></td><td>文本内容(type 为 <code>"text"</code> 时必填)</td></tr>
|
||||
<tr><td><code>content[].image_url</code></td><td><span class="type-tag">object</span></td><td>图像对象(type 为 <code>"image_url"</code> 时必填)</td></tr>
|
||||
<tr><td><code>content[].image_url.url</code></td><td><span class="type-tag">string</span></td><td>图像公网 URL 或 base64 data URI(格式:<code>data:image/png;base64,...</code>)</td></tr>
|
||||
<tr><td><code>content[].image_url.detail</code></td><td><span class="type-tag">string</span></td><td>可选,清晰度:<code>"low"</code> / <code>"high"</code> / <code>"auto"</code>(默认 <code>"auto"</code>)</td></tr>
|
||||
</table>
|
||||
|
||||
<div class="info-box">
|
||||
<strong>📌 支持的图像格式:</strong>
|
||||
<ul style="padding-left:20px;margin:8px 0 0;">
|
||||
<li><strong>URL</strong>:公网可访问的 HTTPS 图片地址</li>
|
||||
<li><strong>Base64 data URI</strong>:<code>data:image/{format};base64,{base64_data}</code>,支持 png / jpeg / gif / webp</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h3>多模态消息示例</h3>
|
||||
<pre><span class="code-lang">json</span>{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{ "type": "text", "text": "请描述这张图片的内容" },
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "https://example.com/image.jpg",
|
||||
"detail": "auto"
|
||||
}
|
||||
}
|
||||
]
|
||||
}</pre>
|
||||
|
||||
<div class="warn-box">
|
||||
<strong>⚠️ 多模态限制:</strong>
|
||||
<ul style="padding-left:20px;margin:8px 0 0;">
|
||||
<li>仅 <code>mimo-v2.5-pro</code> / <code>mimo-v2.5</code> 文本模型支持图像输入</li>
|
||||
<li>TTS 系列模型(<code>mimo-v2.5-tts*</code>)不支持多模态图像输入</li>
|
||||
<li>历史消息中的图片 content 数组应原样保留,不建议在多轮对话中删除</li>
|
||||
<li>base64 data URI 会显著增加请求体大小,建议对大图先压缩或使用公网 URL</li>
|
||||
</ul>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ===== RESPONSE ===== -->
|
||||
<section id="response">
|
||||
<h2>📤 响应对象(非流式输出)</h2>
|
||||
@@ -746,6 +797,100 @@ audio_bytes = base64.b64decode(audio_data)
|
||||
with output("output.wav", "wb") as f:
|
||||
f.write(audio_bytes)
|
||||
print("音频已保存到 output.wav")</pre>
|
||||
|
||||
<h3>多模态图像输入</h3>
|
||||
<div class="tabs">
|
||||
<button class="tab-btn active" onclick="switchTab(event,'tab-mm-curl')">curl</button>
|
||||
<button class="tab-btn" onclick="switchTab(event,'tab-mm-python')">Python (OpenAI SDK)</button>
|
||||
<button class="tab-btn" onclick="switchTab(event,'tab-mm-base64')">Python (Base64 本地图片)</button>
|
||||
</div>
|
||||
|
||||
<div id="tab-mm-curl" class="tab-content active">
|
||||
<pre><span class="code-lang">bash</span>curl --location --request POST 'https://api.xiaomimimo.com/v1/chat/completions' \
|
||||
--header "api-key: $MIMO_API_KEY" \
|
||||
--header "Content-Type: application/json" \
|
||||
--data-raw '{
|
||||
"model": "mimo-v2.5-pro",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "请描述这张图片的内容"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "https://example.com/image.jpg",
|
||||
"detail": "auto"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"max_completion_tokens": 1024,
|
||||
"thinking": {"type": "disabled"}
|
||||
}'</pre>
|
||||
</div>
|
||||
|
||||
<div id="tab-mm-python" class="tab-content">
|
||||
<pre><span class="code-lang">python</span>from openai import OpenAI
|
||||
|
||||
client = OpenAI(api_key="$MIMO_API_KEY", base_url="https://api.xiaomimimo.com/v1")
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="mimo-v2.5-pro",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "请描述这张图片的内容"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "https://example.com/image.jpg",
|
||||
"detail": "auto"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
max_completion_tokens=1024,
|
||||
thinking={"type": "disabled"}
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
print(f"图像 token: {response.usage.prompt_tokens_details.image_tokens}")</pre>
|
||||
</div>
|
||||
|
||||
<div id="tab-mm-base64" class="tab-content">
|
||||
<pre><span class="code-lang">python</span>import base64
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(api_key="$MIMO_API_KEY", base_url="https://api.xiaomimimo.com/v1")
|
||||
|
||||
# 读取本地图片并转为 base64 data URI
|
||||
with open("local_image.jpg", "rb") as f:
|
||||
image_data = base64.b64encode(f.read()).decode("utf-8")
|
||||
image_data_uri = f"data:image/jpeg;base64,{image_data}"
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="mimo-v2.5-pro",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "这张图里有什么?"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data_uri}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
max_completion_tokens=1024
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)</pre>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ===== ERRORS ===== -->
|
||||
|
||||
Reference in New Issue
Block a user