feat: v0.7.4 时序语义修正 · 防线实效补漏 · 全量测试翻倍 — 2406 用例 + jsdom 组件测试全量回归
CI / 类型检查 + Lint + 单元测试 (push) Failing after 6m27s
CI / 产物编译验证 (push) Successful in 9m57s
CI / 全量测试 (Electron ABI) (push) Failing after 5m19s

P1 修复面收口:
- 超时三态区分(aborted→USER_INTERRUPT / ETIMEDOUT→TIMEOUT / 其余→ERROR),
  根治"真实网络超时被误报为用户中断"
- 流空闲超时统一(SSE/Ollama/Anthropic 读循环 60s 无数据抛 504 进重试通道)
- 同会话并发 sendMessage 防重入(isRunning 守卫)+ 会话存在性预检 +
  前置调用移入 try(ERROR+DONE 双事件保证,根治 isStreaming 假死)
- 清空审计后 resetChainCache(根治 verifyChain 误报 TAMPERED)
- DONE 不再提前清理 TRACE(TERMINATED 统一收尾,补全最终迭代录制)
- IME 合成回车不发送(普通 Enter + Cmd/Ctrl+Enter 双分支)+ handleSend 闭包修复

P2 安全纵深:
- preload 移除原始 electronAPI 暴露(渲染层零使用,关掉 XSS invoke 任意通道单点风险)
- CORS 同源回显根治(仅当前浏览页面 Origin,did-navigate 同步)
- MEMORY.md 命令保护正则扩展(括号/$/反引号/< 重定向边界 + 前导路径)
- write_file append TOCTOU 统一(open 后 realpath 校验,新文件分支补漏)
- 敏感键归一化(authKey 驼峰/连字符命中)+ MCP headers 鉴权值加密落库
- ReDoS 检测共享化(search_files/file_editor 统一拦截)
- run_tests/lint_code 升风险 + 需确认 + npx --no-install(执行边界对齐 run_command)
- MCP/SearXNG/llm.baseURL/updateFeedUrl 配置类 URL 高危目标校验(IPv6 去括号 +
  十六进制映射解析 + 尾点剥离)

P3 架构还债:
- temperature/maxTokens 热生效(引擎/编排器/SubAgent 三处接线)+ setBatch 单事务落盘
- SessionRecorder flush 竞态根治(flushPromise 等待 + 超限内联落盘 + stopRecording async)
- 内存收口(lastConsolidationBySession LRU / subTraces 清理 / 会话删除 disposeEngine)
- i18n 全量收口(28 组件 + 353 key 双字典,状态标签改渲染时函数)
- 死代码清理(updateTraceStep/HEADER_HEIGHT/void preA/失实注释)
- 斜杠菜单 MUI 化 + 删除逻辑收敛 resetSessionState + Blob URL 统一释放 +
  用户消息"仅保存"落库(saveMessage 透传前端 id 修复 id 错位)

P4 能力演进:
- 死循环检测拆分(驻留前置 + 乒乓后置带进度信号,合法交替不误报)
- run-lock 30s 超时强制 abort(旧 run 卡死不无限排队)
- RETRY 双通道 stream_reset(前端按 run 归属精确清空,根治重试文本重复)
- FTS5 trigram 中文子串搜索(迁移 9 版本化 SCHEMA_VERSION=2,≤2 字符 LIKE 回退)
- getContextWindow 兜底 1M→128K(未知模型防 413)

测试:
- 855 → 2406 用例(+1551,2.8 倍):服务层 +325(含 MemoryManager 51 新用例)、
  工具实体 +483、IPC/适配器 +390(含 OpenAI/Anthropic/Ollama 独立套件)、
  纯函数表格化 +330;引入 jsdom + @testing-library(14 组件测试文件 249 用例)
- 修复 R1(saveMessage id 透传)/ R2(stream_reset 精确归属)两个回归缺陷
- 遗留低危项清零:git-tools 顺序耦合 / web-fetch 真实时间退避 / slo 内存断言 /
  mcp-security 多余 skipIf / deepseek-balance 命名误导 / 组件 mock 注入脆弱性

版本: 0.7.4; README 同步(工具风险表/版本徽章); 依赖: 移除 @electron-toolkit/preload,
新增 jsdom/@testing-library(devDependencies 不打包)

回归: typecheck 双端 0 错误; ESLint 0/0; Electron ABI 全量 2406/2406 零跳过;
系统 Node 2110 通过 296 跳过(better-sqlite3 ABI)
This commit is contained in:
2026-08-30 19:19:07 +08:00
parent ebe45482b0
commit 99d0c54129
137 changed files with 25190 additions and 1792 deletions
@@ -1,9 +1,9 @@
/**
* network-utils 纯函数层契约测试(v0.7.0 覆盖补齐)
* network-utils 纯函数层契约测试(v0.7.0 覆盖补齐 → v0.7.5 大幅扩充
*
* 此前该共享模块(UA 轮换 / 反爬头 / URL 归一化 / 拦截页特征 / 正文提取 /
* 流式限读 / SearXNG 认证头 / 双 LRU 缓存)只有 web_fetch/web_search 间接触达,
* 直接行为契约零锁定。本文件逐一钉死。
* 共享模块(UA 轮换 / 语言轮换 / 反爬头 / URL 归一化 / 拦截页特征 / 正文提取 /
* 流式限读 / SearXNG 认证头 / CORS / Origin 提取 / HTML→Markdown / 双 LRU 缓存)
* 全部行为契约逐一钉死。
*/
import { describe, it, expect, vi, afterEach } from 'vitest';
@@ -17,12 +17,17 @@ import {
fetchCache,
UA_POOL,
MOBILE_UA,
ACCEPT_LANGUAGE_POOL,
buildAntiCrawlHeaders,
normalizeUrl,
isInterceptedPage,
htmlToText,
readBodyWithLimit,
buildSearXNGAuthHeaders,
htmlToMarkdown,
extractOriginHeader,
corsAllowOrigin,
fetchWithTimeout,
} from '../network-utils';
describe('normalizeUrl — 去重键归一化', () => {
@@ -44,6 +49,33 @@ describe('normalizeUrl — 去重键归一化', () => {
])('%s → %s', (input, expected) => {
expect(normalizeUrl(input)).toBe(expected);
});
it('非默认端口保留', () => {
expect(normalizeUrl('http://a.com:8080/x')).toBe('http://a.com:8080/x');
expect(normalizeUrl('https://a.com:8443/x')).toBe('https://a.com:8443/x');
});
it('ws/wss 默认端口剥离', () => {
expect(normalizeUrl('ws://a.com:80/socket')).toBe('ws://a.com/socket');
expect(normalizeUrl('wss://a.com:443/socket')).toBe('wss://a.com/socket');
});
it('fragment 保留', () => {
expect(normalizeUrl('https://a.com/x?q=1#sec')).toBe('https://a.com/x?q=1#sec');
});
it('非法 URL 原样返回', () => {
expect(normalizeUrl('not-a-url')).toBe('not-a-url');
expect(normalizeUrl('')).toBe('');
});
it('根路径(pathname=/)保留尾斜杠', () => {
expect(normalizeUrl('https://a.com/')).toBe('https://a.com/');
});
it('端口大小写 host 归一同时生效', () => {
expect(normalizeUrl('HTTP://A.COM:80/X')).toBe('http://a.com/X');
});
});
describe('isInterceptedPage — 反爬/验证码拦截特征', () => {
@@ -59,10 +91,32 @@ describe('isInterceptedPage — 反爬/验证码拦截特征', () => {
});
it('正常正文不误报;超短正文触发空壳判定', () => {
const normal = '<html><body>' + '<p>'.repeat(0) + '<article>' + 'x'.repeat(2000) + '</article></body></html>';
const normal =
'<html><body>' +
'<p>'.repeat(0) +
'<article>' +
'x'.repeat(2000) +
'</article></body></html>';
expect(isInterceptedPage(normal)).toBe(false);
expect(isInterceptedPage('<html><body>hi</body></html>')).toBe(true); // <80 字符空壳
});
it('just a moment / DDoS protection 特征', () => {
expect(isInterceptedPage('<div>Just a moment...</div><script>challenge</script>')).toBe(true);
expect(isInterceptedPage('<title>DDoS protection by Cloudflare</title>')).toBe(true);
});
it('challenge-platform / cf-challenge 特征', () => {
expect(
isInterceptedPage('<script src="/cdn-cgi/challenge-platform/h/b/orchestrate/"></script>'),
).toBe(true);
});
it('恰好 80 字符的非拦截正文不误报', () => {
const exact80 = 'x'.repeat(80);
expect(isInterceptedPage(exact80)).toBe(false);
expect(isInterceptedPage('x'.repeat(79))).toBe(true); // <80 空壳
});
});
describe('htmlToText — HTML→纯文本管线', () => {
@@ -78,6 +132,108 @@ describe('htmlToText — HTML→纯文本管线', () => {
expect(text).toContain('第一段 & 符号');
expect(text).toContain('\n'); // 块级元素产生换行
});
it('nav/header/footer/aside/iframe/svg 整体剔除', () => {
const text = htmlToText(
'<nav>导航</nav><header>页头</header><footer>页脚</footer><aside>侧栏</aside>' +
'<iframe src="x">iframe 内容</iframe><svg><text>svg 文本</text></svg>正文',
);
expect(text).not.toContain('导航');
expect(text).not.toContain('页头');
expect(text).not.toContain('页脚');
expect(text).not.toContain('侧栏');
expect(text).not.toContain('iframe');
expect(text).not.toContain('svg 文本');
expect(text).toContain('正文');
});
it('HTML 注释被剔除', () => {
const text = htmlToText('<!-- 隐藏注释 -->可见');
expect(text).not.toContain('隐藏注释');
expect(text).toContain('可见');
});
it('表格单元格转制表符后由空白折叠为单空格(td/th → tab → 空格)', () => {
const text = htmlToText(
'<table><tr><th>头A</th><th>头B</th></tr><tr><td>v1</td><td>v2</td></tr></table>',
);
// 实况契约:td/th 先转 \t,末尾 [ \t]+ 折叠为单空格
expect(text).toContain('头A 头B');
expect(text).toContain('v1 v2');
});
it('数字/十六进制实体解码', () => {
const text = htmlToText('<p>&#65;&#x42;</p>');
expect(text).toContain('AB');
});
it('符号实体解码(nbsp/lt/gt/quot/apos/hellip 等)', () => {
const text = htmlToText('<p>a&nbsp;b &lt;c&gt; &quot;q&quot; &apos;x&apos; &hellip;</p>');
expect(text).toContain('a b <c> "q"');
expect(text).toContain('…');
});
it('连续换行折叠(3+ → 2', () => {
const text = htmlToText('<p>a</p><p>b</p><p>c</p>');
expect(text).not.toContain('\n\n\n');
});
it('br/hr 也产生换行', () => {
const text = htmlToText('a<br>b<hr>c');
expect(text.split('\n').length).toBeGreaterThanOrEqual(2);
});
it('空输入与纯标签输入', () => {
expect(htmlToText('')).toBe('');
expect(htmlToText('<div><span></span></div>')).toBe('');
});
});
describe('htmlToMarkdown — HTML→Markdown 结构化转换(v0.6.4 P4-4', () => {
it('h1-h6 输出 ATX 标题', () => {
expect(htmlToMarkdown('<h1>一级</h1>')).toContain('# 一级');
expect(htmlToMarkdown('<h2>二级</h2>')).toContain('## 二级');
expect(htmlToMarkdown('<h6>六级</h6>')).toContain('###### 六级');
});
it('段落 / 链接 / 强调 / 行内代码', () => {
const md = htmlToMarkdown(
'<p>看 <a href="https://x.test">链接</a> 和 <strong>粗</strong> <code>code</code></p>',
);
expect(md).toContain('[链接](https://x.test)');
expect(md).toContain('**粗**');
expect(md).toContain('`code`');
});
it('pre 围栏代码块与 ul/ol 列表', () => {
const md = htmlToMarkdown(
'<pre><code>const x = 1;</code></pre><ul><li>甲</li><li>乙</li></ul>',
);
expect(md).toContain('```');
expect(md).toContain('- 甲');
expect(md).toContain('- 乙');
});
it('blockquote 与 hr', () => {
const md = htmlToMarkdown('<blockquote>引用</blockquote><hr>');
expect(md).toContain('> 引用');
expect(md).toContain('---');
});
it('script/style/svg/noscript/iframe 整体剔除', () => {
const md = htmlToMarkdown(
'<script>evil()</script><style>.x{}</style><svg><text>t</text></svg>正文',
);
expect(md).not.toContain('evil');
expect(md).not.toContain('.x');
expect(md).not.toContain('t');
expect(md).toContain('正文');
});
it('空输入返回空串', () => {
expect(htmlToMarkdown('')).toBe('');
expect(htmlToMarkdown(' ')).toBe('');
});
});
describe('readBodyWithLimit — 流式硬上限', () => {
@@ -115,6 +271,37 @@ describe('readBodyWithLimit — 流式硬上限', () => {
/Response too large/,
);
});
it('content-length 虚报偏小(真实流量超限)→ 流式累计超限抛错', async () => {
const response = new Response(streamOf(['y'.repeat(900), 'z'.repeat(900)]), {
headers: { 'Content-Length': String(500) }, // 虚报:预检通过,流式读取时超限
});
await expect(readBodyWithLimit(response as unknown as Response, 1000)).rejects.toThrow(
/bytes limit/,
);
});
it('无 body 的响应(null body)→ 返回空串', async () => {
const response = new Response(null);
const body = await readBodyWithLimit(response as unknown as Response);
expect(body).toBe('');
});
it('非 UTF-8 字节以替换字符容错解码(fatal:false', async () => {
const enc = new TextEncoder();
const bad = new Uint8Array([0x48, 0x69, 0xff, 0xfe, 0x21]); // Hi + 非法字节 + !
const response = new Response(
new ReadableStream<Uint8Array>({
start(c) {
c.enqueue(enc.encode(''));
c.enqueue(bad);
c.close();
},
}),
);
const body = await readBodyWithLimit(response as unknown as Response);
expect(body).toContain('Hi');
});
});
describe('buildAntiCrawlHeaders — UA 轮换与移动端分支', () => {
@@ -128,6 +315,20 @@ describe('buildAntiCrawlHeaders — UA 轮换与移动端分支', () => {
}
});
it('UA 轮换取模:attempt=UA_POOL.length 回到首个 UA', () => {
const h0 = buildAntiCrawlHeaders('https://t.test/', 0, false);
const hN = buildAntiCrawlHeaders('https://t.test/', UA_POOL.length, false);
expect(hN['User-Agent']).toBe(h0['User-Agent']);
});
it('语言头随 attempt 轮换(Accept-Language 池)', () => {
const h0 = buildAntiCrawlHeaders('https://t.test/', 0, false);
const h1 = buildAntiCrawlHeaders('https://t.test/', 1, false);
expect(ACCEPT_LANGUAGE_POOL).toContain(h0['Accept-Language']);
expect(ACCEPT_LANGUAGE_POOL).toContain(h1['Accept-Language']);
expect(h0['Accept-Language']).not.toBe(h1['Accept-Language']);
});
it('mobile_ua=true 时固定使用移动 UA,并携带 Sec-Fetch/语言族反爬头', () => {
const h = buildAntiCrawlHeaders('https://t.test/x?lang=zh', 0, true);
const entries = Object.entries(h).map(([k, v]) => [k.toLowerCase(), v] as const);
@@ -136,6 +337,29 @@ describe('buildAntiCrawlHeaders — UA 轮换与移动端分支', () => {
expect(map.has('sec-fetch-site')).toBe(true);
expect(String(map.get('referer'))).toContain('https://t.test');
});
it('构建完整 12 头反爬签名', () => {
const h = buildAntiCrawlHeaders('https://cdn.test/path', 0, false);
expect(h['Accept']).toContain('text/html');
expect(h['Accept-Encoding']).toBe('gzip, deflate, br');
expect(h['Cache-Control']).toBe('no-cache');
expect(h['DNT']).toBe('1');
expect(h['Sec-Fetch-Dest']).toBe('document');
expect(h['Sec-Fetch-Mode']).toBe('navigate');
expect(h['Sec-Fetch-Site']).toBe('none');
expect(h['Sec-Fetch-User']).toBe('?1');
expect(h['Pragma']).toBe('no-cache');
});
it('Referer 使用 URL origin(含路径时只取源)', () => {
const h = buildAntiCrawlHeaders('https://sub.example.com/a/b?x=1', 0, false);
expect(h['Referer']).toBe('https://sub.example.com');
});
it('非法 URL → Referer 为空串(不抛错)', () => {
const h = buildAntiCrawlHeaders('not a url', 0, false);
expect(h['Referer']).toBe('');
});
});
describe('buildSearXNGAuthHeaders — 认证注入规则', () => {
@@ -157,6 +381,59 @@ describe('buildSearXNGAuthHeaders — 认证注入规则', () => {
it('未知 authType 不注入', () => {
expect(buildSearXNGAuthHeaders('k', 'digest')).toEqual({});
expect(buildSearXNGAuthHeaders('k', '')).toEqual({});
});
});
describe('corsAllowOrigin — 仅回显当前浏览页面同源(P2-2 根治)', () => {
it('请求 Origin 与当前页面同源 → 回显该 Origin', async () => {
const result = corsAllowOrigin('https://example.com', 'https://example.com');
expect(result).toEqual(['https://example.com']);
});
it('请求 Origin 与当前页面跨域 → 返回 null(不加 ACAO,保持默认同源策略)', async () => {
expect(corsAllowOrigin('https://evil.com', 'https://example.com')).toBeNull();
});
it('无 Origin / 无当前页面 → 返回 null(不回退 *)', async () => {
expect(corsAllowOrigin(undefined, 'https://example.com')).toBeNull();
expect(corsAllowOrigin('https://example.com', null)).toBeNull();
expect(corsAllowOrigin(undefined, null)).toBeNull();
});
it('大小写/尾斜杠差异不误判(同源归一化)', async () => {
expect(corsAllowOrigin('HTTPS://EXAMPLE.COM/', 'https://example.com')).toEqual([
'HTTPS://EXAMPLE.COM/',
]);
});
it('同源请求回显原始 Origin(含端口差异保留)', () => {
expect(corsAllowOrigin('https://a.com:8443', 'https://a.com:8443')).toEqual([
'https://a.com:8443',
]);
});
it('空白 Origin 视为无 → null', () => {
expect(corsAllowOrigin(' ', 'https://a.com')).toBeNull();
});
});
describe('extractOriginHeader — 请求头 Origin 提取', () => {
it('大小写不敏感提取单值 Origin', () => {
expect(extractOriginHeader({ ORIGIN: 'https://x.com' })).toBe('https://x.com');
expect(extractOriginHeader({ Origin: 'https://x.com' })).toBe('https://x.com');
expect(extractOriginHeader({ origin: 'https://x.com' })).toBe('https://x.com');
});
it('数组值取第一个', () => {
expect(extractOriginHeader({ Origin: ['https://a.com', 'https://b.com'] })).toBe(
'https://a.com',
);
});
it('无 Origin 头 / 无头对象 → undefined', () => {
expect(extractOriginHeader(undefined)).toBeUndefined();
expect(extractOriginHeader({ Referer: 'x' })).toBeUndefined();
});
});
@@ -174,4 +451,121 @@ describe('searchCache / fetchCache — LRU 行为', () => {
expect(searchCache.get('never:/x')).toBeUndefined();
expect(fetchCache.get('never:/x')).toBeUndefined();
});
it('超容量淘汰最旧条目(LRU max 语义)', () => {
searchCache.clear();
for (let i = 0; i < 210; i++) searchCache.set(`s:evict-${i}`, { i });
expect(searchCache.get('s:evict-0')).toBeUndefined(); // 最早写入被淘汰
expect(searchCache.get('s:evict-209')).toEqual({ i: 209 });
});
});
describe('fetchWithTimeout — 超时中止', () => {
afterEach(() => vi.unstubAllGlobals());
it('正常响应透传返回', async () => {
const stub = vi.fn(async () => new Response('ok', { status: 200 }));
vi.stubGlobal('fetch', stub);
const resp = await fetchWithTimeout('https://x.test/', {}, 1000);
expect(resp.status).toBe(200);
expect(await resp.text()).toBe('ok');
expect(stub).toHaveBeenCalledTimes(1);
});
it('超时触发 AbortErrorfetch 收到 abort signal', async () => {
vi.stubGlobal(
'fetch',
vi.fn((_url: string, init: RequestInit) => {
const signal = init.signal as AbortSignal;
return new Promise<Response>((_resolve, reject) => {
signal.addEventListener('abort', () => {
const err = new Error('Aborted');
err.name = 'AbortError';
reject(err);
});
});
}),
);
await expect(fetchWithTimeout('https://slow.test/', {}, 30)).rejects.toMatchObject({
name: 'AbortError',
});
});
it('fetch 拒绝原样向上传播', async () => {
vi.stubGlobal(
'fetch',
vi.fn(async () => {
throw new Error('network down');
}),
);
await expect(fetchWithTimeout('https://x.test/', {}, 100)).rejects.toThrow('network down');
});
});
// ===== assertSafeConfigTarget 补充(配置类 URL 高危目标校验)=====
describe('assertSafeConfigTarget — 配置 URL 校验(P2-9', () => {
it('拦截云元数据地址', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://169.254.169.254/latest/meta-data/')).toThrow();
expect(() => assertSafeConfigTarget('http://169.254.169.254')).toThrow();
expect(() => assertSafeConfigTarget('http://metadata.google.internal/')).toThrow();
});
it('拦截链路本地/组播/保留段与 0.0.0.0', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://0.0.0.0:8080')).toThrow();
expect(() => assertSafeConfigTarget('http://224.0.0.1/')).toThrow();
expect(() => assertSafeConfigTarget('http://240.0.0.1/')).toThrow();
});
it('放行本地回环/私网实例(合法 MCP/SearXNG', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://127.0.0.1:3000')).not.toThrow();
expect(() => assertSafeConfigTarget('http://192.168.1.10:8080')).not.toThrow();
expect(() => assertSafeConfigTarget('http://10.0.0.5:8888')).not.toThrow();
expect(() => assertSafeConfigTarget('https://searxng.example.com')).not.toThrow();
});
it('拦截非 http/https 协议', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('file:///etc/passwd')).toThrow();
expect(() => assertSafeConfigTarget('ftp://example.com')).toThrow();
});
it('拦截 IPv6 高危地址(去括号后判定,P2-9-A 修正)', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://[::ffff:169.254.169.254]/')).toThrow();
expect(() => assertSafeConfigTarget('http://[fe80::1]/')).toThrow();
expect(() => assertSafeConfigTarget('http://[ff02::1]/')).toThrow();
expect(() => assertSafeConfigTarget('http://[::]/')).toThrow();
expect(() => assertSafeConfigTarget('http://[::1]:11434/')).not.toThrow();
});
it('拦截域名尾点绕过(P2-9-B 修正)', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://metadata.google.internal./')).toThrow();
expect(() => assertSafeConfigTarget('http://169.254.169.254./latest/meta-data/')).toThrow();
});
it('拦截 IPv4-mapped 十六进制云元数据(::ffff:a9fe:a9fe', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
// 169.254 = 0xa9fe
expect(() => assertSafeConfigTarget('http://[::ffff:a9fe:a9fe]/')).toThrow();
});
it('放行 IPv4-mapped 公网(::ffff:0808:0808 = 8.8.8.8', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://[::ffff:0808:0808]/')).not.toThrow();
});
it('拦截 169.254 链路本地变体(169.254.0.1', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('http://169.254.0.1/')).toThrow();
});
it('非法 URL → Invalid URL', async () => {
const { assertSafeConfigTarget } = await import('../ssrf-guard');
expect(() => assertSafeConfigTarget('not a url')).toThrow(/Invalid URL/);
});
});