MetonaTeam/searxng-use-cli
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(v1.7.0): AI 友好度增强 + 测试补全 (155→309)

Browse Source 
核心新增（面向 AI Agent 程序化使用）：
- 结构化错误码体系：E_CONFIG/E_AUTH/E_NETWORK/E_RATE_LIMIT/E_PARSE/E_EMPTY/E_INPUT/E_INTERNAL
  classify_error() 自动分类异常，JSON 错误输出含 error_code 字段
- JSON Lines 流式输出 (--stream)：每条结果独立一行，AI 可增量处理
- 进度事件 (--progress)：JSON Lines 事件流到 stderr（start/cache_hit/fetch_ok/done 等）

测试补全（+154 例，覆盖全部高风险盲区）：
- HTML 回退搜索路径 (19)
- --fetch 自动抓取 (21)
- --verify 健康检查 (15)
- 输出格式化 (15)
- 实例解析链 (20)
- 并行多实例搜索 (10)
- CLI 入口与端到端 (17)
- 错误码分类 (27)
- 流式输出与进度事件 (10)

源码改进：
- search.py: h3 内 a 标签 href 作为 url fallback，提升 SearXNG 主题兼容性
- common.py: 新增 classify_error/emit_progress/set_progress_enabled

文档同步：SKILL.md 新增 AI Agent Integration Guide 章节，README.md 更新参数与错误码表
This commit is contained in:
thzxx
2026-08-01 17:40:14 +08:00
parent 9796988da9
commit f983a9377e
14 changed files with 2516 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+144 -2
View File
@@ -1,7 +1,7 @@
---
name: searxng-use-cli
description: Use when you need to search the web via your OWN SearXNG instance (no public-instance discovery). 3 CLI scripts + a shared common.py module — execute privacy-respecting searches against a user-supplied instance (with multi-instance failover, 5xx/429 retry, auto-fetch) or via SEARXNG_INSTANCE env / config file, fetch/extract readable text or markdown from web pages. Zero-config replacement for proprietary search APIs.
version: 1.6.0
version: 1.7.0
author: Metona Team
license: MIT
platforms: [linux, macos, windows]
@@ -33,7 +33,9 @@ SearXNG is a privacy-respecting metasearch engine that aggregates results from 7
**Output formats**
- JSON (default, rich metadata), brief (title+URL+snippet), urls (plain list), CSV (spreadsheet-ready)
- Enhanced Markdown conversion — nested ordered lists (numbered), mixed `ul`/`ol` nesting, `<dl>`/`<dt>`/`<dd>` definition lists, GFM tables, fenced code blocks, blockquotes
- Structured JSON error output (in `--format json` mode) for machine-readable failure reporting
- Structured JSON error output (in `--format json` mode) with `error_code` field for machine-readable failure reporting
- JSON Lines streaming (`--stream`) — each result emitted as a separate JSON line to stdout, enabling incremental processing by AI agents
- Progress events (`--progress`) — structured JSON Lines events to stderr for real-time execution tracking
**Caching & config**
- SQLite result caching (`--cache-ttl`) — identical queries within a TTL skip the network entirely; `--clear-cache` / `--cache-stats` manage it
@@ -157,6 +159,146 @@ python scripts/search.py -q "test" -i https://s.example.com --quiet # errors
- High-frequency production search without a properly-scaled instance — respect your instance's rate limits
- Guaranteed uptime/accuracy — depends entirely on the instance you supply
## AI Agent Integration Guide
This section documents the structured interfaces that AI agents can rely on
for programmatic integration. All features are designed to be machine-readable
and machine-actionable.
### Output Channels
| Channel | Content | Description |
|---------|---------|-------------|
| stdout | Data | JSON/CSV/text — the only source AI should parse |
| stderr | Logs + Progress | Human-readable logs (default) or JSON Lines events (`--progress`) |
| exit 0 | Success | Results available on stdout |
| exit 1 | Fatal error | Error JSON on stdout (in `--format json` mode) or stderr |
| exit 2 | Empty results | Search succeeded but returned no results |
### Error Code System
In `--format json` mode, errors are emitted as structured JSON on stdout:
```json
{
"error": "All 2 instances failed. Last error: connection refused",
"error_code": "E_NETWORK",
"exit_code": 1,
"query": "search term"
}
```
AI agents can use `error_code` to programmatically decide recovery strategy:
| Code | Meaning | AI Recovery Strategy |
|------|---------|---------------------|
| `E_CONFIG` | Configuration error (no instance resolved) | Prompt user to set `-i` / `SEARXNG_INSTANCE` / config file |
| `E_AUTH` | Authentication failed (401/403) | Check token/credentials, prompt user to re-authenticate |
| `E_NETWORK` | Network error (connection refused, timeout, 5xx, all instances failed) | Retry with backoff, switch instance or proxy |
| `E_RATE_LIMIT` | Rate limited (429) | Wait and retry with reduced frequency |
| `E_PARSE` | Parse error (JSON/HTML parsing failed) | Check if instance supports JSON, try HTML fallback |
| `E_EMPTY` | Empty results (exit code 2) | Adjust query terms or time range |
| `E_INPUT` | Input error (bad parameters, file not found) | Fix CLI arguments or file paths |
| `E_INTERNAL` | Internal error (unexpected exception) | Report bug with full error message |
### JSON Lines Streaming (`--stream`)
For large result sets, `--stream` outputs results as JSON Lines (one JSON
object per line) to stdout, allowing AI agents to process results
incrementally without waiting for the full response:
```bash
python scripts/search.py -q "large topic" -i https://your-instance --stream
```
Output format (each line is a separate JSON object):
```
{"type": "result", "result": {"title": "...", "url": "...", "content": "..."}}
{"type": "result", "result": {"title": "...", "url": "...", "content": "..."}}
{"type": "done", "count": 2, "query": "large topic"}
```
- `type: "result"` — one per search result, emitted as soon as available
- `type: "done"` — terminal event with total count, always emitted last
- Exit code 0 on success, 2 on empty results (done event still emitted)
Only valid with `--format json` (the default).
### Progress Events (`--progress`)
For long-running operations, `--progress` emits structured JSON Lines events
to stderr, enabling AI agents to track execution progress in real time:
```bash
python scripts/search.py -q "research topic" -i https://your-instance --progress --fetch 3
```
Event types (each on its own line, JSON Lines format on stderr):
```jsonl
{"event": "start", "query": "research topic", "instances": 2}
{"event": "cache_hit", "query": "research topic", "ttl": 30}
{"event": "cache_store", "query": "research topic", "ttl": 30}
{"event": "fetch_start", "count": 3}
{"event": "fetch_ok", "url": "https://example.com/page", "chars": 12345}
{"event": "fetch_fail", "url": "https://bad.example.com", "error": "HTTP 503"}
{"event": "done", "results": 10, "query": "research topic"}
{"event": "error", "error": "connection refused", "error_code": "E_NETWORK", "query": "..."}
```
AI agents can parse these events to:
- Show progress indicators to users
- Detect cache hits (skip waiting)
- Monitor fetch failures and retry strategies
- Correlate errors with specific queries in batch mode
`--progress` and `--verbose` can be used together (progress events on stderr,
debug logs also on stderr). `--progress` events are JSON Lines; `--verbose`
logs are human-readable text.
### Output JSON Schema
The default `--format json` output shape (for reference, not enforced):
```json
{
"query": "search term",
"number_of_results": 10,
"results": [
{
"title": "Result title",
"url": "https://example.com/page",
"content": "Snippet text...",
"engine": "google",
"score": 1.0,
"category": "general",
"published_date": "2024-01-15T10:30:00"
}
],
"answers": ["Direct answer if available"],
"corrections": [],
"suggestions": ["related suggestion"],
"infoboxes": [],
"unresponsive_engines": [["engine_name", "error reason"]],
"fetched": [
{
"url": "https://example.com/page",
"status": "ok",
"text": "Extracted page content...",
"text_length": 12345,
"truncated": false,
"final_url": "https://example.com/final",
"user_agent_used": "searxng-cli/1.7.0"
}
],
"fetched_source": "json"
}
```
Fields marked as optional may be absent. The `fetched` and `fetched_source`
fields only appear when `--fetch N` is used.
## Cross-Agent Compatibility
These scripts are **agent-agnostic** — they work with any AI agent that can invoke terminal commands: