feat(v2.4.0): 新增 E_BLOCKED 错误码——细分 fetch 403 反爬拦截
真实环境问题: 抓取被墙/反爬站点 (如 baike.baidu.com) 返回 403 时, classify_error 统一归为 E_AUTH, AI Agent 会误判为凭证问题而做无效的 认证重试。被封锁不是认证失败。 改动: - common.py: 新增 E_BLOCKED 错误码 + recovery_hint (提示换 URL/镜像/ 用 Wayback 兜底/--exclude-domain); 新增 classify_fetch_error() 与 _extract_status_code(): fetch 场景 403→E_BLOCKED, 401→E_AUTH, 其余委托 classify_error (搜索场景 403 仍为 E_AUTH, 不变) - fetch.py main(): 错误路径改用 classify_fetch_error (替代 classify_error) - search.py: fetch_page 与 fetch_top_results 线程错误路径同样切换 - 新增 11 个测试 (test_v240_blocked_code.py), 全量 572 测试通过 - 真实环境验证: baike 403 → E_BLOCKED, 正常站点不受影响 - 文档同步 (SKILL.md/README.md 错误码表, 版本号 2.4.0)
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: searxng-use-cli
|
||||
description: Use when you need to search the web via your OWN SearXNG instance (no public-instance discovery). 3 CLI scripts + a shared common.py module — execute privacy-respecting searches against a user-supplied instance (with multi-instance failover, 5xx/429 retry, auto-fetch) or via SEARXNG_INSTANCE env / config file, fetch/extract readable text or markdown from web pages. Zero-config replacement for proprietary search APIs.
|
||||
version: 2.3.0
|
||||
version: 2.4.0
|
||||
author: Metona Team
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
@@ -250,16 +250,20 @@ In `--format json` mode, errors are emitted as structured JSON on stdout:
|
||||
|
||||
AI agents can use `error_code` to programmatically decide recovery strategy, and `recovery_hint` for a ready-to-use actionable suggestion:
|
||||
|
||||
> **v2.4.0:** fetch-scenario 403 (WAF/anti-bot blocking) is classified as `E_BLOCKED`, not `E_AUTH` — a blocked page is not a credentials problem, so agents should not waste attempts re-authenticating. `classify_fetch_error()` applies to page fetches (`fetch.py`, `search.py --fetch`); SearXNG instance 403 still maps to `E_AUTH` (correct for instance auth).
|
||||
|
||||
| Code | Meaning | recovery_hint (abridged) |
|
||||
|------|---------|--------------------------|
|
||||
| `E_CONFIG` | Configuration error (no instance resolved) | Provide `-i`/`SEARXNG_INSTANCE`/config file |
|
||||
| `E_AUTH` | Authentication failed (401/403) | Verify credentials, check token expiry & permissions |
|
||||
| `E_BLOCKED` | Fetch blocked by anti-bot/WAF (403, v2.4.0) | NOT a credentials problem — use Wayback fallback (default on), fetch a mirror/other URL, or drop the domain (`--exclude-domain`) |
|
||||
| `E_NETWORK` | Network error (connection refused, timeout, 5xx, all instances failed) | Retry with backoff, switch instance, check proxy |
|
||||
| `E_RATE_LIMIT` | Rate limited (429) | Wait and retry, reduce frequency, distribute load |
|
||||
| `E_PARSE` | Parse error (JSON/HTML parsing failed) | Try different instance, switch `--method` |
|
||||
| `E_EMPTY` | Empty results (exit code 2) | Refine query, broaden `--time-range`, add `--categories` |
|
||||
| `E_INPUT` | Input error (bad parameters, file not found) | Check query syntax, flag combinations, file paths |
|
||||
| `E_INTERNAL` | Internal error (unexpected exception) | Re-run with `--verbose`, report bug |
|
||||
| `E_BLOCKED` | Fetch blocked by anti-bot/WAF (403, v2.4.0) | Not a credentials issue — Wayback fallback, mirror URL, or `--exclude-domain` |
|
||||
| `E_UNSUPPORTED_MEDIA` | Unsupported binary media type (PDF/docx/xlsx parse failure or unsupported content type) | No auto hint — try a different URL, or install `pdftotext` for PDF parsing |
|
||||
|
||||
### JSON Lines Streaming (`--stream`)
|
||||
|
||||
Reference in New Issue
Block a user