SearXNG CLI Toolkit

A privacy-respecting metasearch CLI toolkit that runs against your own SearXNG instance. Zero external dependencies for search (stdlib only), optional requests + beautifulsoup4 for enhanced page fetching. Works with any AI agent (Hermes, Claude Code, Codex, OpenCode, Cursor, Trae) or directly from your terminal.

Features

Search & results

• Multi-instance failover with parallel probing
• Exponential-backoff retry on transient errors (429/5xx/connection)
• --verify health-check mode (reachability / JSON-API / latency / POST / engines / auth)
• Cross-engine result deduplication (default on; --no-dedup disables)
• Result sorting (--sort-by {score,date,engine,none})
• Domain allowlist/blocklist (--include-domain / --exclude-domain)
• Batch mode (--queries-file)

Output formats

• JSON (default), brief, urls, CSV
• Enhanced Markdown conversion (GFM tables, code blocks, blockquotes, nested lists, definition lists)
• Structured JSON error output for machine-readable failure reporting

Caching & config

• SQLite result caching (--cache-ttl) with TTL management
• searxng.toml config file for defaults; --config FILE for explicit loading
• Instance resolution: -i → SEARXNG_INSTANCE env → config file

Network & auth

• Proxy support (--proxy) for both search and fetch
• Auth via CLI flag, file, or env var (--auth-bearer / --auth-basic + *-file variants)
• Credentials-file permission warning (POSIX)

Engineering

• Shared common.py (unified retry/charset/auth/logging)
• Structured logging (--verbose / --quiet)
• 155 unit + integration tests with pytest

Quick Start

# Prerequisites: Python 3.8+ (optional: pip install requests beautifulsoup4)

# Search against YOUR instance (instance URL is required)
python scripts/search.py -q "python asyncio tutorial" -i https://my-searxng.example.com

# Multiple instances for failover (comma-separated)
python scripts/search.py -q "rust memory safety" \
  -i https://a.example.com,https://b.example.com --format brief

# Search + auto-fetch top 3 result pages
python scripts/search.py -q "climate policy" -i https://my-searxng.example.com --fetch 3

# Fetch a web page
python scripts/fetch.py -u "https://example.com" --extract markdown

# Skip -i via env var
export SEARXNG_INSTANCE="https://my-searxng.example.com"
python scripts/search.py -q "python asyncio tutorial"

Installation

git clone https://git.metona.cn/MetonaTeam/searxng-use-cli.git
cd searxng-use-cli

# Zero deps — search.py runs on stdlib alone
python scripts/search.py --version

# Optional: enhanced fetch quality
pip install requests beautifulsoup4

Usage

search.py — Execute SearXNG Search

python scripts/search.py -q "your query" -i https://your-instance \
  [--format json|brief|urls|csv] \
  [--engines google,bing,brave] \
  [--time-range day|month|year|none] \
  [--language en] \
  [--sort-by score|date|engine|none] \
  [--no-dedup] \
  [--max-results 10] \
  [--fetch 3] \
  [--cache-ttl 30] \
  [--queries-file queries.txt] \
  [--include-domain example.com] \
  [--exclude-domain spam.com] \
  [--proxy http://corp:8080] \
  [--auth-bearer-file ~/.token] \
  [--verify] \
  [--verbose|-v] [--quiet]

fetch.py — Fetch & Extract Web Page Content

python scripts/fetch.py -u https://example.com \
  --extract text|html|markdown \
  [--encoding gbk] \
  [--max-size 5242880] \
  [--proxy http://corp:8080] \
  [--auth-bearer-file ~/.token]

Configuration

Instance URLs resolve in priority order:

1. -i / --instance (comma-separated for failover)
2. SEARXNG_INSTANCE environment variable
3. Config file: ./searxng.toml → ~/.config/searxng-cli/searxng.toml → ./instances.txt → ~/.config/searxng-cli/instances.txt

Example searxng.toml:

[searxng]
instance = "https://my-searxng.example.com"
# or: instances = ["https://a.example.com", "https://b.example.com"]
engines = "google,bing,brave,duckduckgo,startpage,wikipedia,wikidata"
time_range = "year"
safesearch = 0
format = "json"
cache_ttl = 30

Testing

pip install pytest
pytest -q

155 tests cover: cache operations, auth resolution, domain filtering, Markdown conversion, search logic, integration flows, and logging configuration.

Project Structure

├── scripts/
│   ├── search.py      # Search with multi-instance failover, cache, batch, domain filter
│   ├── fetch.py       # Web page fetcher with text/markdown extraction
│   ├── common.py      # Shared utilities (auth, retry, charset, logging)
│   ├── cache.py       # SQLite-backed result cache
│   └── _config.py     # Version + User-Agent constants
├── tests/             # pytest unit + integration tests
├── .gitea/workflows/  # Gitea Actions CI
├── SKILL.md           # Full skill documentation (agent-facing)
├── pytest.ini         # Test configuration
└── README.md

Defaults

Setting	Default	Flag
Instance	required	-i / SEARXNG_INSTANCE / config
Safe search	0 (off)	-s
Time range	year	-t
Output format	json	-f
Engines	google,bing,brave,duckduckgo,startpage,wikipedia,wikidata	--engines
Sort	score descending	--sort-by
Dedup	on	--no-dedup

Cross-Agent Compatibility

These scripts are agent-agnostic — they work with any AI agent that can invoke terminal commands:

Agent	How to invoke
Hermes	python scripts/search.py -q "..." -i https://your-instance
Claude Code	Same — call via terminal tool
Codex (OpenAI)	Same — call via terminal tool
OpenCode	Same — call via terminal tool
Cursor	Same — call via terminal tool
Trae	Same — call via terminal tool
Standalone (human)	Run directly in any terminal

Key design decisions for universal compatibility:

• Zero external dependencies (stdlib-only for search.py)
• Scripts self-inject their directory into sys.path — run from any working directory
• Stdout carries data (JSON/CSV/text), stderr carries progress/warnings
• Exit codes: 0=success, 1=fatal error, 2=no results/empty
• NO agent-specific API calls — purely CLI-based, portable across all agent platforms

License

MIT