thzxx
81c46eb6f2
- PLAN 附录 H：交付物清单、提交顺序不变量、**实施中新发现的 10 个缺陷**（含分片号复用、 读快照与并发 flush 的窗口、checkpoint 仍等 compaction、takeover 世代竞争、提交中冻结表 误报 WRITE_LOST、底部层只剩墓碑的 TypeError、介质读故障被当缺失、元数据损坏静默空库等）， 以及可复现的验收命令与实测数字。 - PLAN 待办表：B-6 行改为"完整实施（非降级选项）"；原"B-1 遗留"给出结论 （compaction/merge 输入只来自已校验数据，补校验反而有害；触发条件写明）。 - README：架构图/核心机制加入单一提交点；新增"存储布局在 v0.8.0 变更"的已知限制与迁移说明； 测试 1935 / 覆盖率 90.34 · 82.16 · 94.06 · 93.23；新增变异验证命令。 - CHANGELOG：0.8.0 条目补齐 B-6 完整实现（含 9 个新错误码与恢复报告）。 - site：错误码表补 7 个新码；AriaEngine 与崩溃恢复卡片按实现改写（不再宣称"空洞截断"）； 首页徽章数字同步。 - CONTRIBUTING：新增"变异验证"一节（修复类提交必须能回答"回退后用例会不会失败"）。
249 lines
9.6 KiB
Markdown
249 lines
9.6 KiB
Markdown
# Contributing to MetonaSqlark
|
||
|
||
Thanks for your interest in contributing! This document outlines the development workflow and conventions.
|
||
|
||
## Prerequisites
|
||
|
||
- Node.js >= 16.0.0
|
||
- npm >= 8.0.0
|
||
|
||
## Setup
|
||
|
||
```bash
|
||
git clone https://git.metona.cn/MetonaTeam/MetonaSqlark.git
|
||
cd MetonaSqlark
|
||
npm install
|
||
```
|
||
|
||
## Development
|
||
|
||
```bash
|
||
# Start dev server with hot reload (port 3001)
|
||
npm run dev
|
||
|
||
# Run tests in watch mode
|
||
npm run test:watch
|
||
|
||
# Type check
|
||
npm run typecheck
|
||
|
||
# Lint
|
||
npm run lint
|
||
npm run lint:fix
|
||
|
||
# Format
|
||
npm run format
|
||
```
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
src/
|
||
├── index.ts # Entry point, global API (MetonaSqlark + MeSqlark)
|
||
├── core.ts # MetonaSqlark main class
|
||
├── constants.ts # Types, defaults, enums, errors
|
||
├── connection-manager.ts # Connection pool (connect/disconnect)
|
||
├── engine/ # Storage engines
|
||
│ ├── interface.ts # IStorageEngine interface
|
||
│ ├── memory.ts # MemoryEngine (Map-based)
|
||
│ ├── kvstore_engine.ts # KVStoreEngine (disk mode, self-built KV store)
|
||
│ ├── kvstore/ # KVStore (log + snapshot + atomic multi-key write)
|
||
│ └── aria/ # AriaEngine (LSM-Tree page storage engine)
|
||
│ ├── index/ # LSM / MemTable / SSTable / Bloom / MergeIterator
|
||
│ ├── page/ # 4KB slotted page format
|
||
│ ├── buffer/ # Buffer Pool (LRU eviction)
|
||
│ ├── wal/ # Write-Ahead Log + Checkpoint
|
||
│ ├── transaction/ # MVCC manager
|
||
│ ├── store/ # Backends (OPFS / KVStore / Memory / Encrypted)
|
||
│ ├── locks.ts # Web Locks multi-tab exclusive lock
|
||
│ └── compression/ # LZ4
|
||
├── hybrid/ # HybridEngine (write-through)
|
||
├── migration/ # Legacy IndexedDB migration tool (one-shot)
|
||
├── table/ # Table management & Schema validation
|
||
├── query/ # Query system
|
||
│ ├── ast.ts # SQL AST type definitions
|
||
│ ├── builder.ts # Chainable QueryBuilder API
|
||
│ ├── compiler.ts # AST → QueryPlan compiler
|
||
│ ├── executor.ts # QueryExecutor with JOIN/GROUP BY support
|
||
│ └── where-matcher.ts # Unified WHERE matching logic
|
||
├── sql/ # SQL parser
|
||
│ ├── tokens.ts # Token types & keywords
|
||
│ ├── lexer.ts # Tokenizer
|
||
│ ├── parser.ts # Recursive descent parser
|
||
│ └── params.ts # Parameter binding (? placeholders)
|
||
├── transaction/ # Transaction manager
|
||
├── plugin/ # Plugin system (14 lifecycle hooks)
|
||
└── integrations/ # React & Vue hooks
|
||
|
||
tests/ # Test suite (1304 test cases, 76 suites + 12 e2e)
|
||
tests/helpers/ # 共享测试工具（OPFS mock 等）
|
||
tests/e2e/ # Playwright e2e（真实 Chromium + OPFS）
|
||
site/ # Documentation site (index / docs / demo)
|
||
```
|
||
|
||
## Code Conventions
|
||
|
||
### TypeScript
|
||
- **Strict mode** is enabled — all code must pass `tsc --noEmit`.
|
||
- Export types explicitly. Avoid `any` where possible.
|
||
- Use `interface` for object shapes, `type` for unions/primitives.
|
||
|
||
### Style
|
||
- Run `npm run format` before committing (uses Prettier).
|
||
- Follow existing comment patterns: JSDoc `/** */` for public APIs, `//` for inline notes.
|
||
- Keep functions focused and under ~60 lines where practical.
|
||
|
||
### Testing
|
||
- **Every new feature must include tests.**
|
||
- Test files mirror source structure: `src/foo.ts` → `tests/foo.test.ts`.
|
||
- Use descriptive test names: `('does X when Y')`.
|
||
- Run the full suite before submitting: `npm test`.
|
||
|
||
### Commits
|
||
- Use conventional commit messages:
|
||
- `feat: add SQL GROUP BY support`
|
||
- `fix: JOIN with multiple ON conditions`
|
||
- `docs: update API reference`
|
||
- `test: add edge case coverage`
|
||
- `chore: optimize rollup dev build`
|
||
|
||
## Architecture
|
||
|
||
MetonaSqlark follows a layered architecture:
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────┐
|
||
│ MetonaSqlark / MeSqlark (Public API) │
|
||
├──────────────────────────────────────────────────┤
|
||
│ SQL Parser │ QueryBuilder │ Transaction │ Plugin │
|
||
│ (Lexer→AST) │ (.select…) │ Manager │ System │
|
||
├──────────────────────────────────────────────────┤
|
||
│ QueryExecutor (AST → Results) │
|
||
│ JOIN · GROUP BY · HAVING · DISTINCT · Agg │
|
||
├──────────────────────────────────────────────────┤
|
||
│ IStorageEngine (Interface) │
|
||
├─────────┬──────────┬──────────┬──────────────────┤
|
||
│ Memory │ IndexedDB │ OPFS │ Hybrid │
|
||
│ Engine │ Engine │ Engine │ Engine │
|
||
└─────────┴──────────┴──────────┴──────────────────┘
|
||
```
|
||
|
||
### Key Design Decisions
|
||
|
||
1. **AST as Universal Intermediate Representation**: Both SQL strings and Query Builder produce the same AST, ensuring consistent behavior regardless of API used.
|
||
|
||
2. **Write-Through Hybrid Strategy**: When using `mode: 'hybrid'`, all writes go to both memory and disk simultaneously. Reads always hit memory for maximum speed.
|
||
|
||
3. **Plugin Hook Pipeline**: 14 lifecycle hooks let plugins observe and adjust database
|
||
operations without modifying core code. **Hook contract (v0.8.0)**:
|
||
- return values are **ignored** (you cannot cancel an operation by returning `false`,
|
||
nor rewrite the SQL by returning a string);
|
||
- **mutate the argument object in place** to change it — this works on the Table API
|
||
path (`db.table(...).insert(rows)`);
|
||
- **throw** to abort the operation (the error propagates to the caller);
|
||
- the SQL path passes a **copy** for `beforeInsert`, so mutating it there has no effect.
|
||
Use the Table API when you need to transform rows.
|
||
These rules are covered by tests; changing them requires updating this section.
|
||
|
||
4. **Hand-Written SQL Parser**: No dependencies on parser generators — a recursive-descent parser keeps the bundle size minimal.
|
||
|
||
## Building
|
||
|
||
```bash
|
||
# Production build (all formats)
|
||
npm run build
|
||
|
||
# Output in dist/
|
||
# ├── metona-sqlark.js UMD
|
||
# ├── metona-sqlark.min.js UMD minified (~105KB / ~27KB gzip)
|
||
# ├── metona-sqlark.esm.js ES Module
|
||
# ├── metona-sqlark.cjs CommonJS
|
||
# └── metona-sqlark.d.ts TypeScript declarations
|
||
```
|
||
|
||
## Plugin Development
|
||
|
||
Plugins follow a simple convention:
|
||
|
||
```typescript
|
||
import type { MetonaPlugin } from '@metona-team/metona-sqlark';
|
||
|
||
const myPlugin: MetonaPlugin = {
|
||
name: 'myPlugin',
|
||
version: '1.0.0',
|
||
description: 'Description of my plugin',
|
||
// higher = executed first（v0.8.0 起真正生效：install 与钩子都按优先级降序；
|
||
// 同优先级保持 plugins 数组顺序）
|
||
priority: 50,
|
||
|
||
install(db) {
|
||
// Use db.on() to subscribe to hooks
|
||
db.on('beforeInsert', async (rows) => {
|
||
// Validate or transform data
|
||
});
|
||
db.on('afterQuery', async (sql, result) => {
|
||
// Log or cache query results
|
||
});
|
||
},
|
||
|
||
destroy() {
|
||
// Clean up event listeners, timers, etc.
|
||
},
|
||
};
|
||
|
||
// Register in config
|
||
const db = await MetonaSqlark.create({
|
||
name: 'my-app',
|
||
plugins: [myPlugin],
|
||
});
|
||
```
|
||
|
||
### Available Lifecycle Hooks
|
||
|
||
| Hook | Trigger | Parameters |
|
||
|------|---------|------------|
|
||
| `beforeCreateTable` | Before table creation | schema |
|
||
| `afterCreateTable` | After table creation | schema |
|
||
| `beforeDropTable` | Before table drop | tableName |
|
||
| `afterDropTable` | After table drop | tableName |
|
||
| `beforeInsert` | Before row insert | rows[] |
|
||
| `afterInsert` | After row insert | rows[] |
|
||
| `beforeUpdate` | Before row update | query, updates |
|
||
| `afterUpdate` | After row update | query, updates, count |
|
||
| `beforeDelete` | Before row delete | query |
|
||
| `afterDelete` | After row delete | query, count |
|
||
| `beforeQuery` | Before SQL query | sql |
|
||
| `afterQuery` | After SQL query | sql, result |
|
||
| `beforeTransaction` | Before transaction | - |
|
||
| `afterTransaction` | After transaction | - |
|
||
|
||
## Releasing
|
||
|
||
1. Update version in `package.json` and `src/constants.ts` (`VERSION` constant).
|
||
2. Update `CHANGELOG.md`.
|
||
3. Run full test suite: `npm test`.
|
||
4. Build: `npm run build`.
|
||
5. Publish: `npm publish`.
|
||
|
||
## Questions?
|
||
|
||
Open an issue at [git.metona.cn/MetonaTeam/MetonaSqlark/issues](https://git.metona.cn/MetonaTeam/MetonaSqlark/issues).
|
||
|
||
---
|
||
|
||
## 变异验证（回归套件的"是否只是陪跑"检查）
|
||
|
||
修复类提交必须能回答一个问题：**把修复回退到修复前的行为，对应用例会不会失败？**
|
||
不会失败的用例等于没有保护。
|
||
|
||
```bash
|
||
# B-6（存储单一提交点 / LSM 结构根治）17 项变异验证
|
||
python3 scripts/mutation-b6.py
|
||
|
||
# 只跑其中一条（按名字子串匹配）
|
||
python3 scripts/mutation-b6.py "WAL 分片号复用"
|
||
```
|
||
|
||
脚本会临时改写 `src/`、运行对应用例、再恢复源码（收到 SIGINT/SIGTERM 也会恢复），
|
||
最后打印每一条是"被拦住"还是"仍然通过"。**出现任何一条"仍然通过"，本次提交不算完成。**
|