MetonaSqlark/CONTRIBUTING.md

Contributing to MetonaSqlark

Thanks for your interest in contributing! This document outlines the development workflow and conventions.

Prerequisites

• Node.js >= 16.0.0
• npm >= 8.0.0

Setup

git clone https://git.metona.cn/MetonaTeam/MetonaSqlark.git
cd MetonaSqlark
npm install

Development

# Start dev server with hot reload (port 3001)
npm run dev

# Run tests in watch mode
npm run test:watch

# Type check
npm run typecheck

# Lint
npm run lint
npm run lint:fix

# Format
npm run format

Project Structure

src/
├── index.ts              # Entry point, global API (MetonaSqlark + MeSqlark)
├── core.ts               # MetonaSqlark main class
├── constants.ts          # Types, defaults, enums, errors
├── connection-manager.ts # Connection pool (connect/disconnect)
├── engine/               # Storage engines
│   ├── interface.ts      # IStorageEngine interface
│   ├── memory.ts         # MemoryEngine (Map-based)
│   ├── kvstore_engine.ts # KVStoreEngine (disk mode, self-built KV store)
│   ├── kvstore/          # KVStore (log + snapshot + atomic multi-key write)
│   └── aria/             # AriaEngine (LSM-Tree page storage engine)
│       ├── index/        # LSM / MemTable / SSTable / Bloom / MergeIterator
│       ├── page/         # 4KB slotted page format
│       ├── buffer/       # Buffer Pool (LRU eviction)
│       ├── wal/          # Write-Ahead Log + Checkpoint
│       ├── transaction/  # MVCC manager
│       ├── store/        # Backends (OPFS / KVStore / Memory / Encrypted)
│       ├── locks.ts      # Web Locks multi-tab exclusive lock
│       └── compression/  # LZ4
├── hybrid/               # HybridEngine (write-through)
├── migration/            # Legacy IndexedDB migration tool (one-shot)
├── table/                # Table management & Schema validation
├── query/                # Query system
│   ├── ast.ts            # SQL AST type definitions
│   ├── builder.ts        # Chainable QueryBuilder API
│   ├── compiler.ts       # AST → QueryPlan compiler
│   ├── executor.ts       # QueryExecutor with JOIN/GROUP BY support
│   └── where-matcher.ts  # Unified WHERE matching logic
├── sql/                  # SQL parser
│   ├── tokens.ts         # Token types & keywords
│   ├── lexer.ts          # Tokenizer
│   ├── parser.ts         # Recursive descent parser
│   └── params.ts         # Parameter binding (? placeholders)
├── transaction/          # Transaction manager
├── plugin/               # Plugin system (14 lifecycle hooks)
└── integrations/         # React & Vue hooks

tests/                    # Test suite (1980 test cases, 92 suites + 14 e2e)
tests/helpers/             # 共享测试工具（OPFS mock 等）
tests/e2e/                # Playwright e2e（真实 Chromium + OPFS）
site/                     # Documentation site (index / docs / demo)

Code Conventions

TypeScript

• Strict mode is enabled — all code must pass tsc --noEmit.
• Export types explicitly. Avoid any where possible.
• Use interface for object shapes, type for unions/primitives.

Style

• Run npm run format before committing (uses Prettier).
• Follow existing comment patterns: JSDoc /** */ for public APIs, // for inline notes.
• Keep functions focused and under ~60 lines where practical.

Testing

• Every new feature must include tests.
• Test files mirror source structure: src/foo.ts → tests/foo.test.ts.
• Use descriptive test names: ('does X when Y').
• Run the full suite before submitting: npm test.

Commits

• Use conventional commit messages:
  • feat: add SQL GROUP BY support
  • fix: JOIN with multiple ON conditions
  • docs: update API reference
  • test: add edge case coverage
  • chore: optimize rollup dev build

Architecture

MetonaSqlark follows a layered architecture:

┌──────────────────────────────────────────────────┐
│         MetonaSqlark / MeSqlark (Public API)      │
├──────────────────────────────────────────────────┤
│  SQL Parser  │ QueryBuilder │ Transaction │ Plugin │
│  (Lexer→AST) │ (.select…)   │  Manager    │ System │
├──────────────────────────────────────────────────┤
│         QueryExecutor (AST → Results)             │
│    JOIN · GROUP BY · HAVING · DISTINCT · Agg     │
├──────────────────────────────────────────────────┤
│         IStorageEngine (Interface)                │
├─────────┬──────────┬──────────┬──────────────────┤
│ Memory  │ IndexedDB │ OPFS     │ Hybrid           │
│ Engine  │ Engine    │ Engine   │ Engine           │
└─────────┴──────────┴──────────┴──────────────────┘

Key Design Decisions

1. AST as Universal Intermediate Representation: Both SQL strings and Query Builder produce the same AST, ensuring consistent behavior regardless of API used.

2. Write-Through Hybrid Strategy: When using mode: 'hybrid', all writes go to both memory and disk simultaneously. Reads always hit memory for maximum speed.

3. Plugin Hook Pipeline: 14 lifecycle hooks let plugins observe and adjust database operations without modifying core code. Hook contract (v0.8.0):

   • return values are ignored (you cannot cancel an operation by returning false, nor rewrite the SQL by returning a string);
   • mutate the argument object in place to change it — this works on the Table API path (db.table(...).insert(rows));
   • throw to abort the operation (the error propagates to the caller);
   • the SQL path passes a copy for beforeInsert, so mutating it there has no effect. Use the Table API when you need to transform rows. These rules are covered by tests; changing them requires updating this section.

4. Hand-Written SQL Parser: No dependencies on parser generators — a recursive-descent parser keeps the bundle size minimal.

Building

# Production build (all formats)
npm run build

# Output in dist/
# ├── metona-sqlark.js        UMD
# ├── metona-sqlark.min.js    UMD minified (251,109 B / gzip 63,145 B)
# ├── metona-sqlark.esm.js    ES Module
# ├── metona-sqlark.cjs    CommonJS
# └── metona-sqlark.d.ts      TypeScript declarations

Plugin Development

Plugins follow a simple convention:

import type { MetonaPlugin } from '@metona-team/metona-sqlark';

const myPlugin: MetonaPlugin = {
  name: 'myPlugin',
  version: '1.0.0',
  description: 'Description of my plugin',
  // higher = executed first（v0.8.0 起真正生效：install 与钩子都按优先级降序；
  // 同优先级保持 plugins 数组顺序）
  priority: 50,

  install(db) {
    // Use db.on() to subscribe to hooks
    db.on('beforeInsert', async (rows) => {
      // Validate or transform data
    });
    db.on('afterQuery', async (sql, result) => {
      // Log or cache query results
    });
  },

  destroy() {
    // Clean up event listeners, timers, etc.
  },
};

// Register in config
const db = await MetonaSqlark.create({
  name: 'my-app',
  plugins: [myPlugin],
});

Available Lifecycle Hooks

Hook	Trigger	Parameters
beforeCreateTable	Before table creation	schema
afterCreateTable	After table creation	schema
beforeDropTable	Before table drop	tableName
afterDropTable	After table drop	tableName
beforeInsert	Before row insert	rows[]
afterInsert	After row insert	rows[]
beforeUpdate	Before row update	query, updates
afterUpdate	After row update	query, updates, count
beforeDelete	Before row delete	query
afterDelete	After row delete	query, count
beforeQuery	Before SQL query	sql
afterQuery	After SQL query	sql, result
beforeTransaction	Before transaction	-
afterTransaction	After transaction	-

Releasing

1. Update version in package.json and src/constants.ts (VERSION constant).
2. Update CHANGELOG.md.
3. Run full test suite: npm test.
4. Build: npm run build.
5. Publish: npm publish.

Questions?

Open an issue at git.metona.cn/MetonaTeam/MetonaSqlark/issues.

---

变异验证（回归套件的"是否只是陪跑"检查）

修复类提交必须能回答一个问题：把修复回退到修复前的行为，对应用例会不会失败？ 不会失败的用例等于没有保护。

# 全部 40 项变异（存储单一提交点 / LSM 结构根治 / v0.8.0 全量审查修复）
python3 scripts/mutation-b6.py

# 只跑其中一条（按名字子串匹配）
python3 scripts/mutation-b6.py "WAL 分片号复用"

脚本会临时改写 src/、运行对应用例、再恢复源码（收到 SIGINT/SIGTERM 也会恢复）， 最后打印每一条是"被拦住"还是"仍然通过"。出现任何一条"仍然通过"，本次提交不算完成。

脚本自身的保证（v0.8.0 审查后加固）

机制	为什么需要
正控	先跑一遍未变异的干净基线（B-6 套件 + 分片 WAL 套件必须全绿）。没有正控就无法区分"变异被测试拦住"与"这套件本来就是红的"
编译失败 ≠ 被拦住	Test suite failed to run / error TS#### 单独归类为 BAD(变异破坏编译)：一个把源码改成语法错误的"变异"什么都没验证
用例未匹配 ≠ 被拦住	Tests: 0 total / No tests found 归类为 BAD(用例未匹配)（模式写错时最容易被误读成"测试通过了"）
锚点唯一性	锚点在文件里出现多次时拒绝变异（否则可能改到另一处、验错位置）
超时	单条变异 300s 上限，超时归类为 BAD(超时) 而不是放过
逐字节恢复校验	每条变异跑完用 sha256 比对恢复结果；不一致立即 FATAL 退出
进程锁	.mutation-b6.lock（O_EXCL）：并发运行会互相踩掉对方的源码，直接拒绝启动

判定顺序固定为：编译失败 → 用例未匹配 → ✕/Tests: N failed（= 被拦住）→ 其余为"仍然通过"。 只有 OK(变异被拦住) 计入成功，其余任何状态都会让脚本以非零退出码结束。