feat: v0.7.0 参数化查询 + 事务增量 flush + 复合主键语义硬化 + EXPLAIN 真实索引信息
- 参数化查询 db.query(sql, params):词法层 ? 绑定 + SQL 字面量安全编码 ('' 转义/注入防护);参数计数不匹配 PARAM_ERROR;对象参数显式拒绝 - KVStoreEngine 事务增量 flush:行级变更追踪,commit 仅写改动行 (1000 行表改 1 行:日志 1 条目 vs 整表 1000 条目);级联影响表漏写修复 (txFullTables 同步加入 txDirtyTables);移除每次 commit 全量 checkpoint (阈值自动 checkpoint + close 统一截断) - 复合主键显式拒绝:createSchema 校验期 SCHEMA_ERROR + ALTER ADD 主键列防护 (此前静默取第一个主键,其余标记失效) - EXPLAIN usingIndex 真实命中信息:pk / index:col / none 测试 1126 → 1147(72 套件);行覆盖率 89.8%;版本 0.7.0
This commit is contained in:
@@ -2,6 +2,52 @@
|
|||||||
|
|
||||||
All notable changes to MetonaSqlark will be documented in this file.
|
All notable changes to MetonaSqlark will be documented in this file.
|
||||||
|
|
||||||
|
## [0.7.0] - 2026-08-13
|
||||||
|
|
||||||
|
### 参数化查询 / 事务性能 / 语义硬化
|
||||||
|
|
||||||
|
> 深度审计第三阶段:能力补全(参数化查询)、大库事务性能、
|
||||||
|
> 复合主键语义陷阱显式化、EXPLAIN 真实索引信息。
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **参数化查询** — `db.query(sql, params)` 位置参数(`?`):
|
||||||
|
- 绑定在词法层完成(仅替换字符串字面量之外的 `?`,`''` 转义与原样保留)
|
||||||
|
- 值按 SQL 字面量安全编码(字符串 `''` 转义 / 数字 / 布尔 / NULL),
|
||||||
|
注入防护从根上成立(`'x'; DROP TABLE users; --` 作为字面量原样入库)
|
||||||
|
- 参数数量不匹配 → `PARAM_ERROR`;对象/数组参数显式拒绝(SQL 方言无 json
|
||||||
|
字面量,防静默类型错配);NaN/Infinity → NULL
|
||||||
|
- **EXPLAIN 真实索引信息** — `usingIndex` 从 'auto' 占位升级为真实命中判定:
|
||||||
|
主键条件 → `pk`、索引/唯一列条件 → `index:col`、无索引 → `none`
|
||||||
|
(引擎无关启发式:按 schema 标记判定,与各引擎 tryIndexLookup 路径一致)
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **KVStoreEngine 事务增量 flush(性能)** — commit 时此前对每个 dirty 表整表
|
||||||
|
diff(大表事务改 1 行也 O(表大小))。现在行级变更追踪:普通 insert/update/delete
|
||||||
|
仅写事务内改动行(单次全表扫描 + 变更集合过滤);主键变更/级联影响表整表 diff
|
||||||
|
兜底;事务内 clear/drop 只删 KV 行;clear 后写入自动切换兜底。实测:1000 行表
|
||||||
|
事务改 1 行 commit 日志仅 1 条记录 1 个条目(此前 1000 条目)
|
||||||
|
- **移除每次 commit 的强制全量 checkpoint** — KVStore 按日志阈值自动 checkpoint
|
||||||
|
(日志重放保证崩溃恢复),close() 时统一 checkpoint 截断(重开更快);
|
||||||
|
大库高频事务不再 O(库大小)
|
||||||
|
- **复合主键显式拒绝(语义陷阱)** — 此前 schema 允许多个 primaryKey 但引擎
|
||||||
|
静默取第一个(其余标记失效)。createSchema 校验期抛 `SCHEMA_ERROR`
|
||||||
|
(含明确指引);ALTER TABLE ADD 主键列同样防护(防绕过建表校验)。
|
||||||
|
复合主键列入 v0.8 路线图
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- 事务级联影响表漏写 — delete/update 的级联影响表加入 txFullTables 但未加入
|
||||||
|
txDirtyTables → commit 循环不处理 → 级联改动(SET NULL 等)不落盘
|
||||||
|
|
||||||
|
### Changed(测试)
|
||||||
|
|
||||||
|
- 测试 1126 → **1147**(72 套件,+21 个 v0.7.0 回归);新增 `tests/v070-features.test.ts`
|
||||||
|
(复合主键拒绝 ×3 / 参数化 ×7 / 事务增量 flush ×8 / EXPLAIN ×2)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [0.6.3] - 2026-08-13
|
## [0.6.3] - 2026-08-13
|
||||||
|
|
||||||
### 原子性 / 一致性 / 资源治理
|
### 原子性 / 一致性 / 资源治理
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
# MetonaSqlark
|
# MetonaSqlark
|
||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<img src="https://img.shields.io/badge/version-0.6.3-blue?style=flat-square" alt="version">
|
<img src="https://img.shields.io/badge/version-0.7.0-blue?style=flat-square" alt="version">
|
||||||
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
||||||
<img src="https://img.shields.io/badge/coverage-89.7%25-brightgreen?style=flat-square" alt="coverage">
|
<img src="https://img.shields.io/badge/coverage-89.8%25-brightgreen?style=flat-square" alt="coverage">
|
||||||
<img src="https://img.shields.io/badge/tests-1126%20passed-success?style=flat-square" alt="tests">
|
<img src="https://img.shields.io/badge/tests-1147%20passed-success?style=flat-square" alt="tests">
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
> 基于 TypeScript 的**前端关系型数据库**:完整 SQL + Query Builder 双 API,
|
> 基于 TypeScript 的**前端关系型数据库**:完整 SQL + Query Builder 双 API,
|
||||||
@@ -116,6 +116,10 @@ await db.defineTable('orders', {
|
|||||||
await db.query("INSERT INTO users VALUES ('1', 'Alice', 30, 'alice@demo.com')");
|
await db.query("INSERT INTO users VALUES ('1', 'Alice', 30, 'alice@demo.com')");
|
||||||
const rows = await db.query('SELECT * FROM users WHERE age > 18 ORDER BY name');
|
const rows = await db.query('SELECT * FROM users WHERE age > 18 ORDER BY name');
|
||||||
|
|
||||||
|
// 参数化查询(v0.7.0)— 位置参数 `?`,安全编码杜绝 SQL 注入
|
||||||
|
await db.query("INSERT INTO users VALUES (?, ?, ?, ?)", ['2', "O'Brien", 25, 'ob@demo.com']);
|
||||||
|
const row2 = await db.query('SELECT * FROM users WHERE name = ?', ["O'Brien"]);
|
||||||
|
|
||||||
// Query Builder
|
// Query Builder
|
||||||
const result = await db.table('users')
|
const result = await db.table('users')
|
||||||
.select(['name', 'age'])
|
.select(['name', 'age'])
|
||||||
@@ -213,7 +217,7 @@ await db2.disconnect(); // 引用计数 -1
|
|||||||
|
|
||||||
| 方法 | 说明 |
|
| 方法 | 说明 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| `db.query(sql)` | 执行 SQL(支持分号多语句) |
|
| `db.query(sql)` / `db.query(sql, params)` | 执行 SQL(支持分号多语句;v0.7.0 位置参数 `?` 绑定) |
|
||||||
| `db.queryStream(sql, onRow)` | 流式查询(逐行回调,不物化) |
|
| `db.queryStream(sql, onRow)` | 流式查询(逐行回调,不物化) |
|
||||||
| `db.table(name)` | 获取表操作对象(`insert` / `select` / `update` / `delete` / `count` / `stream` / `clear` / `drop`) |
|
| `db.table(name)` | 获取表操作对象(`insert` / `select` / `update` / `delete` / `count` / `stream` / `clear` / `drop`) |
|
||||||
| `db.defineTable(name, cols)` | 定义表结构 |
|
| `db.defineTable(name, cols)` | 定义表结构 |
|
||||||
@@ -233,7 +237,7 @@ await db2.disconnect(); // 引用计数 -1
|
|||||||
|
|
||||||
| 语句 | 说明 | 支持引擎 |
|
| 语句 | 说明 | 支持引擎 |
|
||||||
|------|------|----------|
|
|------|------|----------|
|
||||||
| `EXPLAIN SELECT ...` | 输出查询计划(type/table/where/estimatedRows/actualTimeMs) | 全部 |
|
| `EXPLAIN SELECT ...` | 输出查询计划(type/table/where/usingIndex/estimatedRows/actualTimeMs) | 全部 |
|
||||||
| `ANALYZE [TABLE] name` | 收集表统计信息(行数/行大小/索引深度/列基数) | Aria |
|
| `ANALYZE [TABLE] name` | 收集表统计信息(行数/行大小/索引深度/列基数) | Aria |
|
||||||
| `REINDEX [TABLE] name` | 重建表二级索引 | Aria |
|
| `REINDEX [TABLE] name` | 重建表二级索引 | Aria |
|
||||||
| `VACUUM` | 压缩 LSM + 清理 MVCC 碎片 | Aria |
|
| `VACUUM` | 压缩 LSM + 清理 MVCC 碎片 | Aria |
|
||||||
@@ -397,7 +401,7 @@ const { data, loading, error, refresh } = useSqlarkQuery(db, 'SELECT * FROM user
|
|||||||
npm install # 安装依赖
|
npm install # 安装依赖
|
||||||
npm run dev # 开发模式(localhost:3001)
|
npm run dev # 开发模式(localhost:3001)
|
||||||
npm run build # 生产构建(生成 dist/)
|
npm run build # 生产构建(生成 dist/)
|
||||||
npm test # 运行测试(1126 用例 · 71 套件)
|
npm test # 运行测试(1147 用例 · 72 套件)
|
||||||
npm run test:e2e # Playwright e2e(真实 Chromium + OPFS + 崩溃注入,需先 build)
|
npm run test:e2e # Playwright e2e(真实 Chromium + OPFS + 崩溃注入,需先 build)
|
||||||
npm run lint # 代码检查
|
npm run lint # 代码检查
|
||||||
npm run typecheck # 类型检查
|
npm run typecheck # 类型检查
|
||||||
@@ -409,9 +413,9 @@ npm run typecheck # 类型检查
|
|||||||
|
|
||||||
| 指标 | 数值 |
|
| 指标 | 数值 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| 测试用例 | 1126(+12 Playwright e2e) |
|
| 测试用例 | 1147(+12 Playwright e2e) |
|
||||||
| 测试套件 | 70 |
|
| 测试套件 | 70 |
|
||||||
| 行覆盖率 | 89.7% |
|
| 行覆盖率 | 89.8% |
|
||||||
| SQL 关键字 | 72 |
|
| SQL 关键字 | 72 |
|
||||||
| 存储引擎 | 5(Memory / KVStore / OPFS / Hybrid / Aria) |
|
| 存储引擎 | 5(Memory / KVStore / OPFS / Hybrid / Aria) |
|
||||||
| 运行时依赖 | 0 |
|
| 运行时依赖 | 0 |
|
||||||
|
|||||||
Vendored
+264
-13
@@ -34,7 +34,7 @@ class DatabaseError extends Error {
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// 版本
|
// 版本
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
const VERSION = '0.6.3';
|
const VERSION = '0.7.0';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Shared WHERE Matcher — 统一的条件匹配逻辑
|
* metona-sqlark Shared WHERE Matcher — 统一的条件匹配逻辑
|
||||||
@@ -1858,6 +1858,15 @@ class KVStoreEngine {
|
|||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
/**
|
||||||
|
* v0.7.0: 事务行级变更记录(table → pk → put/delete)。
|
||||||
|
* commit 时按行增量 flush(此前整表 diff:大表事务改 1 行也重写全表)。
|
||||||
|
*/
|
||||||
|
this.txChanges = new Map();
|
||||||
|
/** v0.7.0: 无法行级追踪的表(主键变更/级联影响表)→ commit 时整表 diff */
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
/** v0.7.0: 事务内 clear 的表 → commit 时清空 KV 行 */
|
||||||
|
this.txClearedTables = new Set();
|
||||||
this.kv = new KVStore(medium, checkpointThreshold);
|
this.kv = new KVStore(medium, checkpointThreshold);
|
||||||
}
|
}
|
||||||
// ---- 行 key 编解码 ----
|
// ---- 行 key 编解码 ----
|
||||||
@@ -1931,6 +1940,12 @@ class KVStoreEngine {
|
|||||||
}
|
}
|
||||||
catch { /* ignore */ }
|
catch { /* ignore */ }
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 关闭前 checkpoint(截断日志,重开更快)。
|
||||||
|
// 失败不阻塞关闭(日志已持久,重开可全量重放)。
|
||||||
|
try {
|
||||||
|
await this.kv.checkpoint();
|
||||||
|
}
|
||||||
|
catch { /* 数据在日志中,重开放心重放 */ }
|
||||||
await this.kv.close();
|
await this.kv.close();
|
||||||
await this.memory.close();
|
await this.memory.close();
|
||||||
this.opened = false;
|
this.opened = false;
|
||||||
@@ -2033,6 +2048,20 @@ class KVStoreEngine {
|
|||||||
const pks = await this.memory.insert(tableName, rows);
|
const pks = await this.memory.insert(tableName, rows);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内 clear 后又写入 → 清空语义被覆盖,整表 diff 兜底
|
||||||
|
if (this.txClearedTables.has(tableName)) {
|
||||||
|
this.txClearedTables.delete(tableName);
|
||||||
|
this.txFullTables.add(tableName);
|
||||||
|
return pks;
|
||||||
|
}
|
||||||
|
// v0.7.0: 行级变更记录(增量 flush)
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks)
|
||||||
|
changes.set(pk, 'put');
|
||||||
return pks;
|
return pks;
|
||||||
}
|
}
|
||||||
// 增量持久化(原子 putMany)
|
// 增量持久化(原子 putMany)
|
||||||
@@ -2067,6 +2096,30 @@ class KVStoreEngine {
|
|||||||
const count = await this.memory.update(tableName, query, updates);
|
const count = await this.memory.update(tableName, query, updates);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 主键变更无法行级追踪(旧键删除+新键落盘+级联),
|
||||||
|
// 相关表整表 diff 兜底;普通更新记录受影响行
|
||||||
|
if (pkChanged) {
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of affected)
|
||||||
|
changes.set(pk, 'put');
|
||||||
|
// 级联影响表(理论上非主键更新不级联,防御性兜底)
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts = {};
|
const puts = {};
|
||||||
@@ -2116,6 +2169,20 @@ class KVStoreEngine {
|
|||||||
const count = await this.memory.delete(tableName, query);
|
const count = await this.memory.delete(tableName, query);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 删除行记录 delete;级联影响表整表 diff 兜底
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks)
|
||||||
|
changes.set(pk, 'delete');
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts = {};
|
const puts = {};
|
||||||
@@ -2140,6 +2207,9 @@ class KVStoreEngine {
|
|||||||
await this.memory.clear(tableName);
|
await this.memory.clear(tableName);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内清空 → commit 时删除全部 KV 行(比整表 diff 更高效)
|
||||||
|
this.txClearedTables.add(tableName);
|
||||||
|
this.txChanges.delete(tableName);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const diff = await this.collectTableDiff(tableName);
|
const diff = await this.collectTableDiff(tableName);
|
||||||
@@ -2173,22 +2243,23 @@ class KVStoreEngine {
|
|||||||
this.txActive = true;
|
this.txActive = true;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
async commitTransaction() {
|
async commitTransaction() {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
if (!this.txActive)
|
if (!this.txActive)
|
||||||
throw new DatabaseError('No active transaction', 'TX_NONE');
|
throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
||||||
// 多表事务中途崩溃/失败不会出现"部分表已提交")
|
// 多表事务中途崩溃/失败不会出现"部分表已提交")。
|
||||||
|
// v0.7.0: 行级增量 flush —— 普通 insert/update/delete 仅写事务内改动的行
|
||||||
|
// (此前 collectTableDiff 整表重写:大表事务改 1 行也 O(表大小));
|
||||||
|
// 主键变更/级联影响表整表 diff 兜底;clear/drop 表只删 KV 行。
|
||||||
const puts = {};
|
const puts = {};
|
||||||
const deletes = [];
|
const deletes = [];
|
||||||
for (const table of this.txDirtyTables) {
|
for (const table of this.txDirtyTables) {
|
||||||
if (await this.memory.hasTable(table)) {
|
if (!(await this.memory.hasTable(table))) {
|
||||||
const diff = await this.collectTableDiff(table);
|
|
||||||
Object.assign(puts, diff.puts);
|
|
||||||
deletes.push(...diff.deletes);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// 事务内 drop 的表:清理 KV 残留行
|
// 事务内 drop 的表:清理 KV 残留行
|
||||||
const all = await this.kv.getAll();
|
const all = await this.kv.getAll();
|
||||||
const prefix = this.rowPrefix(table);
|
const prefix = this.rowPrefix(table);
|
||||||
@@ -2196,6 +2267,49 @@ class KVStoreEngine {
|
|||||||
if (key.startsWith(prefix))
|
if (key.startsWith(prefix))
|
||||||
deletes.push(key);
|
deletes.push(key);
|
||||||
}
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txClearedTables.has(table)) {
|
||||||
|
// 事务内 clear 的表:删除全部 KV 行
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const prefix = this.rowPrefix(table);
|
||||||
|
for (const [key] of all) {
|
||||||
|
if (key.startsWith(prefix))
|
||||||
|
deletes.push(key);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txFullTables.has(table)) {
|
||||||
|
const diff = await this.collectTableDiff(table);
|
||||||
|
Object.assign(puts, diff.puts);
|
||||||
|
deletes.push(...diff.deletes);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const changes = this.txChanges.get(table);
|
||||||
|
if (changes && changes.size > 0) {
|
||||||
|
// v0.7.0: 增量 flush —— 单次全表扫描 + 变更集合过滤
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (!schema)
|
||||||
|
continue;
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const pending = new Map(changes);
|
||||||
|
const rows = await this.memory.find(table, { table: table });
|
||||||
|
for (const row of rows) {
|
||||||
|
const pkStr = String(row[pkCol]);
|
||||||
|
const kind = pending.get(pkStr);
|
||||||
|
if (kind !== undefined) {
|
||||||
|
if (kind === 'put')
|
||||||
|
puts[this.rowKey(table, pkStr)] = enc(JSON.stringify(row));
|
||||||
|
else
|
||||||
|
deletes.push(this.rowKey(table, pkStr));
|
||||||
|
pending.delete(pkStr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 内存中已不存在的行(后续操作删除)→ KV 行删除
|
||||||
|
for (const [pk, kind] of pending) {
|
||||||
|
if (kind === 'delete')
|
||||||
|
deletes.push(this.rowKey(table, pk));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
await this.kv.writeBatch(puts, deletes);
|
await this.kv.writeBatch(puts, deletes);
|
||||||
@@ -2203,10 +2317,15 @@ class KVStoreEngine {
|
|||||||
if (this.txSchemaChanged) {
|
if (this.txSchemaChanged) {
|
||||||
await this.persistSchema();
|
await this.persistSchema();
|
||||||
}
|
}
|
||||||
await this.kv.checkpoint();
|
// v0.7.0-perf: 移除每次 commit 的强制全量 checkpoint —— KVStore 按日志阈值
|
||||||
|
// 自动 checkpoint(日志重放保证崩溃恢复正确),大库高频事务不再 O(库大小)。
|
||||||
|
// close() 时统一 checkpoint(截断日志,重开更快)。
|
||||||
await this.memory.commitTransaction();
|
await this.memory.commitTransaction();
|
||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
async rollbackTransaction() {
|
async rollbackTransaction() {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
@@ -2216,6 +2335,9 @@ class KVStoreEngine {
|
|||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
// ---- 内部 ----
|
// ---- 内部 ----
|
||||||
ensureOpen() {
|
ensureOpen() {
|
||||||
@@ -2347,6 +2469,13 @@ function validateColumns(columns) {
|
|||||||
if (primaryKeyCount === 0) {
|
if (primaryKeyCount === 0) {
|
||||||
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 复合主键(多列 primaryKey)当前不支持 —— 所有引擎的存储布局与
|
||||||
|
// 外键引用均为单主键假设(此前静默取第一个主键,其余标记被忽略 → 语义陷阱)。
|
||||||
|
// 显式拒绝避免用户误用;复合主键列入 v0.8 路线图。
|
||||||
|
if (primaryKeyCount > 1) {
|
||||||
|
throw new DatabaseError(`Composite primary keys are not supported yet: table has ${primaryKeyCount} primary key columns. ` +
|
||||||
|
'Use a single primary key column (or a unique column combination) instead.', 'SCHEMA_ERROR');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
/** 检查字段类型(含约束校验) */
|
/** 检查字段类型(含约束校验) */
|
||||||
function checkFieldType(tableName, colName, type, value, colDef) {
|
function checkFieldType(tableName, colName, type, value, colDef) {
|
||||||
@@ -10313,6 +10442,32 @@ class QueryExecutor {
|
|||||||
plan = compileStatement(stmt.query);
|
plan = compileStatement(stmt.query);
|
||||||
}
|
}
|
||||||
catch { /* 非查询语句无 QueryPlan */ }
|
catch { /* 非查询语句无 QueryPlan */ }
|
||||||
|
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
|
||||||
|
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
|
||||||
|
let usingIndex = plan?.table ? 'none' : 'none';
|
||||||
|
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
|
||||||
|
try {
|
||||||
|
const schema = await this.engine.getTableSchema(plan.table);
|
||||||
|
if (schema) {
|
||||||
|
for (const col of Object.keys(plan.where)) {
|
||||||
|
if (col.startsWith('$'))
|
||||||
|
continue;
|
||||||
|
const colDef = schema.columns[col];
|
||||||
|
if (!colDef)
|
||||||
|
continue;
|
||||||
|
if (colDef.primaryKey) {
|
||||||
|
usingIndex = 'pk';
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if (colDef.index || colDef.unique) {
|
||||||
|
usingIndex = `index:${col}`;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch { /* schema 读取失败保持 none */ }
|
||||||
|
}
|
||||||
return {
|
return {
|
||||||
type: stmt.query.type,
|
type: stmt.query.type,
|
||||||
table: plan?.table,
|
table: plan?.table,
|
||||||
@@ -10321,7 +10476,7 @@ class QueryExecutor {
|
|||||||
orderBy: plan?.orderBy || [],
|
orderBy: plan?.orderBy || [],
|
||||||
limit: plan?.limit,
|
limit: plan?.limit,
|
||||||
offset: plan?.offset,
|
offset: plan?.offset,
|
||||||
usingIndex: plan?.table ? 'auto' : 'none',
|
usingIndex,
|
||||||
estimatedRows: rows,
|
estimatedRows: rows,
|
||||||
actualTimeMs: elapsed,
|
actualTimeMs: elapsed,
|
||||||
};
|
};
|
||||||
@@ -10826,6 +10981,14 @@ class QueryExecutor {
|
|||||||
const schema = await this.engine.getTableSchema(stmt.name);
|
const schema = await this.engine.getTableSchema(stmt.name);
|
||||||
if (!schema)
|
if (!schema)
|
||||||
return;
|
return;
|
||||||
|
// v0.7.0: ALTER ADD 主键列防护 —— 复合主键不支持(与 createSchema 校验对齐),
|
||||||
|
// 避免绕过建表校验添加第二个主键列导致语义陷阱
|
||||||
|
if (stmt.action === 'ADD' && stmt.column.primaryKey) {
|
||||||
|
const hasPk = Object.values(schema.columns).some((c) => c.primaryKey);
|
||||||
|
if (hasPk) {
|
||||||
|
throw new DatabaseError(`Composite primary keys are not supported yet: table "${stmt.name}" already has a primary key column`, 'SCHEMA_ERROR');
|
||||||
|
}
|
||||||
|
}
|
||||||
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
||||||
if (typeof this.engine.alterTable === 'function') {
|
if (typeof this.engine.alterTable === 'function') {
|
||||||
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
||||||
@@ -11441,6 +11604,88 @@ class QueryExecutor {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* metona-sqlark SQL Parameters — 参数化查询绑定
|
||||||
|
* @module sql/params
|
||||||
|
*
|
||||||
|
* v0.7.0: `db.query(sql, params)` 位置参数(`?`)支持。
|
||||||
|
* 绑定在词法层面完成:仅替换字符串字面量之外的 `?`,
|
||||||
|
* 值按 SQL 字面量编码(字符串 `''` 转义、数字/布尔/JSON 直出),
|
||||||
|
* 从根上规避 SQL 注入(不经过字符串拼接由用户自行转义)。
|
||||||
|
*/
|
||||||
|
/** 将单个参数值编码为 SQL 字面量 */
|
||||||
|
function encodeParam(value) {
|
||||||
|
if (value === null || value === undefined)
|
||||||
|
return 'NULL';
|
||||||
|
if (typeof value === 'number') {
|
||||||
|
if (Number.isFinite(value))
|
||||||
|
return String(value);
|
||||||
|
return 'NULL'; // NaN/Infinity 无 SQL 字面量 → NULL
|
||||||
|
}
|
||||||
|
if (typeof value === 'boolean')
|
||||||
|
return value ? 'TRUE' : 'FALSE';
|
||||||
|
if (typeof value === 'string')
|
||||||
|
return `'${value.replace(/'/g, "''")}'`;
|
||||||
|
// 对象/数组无 SQL 字面量(SQL 方言不支持 json 字面量),显式拒绝而非静默错配
|
||||||
|
throw new DatabaseError('Object/array query parameters are not supported by SQL binding (pass JSON strings explicitly)', 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* 将 SQL 中的位置参数 `?`(字符串字面量之外)替换为编码后的字面量。
|
||||||
|
* @param sql 含 `?` 占位符的 SQL
|
||||||
|
* @param params 位置参数数组
|
||||||
|
* @throws PARAM_ERROR 参数数量不匹配
|
||||||
|
*/
|
||||||
|
function bindParameters(sql, params) {
|
||||||
|
// undefined = 不启用绑定;[] + 含 ? 的 SQL 由循环内报 PARAM_ERROR
|
||||||
|
if (!params)
|
||||||
|
return sql;
|
||||||
|
let out = '';
|
||||||
|
let i = 0;
|
||||||
|
let pIdx = 0;
|
||||||
|
let quote = null;
|
||||||
|
while (i < sql.length) {
|
||||||
|
const ch = sql[i];
|
||||||
|
if (quote !== null) {
|
||||||
|
out += ch;
|
||||||
|
if (ch === quote) {
|
||||||
|
// SQL 标准 '' 转义:两个连续引号 = 一个引号(原样保留)
|
||||||
|
if (sql[i + 1] === quote) {
|
||||||
|
out += sql[i + 1];
|
||||||
|
i += 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
quote = null;
|
||||||
|
}
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === "'" || ch === '"') {
|
||||||
|
quote = ch;
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === '?') {
|
||||||
|
if (pIdx >= params.length) {
|
||||||
|
throw new DatabaseError(`Too few query parameters: placeholder #${pIdx + 1} has no value (got ${params.length} total)`, 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
out += encodeParam(params[pIdx]);
|
||||||
|
pIdx++;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
if (quote !== null) {
|
||||||
|
throw new DatabaseError('Unterminated string literal in SQL', 'PARSE_ERROR');
|
||||||
|
}
|
||||||
|
if (pIdx < params.length) {
|
||||||
|
throw new DatabaseError(`Too many query parameters: ${params.length} provided but only ${pIdx} placeholders`, 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Transaction — 事务管理
|
* metona-sqlark Transaction — 事务管理
|
||||||
* @module transaction
|
* @module transaction
|
||||||
@@ -11716,15 +11961,21 @@ class MetonaSqlark {
|
|||||||
return this.engine.getTableNames();
|
return this.engine.getTableNames();
|
||||||
}
|
}
|
||||||
// ---- SQL 查询 ----
|
// ---- SQL 查询 ----
|
||||||
/** 执行 SQL 字符串查询 */
|
/**
|
||||||
async query(sql) {
|
* 执行 SQL 字符串查询。
|
||||||
|
* v0.7.0: 支持位置参数(`?`)—— `db.query('SELECT * FROM t WHERE id = ?', ['1'])`。
|
||||||
|
* 参数按 SQL 字面量安全编码(字符串 '' 转义),杜绝 SQL 注入。
|
||||||
|
*/
|
||||||
|
async query(sql, params) {
|
||||||
this.ensureReady();
|
this.ensureReady();
|
||||||
const startTime = this.debug ? Date.now() : 0;
|
const startTime = this.debug ? Date.now() : 0;
|
||||||
await this.pluginManager.trigger('beforeQuery', sql);
|
await this.pluginManager.trigger('beforeQuery', sql);
|
||||||
let result;
|
let result;
|
||||||
try {
|
try {
|
||||||
|
// v0.7.0: 参数绑定(仅替换字符串字面量之外的 ?)
|
||||||
|
const boundSql = bindParameters(sql, params);
|
||||||
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
||||||
const statements = parseAll(sql);
|
const statements = parseAll(boundSql);
|
||||||
for (const stmt of statements) {
|
for (const stmt of statements) {
|
||||||
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
||||||
await this.triggerStatementHooks(stmt, 'before');
|
await this.triggerStatementHooks(stmt, 'before');
|
||||||
|
|||||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+16
-3
@@ -164,7 +164,7 @@ interface MetonaPlugin {
|
|||||||
/** 销毁 */
|
/** 销毁 */
|
||||||
destroy(): void;
|
destroy(): void;
|
||||||
}
|
}
|
||||||
declare const VERSION = "0.6.3";
|
declare const VERSION = "0.7.0";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Plugin — 插件系统
|
* metona-sqlark Plugin — 插件系统
|
||||||
@@ -676,8 +676,12 @@ declare class MetonaSqlark {
|
|||||||
dropTable(name: string): Promise<void>;
|
dropTable(name: string): Promise<void>;
|
||||||
/** 获取所有表名 */
|
/** 获取所有表名 */
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
/** 执行 SQL 字符串查询 */
|
/**
|
||||||
query(sql: string): Promise<unknown>;
|
* 执行 SQL 字符串查询。
|
||||||
|
* v0.7.0: 支持位置参数(`?`)—— `db.query('SELECT * FROM t WHERE id = ?', ['1'])`。
|
||||||
|
* 参数按 SQL 字面量安全编码(字符串 '' 转义),杜绝 SQL 注入。
|
||||||
|
*/
|
||||||
|
query(sql: string, params?: unknown[]): Promise<unknown>;
|
||||||
/**
|
/**
|
||||||
* 流式查询:逐行回调,不一次性物化全部结果(大表友好)。
|
* 流式查询:逐行回调,不一次性物化全部结果(大表友好)。
|
||||||
* 支持简单 SELECT(WHERE/LIMIT/OFFSET/列投影);
|
* 支持简单 SELECT(WHERE/LIMIT/OFFSET/列投影);
|
||||||
@@ -895,6 +899,15 @@ declare class KVStoreEngine implements IStorageEngine {
|
|||||||
private txDirtyTables;
|
private txDirtyTables;
|
||||||
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
||||||
private txSchemaChanged;
|
private txSchemaChanged;
|
||||||
|
/**
|
||||||
|
* v0.7.0: 事务行级变更记录(table → pk → put/delete)。
|
||||||
|
* commit 时按行增量 flush(此前整表 diff:大表事务改 1 行也重写全表)。
|
||||||
|
*/
|
||||||
|
private txChanges;
|
||||||
|
/** v0.7.0: 无法行级追踪的表(主键变更/级联影响表)→ commit 时整表 diff */
|
||||||
|
private txFullTables;
|
||||||
|
/** v0.7.0: 事务内 clear 的表 → commit 时清空 KV 行 */
|
||||||
|
private txClearedTables;
|
||||||
constructor(medium?: IStorageBackend, checkpointThreshold?: number);
|
constructor(medium?: IStorageBackend, checkpointThreshold?: number);
|
||||||
private rowKey;
|
private rowKey;
|
||||||
private rowPrefix;
|
private rowPrefix;
|
||||||
|
|||||||
Vendored
+264
-13
@@ -30,7 +30,7 @@ class DatabaseError extends Error {
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// 版本
|
// 版本
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
const VERSION = '0.6.3';
|
const VERSION = '0.7.0';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Shared WHERE Matcher — 统一的条件匹配逻辑
|
* metona-sqlark Shared WHERE Matcher — 统一的条件匹配逻辑
|
||||||
@@ -1854,6 +1854,15 @@ class KVStoreEngine {
|
|||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
/**
|
||||||
|
* v0.7.0: 事务行级变更记录(table → pk → put/delete)。
|
||||||
|
* commit 时按行增量 flush(此前整表 diff:大表事务改 1 行也重写全表)。
|
||||||
|
*/
|
||||||
|
this.txChanges = new Map();
|
||||||
|
/** v0.7.0: 无法行级追踪的表(主键变更/级联影响表)→ commit 时整表 diff */
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
/** v0.7.0: 事务内 clear 的表 → commit 时清空 KV 行 */
|
||||||
|
this.txClearedTables = new Set();
|
||||||
this.kv = new KVStore(medium, checkpointThreshold);
|
this.kv = new KVStore(medium, checkpointThreshold);
|
||||||
}
|
}
|
||||||
// ---- 行 key 编解码 ----
|
// ---- 行 key 编解码 ----
|
||||||
@@ -1927,6 +1936,12 @@ class KVStoreEngine {
|
|||||||
}
|
}
|
||||||
catch { /* ignore */ }
|
catch { /* ignore */ }
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 关闭前 checkpoint(截断日志,重开更快)。
|
||||||
|
// 失败不阻塞关闭(日志已持久,重开可全量重放)。
|
||||||
|
try {
|
||||||
|
await this.kv.checkpoint();
|
||||||
|
}
|
||||||
|
catch { /* 数据在日志中,重开放心重放 */ }
|
||||||
await this.kv.close();
|
await this.kv.close();
|
||||||
await this.memory.close();
|
await this.memory.close();
|
||||||
this.opened = false;
|
this.opened = false;
|
||||||
@@ -2029,6 +2044,20 @@ class KVStoreEngine {
|
|||||||
const pks = await this.memory.insert(tableName, rows);
|
const pks = await this.memory.insert(tableName, rows);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内 clear 后又写入 → 清空语义被覆盖,整表 diff 兜底
|
||||||
|
if (this.txClearedTables.has(tableName)) {
|
||||||
|
this.txClearedTables.delete(tableName);
|
||||||
|
this.txFullTables.add(tableName);
|
||||||
|
return pks;
|
||||||
|
}
|
||||||
|
// v0.7.0: 行级变更记录(增量 flush)
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks)
|
||||||
|
changes.set(pk, 'put');
|
||||||
return pks;
|
return pks;
|
||||||
}
|
}
|
||||||
// 增量持久化(原子 putMany)
|
// 增量持久化(原子 putMany)
|
||||||
@@ -2063,6 +2092,30 @@ class KVStoreEngine {
|
|||||||
const count = await this.memory.update(tableName, query, updates);
|
const count = await this.memory.update(tableName, query, updates);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 主键变更无法行级追踪(旧键删除+新键落盘+级联),
|
||||||
|
// 相关表整表 diff 兜底;普通更新记录受影响行
|
||||||
|
if (pkChanged) {
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of affected)
|
||||||
|
changes.set(pk, 'put');
|
||||||
|
// 级联影响表(理论上非主键更新不级联,防御性兜底)
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts = {};
|
const puts = {};
|
||||||
@@ -2112,6 +2165,20 @@ class KVStoreEngine {
|
|||||||
const count = await this.memory.delete(tableName, query);
|
const count = await this.memory.delete(tableName, query);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 删除行记录 delete;级联影响表整表 diff 兜底
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks)
|
||||||
|
changes.set(pk, 'delete');
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts = {};
|
const puts = {};
|
||||||
@@ -2136,6 +2203,9 @@ class KVStoreEngine {
|
|||||||
await this.memory.clear(tableName);
|
await this.memory.clear(tableName);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内清空 → commit 时删除全部 KV 行(比整表 diff 更高效)
|
||||||
|
this.txClearedTables.add(tableName);
|
||||||
|
this.txChanges.delete(tableName);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const diff = await this.collectTableDiff(tableName);
|
const diff = await this.collectTableDiff(tableName);
|
||||||
@@ -2169,22 +2239,23 @@ class KVStoreEngine {
|
|||||||
this.txActive = true;
|
this.txActive = true;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
async commitTransaction() {
|
async commitTransaction() {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
if (!this.txActive)
|
if (!this.txActive)
|
||||||
throw new DatabaseError('No active transaction', 'TX_NONE');
|
throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
||||||
// 多表事务中途崩溃/失败不会出现"部分表已提交")
|
// 多表事务中途崩溃/失败不会出现"部分表已提交")。
|
||||||
|
// v0.7.0: 行级增量 flush —— 普通 insert/update/delete 仅写事务内改动的行
|
||||||
|
// (此前 collectTableDiff 整表重写:大表事务改 1 行也 O(表大小));
|
||||||
|
// 主键变更/级联影响表整表 diff 兜底;clear/drop 表只删 KV 行。
|
||||||
const puts = {};
|
const puts = {};
|
||||||
const deletes = [];
|
const deletes = [];
|
||||||
for (const table of this.txDirtyTables) {
|
for (const table of this.txDirtyTables) {
|
||||||
if (await this.memory.hasTable(table)) {
|
if (!(await this.memory.hasTable(table))) {
|
||||||
const diff = await this.collectTableDiff(table);
|
|
||||||
Object.assign(puts, diff.puts);
|
|
||||||
deletes.push(...diff.deletes);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// 事务内 drop 的表:清理 KV 残留行
|
// 事务内 drop 的表:清理 KV 残留行
|
||||||
const all = await this.kv.getAll();
|
const all = await this.kv.getAll();
|
||||||
const prefix = this.rowPrefix(table);
|
const prefix = this.rowPrefix(table);
|
||||||
@@ -2192,6 +2263,49 @@ class KVStoreEngine {
|
|||||||
if (key.startsWith(prefix))
|
if (key.startsWith(prefix))
|
||||||
deletes.push(key);
|
deletes.push(key);
|
||||||
}
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txClearedTables.has(table)) {
|
||||||
|
// 事务内 clear 的表:删除全部 KV 行
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const prefix = this.rowPrefix(table);
|
||||||
|
for (const [key] of all) {
|
||||||
|
if (key.startsWith(prefix))
|
||||||
|
deletes.push(key);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txFullTables.has(table)) {
|
||||||
|
const diff = await this.collectTableDiff(table);
|
||||||
|
Object.assign(puts, diff.puts);
|
||||||
|
deletes.push(...diff.deletes);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const changes = this.txChanges.get(table);
|
||||||
|
if (changes && changes.size > 0) {
|
||||||
|
// v0.7.0: 增量 flush —— 单次全表扫描 + 变更集合过滤
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (!schema)
|
||||||
|
continue;
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const pending = new Map(changes);
|
||||||
|
const rows = await this.memory.find(table, { table: table });
|
||||||
|
for (const row of rows) {
|
||||||
|
const pkStr = String(row[pkCol]);
|
||||||
|
const kind = pending.get(pkStr);
|
||||||
|
if (kind !== undefined) {
|
||||||
|
if (kind === 'put')
|
||||||
|
puts[this.rowKey(table, pkStr)] = enc(JSON.stringify(row));
|
||||||
|
else
|
||||||
|
deletes.push(this.rowKey(table, pkStr));
|
||||||
|
pending.delete(pkStr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 内存中已不存在的行(后续操作删除)→ KV 行删除
|
||||||
|
for (const [pk, kind] of pending) {
|
||||||
|
if (kind === 'delete')
|
||||||
|
deletes.push(this.rowKey(table, pk));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
await this.kv.writeBatch(puts, deletes);
|
await this.kv.writeBatch(puts, deletes);
|
||||||
@@ -2199,10 +2313,15 @@ class KVStoreEngine {
|
|||||||
if (this.txSchemaChanged) {
|
if (this.txSchemaChanged) {
|
||||||
await this.persistSchema();
|
await this.persistSchema();
|
||||||
}
|
}
|
||||||
await this.kv.checkpoint();
|
// v0.7.0-perf: 移除每次 commit 的强制全量 checkpoint —— KVStore 按日志阈值
|
||||||
|
// 自动 checkpoint(日志重放保证崩溃恢复正确),大库高频事务不再 O(库大小)。
|
||||||
|
// close() 时统一 checkpoint(截断日志,重开更快)。
|
||||||
await this.memory.commitTransaction();
|
await this.memory.commitTransaction();
|
||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
async rollbackTransaction() {
|
async rollbackTransaction() {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
@@ -2212,6 +2331,9 @@ class KVStoreEngine {
|
|||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
// ---- 内部 ----
|
// ---- 内部 ----
|
||||||
ensureOpen() {
|
ensureOpen() {
|
||||||
@@ -2343,6 +2465,13 @@ function validateColumns(columns) {
|
|||||||
if (primaryKeyCount === 0) {
|
if (primaryKeyCount === 0) {
|
||||||
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 复合主键(多列 primaryKey)当前不支持 —— 所有引擎的存储布局与
|
||||||
|
// 外键引用均为单主键假设(此前静默取第一个主键,其余标记被忽略 → 语义陷阱)。
|
||||||
|
// 显式拒绝避免用户误用;复合主键列入 v0.8 路线图。
|
||||||
|
if (primaryKeyCount > 1) {
|
||||||
|
throw new DatabaseError(`Composite primary keys are not supported yet: table has ${primaryKeyCount} primary key columns. ` +
|
||||||
|
'Use a single primary key column (or a unique column combination) instead.', 'SCHEMA_ERROR');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
/** 检查字段类型(含约束校验) */
|
/** 检查字段类型(含约束校验) */
|
||||||
function checkFieldType(tableName, colName, type, value, colDef) {
|
function checkFieldType(tableName, colName, type, value, colDef) {
|
||||||
@@ -10309,6 +10438,32 @@ class QueryExecutor {
|
|||||||
plan = compileStatement(stmt.query);
|
plan = compileStatement(stmt.query);
|
||||||
}
|
}
|
||||||
catch { /* 非查询语句无 QueryPlan */ }
|
catch { /* 非查询语句无 QueryPlan */ }
|
||||||
|
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
|
||||||
|
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
|
||||||
|
let usingIndex = plan?.table ? 'none' : 'none';
|
||||||
|
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
|
||||||
|
try {
|
||||||
|
const schema = await this.engine.getTableSchema(plan.table);
|
||||||
|
if (schema) {
|
||||||
|
for (const col of Object.keys(plan.where)) {
|
||||||
|
if (col.startsWith('$'))
|
||||||
|
continue;
|
||||||
|
const colDef = schema.columns[col];
|
||||||
|
if (!colDef)
|
||||||
|
continue;
|
||||||
|
if (colDef.primaryKey) {
|
||||||
|
usingIndex = 'pk';
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if (colDef.index || colDef.unique) {
|
||||||
|
usingIndex = `index:${col}`;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch { /* schema 读取失败保持 none */ }
|
||||||
|
}
|
||||||
return {
|
return {
|
||||||
type: stmt.query.type,
|
type: stmt.query.type,
|
||||||
table: plan?.table,
|
table: plan?.table,
|
||||||
@@ -10317,7 +10472,7 @@ class QueryExecutor {
|
|||||||
orderBy: plan?.orderBy || [],
|
orderBy: plan?.orderBy || [],
|
||||||
limit: plan?.limit,
|
limit: plan?.limit,
|
||||||
offset: plan?.offset,
|
offset: plan?.offset,
|
||||||
usingIndex: plan?.table ? 'auto' : 'none',
|
usingIndex,
|
||||||
estimatedRows: rows,
|
estimatedRows: rows,
|
||||||
actualTimeMs: elapsed,
|
actualTimeMs: elapsed,
|
||||||
};
|
};
|
||||||
@@ -10822,6 +10977,14 @@ class QueryExecutor {
|
|||||||
const schema = await this.engine.getTableSchema(stmt.name);
|
const schema = await this.engine.getTableSchema(stmt.name);
|
||||||
if (!schema)
|
if (!schema)
|
||||||
return;
|
return;
|
||||||
|
// v0.7.0: ALTER ADD 主键列防护 —— 复合主键不支持(与 createSchema 校验对齐),
|
||||||
|
// 避免绕过建表校验添加第二个主键列导致语义陷阱
|
||||||
|
if (stmt.action === 'ADD' && stmt.column.primaryKey) {
|
||||||
|
const hasPk = Object.values(schema.columns).some((c) => c.primaryKey);
|
||||||
|
if (hasPk) {
|
||||||
|
throw new DatabaseError(`Composite primary keys are not supported yet: table "${stmt.name}" already has a primary key column`, 'SCHEMA_ERROR');
|
||||||
|
}
|
||||||
|
}
|
||||||
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
||||||
if (typeof this.engine.alterTable === 'function') {
|
if (typeof this.engine.alterTable === 'function') {
|
||||||
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
||||||
@@ -11437,6 +11600,88 @@ class QueryExecutor {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* metona-sqlark SQL Parameters — 参数化查询绑定
|
||||||
|
* @module sql/params
|
||||||
|
*
|
||||||
|
* v0.7.0: `db.query(sql, params)` 位置参数(`?`)支持。
|
||||||
|
* 绑定在词法层面完成:仅替换字符串字面量之外的 `?`,
|
||||||
|
* 值按 SQL 字面量编码(字符串 `''` 转义、数字/布尔/JSON 直出),
|
||||||
|
* 从根上规避 SQL 注入(不经过字符串拼接由用户自行转义)。
|
||||||
|
*/
|
||||||
|
/** 将单个参数值编码为 SQL 字面量 */
|
||||||
|
function encodeParam(value) {
|
||||||
|
if (value === null || value === undefined)
|
||||||
|
return 'NULL';
|
||||||
|
if (typeof value === 'number') {
|
||||||
|
if (Number.isFinite(value))
|
||||||
|
return String(value);
|
||||||
|
return 'NULL'; // NaN/Infinity 无 SQL 字面量 → NULL
|
||||||
|
}
|
||||||
|
if (typeof value === 'boolean')
|
||||||
|
return value ? 'TRUE' : 'FALSE';
|
||||||
|
if (typeof value === 'string')
|
||||||
|
return `'${value.replace(/'/g, "''")}'`;
|
||||||
|
// 对象/数组无 SQL 字面量(SQL 方言不支持 json 字面量),显式拒绝而非静默错配
|
||||||
|
throw new DatabaseError('Object/array query parameters are not supported by SQL binding (pass JSON strings explicitly)', 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* 将 SQL 中的位置参数 `?`(字符串字面量之外)替换为编码后的字面量。
|
||||||
|
* @param sql 含 `?` 占位符的 SQL
|
||||||
|
* @param params 位置参数数组
|
||||||
|
* @throws PARAM_ERROR 参数数量不匹配
|
||||||
|
*/
|
||||||
|
function bindParameters(sql, params) {
|
||||||
|
// undefined = 不启用绑定;[] + 含 ? 的 SQL 由循环内报 PARAM_ERROR
|
||||||
|
if (!params)
|
||||||
|
return sql;
|
||||||
|
let out = '';
|
||||||
|
let i = 0;
|
||||||
|
let pIdx = 0;
|
||||||
|
let quote = null;
|
||||||
|
while (i < sql.length) {
|
||||||
|
const ch = sql[i];
|
||||||
|
if (quote !== null) {
|
||||||
|
out += ch;
|
||||||
|
if (ch === quote) {
|
||||||
|
// SQL 标准 '' 转义:两个连续引号 = 一个引号(原样保留)
|
||||||
|
if (sql[i + 1] === quote) {
|
||||||
|
out += sql[i + 1];
|
||||||
|
i += 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
quote = null;
|
||||||
|
}
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === "'" || ch === '"') {
|
||||||
|
quote = ch;
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === '?') {
|
||||||
|
if (pIdx >= params.length) {
|
||||||
|
throw new DatabaseError(`Too few query parameters: placeholder #${pIdx + 1} has no value (got ${params.length} total)`, 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
out += encodeParam(params[pIdx]);
|
||||||
|
pIdx++;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
if (quote !== null) {
|
||||||
|
throw new DatabaseError('Unterminated string literal in SQL', 'PARSE_ERROR');
|
||||||
|
}
|
||||||
|
if (pIdx < params.length) {
|
||||||
|
throw new DatabaseError(`Too many query parameters: ${params.length} provided but only ${pIdx} placeholders`, 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Transaction — 事务管理
|
* metona-sqlark Transaction — 事务管理
|
||||||
* @module transaction
|
* @module transaction
|
||||||
@@ -11712,15 +11957,21 @@ class MetonaSqlark {
|
|||||||
return this.engine.getTableNames();
|
return this.engine.getTableNames();
|
||||||
}
|
}
|
||||||
// ---- SQL 查询 ----
|
// ---- SQL 查询 ----
|
||||||
/** 执行 SQL 字符串查询 */
|
/**
|
||||||
async query(sql) {
|
* 执行 SQL 字符串查询。
|
||||||
|
* v0.7.0: 支持位置参数(`?`)—— `db.query('SELECT * FROM t WHERE id = ?', ['1'])`。
|
||||||
|
* 参数按 SQL 字面量安全编码(字符串 '' 转义),杜绝 SQL 注入。
|
||||||
|
*/
|
||||||
|
async query(sql, params) {
|
||||||
this.ensureReady();
|
this.ensureReady();
|
||||||
const startTime = this.debug ? Date.now() : 0;
|
const startTime = this.debug ? Date.now() : 0;
|
||||||
await this.pluginManager.trigger('beforeQuery', sql);
|
await this.pluginManager.trigger('beforeQuery', sql);
|
||||||
let result;
|
let result;
|
||||||
try {
|
try {
|
||||||
|
// v0.7.0: 参数绑定(仅替换字符串字面量之外的 ?)
|
||||||
|
const boundSql = bindParameters(sql, params);
|
||||||
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
||||||
const statements = parseAll(sql);
|
const statements = parseAll(boundSql);
|
||||||
for (const stmt of statements) {
|
for (const stmt of statements) {
|
||||||
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
||||||
await this.triggerStatementHooks(stmt, 'before');
|
await this.triggerStatementHooks(stmt, 'before');
|
||||||
|
|||||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+264
-13
@@ -36,7 +36,7 @@
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// 版本
|
// 版本
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
const VERSION = '0.6.3';
|
const VERSION = '0.7.0';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Shared WHERE Matcher — 统一的条件匹配逻辑
|
* metona-sqlark Shared WHERE Matcher — 统一的条件匹配逻辑
|
||||||
@@ -1860,6 +1860,15 @@
|
|||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
/**
|
||||||
|
* v0.7.0: 事务行级变更记录(table → pk → put/delete)。
|
||||||
|
* commit 时按行增量 flush(此前整表 diff:大表事务改 1 行也重写全表)。
|
||||||
|
*/
|
||||||
|
this.txChanges = new Map();
|
||||||
|
/** v0.7.0: 无法行级追踪的表(主键变更/级联影响表)→ commit 时整表 diff */
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
/** v0.7.0: 事务内 clear 的表 → commit 时清空 KV 行 */
|
||||||
|
this.txClearedTables = new Set();
|
||||||
this.kv = new KVStore(medium, checkpointThreshold);
|
this.kv = new KVStore(medium, checkpointThreshold);
|
||||||
}
|
}
|
||||||
// ---- 行 key 编解码 ----
|
// ---- 行 key 编解码 ----
|
||||||
@@ -1933,6 +1942,12 @@
|
|||||||
}
|
}
|
||||||
catch { /* ignore */ }
|
catch { /* ignore */ }
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 关闭前 checkpoint(截断日志,重开更快)。
|
||||||
|
// 失败不阻塞关闭(日志已持久,重开可全量重放)。
|
||||||
|
try {
|
||||||
|
await this.kv.checkpoint();
|
||||||
|
}
|
||||||
|
catch { /* 数据在日志中,重开放心重放 */ }
|
||||||
await this.kv.close();
|
await this.kv.close();
|
||||||
await this.memory.close();
|
await this.memory.close();
|
||||||
this.opened = false;
|
this.opened = false;
|
||||||
@@ -2035,6 +2050,20 @@
|
|||||||
const pks = await this.memory.insert(tableName, rows);
|
const pks = await this.memory.insert(tableName, rows);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内 clear 后又写入 → 清空语义被覆盖,整表 diff 兜底
|
||||||
|
if (this.txClearedTables.has(tableName)) {
|
||||||
|
this.txClearedTables.delete(tableName);
|
||||||
|
this.txFullTables.add(tableName);
|
||||||
|
return pks;
|
||||||
|
}
|
||||||
|
// v0.7.0: 行级变更记录(增量 flush)
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks)
|
||||||
|
changes.set(pk, 'put');
|
||||||
return pks;
|
return pks;
|
||||||
}
|
}
|
||||||
// 增量持久化(原子 putMany)
|
// 增量持久化(原子 putMany)
|
||||||
@@ -2069,6 +2098,30 @@
|
|||||||
const count = await this.memory.update(tableName, query, updates);
|
const count = await this.memory.update(tableName, query, updates);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 主键变更无法行级追踪(旧键删除+新键落盘+级联),
|
||||||
|
// 相关表整表 diff 兜底;普通更新记录受影响行
|
||||||
|
if (pkChanged) {
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of affected)
|
||||||
|
changes.set(pk, 'put');
|
||||||
|
// 级联影响表(理论上非主键更新不级联,防御性兜底)
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts = {};
|
const puts = {};
|
||||||
@@ -2118,6 +2171,20 @@
|
|||||||
const count = await this.memory.delete(tableName, query);
|
const count = await this.memory.delete(tableName, query);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 删除行记录 delete;级联影响表整表 diff 兜底
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks)
|
||||||
|
changes.set(pk, 'delete');
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts = {};
|
const puts = {};
|
||||||
@@ -2142,6 +2209,9 @@
|
|||||||
await this.memory.clear(tableName);
|
await this.memory.clear(tableName);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内清空 → commit 时删除全部 KV 行(比整表 diff 更高效)
|
||||||
|
this.txClearedTables.add(tableName);
|
||||||
|
this.txChanges.delete(tableName);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const diff = await this.collectTableDiff(tableName);
|
const diff = await this.collectTableDiff(tableName);
|
||||||
@@ -2175,22 +2245,23 @@
|
|||||||
this.txActive = true;
|
this.txActive = true;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
async commitTransaction() {
|
async commitTransaction() {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
if (!this.txActive)
|
if (!this.txActive)
|
||||||
throw new DatabaseError('No active transaction', 'TX_NONE');
|
throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
||||||
// 多表事务中途崩溃/失败不会出现"部分表已提交")
|
// 多表事务中途崩溃/失败不会出现"部分表已提交")。
|
||||||
|
// v0.7.0: 行级增量 flush —— 普通 insert/update/delete 仅写事务内改动的行
|
||||||
|
// (此前 collectTableDiff 整表重写:大表事务改 1 行也 O(表大小));
|
||||||
|
// 主键变更/级联影响表整表 diff 兜底;clear/drop 表只删 KV 行。
|
||||||
const puts = {};
|
const puts = {};
|
||||||
const deletes = [];
|
const deletes = [];
|
||||||
for (const table of this.txDirtyTables) {
|
for (const table of this.txDirtyTables) {
|
||||||
if (await this.memory.hasTable(table)) {
|
if (!(await this.memory.hasTable(table))) {
|
||||||
const diff = await this.collectTableDiff(table);
|
|
||||||
Object.assign(puts, diff.puts);
|
|
||||||
deletes.push(...diff.deletes);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// 事务内 drop 的表:清理 KV 残留行
|
// 事务内 drop 的表:清理 KV 残留行
|
||||||
const all = await this.kv.getAll();
|
const all = await this.kv.getAll();
|
||||||
const prefix = this.rowPrefix(table);
|
const prefix = this.rowPrefix(table);
|
||||||
@@ -2198,6 +2269,49 @@
|
|||||||
if (key.startsWith(prefix))
|
if (key.startsWith(prefix))
|
||||||
deletes.push(key);
|
deletes.push(key);
|
||||||
}
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txClearedTables.has(table)) {
|
||||||
|
// 事务内 clear 的表:删除全部 KV 行
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const prefix = this.rowPrefix(table);
|
||||||
|
for (const [key] of all) {
|
||||||
|
if (key.startsWith(prefix))
|
||||||
|
deletes.push(key);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txFullTables.has(table)) {
|
||||||
|
const diff = await this.collectTableDiff(table);
|
||||||
|
Object.assign(puts, diff.puts);
|
||||||
|
deletes.push(...diff.deletes);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const changes = this.txChanges.get(table);
|
||||||
|
if (changes && changes.size > 0) {
|
||||||
|
// v0.7.0: 增量 flush —— 单次全表扫描 + 变更集合过滤
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (!schema)
|
||||||
|
continue;
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const pending = new Map(changes);
|
||||||
|
const rows = await this.memory.find(table, { table: table });
|
||||||
|
for (const row of rows) {
|
||||||
|
const pkStr = String(row[pkCol]);
|
||||||
|
const kind = pending.get(pkStr);
|
||||||
|
if (kind !== undefined) {
|
||||||
|
if (kind === 'put')
|
||||||
|
puts[this.rowKey(table, pkStr)] = enc(JSON.stringify(row));
|
||||||
|
else
|
||||||
|
deletes.push(this.rowKey(table, pkStr));
|
||||||
|
pending.delete(pkStr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 内存中已不存在的行(后续操作删除)→ KV 行删除
|
||||||
|
for (const [pk, kind] of pending) {
|
||||||
|
if (kind === 'delete')
|
||||||
|
deletes.push(this.rowKey(table, pk));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
await this.kv.writeBatch(puts, deletes);
|
await this.kv.writeBatch(puts, deletes);
|
||||||
@@ -2205,10 +2319,15 @@
|
|||||||
if (this.txSchemaChanged) {
|
if (this.txSchemaChanged) {
|
||||||
await this.persistSchema();
|
await this.persistSchema();
|
||||||
}
|
}
|
||||||
await this.kv.checkpoint();
|
// v0.7.0-perf: 移除每次 commit 的强制全量 checkpoint —— KVStore 按日志阈值
|
||||||
|
// 自动 checkpoint(日志重放保证崩溃恢复正确),大库高频事务不再 O(库大小)。
|
||||||
|
// close() 时统一 checkpoint(截断日志,重开更快)。
|
||||||
await this.memory.commitTransaction();
|
await this.memory.commitTransaction();
|
||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
async rollbackTransaction() {
|
async rollbackTransaction() {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
@@ -2218,6 +2337,9 @@
|
|||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
// ---- 内部 ----
|
// ---- 内部 ----
|
||||||
ensureOpen() {
|
ensureOpen() {
|
||||||
@@ -2349,6 +2471,13 @@
|
|||||||
if (primaryKeyCount === 0) {
|
if (primaryKeyCount === 0) {
|
||||||
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 复合主键(多列 primaryKey)当前不支持 —— 所有引擎的存储布局与
|
||||||
|
// 外键引用均为单主键假设(此前静默取第一个主键,其余标记被忽略 → 语义陷阱)。
|
||||||
|
// 显式拒绝避免用户误用;复合主键列入 v0.8 路线图。
|
||||||
|
if (primaryKeyCount > 1) {
|
||||||
|
throw new DatabaseError(`Composite primary keys are not supported yet: table has ${primaryKeyCount} primary key columns. ` +
|
||||||
|
'Use a single primary key column (or a unique column combination) instead.', 'SCHEMA_ERROR');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
/** 检查字段类型(含约束校验) */
|
/** 检查字段类型(含约束校验) */
|
||||||
function checkFieldType(tableName, colName, type, value, colDef) {
|
function checkFieldType(tableName, colName, type, value, colDef) {
|
||||||
@@ -10315,6 +10444,32 @@
|
|||||||
plan = compileStatement(stmt.query);
|
plan = compileStatement(stmt.query);
|
||||||
}
|
}
|
||||||
catch { /* 非查询语句无 QueryPlan */ }
|
catch { /* 非查询语句无 QueryPlan */ }
|
||||||
|
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
|
||||||
|
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
|
||||||
|
let usingIndex = plan?.table ? 'none' : 'none';
|
||||||
|
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
|
||||||
|
try {
|
||||||
|
const schema = await this.engine.getTableSchema(plan.table);
|
||||||
|
if (schema) {
|
||||||
|
for (const col of Object.keys(plan.where)) {
|
||||||
|
if (col.startsWith('$'))
|
||||||
|
continue;
|
||||||
|
const colDef = schema.columns[col];
|
||||||
|
if (!colDef)
|
||||||
|
continue;
|
||||||
|
if (colDef.primaryKey) {
|
||||||
|
usingIndex = 'pk';
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if (colDef.index || colDef.unique) {
|
||||||
|
usingIndex = `index:${col}`;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch { /* schema 读取失败保持 none */ }
|
||||||
|
}
|
||||||
return {
|
return {
|
||||||
type: stmt.query.type,
|
type: stmt.query.type,
|
||||||
table: plan?.table,
|
table: plan?.table,
|
||||||
@@ -10323,7 +10478,7 @@
|
|||||||
orderBy: plan?.orderBy || [],
|
orderBy: plan?.orderBy || [],
|
||||||
limit: plan?.limit,
|
limit: plan?.limit,
|
||||||
offset: plan?.offset,
|
offset: plan?.offset,
|
||||||
usingIndex: plan?.table ? 'auto' : 'none',
|
usingIndex,
|
||||||
estimatedRows: rows,
|
estimatedRows: rows,
|
||||||
actualTimeMs: elapsed,
|
actualTimeMs: elapsed,
|
||||||
};
|
};
|
||||||
@@ -10828,6 +10983,14 @@
|
|||||||
const schema = await this.engine.getTableSchema(stmt.name);
|
const schema = await this.engine.getTableSchema(stmt.name);
|
||||||
if (!schema)
|
if (!schema)
|
||||||
return;
|
return;
|
||||||
|
// v0.7.0: ALTER ADD 主键列防护 —— 复合主键不支持(与 createSchema 校验对齐),
|
||||||
|
// 避免绕过建表校验添加第二个主键列导致语义陷阱
|
||||||
|
if (stmt.action === 'ADD' && stmt.column.primaryKey) {
|
||||||
|
const hasPk = Object.values(schema.columns).some((c) => c.primaryKey);
|
||||||
|
if (hasPk) {
|
||||||
|
throw new DatabaseError(`Composite primary keys are not supported yet: table "${stmt.name}" already has a primary key column`, 'SCHEMA_ERROR');
|
||||||
|
}
|
||||||
|
}
|
||||||
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
||||||
if (typeof this.engine.alterTable === 'function') {
|
if (typeof this.engine.alterTable === 'function') {
|
||||||
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
||||||
@@ -11443,6 +11606,88 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* metona-sqlark SQL Parameters — 参数化查询绑定
|
||||||
|
* @module sql/params
|
||||||
|
*
|
||||||
|
* v0.7.0: `db.query(sql, params)` 位置参数(`?`)支持。
|
||||||
|
* 绑定在词法层面完成:仅替换字符串字面量之外的 `?`,
|
||||||
|
* 值按 SQL 字面量编码(字符串 `''` 转义、数字/布尔/JSON 直出),
|
||||||
|
* 从根上规避 SQL 注入(不经过字符串拼接由用户自行转义)。
|
||||||
|
*/
|
||||||
|
/** 将单个参数值编码为 SQL 字面量 */
|
||||||
|
function encodeParam(value) {
|
||||||
|
if (value === null || value === undefined)
|
||||||
|
return 'NULL';
|
||||||
|
if (typeof value === 'number') {
|
||||||
|
if (Number.isFinite(value))
|
||||||
|
return String(value);
|
||||||
|
return 'NULL'; // NaN/Infinity 无 SQL 字面量 → NULL
|
||||||
|
}
|
||||||
|
if (typeof value === 'boolean')
|
||||||
|
return value ? 'TRUE' : 'FALSE';
|
||||||
|
if (typeof value === 'string')
|
||||||
|
return `'${value.replace(/'/g, "''")}'`;
|
||||||
|
// 对象/数组无 SQL 字面量(SQL 方言不支持 json 字面量),显式拒绝而非静默错配
|
||||||
|
throw new DatabaseError('Object/array query parameters are not supported by SQL binding (pass JSON strings explicitly)', 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* 将 SQL 中的位置参数 `?`(字符串字面量之外)替换为编码后的字面量。
|
||||||
|
* @param sql 含 `?` 占位符的 SQL
|
||||||
|
* @param params 位置参数数组
|
||||||
|
* @throws PARAM_ERROR 参数数量不匹配
|
||||||
|
*/
|
||||||
|
function bindParameters(sql, params) {
|
||||||
|
// undefined = 不启用绑定;[] + 含 ? 的 SQL 由循环内报 PARAM_ERROR
|
||||||
|
if (!params)
|
||||||
|
return sql;
|
||||||
|
let out = '';
|
||||||
|
let i = 0;
|
||||||
|
let pIdx = 0;
|
||||||
|
let quote = null;
|
||||||
|
while (i < sql.length) {
|
||||||
|
const ch = sql[i];
|
||||||
|
if (quote !== null) {
|
||||||
|
out += ch;
|
||||||
|
if (ch === quote) {
|
||||||
|
// SQL 标准 '' 转义:两个连续引号 = 一个引号(原样保留)
|
||||||
|
if (sql[i + 1] === quote) {
|
||||||
|
out += sql[i + 1];
|
||||||
|
i += 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
quote = null;
|
||||||
|
}
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === "'" || ch === '"') {
|
||||||
|
quote = ch;
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === '?') {
|
||||||
|
if (pIdx >= params.length) {
|
||||||
|
throw new DatabaseError(`Too few query parameters: placeholder #${pIdx + 1} has no value (got ${params.length} total)`, 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
out += encodeParam(params[pIdx]);
|
||||||
|
pIdx++;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
if (quote !== null) {
|
||||||
|
throw new DatabaseError('Unterminated string literal in SQL', 'PARSE_ERROR');
|
||||||
|
}
|
||||||
|
if (pIdx < params.length) {
|
||||||
|
throw new DatabaseError(`Too many query parameters: ${params.length} provided but only ${pIdx} placeholders`, 'PARAM_ERROR');
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Transaction — 事务管理
|
* metona-sqlark Transaction — 事务管理
|
||||||
* @module transaction
|
* @module transaction
|
||||||
@@ -11718,15 +11963,21 @@
|
|||||||
return this.engine.getTableNames();
|
return this.engine.getTableNames();
|
||||||
}
|
}
|
||||||
// ---- SQL 查询 ----
|
// ---- SQL 查询 ----
|
||||||
/** 执行 SQL 字符串查询 */
|
/**
|
||||||
async query(sql) {
|
* 执行 SQL 字符串查询。
|
||||||
|
* v0.7.0: 支持位置参数(`?`)—— `db.query('SELECT * FROM t WHERE id = ?', ['1'])`。
|
||||||
|
* 参数按 SQL 字面量安全编码(字符串 '' 转义),杜绝 SQL 注入。
|
||||||
|
*/
|
||||||
|
async query(sql, params) {
|
||||||
this.ensureReady();
|
this.ensureReady();
|
||||||
const startTime = this.debug ? Date.now() : 0;
|
const startTime = this.debug ? Date.now() : 0;
|
||||||
await this.pluginManager.trigger('beforeQuery', sql);
|
await this.pluginManager.trigger('beforeQuery', sql);
|
||||||
let result;
|
let result;
|
||||||
try {
|
try {
|
||||||
|
// v0.7.0: 参数绑定(仅替换字符串字面量之外的 ?)
|
||||||
|
const boundSql = bindParameters(sql, params);
|
||||||
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
||||||
const statements = parseAll(sql);
|
const statements = parseAll(boundSql);
|
||||||
for (const stmt of statements) {
|
for (const stmt of statements) {
|
||||||
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
||||||
await this.triggerStatementHooks(stmt, 'before');
|
await this.triggerStatementHooks(stmt, 'before');
|
||||||
|
|||||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@metona-team/metona-sqlark",
|
"name": "@metona-team/metona-sqlark",
|
||||||
"version": "0.6.3",
|
"version": "0.7.0",
|
||||||
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "dist/metona-sqlark.cjs",
|
"main": "dist/metona-sqlark.cjs",
|
||||||
|
|||||||
+1
-1
@@ -214,4 +214,4 @@ export class DatabaseError extends Error {
|
|||||||
// 版本
|
// 版本
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export const VERSION = '0.6.3';
|
export const VERSION = '0.7.0';
|
||||||
|
|||||||
+10
-3
@@ -16,6 +16,7 @@ import { Table } from './table/table';
|
|||||||
import { createSchema } from './table/schema';
|
import { createSchema } from './table/schema';
|
||||||
import { QueryExecutor } from './query/executor';
|
import { QueryExecutor } from './query/executor';
|
||||||
import { parseAll } from './sql/parser';
|
import { parseAll } from './sql/parser';
|
||||||
|
import { bindParameters } from './sql/params';
|
||||||
import { TransactionManager } from './transaction/index';
|
import { TransactionManager } from './transaction/index';
|
||||||
import { PluginManager } from './plugin/index';
|
import { PluginManager } from './plugin/index';
|
||||||
import type { Statement } from './query/ast';
|
import type { Statement } from './query/ast';
|
||||||
@@ -186,8 +187,12 @@ export class MetonaSqlark {
|
|||||||
|
|
||||||
// ---- SQL 查询 ----
|
// ---- SQL 查询 ----
|
||||||
|
|
||||||
/** 执行 SQL 字符串查询 */
|
/**
|
||||||
async query(sql: string): Promise<unknown> {
|
* 执行 SQL 字符串查询。
|
||||||
|
* v0.7.0: 支持位置参数(`?`)—— `db.query('SELECT * FROM t WHERE id = ?', ['1'])`。
|
||||||
|
* 参数按 SQL 字面量安全编码(字符串 '' 转义),杜绝 SQL 注入。
|
||||||
|
*/
|
||||||
|
async query(sql: string, params?: unknown[]): Promise<unknown> {
|
||||||
this.ensureReady();
|
this.ensureReady();
|
||||||
const startTime = this.debug ? Date.now() : 0;
|
const startTime = this.debug ? Date.now() : 0;
|
||||||
|
|
||||||
@@ -195,8 +200,10 @@ export class MetonaSqlark {
|
|||||||
|
|
||||||
let result: unknown;
|
let result: unknown;
|
||||||
try {
|
try {
|
||||||
|
// v0.7.0: 参数绑定(仅替换字符串字面量之外的 ?)
|
||||||
|
const boundSql = bindParameters(sql, params);
|
||||||
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
// v0.3.0: 支持分号分隔的多语句,逐条顺序执行,返回最后一条的结果
|
||||||
const statements: Statement[] = parseAll(sql);
|
const statements: Statement[] = parseAll(boundSql);
|
||||||
for (const stmt of statements) {
|
for (const stmt of statements) {
|
||||||
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
// v0.5.1: SQL 写语句触发 CRUD 生命周期钩子(与 Table API 路径一致)
|
||||||
await this.triggerStatementHooks(stmt, 'before');
|
await this.triggerStatementHooks(stmt, 'before');
|
||||||
|
|||||||
@@ -46,6 +46,15 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
private txDirtyTables: Set<string> = new Set();
|
private txDirtyTables: Set<string> = new Set();
|
||||||
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
/** 事务中发生 schema 变更(DDL)—— commit 时持久化 schema */
|
||||||
private txSchemaChanged = false;
|
private txSchemaChanged = false;
|
||||||
|
/**
|
||||||
|
* v0.7.0: 事务行级变更记录(table → pk → put/delete)。
|
||||||
|
* commit 时按行增量 flush(此前整表 diff:大表事务改 1 行也重写全表)。
|
||||||
|
*/
|
||||||
|
private txChanges: Map<string, Map<string, 'put' | 'delete'>> = new Map();
|
||||||
|
/** v0.7.0: 无法行级追踪的表(主键变更/级联影响表)→ commit 时整表 diff */
|
||||||
|
private txFullTables: Set<string> = new Set();
|
||||||
|
/** v0.7.0: 事务内 clear 的表 → commit 时清空 KV 行 */
|
||||||
|
private txClearedTables: Set<string> = new Set();
|
||||||
|
|
||||||
constructor(medium?: IStorageBackend, checkpointThreshold?: number) {
|
constructor(medium?: IStorageBackend, checkpointThreshold?: number) {
|
||||||
this.kv = new KVStore(medium, checkpointThreshold);
|
this.kv = new KVStore(medium, checkpointThreshold);
|
||||||
@@ -119,6 +128,9 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
try { await this.rollbackTransaction(); } catch { /* ignore */ }
|
try { await this.rollbackTransaction(); } catch { /* ignore */ }
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 关闭前 checkpoint(截断日志,重开更快)。
|
||||||
|
// 失败不阻塞关闭(日志已持久,重开可全量重放)。
|
||||||
|
try { await this.kv.checkpoint(); } catch { /* 数据在日志中,重开放心重放 */ }
|
||||||
await this.kv.close();
|
await this.kv.close();
|
||||||
await this.memory.close();
|
await this.memory.close();
|
||||||
this.opened = false;
|
this.opened = false;
|
||||||
@@ -239,6 +251,19 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
const pks = await this.memory.insert(tableName, rows);
|
const pks = await this.memory.insert(tableName, rows);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内 clear 后又写入 → 清空语义被覆盖,整表 diff 兜底
|
||||||
|
if (this.txClearedTables.has(tableName)) {
|
||||||
|
this.txClearedTables.delete(tableName);
|
||||||
|
this.txFullTables.add(tableName);
|
||||||
|
return pks;
|
||||||
|
}
|
||||||
|
// v0.7.0: 行级变更记录(增量 flush)
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks) changes.set(pk, 'put');
|
||||||
return pks;
|
return pks;
|
||||||
}
|
}
|
||||||
// 增量持久化(原子 putMany)
|
// 增量持久化(原子 putMany)
|
||||||
@@ -279,6 +304,28 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
const count = await this.memory.update(tableName, query, updates);
|
const count = await this.memory.update(tableName, query, updates);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 主键变更无法行级追踪(旧键删除+新键落盘+级联),
|
||||||
|
// 相关表整表 diff 兜底;普通更新记录受影响行
|
||||||
|
if (pkChanged) {
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of affected) changes.set(pk, 'put');
|
||||||
|
// 级联影响表(理论上非主键更新不级联,防御性兜底)
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -328,6 +375,19 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
const count = await this.memory.delete(tableName, query);
|
const count = await this.memory.delete(tableName, query);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 行级变更记录 —— 删除行记录 delete;级联影响表整表 diff 兜底
|
||||||
|
let changes = this.txChanges.get(tableName);
|
||||||
|
if (!changes) {
|
||||||
|
changes = new Map();
|
||||||
|
this.txChanges.set(tableName, changes);
|
||||||
|
}
|
||||||
|
for (const pk of pks) changes.set(pk, 'delete');
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) {
|
||||||
|
this.txFullTables.add(t);
|
||||||
|
this.txDirtyTables.add(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
const puts: Record<string, ArrayBuffer> = {};
|
const puts: Record<string, ArrayBuffer> = {};
|
||||||
@@ -353,6 +413,9 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
await this.memory.clear(tableName);
|
await this.memory.clear(tableName);
|
||||||
if (this.txActive) {
|
if (this.txActive) {
|
||||||
this.txDirtyTables.add(tableName);
|
this.txDirtyTables.add(tableName);
|
||||||
|
// v0.7.0: 事务内清空 → commit 时删除全部 KV 行(比整表 diff 更高效)
|
||||||
|
this.txClearedTables.add(tableName);
|
||||||
|
this.txChanges.delete(tableName);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const diff = await this.collectTableDiff(tableName);
|
const diff = await this.collectTableDiff(tableName);
|
||||||
@@ -391,27 +454,67 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
this.txActive = true;
|
this.txActive = true;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
|
|
||||||
async commitTransaction(): Promise<void> {
|
async commitTransaction(): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
// v0.6.1: 全部 dirty 表合并为单次原子 flush(一条日志记录 = 真原子,
|
||||||
// 多表事务中途崩溃/失败不会出现"部分表已提交")
|
// 多表事务中途崩溃/失败不会出现"部分表已提交")。
|
||||||
|
// v0.7.0: 行级增量 flush —— 普通 insert/update/delete 仅写事务内改动的行
|
||||||
|
// (此前 collectTableDiff 整表重写:大表事务改 1 行也 O(表大小));
|
||||||
|
// 主键变更/级联影响表整表 diff 兜底;clear/drop 表只删 KV 行。
|
||||||
const puts: Record<string, ArrayBuffer> = {};
|
const puts: Record<string, ArrayBuffer> = {};
|
||||||
const deletes: string[] = [];
|
const deletes: string[] = [];
|
||||||
for (const table of this.txDirtyTables) {
|
for (const table of this.txDirtyTables) {
|
||||||
if (await this.memory.hasTable(table)) {
|
if (!(await this.memory.hasTable(table))) {
|
||||||
const diff = await this.collectTableDiff(table);
|
|
||||||
Object.assign(puts, diff.puts);
|
|
||||||
deletes.push(...diff.deletes);
|
|
||||||
} else {
|
|
||||||
// 事务内 drop 的表:清理 KV 残留行
|
// 事务内 drop 的表:清理 KV 残留行
|
||||||
const all = await this.kv.getAll();
|
const all = await this.kv.getAll();
|
||||||
const prefix = this.rowPrefix(table);
|
const prefix = this.rowPrefix(table);
|
||||||
for (const [key] of all) {
|
for (const [key] of all) {
|
||||||
if (key.startsWith(prefix)) deletes.push(key);
|
if (key.startsWith(prefix)) deletes.push(key);
|
||||||
}
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txClearedTables.has(table)) {
|
||||||
|
// 事务内 clear 的表:删除全部 KV 行
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const prefix = this.rowPrefix(table);
|
||||||
|
for (const [key] of all) {
|
||||||
|
if (key.startsWith(prefix)) deletes.push(key);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (this.txFullTables.has(table)) {
|
||||||
|
const diff = await this.collectTableDiff(table);
|
||||||
|
Object.assign(puts, diff.puts);
|
||||||
|
deletes.push(...diff.deletes);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const changes = this.txChanges.get(table);
|
||||||
|
if (changes && changes.size > 0) {
|
||||||
|
// v0.7.0: 增量 flush —— 单次全表扫描 + 变更集合过滤
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (!schema) continue;
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const pending = new Map(changes);
|
||||||
|
const rows = await this.memory.find(table, { table: table });
|
||||||
|
for (const row of rows) {
|
||||||
|
const pkStr = String(row[pkCol]);
|
||||||
|
const kind = pending.get(pkStr);
|
||||||
|
if (kind !== undefined) {
|
||||||
|
if (kind === 'put') puts[this.rowKey(table, pkStr)] = enc(JSON.stringify(row));
|
||||||
|
else deletes.push(this.rowKey(table, pkStr));
|
||||||
|
pending.delete(pkStr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 内存中已不存在的行(后续操作删除)→ KV 行删除
|
||||||
|
for (const [pk, kind] of pending) {
|
||||||
|
if (kind === 'delete') deletes.push(this.rowKey(table, pk));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
await this.kv.writeBatch(puts, deletes);
|
await this.kv.writeBatch(puts, deletes);
|
||||||
@@ -419,10 +522,15 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
if (this.txSchemaChanged) {
|
if (this.txSchemaChanged) {
|
||||||
await this.persistSchema();
|
await this.persistSchema();
|
||||||
}
|
}
|
||||||
await this.kv.checkpoint();
|
// v0.7.0-perf: 移除每次 commit 的强制全量 checkpoint —— KVStore 按日志阈值
|
||||||
|
// 自动 checkpoint(日志重放保证崩溃恢复正确),大库高频事务不再 O(库大小)。
|
||||||
|
// close() 时统一 checkpoint(截断日志,重开更快)。
|
||||||
await this.memory.commitTransaction();
|
await this.memory.commitTransaction();
|
||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
|
|
||||||
async rollbackTransaction(): Promise<void> {
|
async rollbackTransaction(): Promise<void> {
|
||||||
@@ -432,6 +540,9 @@ export class KVStoreEngine implements IStorageEngine {
|
|||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
this.txDirtyTables = new Set();
|
this.txDirtyTables = new Set();
|
||||||
this.txSchemaChanged = false;
|
this.txSchemaChanged = false;
|
||||||
|
this.txChanges = new Map();
|
||||||
|
this.txFullTables = new Set();
|
||||||
|
this.txClearedTables = new Set();
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---- 内部 ----
|
// ---- 内部 ----
|
||||||
|
|||||||
+31
-1
@@ -207,6 +207,24 @@ export class QueryExecutor {
|
|||||||
plan = compileStatement(stmt.query);
|
plan = compileStatement(stmt.query);
|
||||||
} catch { /* 非查询语句无 QueryPlan */ }
|
} catch { /* 非查询语句无 QueryPlan */ }
|
||||||
|
|
||||||
|
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
|
||||||
|
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
|
||||||
|
let usingIndex: string = plan?.table ? 'none' : 'none';
|
||||||
|
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
|
||||||
|
try {
|
||||||
|
const schema = await this.engine.getTableSchema(plan.table);
|
||||||
|
if (schema) {
|
||||||
|
for (const col of Object.keys(plan.where)) {
|
||||||
|
if (col.startsWith('$')) continue;
|
||||||
|
const colDef = schema.columns[col];
|
||||||
|
if (!colDef) continue;
|
||||||
|
if (colDef.primaryKey) { usingIndex = 'pk'; break; }
|
||||||
|
if (colDef.index || colDef.unique) { usingIndex = `index:${col}`; break; }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch { /* schema 读取失败保持 none */ }
|
||||||
|
}
|
||||||
|
|
||||||
return {
|
return {
|
||||||
type: stmt.query.type,
|
type: stmt.query.type,
|
||||||
table: plan?.table,
|
table: plan?.table,
|
||||||
@@ -215,7 +233,7 @@ export class QueryExecutor {
|
|||||||
orderBy: plan?.orderBy || [],
|
orderBy: plan?.orderBy || [],
|
||||||
limit: plan?.limit,
|
limit: plan?.limit,
|
||||||
offset: plan?.offset,
|
offset: plan?.offset,
|
||||||
usingIndex: plan?.table ? 'auto' : 'none',
|
usingIndex,
|
||||||
estimatedRows: rows,
|
estimatedRows: rows,
|
||||||
actualTimeMs: elapsed,
|
actualTimeMs: elapsed,
|
||||||
};
|
};
|
||||||
@@ -713,6 +731,18 @@ export class QueryExecutor {
|
|||||||
const schema = await this.engine.getTableSchema(stmt.name);
|
const schema = await this.engine.getTableSchema(stmt.name);
|
||||||
if (!schema) return;
|
if (!schema) return;
|
||||||
|
|
||||||
|
// v0.7.0: ALTER ADD 主键列防护 —— 复合主键不支持(与 createSchema 校验对齐),
|
||||||
|
// 避免绕过建表校验添加第二个主键列导致语义陷阱
|
||||||
|
if (stmt.action === 'ADD' && stmt.column.primaryKey) {
|
||||||
|
const hasPk = Object.values(schema.columns).some((c) => c.primaryKey);
|
||||||
|
if (hasPk) {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Composite primary keys are not supported yet: table "${stmt.name}" already has a primary key column`,
|
||||||
|
'SCHEMA_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
// v0.4.1: 引擎级 alterTable(Aria 需重写存储行 + 持久化 schema;其余引擎走通用引用路径)
|
||||||
if (typeof this.engine.alterTable === 'function') {
|
if (typeof this.engine.alterTable === 'function') {
|
||||||
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
return this.engine.alterTable(stmt.name, stmt.action, { ...astColumnToColumnDef(stmt.column), name: stmt.column.name });
|
||||||
|
|||||||
@@ -0,0 +1,96 @@
|
|||||||
|
/**
|
||||||
|
* metona-sqlark SQL Parameters — 参数化查询绑定
|
||||||
|
* @module sql/params
|
||||||
|
*
|
||||||
|
* v0.7.0: `db.query(sql, params)` 位置参数(`?`)支持。
|
||||||
|
* 绑定在词法层面完成:仅替换字符串字面量之外的 `?`,
|
||||||
|
* 值按 SQL 字面量编码(字符串 `''` 转义、数字/布尔/JSON 直出),
|
||||||
|
* 从根上规避 SQL 注入(不经过字符串拼接由用户自行转义)。
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { DatabaseError } from '../constants';
|
||||||
|
|
||||||
|
/** 将单个参数值编码为 SQL 字面量 */
|
||||||
|
function encodeParam(value: unknown): string {
|
||||||
|
if (value === null || value === undefined) return 'NULL';
|
||||||
|
if (typeof value === 'number') {
|
||||||
|
if (Number.isFinite(value)) return String(value);
|
||||||
|
return 'NULL'; // NaN/Infinity 无 SQL 字面量 → NULL
|
||||||
|
}
|
||||||
|
if (typeof value === 'boolean') return value ? 'TRUE' : 'FALSE';
|
||||||
|
if (typeof value === 'string') return `'${value.replace(/'/g, "''")}'`;
|
||||||
|
// 对象/数组无 SQL 字面量(SQL 方言不支持 json 字面量),显式拒绝而非静默错配
|
||||||
|
throw new DatabaseError(
|
||||||
|
'Object/array query parameters are not supported by SQL binding (pass JSON strings explicitly)',
|
||||||
|
'PARAM_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 将 SQL 中的位置参数 `?`(字符串字面量之外)替换为编码后的字面量。
|
||||||
|
* @param sql 含 `?` 占位符的 SQL
|
||||||
|
* @param params 位置参数数组
|
||||||
|
* @throws PARAM_ERROR 参数数量不匹配
|
||||||
|
*/
|
||||||
|
export function bindParameters(sql: string, params?: unknown[]): string {
|
||||||
|
// undefined = 不启用绑定;[] + 含 ? 的 SQL 由循环内报 PARAM_ERROR
|
||||||
|
if (!params) return sql;
|
||||||
|
|
||||||
|
let out = '';
|
||||||
|
let i = 0;
|
||||||
|
let pIdx = 0;
|
||||||
|
let quote: string | null = null;
|
||||||
|
|
||||||
|
while (i < sql.length) {
|
||||||
|
const ch = sql[i];
|
||||||
|
|
||||||
|
if (quote !== null) {
|
||||||
|
out += ch;
|
||||||
|
if (ch === quote) {
|
||||||
|
// SQL 标准 '' 转义:两个连续引号 = 一个引号(原样保留)
|
||||||
|
if (sql[i + 1] === quote) {
|
||||||
|
out += sql[i + 1];
|
||||||
|
i += 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
quote = null;
|
||||||
|
}
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ch === "'" || ch === '"') {
|
||||||
|
quote = ch;
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ch === '?') {
|
||||||
|
if (pIdx >= params.length) {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Too few query parameters: placeholder #${pIdx + 1} has no value (got ${params.length} total)`,
|
||||||
|
'PARAM_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
out += encodeParam(params[pIdx]);
|
||||||
|
pIdx++;
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
out += ch;
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (quote !== null) {
|
||||||
|
throw new DatabaseError('Unterminated string literal in SQL', 'PARSE_ERROR');
|
||||||
|
}
|
||||||
|
if (pIdx < params.length) {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Too many query parameters: ${params.length} provided but only ${pIdx} placeholders`,
|
||||||
|
'PARAM_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
@@ -44,6 +44,16 @@ export function validateColumns(columns: Record<string, ColumnDef>): void {
|
|||||||
if (primaryKeyCount === 0) {
|
if (primaryKeyCount === 0) {
|
||||||
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
throw new DatabaseError('Table must have at least one primary key column', 'SCHEMA_ERROR');
|
||||||
}
|
}
|
||||||
|
// v0.7.0: 复合主键(多列 primaryKey)当前不支持 —— 所有引擎的存储布局与
|
||||||
|
// 外键引用均为单主键假设(此前静默取第一个主键,其余标记被忽略 → 语义陷阱)。
|
||||||
|
// 显式拒绝避免用户误用;复合主键列入 v0.8 路线图。
|
||||||
|
if (primaryKeyCount > 1) {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Composite primary keys are not supported yet: table has ${primaryKeyCount} primary key columns. ` +
|
||||||
|
'Use a single primary key column (or a unique column combination) instead.',
|
||||||
|
'SCHEMA_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 获取主键列名 */
|
/** 获取主键列名 */
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ beforeEach(() => { installOPFSMock(new Map()); });
|
|||||||
|
|
||||||
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
||||||
test('VERSION 常量为当前版本(0.6.0)', () => {
|
test('VERSION 常量为当前版本(0.6.0)', () => {
|
||||||
expect(VERSION).toBe('0.6.3');
|
expect(VERSION).toBe('0.7.0');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -403,7 +403,7 @@ describe('[v0.3.3] P1-9: Savepoint + MVCC 一致性', () => {
|
|||||||
|
|
||||||
describe('[v0.3.3] 端到端', () => {
|
describe('[v0.3.3] 端到端', () => {
|
||||||
test('全部修复点可共存于 MetonaSqlark API', async () => {
|
test('全部修复点可共存于 MetonaSqlark API', async () => {
|
||||||
expect(VERSION).toBe('0.6.3');
|
expect(VERSION).toBe('0.7.0');
|
||||||
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
|
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
|
|||||||
@@ -0,0 +1,403 @@
|
|||||||
|
/**
|
||||||
|
* v0.7.0 功能与性能回归测试
|
||||||
|
*
|
||||||
|
* 覆盖:
|
||||||
|
* - 复合主键显式拒绝(defineTable / SQL / ALTER ADD 主键列)
|
||||||
|
* - 参数化查询 db.query(sql, params)(安全编码/参数计数/注入防护/多语句)
|
||||||
|
* - KVStoreEngine 事务增量 flush(大表事务 O(改动行),非整表重写)
|
||||||
|
* - EXPLAIN 真实索引命中信息(pk / index:col / none)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { MetonaSqlark } from '../src/core';
|
||||||
|
import { KVStoreEngine } from '../src/engine/kvstore_engine';
|
||||||
|
import { createSchema } from '../src/table/schema';
|
||||||
|
import { DatabaseError } from '../src/constants';
|
||||||
|
import { parseLogRecords, KVLogOp } from '../src/engine/kvstore/log';
|
||||||
|
import { SharedMemoryBackend } from '../src/engine/kvstore/shared_memory_medium';
|
||||||
|
import type { KVStore } from '../src/engine/kvstore/index';
|
||||||
|
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `v070-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function expectCode(promise: Promise<unknown>, code: string): Promise<void> {
|
||||||
|
try {
|
||||||
|
await promise;
|
||||||
|
} catch (e) {
|
||||||
|
expect((e as { code?: string }).code).toBe(code);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
throw new Error(`Expected rejection with code "${code}", but promise resolved`);
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('v0.7.0 — 复合主键显式拒绝', () => {
|
||||||
|
it('defineTable 多主键抛 SCHEMA_ERROR(不再静默取第一个)', () => {
|
||||||
|
expect(() => createSchema('t', {
|
||||||
|
a: { type: 'string', primaryKey: true },
|
||||||
|
b: { type: 'string', primaryKey: true },
|
||||||
|
})).toThrow('Composite primary keys are not supported');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('SQL CREATE TABLE 多主键抛 SCHEMA_ERROR', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await expectCode(
|
||||||
|
db.query('CREATE TABLE t (a STRING PRIMARY KEY, b STRING PRIMARY KEY)'),
|
||||||
|
'SCHEMA_ERROR',
|
||||||
|
);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ALTER TABLE ADD 主键列(已有主键)抛 SCHEMA_ERROR', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await expectCode(
|
||||||
|
db.query('ALTER TABLE t ADD COLUMN other STRING PRIMARY KEY'),
|
||||||
|
'SCHEMA_ERROR',
|
||||||
|
);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('v0.7.0 — 参数化查询', () => {
|
||||||
|
async function setup(): Promise<MetonaSqlark> {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string' },
|
||||||
|
age: { type: 'number' },
|
||||||
|
active: { type: 'boolean' },
|
||||||
|
profile: { type: 'json' },
|
||||||
|
});
|
||||||
|
return db;
|
||||||
|
}
|
||||||
|
|
||||||
|
it('位置参数绑定(字符串/数字/布尔/null)', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await db.query("INSERT INTO users VALUES ('1', 'Alice', 30, TRUE, NULL)");
|
||||||
|
await db.query('INSERT INTO users VALUES (?, ?, ?, ?, ?)', ['2', "O'Brien", 25, false, null]);
|
||||||
|
|
||||||
|
const rows = await db.query('SELECT * FROM users WHERE id = ?', ['2']) as Record<string, unknown>[];
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(rows[0].name).toBe("O'Brien");
|
||||||
|
expect(rows[0].age).toBe(25);
|
||||||
|
expect(rows[0].active).toBe(false);
|
||||||
|
expect(rows[0].profile).toBeNull();
|
||||||
|
|
||||||
|
const byName = await db.query("SELECT * FROM users WHERE name = ?", ["O'Brien"]) as Record<string, unknown>[];
|
||||||
|
expect(byName).toHaveLength(1);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('对象参数显式拒绝(PARAM_ERROR,防静默错配)', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await expectCode(
|
||||||
|
db.query('INSERT INTO users VALUES (?, ?, ?, ?, ?)', ['1', 'N', 1, true, { city: 'BJ' }]),
|
||||||
|
'PARAM_ERROR',
|
||||||
|
);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('注入防护:字符串参数含 SQL 片段被当作字面量', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await db.query('INSERT INTO users VALUES (?, ?, ?, ?, NULL)', ['1', "x'; DROP TABLE users; --", 1, true]);
|
||||||
|
const rows = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(rows[0].name).toBe("x'; DROP TABLE users; --");
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('字符串字面量内的 ? 不被替换', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await db.query("INSERT INTO users VALUES ('1', 'what?', 1, TRUE, NULL)");
|
||||||
|
const rows = await db.query("SELECT * FROM users WHERE name = 'what?'") as Record<string, unknown>[];
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('参数数量不匹配抛 PARAM_ERROR', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await expectCode(db.query('SELECT * FROM users WHERE id = ?', []), 'PARAM_ERROR');
|
||||||
|
await expectCode(db.query('SELECT * FROM users WHERE id = ?', ['1', '2']), 'PARAM_ERROR');
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('多语句 + 参数', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await db.query("INSERT INTO users VALUES (?, 'A', 1, TRUE, NULL); INSERT INTO users VALUES (?, 'B', 2, TRUE, NULL)", ['1', '2']);
|
||||||
|
const rows = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||||
|
expect(rows).toHaveLength(2);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('NaN/Infinity 参数编码为 NULL', async () => {
|
||||||
|
const db = await setup();
|
||||||
|
await db.query('INSERT INTO users VALUES (?, ?, ?, ?, NULL)', ['1', 'N', Number.NaN, true]);
|
||||||
|
const rows = await db.query('SELECT * FROM users WHERE id = ?', ['1']) as Record<string, unknown>[];
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(rows[0].age).toBeNull();
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('v0.7.0 — KVStoreEngine 事务增量 flush', () => {
|
||||||
|
it('大表事务改 1 行:日志仅含该行(非整表重写)', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'number', primaryKey: true },
|
||||||
|
v: { type: 'string' },
|
||||||
|
}));
|
||||||
|
const rows = Array.from({ length: 1000 }, (_, i) => ({ id: i, v: `v${i}` }));
|
||||||
|
await eng.insert('t', rows);
|
||||||
|
// 基线 checkpoint(日志清空)
|
||||||
|
const kv0 = (eng as any).kv as KVStore;
|
||||||
|
await kv0.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.update('t', { table: 't', where: { id: 500 } }, { v: 'CHANGED' });
|
||||||
|
await eng.commitTransaction();
|
||||||
|
|
||||||
|
// commit 后日志应只有 1 条记录、1 个 PUT 条目(增量 flush,非 1000 行重写)
|
||||||
|
const medium = (eng as any).kv.medium as SharedMemoryBackend;
|
||||||
|
const raw = await medium.read('__kv_log');
|
||||||
|
const records: { entries: { op: KVLogOp }[] }[] = [];
|
||||||
|
parseLogRecords(new Uint8Array(raw!), (r) => records.push(r as never));
|
||||||
|
expect(records).toHaveLength(1);
|
||||||
|
expect(records[0].entries).toHaveLength(1);
|
||||||
|
expect(records[0].entries[0].op).toBe(KVLogOp.PUT);
|
||||||
|
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
// 重开后数据完整
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const all = await eng2.find('t', { table: 't' });
|
||||||
|
expect(all).toHaveLength(1000);
|
||||||
|
expect(all.find((r) => r.id === 500)!.v).toBe('CHANGED');
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务混合操作(insert/update/delete/clear 语义)持久化正确', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'string' },
|
||||||
|
}));
|
||||||
|
await eng.insert('t', [{ id: '1', v: 'a' }, { id: '2', v: 'b' }, { id: '3', v: 'c' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.insert('t', [{ id: '4', v: 'd' }]);
|
||||||
|
await eng.update('t', { table: 't', where: { id: '1' } }, { v: 'A' });
|
||||||
|
await eng.delete('t', { table: 't', where: { id: '3' } });
|
||||||
|
await eng.commitTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const rows = await eng2.find('t', { table: 't' });
|
||||||
|
expect(rows.map((r) => r.id).sort()).toEqual(['1', '2', '4']);
|
||||||
|
expect(rows.find((r) => r.id === '1')!.v).toBe('A');
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务内 delete 后 reinsert 同主键 → put 最终态', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'string' },
|
||||||
|
}));
|
||||||
|
await eng.insert('t', [{ id: '1', v: 'old' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.delete('t', { table: 't', where: { id: '1' } });
|
||||||
|
await eng.insert('t', [{ id: '1', v: 'new' }]);
|
||||||
|
await eng.commitTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const rows = await eng2.find('t', { table: 't' });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(rows[0].v).toBe('new');
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务内 clear 后 insert → 最终态仅新行', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
}));
|
||||||
|
await eng.insert('t', [{ id: '1' }, { id: '2' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.clear('t');
|
||||||
|
await eng.insert('t', [{ id: '9' }]);
|
||||||
|
await eng.commitTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const rows = await eng2.find('t', { table: 't' });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(rows[0].id).toBe('9');
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务内主键变更 → 整表 diff 兜底路径正确', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
}));
|
||||||
|
await eng.insert('t', [{ id: '1' }, { id: '2' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.update('t', { table: 't', where: { id: '1' } }, { id: '10' });
|
||||||
|
await eng.commitTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const rows = await eng2.find('t', { table: 't' });
|
||||||
|
expect(rows.map((r) => r.id).sort()).toEqual(['10', '2']);
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('多表事务 + 级联(SET NULL)持久化正确', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
}));
|
||||||
|
await eng.createTable(createSchema('orders', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string', references: 'users.id', onDelete: 'SET NULL' },
|
||||||
|
}));
|
||||||
|
await eng.insert('users', [{ id: 'u1' }]);
|
||||||
|
await eng.insert('orders', [{ id: 'o1', user_id: 'u1' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.delete('users', { table: 'users', where: { id: 'u1' } });
|
||||||
|
await eng.insert('users', [{ id: 'u2' }]);
|
||||||
|
await eng.commitTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const users = await eng2.find('users', { table: 'users' });
|
||||||
|
expect(users.map((r) => r.id)).toEqual(['u2']);
|
||||||
|
const orders = await eng2.find('orders', { table: 'orders' });
|
||||||
|
expect(orders).toHaveLength(1);
|
||||||
|
expect(orders[0].user_id).toBeNull();
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务回滚后不落盘(增量追踪状态重置)', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
}));
|
||||||
|
await eng.insert('t', [{ id: '1' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.insert('t', [{ id: '2' }]);
|
||||||
|
await eng.rollbackTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const rows = await eng2.find('t', { table: 't' });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('连续事务互不串扰(第二事务增量正常)', async () => {
|
||||||
|
const name = uniqueDB();
|
||||||
|
const eng = new KVStoreEngine();
|
||||||
|
await eng.open(name, 1);
|
||||||
|
await eng.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'string' },
|
||||||
|
}));
|
||||||
|
await eng.insert('t', [{ id: '1', v: 'a' }, { id: '2', v: 'b' }]);
|
||||||
|
await (eng as any).kv.checkpoint();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.update('t', { table: 't', where: { id: '1' } }, { v: 'A' });
|
||||||
|
await eng.commitTransaction();
|
||||||
|
|
||||||
|
await eng.beginTransaction();
|
||||||
|
await eng.update('t', { table: 't', where: { id: '2' } }, { v: 'B' });
|
||||||
|
await eng.commitTransaction();
|
||||||
|
await eng.close();
|
||||||
|
|
||||||
|
const eng2 = new KVStoreEngine();
|
||||||
|
await eng2.open(name, 1);
|
||||||
|
const rows = await eng2.find('t', { table: 't' });
|
||||||
|
expect(rows.find((r) => r.id === '1')!.v).toBe('A');
|
||||||
|
expect(rows.find((r) => r.id === '2')!.v).toBe('B');
|
||||||
|
await eng2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('v0.7.0 — EXPLAIN 真实索引信息', () => {
|
||||||
|
it('主键/索引/无索引条件分别报告 pk/index:col/none', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'aria', diskEngine: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
tag: { type: 'string', index: true },
|
||||||
|
other: { type: 'string' },
|
||||||
|
});
|
||||||
|
|
||||||
|
const byPk = await db.query("EXPLAIN SELECT * FROM t WHERE id = '1'") as Record<string, unknown>;
|
||||||
|
expect(byPk.usingIndex).toBe('pk');
|
||||||
|
|
||||||
|
const byIdx = await db.query("EXPLAIN SELECT * FROM t WHERE tag = 'x'") as Record<string, unknown>;
|
||||||
|
expect(byIdx.usingIndex).toBe('index:tag');
|
||||||
|
|
||||||
|
const none = await db.query("EXPLAIN SELECT * FROM t WHERE other = 'x'") as Record<string, unknown>;
|
||||||
|
expect(none.usingIndex).toBe('none');
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('EXPLAIN DELETE 同样报告索引信息(不执行)', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.query('INSERT INTO t VALUES (\'1\'), (\'2\')');
|
||||||
|
const plan = await db.query('EXPLAIN DELETE FROM t WHERE id = \'1\'') as Record<string, unknown>;
|
||||||
|
expect(plan.usingIndex).toBe('pk');
|
||||||
|
expect(plan.estimatedRows).toBe(1);
|
||||||
|
const rows = await db.query('SELECT * FROM t') as Record<string, unknown>[];
|
||||||
|
expect(rows).toHaveLength(2);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('v0.7.0 — 错误码导出检查', () => {
|
||||||
|
it('DatabaseError code 可访问', () => {
|
||||||
|
const e = new DatabaseError('m', 'X');
|
||||||
|
expect(e.code).toBe('X');
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user