fix: v0.7.3 数据正确性与边界窗口收尾 — INSERT 语句级原子(三引擎+Aria PK 批内重复)/ 索引列 IS NULL 恒空 / delete RESTRICT 破坏索引 / queryStream 子查询静默空结果 / ALTER DROP 索引残留 / UNIQUE INDEX 存量校验 / SELECT * 别名投影 / WAL BEGIN/ROLLBACK 事务边界 / aria $in 与级联重复扫描性能 / $and 等值下推 / ANALYZE 索引统计 / React-Vue hooks 生命周期 / 迁移主键兜底 + 58 回归
CI / test (18.x) (push) Successful in 17m43s
CI / test (22.x) (push) Successful in 13m45s
CI / test (20.x) (push) Successful in 15m33s
CI / test (24.x) (push) Successful in 24m46s
CI / e2e (push) Successful in 52s

This commit is contained in:
thzxx
2026-08-14 22:53:46 +08:00
parent f8f8d1b2ff
commit 50468b9b0e
29 changed files with 2374 additions and 650 deletions
+88
View File
@@ -2,6 +2,94 @@
All notable changes to MetonaSqlark will be documented in this file. All notable changes to MetonaSqlark will be documented in this file.
## [0.7.3] - 2026-08-14
### INSERT 语句级原子性补全 / 索引一致性 / 流式查询回退安全
> 深度审计第六阶段:修复 INSERT 语句级部分提交(三引擎 + Aria PK 批内重复)、
> 索引列 IS NULL 恒空、delete RESTRICT 预检破坏索引、queryStream 子查询静默空结果
> 等 10 项问题,四引擎语义对齐。
### Fixed
- **INSERT 语句级部分提交(P1Memory/KVStore/Hybrid** — 第 N 行主键重复/唯一冲突
抛错时前 N-1 行已提交(v0.7.2 只修了 UPDATEINSERT 漏修)。改为两阶段:
先全量预检(主键批内 Set 互查 + 索引查 + 唯一批内互查),任何一行失败整句不执行
- **Aria insert 批内主键重复部分提交(P1)** — PK 重复检查在写入循环内:第 N 行
重复抛错时前 N-1 行已 put LSM 且其 WAL 记录随 appendBatch 一起丢失 → 部分提交 +
内存/WAL 不一致。PK 检查移入批预检阶段(与 v0.6.2 的 unique 预检同一阶段)
- **索引列 IS NULL 恒空(P1Memory/KVStore/Hybrid** — `tryIndexLookup`
`colIndex.get(null)` 恒 undefined → `return []` 短路全表扫描(AriaEngine v0.6.2
已修,Memory 漏修)。null/undefined 条件跳过索引路径回退全表扫描
- **delete RESTRICT 预检破坏索引(P1Memory/KVStore/Hybrid** — `removeIndexEntries`
在收集阶段(RESTRICT 预检前)执行:预检抛错时行未删但索引条目已删 → 唯一约束
永久失效 + 索引查询丢行。索引清理移到预检通过之后(与 AriaEngine 对齐)
- **queryStream 子查询静默空结果(P1** — WHERE 含 `$subquery`/`$exists`/`$col`
未回退物化:引擎层 matchWhere 的 `$in` 遇未解析的 `$subquery` 对象返回 false →
所有行被静默过滤(`$col` 会抛 QUERY_ERROR)。streamable 判定增加递归检测,
回退物化路径(resolveSubqueries 正确解析)
- **ALTER DROP 索引列残留(P2Memory/KVStore** — 删列只删 schema.columns
indexes Map 中该列条目残留 → 查询已删列走旧索引(不含新行)→ 结果不完整。
DROP 时同步清理索引 Map(对齐 AriaEngine cleanupTableIndexes
- **CREATE UNIQUE INDEX 存量重复数据静默成功(P2)** — 回填不校验存量唯一性
SQLite 语义应报错)。Memory/Aria 回填时检查重复 → 抛 `UNIQUE_VIOLATION`
失败路径清理半初始化索引(标志未落、索引 Map/LSM 移除),保持原子语义
- **`SELECT *, col AS alias` 解析与投影(P3** — 此前 parser 的 '*' 独占分支使
`SELECT *, name AS nick` 直接 PARSE_ERRORexecutor 侧 columns[0]==='*' 不投影。
parser 支持 '*' 后接列列表;projectRow 以原行全部列为基、其余表达式覆盖/追加
- **INSERT hooks 行键错位(P3** — SQL 省略列名时 beforeInsert/afterInsert 收到
数字键行(与 executor 写入的 schema 列名行不一致)。hooks 行映射对齐 executor
(省略列名时按 schema 列顺序)
- **KVStoreEngine insert 持久化原始行(P3)** — 非事务路径 `JSON.stringify(row)`
写入参原始对象:default 值不落盘、schema 外列被持久化。改为持久化内存中
validated 行(MemoryEngine 新增 `getRow()`O(rows) 无性能回退)
### Changed
- 测试 1198 → **1256**75 套件,+58 个 v0.7.3 回归);新增 `tests/v073-fixes.test.ts`
IS NULL ×6 / RESTRICT 索引 ×3 / INSERT 原子 ×11 / queryStream ×4 / ALTER DROP ×2 /
UNIQUE INDEX ×4 / SELECT * ×3 / hooks ×2 / KVStore validated ×3 / getRow ×1 /
WAL 失败窗口 ×2 / aria $in ×2 / $and 下推 ×4 / 常量转义 ×3 / ANALYZE ×2
+ React config 重建 ×1 + Vue 卸载 ×2 + 迁移无主键 ×2
- 行覆盖率 89.8% → **90.0%**
### Fixed(审计第二阶段:边界窗口 / 性能 / 生态收尾)
- **WAL BEGIN 写失败事务泄漏(P3** — full 模式 BEGIN 记录写失败时 currentTxnId 已设置
→ TX_ACTIVE 永久泄漏(后续无法开始新事务)。失败回滚 mvcc 登记与快照后重抛,
调用方可重试
- **WAL ROLLBACK 顺序(P3** — 内存先回滚、ROLLBACK 记录后写:full 模式写失败时
崩溃重放无 ROLLBACK 记录 → 已回滚事务的数据复活。改为先持久化 ROLLBACK 再回滚内存
(与 commitTransaction 的"WAL 领先内存"对齐):写失败 → 事务仍活跃可重试,
崩溃重放看到 ROLLBACK 记录不复活数据
- **aria $in 逐值 drainChain 性能悬崖(P2** — 索引 `$in` 逐值 indexScanToRows
每个值一次 prefetchRange + prefetchKeys(各一次 drainChain 排空后台链),
compaction 长耗时时 N 倍放大(与 v0.6.1 修的 insert 批量预加载同类)。
批级预加载全部值索引范围 + 主表行各一次,循环内同步 rangeScan/get
- **update 主键变更级联重复全表扫描(P2 性能)** — applyUpdateCascade /
applyForeignKeyUpdateRules 的阶段 1 RESTRICT 扫描与两阶段预检
checkUpdateRestrict / checkForeignKeyUpdateRestrict)完全重复,删除冗余扫描
- **多条件 AND 永不走索引(P2)** — `WHERE a AND b` 解析为顶层 $andMemory/Aria
的 tryIndexLookup 只查顶层键 → 永远全表扫描,索引形同虚设。递归展开 $and 等值
条件下推($or/$not 保守跳过,命中后全条件 matchWhere 过滤,子集语义安全);
EXPLAIN usingIndex 同步递归识别 $and 嵌套(与引擎行为对齐)
- **ANALYZE 统计不含二级索引(P3** — indexDepth/sstableCount/memtableSize 只统计
主 LSM,多索引表严重低估。汇总该表全部二级索引 LSM
- **Vue useSqlarkDatabase 卸载不 closeP3)** — 组件卸载后实例永不关闭(连接/锁/
后端句柄泄漏)。onUnmounted 时 close(失败不阻塞卸载)
- **React useDatabase config 变更不生效(P3** — initRef 只建一次,配置更新永不
重建且旧实例残留。以 config 序列化指纹为依赖:变更时 cleanup 关闭旧实例再重建
close 幂等,未完成 init 亦可安全关闭)
- **migrateFromIndexedDB 无 id 列旧库迁移中断(P3)** — 推断 schema 无主键 →
createSchema 抛 SCHEMA_ERROR 中断整个迁移。第一个非 json 列兜底为主键;
全 json 列无可用主键 → 跳过该表(skippedTables)不中断
- **SELECT 常量列 SQL 标准 '' 转义未还原(P3** — `SELECT 'O''Brien'` 输出
`O''Brien`projectRow 只处理反斜杠转义)。`''` 还原为 `'`(常量列与别名常量两处)
- **死代码清理** — 移除 AriaEngine.getWALEstimatedSizeCheckpointManager 自带
真实缓冲字节估算,此方法无调用者)
---
## [0.7.2] - 2026-08-13 ## [0.7.2] - 2026-08-13
### 语句级原子性 / 事务 DDL 语义统一 / 约束与绑定硬化 ### 语句级原子性 / 事务 DDL 语义统一 / 约束与绑定硬化
+1 -1
View File
@@ -75,7 +75,7 @@ src/
├── plugin/ # Plugin system (14 lifecycle hooks) ├── plugin/ # Plugin system (14 lifecycle hooks)
└── integrations/ # React & Vue hooks └── integrations/ # React & Vue hooks
tests/ # Test suite (1198 test cases, 74 suites + 12 e2e) tests/ # Test suite (1256 test cases, 75 suites + 12 e2e)
tests/helpers/ # 共享测试工具(OPFS mock 等) tests/helpers/ # 共享测试工具(OPFS mock 等)
tests/e2e/ # Playwright e2e(真实 Chromium + OPFS tests/e2e/ # Playwright e2e(真实 Chromium + OPFS
site/ # Documentation site (index / docs / demo) site/ # Documentation site (index / docs / demo)
+6 -6
View File
@@ -1,10 +1,10 @@
# MetonaSqlark # MetonaSqlark
<p align="center"> <p align="center">
<img src="https://img.shields.io/badge/version-0.7.2-blue?style=flat-square" alt="version"> <img src="https://img.shields.io/badge/version-0.7.3-blue?style=flat-square" alt="version">
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license"> <img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
<img src="https://img.shields.io/badge/coverage-89.8%25-brightgreen?style=flat-square" alt="coverage"> <img src="https://img.shields.io/badge/coverage-90.0%25-brightgreen?style=flat-square" alt="coverage">
<img src="https://img.shields.io/badge/tests-1198%20passed-success?style=flat-square" alt="tests"> <img src="https://img.shields.io/badge/tests-1256%20passed-success?style=flat-square" alt="tests">
</p> </p>
> 基于 TypeScript 的**前端关系型数据库**:完整 SQL + Query Builder 双 API > 基于 TypeScript 的**前端关系型数据库**:完整 SQL + Query Builder 双 API
@@ -401,7 +401,7 @@ const { data, loading, error, refresh } = useSqlarkQuery(db, 'SELECT * FROM user
npm install # 安装依赖 npm install # 安装依赖
npm run dev # 开发模式(localhost:3001 npm run dev # 开发模式(localhost:3001
npm run build # 生产构建(生成 dist/ npm run build # 生产构建(生成 dist/
npm test # 运行测试(1198 用例 · 74 套件) npm test # 运行测试(1256 用例 · 75 套件)
npm run test:e2e # Playwright e2e(真实 Chromium + OPFS + 崩溃注入,需先 build npm run test:e2e # Playwright e2e(真实 Chromium + OPFS + 崩溃注入,需先 build
npm run lint # 代码检查 npm run lint # 代码检查
npm run typecheck # 类型检查 npm run typecheck # 类型检查
@@ -413,8 +413,8 @@ npm run typecheck # 类型检查
| 指标 | 数值 | | 指标 | 数值 |
|------|------| |------|------|
| 测试用例 | 1198+12 Playwright e2e | | 测试用例 | 1256+12 Playwright e2e |
| 测试套件 | 74 | | 测试套件 | 75 |
| 行覆盖率 | 89.8% | | 行覆盖率 | 89.8% |
| SQL 关键字 | 72 | | SQL 关键字 | 72 |
| 存储引擎 | 5Memory / KVStore / OPFS / Hybrid / Aria | | 存储引擎 | 5Memory / KVStore / OPFS / Hybrid / Aria |
+335 -131
View File
@@ -34,7 +34,7 @@ class DatabaseError extends Error {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 版本 // 版本
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
const VERSION = '0.7.2'; const VERSION = '0.7.3';
/** /**
* metona-sqlark Shared WHERE Matcher 统一的条件匹配逻辑 * metona-sqlark Shared WHERE Matcher 统一的条件匹配逻辑
@@ -439,6 +439,11 @@ class MemoryEngine {
if (!schema.columns[column.name]) { if (!schema.columns[column.name]) {
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
} }
// v0.7.3: 被删列是索引列 → 同步清理索引 Map —— 此前残留旧索引:
// 查询已删列仍走旧索引(不含新行)→ 结果不完整(对齐 AriaEngine cleanupTableIndexes
if (schema.columns[column.name].index || schema.columns[column.name].unique) {
this.indexes.get(tableName)?.delete(column.name);
}
delete schema.columns[column.name]; delete schema.columns[column.name];
// 清理已有行中该列的值(find 返回行引用,直接删除生效) // 清理已有行中该列的值(find 返回行引用,直接删除生效)
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
@@ -454,18 +459,42 @@ class MemoryEngine {
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
const pkColumn = this.getPrimaryKey(schema); const pkColumn = this.getPrimaryKey(schema);
const pks = []; const pks = [];
// v0.7.3: 语句级原子性 —— 两阶段(先全量预检,后执行)。
// 此前逐行"校验+写入":第 N 行主键重复/唯一冲突抛错时,前 N-1 行已提交
// (无事务下语句级部分提交,与 v0.7.2 修复的 UPDATE 同类问题)。
const validated = [];
const pkSet = new Set();
const batchUnique = new Map();
// 阶段 1:全量预检(任何一行失败 → 整条语句不执行)
for (const row of rows) { for (const row of rows) {
const validatedRow = this.validateRow(schema, row); const validatedRow = this.validateRow(schema, row);
const pkValue = String(validatedRow[pkColumn]); const pkValue = String(validatedRow[pkColumn]);
if (table.has(pkValue)) // 批内主键互查(内存表尚未反映本批写入)
if (table.has(pkValue) || pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY'); throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
this.checkUniqueness(schema, validatedRow); }
pkSet.add(pkValue);
// v0.7.3: 批内唯一互查 + 索引查(此前两行同批写入同一唯一值时,
// 第一行已写入索引 → 第二行 checkUniqueness 抛错 → 第一行残留)
this.checkInsertUniqueness(schema, tableName, validatedRow, batchUnique);
validated.push(validatedRow);
}
// 阶段 2:执行(预检已通过,此阶段不再抛校验类错误)
for (const validatedRow of validated) {
const pkValue = String(validatedRow[pkColumn]);
table.set(pkValue, validatedRow); table.set(pkValue, validatedRow);
this.updateIndexes(tableName, validatedRow, pkValue); this.updateIndexes(tableName, validatedRow, pkValue);
pks.push(pkValue); pks.push(pkValue);
} }
return pks; return pks;
} }
/** v0.7.3: 按主键取已验证行(KVStoreEngine 持久化 validated 行用,含 default/类型归一) */
getRow(tableName, pkValue) {
const table = this.tables.get(tableName);
if (!table)
return null;
return table.get(pkValue) ?? null;
}
async find(tableName, query) { async find(tableName, query) {
this.ensureTable(tableName); this.ensureTable(tableName);
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
@@ -558,7 +587,37 @@ class MemoryEngine {
return count; return count;
} }
/** /**
* v0.7.2: 更新唯一性预检 批内互查多条行更新到同一唯一值+ 索引查 * v0.7.3: 插入唯一性预检 批内互查本批前几行写入同一唯一值
* + 索引查表中已有行 update checkUpdateUniqueness 对称
* 两阶段 insert 预检阶段调用索引尚未反映本批写入
*/
checkInsertUniqueness(schema, tableName, row, batchUnique) {
const tableIndexes = this.indexes.get(tableName);
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique)
continue;
const value = row[colName];
if (value === undefined || value === null)
continue;
let seen = batchUnique.get(colName);
if (!seen) {
seen = new Set();
batchUnique.set(colName, seen);
}
if (seen.has(value)) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
seen.add(value);
if (!tableIndexes)
continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(value)) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/**
* v0.7.3: 更新唯一性预检 批内互查多条行更新到同一唯一值+ 索引查
* 排除自身旧条目阶段 1 中索引尚未更新批内互查避免"两行同时改到 * 排除自身旧条目阶段 1 中索引尚未更新批内互查避免"两行同时改到
* 同一新值"绕过唯一约束 * 同一新值"绕过唯一约束
*/ */
@@ -628,30 +687,11 @@ class MemoryEngine {
/** /**
* v0.4.2-fix: ON UPDATE 外键级联 被引用表主键变更时处理引用表 * v0.4.2-fix: ON UPDATE 外键级联 被引用表主键变更时处理引用表
* RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空 * RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空
* 分两阶段先全量 RESTRICT 检查任何修改前再执行级联防部分修改 * v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 checkUpdateRestrict 已在
* 两阶段 update 预检阶段 1b覆盖 RESTRICT SET NULL+required
* 此处任何修改前重复全表扫描纯属浪费直接执行 CASCADE / SET NULL
*/ */
async applyUpdateCascade(tableName, oldPk, newPk) { async applyUpdateCascade(tableName, oldPk, newPk) {
// 阶段 1: RESTRICT 检查(引用旧主键的行存在即拒绝)
for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName)
continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate)
continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName)
continue;
const refTableData = this.tables.get(refTableName);
if (!refTableData)
continue;
for (const [, refRow] of refTableData) {
if (String(refRow[colName]) === oldPk && colDef.onUpdate === 'RESTRICT') {
throw new DatabaseError(`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`, 'FOREIGN_KEY_VIOLATION');
}
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) if (refTableName === tableName)
continue; continue;
@@ -682,28 +722,27 @@ class MemoryEngine {
const toDelete = []; const toDelete = [];
for (const [pk, row] of table) { for (const [pk, row] of table) {
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) { if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
// v0.3.3: 删除行前清理其索引条目(修复删除后索引残留) toDelete.push({ pk, row });
this.removeIndexEntries(tableName, row, pk);
toDelete.push(pk);
} }
} }
// v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归), // v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归),
// 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的 // 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的
// 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致) // 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致)
//
// v0.7.3-fix: 索引清理移到预检之后 —— 此前 removeIndexEntries 在收集阶段执行,
// RESTRICT 预检抛错时行未删但索引条目已删 → 唯一约束失效、索引查询丢行
const restrictVisited = new Set(); const restrictVisited = new Set();
for (const pk of toDelete) { for (const { pk } of toDelete) {
const row = table.get(pk);
if (row)
this.checkCascadeRestrict(tableName, pk, restrictVisited); this.checkCascadeRestrict(tableName, pk, restrictVisited);
} }
// 级联删除:检查引用此表的其他表(RESTRICT 已预检通过,此阶段不再抛 // 预检通过:清理索引 + 级联删除(此阶段不再抛校验类错误
let cascadeCount = 0; let cascadeCount = 0;
for (const pk of toDelete) { for (const { pk, row } of toDelete) {
const row = table.get(pk); // v0.3.3: 删除行前清理其索引条目(修复删除后索引残留)
if (row) this.removeIndexEntries(tableName, row, pk);
cascadeCount += await this.cascadeDelete(tableName, pk, row); cascadeCount += await this.cascadeDelete(tableName, pk, row);
} }
for (const pk of toDelete) for (const { pk } of toDelete)
table.delete(pk); table.delete(pk);
return toDelete.length + cascadeCount; return toDelete.length + cascadeCount;
} }
@@ -782,23 +821,35 @@ class MemoryEngine {
throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
if (colDef.index || colDef.unique) if (colDef.index || colDef.unique)
return; // 已存在 return; // 已存在
colDef.index = true;
if (unique)
colDef.unique = true;
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes.has(column)) if (!tableIndexes.has(column))
tableIndexes.set(column, new Map()); tableIndexes.set(column, new Map());
const colIndex = tableIndexes.get(column); const colIndex = tableIndexes.get(column);
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
try {
for (const [pk, row] of table) { for (const [pk, row] of table) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// (SQLite 语义应报错),且此后该列唯一约束永远无法满足
if (unique && colIndex.has(value)) {
throw new DatabaseError(`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${String(value)}"`, 'UNIQUE_VIOLATION');
}
if (!colIndex.has(value)) if (!colIndex.has(value))
colIndex.set(value, new Set()); colIndex.set(value, new Set());
colIndex.get(value).add(pk); colIndex.get(value).add(pk);
} }
} }
} }
catch (error) {
// 回填失败(唯一冲突):清理半初始化索引,标志未落,保持原子语义
tableIndexes.delete(column);
throw error;
}
colDef.index = true;
if (unique)
colDef.unique = true;
}
async dropIndex(tableName, column, _indexName) { async dropIndex(tableName, column, _indexName) {
// v0.7.2: 同 createIndex —— 列级标志修改无法通过事务快照回滚,显式拒绝 // v0.7.2: 同 createIndex —— 列级标志修改无法通过事务快照回滚,显式拒绝
if (this.snapshot) { if (this.snapshot) {
@@ -920,26 +971,30 @@ class MemoryEngine {
break; break;
} }
} }
/** O(1) 唯一性检查:利用哈希索引 */
checkUniqueness(schema, row) {
const tableIndexes = this.indexes.get(schema.name);
if (!tableIndexes)
return;
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique || row[colName] === undefined || row[colName] === null)
continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(row[colName])) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/** 索引查找 */ /** 索引查找 */
tryIndexLookup(tableName, table, query) { tryIndexLookup(tableName, table, query) {
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes || !query.where) if (!tableIndexes || !query.where)
return Array.from(table.values()); return Array.from(table.values());
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
// $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat = [];
const collect = (w) => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v)
collect(sub);
continue;
}
if (k === '$or' || k === '$not')
continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
// v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引 // v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引
let targetValue; let targetValue;
if (typeof condition !== 'object' || condition === null) { if (typeof condition !== 'object' || condition === null) {
@@ -951,6 +1006,11 @@ class MemoryEngine {
else { else {
continue; continue;
} }
// v0.7.3: null/undefined 条件不走索引 —— 索引不含 null 条目,
// colIndex.get(null) 恒 undefined → return [] 短路全表扫描 → 索引列
// IS NULL 恒空(对齐 AriaEngine v0.6.2 修复)
if (targetValue === null || targetValue === undefined)
continue;
const colIndex = tableIndexes.get(col); const colIndex = tableIndexes.get(col);
if (colIndex) { if (colIndex) {
const pks = colIndex.get(targetValue); const pks = colIndex.get(targetValue);
@@ -2297,11 +2357,14 @@ class KVStoreEngine {
const schema = await this.memory.getTableSchema(tableName); const schema = await this.memory.getTableSchema(tableName);
if (!schema) if (!schema)
throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND'); throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema); // v0.7.3: 持久化内存中的 validated 行(含 default 值/类型归一/列投影)——
// 此前写原始入参 rowdefault 不落盘、schema 外列被持久化,重启后行不一致
const puts = {}; const puts = {};
rows.forEach((row, i) => { for (const pk of pks) {
puts[this.rowKey(tableName, String(pks[i] ?? row[pkCol]))] = enc(JSON.stringify(row)); const row = this.memory.getRow(tableName, pk);
}); if (row)
puts[this.rowKey(tableName, pk)] = enc(JSON.stringify(row));
}
await this.kv.putMany(puts); await this.kv.putMany(puts);
return pks; return pks;
} }
@@ -4910,7 +4973,8 @@ class WAL {
const computedNew = crc32(recordBytes); const computedNew = crc32(recordBytes);
const computedLegacy = this.legacyChecksum(recordBytes); const computedLegacy = this.legacyChecksum(recordBytes);
if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) { if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) {
// CRC 不匹配,跳过此损坏记录 // CRC 不匹配,跳过此损坏记录(长度字段链完整时后续好记录仍可恢复,
// 行为由 aria-wal-crc 测试锁定)
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`); console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`);
continue; continue;
@@ -6875,6 +6939,22 @@ class AriaEngine {
validatedRows.push({ row: validated, pkValue, key: `${tableName}:${pkValue}` }); validatedRows.push({ row: validated, pkValue, key: `${tableName}:${pkValue}` });
} }
await this.lsm.prefetchKeys(validatedRows.map((v) => v.key)); await this.lsm.prefetchKeys(validatedRows.map((v) => v.key));
// v0.7.3: 主键批内互查 + 预检 —— 此前 PK 重复检查在写入循环内:
// 第 N 行重复抛错时,前 N-1 行已 put LSM 且其 WAL 记录随 appendBatch 一起
// 丢失 → 语句级部分提交 + 内存/WAL 不一致(与 v0.6.2 的 unique 预检同一阶段)。
const pkSet = new Set();
for (const { pkValue, key } of validatedRows) {
if (pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
pkSet.add(pkValue);
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !existing.__txn_deleted) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
}
// v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain // v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain
for (const colName of uniqueCols) { for (const colName of uniqueCols) {
const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`); const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`);
@@ -6909,13 +6989,7 @@ class AriaEngine {
} }
} }
for (const { row: validated, pkValue, key } of validatedRows) { for (const { row: validated, pkValue, key } of validatedRows) {
// Check duplicate in LSM + transaction snapshot // PK 重复已在批预检阶段检查(v0.7.3),此处不再重复查询
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !existing.__txn_deleted) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
if (this.currentTxnId && this.txnSnapshot) { if (this.currentTxnId && this.txnSnapshot) {
// Within transaction: buffer to snapshot + MVCC version chain // Within transaction: buffer to snapshot + MVCC version chain
this.txnSnapshot.set(key, validated); this.txnSnapshot.set(key, validated);
@@ -7153,25 +7227,9 @@ class AriaEngine {
if (visited.has(visitKey)) if (visited.has(visitKey))
return; return;
visited.add(visitKey); visited.add(visitKey);
// 阶段 1: RESTRICT 检查 // v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 —— checkForeignKeyUpdateRestrict
for (const [refTableName, refSchema] of this.schemas) { // 已在两阶段 update 预检(阶段 1b)覆盖 RESTRICT 与 SET NULL+required
if (refTableName === tableName) // 此处任何修改前重复全表扫描纯属浪费。直接执行 CASCADE / SET NULL。
continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate)
continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName)
continue;
if (colDef.onUpdate !== 'RESTRICT')
continue;
const refRows = await this.getAllRows(refTableName);
if (refRows.some((r) => String(r[colName]) === oldPk)) {
throw new DatabaseError(`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`, 'FOREIGN_KEY_VIOLATION');
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) if (refTableName === tableName)
continue; continue;
@@ -7554,9 +7612,6 @@ class AriaEngine {
// 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失) // 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失)
if (this.secondaryIndexes.has(idxKey)) if (this.secondaryIndexes.has(idxKey))
return; return;
colDef.index = true;
if (unique)
colDef.unique = true;
const idxLsm = new LSM({ const idxLsm = new LSM({
memtableSizeThreshold: this.config.memtableSizeThreshold, memtableSizeThreshold: this.config.memtableSizeThreshold,
levelSizeMultiplier: this.config.levelSizeMultiplier, levelSizeMultiplier: this.config.levelSizeMultiplier,
@@ -7567,16 +7622,38 @@ class AriaEngine {
}); });
await idxLsm.init(); await idxLsm.init();
this.secondaryIndexes.set(idxKey, idxLsm); this.secondaryIndexes.set(idxKey, idxLsm);
try {
// 从主 LSM 重建索引数据 // 从主 LSM 重建索引数据
const pkCol = this.tablePKs.get(tableName); const pkCol = this.tablePKs.get(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
const seen = new Set();
for (const row of rows) { for (const row of rows) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
idxLsm.put(`${String(value)}:${row[pkCol]}`, { pk: row[pkCol] }); const v = String(value);
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// SQLite 语义应报错),与 MemoryEngine 对齐
if (unique && seen.has(v)) {
throw new DatabaseError(`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${v}"`, 'UNIQUE_VIOLATION');
}
seen.add(v);
idxLsm.put(`${v}:${row[pkCol]}`, { pk: row[pkCol] });
} }
} }
await idxLsm.flush(); await idxLsm.flush();
}
catch (error) {
// 回填失败(唯一冲突):清理半初始化索引(内存 + 存储),标志未落,保持原子语义
this.secondaryIndexes.delete(idxKey);
try {
await idxLsm.clear();
}
catch { /* 清理失败不阻塞 */ }
throw error;
}
colDef.index = true;
if (unique)
colDef.unique = true;
await this.persistSchemas(); await this.persistSchemas();
} }
async dropIndex(tableName, column, _indexName) { async dropIndex(tableName, column, _indexName) {
@@ -7613,6 +7690,10 @@ class AriaEngine {
throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE'); throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE');
this.currentTxnId = this.mvcc.beginTransaction(); this.currentTxnId = this.mvcc.beginTransaction();
this.txnSnapshot = new Map(); this.txnSnapshot = new Map();
// v0.7.3-fix: WAL BEGIN 写失败回滚内存事务状态 —— 此前 append 抛错(full 模式)
// 时 currentTxnId 已设置 → TX_ACTIVE 永久泄漏(后续无法开始新事务)。
// 回滚 mvcc 登记 + 快照后重抛,调用方可重试。
try {
await this.wal.append({ await this.wal.append({
type: WALRecordType.BEGIN, type: WALRecordType.BEGIN,
txnId: this.currentTxnId, txnId: this.currentTxnId,
@@ -7620,6 +7701,13 @@ class AriaEngine {
key: '', key: '',
}); });
} }
catch (error) {
this.mvcc.rollbackTransaction(this.currentTxnId);
this.currentTxnId = null;
this.txnSnapshot = null;
throw error;
}
}
async commitTransaction() { async commitTransaction() {
if (!this.currentTxnId) if (!this.currentTxnId)
throw new DatabaseError('No active transaction', 'TX_NONE'); throw new DatabaseError('No active transaction', 'TX_NONE');
@@ -7650,6 +7738,18 @@ class AriaEngine {
async rollbackTransaction() { async rollbackTransaction() {
if (!this.currentTxnId) if (!this.currentTxnId)
throw new DatabaseError('No active transaction', 'TX_NONE'); throw new DatabaseError('No active transaction', 'TX_NONE');
const txnId = this.currentTxnId;
// v0.7.3-fix: 先持久化 WAL ROLLBACK,再回滚内存 —— 与 commitTransaction 的
// "WAL 领先内存"v0.4.3-fix)对齐。此前内存先回滚、ROLLBACK 记录后写:
// full 模式写失败时崩溃重放无 ROLLBACK 记录 → 已回滚事务的数据复活。
// 现在写失败 → 内存未回滚、事务仍活跃(调用方可重试),崩溃后重放
// 看到 ROLLBACK 记录同样不会复活数据。
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId,
tableName: '',
key: '',
});
// v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留) // v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留)
const affectedTables = new Set(); const affectedTables = new Set();
if (this.txnSnapshot) { if (this.txnSnapshot) {
@@ -7659,14 +7759,8 @@ class AriaEngine {
affectedTables.add(key.slice(0, idx)); affectedTables.add(key.slice(0, idx));
} }
} }
this.mvcc.rollbackTransaction(this.currentTxnId); this.mvcc.rollbackTransaction(txnId);
this.txnSnapshot = null; this.txnSnapshot = null;
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId: this.currentTxnId,
tableName: '',
key: '',
});
this.currentTxnId = null; this.currentTxnId = null;
// v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引 // v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引
for (const tableName of affectedTables) { for (const tableName of affectedTables) {
@@ -8084,10 +8178,25 @@ class AriaEngine {
if (!schema) if (!schema)
return null; return null;
const pkCol = this.tablePKs.get(tableName); const pkCol = this.tablePKs.get(tableName);
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// 跳过 $and/$or/$not 逻辑组合 // `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
if (col === '$and' || col === '$or' || col === '$not') // $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat = [];
const collect = (w) => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v)
collect(sub);
continue; continue;
}
if (k === '$or' || k === '$not' || k === '$exists')
continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
const colDef = schema.columns[col]; const colDef = schema.columns[col];
const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey); const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey);
if (!hasIndex && col !== pkCol) if (!hasIndex && col !== pkCol)
@@ -8165,18 +8274,31 @@ class AriaEngine {
// v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行) // v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行)
if (c.$in.some((v) => v === null)) if (c.$in.some((v) => v === null))
continue; continue;
// v0.7.3-perf: 批级预加载全部值的索引范围 + 主表行(各一次 drainChain)——
// 此前逐值 indexScanToRows:每个值一次 prefetchRange + prefetchKeys
// 后台 compaction 长耗时时 N 倍放大(与 v0.6.1 修的 insert 批量预加载
// 性能悬崖同类)。批级预加载后循环内同步 rangeScan/get。
const values = c.$in.map((v) => String(v));
await idxLsm.prefetchPrefixRanges(values.map((v) => [v, `${v}\uffff`]));
const results = []; const results = [];
const seenPks = new Set(); // v0.4.1: IN 值可能重复,按 pk 去重 const seenPks = new Set(); // v0.4.1: IN 值可能重复,按 pk 去重
for (const val of c.$in) { const pks = [];
const rows = await this.indexScanToRows(tableName, pkCol, idxLsm, String(val), String(val)); for (const val of values) {
for (const row of rows) { const entries = idxLsm.rangeScan(val, `${val}\uffff`);
const pk = String(row[pkCol]); for (const [, idxEntry] of entries) {
if (!seenPks.has(pk)) { const pk = idxEntry.pk;
if (pk && !seenPks.has(pk)) {
seenPks.add(pk); seenPks.add(pk);
results.push(row); pks.push(pk);
} }
} }
} }
await this.lsm.prefetchKeys(pks.map((pk) => `${tableName}:${pk}`));
for (const pk of pks) {
const row = this.lsm.get(`${tableName}:${pk}`);
if (row)
results.push({ ...row, [pkCol]: pk });
}
return results; return results;
} }
// $gt / $gte / $lt / $lte → 范围扫描 // $gt / $gte / $lt / $lte → 范围扫描
@@ -8240,10 +8362,6 @@ class AriaEngine {
this.mvcc.gc(50); this.mvcc.gc(50);
} }
} }
/** 估算 WAL 大小(字节) */
getWALEstimatedSize() {
return this.wal.getBufferedCount() * 200; // 粗略估算每条 ~200B
}
/** /**
* ANALYZE: 收集表统计信息 * ANALYZE: 收集表统计信息
* 返回行数平均行大小索引深度等 * 返回行数平均行大小索引深度等
@@ -8252,15 +8370,28 @@ class AriaEngine {
this.ensureOpen(); this.ensureOpen();
this.ensureTable(tableName); this.ensureTable(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
// v0.7.3: 统计汇总主 LSM + 该表全部二级索引 LSM —— 此前只统计主 LSM,
// 表带多个索引时索引深度/SSTable 数量严重低估
let sstableCount = this.lsm.getStats().sstableCount;
let memtableSize = this.lsm.getStats().memtableSize;
let indexDepth = this.lsm.getStats().levelCounts.filter((c) => c > 0).length;
for (const [idxKey, idxLsm] of this.secondaryIndexes) {
if (!idxKey.startsWith(`${tableName}:idx:`))
continue;
const s = idxLsm.getStats();
sstableCount += s.sstableCount;
memtableSize += s.memtableSize;
indexDepth = Math.max(indexDepth, s.levelCounts.filter((c) => c > 0).length);
}
const stats = { const stats = {
table: tableName, table: tableName,
rowCount: rows.length, rowCount: rows.length,
avgRowSize: rows.length > 0 avgRowSize: rows.length > 0
? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length) ? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length)
: 0, : 0,
indexDepth: this.lsm.getStats().levelCounts.filter((c) => c > 0).length, indexDepth,
sstableCount: this.lsm.getStats().sstableCount, sstableCount,
memtableSize: this.lsm.getStats().memtableSize, memtableSize,
estimatedMemory: this.lsm.getEstimatedMemory(), estimatedMemory: this.lsm.getEstimatedMemory(),
}; };
// 列基数统计 // 列基数统计
@@ -9602,6 +9733,12 @@ class Parser {
if (this.curTokenIs(TokenType.STAR)) { if (this.curTokenIs(TokenType.STAR)) {
columns.push('*'); columns.push('*');
this.nextToken(); this.nextToken();
// v0.7.3: `SELECT *, col [AS alias], ...` —— '*' 后可继续列列表
// (此前 '*' 独占分支,逗号后直接 PARSE_ERRORexecutor 侧投影已支持混合)
while (this.curTokenIs(TokenType.COMMA)) {
this.nextToken();
columns.push(this.parseColumnWithAlias());
}
} }
else { else {
columns.push(...this.parseColumnList()); columns.push(...this.parseColumnList());
@@ -10727,26 +10864,36 @@ class QueryExecutor {
catch { /* 非查询语句无 QueryPlan */ } catch { /* 非查询语句无 QueryPlan */ }
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。 // v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。 // 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
// v0.7.3: 递归识别 $and 嵌套等值条件(与 Memory/Aria 的 $and 下推行为对齐;
// $or/$not 不下推,保持 none)。
let usingIndex = plan?.table ? 'none' : 'none'; let usingIndex = plan?.table ? 'none' : 'none';
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) { if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
try { try {
const schema = await this.engine.getTableSchema(plan.table); const schema = await this.engine.getTableSchema(plan.table);
if (schema) { if (schema) {
for (const col of Object.keys(plan.where)) { const findIndex = (w) => {
if (col.startsWith('$')) for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v) {
const hit = findIndex(sub);
if (hit)
return hit;
}
continue; continue;
const colDef = schema.columns[col]; }
if (k === '$or' || k === '$not')
continue;
const colDef = schema.columns[k];
if (!colDef) if (!colDef)
continue; continue;
if (colDef.primaryKey) { if (colDef.primaryKey)
usingIndex = 'pk'; return 'pk';
break; if (colDef.index || colDef.unique)
} return `index:${k}`;
if (colDef.index || colDef.unique) {
usingIndex = `index:${col}`;
break;
}
} }
return null;
};
usingIndex = findIndex(plan.where) ?? 'none';
} }
} }
catch { /* schema 读取失败保持 none */ } catch { /* schema 读取失败保持 none */ }
@@ -10877,7 +11024,11 @@ class QueryExecutor {
} }
if (stmt.orderBy && stmt.orderBy.length > 0) if (stmt.orderBy && stmt.orderBy.length > 0)
rows = applyOrderBy(rows, stmt.orderBy); rows = applyOrderBy(rows, stmt.orderBy);
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0 && stmt.columns[0] !== '*') { // v0.7.3: `SELECT *, col AS alias` —— 此前 columns[0]==='*' 直接不投影,
// 别名列/常量列丢失。仅当 '*' 是唯一列时跳过投影(projectRow 对裸 '*'
// 合并原行全部列,其余表达式覆盖/追加)
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0
&& !(stmt.columns.length === 1 && stmt.columns[0] === '*')) {
rows = rows.map((row) => this.projectRow(row, stmt.columns)); rows = rows.map((row) => this.projectRow(row, stmt.columns));
} }
// v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序 // v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序
@@ -11447,9 +11598,13 @@ class QueryExecutor {
const aliasCols = []; const aliasCols = [];
const caseCols = []; const caseCols = [];
const constCols = []; const constCols = [];
// v0.7.3: 裸 '*' 与列表达式混合(SELECT *, name AS nick)→ 原行全部列为基
let hasStar = false;
for (const col of columns) { for (const col of columns) {
if (col === '*') if (col === '*') {
hasStar = true;
continue; continue;
}
const expr = parseCaseExpression(col); const expr = parseCaseExpression(col);
if (expr) { if (expr) {
caseCols.push({ alias: expr.alias ?? col, expr }); caseCols.push({ alias: expr.alias ?? col, expr });
@@ -11463,20 +11618,26 @@ class QueryExecutor {
// v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出 // v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出
const lit = col.match(/^'(.*)'$/s); const lit = col.match(/^'(.*)'$/s);
if (lit) { if (lit) {
const value = lit[1].replace(/\\'/g, "'"); // v0.7.3: SQL 标准 '' 转义还原(readString 已把 '' 合并为单个 '
// 打包回列的文本中相邻两个 ' 即一个引号字面量)
const value = lit[1].replace(/''/g, "'");
constCols.push({ key: col, value }); constCols.push({ key: col, value });
continue; continue;
} }
plain.push(col); plain.push(col);
} }
const projected = plain.length > 0 ? projectColumns(row, plain) : {}; // v0.7.3: hasStar 时以原行全部列为基(projectColumns 仅投影 plain 列,不含 * 的其余列)
const projected = hasStar
? { ...row }
: (plain.length > 0 ? projectColumns(row, plain) : {});
for (const { alias, source } of aliasCols) { for (const { alias, source } of aliasCols) {
if (source === '*') { if (source === '*') {
Object.assign(projected, row); Object.assign(projected, row);
} }
else { else {
const lit = source.match(/^'(.*)'$/s); const lit = source.match(/^'(.*)'$/s);
projected[alias] = lit ? lit[1].replace(/\\'/g, "'") : row[source]; // v0.7.3: 同 constCols —— SQL 标准 '' 转义还原
projected[alias] = lit ? lit[1].replace(/''/g, "'") : row[source];
} }
} }
for (const { key, value } of constCols) { for (const { key, value } of constCols) {
@@ -12348,9 +12509,41 @@ class MetonaSqlark {
const select = stmt; const select = stmt;
// 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY // 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY
const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c)); const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c));
// v0.7.3: WHERE 含子查询($subquery / $exists / 嵌套 $col 列引用)不可流式 ——
// 引擎层 matchWhere 的 $in/$nin 遇未解析的 $subquery 对象返回 false → 所有行
// 被静默过滤(空结果);$col 操作符无对应匹配分支会抛 QUERY_ERROR。
// 递归检测后回退物化路径(resolveSubqueries 正确解析)。
const hasSubquery = (where) => {
if (!where)
return false;
for (const [k, v] of Object.entries(where)) {
if (k === '$and' || k === '$or') {
if (v.some((sub) => hasSubquery(sub)))
return true;
continue;
}
if (k === '$not') {
if (hasSubquery(v))
return true;
continue;
}
if (k === '$exists')
return true;
if (typeof v === 'object' && v !== null) {
for (const [, operand] of Object.entries(v)) {
if (typeof operand === 'object' && operand !== null) {
const ops = operand;
if ('$subquery' in ops || '$col' in ops)
return true;
}
}
}
}
return false;
};
const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct
&& !aggregate && !(select.orderBy && select.orderBy.length > 0) && !aggregate && !(select.orderBy && select.orderBy.length > 0)
&& !(select.where && select.where['$exists'] !== undefined); && !hasSubquery(select.where);
if (streamable && typeof this.engine.findStream === 'function') { if (streamable && typeof this.engine.findStream === 'function') {
// 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化 // 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化
const isAsync = onRow.constructor?.name === 'AsyncFunction'; const isAsync = onRow.constructor?.name === 'AsyncFunction';
@@ -12507,9 +12700,20 @@ class MetonaSqlark {
async triggerStatementHooks(stmt, phase, result) { async triggerStatementHooks(stmt, phase, result) {
switch (stmt.type) { switch (stmt.type) {
case 'INSERT': { case 'INSERT': {
// v0.7.3: 列映射对齐 executor —— 省略列名时按 schema 列顺序映射
// (此前用数字键 String(i),与 executor 写入的真实行键不一致)
let cols = stmt.columns ?? [];
if (cols.length === 0) {
try {
const schema = await this.engine.getTableSchema(stmt.into);
cols = schema ? Object.keys(schema.columns) : [];
}
catch {
cols = [];
}
}
const rows = (stmt.values ?? []).map((vals) => { const rows = (stmt.values ?? []).map((vals) => {
const row = {}; const row = {};
const cols = stmt.columns ?? [];
for (let i = 0; i < vals.length; i++) { for (let i = 0; i < vals.length; i++) {
row[cols[i] ?? String(i)] = vals[i]; row[cols[i] ?? String(i)] = vals[i];
} }
+1 -1
View File
File diff suppressed because one or more lines are too long
+13 -7
View File
@@ -164,7 +164,7 @@ interface MetonaPlugin {
/** 销毁 */ /** 销毁 */
destroy(): void; destroy(): void;
} }
declare const VERSION = "0.7.2"; declare const VERSION = "0.7.3";
/** /**
* metona-sqlark Plugin — 插件系统 * metona-sqlark Plugin — 插件系统
@@ -803,12 +803,20 @@ declare class MemoryEngine implements IStorageEngine {
name: string; name: string;
}): Promise<void>; }): Promise<void>;
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>; insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
/** v0.7.3: 按主键取已验证行(KVStoreEngine 持久化 validated 行用,含 default/类型归一) */
getRow(tableName: string, pkValue: string): Record<string, unknown> | null;
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>; find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
/** v0.4.0: 流式查询 — 逐行回调(单次迭代,不物化结果数组) */ /** v0.4.0: 流式查询 — 逐行回调(单次迭代,不物化结果数组) */
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>; findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>; update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
/** /**
* v0.7.2: 更新唯一性预检 — 批内互查(多条行更新到同一唯一值)+ 索引查 * v0.7.3: 插入唯一性预检 — 批内互查(本批前几行写入同一唯一值)
* + 索引查(表中已有行)。与 update 的 checkUpdateUniqueness 对称,
* 两阶段 insert 预检阶段调用(索引尚未反映本批写入)。
*/
private checkInsertUniqueness;
/**
* v0.7.3: 更新唯一性预检 — 批内互查(多条行更新到同一唯一值)+ 索引查
* (排除自身旧条目)。阶段 1 中索引尚未更新,批内互查避免"两行同时改到 * (排除自身旧条目)。阶段 1 中索引尚未更新,批内互查避免"两行同时改到
* 同一新值"绕过唯一约束。 * 同一新值"绕过唯一约束。
*/ */
@@ -821,7 +829,9 @@ declare class MemoryEngine implements IStorageEngine {
/** /**
* v0.4.2-fix: ON UPDATE 外键级联 — 被引用表主键变更时处理引用表: * v0.4.2-fix: ON UPDATE 外键级联 — 被引用表主键变更时处理引用表:
* RESTRICT 抛错 / CASCADE 更新 FK 值 / SET NULL 置空。 * RESTRICT 抛错 / CASCADE 更新 FK 值 / SET NULL 置空。
* 分两阶段:先全量 RESTRICT 检查(任何修改前),再执行级联(防部分修改)。 * v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 —— checkUpdateRestrict 已在
* 两阶段 update 预检(阶段 1b)覆盖 RESTRICT 与 SET NULL+required
* 此处任何修改前重复全表扫描纯属浪费。直接执行 CASCADE / SET NULL。
*/ */
private applyUpdateCascade; private applyUpdateCascade;
delete(tableName: string, query: QueryPlan): Promise<number>; delete(tableName: string, query: QueryPlan): Promise<number>;
@@ -843,8 +853,6 @@ declare class MemoryEngine implements IStorageEngine {
private getPrimaryKey; private getPrimaryKey;
private validateRow; private validateRow;
private checkType; private checkType;
/** O(1) 唯一性检查:利用哈希索引 */
private checkUniqueness;
/** 索引查找 */ /** 索引查找 */
private tryIndexLookup; private tryIndexLookup;
/** 更新索引 */ /** 更新索引 */
@@ -1145,8 +1153,6 @@ declare class AriaEngine implements IStorageEngine {
private trimAllCaches; private trimAllCaches;
/** 检查内存预算,超出时强制 flush + GC */ /** 检查内存预算,超出时强制 flush + GC */
private checkMemoryBudget; private checkMemoryBudget;
/** 估算 WAL 大小(字节) */
getWALEstimatedSize(): number;
/** /**
* ANALYZE: 收集表统计信息 * ANALYZE: 收集表统计信息
* 返回行数、平均行大小、索引深度等 * 返回行数、平均行大小、索引深度等
+335 -131
View File
@@ -30,7 +30,7 @@ class DatabaseError extends Error {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 版本 // 版本
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
const VERSION = '0.7.2'; const VERSION = '0.7.3';
/** /**
* metona-sqlark Shared WHERE Matcher 统一的条件匹配逻辑 * metona-sqlark Shared WHERE Matcher 统一的条件匹配逻辑
@@ -435,6 +435,11 @@ class MemoryEngine {
if (!schema.columns[column.name]) { if (!schema.columns[column.name]) {
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
} }
// v0.7.3: 被删列是索引列 → 同步清理索引 Map —— 此前残留旧索引:
// 查询已删列仍走旧索引(不含新行)→ 结果不完整(对齐 AriaEngine cleanupTableIndexes
if (schema.columns[column.name].index || schema.columns[column.name].unique) {
this.indexes.get(tableName)?.delete(column.name);
}
delete schema.columns[column.name]; delete schema.columns[column.name];
// 清理已有行中该列的值(find 返回行引用,直接删除生效) // 清理已有行中该列的值(find 返回行引用,直接删除生效)
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
@@ -450,18 +455,42 @@ class MemoryEngine {
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
const pkColumn = this.getPrimaryKey(schema); const pkColumn = this.getPrimaryKey(schema);
const pks = []; const pks = [];
// v0.7.3: 语句级原子性 —— 两阶段(先全量预检,后执行)。
// 此前逐行"校验+写入":第 N 行主键重复/唯一冲突抛错时,前 N-1 行已提交
// (无事务下语句级部分提交,与 v0.7.2 修复的 UPDATE 同类问题)。
const validated = [];
const pkSet = new Set();
const batchUnique = new Map();
// 阶段 1:全量预检(任何一行失败 → 整条语句不执行)
for (const row of rows) { for (const row of rows) {
const validatedRow = this.validateRow(schema, row); const validatedRow = this.validateRow(schema, row);
const pkValue = String(validatedRow[pkColumn]); const pkValue = String(validatedRow[pkColumn]);
if (table.has(pkValue)) // 批内主键互查(内存表尚未反映本批写入)
if (table.has(pkValue) || pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY'); throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
this.checkUniqueness(schema, validatedRow); }
pkSet.add(pkValue);
// v0.7.3: 批内唯一互查 + 索引查(此前两行同批写入同一唯一值时,
// 第一行已写入索引 → 第二行 checkUniqueness 抛错 → 第一行残留)
this.checkInsertUniqueness(schema, tableName, validatedRow, batchUnique);
validated.push(validatedRow);
}
// 阶段 2:执行(预检已通过,此阶段不再抛校验类错误)
for (const validatedRow of validated) {
const pkValue = String(validatedRow[pkColumn]);
table.set(pkValue, validatedRow); table.set(pkValue, validatedRow);
this.updateIndexes(tableName, validatedRow, pkValue); this.updateIndexes(tableName, validatedRow, pkValue);
pks.push(pkValue); pks.push(pkValue);
} }
return pks; return pks;
} }
/** v0.7.3: 按主键取已验证行(KVStoreEngine 持久化 validated 行用,含 default/类型归一) */
getRow(tableName, pkValue) {
const table = this.tables.get(tableName);
if (!table)
return null;
return table.get(pkValue) ?? null;
}
async find(tableName, query) { async find(tableName, query) {
this.ensureTable(tableName); this.ensureTable(tableName);
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
@@ -554,7 +583,37 @@ class MemoryEngine {
return count; return count;
} }
/** /**
* v0.7.2: 更新唯一性预检 批内互查多条行更新到同一唯一值+ 索引查 * v0.7.3: 插入唯一性预检 批内互查本批前几行写入同一唯一值
* + 索引查表中已有行 update checkUpdateUniqueness 对称
* 两阶段 insert 预检阶段调用索引尚未反映本批写入
*/
checkInsertUniqueness(schema, tableName, row, batchUnique) {
const tableIndexes = this.indexes.get(tableName);
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique)
continue;
const value = row[colName];
if (value === undefined || value === null)
continue;
let seen = batchUnique.get(colName);
if (!seen) {
seen = new Set();
batchUnique.set(colName, seen);
}
if (seen.has(value)) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
seen.add(value);
if (!tableIndexes)
continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(value)) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/**
* v0.7.3: 更新唯一性预检 批内互查多条行更新到同一唯一值+ 索引查
* 排除自身旧条目阶段 1 中索引尚未更新批内互查避免"两行同时改到 * 排除自身旧条目阶段 1 中索引尚未更新批内互查避免"两行同时改到
* 同一新值"绕过唯一约束 * 同一新值"绕过唯一约束
*/ */
@@ -624,30 +683,11 @@ class MemoryEngine {
/** /**
* v0.4.2-fix: ON UPDATE 外键级联 被引用表主键变更时处理引用表 * v0.4.2-fix: ON UPDATE 外键级联 被引用表主键变更时处理引用表
* RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空 * RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空
* 分两阶段先全量 RESTRICT 检查任何修改前再执行级联防部分修改 * v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 checkUpdateRestrict 已在
* 两阶段 update 预检阶段 1b覆盖 RESTRICT SET NULL+required
* 此处任何修改前重复全表扫描纯属浪费直接执行 CASCADE / SET NULL
*/ */
async applyUpdateCascade(tableName, oldPk, newPk) { async applyUpdateCascade(tableName, oldPk, newPk) {
// 阶段 1: RESTRICT 检查(引用旧主键的行存在即拒绝)
for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName)
continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate)
continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName)
continue;
const refTableData = this.tables.get(refTableName);
if (!refTableData)
continue;
for (const [, refRow] of refTableData) {
if (String(refRow[colName]) === oldPk && colDef.onUpdate === 'RESTRICT') {
throw new DatabaseError(`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`, 'FOREIGN_KEY_VIOLATION');
}
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) if (refTableName === tableName)
continue; continue;
@@ -678,28 +718,27 @@ class MemoryEngine {
const toDelete = []; const toDelete = [];
for (const [pk, row] of table) { for (const [pk, row] of table) {
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) { if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
// v0.3.3: 删除行前清理其索引条目(修复删除后索引残留) toDelete.push({ pk, row });
this.removeIndexEntries(tableName, row, pk);
toDelete.push(pk);
} }
} }
// v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归), // v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归),
// 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的 // 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的
// 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致) // 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致)
//
// v0.7.3-fix: 索引清理移到预检之后 —— 此前 removeIndexEntries 在收集阶段执行,
// RESTRICT 预检抛错时行未删但索引条目已删 → 唯一约束失效、索引查询丢行
const restrictVisited = new Set(); const restrictVisited = new Set();
for (const pk of toDelete) { for (const { pk } of toDelete) {
const row = table.get(pk);
if (row)
this.checkCascadeRestrict(tableName, pk, restrictVisited); this.checkCascadeRestrict(tableName, pk, restrictVisited);
} }
// 级联删除:检查引用此表的其他表(RESTRICT 已预检通过,此阶段不再抛 // 预检通过:清理索引 + 级联删除(此阶段不再抛校验类错误
let cascadeCount = 0; let cascadeCount = 0;
for (const pk of toDelete) { for (const { pk, row } of toDelete) {
const row = table.get(pk); // v0.3.3: 删除行前清理其索引条目(修复删除后索引残留)
if (row) this.removeIndexEntries(tableName, row, pk);
cascadeCount += await this.cascadeDelete(tableName, pk, row); cascadeCount += await this.cascadeDelete(tableName, pk, row);
} }
for (const pk of toDelete) for (const { pk } of toDelete)
table.delete(pk); table.delete(pk);
return toDelete.length + cascadeCount; return toDelete.length + cascadeCount;
} }
@@ -778,23 +817,35 @@ class MemoryEngine {
throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
if (colDef.index || colDef.unique) if (colDef.index || colDef.unique)
return; // 已存在 return; // 已存在
colDef.index = true;
if (unique)
colDef.unique = true;
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes.has(column)) if (!tableIndexes.has(column))
tableIndexes.set(column, new Map()); tableIndexes.set(column, new Map());
const colIndex = tableIndexes.get(column); const colIndex = tableIndexes.get(column);
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
try {
for (const [pk, row] of table) { for (const [pk, row] of table) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// (SQLite 语义应报错),且此后该列唯一约束永远无法满足
if (unique && colIndex.has(value)) {
throw new DatabaseError(`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${String(value)}"`, 'UNIQUE_VIOLATION');
}
if (!colIndex.has(value)) if (!colIndex.has(value))
colIndex.set(value, new Set()); colIndex.set(value, new Set());
colIndex.get(value).add(pk); colIndex.get(value).add(pk);
} }
} }
} }
catch (error) {
// 回填失败(唯一冲突):清理半初始化索引,标志未落,保持原子语义
tableIndexes.delete(column);
throw error;
}
colDef.index = true;
if (unique)
colDef.unique = true;
}
async dropIndex(tableName, column, _indexName) { async dropIndex(tableName, column, _indexName) {
// v0.7.2: 同 createIndex —— 列级标志修改无法通过事务快照回滚,显式拒绝 // v0.7.2: 同 createIndex —— 列级标志修改无法通过事务快照回滚,显式拒绝
if (this.snapshot) { if (this.snapshot) {
@@ -916,26 +967,30 @@ class MemoryEngine {
break; break;
} }
} }
/** O(1) 唯一性检查:利用哈希索引 */
checkUniqueness(schema, row) {
const tableIndexes = this.indexes.get(schema.name);
if (!tableIndexes)
return;
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique || row[colName] === undefined || row[colName] === null)
continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(row[colName])) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/** 索引查找 */ /** 索引查找 */
tryIndexLookup(tableName, table, query) { tryIndexLookup(tableName, table, query) {
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes || !query.where) if (!tableIndexes || !query.where)
return Array.from(table.values()); return Array.from(table.values());
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
// $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat = [];
const collect = (w) => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v)
collect(sub);
continue;
}
if (k === '$or' || k === '$not')
continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
// v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引 // v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引
let targetValue; let targetValue;
if (typeof condition !== 'object' || condition === null) { if (typeof condition !== 'object' || condition === null) {
@@ -947,6 +1002,11 @@ class MemoryEngine {
else { else {
continue; continue;
} }
// v0.7.3: null/undefined 条件不走索引 —— 索引不含 null 条目,
// colIndex.get(null) 恒 undefined → return [] 短路全表扫描 → 索引列
// IS NULL 恒空(对齐 AriaEngine v0.6.2 修复)
if (targetValue === null || targetValue === undefined)
continue;
const colIndex = tableIndexes.get(col); const colIndex = tableIndexes.get(col);
if (colIndex) { if (colIndex) {
const pks = colIndex.get(targetValue); const pks = colIndex.get(targetValue);
@@ -2293,11 +2353,14 @@ class KVStoreEngine {
const schema = await this.memory.getTableSchema(tableName); const schema = await this.memory.getTableSchema(tableName);
if (!schema) if (!schema)
throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND'); throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema); // v0.7.3: 持久化内存中的 validated 行(含 default 值/类型归一/列投影)——
// 此前写原始入参 rowdefault 不落盘、schema 外列被持久化,重启后行不一致
const puts = {}; const puts = {};
rows.forEach((row, i) => { for (const pk of pks) {
puts[this.rowKey(tableName, String(pks[i] ?? row[pkCol]))] = enc(JSON.stringify(row)); const row = this.memory.getRow(tableName, pk);
}); if (row)
puts[this.rowKey(tableName, pk)] = enc(JSON.stringify(row));
}
await this.kv.putMany(puts); await this.kv.putMany(puts);
return pks; return pks;
} }
@@ -4906,7 +4969,8 @@ class WAL {
const computedNew = crc32(recordBytes); const computedNew = crc32(recordBytes);
const computedLegacy = this.legacyChecksum(recordBytes); const computedLegacy = this.legacyChecksum(recordBytes);
if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) { if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) {
// CRC 不匹配,跳过此损坏记录 // CRC 不匹配,跳过此损坏记录(长度字段链完整时后续好记录仍可恢复,
// 行为由 aria-wal-crc 测试锁定)
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`); console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`);
continue; continue;
@@ -6871,6 +6935,22 @@ class AriaEngine {
validatedRows.push({ row: validated, pkValue, key: `${tableName}:${pkValue}` }); validatedRows.push({ row: validated, pkValue, key: `${tableName}:${pkValue}` });
} }
await this.lsm.prefetchKeys(validatedRows.map((v) => v.key)); await this.lsm.prefetchKeys(validatedRows.map((v) => v.key));
// v0.7.3: 主键批内互查 + 预检 —— 此前 PK 重复检查在写入循环内:
// 第 N 行重复抛错时,前 N-1 行已 put LSM 且其 WAL 记录随 appendBatch 一起
// 丢失 → 语句级部分提交 + 内存/WAL 不一致(与 v0.6.2 的 unique 预检同一阶段)。
const pkSet = new Set();
for (const { pkValue, key } of validatedRows) {
if (pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
pkSet.add(pkValue);
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !existing.__txn_deleted) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
}
// v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain // v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain
for (const colName of uniqueCols) { for (const colName of uniqueCols) {
const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`); const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`);
@@ -6905,13 +6985,7 @@ class AriaEngine {
} }
} }
for (const { row: validated, pkValue, key } of validatedRows) { for (const { row: validated, pkValue, key } of validatedRows) {
// Check duplicate in LSM + transaction snapshot // PK 重复已在批预检阶段检查(v0.7.3),此处不再重复查询
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !existing.__txn_deleted) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
if (this.currentTxnId && this.txnSnapshot) { if (this.currentTxnId && this.txnSnapshot) {
// Within transaction: buffer to snapshot + MVCC version chain // Within transaction: buffer to snapshot + MVCC version chain
this.txnSnapshot.set(key, validated); this.txnSnapshot.set(key, validated);
@@ -7149,25 +7223,9 @@ class AriaEngine {
if (visited.has(visitKey)) if (visited.has(visitKey))
return; return;
visited.add(visitKey); visited.add(visitKey);
// 阶段 1: RESTRICT 检查 // v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 —— checkForeignKeyUpdateRestrict
for (const [refTableName, refSchema] of this.schemas) { // 已在两阶段 update 预检(阶段 1b)覆盖 RESTRICT 与 SET NULL+required
if (refTableName === tableName) // 此处任何修改前重复全表扫描纯属浪费。直接执行 CASCADE / SET NULL。
continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate)
continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName)
continue;
if (colDef.onUpdate !== 'RESTRICT')
continue;
const refRows = await this.getAllRows(refTableName);
if (refRows.some((r) => String(r[colName]) === oldPk)) {
throw new DatabaseError(`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`, 'FOREIGN_KEY_VIOLATION');
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) if (refTableName === tableName)
continue; continue;
@@ -7550,9 +7608,6 @@ class AriaEngine {
// 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失) // 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失)
if (this.secondaryIndexes.has(idxKey)) if (this.secondaryIndexes.has(idxKey))
return; return;
colDef.index = true;
if (unique)
colDef.unique = true;
const idxLsm = new LSM({ const idxLsm = new LSM({
memtableSizeThreshold: this.config.memtableSizeThreshold, memtableSizeThreshold: this.config.memtableSizeThreshold,
levelSizeMultiplier: this.config.levelSizeMultiplier, levelSizeMultiplier: this.config.levelSizeMultiplier,
@@ -7563,16 +7618,38 @@ class AriaEngine {
}); });
await idxLsm.init(); await idxLsm.init();
this.secondaryIndexes.set(idxKey, idxLsm); this.secondaryIndexes.set(idxKey, idxLsm);
try {
// 从主 LSM 重建索引数据 // 从主 LSM 重建索引数据
const pkCol = this.tablePKs.get(tableName); const pkCol = this.tablePKs.get(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
const seen = new Set();
for (const row of rows) { for (const row of rows) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
idxLsm.put(`${String(value)}:${row[pkCol]}`, { pk: row[pkCol] }); const v = String(value);
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// SQLite 语义应报错),与 MemoryEngine 对齐
if (unique && seen.has(v)) {
throw new DatabaseError(`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${v}"`, 'UNIQUE_VIOLATION');
}
seen.add(v);
idxLsm.put(`${v}:${row[pkCol]}`, { pk: row[pkCol] });
} }
} }
await idxLsm.flush(); await idxLsm.flush();
}
catch (error) {
// 回填失败(唯一冲突):清理半初始化索引(内存 + 存储),标志未落,保持原子语义
this.secondaryIndexes.delete(idxKey);
try {
await idxLsm.clear();
}
catch { /* 清理失败不阻塞 */ }
throw error;
}
colDef.index = true;
if (unique)
colDef.unique = true;
await this.persistSchemas(); await this.persistSchemas();
} }
async dropIndex(tableName, column, _indexName) { async dropIndex(tableName, column, _indexName) {
@@ -7609,6 +7686,10 @@ class AriaEngine {
throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE'); throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE');
this.currentTxnId = this.mvcc.beginTransaction(); this.currentTxnId = this.mvcc.beginTransaction();
this.txnSnapshot = new Map(); this.txnSnapshot = new Map();
// v0.7.3-fix: WAL BEGIN 写失败回滚内存事务状态 —— 此前 append 抛错(full 模式)
// 时 currentTxnId 已设置 → TX_ACTIVE 永久泄漏(后续无法开始新事务)。
// 回滚 mvcc 登记 + 快照后重抛,调用方可重试。
try {
await this.wal.append({ await this.wal.append({
type: WALRecordType.BEGIN, type: WALRecordType.BEGIN,
txnId: this.currentTxnId, txnId: this.currentTxnId,
@@ -7616,6 +7697,13 @@ class AriaEngine {
key: '', key: '',
}); });
} }
catch (error) {
this.mvcc.rollbackTransaction(this.currentTxnId);
this.currentTxnId = null;
this.txnSnapshot = null;
throw error;
}
}
async commitTransaction() { async commitTransaction() {
if (!this.currentTxnId) if (!this.currentTxnId)
throw new DatabaseError('No active transaction', 'TX_NONE'); throw new DatabaseError('No active transaction', 'TX_NONE');
@@ -7646,6 +7734,18 @@ class AriaEngine {
async rollbackTransaction() { async rollbackTransaction() {
if (!this.currentTxnId) if (!this.currentTxnId)
throw new DatabaseError('No active transaction', 'TX_NONE'); throw new DatabaseError('No active transaction', 'TX_NONE');
const txnId = this.currentTxnId;
// v0.7.3-fix: 先持久化 WAL ROLLBACK,再回滚内存 —— 与 commitTransaction 的
// "WAL 领先内存"v0.4.3-fix)对齐。此前内存先回滚、ROLLBACK 记录后写:
// full 模式写失败时崩溃重放无 ROLLBACK 记录 → 已回滚事务的数据复活。
// 现在写失败 → 内存未回滚、事务仍活跃(调用方可重试),崩溃后重放
// 看到 ROLLBACK 记录同样不会复活数据。
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId,
tableName: '',
key: '',
});
// v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留) // v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留)
const affectedTables = new Set(); const affectedTables = new Set();
if (this.txnSnapshot) { if (this.txnSnapshot) {
@@ -7655,14 +7755,8 @@ class AriaEngine {
affectedTables.add(key.slice(0, idx)); affectedTables.add(key.slice(0, idx));
} }
} }
this.mvcc.rollbackTransaction(this.currentTxnId); this.mvcc.rollbackTransaction(txnId);
this.txnSnapshot = null; this.txnSnapshot = null;
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId: this.currentTxnId,
tableName: '',
key: '',
});
this.currentTxnId = null; this.currentTxnId = null;
// v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引 // v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引
for (const tableName of affectedTables) { for (const tableName of affectedTables) {
@@ -8080,10 +8174,25 @@ class AriaEngine {
if (!schema) if (!schema)
return null; return null;
const pkCol = this.tablePKs.get(tableName); const pkCol = this.tablePKs.get(tableName);
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// 跳过 $and/$or/$not 逻辑组合 // `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
if (col === '$and' || col === '$or' || col === '$not') // $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat = [];
const collect = (w) => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v)
collect(sub);
continue; continue;
}
if (k === '$or' || k === '$not' || k === '$exists')
continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
const colDef = schema.columns[col]; const colDef = schema.columns[col];
const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey); const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey);
if (!hasIndex && col !== pkCol) if (!hasIndex && col !== pkCol)
@@ -8161,18 +8270,31 @@ class AriaEngine {
// v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行) // v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行)
if (c.$in.some((v) => v === null)) if (c.$in.some((v) => v === null))
continue; continue;
// v0.7.3-perf: 批级预加载全部值的索引范围 + 主表行(各一次 drainChain)——
// 此前逐值 indexScanToRows:每个值一次 prefetchRange + prefetchKeys
// 后台 compaction 长耗时时 N 倍放大(与 v0.6.1 修的 insert 批量预加载
// 性能悬崖同类)。批级预加载后循环内同步 rangeScan/get。
const values = c.$in.map((v) => String(v));
await idxLsm.prefetchPrefixRanges(values.map((v) => [v, `${v}\uffff`]));
const results = []; const results = [];
const seenPks = new Set(); // v0.4.1: IN 值可能重复,按 pk 去重 const seenPks = new Set(); // v0.4.1: IN 值可能重复,按 pk 去重
for (const val of c.$in) { const pks = [];
const rows = await this.indexScanToRows(tableName, pkCol, idxLsm, String(val), String(val)); for (const val of values) {
for (const row of rows) { const entries = idxLsm.rangeScan(val, `${val}\uffff`);
const pk = String(row[pkCol]); for (const [, idxEntry] of entries) {
if (!seenPks.has(pk)) { const pk = idxEntry.pk;
if (pk && !seenPks.has(pk)) {
seenPks.add(pk); seenPks.add(pk);
results.push(row); pks.push(pk);
} }
} }
} }
await this.lsm.prefetchKeys(pks.map((pk) => `${tableName}:${pk}`));
for (const pk of pks) {
const row = this.lsm.get(`${tableName}:${pk}`);
if (row)
results.push({ ...row, [pkCol]: pk });
}
return results; return results;
} }
// $gt / $gte / $lt / $lte → 范围扫描 // $gt / $gte / $lt / $lte → 范围扫描
@@ -8236,10 +8358,6 @@ class AriaEngine {
this.mvcc.gc(50); this.mvcc.gc(50);
} }
} }
/** 估算 WAL 大小(字节) */
getWALEstimatedSize() {
return this.wal.getBufferedCount() * 200; // 粗略估算每条 ~200B
}
/** /**
* ANALYZE: 收集表统计信息 * ANALYZE: 收集表统计信息
* 返回行数平均行大小索引深度等 * 返回行数平均行大小索引深度等
@@ -8248,15 +8366,28 @@ class AriaEngine {
this.ensureOpen(); this.ensureOpen();
this.ensureTable(tableName); this.ensureTable(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
// v0.7.3: 统计汇总主 LSM + 该表全部二级索引 LSM —— 此前只统计主 LSM,
// 表带多个索引时索引深度/SSTable 数量严重低估
let sstableCount = this.lsm.getStats().sstableCount;
let memtableSize = this.lsm.getStats().memtableSize;
let indexDepth = this.lsm.getStats().levelCounts.filter((c) => c > 0).length;
for (const [idxKey, idxLsm] of this.secondaryIndexes) {
if (!idxKey.startsWith(`${tableName}:idx:`))
continue;
const s = idxLsm.getStats();
sstableCount += s.sstableCount;
memtableSize += s.memtableSize;
indexDepth = Math.max(indexDepth, s.levelCounts.filter((c) => c > 0).length);
}
const stats = { const stats = {
table: tableName, table: tableName,
rowCount: rows.length, rowCount: rows.length,
avgRowSize: rows.length > 0 avgRowSize: rows.length > 0
? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length) ? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length)
: 0, : 0,
indexDepth: this.lsm.getStats().levelCounts.filter((c) => c > 0).length, indexDepth,
sstableCount: this.lsm.getStats().sstableCount, sstableCount,
memtableSize: this.lsm.getStats().memtableSize, memtableSize,
estimatedMemory: this.lsm.getEstimatedMemory(), estimatedMemory: this.lsm.getEstimatedMemory(),
}; };
// 列基数统计 // 列基数统计
@@ -9598,6 +9729,12 @@ class Parser {
if (this.curTokenIs(TokenType.STAR)) { if (this.curTokenIs(TokenType.STAR)) {
columns.push('*'); columns.push('*');
this.nextToken(); this.nextToken();
// v0.7.3: `SELECT *, col [AS alias], ...` —— '*' 后可继续列列表
// (此前 '*' 独占分支,逗号后直接 PARSE_ERRORexecutor 侧投影已支持混合)
while (this.curTokenIs(TokenType.COMMA)) {
this.nextToken();
columns.push(this.parseColumnWithAlias());
}
} }
else { else {
columns.push(...this.parseColumnList()); columns.push(...this.parseColumnList());
@@ -10723,26 +10860,36 @@ class QueryExecutor {
catch { /* 非查询语句无 QueryPlan */ } catch { /* 非查询语句无 QueryPlan */ }
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。 // v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。 // 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
// v0.7.3: 递归识别 $and 嵌套等值条件(与 Memory/Aria 的 $and 下推行为对齐;
// $or/$not 不下推,保持 none)。
let usingIndex = plan?.table ? 'none' : 'none'; let usingIndex = plan?.table ? 'none' : 'none';
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) { if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
try { try {
const schema = await this.engine.getTableSchema(plan.table); const schema = await this.engine.getTableSchema(plan.table);
if (schema) { if (schema) {
for (const col of Object.keys(plan.where)) { const findIndex = (w) => {
if (col.startsWith('$')) for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v) {
const hit = findIndex(sub);
if (hit)
return hit;
}
continue; continue;
const colDef = schema.columns[col]; }
if (k === '$or' || k === '$not')
continue;
const colDef = schema.columns[k];
if (!colDef) if (!colDef)
continue; continue;
if (colDef.primaryKey) { if (colDef.primaryKey)
usingIndex = 'pk'; return 'pk';
break; if (colDef.index || colDef.unique)
} return `index:${k}`;
if (colDef.index || colDef.unique) {
usingIndex = `index:${col}`;
break;
}
} }
return null;
};
usingIndex = findIndex(plan.where) ?? 'none';
} }
} }
catch { /* schema 读取失败保持 none */ } catch { /* schema 读取失败保持 none */ }
@@ -10873,7 +11020,11 @@ class QueryExecutor {
} }
if (stmt.orderBy && stmt.orderBy.length > 0) if (stmt.orderBy && stmt.orderBy.length > 0)
rows = applyOrderBy(rows, stmt.orderBy); rows = applyOrderBy(rows, stmt.orderBy);
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0 && stmt.columns[0] !== '*') { // v0.7.3: `SELECT *, col AS alias` —— 此前 columns[0]==='*' 直接不投影,
// 别名列/常量列丢失。仅当 '*' 是唯一列时跳过投影(projectRow 对裸 '*'
// 合并原行全部列,其余表达式覆盖/追加)
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0
&& !(stmt.columns.length === 1 && stmt.columns[0] === '*')) {
rows = rows.map((row) => this.projectRow(row, stmt.columns)); rows = rows.map((row) => this.projectRow(row, stmt.columns));
} }
// v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序 // v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序
@@ -11443,9 +11594,13 @@ class QueryExecutor {
const aliasCols = []; const aliasCols = [];
const caseCols = []; const caseCols = [];
const constCols = []; const constCols = [];
// v0.7.3: 裸 '*' 与列表达式混合(SELECT *, name AS nick)→ 原行全部列为基
let hasStar = false;
for (const col of columns) { for (const col of columns) {
if (col === '*') if (col === '*') {
hasStar = true;
continue; continue;
}
const expr = parseCaseExpression(col); const expr = parseCaseExpression(col);
if (expr) { if (expr) {
caseCols.push({ alias: expr.alias ?? col, expr }); caseCols.push({ alias: expr.alias ?? col, expr });
@@ -11459,20 +11614,26 @@ class QueryExecutor {
// v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出 // v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出
const lit = col.match(/^'(.*)'$/s); const lit = col.match(/^'(.*)'$/s);
if (lit) { if (lit) {
const value = lit[1].replace(/\\'/g, "'"); // v0.7.3: SQL 标准 '' 转义还原(readString 已把 '' 合并为单个 '
// 打包回列的文本中相邻两个 ' 即一个引号字面量)
const value = lit[1].replace(/''/g, "'");
constCols.push({ key: col, value }); constCols.push({ key: col, value });
continue; continue;
} }
plain.push(col); plain.push(col);
} }
const projected = plain.length > 0 ? projectColumns(row, plain) : {}; // v0.7.3: hasStar 时以原行全部列为基(projectColumns 仅投影 plain 列,不含 * 的其余列)
const projected = hasStar
? { ...row }
: (plain.length > 0 ? projectColumns(row, plain) : {});
for (const { alias, source } of aliasCols) { for (const { alias, source } of aliasCols) {
if (source === '*') { if (source === '*') {
Object.assign(projected, row); Object.assign(projected, row);
} }
else { else {
const lit = source.match(/^'(.*)'$/s); const lit = source.match(/^'(.*)'$/s);
projected[alias] = lit ? lit[1].replace(/\\'/g, "'") : row[source]; // v0.7.3: 同 constCols —— SQL 标准 '' 转义还原
projected[alias] = lit ? lit[1].replace(/''/g, "'") : row[source];
} }
} }
for (const { key, value } of constCols) { for (const { key, value } of constCols) {
@@ -12344,9 +12505,41 @@ class MetonaSqlark {
const select = stmt; const select = stmt;
// 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY // 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY
const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c)); const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c));
// v0.7.3: WHERE 含子查询($subquery / $exists / 嵌套 $col 列引用)不可流式 ——
// 引擎层 matchWhere 的 $in/$nin 遇未解析的 $subquery 对象返回 false → 所有行
// 被静默过滤(空结果);$col 操作符无对应匹配分支会抛 QUERY_ERROR。
// 递归检测后回退物化路径(resolveSubqueries 正确解析)。
const hasSubquery = (where) => {
if (!where)
return false;
for (const [k, v] of Object.entries(where)) {
if (k === '$and' || k === '$or') {
if (v.some((sub) => hasSubquery(sub)))
return true;
continue;
}
if (k === '$not') {
if (hasSubquery(v))
return true;
continue;
}
if (k === '$exists')
return true;
if (typeof v === 'object' && v !== null) {
for (const [, operand] of Object.entries(v)) {
if (typeof operand === 'object' && operand !== null) {
const ops = operand;
if ('$subquery' in ops || '$col' in ops)
return true;
}
}
}
}
return false;
};
const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct
&& !aggregate && !(select.orderBy && select.orderBy.length > 0) && !aggregate && !(select.orderBy && select.orderBy.length > 0)
&& !(select.where && select.where['$exists'] !== undefined); && !hasSubquery(select.where);
if (streamable && typeof this.engine.findStream === 'function') { if (streamable && typeof this.engine.findStream === 'function') {
// 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化 // 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化
const isAsync = onRow.constructor?.name === 'AsyncFunction'; const isAsync = onRow.constructor?.name === 'AsyncFunction';
@@ -12503,9 +12696,20 @@ class MetonaSqlark {
async triggerStatementHooks(stmt, phase, result) { async triggerStatementHooks(stmt, phase, result) {
switch (stmt.type) { switch (stmt.type) {
case 'INSERT': { case 'INSERT': {
// v0.7.3: 列映射对齐 executor —— 省略列名时按 schema 列顺序映射
// (此前用数字键 String(i),与 executor 写入的真实行键不一致)
let cols = stmt.columns ?? [];
if (cols.length === 0) {
try {
const schema = await this.engine.getTableSchema(stmt.into);
cols = schema ? Object.keys(schema.columns) : [];
}
catch {
cols = [];
}
}
const rows = (stmt.values ?? []).map((vals) => { const rows = (stmt.values ?? []).map((vals) => {
const row = {}; const row = {};
const cols = stmt.columns ?? [];
for (let i = 0; i < vals.length; i++) { for (let i = 0; i < vals.length; i++) {
row[cols[i] ?? String(i)] = vals[i]; row[cols[i] ?? String(i)] = vals[i];
} }
+1 -1
View File
File diff suppressed because one or more lines are too long
+335 -131
View File
@@ -36,7 +36,7 @@
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 版本 // 版本
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
const VERSION = '0.7.2'; const VERSION = '0.7.3';
/** /**
* metona-sqlark Shared WHERE Matcher 统一的条件匹配逻辑 * metona-sqlark Shared WHERE Matcher 统一的条件匹配逻辑
@@ -441,6 +441,11 @@
if (!schema.columns[column.name]) { if (!schema.columns[column.name]) {
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
} }
// v0.7.3: 被删列是索引列 → 同步清理索引 Map —— 此前残留旧索引:
// 查询已删列仍走旧索引(不含新行)→ 结果不完整(对齐 AriaEngine cleanupTableIndexes
if (schema.columns[column.name].index || schema.columns[column.name].unique) {
this.indexes.get(tableName)?.delete(column.name);
}
delete schema.columns[column.name]; delete schema.columns[column.name];
// 清理已有行中该列的值(find 返回行引用,直接删除生效) // 清理已有行中该列的值(find 返回行引用,直接删除生效)
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
@@ -456,18 +461,42 @@
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
const pkColumn = this.getPrimaryKey(schema); const pkColumn = this.getPrimaryKey(schema);
const pks = []; const pks = [];
// v0.7.3: 语句级原子性 —— 两阶段(先全量预检,后执行)。
// 此前逐行"校验+写入":第 N 行主键重复/唯一冲突抛错时,前 N-1 行已提交
// (无事务下语句级部分提交,与 v0.7.2 修复的 UPDATE 同类问题)。
const validated = [];
const pkSet = new Set();
const batchUnique = new Map();
// 阶段 1:全量预检(任何一行失败 → 整条语句不执行)
for (const row of rows) { for (const row of rows) {
const validatedRow = this.validateRow(schema, row); const validatedRow = this.validateRow(schema, row);
const pkValue = String(validatedRow[pkColumn]); const pkValue = String(validatedRow[pkColumn]);
if (table.has(pkValue)) // 批内主键互查(内存表尚未反映本批写入)
if (table.has(pkValue) || pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY'); throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
this.checkUniqueness(schema, validatedRow); }
pkSet.add(pkValue);
// v0.7.3: 批内唯一互查 + 索引查(此前两行同批写入同一唯一值时,
// 第一行已写入索引 → 第二行 checkUniqueness 抛错 → 第一行残留)
this.checkInsertUniqueness(schema, tableName, validatedRow, batchUnique);
validated.push(validatedRow);
}
// 阶段 2:执行(预检已通过,此阶段不再抛校验类错误)
for (const validatedRow of validated) {
const pkValue = String(validatedRow[pkColumn]);
table.set(pkValue, validatedRow); table.set(pkValue, validatedRow);
this.updateIndexes(tableName, validatedRow, pkValue); this.updateIndexes(tableName, validatedRow, pkValue);
pks.push(pkValue); pks.push(pkValue);
} }
return pks; return pks;
} }
/** v0.7.3: 按主键取已验证行(KVStoreEngine 持久化 validated 行用,含 default/类型归一) */
getRow(tableName, pkValue) {
const table = this.tables.get(tableName);
if (!table)
return null;
return table.get(pkValue) ?? null;
}
async find(tableName, query) { async find(tableName, query) {
this.ensureTable(tableName); this.ensureTable(tableName);
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
@@ -560,7 +589,37 @@
return count; return count;
} }
/** /**
* v0.7.2: 更新唯一性预检 批内互查多条行更新到同一唯一值+ 索引查 * v0.7.3: 插入唯一性预检 批内互查本批前几行写入同一唯一值
* + 索引查表中已有行 update checkUpdateUniqueness 对称
* 两阶段 insert 预检阶段调用索引尚未反映本批写入
*/
checkInsertUniqueness(schema, tableName, row, batchUnique) {
const tableIndexes = this.indexes.get(tableName);
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique)
continue;
const value = row[colName];
if (value === undefined || value === null)
continue;
let seen = batchUnique.get(colName);
if (!seen) {
seen = new Set();
batchUnique.set(colName, seen);
}
if (seen.has(value)) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
seen.add(value);
if (!tableIndexes)
continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(value)) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/**
* v0.7.3: 更新唯一性预检 批内互查多条行更新到同一唯一值+ 索引查
* 排除自身旧条目阶段 1 中索引尚未更新批内互查避免"两行同时改到 * 排除自身旧条目阶段 1 中索引尚未更新批内互查避免"两行同时改到
* 同一新值"绕过唯一约束 * 同一新值"绕过唯一约束
*/ */
@@ -630,30 +689,11 @@
/** /**
* v0.4.2-fix: ON UPDATE 外键级联 被引用表主键变更时处理引用表 * v0.4.2-fix: ON UPDATE 外键级联 被引用表主键变更时处理引用表
* RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空 * RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空
* 分两阶段先全量 RESTRICT 检查任何修改前再执行级联防部分修改 * v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 checkUpdateRestrict 已在
* 两阶段 update 预检阶段 1b覆盖 RESTRICT SET NULL+required
* 此处任何修改前重复全表扫描纯属浪费直接执行 CASCADE / SET NULL
*/ */
async applyUpdateCascade(tableName, oldPk, newPk) { async applyUpdateCascade(tableName, oldPk, newPk) {
// 阶段 1: RESTRICT 检查(引用旧主键的行存在即拒绝)
for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName)
continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate)
continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName)
continue;
const refTableData = this.tables.get(refTableName);
if (!refTableData)
continue;
for (const [, refRow] of refTableData) {
if (String(refRow[colName]) === oldPk && colDef.onUpdate === 'RESTRICT') {
throw new DatabaseError(`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`, 'FOREIGN_KEY_VIOLATION');
}
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) if (refTableName === tableName)
continue; continue;
@@ -684,28 +724,27 @@
const toDelete = []; const toDelete = [];
for (const [pk, row] of table) { for (const [pk, row] of table) {
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) { if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
// v0.3.3: 删除行前清理其索引条目(修复删除后索引残留) toDelete.push({ pk, row });
this.removeIndexEntries(tableName, row, pk);
toDelete.push(pk);
} }
} }
// v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归), // v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归),
// 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的 // 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的
// 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致) // 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致)
//
// v0.7.3-fix: 索引清理移到预检之后 —— 此前 removeIndexEntries 在收集阶段执行,
// RESTRICT 预检抛错时行未删但索引条目已删 → 唯一约束失效、索引查询丢行
const restrictVisited = new Set(); const restrictVisited = new Set();
for (const pk of toDelete) { for (const { pk } of toDelete) {
const row = table.get(pk);
if (row)
this.checkCascadeRestrict(tableName, pk, restrictVisited); this.checkCascadeRestrict(tableName, pk, restrictVisited);
} }
// 级联删除:检查引用此表的其他表(RESTRICT 已预检通过,此阶段不再抛 // 预检通过:清理索引 + 级联删除(此阶段不再抛校验类错误
let cascadeCount = 0; let cascadeCount = 0;
for (const pk of toDelete) { for (const { pk, row } of toDelete) {
const row = table.get(pk); // v0.3.3: 删除行前清理其索引条目(修复删除后索引残留)
if (row) this.removeIndexEntries(tableName, row, pk);
cascadeCount += await this.cascadeDelete(tableName, pk, row); cascadeCount += await this.cascadeDelete(tableName, pk, row);
} }
for (const pk of toDelete) for (const { pk } of toDelete)
table.delete(pk); table.delete(pk);
return toDelete.length + cascadeCount; return toDelete.length + cascadeCount;
} }
@@ -784,23 +823,35 @@
throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
if (colDef.index || colDef.unique) if (colDef.index || colDef.unique)
return; // 已存在 return; // 已存在
colDef.index = true;
if (unique)
colDef.unique = true;
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes.has(column)) if (!tableIndexes.has(column))
tableIndexes.set(column, new Map()); tableIndexes.set(column, new Map());
const colIndex = tableIndexes.get(column); const colIndex = tableIndexes.get(column);
const table = this.tables.get(tableName); const table = this.tables.get(tableName);
try {
for (const [pk, row] of table) { for (const [pk, row] of table) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// (SQLite 语义应报错),且此后该列唯一约束永远无法满足
if (unique && colIndex.has(value)) {
throw new DatabaseError(`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${String(value)}"`, 'UNIQUE_VIOLATION');
}
if (!colIndex.has(value)) if (!colIndex.has(value))
colIndex.set(value, new Set()); colIndex.set(value, new Set());
colIndex.get(value).add(pk); colIndex.get(value).add(pk);
} }
} }
} }
catch (error) {
// 回填失败(唯一冲突):清理半初始化索引,标志未落,保持原子语义
tableIndexes.delete(column);
throw error;
}
colDef.index = true;
if (unique)
colDef.unique = true;
}
async dropIndex(tableName, column, _indexName) { async dropIndex(tableName, column, _indexName) {
// v0.7.2: 同 createIndex —— 列级标志修改无法通过事务快照回滚,显式拒绝 // v0.7.2: 同 createIndex —— 列级标志修改无法通过事务快照回滚,显式拒绝
if (this.snapshot) { if (this.snapshot) {
@@ -922,26 +973,30 @@
break; break;
} }
} }
/** O(1) 唯一性检查:利用哈希索引 */
checkUniqueness(schema, row) {
const tableIndexes = this.indexes.get(schema.name);
if (!tableIndexes)
return;
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique || row[colName] === undefined || row[colName] === null)
continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(row[colName])) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/** 索引查找 */ /** 索引查找 */
tryIndexLookup(tableName, table, query) { tryIndexLookup(tableName, table, query) {
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes || !query.where) if (!tableIndexes || !query.where)
return Array.from(table.values()); return Array.from(table.values());
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
// $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat = [];
const collect = (w) => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v)
collect(sub);
continue;
}
if (k === '$or' || k === '$not')
continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
// v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引 // v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引
let targetValue; let targetValue;
if (typeof condition !== 'object' || condition === null) { if (typeof condition !== 'object' || condition === null) {
@@ -953,6 +1008,11 @@
else { else {
continue; continue;
} }
// v0.7.3: null/undefined 条件不走索引 —— 索引不含 null 条目,
// colIndex.get(null) 恒 undefined → return [] 短路全表扫描 → 索引列
// IS NULL 恒空(对齐 AriaEngine v0.6.2 修复)
if (targetValue === null || targetValue === undefined)
continue;
const colIndex = tableIndexes.get(col); const colIndex = tableIndexes.get(col);
if (colIndex) { if (colIndex) {
const pks = colIndex.get(targetValue); const pks = colIndex.get(targetValue);
@@ -2299,11 +2359,14 @@
const schema = await this.memory.getTableSchema(tableName); const schema = await this.memory.getTableSchema(tableName);
if (!schema) if (!schema)
throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND'); throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema); // v0.7.3: 持久化内存中的 validated 行(含 default 值/类型归一/列投影)——
// 此前写原始入参 rowdefault 不落盘、schema 外列被持久化,重启后行不一致
const puts = {}; const puts = {};
rows.forEach((row, i) => { for (const pk of pks) {
puts[this.rowKey(tableName, String(pks[i] ?? row[pkCol]))] = enc(JSON.stringify(row)); const row = this.memory.getRow(tableName, pk);
}); if (row)
puts[this.rowKey(tableName, pk)] = enc(JSON.stringify(row));
}
await this.kv.putMany(puts); await this.kv.putMany(puts);
return pks; return pks;
} }
@@ -4912,7 +4975,8 @@
const computedNew = crc32(recordBytes); const computedNew = crc32(recordBytes);
const computedLegacy = this.legacyChecksum(recordBytes); const computedLegacy = this.legacyChecksum(recordBytes);
if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) { if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) {
// CRC 不匹配,跳过此损坏记录 // CRC 不匹配,跳过此损坏记录(长度字段链完整时后续好记录仍可恢复,
// 行为由 aria-wal-crc 测试锁定)
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`); console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`);
continue; continue;
@@ -6877,6 +6941,22 @@
validatedRows.push({ row: validated, pkValue, key: `${tableName}:${pkValue}` }); validatedRows.push({ row: validated, pkValue, key: `${tableName}:${pkValue}` });
} }
await this.lsm.prefetchKeys(validatedRows.map((v) => v.key)); await this.lsm.prefetchKeys(validatedRows.map((v) => v.key));
// v0.7.3: 主键批内互查 + 预检 —— 此前 PK 重复检查在写入循环内:
// 第 N 行重复抛错时,前 N-1 行已 put LSM 且其 WAL 记录随 appendBatch 一起
// 丢失 → 语句级部分提交 + 内存/WAL 不一致(与 v0.6.2 的 unique 预检同一阶段)。
const pkSet = new Set();
for (const { pkValue, key } of validatedRows) {
if (pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
pkSet.add(pkValue);
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !existing.__txn_deleted) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
}
// v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain // v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain
for (const colName of uniqueCols) { for (const colName of uniqueCols) {
const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`); const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`);
@@ -6911,13 +6991,7 @@
} }
} }
for (const { row: validated, pkValue, key } of validatedRows) { for (const { row: validated, pkValue, key } of validatedRows) {
// Check duplicate in LSM + transaction snapshot // PK 重复已在批预检阶段检查(v0.7.3),此处不再重复查询
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !existing.__txn_deleted) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
if (this.currentTxnId && this.txnSnapshot) { if (this.currentTxnId && this.txnSnapshot) {
// Within transaction: buffer to snapshot + MVCC version chain // Within transaction: buffer to snapshot + MVCC version chain
this.txnSnapshot.set(key, validated); this.txnSnapshot.set(key, validated);
@@ -7155,25 +7229,9 @@
if (visited.has(visitKey)) if (visited.has(visitKey))
return; return;
visited.add(visitKey); visited.add(visitKey);
// 阶段 1: RESTRICT 检查 // v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 —— checkForeignKeyUpdateRestrict
for (const [refTableName, refSchema] of this.schemas) { // 已在两阶段 update 预检(阶段 1b)覆盖 RESTRICT 与 SET NULL+required
if (refTableName === tableName) // 此处任何修改前重复全表扫描纯属浪费。直接执行 CASCADE / SET NULL。
continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate)
continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName)
continue;
if (colDef.onUpdate !== 'RESTRICT')
continue;
const refRows = await this.getAllRows(refTableName);
if (refRows.some((r) => String(r[colName]) === oldPk)) {
throw new DatabaseError(`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`, 'FOREIGN_KEY_VIOLATION');
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) if (refTableName === tableName)
continue; continue;
@@ -7556,9 +7614,6 @@
// 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失) // 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失)
if (this.secondaryIndexes.has(idxKey)) if (this.secondaryIndexes.has(idxKey))
return; return;
colDef.index = true;
if (unique)
colDef.unique = true;
const idxLsm = new LSM({ const idxLsm = new LSM({
memtableSizeThreshold: this.config.memtableSizeThreshold, memtableSizeThreshold: this.config.memtableSizeThreshold,
levelSizeMultiplier: this.config.levelSizeMultiplier, levelSizeMultiplier: this.config.levelSizeMultiplier,
@@ -7569,16 +7624,38 @@
}); });
await idxLsm.init(); await idxLsm.init();
this.secondaryIndexes.set(idxKey, idxLsm); this.secondaryIndexes.set(idxKey, idxLsm);
try {
// 从主 LSM 重建索引数据 // 从主 LSM 重建索引数据
const pkCol = this.tablePKs.get(tableName); const pkCol = this.tablePKs.get(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
const seen = new Set();
for (const row of rows) { for (const row of rows) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
idxLsm.put(`${String(value)}:${row[pkCol]}`, { pk: row[pkCol] }); const v = String(value);
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// SQLite 语义应报错),与 MemoryEngine 对齐
if (unique && seen.has(v)) {
throw new DatabaseError(`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${v}"`, 'UNIQUE_VIOLATION');
}
seen.add(v);
idxLsm.put(`${v}:${row[pkCol]}`, { pk: row[pkCol] });
} }
} }
await idxLsm.flush(); await idxLsm.flush();
}
catch (error) {
// 回填失败(唯一冲突):清理半初始化索引(内存 + 存储),标志未落,保持原子语义
this.secondaryIndexes.delete(idxKey);
try {
await idxLsm.clear();
}
catch { /* 清理失败不阻塞 */ }
throw error;
}
colDef.index = true;
if (unique)
colDef.unique = true;
await this.persistSchemas(); await this.persistSchemas();
} }
async dropIndex(tableName, column, _indexName) { async dropIndex(tableName, column, _indexName) {
@@ -7615,6 +7692,10 @@
throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE'); throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE');
this.currentTxnId = this.mvcc.beginTransaction(); this.currentTxnId = this.mvcc.beginTransaction();
this.txnSnapshot = new Map(); this.txnSnapshot = new Map();
// v0.7.3-fix: WAL BEGIN 写失败回滚内存事务状态 —— 此前 append 抛错(full 模式)
// 时 currentTxnId 已设置 → TX_ACTIVE 永久泄漏(后续无法开始新事务)。
// 回滚 mvcc 登记 + 快照后重抛,调用方可重试。
try {
await this.wal.append({ await this.wal.append({
type: WALRecordType.BEGIN, type: WALRecordType.BEGIN,
txnId: this.currentTxnId, txnId: this.currentTxnId,
@@ -7622,6 +7703,13 @@
key: '', key: '',
}); });
} }
catch (error) {
this.mvcc.rollbackTransaction(this.currentTxnId);
this.currentTxnId = null;
this.txnSnapshot = null;
throw error;
}
}
async commitTransaction() { async commitTransaction() {
if (!this.currentTxnId) if (!this.currentTxnId)
throw new DatabaseError('No active transaction', 'TX_NONE'); throw new DatabaseError('No active transaction', 'TX_NONE');
@@ -7652,6 +7740,18 @@
async rollbackTransaction() { async rollbackTransaction() {
if (!this.currentTxnId) if (!this.currentTxnId)
throw new DatabaseError('No active transaction', 'TX_NONE'); throw new DatabaseError('No active transaction', 'TX_NONE');
const txnId = this.currentTxnId;
// v0.7.3-fix: 先持久化 WAL ROLLBACK,再回滚内存 —— 与 commitTransaction 的
// "WAL 领先内存"v0.4.3-fix)对齐。此前内存先回滚、ROLLBACK 记录后写:
// full 模式写失败时崩溃重放无 ROLLBACK 记录 → 已回滚事务的数据复活。
// 现在写失败 → 内存未回滚、事务仍活跃(调用方可重试),崩溃后重放
// 看到 ROLLBACK 记录同样不会复活数据。
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId,
tableName: '',
key: '',
});
// v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留) // v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留)
const affectedTables = new Set(); const affectedTables = new Set();
if (this.txnSnapshot) { if (this.txnSnapshot) {
@@ -7661,14 +7761,8 @@
affectedTables.add(key.slice(0, idx)); affectedTables.add(key.slice(0, idx));
} }
} }
this.mvcc.rollbackTransaction(this.currentTxnId); this.mvcc.rollbackTransaction(txnId);
this.txnSnapshot = null; this.txnSnapshot = null;
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId: this.currentTxnId,
tableName: '',
key: '',
});
this.currentTxnId = null; this.currentTxnId = null;
// v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引 // v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引
for (const tableName of affectedTables) { for (const tableName of affectedTables) {
@@ -8086,10 +8180,25 @@
if (!schema) if (!schema)
return null; return null;
const pkCol = this.tablePKs.get(tableName); const pkCol = this.tablePKs.get(tableName);
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// 跳过 $and/$or/$not 逻辑组合 // `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
if (col === '$and' || col === '$or' || col === '$not') // $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat = [];
const collect = (w) => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v)
collect(sub);
continue; continue;
}
if (k === '$or' || k === '$not' || k === '$exists')
continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
const colDef = schema.columns[col]; const colDef = schema.columns[col];
const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey); const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey);
if (!hasIndex && col !== pkCol) if (!hasIndex && col !== pkCol)
@@ -8167,18 +8276,31 @@
// v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行) // v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行)
if (c.$in.some((v) => v === null)) if (c.$in.some((v) => v === null))
continue; continue;
// v0.7.3-perf: 批级预加载全部值的索引范围 + 主表行(各一次 drainChain)——
// 此前逐值 indexScanToRows:每个值一次 prefetchRange + prefetchKeys
// 后台 compaction 长耗时时 N 倍放大(与 v0.6.1 修的 insert 批量预加载
// 性能悬崖同类)。批级预加载后循环内同步 rangeScan/get。
const values = c.$in.map((v) => String(v));
await idxLsm.prefetchPrefixRanges(values.map((v) => [v, `${v}\uffff`]));
const results = []; const results = [];
const seenPks = new Set(); // v0.4.1: IN 值可能重复,按 pk 去重 const seenPks = new Set(); // v0.4.1: IN 值可能重复,按 pk 去重
for (const val of c.$in) { const pks = [];
const rows = await this.indexScanToRows(tableName, pkCol, idxLsm, String(val), String(val)); for (const val of values) {
for (const row of rows) { const entries = idxLsm.rangeScan(val, `${val}\uffff`);
const pk = String(row[pkCol]); for (const [, idxEntry] of entries) {
if (!seenPks.has(pk)) { const pk = idxEntry.pk;
if (pk && !seenPks.has(pk)) {
seenPks.add(pk); seenPks.add(pk);
results.push(row); pks.push(pk);
} }
} }
} }
await this.lsm.prefetchKeys(pks.map((pk) => `${tableName}:${pk}`));
for (const pk of pks) {
const row = this.lsm.get(`${tableName}:${pk}`);
if (row)
results.push({ ...row, [pkCol]: pk });
}
return results; return results;
} }
// $gt / $gte / $lt / $lte → 范围扫描 // $gt / $gte / $lt / $lte → 范围扫描
@@ -8242,10 +8364,6 @@
this.mvcc.gc(50); this.mvcc.gc(50);
} }
} }
/** 估算 WAL 大小(字节) */
getWALEstimatedSize() {
return this.wal.getBufferedCount() * 200; // 粗略估算每条 ~200B
}
/** /**
* ANALYZE: 收集表统计信息 * ANALYZE: 收集表统计信息
* 返回行数平均行大小索引深度等 * 返回行数平均行大小索引深度等
@@ -8254,15 +8372,28 @@
this.ensureOpen(); this.ensureOpen();
this.ensureTable(tableName); this.ensureTable(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
// v0.7.3: 统计汇总主 LSM + 该表全部二级索引 LSM —— 此前只统计主 LSM,
// 表带多个索引时索引深度/SSTable 数量严重低估
let sstableCount = this.lsm.getStats().sstableCount;
let memtableSize = this.lsm.getStats().memtableSize;
let indexDepth = this.lsm.getStats().levelCounts.filter((c) => c > 0).length;
for (const [idxKey, idxLsm] of this.secondaryIndexes) {
if (!idxKey.startsWith(`${tableName}:idx:`))
continue;
const s = idxLsm.getStats();
sstableCount += s.sstableCount;
memtableSize += s.memtableSize;
indexDepth = Math.max(indexDepth, s.levelCounts.filter((c) => c > 0).length);
}
const stats = { const stats = {
table: tableName, table: tableName,
rowCount: rows.length, rowCount: rows.length,
avgRowSize: rows.length > 0 avgRowSize: rows.length > 0
? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length) ? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length)
: 0, : 0,
indexDepth: this.lsm.getStats().levelCounts.filter((c) => c > 0).length, indexDepth,
sstableCount: this.lsm.getStats().sstableCount, sstableCount,
memtableSize: this.lsm.getStats().memtableSize, memtableSize,
estimatedMemory: this.lsm.getEstimatedMemory(), estimatedMemory: this.lsm.getEstimatedMemory(),
}; };
// 列基数统计 // 列基数统计
@@ -9604,6 +9735,12 @@
if (this.curTokenIs(TokenType.STAR)) { if (this.curTokenIs(TokenType.STAR)) {
columns.push('*'); columns.push('*');
this.nextToken(); this.nextToken();
// v0.7.3: `SELECT *, col [AS alias], ...` —— '*' 后可继续列列表
// (此前 '*' 独占分支,逗号后直接 PARSE_ERRORexecutor 侧投影已支持混合)
while (this.curTokenIs(TokenType.COMMA)) {
this.nextToken();
columns.push(this.parseColumnWithAlias());
}
} }
else { else {
columns.push(...this.parseColumnList()); columns.push(...this.parseColumnList());
@@ -10729,26 +10866,36 @@
catch { /* 非查询语句无 QueryPlan */ } catch { /* 非查询语句无 QueryPlan */ }
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。 // v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。 // 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
// v0.7.3: 递归识别 $and 嵌套等值条件(与 Memory/Aria 的 $and 下推行为对齐;
// $or/$not 不下推,保持 none)。
let usingIndex = plan?.table ? 'none' : 'none'; let usingIndex = plan?.table ? 'none' : 'none';
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) { if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
try { try {
const schema = await this.engine.getTableSchema(plan.table); const schema = await this.engine.getTableSchema(plan.table);
if (schema) { if (schema) {
for (const col of Object.keys(plan.where)) { const findIndex = (w) => {
if (col.startsWith('$')) for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of v) {
const hit = findIndex(sub);
if (hit)
return hit;
}
continue; continue;
const colDef = schema.columns[col]; }
if (k === '$or' || k === '$not')
continue;
const colDef = schema.columns[k];
if (!colDef) if (!colDef)
continue; continue;
if (colDef.primaryKey) { if (colDef.primaryKey)
usingIndex = 'pk'; return 'pk';
break; if (colDef.index || colDef.unique)
} return `index:${k}`;
if (colDef.index || colDef.unique) {
usingIndex = `index:${col}`;
break;
}
} }
return null;
};
usingIndex = findIndex(plan.where) ?? 'none';
} }
} }
catch { /* schema 读取失败保持 none */ } catch { /* schema 读取失败保持 none */ }
@@ -10879,7 +11026,11 @@
} }
if (stmt.orderBy && stmt.orderBy.length > 0) if (stmt.orderBy && stmt.orderBy.length > 0)
rows = applyOrderBy(rows, stmt.orderBy); rows = applyOrderBy(rows, stmt.orderBy);
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0 && stmt.columns[0] !== '*') { // v0.7.3: `SELECT *, col AS alias` —— 此前 columns[0]==='*' 直接不投影,
// 别名列/常量列丢失。仅当 '*' 是唯一列时跳过投影(projectRow 对裸 '*'
// 合并原行全部列,其余表达式覆盖/追加)
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0
&& !(stmt.columns.length === 1 && stmt.columns[0] === '*')) {
rows = rows.map((row) => this.projectRow(row, stmt.columns)); rows = rows.map((row) => this.projectRow(row, stmt.columns));
} }
// v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序 // v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序
@@ -11449,9 +11600,13 @@
const aliasCols = []; const aliasCols = [];
const caseCols = []; const caseCols = [];
const constCols = []; const constCols = [];
// v0.7.3: 裸 '*' 与列表达式混合(SELECT *, name AS nick)→ 原行全部列为基
let hasStar = false;
for (const col of columns) { for (const col of columns) {
if (col === '*') if (col === '*') {
hasStar = true;
continue; continue;
}
const expr = parseCaseExpression(col); const expr = parseCaseExpression(col);
if (expr) { if (expr) {
caseCols.push({ alias: expr.alias ?? col, expr }); caseCols.push({ alias: expr.alias ?? col, expr });
@@ -11465,20 +11620,26 @@
// v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出 // v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出
const lit = col.match(/^'(.*)'$/s); const lit = col.match(/^'(.*)'$/s);
if (lit) { if (lit) {
const value = lit[1].replace(/\\'/g, "'"); // v0.7.3: SQL 标准 '' 转义还原(readString 已把 '' 合并为单个 '
// 打包回列的文本中相邻两个 ' 即一个引号字面量)
const value = lit[1].replace(/''/g, "'");
constCols.push({ key: col, value }); constCols.push({ key: col, value });
continue; continue;
} }
plain.push(col); plain.push(col);
} }
const projected = plain.length > 0 ? projectColumns(row, plain) : {}; // v0.7.3: hasStar 时以原行全部列为基(projectColumns 仅投影 plain 列,不含 * 的其余列)
const projected = hasStar
? { ...row }
: (plain.length > 0 ? projectColumns(row, plain) : {});
for (const { alias, source } of aliasCols) { for (const { alias, source } of aliasCols) {
if (source === '*') { if (source === '*') {
Object.assign(projected, row); Object.assign(projected, row);
} }
else { else {
const lit = source.match(/^'(.*)'$/s); const lit = source.match(/^'(.*)'$/s);
projected[alias] = lit ? lit[1].replace(/\\'/g, "'") : row[source]; // v0.7.3: 同 constCols —— SQL 标准 '' 转义还原
projected[alias] = lit ? lit[1].replace(/''/g, "'") : row[source];
} }
} }
for (const { key, value } of constCols) { for (const { key, value } of constCols) {
@@ -12350,9 +12511,41 @@
const select = stmt; const select = stmt;
// 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY // 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY
const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c)); const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c));
// v0.7.3: WHERE 含子查询($subquery / $exists / 嵌套 $col 列引用)不可流式 ——
// 引擎层 matchWhere 的 $in/$nin 遇未解析的 $subquery 对象返回 false → 所有行
// 被静默过滤(空结果);$col 操作符无对应匹配分支会抛 QUERY_ERROR。
// 递归检测后回退物化路径(resolveSubqueries 正确解析)。
const hasSubquery = (where) => {
if (!where)
return false;
for (const [k, v] of Object.entries(where)) {
if (k === '$and' || k === '$or') {
if (v.some((sub) => hasSubquery(sub)))
return true;
continue;
}
if (k === '$not') {
if (hasSubquery(v))
return true;
continue;
}
if (k === '$exists')
return true;
if (typeof v === 'object' && v !== null) {
for (const [, operand] of Object.entries(v)) {
if (typeof operand === 'object' && operand !== null) {
const ops = operand;
if ('$subquery' in ops || '$col' in ops)
return true;
}
}
}
}
return false;
};
const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct
&& !aggregate && !(select.orderBy && select.orderBy.length > 0) && !aggregate && !(select.orderBy && select.orderBy.length > 0)
&& !(select.where && select.where['$exists'] !== undefined); && !hasSubquery(select.where);
if (streamable && typeof this.engine.findStream === 'function') { if (streamable && typeof this.engine.findStream === 'function') {
// 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化 // 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化
const isAsync = onRow.constructor?.name === 'AsyncFunction'; const isAsync = onRow.constructor?.name === 'AsyncFunction';
@@ -12509,9 +12702,20 @@
async triggerStatementHooks(stmt, phase, result) { async triggerStatementHooks(stmt, phase, result) {
switch (stmt.type) { switch (stmt.type) {
case 'INSERT': { case 'INSERT': {
// v0.7.3: 列映射对齐 executor —— 省略列名时按 schema 列顺序映射
// (此前用数字键 String(i),与 executor 写入的真实行键不一致)
let cols = stmt.columns ?? [];
if (cols.length === 0) {
try {
const schema = await this.engine.getTableSchema(stmt.into);
cols = schema ? Object.keys(schema.columns) : [];
}
catch {
cols = [];
}
}
const rows = (stmt.values ?? []).map((vals) => { const rows = (stmt.values ?? []).map((vals) => {
const row = {}; const row = {};
const cols = stmt.columns ?? [];
for (let i = 0; i < vals.length; i++) { for (let i = 0; i < vals.length; i++) {
row[cols[i] ?? String(i)] = vals[i]; row[cols[i] ?? String(i)] = vals[i];
} }
+1 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@metona-team/metona-sqlark", "name": "@metona-team/metona-sqlark",
"version": "0.7.2", "version": "0.7.3",
"description": "Frontend SQL database with in-memory and disk dual-mode storage", "description": "Frontend SQL database with in-memory and disk dual-mode storage",
"type": "module", "type": "module",
"main": "dist/metona-sqlark.cjs", "main": "dist/metona-sqlark.cjs",
+1 -1
View File
@@ -214,4 +214,4 @@ export class DatabaseError extends Error {
// 版本 // 版本
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
export const VERSION = '0.7.2'; export const VERSION = '0.7.3';
+39 -2
View File
@@ -270,9 +270,36 @@ export class MetonaSqlark {
// 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY // 不可流式场景:JOIN / GROUP BY / HAVING / DISTINCT / 聚合 / UNION / 关联子查询 / ORDER BY
const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c)); const aggregate = select.columns.some((c) => /^(COUNT|SUM|AVG|MIN|MAX)\(/i.test(c));
// v0.7.3: WHERE 含子查询($subquery / $exists / 嵌套 $col 列引用)不可流式 ——
// 引擎层 matchWhere 的 $in/$nin 遇未解析的 $subquery 对象返回 false → 所有行
// 被静默过滤(空结果);$col 操作符无对应匹配分支会抛 QUERY_ERROR。
// 递归检测后回退物化路径(resolveSubqueries 正确解析)。
const hasSubquery = (where: import('./constants').WhereCondition | undefined): boolean => {
if (!where) return false;
for (const [k, v] of Object.entries(where)) {
if (k === '$and' || k === '$or') {
if ((v as import('./constants').WhereCondition[]).some((sub) => hasSubquery(sub))) return true;
continue;
}
if (k === '$not') {
if (hasSubquery(v as import('./constants').WhereCondition)) return true;
continue;
}
if (k === '$exists') return true;
if (typeof v === 'object' && v !== null) {
for (const [, operand] of Object.entries(v as Record<string, unknown>)) {
if (typeof operand === 'object' && operand !== null) {
const ops = operand as Record<string, unknown>;
if ('$subquery' in ops || '$col' in ops) return true;
}
}
}
}
return false;
};
const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct const streamable = !select.joins && !select.groupBy && !select.having && !select.distinct
&& !aggregate && !(select.orderBy && select.orderBy.length > 0) && !aggregate && !(select.orderBy && select.orderBy.length > 0)
&& !(select.where && select.where['$exists'] !== undefined); && !hasSubquery(select.where);
if (streamable && typeof this.engine.findStream === 'function') { if (streamable && typeof this.engine.findStream === 'function') {
// 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化 // 用户回调为 async(返回 Promise)时引擎同步扫描无法 await → 回退物化
@@ -441,9 +468,19 @@ export class MetonaSqlark {
private async triggerStatementHooks(stmt: Statement, phase: 'before' | 'after', result?: unknown): Promise<void> { private async triggerStatementHooks(stmt: Statement, phase: 'before' | 'after', result?: unknown): Promise<void> {
switch (stmt.type) { switch (stmt.type) {
case 'INSERT': { case 'INSERT': {
// v0.7.3: 列映射对齐 executor —— 省略列名时按 schema 列顺序映射
// (此前用数字键 String(i),与 executor 写入的真实行键不一致)
let cols: string[] = stmt.columns ?? [];
if (cols.length === 0) {
try {
const schema = await this.engine.getTableSchema(stmt.into);
cols = schema ? Object.keys(schema.columns) : [];
} catch {
cols = [];
}
}
const rows: Record<string, unknown>[] = (stmt.values ?? []).map((vals: unknown[]) => { const rows: Record<string, unknown>[] = (stmt.values ?? []).map((vals: unknown[]) => {
const row: Record<string, unknown> = {}; const row: Record<string, unknown> = {};
const cols = stmt.columns ?? [];
for (let i = 0; i < vals.length; i++) { for (let i = 0; i < vals.length; i++) {
row[cols[i] ?? String(i)] = vals[i]; row[cols[i] ?? String(i)] = vals[i];
} }
+124 -58
View File
@@ -6,7 +6,7 @@
*/ */
import type { IStorageEngine } from '../interface'; import type { IStorageEngine } from '../interface';
import type { QueryPlan, TableSchema, ColumnDef } from '../../constants'; import type { QueryPlan, TableSchema, ColumnDef, WhereCondition } from '../../constants';
import { DatabaseError } from '../../constants'; import { DatabaseError } from '../../constants';
import { matchWhere, applyOrderBy, projectColumns } from '../../query/where-matcher'; import { matchWhere, applyOrderBy, projectColumns } from '../../query/where-matcher';
import { checkFieldType, stripUndefinedUpdates } from '../../table/schema'; import { checkFieldType, stripUndefinedUpdates } from '../../table/schema';
@@ -557,6 +557,28 @@ export class AriaEngine implements IStorageEngine {
} }
await this.lsm.prefetchKeys(validatedRows.map((v) => v.key)); await this.lsm.prefetchKeys(validatedRows.map((v) => v.key));
// v0.7.3: 主键批内互查 + 预检 —— 此前 PK 重复检查在写入循环内:
// 第 N 行重复抛错时,前 N-1 行已 put LSM 且其 WAL 记录随 appendBatch 一起
// 丢失 → 语句级部分提交 + 内存/WAL 不一致(与 v0.6.2 的 unique 预检同一阶段)。
const pkSet = new Set<string>();
for (const { pkValue, key } of validatedRows) {
if (pkSet.has(pkValue)) {
throw new DatabaseError(
`Duplicate primary key "${pkValue}" in table "${tableName}"`,
'DUPLICATE_KEY',
);
}
pkSet.add(pkValue);
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !(existing as unknown as Record<string, unknown>).__txn_deleted) {
throw new DatabaseError(
`Duplicate primary key "${pkValue}" in table "${tableName}"`,
'DUPLICATE_KEY',
);
}
}
// v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain // v0.6.2: 唯一约束 — 批量预加载本批唯一列涉及的索引范围(一次 drainChain
for (const colName of uniqueCols) { for (const colName of uniqueCols) {
const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`)!; const idxLsm = this.secondaryIndexes.get(`${tableName}:idx:${colName}`)!;
@@ -595,16 +617,7 @@ export class AriaEngine implements IStorageEngine {
} }
for (const { row: validated, pkValue, key } of validatedRows) { for (const { row: validated, pkValue, key } of validatedRows) {
// Check duplicate in LSM + transaction snapshot // PK 重复已在批预检阶段检查(v0.7.3),此处不再重复查询
const existing = this.currentTxnId
? (this.txnSnapshot?.get(key) ?? this.lsm.get(key))
: this.lsm.get(key);
if (existing && !(existing as unknown as Record<string, unknown>).__txn_deleted) {
throw new DatabaseError(
`Duplicate primary key "${pkValue}" in table "${tableName}"`,
'DUPLICATE_KEY',
);
}
if (this.currentTxnId && this.txnSnapshot) { if (this.currentTxnId && this.txnSnapshot) {
// Within transaction: buffer to snapshot + MVCC version chain // Within transaction: buffer to snapshot + MVCC version chain
@@ -889,25 +902,9 @@ export class AriaEngine implements IStorageEngine {
if (visited.has(visitKey)) return; if (visited.has(visitKey)) return;
visited.add(visitKey); visited.add(visitKey);
// 阶段 1: RESTRICT 检查 // v0.7.3-perf: 删除冗余的阶段 1 RESTRICT 扫描 —— checkForeignKeyUpdateRestrict
for (const [refTableName, refSchema] of this.schemas) { // 已在两阶段 update 预检(阶段 1b)覆盖 RESTRICT 与 SET NULL+required
if (refTableName === tableName) continue; // 此处任何修改前重复全表扫描纯属浪费。直接执行 CASCADE / SET NULL。
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate) continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName) continue;
if (colDef.onUpdate !== 'RESTRICT') continue;
const refRows = await this.getAllRows(refTableName);
if (refRows.some((r) => String(r[colName]) === oldPk)) {
throw new DatabaseError(
`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`,
'FOREIGN_KEY_VIOLATION',
);
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) continue; if (refTableName === tableName) continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) { for (const [colName, colDef] of Object.entries(refSchema.columns)) {
@@ -1312,8 +1309,6 @@ export class AriaEngine implements IStorageEngine {
// v0.4.2-fix: 以索引 LSM 是否已建为准(schema 标记可能因重启恢复而存在, // v0.4.2-fix: 以索引 LSM 是否已建为准(schema 标记可能因重启恢复而存在,
// 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失) // 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失)
if (this.secondaryIndexes.has(idxKey)) return; if (this.secondaryIndexes.has(idxKey)) return;
colDef.index = true;
if (unique) colDef.unique = true;
const idxLsm = new LSM({ const idxLsm = new LSM({
memtableSizeThreshold: this.config.memtableSizeThreshold, memtableSizeThreshold: this.config.memtableSizeThreshold,
@@ -1326,16 +1321,36 @@ export class AriaEngine implements IStorageEngine {
await idxLsm.init(); await idxLsm.init();
this.secondaryIndexes.set(idxKey, idxLsm); this.secondaryIndexes.set(idxKey, idxLsm);
try {
// 从主 LSM 重建索引数据 // 从主 LSM 重建索引数据
const pkCol = this.tablePKs.get(tableName)!; const pkCol = this.tablePKs.get(tableName)!;
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
const seen = new Set<string>();
for (const row of rows) { for (const row of rows) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
idxLsm.put(`${String(value)}:${row[pkCol]}`, { pk: row[pkCol] }); const v = String(value);
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// SQLite 语义应报错),与 MemoryEngine 对齐
if (unique && seen.has(v)) {
throw new DatabaseError(
`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${v}"`,
'UNIQUE_VIOLATION',
);
}
seen.add(v);
idxLsm.put(`${v}:${row[pkCol]}`, { pk: row[pkCol] });
} }
} }
await idxLsm.flush(); await idxLsm.flush();
} catch (error) {
// 回填失败(唯一冲突):清理半初始化索引(内存 + 存储),标志未落,保持原子语义
this.secondaryIndexes.delete(idxKey);
try { await idxLsm.clear(); } catch { /* 清理失败不阻塞 */ }
throw error;
}
colDef.index = true;
if (unique) colDef.unique = true;
await this.persistSchemas(); await this.persistSchemas();
} }
@@ -1375,12 +1390,22 @@ export class AriaEngine implements IStorageEngine {
this.currentTxnId = this.mvcc.beginTransaction(); this.currentTxnId = this.mvcc.beginTransaction();
this.txnSnapshot = new Map(); this.txnSnapshot = new Map();
// v0.7.3-fix: WAL BEGIN 写失败回滚内存事务状态 —— 此前 append 抛错(full 模式)
// 时 currentTxnId 已设置 → TX_ACTIVE 永久泄漏(后续无法开始新事务)。
// 回滚 mvcc 登记 + 快照后重抛,调用方可重试。
try {
await this.wal.append({ await this.wal.append({
type: WALRecordType.BEGIN, type: WALRecordType.BEGIN,
txnId: this.currentTxnId, txnId: this.currentTxnId,
tableName: '', tableName: '',
key: '', key: '',
}); });
} catch (error) {
this.mvcc.rollbackTransaction(this.currentTxnId);
this.currentTxnId = null;
this.txnSnapshot = null;
throw error;
}
} }
async commitTransaction(): Promise<void> { async commitTransaction(): Promise<void> {
@@ -1416,6 +1441,20 @@ export class AriaEngine implements IStorageEngine {
async rollbackTransaction(): Promise<void> { async rollbackTransaction(): Promise<void> {
if (!this.currentTxnId) throw new DatabaseError('No active transaction', 'TX_NONE'); if (!this.currentTxnId) throw new DatabaseError('No active transaction', 'TX_NONE');
const txnId = this.currentTxnId;
// v0.7.3-fix: 先持久化 WAL ROLLBACK,再回滚内存 —— 与 commitTransaction 的
// "WAL 领先内存"v0.4.3-fix)对齐。此前内存先回滚、ROLLBACK 记录后写:
// full 模式写失败时崩溃重放无 ROLLBACK 记录 → 已回滚事务的数据复活。
// 现在写失败 → 内存未回滚、事务仍活跃(调用方可重试),崩溃后重放
// 看到 ROLLBACK 记录同样不会复活数据。
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId,
tableName: '',
key: '',
});
// v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留) // v0.3.3: 记录事务涉及的表(用于回滚后重建索引,消除索引残留)
const affectedTables = new Set<string>(); const affectedTables = new Set<string>();
if (this.txnSnapshot) { if (this.txnSnapshot) {
@@ -1425,16 +1464,9 @@ export class AriaEngine implements IStorageEngine {
} }
} }
this.mvcc.rollbackTransaction(this.currentTxnId); this.mvcc.rollbackTransaction(txnId);
this.txnSnapshot = null; this.txnSnapshot = null;
await this.wal.append({
type: WALRecordType.ROLLBACK,
txnId: this.currentTxnId,
tableName: '',
key: '',
});
this.currentTxnId = null; this.currentTxnId = null;
// v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引 // v0.3.3: 事务内直接写入了二级索引 LSM,回滚后全量重建受影响表的索引
@@ -1870,9 +1902,24 @@ export class AriaEngine implements IStorageEngine {
if (!schema) return null; if (!schema) return null;
const pkCol = this.tablePKs.get(tableName)!; const pkCol = this.tablePKs.get(tableName)!;
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// 跳过 $and/$or/$not 逻辑组合 // `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
if (col === '$and' || col === '$or' || col === '$not') continue; // $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat: [string, unknown][] = [];
const collect = (w: WhereCondition): void => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of (v as WhereCondition[])) collect(sub);
continue;
}
if (k === '$or' || k === '$not' || k === '$exists') continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
const colDef = schema.columns[col]; const colDef = schema.columns[col];
const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey); const hasIndex = colDef && (colDef.index || colDef.unique || colDef.primaryKey);
@@ -1944,18 +1991,30 @@ export class AriaEngine implements IStorageEngine {
if ('$in' in c && Array.isArray(c.$in)) { if ('$in' in c && Array.isArray(c.$in)) {
// v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行) // v0.6.2-fix(P1): IN 列表含 null 不走索引(索引不含 null 条目,会漏匹配 null 行)
if (c.$in.some((v) => v === null)) continue; if (c.$in.some((v) => v === null)) continue;
// v0.7.3-perf: 批级预加载全部值的索引范围 + 主表行(各一次 drainChain)——
// 此前逐值 indexScanToRows:每个值一次 prefetchRange + prefetchKeys
// 后台 compaction 长耗时时 N 倍放大(与 v0.6.1 修的 insert 批量预加载
// 性能悬崖同类)。批级预加载后循环内同步 rangeScan/get。
const values = c.$in.map((v) => String(v));
await idxLsm.prefetchPrefixRanges(values.map((v): [string, string] => [v, `${v}\uffff`]));
const results: Record<string, unknown>[] = []; const results: Record<string, unknown>[] = [];
const seenPks = new Set<string>(); // v0.4.1: IN 值可能重复,按 pk 去重 const seenPks = new Set<string>(); // v0.4.1: IN 值可能重复,按 pk 去重
for (const val of c.$in) { const pks: string[] = [];
const rows = await this.indexScanToRows(tableName, pkCol, idxLsm, String(val), String(val)); for (const val of values) {
for (const row of rows) { const entries = idxLsm.rangeScan(val, `${val}\uffff`);
const pk = String(row[pkCol]); for (const [, idxEntry] of entries) {
if (!seenPks.has(pk)) { const pk = (idxEntry as { pk?: string }).pk;
if (pk && !seenPks.has(pk)) {
seenPks.add(pk); seenPks.add(pk);
results.push(row); pks.push(pk);
} }
} }
} }
await this.lsm.prefetchKeys(pks.map((pk) => `${tableName}:${pk}`));
for (const pk of pks) {
const row = this.lsm.get(`${tableName}:${pk}`);
if (row) results.push({ ...row, [pkCol]: pk });
}
return results; return results;
} }
// $gt / $gte / $lt / $lte → 范围扫描 // $gt / $gte / $lt / $lte → 范围扫描
@@ -2027,11 +2086,6 @@ export class AriaEngine implements IStorageEngine {
} }
} }
/** 估算 WAL 大小(字节) */
getWALEstimatedSize(): number {
return this.wal.getBufferedCount() * 200; // 粗略估算每条 ~200B
}
/** /**
* ANALYZE: 收集表统计信息 * ANALYZE: 收集表统计信息
* *
@@ -2040,15 +2094,27 @@ export class AriaEngine implements IStorageEngine {
this.ensureOpen(); this.ensureOpen();
this.ensureTable(tableName); this.ensureTable(tableName);
const rows = await this.getAllRows(tableName); const rows = await this.getAllRows(tableName);
// v0.7.3: 统计汇总主 LSM + 该表全部二级索引 LSM —— 此前只统计主 LSM,
// 表带多个索引时索引深度/SSTable 数量严重低估
let sstableCount = this.lsm.getStats().sstableCount;
let memtableSize = this.lsm.getStats().memtableSize;
let indexDepth = this.lsm.getStats().levelCounts.filter((c: number) => c > 0).length;
for (const [idxKey, idxLsm] of this.secondaryIndexes) {
if (!idxKey.startsWith(`${tableName}:idx:`)) continue;
const s = idxLsm.getStats();
sstableCount += s.sstableCount;
memtableSize += s.memtableSize;
indexDepth = Math.max(indexDepth, s.levelCounts.filter((c: number) => c > 0).length);
}
const stats: Record<string, unknown> = { const stats: Record<string, unknown> = {
table: tableName, table: tableName,
rowCount: rows.length, rowCount: rows.length,
avgRowSize: rows.length > 0 avgRowSize: rows.length > 0
? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length) ? Math.round(rows.reduce((s, r) => s + JSON.stringify(r).length, 0) / rows.length)
: 0, : 0,
indexDepth: this.lsm.getStats().levelCounts.filter((c: number) => c > 0).length, indexDepth,
sstableCount: this.lsm.getStats().sstableCount, sstableCount,
memtableSize: this.lsm.getStats().memtableSize, memtableSize,
estimatedMemory: this.lsm.getEstimatedMemory(), estimatedMemory: this.lsm.getEstimatedMemory(),
}; };
+2 -1
View File
@@ -290,7 +290,8 @@ export class WAL {
const computedNew = crc32(recordBytes); const computedNew = crc32(recordBytes);
const computedLegacy = this.legacyChecksum(recordBytes); const computedLegacy = this.legacyChecksum(recordBytes);
if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) { if ((computedNew >>> 0) !== storedCrc && (computedLegacy >>> 0) !== storedCrc) {
// CRC 不匹配,跳过此损坏记录 // CRC 不匹配,跳过此损坏记录(长度字段链完整时后续好记录仍可恢复,
// 行为由 aria-wal-crc 测试锁定)
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`); console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`);
continue; continue;
+6 -4
View File
@@ -278,11 +278,13 @@ export class KVStoreEngine implements IStorageEngine {
// 增量持久化(原子 putMany // 增量持久化(原子 putMany
const schema = await this.memory.getTableSchema(tableName); const schema = await this.memory.getTableSchema(tableName);
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND'); if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema); // v0.7.3: 持久化内存中的 validated 行(含 default 值/类型归一/列投影)——
// 此前写原始入参 rowdefault 不落盘、schema 外列被持久化,重启后行不一致
const puts: Record<string, ArrayBuffer> = {}; const puts: Record<string, ArrayBuffer> = {};
rows.forEach((row, i) => { for (const pk of pks) {
puts[this.rowKey(tableName, String(pks[i] ?? row[pkCol]))] = enc(JSON.stringify(row)); const row = this.memory.getRow(tableName, pk);
}); if (row) puts[this.rowKey(tableName, pk)] = enc(JSON.stringify(row));
}
await this.kv.putMany(puts); await this.kv.putMany(puts);
return pks; return pks;
} }
+128 -53
View File
@@ -4,7 +4,7 @@
*/ */
import type { IStorageEngine } from './interface'; import type { IStorageEngine } from './interface';
import type { QueryPlan, TableSchema } from '../constants'; import type { QueryPlan, TableSchema, WhereCondition } from '../constants';
import { DatabaseError } from '../constants'; import { DatabaseError } from '../constants';
import { matchWhere, applyOrderBy, projectColumns } from '../query/where-matcher'; import { matchWhere, applyOrderBy, projectColumns } from '../query/where-matcher';
import { stripUndefinedUpdates } from '../table/schema'; import { stripUndefinedUpdates } from '../table/schema';
@@ -118,6 +118,11 @@ export class MemoryEngine implements IStorageEngine {
if (!schema.columns[column.name]) { if (!schema.columns[column.name]) {
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
} }
// v0.7.3: 被删列是索引列 → 同步清理索引 Map —— 此前残留旧索引:
// 查询已删列仍走旧索引(不含新行)→ 结果不完整(对齐 AriaEngine cleanupTableIndexes
if (schema.columns[column.name].index || schema.columns[column.name].unique) {
this.indexes.get(tableName)?.delete(column.name);
}
delete schema.columns[column.name]; delete schema.columns[column.name];
// 清理已有行中该列的值(find 返回行引用,直接删除生效) // 清理已有行中该列的值(find 返回行引用,直接删除生效)
const table = this.tables.get(tableName)!; const table = this.tables.get(tableName)!;
@@ -134,11 +139,31 @@ export class MemoryEngine implements IStorageEngine {
const pkColumn = this.getPrimaryKey(schema); const pkColumn = this.getPrimaryKey(schema);
const pks: string[] = []; const pks: string[] = [];
// v0.7.3: 语句级原子性 —— 两阶段(先全量预检,后执行)。
// 此前逐行"校验+写入":第 N 行主键重复/唯一冲突抛错时,前 N-1 行已提交
// (无事务下语句级部分提交,与 v0.7.2 修复的 UPDATE 同类问题)。
const validated: Record<string, unknown>[] = [];
const pkSet = new Set<string>();
const batchUnique: Map<string, Set<unknown>> = new Map();
// 阶段 1:全量预检(任何一行失败 → 整条语句不执行)
for (const row of rows) { for (const row of rows) {
const validatedRow = this.validateRow(schema, row); const validatedRow = this.validateRow(schema, row);
const pkValue = String(validatedRow[pkColumn]); const pkValue = String(validatedRow[pkColumn]);
if (table.has(pkValue)) throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY'); // 批内主键互查(内存表尚未反映本批写入)
this.checkUniqueness(schema, validatedRow); if (table.has(pkValue) || pkSet.has(pkValue)) {
throw new DatabaseError(`Duplicate primary key "${pkValue}" in table "${tableName}"`, 'DUPLICATE_KEY');
}
pkSet.add(pkValue);
// v0.7.3: 批内唯一互查 + 索引查(此前两行同批写入同一唯一值时,
// 第一行已写入索引 → 第二行 checkUniqueness 抛错 → 第一行残留)
this.checkInsertUniqueness(schema, tableName, validatedRow, batchUnique);
validated.push(validatedRow);
}
// 阶段 2:执行(预检已通过,此阶段不再抛校验类错误)
for (const validatedRow of validated) {
const pkValue = String(validatedRow[pkColumn]);
table.set(pkValue, validatedRow); table.set(pkValue, validatedRow);
this.updateIndexes(tableName, validatedRow, pkValue); this.updateIndexes(tableName, validatedRow, pkValue);
pks.push(pkValue); pks.push(pkValue);
@@ -146,6 +171,13 @@ export class MemoryEngine implements IStorageEngine {
return pks; return pks;
} }
/** v0.7.3: 按主键取已验证行(KVStoreEngine 持久化 validated 行用,含 default/类型归一) */
getRow(tableName: string, pkValue: string): Record<string, unknown> | null {
const table = this.tables.get(tableName);
if (!table) return null;
return table.get(pkValue) ?? null;
}
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> { async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
this.ensureTable(tableName); this.ensureTable(tableName);
const table = this.tables.get(tableName)!; const table = this.tables.get(tableName)!;
@@ -242,7 +274,46 @@ export class MemoryEngine implements IStorageEngine {
} }
/** /**
* v0.7.2: 更新唯一性预检 + * v0.7.3: 插入唯一性预检
* + update checkUpdateUniqueness
* insert
*/
private checkInsertUniqueness(
schema: TableSchema,
tableName: string,
row: Record<string, unknown>,
batchUnique: Map<string, Set<unknown>>,
): void {
const tableIndexes = this.indexes.get(tableName);
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique) continue;
const value = row[colName];
if (value === undefined || value === null) continue;
let seen = batchUnique.get(colName);
if (!seen) {
seen = new Set<unknown>();
batchUnique.set(colName, seen);
}
if (seen.has(value)) {
throw new DatabaseError(
`Unique constraint violation on column "${colName}" in table "${schema.name}"`,
'UNIQUE_VIOLATION',
);
}
seen.add(value);
if (!tableIndexes) continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(value)) {
throw new DatabaseError(
`Unique constraint violation on column "${colName}" in table "${schema.name}"`,
'UNIQUE_VIOLATION',
);
}
}
}
/**
* v0.7.3: 更新唯一性预检 +
* 1 " * 1 "
* " * "
*/ */
@@ -324,29 +395,11 @@ export class MemoryEngine implements IStorageEngine {
/** /**
* v0.4.2-fix: ON UPDATE * v0.4.2-fix: ON UPDATE
* RESTRICT / CASCADE FK / SET NULL * RESTRICT / CASCADE FK / SET NULL
* RESTRICT * v0.7.3-perf: 删除冗余的阶段 1 RESTRICT checkUpdateRestrict
* update 1b RESTRICT SET NULL+required
* CASCADE / SET NULL
*/ */
private async applyUpdateCascade(tableName: string, oldPk: string, newPk: string): Promise<void> { private async applyUpdateCascade(tableName: string, oldPk: string, newPk: string): Promise<void> {
// 阶段 1: RESTRICT 检查(引用旧主键的行存在即拒绝)
for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
if (!colDef.references || !colDef.onUpdate) continue;
const [refTable] = colDef.references.split('.');
if (refTable !== tableName) continue;
const refTableData = this.tables.get(refTableName);
if (!refTableData) continue;
for (const [, refRow] of refTableData) {
if (String(refRow[colName]) === oldPk && colDef.onUpdate === 'RESTRICT') {
throw new DatabaseError(
`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`,
'FOREIGN_KEY_VIOLATION',
);
}
}
}
}
// 阶段 2: CASCADE / SET NULL
for (const [refTableName, refSchema] of this.schemas) { for (const [refTableName, refSchema] of this.schemas) {
if (refTableName === tableName) continue; if (refTableName === tableName) continue;
for (const [colName, colDef] of Object.entries(refSchema.columns)) { for (const [colName, colDef] of Object.entries(refSchema.columns)) {
@@ -369,29 +422,30 @@ export class MemoryEngine implements IStorageEngine {
async delete(tableName: string, query: QueryPlan): Promise<number> { async delete(tableName: string, query: QueryPlan): Promise<number> {
this.ensureTable(tableName); this.ensureTable(tableName);
const table = this.tables.get(tableName)!; const table = this.tables.get(tableName)!;
const toDelete: string[] = []; const toDelete: { pk: string; row: Record<string, unknown> }[] = [];
for (const [pk, row] of table) { for (const [pk, row] of table) {
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) { if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
// v0.3.3: 删除行前清理其索引条目(修复删除后索引残留) toDelete.push({ pk, row });
this.removeIndexEntries(tableName, row, pk);
toDelete.push(pk);
} }
} }
// v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归), // v0.6.3-fix: 级联两阶段 —— 先对全部待删行做 RESTRICT 预检(沿 CASCADE 链递归),
// 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的 // 任何一行违规则整体拒绝。此前逐行执行:第 N 行 RESTRICT 抛错时,前 N-1 行的
// 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致) // 级联子行已被删除、父行未删 → 无事务下部分级联(数据不一致)
//
// v0.7.3-fix: 索引清理移到预检之后 —— 此前 removeIndexEntries 在收集阶段执行,
// RESTRICT 预检抛错时行未删但索引条目已删 → 唯一约束失效、索引查询丢行
const restrictVisited = new Set<string>(); const restrictVisited = new Set<string>();
for (const pk of toDelete) { for (const { pk } of toDelete) {
const row = table.get(pk); this.checkCascadeRestrict(tableName, pk, restrictVisited);
if (row) this.checkCascadeRestrict(tableName, pk, restrictVisited);
} }
// 级联删除:检查引用此表的其他表(RESTRICT 已预检通过,此阶段不再抛 // 预检通过:清理索引 + 级联删除(此阶段不再抛校验类错误
let cascadeCount = 0; let cascadeCount = 0;
for (const pk of toDelete) { for (const { pk, row } of toDelete) {
const row = table.get(pk); // v0.3.3: 删除行前清理其索引条目(修复删除后索引残留)
if (row) cascadeCount += await this.cascadeDelete(tableName, pk, row); this.removeIndexEntries(tableName, row, pk);
cascadeCount += await this.cascadeDelete(tableName, pk, row);
} }
for (const pk of toDelete) table.delete(pk); for (const { pk } of toDelete) table.delete(pk);
return toDelete.length + cascadeCount; return toDelete.length + cascadeCount;
} }
@@ -470,20 +524,34 @@ export class MemoryEngine implements IStorageEngine {
const colDef = schema.columns[column]; const colDef = schema.columns[column];
if (!colDef) throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND'); if (!colDef) throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
if (colDef.index || colDef.unique) return; // 已存在 if (colDef.index || colDef.unique) return; // 已存在
colDef.index = true;
if (unique) colDef.unique = true;
const tableIndexes = this.indexes.get(tableName)!; const tableIndexes = this.indexes.get(tableName)!;
if (!tableIndexes.has(column)) tableIndexes.set(column, new Map()); if (!tableIndexes.has(column)) tableIndexes.set(column, new Map());
const colIndex = tableIndexes.get(column)!; const colIndex = tableIndexes.get(column)!;
const table = this.tables.get(tableName)!; const table = this.tables.get(tableName)!;
try {
for (const [pk, row] of table) { for (const [pk, row] of table) {
const value = row[column]; const value = row[column];
if (value !== undefined && value !== null) { if (value !== undefined && value !== null) {
// v0.7.3: UNIQUE 索引回填校验存量唯一性 —— 此前重复数据静默建索引
// (SQLite 语义应报错),且此后该列唯一约束永远无法满足
if (unique && colIndex.has(value)) {
throw new DatabaseError(
`Unique index on column "${column}" in table "${tableName}" cannot be created: duplicate value "${String(value)}"`,
'UNIQUE_VIOLATION',
);
}
if (!colIndex.has(value)) colIndex.set(value, new Set()); if (!colIndex.has(value)) colIndex.set(value, new Set());
colIndex.get(value)!.add(pk); colIndex.get(value)!.add(pk);
} }
} }
} catch (error) {
// 回填失败(唯一冲突):清理半初始化索引,标志未落,保持原子语义
tableIndexes.delete(column);
throw error;
}
colDef.index = true;
if (unique) colDef.unique = true;
} }
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> { async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
@@ -593,26 +661,29 @@ export class MemoryEngine implements IStorageEngine {
} }
} }
/** O(1) 唯一性检查:利用哈希索引 */
private checkUniqueness(schema: TableSchema, row: Record<string, unknown>): void {
const tableIndexes = this.indexes.get(schema.name);
if (!tableIndexes) return;
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (!colDef.unique || row[colName] === undefined || row[colName] === null) continue;
const colIndex = tableIndexes.get(colName);
if (colIndex && colIndex.has(row[colName])) {
throw new DatabaseError(`Unique constraint violation on column "${colName}" in table "${schema.name}"`, 'UNIQUE_VIOLATION');
}
}
}
/** 索引查找 */ /** 索引查找 */
private tryIndexLookup( private tryIndexLookup(
tableName: string, table: Map<string, Record<string, unknown>>, query: QueryPlan, tableName: string, table: Map<string, Record<string, unknown>>, query: QueryPlan,
): Record<string, unknown>[] { ): Record<string, unknown>[] {
const tableIndexes = this.indexes.get(tableName); const tableIndexes = this.indexes.get(tableName);
if (!tableIndexes || !query.where) return Array.from(table.values()); if (!tableIndexes || !query.where) return Array.from(table.values());
for (const [col, condition] of Object.entries(query.where)) { // v0.7.3: 递归展开 $and 中的等值条件 —— 此前仅顶层键,
// `WHERE a AND b`(解析为顶层 $and)永远全表扫描,索引形同虚设。
// $or/$not 语义不适用单索引下推,保守跳过。命中索引后 find 仍以
// 全条件 matchWhere 过滤(子集语义安全)。
const flat: [string, unknown][] = [];
const collect = (w: WhereCondition): void => {
for (const [k, v] of Object.entries(w)) {
if (k === '$and') {
for (const sub of (v as WhereCondition[])) collect(sub);
continue;
}
if (k === '$or' || k === '$not') continue;
flat.push([k, v]);
}
};
collect(query.where);
for (const [col, condition] of flat) {
// v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引 // v0.4.1: 支持 { $eq: value } 形式(SQL 解析器生成的等值条件)走索引
let targetValue: unknown; let targetValue: unknown;
if (typeof condition !== 'object' || condition === null) { if (typeof condition !== 'object' || condition === null) {
@@ -622,6 +693,10 @@ export class MemoryEngine implements IStorageEngine {
} else { } else {
continue; continue;
} }
// v0.7.3: null/undefined 条件不走索引 —— 索引不含 null 条目,
// colIndex.get(null) 恒 undefined → return [] 短路全表扫描 → 索引列
// IS NULL 恒空(对齐 AriaEngine v0.6.2 修复)
if (targetValue === null || targetValue === undefined) continue;
const colIndex = tableIndexes.get(col); const colIndex = tableIndexes.get(col);
if (colIndex) { if (colIndex) {
const pks = colIndex.get(targetValue); const pks = colIndex.get(targetValue);
+10 -4
View File
@@ -72,17 +72,23 @@ export function useDatabase(config: import('../constants').DatabaseConfig): {
const [db, setDb] = useState<MetonaSqlark | null>(null); const [db, setDb] = useState<MetonaSqlark | null>(null);
const [ready, setReady] = useState(false); const [ready, setReady] = useState(false);
const [error, setError] = useState<Error | null>(null); const [error, setError] = useState<Error | null>(null);
const initRef = useRef(false); // v0.7.3: config 变更时重建实例 —— 此前 initRef 只建一次,配置更新永不生效
// (且旧实例残留)。以 config 序列化指纹为依赖:值不变不重建,变更时
// cleanup 关闭旧实例(close 幂等,未完成 init 亦可安全关闭)再建新实例。
const configKey = JSON.stringify(config);
const prevKeyRef = useRef<string | null>(null);
useEffect(() => { useEffect(() => {
if (initRef.current) return; if (prevKeyRef.current === configKey) return;
initRef.current = true; prevKeyRef.current = configKey;
setReady(false);
setError(null);
const instance = new MetonaSqlark(config); const instance = new MetonaSqlark(config);
instance.init() instance.init()
.then(() => { setDb(instance); setReady(true); }) .then(() => { setDb(instance); setReady(true); })
.catch(setError); .catch(setError);
return () => { instance.close(); }; return () => { instance.close(); };
}, []); }, [configKey]);
return { db, ready, error }; return { db, ready, error };
} }
+9 -1
View File
@@ -13,7 +13,7 @@
*/ */
import { MetonaSqlark } from '../core'; import { MetonaSqlark } from '../core';
import { ref, watch, onMounted, type Ref } from 'vue'; import { ref, watch, onMounted, onUnmounted, type Ref } from 'vue';
/** useSqlarkQuery: 执行 SQL 查询 */ /** useSqlarkQuery: 执行 SQL 查询 */
export function useSqlarkQuery( export function useSqlarkQuery(
@@ -81,5 +81,13 @@ export function useSqlarkDatabase(config: import('../constants').DatabaseConfig)
} }
}); });
// v0.7.3: 组件卸载时关闭数据库 —— 此前实例永不 close(连接/锁/后端句柄泄漏)
onUnmounted(() => {
const instance = db.value;
if (instance) {
instance.close().catch(() => { /* 关闭失败不阻塞卸载 */ });
}
});
return { db, ready, error }; return { db, ready, error };
} }
+18 -4
View File
@@ -49,14 +49,22 @@ function inferFieldType(value: unknown): FieldType {
} }
/** 从样例行推断 schema(旧库无持久化 schema 时回退) */ /** 从样例行推断 schema(旧库无持久化 schema 时回退) */
function inferSchema(tableName: string, rows: Record<string, unknown>[]): TableSchema { function inferSchema(tableName: string, rows: Record<string, unknown>[]): TableSchema | null {
const columns: Record<string, ColumnDef> = {}; const columns: Record<string, ColumnDef> = {};
if (rows.length === 0) return { name: tableName, columns }; if (rows.length === 0) return { name: tableName, columns };
const first = rows[0]; const first = rows[0];
for (const key of Object.keys(first)) { const keys = Object.keys(first);
// v0.7.3: 主键推断 —— 优先 id;无 id 列时取第一个非 json 类型列(json 列
// String() 化为 "[object Object]" 会致所有行主键冲突)。全 json 列无可用
// 主键 → 返回 null(调用方跳过该表),此前直接抛 SCHEMA_ERROR 中断整个迁移。
const pkKey = keys.includes('id')
? 'id'
: keys.find((k) => inferFieldType(first[k]) !== 'json');
if (!pkKey) return null;
for (const key of keys) {
columns[key] = { columns[key] = {
type: inferFieldType(first[key]), type: inferFieldType(first[key]),
primaryKey: key === 'id', primaryKey: key === pkKey,
}; };
} }
return { name: tableName, columns }; return { name: tableName, columns };
@@ -151,7 +159,13 @@ export async function migrateFromIndexedDB(opts: MigrationOptions): Promise<Migr
result.skippedTables.push(tableName); result.skippedTables.push(tableName);
continue; continue;
} }
schema = inferSchema(tableName, rows); const inferred = inferSchema(tableName, rows);
// v0.7.3: 全 json 列推断无主键 → 跳过该表(此前抛 SCHEMA_ERROR 中断迁移)
if (!inferred) {
result.skippedTables.push(tableName);
continue;
}
schema = inferred;
} }
// 写入目标引擎 // 写入目标引擎
+39 -11
View File
@@ -209,18 +209,31 @@ export class QueryExecutor {
// v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。 // v0.7.0: 真实索引命中信息(此前 usingIndex 恒为 'auto' 占位)。
// 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。 // 引擎无关启发式:WHERE 中存在主键/索引/唯一列条件 → 对应引擎索引路径。
// v0.7.3: 递归识别 $and 嵌套等值条件(与 Memory/Aria 的 $and 下推行为对齐;
// $or/$not 不下推,保持 none)。
let usingIndex: string = plan?.table ? 'none' : 'none'; let usingIndex: string = plan?.table ? 'none' : 'none';
if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) { if (plan && plan.table && plan.where && Object.keys(plan.where).length > 0) {
try { try {
const schema = await this.engine.getTableSchema(plan.table); const schema = await this.engine.getTableSchema(plan.table);
if (schema) { if (schema) {
for (const col of Object.keys(plan.where)) { const findIndex = (w: import('../constants').WhereCondition): string | null => {
if (col.startsWith('$')) continue; for (const [k, v] of Object.entries(w)) {
const colDef = schema.columns[col]; if (k === '$and') {
if (!colDef) continue; for (const sub of (v as import('../constants').WhereCondition[])) {
if (colDef.primaryKey) { usingIndex = 'pk'; break; } const hit = findIndex(sub);
if (colDef.index || colDef.unique) { usingIndex = `index:${col}`; break; } if (hit) return hit;
} }
continue;
}
if (k === '$or' || k === '$not') continue;
const colDef = schema.columns[k];
if (!colDef) continue;
if (colDef.primaryKey) return 'pk';
if (colDef.index || colDef.unique) return `index:${k}`;
}
return null;
};
usingIndex = findIndex(plan.where) ?? 'none';
} }
} catch { /* schema 读取失败保持 none */ } } catch { /* schema 读取失败保持 none */ }
} }
@@ -341,7 +354,11 @@ export class QueryExecutor {
rows = rows.filter((row) => matchWhere(row, stmt.having!)); rows = rows.filter((row) => matchWhere(row, stmt.having!));
} }
if (stmt.orderBy && stmt.orderBy.length > 0) rows = applyOrderBy(rows, stmt.orderBy); if (stmt.orderBy && stmt.orderBy.length > 0) rows = applyOrderBy(rows, stmt.orderBy);
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0 && stmt.columns[0] !== '*') { // v0.7.3: `SELECT *, col AS alias` —— 此前 columns[0]==='*' 直接不投影,
// 别名列/常量列丢失。仅当 '*' 是唯一列时跳过投影(projectRow 对裸 '*'
// 合并原行全部列,其余表达式覆盖/追加)
if (!hasGroupBy && !hasAggregate && stmt.columns.length > 0
&& !(stmt.columns.length === 1 && stmt.columns[0] === '*')) {
rows = rows.map((row) => this.projectRow(row, stmt.columns)); rows = rows.map((row) => this.projectRow(row, stmt.columns));
} }
// v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序 // v0.3.3: ORDER BY 别名 → 投影后才存在,需在投影后重新排序
@@ -942,8 +959,13 @@ export class QueryExecutor {
const aliasCols: { alias: string; source: string }[] = []; const aliasCols: { alias: string; source: string }[] = [];
const caseCols: { alias: string; expr: CaseExpression }[] = []; const caseCols: { alias: string; expr: CaseExpression }[] = [];
const constCols: { key: string; value: unknown }[] = []; const constCols: { key: string; value: unknown }[] = [];
// v0.7.3: 裸 '*' 与列表达式混合(SELECT *, name AS nick)→ 原行全部列为基
let hasStar = false;
for (const col of columns) { for (const col of columns) {
if (col === '*') continue; if (col === '*') {
hasStar = true;
continue;
}
const expr = parseCaseExpression(col); const expr = parseCaseExpression(col);
if (expr) { if (expr) {
caseCols.push({ alias: expr.alias ?? col, expr }); caseCols.push({ alias: expr.alias ?? col, expr });
@@ -957,19 +979,25 @@ export class QueryExecutor {
// v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出 // v0.4.0: 字符串常量列 SELECT 'lit' → 常量输出
const lit = col.match(/^'(.*)'$/s); const lit = col.match(/^'(.*)'$/s);
if (lit) { if (lit) {
const value = lit[1].replace(/\\'/g, "'"); // v0.7.3: SQL 标准 '' 转义还原(readString 已把 '' 合并为单个 '
// 打包回列的文本中相邻两个 ' 即一个引号字面量)
const value = lit[1].replace(/''/g, "'");
constCols.push({ key: col, value }); constCols.push({ key: col, value });
continue; continue;
} }
plain.push(col); plain.push(col);
} }
const projected = plain.length > 0 ? projectColumns(row, plain) : {}; // v0.7.3: hasStar 时以原行全部列为基(projectColumns 仅投影 plain 列,不含 * 的其余列)
const projected = hasStar
? { ...row }
: (plain.length > 0 ? projectColumns(row, plain) : {});
for (const { alias, source } of aliasCols) { for (const { alias, source } of aliasCols) {
if (source === '*') { if (source === '*') {
Object.assign(projected, row); Object.assign(projected, row);
} else { } else {
const lit = source.match(/^'(.*)'$/s); const lit = source.match(/^'(.*)'$/s);
projected[alias] = lit ? lit[1].replace(/\\'/g, "'") : row[source]; // v0.7.3: 同 constCols —— SQL 标准 '' 转义还原
projected[alias] = lit ? lit[1].replace(/''/g, "'") : row[source];
} }
} }
for (const { key, value } of constCols) { for (const { key, value } of constCols) {
+6
View File
@@ -283,6 +283,12 @@ export class Parser {
if (this.curTokenIs(TokenType.STAR)) { if (this.curTokenIs(TokenType.STAR)) {
columns.push('*'); columns.push('*');
this.nextToken(); this.nextToken();
// v0.7.3: `SELECT *, col [AS alias], ...` —— '*' 后可继续列列表
// (此前 '*' 独占分支,逗号后直接 PARSE_ERRORexecutor 侧投影已支持混合)
while (this.curTokenIs(TokenType.COMMA)) {
this.nextToken();
columns.push(this.parseColumnWithAlias());
}
} else { } else {
columns.push(...this.parseColumnList()); columns.push(...this.parseColumnList());
} }
+59 -3
View File
@@ -16,6 +16,9 @@ jest.mock('react', () => {
const stateStore: any[] = []; const stateStore: any[] = [];
const registeredEffects = new Set<number>(); const registeredEffects = new Set<number>();
const effectQueue: Array<() => void | Promise<void>> = []; const effectQueue: Array<() => void | Promise<void>> = [];
// v0.7.3: 依赖数组指纹 + cleanup 支持(useDatabase config 变更重建测试用)
const effectDeps = new Map<number, string>();
const effectCleanups = new Map<number, () => void>();
let cursor = 0; let cursor = 0;
return { return {
useState: (init: any) => { useState: (init: any) => {
@@ -28,12 +31,26 @@ jest.mock('react', () => {
}, },
]; ];
}, },
// 按 hook 调用位置去重:同一位置的 effect 只在首次 render 注册 // 按 hook 调用位置注册:同一位置首次注册入队;deps 指纹变化时
useEffect: (fn: any, _deps: any[]) => { // 先执行旧 cleanup 再重跑 effect(模拟 React 依赖更新语义)
useEffect: (fn: any, _deps: any[] = []) => {
const idx = cursor; const idx = cursor;
const key = JSON.stringify(_deps ?? []);
if (!registeredEffects.has(idx)) { if (!registeredEffects.has(idx)) {
registeredEffects.add(idx); registeredEffects.add(idx);
effectQueue.push(fn); effectDeps.set(idx, key);
effectQueue.push(() => {
const cleanup = fn();
if (typeof cleanup === 'function') effectCleanups.set(idx, cleanup);
});
} else if (effectDeps.get(idx) !== key) {
effectDeps.set(idx, key);
effectQueue.push(() => {
const c = effectCleanups.get(idx);
if (c) { c(); effectCleanups.delete(idx); }
const cleanup = fn();
if (typeof cleanup === 'function') effectCleanups.set(idx, cleanup);
});
} }
}, },
useCallback: (fn: any) => fn, useCallback: (fn: any) => fn,
@@ -49,6 +66,8 @@ jest.mock('react', () => {
cursor = 0; cursor = 0;
effectQueue.length = 0; effectQueue.length = 0;
registeredEffects.clear(); registeredEffects.clear();
effectDeps.clear();
effectCleanups.clear();
}, },
}; };
}, { virtual: true }); }, { virtual: true });
@@ -197,4 +216,41 @@ describe('useDatabase', () => {
initSpy.mockRestore(); initSpy.mockRestore();
}); });
test('config 变更时关闭旧实例并重建(v0.7.3)', async () => {
const closeSpy = jest.spyOn(MetonaSqlark.prototype, 'close').mockResolvedValue();
// 首次挂载(真实异步 init
const mount = (config: { name: string }) => useDatabase(config);
render(() => mount({ name: 'hook-a' }));
await flushEffects();
// 轮询等待真实 init 完成(stateStore 经 re-render 读取最新值)
for (let i = 0; i < 100 && render(() => mount({ name: 'hook-a' })).db === null; i++) {
await new Promise((r) => setTimeout(r, 5));
}
const first = render(() => mount({ name: 'hook-a' }));
expect(first.db).not.toBeNull();
const firstDb = first.db;
// 同名 config 再渲染 → 不重建(相同实例)
const same = render(() => mount({ name: 'hook-a' }));
expect(same.db).toBe(firstDb);
// config 变更渲染 → cleanup 关闭旧实例 + 重建新实例
render(() => mount({ name: 'hook-b' }));
await flushEffects();
for (let i = 0; i < 100; i++) {
const cur = render(() => mount({ name: 'hook-b' }));
if (cur.db !== null && cur.db !== firstDb && cur.ready) break;
await new Promise((r) => setTimeout(r, 5));
}
const afterChange = render(() => mount({ name: 'hook-b' }));
expect(closeSpy).toHaveBeenCalled();
expect(afterChange.db).not.toBeNull();
expect(afterChange.db).not.toBe(firstDb);
expect(afterChange.ready).toBe(true);
closeSpy.mockRestore();
});
}); });
+35
View File
@@ -13,6 +13,7 @@
// ---- 最小 Vue mock(自包含:jest.mock 工厂不能引用外部变量) ---- // ---- 最小 Vue mock(自包含:jest.mock 工厂不能引用外部变量) ----
jest.mock('vue', () => { jest.mock('vue', () => {
const mountQueue: Array<() => void | Promise<void>> = []; const mountQueue: Array<() => void | Promise<void>> = [];
const unmountQueue: Array<() => void | Promise<void>> = [];
const watchList: Array<{ sources: any[]; cb: () => void | Promise<void> }> = []; const watchList: Array<{ sources: any[]; cb: () => void | Promise<void> }> = [];
return { return {
ref: (init: any) => { ref: (init: any) => {
@@ -25,10 +26,15 @@ jest.mock('vue', () => {
onMounted: (fn: any) => { onMounted: (fn: any) => {
mountQueue.push(fn); mountQueue.push(fn);
}, },
onUnmounted: (fn: any) => {
unmountQueue.push(fn);
},
__mockMounted: mountQueue, __mockMounted: mountQueue,
__mockUnmounted: unmountQueue,
__mockWatch: watchList, __mockWatch: watchList,
__mockReset: () => { __mockReset: () => {
mountQueue.length = 0; mountQueue.length = 0;
unmountQueue.length = 0;
watchList.length = 0; watchList.length = 0;
}, },
}; };
@@ -39,6 +45,7 @@ import { useSqlarkQuery, useSqlarkTable, useSqlarkDatabase } from '../../src/int
/** mock 模块内的挂载/监听队列(自包含作用域) */ /** mock 模块内的挂载/监听队列(自包含作用域) */
const vueMock = jest.requireMock('vue') as { const vueMock = jest.requireMock('vue') as {
__mockMounted: Array<() => void | Promise<void>>; __mockMounted: Array<() => void | Promise<void>>;
__mockUnmounted: Array<() => void | Promise<void>>;
__mockWatch: Array<{ sources: any[]; cb: () => void | Promise<void> }>; __mockWatch: Array<{ sources: any[]; cb: () => void | Promise<void> }>;
__mockReset: () => void; __mockReset: () => void;
}; };
@@ -55,6 +62,14 @@ async function flushMounted(): Promise<void> {
} }
} }
/** 模拟组件卸载:执行 onUnmounted 注册的回调 */
async function flushUnmounted(): Promise<void> {
const fns = vueMock.__mockUnmounted.splice(0);
for (const fn of fns) {
await fn();
}
}
/** 触发 watch 回调 */ /** 触发 watch 回调 */
async function flushWatch(): Promise<void> { async function flushWatch(): Promise<void> {
const pairs = vueMock.__mockWatch.splice(0); const pairs = vueMock.__mockWatch.splice(0);
@@ -163,4 +178,24 @@ describe('useSqlarkDatabase', () => {
expect(hook.ready.value).toBe(false); expect(hook.ready.value).toBe(false);
expect(hook.error.value).toBeInstanceOf(Error); expect(hook.error.value).toBeInstanceOf(Error);
}); });
test('组件卸载时关闭数据库实例(v0.7.3)', async () => {
const hook = useSqlarkDatabase({ name: 'vue-unmount', mode: 'memory' });
await flushMounted();
expect(hook.db.value).not.toBeNull();
const db = hook.db.value!;
// 实例打开中:isReady
expect(db.isReady()).toBe(true);
await flushUnmounted();
// close 后实例不可再查询
await expect(db.query('SELECT 1')).rejects.toMatchObject({ code: 'DB_NOT_READY' });
});
test('卸载时实例为 null 不抛错(初始化失败场景)', async () => {
const hook = useSqlarkDatabase({ name: 'vue-unmount-fail', mode: 'unknown-mode' as any });
await flushMounted();
await expect(flushUnmounted()).resolves.toBeUndefined();
expect(hook.db.value).toBeNull();
});
}); });
+57
View File
@@ -184,4 +184,61 @@ describe('migrateFromIndexedDB', () => {
.rejects.toThrow('not supported'); .rejects.toThrow('not supported');
await target.close(); await target.close();
}); });
it('无 id 列的旧库:第一个非 json 列兜底为主键(v0.7.3', async () => {
const legacyName = uniqueDB();
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(legacyName, 1);
request.onupgradeneeded = () => {
request.result.createObjectStore('nokey_tbl', { autoIncrement: true });
};
request.onsuccess = () => {
const db = request.result;
const tx = db.transaction('nokey_tbl', 'readwrite');
tx.objectStore('nokey_tbl').add({ code: 'c1', name: 'X' });
tx.objectStore('nokey_tbl').add({ code: 'c2', name: 'Y' });
tx.oncomplete = () => { db.close(); resolve(); };
tx.onerror = () => reject(tx.error);
};
request.onerror = () => reject(request.error);
});
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
// 此前无主键 → createSchema 抛 SCHEMA_ERROR 中断整个迁移;现在 code 列兜底为主键
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
expect(result.migratedTables).toEqual(['nokey_tbl']);
expect(result.rowCount).toBe(2);
const rows = await target.table('nokey_tbl').select().execute();
expect(rows).toHaveLength(2);
await target.close();
});
it('全 json 列的旧库:无可用主键 → 跳过该表不中断迁移(v0.7.3)', async () => {
const legacyName = uniqueDB();
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(legacyName, 1);
request.onupgradeneeded = () => {
request.result.createObjectStore('json_tbl', { autoIncrement: true });
request.result.createObjectStore('ok_tbl', { keyPath: 'id' });
};
request.onsuccess = () => {
const db = request.result;
const tx = db.transaction(['json_tbl', 'ok_tbl'], 'readwrite');
tx.objectStore('json_tbl').add({ payload: { a: 1 } });
tx.objectStore('ok_tbl').add({ id: '1', name: 'Z' });
tx.oncomplete = () => { db.close(); resolve(); };
tx.onerror = () => reject(tx.error);
};
request.onerror = () => reject(request.error);
});
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
// json_tbl 无可用主键被跳过;ok_tbl 正常迁移
expect(result.skippedTables).toContain('json_tbl');
expect(result.migratedTables).toEqual(['ok_tbl']);
await target.close();
});
}); });
+1 -1
View File
@@ -28,7 +28,7 @@ beforeEach(() => { installOPFSMock(new Map()); });
describe('[v0.2.5] P0-1: 版本号统一', () => { describe('[v0.2.5] P0-1: 版本号统一', () => {
test('VERSION 常量为当前版本(0.6.0', () => { test('VERSION 常量为当前版本(0.6.0', () => {
expect(VERSION).toBe('0.7.2'); expect(VERSION).toBe('0.7.3');
}); });
}); });
+1 -1
View File
@@ -403,7 +403,7 @@ describe('[v0.3.3] P1-9: Savepoint + MVCC 一致性', () => {
describe('[v0.3.3] 端到端', () => { describe('[v0.3.3] 端到端', () => {
test('全部修复点可共存于 MetonaSqlark API', async () => { test('全部修复点可共存于 MetonaSqlark API', async () => {
expect(VERSION).toBe('0.7.2'); expect(VERSION).toBe('0.7.3');
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' }); const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
+627
View File
@@ -0,0 +1,627 @@
/**
* v0.7.3
*
* 1. IS NULL Memory/KVStore/Hybrid
* 2. delete RESTRICT
* 3. insert PK/unique
* 4. Aria insert PK
* 5. queryStream
* 6. ALTER DROP
* 7. CREATE UNIQUE INDEX
* 8. SELECT *
* 9. INSERT hooks
* 10. KVStore insert validated
*/
import { MetonaSqlark } from '../src/core';
import { MemoryEngine } from '../src/engine/memory';
describe('v0.7.3: 索引列 IS NULL(三引擎对齐)', () => {
test.each([
['memory', { mode: 'memory' } as const],
['disk', { mode: 'disk', diskEngine: 'memory' } as const],
['hybrid', { mode: 'hybrid', diskEngine: 'memory' } as const],
])('%s: 索引列 IS NULL 返回 null 行', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-isnull-${_label}`, ...cfg });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', index: true },
});
await db.query("INSERT INTO users VALUES ('1', NULL), ('2', 'a@b.c')");
const rows = await db.query('SELECT * FROM users WHERE email IS NULL');
expect(rows).toHaveLength(1);
expect((rows[0] as Record<string, unknown>).id).toBe('1');
await db.close();
});
test.each([
['memory', { mode: 'memory' } as const],
['disk', { mode: 'disk', diskEngine: 'memory' } as const],
])('%s: 索引列 $eq: nullQuery Builder)不走索引短路', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-isnull-qb-${_label}`, ...cfg });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', unique: true },
});
await db.query("INSERT INTO users VALUES ('1', NULL), ('2', 'a@b.c')");
const rows = await db.table('users').select(['id']).where({ email: { $eq: null } }).execute();
expect(rows).toHaveLength(1);
await db.close();
});
test('aria: IS NULL 回归护栏(v0.6.2 已修)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-isnull-aria', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', index: true },
});
await db.query("INSERT INTO users VALUES ('1', NULL), ('2', 'a@b.c')");
const rows = await db.query('SELECT * FROM users WHERE email IS NULL');
expect(rows).toHaveLength(1);
await db.close();
});
});
describe('v0.7.3: delete RESTRICT 预检不破坏索引', () => {
test('memory: RESTRICT 抛错后唯一约束与索引查询保持有效', async () => {
const db = await MetonaSqlark.create({ name: 'v073-del-restrict', mode: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', unique: true, index: true },
});
await db.defineTable('orders', {
id: { type: 'string', primaryKey: true },
user_id: { type: 'string', references: 'users.id', onDelete: 'RESTRICT' },
});
await db.query("INSERT INTO users VALUES ('u1', 'x@x.x')");
await db.query("INSERT INTO orders VALUES ('o1', 'u1')");
let threw = false;
try { await db.query("DELETE FROM users WHERE id = 'u1'"); } catch { threw = true; }
expect(threw).toBe(true);
// 行仍在
expect(await db.query('SELECT * FROM users')).toHaveLength(1);
// 唯一约束仍有效
let uniqueThrew = false;
try { await db.query("INSERT INTO users VALUES ('u2', 'x@x.x')"); } catch { uniqueThrew = true; }
expect(uniqueThrew).toBe(true);
// 索引查询仍能命中
const viaIndex = await db.query("SELECT * FROM users WHERE email = 'x@x.x'");
expect(viaIndex).toHaveLength(1);
await db.close();
});
test('memory: 多行匹配删除 RESTRICT 失败 → 全部行索引完好', async () => {
const db = await MetonaSqlark.create({ name: 'v073-del-restrict-multi', mode: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
tag: { type: 'string', index: true },
});
await db.defineTable('orders', {
id: { type: 'string', primaryKey: true },
user_id: { type: 'string', references: 'users.id', onDelete: 'RESTRICT' },
});
await db.query("INSERT INTO users VALUES ('u1', 't1'), ('u2', 't2')");
await db.query("INSERT INTO orders VALUES ('o1', 'u1')");
let threw = false;
try { await db.query('DELETE FROM users'); } catch { threw = true; }
expect(threw).toBe(true);
expect(await db.query('SELECT * FROM users')).toHaveLength(2);
expect(await db.query("SELECT * FROM users WHERE tag = 't1'")).toHaveLength(1);
expect(await db.query("SELECT * FROM users WHERE tag = 't2'")).toHaveLength(1);
await db.close();
});
test('disk: RESTRICT 抛错后索引保持(与 memory 同路径)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-del-restrict-disk', mode: 'disk', diskEngine: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', unique: true, index: true },
});
await db.defineTable('orders', {
id: { type: 'string', primaryKey: true },
user_id: { type: 'string', references: 'users.id', onDelete: 'RESTRICT' },
});
await db.query("INSERT INTO users VALUES ('u1', 'x@x.x')");
await db.query("INSERT INTO orders VALUES ('o1', 'u1')");
let threw = false;
try { await db.query("DELETE FROM users WHERE id = 'u1'"); } catch { threw = true; }
expect(threw).toBe(true);
const viaIndex = await db.query("SELECT * FROM users WHERE email = 'x@x.x'");
expect(viaIndex).toHaveLength(1);
await db.close();
});
});
describe('v0.7.3: insert 语句级原子性', () => {
test.each([
['memory', { mode: 'memory' } as const],
['disk', { mode: 'disk', diskEngine: 'memory' } as const],
['hybrid', { mode: 'hybrid', diskEngine: 'memory' } as const],
])('%s: 批内主键重复 → 整句不执行', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-ins-atomic-pk-${_label}`, ...cfg });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
let threw = false;
try { await db.query("INSERT INTO users VALUES ('a'), ('a')"); } catch { threw = true; }
expect(threw).toBe(true);
expect(await db.query('SELECT * FROM users')).toHaveLength(0);
await db.close();
});
test.each([
['memory', { mode: 'memory' } as const],
['disk', { mode: 'disk', diskEngine: 'memory' } as const],
['hybrid', { mode: 'hybrid', diskEngine: 'memory' } as const],
])('%s: 批内唯一冲突 → 整句不执行', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-ins-atomic-uq-${_label}`, ...cfg });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', unique: true },
});
let threw = false;
try {
await db.query("INSERT INTO users VALUES ('1', 'a@b.c'), ('2', 'a@b.c')");
} catch { threw = true; }
expect(threw).toBe(true);
expect(await db.query('SELECT * FROM users')).toHaveLength(0);
await db.close();
});
test.each([
['memory', { mode: 'memory' } as const],
['disk', { mode: 'disk', diskEngine: 'memory' } as const],
])('%s: 第 N 行撞已有主键 → 整句不执行(含前 N-1 行)', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-ins-atomic-exist-${_label}`, ...cfg });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.query("INSERT INTO users VALUES ('a')");
let threw = false;
try { await db.query("INSERT INTO users VALUES ('b'), ('a')"); } catch { threw = true; }
expect(threw).toBe(true);
const rows = await db.query('SELECT * FROM users');
expect(rows).toHaveLength(1);
expect((rows[0] as Record<string, unknown>).id).toBe('a');
await db.close();
});
test('aria: 批内主键重复 → 整句不执行(此前部分提交 + WAL 不一致)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-ins-atomic-aria', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
let threw = false;
try { await db.query("INSERT INTO users VALUES ('a'), ('a')"); } catch { threw = true; }
expect(threw).toBe(true);
expect(await db.query('SELECT * FROM users')).toHaveLength(0);
await db.close();
});
test('aria: 事务内批内主键重复 → 整句不执行且快照干净', async () => {
const db = await MetonaSqlark.create({ name: 'v073-ins-atomic-aria-tx', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.query('BEGIN');
let threw = false;
try { await db.query("INSERT INTO users VALUES ('a'), ('a')"); } catch { threw = true; }
expect(threw).toBe(true);
await db.query('COMMIT');
expect(await db.query('SELECT * FROM users')).toHaveLength(0);
await db.close();
});
test('aria: 批内唯一冲突整批不落库回归护栏(v0.6.2)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-ins-atomic-aria-uq', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', unique: true },
});
let threw = false;
try {
await db.query("INSERT INTO users VALUES ('1', 'a@b.c'), ('2', 'a@b.c')");
} catch { threw = true; }
expect(threw).toBe(true);
expect(await db.query('SELECT * FROM users')).toHaveLength(0);
await db.close();
});
});
describe('v0.7.3: queryStream 子查询回退物化', () => {
test.each([
['memory', { mode: 'memory' } as const],
['aria', { mode: 'aria', diskEngine: 'memory' } as const],
])('%s: IN 子查询流式查询返回正确结果(回退物化)', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-stream-sub-${_label}`, ...cfg });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.defineTable('orders', { id: { type: 'string', primaryKey: true }, user_id: { type: 'string' } });
await db.query("INSERT INTO users VALUES ('1'), ('2')");
await db.query("INSERT INTO orders VALUES ('o1', '1')");
const collected: Record<string, unknown>[] = [];
const n = await db.queryStream('SELECT * FROM users WHERE id IN (SELECT user_id FROM orders)', (r) => collected.push(r));
expect(n).toBe(1);
expect(collected).toHaveLength(1);
expect(collected[0].id).toBe('1');
await db.close();
});
test('memory: EXISTS 关联子查询流式查询回退物化', async () => {
const db = await MetonaSqlark.create({ name: 'v073-stream-exists', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.defineTable('orders', { id: { type: 'string', primaryKey: true }, user_id: { type: 'string' } });
await db.query("INSERT INTO users VALUES ('1'), ('2')");
await db.query("INSERT INTO orders VALUES ('o1', '1')");
const collected: Record<string, unknown>[] = [];
await db.queryStream(
'SELECT * FROM users u WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)',
(r) => collected.push(r),
);
expect(collected).toHaveLength(1);
await db.close();
});
test('memory: 简单查询仍走引擎流式路径(未误回退)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-stream-simple', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.query("INSERT INTO users VALUES ('1'), ('2')");
const collected: Record<string, unknown>[] = [];
const n = await db.queryStream("SELECT * FROM users WHERE id = '1'", (r) => collected.push(r));
expect(n).toBe(1);
expect(collected).toHaveLength(1);
await db.close();
});
test('memory: WHERE 列引用($col)流式查询回退物化且不抛错', async () => {
const db = await MetonaSqlark.create({ name: 'v073-stream-colref', mode: 'memory' });
await db.defineTable('t1', { id: { type: 'string', primaryKey: true }, x: { type: 'number' }, y: { type: 'number' } });
await db.query("INSERT INTO t1 VALUES ('1', 5, 5), ('2', 10, 3)");
// t1.x = t1.y 解析为 $col 列引用 —— 引擎层 matchWhere 无 $col 匹配分支,
// 此前流式路径会抛 QUERY_ERROR/静默过滤;v0.7.3 回退物化,结果与 query() 一致
const viaQuery = await db.query('SELECT * FROM t1 WHERE t1.x = t1.y');
const collected: Record<string, unknown>[] = [];
await db.queryStream('SELECT * FROM t1 WHERE t1.x = t1.y', (r) => collected.push(r));
expect(collected).toHaveLength((viaQuery as Record<string, unknown>[]).length);
await db.close();
});
});
describe('v0.7.3: ALTER DROP 索引列清理', () => {
test.each([
['memory', { mode: 'memory' } as const],
['disk', { mode: 'disk', diskEngine: 'memory' } as const],
])('%s: DROP 索引列后无旧索引短路(新增行可见)', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-alter-drop-idx-${_label}`, ...cfg });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', index: true },
});
await db.query("INSERT INTO users VALUES ('1', 'a@b.c')");
await db.query('ALTER TABLE users DROP COLUMN email');
await db.query("INSERT INTO users VALUES ('2')");
expect(await db.query('SELECT * FROM users')).toHaveLength(2);
// 已删列不再存在于 schema;查询该列应报错而非走旧索引(executor 层不会到达)
const rows = await db.query('SELECT * FROM users WHERE id = \'2\'');
expect(rows).toHaveLength(1);
await db.close();
});
test('memory: DROP 非索引列不影响其他列索引', async () => {
const db = await MetonaSqlark.create({ name: 'v073-alter-drop-plain', mode: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
name: { type: 'string' },
email: { type: 'string', index: true },
});
await db.query("INSERT INTO users VALUES ('1', 'Alice', 'a@b.c')");
await db.query('ALTER TABLE users DROP COLUMN name');
const viaIndex = await db.query("SELECT * FROM users WHERE email = 'a@b.c'");
expect(viaIndex).toHaveLength(1);
await db.close();
});
});
describe('v0.7.3: CREATE UNIQUE INDEX 存量唯一性', () => {
test.each([
['memory', { mode: 'memory' } as const],
['aria', { mode: 'aria', diskEngine: 'memory' } as const],
])('%s: 存量重复数据 → 抛 UNIQUE_VIOLATION 且无半初始化索引', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-uqidx-dup-${_label}`, ...cfg });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, email: { type: 'string' } });
await db.query("INSERT INTO users VALUES ('1', 'a@b.c'), ('2', 'a@b.c')");
let threw = false;
let code = '';
try { await db.query('CREATE UNIQUE INDEX idx_e ON users (email)'); } catch (e) {
threw = true;
code = (e as { code?: string }).code ?? '';
}
expect(threw).toBe(true);
expect(code).toBe('UNIQUE_VIOLATION');
// 失败后列标志未落:普通 CREATE INDEX 仍可建立
await db.query('CREATE INDEX idx_e ON users (email)');
const viaIndex = await db.query("SELECT * FROM users WHERE email = 'a@b.c'");
expect(viaIndex).toHaveLength(2);
await db.close();
});
test.each([
['memory', { mode: 'memory' } as const],
['aria', { mode: 'aria', diskEngine: 'memory' } as const],
])('%s: 存量数据唯一 → 建索引成功且唯一约束生效', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-uqidx-ok-${_label}`, ...cfg });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, email: { type: 'string' } });
await db.query("INSERT INTO users VALUES ('1', 'a@b.c'), ('2', 'b@b.c')");
await db.query('CREATE UNIQUE INDEX idx_e ON users (email)');
let threw = false;
try { await db.query("INSERT INTO users VALUES ('3', 'a@b.c')"); } catch { threw = true; }
expect(threw).toBe(true);
expect(await db.query('SELECT * FROM users')).toHaveLength(2);
await db.close();
});
});
describe('v0.7.3: SELECT * 混别名列投影', () => {
test('memory: SELECT *, name AS nick 保留全部列 + 别名列', async () => {
const db = await MetonaSqlark.create({ name: 'v073-star-alias', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' }, age: { type: 'number' } });
await db.query("INSERT INTO users VALUES ('1', 'Alice', 30)");
const rows = await db.query('SELECT *, name AS nick FROM users');
expect(rows).toHaveLength(1);
const row = rows[0] as Record<string, unknown>;
expect(row.id).toBe('1');
expect(row.name).toBe('Alice');
expect(row.age).toBe(30);
expect(row.nick).toBe('Alice');
await db.close();
});
test('memory: 纯 SELECT * 行为不变(原行引用键集合)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-star-only', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
await db.query("INSERT INTO users VALUES ('1', 'Alice')");
const rows = await db.query('SELECT * FROM users');
expect(Object.keys(rows[0] as Record<string, unknown>).sort()).toEqual(['id', 'name']);
await db.close();
});
test('memory: SELECT *, 常量列混合', async () => {
const db = await MetonaSqlark.create({ name: 'v073-star-const', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.query("INSERT INTO users VALUES ('1')");
const rows = await db.query("SELECT *, 'lit' AS c FROM users");
const row = rows[0] as Record<string, unknown>;
expect(row.id).toBe('1');
expect(row.c).toBe('lit');
await db.close();
});
});
describe('v0.7.3: INSERT hooks 列映射', () => {
test('SQL 省略列名时 beforeInsert 收到 schema 列名映射', async () => {
const db = await MetonaSqlark.create({ name: 'v073-hooks-insert', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
let seenRows: unknown = null;
db.on('beforeInsert', async (rows: unknown) => { seenRows = rows; });
await db.query("INSERT INTO users VALUES ('1', 'Alice')");
const rows = seenRows as Record<string, unknown>[];
expect(rows).toHaveLength(1);
expect(rows[0].id).toBe('1');
expect(rows[0].name).toBe('Alice');
await db.close();
});
test('SQL 显式列名时 hooks 行键按显式列名', async () => {
const db = await MetonaSqlark.create({ name: 'v073-hooks-insert-cols', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
let seenRows: unknown = null;
db.on('beforeInsert', async (rows: unknown) => { seenRows = rows; });
await db.query("INSERT INTO users (name, id) VALUES ('Alice', '1')");
const rows = seenRows as Record<string, unknown>[];
expect(rows[0].id).toBe('1');
expect(rows[0].name).toBe('Alice');
await db.close();
});
});
describe('v0.7.3: KVStore insert 持久化 validated 行', () => {
test('default 值与列投影落盘(跨实例恢复一致)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-kv-validated', mode: 'disk', diskEngine: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, age: { type: 'number', default: 18 } });
await db.query("INSERT INTO users VALUES ('1')");
await db.close();
const db2 = await MetonaSqlark.create({ name: 'v073-kv-validated', mode: 'disk', diskEngine: 'memory' });
const rows = await db2.query('SELECT * FROM users');
expect(rows).toHaveLength(1);
expect(rows[0]).toEqual({ id: '1', age: 18 });
await db2.close();
});
test('schema 外列不持久化', async () => {
const db = await MetonaSqlark.create({ name: 'v073-kv-extra-col', mode: 'disk', diskEngine: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.table('users').insert({ id: '1', junk: 'x' } as never);
await db.close();
const db2 = await MetonaSqlark.create({ name: 'v073-kv-extra-col', mode: 'disk', diskEngine: 'memory' });
const rows = await db2.query('SELECT * FROM users');
expect(Object.keys(rows[0] as Record<string, unknown>).sort()).toEqual(['id']);
await db2.close();
});
test('MemoryEngine.getRow 暴露 validated 行', async () => {
const engine = new MemoryEngine();
await engine.open('v073-getrow', 1);
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, age: { type: 'number', default: 18 } } });
await engine.insert('users', [{ id: '1' }]);
const row = engine.getRow('users', '1');
expect(row).toEqual({ id: '1', age: 18 });
expect(engine.getRow('users', 'nope')).toBeNull();
expect(engine.getRow('missing', '1')).toBeNull();
await engine.close();
});
});
describe('v0.7.3: WAL BEGIN/ROLLBACK 写失败窗口', () => {
async function createAria(name: string): Promise<{ db: MetonaSqlark; walStore: { append: (d: Uint8Array) => Promise<void> } }> {
const db = await MetonaSqlark.create({ name, mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
const engine = db.getEngine() as unknown as { wal: { store: { append: (d: Uint8Array) => Promise<void> } } };
return { db, walStore: engine.wal.store };
}
test('BEGIN 记录写失败 → 事务状态不泄漏(可重试)', async () => {
const { db, walStore } = await createAria('v073-wal-begin-fail');
const orig = walStore.append.bind(walStore);
walStore.append = async () => { throw new Error('wal boom'); };
await expect(db.query('BEGIN')).rejects.toThrow();
walStore.append = orig;
// 事务状态未泄漏:可正常开始并提交新事务
await db.query('BEGIN');
await db.query("INSERT INTO users VALUES ('1')");
await db.query('COMMIT');
expect(await db.query('SELECT * FROM users')).toHaveLength(1);
await db.close();
});
test('ROLLBACK 记录写失败 → 事务仍活跃(可重试回滚,不复活数据)', async () => {
const { db, walStore } = await createAria('v073-wal-rollback-fail');
await db.query('BEGIN');
await db.query("INSERT INTO users VALUES ('1')");
const orig = walStore.append.bind(walStore);
walStore.append = async () => { throw new Error('wal boom'); };
// ROLLBACK WAL 记录先写失败 → 内存未回滚、事务仍活跃
await expect(db.query('ROLLBACK')).rejects.toThrow();
walStore.append = orig;
// 重试回滚成功,数据未提交
await db.query('ROLLBACK');
expect(await db.query('SELECT * FROM users')).toHaveLength(0);
await db.close();
});
});
describe('v0.7.3: aria $in 批级预加载', () => {
test('多值 IN(含重复值/未命中值)索引查询正确', async () => {
const db = await MetonaSqlark.create({ name: 'v073-in-batch', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
tag: { type: 'string', index: true },
});
for (let i = 0; i < 50; i++) {
await db.query('INSERT INTO users VALUES (?, ?)', [String(i), `t${i % 5}`]);
}
// flush 使索引/主数据 SSTable 化(预加载路径真实生效)
const engine = db.getEngine() as unknown as { lsm: { flush(): Promise<void> }; secondaryIndexes: Map<string, { flush(): Promise<void> }> };
await engine.lsm.flush();
for (const idxLsm of engine.secondaryIndexes.values()) await idxLsm.flush();
const rows = await db.query("SELECT * FROM users WHERE tag IN ('t1', 't2', 't1', 't9')");
expect(rows).toHaveLength(20);
await db.close();
});
test('IN 与 $and 组合条件结果正确(索引子集 + 全条件过滤)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-in-and', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
tag: { type: 'string', index: true },
age: { type: 'number' },
});
for (let i = 0; i < 20; i++) {
await db.query('INSERT INTO users VALUES (?, ?, ?)', [String(i), `t${i % 4}`, i]);
}
const rows = await db.query("SELECT * FROM users WHERE tag IN ('t1', 't2') AND age >= 10");
expect(rows).toHaveLength(5);
await db.close();
});
});
describe('v0.7.3: $and 等值条件下推', () => {
test.each([
['memory', { mode: 'memory' } as const],
['aria', { mode: 'aria', diskEngine: 'memory' } as const],
])('%s: 多条件 AND 查询结果正确', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-and-push-${_label}`, ...cfg });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
tag: { type: 'string', index: true },
age: { type: 'number' },
});
await db.query("INSERT INTO users VALUES ('1', 'a', 10), ('2', 'a', 20), ('3', 'b', 10)");
const rows = await db.query("SELECT * FROM users WHERE tag = 'a' AND age = 10");
expect(rows).toHaveLength(1);
expect((rows[0] as Record<string, unknown>).id).toBe('1');
await db.close();
});
test.each([
['memory', { mode: 'memory' } as const],
['aria', { mode: 'aria', diskEngine: 'memory' } as const],
])('%s: EXPLAIN 识别 $and 嵌套索引条件', async (_label, cfg) => {
const db = await MetonaSqlark.create({ name: `v073-and-explain-${_label}`, ...cfg });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
tag: { type: 'string', index: true },
age: { type: 'number' },
});
await db.query("INSERT INTO users VALUES ('1', 'a', 10)");
// 索引列在 $and 嵌套中 → 递归识别 index:tag
const plan1 = await db.query("EXPLAIN SELECT * FROM users WHERE age > 5 AND tag = 'a'");
expect((plan1 as Record<string, unknown>).usingIndex).toBe('index:tag');
// 主键在 $and 嵌套中 → pk
const plan2 = await db.query("EXPLAIN SELECT * FROM users WHERE age > 5 AND id = '1'");
expect((plan2 as Record<string, unknown>).usingIndex).toBe('pk');
await db.close();
});
});
describe('v0.7.3: SELECT 常量列 SQL 标准转义', () => {
test("SELECT 'O''Brien' 还原为 O'Brien", async () => {
const db = await MetonaSqlark.create({ name: 'v073-escape', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.query("INSERT INTO users VALUES ('1')");
const rows = await db.query("SELECT 'O''Brien' AS name FROM users");
expect((rows[0] as Record<string, unknown>).name).toBe("O'Brien");
await db.close();
});
test("无表查询 SELECT 'a''b' AS x 转义还原", async () => {
const db = await MetonaSqlark.create({ name: 'v073-escape-notable', mode: 'memory' });
const rows = await db.query("SELECT 'a''b' AS x");
expect((rows[0] as Record<string, unknown>).x).toBe("a'b");
await db.close();
});
test("SELECT *, 'x''y' AS c 混合投影转义", async () => {
const db = await MetonaSqlark.create({ name: 'v073-escape-star', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await db.query("INSERT INTO users VALUES ('1')");
const rows = await db.query("SELECT *, 'x''y' AS c FROM users");
const row = rows[0] as Record<string, unknown>;
expect(row.id).toBe('1');
expect(row.c).toBe("x'y");
await db.close();
});
});
describe('v0.7.3: ANALYZE 统计二级索引', () => {
test('aria: 统计含二级索引 LSMsstableCount/memtableSize 汇总)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-analyze', mode: 'aria', diskEngine: 'memory' });
await db.defineTable('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', index: true },
tag: { type: 'string', index: true },
});
for (let i = 0; i < 30; i++) {
await db.query('INSERT INTO users VALUES (?, ?, ?)', [String(i), `e${i}@x.x`, `t${i % 3}`]);
}
const engine = db.getEngine() as unknown as {
lsm: { flush(): Promise<void> };
secondaryIndexes: Map<string, { flush(): Promise<void> }>;
};
await engine.lsm.flush();
for (const idxLsm of engine.secondaryIndexes.values()) await idxLsm.flush();
const stats = await db.query('ANALYZE users');
expect((stats as Record<string, unknown>).rowCount).toBe(30);
expect(typeof (stats as Record<string, unknown>).indexDepth).toBe('number');
expect((stats as Record<string, unknown>).sstableCount).toBeGreaterThanOrEqual(1);
expect(typeof (stats as Record<string, unknown>).memtableSize).toBe('number');
await db.close();
});
test('memory: ANALYZE 不支持抛 NOT_SUPPORTED(护栏)', async () => {
const db = await MetonaSqlark.create({ name: 'v073-analyze-memory', mode: 'memory' });
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
await expect(db.query('ANALYZE users')).rejects.toMatchObject({ code: 'NOT_SUPPORTED' });
await db.close();
});
});