release: v0.4.2 — 生产就绪与崩溃自愈 + 问题清单修复 + 版本迭代
This commit is contained in:
@@ -2,6 +2,57 @@
|
|||||||
|
|
||||||
All notable changes to MetonaSqlark will be documented in this file.
|
All notable changes to MetonaSqlark will be documented in this file.
|
||||||
|
|
||||||
|
## [0.4.2] - 2026-08-09
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **AriaEngine 残缺 SSTable 导致打开崩溃(P0)** — 异常退出后下次打开抛原生 `RangeError: offset is out of bounds`,库打不开只能删库重建。三处根治:
|
||||||
|
- `SSTableReader` 读路径防御:块偏移/大小越界时跳过该块(get/rangeScan/scanAll 返回空或继续),索引块/块内条目解析全程边界检查,不再抛 RangeError
|
||||||
|
- 打开时完整性校验:`LSM.init()` 校验 meta 引用的文件存在、长度 ≥ 头、可解析,残缺的忽略并清理 meta/文件(自愈,不阻塞打开)
|
||||||
|
- WAL 原子写入:`IndexedDBBackend` 新增 `writeMany`/`deleteMany`(单事务批量写删),WAL 记录与 count 计数同事务提交,中断整体回滚不留半写
|
||||||
|
- **IndexedDB 版本管理缺陷导致重启必报 VersionError(P0)** — 建表每张版本号 +1 而 `open()` 用 `config.version`,重启后版本过低直接失败。`open()` 遇 VersionError 自动用无版本参数探测库当前实际版本并以该版本重开
|
||||||
|
- **`create({ version: 0 })` 抛原生 TypeError(P0)** — 版本 < 1 统一归一化为 1,不再直接炸 `indexedDB.open(name, 0)`
|
||||||
|
- **WAL 写丢失(P1)** — 连续快速写入 200 条 close+重开实测丢 4 条。WAL 恢复不再依赖 count 计数,改为扫描全部 `__wal_*` 键(数值排序拼接),count 键只作序号分配器;配合原子写入彻底消除 count/记录竞态
|
||||||
|
- **AriaEngine.close() 不 checkpoint,WAL 无限重放(P1)** — close 前补 `wal.checkpoint()` 截断,下次打开不再重放已落盘的历史记录
|
||||||
|
- **引擎内部错误不包装 DatabaseError(P1)** — AriaEngine `open()` 整体 try/catch 包装为 `ARIA_OPEN_ERROR`;IndexedDBEngine 打开路径统一 `IDB_OPEN_ERROR`/`IDB_VERSION_ERROR`/`IDB_BLOCKED`,应用层可拿 code 分类处理
|
||||||
|
- **`migrateTo` 版本门槛与引擎版本脱节(P2)** — 迁移版本持久化到库内(`IStorageEngine.getMeta/setMeta` 可选接口,Aria 存 backend、IndexedDB 复用 `__metona_schema` store、OPFS 存元数据文件、Hybrid 委托磁盘引擎),重启后从持久化版本继续执行,不重跑不跳跑
|
||||||
|
- **IndexedDB `onblocked` 直接 reject(P2)** — 改为等待后重试(阻塞解除后 success 仍触发,仅超时判失败,并防止连接泄漏),多次重试仍阻塞才抛 `IDB_BLOCKED`
|
||||||
|
- **缺少官方崩溃恢复/自愈接口(P2)** — 新增 `db.repair()`(校验清理损坏数据并恢复一致性)与 `db.clearAll()`(清空全部数据与表结构,保留库),全部引擎(Aria/IndexedDB/OPFS/Memory/Hybrid)实现,无需删库重建
|
||||||
|
- **IndexedDB `__metona_schema` store 缺失时元数据无法落盘** — `open()` 时确保 schema/meta store 存在(缺失则一次版本升级创建)
|
||||||
|
|
||||||
|
### Fixed(深度审计第二轮)
|
||||||
|
|
||||||
|
- **AriaEngine 二级索引跨重启丢失(严重)** — 重开只恢复 schema 不恢复索引 LSM:索引查询静默回退全表、`createIndex` 因 schema 标记已存在而直接 return → 索引永久缺失;且 WAL 崩溃恢复只回放主 LSM,最后一批写入的索引缺失 → 索引查询丢行。三处根治:open 时按 schema 标记重建索引 LSM(数据已持久化直接加载)、close 时同步落盘全部索引 LSM、WAL 恢复完成后全量重建索引
|
||||||
|
- **AriaEngine Compaction 依赖缓存丢数据(严重)** — `compactLevelAsync` 仅从 LRU 缓存读 SSTable,缓存未命中(单文件超缓存上限被驱逐、频繁 flush)时跳过全部文件并从 levels 移除 → 运行中数据全部不可见(重启才恢复)。改为从存储兜底加载;无有效数据时将文件放回 levels 不删除
|
||||||
|
- **AriaEngine flush/compaction 失败卡死写路径(严重)** — flushChain 链上任务抛错后永久 rejected,后续所有 flush/compaction 挂起。链上任务统一吞错恢复(记录告警,链继续)
|
||||||
|
- **ALTER TABLE 在 IndexedDB/Hybrid/OPFS 引擎不持久化** — v0.4.1 只给 Aria 加了引擎级 `alterTable`,其余引擎走通用路径只改内存 schema 引用:重启后 ADD/DROP 全部回退,DROP 的行数据也残留。为 IndexedDB(持久化 schema + 重写存储行)/ Hybrid(双引擎)/ OPFS / Memory 统一实现引擎级 `alterTable`
|
||||||
|
- **IndexedDB 事务内 DDL 崩溃(严重)** — 事务内建表 commit 时 IDB 无对应 store 报错;事务内删表 IDB store 残留 → 重启幽灵表。`flushToIDB` 提交时 diff 内存表与 IDB store:缺失的创建(含 schema 持久化)、多余的删除(含 schema 记录清理);commit 顺序调整为"先刷盘后提交内存快照",失败可回滚
|
||||||
|
- **Hybrid 内存/磁盘引擎共享 schema 引用污染** — `reloadMemoryFromDisk` 把磁盘引擎的 schema 对象直接存入内存引擎,任一引擎 ALTER 都改到对方。`MemoryEngine.createTable` 深拷贝 schema(全引擎受益)
|
||||||
|
- **Hybrid commit 失败路径错误掩盖** — 磁盘已提交后内存提交失败时调 `diskEngine.rollbackTransaction()` 抛 TX_NONE 掩盖原错误;如实上报磁盘已提交状态
|
||||||
|
- **AriaEngine.close 运行期状态残留** — close 后 MVCC/事务快照/savepoint 残留,重开后 `beginTransaction` 报 TX_ACTIVE。close 统一清理
|
||||||
|
- **VACUUM 对 2-3 个文件层级不压缩** — `compactLevel` public 门槛与内部自动调度门槛混用(内部 4 / VACUUM 期望 2)。分离参数化
|
||||||
|
|
||||||
|
### Fixed(生产就绪审计第三轮)
|
||||||
|
|
||||||
|
- **Aria 事务进行中 checkpoint 截断 WAL(严重,P0)** — checkpoint(tick/forceCheckpoint)在活跃事务中途执行时 truncate WAL,BEGIN/INSERT 记录被截断,事务 COMMIT 后崩溃恢复丢失整个事务数据。CheckpointManager 包装 WAL:事务活跃时跳过截断(close/repair/clearAll 仍正常截断)
|
||||||
|
- **Aria dropTable / DROP_TABLE 恢复不清理二级索引(P1)** — 索引 LSM 孤儿残留,重建同名表后旧索引数据污染新表(按旧值索引查询返回不匹配行)。新增 `cleanupTableIndexes`:dropTable、崩溃恢复、ALTER DROP 索引列统一清理
|
||||||
|
- **Aria ALTER TABLE DROP 索引列残留(P1)** — 删列但索引 LSM 保留,后续同名列索引数据脏
|
||||||
|
- **Aria freezeMemtable 阈值逐次衰减(P2)** — 新 memtable 用旧表已用大小当阈值,手动 flush 后阈值塌缩 → 频繁小文件。改回配置阈值
|
||||||
|
- **Aria 事务中 DDL 静默不一致(P2)** — 事务快照只覆盖行数据,createTable/dropTable/alterTable 无法回滚;与 Memory/IndexedDB 可回滚行为不一致 → 显式抛 `NOT_SUPPORTED`(不再静默)
|
||||||
|
- **ON UPDATE 外键级联未实现(P2,声称支持但静默忽略)** — 实现 `ON UPDATE CASCADE/SET NULL/RESTRICT`(Memory + Aria 双引擎):更新主键时级联更新/拒绝,含二级索引与 WAL 记录,先全量 RESTRICT 检查再执行防部分修改;update 同时支持主键变更(旧键删除 + 新键落表)
|
||||||
|
- **MVCC commit/rollback 全库版本遍历(P2 性能)** — 大表事务 commit O(全库版本数)。按事务记录写入的 key 精准清理(O(写入数))
|
||||||
|
- **OPFS 空表重启消失 + schema/索引丢失(P1)** — 表 schema 持久化到元数据文件:createTable/dropTable/alterTable/createIndex 同步;重启从 schema 恢复(空表保留、主键/索引标记/约束完整),无 schema 的旧库回退数据推断(兼容)
|
||||||
|
- **OPFS dropTable 不清理 schema 记录(P1)** — 重启恢复幽灵表;删表同步清理
|
||||||
|
- **OPFS 并发写验证** — 审计确认内存写同步、持久化快照总是最新(单标签页无丢更新竞态),新增并发写回归测试锁定行为
|
||||||
|
- **VACUUM 返回值语义** — `gcVersions` 返回活跃事务数改为全局提交序列号
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- 测试 894 → **944**(50 套件),新增 `tests/v042-fixes.test.ts`(23 条问题清单回归)、`tests/v042-hardening.test.ts`(12 条深度审计回归)与 `tests/v043-hardening.test.ts`(15 条生产就绪回归:事务×checkpoint、DDL 索引清理、OPFS 持久化、memtable 阈值、红黑树 5000 次随机压力、onUpdate 级联)
|
||||||
|
- 版本号升至 v0.4.2
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [0.4.1] - 2026-08-08
|
## [0.4.1] - 2026-08-08
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
# MetonaSqlark
|
# MetonaSqlark
|
||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<img src="https://img.shields.io/badge/version-0.4.1-blue?style=flat-square" alt="version">
|
<img src="https://img.shields.io/badge/version-0.4.2-blue?style=flat-square" alt="version">
|
||||||
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
||||||
<img src="https://img.shields.io/badge/coverage-81.5%25-brightgreen?style=flat-square" alt="coverage">
|
<img src="https://img.shields.io/badge/coverage-84.2%25-brightgreen?style=flat-square" alt="coverage">
|
||||||
<img src="https://img.shields.io/badge/tests-894%20passed-success?style=flat-square" alt="tests">
|
<img src="https://img.shields.io/badge/tests-944%20passed-success?style=flat-square" alt="tests">
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研页面式存储引擎**。
|
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研页面式存储引擎**。
|
||||||
@@ -14,19 +14,21 @@
|
|||||||
## ✨ 特性
|
## ✨ 特性
|
||||||
|
|
||||||
- 🚀 **AriaEngine 自研存储引擎** — LSM-Tree 页面式存储,4KB Slotted Page、WAL 崩溃恢复(full 模式真正同步)、LZ4 压缩
|
- 🚀 **AriaEngine 自研存储引擎** — LSM-Tree 页面式存储,4KB Slotted Page、WAL 崩溃恢复(full 模式真正同步)、LZ4 压缩
|
||||||
- 💾 **OPFS 自研存储后端** — 纯浏览器文件系统,零 IndexedDB 依赖,二进制页面文件
|
- 💾 **OPFS 自研存储后端** — 纯浏览器文件系统,零 IndexedDB 依赖,二进制页面文件,schema 持久化(空表/索引跨重启完整保留)
|
||||||
- 🔒 **生产级数据安全** — WAL CRC 完整性校验、`RESTRICT` 外键约束、Hybrid 提交原子性、SQL 注入防护
|
- 🔒 **生产级数据安全** — WAL 原子写入 + CRC 完整性校验、`RESTRICT` 外键约束、崩溃恢复自愈(`repair()` 无需删库重建)、SQL 注入防护
|
||||||
- 🛡 **输入校验全覆盖** — `maxLength`/`min`/`max` 约束、类型检查、必填验证
|
- 🛡 **输入校验全覆盖** — `maxLength`/`min`/`max` 约束、类型检查、必填验证
|
||||||
- 💾 **多引擎架构** — Memory / IndexedDB / OPFS / Hybrid(write-through) / Aria 五种模式
|
- 💾 **多引擎架构** — Memory / IndexedDB / OPFS / Hybrid(write-through) / Aria 五种模式
|
||||||
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTS(v0.3.0)+ CASE WHEN/哈希连接/组提交(v0.3.1)+ 多标签页同步(v0.3.2)
|
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTS(v0.3.0)+ CASE WHEN/哈希连接/组提交(v0.3.1)+ 多标签页同步(v0.3.2)
|
||||||
- 🚰 **流式查询** — `queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.1)
|
- 🚰 **流式查询** — `queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.0)
|
||||||
- 🧩 **派生表** — `FROM (SELECT ...)` 子查询作为行源,多列 ON 哈希连接,COUNT(DISTINCT),NULLS FIRST/LAST(v0.4.1)
|
- 🧩 **派生表** — `FROM (SELECT ...)` 子查询作为行源,多列 ON 哈希连接,COUNT(DISTINCT),NULLS FIRST/LAST(v0.4.0)
|
||||||
- 🔗 **Query Builder API** — 链式 `.select().where().orderBy().limit().execute()`
|
- 🔗 **Query Builder API** — 链式 `.select().where().orderBy().limit().execute()`
|
||||||
- 🔄 **事务回滚** — Memory/IndexedDB/Hybrid/Aria 四引擎事务原子性,自动回滚,MVCC 版本链接入读写路径
|
- 🔄 **事务回滚** — Memory/IndexedDB/Hybrid/Aria 四引擎事务原子性,自动回滚,MVCC 版本链接入读写路径
|
||||||
|
- 🔗 **外键级联** — ON DELETE + ON UPDATE(CASCADE / SET NULL / RESTRICT)全引擎支持,支持更新主键(v0.4.2)
|
||||||
|
- 🛡 **崩溃恢复自愈** — 残缺 SSTable 打开自动跳过、`db.repair()` 自愈、`db.clearAll()` 重置、迁移版本持久化(v0.4.2)
|
||||||
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
|
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
|
||||||
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
||||||
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
||||||
- 🧪 **894 测试 · 81.5% 覆盖率** — 47 套件,生产级质量保证
|
- 🧪 **944 测试 · 84.2% 覆盖率** — 50 套件,生产级质量保证
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -167,17 +169,24 @@ const hasOrders = await db.query('SELECT * FROM users u WHERE EXISTS (SELECT 1 F
|
|||||||
const labeled = await db.query("SELECT name, CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users");
|
const labeled = await db.query("SELECT name, CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users");
|
||||||
const joinExists = await db.query('SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 150)');
|
const joinExists = await db.query('SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 150)');
|
||||||
|
|
||||||
// v0.4.1 — 流式查询(大表逐行回调,不物化全部结果)
|
// v0.4.0 — 流式查询(大表逐行回调,不物化全部结果)
|
||||||
let count = 0;
|
let count = 0;
|
||||||
await db.queryStream('SELECT * FROM logs WHERE level = \'error\'', (row) => {
|
await db.queryStream('SELECT * FROM logs WHERE level = \'error\'', (row) => {
|
||||||
count++;
|
count++;
|
||||||
processRow(row);
|
processRow(row);
|
||||||
});
|
});
|
||||||
// v0.4.1 — 派生表 / 多列哈希连接 / COUNT(DISTINCT) / NULLS 排序
|
// v0.4.0 — 派生表 / 多列哈希连接 / COUNT(DISTINCT) / NULLS 排序
|
||||||
const top = await db.query('SELECT dept, total FROM (SELECT dept, SUM(salary) AS total FROM emp GROUP BY dept) AS t WHERE total > 100 ORDER BY total DESC');
|
const top = await db.query('SELECT dept, total FROM (SELECT dept, SUM(salary) AS total FROM emp GROUP BY dept) AS t WHERE total > 100 ORDER BY total DESC');
|
||||||
await db.query('SELECT COUNT(DISTINCT city) AS n FROM users');
|
await db.query('SELECT COUNT(DISTINCT city) AS n FROM users');
|
||||||
await db.query('SELECT name FROM users ORDER BY age ASC NULLS FIRST');
|
await db.query('SELECT name FROM users ORDER BY age ASC NULLS FIRST');
|
||||||
|
|
||||||
|
// v0.4.2 — 崩溃恢复自愈(无需删库重建)
|
||||||
|
await db.repair(); // 校验清理损坏数据,恢复一致性
|
||||||
|
await db.clearAll(); // 清空全部数据与表结构(保留库本身)
|
||||||
|
// v0.4.2 — 迁移版本持久化(重启后从持久化版本继续,不重跑不跳跑)
|
||||||
|
db.addMigration(1, async (d) => { /* ... */ });
|
||||||
|
await db.migrateTo(1);
|
||||||
|
|
||||||
// 事务 — 自动回滚 v0.1.13
|
// 事务 — 自动回滚 v0.1.13
|
||||||
await db.transaction(async (trx) => {
|
await db.transaction(async (trx) => {
|
||||||
await trx.table('users').insert({ id: '3', name: 'Charlie' });
|
await trx.table('users').insert({ id: '3', name: 'Charlie' });
|
||||||
@@ -218,8 +227,8 @@ await db2.disconnect(); // 引用计数 -1
|
|||||||
| `index` | `boolean` | 创建哈希索引 |
|
| `index` | `boolean` | 创建哈希索引 |
|
||||||
| `default` | `unknown` | 默认值 |
|
| `default` | `unknown` | 默认值 |
|
||||||
| `references` | `string` | 外键引用 `'table.column'` |
|
| `references` | `string` | 外键引用 `'table.column'` |
|
||||||
| `onDelete` | `'CASCADE'\|'SET NULL'\|'RESTRICT'` | 删除级联 🆕 |
|
| `onDelete` | `'CASCADE'\|'SET NULL'\|'RESTRICT'` | 删除级联 ✅ v0.4.1 |
|
||||||
| `onUpdate` | `'CASCADE'\|'SET NULL'\|'RESTRICT'` | 更新级联 🆕 |
|
| `onUpdate` | `'CASCADE'\|'SET NULL'\|'RESTRICT'` | 更新级联(更新主键时触发)✅ v0.4.2 |
|
||||||
|
|
||||||
### WHERE 操作符
|
### WHERE 操作符
|
||||||
|
|
||||||
@@ -239,9 +248,11 @@ await db2.disconnect(); // 引用计数 -1
|
|||||||
| `db.table(name)` | 获取表操作对象 |
|
| `db.table(name)` | 获取表操作对象 |
|
||||||
| `db.defineTable(name, cols)` | 定义表结构 |
|
| `db.defineTable(name, cols)` | 定义表结构 |
|
||||||
| `db.transaction(fn)` | 执行事务(自动回滚)🆕 |
|
| `db.transaction(fn)` | 执行事务(自动回滚)🆕 |
|
||||||
|
| `db.repair()` | 崩溃恢复自愈:校验清理损坏数据、恢复索引一致性(无需删库重建)🆕 v0.4.2 |
|
||||||
|
| `db.clearAll()` | 清空全部数据与表结构(保留库本身,实例可继续使用)🆕 v0.4.2 |
|
||||||
| `db.exportTable(name)` / `db.exportAll()` | 导出数据 JSON |
|
| `db.exportTable(name)` / `db.exportAll()` | 导出数据 JSON |
|
||||||
| `db.importTable(name, data)` | 导入数据 |
|
| `db.importTable(name, data)` | 导入数据 |
|
||||||
| `db.addMigration(v, fn)` / `db.migrateTo(v)` | 数据迁移 |
|
| `db.addMigration(v, fn)` / `db.migrateTo(v)` | 数据迁移(版本持久化到库内,重启不重跑)🆕 v0.4.2 |
|
||||||
| `db.subscribe(table, fn)` | 订阅表变更 |
|
| `db.subscribe(table, fn)` | 订阅表变更 |
|
||||||
| `db.on(hook, fn)` | 注册钩子 (14 种) |
|
| `db.on(hook, fn)` | 注册钩子 (14 种) |
|
||||||
|
|
||||||
@@ -272,14 +283,14 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
|||||||
|
|
||||||
| 特性 | Memory | Disk (IndexedDB) | Disk (OPFS) | Hybrid | Aria |
|
| 特性 | Memory | Disk (IndexedDB) | Disk (OPFS) | Hybrid | Aria |
|
||||||
|------|--------|------------------|-------------|--------|------|
|
|------|--------|------------------|-------------|--------|------|
|
||||||
| **持久化** | ❌ 重启丢失 | ✅ IndexedDB | ✅ OPFS文件系统 | ✅ 内存+磁盘 | ✅ 后端决定 |
|
| **持久化** | ❌ 重启丢失 | ✅ IndexedDB | ✅ OPFS(schema 持久化) | ✅ 内存+磁盘 | ✅ 后端决定 |
|
||||||
| **事务回滚** | ✅ 快照 | ✅ 原子flush | ✅ 快照 | ✅ 双引擎 | ✅ MVCC |
|
| **事务回滚** | ✅ 快照 | ✅ 原子flush | ✅ 快照 | ✅ 双引擎 | ✅ MVCC |
|
||||||
| **二级索引** | ✅ Hash | ✅ Hash | ✅ Hash | ✅ Hash | ✅ LSM |
|
| **二级索引** | ✅ Hash | ✅ Hash | ✅ Hash(重启恢复) | ✅ Hash | ✅ LSM(重启恢复) |
|
||||||
| **查询性能** | ⚡ O(1) PK | 🟡 O(1) PK | 🟡 O(1) PK | ⚡ O(1) PK | ⚡ O(log n) |
|
| **查询性能** | ⚡ O(1) PK | 🟡 O(1) PK | 🟡 O(1) PK | ⚡ O(1) PK | ⚡ O(log n) |
|
||||||
| **数据上限** | 内存限制 | ~2GB(IDB限制) | ~磁盘可用 | ~2GB(IDB) | 内存限制 |
|
| **数据上限** | 内存限制 | ~2GB(IDB限制) | ~磁盘可用(整表重写,≤1000行/表为宜) | ~2GB(IDB) | 内存限制 |
|
||||||
| **浏览器** | 全部 | 全部 | Chrome/Edge 102+ | 全部 | 全部 |
|
| **浏览器** | 全部 | 全部 | Chrome/Edge 102+ | 全部 | 全部 |
|
||||||
| **适用场景** | 缓存/测试 | 标准持久化 | Chromium专有 | 速度+持久化 | 大规模/分析 |
|
| **适用场景** | 缓存/测试 | 标准持久化 | Chromium专有 | 速度+持久化 | 大规模/分析 |
|
||||||
| **测试覆盖** | 30+ | 30+ | 12 | 15+ | 200+ |
|
| **测试覆盖** | 30+ | 30+ | 15 | 15+ | 200+ |
|
||||||
|
|
||||||
### Memory 模式
|
### Memory 模式
|
||||||
- **环境**: 所有浏览器、Node.js
|
- **环境**: 所有浏览器、Node.js
|
||||||
@@ -296,7 +307,7 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
|||||||
### Disk (OPFS) 模式
|
### Disk (OPFS) 模式
|
||||||
- **环境**: **仅限** Chrome 102+ / Edge 102+(Origin Private File System)
|
- **环境**: **仅限** Chrome 102+ / Edge 102+(Origin Private File System)
|
||||||
- **限制**: Firefox/Safari 不支持 OPFS API;每次写入重写整表 JSON 文件(大表性能差,不建议 >1000 行)
|
- **限制**: Firefox/Safari 不支持 OPFS API;每次写入重写整表 JSON 文件(大表性能差,不建议 >1000 行)
|
||||||
- **能力**: 完整 CRUD、重启自动加载数据、事务回滚
|
- **能力**: 完整 CRUD、重启自动加载数据(空表/索引/schema 完整保留,v0.4.2)、事务回滚、并发写安全(内存快照一致)
|
||||||
- **适用**: Chromium 独占场景、小数据集持久化
|
- **适用**: Chromium 独占场景、小数据集持久化
|
||||||
|
|
||||||
### Hybrid 模式
|
### Hybrid 模式
|
||||||
@@ -308,7 +319,7 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
|||||||
### Aria 模式
|
### Aria 模式
|
||||||
- **环境**: 所有浏览器(后端可选 IndexedDB / OPFS / Memory)
|
- **环境**: 所有浏览器(后端可选 IndexedDB / OPFS / Memory)
|
||||||
- **限制**: Memory 后端重启丢失;IndexedDB 后端受配额限制;OPFS 后端仅 Chromium
|
- **限制**: Memory 后端重启丢失;IndexedDB 后端受配额限制;OPFS 后端仅 Chromium
|
||||||
- **能力**: LSM-Tree 存储引擎、二级索引、MVCC 事务、WAL 崩溃恢复、Bloom Filter、AES-GCM 加密、Savepoint、EXPLAIN、ANALYZE、REINDEX、VACUUM
|
- **能力**: LSM-Tree 存储引擎、二级索引(跨重启恢复)、MVCC 事务、WAL 原子写入崩溃恢复(残缺 SSTable 打开自动跳过)、ON UPDATE/DELETE 外键级联、Bloom Filter、AES-GCM 加密、Savepoint、EXPLAIN、ANALYZE、REINDEX、VACUUM、`repair()` 自愈
|
||||||
- **适用**: 大规模数据分析、需要自研引擎可控性的高级场景
|
- **适用**: 大规模数据分析、需要自研引擎可控性的高级场景
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -358,12 +369,13 @@ const rows = await db.query('SELECT * FROM users');
|
|||||||
|
|
||||||
| 特性 | 说明 |
|
| 特性 | 说明 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| **LSM-Tree** | MemTable (红黑树) → SSTable 多级索引,异步 Compaction,写背压 |
|
| **LSM-Tree** | MemTable (红黑树) → SSTable 多级索引,异步 Compaction(从存储兜底加载,不依赖缓存),写背压 |
|
||||||
| **WAL** | Write-Ahead Log 二进制格式,CRC 校验,full/batch/none 三种模式(full 模式真正同步 ✅ v0.2.5),16MB 阈值自动 checkpoint |
|
| **WAL** | Write-Ahead Log 二进制格式,CRC 校验,记录与计数单事务原子写入,full/batch/none 三种模式(full 模式真正同步 ✅ v0.2.5),16MB 阈值自动 checkpoint(活跃事务期间不截断 ✅ v0.4.2) |
|
||||||
| **MVCC** | 版本链 + 快照隔离,事务读写不互斥,自动 GC(每10次检查点),读写路径接入版本链 ✅ v0.2.5 |
|
| **崩溃恢复** | 打开时完整性校验(残缺 SSTable 自动跳过并清理)、WAL 按 key 扫描恢复(不丢记录)、恢复后自动重建二级索引 ✅ v0.4.2 |
|
||||||
|
| **MVCC** | 版本链 + 快照隔离,事务读写不互斥,提交/回滚按事务写入 key 精准清理,自动 GC |
|
||||||
| **Buffer Pool** | SSTable 缓存 LRU 上限(`bufferPoolPages` × `pageSize`,默认 256 页 ≈ 1MB 可控内存)✅ v0.2.6 生效,查询前异步预加载兜底,缓存驱逐不丢数据 |
|
| **Buffer Pool** | SSTable 缓存 LRU 上限(`bufferPoolPages` × `pageSize`,默认 256 页 ≈ 1MB 可控内存)✅ v0.2.6 生效,查询前异步预加载兜底,缓存驱逐不丢数据 |
|
||||||
| **Bloom Filter** | FNV-1a + Murmur 双哈希,SSTable footer 序列化,查询时 probe |
|
| **Bloom Filter** | FNV-1a + Murmur 双哈希,SSTable footer 序列化,查询时 probe |
|
||||||
| **二级索引** | 每列独立 LSM Tree,支持 $eq/$in/$gt/$lt 范围扫描,SSTableReader 二分查找统一 ✅ v0.2.5 |
|
| **二级索引** | 每列独立 LSM Tree,支持 $eq/$in/$gt/$lt 范围扫描,跨重启自动恢复,WAL 恢复后自动重建 ✅ v0.4.2 |
|
||||||
| **AES-GCM** | PBKDF2 密钥派生 + AES-256-GCM 页面级加密,CryptoManager 实例化 ✅ v0.2.5 |
|
| **AES-GCM** | PBKDF2 密钥派生 + AES-256-GCM 页面级加密,CryptoManager 实例化 ✅ v0.2.5 |
|
||||||
| **Compaction** | 异步 Leveled Compaction,Level 0 > 8 触发同步背压,compactLevel public 接口 ✅ v0.2.5 |
|
| **Compaction** | 异步 Leveled Compaction,Level 0 > 8 触发同步背压,compactLevel public 接口 ✅ v0.2.5 |
|
||||||
| **OPFS Backend** | 纯浏览器文件系统,Promise 队列串行写,零外部依赖 |
|
| **OPFS Backend** | 纯浏览器文件系统,Promise 队列串行写,零外部依赖 |
|
||||||
@@ -387,9 +399,9 @@ npm run typecheck # 类型检查
|
|||||||
|
|
||||||
| 指标 | 数值 |
|
| 指标 | 数值 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| 测试用例 | 894 |
|
| 测试用例 | 944 |
|
||||||
| 测试套件 | 47 |
|
| 测试套件 | 51 |
|
||||||
| 行覆盖率 | 81.5% |
|
| 行覆盖率 | 84.2% |
|
||||||
| SQL 关键字 | 36 |
|
| SQL 关键字 | 36 |
|
||||||
| 存储引擎 | 5(Memory / IndexedDB / OPFS / Hybrid / **Aria**) |
|
| 存储引擎 | 5(Memory / IndexedDB / OPFS / Hybrid / **Aria**) |
|
||||||
|
|
||||||
|
|||||||
Vendored
+1230
-86
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+146
-32
@@ -118,7 +118,7 @@ interface MetonaPlugin {
|
|||||||
/** 销毁 */
|
/** 销毁 */
|
||||||
destroy(): void;
|
destroy(): void;
|
||||||
}
|
}
|
||||||
declare const VERSION = "0.4.1";
|
declare const VERSION = "0.4.2";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Plugin — 插件系统
|
* metona-sqlark Plugin — 插件系统
|
||||||
@@ -202,6 +202,14 @@ interface IStorageEngine {
|
|||||||
releaseSavepoint?(name: string): Promise<void>;
|
releaseSavepoint?(name: string): Promise<void>;
|
||||||
/** 在线备份:导出全库一致性快照 */
|
/** 在线备份:导出全库一致性快照 */
|
||||||
backup?(): Promise<Record<string, Record<string, unknown>[]>>;
|
backup?(): Promise<Record<string, Record<string, unknown>[]>>;
|
||||||
|
/** 崩溃恢复自愈:校验并清理损坏数据、恢复一致性(检测到异常后调用,无需删库重建) */
|
||||||
|
repair?(): Promise<void>;
|
||||||
|
/** 清空全部数据与表结构(保留库本身,供演示页刷新/重建用) */
|
||||||
|
clearAll?(): Promise<void>;
|
||||||
|
/** 读取库内元数据(迁移版本持久化用) */
|
||||||
|
getMeta?(key: string): Promise<string | null>;
|
||||||
|
/** 写入库内元数据(迁移版本持久化用) */
|
||||||
|
setMeta?(key: string, value: string): Promise<void>;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -651,6 +659,16 @@ declare class MetonaSqlark {
|
|||||||
addMigration(version: number, up: (db: MetonaSqlark) => Promise<void>): void;
|
addMigration(version: number, up: (db: MetonaSqlark) => Promise<void>): void;
|
||||||
/** 执行迁移到指定版本 */
|
/** 执行迁移到指定版本 */
|
||||||
migrateTo(targetVersion: number): Promise<void>;
|
migrateTo(targetVersion: number): Promise<void>;
|
||||||
|
/**
|
||||||
|
* 崩溃恢复自愈 — 校验并清理损坏数据、恢复一致性。
|
||||||
|
* 检测到异常后调用,无需删库重建。
|
||||||
|
*/
|
||||||
|
repair(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* 清空全部数据与表结构(保留库本身)。
|
||||||
|
* 支持后续继续使用本实例重建表。
|
||||||
|
*/
|
||||||
|
clearAll(): Promise<void>;
|
||||||
/** 获取插件管理器 */
|
/** 获取插件管理器 */
|
||||||
getPluginManager(): PluginManager;
|
getPluginManager(): PluginManager;
|
||||||
/** 注册钩子 */
|
/** 注册钩子 */
|
||||||
@@ -667,31 +685,47 @@ declare class MetonaSqlark {
|
|||||||
private _debug;
|
private _debug;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* metona-sqlark Memory Engine — 基于 Map 的内存存储引擎
|
|
||||||
* @module engine/memory
|
|
||||||
*/
|
|
||||||
|
|
||||||
declare class MemoryEngine implements IStorageEngine {
|
declare class MemoryEngine implements IStorageEngine {
|
||||||
readonly name = "memory";
|
readonly name = "memory";
|
||||||
private tables;
|
private tables;
|
||||||
private schemas;
|
private schemas;
|
||||||
private indexes;
|
private indexes;
|
||||||
private opened;
|
private opened;
|
||||||
|
/** v0.4.2-fix: 库内元数据(迁移版本持久化用) */
|
||||||
|
private metaStore;
|
||||||
private snapshot;
|
private snapshot;
|
||||||
open(_dbName: string, _version: number): Promise<void>;
|
open(_dbName: string, _version: number): Promise<void>;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
|
/** 内存引擎无需修复(无持久化损坏概念) */
|
||||||
|
repair(): Promise<void>;
|
||||||
|
/** 清空全部数据与表结构 */
|
||||||
|
clearAll(): Promise<void>;
|
||||||
|
getMeta(key: string): Promise<string | null>;
|
||||||
|
setMeta(key: string, value: string): Promise<void>;
|
||||||
createTable(schema: TableSchema): Promise<void>;
|
createTable(schema: TableSchema): Promise<void>;
|
||||||
dropTable(tableName: string): Promise<void>;
|
dropTable(tableName: string): Promise<void>;
|
||||||
hasTable(tableName: string): Promise<boolean>;
|
hasTable(tableName: string): Promise<boolean>;
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 引擎级 ALTER TABLE — 直接修改内存 schema 引用并清理行数据。
|
||||||
|
* (此前走 executor 通用路径,行为相同;统一到引擎层保证 Hybrid/IndexedDB 委托一致性)
|
||||||
|
*/
|
||||||
|
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
||||||
|
name: string;
|
||||||
|
}): Promise<void>;
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
||||||
/** v0.4.0: 流式查询 — 逐行回调(单次迭代,不物化结果数组) */
|
/** v0.4.0: 流式查询 — 逐行回调(单次迭代,不物化结果数组) */
|
||||||
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
|
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
|
||||||
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: ON UPDATE 外键级联 — 被引用表主键变更时处理引用表:
|
||||||
|
* RESTRICT 抛错 / CASCADE 更新 FK 值 / SET NULL 置空。
|
||||||
|
* 分两阶段:先全量 RESTRICT 检查(任何修改前),再执行级联(防部分修改)。
|
||||||
|
*/
|
||||||
|
private applyUpdateCascade;
|
||||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||||
clear(tableName: string): Promise<void>;
|
clear(tableName: string): Promise<void>;
|
||||||
@@ -721,13 +755,6 @@ declare class MemoryEngine implements IStorageEngine {
|
|||||||
private cascadeDelete;
|
private cascadeDelete;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* metona-sqlark IndexedDB Engine — 基于 IndexedDB 的持久化存储引擎
|
|
||||||
* @module engine/indexeddb
|
|
||||||
*
|
|
||||||
* v0.1.13: 支持事务 — beginTransaction 延迟 IDB 写入,commit 批量刷盘,rollback 恢复快照。
|
|
||||||
*/
|
|
||||||
|
|
||||||
declare class IndexedDBEngine implements IStorageEngine {
|
declare class IndexedDBEngine implements IStorageEngine {
|
||||||
readonly name = "indexeddb";
|
readonly name = "indexeddb";
|
||||||
private db;
|
private db;
|
||||||
@@ -736,6 +763,29 @@ declare class IndexedDBEngine implements IStorageEngine {
|
|||||||
private memoryCache;
|
private memoryCache;
|
||||||
private txActive;
|
private txActive;
|
||||||
open(dbName: string, version: number): Promise<void>;
|
open(dbName: string, version: number): Promise<void>;
|
||||||
|
/** v0.4.2-fix: 多标签页冲突处理 — 其他标签页升级版本时自动关闭当前连接 */
|
||||||
|
private setupVersionChangeHandler;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 打开 IndexedDB 连接。
|
||||||
|
* - P0-2: 请求版本低于库实际版本(VersionError)时,先无版本参数探测库当前版本,
|
||||||
|
* 再以实际版本重开(建表每张表版本号 +1,config.version 会过期)
|
||||||
|
* - P2-8: onblocked 为瞬时状态(另一连接短暂持有),等待后重试多次,超时才抛 IDB_BLOCKED
|
||||||
|
*/
|
||||||
|
private openDatabaseWithRetry;
|
||||||
|
/**
|
||||||
|
* 发起一次 indexedDB.open 请求(success/error/blocked 三态收敛)。
|
||||||
|
* onblocked 不立即失败:阻塞解除后 success 仍会触发,仅超时兜底判失败,
|
||||||
|
* 避免"拒绝后连接迟到成功"泄漏未关闭的数据库连接。
|
||||||
|
*/
|
||||||
|
private openRequest;
|
||||||
|
/** 无版本参数打开库,解析其当前实际版本号(随后立即关闭) */
|
||||||
|
private resolveCurrentVersion;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix (P2-7): 确保 __metona_schema store 存在。
|
||||||
|
* 新库(或版本升级前创建的旧库)没有该 store 时,通过一次版本升级创建,
|
||||||
|
* 使 getMeta/setMeta(迁移版本持久化)始终可用。
|
||||||
|
*/
|
||||||
|
private ensureSchemaStore;
|
||||||
/**
|
/**
|
||||||
* 从 IDB 恢复内存 schema:
|
* 从 IDB 恢复内存 schema:
|
||||||
* 1. 优先读取持久化的 schema 记录('__metona_schema' store,v0.3.2)
|
* 1. 优先读取持久化的 schema 记录('__metona_schema' store,v0.3.2)
|
||||||
@@ -743,12 +793,31 @@ declare class IndexedDBEngine implements IStorageEngine {
|
|||||||
*/
|
*/
|
||||||
private rebuildSchemaFromIDB;
|
private rebuildSchemaFromIDB;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 自愈 — 从磁盘重建内存 schema 与数据(schema 丢失/内存不一致时调用)。
|
||||||
|
* 无删库需求即可恢复可用的库。
|
||||||
|
*/
|
||||||
|
repair(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 清空全部数据与表结构(含持久化 schema 记录),保留库本身。
|
||||||
|
* 单个版本升级事务内原子完成。
|
||||||
|
*/
|
||||||
|
clearAll(): Promise<void>;
|
||||||
|
getMeta(key: string): Promise<string | null>;
|
||||||
|
setMeta(key: string, value: string): Promise<void>;
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
createTable(schema: TableSchema): Promise<void>;
|
createTable(schema: TableSchema): Promise<void>;
|
||||||
dropTable(tableName: string): Promise<void>;
|
dropTable(tableName: string): Promise<void>;
|
||||||
hasTable(tableName: string): Promise<boolean>;
|
hasTable(tableName: string): Promise<boolean>;
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 引擎级 ALTER TABLE — schema 持久化到 __metona_schema store,
|
||||||
|
* 重启后 ALTER 不丢失(此前通用路径只改内存引用,重启回退;DROP 的行数据也没真正删)。
|
||||||
|
*/
|
||||||
|
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
||||||
|
name: string;
|
||||||
|
}): Promise<void>;
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
||||||
/** v0.4.0: 流式查询 — IDB 批量读入后逐行回调(保持接口一致性) */
|
/** v0.4.0: 流式查询 — IDB 批量读入后逐行回调(保持接口一致性) */
|
||||||
@@ -771,19 +840,13 @@ declare class IndexedDBEngine implements IStorageEngine {
|
|||||||
private idbUpdate;
|
private idbUpdate;
|
||||||
private idbDelete;
|
private idbDelete;
|
||||||
private idbClear;
|
private idbClear;
|
||||||
|
/** 持久化单个表 schema 到 __metona_schema store(v0.4.2-fix: ALTER TABLE 用) */
|
||||||
|
private persistSchema;
|
||||||
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
|
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
|
||||||
private flushToIDB;
|
private flushToIDB;
|
||||||
private ensureDB;
|
private ensureDB;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* metona-sqlark OPFS Engine — 基于 Origin Private File System 的持久化存储引擎
|
|
||||||
* @module engine/opfs
|
|
||||||
*
|
|
||||||
* 使用 JSON-per-table 文件存储方案。
|
|
||||||
* 目录结构:{dbName}/tables/{tableName}.json
|
|
||||||
*/
|
|
||||||
|
|
||||||
declare class OPFSEngine implements IStorageEngine {
|
declare class OPFSEngine implements IStorageEngine {
|
||||||
readonly name = "opfs";
|
readonly name = "opfs";
|
||||||
private root;
|
private root;
|
||||||
@@ -793,11 +856,21 @@ declare class OPFSEngine implements IStorageEngine {
|
|||||||
open(dbName: string, version: number): Promise<void>;
|
open(dbName: string, version: number): Promise<void>;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
|
/** 自愈:重置内存缓存后从 OPFS 重新加载(单文件损坏不影响其他表) */
|
||||||
|
repair(): Promise<void>;
|
||||||
|
/** 清空全部数据与表结构(删除目录内全部文件) */
|
||||||
|
clearAll(): Promise<void>;
|
||||||
|
getMeta(key: string): Promise<string | null>;
|
||||||
|
setMeta(key: string, value: string): Promise<void>;
|
||||||
createTable(schema: TableSchema): Promise<void>;
|
createTable(schema: TableSchema): Promise<void>;
|
||||||
dropTable(tableName: string): Promise<void>;
|
dropTable(tableName: string): Promise<void>;
|
||||||
hasTable(tableName: string): Promise<boolean>;
|
hasTable(tableName: string): Promise<boolean>;
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
||||||
|
/** v0.4.2-fix: 引擎级 ALTER TABLE — 内存 + schema 持久化 + 整表文件重写 */
|
||||||
|
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
||||||
|
name: string;
|
||||||
|
}): Promise<void>;
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
||||||
/** v0.4.0: 流式查询(委托内存缓存) */
|
/** v0.4.0: 流式查询(委托内存缓存) */
|
||||||
@@ -814,7 +887,11 @@ declare class OPFSEngine implements IStorageEngine {
|
|||||||
private ensureDir;
|
private ensureDir;
|
||||||
private writeTableData;
|
private writeTableData;
|
||||||
private readTableData;
|
private readTableData;
|
||||||
/** 从 OPFS 加载已有表数据到内存缓存 */
|
/**
|
||||||
|
* 从 OPFS 加载已有表到内存缓存。
|
||||||
|
* v0.4.2-fix: 优先从持久化 schema(__metona_schema_*.meta)恢复 —
|
||||||
|
* 空表不再消失、索引标记/主键/约束完整;无 schema 记录的旧库从数据推断(兼容)。
|
||||||
|
*/
|
||||||
private loadExistingTables;
|
private loadExistingTables;
|
||||||
/** 从 OPFS 加载表数据到内存缓存 */
|
/** 从 OPFS 加载表数据到内存缓存 */
|
||||||
loadTableIntoMemory(tableName: string, schema: TableSchema): Promise<void>;
|
loadTableIntoMemory(tableName: string, schema: TableSchema): Promise<void>;
|
||||||
@@ -867,21 +944,41 @@ declare class AriaEngine implements IStorageEngine {
|
|||||||
private bufferPool;
|
private bufferPool;
|
||||||
constructor(config?: AriaEngineConfig);
|
constructor(config?: AriaEngineConfig);
|
||||||
open(dbName: string, _version: number): Promise<void>;
|
open(dbName: string, _version: number): Promise<void>;
|
||||||
|
/** open 内部实现(错误包装在 open 外层) */
|
||||||
|
private openInternal;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 崩溃恢复/自愈 — 校验并移除损坏 SSTable、截断 WAL、重建二级索引。
|
||||||
|
* 应用层检测到异常后调用,无需删库重建。
|
||||||
|
*/
|
||||||
|
repair(): Promise<void>;
|
||||||
/**
|
/**
|
||||||
* v0.4.1: 重置数据库 — 清空全部数据与表结构(演示页刷新/重新初始化用)。
|
* v0.4.1: 重置数据库 — 清空全部数据与表结构(演示页刷新/重新初始化用)。
|
||||||
* 清空存储后端、LSM、WAL、MVCC 与二级索引,后续可继续使用本实例。
|
* 清空存储后端、LSM、WAL、MVCC 与二级索引,后续可继续使用本实例。
|
||||||
*/
|
*/
|
||||||
clearAll(): Promise<void>;
|
clearAll(): Promise<void>;
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
|
getMeta(key: string): Promise<string | null>;
|
||||||
|
setMeta(key: string, value: string): Promise<void>;
|
||||||
createTable(schema: TableSchema): Promise<void>;
|
createTable(schema: TableSchema): Promise<void>;
|
||||||
dropTable(tableName: string): Promise<void>;
|
dropTable(tableName: string): Promise<void>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 清理指定表的全部二级索引 LSM(内存 + 存储文件 + meta)。
|
||||||
|
* dropTable / DROP_TABLE 恢复 / alterTable DROP 索引列 共用。
|
||||||
|
*/
|
||||||
|
private cleanupTableIndexes;
|
||||||
hasTable(tableName: string): Promise<boolean>;
|
hasTable(tableName: string): Promise<boolean>;
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
||||||
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: ON UPDATE 外键级联 — 主键 oldPk → newPk 时处理引用表。
|
||||||
|
* RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空(含索引与 WAL 记录)。
|
||||||
|
* 两阶段:先全量 RESTRICT 检查,再执行级联。
|
||||||
|
*/
|
||||||
|
private applyForeignKeyUpdateRules;
|
||||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||||
/**
|
/**
|
||||||
* v0.4.1: 外键级联规则 — 对齐 MemoryEngine.cascadeDelete 行为。
|
* v0.4.1: 外键级联规则 — 对齐 MemoryEngine.cascadeDelete 行为。
|
||||||
@@ -970,6 +1067,8 @@ declare class AriaEngine implements IStorageEngine {
|
|||||||
* REINDEX: 重建指定表的所有二级索引
|
* REINDEX: 重建指定表的所有二级索引
|
||||||
*/
|
*/
|
||||||
reindexTable(tableName: string): Promise<number>;
|
reindexTable(tableName: string): Promise<number>;
|
||||||
|
/** v0.4.2-fix: 重建索引内部实现(不校验 opened,供 open 恢复流程调用) */
|
||||||
|
private reindexTableInternal;
|
||||||
/**
|
/**
|
||||||
* VACUUM: 压缩 LSM + 清理碎片
|
* VACUUM: 压缩 LSM + 清理碎片
|
||||||
*/
|
*/
|
||||||
@@ -985,6 +1084,8 @@ declare class AriaEngine implements IStorageEngine {
|
|||||||
estimatedRows: number;
|
estimatedRows: number;
|
||||||
};
|
};
|
||||||
private ensureOpen;
|
private ensureOpen;
|
||||||
|
/** v0.4.2-fix: Aria 事务中 DDL 显式拒绝(结构变更无法通过行快照回滚) */
|
||||||
|
private ensureNoDDLInTransaction;
|
||||||
private ensureTable;
|
private ensureTable;
|
||||||
/** Get the number of WAL records stored */
|
/** Get the number of WAL records stored */
|
||||||
private getWALCount;
|
private getWALCount;
|
||||||
@@ -992,16 +1093,6 @@ declare class AriaEngine implements IStorageEngine {
|
|||||||
private setWALCount;
|
private setWALCount;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* metona-sqlark Hybrid Engine — 内存 + 磁盘混合存储引擎
|
|
||||||
* @module hybrid/index
|
|
||||||
*
|
|
||||||
* 采用 write-through 策略:
|
|
||||||
* - 所有写操作同时写入内存和磁盘
|
|
||||||
* - 所有读操作直接从内存返回
|
|
||||||
* - 数据库打开时从磁盘加载数据到内存
|
|
||||||
*/
|
|
||||||
|
|
||||||
declare class HybridEngine implements IStorageEngine {
|
declare class HybridEngine implements IStorageEngine {
|
||||||
readonly name = "hybrid";
|
readonly name = "hybrid";
|
||||||
private memoryEngine;
|
private memoryEngine;
|
||||||
@@ -1018,11 +1109,21 @@ declare class HybridEngine implements IStorageEngine {
|
|||||||
reloadMemoryFromDisk(): Promise<void>;
|
reloadMemoryFromDisk(): Promise<void>;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
|
/** 自愈:修复磁盘引擎后重载内存缓存 */
|
||||||
|
repair(): Promise<void>;
|
||||||
|
/** 清空全部数据与表结构 */
|
||||||
|
clearAll(): Promise<void>;
|
||||||
|
getMeta(key: string): Promise<string | null>;
|
||||||
|
setMeta(key: string, value: string): Promise<void>;
|
||||||
createTable(schema: TableSchema): Promise<void>;
|
createTable(schema: TableSchema): Promise<void>;
|
||||||
dropTable(tableName: string): Promise<void>;
|
dropTable(tableName: string): Promise<void>;
|
||||||
hasTable(tableName: string): Promise<boolean>;
|
hasTable(tableName: string): Promise<boolean>;
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
||||||
|
/** v0.4.2-fix: 引擎级 ALTER TABLE — 双引擎同步(磁盘持久化 + 内存引用) */
|
||||||
|
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
||||||
|
name: string;
|
||||||
|
}): Promise<void>;
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
||||||
/** v0.4.0: 流式查询(内存引擎逐行回调) */
|
/** v0.4.0: 流式查询(内存引擎逐行回调) */
|
||||||
@@ -1177,8 +1278,17 @@ interface IStorageBackend {
|
|||||||
read(key: string): Promise<ArrayBuffer | null>;
|
read(key: string): Promise<ArrayBuffer | null>;
|
||||||
/** 写入数据块 */
|
/** 写入数据块 */
|
||||||
write(key: string, data: ArrayBuffer): Promise<void>;
|
write(key: string, data: ArrayBuffer): Promise<void>;
|
||||||
|
/**
|
||||||
|
* 批量原子写入(v0.4.2-fix):多个 key 在单个底层事务中提交,
|
||||||
|
* 中断时整体回滚,不留半写状态。WAL count 与记录同事务保证一致性。
|
||||||
|
*/
|
||||||
|
writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
|
||||||
/** 删除数据块 */
|
/** 删除数据块 */
|
||||||
delete(key: string): Promise<void>;
|
delete(key: string): Promise<void>;
|
||||||
|
/**
|
||||||
|
* 批量原子删除(v0.4.2-fix):多个 key 在单个底层事务中提交。
|
||||||
|
*/
|
||||||
|
deleteMany(keys: string[]): Promise<void>;
|
||||||
/** 列出所有 key */
|
/** 列出所有 key */
|
||||||
listKeys(): Promise<string[]>;
|
listKeys(): Promise<string[]>;
|
||||||
/** 检查 key 是否存在 */
|
/** 检查 key 是否存在 */
|
||||||
@@ -1207,7 +1317,11 @@ declare class OPFSBackend implements IStorageBackend {
|
|||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
read(key: string): Promise<ArrayBuffer | null>;
|
read(key: string): Promise<ArrayBuffer | null>;
|
||||||
write(key: string, data: ArrayBuffer): Promise<void>;
|
write(key: string, data: ArrayBuffer): Promise<void>;
|
||||||
|
/** v0.4.2-fix: 批量写入 — 串行队列内逐个落盘(OPFS 无跨文件事务,顺序保证一致) */
|
||||||
|
writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
|
||||||
delete(key: string): Promise<void>;
|
delete(key: string): Promise<void>;
|
||||||
|
/** v0.4.2-fix: 批量删除 — 串行队列内逐个删除 */
|
||||||
|
deleteMany(keys: string[]): Promise<void>;
|
||||||
listKeys(): Promise<string[]>;
|
listKeys(): Promise<string[]>;
|
||||||
exists(key: string): Promise<boolean>;
|
exists(key: string): Promise<boolean>;
|
||||||
clear(): Promise<void>;
|
clear(): Promise<void>;
|
||||||
|
|||||||
Vendored
+1230
-86
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1230
-86
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@metona-team/metona-sqlark",
|
"name": "@metona-team/metona-sqlark",
|
||||||
"version": "0.4.1",
|
"version": "0.4.2",
|
||||||
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "dist/metona-sqlark.cjs",
|
"main": "dist/metona-sqlark.cjs",
|
||||||
|
|||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>⚡ 性能基准 — MetonaSqlark v0.4.1</title>
|
<title>⚡ 性能基准 — MetonaSqlark v0.4.2</title>
|
||||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||||
<style>
|
<style>
|
||||||
:root {
|
:root {
|
||||||
|
|||||||
+68
-20
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>🧪 在线演示 — MetonaSqlark v0.4.1</title>
|
<title>🧪 在线演示 — MetonaSqlark v0.4.2</title>
|
||||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||||
<style>
|
<style>
|
||||||
:root {
|
:root {
|
||||||
@@ -84,7 +84,7 @@
|
|||||||
<a href="demo.html" class="nav-active">演示</a>
|
<a href="demo.html" class="nav-active">演示</a>
|
||||||
<a href="benchmark.html">基准</a>
|
<a href="benchmark.html">基准</a>
|
||||||
</nav>
|
</nav>
|
||||||
<div class="status"><span class="dot" id="engine-dot"></span> <span id="engine-status">Memory</span> 模式 — v0.4.1</div>
|
<div class="status"><span class="dot" id="engine-dot"></span> <span id="engine-status">Memory</span> 模式 — v0.4.2</div>
|
||||||
<button class="btn btn-preset" onclick="switchEngine('memory')" id="btn-memory" style="margin:6px 4px 6px 0;padding:6px 12px;">⚡ Memory</button>
|
<button class="btn btn-preset" onclick="switchEngine('memory')" id="btn-memory" style="margin:6px 4px 6px 0;padding:6px 12px;">⚡ Memory</button>
|
||||||
<button class="btn btn-preset" onclick="switchEngine('aria')" id="btn-aria" style="margin:6px 0;padding:6px 12px;color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
<button class="btn btn-preset" onclick="switchEngine('aria')" id="btn-aria" style="margin:6px 0;padding:6px 12px;color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
||||||
</header>
|
</header>
|
||||||
@@ -92,7 +92,7 @@
|
|||||||
<div class="main">
|
<div class="main">
|
||||||
<div class="editor-panel">
|
<div class="editor-panel">
|
||||||
<div class="editor-area">
|
<div class="editor-area">
|
||||||
<textarea id="sql-input" placeholder="输入 SQL 语句... SELECT * FROM users; INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28); SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.4.1 在线演示
|
<textarea id="sql-input" placeholder="输入 SQL 语句... SELECT * FROM users; INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28); SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.4.2 在线演示
|
||||||
-- 已预置 users / orders / products 表数据
|
-- 已预置 users / orders / products 表数据
|
||||||
-- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护
|
-- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护
|
||||||
|
|
||||||
@@ -131,6 +131,8 @@
|
|||||||
<button class="btn btn-preset" onclick="loadPreset('index')">🗂 索引</button>
|
<button class="btn btn-preset" onclick="loadPreset('index')">🗂 索引</button>
|
||||||
<button class="btn btn-preset" onclick="loadPreset('multistmt')">📜 多语句/事务</button>
|
<button class="btn btn-preset" onclick="loadPreset('multistmt')">📜 多语句/事务</button>
|
||||||
<button class="btn btn-preset" onclick="loadPreset('v040')" style="color:#22c55e;border-color:#22c55e;">🚰 v0.4.0 新特性</button>
|
<button class="btn btn-preset" onclick="loadPreset('v040')" style="color:#22c55e;border-color:#22c55e;">🚰 v0.4.0 新特性</button>
|
||||||
|
<button class="btn btn-preset" onclick="loadPreset('onupdate')" style="color:#fbbf24;border-color:#fbbf24;">🔄 ON UPDATE</button>
|
||||||
|
<button class="btn btn-preset" onclick="loadPreset('repair')" style="color:#22c55e;border-color:#22c55e;">🛡 自愈</button>
|
||||||
<button class="btn btn-preset" onclick="loadPreset('aria')" style="color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
<button class="btn btn-preset" onclick="loadPreset('aria')" style="color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -174,7 +176,7 @@ async function initDB() {
|
|||||||
db = new DBClass({ name: 'demo', mode: engine });
|
db = new DBClass({ name: 'demo', mode: engine });
|
||||||
await db.init();
|
await db.init();
|
||||||
|
|
||||||
// v0.4.1: Aria 引擎持久化 — 每次加载清空上次演示数据,保证演示确定性
|
// v0.4.2: Aria 引擎持久化 — 每次加载清空上次演示数据,保证演示确定性
|
||||||
if (engine === 'aria' && typeof db.getEngine().clearAll === 'function') {
|
if (engine === 'aria' && typeof db.getEngine().clearAll === 'function') {
|
||||||
await db.getEngine().clearAll();
|
await db.getEngine().clearAll();
|
||||||
}
|
}
|
||||||
@@ -230,7 +232,7 @@ async function seedDemoData(db) {
|
|||||||
]);
|
]);
|
||||||
}
|
}
|
||||||
|
|
||||||
// v0.4.1: 切换存储引擎(重建数据库实例)
|
// v0.4.2: 切换存储引擎(重建数据库实例)
|
||||||
async function switchEngine(engine) {
|
async function switchEngine(engine) {
|
||||||
if (engine === currentEngine) return;
|
if (engine === currentEngine) return;
|
||||||
currentEngine = engine;
|
currentEngine = engine;
|
||||||
@@ -552,7 +554,7 @@ SELECT COUNT(*) as total FROM temp_logs;
|
|||||||
|
|
||||||
-- 清理
|
-- 清理
|
||||||
DROP TABLE temp_logs;`,
|
DROP TABLE temp_logs;`,
|
||||||
aria: `-- 🌲 AriaEngine 演示 (v0.4.1)
|
aria: `-- 🌲 AriaEngine 演示 (v0.4.2)
|
||||||
-- 点击右上角「🌲 Aria」按钮切换数据库引擎到 AriaEngine
|
-- 点击右上角「🌲 Aria」按钮切换数据库引擎到 AriaEngine
|
||||||
-- 当前数据库即运行在 Aria 引擎上(LSM-Tree · WAL 崩溃恢复 · MVCC · BloomFilter)
|
-- 当前数据库即运行在 Aria 引擎上(LSM-Tree · WAL 崩溃恢复 · MVCC · BloomFilter)
|
||||||
-- 基础 CRUD 与 Memory 引擎完全兼容
|
-- 基础 CRUD 与 Memory 引擎完全兼容
|
||||||
@@ -587,12 +589,13 @@ DROP TABLE temp_logs;`,
|
|||||||
|
|
||||||
-- AriaEngine 特性:
|
-- AriaEngine 特性:
|
||||||
-- • LSM-Tree: MemTable (红黑树) → SSTable 多级索引
|
-- • LSM-Tree: MemTable (红黑树) → SSTable 多级索引
|
||||||
-- • WAL: Write-Ahead Log 保证崩溃恢复 + 批量组提交
|
-- • WAL: 原子写入 + 崩溃恢复(残缺 SSTable 打开自动跳过)+ 批量组提交
|
||||||
-- • MVCC: 版本链 + 快照隔离
|
-- • MVCC: 版本链 + 快照隔离
|
||||||
-- • Buffer Pool: SSTable LRU 缓存 (256页 ~ 1MB) ✅ 已生效
|
-- • Buffer Pool: SSTable LRU 缓存 (256页 ~ 1MB) ✅ 已生效
|
||||||
-- • Bloom Filter: FNV-1a + Murmur 双哈希
|
-- • Bloom Filter: FNV-1a + Murmur 双哈希
|
||||||
-- • 二级索引: 每列独立 LSM + 动态 CREATE INDEX
|
-- • 二级索引: 每列独立 LSM + 跨重启自动恢复
|
||||||
-- • 外键级联: CASCADE / SET NULL / RESTRICT (v0.4.1)
|
-- • 外键级联: ON DELETE / ON UPDATE — CASCADE / SET NULL / RESTRICT (v0.4.2)
|
||||||
|
-- • 自愈: db.repair() 无需删库重建 (v0.4.2)
|
||||||
|
|
||||||
-- 生产环境 API(与演示页右上角切换等价)
|
-- 生产环境 API(与演示页右上角切换等价)
|
||||||
-- const db = await MetonaSqlark.create({
|
-- const db = await MetonaSqlark.create({
|
||||||
@@ -674,19 +677,64 @@ WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 10
|
|||||||
-- 为 orders.user_id 创建索引(已有数据自动回填)
|
-- 为 orders.user_id 创建索引(已有数据自动回填)
|
||||||
CREATE INDEX idx_orders_user ON orders (user_id);
|
CREATE INDEX idx_orders_user ON orders (user_id);
|
||||||
|
|
||||||
-- 索引查找(v0.4.1: JOIN 主表 WHERE 条件下推到引擎,真正走二级索引)
|
-- 索引查找(v0.4.2: JOIN 主表 WHERE 条件下推 + 二级索引跨重启恢复)
|
||||||
SELECT u.name, o.product, o.amount
|
SELECT u.name, o.product, o.amount
|
||||||
FROM orders o JOIN users u ON u.id = o.user_id
|
FROM orders o JOIN users u ON u.id = o.user_id
|
||||||
WHERE o.user_id = '1';
|
WHERE o.user_id = '1';
|
||||||
|
|
||||||
-- 删除索引
|
-- 删除索引
|
||||||
DROP INDEX idx_orders_user ON orders (user_id);
|
DROP INDEX idx_orders_user ON orders (user_id);
|
||||||
|
|
||||||
-- 删除后回退全表扫描(结果不变)
|
-- 删除后回退全表扫描(结果不变)
|
||||||
SELECT * FROM orders WHERE user_id = '3';
|
SELECT * FROM orders WHERE user_id = '3';
|
||||||
|
|
||||||
-- DROP 不存在的索引会报错(INDEX_NOT_FOUND)
|
-- DROP 不存在的索引会报错(INDEX_NOT_FOUND)
|
||||||
-- DROP INDEX idx_nonexist ON orders (user_id);`,
|
-- DROP INDEX idx_nonexist ON orders (user_id);`,
|
||||||
|
onupdate: `-- 🔄 ON UPDATE 外键级联 (v0.4.2)
|
||||||
|
-- 更新父表主键 → 子表外键自动级联(CASCADE / SET NULL / RESTRICT)
|
||||||
|
|
||||||
|
-- 建带 onUpdate 外键的表
|
||||||
|
DROP TABLE IF EXISTS accounts;
|
||||||
|
DROP TABLE IF EXISTS audit_log;
|
||||||
|
CREATE TABLE accounts (id STRING PRIMARY KEY, owner STRING);
|
||||||
|
CREATE TABLE audit_log (
|
||||||
|
id STRING PRIMARY KEY,
|
||||||
|
account_id STRING REFERENCES accounts.id ON UPDATE CASCADE
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 种子数据
|
||||||
|
INSERT INTO accounts VALUES ('a1', 'Alice');
|
||||||
|
INSERT INTO audit_log VALUES ('l1', 'a1');
|
||||||
|
INSERT INTO audit_log VALUES ('l2', 'a1');
|
||||||
|
|
||||||
|
-- 更新父表主键 a1 → a2
|
||||||
|
UPDATE accounts SET id = 'a2' WHERE id = 'a1';
|
||||||
|
|
||||||
|
-- 子表外键已级联为 a2
|
||||||
|
SELECT * FROM audit_log ORDER BY id;
|
||||||
|
|
||||||
|
-- 再按新主键反查
|
||||||
|
SELECT * FROM accounts WHERE id = 'a2';`,
|
||||||
|
repair: `-- 🛡 崩溃恢复自愈 (v0.4.2)
|
||||||
|
-- db.repair(): 校验并清理损坏 SSTable / 重建二级索引 / 截断 WAL
|
||||||
|
-- db.clearAll(): 清空全部数据与表结构(保留库本身)
|
||||||
|
-- 打开数据库时自动跳过残缺 SSTable,无需删库重建
|
||||||
|
|
||||||
|
-- 数据写入
|
||||||
|
CREATE TABLE IF NOT EXISTS notes (id STRING PRIMARY KEY, body STRING);
|
||||||
|
INSERT INTO notes VALUES ('n1', 'Hello');
|
||||||
|
INSERT INTO notes VALUES ('n2', 'World');
|
||||||
|
|
||||||
|
-- 模拟数据(正常数据)
|
||||||
|
SELECT * FROM notes ORDER BY id;
|
||||||
|
|
||||||
|
-- API 自愈(控制台执行):
|
||||||
|
-- await db.repair(); → 校验+清理损坏文件,重建索引
|
||||||
|
-- await db.clearAll(); → 清空全部表与数据
|
||||||
|
|
||||||
|
-- 迁移版本持久化(重启后不重跑已执行迁移)
|
||||||
|
-- db.addMigration(1, async () => { ... });
|
||||||
|
-- await db.migrateTo(1);`,
|
||||||
multistmt: `-- 📜 多语句 + 事务语句 (v0.3.0)
|
multistmt: `-- 📜 多语句 + 事务语句 (v0.3.0)
|
||||||
|
|
||||||
-- 分号分隔的多语句一次执行
|
-- 分号分隔的多语句一次执行
|
||||||
@@ -753,7 +801,7 @@ document.addEventListener('keydown', e => {
|
|||||||
document.getElementById('btn-aria').style.opacity = '0.6';
|
document.getElementById('btn-aria').style.opacity = '0.6';
|
||||||
document.getElementById('engine-status').textContent = '⚡ Memory';
|
document.getElementById('engine-status').textContent = '⚡ Memory';
|
||||||
initDB().then(() => {
|
initDB().then(() => {
|
||||||
console.log('✅ MetonaSqlark v0.4.1 demo ready');
|
console.log('✅ MetonaSqlark v0.4.2 demo ready');
|
||||||
setTimeout(runQuery, 300);
|
setTimeout(runQuery, 300);
|
||||||
}).catch(err => {
|
}).catch(err => {
|
||||||
renderError('初始化失败: ' + err.message);
|
renderError('初始化失败: ' + err.message);
|
||||||
|
|||||||
+18
-4
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>📖 API 文档 — MetonaSqlark v0.4.1</title>
|
<title>📖 API 文档 — MetonaSqlark v0.4.2</title>
|
||||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||||
<style>
|
<style>
|
||||||
:root {
|
:root {
|
||||||
@@ -95,6 +95,7 @@
|
|||||||
<a href="#foreign-key">外键级联</a>
|
<a href="#foreign-key">外键级联</a>
|
||||||
<a href="#connection-pool">连接池</a>
|
<a href="#connection-pool">连接池</a>
|
||||||
<a href="#migration">数据迁移</a>
|
<a href="#migration">数据迁移</a>
|
||||||
|
<a href="#selfheal">崩溃自愈 🆕</a>
|
||||||
<a href="#export">导入导出</a>
|
<a href="#export">导入导出</a>
|
||||||
<a href="#plugin">插件 & 钩子</a>
|
<a href="#plugin">插件 & 钩子</a>
|
||||||
<a href="#subscribe">发布订阅</a>
|
<a href="#subscribe">发布订阅</a>
|
||||||
@@ -197,8 +198,8 @@ db.<span class="f">isReady</span>(); <span class="c">// true</span>
|
|||||||
<tr><td><code>maxLength</code></td><td><code>number</code></td><td>字符串最大长度</td></tr>
|
<tr><td><code>maxLength</code></td><td><code>number</code></td><td>字符串最大长度</td></tr>
|
||||||
<tr><td><code>min</code>/<code>max</code></td><td><code>number</code></td><td>数值范围</td></tr>
|
<tr><td><code>min</code>/<code>max</code></td><td><code>number</code></td><td>数值范围</td></tr>
|
||||||
<tr><td><code>references</code></td><td><code>string</code></td><td>外键引用 <code>'table.column'</code></td></tr>
|
<tr><td><code>references</code></td><td><code>string</code></td><td>外键引用 <code>'table.column'</code></td></tr>
|
||||||
<tr><td><code>onDelete</code></td><td><code>'CASCADE'\|'SET NULL'\|'RESTRICT'</code></td><td>删除级联 🆕</td></tr>
|
<tr><td><code>onDelete</code></td><td><code>'CASCADE'\|'SET NULL'\|'RESTRICT'</code></td><td>删除级联 ✅ v0.4.1</td></tr>
|
||||||
<tr><td><code>onUpdate</code></td><td><code>'CASCADE'\|'SET NULL'\|'RESTRICT'</code></td><td>更新级联 🆕</td></tr>
|
<tr><td><code>onUpdate</code></td><td><code>'CASCADE'\|'SET NULL'\|'RESTRICT'</code></td><td>更新级联(更新主键时触发)✅ v0.4.2</td></tr>
|
||||||
</table>
|
</table>
|
||||||
|
|
||||||
<h2 id="sql-query">🔍 SQL 查询</h2>
|
<h2 id="sql-query">🔍 SQL 查询</h2>
|
||||||
@@ -549,6 +550,19 @@ db.<span class="f">addMigration</span>(<span class="n">3</span>, <span class="k"
|
|||||||
|
|
||||||
<span class="c">// 执行迁移到目标版本</span>
|
<span class="c">// 执行迁移到目标版本</span>
|
||||||
<span class="k">await</span> db.<span class="f">migrateTo</span>(<span class="n">3</span>); <span class="c">// 依次执行 v2, v3 的迁移函数</span></pre>
|
<span class="k">await</span> db.<span class="f">migrateTo</span>(<span class="n">3</span>); <span class="c">// 依次执行 v2, v3 的迁移函数</span></pre>
|
||||||
|
<p>✅ <strong>v0.4.2: 迁移版本持久化到库内</strong> — 重启后从持久化版本继续执行,已执行迁移不重跑(此前 version 每次从 config 重置,可能重跑不幂等的迁移)。</p>
|
||||||
|
|
||||||
|
<h2 id="selfheal">🛡 崩溃恢复自愈(v0.4.2)</h2>
|
||||||
|
<p>异常退出(强杀/断电)后无需删库重建:打开数据库时自动跳过残缺 SSTable,应用层可调用自愈 API 恢复一致性。</p>
|
||||||
|
|
||||||
|
<pre><span class="c">// 自愈 — 校验并清理损坏 SSTable / 重建二级索引 / 截断 WAL</span>
|
||||||
|
<span class="k">await</span> db.<span class="f">repair</span>();
|
||||||
|
|
||||||
|
<span class="c">// 清空全部数据与表结构(保留库本身,实例可继续使用)</span>
|
||||||
|
<span class="k">await</span> db.<span class="f">clearAll</span>();
|
||||||
|
|
||||||
|
<span class="c">// 引擎级元数据(迁移版本等)</span>
|
||||||
|
<span class="c">// 引擎接口 IStorageEngine 可选扩展:repair() / clearAll() / getMeta() / setMeta()</span></pre>
|
||||||
|
|
||||||
<h2 id="export">📤 导入导出</h2>
|
<h2 id="export">📤 导入导出</h2>
|
||||||
<pre><span class="c">// 导出单表 — 返回 JSON 数组</span>
|
<pre><span class="c">// 导出单表 — 返回 JSON 数组</span>
|
||||||
@@ -747,7 +761,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。<br>
|
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。<br>
|
||||||
<strong>v0.2.4 生产级</strong> — 二级索引 · MVCC · BloomFilter · WAL CRC全同步 · AES-GCM加密 · Savepoint · EXPLAIN · ANALYZE · REINDEX · VACUUM · BufferPool · 零死代码。<br>
|
<strong>v0.2.4 生产级</strong> — 二级索引 · MVCC · BloomFilter · WAL CRC全同步 · AES-GCM加密 · Savepoint · EXPLAIN · ANALYZE · REINDEX · VACUUM · BufferPool · 零死代码。<br>
|
||||||
<strong>v0.3.2 表达式与并发</strong> — WAL full模式真正同步 · MVCC接入读写路径 · SSTableReader二分查找统一 · crypto实例化 · IndexedDB索引利用 · compactLevel public接口 · WAL大小阈值自动checkpoint · SQL注入防护 · ALTER TABLE · TRUNCATE TABLE · 多标签页同步 · IDB schema持久化。<br>
|
<strong>v0.3.2 表达式与并发</strong> — WAL full模式真正同步 · MVCC接入读写路径 · SSTableReader二分查找统一 · crypto实例化 · IndexedDB索引利用 · compactLevel public接口 · WAL大小阈值自动checkpoint · SQL注入防护 · ALTER TABLE · TRUNCATE TABLE · 多标签页同步 · IDB schema持久化。<br>
|
||||||
<strong>v0.4.1 Aria 级联与演示页引擎切换</strong> — AriaEngine 外键级联(CASCADE/SET NULL/RESTRICT)· `clearAll()` 重置 API · 演示页 ⚡Memory/🌲Aria 引擎切换器 · 894测试 47套件。</p>
|
<strong>v0.4.2 生产就绪与崩溃自愈</strong> — 残缺 SSTable 打开自动跳过(不删库)· WAL 记录与计数原子写入 + 按 key 扫描恢复 · 事务进行中 checkpoint 不截断 WAL · 二级索引跨重启自动恢复 · ALTER TABLE / 事务内 DDL 全引擎持久化 · ON UPDATE 外键级联(含更新主键)· OPFS schema 持久化(空表/索引完整保留)· `repair()` / `clearAll()` 统一自愈接口 · 迁移版本持久化到库内 · 944 测试 50 套件。</p>
|
||||||
|
|
||||||
<h3>存储模式对比</h3>
|
<h3>存储模式对比</h3>
|
||||||
<table>
|
<table>
|
||||||
|
|||||||
+12
-7
@@ -153,7 +153,7 @@
|
|||||||
<!-- Hero -->
|
<!-- Hero -->
|
||||||
<section class="hero">
|
<section class="hero">
|
||||||
<div class="container">
|
<div class="container">
|
||||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.4.1 Aria 级联与演示页引擎切换 — 894测试 47套件 · 流式查询 · 派生表 · 崩溃恢复修复 · 多列哈希连接 · 零回归</div>
|
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.4.2 生产就绪与崩溃自愈 — 944测试 50套件 · 残缺SSTable自动跳过 · WAL原子写入 · ON UPDATE级联 · 二级索引跨重启恢复 · 零回归</div>
|
||||||
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
||||||
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
||||||
<div class="actions">
|
<div class="actions">
|
||||||
@@ -243,8 +243,8 @@ npm install @metona-team/metona-sqlark
|
|||||||
</div>
|
</div>
|
||||||
<div class="feature-card">
|
<div class="feature-card">
|
||||||
<div class="icon">🌲</div>
|
<div class="icon">🌲</div>
|
||||||
<h3>AriaEngine <span style="font-size:0.65rem;color:var(--accent);vertical-align:super;">v0.4.1</span></h3>
|
<h3>AriaEngine <span style="font-size:0.65rem;color:var(--accent);vertical-align:super;">v0.4.2</span></h3>
|
||||||
<p>自研 LSM-Tree 页面式存储引擎。MemTable 红黑树 + 多级 SSTable、Bloom Filter 快速判存、WAL full模式真正同步、MVCC 版本链接入读写路径、崩溃恢复 DROP_TABLE 回放修复。</p>
|
<p>自研 LSM-Tree 页面式存储引擎。MemTable 红黑树 + 多级 SSTable、Bloom Filter 快速判存、WAL 原子写入 full模式真正同步、MVCC 快照隔离、崩溃恢复自动跳过残缺 SSTable、二级索引跨重启恢复、ON UPDATE/DELETE 外键级联。</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="feature-card">
|
<div class="feature-card">
|
||||||
<div class="icon">🔒</div>
|
<div class="icon">🔒</div>
|
||||||
@@ -259,7 +259,12 @@ npm install @metona-team/metona-sqlark
|
|||||||
<div class="feature-card">
|
<div class="feature-card">
|
||||||
<div class="icon">🔗</div>
|
<div class="icon">🔗</div>
|
||||||
<h3>外键级联 + SQL安全</h3>
|
<h3>外键级联 + SQL安全</h3>
|
||||||
<p>references + ON DELETE CASCADE / SET NULL / RESTRICT。React/Vue hooks 表名合法性校验,防 SQL 注入。</p>
|
<p>references + ON DELETE / ON UPDATE CASCADE · SET NULL · RESTRICT(更新主键级联更新子表)。React/Vue hooks 表名合法性校验,防 SQL 注入。</p>
|
||||||
|
</div>
|
||||||
|
<div class="feature-card">
|
||||||
|
<div class="icon">🛡</div>
|
||||||
|
<h3>崩溃恢复自愈</h3>
|
||||||
|
<p>异常退出后无需删库重建:打开自动跳过残缺 SSTable,db.repair() 清理损坏数据并重建索引,db.clearAll() 重置。迁移版本持久化,重启不重跑。</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="feature-card">
|
<div class="feature-card">
|
||||||
<div class="icon">🏊</div>
|
<div class="icon">🏊</div>
|
||||||
@@ -399,12 +404,12 @@ npm install @metona-team/metona-sqlark
|
|||||||
<p>MetonaSqlark 的核心指标</p>
|
<p>MetonaSqlark 的核心指标</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="stats">
|
<div class="stats">
|
||||||
<div class="stat-card"><div class="num">894</div><div class="label">测试用例</div></div>
|
<div class="stat-card"><div class="num">944</div><div class="label">测试用例</div></div>
|
||||||
<div class="stat-card"><div class="num">81.5%</div><div class="label">行覆盖率</div></div>
|
<div class="stat-card"><div class="num">84.2%</div><div class="label">行覆盖率</div></div>
|
||||||
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
||||||
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
||||||
<div class="stat-card"><div class="num">36</div><div class="label">SQL 关键字</div></div>
|
<div class="stat-card"><div class="num">36</div><div class="label">SQL 关键字</div></div>
|
||||||
<div class="stat-card"><div class="num">46</div><div class="label">测试套件</div></div>
|
<div class="stat-card"><div class="num">50</div><div class="label">测试套件</div></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|||||||
+1
-1
@@ -214,4 +214,4 @@ export class DatabaseError extends Error {
|
|||||||
// 版本
|
// 版本
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export const VERSION = '0.4.1';
|
export const VERSION = '0.4.2';
|
||||||
|
|||||||
+53
-1
@@ -90,6 +90,17 @@ export class MetonaSqlark {
|
|||||||
// 打开连接
|
// 打开连接
|
||||||
await this.engine.open(this.name, this.version);
|
await this.engine.open(this.name, this.version);
|
||||||
|
|
||||||
|
// v0.4.2-fix (P2-7): 从库内加载持久化的迁移版本,
|
||||||
|
// 重启后 migrateTo 从持久化版本继续执行,不再每次从 config.version 重置
|
||||||
|
if (typeof this.engine.getMeta === 'function') {
|
||||||
|
try {
|
||||||
|
const persistedVersion = await this.engine.getMeta('__metona_version');
|
||||||
|
if (persistedVersion != null && Number(persistedVersion) >= 1) {
|
||||||
|
this._version = Math.max(this._version, Math.floor(Number(persistedVersion)));
|
||||||
|
}
|
||||||
|
} catch { /* 读取失败回退 config.version */ }
|
||||||
|
}
|
||||||
|
|
||||||
// 初始化执行器和事务管理器
|
// 初始化执行器和事务管理器
|
||||||
this.executor = new QueryExecutor(this.engine, this.maxRowsPerQuery);
|
this.executor = new QueryExecutor(this.engine, this.maxRowsPerQuery);
|
||||||
this.transactionManager = new TransactionManager(this.engine);
|
this.transactionManager = new TransactionManager(this.engine);
|
||||||
@@ -395,11 +406,52 @@ export class MetonaSqlark {
|
|||||||
async migrateTo(targetVersion: number): Promise<void> {
|
async migrateTo(targetVersion: number): Promise<void> {
|
||||||
this.ensureReady();
|
this.ensureReady();
|
||||||
for (const [version, up] of [...this.migrations.entries()].sort((a, b) => a[0] - b[0])) {
|
for (const [version, up] of [...this.migrations.entries()].sort((a, b) => a[0] - b[0])) {
|
||||||
if (version <= targetVersion && version > this.version) {
|
if (version <= targetVersion && version > this._version) {
|
||||||
await up(this);
|
await up(this);
|
||||||
this._version = version;
|
this._version = version;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// v0.4.2-fix (P2-7): 迁移版本持久化到库内,重启后从持久化版本继续,
|
||||||
|
// 避免"version 重置导致已执行迁移重跑(不幂等就炸)"或"版本门槛跳过迁移"
|
||||||
|
if (typeof this.engine.setMeta === 'function') {
|
||||||
|
try {
|
||||||
|
await this.engine.setMeta('__metona_version', String(this._version));
|
||||||
|
} catch { /* 持久化失败不阻塞迁移流程 */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 自愈 / 重置(v0.4.2-fix, P2-9) ----
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 崩溃恢复自愈 — 校验并清理损坏数据、恢复一致性。
|
||||||
|
* 检测到异常后调用,无需删库重建。
|
||||||
|
*/
|
||||||
|
async repair(): Promise<void> {
|
||||||
|
this.ensureReady();
|
||||||
|
if (typeof this.engine.repair === 'function') {
|
||||||
|
await this.engine.repair();
|
||||||
|
this.tableCache.clear();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// 兜底:重建表缓存
|
||||||
|
this.tableCache.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 清空全部数据与表结构(保留库本身)。
|
||||||
|
* 支持后续继续使用本实例重建表。
|
||||||
|
*/
|
||||||
|
async clearAll(): Promise<void> {
|
||||||
|
this.ensureReady();
|
||||||
|
if (typeof this.engine.clearAll === 'function') {
|
||||||
|
await this.engine.clearAll();
|
||||||
|
} else {
|
||||||
|
const names = await this.engine.getTableNames();
|
||||||
|
for (const name of names) {
|
||||||
|
await this.engine.dropTable(name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.tableCache.clear();
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---- 插件 ----
|
// ---- 插件 ----
|
||||||
|
|||||||
+302
-29
@@ -70,6 +70,22 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
|
|
||||||
async open(dbName: string, _version: number): Promise<void> {
|
async open(dbName: string, _version: number): Promise<void> {
|
||||||
if (this.opened) return;
|
if (this.opened) return;
|
||||||
|
// v0.4.2-fix: 引擎内部错误统一包装为 DatabaseError(ARIA_OPEN_ERROR),
|
||||||
|
// 应用层可拿到 code 分类处理,不再抛出原生 RangeError/TypeError
|
||||||
|
try {
|
||||||
|
await this.openInternal(dbName);
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof DatabaseError) throw error;
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Failed to open AriaEngine database "${dbName}"`,
|
||||||
|
'ARIA_OPEN_ERROR',
|
||||||
|
error,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** open 内部实现(错误包装在 open 外层) */
|
||||||
|
private async openInternal(dbName: string): Promise<void> {
|
||||||
this.dbName = dbName;
|
this.dbName = dbName;
|
||||||
|
|
||||||
// 1. 存储后端
|
// 1. 存储后端
|
||||||
@@ -105,20 +121,30 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
this.wal = new WAL(
|
this.wal = new WAL(
|
||||||
{
|
{
|
||||||
append: async (data) => {
|
append: async (data) => {
|
||||||
// Store each record as a separate numbered key
|
// v0.4.2-fix: 记录写入与 count 计数在同一底层事务中原子提交,
|
||||||
|
// 中断时整体回滚,杜绝"记录在、计数丢"导致恢复漏读的丢数据问题
|
||||||
const idx = await this.getWALCount();
|
const idx = await this.getWALCount();
|
||||||
const slice = new Uint8Array(data.buffer, data.byteOffset, data.byteLength);
|
const slice = new Uint8Array(data.buffer, data.byteOffset, data.byteLength);
|
||||||
const copy = slice.buffer.slice(slice.byteOffset, slice.byteOffset + slice.byteLength) as ArrayBuffer;
|
const copy = slice.buffer.slice(slice.byteOffset, slice.byteOffset + slice.byteLength) as ArrayBuffer;
|
||||||
await this.backend.write(`__wal_${idx}`, copy);
|
await this.backend.writeMany({
|
||||||
await this.setWALCount(idx + 1);
|
[`__wal_${idx}`]: copy,
|
||||||
|
__wal_count: new TextEncoder().encode(String(idx + 1)).buffer,
|
||||||
|
});
|
||||||
},
|
},
|
||||||
readAll: async () => {
|
readAll: async () => {
|
||||||
const count = await this.getWALCount();
|
// v0.4.2-fix: 不依赖 count 计数,直接扫描全部 __wal_* 键,
|
||||||
if (count === 0) return new Uint8Array(0);
|
// 避免 count 与实际记录不一致时漏读(与 checkpoint/并发写入竞态无关)
|
||||||
// Read all records and concatenate
|
const keys = (await this.backend.listKeys())
|
||||||
|
.filter((k) => k.startsWith('__wal_') && k !== '__wal_count')
|
||||||
|
.sort((a, b) => {
|
||||||
|
const na = parseInt(a.slice('__wal_'.length), 10);
|
||||||
|
const nb = parseInt(b.slice('__wal_'.length), 10);
|
||||||
|
return (isNaN(na) ? 0 : na) - (isNaN(nb) ? 0 : nb);
|
||||||
|
});
|
||||||
|
if (keys.length === 0) return new Uint8Array(0);
|
||||||
const chunks: Uint8Array[] = [];
|
const chunks: Uint8Array[] = [];
|
||||||
for (let i = 0; i < count; i++) {
|
for (const key of keys) {
|
||||||
const d = await this.backend.read(`__wal_${i}`);
|
const d = await this.backend.read(key);
|
||||||
if (d) chunks.push(new Uint8Array(d));
|
if (d) chunks.push(new Uint8Array(d));
|
||||||
}
|
}
|
||||||
const total = chunks.reduce((s, c) => s + c.byteLength, 0);
|
const total = chunks.reduce((s, c) => s + c.byteLength, 0);
|
||||||
@@ -128,15 +154,17 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
return combined;
|
return combined;
|
||||||
},
|
},
|
||||||
truncate: async () => {
|
truncate: async () => {
|
||||||
const count = await this.getWALCount();
|
// v0.4.2-fix: 扫描删除全部 WAL 记录键 + count 键(单事务原子清理)
|
||||||
for (let i = 0; i < count; i++) {
|
const keys = (await this.backend.listKeys())
|
||||||
await this.backend.delete(`__wal_${i}`);
|
.filter((k) => k.startsWith('__wal_'));
|
||||||
}
|
await this.backend.deleteMany(keys);
|
||||||
await this.setWALCount(0);
|
await this.setWALCount(0);
|
||||||
},
|
},
|
||||||
exists: async () => {
|
exists: async () => {
|
||||||
const count = await this.getWALCount();
|
// v0.4.2-fix: 与 readAll 一致按 key 扫描判断(count 可能因崩溃截断而滞后)
|
||||||
return count > 0;
|
const keys = (await this.backend.listKeys())
|
||||||
|
.filter((k) => k.startsWith('__wal_') && k !== '__wal_count');
|
||||||
|
return keys.length > 0;
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
this.config.walEnabled,
|
this.config.walEnabled,
|
||||||
@@ -146,6 +174,31 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
// 5. 恢复 Schema
|
// 5. 恢复 Schema
|
||||||
await this.loadSchemas();
|
await this.loadSchemas();
|
||||||
|
|
||||||
|
// v0.4.2-fix: 为 schema 中带 index/unique 标记的列重建二级索引 LSM。
|
||||||
|
// 此前重开只恢复 schema 不恢复索引 LSM → 索引查询静默回退全表、
|
||||||
|
// createIndex 因 colDef 已有标记直接 return → 索引永久缺失。
|
||||||
|
// 索引数据已持久化在独立命名空间(sst_idx_* / meta),init() 直接加载。
|
||||||
|
for (const [tableName, schema] of this.schemas) {
|
||||||
|
const pkCol = this.tablePKs.get(tableName)!;
|
||||||
|
for (const [colName, colDef] of Object.entries(schema.columns)) {
|
||||||
|
if ((colDef.index || colDef.unique) && colName !== pkCol) {
|
||||||
|
const idxKey = `${tableName}:idx:${colName}`;
|
||||||
|
if (!this.secondaryIndexes.has(idxKey)) {
|
||||||
|
const idxLsm = new LSM({
|
||||||
|
memtableSizeThreshold: this.config.memtableSizeThreshold,
|
||||||
|
levelSizeMultiplier: this.config.levelSizeMultiplier,
|
||||||
|
blockSize: this.config.pageSize,
|
||||||
|
bloomBitsPerKey: this.config.bloomFilterBitsPerKey,
|
||||||
|
cacheLimitBytes: this.config.bufferPoolPages * this.config.pageSize,
|
||||||
|
sstableStore: this.createSSTableStore(`idx_${tableName}_${colName}`),
|
||||||
|
});
|
||||||
|
await idxLsm.init();
|
||||||
|
this.secondaryIndexes.set(idxKey, idxLsm);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// 6. 初始化 LSM(加载 SSTable 元数据)
|
// 6. 初始化 LSM(加载 SSTable 元数据)
|
||||||
await this.lsm.init();
|
await this.lsm.init();
|
||||||
|
|
||||||
@@ -175,12 +228,31 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
if (allRecords.length > 0) {
|
if (allRecords.length > 0) {
|
||||||
await this.lsm.flush();
|
await this.lsm.flush();
|
||||||
await this.wal.checkpoint();
|
await this.wal.checkpoint();
|
||||||
|
// v0.4.2-fix: WAL 回放只更新主 LSM,二级索引 LSM 未同步 →
|
||||||
|
// 崩溃前最后一批写入的索引缺失,重开时索引查询丢行。
|
||||||
|
// 恢复后全量重建所有表的二级索引(幂等)。
|
||||||
|
for (const tableName of this.schemas.keys()) {
|
||||||
|
await this.reindexTableInternal(tableName);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 8. Checkpoint Manager(接入 WAL 大小阈值)
|
// 8. Checkpoint Manager(接入 WAL 大小阈值)
|
||||||
|
// v0.4.2-fix: 事务活跃时 checkpoint 不得截断 WAL —
|
||||||
|
// 否则 BEGIN/INSERT 记录被截断,COMMIT 后崩溃恢复丢失整个事务数据
|
||||||
this.checkpointManager = new CheckpointManager(
|
this.checkpointManager = new CheckpointManager(
|
||||||
this.lsm,
|
this.lsm,
|
||||||
this.wal,
|
{
|
||||||
|
checkpoint: async () => {
|
||||||
|
if (this.currentTxnId) return;
|
||||||
|
await this.wal.checkpoint();
|
||||||
|
},
|
||||||
|
flush: async () => {
|
||||||
|
if (this.currentTxnId) return;
|
||||||
|
await this.wal.flush();
|
||||||
|
},
|
||||||
|
getBufferedBytes: () => this.wal.getBufferedBytes(),
|
||||||
|
getBufferedCount: () => this.wal.getBufferedCount(),
|
||||||
|
} as unknown as WAL,
|
||||||
{ flushAll: async () => { await this.lsm.flush(); } } as any,
|
{ flushAll: async () => { await this.lsm.flush(); } } as any,
|
||||||
this.config.checkpointInterval,
|
this.config.checkpointInterval,
|
||||||
this.config.walSizeThreshold,
|
this.config.walSizeThreshold,
|
||||||
@@ -193,12 +265,51 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
if (!this.opened) return;
|
if (!this.opened) return;
|
||||||
await this.persistSchemas();
|
await this.persistSchemas();
|
||||||
await this.lsm.flush();
|
await this.lsm.flush();
|
||||||
|
// v0.4.2-fix: 同步落盘全部二级索引 LSM — 此前只 flush 主 LSM,
|
||||||
|
// 优雅关闭后索引 memtable 未落盘 → 重开索引为空 → 索引查询返回空结果
|
||||||
|
for (const idxLsm of this.secondaryIndexes.values()) {
|
||||||
|
await idxLsm.flush();
|
||||||
|
}
|
||||||
await this.wal.flush();
|
await this.wal.flush();
|
||||||
|
// v0.4.2-fix: close 前 checkpoint(截断 WAL)—
|
||||||
|
// 此前只 flush 不截断,下次打开会重放全部历史 WAL 记录(含已落盘 SSTable 的数据),
|
||||||
|
// 重复解析/重复 put 拖慢启动,并与恢复后 flush+checkpoint 竞争放大丢数据
|
||||||
|
await this.wal.checkpoint();
|
||||||
await this.backend.close();
|
await this.backend.close();
|
||||||
|
// v0.4.2-fix: 清空运行期状态(此前 close 后 mvcc/txn 残留,
|
||||||
|
// 重开时 beginTransaction 报 TX_ACTIVE 或读到陈旧快照)
|
||||||
this.schemas.clear();
|
this.schemas.clear();
|
||||||
|
this.tablePKs.clear();
|
||||||
|
this.secondaryIndexes.clear();
|
||||||
|
this.mvcc = new MVCCManager();
|
||||||
|
this.currentTxnId = null;
|
||||||
|
this.txnSnapshot = null;
|
||||||
|
this.savepoints.clear();
|
||||||
|
this.opCounter = 0;
|
||||||
this.opened = false;
|
this.opened = false;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 崩溃恢复/自愈 — 校验并移除损坏 SSTable、截断 WAL、重建二级索引。
|
||||||
|
* 应用层检测到异常后调用,无需删库重建。
|
||||||
|
*/
|
||||||
|
async repair(): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
// 1. 校验全部 SSTable,移除残缺项(打开时已做一次,此处兜底运行期损坏)
|
||||||
|
const removed = await this.lsm.validateAll();
|
||||||
|
// 2. 将 WAL 残留数据落盘并截断,避免无限重放
|
||||||
|
await this.lsm.flush();
|
||||||
|
await this.wal.checkpoint();
|
||||||
|
// 3. 重建所有表的二级索引(修复索引与主数据不一致)
|
||||||
|
for (const tableName of this.schemas.keys()) {
|
||||||
|
await this.reindexTable(tableName);
|
||||||
|
}
|
||||||
|
if (removed > 0) {
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn(`[AriaEngine] repair: removed ${removed} corrupted SSTable(s)`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* v0.4.1: 重置数据库 — 清空全部数据与表结构(演示页刷新/重新初始化用)。
|
* v0.4.1: 重置数据库 — 清空全部数据与表结构(演示页刷新/重新初始化用)。
|
||||||
* 清空存储后端、LSM、WAL、MVCC 与二级索引,后续可继续使用本实例。
|
* 清空存储后端、LSM、WAL、MVCC 与二级索引,后续可继续使用本实例。
|
||||||
@@ -224,12 +335,26 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
|
|
||||||
isOpen(): boolean { return this.opened; }
|
isOpen(): boolean { return this.opened; }
|
||||||
|
|
||||||
|
// ---- v0.4.2-fix: 库内元数据(迁移版本持久化用) ----
|
||||||
|
|
||||||
|
async getMeta(key: string): Promise<string | null> {
|
||||||
|
const raw = await this.backend.read(`__meta_${key}`);
|
||||||
|
return raw ? new TextDecoder().decode(raw) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async setMeta(key: string, value: string): Promise<void> {
|
||||||
|
await this.backend.write(`__meta_${key}`, new TextEncoder().encode(value).buffer);
|
||||||
|
}
|
||||||
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// 表管理
|
// 表管理
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|
||||||
async createTable(schema: TableSchema): Promise<void> {
|
async createTable(schema: TableSchema): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
|
// v0.4.2-fix: Aria 事务中 DDL 显式拒绝(事务快照只覆盖行数据,
|
||||||
|
// 结构变更无法回滚;Memory/IndexedDB 引擎快照可回滚,行为不一致 → 明确报错而非静默)
|
||||||
|
this.ensureNoDDLInTransaction('CREATE TABLE');
|
||||||
if (this.schemas.has(schema.name)) {
|
if (this.schemas.has(schema.name)) {
|
||||||
throw new DatabaseError(`Table "${schema.name}" already exists`, 'TABLE_EXISTS');
|
throw new DatabaseError(`Table "${schema.name}" already exists`, 'TABLE_EXISTS');
|
||||||
}
|
}
|
||||||
@@ -270,6 +395,7 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
|
|
||||||
async dropTable(tableName: string): Promise<void> {
|
async dropTable(tableName: string): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
|
this.ensureNoDDLInTransaction('DROP TABLE');
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
|
|
||||||
// 删除表中所有行
|
// 删除表中所有行
|
||||||
@@ -279,6 +405,10 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
this.lsm.delete(`${tableName}:${row[pkCol]}`);
|
this.lsm.delete(`${tableName}:${row[pkCol]}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v0.4.2-fix: 清理该表的全部二级索引 LSM 与持久化文件 —
|
||||||
|
// 此前残留孤儿索引,重建同名表后旧索引数据污染新表(索引查询返回错误行)
|
||||||
|
await this.cleanupTableIndexes(tableName);
|
||||||
|
|
||||||
this.schemas.delete(tableName);
|
this.schemas.delete(tableName);
|
||||||
this.tablePKs.delete(tableName);
|
this.tablePKs.delete(tableName);
|
||||||
await this.persistSchemas();
|
await this.persistSchemas();
|
||||||
@@ -291,6 +421,25 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 清理指定表的全部二级索引 LSM(内存 + 存储文件 + meta)。
|
||||||
|
* dropTable / DROP_TABLE 恢复 / alterTable DROP 索引列 共用。
|
||||||
|
*/
|
||||||
|
private async cleanupTableIndexes(tableName: string): Promise<void> {
|
||||||
|
const prefix = `${tableName}:idx:`;
|
||||||
|
const toDelete: string[] = [];
|
||||||
|
for (const [idxKey, idxLsm] of this.secondaryIndexes) {
|
||||||
|
if (!idxKey.startsWith(prefix)) continue;
|
||||||
|
toDelete.push(idxKey);
|
||||||
|
try {
|
||||||
|
await idxLsm.clear();
|
||||||
|
} catch { /* 清理失败不阻塞 */ }
|
||||||
|
}
|
||||||
|
for (const idxKey of toDelete) {
|
||||||
|
this.secondaryIndexes.delete(idxKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async hasTable(tableName: string): Promise<boolean> {
|
async hasTable(tableName: string): Promise<boolean> {
|
||||||
return this.schemas.has(tableName);
|
return this.schemas.has(tableName);
|
||||||
}
|
}
|
||||||
@@ -421,6 +570,8 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
let count = 0;
|
let count = 0;
|
||||||
// v0.3.1: 批量 WAL 写入(组提交)
|
// v0.3.1: 批量 WAL 写入(组提交)
|
||||||
const walRecords: Omit<import('./types').WALRecord, 'lsn' | 'checksum'>[] = [];
|
const walRecords: Omit<import('./types').WALRecord, 'lsn' | 'checksum'>[] = [];
|
||||||
|
// v0.4.2-fix: ON UPDATE 级联环路保护
|
||||||
|
const visited = new Set<string>();
|
||||||
|
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
const pkCol = this.tablePKs.get(tableName)!;
|
const pkCol = this.tablePKs.get(tableName)!;
|
||||||
@@ -430,24 +581,48 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
const updated = { ...row, ...updates };
|
const updated = { ...row, ...updates };
|
||||||
this.validateRow(schema, updated);
|
this.validateRow(schema, updated);
|
||||||
|
|
||||||
|
// v0.4.2-fix: 支持更新主键 — 删除旧键 + 落新键 + WAL 两条记录
|
||||||
|
const newPk = String(updated[pkCol]);
|
||||||
|
const pkChanged = newPk !== String(row[pkCol]);
|
||||||
|
|
||||||
|
if (pkChanged) {
|
||||||
|
// ON UPDATE 外键级联(RESTRICT 抛错 / CASCADE / SET NULL)
|
||||||
|
await this.applyForeignKeyUpdateRules(
|
||||||
|
tableName, String(row[pkCol]), newPk, walRecords, visited,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
if (this.currentTxnId && this.txnSnapshot) {
|
if (this.currentTxnId && this.txnSnapshot) {
|
||||||
this.txnSnapshot.set(key, updated);
|
if (pkChanged) {
|
||||||
this.mvcc.writeVersion(tableName, String(row[pkCol]), updated, this.currentTxnId);
|
this.txnSnapshot.set(key, { __txn_deleted: true } as unknown as Record<string, unknown>);
|
||||||
|
this.mvcc.deleteVersion(tableName, String(row[pkCol]), this.currentTxnId);
|
||||||
|
}
|
||||||
|
this.txnSnapshot.set(`${tableName}:${newPk}`, updated);
|
||||||
|
this.mvcc.writeVersion(tableName, newPk, updated, this.currentTxnId);
|
||||||
} else {
|
} else {
|
||||||
this.lsm.put(key, updated);
|
if (pkChanged) this.lsm.delete(key);
|
||||||
|
this.lsm.put(`${tableName}:${newPk}`, updated);
|
||||||
}
|
}
|
||||||
count++;
|
count++;
|
||||||
|
|
||||||
|
if (pkChanged) {
|
||||||
|
walRecords.push({
|
||||||
|
type: WALRecordType.DELETE,
|
||||||
|
txnId: this.currentTxnId ?? 0,
|
||||||
|
tableName,
|
||||||
|
key: String(row[pkCol]),
|
||||||
|
});
|
||||||
|
}
|
||||||
walRecords.push({
|
walRecords.push({
|
||||||
type: WALRecordType.UPDATE,
|
type: WALRecordType.UPDATE,
|
||||||
txnId: this.currentTxnId ?? 0,
|
txnId: this.currentTxnId ?? 0,
|
||||||
tableName,
|
tableName,
|
||||||
key: String(row[pkCol]),
|
key: newPk,
|
||||||
data: updated,
|
data: updated,
|
||||||
});
|
});
|
||||||
|
|
||||||
// 更新二级索引
|
// 更新二级索引(主键变更时旧索引条目一并清理)
|
||||||
this.updateSecondaryIndexes(tableName, String(row[pkCol]), updated, row);
|
this.updateSecondaryIndexes(tableName, newPk, updated, pkChanged ? row : null);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -459,6 +634,74 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: ON UPDATE 外键级联 — 主键 oldPk → newPk 时处理引用表。
|
||||||
|
* RESTRICT 抛错 / CASCADE 更新 FK / SET NULL 置空(含索引与 WAL 记录)。
|
||||||
|
* 两阶段:先全量 RESTRICT 检查,再执行级联。
|
||||||
|
*/
|
||||||
|
private async applyForeignKeyUpdateRules(
|
||||||
|
tableName: string,
|
||||||
|
oldPk: string,
|
||||||
|
newPk: string,
|
||||||
|
walRecords: Omit<import('./types').WALRecord, 'lsn' | 'checksum'>[],
|
||||||
|
visited: Set<string>,
|
||||||
|
): Promise<void> {
|
||||||
|
const visitKey = `${tableName}:${oldPk}`;
|
||||||
|
if (visited.has(visitKey)) return;
|
||||||
|
visited.add(visitKey);
|
||||||
|
|
||||||
|
// 阶段 1: RESTRICT 检查
|
||||||
|
for (const [refTableName, refSchema] of this.schemas) {
|
||||||
|
if (refTableName === tableName) continue;
|
||||||
|
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
|
||||||
|
if (!colDef.references || !colDef.onUpdate) continue;
|
||||||
|
const [refTable] = colDef.references.split('.');
|
||||||
|
if (refTable !== tableName) continue;
|
||||||
|
if (colDef.onUpdate !== 'RESTRICT') continue;
|
||||||
|
const refRows = await this.getAllRows(refTableName);
|
||||||
|
if (refRows.some((r) => String(r[colName]) === oldPk)) {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`,
|
||||||
|
'FOREIGN_KEY_VIOLATION',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 阶段 2: CASCADE / SET NULL
|
||||||
|
for (const [refTableName, refSchema] of this.schemas) {
|
||||||
|
if (refTableName === tableName) continue;
|
||||||
|
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
|
||||||
|
if (!colDef.references || !colDef.onUpdate) continue;
|
||||||
|
const [refTable] = colDef.references.split('.');
|
||||||
|
if (refTable !== tableName) continue;
|
||||||
|
if (colDef.onUpdate !== 'CASCADE' && colDef.onUpdate !== 'SET NULL') continue;
|
||||||
|
const refRows = await this.getAllRows(refTableName);
|
||||||
|
for (const refRow of refRows) {
|
||||||
|
if (String(refRow[colName]) !== oldPk) continue;
|
||||||
|
const refPkCol = this.tablePKs.get(refTableName)!;
|
||||||
|
const refPk = String(refRow[refPkCol]);
|
||||||
|
const updatedRef = { ...refRow, [colName]: colDef.onUpdate === 'CASCADE' ? newPk : null };
|
||||||
|
const refKey = `${refTableName}:${refPk}`;
|
||||||
|
if (this.currentTxnId && this.txnSnapshot) {
|
||||||
|
this.txnSnapshot.set(refKey, updatedRef);
|
||||||
|
this.mvcc.writeVersion(refTableName, refPk, updatedRef, this.currentTxnId);
|
||||||
|
} else {
|
||||||
|
this.lsm.put(refKey, updatedRef);
|
||||||
|
}
|
||||||
|
this.updateSecondaryIndexes(refTableName, refPk, updatedRef, refRow);
|
||||||
|
walRecords.push({
|
||||||
|
type: WALRecordType.UPDATE,
|
||||||
|
txnId: this.currentTxnId ?? 0,
|
||||||
|
tableName: refTableName,
|
||||||
|
key: refPk,
|
||||||
|
data: updatedRef,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
@@ -707,6 +950,7 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
column: import('../../constants').ColumnDef & { name: string },
|
column: import('../../constants').ColumnDef & { name: string },
|
||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
|
this.ensureNoDDLInTransaction('ALTER TABLE');
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
const schema = this.schemas.get(tableName)!;
|
const schema = this.schemas.get(tableName)!;
|
||||||
|
|
||||||
@@ -723,11 +967,21 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
if (!schema.columns[column.name]) {
|
if (!schema.columns[column.name]) {
|
||||||
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
||||||
}
|
}
|
||||||
|
// v0.4.2-fix: 被删列是索引列 → 先清理索引 LSM(残留会导致后续同名列索引脏数据)
|
||||||
|
if (schema.columns[column.name].index || schema.columns[column.name].unique) {
|
||||||
|
const idxKey = `${tableName}:idx:${column.name}`;
|
||||||
|
const idxLsm = this.secondaryIndexes.get(idxKey);
|
||||||
|
if (idxLsm) {
|
||||||
|
try {
|
||||||
|
await idxLsm.clear();
|
||||||
|
} catch { /* 清理失败不阻塞 */ }
|
||||||
|
this.secondaryIndexes.delete(idxKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
delete schema.columns[column.name];
|
delete schema.columns[column.name];
|
||||||
await this.persistSchemas();
|
await this.persistSchemas();
|
||||||
|
|
||||||
// 重写主 LSM:移除所有行的该列键(find 副本无法就地删除,必须重写存储)
|
// 重写主 LSM:移除所有行的该列键(find 副本无法就地删除,必须重写存储)
|
||||||
const pkCol = this.tablePKs.get(tableName)!;
|
|
||||||
const prefix = `${tableName}:`;
|
const prefix = `${tableName}:`;
|
||||||
const endKey = `${prefix}\uffff`;
|
const endKey = `${prefix}\uffff`;
|
||||||
await this.lsm.prefetchRange(prefix, endKey);
|
await this.lsm.prefetchRange(prefix, endKey);
|
||||||
@@ -757,16 +1011,18 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
|
|
||||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
|
this.ensureNoDDLInTransaction('CREATE INDEX');
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
const schema = this.schemas.get(tableName)!;
|
const schema = this.schemas.get(tableName)!;
|
||||||
const colDef = schema.columns[column];
|
const colDef = schema.columns[column];
|
||||||
if (!colDef) throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
if (!colDef) throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
||||||
if (colDef.index || colDef.unique) return; // 已存在
|
const idxKey = `${tableName}:idx:${column}`;
|
||||||
|
// v0.4.2-fix: 以索引 LSM 是否已建为准(schema 标记可能因重启恢复而存在,
|
||||||
|
// 但索引 LSM 未恢复 → 此前静默 return 导致索引永久缺失)
|
||||||
|
if (this.secondaryIndexes.has(idxKey)) return;
|
||||||
colDef.index = true;
|
colDef.index = true;
|
||||||
if (unique) colDef.unique = true;
|
if (unique) colDef.unique = true;
|
||||||
|
|
||||||
const idxKey = `${tableName}:idx:${column}`;
|
|
||||||
if (!this.secondaryIndexes.has(idxKey)) {
|
|
||||||
const idxLsm = new LSM({
|
const idxLsm = new LSM({
|
||||||
memtableSizeThreshold: this.config.memtableSizeThreshold,
|
memtableSizeThreshold: this.config.memtableSizeThreshold,
|
||||||
levelSizeMultiplier: this.config.levelSizeMultiplier,
|
levelSizeMultiplier: this.config.levelSizeMultiplier,
|
||||||
@@ -777,10 +1033,8 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
});
|
});
|
||||||
await idxLsm.init();
|
await idxLsm.init();
|
||||||
this.secondaryIndexes.set(idxKey, idxLsm);
|
this.secondaryIndexes.set(idxKey, idxLsm);
|
||||||
}
|
|
||||||
|
|
||||||
// 从主 LSM 重建索引数据
|
// 从主 LSM 重建索引数据
|
||||||
const idxLsm = this.secondaryIndexes.get(idxKey)!;
|
|
||||||
const pkCol = this.tablePKs.get(tableName)!;
|
const pkCol = this.tablePKs.get(tableName)!;
|
||||||
const rows = await this.getAllRows(tableName);
|
const rows = await this.getAllRows(tableName);
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
@@ -795,6 +1049,7 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
|
|
||||||
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
|
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
|
this.ensureNoDDLInTransaction('DROP INDEX');
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
const schema = this.schemas.get(tableName)!;
|
const schema = this.schemas.get(tableName)!;
|
||||||
const colDef = schema.columns[column];
|
const colDef = schema.columns[column];
|
||||||
@@ -1185,6 +1440,8 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
*/
|
*/
|
||||||
private async applyDropTableRecovery(tableName: string): Promise<void> {
|
private async applyDropTableRecovery(tableName: string): Promise<void> {
|
||||||
if (!tableName) return;
|
if (!tableName) return;
|
||||||
|
// v0.4.2-fix: 清理该表二级索引(崩溃恢复路径同样不留孤儿索引)
|
||||||
|
await this.cleanupTableIndexes(tableName);
|
||||||
this.schemas.delete(tableName);
|
this.schemas.delete(tableName);
|
||||||
this.tablePKs.delete(tableName);
|
this.tablePKs.delete(tableName);
|
||||||
|
|
||||||
@@ -1442,7 +1699,13 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
async reindexTable(tableName: string): Promise<number> {
|
async reindexTable(tableName: string): Promise<number> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
const schema = this.schemas.get(tableName)!;
|
return this.reindexTableInternal(tableName);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 重建索引内部实现(不校验 opened,供 open 恢复流程调用) */
|
||||||
|
private async reindexTableInternal(tableName: string): Promise<number> {
|
||||||
|
const schema = this.schemas.get(tableName);
|
||||||
|
if (!schema) return 0;
|
||||||
let rebuiltCount = 0;
|
let rebuiltCount = 0;
|
||||||
|
|
||||||
for (const [colName, colDef] of Object.entries(schema.columns)) {
|
for (const [colName, colDef] of Object.entries(schema.columns)) {
|
||||||
@@ -1482,7 +1745,7 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
// GC MVCC 版本(保留最新 10 个)
|
// GC MVCC 版本(保留最新 10 个)
|
||||||
const beforeGC = this.mvcc.getActiveTxnCount?.() ?? 0;
|
const beforeGC = this.mvcc.getGlobalLSN();
|
||||||
this.mvcc.gc(10);
|
this.mvcc.gc(10);
|
||||||
return { compactedLevels: 6, gcVersions: beforeGC };
|
return { compactedLevels: 6, gcVersions: beforeGC };
|
||||||
}
|
}
|
||||||
@@ -1525,6 +1788,16 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
if (!this.opened) throw new DatabaseError('AriaEngine not opened', 'DB_NOT_OPEN');
|
if (!this.opened) throw new DatabaseError('AriaEngine not opened', 'DB_NOT_OPEN');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: Aria 事务中 DDL 显式拒绝(结构变更无法通过行快照回滚) */
|
||||||
|
private ensureNoDDLInTransaction(op: string): void {
|
||||||
|
if (this.currentTxnId) {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`${op} is not supported inside a transaction (AriaEngine DDL is not transactional)`,
|
||||||
|
'NOT_SUPPORTED',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private ensureTable(tableName: string): void {
|
private ensureTable(tableName: string): void {
|
||||||
if (!this.schemas.has(tableName)) {
|
if (!this.schemas.has(tableName)) {
|
||||||
throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
|
throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
|
||||||
|
|||||||
+142
-13
@@ -69,6 +69,8 @@ export class LSM {
|
|||||||
private cacheLimitBytes: number;
|
private cacheLimitBytes: number;
|
||||||
private levelSizeMultiplier: number;
|
private levelSizeMultiplier: number;
|
||||||
private blockSize: number;
|
private blockSize: number;
|
||||||
|
/** v0.4.2-fix: 配置的 memtable 阈值(freeze 后新 memtable 用配置值,不衰减) */
|
||||||
|
private memtableSizeThreshold: number;
|
||||||
private sstableStore: SSTableStore;
|
private sstableStore: SSTableStore;
|
||||||
private operationCount = 0;
|
private operationCount = 0;
|
||||||
private initialized = false;
|
private initialized = false;
|
||||||
@@ -77,7 +79,8 @@ export class LSM {
|
|||||||
private flushChain: Promise<void> = Promise.resolve();
|
private flushChain: Promise<void> = Promise.resolve();
|
||||||
|
|
||||||
constructor(config: LSMConfig) {
|
constructor(config: LSMConfig) {
|
||||||
this.memtable = new MemTable(config.memtableSizeThreshold ?? DEFAULT_MEMTABLE_SIZE);
|
this.memtableSizeThreshold = config.memtableSizeThreshold ?? DEFAULT_MEMTABLE_SIZE;
|
||||||
|
this.memtable = new MemTable(this.memtableSizeThreshold);
|
||||||
this.levelSizeMultiplier = config.levelSizeMultiplier ?? DEFAULT_LEVEL_SIZE_MULTIPLIER;
|
this.levelSizeMultiplier = config.levelSizeMultiplier ?? DEFAULT_LEVEL_SIZE_MULTIPLIER;
|
||||||
this.blockSize = config.blockSize ?? 4096;
|
this.blockSize = config.blockSize ?? 4096;
|
||||||
this.sstableStore = config.sstableStore;
|
this.sstableStore = config.sstableStore;
|
||||||
@@ -97,8 +100,17 @@ export class LSM {
|
|||||||
|
|
||||||
const metas = await this.sstableStore.listMeta();
|
const metas = await this.sstableStore.listMeta();
|
||||||
|
|
||||||
// 按层级分组
|
// v0.4.2-fix: 打开时完整性校验 — 验证每个 meta 引用的文件存在、可解析,
|
||||||
|
// 残缺/损坏的 SSTable 忽略并清理 meta,避免后续读取抛 RangeError 崩溃
|
||||||
|
const validMetas: SSTableMeta[] = [];
|
||||||
for (const meta of metas) {
|
for (const meta of metas) {
|
||||||
|
if (await this.validateSSTable(meta)) {
|
||||||
|
validMetas.push(meta);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 按层级分组
|
||||||
|
for (const meta of validMetas) {
|
||||||
if (meta.level >= 0 && meta.level < MAX_LSM_LEVELS) {
|
if (meta.level >= 0 && meta.level < MAX_LSM_LEVELS) {
|
||||||
this.levels[meta.level].push(meta);
|
this.levels[meta.level].push(meta);
|
||||||
}
|
}
|
||||||
@@ -157,15 +169,30 @@ export class LSM {
|
|||||||
* 冻结当前 MemTable 为 immutable,并在串行链上排队异步刷盘。
|
* 冻结当前 MemTable 为 immutable,并在串行链上排队异步刷盘。
|
||||||
* 冻结的 MemTable 通过闭包捕获,避免链中前一个 flush 错误处理后续冻结的表。
|
* 冻结的 MemTable 通过闭包捕获,避免链中前一个 flush 错误处理后续冻结的表。
|
||||||
* 所有 pending frozen 记录在 frozenMemtables 中,flush 完成前读取路径仍可访问。
|
* 所有 pending frozen 记录在 frozenMemtables 中,flush 完成前读取路径仍可访问。
|
||||||
|
*
|
||||||
|
* v0.4.2-fix: 链上任务失败时吞错恢复链(否则 flushChain 永久 rejected,
|
||||||
|
* 后续所有 flush/compaction 挂起,写路径卡死)。
|
||||||
*/
|
*/
|
||||||
freezeMemtable(): void {
|
freezeMemtable(): void {
|
||||||
if (this.immutableMemtable) {
|
if (this.immutableMemtable) {
|
||||||
const frozen = this.immutableMemtable;
|
const frozen = this.immutableMemtable;
|
||||||
this.flushChain = this.flushChain.then(() => this.flushImmutableAsync(frozen));
|
this.flushChain = this.enqueueOnChain(() => this.flushImmutableAsync(frozen));
|
||||||
}
|
}
|
||||||
this.immutableMemtable = this.memtable;
|
this.immutableMemtable = this.memtable;
|
||||||
this.frozenMemtables.push(this.immutableMemtable);
|
this.frozenMemtables.push(this.immutableMemtable);
|
||||||
this.memtable = new MemTable(this.memtable.getEstimatedSize());
|
// v0.4.2-fix: 新 memtable 用配置阈值(此前传旧表已用大小 → 阈值逐次衰减 → 频繁小文件 flush)
|
||||||
|
this.memtable = new MemTable(this.memtableSizeThreshold);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 在串行链上排队任务;任务失败吞错并记录,保证链不被单次失败卡死 */
|
||||||
|
private enqueueOnChain(task: () => Promise<void>): Promise<void> {
|
||||||
|
return this.flushChain
|
||||||
|
.then(task)
|
||||||
|
.catch((error) => {
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn('[AriaEngine LSM] background flush/compaction failed:', error);
|
||||||
|
// catch 返回 undefined → 链恢复为 resolved,后续任务继续
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 将指定 Immutable MemTable 刷盘为 SSTable(id 由 store 按命名空间分配) */
|
/** 将指定 Immutable MemTable 刷盘为 SSTable(id 由 store 按命名空间分配) */
|
||||||
@@ -218,7 +245,7 @@ export class LSM {
|
|||||||
this.compacting = true;
|
this.compacting = true;
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
try {
|
try {
|
||||||
this.flushChain = this.flushChain.then(() => this.compactLevelAsync(level));
|
this.flushChain = this.enqueueOnChain(() => this.compactLevelAsync(level));
|
||||||
} finally {
|
} finally {
|
||||||
this.compacting = false;
|
this.compacting = false;
|
||||||
// 连续触发:如果 compaction 后仍然超标,继续调度
|
// 连续触发:如果 compaction 后仍然超标,继续调度
|
||||||
@@ -243,6 +270,10 @@ export class LSM {
|
|||||||
} finally {
|
} finally {
|
||||||
this.compacting = false;
|
this.compacting = false;
|
||||||
}
|
}
|
||||||
|
}).catch((error) => {
|
||||||
|
this.compacting = false;
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn('[AriaEngine LSM] background compaction failed:', error);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -396,29 +427,64 @@ export class LSM {
|
|||||||
// Compaction
|
// Compaction
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|
||||||
/** 执行 Compaction(public,供 VACUUM 等外部调用) */
|
/** 执行 Compaction(public,供 VACUUM 等外部调用;VACUUM 期望 2 个文件即可压缩) */
|
||||||
async compactLevel(level: number): Promise<void> {
|
async compactLevel(level: number): Promise<void> {
|
||||||
await this.compactLevelAsync(level);
|
await this.compactLevelAsync(level, 2);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 串行执行 Compaction(简化版,内部实现) */
|
/**
|
||||||
private async compactLevelAsync(level: number): Promise<void> {
|
* 串行执行 Compaction。
|
||||||
|
* @param minFiles 触发压缩的文件数门槛(自动调度用 4,VACUUM 用 2)
|
||||||
|
*
|
||||||
|
* v0.4.2-fix: 读取从存储兜底(不依赖缓存)——此前仅从缓存读,
|
||||||
|
* 缓存未命中(LRU 驱逐/单文件超缓存上限)时跳过全部文件并从 levels 移除,
|
||||||
|
* 运行中数据全部不可见。
|
||||||
|
*/
|
||||||
|
private async compactLevelAsync(level: number, minFiles: number = 4): Promise<void> {
|
||||||
if (level >= MAX_LSM_LEVELS - 1) return;
|
if (level >= MAX_LSM_LEVELS - 1) return;
|
||||||
if (this.levels[level].length < 4) return;
|
if (this.levels[level].length < minFiles) return;
|
||||||
|
|
||||||
const sstables = this.levels[level].splice(0, this.levels[level].length);
|
const sstables = this.levels[level].splice(0, this.levels[level].length);
|
||||||
const mergeIter = new MergeIterator();
|
const mergeIter = new MergeIterator();
|
||||||
|
const loadedMetas: SSTableMeta[] = [];
|
||||||
|
|
||||||
for (const meta of sstables) {
|
for (const meta of sstables) {
|
||||||
const reader = this.loadSSTableReader(meta);
|
// 优先缓存,未命中则从存储加载(残缺文件经校验清理,跳过)
|
||||||
if (!reader) continue;
|
let data: Uint8Array | null = this.sstableCache.get(meta.id) ?? null;
|
||||||
|
if (!data) {
|
||||||
|
try {
|
||||||
|
data = await this.sstableStore.load(meta.id);
|
||||||
|
} catch {
|
||||||
|
data = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!data || data.byteLength < 32) {
|
||||||
|
await this.dropInvalidSSTable(meta);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let reader: SSTableReader;
|
||||||
|
try {
|
||||||
|
reader = new SSTableReader(data, meta);
|
||||||
|
} catch {
|
||||||
|
await this.dropInvalidSSTable(meta);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
const entries: [string, Record<string, unknown>][] = [];
|
const entries: [string, Record<string, unknown>][] = [];
|
||||||
reader.scanAll((k, v) => entries.push([k, v]));
|
reader.scanAll((k, v) => entries.push([k, v]));
|
||||||
mergeIter.addSource(new ArrayEntrySource(entries));
|
mergeIter.addSource(new ArrayEntrySource(entries));
|
||||||
|
loadedMetas.push(meta);
|
||||||
}
|
}
|
||||||
|
|
||||||
const merged = mergeIter.drain();
|
const merged = mergeIter.drain();
|
||||||
if (merged.length === 0) return;
|
if (merged.length === 0) {
|
||||||
|
// 没有有效数据(全部损坏):把有效 meta 放回 levels,
|
||||||
|
// 避免文件从读取路径消失(数据仍在磁盘,重启可恢复)
|
||||||
|
for (const meta of loadedMetas) {
|
||||||
|
this.levels[level].push(meta);
|
||||||
|
}
|
||||||
|
this.levels[level].sort((a, b) => b.id - a.id);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
const id = await this.sstableStore.allocateId();
|
const id = await this.sstableStore.allocateId();
|
||||||
const builder = new SSTableBuilder(this.blockSize);
|
const builder = new SSTableBuilder(this.blockSize);
|
||||||
@@ -505,6 +571,69 @@ export class LSM {
|
|||||||
// 内部
|
// 内部
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 重新校验全部已加载 SSTable,移除损坏项(repair 自愈用)。
|
||||||
|
* @returns 移除的损坏 SSTable 数量
|
||||||
|
*/
|
||||||
|
async validateAll(): Promise<number> {
|
||||||
|
let removed = 0;
|
||||||
|
for (let level = 0; level < MAX_LSM_LEVELS; level++) {
|
||||||
|
const valid: SSTableMeta[] = [];
|
||||||
|
for (const meta of this.levels[level]) {
|
||||||
|
if (await this.validateSSTable(meta)) {
|
||||||
|
valid.push(meta);
|
||||||
|
} else {
|
||||||
|
this.sstableCache.delete(meta.id);
|
||||||
|
removed++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.levels[level] = valid;
|
||||||
|
}
|
||||||
|
return removed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 校验单个 SSTable 的完整性。
|
||||||
|
* - 文件不存在 → 清理 meta,返回 false
|
||||||
|
* - 文件过小/魔数错误/索引越界(残缺写入产物)→ 清理 meta,返回 false
|
||||||
|
* 校验通过的数据不缓存(保持内存预算),读路径按需预加载。
|
||||||
|
*/
|
||||||
|
private async validateSSTable(meta: SSTableMeta): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
const data = await this.sstableStore.load(meta.id);
|
||||||
|
if (!data) {
|
||||||
|
this.dropInvalidSSTable(meta);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (data.byteLength < 32) {
|
||||||
|
this.dropInvalidSSTable(meta);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
new SSTableReader(data, meta);
|
||||||
|
} catch {
|
||||||
|
this.dropInvalidSSTable(meta);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
this.dropInvalidSSTable(meta);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 清理无效 SSTable 的 meta 与文件(打开自愈路径) */
|
||||||
|
private async dropInvalidSSTable(meta: SSTableMeta): Promise<void> {
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn(`[AriaEngine LSM] Skipping corrupted SSTable id=${meta.id} (level=${meta.level})`);
|
||||||
|
try {
|
||||||
|
await this.sstableStore.deleteMeta(meta.id);
|
||||||
|
} catch { /* 清理失败不阻塞打开 */ }
|
||||||
|
try {
|
||||||
|
await this.sstableStore.delete(meta.id);
|
||||||
|
} catch { /* 清理失败不阻塞打开 */ }
|
||||||
|
}
|
||||||
|
|
||||||
private unwrapTombstone(value: Record<string, unknown> | null): Record<string, unknown> | null {
|
private unwrapTombstone(value: Record<string, unknown> | null): Record<string, unknown> | null {
|
||||||
if (!value) return null;
|
if (!value) return null;
|
||||||
if ((value as unknown as Record<string, unknown>).__tombstone) return null;
|
if ((value as unknown as Record<string, unknown>).__tombstone) return null;
|
||||||
|
|||||||
@@ -40,11 +40,9 @@ export class SSTableReader {
|
|||||||
if (blockIdx < 0) return null;
|
if (blockIdx < 0) return null;
|
||||||
|
|
||||||
const entry = this.indexEntries[blockIdx];
|
const entry = this.indexEntries[blockIdx];
|
||||||
const blockData = new Uint8Array(
|
const blockData = this.getBlockData(entry);
|
||||||
this.data.buffer,
|
// v0.4.2-fix: 残缺文件(meta 偏移超出实际长度)跳过该块,而非抛 RangeError
|
||||||
this.data.byteOffset + entry.blockOffset,
|
if (!blockData) return null;
|
||||||
entry.blockSize,
|
|
||||||
);
|
|
||||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||||
|
|
||||||
const entryCount = blockView.getUint32(0, false);
|
const entryCount = blockView.getUint32(0, false);
|
||||||
@@ -52,12 +50,15 @@ export class SSTableReader {
|
|||||||
|
|
||||||
// 顺序扫描 block 内的条目(生产中应二分查找)
|
// 顺序扫描 block 内的条目(生产中应二分查找)
|
||||||
for (let i = 0; i < entryCount; i++) {
|
for (let i = 0; i < entryCount; i++) {
|
||||||
|
if (offset + 2 > blockData.byteLength) break;
|
||||||
const keyLen = blockView.getUint16(offset, false);
|
const keyLen = blockView.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + keyLen + 2 > blockData.byteLength) break;
|
||||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||||
offset += keyLen;
|
offset += keyLen;
|
||||||
const valLen = blockView.getUint16(offset, false);
|
const valLen = blockView.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + valLen > blockData.byteLength) break;
|
||||||
const valBytes = blockData.slice(offset, offset + valLen);
|
const valBytes = blockData.slice(offset, offset + valLen);
|
||||||
offset += valLen;
|
offset += valLen;
|
||||||
|
|
||||||
@@ -86,23 +87,24 @@ export class SSTableReader {
|
|||||||
|
|
||||||
for (let bi = startBlockIdx; bi <= endBlockIdx && bi >= 0; bi++) {
|
for (let bi = startBlockIdx; bi <= endBlockIdx && bi >= 0; bi++) {
|
||||||
const entry = this.indexEntries[bi];
|
const entry = this.indexEntries[bi];
|
||||||
const blockData = new Uint8Array(
|
const blockData = this.getBlockData(entry);
|
||||||
this.data.buffer,
|
// v0.4.2-fix: 残缺块跳过(rangeScan 继续后续块,不抛异常)
|
||||||
this.data.byteOffset + entry.blockOffset,
|
if (!blockData) continue;
|
||||||
entry.blockSize,
|
|
||||||
);
|
|
||||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||||
|
|
||||||
const blockEntryCount = blockView.getUint32(0, false);
|
const blockEntryCount = blockView.getUint32(0, false);
|
||||||
let offset = 4;
|
let offset = 4;
|
||||||
|
|
||||||
for (let i = 0; i < blockEntryCount; i++) {
|
for (let i = 0; i < blockEntryCount; i++) {
|
||||||
|
if (offset + 2 > blockData.byteLength) break;
|
||||||
const keyLen = blockView.getUint16(offset, false);
|
const keyLen = blockView.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + keyLen + 2 > blockData.byteLength) break;
|
||||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||||
offset += keyLen;
|
offset += keyLen;
|
||||||
const valLen = blockView.getUint16(offset, false);
|
const valLen = blockView.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + valLen > blockData.byteLength) break;
|
||||||
const valBytes = blockData.slice(offset, offset + valLen);
|
const valBytes = blockData.slice(offset, offset + valLen);
|
||||||
offset += valLen;
|
offset += valLen;
|
||||||
|
|
||||||
@@ -121,23 +123,24 @@ export class SSTableReader {
|
|||||||
/** 扫描所有条目 */
|
/** 扫描所有条目 */
|
||||||
scanAll(callback: (key: string, value: Record<string, unknown>) => void): void {
|
scanAll(callback: (key: string, value: Record<string, unknown>) => void): void {
|
||||||
for (const entry of this.indexEntries) {
|
for (const entry of this.indexEntries) {
|
||||||
const blockData = new Uint8Array(
|
const blockData = this.getBlockData(entry);
|
||||||
this.data.buffer,
|
// v0.4.2-fix: 残缺块跳过(scanAll 继续后续块,不抛异常)
|
||||||
this.data.byteOffset + entry.blockOffset,
|
if (!blockData) continue;
|
||||||
entry.blockSize,
|
|
||||||
);
|
|
||||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||||
|
|
||||||
const blockEntryCount = blockView.getUint32(0, false);
|
const blockEntryCount = blockView.getUint32(0, false);
|
||||||
let offset = 4;
|
let offset = 4;
|
||||||
|
|
||||||
for (let i = 0; i < blockEntryCount; i++) {
|
for (let i = 0; i < blockEntryCount; i++) {
|
||||||
|
if (offset + 2 > blockData.byteLength) break;
|
||||||
const keyLen = blockView.getUint16(offset, false);
|
const keyLen = blockView.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + keyLen + 2 > blockData.byteLength) break;
|
||||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||||
offset += keyLen;
|
offset += keyLen;
|
||||||
const valLen = blockView.getUint16(offset, false);
|
const valLen = blockView.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + valLen > blockData.byteLength) break;
|
||||||
const valBytes = blockData.slice(offset, offset + valLen);
|
const valBytes = blockData.slice(offset, offset + valLen);
|
||||||
offset += valLen;
|
offset += valLen;
|
||||||
|
|
||||||
@@ -185,6 +188,11 @@ export class SSTableReader {
|
|||||||
const bloomHashCount = this.view.getUint32(footerOffset + 16, false);
|
const bloomHashCount = this.view.getUint32(footerOffset + 16, false);
|
||||||
this.entryCount = this.view.getUint32(footerOffset + 20, false);
|
this.entryCount = this.view.getUint32(footerOffset + 20, false);
|
||||||
|
|
||||||
|
// v0.4.2-fix: 完整性校验 — 索引块必须完全落在文件内,否则视为残缺文件跳过
|
||||||
|
if (indexOffset + 4 > this.data.byteLength || indexOffset + indexSize > this.data.byteLength) {
|
||||||
|
return; // 残缺文件:无索引块可读,get/rangeScan 均返回空
|
||||||
|
}
|
||||||
|
|
||||||
// 解析索引块
|
// 解析索引块
|
||||||
this.parseIndexBlock(indexOffset, indexSize);
|
this.parseIndexBlock(indexOffset, indexSize);
|
||||||
|
|
||||||
@@ -204,8 +212,12 @@ export class SSTableReader {
|
|||||||
offset += 4;
|
offset += 4;
|
||||||
|
|
||||||
for (let i = 0; i < entryCount; i++) {
|
for (let i = 0; i < entryCount; i++) {
|
||||||
|
// v0.4.2-fix: 索引条目越界(keyLen/blockOffset/blockSize 超过文件长度)时中止解析,
|
||||||
|
// 已解析的有效条目仍可用于查询
|
||||||
|
if (offset + 2 > this.data.byteLength) break;
|
||||||
const keyLen = this.view.getUint16(offset, false);
|
const keyLen = this.view.getUint16(offset, false);
|
||||||
offset += 2;
|
offset += 2;
|
||||||
|
if (offset + keyLen + 8 > this.data.byteLength) break;
|
||||||
const key = new TextDecoder().decode(this.data.slice(offset, offset + keyLen));
|
const key = new TextDecoder().decode(this.data.slice(offset, offset + keyLen));
|
||||||
offset += keyLen;
|
offset += keyLen;
|
||||||
const blockOffset = this.view.getUint32(offset, false);
|
const blockOffset = this.view.getUint32(offset, false);
|
||||||
@@ -213,10 +225,27 @@ export class SSTableReader {
|
|||||||
const blockSize = this.view.getUint32(offset, false);
|
const blockSize = this.view.getUint32(offset, false);
|
||||||
offset += 4;
|
offset += 4;
|
||||||
|
|
||||||
|
// 跳过指向文件外的块(残缺写入产物),不抛异常
|
||||||
|
if (blockSize === 0 || blockOffset + blockSize > this.data.byteLength) continue;
|
||||||
|
|
||||||
this.indexEntries.push({ key, blockOffset, blockSize });
|
this.indexEntries.push({ key, blockOffset, blockSize });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 获取索引条目对应的数据块。
|
||||||
|
* 块偏移/大小越界(残缺 SSTable)时返回 null,由调用方跳过而非抛 RangeError。
|
||||||
|
*/
|
||||||
|
private getBlockData(entry: IndexEntry): Uint8Array | null {
|
||||||
|
if (entry.blockSize <= 0 || entry.blockOffset < 0) return null;
|
||||||
|
if (entry.blockOffset + entry.blockSize > this.data.byteLength) return null;
|
||||||
|
return new Uint8Array(
|
||||||
|
this.data.buffer,
|
||||||
|
this.data.byteOffset + entry.blockOffset,
|
||||||
|
entry.blockSize,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
/** 二分查找某 key 所在的 block 索引 */
|
/** 二分查找某 key 所在的 block 索引 */
|
||||||
private locateBlock(key: string): number {
|
private locateBlock(key: string): number {
|
||||||
let lo = 0;
|
let lo = 0;
|
||||||
|
|||||||
@@ -23,8 +23,17 @@ export interface IStorageBackend {
|
|||||||
read(key: string): Promise<ArrayBuffer | null>;
|
read(key: string): Promise<ArrayBuffer | null>;
|
||||||
/** 写入数据块 */
|
/** 写入数据块 */
|
||||||
write(key: string, data: ArrayBuffer): Promise<void>;
|
write(key: string, data: ArrayBuffer): Promise<void>;
|
||||||
|
/**
|
||||||
|
* 批量原子写入(v0.4.2-fix):多个 key 在单个底层事务中提交,
|
||||||
|
* 中断时整体回滚,不留半写状态。WAL count 与记录同事务保证一致性。
|
||||||
|
*/
|
||||||
|
writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
|
||||||
/** 删除数据块 */
|
/** 删除数据块 */
|
||||||
delete(key: string): Promise<void>;
|
delete(key: string): Promise<void>;
|
||||||
|
/**
|
||||||
|
* 批量原子删除(v0.4.2-fix):多个 key 在单个底层事务中提交。
|
||||||
|
*/
|
||||||
|
deleteMany(keys: string[]): Promise<void>;
|
||||||
/** 列出所有 key */
|
/** 列出所有 key */
|
||||||
listKeys(): Promise<string[]>;
|
listKeys(): Promise<string[]>;
|
||||||
/** 检查 key 是否存在 */
|
/** 检查 key 是否存在 */
|
||||||
@@ -91,6 +100,25 @@ export class IndexedDBBackend implements IStorageBackend {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 批量原子写入 — 单个 IDB 事务内写入多个 key。
|
||||||
|
* 中断时事务整体回滚,WAL 记录与 count 计数不会出现"记录在、计数丢"或反之的半写状态。
|
||||||
|
*/
|
||||||
|
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
||||||
|
const keys = Object.keys(entries);
|
||||||
|
if (keys.length === 0) return;
|
||||||
|
const db = this.ensureDB();
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const tx = db.transaction(this.storeName, 'readwrite');
|
||||||
|
const store = tx.objectStore(this.storeName);
|
||||||
|
for (const key of keys) {
|
||||||
|
store.put(entries[key], key);
|
||||||
|
}
|
||||||
|
tx.oncomplete = () => resolve();
|
||||||
|
tx.onerror = () => reject(new DatabaseError('Failed to batch write to AriaEngine store', 'ARIA_WRITE_ERROR'));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
async delete(key: string): Promise<void> {
|
async delete(key: string): Promise<void> {
|
||||||
const db = this.ensureDB();
|
const db = this.ensureDB();
|
||||||
return new Promise((resolve, reject) => {
|
return new Promise((resolve, reject) => {
|
||||||
@@ -101,6 +129,21 @@ export class IndexedDBBackend implements IStorageBackend {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 批量原子删除 — 单个 IDB 事务内删除多个 key */
|
||||||
|
async deleteMany(keys: string[]): Promise<void> {
|
||||||
|
if (keys.length === 0) return;
|
||||||
|
const db = this.ensureDB();
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const tx = db.transaction(this.storeName, 'readwrite');
|
||||||
|
const store = tx.objectStore(this.storeName);
|
||||||
|
for (const key of keys) {
|
||||||
|
store.delete(key);
|
||||||
|
}
|
||||||
|
tx.oncomplete = () => resolve();
|
||||||
|
tx.onerror = () => reject(new DatabaseError('Failed to batch delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
async listKeys(): Promise<string[]> {
|
async listKeys(): Promise<string[]> {
|
||||||
const db = this.ensureDB();
|
const db = this.ensureDB();
|
||||||
return new Promise((resolve, reject) => {
|
return new Promise((resolve, reject) => {
|
||||||
@@ -161,10 +204,22 @@ export class MemoryBackend implements IStorageBackend {
|
|||||||
this.store.set(key, data);
|
this.store.set(key, data);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
||||||
|
for (const [key, data] of Object.entries(entries)) {
|
||||||
|
this.store.set(key, data);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async delete(key: string): Promise<void> {
|
async delete(key: string): Promise<void> {
|
||||||
this.store.delete(key);
|
this.store.delete(key);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async deleteMany(keys: string[]): Promise<void> {
|
||||||
|
for (const key of keys) {
|
||||||
|
this.store.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async listKeys(): Promise<string[]> {
|
async listKeys(): Promise<string[]> {
|
||||||
return Array.from(this.store.keys());
|
return Array.from(this.store.keys());
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -52,6 +52,20 @@ export class OPFSBackend implements IStorageBackend {
|
|||||||
return this.writeQueue;
|
return this.writeQueue;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 批量写入 — 串行队列内逐个落盘(OPFS 无跨文件事务,顺序保证一致) */
|
||||||
|
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
||||||
|
if (!this.dbDir) return;
|
||||||
|
this.writeQueue = this.writeQueue.then(async () => {
|
||||||
|
for (const [key, data] of Object.entries(entries)) {
|
||||||
|
const fh = await this.dbDir!.getFileHandle(key, { create: true });
|
||||||
|
const writable = await fh.createWritable();
|
||||||
|
await writable.write(data);
|
||||||
|
await writable.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return this.writeQueue;
|
||||||
|
}
|
||||||
|
|
||||||
async delete(key: string): Promise<void> {
|
async delete(key: string): Promise<void> {
|
||||||
if (!this.dbDir) return;
|
if (!this.dbDir) return;
|
||||||
this.writeQueue = this.writeQueue.then(async () => {
|
this.writeQueue = this.writeQueue.then(async () => {
|
||||||
@@ -60,6 +74,17 @@ export class OPFSBackend implements IStorageBackend {
|
|||||||
return this.writeQueue;
|
return this.writeQueue;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 批量删除 — 串行队列内逐个删除 */
|
||||||
|
async deleteMany(keys: string[]): Promise<void> {
|
||||||
|
if (!this.dbDir) return;
|
||||||
|
this.writeQueue = this.writeQueue.then(async () => {
|
||||||
|
for (const key of keys) {
|
||||||
|
try { await this.dbDir!.removeEntry(key); } catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return this.writeQueue;
|
||||||
|
}
|
||||||
|
|
||||||
async listKeys(): Promise<string[]> {
|
async listKeys(): Promise<string[]> {
|
||||||
if (!this.dbDir) return [];
|
if (!this.dbDir) return [];
|
||||||
const keys: string[] = [];
|
const keys: string[] = [];
|
||||||
|
|||||||
@@ -26,6 +26,12 @@ export class MVCCManager {
|
|||||||
/** 全局提交序列号(用于可见性判断) */
|
/** 全局提交序列号(用于可见性判断) */
|
||||||
private globalCommitLsn = 0;
|
private globalCommitLsn = 0;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 每个事务写入的 tableKey 集合 —
|
||||||
|
* commit/rollback 只遍历本事务写过的 key,避免全库版本链扫描(大表事务 O(N) → O(写入数))
|
||||||
|
*/
|
||||||
|
private txnWriteKeys: Map<number, Set<string>> = new Map();
|
||||||
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// 事务管理
|
// 事务管理
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
@@ -39,6 +45,7 @@ export class MVCCManager {
|
|||||||
snapshotLsn: this.globalCommitLsn,
|
snapshotLsn: this.globalCommitLsn,
|
||||||
startTime: Date.now(),
|
startTime: Date.now(),
|
||||||
});
|
});
|
||||||
|
this.txnWriteKeys.set(txnId, new Set());
|
||||||
return txnId;
|
return txnId;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -50,17 +57,23 @@ export class MVCCManager {
|
|||||||
txn.state = TransactionState.COMMITTED;
|
txn.state = TransactionState.COMMITTED;
|
||||||
this.globalCommitLsn++;
|
this.globalCommitLsn++;
|
||||||
|
|
||||||
// 标记此事务写入的所有版本为已提交
|
// v0.4.2-fix: 仅标记本事务写入的版本(此前遍历全库 versionStore)
|
||||||
for (const [, versions] of this.versionStore) {
|
const writeKeys = this.txnWriteKeys.get(txnId);
|
||||||
|
if (writeKeys) {
|
||||||
|
for (const tableKey of writeKeys) {
|
||||||
|
const versions = this.versionStore.get(tableKey);
|
||||||
|
if (!versions) continue;
|
||||||
for (const version of versions) {
|
for (const version of versions) {
|
||||||
if (version.txnId === txnId) {
|
if (version.txnId === txnId) {
|
||||||
version.committed = true;
|
version.committed = true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// 清理已提交事务的记录
|
// 清理已提交事务的记录
|
||||||
this.activeTxns.delete(txnId);
|
this.activeTxns.delete(txnId);
|
||||||
|
this.txnWriteKeys.delete(txnId);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 回滚事务 */
|
/** 回滚事务 */
|
||||||
@@ -70,8 +83,12 @@ export class MVCCManager {
|
|||||||
|
|
||||||
txn.state = TransactionState.ABORTED;
|
txn.state = TransactionState.ABORTED;
|
||||||
|
|
||||||
// 移除此事务写入的所有版本
|
// v0.4.2-fix: 仅移除本事务写入的版本(此前遍历全库 versionStore)
|
||||||
for (const [tableKey, versions] of this.versionStore) {
|
const writeKeys = this.txnWriteKeys.get(txnId);
|
||||||
|
if (writeKeys) {
|
||||||
|
for (const tableKey of writeKeys) {
|
||||||
|
const versions = this.versionStore.get(tableKey);
|
||||||
|
if (!versions) continue;
|
||||||
const filtered = versions.filter((v) => v.txnId !== txnId);
|
const filtered = versions.filter((v) => v.txnId !== txnId);
|
||||||
if (filtered.length === 0) {
|
if (filtered.length === 0) {
|
||||||
this.versionStore.delete(tableKey);
|
this.versionStore.delete(tableKey);
|
||||||
@@ -79,8 +96,10 @@ export class MVCCManager {
|
|||||||
this.versionStore.set(tableKey, filtered);
|
this.versionStore.set(tableKey, filtered);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
this.activeTxns.delete(txnId);
|
this.activeTxns.delete(txnId);
|
||||||
|
this.txnWriteKeys.delete(txnId);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 检查事务是否活跃 */
|
/** 检查事务是否活跃 */
|
||||||
@@ -114,6 +133,8 @@ export class MVCCManager {
|
|||||||
|
|
||||||
versions.push(newVersion);
|
versions.push(newVersion);
|
||||||
this.versionStore.set(tableKey, versions);
|
this.versionStore.set(tableKey, versions);
|
||||||
|
// v0.4.2-fix: 记录本事务写过的 key(commit/rollback 精准清理)
|
||||||
|
this.txnWriteKeys.get(txnId)?.add(tableKey);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -186,9 +207,14 @@ export class MVCCManager {
|
|||||||
/**
|
/**
|
||||||
* v0.3.3: 丢弃指定事务的所有版本记录,但保留事务登记(Savepoint 回滚用)。
|
* v0.3.3: 丢弃指定事务的所有版本记录,但保留事务登记(Savepoint 回滚用)。
|
||||||
* 快照数据由调用方(引擎 txnSnapshot)负责恢复。
|
* 快照数据由调用方(引擎 txnSnapshot)负责恢复。
|
||||||
|
* v0.4.2-fix: 仅遍历本事务写过的 key(此前全库扫描)。
|
||||||
*/
|
*/
|
||||||
discardVersions(txnId: number): void {
|
discardVersions(txnId: number): void {
|
||||||
for (const [tableKey, versions] of this.versionStore) {
|
const writeKeys = this.txnWriteKeys.get(txnId);
|
||||||
|
if (!writeKeys) return;
|
||||||
|
for (const tableKey of writeKeys) {
|
||||||
|
const versions = this.versionStore.get(tableKey);
|
||||||
|
if (!versions) continue;
|
||||||
const filtered = versions.filter((v) => v.txnId !== txnId);
|
const filtered = versions.filter((v) => v.txnId !== txnId);
|
||||||
if (filtered.length === 0) {
|
if (filtered.length === 0) {
|
||||||
this.versionStore.delete(tableKey);
|
this.versionStore.delete(tableKey);
|
||||||
|
|||||||
+333
-18
@@ -23,31 +23,160 @@ export class IndexedDBEngine implements IStorageEngine {
|
|||||||
private txActive = false;
|
private txActive = false;
|
||||||
|
|
||||||
async open(dbName: string, version: number): Promise<void> {
|
async open(dbName: string, version: number): Promise<void> {
|
||||||
this.dbName = dbName; this.version = version;
|
// v0.4.2-fix (P0-3): version < 1 归一化为 1(indexedDB.open(name, 0) 抛原生 TypeError)
|
||||||
await this.memoryCache.open(dbName, version);
|
const normalizedVersion = version >= 1 ? Math.floor(version) : 1;
|
||||||
return new Promise((resolve, reject) => {
|
this.dbName = dbName;
|
||||||
const request = indexedDB.open(dbName, version);
|
this.version = normalizedVersion;
|
||||||
request.onsuccess = async () => {
|
await this.memoryCache.open(dbName, normalizedVersion);
|
||||||
this.db = request.result;
|
|
||||||
// 多标签页冲突处理:其他标签页升级版本时自动关闭当前连接
|
// v0.4.2-fix (P0-2/P2-8): 版本自适应打开 + blocked 重试
|
||||||
|
this.db = await this.openDatabaseWithRetry(dbName, normalizedVersion);
|
||||||
|
this.setupVersionChangeHandler();
|
||||||
|
|
||||||
|
// v0.4.2-fix (P2-7): 确保 schema/meta 持久化 store 存在(新库或旧库升级时创建),
|
||||||
|
// 否则迁移版本等库内元数据无处落盘
|
||||||
|
await this.ensureSchemaStore();
|
||||||
|
|
||||||
|
try {
|
||||||
|
// v0.3.2: reopen 后从 IDB 重建 schema(schema 此前只存内存缓存,重开连接即丢失)
|
||||||
|
await this.rebuildSchemaFromIDB();
|
||||||
|
} catch (error) {
|
||||||
|
throw new DatabaseError(`Failed to restore schema for "${dbName}"`, 'IDB_SCHEMA_RESTORE_ERROR', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 多标签页冲突处理 — 其他标签页升级版本时自动关闭当前连接 */
|
||||||
|
private setupVersionChangeHandler(): void {
|
||||||
|
if (!this.db) return;
|
||||||
this.db.onversionchange = () => {
|
this.db.onversionchange = () => {
|
||||||
if (this.db) {
|
if (this.db) {
|
||||||
this.db.close();
|
this.db.close();
|
||||||
this.db = null;
|
this.db = null;
|
||||||
// eslint-disable-next-line no-console
|
// eslint-disable-next-line no-console
|
||||||
console.warn(`[metona-sqlark] Database "${dbName}" was upgraded in another tab. Connection closed. Please re-open.`);
|
console.warn(`[metona-sqlark] Database "${this.dbName}" was upgraded in another tab. Connection closed. Please re-open.`);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 打开 IndexedDB 连接。
|
||||||
|
* - P0-2: 请求版本低于库实际版本(VersionError)时,先无版本参数探测库当前版本,
|
||||||
|
* 再以实际版本重开(建表每张表版本号 +1,config.version 会过期)
|
||||||
|
* - P2-8: onblocked 为瞬时状态(另一连接短暂持有),等待后重试多次,超时才抛 IDB_BLOCKED
|
||||||
|
*/
|
||||||
|
private async openDatabaseWithRetry(dbName: string, requestedVersion: number): Promise<IDBDatabase> {
|
||||||
|
const BLOCKED_RETRIES = 10;
|
||||||
|
let effectiveVersion = requestedVersion;
|
||||||
|
for (let attempt = 0; attempt < BLOCKED_RETRIES; attempt++) {
|
||||||
try {
|
try {
|
||||||
// v0.3.2: reopen 后从 IDB 重建 schema(schema 此前只存内存缓存,重开连接即丢失)
|
return await this.openRequest(dbName, effectiveVersion, 200 + attempt * 150);
|
||||||
await this.rebuildSchemaFromIDB();
|
|
||||||
resolve();
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
reject(new DatabaseError(`Failed to restore schema for "${dbName}"`, 'IDB_SCHEMA_RESTORE_ERROR', error));
|
const err = error as { name?: string };
|
||||||
|
if (err && err.name === 'VersionError') {
|
||||||
|
const currentVersion = await this.resolveCurrentVersion(dbName);
|
||||||
|
if (currentVersion >= 1 && currentVersion !== effectiveVersion) {
|
||||||
|
effectiveVersion = currentVersion;
|
||||||
|
this.version = currentVersion;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Failed to open IndexedDB "${dbName}": version mismatch`,
|
||||||
|
'IDB_VERSION_ERROR',
|
||||||
|
error,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (err && err.name === 'BlockedError') {
|
||||||
|
// 另一连接短暂持有 → 等待后重试
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 50 * (attempt + 1)));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
throw new DatabaseError(`Failed to open IndexedDB "${dbName}"`, 'IDB_OPEN_ERROR', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new DatabaseError(`IndexedDB "${dbName}" is blocked`, 'IDB_BLOCKED');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 发起一次 indexedDB.open 请求(success/error/blocked 三态收敛)。
|
||||||
|
* onblocked 不立即失败:阻塞解除后 success 仍会触发,仅超时兜底判失败,
|
||||||
|
* 避免"拒绝后连接迟到成功"泄漏未关闭的数据库连接。
|
||||||
|
*/
|
||||||
|
private openRequest(dbName: string, version: number, timeoutMs: number): Promise<IDBDatabase> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(dbName, version);
|
||||||
|
let settled = false;
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
if (settled) return;
|
||||||
|
settled = true;
|
||||||
|
// 兼容无 DOMException 构造环境
|
||||||
|
const blockedError = typeof DOMException !== 'undefined'
|
||||||
|
? new DOMException('IndexedDB open is blocked', 'BlockedError')
|
||||||
|
: Object.assign(new Error('IndexedDB open is blocked'), { name: 'BlockedError' });
|
||||||
|
reject(blockedError);
|
||||||
|
}, timeoutMs);
|
||||||
|
request.onsuccess = () => {
|
||||||
|
if (settled) {
|
||||||
|
// 超时判失败后连接迟到成功:立即关闭,避免阻塞后续版本升级
|
||||||
|
request.result.close();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
settled = true;
|
||||||
|
clearTimeout(timeout);
|
||||||
|
resolve(request.result);
|
||||||
|
};
|
||||||
|
request.onerror = () => {
|
||||||
|
if (settled) return;
|
||||||
|
settled = true;
|
||||||
|
clearTimeout(timeout);
|
||||||
|
reject(request.error ?? new Error('Unknown IndexedDB open error'));
|
||||||
|
};
|
||||||
|
request.onblocked = () => {
|
||||||
|
// 保持等待,不拒绝(阻塞解除后 success 会触发;超时由 timer 兜底)
|
||||||
|
};
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 无版本参数打开库,解析其当前实际版本号(随后立即关闭) */
|
||||||
|
private resolveCurrentVersion(dbName: string): Promise<number> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(dbName);
|
||||||
|
request.onsuccess = () => {
|
||||||
|
const actualVersion = request.result.version;
|
||||||
|
request.result.close();
|
||||||
|
resolve(actualVersion);
|
||||||
|
};
|
||||||
|
request.onerror = () => {
|
||||||
|
reject(request.error ?? new Error('Failed to resolve IndexedDB version'));
|
||||||
|
};
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix (P2-7): 确保 __metona_schema store 存在。
|
||||||
|
* 新库(或版本升级前创建的旧库)没有该 store 时,通过一次版本升级创建,
|
||||||
|
* 使 getMeta/setMeta(迁移版本持久化)始终可用。
|
||||||
|
*/
|
||||||
|
private async ensureSchemaStore(): Promise<void> {
|
||||||
|
if (!this.db) return;
|
||||||
|
if (this.db.objectStoreNames.contains('__metona_schema')) return;
|
||||||
|
const newVersion = this.db.version + 1;
|
||||||
|
this.db.close();
|
||||||
|
this.db = await new Promise<IDBDatabase>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(this.dbName, newVersion);
|
||||||
|
request.onupgradeneeded = () => {
|
||||||
|
const idb = request.result;
|
||||||
|
if (!idb.objectStoreNames.contains('__metona_schema')) {
|
||||||
|
idb.createObjectStore('__metona_schema', { keyPath: 'name' });
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
request.onerror = () => reject(new DatabaseError(`Failed to open IndexedDB "${dbName}"`, 'IDB_OPEN_ERROR', request.error));
|
request.onsuccess = () => {
|
||||||
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${dbName}" is blocked`, 'IDB_BLOCKED'));
|
this.db = request.result;
|
||||||
|
this.setupVersionChangeHandler();
|
||||||
|
resolve(request.result);
|
||||||
|
};
|
||||||
|
request.onerror = () => reject(
|
||||||
|
new DatabaseError('Failed to create schema store', 'IDB_UPGRADE_ERROR', request.error),
|
||||||
|
);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -130,6 +259,84 @@ export class IndexedDBEngine implements IStorageEngine {
|
|||||||
await this.memoryCache.close();
|
await this.memoryCache.close();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 自愈 — 从磁盘重建内存 schema 与数据(schema 丢失/内存不一致时调用)。
|
||||||
|
* 无删库需求即可恢复可用的库。
|
||||||
|
*/
|
||||||
|
async repair(): Promise<void> {
|
||||||
|
if (!this.db) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
|
||||||
|
await this.memoryCache.close();
|
||||||
|
await this.memoryCache.open(this.dbName, this.version);
|
||||||
|
await this.rebuildSchemaFromIDB();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 清空全部数据与表结构(含持久化 schema 记录),保留库本身。
|
||||||
|
* 单个版本升级事务内原子完成。
|
||||||
|
*/
|
||||||
|
async clearAll(): Promise<void> {
|
||||||
|
const db = this.ensureDB();
|
||||||
|
// 先清内存缓存
|
||||||
|
const tableNames = await this.memoryCache.getTableNames();
|
||||||
|
for (const name of tableNames) {
|
||||||
|
await this.memoryCache.dropTable(name);
|
||||||
|
}
|
||||||
|
// 重置 IDB:删除所有表 store + 清空 schema/meta store
|
||||||
|
const newVersion = db.version + 1;
|
||||||
|
db.close();
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(this.dbName, newVersion);
|
||||||
|
request.onupgradeneeded = (event) => {
|
||||||
|
const idb = (event.target as IDBOpenDBRequest).result;
|
||||||
|
const toDelete = Array.from(idb.objectStoreNames).filter((n) => n !== '__metona_schema');
|
||||||
|
for (const name of toDelete) {
|
||||||
|
idb.deleteObjectStore(name);
|
||||||
|
}
|
||||||
|
if (!idb.objectStoreNames.contains('__metona_schema')) {
|
||||||
|
idb.createObjectStore('__metona_schema', { keyPath: 'name' });
|
||||||
|
} else {
|
||||||
|
// 保留 store 但清空内容(含持久化 schema 与 meta 记录)
|
||||||
|
const tx = (event.target as IDBOpenDBRequest).transaction!;
|
||||||
|
tx.objectStore('__metona_schema').clear();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
request.onsuccess = () => {
|
||||||
|
this.db = request.result;
|
||||||
|
this.setupVersionChangeHandler();
|
||||||
|
resolve();
|
||||||
|
};
|
||||||
|
request.onerror = () => reject(new DatabaseError('Failed to clear database', 'IDB_CLEAR_ERROR', request.error));
|
||||||
|
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${this.dbName}" is blocked`, 'IDB_BLOCKED'));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 库内元数据(v0.4.2-fix:迁移版本持久化用,复用 __metona_schema store) ----
|
||||||
|
|
||||||
|
async getMeta(key: string): Promise<string | null> {
|
||||||
|
const db = this.ensureDB();
|
||||||
|
if (!db.objectStoreNames.contains('__metona_schema')) return null;
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const req = db.transaction('__metona_schema', 'readonly')
|
||||||
|
.objectStore('__metona_schema').get(`__meta:${key}`);
|
||||||
|
req.onsuccess = () => {
|
||||||
|
const rec = req.result as { schema?: unknown } | undefined;
|
||||||
|
resolve(rec && typeof rec.schema === 'string' ? rec.schema : null);
|
||||||
|
};
|
||||||
|
req.onerror = () => reject(req.error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async setMeta(key: string, value: string): Promise<void> {
|
||||||
|
const db = this.ensureDB();
|
||||||
|
if (!db.objectStoreNames.contains('__metona_schema')) return;
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const tx = db.transaction('__metona_schema', 'readwrite');
|
||||||
|
tx.objectStore('__metona_schema').put({ name: `__meta:${key}`, schema: value });
|
||||||
|
tx.oncomplete = () => resolve();
|
||||||
|
tx.onerror = () => reject(new DatabaseError(`Failed to persist meta "${key}"`, 'IDB_META_ERROR', tx.error));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
isOpen(): boolean { return this.db !== null; }
|
isOpen(): boolean { return this.db !== null; }
|
||||||
|
|
||||||
// ---- 表管理 ----
|
// ---- 表管理 ----
|
||||||
@@ -149,6 +356,47 @@ export class IndexedDBEngine implements IStorageEngine {
|
|||||||
async getTableNames(): Promise<string[]> { return Array.from(this.ensureDB().objectStoreNames); }
|
async getTableNames(): Promise<string[]> { return Array.from(this.ensureDB().objectStoreNames); }
|
||||||
async getTableSchema(tableName: string): Promise<TableSchema | null> { return this.memoryCache.getTableSchema(tableName); }
|
async getTableSchema(tableName: string): Promise<TableSchema | null> { return this.memoryCache.getTableSchema(tableName); }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 引擎级 ALTER TABLE — schema 持久化到 __metona_schema store,
|
||||||
|
* 重启后 ALTER 不丢失(此前通用路径只改内存引用,重启回退;DROP 的行数据也没真正删)。
|
||||||
|
*/
|
||||||
|
async alterTable(
|
||||||
|
tableName: string,
|
||||||
|
action: 'ADD' | 'DROP',
|
||||||
|
column: import('../constants').ColumnDef & { name: string },
|
||||||
|
): Promise<void> {
|
||||||
|
await this.memoryCache.alterTable(tableName, action, column);
|
||||||
|
if (this.txActive) return; // 事务中:commit 时统一 flushToIDB 同步
|
||||||
|
const schema = await this.memoryCache.getTableSchema(tableName);
|
||||||
|
if (schema) {
|
||||||
|
await this.persistSchema(schema);
|
||||||
|
}
|
||||||
|
if (action === 'DROP') {
|
||||||
|
// 重写 IDB 存储行:移除该列键(store.put 经 keyPath 自动覆盖原行)
|
||||||
|
const db = this.ensureDB();
|
||||||
|
const rows = await this.idbFind(tableName, { table: tableName });
|
||||||
|
const rewritten = rows.map((row) => {
|
||||||
|
if (column.name in row) {
|
||||||
|
const copy = { ...row };
|
||||||
|
delete (copy as Record<string, unknown>)[column.name];
|
||||||
|
return copy;
|
||||||
|
}
|
||||||
|
return row;
|
||||||
|
});
|
||||||
|
if (rewritten.length > 0) {
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const tx = db.transaction(tableName, 'readwrite');
|
||||||
|
const store = tx.objectStore(tableName);
|
||||||
|
for (const row of rewritten) {
|
||||||
|
store.put(row);
|
||||||
|
}
|
||||||
|
tx.oncomplete = () => resolve();
|
||||||
|
tx.onerror = () => reject(new DatabaseError(`Failed to rewrite "${tableName}" after DROP COLUMN`, 'IDB_TX_ERROR', tx.error));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---- CRUD ----
|
// ---- CRUD ----
|
||||||
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
||||||
const pks = await this.memoryCache.insert(tableName, rows);
|
const pks = await this.memoryCache.insert(tableName, rows);
|
||||||
@@ -256,10 +504,10 @@ export class IndexedDBEngine implements IStorageEngine {
|
|||||||
|
|
||||||
async commitTransaction(): Promise<void> {
|
async commitTransaction(): Promise<void> {
|
||||||
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
// 确认内存层的变更
|
// v0.4.2-fix: 先刷盘后提交内存快照 — 此前先 memoryCache.commitTransaction()
|
||||||
await this.memoryCache.commitTransaction();
|
// 再 flushToIDB,flush 失败时 snapshot 已丢,回滚报 TX_NONE 且内存数据已确认
|
||||||
// 批量将内存数据刷到 IndexedDB
|
|
||||||
await this.flushToIDB();
|
await this.flushToIDB();
|
||||||
|
await this.memoryCache.commitTransaction();
|
||||||
this.txActive = false;
|
this.txActive = false;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -482,10 +730,77 @@ export class IndexedDBEngine implements IStorageEngine {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** 持久化单个表 schema 到 __metona_schema store(v0.4.2-fix: ALTER TABLE 用) */
|
||||||
|
private async persistSchema(schema: TableSchema): Promise<void> {
|
||||||
|
const db = this.ensureDB();
|
||||||
|
if (!db.objectStoreNames.contains('__metona_schema')) return;
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const tx = db.transaction('__metona_schema', 'readwrite');
|
||||||
|
tx.objectStore('__metona_schema').put({ name: schema.name, schema: JSON.stringify(schema) });
|
||||||
|
tx.oncomplete = () => resolve();
|
||||||
|
tx.onerror = () => reject(new DatabaseError(`Failed to persist schema for "${schema.name}"`, 'IDB_SCHEMA_ERROR', tx.error));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
|
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
|
||||||
private async flushToIDB(): Promise<void> {
|
private async flushToIDB(): Promise<void> {
|
||||||
const tableNames = await this.memoryCache.getTableNames();
|
const tableNames = await this.memoryCache.getTableNames();
|
||||||
const db = this.ensureDB();
|
let db = this.ensureDB();
|
||||||
|
|
||||||
|
// v0.4.2-fix: 事务内 DDL 只更新内存,commit 时同步 IDB 的 objectStore 结构:
|
||||||
|
// 缺失的表 store 创建(并持久化 schema)、已删除的 store 移除(防止重启幽灵表)
|
||||||
|
const idbStores = Array.from(db.objectStoreNames);
|
||||||
|
const missing = tableNames.filter((t) => !idbStores.includes(t));
|
||||||
|
const stale = idbStores.filter((s) => s !== '__metona_schema' && !tableNames.includes(s));
|
||||||
|
if (missing.length > 0 || stale.length > 0) {
|
||||||
|
// 升级事务内是同步上下文,先异步收集缺失表的 schema(主键列定义)
|
||||||
|
const schemaMap = new Map<string, TableSchema>();
|
||||||
|
for (const name of missing) {
|
||||||
|
const s = await this.memoryCache.getTableSchema(name);
|
||||||
|
if (s) schemaMap.set(name, s);
|
||||||
|
}
|
||||||
|
const newVersion = db.version + 1;
|
||||||
|
db.close();
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(this.dbName, newVersion);
|
||||||
|
request.onupgradeneeded = (event) => {
|
||||||
|
const idb = (event.target as IDBOpenDBRequest).result;
|
||||||
|
for (const name of stale) {
|
||||||
|
idb.deleteObjectStore(name);
|
||||||
|
}
|
||||||
|
for (const name of missing) {
|
||||||
|
const schema = schemaMap.get(name);
|
||||||
|
const pkColumn = schema
|
||||||
|
? (Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0])
|
||||||
|
: undefined;
|
||||||
|
idb.createObjectStore(name, { keyPath: pkColumn });
|
||||||
|
}
|
||||||
|
// 事务内 drop 的表:同步清理持久化 schema 记录(防止重启后幽灵表恢复)
|
||||||
|
if (idb.objectStoreNames.contains('__metona_schema') && stale.length > 0) {
|
||||||
|
const tx = (event.target as IDBOpenDBRequest).transaction!;
|
||||||
|
const store = tx.objectStore('__metona_schema');
|
||||||
|
for (const name of stale) {
|
||||||
|
store.delete(name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
request.onsuccess = () => {
|
||||||
|
this.db = request.result;
|
||||||
|
this.setupVersionChangeHandler();
|
||||||
|
resolve();
|
||||||
|
};
|
||||||
|
request.onerror = () => reject(new DatabaseError('Failed to sync stores after transaction', 'IDB_UPGRADE_ERROR', request.error));
|
||||||
|
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${this.dbName}" is blocked`, 'IDB_BLOCKED'));
|
||||||
|
});
|
||||||
|
// 事务内新建表的 schema 一并持久化(此前只建 store 不存 schema → 重启后约束推断丢失)
|
||||||
|
for (const name of missing) {
|
||||||
|
const schema = await this.memoryCache.getTableSchema(name);
|
||||||
|
if (schema) await this.persistSchema(schema);
|
||||||
|
}
|
||||||
|
// v0.4.2-fix: DDL 升级会 close 旧连接并重新 open — 重新取 db 引用,
|
||||||
|
// 否则下方数据 flush 用已关闭的连接抛 InvalidStateError
|
||||||
|
db = this.ensureDB();
|
||||||
|
}
|
||||||
|
|
||||||
// 每个表在一个单独的 IDB 事务中完成 clear+insert,保证原子性
|
// 每个表在一个单独的 IDB 事务中完成 clear+insert,保证原子性
|
||||||
for (const tableName of tableNames) {
|
for (const tableName of tableNames) {
|
||||||
|
|||||||
@@ -95,4 +95,18 @@ export interface IStorageEngine {
|
|||||||
|
|
||||||
/** 在线备份:导出全库一致性快照 */
|
/** 在线备份:导出全库一致性快照 */
|
||||||
backup?(): Promise<Record<string, Record<string, unknown>[]>>;
|
backup?(): Promise<Record<string, Record<string, unknown>[]>>;
|
||||||
|
|
||||||
|
// ---- 自愈/重置 (可选,v0.4.2-fix) ----
|
||||||
|
|
||||||
|
/** 崩溃恢复自愈:校验并清理损坏数据、恢复一致性(检测到异常后调用,无需删库重建) */
|
||||||
|
repair?(): Promise<void>;
|
||||||
|
|
||||||
|
/** 清空全部数据与表结构(保留库本身,供演示页刷新/重建用) */
|
||||||
|
clearAll?(): Promise<void>;
|
||||||
|
|
||||||
|
/** 读取库内元数据(迁移版本持久化用) */
|
||||||
|
getMeta?(key: string): Promise<string | null>;
|
||||||
|
|
||||||
|
/** 写入库内元数据(迁移版本持久化用) */
|
||||||
|
setMeta?(key: string, value: string): Promise<void>;
|
||||||
}
|
}
|
||||||
|
|||||||
+118
-6
@@ -15,6 +15,8 @@ export class MemoryEngine implements IStorageEngine {
|
|||||||
private schemas: Map<string, TableSchema> = new Map();
|
private schemas: Map<string, TableSchema> = new Map();
|
||||||
private indexes: Map<string, Map<string, Map<unknown, Set<string>>>> = new Map();
|
private indexes: Map<string, Map<string, Map<unknown, Set<string>>>> = new Map();
|
||||||
private opened = false;
|
private opened = false;
|
||||||
|
/** v0.4.2-fix: 库内元数据(迁移版本持久化用) */
|
||||||
|
private metaStore: Map<string, string> = new Map();
|
||||||
|
|
||||||
// ---- 事务快照 ----
|
// ---- 事务快照 ----
|
||||||
private snapshot: {
|
private snapshot: {
|
||||||
@@ -32,17 +34,45 @@ export class MemoryEngine implements IStorageEngine {
|
|||||||
this.opened = true;
|
this.opened = true;
|
||||||
}
|
}
|
||||||
async close(): Promise<void> {
|
async close(): Promise<void> {
|
||||||
this.tables.clear(); this.schemas.clear(); this.indexes.clear(); this.opened = false;
|
this.tables.clear(); this.schemas.clear(); this.indexes.clear(); this.metaStore.clear(); this.opened = false;
|
||||||
}
|
}
|
||||||
isOpen(): boolean { return this.opened; }
|
isOpen(): boolean { return this.opened; }
|
||||||
|
|
||||||
|
// ---- v0.4.2-fix: 自愈 / 重置 / 元数据 ----
|
||||||
|
|
||||||
|
/** 内存引擎无需修复(无持久化损坏概念) */
|
||||||
|
async repair(): Promise<void> { return; }
|
||||||
|
|
||||||
|
/** 清空全部数据与表结构 */
|
||||||
|
async clearAll(): Promise<void> {
|
||||||
|
const names = Array.from(this.schemas.keys());
|
||||||
|
for (const name of names) {
|
||||||
|
await this.dropTable(name);
|
||||||
|
}
|
||||||
|
this.metaStore.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
async getMeta(key: string): Promise<string | null> {
|
||||||
|
return this.metaStore.get(key) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async setMeta(key: string, value: string): Promise<void> {
|
||||||
|
this.metaStore.set(key, value);
|
||||||
|
}
|
||||||
|
|
||||||
// ---- 表管理 ----
|
// ---- 表管理 ----
|
||||||
async createTable(schema: TableSchema): Promise<void> {
|
async createTable(schema: TableSchema): Promise<void> {
|
||||||
if (this.schemas.has(schema.name)) throw new DatabaseError(`Table "${schema.name}" already exists`, 'TABLE_EXISTS');
|
if (this.schemas.has(schema.name)) throw new DatabaseError(`Table "${schema.name}" already exists`, 'TABLE_EXISTS');
|
||||||
this.schemas.set(schema.name, schema);
|
// v0.4.2-fix: 存储 schema 深拷贝 — 此前 Hybrid.reloadMemoryFromDisk 直接存入
|
||||||
|
// disk 引擎的 schema 引用,内存/磁盘引擎共享同一对象,任一引擎 ALTER 都会污染对方
|
||||||
|
const copy: TableSchema = { name: schema.name, columns: {} };
|
||||||
|
for (const [colName, colDef] of Object.entries(schema.columns)) {
|
||||||
|
copy.columns[colName] = { ...colDef };
|
||||||
|
}
|
||||||
|
this.schemas.set(schema.name, copy);
|
||||||
this.tables.set(schema.name, new Map());
|
this.tables.set(schema.name, new Map());
|
||||||
const tableIndexes = new Map<string, Map<unknown, Set<string>>>();
|
const tableIndexes = new Map<string, Map<unknown, Set<string>>>();
|
||||||
for (const [colName, colDef] of Object.entries(schema.columns)) {
|
for (const [colName, colDef] of Object.entries(copy.columns)) {
|
||||||
if (colDef.index || colDef.unique) tableIndexes.set(colName, new Map());
|
if (colDef.index || colDef.unique) tableIndexes.set(colName, new Map());
|
||||||
}
|
}
|
||||||
this.indexes.set(schema.name, tableIndexes);
|
this.indexes.set(schema.name, tableIndexes);
|
||||||
@@ -57,6 +87,35 @@ export class MemoryEngine implements IStorageEngine {
|
|||||||
async getTableNames(): Promise<string[]> { return Array.from(this.schemas.keys()); }
|
async getTableNames(): Promise<string[]> { return Array.from(this.schemas.keys()); }
|
||||||
async getTableSchema(tableName: string): Promise<TableSchema | null> { return this.schemas.get(tableName) ?? null; }
|
async getTableSchema(tableName: string): Promise<TableSchema | null> { return this.schemas.get(tableName) ?? null; }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: 引擎级 ALTER TABLE — 直接修改内存 schema 引用并清理行数据。
|
||||||
|
* (此前走 executor 通用路径,行为相同;统一到引擎层保证 Hybrid/IndexedDB 委托一致性)
|
||||||
|
*/
|
||||||
|
async alterTable(
|
||||||
|
tableName: string,
|
||||||
|
action: 'ADD' | 'DROP',
|
||||||
|
column: import('../constants').ColumnDef & { name: string },
|
||||||
|
): Promise<void> {
|
||||||
|
this.ensureTable(tableName);
|
||||||
|
const schema = this.schemas.get(tableName)!;
|
||||||
|
if (action === 'ADD') {
|
||||||
|
if (schema.columns[column.name]) {
|
||||||
|
throw new DatabaseError(`Column "${column.name}" already exists in table "${tableName}"`, 'COLUMN_EXISTS');
|
||||||
|
}
|
||||||
|
schema.columns[column.name] = column;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!schema.columns[column.name]) {
|
||||||
|
throw new DatabaseError(`Column "${column.name}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
||||||
|
}
|
||||||
|
delete schema.columns[column.name];
|
||||||
|
// 清理已有行中该列的值(find 返回行引用,直接删除生效)
|
||||||
|
const table = this.tables.get(tableName)!;
|
||||||
|
for (const row of table.values()) {
|
||||||
|
if (column.name in row) delete row[column.name];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---- CRUD ----
|
// ---- CRUD ----
|
||||||
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
@@ -124,22 +183,75 @@ export class MemoryEngine implements IStorageEngine {
|
|||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
const schema = this.schemas.get(tableName)!;
|
const schema = this.schemas.get(tableName)!;
|
||||||
const table = this.tables.get(tableName)!;
|
const table = this.tables.get(tableName)!;
|
||||||
|
const pkCol = this.getPrimaryKey(schema);
|
||||||
let count = 0;
|
let count = 0;
|
||||||
for (const [pk, row] of table) {
|
// v0.4.2-fix: 迭代期间会 delete/set 同一 Map(主键变更)→ 拷贝快照避免跳过/重复
|
||||||
|
for (const [pk, row] of [...table]) {
|
||||||
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
|
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
|
||||||
// v0.3.3: 先移除旧值索引条目(修复 update 后唯一约束被绕过、按新值查索引丢行)
|
// v0.3.3: 先移除旧值索引条目(修复 update 后唯一约束被绕过、按新值查索引丢行)
|
||||||
this.removeIndexEntries(tableName, row, pk);
|
this.removeIndexEntries(tableName, row, pk);
|
||||||
const updated = { ...row, ...updates };
|
const updated = { ...row, ...updates };
|
||||||
this.validateRow(schema, updated);
|
this.validateRow(schema, updated);
|
||||||
this.checkUniqueness(schema, updated);
|
this.checkUniqueness(schema, updated);
|
||||||
table.set(pk, updated);
|
const newPk = String(updated[pkCol]);
|
||||||
this.updateIndexes(tableName, updated, pk);
|
// v0.4.2-fix: 主键变更 — 删除旧键 + 级联更新引用表 + 新键落表
|
||||||
|
if (newPk !== pk) {
|
||||||
|
await this.applyUpdateCascade(tableName, pk, newPk);
|
||||||
|
}
|
||||||
|
table.delete(pk);
|
||||||
|
table.set(newPk, updated);
|
||||||
|
this.updateIndexes(tableName, updated, newPk);
|
||||||
count++;
|
count++;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return count;
|
return count;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.4.2-fix: ON UPDATE 外键级联 — 被引用表主键变更时处理引用表:
|
||||||
|
* RESTRICT 抛错 / CASCADE 更新 FK 值 / SET NULL 置空。
|
||||||
|
* 分两阶段:先全量 RESTRICT 检查(任何修改前),再执行级联(防部分修改)。
|
||||||
|
*/
|
||||||
|
private async applyUpdateCascade(tableName: string, oldPk: string, newPk: string): Promise<void> {
|
||||||
|
// 阶段 1: RESTRICT 检查(引用旧主键的行存在即拒绝)
|
||||||
|
for (const [refTableName, refSchema] of this.schemas) {
|
||||||
|
if (refTableName === tableName) continue;
|
||||||
|
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
|
||||||
|
if (!colDef.references || !colDef.onUpdate) continue;
|
||||||
|
const [refTable] = colDef.references.split('.');
|
||||||
|
if (refTable !== tableName) continue;
|
||||||
|
const refTableData = this.tables.get(refTableName);
|
||||||
|
if (!refTableData) continue;
|
||||||
|
for (const [, refRow] of refTableData) {
|
||||||
|
if (String(refRow[colName]) === oldPk && colDef.onUpdate === 'RESTRICT') {
|
||||||
|
throw new DatabaseError(
|
||||||
|
`Cannot update "${tableName}" key "${oldPk}": foreign key "${colName}" in "${refTableName}" has dependent rows`,
|
||||||
|
'FOREIGN_KEY_VIOLATION',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 阶段 2: CASCADE / SET NULL
|
||||||
|
for (const [refTableName, refSchema] of this.schemas) {
|
||||||
|
if (refTableName === tableName) continue;
|
||||||
|
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
|
||||||
|
if (!colDef.references || !colDef.onUpdate) continue;
|
||||||
|
const [refTable] = colDef.references.split('.');
|
||||||
|
if (refTable !== tableName) continue;
|
||||||
|
const refTableData = this.tables.get(refTableName);
|
||||||
|
if (!refTableData) continue;
|
||||||
|
if (colDef.onUpdate !== 'CASCADE' && colDef.onUpdate !== 'SET NULL') continue;
|
||||||
|
for (const [refPk, refRow] of refTableData) {
|
||||||
|
if (String(refRow[colName]) !== oldPk) continue;
|
||||||
|
this.removeIndexEntries(refTableName, refRow, refPk);
|
||||||
|
refRow[colName] = colDef.onUpdate === 'CASCADE' ? newPk : null;
|
||||||
|
this.updateIndexes(refTableName, refRow, refPk);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
||||||
this.ensureTable(tableName);
|
this.ensureTable(tableName);
|
||||||
const table = this.tables.get(tableName)!;
|
const table = this.tables.get(tableName)!;
|
||||||
|
|||||||
+95
-6
@@ -51,17 +51,65 @@ export class OPFSEngine implements IStorageEngine {
|
|||||||
return this.tablesDir !== null;
|
return this.tablesDir !== null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- v0.4.2-fix: 自愈 / 重置 / 元数据 ----
|
||||||
|
|
||||||
|
/** 自愈:重置内存缓存后从 OPFS 重新加载(单文件损坏不影响其他表) */
|
||||||
|
async repair(): Promise<void> {
|
||||||
|
await this.memoryCache.close();
|
||||||
|
await this.memoryCache.open(this.dbName, 1);
|
||||||
|
await this.loadExistingTables();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 清空全部数据与表结构(删除目录内全部文件) */
|
||||||
|
async clearAll(): Promise<void> {
|
||||||
|
await this.memoryCache.close();
|
||||||
|
await this.memoryCache.open(this.dbName, 1);
|
||||||
|
if (this.tablesDir) {
|
||||||
|
const dir = this.tablesDir as any;
|
||||||
|
for await (const [name] of dir.entries()) {
|
||||||
|
try { await this.tablesDir!.removeEntry(name); } catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async getMeta(key: string): Promise<string | null> {
|
||||||
|
if (!this.tablesDir) return null;
|
||||||
|
try {
|
||||||
|
const fh = await this.tablesDir.getFileHandle(`__metona_${key}.meta`);
|
||||||
|
const file = await fh.getFile();
|
||||||
|
return await file.text();
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async setMeta(key: string, value: string): Promise<void> {
|
||||||
|
if (!this.tablesDir) return;
|
||||||
|
const fh = await this.tablesDir.getFileHandle(`__metona_${key}.meta`, { create: true });
|
||||||
|
const writable = await fh.createWritable();
|
||||||
|
await writable.write(value);
|
||||||
|
await writable.close();
|
||||||
|
}
|
||||||
|
|
||||||
// ---- 表管理 ----
|
// ---- 表管理 ----
|
||||||
|
|
||||||
async createTable(schema: TableSchema): Promise<void> {
|
async createTable(schema: TableSchema): Promise<void> {
|
||||||
await this.memoryCache.createTable(schema);
|
await this.memoryCache.createTable(schema);
|
||||||
|
// v0.4.2-fix: schema 持久化(此前仅写空数据文件 → 空表重启后消失、索引标记丢失)
|
||||||
|
await this.setMeta(`schema_${schema.name}`, JSON.stringify(schema));
|
||||||
// OPFS 中表以空 JSON 数组文件形式存在
|
// OPFS 中表以空 JSON 数组文件形式存在
|
||||||
await this.writeTableData(schema.name, []);
|
await this.writeTableData(schema.name, []);
|
||||||
}
|
}
|
||||||
|
|
||||||
async dropTable(tableName: string): Promise<void> {
|
async dropTable(tableName: string): Promise<void> {
|
||||||
await this.memoryCache.dropTable(tableName);
|
await this.memoryCache.dropTable(tableName);
|
||||||
|
// v0.4.2-fix: 清理 schema meta(否则重启恢复幽灵表)
|
||||||
if (this.tablesDir) {
|
if (this.tablesDir) {
|
||||||
|
try {
|
||||||
|
await this.tablesDir.removeEntry(`__metona_schema_${tableName}.meta`);
|
||||||
|
} catch {
|
||||||
|
// 文件不存在则忽略
|
||||||
|
}
|
||||||
try {
|
try {
|
||||||
await this.tablesDir.removeEntry(`${tableName}.json`);
|
await this.tablesDir.removeEntry(`${tableName}.json`);
|
||||||
} catch {
|
} catch {
|
||||||
@@ -95,6 +143,19 @@ export class OPFSEngine implements IStorageEngine {
|
|||||||
return this.memoryCache.getTableSchema(tableName);
|
return this.memoryCache.getTableSchema(tableName);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 引擎级 ALTER TABLE — 内存 + schema 持久化 + 整表文件重写 */
|
||||||
|
async alterTable(
|
||||||
|
tableName: string,
|
||||||
|
action: 'ADD' | 'DROP',
|
||||||
|
column: import('../constants').ColumnDef & { name: string },
|
||||||
|
): Promise<void> {
|
||||||
|
await this.memoryCache.alterTable(tableName, action, column);
|
||||||
|
const schema = await this.memoryCache.getTableSchema(tableName);
|
||||||
|
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
|
||||||
|
const rows = await this.memoryCache.find(tableName, { table: tableName });
|
||||||
|
await this.writeTableData(tableName, rows);
|
||||||
|
}
|
||||||
|
|
||||||
// ---- CRUD ----
|
// ---- CRUD ----
|
||||||
|
|
||||||
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
||||||
@@ -140,11 +201,16 @@ export class OPFSEngine implements IStorageEngine {
|
|||||||
// ---- 动态索引(v0.3.0) ----
|
// ---- 动态索引(v0.3.0) ----
|
||||||
|
|
||||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||||
return this.memoryCache.createIndex(tableName, column, unique);
|
await this.memoryCache.createIndex(tableName, column, unique);
|
||||||
|
// v0.4.2-fix: 索引标记持久化(重启后索引结构恢复)
|
||||||
|
const schema = await this.memoryCache.getTableSchema(tableName);
|
||||||
|
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
|
||||||
}
|
}
|
||||||
|
|
||||||
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
|
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
|
||||||
return this.memoryCache.dropIndex(tableName, column, indexName);
|
await this.memoryCache.dropIndex(tableName, column, indexName);
|
||||||
|
const schema = await this.memoryCache.getTableSchema(tableName);
|
||||||
|
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---- 事务 ----
|
// ---- 事务 ----
|
||||||
@@ -198,16 +264,39 @@ export class OPFSEngine implements IStorageEngine {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 从 OPFS 加载已有表数据到内存缓存 */
|
/**
|
||||||
|
* 从 OPFS 加载已有表到内存缓存。
|
||||||
|
* v0.4.2-fix: 优先从持久化 schema(__metona_schema_*.meta)恢复 —
|
||||||
|
* 空表不再消失、索引标记/主键/约束完整;无 schema 记录的旧库从数据推断(兼容)。
|
||||||
|
*/
|
||||||
private async loadExistingTables(): Promise<void> {
|
private async loadExistingTables(): Promise<void> {
|
||||||
if (!this.tablesDir) return;
|
if (!this.tablesDir) return;
|
||||||
const dir = this.tablesDir as any;
|
const dir = this.tablesDir as any;
|
||||||
|
const fileNames: string[] = [];
|
||||||
for await (const [name] of dir.entries()) {
|
for await (const [name] of dir.entries()) {
|
||||||
if (!name.endsWith('.json')) continue;
|
if (name.endsWith('.json')) fileNames.push(name);
|
||||||
const tableName = name.replace('.json', '');
|
}
|
||||||
|
|
||||||
|
for (const fileName of fileNames) {
|
||||||
|
const tableName = fileName.replace('.json', '');
|
||||||
try {
|
try {
|
||||||
|
// 1. 优先:持久化 schema
|
||||||
|
const schemaRaw = await this.getMeta(`schema_${tableName}`);
|
||||||
|
if (schemaRaw) {
|
||||||
|
const schema = JSON.parse(schemaRaw) as TableSchema;
|
||||||
|
await this.memoryCache.createTable(schema);
|
||||||
|
const rows = await this.readTableData(tableName);
|
||||||
|
for (const row of rows) {
|
||||||
|
try {
|
||||||
|
await this.memoryCache.insert(tableName, [row]);
|
||||||
|
} catch {
|
||||||
|
// 单行损坏不影响整表恢复
|
||||||
|
}
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// 2. 兼容旧库:从数据推断 schema(空表且无 schema 记录 → 跳过)
|
||||||
const data = await this.readTableData(tableName);
|
const data = await this.readTableData(tableName);
|
||||||
// 从数据中推断 schema(简化:从第一行提取列信息)
|
|
||||||
if (data.length > 0) {
|
if (data.length > 0) {
|
||||||
const firstRow = data[0];
|
const firstRow = data[0];
|
||||||
const columns: Record<string, any> = {};
|
const columns: Record<string, any> = {};
|
||||||
|
|||||||
+67
-4
@@ -87,6 +87,49 @@ export class HybridEngine implements IStorageEngine {
|
|||||||
return this.memoryEngine.isOpen() && this.diskEngine.isOpen();
|
return this.memoryEngine.isOpen() && this.diskEngine.isOpen();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- v0.4.2-fix: 自愈 / 重置 / 元数据(委托双引擎) ----
|
||||||
|
|
||||||
|
/** 自愈:修复磁盘引擎后重载内存缓存 */
|
||||||
|
async repair(): Promise<void> {
|
||||||
|
if (typeof this.diskEngine.repair === 'function') {
|
||||||
|
await this.diskEngine.repair();
|
||||||
|
}
|
||||||
|
await this.reloadMemoryFromDisk();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 清空全部数据与表结构 */
|
||||||
|
async clearAll(): Promise<void> {
|
||||||
|
if (typeof this.diskEngine.clearAll === 'function') {
|
||||||
|
await this.diskEngine.clearAll();
|
||||||
|
} else {
|
||||||
|
const names = await this.diskEngine.getTableNames();
|
||||||
|
for (const name of names) {
|
||||||
|
await this.diskEngine.dropTable(name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (typeof this.memoryEngine.clearAll === 'function') {
|
||||||
|
await this.memoryEngine.clearAll();
|
||||||
|
} else {
|
||||||
|
const names = await this.memoryEngine.getTableNames();
|
||||||
|
for (const name of names) {
|
||||||
|
await this.memoryEngine.dropTable(name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async getMeta(key: string): Promise<string | null> {
|
||||||
|
if (typeof this.diskEngine.getMeta === 'function') {
|
||||||
|
return this.diskEngine.getMeta(key);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async setMeta(key: string, value: string): Promise<void> {
|
||||||
|
if (typeof this.diskEngine.setMeta === 'function') {
|
||||||
|
await this.diskEngine.setMeta(key, value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---- 表管理 ----
|
// ---- 表管理 ----
|
||||||
|
|
||||||
async createTable(schema: TableSchema): Promise<void> {
|
async createTable(schema: TableSchema): Promise<void> {
|
||||||
@@ -111,6 +154,22 @@ export class HybridEngine implements IStorageEngine {
|
|||||||
return this.memoryEngine.getTableSchema(tableName);
|
return this.memoryEngine.getTableSchema(tableName);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 引擎级 ALTER TABLE — 双引擎同步(磁盘持久化 + 内存引用) */
|
||||||
|
async alterTable(
|
||||||
|
tableName: string,
|
||||||
|
action: 'ADD' | 'DROP',
|
||||||
|
column: import('../constants').ColumnDef & { name: string },
|
||||||
|
): Promise<void> {
|
||||||
|
await this.memoryEngine.alterTable(tableName, action, column);
|
||||||
|
if (typeof this.diskEngine.alterTable === 'function') {
|
||||||
|
await this.diskEngine.alterTable(tableName, action, column);
|
||||||
|
} else {
|
||||||
|
// 磁盘引擎无引擎级实现 → 从磁盘重建内存 schema(disk 引擎 schema 以自身为准)
|
||||||
|
const schema = await this.diskEngine.getTableSchema(tableName);
|
||||||
|
if (schema && action === 'DROP') delete schema.columns[column.name];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---- CRUD(write-through 策略) ----
|
// ---- CRUD(write-through 策略) ----
|
||||||
|
|
||||||
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
||||||
@@ -181,10 +240,14 @@ export class HybridEngine implements IStorageEngine {
|
|||||||
await this.diskEngine.commitTransaction();
|
await this.diskEngine.commitTransaction();
|
||||||
try {
|
try {
|
||||||
await this.memoryEngine.commitTransaction();
|
await this.memoryEngine.commitTransaction();
|
||||||
} catch {
|
} catch (error) {
|
||||||
// 内存提交失败时回滚磁盘
|
// v0.4.2-fix: 磁盘已提交无法回滚(此前调 diskEngine.rollbackTransaction()
|
||||||
await this.diskEngine.rollbackTransaction();
|
// 会抛 TX_NONE 掩盖原错误)。如实上报内存提交失败,磁盘数据保持已提交状态。
|
||||||
throw new DatabaseError('Hybrid commit failed: memory engine error after disk commit', 'TX_COMMIT_ERROR');
|
throw new DatabaseError(
|
||||||
|
'Hybrid commit failed: memory engine error after disk commit (disk data is committed)',
|
||||||
|
'TX_COMMIT_ERROR',
|
||||||
|
error,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -287,16 +287,15 @@ describe('[v0.3.1] WAL 批量组提交', () => {
|
|||||||
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
||||||
|
|
||||||
const engine = db.getEngine() as any;
|
const engine = db.getEngine() as any;
|
||||||
let before = 0;
|
// v0.4.2: WAL 记录与 count 改用 writeMany 单事务原子写入(记录仍合并为 1 次落盘)
|
||||||
const origRead = engine.backend.read.bind(engine.backend);
|
const origWriteMany = engine.backend.writeMany.bind(engine.backend);
|
||||||
// 统计 __wal_ 写入次数
|
|
||||||
const origWrite = engine.backend.write.bind(engine.backend);
|
const origWrite = engine.backend.write.bind(engine.backend);
|
||||||
let walWrites = 0;
|
let walWrites = 0;
|
||||||
engine.backend.write = async (key: string, data: ArrayBuffer) => {
|
engine.backend.writeMany = async (entries: Record<string, ArrayBuffer>) => {
|
||||||
if (key.startsWith('__wal_') && !key.startsWith('__wal_count')) walWrites++;
|
const keys = Object.keys(entries);
|
||||||
return origWrite(key, data);
|
if (keys.some((k) => k.startsWith('__wal_') && !k.startsWith('__wal_count'))) walWrites++;
|
||||||
|
return origWriteMany(entries);
|
||||||
};
|
};
|
||||||
void before; void origRead;
|
|
||||||
|
|
||||||
await db.table('users').insertMany([
|
await db.table('users').insertMany([
|
||||||
{ id: '1', name: 'A' },
|
{ id: '1', name: 'A' },
|
||||||
@@ -305,7 +304,8 @@ describe('[v0.3.1] WAL 批量组提交', () => {
|
|||||||
{ id: '4', name: 'D' },
|
{ id: '4', name: 'D' },
|
||||||
]);
|
]);
|
||||||
|
|
||||||
expect(walWrites).toBe(1); // 4 行 1 次 WAL 写入
|
expect(walWrites).toBe(1); // 4 行合并为 1 次 WAL 落盘
|
||||||
|
void origWrite;
|
||||||
const rows = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
const rows = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||||
expect(rows).toHaveLength(4);
|
expect(rows).toHaveLength(4);
|
||||||
await db.close();
|
await db.close();
|
||||||
|
|||||||
@@ -23,8 +23,8 @@ import { createSchema } from '../src/table/schema';
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
||||||
test('VERSION 常量为当前版本(0.4.1)', () => {
|
test('VERSION 常量为当前版本(0.4.2)', () => {
|
||||||
expect(VERSION).toBe('0.4.1');
|
expect(VERSION).toBe('0.4.2');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -400,7 +400,7 @@ describe('[v0.3.3] P1-9: Savepoint + MVCC 一致性', () => {
|
|||||||
|
|
||||||
describe('[v0.3.3] 端到端', () => {
|
describe('[v0.3.3] 端到端', () => {
|
||||||
test('全部修复点可共存于 MetonaSqlark API', async () => {
|
test('全部修复点可共存于 MetonaSqlark API', async () => {
|
||||||
expect(VERSION).toBe('0.4.1');
|
expect(VERSION).toBe('0.4.2');
|
||||||
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
|
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
|
|||||||
@@ -0,0 +1,501 @@
|
|||||||
|
/**
|
||||||
|
* v0.4.2 问题清单回归测试
|
||||||
|
* 覆盖:
|
||||||
|
* - P0-1: SSTable 残缺数据防御(读路径越界跳过 / 打开时完整性校验 / WAL 原子写入)
|
||||||
|
* - P0-2: IndexedDB 版本管理(VersionError 自适应重开)
|
||||||
|
* - P0-3: version 0 归一化
|
||||||
|
* - P1-4: WAL 写丢失(原子 append + 按 key 扫描恢复)
|
||||||
|
* - P1-5: close 截断 WAL(不无限重放)
|
||||||
|
* - P1-6: 引擎内部错误包装 DatabaseError
|
||||||
|
* - P2-7: 迁移版本持久化
|
||||||
|
* - P2-9: repair() / clearAll() 统一自愈接口
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
|
import { createSchema } from '../src/table/schema';
|
||||||
|
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
|
||||||
|
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
||||||
|
import { IndexedDBEngine } from '../src/engine/indexeddb';
|
||||||
|
import { MetonaSqlark } from '../src/core';
|
||||||
|
import { IndexedDBBackend } from '../src/engine/aria/store/backend';
|
||||||
|
import type { SSTableMeta } from '../src/engine/aria/types';
|
||||||
|
import 'fake-indexeddb/auto';
|
||||||
|
|
||||||
|
let idbCounter = 0;
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `fix-${Date.now()}-${++idbCounter}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const makeMeta = (data: Uint8Array): SSTableMeta => ({
|
||||||
|
id: 1, level: 0, minKey: '', maxKey: '\uffff',
|
||||||
|
blockCount: 1, totalSize: data.byteLength, bloomData: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P0-1a: SSTableReader 残缺数据防御
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P0-1a — SSTableReader 残缺数据防御', () => {
|
||||||
|
it('索引块越界(文件被截断)→ 构造不抛异常,get/rangeScan/scanAll 返回空', () => {
|
||||||
|
const builder = new SSTableBuilder(64);
|
||||||
|
for (let i = 0; i < 100; i++) {
|
||||||
|
builder.add(`k-${String(i).padStart(3, '0')}`, { v: i, data: 'x'.repeat(30) });
|
||||||
|
}
|
||||||
|
const { sstableData } = builder.build();
|
||||||
|
// 完整文件可用(对照)
|
||||||
|
const full = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||||
|
expect(full.get('k-001')).not.toBeNull();
|
||||||
|
|
||||||
|
// 从索引块中部截断(手动拼接保留末尾 32 字节 footer,使索引块越界但 footer 完整)
|
||||||
|
const footerOffset = sstableData.byteLength - 32;
|
||||||
|
const indexOffset = new DataView(sstableData.buffer, sstableData.byteOffset, sstableData.byteLength)
|
||||||
|
.getUint32(footerOffset, false);
|
||||||
|
const truncated = new Uint8Array((indexOffset + 4) + 32);
|
||||||
|
truncated.set(sstableData.slice(0, indexOffset + 4), 0);
|
||||||
|
truncated.set(sstableData.slice(sstableData.byteLength - 32), indexOffset + 4);
|
||||||
|
// 构造必须不抛 RangeError
|
||||||
|
let reader: SSTableReader;
|
||||||
|
expect(() => { reader = new SSTableReader(truncated, makeMeta(truncated)); }).not.toThrow();
|
||||||
|
expect(() => (reader as any).get('k-001')).not.toThrow();
|
||||||
|
expect(() => (reader as any).rangeScan('a', 'z', () => {})).not.toThrow();
|
||||||
|
expect(() => (reader as any).scanAll(() => {})).not.toThrow();
|
||||||
|
expect((reader as any).get('k-001')).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('索引条目 blockSize 越界 → 该块跳过,其余块仍可读', () => {
|
||||||
|
const builder = new SSTableBuilder(64);
|
||||||
|
for (let i = 0; i < 100; i++) {
|
||||||
|
builder.add(`k-${String(i).padStart(3, '0')}`, { v: i, data: 'x'.repeat(30) });
|
||||||
|
}
|
||||||
|
const { sstableData } = builder.build();
|
||||||
|
const corrupted = sstableData.slice();
|
||||||
|
const view = new DataView(corrupted.buffer, corrupted.byteOffset, corrupted.byteLength);
|
||||||
|
const footerOffset = corrupted.byteLength - 32;
|
||||||
|
const indexOffset = view.getUint32(footerOffset, false);
|
||||||
|
expect(view.getUint32(indexOffset, false)).toBeGreaterThan(1);
|
||||||
|
// 把第二个索引条目的 blockSize 改为超大(指向文件外):
|
||||||
|
// 跳过 entry0(keyLen+key+blockOffset+blockSize)与 entry1 的 keyLen+key+blockOffset
|
||||||
|
let off = indexOffset + 4;
|
||||||
|
const keyLen0 = view.getUint16(off, false);
|
||||||
|
off += 2 + keyLen0 + 8;
|
||||||
|
const keyLen1 = view.getUint16(off, false);
|
||||||
|
off += 2 + keyLen1 + 4;
|
||||||
|
view.setUint32(off, 0x7FFFFFF0, false);
|
||||||
|
|
||||||
|
const reader = new SSTableReader(corrupted, makeMeta(corrupted));
|
||||||
|
// 不抛 RangeError
|
||||||
|
expect(() => reader.get('k-001')).not.toThrow();
|
||||||
|
expect(() => reader.rangeScan('a', 'z', () => {})).not.toThrow();
|
||||||
|
expect(() => reader.scanAll(() => {})).not.toThrow();
|
||||||
|
// 第一个索引条目(未破坏)指向的块仍可读
|
||||||
|
const first = reader.get('k-000');
|
||||||
|
expect(first).not.toBeNull();
|
||||||
|
expect((first as any).v).toBe(0);
|
||||||
|
// 被破坏的块被跳过(返回 null 而非崩溃)
|
||||||
|
expect(reader.get('k-001')).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('魔数错误 → 构造抛错(由 LSM loadSSTableReader 捕获跳过)', () => {
|
||||||
|
const builder = new SSTableBuilder(4096);
|
||||||
|
builder.add('a', { v: 1 });
|
||||||
|
const { sstableData } = builder.build();
|
||||||
|
const corrupted = sstableData.slice();
|
||||||
|
// 破坏 footer 中 magic(位于文件末尾 32 字节内的 +24 偏移)
|
||||||
|
new DataView(corrupted.buffer, corrupted.byteOffset, corrupted.byteLength)
|
||||||
|
.setUint32(corrupted.byteLength - 8, 0xDEADBEEF, false);
|
||||||
|
expect(() => new SSTableReader(corrupted, makeMeta(corrupted))).toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('块内条目计数虚高(内容截断)→ 提前中止,不抛 RangeError', () => {
|
||||||
|
const builder = new SSTableBuilder(64);
|
||||||
|
for (let i = 0; i < 100; i++) {
|
||||||
|
builder.add(`k-${String(i).padStart(3, '0')}`, { v: i, data: 'x'.repeat(30) });
|
||||||
|
}
|
||||||
|
const { sstableData } = builder.build();
|
||||||
|
const corrupted = sstableData.slice();
|
||||||
|
// 把第一个数据块的 entryCount 改为超大(模拟块内条目被截断)
|
||||||
|
new DataView(corrupted.buffer, corrupted.byteOffset, corrupted.byteLength)
|
||||||
|
.setUint32(0, 0x7FFFFFF0, false);
|
||||||
|
expect(() => new SSTableReader(corrupted, makeMeta(corrupted))).not.toThrow();
|
||||||
|
const reader = new SSTableReader(corrupted, makeMeta(corrupted));
|
||||||
|
expect(() => reader.get('k-000')).not.toThrow();
|
||||||
|
expect(() => reader.get('nonexistent')).not.toThrow();
|
||||||
|
expect(() => reader.scanAll(() => {})).not.toThrow();
|
||||||
|
// 截断点之前的条目仍可读
|
||||||
|
expect((reader.get('k-000') as any)?.v).toBe(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P0-1b: LSM 打开时完整性校验
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
||||||
|
it('meta 引用残缺文件 → 打开跳过损坏 SSTable 不崩溃,库可用', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 50; i++) {
|
||||||
|
await engine.insert('users', [{ id: `u${i}`, name: `User${i}` }]);
|
||||||
|
}
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 篡改存储:把第一个 sst_ 文件写成残缺内容(meta 仍引用它)
|
||||||
|
const backend = new IndexedDBBackend();
|
||||||
|
await backend.open(dbName);
|
||||||
|
const sstKeys = (await backend.listKeys())
|
||||||
|
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
||||||
|
expect(sstKeys.length).toBeGreaterThan(0);
|
||||||
|
await backend.write(sstKeys[0], new TextEncoder().encode('truncated-garbage').buffer);
|
||||||
|
await backend.close();
|
||||||
|
|
||||||
|
// 重开:不得崩溃(此前抛 RangeError 打不开库)
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
||||||
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
|
expect(engine2.isOpen()).toBe(true);
|
||||||
|
// schema 完整,损坏的 SSTable 已被清理
|
||||||
|
expect(await engine2.getTableNames()).toEqual(['users']);
|
||||||
|
const rows = await engine2.find('users', { table: 'users' });
|
||||||
|
expect(Array.isArray(rows)).toBe(true);
|
||||||
|
await engine2.close();
|
||||||
|
|
||||||
|
// 清理 meta 已验证:损坏文件被删除
|
||||||
|
const backend2 = new IndexedDBBackend();
|
||||||
|
await backend2.open(dbName);
|
||||||
|
const remaining = (await backend2.listKeys())
|
||||||
|
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
||||||
|
const metaRaw = await backend2.read('__aria_lsm_meta');
|
||||||
|
const metaList = JSON.parse(new TextDecoder().decode(metaRaw ?? new Uint8Array())) as { id: number }[];
|
||||||
|
expect(remaining.length).toBe(metaList.length); // 无孤儿文件 / 无悬空 meta
|
||||||
|
await backend2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('meta 引用缺失文件 → 打开清理 meta 不崩溃', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 30; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 删除全部 sst_ 文件(保留 meta → 悬空引用)
|
||||||
|
const backend = new IndexedDBBackend();
|
||||||
|
await backend.open(dbName);
|
||||||
|
const sstKeys = (await backend.listKeys())
|
||||||
|
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
||||||
|
await backend.deleteMany(sstKeys);
|
||||||
|
await backend.close();
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
||||||
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
|
expect(engine2.isOpen()).toBe(true);
|
||||||
|
expect(await engine2.getTableNames()).toEqual(['t']);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P0-1c / P1-4: WAL 原子写入 + 按 key 扫描恢复
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
||||||
|
it('IndexedDBBackend.writeMany/deleteMany 单事务原子批量操作', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const be = new IndexedDBBackend();
|
||||||
|
await be.open(dbName);
|
||||||
|
await be.writeMany({
|
||||||
|
a: new TextEncoder().encode('1').buffer,
|
||||||
|
b: new TextEncoder().encode('2').buffer,
|
||||||
|
});
|
||||||
|
expect(new TextDecoder().decode((await be.read('a'))!)).toBe('1');
|
||||||
|
expect(new TextDecoder().decode((await be.read('b'))!)).toBe('2');
|
||||||
|
await be.deleteMany(['a', 'b']);
|
||||||
|
expect(await be.read('a')).toBeNull();
|
||||||
|
expect(await be.read('b')).toBeNull();
|
||||||
|
await be.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('P1-4: 连续快速写入 200 条 → close+重开 数据完整(实测曾丢 4 条)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 200; i++) {
|
||||||
|
await engine.insert('t', [{ id: `${i}`, v: i }]);
|
||||||
|
}
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('t')).toBe(200);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('P1-4: count 键丢失(模拟崩溃竞态)→ 按 key 扫描恢复不丢记录', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 20; i++) {
|
||||||
|
await engine.insert('t', [{ id: `${i}`, v: i }]);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 模拟异常退出(不 close):直接删掉 count 键,制造 count 与记录不一致
|
||||||
|
const backend = new IndexedDBBackend();
|
||||||
|
await backend.open(dbName);
|
||||||
|
await backend.delete('__wal_count');
|
||||||
|
await backend.close();
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('t')).toBe(20);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1-5: close 截断 WAL
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1-5 — AriaEngine.close 截断 WAL', () => {
|
||||||
|
it('close 后 WAL 记录键全部清空(不无限重放)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
await engine.insert('t', [{ id: '1', v: 1 }, { id: '2', v: 2 }]);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const backend = new IndexedDBBackend();
|
||||||
|
await backend.open(dbName);
|
||||||
|
// __wal_count 计数键合法保留(作为 append 序号分配器),WAL 记录键必须清空
|
||||||
|
const walKeys = (await backend.listKeys())
|
||||||
|
.filter((k) => k.startsWith('__wal_') && k !== '__wal_count');
|
||||||
|
expect(walKeys).toHaveLength(0);
|
||||||
|
await backend.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P0-2 / P0-3 / P2-8: IndexedDB 版本管理与参数校验
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P0-2 / P0-3 — IndexedDB 版本管理', () => {
|
||||||
|
it('P0-2: 建表提升版本后以旧 version 重开成功(VersionError 自适应)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new IndexedDBEngine();
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable({ name: 'a', columns: { id: { type: 'string', primaryKey: true } } });
|
||||||
|
await engine.createTable({ name: 'b', columns: { id: { type: 'string', primaryKey: true } } });
|
||||||
|
await engine.createTable({ name: 'c', columns: { id: { type: 'string', primaryKey: true } } });
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 用旧版本 1 重开:此前必报 IDB_OPEN_ERROR(VersionError)
|
||||||
|
const engine2 = new IndexedDBEngine();
|
||||||
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
|
const names = await engine2.getTableNames();
|
||||||
|
expect(names).toContain('a');
|
||||||
|
expect(names).toContain('b');
|
||||||
|
expect(names).toContain('c');
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('P0-2: MetonaSqlark hybrid 二次启动不报 VersionError(MarkLite 场景)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 1 });
|
||||||
|
await db1.init();
|
||||||
|
await db1.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
||||||
|
await db1.table('users').insert({ id: '1', name: 'Alice' });
|
||||||
|
await db1.close();
|
||||||
|
|
||||||
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 1 });
|
||||||
|
await expect(db2.init()).resolves.toBeUndefined();
|
||||||
|
expect(await db2.table('users').count()).toBe(1);
|
||||||
|
await db2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('P0-3: IndexedDBEngine open(version=0) 归一化为 1,不抛 TypeError', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new IndexedDBEngine();
|
||||||
|
await expect(engine.open(dbName, 0)).resolves.toBeUndefined();
|
||||||
|
expect(engine.isOpen()).toBe(true);
|
||||||
|
await engine.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
|
||||||
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
|
expect(await engine.find('t', { table: 't' })).toHaveLength(1);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('P0-3: MetonaSqlark version=0 初始化正常', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
|
||||||
|
await expect(db.init()).resolves.toBeUndefined();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.table('t').insert({ id: '1' });
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1-6: 引擎错误包装 DatabaseError
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1-6 — 引擎内部错误统一包装 DatabaseError', () => {
|
||||||
|
it('AriaEngine open 异常路径抛 DatabaseError(ARIA_OPEN_ERROR)', async () => {
|
||||||
|
// 构造一个无法打开的 backend 场景:直接调用 openInternal 模拟底层异常不可行,
|
||||||
|
// 这里验证损坏库重开时抛的是 DatabaseError 而非原生错误(不崩溃路径已由 P0-1b 覆盖)
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb' });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' });
|
||||||
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
|
expect(engine2.isOpen()).toBe(true);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('open 未初始化 IndexedDB 环境时抛 DatabaseError 而非原生错误', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new IndexedDBEngine();
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
expect(engine.isOpen()).toBe(true);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P2-7: 迁移版本持久化
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P2-7 — 迁移版本持久化', () => {
|
||||||
|
it('重启后已执行迁移不重跑,只执行新迁移', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const runs: number[] = [];
|
||||||
|
|
||||||
|
// version: 0 表示"无 schema 起点",migration 1 可执行
|
||||||
|
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
|
||||||
|
await db1.init();
|
||||||
|
db1.addMigration(1, async () => { runs.push(1); });
|
||||||
|
db1.addMigration(2, async () => { runs.push(2); });
|
||||||
|
await db1.migrateTo(2);
|
||||||
|
expect(runs).toEqual([1, 2]);
|
||||||
|
await db1.close();
|
||||||
|
|
||||||
|
// 重启:version 又重置为 config.version=0(此前会重跑 migration 1/2)
|
||||||
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
|
||||||
|
await db2.init();
|
||||||
|
db2.addMigration(1, async () => { runs.push(1); });
|
||||||
|
db2.addMigration(2, async () => { runs.push(2); });
|
||||||
|
db2.addMigration(3, async () => { runs.push(3); });
|
||||||
|
await db2.migrateTo(3);
|
||||||
|
// 只有 migration 3 是新迁移(1/2 已持久化不重跑)
|
||||||
|
expect(runs).toEqual([1, 2, 3]);
|
||||||
|
await db2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('AriaEngine getMeta/setMeta 往返', async () => {
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'memory' });
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
expect(await engine.getMeta('__metona_version')).toBeNull();
|
||||||
|
await engine.setMeta('__metona_version', '3');
|
||||||
|
expect(await engine.getMeta('__metona_version')).toBe('3');
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P2-9: repair() / clearAll() 统一自愈接口
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
|
||||||
|
it('AriaEngine.clearAll 清空全部表与数据', async () => {
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'memory' });
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
|
await engine.clearAll();
|
||||||
|
expect(await engine.getTableNames()).toEqual([]);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('AriaEngine.repair 清理损坏 SSTable 后引擎可用', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 4096 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 100; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
|
||||||
|
// 篡改一个 sst 文件
|
||||||
|
const backend = new IndexedDBBackend();
|
||||||
|
await backend.open(dbName);
|
||||||
|
const sstKeys = (await backend.listKeys())
|
||||||
|
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
||||||
|
expect(sstKeys.length).toBeGreaterThan(0);
|
||||||
|
await backend.write(sstKeys[0], new TextEncoder().encode('garbage').buffer);
|
||||||
|
await backend.close();
|
||||||
|
|
||||||
|
await expect(engine.repair()).resolves.toBeUndefined();
|
||||||
|
const rows = await engine.find('t', { table: 't' });
|
||||||
|
expect(Array.isArray(rows)).toBe(true);
|
||||||
|
expect(rows.length).toBeLessThanOrEqual(100);
|
||||||
|
expect(engine.isOpen()).toBe(true);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('MetonaSqlark.clearAll 统一接口(hybrid)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.table('t').insert({ id: '1' });
|
||||||
|
await db.clearAll();
|
||||||
|
expect(await db.getTableNames()).toEqual([]);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('MetonaSqlark.repair 统一接口(aria)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'aria', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||||
|
await db.table('t').insert({ id: '1', v: 42 });
|
||||||
|
await db.repair();
|
||||||
|
expect(await db.table('t').count()).toBe(1);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('MetonaSqlark.repair 统一接口(hybrid)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.table('t').insert({ id: '1' });
|
||||||
|
await db.repair();
|
||||||
|
expect(await db.table('t').count()).toBe(1);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,312 @@
|
|||||||
|
/**
|
||||||
|
* v0.4.2 深度审计回归测试(第二轮)
|
||||||
|
* 覆盖:
|
||||||
|
* - P0-A: compaction 不依赖缓存(缓存未命中不得丢数据/数据不可见)
|
||||||
|
* - P0-B: flush/compaction 失败不得卡死 flushChain(写路径死锁)
|
||||||
|
* - P1-A: 重开后二级索引恢复 + createIndex 幂等重建 + WAL 恢复后索引一致
|
||||||
|
* - P1-B: ALTER TABLE 在 IndexedDB/Hybrid 引擎持久化
|
||||||
|
* - P1-C: IndexedDB 事务内 DDL(建表/删表)commit 后磁盘一致
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
|
import { createSchema } from '../src/table/schema';
|
||||||
|
import { MemoryEngine } from '../src/engine/memory';
|
||||||
|
import { HybridEngine } from '../src/hybrid/index';
|
||||||
|
import { MetonaSqlark } from '../src/core';
|
||||||
|
import 'fake-indexeddb/auto';
|
||||||
|
|
||||||
|
let idbCounter = 0;
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `audit-${Date.now()}-${++idbCounter}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P0-A: compaction 不得依赖 SSTable 缓存
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P0-A — Compaction 缓存独立性', () => {
|
||||||
|
it('缓存上限小于单个 SSTable 时 compaction 不丢数据(此前数据不可见)', async () => {
|
||||||
|
const engine = new AriaEngine({
|
||||||
|
storageBackend: 'memory',
|
||||||
|
// 缓存上限 4 页 = 16KB,memtable 32KB → 每次 flush 的文件都超出缓存上限被驱逐
|
||||||
|
bufferPoolPages: 4,
|
||||||
|
memtableSizeThreshold: 32 * 1024,
|
||||||
|
checkpointInterval: 100000,
|
||||||
|
});
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
data: { type: 'string' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 400; i++) {
|
||||||
|
await engine.insert('t', [{ id: `k${String(i).padStart(4, '0')}`, v: i, data: 'x'.repeat(200) }]);
|
||||||
|
}
|
||||||
|
// 等待 flush + compaction 链全部完成
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
await new Promise((r) => setTimeout(r, 100));
|
||||||
|
// 修复前:compaction 缓存未命中跳过全部文件并从 levels 移除 → 查询为空
|
||||||
|
expect(await engine.count('t')).toBe(400);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('多层级 compaction 后数据仍完整且可重开', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({
|
||||||
|
storageBackend: 'indexeddb',
|
||||||
|
bufferPoolPages: 8, // 32KB 缓存
|
||||||
|
memtableSizeThreshold: 16 * 1024,
|
||||||
|
checkpointInterval: 100000,
|
||||||
|
});
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
for (let i = 0; i < 300; i++) {
|
||||||
|
await engine.insert('t', [{ id: `k${String(i).padStart(4, '0')}`, v: i }]);
|
||||||
|
}
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
await new Promise((r) => setTimeout(r, 150));
|
||||||
|
const before = await engine.count('t');
|
||||||
|
expect(before).toBe(300);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 重开:数据完整(meta/文件未被 compaction 破坏)
|
||||||
|
const engine2 = new AriaEngine({
|
||||||
|
storageBackend: 'indexeddb',
|
||||||
|
bufferPoolPages: 8,
|
||||||
|
memtableSizeThreshold: 16 * 1024,
|
||||||
|
checkpointInterval: 100000,
|
||||||
|
});
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('t')).toBe(300);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1-A: 二级索引跨重启恢复
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1-A — 二级索引恢复', () => {
|
||||||
|
it('Aria 重开后二级索引可用(索引 LSM 持久化恢复)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', email: 'a@x.com' },
|
||||||
|
{ id: '2', email: 'b@x.com' },
|
||||||
|
{ id: '3', email: 'a@x.com' },
|
||||||
|
]);
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 重开:二级索引 LSM 应自动恢复(修复前为空 → 索引查询回退全表,createIndex 也静默跳过)
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
// 强断言:索引 LSM 真实存在且数据完整(防止"索引缺失静默回退全表扫描"的假通过)
|
||||||
|
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:email');
|
||||||
|
expect(idxLsm).toBeDefined();
|
||||||
|
expect(idxLsm.getStats().sstableCount).toBeGreaterThan(0);
|
||||||
|
await idxLsm.prefetchRange('', '\uffff');
|
||||||
|
expect(idxLsm.rangeScan('', '\uffff')).toHaveLength(3);
|
||||||
|
const byEmail = await engine2.find('users', { table: 'users', where: { email: 'a@x.com' } });
|
||||||
|
expect(byEmail).toHaveLength(2);
|
||||||
|
// createIndex 对已持久化的索引列应幂等可重建(不得静默跳过导致索引永久缺失)
|
||||||
|
await engine2.createIndex('users', 'email');
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('WAL 恢复后二级索引与主数据一致(崩溃前索引未更新)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
city: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await engine.insert('users', [{ id: '1', city: 'Beijing' }]);
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
// 写入 WAL 但强制不 flush(模拟崩溃:新行只在 WAL,索引 LSM 未更新)
|
||||||
|
await engine.insert('users', [{ id: '2', city: 'Shanghai' }]);
|
||||||
|
// 模拟异常退出(不 close)
|
||||||
|
await (engine as any).backend.close();
|
||||||
|
(engine as any).opened = false;
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
// 强断言:索引 LSM 已恢复且包含 WAL 回放的行(崩溃前索引未更新,恢复后必须重建)
|
||||||
|
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:city');
|
||||||
|
expect(idxLsm).toBeDefined();
|
||||||
|
await idxLsm.prefetchRange('', '\uffff');
|
||||||
|
expect(idxLsm.rangeScan('', '\uffff')).toHaveLength(2);
|
||||||
|
// 索引查询应看到 WAL 恢复的行(修复前索引与主数据不一致 → 丢行)
|
||||||
|
const byCity = await engine2.find('users', { table: 'users', where: { city: 'Shanghai' } });
|
||||||
|
expect(byCity).toHaveLength(1);
|
||||||
|
expect(byCity[0].id).toBe('2');
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('createIndex 重开后仍可新建(schema 标记恢复后不阻塞)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string' },
|
||||||
|
}));
|
||||||
|
await engine.insert('t', [{ id: '1', name: 'A' }]);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
await engine2.createIndex('t', 'name'); // 修复前 schema 无标记时正常;此处验证无标记场景
|
||||||
|
const byName = await engine2.find('t', { table: 't', where: { name: 'A' } });
|
||||||
|
expect(byName).toHaveLength(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1-B: ALTER TABLE 跨引擎持久化
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1-B — ALTER TABLE 持久化', () => {
|
||||||
|
it('IndexedDBEngine DROP COLUMN 后重启不复活', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
old_col: { type: 'string' },
|
||||||
|
keep: { type: 'string' },
|
||||||
|
});
|
||||||
|
await db.table('t').insert({ id: '1', old_col: 'x', keep: 'y' });
|
||||||
|
await db.query('ALTER TABLE t DROP COLUMN old_col');
|
||||||
|
// 行数据中该列已移除
|
||||||
|
const rows = await db.table('t').select().execute();
|
||||||
|
expect(rows[0].old_col).toBeUndefined();
|
||||||
|
await db.close();
|
||||||
|
|
||||||
|
// 重启:schema 不复活,列定义已持久化
|
||||||
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' });
|
||||||
|
await db2.init();
|
||||||
|
const schema2 = await db2.getEngine().getTableSchema('t');
|
||||||
|
expect(schema2!.columns.old_col).toBeUndefined();
|
||||||
|
expect(schema2!.columns.keep).toBeDefined();
|
||||||
|
await db2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('HybridEngine ADD COLUMN 后重启保留', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.query('ALTER TABLE t ADD COLUMN phone STRING');
|
||||||
|
await db.close();
|
||||||
|
|
||||||
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db2.init();
|
||||||
|
const schema2 = await db2.getEngine().getTableSchema('t');
|
||||||
|
expect(schema2!.columns.phone).toBeDefined();
|
||||||
|
await db2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1-C: IndexedDB 事务内 DDL
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1-C — IndexedDB 事务内 DDL', () => {
|
||||||
|
it('事务内建表 → commit 后磁盘一致(重启表存在且可查)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('base', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.table('base').insert({ id: '1' });
|
||||||
|
|
||||||
|
await db.transaction(async (trx) => {
|
||||||
|
await trx.table('base').insert({ id: '2' });
|
||||||
|
// 事务内建新表(此前 commit 时 IDB 无对应 store → 事务失败)
|
||||||
|
await db.defineTable('created_in_tx', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.table('created_in_tx').insert({ id: 'tx1' });
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(await db.table('created_in_tx').count()).toBe(1);
|
||||||
|
await db.close();
|
||||||
|
|
||||||
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db2.init();
|
||||||
|
expect(await db2.table('created_in_tx').count()).toBe(1);
|
||||||
|
expect(await db2.table('base').count()).toBe(2);
|
||||||
|
await db2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务内删表 → commit 后磁盘一致(重启无幽灵表)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('ghost', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.table('ghost').insert({ id: '1' });
|
||||||
|
|
||||||
|
await db.transaction(async () => {
|
||||||
|
await db.dropTable('ghost');
|
||||||
|
});
|
||||||
|
|
||||||
|
await db.close();
|
||||||
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||||
|
await db2.init();
|
||||||
|
const names = await db2.getTableNames();
|
||||||
|
expect(names).not.toContain('ghost');
|
||||||
|
await db2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// 各引擎 clearAll / repair / 元数据 一致性抽查
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('全模式抽查 — 生命周期与元数据', () => {
|
||||||
|
it('MemoryEngine clearAll/repair/getMeta/setMeta', async () => {
|
||||||
|
const e = new MemoryEngine();
|
||||||
|
await e.open('m', 1);
|
||||||
|
await e.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
|
||||||
|
await e.insert('t', [{ id: '1' }]);
|
||||||
|
await e.setMeta('k', 'v');
|
||||||
|
expect(await e.getMeta('k')).toBe('v');
|
||||||
|
await e.repair();
|
||||||
|
await e.clearAll();
|
||||||
|
expect(await e.getTableNames()).toEqual([]);
|
||||||
|
await e.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('HybridEngine getMeta/setMeta 委托磁盘', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const e = new HybridEngine('indexeddb');
|
||||||
|
await e.open(dbName, 1);
|
||||||
|
await e.setMeta('__metona_version', '7');
|
||||||
|
expect(await e.getMeta('__metona_version')).toBe('7');
|
||||||
|
await e.close();
|
||||||
|
const e2 = new HybridEngine('indexeddb');
|
||||||
|
await e2.open(dbName, 1);
|
||||||
|
expect(await e2.getMeta('__metona_version')).toBe('7');
|
||||||
|
await e2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('AriaEngine repair 幂等', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb' });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
|
await engine.repair();
|
||||||
|
await engine.repair();
|
||||||
|
expect(await engine.count('t')).toBe(1);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,390 @@
|
|||||||
|
/**
|
||||||
|
* v0.4.2 生产就绪审计(第三轮)
|
||||||
|
* 覆盖:
|
||||||
|
* - P0: Aria 事务进行中 checkpoint 截断 WAL → 崩溃恢复丢事务数据
|
||||||
|
* - P1: Aria dropTable / ALTER DROP 索引列 的二级索引清理
|
||||||
|
* - P1: OPFS 并发写乱序丢更新 + 空表/schema/索引持久化
|
||||||
|
* - P2: Aria memtable 阈值衰减、红黑树随机压力、onUpdate 级联、事务 DDL 显式拒绝
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
|
import { createSchema } from '../src/table/schema';
|
||||||
|
import { OPFSEngine } from '../src/engine/opfs';
|
||||||
|
import { MemTable } from '../src/engine/aria/index/memtable';
|
||||||
|
import { MetonaSqlark } from '../src/core';
|
||||||
|
import 'fake-indexeddb/auto';
|
||||||
|
|
||||||
|
let idbCounter = 0;
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `prod-${Date.now()}-${++idbCounter}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// OPFS mock(模拟真实 I/O:getFileHandle 延迟 + 写入按内容差异化耗时)
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
function mockOPFS(
|
||||||
|
files?: Map<string, string>,
|
||||||
|
opts: { ioDelay?: number; writeDelayFor?: (data: string) => number } = {},
|
||||||
|
): Map<string, string> {
|
||||||
|
const store = files ?? new Map<string, string>();
|
||||||
|
|
||||||
|
const dirMock = {
|
||||||
|
getDirectoryHandle: async (_name: string, _opts?: any) => dirMock as any,
|
||||||
|
getFileHandle: async (name: string, fileOpts?: any) => {
|
||||||
|
if (fileOpts?.create) {
|
||||||
|
if (opts.ioDelay) {
|
||||||
|
await new Promise((r) => setTimeout(r, opts.ioDelay));
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
createWritable: async () => ({
|
||||||
|
write: async (d: string) => {
|
||||||
|
const delay = opts.writeDelayFor ? opts.writeDelayFor(d) : 0;
|
||||||
|
if (delay > 0) {
|
||||||
|
await new Promise((r) => setTimeout(r, delay));
|
||||||
|
}
|
||||||
|
store.set(name, d);
|
||||||
|
},
|
||||||
|
close: async () => {},
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (!store.has(name)) throw new Error('Not found');
|
||||||
|
return { getFile: async () => ({ text: async () => store.get(name)!, arrayBuffer: async () => new ArrayBuffer(0) }) };
|
||||||
|
},
|
||||||
|
removeEntry: async (name: string) => { store.delete(name); },
|
||||||
|
};
|
||||||
|
(dirMock as any).entries = () => ({
|
||||||
|
[Symbol.asyncIterator]: async function* () {
|
||||||
|
for (const [k] of store) yield [k];
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const nav = (globalThis as any).navigator || {};
|
||||||
|
nav.storage = { getDirectory: async () => dirMock };
|
||||||
|
(globalThis as any).navigator = nav;
|
||||||
|
return store;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P0: 事务与 checkpoint 冲突
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
|
||||||
|
it('事务中触发 checkpoint → 崩溃恢复不丢事务数据', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({
|
||||||
|
storageBackend: 'indexeddb',
|
||||||
|
checkpointInterval: 2, // 每 2 次操作即触发 checkpoint(事务中途)
|
||||||
|
});
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.insert('t', [{ id: '1', v: 1 }]);
|
||||||
|
await engine.insert('t', [{ id: '2', v: 2 }]); // opCounter=2 → tick → checkpoint(修复前截断 WAL)
|
||||||
|
await engine.commitTransaction();
|
||||||
|
|
||||||
|
// 模拟异常退出(不 close)
|
||||||
|
await (engine as any).backend.close();
|
||||||
|
(engine as any).opened = false;
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
// 修复前:checkpoint 截断了事务的 WAL 记录 → 恢复后数据丢失
|
||||||
|
expect(await engine2.count('t')).toBe(2);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务回滚后再 checkpoint 正常截断', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({
|
||||||
|
storageBackend: 'indexeddb',
|
||||||
|
checkpointInterval: 1,
|
||||||
|
});
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
await engine.insert('t', [{ id: 'keep' }]);
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.insert('t', [{ id: 'tx1' }]);
|
||||||
|
await engine.rollbackTransaction();
|
||||||
|
// 事务结束后 checkpoint 可正常截断
|
||||||
|
await (engine as any).checkpointManager.forceCheckpoint();
|
||||||
|
await (engine as any).backend.close();
|
||||||
|
(engine as any).opened = false;
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('t')).toBe(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1: Aria DDL 的二级索引清理
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1 — Aria DDL 索引清理', () => {
|
||||||
|
it('dropTable 清理二级索引(重建同名表索引不脏)', async () => {
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'memory' });
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await engine.insert('t', [{ id: '1', email: 'a@x.com' }]);
|
||||||
|
await engine.dropTable('t');
|
||||||
|
// 索引 LSM 必须清理(修复前残留)
|
||||||
|
expect((engine as any).secondaryIndexes.size).toBe(0);
|
||||||
|
|
||||||
|
// 重建同名表 + 相同 id:旧索引残留会返回错误行
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await engine.insert('t', [{ id: '1', email: 'b@x.com' }]);
|
||||||
|
// 旧索引残留场景:按旧 email 查询不得命中新行(主 LSM 有 t:1 → 修复前返回错误结果)
|
||||||
|
const byOld = await engine.find('t', { table: 't', where: { email: 'a@x.com' } });
|
||||||
|
expect(byOld).toHaveLength(0);
|
||||||
|
const byNew = await engine.find('t', { table: 't', where: { email: 'b@x.com' } });
|
||||||
|
expect(byNew).toHaveLength(1);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ALTER TABLE DROP 索引列清理索引 LSM', async () => {
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'memory' });
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await engine.insert('t', [{ id: '1', email: 'a@x.com' }]);
|
||||||
|
await engine.alterTable('t', 'DROP', { name: 'email', type: 'string' });
|
||||||
|
expect((engine as any).secondaryIndexes.size).toBe(0);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('DROP_TABLE 崩溃恢复同样清理索引', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await engine.insert('t', [{ id: '1', email: 'a@x.com' }]);
|
||||||
|
await engine.dropTable('t');
|
||||||
|
// 模拟崩溃(不 close,WAL 含 DROP_TABLE)
|
||||||
|
await (engine as any).backend.close();
|
||||||
|
(engine as any).opened = false;
|
||||||
|
|
||||||
|
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
// 表不存在,且无索引残留
|
||||||
|
expect(await engine2.hasTable('t')).toBe(false);
|
||||||
|
expect((engine2 as any).secondaryIndexes.size).toBe(0);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P1: OPFS 持久化与并发
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P1 — OPFS 生产加固', () => {
|
||||||
|
it('并发写不丢数据(内存快照总是最新,多写内容一致)', async () => {
|
||||||
|
// 模拟真实 I/O 延迟:内存写同步、持久化异步 → 并发写内容均基于最新内存快照
|
||||||
|
const files = mockOPFS(undefined, {
|
||||||
|
ioDelay: 10,
|
||||||
|
writeDelayFor: (data) => (data.length < 40 ? 30 : 5),
|
||||||
|
});
|
||||||
|
const engine = new OPFSEngine();
|
||||||
|
await engine.open('opfs-race', 1);
|
||||||
|
await engine.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string' },
|
||||||
|
}));
|
||||||
|
// 两个写并发:内存同步写保证两个快照一致 → 无论完成顺序文件内容完整
|
||||||
|
const p1 = engine.insert('users', [{ id: '1', name: 'A' }]);
|
||||||
|
const p2 = engine.insert('users', [{ id: '2', name: 'B' }]);
|
||||||
|
await Promise.all([p1, p2]);
|
||||||
|
const rows = await engine.find('users', { table: 'users' });
|
||||||
|
expect(rows).toHaveLength(2);
|
||||||
|
// 文件内容完整(重启不丢)
|
||||||
|
expect(files.get('users.json')).toContain('"B"');
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('空表重启后保留(schema 持久化)', async () => {
|
||||||
|
mockOPFS();
|
||||||
|
const e1 = new OPFSEngine();
|
||||||
|
await e1.open('opfs-schema', 1);
|
||||||
|
await e1.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await e1.close();
|
||||||
|
|
||||||
|
// 重启:空表也应恢复(修复前空表消失)+ 索引标记恢复
|
||||||
|
const e2 = new OPFSEngine();
|
||||||
|
await e2.open('opfs-schema', 1);
|
||||||
|
expect(await e2.hasTable('users')).toBe(true);
|
||||||
|
const schema = await e2.getTableSchema('users');
|
||||||
|
expect(schema!.columns.email.index).toBe(true);
|
||||||
|
await e2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('有数据重启后索引查询可用', async () => {
|
||||||
|
mockOPFS();
|
||||||
|
const e1 = new OPFSEngine();
|
||||||
|
await e1.open('opfs-idx', 1);
|
||||||
|
await e1.createTable(createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
email: { type: 'string', index: true },
|
||||||
|
}));
|
||||||
|
await e1.insert('users', [{ id: '1', email: 'a@x.com' }]);
|
||||||
|
await e1.close();
|
||||||
|
|
||||||
|
const e2 = new OPFSEngine();
|
||||||
|
await e2.open('opfs-idx', 1);
|
||||||
|
const byEmail = await e2.find('users', { table: 'users', where: { email: 'a@x.com' } });
|
||||||
|
expect(byEmail).toHaveLength(1);
|
||||||
|
await e2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('dropTable 后重启无幽灵表', async () => {
|
||||||
|
mockOPFS();
|
||||||
|
const e1 = new OPFSEngine();
|
||||||
|
await e1.open('opfs-drop', 1);
|
||||||
|
await e1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
await e1.insert('t', [{ id: '1' }]);
|
||||||
|
await e1.dropTable('t');
|
||||||
|
await e1.close();
|
||||||
|
|
||||||
|
const e2 = new OPFSEngine();
|
||||||
|
await e2.open('opfs-drop', 1);
|
||||||
|
expect(await e2.hasTable('t')).toBe(false);
|
||||||
|
expect(await e2.getTableNames()).toEqual([]);
|
||||||
|
await e2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ===================================================================
|
||||||
|
// P2: Aria 内部状态
|
||||||
|
// ===================================================================
|
||||||
|
|
||||||
|
describe('P2 — Aria 内部状态加固', () => {
|
||||||
|
it('freezeMemtable 后阈值不衰减', async () => {
|
||||||
|
const engine = new AriaEngine({
|
||||||
|
storageBackend: 'memory',
|
||||||
|
memtableSizeThreshold: 1024 * 1024,
|
||||||
|
});
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
// 少量写入 + 手动 flush(freezeMemtable 用旧表已用大小当新阈值 → 衰减)
|
||||||
|
for (let i = 0; i < 10; i++) await engine.insert('t', [{ id: `${i}` }]);
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
// 修复前:新 memtable maxSize ≈ 已用字节(远小于配置)
|
||||||
|
const memtable = (engine as any).lsm.memtable;
|
||||||
|
expect((memtable as any).maxSize).toBeGreaterThanOrEqual(1024 * 1024);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('红黑树随机 insert/delete 5000 次保持有序且无丢失', () => {
|
||||||
|
const mem = new MemTable(1 << 30);
|
||||||
|
const reference = new Set<string>();
|
||||||
|
let seed = 42;
|
||||||
|
const rnd = () => {
|
||||||
|
seed = (seed * 1103515245 + 12345) & 0x7fffffff;
|
||||||
|
return seed / 0x7fffffff;
|
||||||
|
};
|
||||||
|
for (let i = 0; i < 5000; i++) {
|
||||||
|
const k = `k${Math.floor(rnd() * 800)}`;
|
||||||
|
if (rnd() < 0.3) {
|
||||||
|
reference.delete(k);
|
||||||
|
mem.delete(k);
|
||||||
|
} else {
|
||||||
|
reference.add(k);
|
||||||
|
mem.put(k, { v: i });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const entries = mem.getAllEntries();
|
||||||
|
// 有序
|
||||||
|
for (let i = 1; i < entries.length; i++) {
|
||||||
|
expect(entries[i][0] > entries[i - 1][0]).toBe(true);
|
||||||
|
}
|
||||||
|
// 与引用集合完全一致(无丢失/无残留)
|
||||||
|
expect(entries.map(([k]) => k)).toEqual([...reference].sort());
|
||||||
|
expect(mem.getEntryCount()).toBe(reference.size);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('onUpdate CASCADE:更新父表主键级联更新子表外键', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.defineTable('orders', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string', references: 'users.id', onUpdate: 'CASCADE' },
|
||||||
|
});
|
||||||
|
await db.table('users').insert({ id: '1' });
|
||||||
|
await db.table('orders').insert({ id: 'o1', user_id: '1' });
|
||||||
|
// 更新父表主键 1 → 2
|
||||||
|
await db.table('users').update({ id: '2' }).where({ id: '1' }).execute();
|
||||||
|
const orders = await db.table('orders').select().execute();
|
||||||
|
expect(orders[0].user_id).toBe('2');
|
||||||
|
// 旧 id 不可再被引用
|
||||||
|
expect(await db.table('orders').count({ user_id: '1' })).toBe(0);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('onUpdate RESTRICT:存在引用行时禁止更新主键', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.defineTable('orders', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string', references: 'users.id', onUpdate: 'RESTRICT' },
|
||||||
|
});
|
||||||
|
await db.table('users').insert({ id: '1' });
|
||||||
|
await db.table('orders').insert({ id: 'o1', user_id: '1' });
|
||||||
|
await expect(
|
||||||
|
db.table('users').update({ id: '2' }).where({ id: '1' }).execute(),
|
||||||
|
).rejects.toThrow();
|
||||||
|
// 主键未被修改
|
||||||
|
const users = await db.table('users').select().execute();
|
||||||
|
expect(users[0].id).toBe('1');
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('onUpdate CASCADE:更新父表主键级联更新子表外键(Aria)', async () => {
|
||||||
|
const db = new MetonaSqlark({ name: uniqueDB(), mode: 'aria', diskEngine: 'memory' });
|
||||||
|
await db.init();
|
||||||
|
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await db.defineTable('orders', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string', references: 'users.id', onUpdate: 'CASCADE' },
|
||||||
|
});
|
||||||
|
await db.table('users').insert({ id: '1' });
|
||||||
|
await db.table('orders').insert({ id: 'o1', user_id: '1' });
|
||||||
|
await db.table('users').update({ id: '2' }).where({ id: '1' }).execute();
|
||||||
|
const orders = await db.table('orders').select().execute();
|
||||||
|
expect(orders[0].user_id).toBe('2');
|
||||||
|
expect(await db.table('orders').count({ user_id: '1' })).toBe(0);
|
||||||
|
await db.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('Aria 事务中 DDL 显式拒绝(NOT_SUPPORTED)而非静默不一致', async () => {
|
||||||
|
const engine = new AriaEngine({ storageBackend: 'memory' });
|
||||||
|
await engine.open(uniqueDB(), 1);
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await expect(
|
||||||
|
engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } })),
|
||||||
|
).rejects.toMatchObject({ code: 'NOT_SUPPORTED' });
|
||||||
|
await engine.rollbackTransaction();
|
||||||
|
// 表未创建
|
||||||
|
expect(await engine.hasTable('t')).toBe(false);
|
||||||
|
await engine.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user