release: v0.6.0 — 完全移除 IndexedDB,自研 KVStore 事务存储引擎(多key原子写/快照日志恢复/CRC自愈)+ KVStoreEngine + 旧库迁移工具 + 10万级压力验证 + 崩溃注入e2e
This commit is contained in:
@@ -2,6 +2,59 @@
|
|||||||
|
|
||||||
All notable changes to MetonaSqlark will be documented in this file.
|
All notable changes to MetonaSqlark will be documented in this file.
|
||||||
|
|
||||||
|
## [0.6.0] - 2026-08-10
|
||||||
|
|
||||||
|
### 里程碑:完全移除 IndexedDB,自研 KV 事务存储引擎
|
||||||
|
|
||||||
|
> 达成"彻底删除 IndexedDB + 自研类似 IndexedDB 的存储后端"目标:
|
||||||
|
> KVStore 引擎在 OPFS 之上实现多 key 原子事务(IndexedDB 的核心能力),
|
||||||
|
> disk 模式全面切换,旧库一键迁移。
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **KVStore 自研 KV 事务引擎**(`src/engine/kvstore/`):
|
||||||
|
- 多 key 原子写:putMany/deleteMany = 单条日志记录(单文件 COW 原子追加)→ 崩溃全有或全无
|
||||||
|
- 持久化与崩溃恢复:快照(checkpoint)+ 追加日志,两阶段恢复(快照水位跳过)
|
||||||
|
- 自愈:快照损坏回退全量日志重放;日志损坏截断至损坏处;`repair()` 清理
|
||||||
|
- 容错时序:checkpoint = 写快照 → 写 meta → 清空日志(meta 先于截断,任何崩溃窗口不丢数据)
|
||||||
|
- 写操作与 checkpoint 串行队列(无交错窗口);标准 CRC-32 全程校验
|
||||||
|
- 介质层:OPFS(浏览器)/ SharedMemoryBackend(Node/测试,跨实例共享模拟持久化)
|
||||||
|
- **KVStoreEngine**(disk 模式,替代 IndexedDBEngine + OPFSEngine):
|
||||||
|
- 内存热路径 + KVStore 原子持久化;读 O(1),写增量/整表 diff 分级
|
||||||
|
- 事务:内存快照 + commit 原子 flush(受影响表),事务内 DDL 支持(schema 一并持久化)
|
||||||
|
- 外键级联(CASCADE/SET NULL/RESTRICT)持久化、主键变更、二级索引跨重启恢复
|
||||||
|
- `reload()` 支持多标签页同步重载(Hybrid 场景)
|
||||||
|
- **迁移工具** `migrateFromIndexedDB()`:一次性把旧 IndexedDB 库(schema/索引/行)导入新引擎;
|
||||||
|
旧库不存在/aria 私有格式(SSTable/WAL)明确报错;目标库已有表跳过不覆盖
|
||||||
|
- **可靠性强化**:
|
||||||
|
- 10 万 key 写入+checkpoint+重开全量验证(928ms)
|
||||||
|
- 5 万混合操作+崩溃模拟零丢失
|
||||||
|
- e2e 新增写入中途崩溃 / checkpoint 前后崩溃注入(9 用例)
|
||||||
|
- **`MetonaSqlark` 配置**:`diskEngine` 类型收敛为 `'opfs' | 'memory'`(默认 'opfs')
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
|
||||||
|
- `src/engine/indexeddb.ts`(840 行 IndexedDBEngine)—— 版本冲突/blocked 重试/onversionchange/flushToIDB 全部消失
|
||||||
|
- `src/engine/opfs.ts`(375 行 OPFSEngine)—— 能力被 KVStoreEngine 覆盖
|
||||||
|
- `IndexedDBBackend`(aria 后端)—— aria storageBackend 收敛为 'opfs' | 'memory'
|
||||||
|
- 26 个测试文件的 fake-indexeddb 依赖改造为 KVStore SharedMemory / OPFS mock
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Hybrid 多标签页重载失效**(KVStoreEngine 读内存 vs 磁盘引擎读穿透)—— 新增 `reload()` 从介质重载
|
||||||
|
- **事务内 DDL schema 不持久化**—— commit 时统一 persistSchema(重启后表结构完整)
|
||||||
|
- **repair 缓存掩盖损坏**—— 先清 BufferPool 再校验(缓存中"完好页面"不再掩盖磁盘损坏)
|
||||||
|
- **dropInvalidSSTable 孤儿页面**—— 先删数据文件再删 meta(页面化删除依赖 pageIds 定位)
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- 测试 1009 → **1021**(64 套件;+58 新增 kvstore/kvstore-engine/压力/迁移测试,-46 删除/改造 IDB 测试)
|
||||||
|
- 行覆盖率 87.25% → **89.13%**
|
||||||
|
- 浏览器支持收敛到 OPFS 矩阵(Chrome/Edge 102+、Firefox 111+、Safari 15.2+);Node 用内存介质
|
||||||
|
- fake-indexeddb 保留为 devDependency(仅迁移工具测试用)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [0.5.1] - 2026-08-10
|
## [0.5.1] - 2026-08-10
|
||||||
|
|
||||||
### 深度审查:消除假实现/半成品/死代码
|
### 深度审查:消除假实现/半成品/死代码
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
# MetonaSqlark
|
# MetonaSqlark
|
||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<img src="https://img.shields.io/badge/version-0.5.1-blue?style=flat-square" alt="version">
|
<img src="https://img.shields.io/badge/version-0.6.0-blue?style=flat-square" alt="version">
|
||||||
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
||||||
<img src="https://img.shields.io/badge/coverage-87.3%25-brightgreen?style=flat-square" alt="coverage">
|
<img src="https://img.shields.io/badge/coverage-89.1%25-brightgreen?style=flat-square" alt="coverage">
|
||||||
<img src="https://img.shields.io/badge/tests-1009%20passed-success?style=flat-square" alt="tests">
|
<img src="https://img.shields.io/badge/tests-1021%20passed-success?style=flat-square" alt="tests">
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。
|
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。
|
||||||
@@ -19,7 +19,9 @@
|
|||||||
- 🔐 **多标签页独占锁** — Web Locks API,第二个标签页打开同一库抛 `ARIA_LOCKED`(v0.5.0)
|
- 🔐 **多标签页独占锁** — Web Locks API,第二个标签页打开同一库抛 `ARIA_LOCKED`(v0.5.0)
|
||||||
- 🛠 **维护语句 SQL 入口** — `EXPLAIN`/`ANALYZE`/`REINDEX`/`VACUUM`/`SAVEPOINT` 原生 SQL 支持(v0.5.1 补齐)
|
- 🛠 **维护语句 SQL 入口** — `EXPLAIN`/`ANALYZE`/`REINDEX`/`VACUUM`/`SAVEPOINT` 原生 SQL 支持(v0.5.1 补齐)
|
||||||
- 🛡 **输入校验全覆盖** — `maxLength`/`min`/`max` 约束、类型检查、必填验证
|
- 🛡 **输入校验全覆盖** — `maxLength`/`min`/`max` 约束、类型检查、必填验证
|
||||||
- 💾 **多引擎架构** — Memory / IndexedDB / OPFS / Hybrid(write-through) / Aria 五种模式
|
- 💾 **多引擎架构** — Memory / **KVStore**(自研 KV 引擎)/ OPFS / Hybrid(write-through) / Aria 五种模式(v0.6.0: IndexedDB 完全移除)
|
||||||
|
- 💾 **KVStore 自研 KV 引擎** — 日志结构化事务存储:多 key 原子写(putMany/deleteMany 单记录原子追加)、快照 checkpoint、崩溃两阶段恢复、CRC-32 自愈;替代 IndexedDB(v0.6.0)
|
||||||
|
- 🔄 **旧库一键迁移** — `migrateFromIndexedDB()` 把旧 IndexedDB 数据(schema/索引/行)导入新引擎(v0.6.0)
|
||||||
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTS(v0.3.0)+ CASE WHEN/哈希连接/组提交(v0.3.1)+ 多标签页同步(v0.3.2)
|
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTS(v0.3.0)+ CASE WHEN/哈希连接/组提交(v0.3.1)+ 多标签页同步(v0.3.2)
|
||||||
- 🚰 **流式查询** — `queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.0)
|
- 🚰 **流式查询** — `queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.0)
|
||||||
- 🧩 **派生表** — `FROM (SELECT ...)` 子查询作为行源,多列 ON 哈希连接,COUNT(DISTINCT),NULLS FIRST/LAST(v0.4.0)
|
- 🧩 **派生表** — `FROM (SELECT ...)` 子查询作为行源,多列 ON 哈希连接,COUNT(DISTINCT),NULLS FIRST/LAST(v0.4.0)
|
||||||
@@ -32,7 +34,7 @@
|
|||||||
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
|
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
|
||||||
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
||||||
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
||||||
- 🧪 **1009 测试 · 87.3% 覆盖率** — 62 套件 + 7 个 Playwright 真实 Chromium e2e,生产级质量保证
|
- 🧪 **1021 测试 · 89.1% 覆盖率** — 64 套件 + 9 个 Playwright 真实 Chromium e2e(含崩溃注入),生产级质量保证
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -276,6 +278,30 @@ await db2.disconnect(); // 引用计数 -1
|
|||||||
|
|
||||||
> 不支持的引擎执行维护语句抛 `NOT_SUPPORTED`。
|
> 不支持的引擎执行维护语句抛 `NOT_SUPPORTED`。
|
||||||
|
|
||||||
|
### 旧库迁移(v0.6.0)
|
||||||
|
|
||||||
|
> IndexedDB 已从引擎中完全移除。使用旧版本(v0.5.x 及更早)的用户,可通过
|
||||||
|
> 一次性迁移工具把磁盘模式(IndexedDBEngine)旧库导入新引擎(KVStore)。
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { migrateFromIndexedDB } from '@metona-team/metona-sqlark/migration';
|
||||||
|
|
||||||
|
// 目标库(新引擎,disk 模式)
|
||||||
|
const target = await MetonaSqlark.create({ name: 'my-app-new', mode: 'disk' });
|
||||||
|
|
||||||
|
// 从旧 IndexedDB 库导入(旧库名 'my-app',旧引擎 disk 模式)
|
||||||
|
const result = await migrateFromIndexedDB({
|
||||||
|
dbName: 'my-app',
|
||||||
|
engine: 'disk', // 仅支持 disk 模式(IndexedDBEngine)
|
||||||
|
target,
|
||||||
|
onProgress: (done, total, table) => console.log(`迁移 ${done}/${total}: ${table}`),
|
||||||
|
});
|
||||||
|
// result: { migratedTables, rowCount, skippedTables }
|
||||||
|
```
|
||||||
|
|
||||||
|
> 注:aria 模式旧库(IndexedDBBackend)数据为引擎私有格式(SSTable/WAL),
|
||||||
|
> 无法按行迁移——此类用户请从应用层导出(exportAll)后重新导入。
|
||||||
|
|
||||||
### 连接池(v0.1.13)
|
### 连接池(v0.1.13)
|
||||||
|
|
||||||
| 静态方法 | 说明 |
|
| 静态方法 | 说明 |
|
||||||
@@ -301,18 +327,19 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
|||||||
|
|
||||||
## 📊 存储模式对比
|
## 📊 存储模式对比
|
||||||
|
|
||||||
| 特性 | Memory | Disk (IndexedDB) | Disk (OPFS) | Hybrid | Aria |
|
| 特性 | Memory | Disk (KVStore) | Disk (OPFS) | Hybrid | Aria |
|
||||||
|------|--------|------------------|-------------|--------|------|
|
|------|--------|----------------|-------------|--------|------|
|
||||||
| **持久化** | ❌ 重启丢失 | ✅ IndexedDB | ✅ OPFS(schema 持久化) | ✅ 内存+磁盘 | ✅ 后端决定 |
|
| **持久化** | ❌ 重启丢失 | ✅ KVStore(OPFS) | ✅ OPFS(schema 持久化) | ✅ 内存+磁盘 | ✅ 后端决定 |
|
||||||
| **事务回滚** | ✅ 快照 | ✅ 原子flush | ✅ 快照 | ✅ 双引擎 | ✅ MVCC |
|
| **事务回滚** | ✅ 快照 | ✅ 原子日志 flush | ✅ 快照 | ✅ 双引擎 | ✅ MVCC |
|
||||||
| **二级索引** | ✅ Hash | ✅ Hash | ✅ Hash(重启恢复) | ✅ Hash | ✅ LSM(重启恢复) |
|
| **多 key 原子写** | — | ✅ 单日志记录原子(v0.6.0) | — | ✅ 委托磁盘 | ✅ WAL 单文件原子 |
|
||||||
| **查询性能** | ⚡ O(1) PK | 🟡 O(1) PK | 🟡 O(1) PK | ⚡ O(1) PK | ⚡ O(log n) |
|
| **二级索引** | ✅ Hash | ✅ Hash(重启恢复) | ✅ Hash(重启恢复) | ✅ Hash | ✅ LSM(重启恢复) |
|
||||||
| **数据上限** | 内存限制 | ~2GB(IDB限制) | ~磁盘可用(页面化后大表可行) | ~2GB(IDB) | 内存限制 |
|
| **查询性能** | ⚡ O(1) PK | ⚡ O(1) PK(内存热路径) | 🟡 O(1) PK | ⚡ O(1) PK | ⚡ O(log n) |
|
||||||
| **浏览器** | 全部 | 全部 | Chrome/Edge 102+ / Firefox 111+ / Safari 15.2+ | 全部 | 全部 |
|
| **数据上限** | 内存限制 | 磁盘可用(行级存储,大表可行) | ~磁盘可用(整表 JSON,≤1000 行) | 磁盘可用 | 内存限制 |
|
||||||
| **多标签页** | — | ✅ versionchange | ❌ 无保护 | ❌ 无保护 | ✅ Web Locks 独占锁(v0.5.0) |
|
| **浏览器** | 全部 | Chrome/Edge 102+ / Firefox 111+ / Safari 15.2+ | 同上 | 同上 | 同上 |
|
||||||
|
| **多标签页** | — | —(无事务锁,Hybrid 场景) | ❌ 无保护 | ❌ 无保护 | ✅ Web Locks 独占锁(v0.5.0) |
|
||||||
| **全库加密** | — | — | — | — | ✅ AES-GCM(v0.5.0) |
|
| **全库加密** | — | — | — | — | ✅ AES-GCM(v0.5.0) |
|
||||||
| **适用场景** | 缓存/测试 | 标准持久化 | Chromium+ 持久化 | 速度+持久化 | 大规模/分析 |
|
| **适用场景** | 缓存/测试 | 标准持久化(替代 IndexedDB) | 小数据集 | 速度+持久化 | 大规模/分析 |
|
||||||
| **测试覆盖** | 30+ | 30+ | 15+e2e | 15+ | 400+ |
|
| **测试覆盖** | 30+ | 50+(含 10 万级压力) | 15 | 15+ | 400+ |
|
||||||
|
|
||||||
### Memory 模式
|
### Memory 模式
|
||||||
- **环境**: 所有浏览器、Node.js
|
- **环境**: 所有浏览器、Node.js
|
||||||
@@ -320,11 +347,12 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
|||||||
- **能力**: 完整 CRUD、事务回滚、外键级联、二级索引、SQL 全支持
|
- **能力**: 完整 CRUD、事务回滚、外键级联、二级索引、SQL 全支持
|
||||||
- **适用**: 临时数据、单元测试、缓存层
|
- **适用**: 临时数据、单元测试、缓存层
|
||||||
|
|
||||||
### Disk (IndexedDB) 模式
|
### Disk (KVStore) 模式
|
||||||
- **环境**: 所有现代浏览器(Chrome/Firefox/Safari/Edge)、Node.js(fake-indexeddb)
|
- **环境**: Chrome/Edge 102+ / Firefox 111+ / Safari 15.2+(OPFS)、Node.js(内存介质)
|
||||||
- **限制**: 受浏览器 IndexedDB 配额限制(通常 ~2GB),多标签页需处理版本冲突
|
- **限制**: 依赖 OPFS;多标签页并发写无锁保护(单标签页内可靠)
|
||||||
- **能力**: 完整 CRUD、事务原子性(单 IDB 事务包裹)、外键级联、onversionchange 感知
|
- **能力**: 完整 CRUD、**多 key 原子事务**(单日志记录原子追加)、外键级联、事务内 DDL、
|
||||||
- **适用**: 标准前端数据库持久化场景
|
二级索引(重启恢复)、schema/数据/索引完整持久化、10 万级数据量压力验证
|
||||||
|
- **适用**: 标准前端数据库持久化(v0.6.0 替代 IndexedDB)
|
||||||
|
|
||||||
### Disk (OPFS) 模式
|
### Disk (OPFS) 模式
|
||||||
- **环境**: Chrome 102+ / Edge 102+ / Firefox 111+ / Safari 15.2+(Origin Private File System)
|
- **环境**: Chrome 102+ / Edge 102+ / Firefox 111+ / Safari 15.2+(Origin Private File System)
|
||||||
@@ -418,7 +446,7 @@ npm install # 安装依赖
|
|||||||
npm run dev # 开发模式(localhost:3001)
|
npm run dev # 开发模式(localhost:3001)
|
||||||
npm run build # 生产构建(生成 dist/)
|
npm run build # 生产构建(生成 dist/)
|
||||||
npm test # 运行测试
|
npm test # 运行测试
|
||||||
npm run test:e2e # Playwright e2e(真实 Chromium + OPFS,需先 build)
|
npm run test:e2e # Playwright e2e(真实 Chromium + OPFS + 崩溃注入,需先 build)
|
||||||
npm run lint # 代码检查
|
npm run lint # 代码检查
|
||||||
npm run typecheck # 类型检查
|
npm run typecheck # 类型检查
|
||||||
```
|
```
|
||||||
@@ -429,21 +457,21 @@ npm run typecheck # 类型检查
|
|||||||
|
|
||||||
| 指标 | 数值 |
|
| 指标 | 数值 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| 测试用例 | 1009 |
|
| 测试用例 | 1021 |
|
||||||
| 测试套件 | 62(+7 Playwright e2e) |
|
| 测试套件 | 64(+9 Playwright e2e) |
|
||||||
| 行覆盖率 | 87.3% |
|
| 行覆盖率 | 89.1% |
|
||||||
| SQL 关键字 | 72 |
|
| SQL 关键字 | 72 |
|
||||||
| 存储引擎 | 5(Memory / IndexedDB / OPFS / Hybrid / **Aria**) |
|
| 存储引擎 | 5(Memory / **KVStore** / OPFS / Hybrid / **Aria**) |
|
||||||
|
|
||||||
### 🌐 浏览器兼容性
|
### 🌐 浏览器兼容性
|
||||||
|
|
||||||
| 浏览器 | 最低版本 | Memory | IndexedDB | OPFS | Web Locks | Aria |
|
| 浏览器 | 最低版本 | Memory | KVStore(OPFS) | OPFS | Web Locks | Aria |
|
||||||
|--------|----------|--------|-----------|------|-----------|------|
|
|--------|----------|--------|---------------|------|-----------|------|
|
||||||
| Chrome | 80+ | ✅ | ✅ | ✅ (102+) | ✅ (69+) | ✅ |
|
| Chrome | 102+ | ✅ | ✅ | ✅ | ✅ (69+) | ✅ |
|
||||||
| Firefox | 80+ | ✅ | ✅ | ✅ (111+) | ✅ (96+) | ✅ |
|
| Firefox | 111+ | ✅ | ✅ | ✅ | ✅ (96+) | ✅ |
|
||||||
| Safari | 14+ | ✅ | ✅ | ✅ (15.2+) | ✅ (15.4+) | ✅ |
|
| Safari | 15.2+ | ✅ | ✅ | ✅ | ✅ (15.4+) | ✅ |
|
||||||
| Edge | 80+ | ✅ | ✅ | ✅ (102+) | ✅ (79+) | ✅ |
|
| Edge | 102+ | ✅ | ✅ | ✅ | ✅ (79+) | ✅ |
|
||||||
| Node.js | 16+ | ✅ | ✅ (fake-idb) | ❌ (测试用 mock) | ❌ | ✅ |
|
| Node.js | 16+ | ✅ | ✅ (内存介质) | ❌ (测试用 mock) | ❌ | ✅ |
|
||||||
|
|
||||||
> **OPFS 支持说明(v0.4.5 更新)**: Chromium(Chrome/Edge 102+)、Firefox 111+、Safari 15.2+
|
> **OPFS 支持说明(v0.4.5 更新)**: Chromium(Chrome/Edge 102+)、Firefox 111+、Safari 15.2+
|
||||||
> 均已支持基础 OPFS API(`createWritable` 原子写)。OPFS 无跨文件事务,
|
> 均已支持基础 OPFS API(`createWritable` 原子写)。OPFS 无跨文件事务,
|
||||||
|
|||||||
Vendored
+1293
-1402
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+76
-158
@@ -18,7 +18,7 @@ interface AriaEngineConfig {
|
|||||||
/** 是否启用页面压缩(默认 false) */
|
/** 是否启用页面压缩(默认 false) */
|
||||||
compression?: boolean;
|
compression?: boolean;
|
||||||
/** 存储后端 */
|
/** 存储后端 */
|
||||||
storageBackend?: 'indexeddb' | 'opfs' | 'memory';
|
storageBackend?: 'opfs' | 'memory';
|
||||||
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB) */
|
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB) */
|
||||||
walSizeThreshold?: number;
|
walSizeThreshold?: number;
|
||||||
/** 最大内存预算(MB,默认 64) */
|
/** 最大内存预算(MB,默认 64) */
|
||||||
@@ -45,8 +45,8 @@ interface AriaEngineConfig {
|
|||||||
*/
|
*/
|
||||||
/** 存储模式 */
|
/** 存储模式 */
|
||||||
type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
||||||
/** 磁盘引擎类型 */
|
/** 磁盘引擎类型(v0.6.0: IndexedDB 已移除,'memory' 供 aria 内存后端) */
|
||||||
type DiskEngine = 'indexeddb' | 'opfs';
|
type DiskEngine = 'opfs' | 'memory';
|
||||||
/** 字段数据类型 */
|
/** 字段数据类型 */
|
||||||
type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json';
|
type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json';
|
||||||
/** 列定义 */
|
/** 列定义 */
|
||||||
@@ -164,7 +164,7 @@ interface MetonaPlugin {
|
|||||||
/** 销毁 */
|
/** 销毁 */
|
||||||
destroy(): void;
|
destroy(): void;
|
||||||
}
|
}
|
||||||
declare const VERSION = "0.5.1";
|
declare const VERSION = "0.6.0";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* metona-sqlark Plugin — 插件系统
|
* metona-sqlark Plugin — 插件系统
|
||||||
@@ -832,118 +832,75 @@ declare class MemoryEngine implements IStorageEngine {
|
|||||||
private cascadeDelete;
|
private cascadeDelete;
|
||||||
}
|
}
|
||||||
|
|
||||||
declare class IndexedDBEngine implements IStorageEngine {
|
/**
|
||||||
readonly name = "indexeddb";
|
* AriaEngine Storage Backend — 存储后端抽象层
|
||||||
private db;
|
* @module engine/aria/store/backend
|
||||||
private dbName;
|
*
|
||||||
private version;
|
* 封装底层浏览器存储 API(IndexedDB / OPFS / Memory 回退),
|
||||||
private memoryCache;
|
* 供 Buffer Pool 的 PageIO 和 WAL 的 WALStore 使用。
|
||||||
private txActive;
|
|
||||||
open(dbName: string, version: number): Promise<void>;
|
|
||||||
/** v0.4.2-fix: 多标签页冲突处理 — 其他标签页升级版本时自动关闭当前连接 */
|
|
||||||
private setupVersionChangeHandler;
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 打开 IndexedDB 连接。
|
|
||||||
* - P0-2: 请求版本低于库实际版本(VersionError)时,先无版本参数探测库当前版本,
|
|
||||||
* 再以实际版本重开(建表每张表版本号 +1,config.version 会过期)
|
|
||||||
* - P2-8: onblocked 为瞬时状态(另一连接短暂持有),等待后重试多次,超时才抛 IDB_BLOCKED
|
|
||||||
*/
|
*/
|
||||||
private openDatabaseWithRetry;
|
interface IStorageBackend {
|
||||||
/**
|
/** 打开存储 */
|
||||||
* 发起一次 indexedDB.open 请求(success/error/blocked 三态收敛)。
|
open(name: string): Promise<void>;
|
||||||
* onblocked 不立即失败:阻塞解除后 success 仍会触发,仅超时兜底判失败,
|
/** 关闭存储 */
|
||||||
* 避免"拒绝后连接迟到成功"泄漏未关闭的数据库连接。
|
|
||||||
*/
|
|
||||||
private openRequest;
|
|
||||||
/** 无版本参数打开库,解析其当前实际版本号(随后立即关闭) */
|
|
||||||
private resolveCurrentVersion;
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix (P2-7): 确保 __metona_schema store 存在。
|
|
||||||
* 新库(或版本升级前创建的旧库)没有该 store 时,通过一次版本升级创建,
|
|
||||||
* 使 getMeta/setMeta(迁移版本持久化)始终可用。
|
|
||||||
*/
|
|
||||||
private ensureSchemaStore;
|
|
||||||
/**
|
|
||||||
* 从 IDB 恢复内存 schema:
|
|
||||||
* 1. 优先读取持久化的 schema 记录('__metona_schema' store,v0.3.2)
|
|
||||||
* 2. 旧数据回退:从 objectStore 主键 / 索引 / 样例数据推断
|
|
||||||
*/
|
|
||||||
private rebuildSchemaFromIDB;
|
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
/**
|
/** 是否已打开 */
|
||||||
* v0.4.2-fix: 自愈 — 从磁盘重建内存 schema 与数据(schema 丢失/内存不一致时调用)。
|
|
||||||
* 无删库需求即可恢复可用的库。
|
|
||||||
*/
|
|
||||||
repair(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 清空全部数据与表结构(含持久化 schema 记录),保留库本身。
|
|
||||||
* 单个版本升级事务内原子完成。
|
|
||||||
*/
|
|
||||||
clearAll(): Promise<void>;
|
|
||||||
getMeta(key: string): Promise<string | null>;
|
|
||||||
setMeta(key: string, value: string): Promise<void>;
|
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
createTable(schema: TableSchema): Promise<void>;
|
/** 读取数据块 */
|
||||||
dropTable(tableName: string): Promise<void>;
|
read(key: string): Promise<ArrayBuffer | null>;
|
||||||
hasTable(tableName: string): Promise<boolean>;
|
/** 写入数据块 */
|
||||||
getTableNames(): Promise<string[]>;
|
write(key: string, data: ArrayBuffer): Promise<void>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
|
||||||
/**
|
/**
|
||||||
* v0.4.2-fix: 引擎级 ALTER TABLE — schema 持久化到 __metona_schema store,
|
* 追加写入(v0.4.5 WAL 分片用,可选):
|
||||||
* 重启后 ALTER 不丢失(此前通用路径只改内存引用,重启回退;DROP 的行数据也没真正删)。
|
* - OPFS 后端实现真追加(createWritable keepExistingData + seek,O(chunk))
|
||||||
|
* - 未实现的后端由调用方回退 read+write(EncryptedBackend 包装时整体重写保正确性)
|
||||||
|
* 语义:在 key 现有内容末尾追加 data;key 不存在时等同 write。
|
||||||
*/
|
*/
|
||||||
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
append?(key: string, data: ArrayBuffer): Promise<void>;
|
||||||
name: string;
|
/**
|
||||||
}): Promise<void>;
|
* 批量原子写入(v0.4.2-fix):多个 key 在单个底层事务中提交,
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
* 中断时整体回滚,不留半写状态。WAL count 与记录同事务保证一致性。
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
*/
|
||||||
/** v0.4.0: 流式查询 — IDB 批量读入后逐行回调(保持接口一致性) */
|
writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
|
||||||
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
|
/** 删除数据块 */
|
||||||
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
delete(key: string): Promise<void>;
|
||||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
/**
|
||||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
* 批量原子删除(v0.4.2-fix):多个 key 在单个底层事务中提交。
|
||||||
clear(tableName: string): Promise<void>;
|
*/
|
||||||
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>;
|
deleteMany(keys: string[]): Promise<void>;
|
||||||
dropIndex(tableName: string, column: string, _indexName?: string): Promise<void>;
|
/** 列出所有 key */
|
||||||
beginTransaction(): Promise<void>;
|
listKeys(): Promise<string[]>;
|
||||||
commitTransaction(): Promise<void>;
|
/** 检查 key 是否存在 */
|
||||||
rollbackTransaction(): Promise<void>;
|
exists(key: string): Promise<boolean>;
|
||||||
private idbCreateTable;
|
/** 清空所有数据 */
|
||||||
private idbDropTable;
|
clear(): Promise<void>;
|
||||||
private idbInsert;
|
|
||||||
private idbFind;
|
|
||||||
/** 尝试使用 IDB 索引进行等值查询,成功返回结果,不适用返回 null */
|
|
||||||
private tryIDBIndexLookup;
|
|
||||||
private idbUpdate;
|
|
||||||
private idbDelete;
|
|
||||||
private idbClear;
|
|
||||||
/** 持久化单个表 schema 到 __metona_schema store(v0.4.2-fix: ALTER TABLE 用) */
|
|
||||||
private persistSchema;
|
|
||||||
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
|
|
||||||
private flushToIDB;
|
|
||||||
private ensureDB;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
declare class OPFSEngine implements IStorageEngine {
|
declare class KVStoreEngine implements IStorageEngine {
|
||||||
readonly name = "opfs";
|
readonly name = "kv";
|
||||||
private root;
|
private kv;
|
||||||
private tablesDir;
|
private memory;
|
||||||
private dbName;
|
private dbName;
|
||||||
private memoryCache;
|
private version;
|
||||||
/**
|
private opened;
|
||||||
* v0.4.3-fix: 写操作串行队列 — 内存写 + 快照 + 文件持久化整体排队执行,
|
/** 活跃事务标记 */
|
||||||
* close() 等待队列排空后再释放目录句柄(避免 close 后挂起写泄漏/读旧数据)。
|
private txActive;
|
||||||
* 前一个操作失败不阻塞后续(错误仍返回给调用方)。
|
/** 事务中写过的表(commit 时只 flush 这些表) */
|
||||||
*/
|
private txDirtyTables;
|
||||||
private opQueue;
|
constructor(medium?: IStorageBackend, checkpointThreshold?: number);
|
||||||
/** 将写操作加入串行队列(快照在队列内取,始终最新) */
|
private rowKey;
|
||||||
private enqueueOp;
|
private rowPrefix;
|
||||||
open(dbName: string, version: number): Promise<void>;
|
open(dbName: string, version: number): Promise<void>;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
isOpen(): boolean;
|
isOpen(): boolean;
|
||||||
/** 自愈:重置内存缓存后从 OPFS 重新加载(单文件损坏不影响其他表) */
|
/**
|
||||||
|
* v0.6.0: 从 KVStore 重新加载全部数据到内存(多标签页同步重载用)。
|
||||||
|
* Hybrid 引擎的 reloadMemoryFromDisk 依赖磁盘引擎"读穿透",
|
||||||
|
* KVStoreEngine 读内存 → 提供 reload 重新加载磁盘最新数据。
|
||||||
|
*/
|
||||||
|
reload(): Promise<void>;
|
||||||
|
/** v0.4.2-fix: 自愈 — 校验 KVStore 日志/快照完整性并重建内存 */
|
||||||
repair(): Promise<void>;
|
repair(): Promise<void>;
|
||||||
/** 清空全部数据与表结构(删除目录内全部文件) */
|
|
||||||
clearAll(): Promise<void>;
|
clearAll(): Promise<void>;
|
||||||
getMeta(key: string): Promise<string | null>;
|
getMeta(key: string): Promise<string | null>;
|
||||||
setMeta(key: string, value: string): Promise<void>;
|
setMeta(key: string, value: string): Promise<void>;
|
||||||
@@ -952,13 +909,11 @@ declare class OPFSEngine implements IStorageEngine {
|
|||||||
hasTable(tableName: string): Promise<boolean>;
|
hasTable(tableName: string): Promise<boolean>;
|
||||||
getTableNames(): Promise<string[]>;
|
getTableNames(): Promise<string[]>;
|
||||||
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
getTableSchema(tableName: string): Promise<TableSchema | null>;
|
||||||
/** v0.4.2-fix: 引擎级 ALTER TABLE — 内存 + schema 持久化 + 整表文件重写 */
|
|
||||||
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
|
||||||
name: string;
|
name: string;
|
||||||
}): Promise<void>;
|
}): Promise<void>;
|
||||||
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
|
||||||
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
|
||||||
/** v0.4.0: 流式查询(委托内存缓存) */
|
|
||||||
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
|
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
|
||||||
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
|
||||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||||
@@ -969,15 +924,22 @@ declare class OPFSEngine implements IStorageEngine {
|
|||||||
beginTransaction(): Promise<void>;
|
beginTransaction(): Promise<void>;
|
||||||
commitTransaction(): Promise<void>;
|
commitTransaction(): Promise<void>;
|
||||||
rollbackTransaction(): Promise<void>;
|
rollbackTransaction(): Promise<void>;
|
||||||
private ensureDir;
|
private ensureOpen;
|
||||||
private writeTableData;
|
private getPK;
|
||||||
private readTableData;
|
/** 收集匹配查询的内存行主键(持久化差异计算用) */
|
||||||
|
private collectMatchingPks;
|
||||||
/**
|
/**
|
||||||
* 从 OPFS 加载已有表到内存缓存。
|
* 计算外键级联影响的表集合(传递闭包:A 被 B 引用,B 被 C 引用 → {A, B, C})。
|
||||||
* v0.4.2-fix: 优先从持久化 schema(__metona_schema_*.meta)恢复 —
|
* 级联操作(delete/update 主键)需要把这些表一并重写持久化。
|
||||||
* 空表不再消失、索引标记/主键/约束完整;无 schema 记录的旧库从数据推断(兼容)。
|
|
||||||
*/
|
*/
|
||||||
private loadExistingTables;
|
private affectedTables;
|
||||||
|
/** 持久化 schema(全部表) */
|
||||||
|
private persistSchema;
|
||||||
|
/**
|
||||||
|
* 整表 diff 持久化:内存行全部 put + KV 残留行删除(原子 putMany + deleteMany)。
|
||||||
|
* 用于主键变更 / 级联 / dropTable / clear / alterTable DROP / 事务 commit。
|
||||||
|
*/
|
||||||
|
private flushTable;
|
||||||
}
|
}
|
||||||
|
|
||||||
declare class AriaEngine implements IStorageEngine {
|
declare class AriaEngine implements IStorageEngine {
|
||||||
@@ -1322,50 +1284,6 @@ interface Token {
|
|||||||
/** 将 SQL 字符串解析为 Token 列表 */
|
/** 将 SQL 字符串解析为 Token 列表 */
|
||||||
declare function tokenize(sql: string): Token[];
|
declare function tokenize(sql: string): Token[];
|
||||||
|
|
||||||
/**
|
|
||||||
* AriaEngine Storage Backend — 存储后端抽象层
|
|
||||||
* @module engine/aria/store/backend
|
|
||||||
*
|
|
||||||
* 封装底层浏览器存储 API(IndexedDB / OPFS / Memory 回退),
|
|
||||||
* 供 Buffer Pool 的 PageIO 和 WAL 的 WALStore 使用。
|
|
||||||
*/
|
|
||||||
interface IStorageBackend {
|
|
||||||
/** 打开存储 */
|
|
||||||
open(name: string): Promise<void>;
|
|
||||||
/** 关闭存储 */
|
|
||||||
close(): Promise<void>;
|
|
||||||
/** 是否已打开 */
|
|
||||||
isOpen(): boolean;
|
|
||||||
/** 读取数据块 */
|
|
||||||
read(key: string): Promise<ArrayBuffer | null>;
|
|
||||||
/** 写入数据块 */
|
|
||||||
write(key: string, data: ArrayBuffer): Promise<void>;
|
|
||||||
/**
|
|
||||||
* 追加写入(v0.4.5 WAL 分片用,可选):
|
|
||||||
* - OPFS 后端实现真追加(createWritable keepExistingData + seek,O(chunk))
|
|
||||||
* - 未实现的后端由调用方回退 read+write(EncryptedBackend 包装时整体重写保正确性)
|
|
||||||
* 语义:在 key 现有内容末尾追加 data;key 不存在时等同 write。
|
|
||||||
*/
|
|
||||||
append?(key: string, data: ArrayBuffer): Promise<void>;
|
|
||||||
/**
|
|
||||||
* 批量原子写入(v0.4.2-fix):多个 key 在单个底层事务中提交,
|
|
||||||
* 中断时整体回滚,不留半写状态。WAL count 与记录同事务保证一致性。
|
|
||||||
*/
|
|
||||||
writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
|
|
||||||
/** 删除数据块 */
|
|
||||||
delete(key: string): Promise<void>;
|
|
||||||
/**
|
|
||||||
* 批量原子删除(v0.4.2-fix):多个 key 在单个底层事务中提交。
|
|
||||||
*/
|
|
||||||
deleteMany(keys: string[]): Promise<void>;
|
|
||||||
/** 列出所有 key */
|
|
||||||
listKeys(): Promise<string[]>;
|
|
||||||
/** 检查 key 是否存在 */
|
|
||||||
exists(key: string): Promise<boolean>;
|
|
||||||
/** 清空所有数据 */
|
|
||||||
clear(): Promise<void>;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* AriaEngine OPFS Backend — 基于 Origin Private File System 的自研存储后端
|
* AriaEngine OPFS Backend — 基于 Origin Private File System 的自研存储后端
|
||||||
* @module engine/aria/store/opfs_backend
|
* @module engine/aria/store/opfs_backend
|
||||||
@@ -1463,4 +1381,4 @@ declare global {
|
|||||||
|
|
||||||
declare const MeSqlark: typeof MetonaSqlark;
|
declare const MeSqlark: typeof MetonaSqlark;
|
||||||
|
|
||||||
export { AriaEngine, AriaEngineConfig, ColumnDef, DatabaseConfig, DeleteStatement, DiskEngine, FieldType, HybridEngine, IStorageEngine, IndexedDBEngine, InsertStatement, MeSqlark, MemoryEngine, MetonaSqlark, OPFSBackend, OPFSEngine, SelectStatement, Statement, StorageMode, Table, TableSchema, UpdateStatement, VERSION, api, create, api as default, parse, parseAll, tokenize };
|
export { AriaEngine, AriaEngineConfig, ColumnDef, DatabaseConfig, DeleteStatement, DiskEngine, FieldType, HybridEngine, IStorageEngine, InsertStatement, KVStoreEngine, MeSqlark, MemoryEngine, MetonaSqlark, OPFSBackend, SelectStatement, Statement, StorageMode, Table, TableSchema, UpdateStatement, VERSION, api, create, api as default, parse, parseAll, tokenize };
|
||||||
|
|||||||
Vendored
+1293
-1401
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1293
-1402
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@metona-team/metona-sqlark",
|
"name": "@metona-team/metona-sqlark",
|
||||||
"version": "0.5.1",
|
"version": "0.6.0",
|
||||||
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "dist/metona-sqlark.cjs",
|
"main": "dist/metona-sqlark.cjs",
|
||||||
|
|||||||
+3
-3
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>⚡ 性能基准 — MetonaSqlark v0.5.1</title>
|
<title>⚡ 性能基准 — MetonaSqlark v0.6.0</title>
|
||||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||||
<style>
|
<style>
|
||||||
:root {
|
:root {
|
||||||
@@ -116,7 +116,7 @@ const SIZES_FAST = [1000];
|
|||||||
const ENGINES = [
|
const ENGINES = [
|
||||||
['memory', 'memory', 'memory'],
|
['memory', 'memory', 'memory'],
|
||||||
['aria-memory', 'aria', 'memory'],
|
['aria-memory', 'aria', 'memory'],
|
||||||
['aria-indexeddb', 'aria', 'indexeddb'],
|
['kvstore', 'disk', 'opfs'],
|
||||||
['aria-opfs', 'aria', 'opfs'],
|
['aria-opfs', 'aria', 'opfs'],
|
||||||
];
|
];
|
||||||
|
|
||||||
@@ -264,7 +264,7 @@ function renderResults(results) {
|
|||||||
const GROUP_LABEL = {
|
const GROUP_LABEL = {
|
||||||
'memory': '🚀 Memory 引擎',
|
'memory': '🚀 Memory 引擎',
|
||||||
'aria-memory': '🌲 Aria 引擎(内存后端)',
|
'aria-memory': '🌲 Aria 引擎(内存后端)',
|
||||||
'aria-indexeddb': '🌲 Aria 引擎(IndexedDB 后端)',
|
'kvstore': '💾 KVStore 引擎(disk 模式 · OPFS 介质)',
|
||||||
'aria-opfs': '🌲 Aria 引擎(OPFS · 4KB 页面化)',
|
'aria-opfs': '🌲 Aria 引擎(OPFS · 4KB 页面化)',
|
||||||
};
|
};
|
||||||
let lastMode = '';
|
let lastMode = '';
|
||||||
|
|||||||
+3
-3
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>🧪 在线演示 — MetonaSqlark v0.5.1</title>
|
<title>🧪 在线演示 — MetonaSqlark v0.6.0</title>
|
||||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||||
<style>
|
<style>
|
||||||
:root {
|
:root {
|
||||||
@@ -84,7 +84,7 @@
|
|||||||
<a href="demo.html" class="nav-active">演示</a>
|
<a href="demo.html" class="nav-active">演示</a>
|
||||||
<a href="benchmark.html">基准</a>
|
<a href="benchmark.html">基准</a>
|
||||||
</nav>
|
</nav>
|
||||||
<div class="status"><span class="dot" id="engine-dot"></span> <span id="engine-status">Memory</span> 模式 — v0.5.1</div>
|
<div class="status"><span class="dot" id="engine-dot"></span> <span id="engine-status">Memory</span> 模式 — v0.6.0</div>
|
||||||
<button class="btn btn-preset" onclick="switchEngine('memory')" id="btn-memory" style="margin:6px 4px 6px 0;padding:6px 12px;">⚡ Memory</button>
|
<button class="btn btn-preset" onclick="switchEngine('memory')" id="btn-memory" style="margin:6px 4px 6px 0;padding:6px 12px;">⚡ Memory</button>
|
||||||
<button class="btn btn-preset" onclick="switchEngine('aria')" id="btn-aria" style="margin:6px 0;padding:6px 12px;color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
<button class="btn btn-preset" onclick="switchEngine('aria')" id="btn-aria" style="margin:6px 0;padding:6px 12px;color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
||||||
</header>
|
</header>
|
||||||
@@ -92,7 +92,7 @@
|
|||||||
<div class="main">
|
<div class="main">
|
||||||
<div class="editor-panel">
|
<div class="editor-panel">
|
||||||
<div class="editor-area">
|
<div class="editor-area">
|
||||||
<textarea id="sql-input" placeholder="输入 SQL 语句... SELECT * FROM users; INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28); SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.5.1 在线演示
|
<textarea id="sql-input" placeholder="输入 SQL 语句... SELECT * FROM users; INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28); SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.6.0 在线演示
|
||||||
-- 已预置 users / orders / products 表数据
|
-- 已预置 users / orders / products 表数据
|
||||||
-- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护
|
-- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护
|
||||||
|
|
||||||
|
|||||||
+5
-4
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>📖 API 文档 — MetonaSqlark v0.5.1</title>
|
<title>📖 API 文档 — MetonaSqlark v0.6.0</title>
|
||||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||||
<style>
|
<style>
|
||||||
:root {
|
:root {
|
||||||
@@ -752,11 +752,11 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<table>
|
<table>
|
||||||
<tr><th>引擎</th><th>模式</th><th>持久化</th><th>索引</th><th>事务</th><th>适用场景</th></tr>
|
<tr><th>引擎</th><th>模式</th><th>持久化</th><th>索引</th><th>事务</th><th>适用场景</th></tr>
|
||||||
<tr><td><code>MemoryEngine</code></td><td>memory</td><td>❌</td><td>哈希</td><td>快照回滚</td><td>临时数据、缓存、测试</td></tr>
|
<tr><td><code>MemoryEngine</code></td><td>memory</td><td>❌</td><td>哈希</td><td>快照回滚</td><td>临时数据、缓存、测试</td></tr>
|
||||||
<tr><td><code>IndexedDBEngine</code></td><td>disk</td><td>✅ IDB</td><td>IDB 索引</td><td>延迟写入</td><td>通用持久化,兼容性最好</td></tr>
|
<tr><td><code>KVStoreEngine</code> 🆕</td><td>disk</td><td>✅ KVStore(OPFS)</td><td>哈希</td><td>原子日志 flush</td><td>标准持久化(v0.6.0 替代 IndexedDB)</td></tr>
|
||||||
<tr><td><code>OPFSEngine</code></td><td>disk</td><td>✅ OPFS</td><td>哈希</td><td>快照回滚</td><td>现代浏览器,文件级存储</td></tr>
|
|
||||||
<tr><td><code>HybridEngine</code></td><td>hybrid</td><td>✅ Write-Through</td><td>哈希</td><td>双引擎代理</td><td>生产推荐,读写均走内存</td></tr>
|
<tr><td><code>HybridEngine</code></td><td>hybrid</td><td>✅ Write-Through</td><td>哈希</td><td>双引擎代理</td><td>生产推荐,读写均走内存</td></tr>
|
||||||
<tr style="border-top:2px solid var(--primary);"><td><code style="color:#ec4899;font-weight:700;">AriaEngine</code></td><td>aria</td><td>✅ WAL + SSTable</td><td>LSM-Tree</td><td>MVCC 快照隔离</td><td>自研引擎:大表、高并发、需崩溃恢复</td></tr>
|
<tr style="border-top:2px solid var(--primary);"><td><code style="color:#ec4899;font-weight:700;">AriaEngine</code></td><td>aria</td><td>✅ WAL + SSTable</td><td>LSM-Tree</td><td>MVCC 快照隔离</td><td>自研引擎:大表、高并发、需崩溃恢复</td></tr>
|
||||||
</table>
|
</table>
|
||||||
|
<p>✅ <strong>v0.6.0: IndexedDB 已完全移除</strong> — disk 模式改用自研 KVStore 引擎(多 key 原子写 + 快照/日志崩溃恢复),旧库可经 <code>migrateFromIndexedDB()</code> 一键迁移。</p>
|
||||||
|
|
||||||
<h2 id="aria-engine">🌲 AriaEngine 自研存储引擎</h2>
|
<h2 id="aria-engine">🌲 AriaEngine 自研存储引擎</h2>
|
||||||
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。<br>
|
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。<br>
|
||||||
@@ -766,7 +766,8 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<strong>v0.4.3 关闭时序与后台任务加固</strong> — 后台 flush/compaction 不再使用 setTimeout 延迟(close 排空全部任务后才关闭存储,杜绝"backend 关闭后写存储/重开污染")· 后台失败在 `flush()`/`close()` 显式报告(`ARIA_BACKGROUND_ERROR`,不静默吞错)· 预加载等待链稳定(修复 compaction 竞态跳块丢数据)· 事务提交先落 WAL 再合并快照(崩溃一致)· OPFS 写操作串行队列 + close 等待。<br>
|
<strong>v0.4.3 关闭时序与后台任务加固</strong> — 后台 flush/compaction 不再使用 setTimeout 延迟(close 排空全部任务后才关闭存储,杜绝"backend 关闭后写存储/重开污染")· 后台失败在 `flush()`/`close()` 显式报告(`ARIA_BACKGROUND_ERROR`,不静默吞错)· 预加载等待链稳定(修复 compaction 竞态跳块丢数据)· 事务提交先落 WAL 再合并快照(崩溃一致)· OPFS 写操作串行队列 + close 等待。<br>
|
||||||
<strong>v0.4.4 SSTable 编码修复</strong> — 大段中文内容(如 300KB 笔记)写入 AriaEngine 不再崩溃:块大小估算改 UTF-8 字节精确计算(修复中文 3 字节 vs 1 码元导致的缓冲区低估越界)· 长度字段 u16 → u32(修复 >64KB value 截断)· 大 value 独立成块 · 格式 v2("SSTC")与 v1("SSTB")双格式兼容(旧库数据不丢)· 中文主键 / 大内容索引列同步支持。<br>
|
<strong>v0.4.4 SSTable 编码修复</strong> — 大段中文内容(如 300KB 笔记)写入 AriaEngine 不再崩溃:块大小估算改 UTF-8 字节精确计算(修复中文 3 字节 vs 1 码元导致的缓冲区低估越界)· 长度字段 u16 → u32(修复 >64KB value 截断)· 大 value 独立成块 · 格式 v2("SSTC")与 v1("SSTB")双格式兼容(旧库数据不丢)· 中文主键 / 大内容索引列同步支持。<br>
|
||||||
<strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e(7 用例)· DatabaseConfig.aria 配置透传。<br>
|
<strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e(7 用例)· DatabaseConfig.aria 配置透传。<br>
|
||||||
<strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)· 1009 测试 62 套件 · 87.3% 行覆盖率。</p>
|
<strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)。<br>
|
||||||
|
<strong>v0.6.0 完全移除 IndexedDB</strong> — 自研 KVStore 事务存储引擎(多 key 原子写 = 单日志记录原子追加 · 快照 checkpoint + 两阶段崩溃恢复 · CRC-32 自愈)· disk 模式切换 KVStoreEngine(替代 IndexedDBEngine + OPFSEngine)· 事务内 DDL / 外键级联 / 二级索引完整持久化 · migrateFromIndexedDB() 旧库一键迁移 · 10 万 key 压力验证 · e2e 崩溃注入(9 用例)· 1021 测试 64 套件 · 89.1% 行覆盖率。</p>
|
||||||
|
|
||||||
<h3>存储模式对比</h3>
|
<h3>存储模式对比</h3>
|
||||||
<table>
|
<table>
|
||||||
|
|||||||
+4
-4
@@ -153,7 +153,7 @@
|
|||||||
<!-- Hero -->
|
<!-- Hero -->
|
||||||
<section class="hero">
|
<section class="hero">
|
||||||
<div class="container">
|
<div class="container">
|
||||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.5.1 存储后端生产级硬化 — 1009测试 62套件 · 全库 AES-GCM 加密 · WAL 分片 · SSTable 页面化 · 多标签页锁 · 真实 Chromium e2e</div>
|
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.6.0 完全移除 IndexedDB — 1021测试 64套件 · 自研 KVStore 事务引擎 · 多 key 原子写 · 崩溃恢复 · 旧库一键迁移</div>
|
||||||
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
||||||
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
||||||
<div class="actions">
|
<div class="actions">
|
||||||
@@ -414,12 +414,12 @@ npm install @metona-team/metona-sqlark
|
|||||||
<p>MetonaSqlark 的核心指标</p>
|
<p>MetonaSqlark 的核心指标</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="stats">
|
<div class="stats">
|
||||||
<div class="stat-card"><div class="num">1009</div><div class="label">测试用例</div></div>
|
<div class="stat-card"><div class="num">1021</div><div class="label">测试用例</div></div>
|
||||||
<div class="stat-card"><div class="num">87.3%</div><div class="label">行覆盖率</div></div>
|
<div class="stat-card"><div class="num">89.1%</div><div class="label">行覆盖率</div></div>
|
||||||
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
||||||
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
||||||
<div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div>
|
<div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div>
|
||||||
<div class="stat-card"><div class="num">62</div><div class="label">测试套件</div></div>
|
<div class="stat-card"><div class="num">64</div><div class="label">测试套件</div></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|||||||
+4
-4
@@ -10,8 +10,8 @@
|
|||||||
/** 存储模式 */
|
/** 存储模式 */
|
||||||
export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
||||||
|
|
||||||
/** 磁盘引擎类型 */
|
/** 磁盘引擎类型(v0.6.0: IndexedDB 已移除,'memory' 供 aria 内存后端) */
|
||||||
export type DiskEngine = 'indexeddb' | 'opfs';
|
export type DiskEngine = 'opfs' | 'memory';
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// 字段类型
|
// 字段类型
|
||||||
@@ -100,7 +100,7 @@ export interface DatabaseConfig {
|
|||||||
export const DB_DEFAULTS: Readonly<Required<Omit<DatabaseConfig, 'plugins' | 'onReady' | 'onError' | 'aria'>>> & { aria: undefined } = Object.freeze({
|
export const DB_DEFAULTS: Readonly<Required<Omit<DatabaseConfig, 'plugins' | 'onReady' | 'onError' | 'aria'>>> & { aria: undefined } = Object.freeze({
|
||||||
name: 'metona-sqlark',
|
name: 'metona-sqlark',
|
||||||
mode: 'hybrid' as const,
|
mode: 'hybrid' as const,
|
||||||
diskEngine: 'indexeddb' as const,
|
diskEngine: 'opfs' as const,
|
||||||
version: 1,
|
version: 1,
|
||||||
maxRowsPerQuery: 0, // 0 = 不限制
|
maxRowsPerQuery: 0, // 0 = 不限制
|
||||||
debug: false,
|
debug: false,
|
||||||
@@ -214,4 +214,4 @@ export class DatabaseError extends Error {
|
|||||||
// 版本
|
// 版本
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export const VERSION = '0.5.1';
|
export const VERSION = '0.6.0';
|
||||||
|
|||||||
+5
-5
@@ -9,8 +9,7 @@ import type { IStorageEngine } from './engine/interface';
|
|||||||
import type { DatabaseConfig, ColumnDef } from './constants';
|
import type { DatabaseConfig, ColumnDef } from './constants';
|
||||||
import { DB_DEFAULTS, DatabaseError } from './constants';
|
import { DB_DEFAULTS, DatabaseError } from './constants';
|
||||||
import { MemoryEngine } from './engine/memory';
|
import { MemoryEngine } from './engine/memory';
|
||||||
import { IndexedDBEngine } from './engine/indexeddb';
|
import { KVStoreEngine } from './engine/kvstore_engine';
|
||||||
import { OPFSEngine } from './engine/opfs';
|
|
||||||
import { AriaEngine } from './engine/aria/index';
|
import { AriaEngine } from './engine/aria/index';
|
||||||
import { HybridEngine } from './hybrid/index';
|
import { HybridEngine } from './hybrid/index';
|
||||||
import { Table } from './table/table';
|
import { Table } from './table/table';
|
||||||
@@ -548,17 +547,18 @@ export class MetonaSqlark {
|
|||||||
|
|
||||||
private createEngine(): IStorageEngine {
|
private createEngine(): IStorageEngine {
|
||||||
const mode = this.mode;
|
const mode = this.mode;
|
||||||
const diskEngine = this.config.diskEngine ?? 'indexeddb';
|
const diskEngine = this.config.diskEngine ?? 'opfs';
|
||||||
|
|
||||||
switch (mode) {
|
switch (mode) {
|
||||||
case 'memory':
|
case 'memory':
|
||||||
return new MemoryEngine();
|
return new MemoryEngine();
|
||||||
case 'disk':
|
case 'disk':
|
||||||
return diskEngine === 'opfs' ? new OPFSEngine() : new IndexedDBEngine();
|
// v0.6.0: 自研 KVStoreEngine(完全移除 IndexedDB)
|
||||||
|
return new KVStoreEngine();
|
||||||
case 'aria':
|
case 'aria':
|
||||||
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
|
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
|
||||||
return new AriaEngine({
|
return new AriaEngine({
|
||||||
storageBackend: diskEngine === 'opfs' ? 'opfs' : 'indexeddb',
|
storageBackend: diskEngine === 'memory' ? 'memory' : 'opfs',
|
||||||
...(this.config.aria ?? {}),
|
...(this.config.aria ?? {}),
|
||||||
});
|
});
|
||||||
case 'hybrid':
|
case 'hybrid':
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ import { SegmentedWALStore } from './wal/segmented_store';
|
|||||||
import { DatabaseLock } from './locks';
|
import { DatabaseLock } from './locks';
|
||||||
import { WALRecordType, type WALRecord } from './types';
|
import { WALRecordType, type WALRecord } from './types';
|
||||||
import { CheckpointManager } from './wal/checkpoint';
|
import { CheckpointManager } from './wal/checkpoint';
|
||||||
import { IndexedDBBackend, MemoryBackend, type IStorageBackend } from './store/backend';
|
import { MemoryBackend, type IStorageBackend } from './store/backend';
|
||||||
import { OPFSBackend } from './store/opfs_backend';
|
import { OPFSBackend } from './store/opfs_backend';
|
||||||
import { EncryptedBackend } from './store/encrypted_backend';
|
import { EncryptedBackend } from './store/encrypted_backend';
|
||||||
import { PageSSTableStore } from './store/page_sstable_store';
|
import { PageSSTableStore } from './store/page_sstable_store';
|
||||||
@@ -117,8 +117,6 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
let baseBackend: IStorageBackend;
|
let baseBackend: IStorageBackend;
|
||||||
if (this.config.storageBackend === 'opfs') {
|
if (this.config.storageBackend === 'opfs') {
|
||||||
baseBackend = new OPFSBackend();
|
baseBackend = new OPFSBackend();
|
||||||
} else if (this.config.storageBackend === 'indexeddb') {
|
|
||||||
baseBackend = new IndexedDBBackend();
|
|
||||||
} else {
|
} else {
|
||||||
baseBackend = new MemoryBackend();
|
baseBackend = new MemoryBackend();
|
||||||
}
|
}
|
||||||
@@ -300,10 +298,10 @@ export class AriaEngine implements IStorageEngine {
|
|||||||
*/
|
*/
|
||||||
async repair(): Promise<void> {
|
async repair(): Promise<void> {
|
||||||
this.ensureOpen();
|
this.ensureOpen();
|
||||||
|
// v0.6.0-fix: 先清页面缓存再校验 — 缓存中的"完好页面"会掩盖磁盘损坏
|
||||||
|
await this.bufferPool.clear();
|
||||||
// 1. 校验全部 SSTable,移除残缺项(打开时已做一次,此处兜底运行期损坏)
|
// 1. 校验全部 SSTable,移除残缺项(打开时已做一次,此处兜底运行期损坏)
|
||||||
const removed = await this.lsm.validateAll();
|
const removed = await this.lsm.validateAll();
|
||||||
// v0.4.5: 清空页面缓存(损坏数据可能驻留 BufferPool,重新加载)
|
|
||||||
await this.bufferPool.clear();
|
|
||||||
// 2. 将 WAL 残留数据落盘并截断,避免无限重放(含空洞截断落地)
|
// 2. 将 WAL 残留数据落盘并截断,避免无限重放(含空洞截断落地)
|
||||||
await this.lsm.flush();
|
await this.lsm.flush();
|
||||||
await this.wal.checkpoint();
|
await this.wal.checkpoint();
|
||||||
|
|||||||
@@ -683,12 +683,14 @@ export class LSM {
|
|||||||
private async dropInvalidSSTable(meta: SSTableMeta): Promise<void> {
|
private async dropInvalidSSTable(meta: SSTableMeta): Promise<void> {
|
||||||
// eslint-disable-next-line no-console
|
// eslint-disable-next-line no-console
|
||||||
console.warn(`[AriaEngine LSM] Skipping corrupted SSTable id=${meta.id} (level=${meta.level})`);
|
console.warn(`[AriaEngine LSM] Skipping corrupted SSTable id=${meta.id} (level=${meta.level})`);
|
||||||
try {
|
// v0.6.0-fix: 先删数据文件再删 meta — 页面化存储的 delete 依赖 meta.pageIds
|
||||||
await this.sstableStore.deleteMeta(meta.id);
|
// 定位页面文件;先删 meta 会丢失 pageIds 导致孤儿页面残留
|
||||||
} catch { /* 清理失败不阻塞打开 */ }
|
|
||||||
try {
|
try {
|
||||||
await this.sstableStore.delete(meta.id);
|
await this.sstableStore.delete(meta.id);
|
||||||
} catch { /* 清理失败不阻塞打开 */ }
|
} catch { /* 清理失败不阻塞打开 */ }
|
||||||
|
try {
|
||||||
|
await this.sstableStore.deleteMeta(meta.id);
|
||||||
|
} catch { /* 清理失败不阻塞打开 */ }
|
||||||
}
|
}
|
||||||
|
|
||||||
private unwrapTombstone(value: Record<string, unknown> | null): Record<string, unknown> | null {
|
private unwrapTombstone(value: Record<string, unknown> | null): Record<string, unknown> | null {
|
||||||
|
|||||||
@@ -49,139 +49,6 @@ export interface IStorageBackend {
|
|||||||
clear(): Promise<void>;
|
clear(): Promise<void>;
|
||||||
}
|
}
|
||||||
|
|
||||||
// =======================================================================
|
|
||||||
// IndexedDB Backend
|
|
||||||
// =======================================================================
|
|
||||||
|
|
||||||
export class IndexedDBBackend implements IStorageBackend {
|
|
||||||
private db: IDBDatabase | null = null;
|
|
||||||
private dbName = '';
|
|
||||||
private storeName = 'data';
|
|
||||||
|
|
||||||
async open(name: string): Promise<void> {
|
|
||||||
this.dbName = `aria-${name}`;
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const request = indexedDB.open(this.dbName, 1);
|
|
||||||
request.onupgradeneeded = () => {
|
|
||||||
const db = request.result;
|
|
||||||
if (!db.objectStoreNames.contains(this.storeName)) {
|
|
||||||
db.createObjectStore(this.storeName);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => {
|
|
||||||
this.db = request.result;
|
|
||||||
resolve();
|
|
||||||
};
|
|
||||||
request.onerror = () => reject(new DatabaseError('Failed to open AriaEngine IndexedDB', 'ARIA_IDB_OPEN_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async close(): Promise<void> {
|
|
||||||
if (this.db) {
|
|
||||||
this.db.close();
|
|
||||||
this.db = null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
isOpen(): boolean {
|
|
||||||
return this.db !== null;
|
|
||||||
}
|
|
||||||
|
|
||||||
async read(key: string): Promise<ArrayBuffer | null> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readonly');
|
|
||||||
const req = tx.objectStore(this.storeName).get(key);
|
|
||||||
req.onsuccess = () => resolve(req.result ?? null);
|
|
||||||
req.onerror = () => reject(new DatabaseError('Failed to read from AriaEngine store', 'ARIA_READ_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async write(key: string, data: ArrayBuffer): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readwrite');
|
|
||||||
tx.objectStore(this.storeName).put(data, key);
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError('Failed to write to AriaEngine store', 'ARIA_WRITE_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 批量原子写入 — 单个 IDB 事务内写入多个 key。
|
|
||||||
* 中断时事务整体回滚,WAL 记录与 count 计数不会出现"记录在、计数丢"或反之的半写状态。
|
|
||||||
*/
|
|
||||||
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
|
||||||
const keys = Object.keys(entries);
|
|
||||||
if (keys.length === 0) return;
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readwrite');
|
|
||||||
const store = tx.objectStore(this.storeName);
|
|
||||||
for (const key of keys) {
|
|
||||||
store.put(entries[key], key);
|
|
||||||
}
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError('Failed to batch write to AriaEngine store', 'ARIA_WRITE_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async delete(key: string): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readwrite');
|
|
||||||
tx.objectStore(this.storeName).delete(key);
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError('Failed to delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/** v0.4.2-fix: 批量原子删除 — 单个 IDB 事务内删除多个 key */
|
|
||||||
async deleteMany(keys: string[]): Promise<void> {
|
|
||||||
if (keys.length === 0) return;
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readwrite');
|
|
||||||
const store = tx.objectStore(this.storeName);
|
|
||||||
for (const key of keys) {
|
|
||||||
store.delete(key);
|
|
||||||
}
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError('Failed to batch delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async listKeys(): Promise<string[]> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readonly');
|
|
||||||
const req = tx.objectStore(this.storeName).getAllKeys();
|
|
||||||
req.onsuccess = () => resolve((req.result ?? []) as string[]);
|
|
||||||
req.onerror = () => reject(new DatabaseError('Failed to list keys', 'ARIA_LIST_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async exists(key: string): Promise<boolean> {
|
|
||||||
const result = await this.read(key);
|
|
||||||
return result !== null;
|
|
||||||
}
|
|
||||||
|
|
||||||
async clear(): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(this.storeName, 'readwrite');
|
|
||||||
tx.objectStore(this.storeName).clear();
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError('Failed to clear AriaEngine store', 'ARIA_CLEAR_ERROR'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private ensureDB(): IDBDatabase {
|
|
||||||
if (!this.db) throw new DatabaseError('AriaEngine storage not opened', 'ARIA_DB_NOT_OPEN');
|
|
||||||
return this.db;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// Memory Backend(回退 / 测试用)
|
// Memory Backend(回退 / 测试用)
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|||||||
@@ -207,7 +207,7 @@ export interface AriaEngineConfig {
|
|||||||
/** 是否启用页面压缩(默认 false) */
|
/** 是否启用页面压缩(默认 false) */
|
||||||
compression?: boolean;
|
compression?: boolean;
|
||||||
/** 存储后端 */
|
/** 存储后端 */
|
||||||
storageBackend?: 'indexeddb' | 'opfs' | 'memory';
|
storageBackend?: 'opfs' | 'memory';
|
||||||
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB) */
|
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB) */
|
||||||
walSizeThreshold?: number;
|
walSizeThreshold?: number;
|
||||||
/** 最大内存预算(MB,默认 64) */
|
/** 最大内存预算(MB,默认 64) */
|
||||||
@@ -241,7 +241,7 @@ export const DEFAULT_ARIA_CONFIG: Required<Omit<AriaEngineConfig, 'encryption' |
|
|||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
checkpointInterval: 1000,
|
checkpointInterval: 1000,
|
||||||
compression: false,
|
compression: false,
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
walSizeThreshold: 16 * 1024 * 1024, // 16MB
|
walSizeThreshold: 16 * 1024 * 1024, // 16MB
|
||||||
maxMemoryMB: 64,
|
maxMemoryMB: 64,
|
||||||
encryption: undefined,
|
encryption: undefined,
|
||||||
|
|||||||
+1
-2
@@ -5,7 +5,6 @@
|
|||||||
|
|
||||||
export type { IStorageEngine } from './interface';
|
export type { IStorageEngine } from './interface';
|
||||||
export { MemoryEngine } from './memory';
|
export { MemoryEngine } from './memory';
|
||||||
export { IndexedDBEngine } from './indexeddb';
|
export { KVStoreEngine } from './kvstore_engine';
|
||||||
export { OPFSEngine } from './opfs';
|
|
||||||
export { AriaEngine } from './aria/index';
|
export { AriaEngine } from './aria/index';
|
||||||
export type { AriaEngineConfig } from './aria/types';
|
export type { AriaEngineConfig } from './aria/types';
|
||||||
|
|||||||
@@ -1,840 +0,0 @@
|
|||||||
/**
|
|
||||||
* metona-sqlark IndexedDB Engine — 基于 IndexedDB 的持久化存储引擎
|
|
||||||
* @module engine/indexeddb
|
|
||||||
*
|
|
||||||
* v0.1.13: 支持事务 — beginTransaction 延迟 IDB 写入,commit 批量刷盘,rollback 恢复快照。
|
|
||||||
*/
|
|
||||||
|
|
||||||
import type { IStorageEngine } from './interface';
|
|
||||||
import type { QueryPlan, TableSchema } from '../constants';
|
|
||||||
import { DatabaseError } from '../constants';
|
|
||||||
import { MemoryEngine } from './memory';
|
|
||||||
import { matchWhere, applyOrderBy, projectColumns } from '../query/where-matcher';
|
|
||||||
|
|
||||||
export class IndexedDBEngine implements IStorageEngine {
|
|
||||||
readonly name = 'indexeddb';
|
|
||||||
|
|
||||||
private db: IDBDatabase | null = null;
|
|
||||||
private dbName = '';
|
|
||||||
private version = 1;
|
|
||||||
private memoryCache: MemoryEngine = new MemoryEngine();
|
|
||||||
|
|
||||||
// ---- 事务状态 ----
|
|
||||||
private txActive = false;
|
|
||||||
|
|
||||||
async open(dbName: string, version: number): Promise<void> {
|
|
||||||
// v0.4.2-fix (P0-3): version < 1 归一化为 1(indexedDB.open(name, 0) 抛原生 TypeError)
|
|
||||||
const normalizedVersion = version >= 1 ? Math.floor(version) : 1;
|
|
||||||
this.dbName = dbName;
|
|
||||||
this.version = normalizedVersion;
|
|
||||||
await this.memoryCache.open(dbName, normalizedVersion);
|
|
||||||
|
|
||||||
// v0.4.2-fix (P0-2/P2-8): 版本自适应打开 + blocked 重试
|
|
||||||
this.db = await this.openDatabaseWithRetry(dbName, normalizedVersion);
|
|
||||||
this.setupVersionChangeHandler();
|
|
||||||
|
|
||||||
// v0.4.2-fix (P2-7): 确保 schema/meta 持久化 store 存在(新库或旧库升级时创建),
|
|
||||||
// 否则迁移版本等库内元数据无处落盘
|
|
||||||
await this.ensureSchemaStore();
|
|
||||||
|
|
||||||
try {
|
|
||||||
// v0.3.2: reopen 后从 IDB 重建 schema(schema 此前只存内存缓存,重开连接即丢失)
|
|
||||||
await this.rebuildSchemaFromIDB();
|
|
||||||
} catch (error) {
|
|
||||||
throw new DatabaseError(`Failed to restore schema for "${dbName}"`, 'IDB_SCHEMA_RESTORE_ERROR', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** v0.4.2-fix: 多标签页冲突处理 — 其他标签页升级版本时自动关闭当前连接 */
|
|
||||||
private setupVersionChangeHandler(): void {
|
|
||||||
if (!this.db) return;
|
|
||||||
this.db.onversionchange = () => {
|
|
||||||
if (this.db) {
|
|
||||||
this.db.close();
|
|
||||||
this.db = null;
|
|
||||||
// eslint-disable-next-line no-console
|
|
||||||
console.warn(`[metona-sqlark] Database "${this.dbName}" was upgraded in another tab. Connection closed. Please re-open.`);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 打开 IndexedDB 连接。
|
|
||||||
* - P0-2: 请求版本低于库实际版本(VersionError)时,先无版本参数探测库当前版本,
|
|
||||||
* 再以实际版本重开(建表每张表版本号 +1,config.version 会过期)
|
|
||||||
* - P2-8: onblocked 为瞬时状态(另一连接短暂持有),等待后重试多次,超时才抛 IDB_BLOCKED
|
|
||||||
*/
|
|
||||||
private async openDatabaseWithRetry(dbName: string, requestedVersion: number): Promise<IDBDatabase> {
|
|
||||||
const BLOCKED_RETRIES = 10;
|
|
||||||
let effectiveVersion = requestedVersion;
|
|
||||||
for (let attempt = 0; attempt < BLOCKED_RETRIES; attempt++) {
|
|
||||||
try {
|
|
||||||
return await this.openRequest(dbName, effectiveVersion, 200 + attempt * 150);
|
|
||||||
} catch (error) {
|
|
||||||
const err = error as { name?: string };
|
|
||||||
if (err && err.name === 'VersionError') {
|
|
||||||
const currentVersion = await this.resolveCurrentVersion(dbName);
|
|
||||||
if (currentVersion >= 1 && currentVersion !== effectiveVersion) {
|
|
||||||
effectiveVersion = currentVersion;
|
|
||||||
this.version = currentVersion;
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
throw new DatabaseError(
|
|
||||||
`Failed to open IndexedDB "${dbName}": version mismatch`,
|
|
||||||
'IDB_VERSION_ERROR',
|
|
||||||
error,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if (err && err.name === 'BlockedError') {
|
|
||||||
// 另一连接短暂持有 → 等待后重试
|
|
||||||
await new Promise((resolve) => setTimeout(resolve, 50 * (attempt + 1)));
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
throw new DatabaseError(`Failed to open IndexedDB "${dbName}"`, 'IDB_OPEN_ERROR', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
throw new DatabaseError(`IndexedDB "${dbName}" is blocked`, 'IDB_BLOCKED');
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 发起一次 indexedDB.open 请求(success/error/blocked 三态收敛)。
|
|
||||||
* onblocked 不立即失败:阻塞解除后 success 仍会触发,仅超时兜底判失败,
|
|
||||||
* 避免"拒绝后连接迟到成功"泄漏未关闭的数据库连接。
|
|
||||||
*/
|
|
||||||
private openRequest(dbName: string, version: number, timeoutMs: number): Promise<IDBDatabase> {
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const request = indexedDB.open(dbName, version);
|
|
||||||
let settled = false;
|
|
||||||
const timeout = setTimeout(() => {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
// 兼容无 DOMException 构造环境
|
|
||||||
const blockedError = typeof DOMException !== 'undefined'
|
|
||||||
? new DOMException('IndexedDB open is blocked', 'BlockedError')
|
|
||||||
: Object.assign(new Error('IndexedDB open is blocked'), { name: 'BlockedError' });
|
|
||||||
reject(blockedError);
|
|
||||||
}, timeoutMs);
|
|
||||||
request.onsuccess = () => {
|
|
||||||
if (settled) {
|
|
||||||
// 超时判失败后连接迟到成功:立即关闭,避免阻塞后续版本升级
|
|
||||||
request.result.close();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
settled = true;
|
|
||||||
clearTimeout(timeout);
|
|
||||||
resolve(request.result);
|
|
||||||
};
|
|
||||||
request.onerror = () => {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
clearTimeout(timeout);
|
|
||||||
reject(request.error ?? new Error('Unknown IndexedDB open error'));
|
|
||||||
};
|
|
||||||
request.onblocked = () => {
|
|
||||||
// 保持等待,不拒绝(阻塞解除后 success 会触发;超时由 timer 兜底)
|
|
||||||
};
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 无版本参数打开库,解析其当前实际版本号(随后立即关闭) */
|
|
||||||
private resolveCurrentVersion(dbName: string): Promise<number> {
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const request = indexedDB.open(dbName);
|
|
||||||
request.onsuccess = () => {
|
|
||||||
const actualVersion = request.result.version;
|
|
||||||
request.result.close();
|
|
||||||
resolve(actualVersion);
|
|
||||||
};
|
|
||||||
request.onerror = () => {
|
|
||||||
reject(request.error ?? new Error('Failed to resolve IndexedDB version'));
|
|
||||||
};
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix (P2-7): 确保 __metona_schema store 存在。
|
|
||||||
* 新库(或版本升级前创建的旧库)没有该 store 时,通过一次版本升级创建,
|
|
||||||
* 使 getMeta/setMeta(迁移版本持久化)始终可用。
|
|
||||||
*/
|
|
||||||
private async ensureSchemaStore(): Promise<void> {
|
|
||||||
if (!this.db) return;
|
|
||||||
if (this.db.objectStoreNames.contains('__metona_schema')) return;
|
|
||||||
const newVersion = this.db.version + 1;
|
|
||||||
this.db.close();
|
|
||||||
this.db = await new Promise<IDBDatabase>((resolve, reject) => {
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = () => {
|
|
||||||
const idb = request.result;
|
|
||||||
if (!idb.objectStoreNames.contains('__metona_schema')) {
|
|
||||||
idb.createObjectStore('__metona_schema', { keyPath: 'name' });
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => {
|
|
||||||
this.db = request.result;
|
|
||||||
this.setupVersionChangeHandler();
|
|
||||||
resolve(request.result);
|
|
||||||
};
|
|
||||||
request.onerror = () => reject(
|
|
||||||
new DatabaseError('Failed to create schema store', 'IDB_UPGRADE_ERROR', request.error),
|
|
||||||
);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 从 IDB 恢复内存 schema:
|
|
||||||
* 1. 优先读取持久化的 schema 记录('__metona_schema' store,v0.3.2)
|
|
||||||
* 2. 旧数据回退:从 objectStore 主键 / 索引 / 样例数据推断
|
|
||||||
*/
|
|
||||||
private async rebuildSchemaFromIDB(): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
|
|
||||||
// 1. 持久化 schema
|
|
||||||
if (db.objectStoreNames.contains('__metona_schema')) {
|
|
||||||
const records: { name: string; schema: string }[] = await new Promise((resolve, reject) => {
|
|
||||||
const req = db.transaction('__metona_schema', 'readonly').objectStore('__metona_schema').getAll();
|
|
||||||
req.onsuccess = () => resolve((req.result ?? []) as { name: string; schema: string }[]);
|
|
||||||
req.onerror = () => reject(req.error);
|
|
||||||
});
|
|
||||||
for (const rec of records) {
|
|
||||||
try {
|
|
||||||
const schema = JSON.parse(rec.schema) as TableSchema;
|
|
||||||
if (!(await this.memoryCache.getTableSchema(schema.name))) {
|
|
||||||
await this.memoryCache.createTable(schema);
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
// 损坏的 schema 记录忽略
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// 2. 回退:无持久化 schema 的表从 IDB 结构推断
|
|
||||||
const storeNames = Array.from(db.objectStoreNames).filter((n) => n !== '__metona_schema');
|
|
||||||
for (const tableName of storeNames) {
|
|
||||||
// 已有 schema(持久化恢复或连续 open)则跳过
|
|
||||||
const existing = await this.memoryCache.getTableSchema(tableName);
|
|
||||||
if (existing) continue;
|
|
||||||
|
|
||||||
const columns: Record<string, import('../constants').ColumnDef> = {};
|
|
||||||
const tx = db.transaction(tableName, 'readonly');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
|
|
||||||
// 主键列
|
|
||||||
const pk = store.keyPath as string;
|
|
||||||
columns[pk] = { type: 'string', primaryKey: true };
|
|
||||||
|
|
||||||
// 索引列(idx_ 前缀约定)
|
|
||||||
for (const idxName of Array.from(store.indexNames)) {
|
|
||||||
if (idxName.startsWith('idx_')) {
|
|
||||||
const col = idxName.slice(4);
|
|
||||||
if (!columns[col]) {
|
|
||||||
columns[col] = { type: 'string', index: true };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// 从样例数据推断其余列的类型
|
|
||||||
const rows: Record<string, unknown>[] = await new Promise((resolve, reject) => {
|
|
||||||
const req = store.getAll();
|
|
||||||
req.onsuccess = () => resolve((req.result ?? []) as Record<string, unknown>[]);
|
|
||||||
req.onerror = () => reject(req.error);
|
|
||||||
});
|
|
||||||
if (rows.length > 0) {
|
|
||||||
for (const [key, value] of Object.entries(rows[0])) {
|
|
||||||
if (!columns[key]) {
|
|
||||||
columns[key] = { type: inferFieldType(value) };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
await this.memoryCache.createTable({ name: tableName, columns });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async close(): Promise<void> {
|
|
||||||
// v0.4.3-fix: 活跃事务时先回滚(避免 commit 对已关闭连接报错)
|
|
||||||
if (this.txActive) {
|
|
||||||
try {
|
|
||||||
await this.rollbackTransaction();
|
|
||||||
} catch { /* 回滚失败不阻塞关闭 */ }
|
|
||||||
}
|
|
||||||
if (this.db) {
|
|
||||||
this.db.onversionchange = null; // 清理监听器
|
|
||||||
this.db.close();
|
|
||||||
this.db = null;
|
|
||||||
}
|
|
||||||
await this.memoryCache.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 自愈 — 从磁盘重建内存 schema 与数据(schema 丢失/内存不一致时调用)。
|
|
||||||
* 无删库需求即可恢复可用的库。
|
|
||||||
*/
|
|
||||||
async repair(): Promise<void> {
|
|
||||||
if (!this.db) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
|
|
||||||
await this.memoryCache.close();
|
|
||||||
await this.memoryCache.open(this.dbName, this.version);
|
|
||||||
await this.rebuildSchemaFromIDB();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 清空全部数据与表结构(含持久化 schema 记录),保留库本身。
|
|
||||||
* 单个版本升级事务内原子完成。
|
|
||||||
*/
|
|
||||||
async clearAll(): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
// 先清内存缓存
|
|
||||||
const tableNames = await this.memoryCache.getTableNames();
|
|
||||||
for (const name of tableNames) {
|
|
||||||
await this.memoryCache.dropTable(name);
|
|
||||||
}
|
|
||||||
// 重置 IDB:删除所有表 store + 清空 schema/meta store
|
|
||||||
const newVersion = db.version + 1;
|
|
||||||
db.close();
|
|
||||||
await new Promise<void>((resolve, reject) => {
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = (event) => {
|
|
||||||
const idb = (event.target as IDBOpenDBRequest).result;
|
|
||||||
const toDelete = Array.from(idb.objectStoreNames).filter((n) => n !== '__metona_schema');
|
|
||||||
for (const name of toDelete) {
|
|
||||||
idb.deleteObjectStore(name);
|
|
||||||
}
|
|
||||||
if (!idb.objectStoreNames.contains('__metona_schema')) {
|
|
||||||
idb.createObjectStore('__metona_schema', { keyPath: 'name' });
|
|
||||||
} else {
|
|
||||||
// 保留 store 但清空内容(含持久化 schema 与 meta 记录)
|
|
||||||
const tx = (event.target as IDBOpenDBRequest).transaction!;
|
|
||||||
tx.objectStore('__metona_schema').clear();
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => {
|
|
||||||
this.db = request.result;
|
|
||||||
this.setupVersionChangeHandler();
|
|
||||||
resolve();
|
|
||||||
};
|
|
||||||
request.onerror = () => reject(new DatabaseError('Failed to clear database', 'IDB_CLEAR_ERROR', request.error));
|
|
||||||
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${this.dbName}" is blocked`, 'IDB_BLOCKED'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 库内元数据(v0.4.2-fix:迁移版本持久化用,复用 __metona_schema store) ----
|
|
||||||
|
|
||||||
async getMeta(key: string): Promise<string | null> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
if (!db.objectStoreNames.contains('__metona_schema')) return null;
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const req = db.transaction('__metona_schema', 'readonly')
|
|
||||||
.objectStore('__metona_schema').get(`__meta:${key}`);
|
|
||||||
req.onsuccess = () => {
|
|
||||||
const rec = req.result as { schema?: unknown } | undefined;
|
|
||||||
resolve(rec && typeof rec.schema === 'string' ? rec.schema : null);
|
|
||||||
};
|
|
||||||
req.onerror = () => reject(req.error);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async setMeta(key: string, value: string): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
if (!db.objectStoreNames.contains('__metona_schema')) return;
|
|
||||||
await new Promise<void>((resolve, reject) => {
|
|
||||||
const tx = db.transaction('__metona_schema', 'readwrite');
|
|
||||||
tx.objectStore('__metona_schema').put({ name: `__meta:${key}`, schema: value });
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Failed to persist meta "${key}"`, 'IDB_META_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
isOpen(): boolean { return this.db !== null; }
|
|
||||||
|
|
||||||
// ---- 表管理 ----
|
|
||||||
async createTable(schema: TableSchema): Promise<void> {
|
|
||||||
await this.memoryCache.createTable(schema);
|
|
||||||
if (this.txActive) return; // 事务中延迟 IDB 操作
|
|
||||||
await this.idbCreateTable(schema);
|
|
||||||
}
|
|
||||||
|
|
||||||
async dropTable(tableName: string): Promise<void> {
|
|
||||||
await this.memoryCache.dropTable(tableName);
|
|
||||||
if (this.txActive) return;
|
|
||||||
await this.idbDropTable(tableName);
|
|
||||||
}
|
|
||||||
|
|
||||||
async hasTable(tableName: string): Promise<boolean> { return this.ensureDB().objectStoreNames.contains(tableName); }
|
|
||||||
async getTableNames(): Promise<string[]> { return Array.from(this.ensureDB().objectStoreNames); }
|
|
||||||
async getTableSchema(tableName: string): Promise<TableSchema | null> { return this.memoryCache.getTableSchema(tableName); }
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.2-fix: 引擎级 ALTER TABLE — schema 持久化到 __metona_schema store,
|
|
||||||
* 重启后 ALTER 不丢失(此前通用路径只改内存引用,重启回退;DROP 的行数据也没真正删)。
|
|
||||||
*/
|
|
||||||
async alterTable(
|
|
||||||
tableName: string,
|
|
||||||
action: 'ADD' | 'DROP',
|
|
||||||
column: import('../constants').ColumnDef & { name: string },
|
|
||||||
): Promise<void> {
|
|
||||||
await this.memoryCache.alterTable(tableName, action, column);
|
|
||||||
if (this.txActive) return; // 事务中:commit 时统一 flushToIDB 同步
|
|
||||||
const schema = await this.memoryCache.getTableSchema(tableName);
|
|
||||||
if (schema) {
|
|
||||||
await this.persistSchema(schema);
|
|
||||||
}
|
|
||||||
if (action === 'DROP') {
|
|
||||||
// 重写 IDB 存储行:移除该列键(store.put 经 keyPath 自动覆盖原行)
|
|
||||||
const db = this.ensureDB();
|
|
||||||
const rows = await this.idbFind(tableName, { table: tableName });
|
|
||||||
const rewritten = rows.map((row) => {
|
|
||||||
if (column.name in row) {
|
|
||||||
const copy = { ...row };
|
|
||||||
delete (copy as Record<string, unknown>)[column.name];
|
|
||||||
return copy;
|
|
||||||
}
|
|
||||||
return row;
|
|
||||||
});
|
|
||||||
if (rewritten.length > 0) {
|
|
||||||
await new Promise<void>((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
for (const row of rewritten) {
|
|
||||||
store.put(row);
|
|
||||||
}
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Failed to rewrite "${tableName}" after DROP COLUMN`, 'IDB_TX_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- CRUD ----
|
|
||||||
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
|
||||||
const pks = await this.memoryCache.insert(tableName, rows);
|
|
||||||
if (this.txActive) return pks; // 事务中延迟写入
|
|
||||||
await this.idbInsert(tableName, rows);
|
|
||||||
return pks;
|
|
||||||
}
|
|
||||||
|
|
||||||
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
|
|
||||||
// 事务中从内存缓存读取(保证读到未提交的变更),否则走 IDB
|
|
||||||
if (this.txActive) return this.memoryCache.find(tableName, query);
|
|
||||||
return this.idbFind(tableName, query);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** v0.4.0: 流式查询 — IDB 批量读入后逐行回调(保持接口一致性) */
|
|
||||||
async findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number> {
|
|
||||||
if (this.txActive) {
|
|
||||||
return this.memoryCache.findStream(tableName, query, onRow);
|
|
||||||
}
|
|
||||||
const rows = await this.idbFind(tableName, { ...query, orderBy: undefined, limit: undefined, offset: undefined });
|
|
||||||
let count = 0;
|
|
||||||
for (const row of rows) {
|
|
||||||
onRow(row);
|
|
||||||
count++;
|
|
||||||
}
|
|
||||||
return count;
|
|
||||||
}
|
|
||||||
|
|
||||||
async update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number> {
|
|
||||||
const count = await this.memoryCache.update(tableName, query, updates);
|
|
||||||
if (this.txActive) return count;
|
|
||||||
await this.idbUpdate(tableName, query, updates);
|
|
||||||
return count;
|
|
||||||
}
|
|
||||||
|
|
||||||
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
|
||||||
const count = await this.memoryCache.delete(tableName, query);
|
|
||||||
if (this.txActive) return count;
|
|
||||||
await this.idbDelete(tableName, query);
|
|
||||||
return count;
|
|
||||||
}
|
|
||||||
|
|
||||||
async count(tableName: string, query?: QueryPlan): Promise<number> {
|
|
||||||
if (this.txActive) return this.memoryCache.count(tableName, query);
|
|
||||||
const results = await this.idbFind(tableName, { table: tableName, where: query?.where ?? {} });
|
|
||||||
return results.length;
|
|
||||||
}
|
|
||||||
|
|
||||||
async clear(tableName: string): Promise<void> {
|
|
||||||
await this.memoryCache.clear(tableName);
|
|
||||||
if (this.txActive) return;
|
|
||||||
await this.idbClear(tableName);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 动态索引(v0.3.0):通过版本升级创建/删除 IDB 索引 ----
|
|
||||||
|
|
||||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
|
||||||
await this.memoryCache.createIndex(tableName, column, unique);
|
|
||||||
if (this.txActive) return;
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const newVersion = db.version + 1; db.close();
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = (event) => {
|
|
||||||
const idb = (event.target as IDBOpenDBRequest).result;
|
|
||||||
const tx = idb.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
if (!store.indexNames.contains(`idx_${column}`)) {
|
|
||||||
store.createIndex(`idx_${column}`, column, { unique: unique ?? false });
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => { this.db = request.result; resolve(); };
|
|
||||||
request.onerror = () => reject(new DatabaseError(`Failed to create index "${tableName}.${column}"`, 'IDB_UPGRADE_ERROR', request.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
|
|
||||||
await this.memoryCache.dropIndex(tableName, column);
|
|
||||||
if (this.txActive) return;
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const newVersion = db.version + 1; db.close();
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = (event) => {
|
|
||||||
const idb = (event.target as IDBOpenDBRequest).result;
|
|
||||||
const tx = idb.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
if (store.indexNames.contains(`idx_${column}`)) {
|
|
||||||
store.deleteIndex(`idx_${column}`);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => { this.db = request.result; resolve(); };
|
|
||||||
request.onerror = () => reject(new DatabaseError(`Failed to drop index "${tableName}.${column}"`, 'IDB_UPGRADE_ERROR', request.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 事务 ----
|
|
||||||
|
|
||||||
async beginTransaction(): Promise<void> {
|
|
||||||
if (this.txActive) throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE');
|
|
||||||
this.txActive = true;
|
|
||||||
// 保存内存快照到 MemoryEngine 内部的 beginTransaction
|
|
||||||
await this.memoryCache.beginTransaction();
|
|
||||||
}
|
|
||||||
|
|
||||||
async commitTransaction(): Promise<void> {
|
|
||||||
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
|
||||||
// v0.4.2-fix: 先刷盘后提交内存快照 — 此前先 memoryCache.commitTransaction()
|
|
||||||
// 再 flushToIDB,flush 失败时 snapshot 已丢,回滚报 TX_NONE 且内存数据已确认
|
|
||||||
await this.flushToIDB();
|
|
||||||
await this.memoryCache.commitTransaction();
|
|
||||||
this.txActive = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
async rollbackTransaction(): Promise<void> {
|
|
||||||
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
|
||||||
// 恢复内存层到快照状态
|
|
||||||
await this.memoryCache.rollbackTransaction();
|
|
||||||
this.txActive = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- IDB 原生操作 ----
|
|
||||||
|
|
||||||
private async idbCreateTable(schema: TableSchema): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const newVersion = db.version + 1; db.close();
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = (event) => {
|
|
||||||
const db = (event.target as IDBOpenDBRequest).result;
|
|
||||||
const pkColumn = Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0];
|
|
||||||
const store = db.createObjectStore(schema.name, { keyPath: pkColumn });
|
|
||||||
for (const [colName, colDef] of Object.entries(schema.columns)) {
|
|
||||||
if (colDef.index && colName !== pkColumn) {
|
|
||||||
store.createIndex(`idx_${colName}`, colName, { unique: colDef.unique ?? false });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// v0.3.2: schema 持久化 store(记录在升级完成后的 onsuccess 写入)
|
|
||||||
if (!db.objectStoreNames.contains('__metona_schema')) {
|
|
||||||
db.createObjectStore('__metona_schema', { keyPath: 'name' });
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => {
|
|
||||||
this.db = request.result;
|
|
||||||
// v0.3.2: 升级完成后持久化 schema(upgrade 事务内异步写会失败)
|
|
||||||
const schemaTx = this.db.transaction('__metona_schema', 'readwrite');
|
|
||||||
schemaTx.objectStore('__metona_schema').put({ name: schema.name, schema: JSON.stringify(schema) });
|
|
||||||
schemaTx.oncomplete = () => resolve();
|
|
||||||
schemaTx.onerror = () => reject(new DatabaseError(`Failed to persist schema for "${schema.name}"`, 'IDB_SCHEMA_ERROR', schemaTx.error));
|
|
||||||
};
|
|
||||||
request.onerror = () => reject(new DatabaseError(`Failed to create table "${schema.name}"`, 'IDB_UPGRADE_ERROR', request.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private async idbDropTable(tableName: string): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const newVersion = db.version + 1; db.close();
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = (event) => {
|
|
||||||
const db = (event.target as IDBOpenDBRequest).result;
|
|
||||||
if (db.objectStoreNames.contains(tableName)) db.deleteObjectStore(tableName);
|
|
||||||
};
|
|
||||||
request.onsuccess = () => {
|
|
||||||
this.db = request.result;
|
|
||||||
// v0.3.2: 清理持久化 schema 记录(upgrade 后执行,失败不阻塞删除)
|
|
||||||
if (this.db.objectStoreNames.contains('__metona_schema')) {
|
|
||||||
const schemaTx = this.db.transaction('__metona_schema', 'readwrite');
|
|
||||||
schemaTx.objectStore('__metona_schema').delete(tableName);
|
|
||||||
schemaTx.onerror = () => { /* 忽略:旧库可能无此记录 */ };
|
|
||||||
}
|
|
||||||
resolve();
|
|
||||||
};
|
|
||||||
request.onerror = () => reject(new DatabaseError(`Failed to drop table "${tableName}"`, 'IDB_UPGRADE_ERROR', request.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private async idbInsert(tableName: string, rows: Record<string, unknown>[]): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
for (const row of rows) store.add(row);
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Insert failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private async idbFind(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
|
|
||||||
// 尝试使用 IDB 索引进行等值查询
|
|
||||||
if (query.where) {
|
|
||||||
const indexResult = await this.tryIDBIndexLookup(db, tableName, query);
|
|
||||||
if (indexResult !== null) return indexResult;
|
|
||||||
}
|
|
||||||
|
|
||||||
// 回退到全量 getAll + 内存过滤
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readonly');
|
|
||||||
const req = tx.objectStore(tableName).getAll();
|
|
||||||
req.onsuccess = () => {
|
|
||||||
let results: Record<string, unknown>[] = req.result ?? [];
|
|
||||||
if (query.where && Object.keys(query.where).length > 0) {
|
|
||||||
results = results.filter((row) => matchWhere(row, query.where!));
|
|
||||||
}
|
|
||||||
if (query.orderBy && query.orderBy.length > 0) {
|
|
||||||
results = applyOrderBy(results, query.orderBy);
|
|
||||||
}
|
|
||||||
const offset = query.offset ?? 0;
|
|
||||||
const limit = query.limit ?? results.length;
|
|
||||||
results = results.slice(offset, offset + limit);
|
|
||||||
if (query.columns && query.columns.length > 0 && query.columns[0] !== '*') {
|
|
||||||
results = results.map((r) => projectColumns(r, query.columns!));
|
|
||||||
}
|
|
||||||
resolve(results);
|
|
||||||
};
|
|
||||||
req.onerror = () => reject(new DatabaseError(`Find failed for "${tableName}"`, 'IDB_READ_ERROR', req.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 尝试使用 IDB 索引进行等值查询,成功返回结果,不适用返回 null */
|
|
||||||
private async tryIDBIndexLookup(
|
|
||||||
db: IDBDatabase,
|
|
||||||
tableName: string,
|
|
||||||
query: QueryPlan,
|
|
||||||
): Promise<Record<string, unknown>[] | null> {
|
|
||||||
if (!query.where) return null;
|
|
||||||
|
|
||||||
for (const [col, condition] of Object.entries(query.where)) {
|
|
||||||
// 跳过逻辑组合符
|
|
||||||
if (col === '$and' || col === '$or' || col === '$not') continue;
|
|
||||||
|
|
||||||
// 只处理等值查询
|
|
||||||
let targetValue: unknown;
|
|
||||||
if (typeof condition !== 'object' || condition === null) {
|
|
||||||
targetValue = condition;
|
|
||||||
} else if ('$eq' in (condition as Record<string, unknown>)) {
|
|
||||||
targetValue = (condition as Record<string, unknown>).$eq;
|
|
||||||
} else {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
// 检查是否有对应的 IDB 索引
|
|
||||||
const indexName = `idx_${col}`;
|
|
||||||
try {
|
|
||||||
return await new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readonly');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
if (!store.indexNames.contains(indexName)) {
|
|
||||||
resolve(null); // 没有索引,回退
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const index = store.index(indexName);
|
|
||||||
const req = index.getAll(targetValue as IDBValidKey);
|
|
||||||
req.onsuccess = () => {
|
|
||||||
let results: Record<string, unknown>[] = req.result ?? [];
|
|
||||||
// 如果有其他 WHERE 条件,继续过滤
|
|
||||||
const otherKeys = Object.keys(query.where!).filter(
|
|
||||||
(k) => k !== col && k !== '$and' && k !== '$or' && k !== '$not',
|
|
||||||
);
|
|
||||||
if (otherKeys.length > 0) {
|
|
||||||
results = results.filter((row) => matchWhere(row, query.where!));
|
|
||||||
}
|
|
||||||
if (query.orderBy && query.orderBy.length > 0) {
|
|
||||||
results = applyOrderBy(results, query.orderBy);
|
|
||||||
}
|
|
||||||
const offset = query.offset ?? 0;
|
|
||||||
const limit = query.limit ?? results.length;
|
|
||||||
results = results.slice(offset, offset + limit);
|
|
||||||
if (query.columns && query.columns.length > 0 && query.columns[0] !== '*') {
|
|
||||||
results = results.map((r) => projectColumns(r, query.columns!));
|
|
||||||
}
|
|
||||||
resolve(results);
|
|
||||||
};
|
|
||||||
req.onerror = () => reject(new DatabaseError(`Index lookup failed for "${tableName}.${col}"`, 'IDB_READ_ERROR', req.error));
|
|
||||||
});
|
|
||||||
} catch {
|
|
||||||
return null; // 索引不可用,回退
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
private async idbUpdate(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
const getAllReq = store.getAll();
|
|
||||||
getAllReq.onsuccess = () => {
|
|
||||||
for (const row of getAllReq.result ?? []) {
|
|
||||||
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
|
|
||||||
Object.assign(row, updates); store.put(row);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Update failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private async idbDelete(tableName: string, query: QueryPlan): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
const schema = await this.memoryCache.getTableSchema(tableName);
|
|
||||||
if (!schema) throw new DatabaseError(`Table "${tableName}" not found`, 'TABLE_NOT_FOUND');
|
|
||||||
const pkColumn = Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0];
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
const getAllReq = store.getAll();
|
|
||||||
getAllReq.onsuccess = () => {
|
|
||||||
for (const row of getAllReq.result ?? []) {
|
|
||||||
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
|
|
||||||
store.delete(row[pkColumn] as IDBValidKey);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Delete failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private async idbClear(tableName: string): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readwrite');
|
|
||||||
tx.objectStore(tableName).clear();
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Clear failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 持久化单个表 schema 到 __metona_schema store(v0.4.2-fix: ALTER TABLE 用) */
|
|
||||||
private async persistSchema(schema: TableSchema): Promise<void> {
|
|
||||||
const db = this.ensureDB();
|
|
||||||
if (!db.objectStoreNames.contains('__metona_schema')) return;
|
|
||||||
await new Promise<void>((resolve, reject) => {
|
|
||||||
const tx = db.transaction('__metona_schema', 'readwrite');
|
|
||||||
tx.objectStore('__metona_schema').put({ name: schema.name, schema: JSON.stringify(schema) });
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Failed to persist schema for "${schema.name}"`, 'IDB_SCHEMA_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
|
|
||||||
private async flushToIDB(): Promise<void> {
|
|
||||||
const tableNames = await this.memoryCache.getTableNames();
|
|
||||||
let db = this.ensureDB();
|
|
||||||
|
|
||||||
// v0.4.2-fix: 事务内 DDL 只更新内存,commit 时同步 IDB 的 objectStore 结构:
|
|
||||||
// 缺失的表 store 创建(并持久化 schema)、已删除的 store 移除(防止重启幽灵表)
|
|
||||||
const idbStores = Array.from(db.objectStoreNames);
|
|
||||||
const missing = tableNames.filter((t) => !idbStores.includes(t));
|
|
||||||
const stale = idbStores.filter((s) => s !== '__metona_schema' && !tableNames.includes(s));
|
|
||||||
if (missing.length > 0 || stale.length > 0) {
|
|
||||||
// 升级事务内是同步上下文,先异步收集缺失表的 schema(主键列定义)
|
|
||||||
const schemaMap = new Map<string, TableSchema>();
|
|
||||||
for (const name of missing) {
|
|
||||||
const s = await this.memoryCache.getTableSchema(name);
|
|
||||||
if (s) schemaMap.set(name, s);
|
|
||||||
}
|
|
||||||
const newVersion = db.version + 1;
|
|
||||||
db.close();
|
|
||||||
await new Promise<void>((resolve, reject) => {
|
|
||||||
const request = indexedDB.open(this.dbName, newVersion);
|
|
||||||
request.onupgradeneeded = (event) => {
|
|
||||||
const idb = (event.target as IDBOpenDBRequest).result;
|
|
||||||
for (const name of stale) {
|
|
||||||
idb.deleteObjectStore(name);
|
|
||||||
}
|
|
||||||
for (const name of missing) {
|
|
||||||
const schema = schemaMap.get(name);
|
|
||||||
const pkColumn = schema
|
|
||||||
? (Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0])
|
|
||||||
: undefined;
|
|
||||||
idb.createObjectStore(name, { keyPath: pkColumn });
|
|
||||||
}
|
|
||||||
// 事务内 drop 的表:同步清理持久化 schema 记录(防止重启后幽灵表恢复)
|
|
||||||
if (idb.objectStoreNames.contains('__metona_schema') && stale.length > 0) {
|
|
||||||
const tx = (event.target as IDBOpenDBRequest).transaction!;
|
|
||||||
const store = tx.objectStore('__metona_schema');
|
|
||||||
for (const name of stale) {
|
|
||||||
store.delete(name);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
request.onsuccess = () => {
|
|
||||||
this.db = request.result;
|
|
||||||
this.setupVersionChangeHandler();
|
|
||||||
resolve();
|
|
||||||
};
|
|
||||||
request.onerror = () => reject(new DatabaseError('Failed to sync stores after transaction', 'IDB_UPGRADE_ERROR', request.error));
|
|
||||||
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${this.dbName}" is blocked`, 'IDB_BLOCKED'));
|
|
||||||
});
|
|
||||||
// 事务内新建表的 schema 一并持久化(此前只建 store 不存 schema → 重启后约束推断丢失)
|
|
||||||
for (const name of missing) {
|
|
||||||
const schema = await this.memoryCache.getTableSchema(name);
|
|
||||||
if (schema) await this.persistSchema(schema);
|
|
||||||
}
|
|
||||||
// v0.4.2-fix: DDL 升级会 close 旧连接并重新 open — 重新取 db 引用,
|
|
||||||
// 否则下方数据 flush 用已关闭的连接抛 InvalidStateError
|
|
||||||
db = this.ensureDB();
|
|
||||||
}
|
|
||||||
|
|
||||||
// 每个表在一个单独的 IDB 事务中完成 clear+insert,保证原子性
|
|
||||||
for (const tableName of tableNames) {
|
|
||||||
const rows = await this.memoryCache.find(tableName, { table: tableName });
|
|
||||||
await new Promise<void>((resolve, reject) => {
|
|
||||||
const tx = db.transaction(tableName, 'readwrite');
|
|
||||||
const store = tx.objectStore(tableName);
|
|
||||||
store.clear(); // 清空
|
|
||||||
for (const row of rows) store.add(row); // 批量写入
|
|
||||||
tx.oncomplete = () => resolve();
|
|
||||||
tx.onerror = () => reject(new DatabaseError(`Flush failed for "${tableName}"`, 'IDB_FLUSH_ERROR', tx.error));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private ensureDB(): IDBDatabase {
|
|
||||||
if (!this.db) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
|
|
||||||
return this.db;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 从存储值推断字段类型(schema 重建用,v0.3.2) */
|
|
||||||
function inferFieldType(value: unknown): import('../constants').FieldType {
|
|
||||||
if (typeof value === 'number') return 'number';
|
|
||||||
if (typeof value === 'boolean') return 'boolean';
|
|
||||||
if (typeof value === 'object' && value !== null) return 'json';
|
|
||||||
if (typeof value === 'string') {
|
|
||||||
return isNaN(Date.parse(value)) ? 'string' : 'string';
|
|
||||||
}
|
|
||||||
return 'string';
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,394 @@
|
|||||||
|
/**
|
||||||
|
* KVStore — 自研 KV 事务存储引擎(替代 IndexedDB)
|
||||||
|
* @module engine/kvstore/index
|
||||||
|
*
|
||||||
|
* v0.6.0: 在浏览器文件系统(OPFS)之上实现 IndexedDB 级能力:
|
||||||
|
* - 多 key 原子事务:putMany/deleteMany 写入单条日志记录(单文件 COW 原子追加)→
|
||||||
|
* 崩溃时记录全有或全无(IndexedDB 事务同等的原子性,但完全自研)
|
||||||
|
* - 持久化与崩溃恢复:快照(checkpoint)+ 追加日志(WAL 式),两阶段恢复
|
||||||
|
* - 自愈:快照损坏回退全量日志重放;日志损坏截断至损坏处(丢弃未确认尾部)
|
||||||
|
* - 容错时序:checkpoint = 写快照 → 写 meta → 清空日志(meta 先于截断,
|
||||||
|
* 任何崩溃窗口数据不丢)
|
||||||
|
*
|
||||||
|
* 介质层为 IStorageBackend(OPFSBackend / SharedMemoryBackend):
|
||||||
|
* - 浏览器:自动选择 OPFS(navigator.storage)
|
||||||
|
* - Node/测试:SharedMemoryBackend(跨实例共享,模拟持久化)
|
||||||
|
*
|
||||||
|
* 可靠性设计:
|
||||||
|
* - 所有写操作与 checkpoint 经内部串行队列(快照与日志水位一致,无交错窗口)
|
||||||
|
* - 日志记录与快照均有标准 CRC-32 校验
|
||||||
|
* - 内存索引为热路径(get O(1)),checkpoint 后日志截断
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { IStorageBackend } from '../aria/store/backend';
|
||||||
|
import { OPFSBackend } from '../aria/store/opfs_backend';
|
||||||
|
import { SharedMemoryBackend } from './shared_memory_medium';
|
||||||
|
import { DatabaseError } from '../../constants';
|
||||||
|
import { encodeLogRecord, parseLogRecords, KVLogOp } from './log';
|
||||||
|
import { encodeSnapshot, decodeSnapshot } from './snapshot';
|
||||||
|
import { crc32 } from '../aria/crc32';
|
||||||
|
|
||||||
|
/** 存储键 */
|
||||||
|
const LOG_KEY = '__kv_log';
|
||||||
|
const SNAPSHOT_KEY = '__kv_snapshot';
|
||||||
|
const META_KEY = '__kv_meta';
|
||||||
|
|
||||||
|
/** checkpoint 自动触发阈值(日志字节数,0=不自动) */
|
||||||
|
const DEFAULT_CHECKPOINT_THRESHOLD = 16 * 1024 * 1024;
|
||||||
|
|
||||||
|
interface KVStoreMeta {
|
||||||
|
/** 当前日志水位(快照内嵌;无快照时 0) */
|
||||||
|
seq: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultMedium(): IStorageBackend {
|
||||||
|
const nav = (globalThis as { navigator?: { storage?: { getDirectory?: unknown } } }).navigator;
|
||||||
|
if (typeof nav !== 'undefined' && nav.storage && typeof nav.storage.getDirectory === 'function') {
|
||||||
|
return new OPFSBackend();
|
||||||
|
}
|
||||||
|
return new SharedMemoryBackend();
|
||||||
|
}
|
||||||
|
|
||||||
|
export class KVStore {
|
||||||
|
private medium: IStorageBackend;
|
||||||
|
private dbName = '';
|
||||||
|
private opened = false;
|
||||||
|
|
||||||
|
/** 内存索引(热路径权威视图) */
|
||||||
|
private index = new Map<string, ArrayBuffer>();
|
||||||
|
/** 日志水位(最后一条已应用日志记录序号) */
|
||||||
|
private seq = 0;
|
||||||
|
/** 日志累计字节数(checkpoint 阈值) */
|
||||||
|
private logBytes = 0;
|
||||||
|
/** checkpoint 自动触发阈值(字节) */
|
||||||
|
private checkpointThreshold: number;
|
||||||
|
|
||||||
|
/** 写操作串行队列(checkpoint 与写入无交错窗口) */
|
||||||
|
private opQueue: Promise<unknown> = Promise.resolve();
|
||||||
|
/** 最近一次后台操作失败(checkpoint 时报告) */
|
||||||
|
private lastBackgroundError: unknown = null;
|
||||||
|
|
||||||
|
constructor(medium?: IStorageBackend, checkpointThreshold: number = DEFAULT_CHECKPOINT_THRESHOLD) {
|
||||||
|
this.medium = medium ?? defaultMedium();
|
||||||
|
this.checkpointThreshold = checkpointThreshold;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 底层介质(测试/诊断用) */
|
||||||
|
getMedium(): IStorageBackend {
|
||||||
|
return this.medium;
|
||||||
|
}
|
||||||
|
|
||||||
|
isOpen(): boolean {
|
||||||
|
return this.opened;
|
||||||
|
}
|
||||||
|
|
||||||
|
// =======================================================================
|
||||||
|
// 生命周期
|
||||||
|
// =======================================================================
|
||||||
|
|
||||||
|
/** 打开(加载快照 + 重放日志) */
|
||||||
|
async open(dbName: string): Promise<void> {
|
||||||
|
if (this.opened) return;
|
||||||
|
this.dbName = dbName;
|
||||||
|
await this.medium.open(dbName);
|
||||||
|
this.index = new Map();
|
||||||
|
this.seq = 0;
|
||||||
|
this.logBytes = 0;
|
||||||
|
|
||||||
|
// 1. 读 meta(可能缺失/损坏)
|
||||||
|
let metaSeq = 0;
|
||||||
|
const metaRaw = await this.medium.read(META_KEY);
|
||||||
|
if (metaRaw) {
|
||||||
|
try {
|
||||||
|
const meta = JSON.parse(new TextDecoder().decode(metaRaw)) as KVStoreMeta;
|
||||||
|
metaSeq = Number(meta.seq) || 0;
|
||||||
|
} catch { /* meta 损坏:回退全量日志 */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. 加载快照(损坏则全量日志重放)
|
||||||
|
let snapshotSeq = 0;
|
||||||
|
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
||||||
|
if (snapshotRaw) {
|
||||||
|
const snap = decodeSnapshot(new Uint8Array(snapshotRaw));
|
||||||
|
if (snap) {
|
||||||
|
this.index = new Map(snap.entries);
|
||||||
|
this.seq = snap.seq;
|
||||||
|
snapshotSeq = snap.seq;
|
||||||
|
} else {
|
||||||
|
// 快照损坏:从空索引 + 全量日志重放
|
||||||
|
this.index = new Map();
|
||||||
|
this.seq = 0;
|
||||||
|
snapshotSeq = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. 重放日志(seq > 快照水位的记录)
|
||||||
|
const logRaw = await this.medium.read(LOG_KEY);
|
||||||
|
if (logRaw && logRaw.byteLength > 0) {
|
||||||
|
const log = new Uint8Array(logRaw);
|
||||||
|
const baseSeq = Math.max(metaSeq, snapshotSeq);
|
||||||
|
const corruptOffsets: number[] = [];
|
||||||
|
const applied = parseLogRecords(log, (record) => {
|
||||||
|
if (record.seq <= baseSeq) return; // 快照已包含,跳过(幂等)
|
||||||
|
this.applyRecord(record.entries);
|
||||||
|
this.seq = record.seq;
|
||||||
|
}, (offset) => {
|
||||||
|
corruptOffsets.push(offset);
|
||||||
|
return true; // 记录损坏位置后停止(日志是顺序流,无法跳过继续)
|
||||||
|
});
|
||||||
|
if (applied > 0 || corruptOffsets.length > 0) {
|
||||||
|
this.logBytes = log.byteLength;
|
||||||
|
}
|
||||||
|
if (corruptOffsets.length > 0) {
|
||||||
|
// 损坏尾部:截断日志(丢弃未确认记录),下次 checkpoint 落盘
|
||||||
|
await this.truncateLog();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.opened = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.6.0: 从介质重新加载(多标签页同步/外部写入可见用)。
|
||||||
|
* KVStore 的内存索引非跨实例共享,重新 open 读取介质最新数据。
|
||||||
|
*/
|
||||||
|
async reload(): Promise<void> {
|
||||||
|
if (!this.opened) return;
|
||||||
|
try { await this.opQueue; } catch { /* ignore */ }
|
||||||
|
this.index = new Map();
|
||||||
|
this.seq = 0;
|
||||||
|
this.logBytes = 0;
|
||||||
|
this.opened = false;
|
||||||
|
await this.open(this.dbName);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 关闭(不丢弃数据;下次 open 同名库恢复) */
|
||||||
|
async close(): Promise<void> {
|
||||||
|
if (!this.opened) return;
|
||||||
|
// 排空写队列
|
||||||
|
try { await this.opQueue; } catch { /* 写失败已返回 */ }
|
||||||
|
await this.medium.close();
|
||||||
|
this.index.clear();
|
||||||
|
this.seq = 0;
|
||||||
|
this.logBytes = 0;
|
||||||
|
this.opened = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// =======================================================================
|
||||||
|
// 读写(内存热路径)
|
||||||
|
// =======================================================================
|
||||||
|
|
||||||
|
async get(key: string): Promise<ArrayBuffer | null> {
|
||||||
|
return this.index.get(key) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getAll(): Promise<[string, ArrayBuffer][]> {
|
||||||
|
return Array.from(this.index.entries());
|
||||||
|
}
|
||||||
|
|
||||||
|
async listKeys(): Promise<string[]> {
|
||||||
|
return Array.from(this.index.keys());
|
||||||
|
}
|
||||||
|
|
||||||
|
async exists(key: string): Promise<boolean> {
|
||||||
|
return this.index.has(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
size(): number {
|
||||||
|
return this.index.size;
|
||||||
|
}
|
||||||
|
|
||||||
|
getSeq(): number {
|
||||||
|
return this.seq;
|
||||||
|
}
|
||||||
|
|
||||||
|
// =======================================================================
|
||||||
|
// 写入(原子事务)
|
||||||
|
// =======================================================================
|
||||||
|
|
||||||
|
/** 单 key 写入(原子) */
|
||||||
|
async put(key: string, value: ArrayBuffer): Promise<void> {
|
||||||
|
await this.enqueue(async () => {
|
||||||
|
await this.appendRecord({ [key]: value }, []);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 多 key 原子写入(单条日志记录,崩溃全有或全无) */
|
||||||
|
async putMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
||||||
|
if (Object.keys(entries).length === 0) return;
|
||||||
|
await this.enqueue(async () => {
|
||||||
|
await this.appendRecord(entries, []);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 单 key 删除(原子) */
|
||||||
|
async delete(key: string): Promise<void> {
|
||||||
|
await this.enqueue(async () => {
|
||||||
|
await this.appendRecord({}, [key]);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 多 key 原子删除(单条日志记录) */
|
||||||
|
async deleteMany(keys: string[]): Promise<void> {
|
||||||
|
if (keys.length === 0) return;
|
||||||
|
await this.enqueue(async () => {
|
||||||
|
await this.appendRecord({}, keys);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// =======================================================================
|
||||||
|
// 维护
|
||||||
|
// =======================================================================
|
||||||
|
|
||||||
|
/** checkpoint:快照 → meta → 截断日志(时序保证任何崩溃窗口不丢数据) */
|
||||||
|
async checkpoint(): Promise<void> {
|
||||||
|
await this.enqueue(async () => {
|
||||||
|
// 报告上次后台失败
|
||||||
|
if (this.lastBackgroundError !== null) {
|
||||||
|
const error = this.lastBackgroundError;
|
||||||
|
this.lastBackgroundError = null;
|
||||||
|
throw new DatabaseError('KVStore background write failed', 'KV_BACKGROUND_ERROR', error);
|
||||||
|
}
|
||||||
|
if (this.logBytes === 0 && this.index.size === 0) return;
|
||||||
|
|
||||||
|
// 1. 写快照(COW 原子)
|
||||||
|
const snapBytes = encodeSnapshot(this.seq, this.index);
|
||||||
|
await this.medium.write(SNAPSHOT_KEY, snapBytes.buffer as ArrayBuffer);
|
||||||
|
// 2. 写 meta(指向新水位)
|
||||||
|
const meta: KVStoreMeta = { seq: this.seq };
|
||||||
|
await this.medium.write(META_KEY, new TextEncoder().encode(JSON.stringify(meta)).buffer);
|
||||||
|
// 3. 截断日志(meta 已更新 → 截断安全)
|
||||||
|
await this.truncateLog();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 清空全部数据(保留库本身) */
|
||||||
|
async clear(): Promise<void> {
|
||||||
|
await this.enqueue(async () => {
|
||||||
|
await this.medium.clear();
|
||||||
|
this.index.clear();
|
||||||
|
this.seq = 0;
|
||||||
|
this.logBytes = 0;
|
||||||
|
// 写空 meta(下次 open 正常初始化)
|
||||||
|
const meta: KVStoreMeta = { seq: 0 };
|
||||||
|
await this.medium.write(META_KEY, new TextEncoder().encode(JSON.stringify(meta)).buffer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 自愈:校验快照/日志完整性,清理损坏数据。
|
||||||
|
* @returns 丢弃的损坏日志字节数(0 = 无损坏)
|
||||||
|
*/
|
||||||
|
async repair(): Promise<number> {
|
||||||
|
return this.enqueue(async () => {
|
||||||
|
let discarded = 0;
|
||||||
|
// 1. 校验快照:损坏则删除(下次打开全量日志重放)
|
||||||
|
const snapRaw = await this.medium.read(SNAPSHOT_KEY);
|
||||||
|
if (snapRaw && !decodeSnapshot(new Uint8Array(snapRaw))) {
|
||||||
|
await this.medium.delete(SNAPSHOT_KEY);
|
||||||
|
discarded++;
|
||||||
|
}
|
||||||
|
// 2. 校验日志:损坏尾部截断
|
||||||
|
const logRaw = await this.medium.read(LOG_KEY);
|
||||||
|
if (logRaw && logRaw.byteLength > 0) {
|
||||||
|
const log = new Uint8Array(logRaw);
|
||||||
|
const validBytes = this.findValidLogLength(log);
|
||||||
|
if (validBytes < log.byteLength) {
|
||||||
|
discarded += log.byteLength - validBytes;
|
||||||
|
const truncated = log.subarray(0, validBytes).slice().buffer as ArrayBuffer;
|
||||||
|
await this.medium.write(LOG_KEY, truncated);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return discarded;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// =======================================================================
|
||||||
|
// 内部
|
||||||
|
// =======================================================================
|
||||||
|
|
||||||
|
private enqueue<T>(fn: () => Promise<T>): Promise<T> {
|
||||||
|
const run = this.opQueue.then(fn, fn);
|
||||||
|
this.opQueue = run.then(() => undefined, () => undefined);
|
||||||
|
return run;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 追加一条日志记录并更新内存索引(队列内调用,无并发) */
|
||||||
|
private async appendRecord(puts: Record<string, ArrayBuffer>, deletes: string[]): Promise<void> {
|
||||||
|
this.seq++;
|
||||||
|
const record = encodeLogRecord(this.seq, puts, deletes);
|
||||||
|
try {
|
||||||
|
// 日志追加:介质 append(真追加)或回退读+拼+写
|
||||||
|
const data = record.buffer.slice(record.byteOffset, record.byteOffset + record.byteLength) as ArrayBuffer;
|
||||||
|
if (typeof this.medium.append === 'function') {
|
||||||
|
await this.medium.append(LOG_KEY, data);
|
||||||
|
} else {
|
||||||
|
const existing = await this.medium.read(LOG_KEY);
|
||||||
|
if (existing) {
|
||||||
|
const combined = new Uint8Array(existing.byteLength + data.byteLength);
|
||||||
|
combined.set(new Uint8Array(existing), 0);
|
||||||
|
combined.set(new Uint8Array(data), existing.byteLength);
|
||||||
|
await this.medium.write(LOG_KEY, combined.buffer as ArrayBuffer);
|
||||||
|
} else {
|
||||||
|
await this.medium.write(LOG_KEY, data);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
// 记录写入失败:内存索引不更新(原子性),记录后台错误
|
||||||
|
this.seq--; // 回滚水位
|
||||||
|
this.lastBackgroundError = error;
|
||||||
|
throw new DatabaseError('KVStore log append failed', 'KV_LOG_ERROR', error);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 日志成功:更新内存索引(原子语义)
|
||||||
|
for (const [key, value] of Object.entries(puts)) {
|
||||||
|
this.index.set(key, value);
|
||||||
|
}
|
||||||
|
for (const key of deletes) {
|
||||||
|
this.index.delete(key);
|
||||||
|
}
|
||||||
|
this.logBytes += record.byteLength;
|
||||||
|
|
||||||
|
// 自动 checkpoint(日志超阈值)
|
||||||
|
if (this.checkpointThreshold > 0 && this.logBytes >= this.checkpointThreshold) {
|
||||||
|
await this.medium.write(SNAPSHOT_KEY, encodeSnapshot(this.seq, this.index).buffer as ArrayBuffer);
|
||||||
|
const meta: KVStoreMeta = { seq: this.seq };
|
||||||
|
await this.medium.write(META_KEY, new TextEncoder().encode(JSON.stringify(meta)).buffer);
|
||||||
|
await this.truncateLog();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 截断日志(清空文件) */
|
||||||
|
private async truncateLog(): Promise<void> {
|
||||||
|
try {
|
||||||
|
await this.medium.write(LOG_KEY, new ArrayBuffer(0));
|
||||||
|
} catch { /* 截断失败:下次 checkpoint 重试 */ }
|
||||||
|
this.logBytes = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 应用记录条目到内存索引 */
|
||||||
|
private applyRecord(entries: { op: KVLogOp; key: string; value: ArrayBuffer }[]): void {
|
||||||
|
for (const e of entries) {
|
||||||
|
if (e.op === KVLogOp.PUT) {
|
||||||
|
this.index.set(e.key, e.value);
|
||||||
|
} else {
|
||||||
|
this.index.delete(e.key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 确定日志中有效字节长度(从 0 开始连续解析到第一条损坏/残缺记录) */
|
||||||
|
private findValidLogLength(log: Uint8Array): number {
|
||||||
|
let offset = 0;
|
||||||
|
const view = new DataView(log.buffer, log.byteOffset, log.byteLength);
|
||||||
|
while (offset + 4 <= log.byteLength) {
|
||||||
|
const recordLen = view.getUint32(offset, false);
|
||||||
|
if (recordLen < 12 || offset + 4 + recordLen > log.byteLength) break;
|
||||||
|
const raw = log.subarray(offset, offset + 4 + recordLen);
|
||||||
|
const storedCrc = view.getUint32(offset + recordLen, false);
|
||||||
|
if (crc32(raw.subarray(0, recordLen)) !== storedCrc) break;
|
||||||
|
offset += 4 + recordLen;
|
||||||
|
}
|
||||||
|
return offset;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
/**
|
||||||
|
* KVStore Log — 追加式事务日志编解码
|
||||||
|
* @module engine/kvstore/log
|
||||||
|
*
|
||||||
|
* v0.6.0: 自研 KV 引擎的原子写载体。
|
||||||
|
* 每条日志记录 = 一个原子事务(putMany 多键写入 / deleteMany 多键删除)。
|
||||||
|
* 单文件追加(介质 append,COW 原子)→ 崩溃时记录全有或全无。
|
||||||
|
*
|
||||||
|
* 记录格式(大端序):
|
||||||
|
* [recordLen u32] — 本条记录长度(含自身,不含 CRC)
|
||||||
|
* [seq u32] — 日志序号(递增,恢复时与快照水位比对去重)
|
||||||
|
* [entryCount u32] — 条目数
|
||||||
|
* 每条 entry:
|
||||||
|
* [op u8] — 1=PUT, 2=DELETE
|
||||||
|
* [keyLen u32][key bytes]
|
||||||
|
* [valueLen u32][value bytes] (DELETE 时 valueLen=0)
|
||||||
|
* [crc u32] — 覆盖本条记录除 CRC 外全部字节的标准 CRC-32
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { crc32 } from '../aria/crc32';
|
||||||
|
|
||||||
|
/** 日志操作类型 */
|
||||||
|
export const enum KVLogOp {
|
||||||
|
PUT = 1,
|
||||||
|
DELETE = 2,
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 解析后的日志记录 */
|
||||||
|
export interface KVLogRecord {
|
||||||
|
/** 日志序号 */
|
||||||
|
seq: number;
|
||||||
|
/** 条目列表(op, key, value) */
|
||||||
|
entries: { op: KVLogOp; key: string; value: ArrayBuffer }[];
|
||||||
|
/** 记录原始字节(CRC 校验用) */
|
||||||
|
raw: Uint8Array;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 编码一条日志记录。
|
||||||
|
* @param seq 日志序号
|
||||||
|
* @param puts key → value 写入条目
|
||||||
|
* @param deletes 删除 key 列表
|
||||||
|
*/
|
||||||
|
export function encodeLogRecord(
|
||||||
|
seq: number,
|
||||||
|
puts: Record<string, ArrayBuffer>,
|
||||||
|
deletes: string[] = [],
|
||||||
|
): Uint8Array {
|
||||||
|
const encoder = new TextEncoder();
|
||||||
|
const entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] = [];
|
||||||
|
for (const [key, value] of Object.entries(puts)) {
|
||||||
|
entries.push({ op: KVLogOp.PUT, key, value });
|
||||||
|
}
|
||||||
|
for (const key of deletes) {
|
||||||
|
entries.push({ op: KVLogOp.DELETE, key, value: new ArrayBuffer(0) });
|
||||||
|
}
|
||||||
|
|
||||||
|
// 预编码 key 字节,计算总长度
|
||||||
|
const entryBytes: { op: KVLogOp; key: Uint8Array; value: Uint8Array }[] = [];
|
||||||
|
let total = 4 + 4 + 4; // recordLen + seq + entryCount
|
||||||
|
for (const e of entries) {
|
||||||
|
const kb = encoder.encode(e.key);
|
||||||
|
const vb = new Uint8Array(e.value);
|
||||||
|
entryBytes.push({ op: e.op, key: kb, value: vb });
|
||||||
|
total += 1 + 4 + kb.byteLength + 4 + vb.byteLength;
|
||||||
|
}
|
||||||
|
total += 4; // crc
|
||||||
|
|
||||||
|
const buf = new Uint8Array(total);
|
||||||
|
const view = new DataView(buf.buffer);
|
||||||
|
let offset = 0;
|
||||||
|
view.setUint32(offset, total - 4, false); offset += 4; // recordLen(不含 CRC)
|
||||||
|
view.setUint32(offset, seq, false); offset += 4;
|
||||||
|
view.setUint32(offset, entryBytes.length, false); offset += 4;
|
||||||
|
for (const e of entryBytes) {
|
||||||
|
view.setUint8(offset, e.op); offset += 1;
|
||||||
|
view.setUint32(offset, e.key.byteLength, false); offset += 4;
|
||||||
|
buf.set(e.key, offset); offset += e.key.byteLength;
|
||||||
|
view.setUint32(offset, e.value.byteLength, false); offset += 4;
|
||||||
|
buf.set(e.value, offset); offset += e.value.byteLength;
|
||||||
|
}
|
||||||
|
const crc = crc32(buf.subarray(0, total - 4));
|
||||||
|
view.setUint32(total - 4, crc, false);
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 解析日志中的全部记录(顺序扫描)。
|
||||||
|
* @param data 日志字节流
|
||||||
|
* @param onRecord 每条有效记录回调(CRC 通过)
|
||||||
|
* @param onCorrupt 损坏记录位置回调(返回 false 停止扫描,或继续尝试下一条)
|
||||||
|
* @returns 有效记录数
|
||||||
|
*/
|
||||||
|
export function parseLogRecords(
|
||||||
|
data: Uint8Array,
|
||||||
|
onRecord: (record: KVLogRecord) => void,
|
||||||
|
onCorrupt?: (offset: number) => boolean,
|
||||||
|
): number {
|
||||||
|
let offset = 0;
|
||||||
|
let count = 0;
|
||||||
|
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
|
||||||
|
while (offset + 4 <= data.byteLength) {
|
||||||
|
const recordLen = view.getUint32(offset, false);
|
||||||
|
if (recordLen < 12 || offset + 4 + recordLen > data.byteLength) {
|
||||||
|
// 尾部残缺记录(最后一批写入被截断):损坏
|
||||||
|
if (onCorrupt) {
|
||||||
|
if (!onCorrupt(offset)) break;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
const recordStart = offset;
|
||||||
|
const recordEnd = offset + 4 + recordLen;
|
||||||
|
const raw = data.subarray(recordStart, recordEnd);
|
||||||
|
|
||||||
|
const recView = new DataView(data.buffer, data.byteOffset + recordStart, recordLen + 4);
|
||||||
|
const storedCrc = recView.getUint32(recordLen, false);
|
||||||
|
const computedCrc = crc32(raw.subarray(0, recordLen));
|
||||||
|
if (storedCrc !== computedCrc) {
|
||||||
|
if (onCorrupt) {
|
||||||
|
if (!onCorrupt(recordStart)) break;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 解析条目
|
||||||
|
let p = 4;
|
||||||
|
const seq = recView.getUint32(p, false); p += 4;
|
||||||
|
const entryCount = recView.getUint32(p, false); p += 4;
|
||||||
|
const entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] = [];
|
||||||
|
let valid = true;
|
||||||
|
for (let i = 0; i < entryCount; i++) {
|
||||||
|
if (p + 1 + 4 > recordLen + 4) { valid = false; break; }
|
||||||
|
const op = recView.getUint8(p) as KVLogOp; p += 1;
|
||||||
|
const keyLen = recView.getUint32(p, false); p += 4;
|
||||||
|
if (p + keyLen + 4 > recordLen + 4) { valid = false; break; }
|
||||||
|
const key = decoder.decode(raw.subarray(p, p + keyLen)); p += keyLen;
|
||||||
|
const valueLen = recView.getUint32(p, false); p += 4;
|
||||||
|
if (p + valueLen > recordLen + 4) { valid = false; break; }
|
||||||
|
const value = raw.slice(p, p + valueLen).buffer as ArrayBuffer; p += valueLen;
|
||||||
|
entries.push({ op, key, value });
|
||||||
|
}
|
||||||
|
if (!valid) {
|
||||||
|
if (onCorrupt) {
|
||||||
|
if (!onCorrupt(recordStart)) break;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
onRecord({ seq, entries, raw });
|
||||||
|
count++;
|
||||||
|
offset = recordEnd;
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
/**
|
||||||
|
* KVStore SharedMemory Medium — 跨实例共享的内存介质
|
||||||
|
* @module engine/kvstore/shared_memory_medium
|
||||||
|
*
|
||||||
|
* v0.6.0: 替代 fake-indexeddb 的测试/Node 环境介质。
|
||||||
|
* 与 MemoryBackend 的区别:数据按库名存于全局注册表,close() 不清除
|
||||||
|
* (模拟"磁盘持久化"语义——重新 open 同名库可读到上次写入的数据)。
|
||||||
|
*
|
||||||
|
* 仅用于测试与 Node 环境;浏览器使用 OPFS 介质(KVStore 默认自动选择)。
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { IStorageBackend } from '../aria/store/backend';
|
||||||
|
|
||||||
|
/** 全局注册表:dbName → key → ArrayBuffer(跨实例共享,模拟持久化) */
|
||||||
|
const registry = new Map<string, Map<string, ArrayBuffer>>();
|
||||||
|
|
||||||
|
export class SharedMemoryBackend implements IStorageBackend {
|
||||||
|
private dbName = '';
|
||||||
|
private store: Map<string, ArrayBuffer> | null = null;
|
||||||
|
|
||||||
|
/** 清空全局注册表(测试隔离用) */
|
||||||
|
static clearRegistry(): void {
|
||||||
|
registry.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 注册表中的库数量(测试诊断用) */
|
||||||
|
static registrySize(): number {
|
||||||
|
return registry.size;
|
||||||
|
}
|
||||||
|
|
||||||
|
async open(name: string): Promise<void> {
|
||||||
|
this.dbName = name;
|
||||||
|
if (!registry.has(name)) {
|
||||||
|
registry.set(name, new Map());
|
||||||
|
}
|
||||||
|
this.store = registry.get(name)!;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** close 不清除数据(持久化语义:重开同名库数据仍在) */
|
||||||
|
async close(): Promise<void> {
|
||||||
|
this.store = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
isOpen(): boolean {
|
||||||
|
return this.store !== null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async read(key: string): Promise<ArrayBuffer | null> {
|
||||||
|
return this.store?.get(key) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async write(key: string, data: ArrayBuffer): Promise<void> {
|
||||||
|
this.store?.set(key, data);
|
||||||
|
}
|
||||||
|
|
||||||
|
async append(key: string, data: ArrayBuffer): Promise<void> {
|
||||||
|
if (!this.store) return;
|
||||||
|
const existing = this.store.get(key);
|
||||||
|
if (existing) {
|
||||||
|
const combined = new Uint8Array(existing.byteLength + data.byteLength);
|
||||||
|
combined.set(new Uint8Array(existing), 0);
|
||||||
|
combined.set(new Uint8Array(data), existing.byteLength);
|
||||||
|
this.store.set(key, combined.buffer as ArrayBuffer);
|
||||||
|
} else {
|
||||||
|
this.store.set(key, data);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
||||||
|
if (!this.store) return;
|
||||||
|
// 同步批量写入 = 原子(JS 单线程,无中间 await 点)
|
||||||
|
for (const [key, data] of Object.entries(entries)) {
|
||||||
|
this.store.set(key, data);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async delete(key: string): Promise<void> {
|
||||||
|
this.store?.delete(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
async deleteMany(keys: string[]): Promise<void> {
|
||||||
|
if (!this.store) return;
|
||||||
|
for (const key of keys) {
|
||||||
|
this.store.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async listKeys(): Promise<string[]> {
|
||||||
|
return this.store ? Array.from(this.store.keys()) : [];
|
||||||
|
}
|
||||||
|
|
||||||
|
async exists(key: string): Promise<boolean> {
|
||||||
|
return this.store?.has(key) ?? false;
|
||||||
|
}
|
||||||
|
|
||||||
|
async clear(): Promise<void> {
|
||||||
|
this.store?.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
/**
|
||||||
|
* KVStore Snapshot — 快照序列化/反序列化
|
||||||
|
* @module engine/kvstore/snapshot
|
||||||
|
*
|
||||||
|
* v0.6.0: checkpoint 时把全部 key-value 序列化为快照文件(COW 原子写),
|
||||||
|
* 快照内嵌"日志水位 seq"(快照包含的最后一条日志序号),恢复时只重放 seq > 水位 的记录。
|
||||||
|
*
|
||||||
|
* 格式(大端序):
|
||||||
|
* [magic u32] — 0x4B56534E ("KVSN")
|
||||||
|
* [seq u32] — 日志水位(快照包含的数据对应的日志序号)
|
||||||
|
* [entryCount u32]
|
||||||
|
* 每条: [keyLen u32][key bytes][valueLen u32][value bytes]
|
||||||
|
* [crc u32] — 覆盖除 CRC 外全部字节的标准 CRC-32
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { crc32 } from '../aria/crc32';
|
||||||
|
|
||||||
|
const SNAPSHOT_MAGIC = 0x4b56534e; // "KVSN"
|
||||||
|
|
||||||
|
/** 快照内容 */
|
||||||
|
export interface KVSsnapshot {
|
||||||
|
/** 日志水位 */
|
||||||
|
seq: number;
|
||||||
|
/** key → value */
|
||||||
|
entries: Map<string, ArrayBuffer>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 序列化快照 */
|
||||||
|
export function encodeSnapshot(seq: number, entries: Map<string, ArrayBuffer>): Uint8Array {
|
||||||
|
const encoder = new TextEncoder();
|
||||||
|
const keys = Array.from(entries.keys());
|
||||||
|
|
||||||
|
// 预编码
|
||||||
|
const encoded: { key: Uint8Array; value: Uint8Array }[] = [];
|
||||||
|
let total = 4 + 4 + 4; // magic + seq + entryCount
|
||||||
|
for (const key of keys) {
|
||||||
|
const kb = encoder.encode(key);
|
||||||
|
const vb = new Uint8Array(entries.get(key)!);
|
||||||
|
encoded.push({ key: kb, value: vb });
|
||||||
|
total += 4 + kb.byteLength + 4 + vb.byteLength;
|
||||||
|
}
|
||||||
|
total += 4; // crc
|
||||||
|
|
||||||
|
const buf = new Uint8Array(total);
|
||||||
|
const view = new DataView(buf.buffer);
|
||||||
|
let offset = 0;
|
||||||
|
view.setUint32(offset, SNAPSHOT_MAGIC, false); offset += 4;
|
||||||
|
view.setUint32(offset, seq, false); offset += 4;
|
||||||
|
view.setUint32(offset, encoded.length, false); offset += 4;
|
||||||
|
for (const e of encoded) {
|
||||||
|
view.setUint32(offset, e.key.byteLength, false); offset += 4;
|
||||||
|
buf.set(e.key, offset); offset += e.key.byteLength;
|
||||||
|
view.setUint32(offset, e.value.byteLength, false); offset += 4;
|
||||||
|
buf.set(e.value, offset); offset += e.value.byteLength;
|
||||||
|
}
|
||||||
|
const crc = crc32(buf.subarray(0, total - 4));
|
||||||
|
view.setUint32(total - 4, crc, false);
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 解析快照。
|
||||||
|
* @returns 快照内容;损坏(magic 错误/CRC 失败/越界)返回 null
|
||||||
|
*/
|
||||||
|
export function decodeSnapshot(data: Uint8Array): KVSsnapshot | null {
|
||||||
|
if (data.byteLength < 16) return null;
|
||||||
|
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
||||||
|
if (view.getUint32(0, false) !== SNAPSHOT_MAGIC) return null;
|
||||||
|
|
||||||
|
const storedCrc = view.getUint32(data.byteLength - 4, false);
|
||||||
|
const computedCrc = crc32(data.subarray(0, data.byteLength - 4));
|
||||||
|
if (storedCrc !== computedCrc) return null;
|
||||||
|
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
const entries = new Map<string, ArrayBuffer>();
|
||||||
|
let p = 4;
|
||||||
|
const seq = view.getUint32(p, false); p += 4;
|
||||||
|
const entryCount = view.getUint32(p, false); p += 4;
|
||||||
|
|
||||||
|
for (let i = 0; i < entryCount; i++) {
|
||||||
|
if (p + 4 > data.byteLength - 4) return null;
|
||||||
|
const keyLen = view.getUint32(p, false); p += 4;
|
||||||
|
if (p + keyLen + 4 > data.byteLength - 4) return null;
|
||||||
|
const key = decoder.decode(data.subarray(p, p + keyLen)); p += keyLen;
|
||||||
|
const valueLen = view.getUint32(p, false); p += 4;
|
||||||
|
if (p + valueLen > data.byteLength - 4) return null;
|
||||||
|
const value = data.slice(p, p + valueLen).buffer as ArrayBuffer; p += valueLen;
|
||||||
|
entries.set(key, value);
|
||||||
|
}
|
||||||
|
|
||||||
|
return { seq, entries };
|
||||||
|
}
|
||||||
@@ -0,0 +1,492 @@
|
|||||||
|
/**
|
||||||
|
* KVStoreEngine — 基于自研 KVStore 的磁盘存储引擎(替代 IndexedDBEngine / OPFSEngine)
|
||||||
|
* @module engine/kvstore_engine
|
||||||
|
*
|
||||||
|
* v0.6.0: 完全移除 IndexedDB 后的 disk 模式引擎。
|
||||||
|
*
|
||||||
|
* 架构:MemoryEngine(内存热路径 + 事务快照)+ KVStore(持久化 + 原子写)
|
||||||
|
* - 读:始终走内存(写路径同步落盘,重启从 KVStore 恢复)
|
||||||
|
* - 写:内存先行 + KVStore 增量持久化(insert 增量 putMany;update/delete 受影响行重写;
|
||||||
|
* 主键变更/级联场景整表 diff;全部原子)
|
||||||
|
* - 事务:内存快照 + commit 时受影响表原子 flush(putMany 单记录 = 真原子,
|
||||||
|
* 此前 IndexedDBEngine 依赖 IDB 事务,现在完全自研)
|
||||||
|
*
|
||||||
|
* 数据布局(KVStore keys):
|
||||||
|
* `__schema` — JSON { tableName: TableSchema }
|
||||||
|
* `__meta:{key}` — 库内元数据(迁移版本等)
|
||||||
|
* `t:{table}:{pk}` — 行数据(JSON)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { IStorageEngine } from './interface';
|
||||||
|
import type { QueryPlan, TableSchema } from '../constants';
|
||||||
|
import { DatabaseError } from '../constants';
|
||||||
|
import { MemoryEngine } from './memory';
|
||||||
|
import { KVStore } from './kvstore/index';
|
||||||
|
import { SharedMemoryBackend } from './kvstore/shared_memory_medium';
|
||||||
|
import type { IStorageBackend } from './aria/store/backend';
|
||||||
|
|
||||||
|
const SCHEMA_KEY = '__schema';
|
||||||
|
const ROW_PREFIX = 't:';
|
||||||
|
|
||||||
|
const enc = (s: string) => new TextEncoder().encode(s).buffer as ArrayBuffer;
|
||||||
|
const dec = (b: ArrayBuffer) => new TextDecoder().decode(b);
|
||||||
|
|
||||||
|
export class KVStoreEngine implements IStorageEngine {
|
||||||
|
readonly name = 'kv';
|
||||||
|
|
||||||
|
private kv!: KVStore;
|
||||||
|
private memory: MemoryEngine = new MemoryEngine();
|
||||||
|
private dbName = '';
|
||||||
|
private version = 1;
|
||||||
|
private opened = false;
|
||||||
|
|
||||||
|
/** 活跃事务标记 */
|
||||||
|
private txActive = false;
|
||||||
|
/** 事务中写过的表(commit 时只 flush 这些表) */
|
||||||
|
private txDirtyTables: Set<string> = new Set();
|
||||||
|
|
||||||
|
constructor(medium?: IStorageBackend, checkpointThreshold?: number) {
|
||||||
|
this.kv = new KVStore(medium, checkpointThreshold);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 行 key 编解码 ----
|
||||||
|
|
||||||
|
private rowKey(table: string, pk: string): string {
|
||||||
|
return `${ROW_PREFIX}${table}:${pk}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
private rowPrefix(table: string): string {
|
||||||
|
return `${ROW_PREFIX}${table}:`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 生命周期 ----
|
||||||
|
|
||||||
|
async open(dbName: string, version: number): Promise<void> {
|
||||||
|
if (this.opened) return;
|
||||||
|
this.dbName = dbName;
|
||||||
|
this.version = version;
|
||||||
|
await this.kv.open(dbName);
|
||||||
|
await this.memory.open(dbName, version);
|
||||||
|
|
||||||
|
// 恢复 schema
|
||||||
|
const schemaRaw = await this.kv.get(SCHEMA_KEY);
|
||||||
|
if (schemaRaw) {
|
||||||
|
try {
|
||||||
|
const schemas = JSON.parse(dec(schemaRaw)) as Record<string, TableSchema>;
|
||||||
|
for (const schema of Object.values(schemas)) {
|
||||||
|
await this.memory.createTable(schema);
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
throw new DatabaseError('Corrupted schema in KVStore', 'KV_SCHEMA_ERROR');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 恢复行数据 + 重建索引
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
for (const [key, value] of all) {
|
||||||
|
if (!key.startsWith(ROW_PREFIX)) continue;
|
||||||
|
const sep = key.indexOf(':', ROW_PREFIX.length);
|
||||||
|
if (sep < 0) continue;
|
||||||
|
const table = key.slice(ROW_PREFIX.length, sep);
|
||||||
|
if (!(await this.memory.hasTable(table))) continue;
|
||||||
|
try {
|
||||||
|
const row = JSON.parse(dec(value));
|
||||||
|
await this.memory.insert(table, [row]);
|
||||||
|
} catch {
|
||||||
|
// 单行损坏跳过(repair 可清理)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 重建二级索引(schema 标记的索引列)
|
||||||
|
const tables = await this.memory.getTableNames();
|
||||||
|
for (const table of tables) {
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (!schema) continue;
|
||||||
|
for (const [col, colDef] of Object.entries(schema.columns)) {
|
||||||
|
if (colDef.index || colDef.unique) {
|
||||||
|
await this.memory.createIndex(table, col, colDef.unique);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.opened = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
async close(): Promise<void> {
|
||||||
|
if (!this.opened) return;
|
||||||
|
// 活跃事务先回滚
|
||||||
|
if (this.txActive) {
|
||||||
|
try { await this.rollbackTransaction(); } catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
await this.kv.close();
|
||||||
|
await this.memory.close();
|
||||||
|
this.opened = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
isOpen(): boolean {
|
||||||
|
return this.opened;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* v0.6.0: 从 KVStore 重新加载全部数据到内存(多标签页同步重载用)。
|
||||||
|
* Hybrid 引擎的 reloadMemoryFromDisk 依赖磁盘引擎"读穿透",
|
||||||
|
* KVStoreEngine 读内存 → 提供 reload 重新加载磁盘最新数据。
|
||||||
|
*/
|
||||||
|
async reload(): Promise<void> {
|
||||||
|
if (!this.opened) return;
|
||||||
|
// 1. KVStore 重新从介质加载(外部写入可见)
|
||||||
|
await this.kv.reload();
|
||||||
|
// 2. 内存缓存重载
|
||||||
|
await this.memory.close();
|
||||||
|
await this.memory.open(this.dbName, this.version);
|
||||||
|
this.opened = false;
|
||||||
|
await this.open(this.dbName, this.version);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** v0.4.2-fix: 自愈 — 校验 KVStore 日志/快照完整性并重建内存 */
|
||||||
|
async repair(): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.kv.repair();
|
||||||
|
await this.memory.close();
|
||||||
|
await this.memory.open(this.dbName, this.version);
|
||||||
|
// 重新恢复(复用 open 的恢复逻辑)
|
||||||
|
this.opened = false;
|
||||||
|
await this.open(this.dbName, this.version);
|
||||||
|
}
|
||||||
|
|
||||||
|
async clearAll(): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.kv.clear();
|
||||||
|
await this.memory.clearAll();
|
||||||
|
}
|
||||||
|
|
||||||
|
async getMeta(key: string): Promise<string | null> {
|
||||||
|
const raw = await this.kv.get(`__meta:${key}`);
|
||||||
|
return raw ? dec(raw) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async setMeta(key: string, value: string): Promise<void> {
|
||||||
|
await this.kv.put(`__meta:${key}`, enc(value));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 表管理 ----
|
||||||
|
|
||||||
|
async createTable(schema: TableSchema): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.createTable(schema);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(schema.name);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await this.persistSchema();
|
||||||
|
}
|
||||||
|
|
||||||
|
async dropTable(tableName: string): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.dropTable(tableName);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await this.persistSchema();
|
||||||
|
// 删除该表全部行(KV 中残留清理)
|
||||||
|
await this.flushTable(tableName);
|
||||||
|
}
|
||||||
|
|
||||||
|
async hasTable(tableName: string): Promise<boolean> {
|
||||||
|
return this.memory.hasTable(tableName);
|
||||||
|
}
|
||||||
|
|
||||||
|
async getTableNames(): Promise<string[]> {
|
||||||
|
return this.memory.getTableNames();
|
||||||
|
}
|
||||||
|
|
||||||
|
async getTableSchema(tableName: string): Promise<TableSchema | null> {
|
||||||
|
return this.memory.getTableSchema(tableName);
|
||||||
|
}
|
||||||
|
|
||||||
|
async alterTable(
|
||||||
|
tableName: string,
|
||||||
|
action: 'ADD' | 'DROP',
|
||||||
|
column: import('../constants').ColumnDef & { name: string },
|
||||||
|
): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.alterTable(tableName, action, column);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await this.persistSchema();
|
||||||
|
if (action === 'DROP') {
|
||||||
|
// 重写存储行(移除该列)
|
||||||
|
await this.flushTable(tableName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- CRUD ----
|
||||||
|
|
||||||
|
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
||||||
|
this.ensureOpen();
|
||||||
|
const pks = await this.memory.insert(tableName, rows);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return pks;
|
||||||
|
}
|
||||||
|
// 增量持久化(原子 putMany)
|
||||||
|
const schema = await this.memory.getTableSchema(tableName);
|
||||||
|
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const puts: Record<string, ArrayBuffer> = {};
|
||||||
|
rows.forEach((row, i) => {
|
||||||
|
puts[this.rowKey(tableName, String(pks[i] ?? row[pkCol]))] = enc(JSON.stringify(row));
|
||||||
|
});
|
||||||
|
await this.kv.putMany(puts);
|
||||||
|
return pks;
|
||||||
|
}
|
||||||
|
|
||||||
|
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
|
||||||
|
this.ensureOpen();
|
||||||
|
return this.memory.find(tableName, query);
|
||||||
|
}
|
||||||
|
|
||||||
|
async findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number> {
|
||||||
|
this.ensureOpen();
|
||||||
|
return this.memory.findStream(tableName, query, onRow);
|
||||||
|
}
|
||||||
|
|
||||||
|
async update(
|
||||||
|
tableName: string,
|
||||||
|
query: QueryPlan,
|
||||||
|
updates: Record<string, unknown>,
|
||||||
|
): Promise<number> {
|
||||||
|
this.ensureOpen();
|
||||||
|
const schema = await this.memory.getTableSchema(tableName);
|
||||||
|
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const pkChanged = pkCol in updates;
|
||||||
|
|
||||||
|
// 收集受影响旧主键(内存匹配)
|
||||||
|
const affected = pkChanged ? [] : await this.collectMatchingPks(tableName, query);
|
||||||
|
const count = await this.memory.update(tableName, query, updates);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (pkChanged) {
|
||||||
|
// 主键变更:相关表整表 diff(罕见操作,可靠性优先)
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
await this.flushTable(t);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// 增量重写受影响行
|
||||||
|
const puts: Record<string, ArrayBuffer> = {};
|
||||||
|
const deletes: string[] = [];
|
||||||
|
for (const pk of affected) {
|
||||||
|
const row = await this.memory.find(tableName, { table: tableName, where: { [pkCol]: pk } });
|
||||||
|
if (row.length > 0) {
|
||||||
|
puts[this.rowKey(tableName, pk)] = enc(JSON.stringify(row[0]));
|
||||||
|
} else {
|
||||||
|
deletes.push(this.rowKey(tableName, pk));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (Object.keys(puts).length > 0) await this.kv.putMany(puts);
|
||||||
|
if (deletes.length > 0) await this.kv.deleteMany(deletes);
|
||||||
|
// 级联影响表(SET NULL/CASCADE 外键)整表 diff
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) await this.flushTable(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
||||||
|
this.ensureOpen();
|
||||||
|
// 收集受影响主键(内存匹配)
|
||||||
|
const pks = await this.collectMatchingPks(tableName, query);
|
||||||
|
const count = await this.memory.delete(tableName, query);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
const deletes = pks.map((pk) => this.rowKey(tableName, pk));
|
||||||
|
if (deletes.length > 0) await this.kv.deleteMany(deletes);
|
||||||
|
// 级联影响表整表 diff
|
||||||
|
for (const t of await this.affectedTables(tableName)) {
|
||||||
|
if (t !== tableName) await this.flushTable(t);
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
async count(tableName: string, query?: QueryPlan): Promise<number> {
|
||||||
|
this.ensureOpen();
|
||||||
|
return this.memory.count(tableName, query);
|
||||||
|
}
|
||||||
|
|
||||||
|
async clear(tableName: string): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.clear(tableName);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await this.flushTable(tableName);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 动态索引 ----
|
||||||
|
|
||||||
|
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.createIndex(tableName, column, unique);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await this.persistSchema();
|
||||||
|
}
|
||||||
|
|
||||||
|
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.dropIndex(tableName, column, indexName);
|
||||||
|
if (this.txActive) {
|
||||||
|
this.txDirtyTables.add(tableName);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await this.persistSchema();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 事务(原子 flush) ----
|
||||||
|
|
||||||
|
async beginTransaction(): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
await this.memory.beginTransaction();
|
||||||
|
this.txActive = true;
|
||||||
|
this.txDirtyTables = new Set();
|
||||||
|
}
|
||||||
|
|
||||||
|
async commitTransaction(): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
|
// 先持久化(原子),再提交内存快照(失败可回滚)
|
||||||
|
for (const table of this.txDirtyTables) {
|
||||||
|
if (await this.memory.hasTable(table)) {
|
||||||
|
await this.flushTable(table);
|
||||||
|
} else {
|
||||||
|
// 事务内 drop 的表:清理 KV 残留行
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const prefix = this.rowPrefix(table);
|
||||||
|
const deletes = all.filter(([key]) => key.startsWith(prefix)).map(([key]) => key);
|
||||||
|
if (deletes.length > 0) await this.kv.deleteMany(deletes);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// v0.6.0-fix: 事务内 DDL(create/drop/alter)的 schema 一并持久化
|
||||||
|
await this.persistSchema();
|
||||||
|
await this.kv.checkpoint();
|
||||||
|
await this.memory.commitTransaction();
|
||||||
|
this.txActive = false;
|
||||||
|
this.txDirtyTables = new Set();
|
||||||
|
}
|
||||||
|
|
||||||
|
async rollbackTransaction(): Promise<void> {
|
||||||
|
this.ensureOpen();
|
||||||
|
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
|
||||||
|
await this.memory.rollbackTransaction();
|
||||||
|
this.txActive = false;
|
||||||
|
this.txDirtyTables = new Set();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 内部 ----
|
||||||
|
|
||||||
|
private ensureOpen(): void {
|
||||||
|
if (!this.opened) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
|
||||||
|
}
|
||||||
|
|
||||||
|
private getPK(schema: TableSchema): string {
|
||||||
|
for (const [name, col] of Object.entries(schema.columns)) {
|
||||||
|
if (col.primaryKey) return name;
|
||||||
|
}
|
||||||
|
return Object.keys(schema.columns)[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 收集匹配查询的内存行主键(持久化差异计算用) */
|
||||||
|
private async collectMatchingPks(tableName: string, query: QueryPlan): Promise<string[]> {
|
||||||
|
const schema = await this.memory.getTableSchema(tableName);
|
||||||
|
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const rows = await this.memory.find(tableName, query);
|
||||||
|
return rows.map((r) => String(r[pkCol]));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 计算外键级联影响的表集合(传递闭包:A 被 B 引用,B 被 C 引用 → {A, B, C})。
|
||||||
|
* 级联操作(delete/update 主键)需要把这些表一并重写持久化。
|
||||||
|
*/
|
||||||
|
private async affectedTables(tableName: string): Promise<Set<string>> {
|
||||||
|
const set = new Set<string>([tableName]);
|
||||||
|
let changed = true;
|
||||||
|
while (changed) {
|
||||||
|
changed = false;
|
||||||
|
for (const table of await this.memory.getTableNames()) {
|
||||||
|
if (set.has(table)) continue;
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (!schema) continue;
|
||||||
|
for (const col of Object.values(schema.columns)) {
|
||||||
|
if (col.references) {
|
||||||
|
const ref = col.references.split('.')[0];
|
||||||
|
if (set.has(ref)) {
|
||||||
|
set.add(table);
|
||||||
|
changed = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return set;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 持久化 schema(全部表) */
|
||||||
|
private async persistSchema(): Promise<void> {
|
||||||
|
const schemas: Record<string, TableSchema> = {};
|
||||||
|
for (const table of await this.memory.getTableNames()) {
|
||||||
|
const schema = await this.memory.getTableSchema(table);
|
||||||
|
if (schema) schemas[table] = schema;
|
||||||
|
}
|
||||||
|
await this.kv.put(SCHEMA_KEY, enc(JSON.stringify(schemas)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 整表 diff 持久化:内存行全部 put + KV 残留行删除(原子 putMany + deleteMany)。
|
||||||
|
* 用于主键变更 / 级联 / dropTable / clear / alterTable DROP / 事务 commit。
|
||||||
|
*/
|
||||||
|
private async flushTable(tableName: string): Promise<void> {
|
||||||
|
const prefix = this.rowPrefix(tableName);
|
||||||
|
// 表已删除:仅清理 KV 残留行
|
||||||
|
const schema = await this.memory.getTableSchema(tableName);
|
||||||
|
if (!schema) {
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const deletes = all.filter(([key]) => key.startsWith(prefix)).map(([key]) => key);
|
||||||
|
if (deletes.length > 0) await this.kv.deleteMany(deletes);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const pkCol = this.getPK(schema);
|
||||||
|
const rows = await this.memory.find(tableName, { table: tableName });
|
||||||
|
|
||||||
|
const puts: Record<string, ArrayBuffer> = {};
|
||||||
|
const current = new Set<string>();
|
||||||
|
for (const row of rows) {
|
||||||
|
const key = this.rowKey(tableName, String(row[pkCol]));
|
||||||
|
current.add(key);
|
||||||
|
puts[key] = enc(JSON.stringify(row));
|
||||||
|
}
|
||||||
|
// KV 残留行(内存中已不存在)删除
|
||||||
|
const all = await this.kv.getAll();
|
||||||
|
const deletes: string[] = [];
|
||||||
|
for (const [key] of all) {
|
||||||
|
if (key.startsWith(prefix) && !current.has(key)) {
|
||||||
|
deletes.push(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (Object.keys(puts).length > 0) await this.kv.putMany(puts);
|
||||||
|
if (deletes.length > 0) await this.kv.deleteMany(deletes);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,363 +0,0 @@
|
|||||||
/**
|
|
||||||
* metona-sqlark OPFS Engine — 基于 Origin Private File System 的持久化存储引擎
|
|
||||||
* @module engine/opfs
|
|
||||||
*
|
|
||||||
* 使用 JSON-per-table 文件存储方案。
|
|
||||||
* 目录结构:{dbName}/tables/{tableName}.json
|
|
||||||
*/
|
|
||||||
|
|
||||||
import type { IStorageEngine } from './interface';
|
|
||||||
import type { QueryPlan, TableSchema } from '../constants';
|
|
||||||
import { DatabaseError } from '../constants';
|
|
||||||
import { MemoryEngine } from './memory';
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// OPFSEngine
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
export class OPFSEngine implements IStorageEngine {
|
|
||||||
readonly name = 'opfs';
|
|
||||||
|
|
||||||
private root: FileSystemDirectoryHandle | null = null;
|
|
||||||
private tablesDir: FileSystemDirectoryHandle | null = null;
|
|
||||||
private dbName = '';
|
|
||||||
|
|
||||||
// 运行时内存缓存(OPFS 文件读写有延迟)
|
|
||||||
private memoryCache: MemoryEngine = new MemoryEngine();
|
|
||||||
|
|
||||||
/**
|
|
||||||
* v0.4.3-fix: 写操作串行队列 — 内存写 + 快照 + 文件持久化整体排队执行,
|
|
||||||
* close() 等待队列排空后再释放目录句柄(避免 close 后挂起写泄漏/读旧数据)。
|
|
||||||
* 前一个操作失败不阻塞后续(错误仍返回给调用方)。
|
|
||||||
*/
|
|
||||||
private opQueue: Promise<unknown> = Promise.resolve();
|
|
||||||
|
|
||||||
/** 将写操作加入串行队列(快照在队列内取,始终最新) */
|
|
||||||
private enqueueOp<T>(fn: () => Promise<T>): Promise<T> {
|
|
||||||
const run = this.opQueue.then(fn, fn);
|
|
||||||
this.opQueue = run.then(() => undefined, () => undefined);
|
|
||||||
return run;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 生命周期 ----
|
|
||||||
|
|
||||||
async open(dbName: string, version: number): Promise<void> {
|
|
||||||
this.dbName = dbName;
|
|
||||||
await this.memoryCache.open(dbName, version);
|
|
||||||
|
|
||||||
// 获取 OPFS 根目录
|
|
||||||
this.root = await navigator.storage.getDirectory();
|
|
||||||
|
|
||||||
// 创建/打开数据库目录
|
|
||||||
this.tablesDir = await this.root.getDirectoryHandle(dbName, { create: true });
|
|
||||||
|
|
||||||
// 从 OPFS 恢复已有表数据到内存缓存
|
|
||||||
await this.loadExistingTables();
|
|
||||||
}
|
|
||||||
|
|
||||||
async close(): Promise<void> {
|
|
||||||
// v0.4.3-fix: 等待所有挂起写操作完成(否则 close 后写仍在进行 → 重启读旧数据)
|
|
||||||
try {
|
|
||||||
await this.opQueue;
|
|
||||||
} catch { /* 写失败已返回给调用方 */ }
|
|
||||||
this.root = null;
|
|
||||||
this.tablesDir = null;
|
|
||||||
await this.memoryCache.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
isOpen(): boolean {
|
|
||||||
return this.tablesDir !== null;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- v0.4.2-fix: 自愈 / 重置 / 元数据 ----
|
|
||||||
|
|
||||||
/** 自愈:重置内存缓存后从 OPFS 重新加载(单文件损坏不影响其他表) */
|
|
||||||
async repair(): Promise<void> {
|
|
||||||
// v0.4.3-fix: 先等写队列排空(避免与挂起写竞态)
|
|
||||||
try { await this.opQueue; } catch { /* ignore */ }
|
|
||||||
await this.memoryCache.close();
|
|
||||||
await this.memoryCache.open(this.dbName, 1);
|
|
||||||
await this.loadExistingTables();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 清空全部数据与表结构(删除目录内全部文件) */
|
|
||||||
async clearAll(): Promise<void> {
|
|
||||||
// v0.4.3-fix: 先等写队列排空
|
|
||||||
try { await this.opQueue; } catch { /* ignore */ }
|
|
||||||
await this.memoryCache.close();
|
|
||||||
await this.memoryCache.open(this.dbName, 1);
|
|
||||||
if (this.tablesDir) {
|
|
||||||
const dir = this.tablesDir as any;
|
|
||||||
for await (const [name] of dir.entries()) {
|
|
||||||
try { await this.tablesDir!.removeEntry(name); } catch { /* ignore */ }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getMeta(key: string): Promise<string | null> {
|
|
||||||
if (!this.tablesDir) return null;
|
|
||||||
try {
|
|
||||||
const fh = await this.tablesDir.getFileHandle(`__metona_${key}.meta`);
|
|
||||||
const file = await fh.getFile();
|
|
||||||
return await file.text();
|
|
||||||
} catch {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async setMeta(key: string, value: string): Promise<void> {
|
|
||||||
if (!this.tablesDir) return;
|
|
||||||
const fh = await this.tablesDir.getFileHandle(`__metona_${key}.meta`, { create: true });
|
|
||||||
const writable = await fh.createWritable();
|
|
||||||
await writable.write(value);
|
|
||||||
await writable.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 表管理 ----
|
|
||||||
|
|
||||||
async createTable(schema: TableSchema): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.createTable(schema);
|
|
||||||
// v0.4.2-fix: schema 持久化(此前仅写空数据文件 → 空表重启后消失、索引标记丢失)
|
|
||||||
await this.setMeta(`schema_${schema.name}`, JSON.stringify(schema));
|
|
||||||
// OPFS 中表以空 JSON 数组文件形式存在
|
|
||||||
await this.writeTableData(schema.name, []);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async dropTable(tableName: string): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.dropTable(tableName);
|
|
||||||
// v0.4.2-fix: 清理 schema meta(否则重启恢复幽灵表)
|
|
||||||
if (this.tablesDir) {
|
|
||||||
try {
|
|
||||||
await this.tablesDir.removeEntry(`__metona_schema_${tableName}.meta`);
|
|
||||||
} catch {
|
|
||||||
// 文件不存在则忽略
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
await this.tablesDir.removeEntry(`${tableName}.json`);
|
|
||||||
} catch {
|
|
||||||
// 文件不存在则忽略
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async hasTable(tableName: string): Promise<boolean> {
|
|
||||||
if (!this.tablesDir) return false;
|
|
||||||
try {
|
|
||||||
await this.tablesDir.getFileHandle(`${tableName}.json`);
|
|
||||||
return true;
|
|
||||||
} catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getTableNames(): Promise<string[]> {
|
|
||||||
if (!this.tablesDir) return [];
|
|
||||||
const names: string[] = [];
|
|
||||||
for await (const [name] of (this.tablesDir as any).entries()) {
|
|
||||||
if (name.endsWith('.json')) {
|
|
||||||
names.push(name.replace('.json', ''));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return names;
|
|
||||||
}
|
|
||||||
|
|
||||||
async getTableSchema(tableName: string): Promise<TableSchema | null> {
|
|
||||||
return this.memoryCache.getTableSchema(tableName);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** v0.4.2-fix: 引擎级 ALTER TABLE — 内存 + schema 持久化 + 整表文件重写 */
|
|
||||||
async alterTable(
|
|
||||||
tableName: string,
|
|
||||||
action: 'ADD' | 'DROP',
|
|
||||||
column: import('../constants').ColumnDef & { name: string },
|
|
||||||
): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.alterTable(tableName, action, column);
|
|
||||||
const schema = await this.memoryCache.getTableSchema(tableName);
|
|
||||||
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
|
|
||||||
const rows = await this.memoryCache.find(tableName, { table: tableName });
|
|
||||||
await this.writeTableData(tableName, rows);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- CRUD ----
|
|
||||||
|
|
||||||
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
const pks = await this.memoryCache.insert(tableName, rows);
|
|
||||||
// 持久化到 OPFS(快照在队列内取,始终最新)
|
|
||||||
const allRows = await this.memoryCache.find(tableName, { table: tableName });
|
|
||||||
await this.writeTableData(tableName, allRows);
|
|
||||||
return pks;
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
|
|
||||||
return this.memoryCache.find(tableName, query);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** v0.4.0: 流式查询(委托内存缓存) */
|
|
||||||
async findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number> {
|
|
||||||
return this.memoryCache.findStream(tableName, query, onRow);
|
|
||||||
}
|
|
||||||
|
|
||||||
async update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
const count = await this.memoryCache.update(tableName, query, updates);
|
|
||||||
const allRows = await this.memoryCache.find(tableName, { table: tableName });
|
|
||||||
await this.writeTableData(tableName, allRows);
|
|
||||||
return count;
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async delete(tableName: string, query: QueryPlan): Promise<number> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
const count = await this.memoryCache.delete(tableName, query);
|
|
||||||
const allRows = await this.memoryCache.find(tableName, { table: tableName });
|
|
||||||
await this.writeTableData(tableName, allRows);
|
|
||||||
return count;
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async count(tableName: string, query?: QueryPlan): Promise<number> {
|
|
||||||
return this.memoryCache.count(tableName, query);
|
|
||||||
}
|
|
||||||
|
|
||||||
async clear(tableName: string): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.clear(tableName);
|
|
||||||
await this.writeTableData(tableName, []);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 动态索引(v0.3.0) ----
|
|
||||||
|
|
||||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.createIndex(tableName, column, unique);
|
|
||||||
// v0.4.2-fix: 索引标记持久化(重启后索引结构恢复)
|
|
||||||
const schema = await this.memoryCache.getTableSchema(tableName);
|
|
||||||
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.dropIndex(tableName, column, indexName);
|
|
||||||
const schema = await this.memoryCache.getTableSchema(tableName);
|
|
||||||
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 事务 ----
|
|
||||||
|
|
||||||
async beginTransaction(): Promise<void> {
|
|
||||||
await this.memoryCache.beginTransaction();
|
|
||||||
}
|
|
||||||
|
|
||||||
async commitTransaction(): Promise<void> {
|
|
||||||
return this.enqueueOp(async () => {
|
|
||||||
await this.memoryCache.commitTransaction();
|
|
||||||
// 将内存数据刷到 OPFS
|
|
||||||
const tableNames = await this.memoryCache.getTableNames();
|
|
||||||
for (const tableName of tableNames) {
|
|
||||||
const rows = await this.memoryCache.find(tableName, { table: tableName });
|
|
||||||
await this.writeTableData(tableName, rows);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async rollbackTransaction(): Promise<void> {
|
|
||||||
await this.memoryCache.rollbackTransaction();
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- 内部辅助 ----
|
|
||||||
|
|
||||||
private ensureDir(): FileSystemDirectoryHandle {
|
|
||||||
if (!this.tablesDir) {
|
|
||||||
throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
|
|
||||||
}
|
|
||||||
return this.tablesDir;
|
|
||||||
}
|
|
||||||
|
|
||||||
private async writeTableData(tableName: string, data: Record<string, unknown>[]): Promise<void> {
|
|
||||||
// 由 enqueueOp 串行化调用,此处直接写文件
|
|
||||||
const dir = this.ensureDir();
|
|
||||||
const fileName = `${tableName}.json`;
|
|
||||||
const fileHandle = await dir.getFileHandle(fileName, { create: true });
|
|
||||||
const writable = await fileHandle.createWritable();
|
|
||||||
await writable.write(JSON.stringify(data));
|
|
||||||
await writable.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
private async readTableData(tableName: string): Promise<Record<string, unknown>[]> {
|
|
||||||
const dir = this.ensureDir();
|
|
||||||
const fileName = `${tableName}.json`;
|
|
||||||
try {
|
|
||||||
const fileHandle = await dir.getFileHandle(fileName);
|
|
||||||
const file = await fileHandle.getFile();
|
|
||||||
const text = await file.text();
|
|
||||||
return JSON.parse(text);
|
|
||||||
} catch {
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 从 OPFS 加载已有表到内存缓存。
|
|
||||||
* v0.4.2-fix: 优先从持久化 schema(__metona_schema_*.meta)恢复 —
|
|
||||||
* 空表不再消失、索引标记/主键/约束完整;无 schema 记录的旧库从数据推断(兼容)。
|
|
||||||
*/
|
|
||||||
private async loadExistingTables(): Promise<void> {
|
|
||||||
if (!this.tablesDir) return;
|
|
||||||
const dir = this.tablesDir as any;
|
|
||||||
const fileNames: string[] = [];
|
|
||||||
for await (const [name] of dir.entries()) {
|
|
||||||
if (name.endsWith('.json')) fileNames.push(name);
|
|
||||||
}
|
|
||||||
|
|
||||||
for (const fileName of fileNames) {
|
|
||||||
const tableName = fileName.replace('.json', '');
|
|
||||||
try {
|
|
||||||
// 1. 优先:持久化 schema
|
|
||||||
const schemaRaw = await this.getMeta(`schema_${tableName}`);
|
|
||||||
if (schemaRaw) {
|
|
||||||
const schema = JSON.parse(schemaRaw) as TableSchema;
|
|
||||||
await this.memoryCache.createTable(schema);
|
|
||||||
const rows = await this.readTableData(tableName);
|
|
||||||
for (const row of rows) {
|
|
||||||
try {
|
|
||||||
await this.memoryCache.insert(tableName, [row]);
|
|
||||||
} catch {
|
|
||||||
// 单行损坏不影响整表恢复
|
|
||||||
}
|
|
||||||
}
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// 2. 兼容旧库:从数据推断 schema(空表且无 schema 记录 → 跳过)
|
|
||||||
const data = await this.readTableData(tableName);
|
|
||||||
if (data.length > 0) {
|
|
||||||
const firstRow = data[0];
|
|
||||||
const columns: Record<string, any> = {};
|
|
||||||
for (const key of Object.keys(firstRow)) {
|
|
||||||
const val = firstRow[key];
|
|
||||||
const type = typeof val === 'number' ? 'number' :
|
|
||||||
typeof val === 'boolean' ? 'boolean' :
|
|
||||||
typeof val === 'object' ? 'json' : 'string';
|
|
||||||
columns[key] = { type, primaryKey: key === 'id' };
|
|
||||||
}
|
|
||||||
await this.memoryCache.createTable({ name: tableName, columns });
|
|
||||||
for (const row of data) {
|
|
||||||
await this.memoryCache.insert(tableName, [row]);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
// 单个文件损坏不影响其他表
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+10
-4
@@ -12,8 +12,7 @@ import type { IStorageEngine } from '../engine/interface';
|
|||||||
import type { QueryPlan, TableSchema, DiskEngine } from '../constants';
|
import type { QueryPlan, TableSchema, DiskEngine } from '../constants';
|
||||||
import { DatabaseError } from '../constants';
|
import { DatabaseError } from '../constants';
|
||||||
import { MemoryEngine } from '../engine/memory';
|
import { MemoryEngine } from '../engine/memory';
|
||||||
import { IndexedDBEngine } from '../engine/indexeddb';
|
import { KVStoreEngine } from '../engine/kvstore_engine';
|
||||||
import { OPFSEngine } from '../engine/opfs';
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// HybridEngine
|
// HybridEngine
|
||||||
@@ -28,10 +27,11 @@ export class HybridEngine implements IStorageEngine {
|
|||||||
private dbName = '';
|
private dbName = '';
|
||||||
private version = 1;
|
private version = 1;
|
||||||
|
|
||||||
constructor(diskEngine: DiskEngine = 'indexeddb') {
|
constructor(diskEngine: DiskEngine = 'opfs') {
|
||||||
this.memoryEngine = new MemoryEngine();
|
this.memoryEngine = new MemoryEngine();
|
||||||
this.diskEngineType = diskEngine;
|
this.diskEngineType = diskEngine;
|
||||||
this.diskEngine = diskEngine === 'opfs' ? new OPFSEngine() : new IndexedDBEngine();
|
// v0.6.0: 磁盘层统一为自研 KVStoreEngine
|
||||||
|
this.diskEngine = new KVStoreEngine();
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---- 生命周期 ----
|
// ---- 生命周期 ----
|
||||||
@@ -57,6 +57,12 @@ export class HybridEngine implements IStorageEngine {
|
|||||||
await this.memoryEngine.close();
|
await this.memoryEngine.close();
|
||||||
await this.memoryEngine.open(this.dbName, this.version);
|
await this.memoryEngine.open(this.dbName, this.version);
|
||||||
|
|
||||||
|
// v0.6.0: KVStoreEngine 读内存 → 先重载磁盘最新数据
|
||||||
|
const disk = this.diskEngine as IStorageEngine & { reload?: () => Promise<void> };
|
||||||
|
if (typeof disk.reload === 'function') {
|
||||||
|
await disk.reload();
|
||||||
|
}
|
||||||
|
|
||||||
const tableNames = await this.diskEngine.getTableNames();
|
const tableNames = await this.diskEngine.getTableNames();
|
||||||
for (const tableName of tableNames) {
|
for (const tableName of tableNames) {
|
||||||
const schema = await this.diskEngine.getTableSchema(tableName);
|
const schema = await this.diskEngine.getTableSchema(tableName);
|
||||||
|
|||||||
+1
-2
@@ -78,8 +78,7 @@ export type { DatabaseConfig, TableSchema, ColumnDef, FieldType, StorageMode, Di
|
|||||||
export type { IStorageEngine } from './engine/interface';
|
export type { IStorageEngine } from './engine/interface';
|
||||||
export type { Statement, SelectStatement, InsertStatement, UpdateStatement, DeleteStatement } from './query/ast';
|
export type { Statement, SelectStatement, InsertStatement, UpdateStatement, DeleteStatement } from './query/ast';
|
||||||
export { MemoryEngine } from './engine/memory';
|
export { MemoryEngine } from './engine/memory';
|
||||||
export { IndexedDBEngine } from './engine/indexeddb';
|
export { KVStoreEngine } from './engine/kvstore_engine';
|
||||||
export { OPFSEngine } from './engine/opfs';
|
|
||||||
export { AriaEngine } from './engine/aria/index';
|
export { AriaEngine } from './engine/aria/index';
|
||||||
export { HybridEngine } from './hybrid/index';
|
export { HybridEngine } from './hybrid/index';
|
||||||
export { Table } from './table/table';
|
export { Table } from './table/table';
|
||||||
|
|||||||
@@ -0,0 +1,170 @@
|
|||||||
|
/**
|
||||||
|
* migrateFromIndexedDB — 旧 IndexedDB 数据迁移到自研 KV 引擎
|
||||||
|
* @module migration/index
|
||||||
|
*
|
||||||
|
* v0.6.0: IndexedDB 从引擎中完全移除后,提供一次性迁移工具把旧库数据
|
||||||
|
* 导入新引擎(KVStoreEngine disk 模式 / AriaEngine)。
|
||||||
|
*
|
||||||
|
* 旧库命名:
|
||||||
|
* - disk 模式(IndexedDBEngine):库名 = dbName
|
||||||
|
* - aria 模式(IndexedDBBackend):库名 = `aria-${dbName}`
|
||||||
|
*
|
||||||
|
* 仅此模块保留原生 IndexedDB 读取代码(一次性迁移用途,不参与运行时)。
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { MetonaSqlark } from '../core';
|
||||||
|
import type { TableSchema, ColumnDef, FieldType } from '../constants';
|
||||||
|
|
||||||
|
export interface MigrationOptions {
|
||||||
|
/** 旧库名(业务名,不含 aria- 前缀) */
|
||||||
|
dbName: string;
|
||||||
|
/**
|
||||||
|
* 旧引擎类型:仅支持 disk(IndexedDBEngine,每表一个 objectStore,行数据可直接读取)。
|
||||||
|
* aria 旧库(IndexedDBBackend)数据为引擎私有格式(SSTable/WAL),无法按行迁移。
|
||||||
|
*/
|
||||||
|
engine: 'disk';
|
||||||
|
/** 目标数据库实例(已初始化,新引擎) */
|
||||||
|
target: MetonaSqlark;
|
||||||
|
/** 进度回调 */
|
||||||
|
onProgress?: (done: number, total: number, table?: string) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MigrationResult {
|
||||||
|
/** 已迁移的表 */
|
||||||
|
migratedTables: string[];
|
||||||
|
/** 迁移的行总数 */
|
||||||
|
rowCount: number;
|
||||||
|
/** 跳过(无 schema 且无数据)的表 */
|
||||||
|
skippedTables: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 旧库中持久化 schema 的 store 名(IndexedDBEngine v0.3.2+) */
|
||||||
|
const SCHEMA_STORE = '__metona_schema';
|
||||||
|
|
||||||
|
function inferFieldType(value: unknown): FieldType {
|
||||||
|
if (typeof value === 'number') return 'number';
|
||||||
|
if (typeof value === 'boolean') return 'boolean';
|
||||||
|
if (typeof value === 'object' && value !== null) return 'json';
|
||||||
|
return 'string';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 从样例行推断 schema(旧库无持久化 schema 时回退) */
|
||||||
|
function inferSchema(tableName: string, rows: Record<string, unknown>[]): TableSchema {
|
||||||
|
const columns: Record<string, ColumnDef> = {};
|
||||||
|
if (rows.length === 0) return { name: tableName, columns };
|
||||||
|
const first = rows[0];
|
||||||
|
for (const key of Object.keys(first)) {
|
||||||
|
columns[key] = {
|
||||||
|
type: inferFieldType(first[key]),
|
||||||
|
primaryKey: key === 'id',
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return { name: tableName, columns };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 打开旧 IndexedDB 库(只读) */
|
||||||
|
function openLegacyDB(idbName: string): Promise<IDBDatabase> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(idbName);
|
||||||
|
request.onsuccess = () => resolve(request.result);
|
||||||
|
request.onerror = () => reject(request.error ?? new Error(`Failed to open legacy IndexedDB "${idbName}"`));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 读取 object store 全部记录 */
|
||||||
|
function readAllRecords(store: IDBObjectStore): Promise<Record<string, unknown>[]> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const req = store.getAll();
|
||||||
|
req.onsuccess = () => resolve((req.result ?? []) as Record<string, unknown>[]);
|
||||||
|
req.onerror = () => reject(req.error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 读取持久化 schema 记录 */
|
||||||
|
function readSchemas(db: IDBDatabase): Promise<Record<string, TableSchema>> {
|
||||||
|
if (!db.objectStoreNames.contains(SCHEMA_STORE)) {
|
||||||
|
return Promise.resolve({});
|
||||||
|
}
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const req = db.transaction(SCHEMA_STORE, 'readonly').objectStore(SCHEMA_STORE).getAll();
|
||||||
|
req.onsuccess = () => {
|
||||||
|
const result: Record<string, TableSchema> = {};
|
||||||
|
for (const rec of (req.result ?? []) as { name: string; schema?: string }[]) {
|
||||||
|
if (!rec.schema) continue;
|
||||||
|
try {
|
||||||
|
const schema = JSON.parse(rec.schema) as TableSchema;
|
||||||
|
result[schema.name] = schema;
|
||||||
|
} catch { /* 损坏记录跳过 */ }
|
||||||
|
}
|
||||||
|
resolve(result);
|
||||||
|
};
|
||||||
|
req.onerror = () => reject(req.error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 将旧 IndexedDB 库迁移到目标引擎。
|
||||||
|
* @returns 迁移结果(表/行数统计)
|
||||||
|
*/
|
||||||
|
export async function migrateFromIndexedDB(opts: MigrationOptions): Promise<MigrationResult> {
|
||||||
|
// v0.6.0: aria 旧库为引擎私有格式(SSTable/WAL),不支持按行迁移(运行时防御)
|
||||||
|
if ((opts.engine as string) === 'aria') {
|
||||||
|
throw new Error(
|
||||||
|
'Migration from AriaEngine IndexedDB backend is not supported ' +
|
||||||
|
'(data is stored in engine-private SSTable/WAL format). ' +
|
||||||
|
'Only disk-mode IndexedDBEngine databases can be migrated.',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const idbName = opts.dbName;
|
||||||
|
|
||||||
|
let db: IDBDatabase | null = null;
|
||||||
|
try {
|
||||||
|
db = await openLegacyDB(idbName);
|
||||||
|
} catch (error) {
|
||||||
|
throw new Error(
|
||||||
|
`Legacy IndexedDB database "${idbName}" not found or unreadable: ${(error as Error).message}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const result: MigrationResult = { migratedTables: [], rowCount: 0, skippedTables: [] };
|
||||||
|
const schemas = await readSchemas(db);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const storeNames = Array.from(db.objectStoreNames).filter((n) => n !== SCHEMA_STORE);
|
||||||
|
for (let i = 0; i < storeNames.length; i++) {
|
||||||
|
const tableName = storeNames[i];
|
||||||
|
opts.onProgress?.(i, storeNames.length, tableName);
|
||||||
|
|
||||||
|
const rows = await readAllRecords(db.transaction(tableName, 'readonly').objectStore(tableName));
|
||||||
|
|
||||||
|
// 表已存在于目标库 → 跳过(避免覆盖)
|
||||||
|
const names = await opts.target.getTableNames();
|
||||||
|
if (names.includes(tableName)) {
|
||||||
|
result.skippedTables.push(tableName);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// schema:持久化优先,否则从数据推断(空表且无 schema → 跳过)
|
||||||
|
let schema = schemas[tableName];
|
||||||
|
if (!schema) {
|
||||||
|
if (rows.length === 0) {
|
||||||
|
result.skippedTables.push(tableName);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
schema = inferSchema(tableName, rows);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 写入目标引擎
|
||||||
|
await opts.target.defineTable(tableName, schema.columns);
|
||||||
|
if (rows.length > 0) {
|
||||||
|
await opts.target.table(tableName).insertMany(rows);
|
||||||
|
}
|
||||||
|
result.migratedTables.push(tableName);
|
||||||
|
result.rowCount += rows.length;
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
db.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
@@ -2,11 +2,14 @@
|
|||||||
* v0.4.1 测试 — AriaEngine 外键级联 + clearAll 重置
|
* v0.4.1 测试 — AriaEngine 外键级联 + clearAll 重置
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { AriaEngine } from '../src/engine/aria/index';
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
const mkEngine = async (name: string): Promise<AriaEngine> => {
|
const mkEngine = async (name: string): Promise<AriaEngine> => {
|
||||||
const e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(`cascade-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, 1);
|
await e.open(`cascade-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, 1);
|
||||||
return e;
|
return e;
|
||||||
};
|
};
|
||||||
@@ -126,7 +129,7 @@ describe('[v0.4.1] AriaEngine clearAll', () => {
|
|||||||
|
|
||||||
test('clearAll 后重启(模拟刷新)无残留数据', async () => {
|
test('clearAll 后重启(模拟刷新)无残留数据', async () => {
|
||||||
const name = `clr2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
const name = `clr2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
let e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
let e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
await e.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
|
await e.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
|
||||||
await e.insert('t', [{ id: '1' }]);
|
await e.insert('t', [{ id: '1' }]);
|
||||||
@@ -134,7 +137,7 @@ describe('[v0.4.1] AriaEngine clearAll', () => {
|
|||||||
await e.close();
|
await e.close();
|
||||||
|
|
||||||
// 重新打开(模拟页面刷新):不应有残留表
|
// 重新打开(模拟页面刷新):不应有残留表
|
||||||
e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
expect(await e.getTableNames()).toHaveLength(0);
|
expect(await e.getTableNames()).toHaveLength(0);
|
||||||
await e.close();
|
await e.close();
|
||||||
@@ -159,7 +162,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
|
|||||||
|
|
||||||
test('ALTER 持久化:重启后 schema 与行一致', async () => {
|
test('ALTER 持久化:重启后 schema 与行一致', async () => {
|
||||||
const name = `alt2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
const name = `alt2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
let e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
let e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
||||||
await e.insert('users', [{ id: '1', name: 'Alice' }]);
|
await e.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
@@ -168,7 +171,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
|
|||||||
await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' });
|
await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' });
|
||||||
await e.close();
|
await e.close();
|
||||||
|
|
||||||
e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
const schema = await e.getTableSchema('users');
|
const schema = await e.getTableSchema('users');
|
||||||
expect(Object.keys(schema!.columns)).not.toContain('phone');
|
expect(Object.keys(schema!.columns)).not.toContain('phone');
|
||||||
@@ -179,7 +182,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
|
|||||||
|
|
||||||
test('ALTER 模拟崩溃(不 close)重启:schema 与行一致', async () => {
|
test('ALTER 模拟崩溃(不 close)重启:schema 与行一致', async () => {
|
||||||
const name = `alt3-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
const name = `alt3-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
let e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
let e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
||||||
await e.insert('users', [{ id: '1', name: 'Alice' }]);
|
await e.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
@@ -188,7 +191,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
|
|||||||
await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' });
|
await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' });
|
||||||
// 不 close,模拟崩溃
|
// 不 close,模拟崩溃
|
||||||
|
|
||||||
e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
const schema = await e.getTableSchema('users');
|
const schema = await e.getTableSchema('users');
|
||||||
expect(Object.keys(schema!.columns)).not.toContain('phone');
|
expect(Object.keys(schema!.columns)).not.toContain('phone');
|
||||||
|
|||||||
+2
-5
@@ -3,7 +3,6 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
|
||||||
let diskCounter = 0;
|
let diskCounter = 0;
|
||||||
|
|
||||||
@@ -215,13 +214,12 @@ describe('MetonaSqlark Disk 模式', () => {
|
|||||||
|
|
||||||
beforeEach(async () => {
|
beforeEach(async () => {
|
||||||
dbName = `test-disk-${++diskCounter}`;
|
dbName = `test-disk-${++diskCounter}`;
|
||||||
db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' });
|
db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
});
|
});
|
||||||
|
|
||||||
afterEach(async () => {
|
afterEach(async () => {
|
||||||
await db.close();
|
await db.close();
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('创建 disk 模式数据库', () => {
|
it('创建 disk 模式数据库', () => {
|
||||||
@@ -250,13 +248,12 @@ describe('MetonaSqlark Hybrid 模式', () => {
|
|||||||
|
|
||||||
beforeEach(async () => {
|
beforeEach(async () => {
|
||||||
dbName = `test-hybrid-core-${++diskCounter}`;
|
dbName = `test-hybrid-core-${++diskCounter}`;
|
||||||
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
});
|
});
|
||||||
|
|
||||||
afterEach(async () => {
|
afterEach(async () => {
|
||||||
await db.close();
|
await db.close();
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('创建 hybrid 模式数据库', () => {
|
it('创建 hybrid 模式数据库', () => {
|
||||||
|
|||||||
@@ -119,6 +119,74 @@ test.describe('OPFS 真实环境', () => {
|
|||||||
await page2.close();
|
await page2.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('写入进行中崩溃(不等待完成)→ 重开数据不损坏且可恢复已确认写入', async ({ page }) => {
|
||||||
|
await openPage(page);
|
||||||
|
await run(page, 'open', {
|
||||||
|
name: 'e2e-crash-mid', diskEngine: 'opfs', checkpointInterval: 999999999,
|
||||||
|
});
|
||||||
|
await run(page, 'createTable', {
|
||||||
|
table: 't', columns: [{ name: 'id', type: 'string', primaryKey: true }],
|
||||||
|
});
|
||||||
|
// 前 20 条确认完成
|
||||||
|
for (let i = 0; i < 20; i++) {
|
||||||
|
await run(page, 'insert', { table: 't', rows: [{ id: `c-${i}` }] });
|
||||||
|
}
|
||||||
|
// 发起一批写入后立即崩溃(不等 Promise 完成)
|
||||||
|
const pending = page.evaluate(async () => {
|
||||||
|
const ms = (window as unknown as { __ms: { insert: (o: unknown) => Promise<unknown> } }).__ms;
|
||||||
|
const rows = [];
|
||||||
|
for (let i = 0; i < 30; i++) rows.push({ id: `p-${i}` });
|
||||||
|
await ms.insert({ table: 't', rows });
|
||||||
|
});
|
||||||
|
await new Promise((r) => setTimeout(r, 50));
|
||||||
|
await crashPage(page);
|
||||||
|
await pending.catch(() => {});
|
||||||
|
|
||||||
|
// 重开:库可打开,已确认 20 条完整;写入中数据要么全部可见要么部分(日志尾部截断)
|
||||||
|
const page2 = await page.context().newPage();
|
||||||
|
await openPage(page2);
|
||||||
|
await run(page2, 'open', {
|
||||||
|
name: 'e2e-crash-mid', diskEngine: 'opfs', checkpointInterval: 999999999,
|
||||||
|
});
|
||||||
|
const res = await run<{ count: number }>(page2, 'count', { table: 't' });
|
||||||
|
expect(res.count).toBeGreaterThanOrEqual(20);
|
||||||
|
// 已确认的 20 条必须完整
|
||||||
|
for (let i = 0; i < 20; i++) {
|
||||||
|
const found = await run<{ rows: Record<string, unknown>[] }>(page2, 'find', {
|
||||||
|
table: 't', where: { id: `c-${i}` },
|
||||||
|
});
|
||||||
|
expect(found.rows).toHaveLength(1);
|
||||||
|
}
|
||||||
|
await run(page2, 'close');
|
||||||
|
await page2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('checkpoint 前后崩溃 → 快照/日志双路径恢复一致', async ({ page }) => {
|
||||||
|
await openPage(page);
|
||||||
|
await run(page, 'open', { name: 'e2e-crash-cp', diskEngine: 'opfs', checkpointInterval: 999999999 });
|
||||||
|
await run(page, 'createTable', {
|
||||||
|
table: 't', columns: [{ name: 'id', type: 'string', primaryKey: true }],
|
||||||
|
});
|
||||||
|
for (let i = 0; i < 10; i++) {
|
||||||
|
await run(page, 'insert', { table: 't', rows: [{ id: `a-${i}` }] });
|
||||||
|
}
|
||||||
|
// 触发 checkpoint(强制落盘)
|
||||||
|
await run(page, 'repair'); // repair 内含 flush+checkpoint
|
||||||
|
for (let i = 0; i < 10; i++) {
|
||||||
|
await run(page, 'insert', { table: 't', rows: [{ id: `b-${i}` }] });
|
||||||
|
}
|
||||||
|
// 崩溃(checkpoint 后数据在快照,后续在日志)
|
||||||
|
await crashPage(page);
|
||||||
|
|
||||||
|
const page2 = await page.context().newPage();
|
||||||
|
await openPage(page2);
|
||||||
|
await run(page2, 'open', { name: 'e2e-crash-cp', diskEngine: 'opfs', checkpointInterval: 999999999 });
|
||||||
|
const res = await run<{ count: number }>(page2, 'count', { table: 't' });
|
||||||
|
expect(res.count).toBe(20);
|
||||||
|
await run(page2, 'close');
|
||||||
|
await page2.close();
|
||||||
|
});
|
||||||
|
|
||||||
test('多标签页锁:第二个标签页打开同一库 → ARIA_LOCKED', async ({ page }) => {
|
test('多标签页锁:第二个标签页打开同一库 → ARIA_LOCKED', async ({ page }) => {
|
||||||
await openPage(page);
|
await openPage(page);
|
||||||
await run(page, 'open', { name: 'e2e-lock', diskEngine: 'opfs' });
|
await run(page, 'open', { name: 'e2e-lock', diskEngine: 'opfs' });
|
||||||
|
|||||||
@@ -10,6 +10,10 @@ import { BufferPool } from '../../src/engine/aria/buffer/pool';
|
|||||||
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — Bloom + WAL + BufferPool', () => {
|
describe('AriaEngine — Bloom + WAL + BufferPool', () => {
|
||||||
// ---- BloomFilter 完整测试 ----
|
// ---- BloomFilter 完整测试 ----
|
||||||
describe('BloomFilter', () => {
|
describe('BloomFilter', () => {
|
||||||
|
|||||||
@@ -5,6 +5,10 @@ import { AriaEngine } from '../../src/engine/aria/index';
|
|||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — 批量扩展测试', () => {
|
describe('AriaEngine — 批量扩展测试', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('bat', 1); });
|
beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('bat', 1); });
|
||||||
|
|||||||
@@ -10,6 +10,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
/** 构造小缓存 + 小 MemTable 阈值的引擎,快速产生多个 SSTable */
|
/** 构造小缓存 + 小 MemTable 阈值的引擎,快速产生多个 SSTable */
|
||||||
function createSmallCacheEngine(bufferPoolPages = 2) {
|
function createSmallCacheEngine(bufferPoolPages = 2) {
|
||||||
return new AriaEngine({
|
return new AriaEngine({
|
||||||
|
|||||||
@@ -8,7 +8,10 @@
|
|||||||
*/
|
*/
|
||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -21,7 +24,7 @@ const SCHEMA = () => createSchema('users', {
|
|||||||
age: { type: 'number' },
|
age: { type: 'number' },
|
||||||
});
|
});
|
||||||
|
|
||||||
/** 篡改 MemoryBackend 中指定 key 的一个字节 */
|
/** 篡改指定 pg_ 页面文件的一个字节 */
|
||||||
async function corruptKey(engine: AriaEngine, key: string, byteOffset: number): Promise<void> {
|
async function corruptKey(engine: AriaEngine, key: string, byteOffset: number): Promise<void> {
|
||||||
const backend = (engine as any).backend;
|
const backend = (engine as any).backend;
|
||||||
const raw = await backend.read(key);
|
const raw = await backend.read(key);
|
||||||
@@ -31,16 +34,24 @@ async function corruptKey(engine: AriaEngine, key: string, byteOffset: number):
|
|||||||
await backend.write(key, buf.buffer as ArrayBuffer);
|
await backend.write(key, buf.buffer as ArrayBuffer);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 列出 backend 中所有 sst_ 前缀 key */
|
/** 列出全部 SSTable 的页面文件 key(页面化存储:pg_ 前缀) */
|
||||||
async function listSSTKeys(engine: AriaEngine): Promise<string[]> {
|
async function listSSTKeys(engine: AriaEngine): Promise<string[]> {
|
||||||
const backend = (engine as any).backend;
|
const backend = (engine as any).backend;
|
||||||
const keys = await backend.listKeys();
|
const keys = (await backend.listKeys()) as string[];
|
||||||
return (keys as string[]).filter((k) => k.startsWith('sst_'));
|
return keys.filter((k) => k.startsWith('pg_'));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 读取主 LSM 的 SSTable meta 列表 */
|
||||||
|
async function listSSTMetas(engine: AriaEngine): Promise<{ id: number; pageIds?: number[] }[]> {
|
||||||
|
const backend = (engine as any).backend;
|
||||||
|
const raw = await backend.read('__aria_lsm_meta');
|
||||||
|
if (!raw) return [];
|
||||||
|
return JSON.parse(new TextDecoder().decode(raw)) as { id: number; pageIds?: number[] }[];
|
||||||
}
|
}
|
||||||
|
|
||||||
describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
||||||
it('打开时 CRC 损坏的 SSTable 被清理,其余数据可读', async () => {
|
it('打开时 CRC 损坏的 SSTable 被清理,其余数据可读', async () => {
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
@@ -57,7 +68,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// 重新打开,篡改第一个 SSTable 文件的数据区
|
// 重新打开,篡改第一个 SSTable 文件的数据区
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
const sstKeys = await listSSTKeys(engine2);
|
const sstKeys = await listSSTKeys(engine2);
|
||||||
expect(sstKeys.length).toBeGreaterThanOrEqual(2);
|
expect(sstKeys.length).toBeGreaterThanOrEqual(2);
|
||||||
@@ -65,7 +76,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
|
|
||||||
// 再次打开:损坏文件应被跳过并清理,打开不抛错
|
// 再次打开:损坏文件应被跳过并清理,打开不抛错
|
||||||
await engine2.close();
|
await engine2.close();
|
||||||
const engine3 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine3 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine3.open(dbName, 1);
|
await engine3.open(dbName, 1);
|
||||||
|
|
||||||
// 剩余未损坏文件的数据应可查询
|
// 剩余未损坏文件的数据应可查询
|
||||||
@@ -73,18 +84,20 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
expect(remaining.length).toBeGreaterThan(0);
|
expect(remaining.length).toBeGreaterThan(0);
|
||||||
expect(remaining.length).toBeLessThan(200);
|
expect(remaining.length).toBeLessThan(200);
|
||||||
|
|
||||||
// 损坏文件已被清理(meta 移除)
|
// 损坏文件已被清理(页面 + meta 移除)
|
||||||
const afterKeys = await listSSTKeys(engine3);
|
const afterKeys = await listSSTKeys(engine3);
|
||||||
expect(afterKeys).not.toContain(sstKeys[0]);
|
expect(afterKeys).not.toContain(sstKeys[0]);
|
||||||
const lsm = (engine3 as any).lsm;
|
const metas = await listSSTMetas(engine3);
|
||||||
const metas = await lsm.sstableStore.listMeta();
|
// 被篡改页面所属的 SSTable meta 应被移除
|
||||||
expect(metas.some((m: { id: number }) => m.id === Number(sstKeys[0].replace('sst_', '')))).toBe(false);
|
const victimMetas = await listSSTMetas(engine2);
|
||||||
|
const victimMeta = victimMetas.find((m) => m.pageIds?.includes(Number(sstKeys[0].slice(3))));
|
||||||
|
expect(metas.some((m) => m.id === victimMeta?.id)).toBe(false);
|
||||||
|
|
||||||
await engine3.close();
|
await engine3.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('打开时损坏全部 SSTable → 库仍可打开,数据为空但不崩溃', async () => {
|
it('打开时损坏全部 SSTable → 库仍可打开,数据为空但不崩溃', async () => {
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
@@ -98,14 +111,14 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
}
|
}
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
for (const key of await listSSTKeys(engine2)) {
|
for (const key of await listSSTKeys(engine2)) {
|
||||||
await corruptKey(engine2, key, 16);
|
await corruptKey(engine2, key, 16);
|
||||||
}
|
}
|
||||||
await engine2.close();
|
await engine2.close();
|
||||||
|
|
||||||
const engine3 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine3 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
// 不应抛 ARIA_OPEN_ERROR
|
// 不应抛 ARIA_OPEN_ERROR
|
||||||
await engine3.open(dbName, 1);
|
await engine3.open(dbName, 1);
|
||||||
const rows3 = await engine3.find('users', { table: 'users' });
|
const rows3 = await engine3.find('users', { table: 'users' });
|
||||||
@@ -114,7 +127,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('运行期 CRC 损坏 → 预加载不缓存损坏文件并清理(自愈)', async () => {
|
it('运行期 CRC 损坏 → 预加载不缓存损坏文件并清理(自愈)', async () => {
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
@@ -133,22 +146,23 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
const victim = sstKeys[sstKeys.length - 1]; // 篡改最新(memtable 已 flush 后的文件)
|
const victim = sstKeys[sstKeys.length - 1]; // 篡改最新(memtable 已 flush 后的文件)
|
||||||
await corruptKey(engine, victim, 100);
|
await corruptKey(engine, victim, 100);
|
||||||
|
|
||||||
// 缓存里已有该文件(flush 时缓存)→ 先清缓存模拟运行期重载
|
// 缓存里已有该文件(flush 时缓存)→ 清 LSM 层与 BufferPool 页面缓存模拟运行期磁盘损坏
|
||||||
const lsm = (engine as any).lsm;
|
const lsm = (engine as any).lsm;
|
||||||
lsm.sstableCache.delete(Number(victim.replace('sst_', '')));
|
lsm.sstableCache.clear();
|
||||||
|
await (engine as any).bufferPool.clear();
|
||||||
|
|
||||||
// 查询触发 prefetchRange → preloadSSTable 发现 CRC 失败 → 清理 + 不缓存
|
// 查询触发 prefetchRange → preloadSSTable 发现 CRC 失败 → 清理 + 不缓存
|
||||||
const remaining = await engine.find('users', { table: 'users' });
|
const remaining = await engine.find('users', { table: 'users' });
|
||||||
expect(remaining.length).toBeLessThan(100);
|
expect(remaining.length).toBeLessThan(100);
|
||||||
expect(lsm.sstableCache.has(Number(victim.replace('sst_', '')))).toBe(false);
|
expect(lsm.sstableCache.has(Number(victim.slice(3)))).toBe(false);
|
||||||
const metas = await lsm.sstableStore.listMeta();
|
const metas = await lsm.sstableStore.listMeta();
|
||||||
expect(metas.some((m: { id: number }) => m.id === Number(victim.replace('sst_', '')))).toBe(false);
|
expect(metas.some((m: { id: number }) => m.id === Number(victim.slice(3)))).toBe(false);
|
||||||
|
|
||||||
await engine.close();
|
await engine.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('旧版无校验文件(checksum=0)在 LSM 中正常加载', async () => {
|
it('旧版无校验文件(checksum=0)在 LSM 中正常加载', async () => {
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
@@ -159,7 +173,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
await (engine as any).lsm.flush();
|
await (engine as any).lsm.flush();
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
const sstKeys = await listSSTKeys(engine2);
|
const sstKeys = await listSSTKeys(engine2);
|
||||||
expect(sstKeys.length).toBe(1);
|
expect(sstKeys.length).toBe(1);
|
||||||
@@ -171,7 +185,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
|
|||||||
await engine2.close();
|
await engine2.close();
|
||||||
|
|
||||||
// 重开:checksum=0 跳过校验,数据完整可读
|
// 重开:checksum=0 跳过校验,数据完整可读
|
||||||
const engine3 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 });
|
const engine3 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine3.open(dbName, 1);
|
await engine3.open(dbName, 1);
|
||||||
const rows3 = await engine3.find('users', { table: 'users' });
|
const rows3 = await engine3.find('users', { table: 'users' });
|
||||||
expect(rows3.length).toBe(2);
|
expect(rows3.length).toBe(2);
|
||||||
|
|||||||
@@ -4,7 +4,6 @@
|
|||||||
*
|
*
|
||||||
* v0.2.6 补强:此前仅验证实例化,现在验证真实的加解密往返一致性。
|
* v0.2.6 补强:此前仅验证实例化,现在验证真实的加解密往返一致性。
|
||||||
*/
|
*/
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { CryptoManager } from '../../src/engine/aria/crypto';
|
import { CryptoManager } from '../../src/engine/aria/crypto';
|
||||||
|
|
||||||
function toBytes(data: ArrayBuffer): number[] {
|
function toBytes(data: ArrayBuffer): number[] {
|
||||||
|
|||||||
@@ -5,6 +5,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — 扩展边缘测试', () => {
|
describe('AriaEngine — 扩展边缘测试', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
|
|
||||||
|
|||||||
@@ -14,7 +14,10 @@ import { AriaEngine } from '../../src/engine/aria/index';
|
|||||||
import { EncryptedBackend } from '../../src/engine/aria/store/encrypted_backend';
|
import { EncryptedBackend } from '../../src/engine/aria/store/encrypted_backend';
|
||||||
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -182,7 +185,7 @@ describe('AriaEngine — EncryptedBackend 单元', () => {
|
|||||||
describe('AriaEngine — 全库加密(集成)', () => {
|
describe('AriaEngine — 全库加密(集成)', () => {
|
||||||
it('加密库:写入 → close → 同密码重开数据完整', async () => {
|
it('加密库:写入 → close → 同密码重开数据完整', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'master-pass' } });
|
const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'master-pass' } });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.insert('users', [
|
await engine.insert('users', [
|
||||||
@@ -193,7 +196,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// 重开(同密码)
|
// 重开(同密码)
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'master-pass' } });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'master-pass' } });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
const rows = await engine2.find('users', { table: 'users' });
|
const rows = await engine2.find('users', { table: 'users' });
|
||||||
expect(rows).toHaveLength(2);
|
expect(rows).toHaveLength(2);
|
||||||
@@ -206,42 +209,42 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
|
|
||||||
it('加密库:错误密码重开 → ARIA_DECRYPT_ERROR', async () => {
|
it('加密库:错误密码重开 → ARIA_DECRYPT_ERROR', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'right-pass' } });
|
const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'right-pass' } });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]);
|
await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]);
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'wrong-pass' } });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'wrong-pass' } });
|
||||||
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
|
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('加密库:无密码打开 → ARIA_ENCRYPT_REQUIRED', async () => {
|
it('加密库:无密码打开 → ARIA_ENCRYPT_REQUIRED', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } });
|
const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_REQUIRED' });
|
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_REQUIRED' });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('明文库:加密配置打开 → ARIA_ENCRYPT_CONFIG_ERROR', async () => {
|
it('明文库:加密配置打开 → ARIA_ENCRYPT_CONFIG_ERROR', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.insert('users', [{ id: '1', name: 'A', secret: 'plain' }]);
|
await engine.insert('users', [{ id: '1', name: 'A', secret: 'plain' }]);
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
|
||||||
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_CONFIG_ERROR' });
|
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_CONFIG_ERROR' });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('加密库:底层存储全部为密文(SSTable/Schema/WAL 均不可见明文)', async () => {
|
it('加密库:底层存储全部为密文(SSTable/Schema/WAL 均不可见明文)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } });
|
const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice', secret: 'needle-in-cipher' }]);
|
await engine.insert('users', [{ id: '1', name: 'Alice', secret: 'needle-in-cipher' }]);
|
||||||
@@ -267,7 +270,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
it('加密 + 压缩组合:往返完整(压缩层先压,加密层后加密)', async () => {
|
it('加密 + 压缩组合:往返完整(压缩层先压,加密层后加密)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
compression: true,
|
compression: true,
|
||||||
encryption: { password: 'pw-compress' },
|
encryption: { password: 'pw-compress' },
|
||||||
memtableSizeThreshold: 64 * 1024 * 1024,
|
memtableSizeThreshold: 64 * 1024 * 1024,
|
||||||
@@ -283,7 +286,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({
|
const engine2 = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
compression: true,
|
compression: true,
|
||||||
encryption: { password: 'pw-compress' },
|
encryption: { password: 'pw-compress' },
|
||||||
});
|
});
|
||||||
@@ -296,7 +299,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
|
|
||||||
it('加密库:clearAll 保留密钥(库身份),同密码重开重建,新密码被拒', async () => {
|
it('加密库:clearAll 保留密钥(库身份),同密码重开重建,新密码被拒', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'old-pw' } });
|
const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'old-pw' } });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]);
|
await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]);
|
||||||
@@ -304,11 +307,11 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// clearAll 保留 keymeta → 换新密码会被拒绝
|
// clearAll 保留 keymeta → 换新密码会被拒绝
|
||||||
const engineWrong = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'new-pw' } });
|
const engineWrong = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'new-pw' } });
|
||||||
await expect(engineWrong.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
|
await expect(engineWrong.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
|
||||||
|
|
||||||
// 同密码重开可重建
|
// 同密码重开可重建
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'old-pw' } });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'old-pw' } });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
await engine2.createTable(SCHEMA());
|
await engine2.createTable(SCHEMA());
|
||||||
await engine2.insert('users', [{ id: '1', name: 'A', secret: 'new' }]);
|
await engine2.insert('users', [{ id: '1', name: 'A', secret: 'new' }]);
|
||||||
@@ -319,7 +322,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
it('加密库:WAL 崩溃恢复路径可用(含加密 WAL)', async () => {
|
it('加密库:WAL 崩溃恢复路径可用(含加密 WAL)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
encryption: { password: 'pw' },
|
encryption: { password: 'pw' },
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
@@ -336,7 +339,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
|
|||||||
|
|
||||||
// 重开:WAL 重放(加密 WAL 解密后解析)
|
// 重开:WAL 重放(加密 WAL 解密后解析)
|
||||||
const engine2 = new AriaEngine({
|
const engine2 = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
encryption: { password: 'pw' },
|
encryption: { password: 'pw' },
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
|
|||||||
@@ -4,6 +4,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — 补充测试', () => {
|
describe('AriaEngine — 补充测试', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('sup', 1); });
|
beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('sup', 1); });
|
||||||
|
|||||||
@@ -7,6 +7,10 @@ import { createSchema } from '../../src/table/schema';
|
|||||||
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
||||||
import { checkFieldType } from '../../src/table/schema';
|
import { checkFieldType } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — 最终扩展测试', () => {
|
describe('AriaEngine — 最终扩展测试', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — 二级索引查询', () => {
|
describe('AriaEngine — 二级索引查询', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
|
|
||||||
|
|||||||
@@ -11,7 +11,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { DatabaseLock, lockName } from '../../src/engine/aria/locks';
|
import { DatabaseLock, lockName } from '../../src/engine/aria/locks';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -25,8 +28,12 @@ let lockHolders: Map<string, true>;
|
|||||||
|
|
||||||
function installLocksMock() {
|
function installLocksMock() {
|
||||||
lockHolders = new Map();
|
lockHolders = new Map();
|
||||||
|
// 保留 storage(OPFS mock 设置过),与 locks 共存
|
||||||
|
const existingNav = (globalThis as { navigator?: { storage?: unknown } }).navigator;
|
||||||
|
const storage = existingNav?.storage;
|
||||||
Object.defineProperty(globalThis, 'navigator', {
|
Object.defineProperty(globalThis, 'navigator', {
|
||||||
value: {
|
value: {
|
||||||
|
storage,
|
||||||
locks: {
|
locks: {
|
||||||
request: async (_name: string, opts: { ifAvailable?: boolean }, cb: (lock: { name: string } | null) => Promise<void> | void) => {
|
request: async (_name: string, opts: { ifAvailable?: boolean }, cb: (lock: { name: string } | null) => Promise<void> | void) => {
|
||||||
const name = _name;
|
const name = _name;
|
||||||
@@ -53,8 +60,10 @@ function installLocksMock() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function removeLocksMock() {
|
function removeLocksMock() {
|
||||||
|
// 只移除 locks,保留 storage(OPFS mock 需要)
|
||||||
|
const existingNav = (globalThis as { navigator?: { storage?: unknown } }).navigator;
|
||||||
Object.defineProperty(globalThis, 'navigator', {
|
Object.defineProperty(globalThis, 'navigator', {
|
||||||
value: {},
|
value: { storage: existingNav?.storage },
|
||||||
configurable: true,
|
configurable: true,
|
||||||
writable: true,
|
writable: true,
|
||||||
});
|
});
|
||||||
@@ -120,11 +129,11 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
|
|||||||
|
|
||||||
it('第二个标签页打开同一库 → ARIA_LOCKED', async () => {
|
it('第二个标签页打开同一库 → ARIA_LOCKED', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine1 = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine1 = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await engine1.open(dbName, 1);
|
await engine1.open(dbName, 1);
|
||||||
await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_LOCKED' });
|
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_LOCKED' });
|
||||||
// engine2 未持锁、未 opened
|
// engine2 未持锁、未 opened
|
||||||
expect((engine2 as any).opened).toBe(false);
|
expect((engine2 as any).opened).toBe(false);
|
||||||
@@ -132,7 +141,7 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
|
|||||||
await engine1.close();
|
await engine1.close();
|
||||||
|
|
||||||
// 释放后可打开
|
// 释放后可打开
|
||||||
const engine3 = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine3 = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await engine3.open(dbName, 1);
|
await engine3.open(dbName, 1);
|
||||||
expect(await engine3.count('t')).toBe(0);
|
expect(await engine3.count('t')).toBe(0);
|
||||||
await engine3.close();
|
await engine3.close();
|
||||||
@@ -140,19 +149,19 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
|
|||||||
|
|
||||||
it('open 失败(错误密码)→ 锁释放,其他标签页可打开', async () => {
|
it('open 失败(错误密码)→ 锁释放,其他标签页可打开', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine1 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } });
|
const engine1 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
|
||||||
await engine1.open(dbName, 1);
|
await engine1.open(dbName, 1);
|
||||||
await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
await engine1.close();
|
await engine1.close();
|
||||||
|
|
||||||
// 错误密码 → open 失败
|
// 错误密码 → open 失败
|
||||||
const engineBad = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'wrong' } });
|
const engineBad = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'wrong' } });
|
||||||
await expect(engineBad.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
|
await expect(engineBad.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
|
||||||
// 失败后锁已释放(未泄漏)
|
// 失败后锁已释放(未泄漏)
|
||||||
expect(lockHolders.has(lockName(dbName))).toBe(false);
|
expect(lockHolders.has(lockName(dbName))).toBe(false);
|
||||||
|
|
||||||
// 正确密码可打开
|
// 正确密码可打开
|
||||||
const engineOk = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } });
|
const engineOk = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
|
||||||
await engineOk.open(dbName, 1);
|
await engineOk.open(dbName, 1);
|
||||||
await engineOk.close();
|
await engineOk.close();
|
||||||
});
|
});
|
||||||
@@ -160,7 +169,7 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
|
|||||||
it('无 Web Locks 环境 → 降级打开(不抛错)', async () => {
|
it('无 Web Locks 环境 → 降级打开(不抛错)', async () => {
|
||||||
removeLocksMock();
|
removeLocksMock();
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
await engine.insert('t', [{ id: '1' }]);
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
|
|||||||
@@ -7,6 +7,10 @@ import { createSchema } from '../../src/table/schema';
|
|||||||
import { QueryExecutor } from '../../src/query/executor';
|
import { QueryExecutor } from '../../src/query/executor';
|
||||||
import { parse } from '../../src/sql/parser';
|
import { parse } from '../../src/sql/parser';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — ANALYZE/VACUUM/REINDEX + EXPLAIN', () => {
|
describe('AriaEngine — ANALYZE/VACUUM/REINDEX + EXPLAIN', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
describe('AriaEngine — MVCC 事务 + Savepoint', () => {
|
describe('AriaEngine — MVCC 事务 + Savepoint', () => {
|
||||||
let engine: AriaEngine;
|
let engine: AriaEngine;
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,8 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import { installOPFSMock } from '../helpers/opfs-mock';
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -32,7 +33,7 @@ describe('AriaEngine — repair 自愈增强', () => {
|
|||||||
it('清理孤儿页面(meta 未引用的 pg_ 文件)', async () => {
|
it('清理孤儿页面(meta 未引用的 pg_ 文件)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
pageStorage: true, // 强制页面化(IDB 上也启用,便于断言页面文件)
|
pageStorage: true, // 强制页面化(IDB 上也启用,便于断言页面文件)
|
||||||
memtableSizeThreshold: 64 * 1024 * 1024,
|
memtableSizeThreshold: 64 * 1024 * 1024,
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
@@ -86,7 +87,7 @@ describe('AriaEngine — repair 自愈增强', () => {
|
|||||||
|
|
||||||
it('WAL 空洞 + repair → 截断清空(不再重放错位数据)', async () => {
|
it('WAL 空洞 + repair → 截断清空(不再重放错位数据)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
// 数据落盘 + checkpoint 清 WAL
|
// 数据落盘 + checkpoint 清 WAL
|
||||||
@@ -116,7 +117,7 @@ describe('AriaEngine — repair 自愈增强', () => {
|
|||||||
|
|
||||||
it('repair 幂等(连续调用无副作用)', async () => {
|
it('repair 幂等(连续调用无副作用)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(SCHEMA());
|
await engine.createTable(SCHEMA());
|
||||||
await engine.insert('items', [{ id: 'a', val: 1, tag: 'x' }]);
|
await engine.insert('items', [{ id: 'a', val: 1, tag: 'x' }]);
|
||||||
@@ -135,7 +136,7 @@ describe('AriaEngine — 随机操作压力 + 模拟崩溃', () => {
|
|||||||
it('500 随机操作(insert/update/delete)→ 模拟崩溃 → 重开验证全部已确认写入', async () => {
|
it('500 随机操作(insert/update/delete)→ 模拟崩溃 → 重开验证全部已确认写入', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
memtableSizeThreshold: 8 * 1024,
|
memtableSizeThreshold: 8 * 1024,
|
||||||
checkpointInterval: 50,
|
checkpointInterval: 50,
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
@@ -178,7 +179,7 @@ describe('AriaEngine — 随机操作压力 + 模拟崩溃', () => {
|
|||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
// 重开:WAL 重放 + SSTable 加载
|
// 重开:WAL 重放 + SSTable 加载
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 50, walSyncMode: 'full' });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 50, walSyncMode: 'full' });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
const rows = await engine2.find('items', { table: 'items' });
|
const rows = await engine2.find('items', { table: 'items' });
|
||||||
const byId = new Map(rows.map((r) => [r.id, r]));
|
const byId = new Map(rows.map((r) => [r.id, r]));
|
||||||
|
|||||||
@@ -13,7 +13,10 @@ import { SegmentedWALStore } from '../../src/engine/aria/wal/segmented_store';
|
|||||||
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
import { MemoryBackend } from '../../src/engine/aria/store/backend';
|
||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -153,7 +156,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
it('高频写入(多条 WAL 记录)→ close → reopen 数据完整', async () => {
|
it('高频写入(多条 WAL 记录)→ close → reopen 数据完整', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
});
|
});
|
||||||
@@ -171,7 +174,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
const engine2 = new AriaEngine({
|
const engine2 = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
});
|
});
|
||||||
@@ -185,7 +188,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
it('分片结构落盘验证(backend 中是分片文件而非旧单记录键)', async () => {
|
it('分片结构落盘验证(backend 中是分片文件而非旧单记录键)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
});
|
});
|
||||||
@@ -208,7 +211,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
it('WAL 分片部分残留(空洞)→ 重开恢复已落盘数据,不丢已 checkpoint 数据', async () => {
|
it('WAL 分片部分残留(空洞)→ 重开恢复已落盘数据,不丢已 checkpoint 数据', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
});
|
});
|
||||||
@@ -228,7 +231,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
await backend.delete('__wal_000000.bin');
|
await backend.delete('__wal_000000.bin');
|
||||||
|
|
||||||
await engine.close();
|
await engine.close();
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
// 已落盘的 a/b 必须保留;c/d 在 WAL 中且文件被删 → 恢复不到(可接受的保守丢弃)
|
// 已落盘的 a/b 必须保留;c/d 在 WAL 中且文件被删 → 恢复不到(可接受的保守丢弃)
|
||||||
const rows = await engine2.find('t', { table: 't' });
|
const rows = await engine2.find('t', { table: 't' });
|
||||||
@@ -241,7 +244,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
// 手工构造旧格式 WAL 库:直接写旧键(模拟 v0.4.4 库崩溃现场)
|
// 手工构造旧格式 WAL 库:直接写旧键(模拟 v0.4.4 库崩溃现场)
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
walSyncMode: 'full',
|
walSyncMode: 'full',
|
||||||
});
|
});
|
||||||
@@ -262,7 +265,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
|
|||||||
await (engine as any).backend.close();
|
await (engine as any).backend.close();
|
||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
const rows = await engine2.find('t', { table: 't' });
|
const rows = await engine2.find('t', { table: 't' });
|
||||||
expect(rows).toHaveLength(1);
|
expect(rows).toHaveLength(1);
|
||||||
|
|||||||
@@ -8,7 +8,10 @@
|
|||||||
import { AriaEngine } from '../../src/engine/aria/index';
|
import { AriaEngine } from '../../src/engine/aria/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import { MetonaSqlark } from '../../src/core';
|
import { MetonaSqlark } from '../../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from '../helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
// AriaEngine 引擎级测试 (Memory Backend)
|
// AriaEngine 引擎级测试 (Memory Backend)
|
||||||
@@ -549,7 +552,7 @@ describe('MetonaSqlark with mode=aria (Memory)', () => {
|
|||||||
let db: MetonaSqlark;
|
let db: MetonaSqlark;
|
||||||
|
|
||||||
beforeEach(async () => {
|
beforeEach(async () => {
|
||||||
db = new MetonaSqlark({ name: `ms-aria-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'indexeddb' });
|
db = new MetonaSqlark({ name: `ms-aria-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -1,323 +0,0 @@
|
|||||||
/**
|
|
||||||
* IndexedDBEngine 完整测试(使用 fake-indexeddb)
|
|
||||||
* 每次测试使用独立的数据库名避免版本冲突
|
|
||||||
*/
|
|
||||||
|
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { IndexedDBEngine } from '../../src/engine/indexeddb';
|
|
||||||
import { createSchema } from '../../src/table/schema';
|
|
||||||
|
|
||||||
let dbCounter = 0;
|
|
||||||
|
|
||||||
describe('IndexedDBEngine', () => {
|
|
||||||
let engine: IndexedDBEngine;
|
|
||||||
let dbName: string;
|
|
||||||
|
|
||||||
const userSchema = createSchema('users', {
|
|
||||||
id: { type: 'string', primaryKey: true },
|
|
||||||
name: { type: 'string', required: true },
|
|
||||||
age: { type: 'number', default: 0 },
|
|
||||||
email: { type: 'string', unique: true },
|
|
||||||
});
|
|
||||||
|
|
||||||
beforeEach(async () => {
|
|
||||||
dbName = `test-idb-${++dbCounter}`;
|
|
||||||
engine = new IndexedDBEngine();
|
|
||||||
await engine.open(dbName, 1);
|
|
||||||
});
|
|
||||||
|
|
||||||
afterEach(async () => {
|
|
||||||
if (engine.isOpen()) {
|
|
||||||
await engine.close();
|
|
||||||
}
|
|
||||||
// 清理:删除 IndexedDB 数据库
|
|
||||||
try {
|
|
||||||
indexedDB.deleteDatabase(dbName);
|
|
||||||
} catch {}
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 生命周期 ----
|
|
||||||
|
|
||||||
describe('生命周期', () => {
|
|
||||||
it('打开和关闭', async () => {
|
|
||||||
expect(engine.isOpen()).toBe(true);
|
|
||||||
await engine.close();
|
|
||||||
expect(engine.isOpen()).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('未打开时操作抛出错误', async () => {
|
|
||||||
await engine.close();
|
|
||||||
await expect(engine.count('users')).rejects.toThrow('not opened');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('可以重新打开', async () => {
|
|
||||||
await engine.close();
|
|
||||||
await engine.open(dbName, 2);
|
|
||||||
expect(engine.isOpen()).toBe(true);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 表管理 ----
|
|
||||||
|
|
||||||
describe('表管理', () => {
|
|
||||||
it('创建和删除表', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
expect(await engine.hasTable('users')).toBe(true);
|
|
||||||
|
|
||||||
await engine.dropTable('users');
|
|
||||||
expect(await engine.hasTable('users')).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('表名列表', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
const names = await engine.getTableNames();
|
|
||||||
expect(names).toContain('users');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('获取表结构', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
const schema = await engine.getTableSchema('users');
|
|
||||||
expect(schema).not.toBeNull();
|
|
||||||
expect(schema!.name).toBe('users');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('获取不存在的表结构返回 null', async () => {
|
|
||||||
const schema = await engine.getTableSchema('nonexistent');
|
|
||||||
expect(schema).toBeNull();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 插入 ----
|
|
||||||
|
|
||||||
describe('插入', () => {
|
|
||||||
beforeEach(async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('插入单行', async () => {
|
|
||||||
const pks = await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'alice@test.com' },
|
|
||||||
]);
|
|
||||||
expect(pks).toEqual(['1']);
|
|
||||||
expect(await engine.count('users')).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('插入多行', async () => {
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
|
|
||||||
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
|
|
||||||
]);
|
|
||||||
expect(await engine.count('users')).toBe(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('必填字段缺失抛出错误', async () => {
|
|
||||||
await expect(
|
|
||||||
engine.insert('users', [{ id: '1', email: 'a@t.com' }]),
|
|
||||||
).rejects.toThrow('required');
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 查询 ----
|
|
||||||
|
|
||||||
describe('查询', () => {
|
|
||||||
beforeEach(async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'alice@test.com' },
|
|
||||||
{ id: '2', name: 'Bob', age: 25, email: 'bob@test.com' },
|
|
||||||
{ id: '3', name: 'Charlie', age: 35, email: 'charlie@test.com' },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('查询所有', async () => {
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(3);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('WHERE 条件', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { age: { $gt: 28 } },
|
|
||||||
});
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('ORDER BY + LIMIT', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
orderBy: [{ column: 'age', direction: 'desc' }],
|
|
||||||
limit: 2,
|
|
||||||
});
|
|
||||||
expect(rows.map(r => r.age)).toEqual([35, 30]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('列选择', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
columns: ['id'],
|
|
||||||
where: { id: '1' },
|
|
||||||
});
|
|
||||||
expect(Object.keys(rows[0])).toEqual(['id']);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 更新 ----
|
|
||||||
|
|
||||||
describe('更新', () => {
|
|
||||||
beforeEach(async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
|
|
||||||
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('更新匹配行', async () => {
|
|
||||||
const count = await engine.update(
|
|
||||||
'users',
|
|
||||||
{ table: 'users', where: { id: '1' } },
|
|
||||||
{ age: 31 },
|
|
||||||
);
|
|
||||||
expect(count).toBe(1);
|
|
||||||
const rows = await engine.find('users', { table: 'users', where: { id: '1' } });
|
|
||||||
expect(rows[0].age).toBe(31);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('更新所有行', async () => {
|
|
||||||
const count = await engine.update(
|
|
||||||
'users',
|
|
||||||
{ table: 'users' },
|
|
||||||
{ age: 100 },
|
|
||||||
);
|
|
||||||
expect(count).toBe(2);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 删除 ----
|
|
||||||
|
|
||||||
describe('删除', () => {
|
|
||||||
beforeEach(async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
|
|
||||||
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('条件删除', async () => {
|
|
||||||
const count = await engine.delete('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { id: '1' },
|
|
||||||
});
|
|
||||||
expect(count).toBe(1);
|
|
||||||
expect(await engine.count('users')).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('清空表', async () => {
|
|
||||||
await engine.clear('users');
|
|
||||||
expect(await engine.count('users')).toBe(0);
|
|
||||||
// 表结构仍存在
|
|
||||||
expect(await engine.hasTable('users')).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('删除不存在的表抛出错误', async () => {
|
|
||||||
await expect(
|
|
||||||
engine.delete('nonexistent', { table: 'nonexistent' }),
|
|
||||||
).rejects.toThrow();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- Count ----
|
|
||||||
|
|
||||||
describe('Count', () => {
|
|
||||||
beforeEach(async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
|
|
||||||
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('count 全部', async () => {
|
|
||||||
expect(await engine.count('users')).toBe(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('count with where', async () => {
|
|
||||||
expect(await engine.count('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { age: { $gt: 28 } },
|
|
||||||
})).toBe(1);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 复杂 Where 条件 ----
|
|
||||||
|
|
||||||
describe('复杂 Where', () => {
|
|
||||||
beforeEach(async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
|
|
||||||
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
|
|
||||||
{ id: '3', name: 'Charlie', age: 35, email: 'c@t.com' },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('$like 模糊匹配', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { name: { $like: 'A%' } },
|
|
||||||
});
|
|
||||||
expect(rows).toHaveLength(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('$in 列表', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { id: { $in: ['1', '3'] } },
|
|
||||||
});
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('$ne 不等于', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { age: { $ne: 25 } },
|
|
||||||
});
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('$gte $lte 范围', async () => {
|
|
||||||
const rows = await engine.find('users', {
|
|
||||||
table: 'users',
|
|
||||||
where: { age: { $gte: 25, $lte: 30 } },
|
|
||||||
});
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 并发/多次操作 ----
|
|
||||||
|
|
||||||
describe('多次操作', () => {
|
|
||||||
it('连续创建删除表', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
expect(await engine.hasTable('users')).toBe(true);
|
|
||||||
await engine.dropTable('users');
|
|
||||||
expect(await engine.hasTable('users')).toBe(false);
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
expect(await engine.hasTable('users')).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('插入后查询再更新再查询', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
|
|
||||||
]);
|
|
||||||
let rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows[0].age).toBe(30);
|
|
||||||
|
|
||||||
await engine.update('users', { table: 'users' }, { age: 31 });
|
|
||||||
rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows[0].age).toBe(31);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -0,0 +1,508 @@
|
|||||||
|
/**
|
||||||
|
* 介质:SharedMemoryBackend(跨实例共享,模拟磁盘持久化)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { KVStoreEngine } from '../../src/engine/kvstore_engine';
|
||||||
|
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
|
||||||
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
let dbCounter = 0;
|
||||||
|
|
||||||
|
describe('KVStoreEngine', () => {
|
||||||
|
let engine: KVStoreEngine;
|
||||||
|
let dbName: string;
|
||||||
|
|
||||||
|
const userSchema = createSchema('users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string', required: true },
|
||||||
|
age: { type: 'number', default: 0 },
|
||||||
|
email: { type: 'string', unique: true },
|
||||||
|
});
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
dbName = `test-kv-${++dbCounter}-${Math.random().toString(36).slice(2, 6)}`;
|
||||||
|
engine = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
if (engine.isOpen()) {
|
||||||
|
await engine.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 生命周期 ----
|
||||||
|
|
||||||
|
describe('生命周期', () => {
|
||||||
|
it('打开和关闭', async () => {
|
||||||
|
expect(engine.isOpen()).toBe(true);
|
||||||
|
await engine.close();
|
||||||
|
expect(engine.isOpen()).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('未打开时操作抛出错误', async () => {
|
||||||
|
await engine.close();
|
||||||
|
await expect(engine.count('users')).rejects.toThrow('not opened');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('可以重新打开', async () => {
|
||||||
|
await engine.close();
|
||||||
|
await engine.open(dbName, 2);
|
||||||
|
expect(engine.isOpen()).toBe(true);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 表管理 ----
|
||||||
|
|
||||||
|
describe('表管理', () => {
|
||||||
|
it('创建表', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
expect(await engine.hasTable('users')).toBe(true);
|
||||||
|
const names = await engine.getTableNames();
|
||||||
|
expect(names).toContain('users');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('重复创建表抛错', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await expect(engine.createTable(userSchema)).rejects.toThrow('already exists');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('删除表', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
await engine.dropTable('users');
|
||||||
|
expect(await engine.hasTable('users')).toBe(false);
|
||||||
|
await expect(engine.count('users')).rejects.toThrow('does not exist');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('获取表结构', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
const schema = await engine.getTableSchema('users');
|
||||||
|
expect(schema).not.toBeNull();
|
||||||
|
expect(schema!.columns.id.primaryKey).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('schema 持久化:重启后表结构保留', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.hasTable('users')).toBe(true);
|
||||||
|
const schema = await engine2.getTableSchema('users');
|
||||||
|
expect(schema!.columns.email.unique).toBe(true);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 插入 ----
|
||||||
|
|
||||||
|
describe('插入', () => {
|
||||||
|
beforeEach(async () => { await engine.createTable(userSchema); });
|
||||||
|
|
||||||
|
it('插入单行并返回主键', async () => {
|
||||||
|
const pks = await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
|
||||||
|
expect(pks).toEqual(['1']);
|
||||||
|
expect(await engine.count('users')).toBe(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('默认值生效', async () => {
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
const rows = await engine.find('users', { table: 'users' });
|
||||||
|
expect(rows[0].age).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('重复主键抛错', async () => {
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'A' }]);
|
||||||
|
await expect(engine.insert('users', [{ id: '1', name: 'B' }])).rejects.toThrow('Duplicate primary key');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('唯一约束冲突抛错', async () => {
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'A', email: 'a@x.com' }]);
|
||||||
|
await expect(engine.insert('users', [{ id: '2', name: 'B', email: 'a@x.com' }]))
|
||||||
|
.rejects.toThrow('Unique constraint violation');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('必填字段校验', async () => {
|
||||||
|
await expect(engine.insert('users', [{ id: '1' }])).rejects.toThrow('is required');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('批量插入', async () => {
|
||||||
|
const pks = await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'A' }, { id: '2', name: 'B' }, { id: '3', name: 'C' },
|
||||||
|
]);
|
||||||
|
expect(pks).toEqual(['1', '2', '3']);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 查询 ----
|
||||||
|
|
||||||
|
describe('查询', () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'Alice', age: 30 },
|
||||||
|
{ id: '2', name: 'Bob', age: 25 },
|
||||||
|
{ id: '3', name: 'Carol', age: 35 },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('全表查询', async () => {
|
||||||
|
const rows = await engine.find('users', { table: 'users' });
|
||||||
|
expect(rows).toHaveLength(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('WHERE 等值过滤', async () => {
|
||||||
|
const rows = await engine.find('users', { table: 'users', where: { name: 'Bob' } });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(rows[0].id).toBe('2');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('WHERE 操作符', async () => {
|
||||||
|
const rows = await engine.find('users', { table: 'users', where: { age: { $gt: 26 } } });
|
||||||
|
expect(rows.map((r) => r.id).sort()).toEqual(['1', '3']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ORDER BY', async () => {
|
||||||
|
const rows = await engine.find('users', { table: 'users', orderBy: [{ column: 'age', direction: 'desc' }] });
|
||||||
|
expect(rows[0].id).toBe('3');
|
||||||
|
expect(rows[2].id).toBe('2');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('LIMIT / OFFSET', async () => {
|
||||||
|
const rows = await engine.find('users', { table: 'users', limit: 2, offset: 1 });
|
||||||
|
expect(rows).toHaveLength(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('列投影', async () => {
|
||||||
|
const rows = await engine.find('users', { table: 'users', columns: ['name'] });
|
||||||
|
expect(Object.keys(rows[0])).toEqual(['name']);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 更新 ----
|
||||||
|
|
||||||
|
describe('更新', () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'Alice', age: 30 },
|
||||||
|
{ id: '2', name: 'Bob', age: 25 },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('按条件更新', async () => {
|
||||||
|
const count = await engine.update('users', { table: 'users', where: { id: '1' } }, { age: 31 });
|
||||||
|
expect(count).toBe(1);
|
||||||
|
const rows = await engine.find('users', { table: 'users', where: { id: '1' } });
|
||||||
|
expect(rows[0].age).toBe(31);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('无匹配行影响 0 行', async () => {
|
||||||
|
const count = await engine.update('users', { table: 'users', where: { id: 'nope' } }, { age: 1 });
|
||||||
|
expect(count).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('更新主键(旧键删除 + 新键写入)', async () => {
|
||||||
|
const count = await engine.update('users', { table: 'users', where: { id: '1' } }, { id: '10', age: 99 });
|
||||||
|
expect(count).toBe(1);
|
||||||
|
expect(await engine.count('users')).toBe(2);
|
||||||
|
const rows = await engine.find('users', { table: 'users', where: { id: '10' } });
|
||||||
|
expect(rows[0].age).toBe(99);
|
||||||
|
expect(await engine.find('users', { table: 'users', where: { id: '1' } })).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('更新持久化:重启后保留', async () => {
|
||||||
|
await engine.update('users', { table: 'users', where: { id: '1' } }, { age: 99 });
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
const rows = await engine2.find('users', { table: 'users', where: { id: '1' } });
|
||||||
|
expect(rows[0].age).toBe(99);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 删除 ----
|
||||||
|
|
||||||
|
describe('删除', () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }, { id: '3', name: 'Carol' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('按条件删除', async () => {
|
||||||
|
const count = await engine.delete('users', { table: 'users', where: { id: '1' } });
|
||||||
|
expect(count).toBe(1);
|
||||||
|
expect(await engine.count('users')).toBe(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('删除持久化:重启后保留', async () => {
|
||||||
|
await engine.delete('users', { table: 'users', where: { id: '1' } });
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('users')).toBe(2);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('级联删除(CASCADE)', async () => {
|
||||||
|
const ordersSchema = createSchema('orders', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string', references: 'users.id', onDelete: 'CASCADE' },
|
||||||
|
});
|
||||||
|
await engine.createTable(ordersSchema);
|
||||||
|
await engine.insert('orders', [{ id: 'o1', user_id: '1' }, { id: 'o2', user_id: '2' }]);
|
||||||
|
|
||||||
|
await engine.delete('users', { table: 'users', where: { id: '1' } });
|
||||||
|
// 级联:orders 中 user_id=1 的行被删
|
||||||
|
expect(await engine.count('orders')).toBe(1);
|
||||||
|
|
||||||
|
// 重启后级联结果持久化
|
||||||
|
await engine.close();
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('users')).toBe(2);
|
||||||
|
expect(await engine2.count('orders')).toBe(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- Count ----
|
||||||
|
|
||||||
|
describe('Count', () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'A', age: 10 }, { id: '2', name: 'B', age: 20 }, { id: '3', name: 'C', age: 30 },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('总数', async () => {
|
||||||
|
expect(await engine.count('users')).toBe(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('条件计数', async () => {
|
||||||
|
expect(await engine.count('users', { table: 'users', where: { age: { $gte: 20 } } })).toBe(2);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 事务 ----
|
||||||
|
|
||||||
|
describe('事务', () => {
|
||||||
|
beforeEach(async () => { await engine.createTable(userSchema); });
|
||||||
|
|
||||||
|
it('commit 后数据可见且持久化', async () => {
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
await engine.commitTransaction();
|
||||||
|
expect(await engine.count('users')).toBe(1);
|
||||||
|
|
||||||
|
await engine.close();
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('users')).toBe(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rollback 后数据消失', async () => {
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
await engine.rollbackTransaction();
|
||||||
|
expect(await engine.count('users')).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务内读到自己写入的行', async () => {
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
const rows = await engine.find('users', { table: 'users' });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
await engine.rollbackTransaction();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('commit 多表原子(全部生效或全部不生效)', async () => {
|
||||||
|
await engine.createTable(createSchema('orders', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string' },
|
||||||
|
}));
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
await engine.insert('orders', [{ id: 'o1', user_id: '1' }]);
|
||||||
|
await engine.commitTransaction();
|
||||||
|
|
||||||
|
await engine.close();
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('users')).toBe(1);
|
||||||
|
expect(await engine2.count('orders')).toBe(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('事务中 DDL 支持(create/drop 随 commit 生效)', async () => {
|
||||||
|
const txSchema = createSchema('tx_users', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string' },
|
||||||
|
});
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.createTable(txSchema);
|
||||||
|
await engine.insert('tx_users', [{ id: '1', name: 'A' }]);
|
||||||
|
await engine.commitTransaction();
|
||||||
|
expect(await engine.hasTable('tx_users')).toBe(true);
|
||||||
|
expect(await engine.count('tx_users')).toBe(1);
|
||||||
|
|
||||||
|
// 事务中 drop 表
|
||||||
|
await engine.beginTransaction();
|
||||||
|
await engine.dropTable('tx_users');
|
||||||
|
await engine.commitTransaction();
|
||||||
|
expect(await engine.hasTable('tx_users')).toBe(false);
|
||||||
|
|
||||||
|
// 重启后 drop 生效(KV 残留清理)
|
||||||
|
await engine.close();
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.hasTable('tx_users')).toBe(false);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 持久化与崩溃恢复 ----
|
||||||
|
|
||||||
|
describe('持久化与崩溃恢复', () => {
|
||||||
|
it('未 checkpoint 数据重启后经日志重放恢复', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'Alice', age: 30 },
|
||||||
|
{ id: '2', name: 'Bob', age: 25 },
|
||||||
|
]);
|
||||||
|
await engine.close(); // 模拟"崩溃后重开"(未 checkpoint,数据在 KVStore 日志)
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('users')).toBe(2);
|
||||||
|
const rows = await engine2.find('users', { table: 'users' });
|
||||||
|
expect(rows.map((r) => r.id).sort()).toEqual(['1', '2']);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('checkpoint 后重启(快照恢复)', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
await (engine as any).kv.checkpoint();
|
||||||
|
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('users')).toBe(2);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('索引列跨重启恢复(查询走索引)', async () => {
|
||||||
|
const idxSchema = createSchema('items', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
tag: { type: 'string', index: true },
|
||||||
|
});
|
||||||
|
await engine.createTable(idxSchema);
|
||||||
|
await engine.insert('items', [
|
||||||
|
{ id: '1', tag: 'a' }, { id: '2', tag: 'b' }, { id: '3', tag: 'a' },
|
||||||
|
]);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
const rows = await engine2.find('items', { table: 'items', where: { tag: 'a' } });
|
||||||
|
expect(rows).toHaveLength(2);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('CREATE INDEX 持久化(重启后索引保留)', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
|
||||||
|
await engine.createIndex('users', 'name');
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
const schema = await engine2.getTableSchema('users');
|
||||||
|
expect(schema!.columns.name.index).toBe(true);
|
||||||
|
const rows = await engine2.find('users', { table: 'users', where: { name: 'Alice' } });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ALTER TABLE ADD/DROP 持久化', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
|
||||||
|
await engine.alterTable('users', 'ADD', { name: 'phone', type: 'string' });
|
||||||
|
await engine.alterTable('users', 'DROP', { name: 'age', type: 'number' });
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
const schema = await engine2.getTableSchema('users');
|
||||||
|
expect(schema!.columns.phone).toBeDefined();
|
||||||
|
expect(schema!.columns.age).toBeUndefined();
|
||||||
|
const rows = await engine2.find('users', { table: 'users' });
|
||||||
|
expect(rows[0].phone).toBeUndefined();
|
||||||
|
expect(rows[0].age).toBeUndefined();
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('元数据 getMeta/setMeta 持久化', async () => {
|
||||||
|
await engine.setMeta('__metona_version', '3');
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.getMeta('__metona_version')).toBe('3');
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('数据损坏自愈:日志损坏 → open 不崩,repair 截断', async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 篡改日志尾部 CRC(模拟最后一条记录写入中断)
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
await medium.open(dbName);
|
||||||
|
const log = await medium.read('__kv_log');
|
||||||
|
expect(log).not.toBeNull();
|
||||||
|
const corrupted = new Uint8Array(log as ArrayBuffer);
|
||||||
|
corrupted[corrupted.byteLength - 5] ^= 0xff;
|
||||||
|
await medium.write('__kv_log', corrupted.buffer as ArrayBuffer);
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1); // 损坏尾部截断,打开不崩
|
||||||
|
expect(await engine2.hasTable('users')).toBe(true);
|
||||||
|
await engine2.repair();
|
||||||
|
expect(await engine2.hasTable('users')).toBe(true);
|
||||||
|
// repair 后新写入正常
|
||||||
|
await engine2.insert('users', [{ id: '2', name: 'Bob' }]);
|
||||||
|
expect(await engine2.count('users')).toBeGreaterThanOrEqual(1);
|
||||||
|
await engine2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- 流式查询 ----
|
||||||
|
|
||||||
|
describe('流式查询', () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
await engine.createTable(userSchema);
|
||||||
|
await engine.insert('users', [
|
||||||
|
{ id: '1', name: 'A' }, { id: '2', name: 'B' }, { id: '3', name: 'C' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('findStream 逐行回调', async () => {
|
||||||
|
const rows: Record<string, unknown>[] = [];
|
||||||
|
const count = await engine.findStream!('users', { table: 'users' }, (r) => rows.push(r));
|
||||||
|
expect(count).toBe(3);
|
||||||
|
expect(rows.map((r) => r.id).sort()).toEqual(['1', '2', '3']);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,139 @@
|
|||||||
|
/**
|
||||||
|
* KVStoreEngine — 大规模压力测试(10 万级)
|
||||||
|
*
|
||||||
|
* 验证生产级可靠性:
|
||||||
|
* 1. 10 万 key 批量写入 + checkpoint + 重开全量验证
|
||||||
|
* 2. 混合操作(insert/update/delete)5 万级 + 崩溃模拟(不 checkpoint 断开)→ 重开验证已确认写入
|
||||||
|
* 3. 多 checkpoint 循环下数据不丢
|
||||||
|
*/
|
||||||
|
import { KVStoreEngine } from '../../src/engine/kvstore_engine';
|
||||||
|
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
|
||||||
|
import { createSchema } from '../../src/table/schema';
|
||||||
|
|
||||||
|
let counter = 0;
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `stress-${Date.now()}-${++counter}-${Math.random().toString(36).slice(2, 6)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
SharedMemoryBackend.clearRegistry();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('KVStoreEngine — 10 万级压力', () => {
|
||||||
|
it('10 万 key 写入 + checkpoint + 重开全量验证', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('big', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
val: { type: 'number' },
|
||||||
|
name: { type: 'string' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
const TOTAL = 100000;
|
||||||
|
// 分批写入(每批 1000)
|
||||||
|
for (let batch = 0; batch < TOTAL / 1000; batch++) {
|
||||||
|
const rows = [] as Record<string, unknown>[];
|
||||||
|
for (let i = 0; i < 1000; i++) {
|
||||||
|
const idx = batch * 1000 + i;
|
||||||
|
rows.push({ id: `k${idx}`, val: idx, name: `User${idx}` });
|
||||||
|
}
|
||||||
|
await engine.insert('big', rows);
|
||||||
|
// 每 20 批 checkpoint 一次
|
||||||
|
if ((batch + 1) % 20 === 0) {
|
||||||
|
await (engine as any).kv.checkpoint();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
expect(await engine.count('big')).toBe(TOTAL);
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
// 重开:全量验证
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('big')).toBe(TOTAL);
|
||||||
|
// 抽样验证
|
||||||
|
for (const id of ['k0', 'k49999', 'k99999', 'k12345']) {
|
||||||
|
const rows = await engine2.find('big', { table: 'big', where: { id } });
|
||||||
|
expect(rows).toHaveLength(1);
|
||||||
|
expect(Number(rows[0].val)).toBe(Number(id.slice(1)));
|
||||||
|
}
|
||||||
|
await engine2.close();
|
||||||
|
}, 120000);
|
||||||
|
|
||||||
|
it('5 万混合操作 + 崩溃模拟(不 checkpoint)→ 重开全部已确认写入可见', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('ops', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
val: { type: 'number' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
let seed = 42;
|
||||||
|
const rand = () => { seed = (seed * 1103515245 + 12345) & 0x7fffffff; return seed / 0x7fffffff; };
|
||||||
|
const confirmed = new Map<string, number>();
|
||||||
|
|
||||||
|
const TOTAL = 50000;
|
||||||
|
for (let i = 0; i < TOTAL; i++) {
|
||||||
|
const r = rand();
|
||||||
|
const id = `k${Math.floor(rand() * 20000)}`;
|
||||||
|
if (r < 0.6) {
|
||||||
|
const val = Math.floor(rand() * 1000000);
|
||||||
|
try {
|
||||||
|
await engine.insert('ops', [{ id, val }]);
|
||||||
|
confirmed.set(id, val);
|
||||||
|
} catch (e) {
|
||||||
|
if ((e as { code?: string }).code !== 'DUPLICATE_KEY') throw e;
|
||||||
|
}
|
||||||
|
} else if (r < 0.8) {
|
||||||
|
const val = Math.floor(rand() * 1000000);
|
||||||
|
await engine.update('ops', { table: 'ops', where: { id } }, { val });
|
||||||
|
if (confirmed.has(id)) confirmed.set(id, val);
|
||||||
|
} else {
|
||||||
|
await engine.delete('ops', { table: 'ops', where: { id } });
|
||||||
|
confirmed.delete(id);
|
||||||
|
}
|
||||||
|
// 每 5000 次 checkpoint
|
||||||
|
if ((i + 1) % 5000 === 0) {
|
||||||
|
await (engine as any).kv.checkpoint();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 崩溃模拟:直接断开(不 close → 数据在 KVStore 日志/快照)
|
||||||
|
await engine.close();
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('ops')).toBe(confirmed.size);
|
||||||
|
// 抽样验证值
|
||||||
|
let sampled = 0;
|
||||||
|
for (const [id, val] of confirmed) {
|
||||||
|
if (sampled++ > 1000) break;
|
||||||
|
const rows = await engine2.find('ops', { table: 'ops', where: { id } });
|
||||||
|
expect(rows[0].val).toBe(val);
|
||||||
|
}
|
||||||
|
await engine2.close();
|
||||||
|
}, 120000);
|
||||||
|
|
||||||
|
it('多次 checkpoint 循环(500 次)数据不丢', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const engine = new KVStoreEngine(new SharedMemoryBackend(), 0);
|
||||||
|
await engine.open(dbName, 1);
|
||||||
|
await engine.createTable(createSchema('t', {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
v: { type: 'number' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
for (let i = 0; i < 500; i++) {
|
||||||
|
await engine.insert('t', [{ id: `k${i}`, v: i }]);
|
||||||
|
await (engine as any).kv.checkpoint();
|
||||||
|
}
|
||||||
|
await engine.close();
|
||||||
|
|
||||||
|
const engine2 = new KVStoreEngine(new SharedMemoryBackend(), 0);
|
||||||
|
await engine2.open(dbName, 1);
|
||||||
|
expect(await engine2.count('t')).toBe(500);
|
||||||
|
const last = await engine2.find('t', { table: 't', where: { id: 'k499' } });
|
||||||
|
expect(last[0].v).toBe(499);
|
||||||
|
await engine2.close();
|
||||||
|
}, 60000);
|
||||||
|
});
|
||||||
@@ -0,0 +1,342 @@
|
|||||||
|
/**
|
||||||
|
* KVStore — 自研 KV 引擎单元测试
|
||||||
|
*
|
||||||
|
* 覆盖:
|
||||||
|
* 1. 基本读写(put/get/delete/exists/listKeys)
|
||||||
|
* 2. putMany/deleteMany 原子性(多 key 一次落盘)
|
||||||
|
* 3. 跨实例持久化(SharedMemory 全局注册表,close 不清数据)
|
||||||
|
* 4. 崩溃恢复:日志重放(未 checkpoint 数据恢复)
|
||||||
|
* 5. checkpoint 后恢复(快照 + 水位跳过)
|
||||||
|
* 6. 快照损坏 → 全量日志重放自愈
|
||||||
|
* 7. 日志损坏 → 截断至损坏处(丢弃未确认尾部)
|
||||||
|
* 8. 写入失败原子性(日志失败不更新内存索引)
|
||||||
|
* 9. clear / repair
|
||||||
|
* 10. 并发写与 checkpoint 串行(无交错丢数据)
|
||||||
|
*/
|
||||||
|
import { KVStore } from '../../src/engine/kvstore/index';
|
||||||
|
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
|
||||||
|
import { encodeLogRecord, parseLogRecords, KVLogOp } from '../../src/engine/kvstore/log';
|
||||||
|
import { encodeSnapshot, decodeSnapshot } from '../../src/engine/kvstore/snapshot';
|
||||||
|
|
||||||
|
let dbCounter = 0;
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `kv-${Date.now()}-${++dbCounter}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const enc = (s: string) => new TextEncoder().encode(s).buffer as ArrayBuffer;
|
||||||
|
const dec = (b: ArrayBuffer | null) => (b ? new TextDecoder().decode(b) : null);
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
SharedMemoryBackend.clearRegistry();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('KVStore — 基本读写', () => {
|
||||||
|
it('put/get/delete/exists/listKeys/size', async () => {
|
||||||
|
const kv = new KVStore(new SharedMemoryBackend());
|
||||||
|
await kv.open(uniqueDB());
|
||||||
|
await kv.put('a', enc('AAA'));
|
||||||
|
await kv.put('b', enc('BBB'));
|
||||||
|
expect(dec(await kv.get('a'))).toBe('AAA');
|
||||||
|
expect(await kv.exists('b')).toBe(true);
|
||||||
|
expect(await kv.exists('nope')).toBe(false);
|
||||||
|
expect((await kv.listKeys()).sort()).toEqual(['a', 'b']);
|
||||||
|
expect(kv.size()).toBe(2);
|
||||||
|
|
||||||
|
await kv.delete('a');
|
||||||
|
expect(await kv.exists('a')).toBe(false);
|
||||||
|
expect(kv.size()).toBe(1);
|
||||||
|
await kv.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('putMany 多 key 原子写入', async () => {
|
||||||
|
const kv = new KVStore(new SharedMemoryBackend());
|
||||||
|
await kv.open(uniqueDB());
|
||||||
|
await kv.putMany({ a: enc('1'), b: enc('2'), c: enc('3') });
|
||||||
|
expect(dec(await kv.get('a'))).toBe('1');
|
||||||
|
expect(dec(await kv.get('c'))).toBe('3');
|
||||||
|
await kv.deleteMany(['a', 'c']);
|
||||||
|
expect(await kv.exists('a')).toBe(false);
|
||||||
|
expect(await kv.exists('c')).toBe(false);
|
||||||
|
expect(await kv.exists('b')).toBe(true);
|
||||||
|
await kv.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('KVStore — 持久化与崩溃恢复', () => {
|
||||||
|
it('未 checkpoint 的数据:close 后重开经日志重放恢复', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.putMany({ a: enc('AAA'), b: enc('BBB') });
|
||||||
|
await kv1.put('c', enc('CCC'));
|
||||||
|
await kv1.delete('b');
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
// 模拟"崩溃后重开"(新实例,共享介质)
|
||||||
|
const kv2 = new KVStore(medium);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(dec(await kv2.get('a'))).toBe('AAA');
|
||||||
|
expect(await kv2.exists('b')).toBe(false);
|
||||||
|
expect(dec(await kv2.get('c'))).toBe('CCC');
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('checkpoint 后重开:快照加载 + 日志水位跳过(无重复/无丢失)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0); // 关闭自动 checkpoint
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.putMany({ a: enc('A1'), b: enc('B1') });
|
||||||
|
await kv1.checkpoint();
|
||||||
|
// checkpoint 后新写入(进日志)
|
||||||
|
await kv1.put('c', enc('C1'));
|
||||||
|
await kv1.put('a', enc('A2'));
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(dec(await kv2.get('a'))).toBe('A2'); // 日志重放覆盖快照值
|
||||||
|
expect(dec(await kv2.get('b'))).toBe('B1');
|
||||||
|
expect(dec(await kv2.get('c'))).toBe('C1');
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('多次 checkpoint + 截断日志后重开正确', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
for (let i = 0; i < 10; i++) {
|
||||||
|
await kv1.put(`k${i}`, enc(`v${i}`));
|
||||||
|
await kv1.checkpoint();
|
||||||
|
}
|
||||||
|
await kv1.put('last', enc('L'));
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(kv2.size()).toBe(11);
|
||||||
|
expect(dec(await kv2.get('k9'))).toBe('v9');
|
||||||
|
expect(dec(await kv2.get('last'))).toBe('L');
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('快照损坏 → 全量日志重放自愈(数据不丢)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.putMany({ a: enc('AAA'), b: enc('BBB') });
|
||||||
|
await kv1.checkpoint();
|
||||||
|
await kv1.put('c', enc('CCC'));
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
// 篡改快照(模拟损坏)
|
||||||
|
const snap = await medium.read('__kv_snapshot');
|
||||||
|
const corrupted = new Uint8Array(snap as ArrayBuffer);
|
||||||
|
corrupted[20] ^= 0xff;
|
||||||
|
await medium.write('__kv_snapshot', corrupted.buffer as ArrayBuffer);
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
// 快照损坏 → 全量日志重放
|
||||||
|
expect(dec(await kv2.get('a'))).toBe('AAA');
|
||||||
|
expect(dec(await kv2.get('b'))).toBe('BBB');
|
||||||
|
expect(dec(await kv2.get('c'))).toBe('CCC');
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('日志损坏尾部 → 截断至损坏处(丢弃未确认尾部,已确认数据保留)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.put('confirmed', enc('KEEP'));
|
||||||
|
await kv1.put('confirmed2', enc('KEEP2'));
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
// 追加一条损坏记录(模拟写入中断:半写记录)
|
||||||
|
const bad = encodeLogRecord(999, { damaged: enc('X') });
|
||||||
|
const corrupted = new Uint8Array(bad);
|
||||||
|
corrupted[20] ^= 0xff; // 破坏 CRC
|
||||||
|
// close 后介质实例不可用,重新打开访问"磁盘"
|
||||||
|
const disk = new SharedMemoryBackend();
|
||||||
|
await disk.open(dbName);
|
||||||
|
const log = await disk.read('__kv_log');
|
||||||
|
const combined = new Uint8Array((log as ArrayBuffer).byteLength + corrupted.byteLength);
|
||||||
|
combined.set(new Uint8Array(log as ArrayBuffer), 0);
|
||||||
|
combined.set(corrupted, (log as ArrayBuffer).byteLength);
|
||||||
|
await disk.write('__kv_log', combined.buffer as ArrayBuffer);
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(dec(await kv2.get('confirmed'))).toBe('KEEP');
|
||||||
|
expect(dec(await kv2.get('confirmed2'))).toBe('KEEP2');
|
||||||
|
expect(await kv2.exists('damaged')).toBe(false);
|
||||||
|
// 损坏日志已被截断(重开后再写正常)
|
||||||
|
await kv2.put('after', enc('OK'));
|
||||||
|
await kv2.close();
|
||||||
|
const kv3 = new KVStore(medium, 0);
|
||||||
|
await kv3.open(dbName);
|
||||||
|
expect(dec(await kv3.get('after'))).toBe('OK');
|
||||||
|
await kv3.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('写入失败 → 内存索引不更新(原子性)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv = new KVStore(medium, 0);
|
||||||
|
await kv.open(dbName);
|
||||||
|
|
||||||
|
// 注入日志追加失败
|
||||||
|
const origAppend = medium.append!.bind(medium);
|
||||||
|
medium.append = async () => { throw new Error('disk full'); };
|
||||||
|
await expect(kv.put('x', enc('X'))).rejects.toMatchObject({ code: 'KV_LOG_ERROR' });
|
||||||
|
expect(await kv.exists('x')).toBe(false);
|
||||||
|
|
||||||
|
medium.append = origAppend;
|
||||||
|
await kv.put('y', enc('Y'));
|
||||||
|
expect(dec(await kv.get('y'))).toBe('Y');
|
||||||
|
await kv.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('KVStore — 维护与并发', () => {
|
||||||
|
it('clear 清空全部(保留库),重开为空', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.putMany({ a: enc('1'), b: enc('2') });
|
||||||
|
await kv1.checkpoint();
|
||||||
|
await kv1.put('c', enc('3'));
|
||||||
|
await kv1.clear();
|
||||||
|
expect(kv1.size()).toBe(0);
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(kv2.size()).toBe(0);
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('repair 清理损坏快照与日志(损坏快照数据无法恢复,日志数据保留)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.put('a', enc('A'));
|
||||||
|
await kv1.checkpoint(); // 快照成为 a 的唯一副本(日志已截断)
|
||||||
|
await kv1.put('b', enc('B')); // 日志中(seq=2)
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
// 快照损坏 + 日志尾部损坏
|
||||||
|
const disk = new SharedMemoryBackend();
|
||||||
|
await disk.open(dbName);
|
||||||
|
const snap = new Uint8Array(await disk.read('__kv_snapshot') as ArrayBuffer);
|
||||||
|
snap[10] ^= 0xff;
|
||||||
|
await disk.write('__kv_snapshot', snap.buffer as ArrayBuffer);
|
||||||
|
const bad = encodeLogRecord(500, { junk: enc('J') });
|
||||||
|
bad[15] ^= 0xff;
|
||||||
|
const log = new Uint8Array(await disk.read('__kv_log') as ArrayBuffer);
|
||||||
|
const combined = new Uint8Array(log.byteLength + bad.byteLength);
|
||||||
|
combined.set(log, 0); combined.set(bad, log.byteLength);
|
||||||
|
await disk.write('__kv_log', combined.buffer as ArrayBuffer);
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
const discarded = await kv2.repair();
|
||||||
|
expect(discarded).toBeGreaterThan(0);
|
||||||
|
// 快照损坏(a 的唯一副本丢失)+ 损坏日志尾部被截断 → 日志数据 b 完整
|
||||||
|
expect(await kv2.exists('a')).toBe(false);
|
||||||
|
expect(dec(await kv2.get('b'))).toBe('B');
|
||||||
|
// repair 后介质上的损坏快照已清理
|
||||||
|
expect(await medium.exists('__kv_snapshot')).toBe(false);
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('并发写 + checkpoint 串行(无交错丢数据)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv = new KVStore(medium, 0);
|
||||||
|
await kv.open(dbName);
|
||||||
|
|
||||||
|
// 并发发起大量写入 + 中途 checkpoint
|
||||||
|
const writes = [] as Promise<void>[];
|
||||||
|
for (let i = 0; i < 200; i++) {
|
||||||
|
writes.push(kv.put(`k${i}`, enc(`v${i}`)));
|
||||||
|
if (i === 100) writes.push(kv.checkpoint());
|
||||||
|
}
|
||||||
|
await Promise.all(writes);
|
||||||
|
|
||||||
|
expect(kv.size()).toBe(200);
|
||||||
|
await kv.close();
|
||||||
|
|
||||||
|
// 重开验证全部恢复
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(kv2.size()).toBe(200);
|
||||||
|
expect(dec(await kv2.get('k150'))).toBe('v150');
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('自动 checkpoint 阈值触发(日志不无限增长)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv = new KVStore(medium, 256); // 小阈值
|
||||||
|
await kv.open(dbName);
|
||||||
|
for (let i = 0; i < 50; i++) {
|
||||||
|
await kv.put(`k${i}`, enc(`value-${i}-`.repeat(10)));
|
||||||
|
}
|
||||||
|
// 自动 checkpoint 后日志应被截断
|
||||||
|
const log = await medium.read('__kv_log');
|
||||||
|
expect((log as ArrayBuffer).byteLength).toBeLessThan(4096);
|
||||||
|
await kv.close();
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 256);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
expect(kv2.size()).toBe(50);
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('KVStore — 编解码单元', () => {
|
||||||
|
it('encodeLogRecord / parseLogRecords 往返', () => {
|
||||||
|
const rec = encodeLogRecord(1, { a: enc('A'), b: enc('BB') }, ['del']);
|
||||||
|
const records: { seq: number; entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] }[] = [];
|
||||||
|
parseLogRecords(rec, (r) => records.push(r));
|
||||||
|
expect(records).toHaveLength(1);
|
||||||
|
expect(records[0].seq).toBe(1);
|
||||||
|
expect(records[0].entries).toHaveLength(3);
|
||||||
|
expect(dec(records[0].entries[0].value)).toBe('A');
|
||||||
|
expect(records[0].entries[2].op).toBe(KVLogOp.DELETE);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('多记录日志顺序解析', () => {
|
||||||
|
const r1 = encodeLogRecord(1, { a: enc('1') });
|
||||||
|
const r2 = encodeLogRecord(2, { b: enc('2') });
|
||||||
|
const combined = new Uint8Array(r1.byteLength + r2.byteLength);
|
||||||
|
combined.set(r1, 0); combined.set(r2, r1.byteLength);
|
||||||
|
const seqs: number[] = [];
|
||||||
|
parseLogRecords(combined, (r) => seqs.push(r.seq));
|
||||||
|
expect(seqs).toEqual([1, 2]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('encodeSnapshot / decodeSnapshot 往返', () => {
|
||||||
|
const entries = new Map<string, ArrayBuffer>([['a', enc('A')], ['b', enc('BB')]]);
|
||||||
|
const bytes = encodeSnapshot(42, entries);
|
||||||
|
const snap = decodeSnapshot(bytes);
|
||||||
|
expect(snap).not.toBeNull();
|
||||||
|
expect(snap!.seq).toBe(42);
|
||||||
|
expect(dec(snap!.entries.get('a'))).toBe('A');
|
||||||
|
expect(dec(snap!.entries.get('b'))).toBe('BB');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('损坏快照 decode 返回 null', () => {
|
||||||
|
const entries = new Map<string, ArrayBuffer>([['a', enc('A')]]);
|
||||||
|
const bytes = encodeSnapshot(1, entries);
|
||||||
|
bytes[10] ^= 0xff;
|
||||||
|
expect(decodeSnapshot(bytes)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,158 +0,0 @@
|
|||||||
/**
|
|
||||||
* OPFSEngine 测试 — mock FileSystemDirectoryHandle,验证逻辑不卡死
|
|
||||||
* 注:jsdom 无 OPFS API,全部使用 mock 模拟
|
|
||||||
*/
|
|
||||||
import { OPFSEngine } from '../../src/engine/opfs';
|
|
||||||
import { createSchema } from '../../src/table/schema';
|
|
||||||
|
|
||||||
// Mock navigator.storage.getDirectory + FileSystemDirectoryHandle
|
|
||||||
function mockOPFS() {
|
|
||||||
const files = new Map<string, string>();
|
|
||||||
|
|
||||||
const dirMock = {
|
|
||||||
getDirectoryHandle: async (_name: string, _opts?: any) => dirMock as any,
|
|
||||||
getFileHandle: async (name: string, opts?: any) => {
|
|
||||||
if (opts?.create) {
|
|
||||||
return { createWritable: async () => ({ write: async (d: string) => { files.set(name, d); }, close: async () => {} }) };
|
|
||||||
}
|
|
||||||
if (!files.has(name)) throw new Error('Not found');
|
|
||||||
return { getFile: async () => ({ text: async () => files.get(name)!, arrayBuffer: async () => new ArrayBuffer(0) }) };
|
|
||||||
},
|
|
||||||
removeEntry: async (name: string) => { files.delete(name); },
|
|
||||||
};
|
|
||||||
|
|
||||||
// 为 entries() 添加可迭代接口
|
|
||||||
(dirMock as any).entries = () => ({
|
|
||||||
[Symbol.asyncIterator]: async function* () {
|
|
||||||
for (const [k] of files) yield [k];
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
// jsdom 环境下 navigator 可能已存在,直接设置 storage 属性
|
|
||||||
const nav = (globalThis as any).navigator || {};
|
|
||||||
nav.storage = { getDirectory: async () => dirMock };
|
|
||||||
(globalThis as any).navigator = nav;
|
|
||||||
|
|
||||||
return { files };
|
|
||||||
}
|
|
||||||
|
|
||||||
const userSchema = createSchema('users', {
|
|
||||||
id: { type: 'string', primaryKey: true },
|
|
||||||
name: { type: 'string', required: true },
|
|
||||||
age: { type: 'number', default: 0 },
|
|
||||||
});
|
|
||||||
|
|
||||||
describe('OPFSEngine', () => {
|
|
||||||
let engine: OPFSEngine;
|
|
||||||
let mocks: ReturnType<typeof mockOPFS>;
|
|
||||||
|
|
||||||
beforeEach(async () => {
|
|
||||||
mocks = mockOPFS();
|
|
||||||
engine = new OPFSEngine();
|
|
||||||
await engine.open('test-opfs', 1);
|
|
||||||
});
|
|
||||||
|
|
||||||
afterEach(async () => {
|
|
||||||
await engine.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 生命周期 ----
|
|
||||||
it('打开后 isOpen 返回 true', () => {
|
|
||||||
expect(engine.isOpen()).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('关闭后 isOpen 返回 false', async () => {
|
|
||||||
await engine.close();
|
|
||||||
expect(engine.isOpen()).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 表管理 ----
|
|
||||||
it('创建表后 hasTable 返回 true', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
expect(await engine.hasTable('users')).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('获取表名列表', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
const names = await engine.getTableNames();
|
|
||||||
expect(names).toContain('users');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('删除表后 hasTable 返回 false', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.dropTable('users');
|
|
||||||
expect(await engine.hasTable('users')).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- CRUD ----
|
|
||||||
it('插入后立即可查询', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30 },
|
|
||||||
{ id: '2', name: 'Bob', age: 25 },
|
|
||||||
]);
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('count 返回正确行数', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
|
||||||
expect(await engine.count('users')).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('更新后数据变化', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
|
|
||||||
await engine.update('users', { table: 'users', where: { id: '1' } }, { age: 31 });
|
|
||||||
const rows = await engine.find('users', { table: 'users', where: { id: '1' } });
|
|
||||||
expect(rows[0].age).toBe(31);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('删除后 count 减少', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }]);
|
|
||||||
await engine.delete('users', { table: 'users', where: { id: '1' } });
|
|
||||||
expect(await engine.count('users')).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 数据持久化(模拟重启) ----
|
|
||||||
it('close + reopen 后数据恢复', async () => {
|
|
||||||
// 写入数据
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [
|
|
||||||
{ id: '1', name: 'Alice', age: 30 },
|
|
||||||
{ id: '2', name: 'Bob', age: 25 },
|
|
||||||
]);
|
|
||||||
const beforeRows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(beforeRows).toHaveLength(2);
|
|
||||||
|
|
||||||
// 关闭后重新打开(模拟重启)
|
|
||||||
await engine.close();
|
|
||||||
engine = new OPFSEngine();
|
|
||||||
await engine.open('test-opfs', 1);
|
|
||||||
|
|
||||||
// 数据应恢复
|
|
||||||
const afterRows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(afterRows).toHaveLength(2);
|
|
||||||
expect(afterRows.map((r: any) => r.name).sort()).toEqual(['Alice', 'Bob']);
|
|
||||||
});
|
|
||||||
|
|
||||||
// ---- 事务 ----
|
|
||||||
it('事务 commit 后数据持久化', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.beginTransaction();
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
|
||||||
await engine.commitTransaction();
|
|
||||||
expect(await engine.count('users')).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('事务 rollback 后数据恢复', async () => {
|
|
||||||
await engine.createTable(userSchema);
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
|
||||||
await engine.beginTransaction();
|
|
||||||
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
|
|
||||||
await engine.rollbackTransaction();
|
|
||||||
expect(await engine.count('users')).toBe(1);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -6,7 +6,6 @@
|
|||||||
* 本次验证 Table API 与 SQL query 两条路径均真实触发全部 14 个钩子。
|
* 本次验证 Table API 与 SQL query 两条路径均真实触发全部 14 个钩子。
|
||||||
*/
|
*/
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
|
||||||
async function createDb(name: string): Promise<MetonaSqlark> {
|
async function createDb(name: string): Promise<MetonaSqlark> {
|
||||||
const db = new MetonaSqlark({ name: `hooks-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`, mode: 'memory' });
|
const db = new MetonaSqlark({ name: `hooks-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`, mode: 'memory' });
|
||||||
|
|||||||
@@ -6,7 +6,6 @@
|
|||||||
|
|
||||||
import { HybridEngine } from '../../src/hybrid/index';
|
import { HybridEngine } from '../../src/hybrid/index';
|
||||||
import { createSchema } from '../../src/table/schema';
|
import { createSchema } from '../../src/table/schema';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
|
||||||
let hybridCounter = 0;
|
let hybridCounter = 0;
|
||||||
|
|
||||||
@@ -30,7 +29,6 @@ describe('HybridEngine', () => {
|
|||||||
if (engine.isOpen()) {
|
if (engine.isOpen()) {
|
||||||
await engine.close();
|
await engine.close();
|
||||||
}
|
}
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('引擎名称', () => {
|
it('引擎名称', () => {
|
||||||
|
|||||||
@@ -9,7 +9,6 @@
|
|||||||
* - refresh 手动刷新
|
* - refresh 手动刷新
|
||||||
* - 表名校验(SQL 注入防护)
|
* - 表名校验(SQL 注入防护)
|
||||||
*/
|
*/
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
|
||||||
// ---- 最小 Vue mock(自包含:jest.mock 工厂不能引用外部变量) ----
|
// ---- 最小 Vue mock(自包含:jest.mock 工厂不能引用外部变量) ----
|
||||||
jest.mock('vue', () => {
|
jest.mock('vue', () => {
|
||||||
|
|||||||
@@ -7,7 +7,10 @@
|
|||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import { parse } from '../src/sql/parser';
|
import { parse } from '../src/sql/parser';
|
||||||
import { createSchema } from '../src/table/schema';
|
import { createSchema } from '../src/table/schema';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let dbCounter = 0;
|
let dbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -53,7 +56,7 @@ describe('v0.5.1 — 维护语句执行(Aria 引擎)', () => {
|
|||||||
const db = new MetonaSqlark({
|
const db = new MetonaSqlark({
|
||||||
name: uniqueDB(),
|
name: uniqueDB(),
|
||||||
mode: 'aria',
|
mode: 'aria',
|
||||||
diskEngine: 'indexeddb',
|
diskEngine: 'opfs',
|
||||||
aria: { checkpointInterval: 100000, walSyncMode: 'full' },
|
aria: { checkpointInterval: 100000, walSyncMode: 'full' },
|
||||||
});
|
});
|
||||||
await db.init();
|
await db.init();
|
||||||
|
|||||||
@@ -0,0 +1,187 @@
|
|||||||
|
/**
|
||||||
|
* migrateFromIndexedDB 迁移工具测试
|
||||||
|
*
|
||||||
|
* 用 fake-indexeddb 构造旧库(模拟 v0.5.x 及更早的 IndexedDB 引擎数据),
|
||||||
|
* 验证迁移到自研 KV 引擎(KVStoreEngine)后数据/schema/索引完整。
|
||||||
|
*/
|
||||||
|
import 'fake-indexeddb/auto';
|
||||||
|
import { MetonaSqlark } from '../src/core';
|
||||||
|
import { migrateFromIndexedDB } from '../src/migration/index';
|
||||||
|
|
||||||
|
let counter = 0;
|
||||||
|
function uniqueDB(): string {
|
||||||
|
return `mig-${Date.now()}-${++counter}-${Math.random().toString(36).slice(2, 6)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 构造旧 disk 模式库(IndexedDBEngine 布局:每表一个 objectStore + __metona_schema) */
|
||||||
|
async function createLegacyDiskDB(dbName: string): Promise<void> {
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(dbName, 2);
|
||||||
|
request.onupgradeneeded = () => {
|
||||||
|
const db = request.result;
|
||||||
|
const users = db.createObjectStore('users', { keyPath: 'id' });
|
||||||
|
users.createIndex('idx_email', 'email', { unique: true });
|
||||||
|
db.createObjectStore('orders', { keyPath: 'id' });
|
||||||
|
db.createObjectStore('__metona_schema', { keyPath: 'name' });
|
||||||
|
};
|
||||||
|
request.onsuccess = () => {
|
||||||
|
const db = request.result;
|
||||||
|
const tx = db.transaction(['users', 'orders', '__metona_schema'], 'readwrite');
|
||||||
|
const users = tx.objectStore('users');
|
||||||
|
users.add({ id: '1', name: 'Alice', email: 'alice@x.com', age: 30 });
|
||||||
|
users.add({ id: '2', name: 'Bob', email: 'bob@x.com', age: 25 });
|
||||||
|
const orders = tx.objectStore('orders');
|
||||||
|
orders.add({ id: 'o1', user_id: '1', amount: 100 });
|
||||||
|
// 持久化 schema(IndexedDBEngine v0.3.2+ 布局)
|
||||||
|
tx.objectStore('__metona_schema').put({
|
||||||
|
name: 'users',
|
||||||
|
schema: JSON.stringify({
|
||||||
|
name: 'users',
|
||||||
|
columns: {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
name: { type: 'string', required: true },
|
||||||
|
email: { type: 'string', unique: true, index: true },
|
||||||
|
age: { type: 'number', default: 0 },
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
tx.objectStore('__metona_schema').put({
|
||||||
|
name: 'orders',
|
||||||
|
schema: JSON.stringify({
|
||||||
|
name: 'orders',
|
||||||
|
columns: {
|
||||||
|
id: { type: 'string', primaryKey: true },
|
||||||
|
user_id: { type: 'string' },
|
||||||
|
amount: { type: 'number' },
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
tx.oncomplete = () => { db.close(); resolve(); };
|
||||||
|
tx.onerror = () => reject(tx.error);
|
||||||
|
};
|
||||||
|
request.onerror = () => reject(request.error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 构造旧 aria 模式库(IndexedDBBackend 布局:单 data store + aria- 前缀) */
|
||||||
|
async function createLegacyAriaDB(dbName: string): Promise<void> {
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(`aria-${dbName}`, 1);
|
||||||
|
request.onupgradeneeded = () => {
|
||||||
|
request.result.createObjectStore('data');
|
||||||
|
};
|
||||||
|
request.onsuccess = () => {
|
||||||
|
const db = request.result;
|
||||||
|
const tx = db.transaction('data', 'readwrite');
|
||||||
|
const store = tx.objectStore('data');
|
||||||
|
store.put(new TextEncoder().encode('legacy').buffer, '__aria_keymeta_test');
|
||||||
|
tx.oncomplete = () => { db.close(); resolve(); };
|
||||||
|
tx.onerror = () => reject(tx.error);
|
||||||
|
};
|
||||||
|
request.onerror = () => reject(request.error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('migrateFromIndexedDB', () => {
|
||||||
|
it('disk 模式旧库:schema + 数据完整迁移(含索引标记)', async () => {
|
||||||
|
const legacyName = uniqueDB();
|
||||||
|
await createLegacyDiskDB(legacyName);
|
||||||
|
|
||||||
|
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
|
||||||
|
await target.init();
|
||||||
|
|
||||||
|
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
|
||||||
|
|
||||||
|
expect(result.migratedTables.sort()).toEqual(['orders', 'users']);
|
||||||
|
expect(result.rowCount).toBe(3);
|
||||||
|
expect(await target.table('users').count()).toBe(2);
|
||||||
|
expect(await target.table('orders').count()).toBe(1);
|
||||||
|
|
||||||
|
// schema 保留(索引/唯一标记)
|
||||||
|
const schema = await target.getEngine().getTableSchema('users');
|
||||||
|
expect(schema!.columns.email.unique).toBe(true);
|
||||||
|
expect(schema!.columns.email.index).toBe(true);
|
||||||
|
|
||||||
|
// 数据正确
|
||||||
|
const alice = await target.table('users').select().where({ id: '1' }).execute();
|
||||||
|
expect(alice[0].email).toBe('alice@x.com');
|
||||||
|
|
||||||
|
// 重启后(持久化)数据保留
|
||||||
|
await target.close();
|
||||||
|
const target2 = new MetonaSqlark({ name: target.name, mode: 'disk' });
|
||||||
|
await target2.init();
|
||||||
|
expect(await target2.table('users').count()).toBe(2);
|
||||||
|
await target2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('无持久化 schema 的旧库:从数据推断', async () => {
|
||||||
|
const legacyName = uniqueDB();
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(legacyName, 1);
|
||||||
|
request.onupgradeneeded = () => {
|
||||||
|
request.result.createObjectStore('legacy_tbl', { keyPath: 'id' });
|
||||||
|
};
|
||||||
|
request.onsuccess = () => {
|
||||||
|
const db = request.result;
|
||||||
|
const tx = db.transaction('legacy_tbl', 'readwrite');
|
||||||
|
tx.objectStore('legacy_tbl').add({ id: 'a', name: 'X', age: 10 });
|
||||||
|
tx.oncomplete = () => { db.close(); resolve(); };
|
||||||
|
tx.onerror = () => reject(tx.error);
|
||||||
|
};
|
||||||
|
request.onerror = () => reject(request.error);
|
||||||
|
});
|
||||||
|
|
||||||
|
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
|
||||||
|
await target.init();
|
||||||
|
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
|
||||||
|
|
||||||
|
expect(result.migratedTables).toEqual(['legacy_tbl']);
|
||||||
|
expect(result.rowCount).toBe(1);
|
||||||
|
const rows = await target.table('legacy_tbl').select().execute();
|
||||||
|
expect(rows[0].name).toBe('X');
|
||||||
|
await target.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('目标库已存在的表跳过(不覆盖)', async () => {
|
||||||
|
const legacyName = uniqueDB();
|
||||||
|
await createLegacyDiskDB(legacyName);
|
||||||
|
|
||||||
|
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
|
||||||
|
await target.init();
|
||||||
|
await target.defineTable('users', { id: { type: 'string', primaryKey: true } });
|
||||||
|
await target.table('users').insert({ id: 'keep' });
|
||||||
|
|
||||||
|
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
|
||||||
|
expect(result.skippedTables).toContain('users');
|
||||||
|
// 已存在的表未被覆盖
|
||||||
|
expect(await target.table('users').count()).toBe(1);
|
||||||
|
await target.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('空旧库(无业务 store)→ 返回空迁移结果', async () => {
|
||||||
|
const legacyName = uniqueDB();
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const request = indexedDB.open(legacyName, 1);
|
||||||
|
request.onsuccess = () => { request.result.close(); resolve(); };
|
||||||
|
request.onerror = () => reject(request.error);
|
||||||
|
});
|
||||||
|
|
||||||
|
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
|
||||||
|
await target.init();
|
||||||
|
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
|
||||||
|
expect(result.migratedTables).toEqual([]);
|
||||||
|
expect(result.rowCount).toBe(0);
|
||||||
|
await target.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('aria 模式旧库(引擎私有格式)→ 明确抛错说明不支持', async () => {
|
||||||
|
const legacyName = uniqueDB();
|
||||||
|
await createLegacyAriaDB(legacyName);
|
||||||
|
|
||||||
|
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
|
||||||
|
await target.init();
|
||||||
|
await expect(migrateFromIndexedDB({ dbName: legacyName, engine: 'aria' as never, target }))
|
||||||
|
.rejects.toThrow('not supported');
|
||||||
|
await target.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -4,9 +4,7 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { MemoryEngine } from '../src/engine/memory';
|
import { MemoryEngine } from '../src/engine/memory';
|
||||||
import { IndexedDBEngine } from '../src/engine/indexeddb';
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
|
|
||||||
@@ -51,75 +49,6 @@ describe('v0.1.14 幂等 init — MemoryEngine', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
// IndexedDB flushToIDB 原子性
|
|
||||||
// ===================================================================
|
|
||||||
|
|
||||||
describe('v0.1.14 原子性 flush — IndexedDBEngine', () => {
|
|
||||||
let engine: IndexedDBEngine;
|
|
||||||
let dbName: string;
|
|
||||||
|
|
||||||
beforeEach(async () => {
|
|
||||||
dbName = uniqueDB();
|
|
||||||
engine = new IndexedDBEngine();
|
|
||||||
await engine.open(dbName, 1);
|
|
||||||
});
|
|
||||||
|
|
||||||
afterEach(async () => {
|
|
||||||
try { await engine.close(); } catch {}
|
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
|
||||||
|
|
||||||
it('事务 commit 后内存数据完整', async () => {
|
|
||||||
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
|
||||||
await engine.beginTransaction();
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }]);
|
|
||||||
await engine.commitTransaction();
|
|
||||||
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('事务 commit 后混合操作数据完整', async () => {
|
|
||||||
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
|
||||||
|
|
||||||
await engine.beginTransaction();
|
|
||||||
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
|
|
||||||
await engine.update('users', { table: 'users', where: { id: '1' } }, { name: 'Alice2' });
|
|
||||||
await engine.delete('users', { table: 'users', where: { id: '2' } });
|
|
||||||
await engine.commitTransaction();
|
|
||||||
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(1);
|
|
||||||
expect(rows[0].name).toBe('Alice2');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('事务回滚后数据完全恢复', async () => {
|
|
||||||
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
|
||||||
|
|
||||||
await engine.beginTransaction();
|
|
||||||
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
|
|
||||||
await engine.rollbackTransaction();
|
|
||||||
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(1);
|
|
||||||
expect(rows[0].name).toBe('Alice');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('事务内的 find 能读到未提交的变更', async () => {
|
|
||||||
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
|
|
||||||
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
|
|
||||||
|
|
||||||
await engine.beginTransaction();
|
|
||||||
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
|
|
||||||
// 事务内的 find 应能读到事务写入的数据
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
await engine.rollbackTransaction();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
// Hybrid 引擎事务持久化
|
// Hybrid 引擎事务持久化
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
@@ -130,7 +59,7 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
|
|||||||
|
|
||||||
beforeEach(async () => {
|
beforeEach(async () => {
|
||||||
dbName = uniqueDB();
|
dbName = uniqueDB();
|
||||||
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -140,7 +69,6 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
|
|||||||
|
|
||||||
afterEach(async () => {
|
afterEach(async () => {
|
||||||
await db.close();
|
await db.close();
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('Hybrid 事务 commit 后内存和磁盘一致', async () => {
|
it('Hybrid 事务 commit 后内存和磁盘一致', async () => {
|
||||||
@@ -153,16 +81,11 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
|
|||||||
const rows = await db.table('users').select().execute();
|
const rows = await db.table('users').select().execute();
|
||||||
expect(rows).toHaveLength(2);
|
expect(rows).toHaveLength(2);
|
||||||
|
|
||||||
// 重新连接同一数据库(先关闭再创建新实例,但要先 deleteDatabase 让 fake-idb 干净)
|
// 重新打开同一库:Hybrid 从磁盘(KVStore)恢复已提交事务数据
|
||||||
await db.close();
|
await db.close();
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
|
|
||||||
// 用新实例打开(验证内存+磁盘一致性:Hybrid 应该从磁盘恢复)
|
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
|
||||||
await db2.init();
|
await db2.init();
|
||||||
// 新数据库是空的(因为删了 IDB),Hybrid 从磁盘加载也是空的
|
expect(await db2.table('users').count()).toBe(2);
|
||||||
const names2 = await db2.getTableNames();
|
|
||||||
expect(names2).toHaveLength(0);
|
|
||||||
await db2.close();
|
await db2.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -181,33 +104,3 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
|
|||||||
expect(rows[0].id).toBe('1');
|
expect(rows[0].id).toBe('1');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// ===================================================================
|
|
||||||
// onversionchange 监听
|
|
||||||
// ===================================================================
|
|
||||||
|
|
||||||
describe('v0.1.14 onversionchange — IndexedDBEngine', () => {
|
|
||||||
it('open 后 onversionchange 回调已绑定', async () => {
|
|
||||||
const dbName = uniqueDB();
|
|
||||||
const engine = new IndexedDBEngine();
|
|
||||||
await engine.open(dbName, 1);
|
|
||||||
|
|
||||||
const rawDb = (engine as any).db;
|
|
||||||
expect(rawDb).toBeTruthy();
|
|
||||||
expect(typeof rawDb.onversionchange).toBe('function');
|
|
||||||
|
|
||||||
await engine.close();
|
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
|
||||||
|
|
||||||
it('close 后 db 引用和监听器均被清理', async () => {
|
|
||||||
const dbName = uniqueDB();
|
|
||||||
const engine = new IndexedDBEngine();
|
|
||||||
await engine.open(dbName, 1);
|
|
||||||
await engine.close();
|
|
||||||
|
|
||||||
const rawDb = (engine as any).db;
|
|
||||||
expect(rawDb).toBeNull();
|
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|||||||
@@ -4,13 +4,16 @@
|
|||||||
*
|
*
|
||||||
* 覆盖:多语句 / 事务语句 / INSERT...SELECT / UNION / CREATE INDEX / EXISTS
|
* 覆盖:多语句 / 事务语句 / INSERT...SELECT / UNION / CREATE INDEX / EXISTS
|
||||||
*/
|
*/
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import '../src/connection-manager';
|
import '../src/connection-manager';
|
||||||
import { parse, parseAll } from '../src/sql/parser';
|
import { parse, parseAll } from '../src/sql/parser';
|
||||||
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
async function createDb(mode: 'memory' | 'disk' | 'aria' = 'memory') {
|
async function createDb(mode: 'memory' | 'disk' | 'aria' = 'memory') {
|
||||||
const db = new MetonaSqlark({ name: `sql-ext-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: `sql-ext-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
|
|||||||
@@ -4,14 +4,17 @@
|
|||||||
*
|
*
|
||||||
* 覆盖:CASE WHEN 表达式 / JOIN + 关联子查询 / WAL 批量组提交
|
* 覆盖:CASE WHEN 表达式 / JOIN + 关联子查询 / WAL 批量组提交
|
||||||
*/
|
*/
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import { parse } from '../src/sql/parser';
|
import { parse } from '../src/sql/parser';
|
||||||
import { WAL, type WALStore } from '../src/engine/aria/wal/log';
|
import { WAL, type WALStore } from '../src/engine/aria/wal/log';
|
||||||
import { WALRecordType } from '../src/engine/aria/types';
|
import { WALRecordType } from '../src/engine/aria/types';
|
||||||
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
async function createDb(mode: 'memory' | 'aria' = 'memory') {
|
async function createDb(mode: 'memory' | 'aria' = 'memory') {
|
||||||
const db = new MetonaSqlark({ name: `sql-ext2-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: `sql-ext2-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
|
|||||||
+17
-18
@@ -4,14 +4,13 @@
|
|||||||
*
|
*
|
||||||
* 覆盖:CASE WHEN 用于 WHERE/聚合 / JOIN 哈希连接 / 多标签页同步
|
* 覆盖:CASE WHEN 用于 WHERE/聚合 / JOIN 哈希连接 / 多标签页同步
|
||||||
*/
|
*/
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
|
|
||||||
async function createDb(mode: 'memory' | 'hybrid' = 'memory', extra: Record<string, unknown> = {}) {
|
async function createDb(mode: 'memory' | 'hybrid' = 'memory', extra: Record<string, unknown> = {}) {
|
||||||
const db = new MetonaSqlark({
|
const db = new MetonaSqlark({
|
||||||
name: `sql-ext3-${mode}-${Date.now()}-${Math.random()}`,
|
name: `sql-ext3-${mode}-${Date.now()}-${Math.random()}`,
|
||||||
mode,
|
mode,
|
||||||
diskEngine: 'indexeddb',
|
diskEngine: 'opfs',
|
||||||
...extra,
|
...extra,
|
||||||
});
|
});
|
||||||
await db.init();
|
await db.init();
|
||||||
@@ -249,13 +248,13 @@ describe('[v0.3.2] 多标签页同步', () => {
|
|||||||
|
|
||||||
test('SQL 写语句广播表变更,其他标签页订阅收到 external 事件', async () => {
|
test('SQL 写语句广播表变更,其他标签页订阅收到 external 事件', async () => {
|
||||||
// 先建表(DDL 版本升级会触发其他标签页 onversionchange 关闭连接,故先建表再开第二连接)
|
// 先建表(DDL 版本升级会触发其他标签页 onversionchange 关闭连接,故先建表再开第二连接)
|
||||||
const setup = new MetonaSqlark({ name: 'mt-a', mode: 'hybrid', diskEngine: 'indexeddb' });
|
const setup = new MetonaSqlark({ name: 'mt-a', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||||
await setup.close();
|
await setup.close();
|
||||||
|
|
||||||
const dbA = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbA = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
const dbB = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbB = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
await dbA.init();
|
await dbA.init();
|
||||||
await dbB.init();
|
await dbB.init();
|
||||||
|
|
||||||
@@ -274,13 +273,13 @@ describe('[v0.3.2] 多标签页同步', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test('Hybrid 标签页收到广播后内存重载(读到其他标签页的新数据)', async () => {
|
test('Hybrid 标签页收到广播后内存重载(读到其他标签页的新数据)', async () => {
|
||||||
const setup = new MetonaSqlark({ name: 'mt-b', mode: 'hybrid', diskEngine: 'indexeddb' });
|
const setup = new MetonaSqlark({ name: 'mt-b', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||||
await setup.close();
|
await setup.close();
|
||||||
|
|
||||||
const dbA = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbA = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
const dbB = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbB = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
await dbA.init();
|
await dbA.init();
|
||||||
await dbB.init();
|
await dbB.init();
|
||||||
|
|
||||||
@@ -305,13 +304,13 @@ describe('[v0.3.2] 多标签页同步', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test('未启用 multiTabSync 不广播', async () => {
|
test('未启用 multiTabSync 不广播', async () => {
|
||||||
const setup = new MetonaSqlark({ name: 'mt-c', mode: 'hybrid', diskEngine: 'indexeddb' });
|
const setup = new MetonaSqlark({ name: 'mt-c', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||||
await setup.close();
|
await setup.close();
|
||||||
|
|
||||||
const dbA = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const dbA = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
const dbB = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbB = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
await dbA.init();
|
await dbA.init();
|
||||||
await dbB.init();
|
await dbB.init();
|
||||||
|
|
||||||
@@ -327,13 +326,13 @@ describe('[v0.3.2] 多标签页同步', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test('Table API 写入也广播', async () => {
|
test('Table API 写入也广播', async () => {
|
||||||
const setup = new MetonaSqlark({ name: 'mt-d', mode: 'hybrid', diskEngine: 'indexeddb' });
|
const setup = new MetonaSqlark({ name: 'mt-d', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||||
await setup.close();
|
await setup.close();
|
||||||
|
|
||||||
const dbA = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbA = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
const dbB = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
const dbB = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
|
||||||
await dbA.init();
|
await dbA.init();
|
||||||
await dbB.init();
|
await dbB.init();
|
||||||
|
|
||||||
@@ -354,7 +353,7 @@ describe('[v0.3.2] 多标签页同步', () => {
|
|||||||
|
|
||||||
describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
||||||
test('close 后重新 open 表结构与数据完整', async () => {
|
test('close 后重新 open 表结构与数据完整', async () => {
|
||||||
const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'indexeddb' });
|
const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
await setup.defineTable('t', {
|
await setup.defineTable('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -365,7 +364,7 @@ describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
|||||||
await setup.close();
|
await setup.close();
|
||||||
|
|
||||||
// 重新打开(模拟页面刷新)
|
// 重新打开(模拟页面刷新)
|
||||||
const db = new MetonaSqlark({ name: 'reopen-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: 'reopen-a', version: 2, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
|
|
||||||
const schema = await db.getEngine().getTableSchema('t');
|
const schema = await db.getEngine().getTableSchema('t');
|
||||||
@@ -383,7 +382,7 @@ describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test('reopen 后 UPDATE 全列生效', async () => {
|
test('reopen 后 UPDATE 全列生效', async () => {
|
||||||
const setup = new MetonaSqlark({ name: 'reopen-b', mode: 'hybrid', diskEngine: 'indexeddb' });
|
const setup = new MetonaSqlark({ name: 'reopen-b', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
await setup.defineTable('t', {
|
await setup.defineTable('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -393,7 +392,7 @@ describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
|||||||
await setup.query(`INSERT INTO t VALUES ('1', 'Alice', 42)`);
|
await setup.query(`INSERT INTO t VALUES ('1', 'Alice', 42)`);
|
||||||
await setup.close();
|
await setup.close();
|
||||||
|
|
||||||
const db = new MetonaSqlark({ name: 'reopen-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: 'reopen-b', version: 2, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.query(`UPDATE t SET name = 'Renamed', v = 99 WHERE id = '1'`);
|
await db.query(`UPDATE t SET name = 'Renamed', v = 99 WHERE id = '1'`);
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,6 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
|
|
||||||
@@ -128,7 +127,7 @@ describe('v0.1.13 事务回滚 — Disk 模式 (IndexedDB)', () => {
|
|||||||
|
|
||||||
beforeEach(async () => {
|
beforeEach(async () => {
|
||||||
dbName = `tx-disk-${++idbCounter}`;
|
dbName = `tx-disk-${++idbCounter}`;
|
||||||
db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' });
|
db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -138,7 +137,6 @@ describe('v0.1.13 事务回滚 — Disk 模式 (IndexedDB)', () => {
|
|||||||
|
|
||||||
afterEach(async () => {
|
afterEach(async () => {
|
||||||
await db.close();
|
await db.close();
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('commit: 事务成功数据持久化', async () => {
|
it('commit: 事务成功数据持久化', async () => {
|
||||||
@@ -175,7 +173,7 @@ describe('v0.1.13 事务回滚 — Hybrid 模式', () => {
|
|||||||
|
|
||||||
beforeEach(async () => {
|
beforeEach(async () => {
|
||||||
dbName = `tx-hybrid-${++idbCounter}`;
|
dbName = `tx-hybrid-${++idbCounter}`;
|
||||||
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -185,7 +183,6 @@ describe('v0.1.13 事务回滚 — Hybrid 模式', () => {
|
|||||||
|
|
||||||
afterEach(async () => {
|
afterEach(async () => {
|
||||||
await db.close();
|
await db.close();
|
||||||
try { indexedDB.deleteDatabase(dbName); } catch {}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('commit: 事务成功', async () => {
|
it('commit: 事务成功', async () => {
|
||||||
|
|||||||
@@ -18,13 +18,17 @@ import type { SSTableMeta } from '../src/engine/aria/types';
|
|||||||
import type { MetonaPlugin } from '../src/constants';
|
import type { MetonaPlugin } from '../src/constants';
|
||||||
import { createSchema } from '../src/table/schema';
|
import { createSchema } from '../src/table/schema';
|
||||||
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// P0-1: 版本号统一
|
// P0-1: 版本号统一
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
||||||
test('VERSION 常量为当前版本(0.5.1)', () => {
|
test('VERSION 常量为当前版本(0.6.0)', () => {
|
||||||
expect(VERSION).toBe('0.5.1');
|
expect(VERSION).toBe('0.6.0');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -41,7 +45,7 @@ describe('[v0.2.5] P0-2: AriaEngine OPFS 后端映射', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test('mode=aria + diskEngine=indexeddb 时应使用 indexeddb 后端', () => {
|
test('mode=aria + diskEngine=indexeddb 时应使用 indexeddb 后端', () => {
|
||||||
const db = new MetonaSqlark({ name: `test-idb-map-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: `test-idb-map-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'opfs' });
|
||||||
expect(db).toBeDefined();
|
expect(db).toBeDefined();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -12,7 +12,6 @@
|
|||||||
* - P1-9: Savepoint 回滚后 MVCC 版本链一致 + rollback 索引重建
|
* - P1-9: Savepoint 回滚后 MVCC 版本链一致 + rollback 索引重建
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { VERSION } from '../src/constants';
|
import { VERSION } from '../src/constants';
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import { AriaEngine } from '../src/engine/aria/index';
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
@@ -20,13 +19,17 @@ import { MemoryEngine } from '../src/engine/memory';
|
|||||||
import { WAL } from '../src/engine/aria/wal/log';
|
import { WAL } from '../src/engine/aria/wal/log';
|
||||||
import { tokenize } from '../src/sql/lexer';
|
import { tokenize } from '../src/sql/lexer';
|
||||||
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// P0-1: Aria WAL DROP_TABLE 崩溃恢复
|
// P0-1: Aria WAL DROP_TABLE 崩溃恢复
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
describe('[v0.3.3] P0-1: WAL DROP_TABLE 崩溃恢复', () => {
|
describe('[v0.3.3] P0-1: WAL DROP_TABLE 崩溃恢复', () => {
|
||||||
const mkEngine = async (name: string): Promise<AriaEngine> => {
|
const mkEngine = async (name: string): Promise<AriaEngine> => {
|
||||||
const e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await e.open(name, 1);
|
await e.open(name, 1);
|
||||||
return e;
|
return e;
|
||||||
};
|
};
|
||||||
@@ -400,7 +403,7 @@ describe('[v0.3.3] P1-9: Savepoint + MVCC 一致性', () => {
|
|||||||
|
|
||||||
describe('[v0.3.3] 端到端', () => {
|
describe('[v0.3.3] 端到端', () => {
|
||||||
test('全部修复点可共存于 MetonaSqlark API', async () => {
|
test('全部修复点可共存于 MetonaSqlark API', async () => {
|
||||||
expect(VERSION).toBe('0.5.1');
|
expect(VERSION).toBe('0.6.0');
|
||||||
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
|
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('users', {
|
await db.defineTable('users', {
|
||||||
|
|||||||
@@ -6,10 +6,13 @@
|
|||||||
* - B-4: COUNT(DISTINCT) + NULLS FIRST/LAST
|
* - B-4: COUNT(DISTINCT) + NULLS FIRST/LAST
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import { AriaEngine } from '../src/engine/aria/index';
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
const uniqueName = (prefix: string): string =>
|
const uniqueName = (prefix: string): string =>
|
||||||
`${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
`${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
||||||
|
|
||||||
@@ -366,7 +369,7 @@ describe('[v0.4.0] B-4: COUNT(DISTINCT) + NULLS FIRST/LAST', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test('Aria $in 查询去重(IN 子查询含重复值不返回重复行)', async () => {
|
test('Aria $in 查询去重(IN 子查询含重复值不返回重复行)', async () => {
|
||||||
const db = new MetonaSqlark({ name: uniqueName('indup'), mode: 'aria', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: uniqueName('indup'), mode: 'aria', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.getEngine().clearAll();
|
await db.getEngine().clearAll();
|
||||||
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
||||||
|
|||||||
+80
-117
@@ -15,11 +15,15 @@ import { AriaEngine } from '../src/engine/aria/index';
|
|||||||
import { createSchema } from '../src/table/schema';
|
import { createSchema } from '../src/table/schema';
|
||||||
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
|
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
|
||||||
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
||||||
import { IndexedDBEngine } from '../src/engine/indexeddb';
|
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import { IndexedDBBackend } from '../src/engine/aria/store/backend';
|
import { OPFSBackend } from '../src/engine/aria/store/opfs_backend';
|
||||||
|
import { KVStoreEngine } from '../src/engine/kvstore_engine';
|
||||||
|
import { SharedMemoryBackend } from '../src/engine/kvstore/shared_memory_medium';
|
||||||
import type { SSTableMeta } from '../src/engine/aria/types';
|
import type { SSTableMeta } from '../src/engine/aria/types';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -133,7 +137,7 @@ describe('P0-1a — SSTableReader 残缺数据防御', () => {
|
|||||||
describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
||||||
it('meta 引用残缺文件 → 打开跳过损坏 SSTable 不崩溃,库可用', async () => {
|
it('meta 引用残缺文件 → 打开跳过损坏 SSTable 不崩溃,库可用', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('users', {
|
await engine.createTable(createSchema('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -145,17 +149,18 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
|||||||
await (engine as any).lsm.flush();
|
await (engine as any).lsm.flush();
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// 篡改存储:把第一个 sst_ 文件写成残缺内容(meta 仍引用它)
|
// 篡改存储:把第一个 SSTable 的第一个页面写成残缺内容(meta 仍引用它)
|
||||||
const backend = new IndexedDBBackend();
|
const backend = new OPFSBackend();
|
||||||
await backend.open(dbName);
|
await backend.open(dbName);
|
||||||
const sstKeys = (await backend.listKeys())
|
const metas = JSON.parse(new TextDecoder().decode(
|
||||||
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
await backend.read('__aria_lsm_meta') as ArrayBuffer)) as { id: number; pageIds: number[] }[];
|
||||||
expect(sstKeys.length).toBeGreaterThan(0);
|
expect(metas.length).toBeGreaterThan(0);
|
||||||
await backend.write(sstKeys[0], new TextEncoder().encode('truncated-garbage').buffer);
|
const pageId = metas[0].pageIds[0];
|
||||||
|
await backend.write(`pg_${pageId}`, new TextEncoder().encode('truncated-garbage').buffer);
|
||||||
await backend.close();
|
await backend.close();
|
||||||
|
|
||||||
// 重开:不得崩溃(此前抛 RangeError 打不开库)
|
// 重开:不得崩溃(此前抛 RangeError 打不开库)
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
expect(engine2.isOpen()).toBe(true);
|
expect(engine2.isOpen()).toBe(true);
|
||||||
// schema 完整,损坏的 SSTable 已被清理
|
// schema 完整,损坏的 SSTable 已被清理
|
||||||
@@ -165,19 +170,21 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
|||||||
await engine2.close();
|
await engine2.close();
|
||||||
|
|
||||||
// 清理 meta 已验证:损坏文件被删除
|
// 清理 meta 已验证:损坏文件被删除
|
||||||
const backend2 = new IndexedDBBackend();
|
const backend2 = new OPFSBackend();
|
||||||
await backend2.open(dbName);
|
await backend2.open(dbName);
|
||||||
const remaining = (await backend2.listKeys())
|
const remaining = (await backend2.listKeys()).filter((k) => k.startsWith('pg_'));
|
||||||
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
|
||||||
const metaRaw = await backend2.read('__aria_lsm_meta');
|
const metaRaw = await backend2.read('__aria_lsm_meta');
|
||||||
const metaList = JSON.parse(new TextDecoder().decode(metaRaw ?? new Uint8Array())) as { id: number }[];
|
const metaList = JSON.parse(new TextDecoder().decode(metaRaw ?? new Uint8Array())) as { pageIds?: number[] }[];
|
||||||
expect(remaining.length).toBe(metaList.length); // 无孤儿文件 / 无悬空 meta
|
const livePages = new Set<number>();
|
||||||
|
for (const m of metaList) if (m.pageIds) for (const pid of m.pageIds) livePages.add(pid);
|
||||||
|
const orphan = remaining.filter((k) => !livePages.has(Number(k.slice(3))));
|
||||||
|
expect(orphan.length).toBe(0); // 无孤儿页面
|
||||||
await backend2.close();
|
await backend2.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('meta 引用缺失文件 → 打开清理 meta 不崩溃', async () => {
|
it('meta 引用缺失页面 → 打开清理 meta 不崩溃', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -187,15 +194,14 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
|||||||
await (engine as any).lsm.flush();
|
await (engine as any).lsm.flush();
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// 删除全部 sst_ 文件(保留 meta → 悬空引用)
|
// 删除全部 pg_ 页面(保留 meta → 悬空引用)
|
||||||
const backend = new IndexedDBBackend();
|
const backend = new OPFSBackend();
|
||||||
await backend.open(dbName);
|
await backend.open(dbName);
|
||||||
const sstKeys = (await backend.listKeys())
|
const pgKeys = (await backend.listKeys()).filter((k) => k.startsWith('pg_'));
|
||||||
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
await backend.deleteMany(pgKeys);
|
||||||
await backend.deleteMany(sstKeys);
|
|
||||||
await backend.close();
|
await backend.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
expect(engine2.isOpen()).toBe(true);
|
expect(engine2.isOpen()).toBe(true);
|
||||||
expect(await engine2.getTableNames()).toEqual(['t']);
|
expect(await engine2.getTableNames()).toEqual(['t']);
|
||||||
@@ -210,7 +216,7 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
|||||||
describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
||||||
it('IndexedDBBackend.writeMany/deleteMany 单事务原子批量操作', async () => {
|
it('IndexedDBBackend.writeMany/deleteMany 单事务原子批量操作', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const be = new IndexedDBBackend();
|
const be = new OPFSBackend();
|
||||||
await be.open(dbName);
|
await be.open(dbName);
|
||||||
await be.writeMany({
|
await be.writeMany({
|
||||||
a: new TextEncoder().encode('1').buffer,
|
a: new TextEncoder().encode('1').buffer,
|
||||||
@@ -226,7 +232,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
|||||||
|
|
||||||
it('P1-4: 连续快速写入 200 条 → close+重开 数据完整(实测曾丢 4 条)', async () => {
|
it('P1-4: 连续快速写入 200 条 → close+重开 数据完整(实测曾丢 4 条)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const engine = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -237,7 +243,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
|||||||
}
|
}
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
expect(await engine2.count('t')).toBe(200);
|
expect(await engine2.count('t')).toBe(200);
|
||||||
await engine2.close();
|
await engine2.close();
|
||||||
@@ -245,7 +251,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
|||||||
|
|
||||||
it('P1-4: count 键丢失(模拟崩溃竞态)→ 按 key 扫描恢复不丢记录', async () => {
|
it('P1-4: count 键丢失(模拟崩溃竞态)→ 按 key 扫描恢复不丢记录', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const engine = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -256,12 +262,12 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 模拟异常退出(不 close):直接删掉 count 键,制造 count 与记录不一致
|
// 模拟异常退出(不 close):直接删掉 count 键,制造 count 与记录不一致
|
||||||
const backend = new IndexedDBBackend();
|
const backend = new OPFSBackend();
|
||||||
await backend.open(dbName);
|
await backend.open(dbName);
|
||||||
await backend.delete('__wal_count');
|
await backend.delete('__wal_count');
|
||||||
await backend.close();
|
await backend.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
expect(await engine2.count('t')).toBe(20);
|
expect(await engine2.count('t')).toBe(20);
|
||||||
await engine2.close();
|
await engine2.close();
|
||||||
@@ -275,7 +281,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
|||||||
describe('P1-5 — AriaEngine.close 截断 WAL', () => {
|
describe('P1-5 — AriaEngine.close 截断 WAL', () => {
|
||||||
it('close 后 WAL 记录键全部清空(不无限重放)', async () => {
|
it('close 后 WAL 记录键全部清空(不无限重放)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' });
|
const engine = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -284,7 +290,7 @@ describe('P1-5 — AriaEngine.close 截断 WAL', () => {
|
|||||||
await engine.insert('t', [{ id: '1', v: 1 }, { id: '2', v: 2 }]);
|
await engine.insert('t', [{ id: '1', v: 1 }, { id: '2', v: 2 }]);
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const backend = new IndexedDBBackend();
|
const backend = new OPFSBackend();
|
||||||
await backend.open(dbName);
|
await backend.open(dbName);
|
||||||
// __wal_count 计数键合法保留(作为 append 序号分配器),WAL 记录键必须清空
|
// __wal_count 计数键合法保留(作为 append 序号分配器),WAL 记录键必须清空
|
||||||
const walKeys = (await backend.listKeys())
|
const walKeys = (await backend.listKeys())
|
||||||
@@ -294,65 +300,6 @@ describe('P1-5 — AriaEngine.close 截断 WAL', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// ===================================================================
|
|
||||||
// P0-2 / P0-3 / P2-8: IndexedDB 版本管理与参数校验
|
|
||||||
// ===================================================================
|
|
||||||
|
|
||||||
describe('P0-2 / P0-3 — IndexedDB 版本管理', () => {
|
|
||||||
it('P0-2: 建表提升版本后以旧 version 重开成功(VersionError 自适应)', async () => {
|
|
||||||
const dbName = uniqueDB();
|
|
||||||
const engine = new IndexedDBEngine();
|
|
||||||
await engine.open(dbName, 1);
|
|
||||||
await engine.createTable({ name: 'a', columns: { id: { type: 'string', primaryKey: true } } });
|
|
||||||
await engine.createTable({ name: 'b', columns: { id: { type: 'string', primaryKey: true } } });
|
|
||||||
await engine.createTable({ name: 'c', columns: { id: { type: 'string', primaryKey: true } } });
|
|
||||||
await engine.close();
|
|
||||||
|
|
||||||
// 用旧版本 1 重开:此前必报 IDB_OPEN_ERROR(VersionError)
|
|
||||||
const engine2 = new IndexedDBEngine();
|
|
||||||
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
|
||||||
const names = await engine2.getTableNames();
|
|
||||||
expect(names).toContain('a');
|
|
||||||
expect(names).toContain('b');
|
|
||||||
expect(names).toContain('c');
|
|
||||||
await engine2.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('P0-2: MetonaSqlark hybrid 二次启动不报 VersionError(MarkLite 场景)', async () => {
|
|
||||||
const dbName = uniqueDB();
|
|
||||||
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 1 });
|
|
||||||
await db1.init();
|
|
||||||
await db1.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
|
||||||
await db1.table('users').insert({ id: '1', name: 'Alice' });
|
|
||||||
await db1.close();
|
|
||||||
|
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 1 });
|
|
||||||
await expect(db2.init()).resolves.toBeUndefined();
|
|
||||||
expect(await db2.table('users').count()).toBe(1);
|
|
||||||
await db2.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('P0-3: IndexedDBEngine open(version=0) 归一化为 1,不抛 TypeError', async () => {
|
|
||||||
const dbName = uniqueDB();
|
|
||||||
const engine = new IndexedDBEngine();
|
|
||||||
await expect(engine.open(dbName, 0)).resolves.toBeUndefined();
|
|
||||||
expect(engine.isOpen()).toBe(true);
|
|
||||||
await engine.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
|
|
||||||
await engine.insert('t', [{ id: '1' }]);
|
|
||||||
expect(await engine.find('t', { table: 't' })).toHaveLength(1);
|
|
||||||
await engine.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('P0-3: MetonaSqlark version=0 初始化正常', async () => {
|
|
||||||
const dbName = uniqueDB();
|
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
|
|
||||||
await expect(db.init()).resolves.toBeUndefined();
|
|
||||||
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
|
||||||
await db.table('t').insert({ id: '1' });
|
|
||||||
await db.close();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
// P1-6: 引擎错误包装 DatabaseError
|
// P1-6: 引擎错误包装 DatabaseError
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
@@ -362,24 +309,36 @@ describe('P1-6 — 引擎内部错误统一包装 DatabaseError', () => {
|
|||||||
// 构造一个无法打开的 backend 场景:直接调用 openInternal 模拟底层异常不可行,
|
// 构造一个无法打开的 backend 场景:直接调用 openInternal 模拟底层异常不可行,
|
||||||
// 这里验证损坏库重开时抛的是 DatabaseError 而非原生错误(不崩溃路径已由 P0-1b 覆盖)
|
// 这里验证损坏库重开时抛的是 DatabaseError 而非原生错误(不崩溃路径已由 P0-1b 覆盖)
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
await engine.insert('t', [{ id: '1' }]);
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
expect(engine2.isOpen()).toBe(true);
|
expect(engine2.isOpen()).toBe(true);
|
||||||
await engine2.close();
|
await engine2.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('open 未初始化 IndexedDB 环境时抛 DatabaseError 而非原生错误', async () => {
|
it('底层介质故障时抛 DatabaseError 而非原生错误', async () => {
|
||||||
const dbName = uniqueDB();
|
const failing = {
|
||||||
const engine = new IndexedDBEngine();
|
open: async () => { throw new Error('disk failure'); },
|
||||||
await engine.open(dbName, 1);
|
close: async () => {},
|
||||||
expect(engine.isOpen()).toBe(true);
|
isOpen: () => false,
|
||||||
await engine.close();
|
read: async () => null,
|
||||||
|
write: async () => { throw new Error('disk failure'); },
|
||||||
|
append: async () => {},
|
||||||
|
writeMany: async () => {},
|
||||||
|
delete: async () => {},
|
||||||
|
deleteMany: async () => {},
|
||||||
|
listKeys: async () => [],
|
||||||
|
exists: async () => false,
|
||||||
|
clear: async () => {},
|
||||||
|
};
|
||||||
|
const engine = new KVStoreEngine(failing as never);
|
||||||
|
// 介质故障如实传播(不吞错),AriaEngine 路径由 ARIA_OPEN_ERROR 包装(上方测试覆盖)
|
||||||
|
await expect(engine.open(uniqueDB(), 1)).rejects.toThrow('disk failure');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -393,7 +352,7 @@ describe('P2-7 — 迁移版本持久化', () => {
|
|||||||
const runs: number[] = [];
|
const runs: number[] = [];
|
||||||
|
|
||||||
// version: 0 表示"无 schema 起点",migration 1 可执行
|
// version: 0 表示"无 schema 起点",migration 1 可执行
|
||||||
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
|
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs', version: 0 });
|
||||||
await db1.init();
|
await db1.init();
|
||||||
db1.addMigration(1, async () => { runs.push(1); });
|
db1.addMigration(1, async () => { runs.push(1); });
|
||||||
db1.addMigration(2, async () => { runs.push(2); });
|
db1.addMigration(2, async () => { runs.push(2); });
|
||||||
@@ -402,7 +361,7 @@ describe('P2-7 — 迁移版本持久化', () => {
|
|||||||
await db1.close();
|
await db1.close();
|
||||||
|
|
||||||
// 重启:version 又重置为 config.version=0(此前会重跑 migration 1/2)
|
// 重启:version 又重置为 config.version=0(此前会重跑 migration 1/2)
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs', version: 0 });
|
||||||
await db2.init();
|
await db2.init();
|
||||||
db2.addMigration(1, async () => { runs.push(1); });
|
db2.addMigration(1, async () => { runs.push(1); });
|
||||||
db2.addMigration(2, async () => { runs.push(2); });
|
db2.addMigration(2, async () => { runs.push(2); });
|
||||||
@@ -440,35 +399,39 @@ describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
|
|||||||
|
|
||||||
it('AriaEngine.repair 清理损坏 SSTable 后引擎可用', async () => {
|
it('AriaEngine.repair 清理损坏 SSTable 后引擎可用', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 4096 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
v: { type: 'number' },
|
v: { type: 'number' },
|
||||||
}));
|
}));
|
||||||
for (let i = 0; i < 100; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
|
for (let i = 0; i < 50; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
|
||||||
|
await (engine as any).lsm.flush();
|
||||||
|
for (let i = 50; i < 100; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
|
||||||
await (engine as any).lsm.flush();
|
await (engine as any).lsm.flush();
|
||||||
|
|
||||||
// 篡改一个 sst 文件
|
// 篡改一个 SSTable 的页面文件
|
||||||
const backend = new IndexedDBBackend();
|
const backend = new OPFSBackend();
|
||||||
await backend.open(dbName);
|
await backend.open(dbName);
|
||||||
const sstKeys = (await backend.listKeys())
|
const metas = JSON.parse(new TextDecoder().decode(
|
||||||
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
|
await backend.read('__aria_lsm_meta') as ArrayBuffer)) as { id: number; pageIds: number[] }[];
|
||||||
expect(sstKeys.length).toBeGreaterThan(0);
|
expect(metas.length).toBeGreaterThan(0);
|
||||||
await backend.write(sstKeys[0], new TextEncoder().encode('garbage').buffer);
|
const victimPage = metas[0].pageIds[0];
|
||||||
|
const raw = new Uint8Array(await backend.read(`pg_${victimPage}`) as ArrayBuffer);
|
||||||
|
raw[50] ^= 0xff;
|
||||||
|
await backend.write(`pg_${victimPage}`, raw.buffer as ArrayBuffer);
|
||||||
await backend.close();
|
await backend.close();
|
||||||
|
|
||||||
await expect(engine.repair()).resolves.toBeUndefined();
|
// repair:校验清理损坏 SSTable,其余数据可用
|
||||||
const rows = await engine.find('t', { table: 't' });
|
await (engine as any).repair();
|
||||||
expect(Array.isArray(rows)).toBe(true);
|
expect(await engine.count('t')).toBeGreaterThan(0);
|
||||||
expect(rows.length).toBeLessThanOrEqual(100);
|
expect(await engine.count('t')).toBeLessThan(100);
|
||||||
expect(engine.isOpen()).toBe(true);
|
|
||||||
await engine.close();
|
await engine.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('MetonaSqlark.clearAll 统一接口(hybrid)', async () => {
|
it('MetonaSqlark.clearAll 统一接口(hybrid)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
await db.table('t').insert({ id: '1' });
|
await db.table('t').insert({ id: '1' });
|
||||||
@@ -479,7 +442,7 @@ describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
|
|||||||
|
|
||||||
it('MetonaSqlark.repair 统一接口(aria)', async () => {
|
it('MetonaSqlark.repair 统一接口(aria)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'aria', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'aria', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||||
await db.table('t').insert({ id: '1', v: 42 });
|
await db.table('t').insert({ id: '1', v: 42 });
|
||||||
@@ -490,7 +453,7 @@ describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
|
|||||||
|
|
||||||
it('MetonaSqlark.repair 统一接口(hybrid)', async () => {
|
it('MetonaSqlark.repair 统一接口(hybrid)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
await db.table('t').insert({ id: '1' });
|
await db.table('t').insert({ id: '1' });
|
||||||
|
|||||||
@@ -13,7 +13,10 @@ import { createSchema } from '../src/table/schema';
|
|||||||
import { MemoryEngine } from '../src/engine/memory';
|
import { MemoryEngine } from '../src/engine/memory';
|
||||||
import { HybridEngine } from '../src/hybrid/index';
|
import { HybridEngine } from '../src/hybrid/index';
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -53,7 +56,7 @@ describe('P0-A — Compaction 缓存独立性', () => {
|
|||||||
it('多层级 compaction 后数据仍完整且可重开', async () => {
|
it('多层级 compaction 后数据仍完整且可重开', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
bufferPoolPages: 8, // 32KB 缓存
|
bufferPoolPages: 8, // 32KB 缓存
|
||||||
memtableSizeThreshold: 16 * 1024,
|
memtableSizeThreshold: 16 * 1024,
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
@@ -74,7 +77,7 @@ describe('P0-A — Compaction 缓存独立性', () => {
|
|||||||
|
|
||||||
// 重开:数据完整(meta/文件未被 compaction 破坏)
|
// 重开:数据完整(meta/文件未被 compaction 破坏)
|
||||||
const engine2 = new AriaEngine({
|
const engine2 = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
bufferPoolPages: 8,
|
bufferPoolPages: 8,
|
||||||
memtableSizeThreshold: 16 * 1024,
|
memtableSizeThreshold: 16 * 1024,
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
@@ -92,7 +95,7 @@ describe('P0-A — Compaction 缓存独立性', () => {
|
|||||||
describe('P1-A — 二级索引恢复', () => {
|
describe('P1-A — 二级索引恢复', () => {
|
||||||
it('Aria 重开后二级索引可用(索引 LSM 持久化恢复)', async () => {
|
it('Aria 重开后二级索引可用(索引 LSM 持久化恢复)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('users', {
|
await engine.createTable(createSchema('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -107,7 +110,7 @@ describe('P1-A — 二级索引恢复', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// 重开:二级索引 LSM 应自动恢复(修复前为空 → 索引查询回退全表,createIndex 也静默跳过)
|
// 重开:二级索引 LSM 应自动恢复(修复前为空 → 索引查询回退全表,createIndex 也静默跳过)
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
// 强断言:索引 LSM 真实存在且数据完整(防止"索引缺失静默回退全表扫描"的假通过)
|
// 强断言:索引 LSM 真实存在且数据完整(防止"索引缺失静默回退全表扫描"的假通过)
|
||||||
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:email');
|
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:email');
|
||||||
@@ -124,7 +127,7 @@ describe('P1-A — 二级索引恢复', () => {
|
|||||||
|
|
||||||
it('WAL 恢复后二级索引与主数据一致(崩溃前索引未更新)', async () => {
|
it('WAL 恢复后二级索引与主数据一致(崩溃前索引未更新)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('users', {
|
await engine.createTable(createSchema('users', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -138,7 +141,7 @@ describe('P1-A — 二级索引恢复', () => {
|
|||||||
await (engine as any).backend.close();
|
await (engine as any).backend.close();
|
||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
// 强断言:索引 LSM 已恢复且包含 WAL 回放的行(崩溃前索引未更新,恢复后必须重建)
|
// 强断言:索引 LSM 已恢复且包含 WAL 回放的行(崩溃前索引未更新,恢复后必须重建)
|
||||||
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:city');
|
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:city');
|
||||||
@@ -154,7 +157,7 @@ describe('P1-A — 二级索引恢复', () => {
|
|||||||
|
|
||||||
it('createIndex 重开后仍可新建(schema 标记恢复后不阻塞)', async () => {
|
it('createIndex 重开后仍可新建(schema 标记恢复后不阻塞)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -163,7 +166,7 @@ describe('P1-A — 二级索引恢复', () => {
|
|||||||
await engine.insert('t', [{ id: '1', name: 'A' }]);
|
await engine.insert('t', [{ id: '1', name: 'A' }]);
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
await engine2.createIndex('t', 'name'); // 修复前 schema 无标记时正常;此处验证无标记场景
|
await engine2.createIndex('t', 'name'); // 修复前 schema 无标记时正常;此处验证无标记场景
|
||||||
const byName = await engine2.find('t', { table: 't', where: { name: 'A' } });
|
const byName = await engine2.find('t', { table: 't', where: { name: 'A' } });
|
||||||
@@ -177,9 +180,9 @@ describe('P1-A — 二级索引恢复', () => {
|
|||||||
// ===================================================================
|
// ===================================================================
|
||||||
|
|
||||||
describe('P1-B — ALTER TABLE 持久化', () => {
|
describe('P1-B — ALTER TABLE 持久化', () => {
|
||||||
it('IndexedDBEngine DROP COLUMN 后重启不复活', async () => {
|
it('KVStoreEngine DROP COLUMN 后重启不复活', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('t', {
|
await db.defineTable('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -194,7 +197,7 @@ describe('P1-B — ALTER TABLE 持久化', () => {
|
|||||||
await db.close();
|
await db.close();
|
||||||
|
|
||||||
// 重启:schema 不复活,列定义已持久化
|
// 重启:schema 不复活,列定义已持久化
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' });
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
|
||||||
await db2.init();
|
await db2.init();
|
||||||
const schema2 = await db2.getEngine().getTableSchema('t');
|
const schema2 = await db2.getEngine().getTableSchema('t');
|
||||||
expect(schema2!.columns.old_col).toBeUndefined();
|
expect(schema2!.columns.old_col).toBeUndefined();
|
||||||
@@ -204,13 +207,13 @@ describe('P1-B — ALTER TABLE 持久化', () => {
|
|||||||
|
|
||||||
it('HybridEngine ADD COLUMN 后重启保留', async () => {
|
it('HybridEngine ADD COLUMN 后重启保留', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
|
||||||
await db.query('ALTER TABLE t ADD COLUMN phone STRING');
|
await db.query('ALTER TABLE t ADD COLUMN phone STRING');
|
||||||
await db.close();
|
await db.close();
|
||||||
|
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db2.init();
|
await db2.init();
|
||||||
const schema2 = await db2.getEngine().getTableSchema('t');
|
const schema2 = await db2.getEngine().getTableSchema('t');
|
||||||
expect(schema2!.columns.phone).toBeDefined();
|
expect(schema2!.columns.phone).toBeDefined();
|
||||||
@@ -219,13 +222,13 @@ describe('P1-B — ALTER TABLE 持久化', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
// P1-C: IndexedDB 事务内 DDL
|
// P1-C: KVStore 事务内 DDL
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
|
|
||||||
describe('P1-C — IndexedDB 事务内 DDL', () => {
|
describe('P1-C — KVStore 事务内 DDL', () => {
|
||||||
it('事务内建表 → commit 后磁盘一致(重启表存在且可查)', async () => {
|
it('事务内建表 → commit 后磁盘一致(重启表存在且可查)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('base', { id: { type: 'string', primaryKey: true } });
|
await db.defineTable('base', { id: { type: 'string', primaryKey: true } });
|
||||||
await db.table('base').insert({ id: '1' });
|
await db.table('base').insert({ id: '1' });
|
||||||
@@ -240,7 +243,7 @@ describe('P1-C — IndexedDB 事务内 DDL', () => {
|
|||||||
expect(await db.table('created_in_tx').count()).toBe(1);
|
expect(await db.table('created_in_tx').count()).toBe(1);
|
||||||
await db.close();
|
await db.close();
|
||||||
|
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db2.init();
|
await db2.init();
|
||||||
expect(await db2.table('created_in_tx').count()).toBe(1);
|
expect(await db2.table('created_in_tx').count()).toBe(1);
|
||||||
expect(await db2.table('base').count()).toBe(2);
|
expect(await db2.table('base').count()).toBe(2);
|
||||||
@@ -249,7 +252,7 @@ describe('P1-C — IndexedDB 事务内 DDL', () => {
|
|||||||
|
|
||||||
it('事务内删表 → commit 后磁盘一致(重启无幽灵表)', async () => {
|
it('事务内删表 → commit 后磁盘一致(重启无幽灵表)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db.init();
|
await db.init();
|
||||||
await db.defineTable('ghost', { id: { type: 'string', primaryKey: true } });
|
await db.defineTable('ghost', { id: { type: 'string', primaryKey: true } });
|
||||||
await db.table('ghost').insert({ id: '1' });
|
await db.table('ghost').insert({ id: '1' });
|
||||||
@@ -259,7 +262,7 @@ describe('P1-C — IndexedDB 事务内 DDL', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
await db.close();
|
await db.close();
|
||||||
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
|
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await db2.init();
|
await db2.init();
|
||||||
const names = await db2.getTableNames();
|
const names = await db2.getTableNames();
|
||||||
expect(names).not.toContain('ghost');
|
expect(names).not.toContain('ghost');
|
||||||
@@ -300,7 +303,7 @@ describe('全模式抽查 — 生命周期与元数据', () => {
|
|||||||
|
|
||||||
it('AriaEngine repair 幂等', async () => {
|
it('AriaEngine repair 幂等', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb' });
|
const engine = new AriaEngine({ storageBackend: 'opfs' });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
||||||
await engine.insert('t', [{ id: '1' }]);
|
await engine.insert('t', [{ id: '1' }]);
|
||||||
|
|||||||
@@ -9,10 +9,12 @@
|
|||||||
|
|
||||||
import { AriaEngine } from '../src/engine/aria/index';
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
import { createSchema } from '../src/table/schema';
|
import { createSchema } from '../src/table/schema';
|
||||||
import { OPFSEngine } from '../src/engine/opfs';
|
|
||||||
import { MemTable } from '../src/engine/aria/index/memtable';
|
import { MemTable } from '../src/engine/aria/index/memtable';
|
||||||
import { MetonaSqlark } from '../src/core';
|
import { MetonaSqlark } from '../src/core';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -73,7 +75,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
|
|||||||
it('事务中触发 checkpoint → 崩溃恢复不丢事务数据', async () => {
|
it('事务中触发 checkpoint → 崩溃恢复不丢事务数据', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 2, // 每 2 次操作即触发 checkpoint(事务中途)
|
checkpointInterval: 2, // 每 2 次操作即触发 checkpoint(事务中途)
|
||||||
});
|
});
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
@@ -91,7 +93,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
|
|||||||
await (engine as any).backend.close();
|
await (engine as any).backend.close();
|
||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
// 修复前:checkpoint 截断了事务的 WAL 记录 → 恢复后数据丢失
|
// 修复前:checkpoint 截断了事务的 WAL 记录 → 恢复后数据丢失
|
||||||
expect(await engine2.count('t')).toBe(2);
|
expect(await engine2.count('t')).toBe(2);
|
||||||
@@ -101,7 +103,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
|
|||||||
it('事务回滚后再 checkpoint 正常截断', async () => {
|
it('事务回滚后再 checkpoint 正常截断', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
checkpointInterval: 1,
|
checkpointInterval: 1,
|
||||||
});
|
});
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
@@ -115,7 +117,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
|
|||||||
await (engine as any).backend.close();
|
await (engine as any).backend.close();
|
||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
expect(await engine2.count('t')).toBe(1);
|
expect(await engine2.count('t')).toBe(1);
|
||||||
await engine2.close();
|
await engine2.close();
|
||||||
@@ -168,7 +170,7 @@ describe('P1 — Aria DDL 索引清理', () => {
|
|||||||
|
|
||||||
it('DROP_TABLE 崩溃恢复同样清理索引', async () => {
|
it('DROP_TABLE 崩溃恢复同样清理索引', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('t', {
|
await engine.createTable(createSchema('t', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -180,7 +182,7 @@ describe('P1 — Aria DDL 索引清理', () => {
|
|||||||
await (engine as any).backend.close();
|
await (engine as any).backend.close();
|
||||||
(engine as any).opened = false;
|
(engine as any).opened = false;
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
// 表不存在,且无索引残留
|
// 表不存在,且无索引残留
|
||||||
expect(await engine2.hasTable('t')).toBe(false);
|
expect(await engine2.hasTable('t')).toBe(false);
|
||||||
@@ -189,88 +191,6 @@ describe('P1 — Aria DDL 索引清理', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// ===================================================================
|
|
||||||
// P1: OPFS 持久化与并发
|
|
||||||
// ===================================================================
|
|
||||||
|
|
||||||
describe('P1 — OPFS 生产加固', () => {
|
|
||||||
it('并发写不丢数据(内存快照总是最新,多写内容一致)', async () => {
|
|
||||||
// 模拟真实 I/O 延迟:内存写同步、持久化异步 → 并发写内容均基于最新内存快照
|
|
||||||
const files = mockOPFS(undefined, {
|
|
||||||
ioDelay: 10,
|
|
||||||
writeDelayFor: (data) => (data.length < 40 ? 30 : 5),
|
|
||||||
});
|
|
||||||
const engine = new OPFSEngine();
|
|
||||||
await engine.open('opfs-race', 1);
|
|
||||||
await engine.createTable(createSchema('users', {
|
|
||||||
id: { type: 'string', primaryKey: true },
|
|
||||||
name: { type: 'string' },
|
|
||||||
}));
|
|
||||||
// 两个写并发:内存同步写保证两个快照一致 → 无论完成顺序文件内容完整
|
|
||||||
const p1 = engine.insert('users', [{ id: '1', name: 'A' }]);
|
|
||||||
const p2 = engine.insert('users', [{ id: '2', name: 'B' }]);
|
|
||||||
await Promise.all([p1, p2]);
|
|
||||||
const rows = await engine.find('users', { table: 'users' });
|
|
||||||
expect(rows).toHaveLength(2);
|
|
||||||
// 文件内容完整(重启不丢)
|
|
||||||
expect(files.get('users.json')).toContain('"B"');
|
|
||||||
await engine.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('空表重启后保留(schema 持久化)', async () => {
|
|
||||||
mockOPFS();
|
|
||||||
const e1 = new OPFSEngine();
|
|
||||||
await e1.open('opfs-schema', 1);
|
|
||||||
await e1.createTable(createSchema('users', {
|
|
||||||
id: { type: 'string', primaryKey: true },
|
|
||||||
email: { type: 'string', index: true },
|
|
||||||
}));
|
|
||||||
await e1.close();
|
|
||||||
|
|
||||||
// 重启:空表也应恢复(修复前空表消失)+ 索引标记恢复
|
|
||||||
const e2 = new OPFSEngine();
|
|
||||||
await e2.open('opfs-schema', 1);
|
|
||||||
expect(await e2.hasTable('users')).toBe(true);
|
|
||||||
const schema = await e2.getTableSchema('users');
|
|
||||||
expect(schema!.columns.email.index).toBe(true);
|
|
||||||
await e2.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('有数据重启后索引查询可用', async () => {
|
|
||||||
mockOPFS();
|
|
||||||
const e1 = new OPFSEngine();
|
|
||||||
await e1.open('opfs-idx', 1);
|
|
||||||
await e1.createTable(createSchema('users', {
|
|
||||||
id: { type: 'string', primaryKey: true },
|
|
||||||
email: { type: 'string', index: true },
|
|
||||||
}));
|
|
||||||
await e1.insert('users', [{ id: '1', email: 'a@x.com' }]);
|
|
||||||
await e1.close();
|
|
||||||
|
|
||||||
const e2 = new OPFSEngine();
|
|
||||||
await e2.open('opfs-idx', 1);
|
|
||||||
const byEmail = await e2.find('users', { table: 'users', where: { email: 'a@x.com' } });
|
|
||||||
expect(byEmail).toHaveLength(1);
|
|
||||||
await e2.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('dropTable 后重启无幽灵表', async () => {
|
|
||||||
mockOPFS();
|
|
||||||
const e1 = new OPFSEngine();
|
|
||||||
await e1.open('opfs-drop', 1);
|
|
||||||
await e1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
|
||||||
await e1.insert('t', [{ id: '1' }]);
|
|
||||||
await e1.dropTable('t');
|
|
||||||
await e1.close();
|
|
||||||
|
|
||||||
const e2 = new OPFSEngine();
|
|
||||||
await e2.open('opfs-drop', 1);
|
|
||||||
expect(await e2.hasTable('t')).toBe(false);
|
|
||||||
expect(await e2.getTableNames()).toEqual([]);
|
|
||||||
await e2.close();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
// P2: Aria 内部状态
|
// P2: Aria 内部状态
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
|
|||||||
@@ -10,8 +10,10 @@
|
|||||||
import { AriaEngine } from '../src/engine/aria/index';
|
import { AriaEngine } from '../src/engine/aria/index';
|
||||||
import { createSchema } from '../src/table/schema';
|
import { createSchema } from '../src/table/schema';
|
||||||
import { LSM } from '../src/engine/aria/index/lsm';
|
import { LSM } from '../src/engine/aria/index/lsm';
|
||||||
import { OPFSEngine } from '../src/engine/opfs';
|
|
||||||
import 'fake-indexeddb/auto';
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -56,7 +58,7 @@ describe('P0 — close 后无残留后台任务', () => {
|
|||||||
it('close 后立即 reopen 不被旧后台任务污染', async () => {
|
it('close 后立即 reopen 不被旧后台任务污染', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
bufferPoolPages: 4,
|
bufferPoolPages: 4,
|
||||||
memtableSizeThreshold: 32 * 1024,
|
memtableSizeThreshold: 32 * 1024,
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
@@ -162,47 +164,4 @@ describe('P1 — 提交顺序与 close 等待', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('OPFS close 等待挂起写完成(文件完整)', async () => {
|
|
||||||
// mock OPFS(跨实例共享)
|
|
||||||
const files = new Map<string, string>();
|
|
||||||
const dirMock = {
|
|
||||||
getDirectoryHandle: async (_n: string, _o?: any) => dirMock as any,
|
|
||||||
getFileHandle: async (name: string, opts?: any) => {
|
|
||||||
if (opts?.create) {
|
|
||||||
return {
|
|
||||||
createWritable: async () => ({
|
|
||||||
write: async (d: string) => {
|
|
||||||
// 模拟慢 I/O
|
|
||||||
await new Promise((r) => setTimeout(r, 30));
|
|
||||||
files.set(name, d);
|
|
||||||
},
|
|
||||||
close: async () => {},
|
|
||||||
}),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (!files.has(name)) throw new Error('Not found');
|
|
||||||
return { getFile: async () => ({ text: async () => files.get(name)!, arrayBuffer: async () => new ArrayBuffer(0) }) };
|
|
||||||
},
|
|
||||||
removeEntry: async (n: string) => { files.delete(n); },
|
|
||||||
};
|
|
||||||
(dirMock as any).entries = () => ({
|
|
||||||
[Symbol.asyncIterator]: async function* () {
|
|
||||||
for (const [k] of files) yield [k];
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
const nav = (globalThis as any).navigator || {};
|
|
||||||
nav.storage = { getDirectory: async () => dirMock };
|
|
||||||
(globalThis as any).navigator = nav;
|
|
||||||
|
|
||||||
const engine = new OPFSEngine();
|
|
||||||
await engine.open('opfs-close', 1);
|
|
||||||
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
|
|
||||||
// 写入不等待(写 I/O 30ms 挂起)
|
|
||||||
const p = engine.insert('t', [{ id: '1' }]);
|
|
||||||
// 立即 close:必须等待挂起写完成
|
|
||||||
await engine.close();
|
|
||||||
await p;
|
|
||||||
// 文件完整(修复前 close 不等写 → 可能读到旧/空文件)
|
|
||||||
expect(files.get('t.json')).toContain('"1"');
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|||||||
@@ -13,7 +13,10 @@ import { createSchema } from '../src/table/schema';
|
|||||||
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
|
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
|
||||||
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
||||||
import type { SSTableMeta } from '../src/engine/aria/types';
|
import type { SSTableMeta } from '../src/engine/aria/types';
|
||||||
import 'fake-indexeddb/auto';
|
|
||||||
|
import { installOPFSMock } from './helpers/opfs-mock';
|
||||||
|
|
||||||
|
beforeEach(() => { installOPFSMock(new Map()); });
|
||||||
|
|
||||||
let idbCounter = 0;
|
let idbCounter = 0;
|
||||||
function uniqueDB(): string {
|
function uniqueDB(): string {
|
||||||
@@ -164,7 +167,7 @@ describe('P1 — v1 旧格式兼容', () => {
|
|||||||
describe('P0 — 大内容端到端(aria 模式)', () => {
|
describe('P0 — 大内容端到端(aria 模式)', () => {
|
||||||
it('300KB 中文写入 → close → reopen 数据完整(修复前打开必崩)', async () => {
|
it('300KB 中文写入 → close → reopen 数据完整(修复前打开必崩)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine.open(dbName, 1);
|
await engine.open(dbName, 1);
|
||||||
await engine.createTable(createSchema('notes', {
|
await engine.createTable(createSchema('notes', {
|
||||||
id: { type: 'string', primaryKey: true },
|
id: { type: 'string', primaryKey: true },
|
||||||
@@ -182,7 +185,7 @@ describe('P0 — 大内容端到端(aria 模式)', () => {
|
|||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
// 修复前:打开解析半写文件 → ARIA_OPEN_ERROR
|
// 修复前:打开解析半写文件 → ARIA_OPEN_ERROR
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
|
||||||
const rows = await engine2.find('notes', { table: 'notes' });
|
const rows = await engine2.find('notes', { table: 'notes' });
|
||||||
expect(rows).toHaveLength(2);
|
expect(rows).toHaveLength(2);
|
||||||
@@ -196,7 +199,7 @@ describe('P0 — 大内容端到端(aria 模式)', () => {
|
|||||||
it('多行 >64KB 内容批量写入重开完整(含索引列)', async () => {
|
it('多行 >64KB 内容批量写入重开完整(含索引列)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const engine = new AriaEngine({
|
const engine = new AriaEngine({
|
||||||
storageBackend: 'indexeddb',
|
storageBackend: 'opfs',
|
||||||
memtableSizeThreshold: 32 * 1024,
|
memtableSizeThreshold: 32 * 1024,
|
||||||
checkpointInterval: 100000,
|
checkpointInterval: 100000,
|
||||||
});
|
});
|
||||||
@@ -214,7 +217,7 @@ describe('P0 — 大内容端到端(aria 模式)', () => {
|
|||||||
}
|
}
|
||||||
await engine.close();
|
await engine.close();
|
||||||
|
|
||||||
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 });
|
const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
|
||||||
await engine2.open(dbName, 1);
|
await engine2.open(dbName, 1);
|
||||||
expect(await engine2.count('docs')).toBe(10);
|
expect(await engine2.count('docs')).toBe(10);
|
||||||
const evens = await engine2.find('docs', { table: 'docs', where: { tag: 'even' } });
|
const evens = await engine2.find('docs', { table: 'docs', where: { tag: 'even' } });
|
||||||
|
|||||||
Reference in New Issue
Block a user