release: v0.6.0 — 完全移除 IndexedDB,自研 KVStore 事务存储引擎(多key原子写/快照日志恢复/CRC自愈)+ KVStoreEngine + 旧库迁移工具 + 10万级压力验证 + 崩溃注入e2e
CI / test (20.x) (push) Successful in 10m53s
CI / test (22.x) (push) Successful in 10m47s
CI / test (24.x) (push) Successful in 10m42s
CI / e2e (push) Successful in 10m27s
CI / test (18.x) (push) Successful in 11m1s

This commit is contained in:
thzxx
2026-08-10 13:56:04 +08:00
parent 2668bd47cf
commit 24b3ca1079
74 changed files with 8016 additions and 7688 deletions
+53
View File
@@ -2,6 +2,59 @@
All notable changes to MetonaSqlark will be documented in this file. All notable changes to MetonaSqlark will be documented in this file.
## [0.6.0] - 2026-08-10
### 里程碑:完全移除 IndexedDB,自研 KV 事务存储引擎
> 达成"彻底删除 IndexedDB + 自研类似 IndexedDB 的存储后端"目标:
> KVStore 引擎在 OPFS 之上实现多 key 原子事务(IndexedDB 的核心能力),
> disk 模式全面切换,旧库一键迁移。
### Added
- **KVStore 自研 KV 事务引擎**`src/engine/kvstore/`):
- 多 key 原子写:putMany/deleteMany = 单条日志记录(单文件 COW 原子追加)→ 崩溃全有或全无
- 持久化与崩溃恢复:快照(checkpoint)+ 追加日志,两阶段恢复(快照水位跳过)
- 自愈:快照损坏回退全量日志重放;日志损坏截断至损坏处;`repair()` 清理
- 容错时序:checkpoint = 写快照 → 写 meta → 清空日志(meta 先于截断,任何崩溃窗口不丢数据)
- 写操作与 checkpoint 串行队列(无交错窗口);标准 CRC-32 全程校验
- 介质层:OPFS(浏览器)/ SharedMemoryBackendNode/测试,跨实例共享模拟持久化)
- **KVStoreEngine**disk 模式,替代 IndexedDBEngine + OPFSEngine):
- 内存热路径 + KVStore 原子持久化;读 O(1),写增量/整表 diff 分级
- 事务:内存快照 + commit 原子 flush(受影响表),事务内 DDL 支持(schema 一并持久化)
- 外键级联(CASCADE/SET NULL/RESTRICT)持久化、主键变更、二级索引跨重启恢复
- `reload()` 支持多标签页同步重载(Hybrid 场景)
- **迁移工具** `migrateFromIndexedDB()`:一次性把旧 IndexedDB 库(schema/索引/行)导入新引擎;
旧库不存在/aria 私有格式(SSTable/WAL)明确报错;目标库已有表跳过不覆盖
- **可靠性强化**
- 10 万 key 写入+checkpoint+重开全量验证(928ms
- 5 万混合操作+崩溃模拟零丢失
- e2e 新增写入中途崩溃 / checkpoint 前后崩溃注入(9 用例)
- **`MetonaSqlark` 配置**`diskEngine` 类型收敛为 `'opfs' | 'memory'`(默认 'opfs'
### Removed
- `src/engine/indexeddb.ts`840 行 IndexedDBEngine)—— 版本冲突/blocked 重试/onversionchange/flushToIDB 全部消失
- `src/engine/opfs.ts`375 行 OPFSEngine)—— 能力被 KVStoreEngine 覆盖
- `IndexedDBBackend`aria 后端)—— aria storageBackend 收敛为 'opfs' | 'memory'
- 26 个测试文件的 fake-indexeddb 依赖改造为 KVStore SharedMemory / OPFS mock
### Fixed
- **Hybrid 多标签页重载失效**KVStoreEngine 读内存 vs 磁盘引擎读穿透)—— 新增 `reload()` 从介质重载
- **事务内 DDL schema 不持久化**—— commit 时统一 persistSchema(重启后表结构完整)
- **repair 缓存掩盖损坏**—— 先清 BufferPool 再校验(缓存中"完好页面"不再掩盖磁盘损坏)
- **dropInvalidSSTable 孤儿页面**—— 先删数据文件再删 meta(页面化删除依赖 pageIds 定位)
### Changed
- 测试 1009 → **1021**64 套件;+58 新增 kvstore/kvstore-engine/压力/迁移测试,-46 删除/改造 IDB 测试)
- 行覆盖率 87.25% → **89.13%**
- 浏览器支持收敛到 OPFS 矩阵(Chrome/Edge 102+、Firefox 111+、Safari 15.2+);Node 用内存介质
- fake-indexeddb 保留为 devDependency(仅迁移工具测试用)
---
## [0.5.1] - 2026-08-10 ## [0.5.1] - 2026-08-10
### 深度审查:消除假实现/半成品/死代码 ### 深度审查:消除假实现/半成品/死代码
+61 -33
View File
@@ -1,10 +1,10 @@
# MetonaSqlark # MetonaSqlark
<p align="center"> <p align="center">
<img src="https://img.shields.io/badge/version-0.5.1-blue?style=flat-square" alt="version"> <img src="https://img.shields.io/badge/version-0.6.0-blue?style=flat-square" alt="version">
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license"> <img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
<img src="https://img.shields.io/badge/coverage-87.3%25-brightgreen?style=flat-square" alt="coverage"> <img src="https://img.shields.io/badge/coverage-89.1%25-brightgreen?style=flat-square" alt="coverage">
<img src="https://img.shields.io/badge/tests-1009%20passed-success?style=flat-square" alt="tests"> <img src="https://img.shields.io/badge/tests-1021%20passed-success?style=flat-square" alt="tests">
</p> </p>
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。 > 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。
@@ -19,7 +19,9 @@
- 🔐 **多标签页独占锁** — Web Locks API,第二个标签页打开同一库抛 `ARIA_LOCKED`v0.5.0 - 🔐 **多标签页独占锁** — Web Locks API,第二个标签页打开同一库抛 `ARIA_LOCKED`v0.5.0
- 🛠 **维护语句 SQL 入口**`EXPLAIN`/`ANALYZE`/`REINDEX`/`VACUUM`/`SAVEPOINT` 原生 SQL 支持(v0.5.1 补齐) - 🛠 **维护语句 SQL 入口**`EXPLAIN`/`ANALYZE`/`REINDEX`/`VACUUM`/`SAVEPOINT` 原生 SQL 支持(v0.5.1 补齐)
- 🛡 **输入校验全覆盖**`maxLength`/`min`/`max` 约束、类型检查、必填验证 - 🛡 **输入校验全覆盖**`maxLength`/`min`/`max` 约束、类型检查、必填验证
- 💾 **多引擎架构** — Memory / IndexedDB / OPFS / Hybrid(write-through) / Aria 五种模式 - 💾 **多引擎架构** — Memory / **KVStore**(自研 KV 引擎)/ OPFS / Hybrid(write-through) / Aria 五种模式v0.6.0: IndexedDB 完全移除)
- 💾 **KVStore 自研 KV 引擎** — 日志结构化事务存储:多 key 原子写(putMany/deleteMany 单记录原子追加)、快照 checkpoint、崩溃两阶段恢复、CRC-32 自愈;替代 IndexedDBv0.6.0
- 🔄 **旧库一键迁移**`migrateFromIndexedDB()` 把旧 IndexedDB 数据(schema/索引/行)导入新引擎(v0.6.0)
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTSv0.3.0+ CASE WHEN/哈希连接/组提交(v0.3.1+ 多标签页同步(v0.3.2 - 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTSv0.3.0+ CASE WHEN/哈希连接/组提交(v0.3.1+ 多标签页同步(v0.3.2
- 🚰 **流式查询**`queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.0) - 🚰 **流式查询**`queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.0)
- 🧩 **派生表**`FROM (SELECT ...)` 子查询作为行源,多列 ON 哈希连接,COUNT(DISTINCT)NULLS FIRST/LASTv0.4.0 - 🧩 **派生表**`FROM (SELECT ...)` 子查询作为行源,多列 ON 哈希连接,COUNT(DISTINCT)NULLS FIRST/LASTv0.4.0
@@ -32,7 +34,7 @@
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证 - 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
-**性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态 -**性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+ - 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
- 🧪 **1009 测试 · 87.3% 覆盖率** — 62 套件 + 7 个 Playwright 真实 Chromium e2e,生产级质量保证 - 🧪 **1021 测试 · 89.1% 覆盖率** — 64 套件 + 9 个 Playwright 真实 Chromium e2e(含崩溃注入),生产级质量保证
--- ---
@@ -276,6 +278,30 @@ await db2.disconnect(); // 引用计数 -1
> 不支持的引擎执行维护语句抛 `NOT_SUPPORTED`。 > 不支持的引擎执行维护语句抛 `NOT_SUPPORTED`。
### 旧库迁移(v0.6.0
> IndexedDB 已从引擎中完全移除。使用旧版本(v0.5.x 及更早)的用户,可通过
> 一次性迁移工具把磁盘模式(IndexedDBEngine)旧库导入新引擎(KVStore)。
```typescript
import { migrateFromIndexedDB } from '@metona-team/metona-sqlark/migration';
// 目标库(新引擎,disk 模式)
const target = await MetonaSqlark.create({ name: 'my-app-new', mode: 'disk' });
// 从旧 IndexedDB 库导入(旧库名 'my-app',旧引擎 disk 模式)
const result = await migrateFromIndexedDB({
dbName: 'my-app',
engine: 'disk', // 仅支持 disk 模式(IndexedDBEngine
target,
onProgress: (done, total, table) => console.log(`迁移 ${done}/${total}: ${table}`),
});
// result: { migratedTables, rowCount, skippedTables }
```
> 注:aria 模式旧库(IndexedDBBackend)数据为引擎私有格式(SSTable/WAL),
> 无法按行迁移——此类用户请从应用层导出(exportAll)后重新导入。
### 连接池(v0.1.13 ### 连接池(v0.1.13
| 静态方法 | 说明 | | 静态方法 | 说明 |
@@ -301,18 +327,19 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
## 📊 存储模式对比 ## 📊 存储模式对比
| 特性 | Memory | Disk (IndexedDB) | Disk (OPFS) | Hybrid | Aria | | 特性 | Memory | Disk (KVStore) | Disk (OPFS) | Hybrid | Aria |
|------|--------|------------------|-------------|--------|------| |------|--------|----------------|-------------|--------|------|
| **持久化** | ❌ 重启丢失 | ✅ IndexedDB | ✅ OPFSschema 持久化) | ✅ 内存+磁盘 | ✅ 后端决定 | | **持久化** | ❌ 重启丢失 | ✅ KVStoreOPFS | ✅ OPFSschema 持久化) | ✅ 内存+磁盘 | ✅ 后端决定 |
| **事务回滚** | ✅ 快照 | ✅ 原子flush | ✅ 快照 | ✅ 双引擎 | ✅ MVCC | | **事务回滚** | ✅ 快照 | ✅ 原子日志 flush | ✅ 快照 | ✅ 双引擎 | ✅ MVCC |
| **二级索引** | ✅ Hash | ✅ Hash | ✅ Hash(重启恢复) | ✅ Hash | ✅ LSM(重启恢复) | | **多 key 原子写** | — | ✅ 单日志记录原子(v0.6.0) | — | ✅ 委托磁盘 | ✅ WAL 单文件原子 |
| **查询性能** | ⚡ O(1) PK | 🟡 O(1) PK | 🟡 O(1) PK | ⚡ O(1) PK | ⚡ O(log n) | | **二级索引** | ✅ Hash | ✅ Hash(重启恢复) | ✅ Hash(重启恢复) | ✅ Hash | ✅ LSM(重启恢复) |
| **数据上限** | 内存限制 | ~2GB(IDB限制) | ~磁盘可用(页面化后大表可行) | ~2GB(IDB) | 内存限制 | | **查询性能** | ⚡ O(1) PK | ⚡ O(1) PK(内存热路径) | 🟡 O(1) PK | ⚡ O(1) PK | ⚡ O(log n) |
| **浏览器** | 全部 | 全部 | Chrome/Edge 102+ / Firefox 111+ / Safari 15.2+ | 全部 | 全部 | | **数据上限** | 内存限制 | 磁盘可用(行级存储,大表可行) | ~磁盘可用(整表 JSON,≤1000 行) | 磁盘可用 | 内存限制 |
| **多标签页** | | ✅ versionchange | ❌ 无保护 | ❌ 无保护 | ✅ Web Locks 独占锁(v0.5.0 | | **浏览器** | 全部 | Chrome/Edge 102+ / Firefox 111+ / Safari 15.2+ | 同上 | 同上 | 同上 |
| **多标签页** | — | —(无事务锁,Hybrid 场景) | ❌ 无保护 | ❌ 无保护 | ✅ Web Locks 独占锁(v0.5.0 |
| **全库加密** | — | — | — | — | ✅ AES-GCMv0.5.0 | | **全库加密** | — | — | — | — | ✅ AES-GCMv0.5.0 |
| **适用场景** | 缓存/测试 | 标准持久化 | Chromium+ 持久化 | 速度+持久化 | 大规模/分析 | | **适用场景** | 缓存/测试 | 标准持久化(替代 IndexedDB | 小数据集 | 速度+持久化 | 大规模/分析 |
| **测试覆盖** | 30+ | 30+ | 15+e2e | 15+ | 400+ | | **测试覆盖** | 30+ | 50+(含 10 万级压力) | 15 | 15+ | 400+ |
### Memory 模式 ### Memory 模式
- **环境**: 所有浏览器、Node.js - **环境**: 所有浏览器、Node.js
@@ -320,11 +347,12 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
- **能力**: 完整 CRUD、事务回滚、外键级联、二级索引、SQL 全支持 - **能力**: 完整 CRUD、事务回滚、外键级联、二级索引、SQL 全支持
- **适用**: 临时数据、单元测试、缓存层 - **适用**: 临时数据、单元测试、缓存层
### Disk (IndexedDB) 模式 ### Disk (KVStore) 模式
- **环境**: 所有现代浏览器(Chrome/Firefox/Safari/Edge)、Node.jsfake-indexeddb - **环境**: Chrome/Edge 102+ / Firefox 111+ / Safari 15.2+OPFS)、Node.js(内存介质
- **限制**: 受浏览器 IndexedDB 配额限制(通常 ~2GB),多标签页需处理版本冲突 - **限制**: 依赖 OPFS;多标签页并发写无锁保护(单标签页内可靠)
- **能力**: 完整 CRUD、事务原子性(单 IDB 事务包裹)、外键级联、onversionchange 感知 - **能力**: 完整 CRUD、**多 key 原子事务**(单日志记录原子追加)、外键级联、事务内 DDL、
- **适用**: 标准前端数据库持久化场景 二级索引(重启恢复)、schema/数据/索引完整持久化、10 万级数据量压力验证
- **适用**: 标准前端数据库持久化(v0.6.0 替代 IndexedDB
### Disk (OPFS) 模式 ### Disk (OPFS) 模式
- **环境**: Chrome 102+ / Edge 102+ / Firefox 111+ / Safari 15.2+Origin Private File System - **环境**: Chrome 102+ / Edge 102+ / Firefox 111+ / Safari 15.2+Origin Private File System
@@ -418,7 +446,7 @@ npm install # 安装依赖
npm run dev # 开发模式(localhost:3001 npm run dev # 开发模式(localhost:3001
npm run build # 生产构建(生成 dist/ npm run build # 生产构建(生成 dist/
npm test # 运行测试 npm test # 运行测试
npm run test:e2e # Playwright e2e(真实 Chromium + OPFS,需先 build npm run test:e2e # Playwright e2e(真实 Chromium + OPFS + 崩溃注入,需先 build
npm run lint # 代码检查 npm run lint # 代码检查
npm run typecheck # 类型检查 npm run typecheck # 类型检查
``` ```
@@ -429,21 +457,21 @@ npm run typecheck # 类型检查
| 指标 | 数值 | | 指标 | 数值 |
|------|------| |------|------|
| 测试用例 | 1009 | | 测试用例 | 1021 |
| 测试套件 | 62+7 Playwright e2e | | 测试套件 | 64+9 Playwright e2e |
| 行覆盖率 | 87.3% | | 行覆盖率 | 89.1% |
| SQL 关键字 | 72 | | SQL 关键字 | 72 |
| 存储引擎 | 5Memory / IndexedDB / OPFS / Hybrid / **Aria** | | 存储引擎 | 5Memory / **KVStore** / OPFS / Hybrid / **Aria** |
### 🌐 浏览器兼容性 ### 🌐 浏览器兼容性
| 浏览器 | 最低版本 | Memory | IndexedDB | OPFS | Web Locks | Aria | | 浏览器 | 最低版本 | Memory | KVStore(OPFS) | OPFS | Web Locks | Aria |
|--------|----------|--------|-----------|------|-----------|------| |--------|----------|--------|---------------|------|-----------|------|
| Chrome | 80+ | ✅ | ✅ | ✅ (102+) | ✅ (69+) | ✅ | | Chrome | 102+ | ✅ | ✅ | ✅ | ✅ (69+) | ✅ |
| Firefox | 80+ | ✅ | ✅ | ✅ (111+) | ✅ (96+) | ✅ | | Firefox | 111+ | ✅ | ✅ | ✅ | ✅ (96+) | ✅ |
| Safari | 14+ | ✅ | ✅ | ✅ (15.2+) | ✅ (15.4+) | ✅ | | Safari | 15.2+ | ✅ | ✅ | ✅ | ✅ (15.4+) | ✅ |
| Edge | 80+ | ✅ | ✅ | ✅ (102+) | ✅ (79+) | ✅ | | Edge | 102+ | ✅ | ✅ | ✅ | ✅ (79+) | ✅ |
| Node.js | 16+ | ✅ | ✅ (fake-idb) | ❌ (测试用 mock) | ❌ | ✅ | | Node.js | 16+ | ✅ | ✅ (内存介质) | ❌ (测试用 mock) | ❌ | ✅ |
> **OPFS 支持说明(v0.4.5 更新)**: ChromiumChrome/Edge 102+)、Firefox 111+、Safari 15.2+ > **OPFS 支持说明(v0.4.5 更新)**: ChromiumChrome/Edge 102+)、Firefox 111+、Safari 15.2+
> 均已支持基础 OPFS API`createWritable` 原子写)。OPFS 无跨文件事务, > 均已支持基础 OPFS API`createWritable` 原子写)。OPFS 无跨文件事务,
+1293 -1402
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
File diff suppressed because one or more lines are too long
+76 -158
View File
@@ -18,7 +18,7 @@ interface AriaEngineConfig {
/** 是否启用页面压缩(默认 false) */ /** 是否启用页面压缩(默认 false) */
compression?: boolean; compression?: boolean;
/** 存储后端 */ /** 存储后端 */
storageBackend?: 'indexeddb' | 'opfs' | 'memory'; storageBackend?: 'opfs' | 'memory';
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB */ /** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB */
walSizeThreshold?: number; walSizeThreshold?: number;
/** 最大内存预算(MB,默认 64) */ /** 最大内存预算(MB,默认 64) */
@@ -45,8 +45,8 @@ interface AriaEngineConfig {
*/ */
/** 存储模式 */ /** 存储模式 */
type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria'; type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
/** 磁盘引擎类型 */ /** 磁盘引擎类型v0.6.0: IndexedDB 已移除,'memory' 供 aria 内存后端) */
type DiskEngine = 'indexeddb' | 'opfs'; type DiskEngine = 'opfs' | 'memory';
/** 字段数据类型 */ /** 字段数据类型 */
type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json'; type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json';
/** 列定义 */ /** 列定义 */
@@ -164,7 +164,7 @@ interface MetonaPlugin {
/** 销毁 */ /** 销毁 */
destroy(): void; destroy(): void;
} }
declare const VERSION = "0.5.1"; declare const VERSION = "0.6.0";
/** /**
* metona-sqlark Plugin — 插件系统 * metona-sqlark Plugin — 插件系统
@@ -832,118 +832,75 @@ declare class MemoryEngine implements IStorageEngine {
private cascadeDelete; private cascadeDelete;
} }
declare class IndexedDBEngine implements IStorageEngine { /**
readonly name = "indexeddb"; * AriaEngine Storage Backend — 存储后端抽象层
private db; * @module engine/aria/store/backend
private dbName; *
private version; * 封装底层浏览器存储 APIIndexedDB / OPFS / Memory 回退),
private memoryCache; * 供 Buffer Pool 的 PageIO 和 WAL 的 WALStore 使用。
private txActive;
open(dbName: string, version: number): Promise<void>;
/** v0.4.2-fix: 多标签页冲突处理 — 其他标签页升级版本时自动关闭当前连接 */
private setupVersionChangeHandler;
/**
* v0.4.2-fix: 打开 IndexedDB 连接。
* - P0-2: 请求版本低于库实际版本(VersionError)时,先无版本参数探测库当前版本,
* 再以实际版本重开(建表每张表版本号 +1,config.version 会过期)
* - P2-8: onblocked 为瞬时状态(另一连接短暂持有),等待后重试多次,超时才抛 IDB_BLOCKED
*/ */
private openDatabaseWithRetry; interface IStorageBackend {
/** /** 打开存储 */
* 发起一次 indexedDB.open 请求(success/error/blocked 三态收敛)。 open(name: string): Promise<void>;
* onblocked 不立即失败:阻塞解除后 success 仍会触发,仅超时兜底判失败, /** 关闭存储 */
* 避免"拒绝后连接迟到成功"泄漏未关闭的数据库连接。
*/
private openRequest;
/** 无版本参数打开库,解析其当前实际版本号(随后立即关闭) */
private resolveCurrentVersion;
/**
* v0.4.2-fix (P2-7): 确保 __metona_schema store 存在。
* 新库(或版本升级前创建的旧库)没有该 store 时,通过一次版本升级创建,
* 使 getMeta/setMeta(迁移版本持久化)始终可用。
*/
private ensureSchemaStore;
/**
* 从 IDB 恢复内存 schema
* 1. 优先读取持久化的 schema 记录('__metona_schema' storev0.3.2
* 2. 旧数据回退:从 objectStore 主键 / 索引 / 样例数据推断
*/
private rebuildSchemaFromIDB;
close(): Promise<void>; close(): Promise<void>;
/** /** 是否已打开 */
* v0.4.2-fix: 自愈 — 从磁盘重建内存 schema 与数据(schema 丢失/内存不一致时调用)。
* 无删库需求即可恢复可用的库。
*/
repair(): Promise<void>;
/**
* v0.4.2-fix: 清空全部数据与表结构(含持久化 schema 记录),保留库本身。
* 单个版本升级事务内原子完成。
*/
clearAll(): Promise<void>;
getMeta(key: string): Promise<string | null>;
setMeta(key: string, value: string): Promise<void>;
isOpen(): boolean; isOpen(): boolean;
createTable(schema: TableSchema): Promise<void>; /** 读取数据块 */
dropTable(tableName: string): Promise<void>; read(key: string): Promise<ArrayBuffer | null>;
hasTable(tableName: string): Promise<boolean>; /** 写入数据块 */
getTableNames(): Promise<string[]>; write(key: string, data: ArrayBuffer): Promise<void>;
getTableSchema(tableName: string): Promise<TableSchema | null>;
/** /**
* v0.4.2-fix: 引擎级 ALTER TABLE — schema 持久化到 __metona_schema store * 追加写入(v0.4.5 WAL 分片用,可选):
* 重启后 ALTER 不丢失(此前通用路径只改内存引用,重启回退;DROP 的行数据也没真正删)。 * - OPFS 后端实现真追加(createWritable keepExistingData + seekO(chunk)
* - 未实现的后端由调用方回退 read+writeEncryptedBackend 包装时整体重写保正确性)
* 语义:在 key 现有内容末尾追加 data;key 不存在时等同 write。
*/ */
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & { append?(key: string, data: ArrayBuffer): Promise<void>;
name: string; /**
}): Promise<void>; * 批量原子写入(v0.4.2-fix):多个 key 在单个底层事务中提交,
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>; * 中断时整体回滚,不留半写状态。WAL count 与记录同事务保证一致性。
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>; */
/** v0.4.0: 流式查询 — IDB 批量读入后逐行回调(保持接口一致性) */ writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>; /** 删除数据块 */
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>; delete(key: string): Promise<void>;
delete(tableName: string, query: QueryPlan): Promise<number>; /**
count(tableName: string, query?: QueryPlan): Promise<number>; * 批量原子删除(v0.4.2-fix):多个 key 在单个底层事务中提交。
clear(tableName: string): Promise<void>; */
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>; deleteMany(keys: string[]): Promise<void>;
dropIndex(tableName: string, column: string, _indexName?: string): Promise<void>; /** 列出所有 key */
beginTransaction(): Promise<void>; listKeys(): Promise<string[]>;
commitTransaction(): Promise<void>; /** 检查 key 是否存在 */
rollbackTransaction(): Promise<void>; exists(key: string): Promise<boolean>;
private idbCreateTable; /** 清空所有数据 */
private idbDropTable; clear(): Promise<void>;
private idbInsert;
private idbFind;
/** 尝试使用 IDB 索引进行等值查询,成功返回结果,不适用返回 null */
private tryIDBIndexLookup;
private idbUpdate;
private idbDelete;
private idbClear;
/** 持久化单个表 schema 到 __metona_schema storev0.4.2-fix: ALTER TABLE 用) */
private persistSchema;
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
private flushToIDB;
private ensureDB;
} }
declare class OPFSEngine implements IStorageEngine { declare class KVStoreEngine implements IStorageEngine {
readonly name = "opfs"; readonly name = "kv";
private root; private kv;
private tablesDir; private memory;
private dbName; private dbName;
private memoryCache; private version;
/** private opened;
* v0.4.3-fix: 写操作串行队列 — 内存写 + 快照 + 文件持久化整体排队执行, /** 活跃事务标记 */
* close() 等待队列排空后再释放目录句柄(避免 close 后挂起写泄漏/读旧数据)。 private txActive;
* 前一个操作失败不阻塞后续(错误仍返回给调用方)。 /** 事务中写过的表(commit 时只 flush 这些表) */
*/ private txDirtyTables;
private opQueue; constructor(medium?: IStorageBackend, checkpointThreshold?: number);
/** 将写操作加入串行队列(快照在队列内取,始终最新) */ private rowKey;
private enqueueOp; private rowPrefix;
open(dbName: string, version: number): Promise<void>; open(dbName: string, version: number): Promise<void>;
close(): Promise<void>; close(): Promise<void>;
isOpen(): boolean; isOpen(): boolean;
/** 自愈:重置内存缓存后从 OPFS 重新加载(单文件损坏不影响其他表) */ /**
* v0.6.0: 从 KVStore 重新加载全部数据到内存(多标签页同步重载用)。
* Hybrid 引擎的 reloadMemoryFromDisk 依赖磁盘引擎"读穿透"
* KVStoreEngine 读内存 → 提供 reload 重新加载磁盘最新数据。
*/
reload(): Promise<void>;
/** v0.4.2-fix: 自愈 — 校验 KVStore 日志/快照完整性并重建内存 */
repair(): Promise<void>; repair(): Promise<void>;
/** 清空全部数据与表结构(删除目录内全部文件) */
clearAll(): Promise<void>; clearAll(): Promise<void>;
getMeta(key: string): Promise<string | null>; getMeta(key: string): Promise<string | null>;
setMeta(key: string, value: string): Promise<void>; setMeta(key: string, value: string): Promise<void>;
@@ -952,13 +909,11 @@ declare class OPFSEngine implements IStorageEngine {
hasTable(tableName: string): Promise<boolean>; hasTable(tableName: string): Promise<boolean>;
getTableNames(): Promise<string[]>; getTableNames(): Promise<string[]>;
getTableSchema(tableName: string): Promise<TableSchema | null>; getTableSchema(tableName: string): Promise<TableSchema | null>;
/** v0.4.2-fix: 引擎级 ALTER TABLE — 内存 + schema 持久化 + 整表文件重写 */
alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & { alterTable(tableName: string, action: 'ADD' | 'DROP', column: ColumnDef & {
name: string; name: string;
}): Promise<void>; }): Promise<void>;
insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>; insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]>;
find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>; find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]>;
/** v0.4.0: 流式查询(委托内存缓存) */
findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>; findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number>;
update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>; update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number>;
delete(tableName: string, query: QueryPlan): Promise<number>; delete(tableName: string, query: QueryPlan): Promise<number>;
@@ -969,15 +924,22 @@ declare class OPFSEngine implements IStorageEngine {
beginTransaction(): Promise<void>; beginTransaction(): Promise<void>;
commitTransaction(): Promise<void>; commitTransaction(): Promise<void>;
rollbackTransaction(): Promise<void>; rollbackTransaction(): Promise<void>;
private ensureDir; private ensureOpen;
private writeTableData; private getPK;
private readTableData; /** 收集匹配查询的内存行主键(持久化差异计算用) */
private collectMatchingPks;
/** /**
* 从 OPFS 加载已有表到内存缓存 * 计算外键级联影响的表集合(传递闭包:A 被 B 引用,B 被 C 引用 → {A, B, C}
* v0.4.2-fix: 优先从持久化 schema__metona_schema_*.meta)恢复 — * 级联操作(delete/update 主键)需要把这些表一并重写持久化。
* 空表不再消失、索引标记/主键/约束完整;无 schema 记录的旧库从数据推断(兼容)。
*/ */
private loadExistingTables; private affectedTables;
/** 持久化 schema(全部表) */
private persistSchema;
/**
* 整表 diff 持久化:内存行全部 put + KV 残留行删除(原子 putMany + deleteMany)。
* 用于主键变更 / 级联 / dropTable / clear / alterTable DROP / 事务 commit。
*/
private flushTable;
} }
declare class AriaEngine implements IStorageEngine { declare class AriaEngine implements IStorageEngine {
@@ -1322,50 +1284,6 @@ interface Token {
/** 将 SQL 字符串解析为 Token 列表 */ /** 将 SQL 字符串解析为 Token 列表 */
declare function tokenize(sql: string): Token[]; declare function tokenize(sql: string): Token[];
/**
* AriaEngine Storage Backend — 存储后端抽象层
* @module engine/aria/store/backend
*
* 封装底层浏览器存储 APIIndexedDB / OPFS / Memory 回退),
* 供 Buffer Pool 的 PageIO 和 WAL 的 WALStore 使用。
*/
interface IStorageBackend {
/** 打开存储 */
open(name: string): Promise<void>;
/** 关闭存储 */
close(): Promise<void>;
/** 是否已打开 */
isOpen(): boolean;
/** 读取数据块 */
read(key: string): Promise<ArrayBuffer | null>;
/** 写入数据块 */
write(key: string, data: ArrayBuffer): Promise<void>;
/**
* 追加写入(v0.4.5 WAL 分片用,可选):
* - OPFS 后端实现真追加(createWritable keepExistingData + seekO(chunk)
* - 未实现的后端由调用方回退 read+writeEncryptedBackend 包装时整体重写保正确性)
* 语义:在 key 现有内容末尾追加 data;key 不存在时等同 write。
*/
append?(key: string, data: ArrayBuffer): Promise<void>;
/**
* 批量原子写入(v0.4.2-fix):多个 key 在单个底层事务中提交,
* 中断时整体回滚,不留半写状态。WAL count 与记录同事务保证一致性。
*/
writeMany(entries: Record<string, ArrayBuffer>): Promise<void>;
/** 删除数据块 */
delete(key: string): Promise<void>;
/**
* 批量原子删除(v0.4.2-fix):多个 key 在单个底层事务中提交。
*/
deleteMany(keys: string[]): Promise<void>;
/** 列出所有 key */
listKeys(): Promise<string[]>;
/** 检查 key 是否存在 */
exists(key: string): Promise<boolean>;
/** 清空所有数据 */
clear(): Promise<void>;
}
/** /**
* AriaEngine OPFS Backend — 基于 Origin Private File System 的自研存储后端 * AriaEngine OPFS Backend — 基于 Origin Private File System 的自研存储后端
* @module engine/aria/store/opfs_backend * @module engine/aria/store/opfs_backend
@@ -1463,4 +1381,4 @@ declare global {
declare const MeSqlark: typeof MetonaSqlark; declare const MeSqlark: typeof MetonaSqlark;
export { AriaEngine, AriaEngineConfig, ColumnDef, DatabaseConfig, DeleteStatement, DiskEngine, FieldType, HybridEngine, IStorageEngine, IndexedDBEngine, InsertStatement, MeSqlark, MemoryEngine, MetonaSqlark, OPFSBackend, OPFSEngine, SelectStatement, Statement, StorageMode, Table, TableSchema, UpdateStatement, VERSION, api, create, api as default, parse, parseAll, tokenize }; export { AriaEngine, AriaEngineConfig, ColumnDef, DatabaseConfig, DeleteStatement, DiskEngine, FieldType, HybridEngine, IStorageEngine, InsertStatement, KVStoreEngine, MeSqlark, MemoryEngine, MetonaSqlark, OPFSBackend, SelectStatement, Statement, StorageMode, Table, TableSchema, UpdateStatement, VERSION, api, create, api as default, parse, parseAll, tokenize };
+1293 -1401
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
File diff suppressed because one or more lines are too long
+1293 -1402
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@metona-team/metona-sqlark", "name": "@metona-team/metona-sqlark",
"version": "0.5.1", "version": "0.6.0",
"description": "Frontend SQL database with in-memory and disk dual-mode storage", "description": "Frontend SQL database with in-memory and disk dual-mode storage",
"type": "module", "type": "module",
"main": "dist/metona-sqlark.cjs", "main": "dist/metona-sqlark.cjs",
+3 -3
View File
@@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"> <meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>⚡ 性能基准 — MetonaSqlark v0.5.1</title> <title>⚡ 性能基准 — MetonaSqlark v0.6.0</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>"> <link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
<style> <style>
:root { :root {
@@ -116,7 +116,7 @@ const SIZES_FAST = [1000];
const ENGINES = [ const ENGINES = [
['memory', 'memory', 'memory'], ['memory', 'memory', 'memory'],
['aria-memory', 'aria', 'memory'], ['aria-memory', 'aria', 'memory'],
['aria-indexeddb', 'aria', 'indexeddb'], ['kvstore', 'disk', 'opfs'],
['aria-opfs', 'aria', 'opfs'], ['aria-opfs', 'aria', 'opfs'],
]; ];
@@ -264,7 +264,7 @@ function renderResults(results) {
const GROUP_LABEL = { const GROUP_LABEL = {
'memory': '🚀 Memory 引擎', 'memory': '🚀 Memory 引擎',
'aria-memory': '🌲 Aria 引擎(内存后端)', 'aria-memory': '🌲 Aria 引擎(内存后端)',
'aria-indexeddb': '🌲 Aria 引擎(IndexedDB 后端', 'kvstore': '💾 KVStore 引擎(disk 模式 · OPFS 介质',
'aria-opfs': '🌲 Aria 引擎(OPFS · 4KB 页面化)', 'aria-opfs': '🌲 Aria 引擎(OPFS · 4KB 页面化)',
}; };
let lastMode = ''; let lastMode = '';
+3 -3
View File
@@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"> <meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>🧪 在线演示 — MetonaSqlark v0.5.1</title> <title>🧪 在线演示 — MetonaSqlark v0.6.0</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>"> <link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
<style> <style>
:root { :root {
@@ -84,7 +84,7 @@
<a href="demo.html" class="nav-active">演示</a> <a href="demo.html" class="nav-active">演示</a>
<a href="benchmark.html">基准</a> <a href="benchmark.html">基准</a>
</nav> </nav>
<div class="status"><span class="dot" id="engine-dot"></span> <span id="engine-status">Memory</span> 模式 — v0.5.1</div> <div class="status"><span class="dot" id="engine-dot"></span> <span id="engine-status">Memory</span> 模式 — v0.6.0</div>
<button class="btn btn-preset" onclick="switchEngine('memory')" id="btn-memory" style="margin:6px 4px 6px 0;padding:6px 12px;">⚡ Memory</button> <button class="btn btn-preset" onclick="switchEngine('memory')" id="btn-memory" style="margin:6px 4px 6px 0;padding:6px 12px;">⚡ Memory</button>
<button class="btn btn-preset" onclick="switchEngine('aria')" id="btn-aria" style="margin:6px 0;padding:6px 12px;color:#ec4899;border-color:#ec4899;">🌲 Aria</button> <button class="btn btn-preset" onclick="switchEngine('aria')" id="btn-aria" style="margin:6px 0;padding:6px 12px;color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
</header> </header>
@@ -92,7 +92,7 @@
<div class="main"> <div class="main">
<div class="editor-panel"> <div class="editor-panel">
<div class="editor-area"> <div class="editor-area">
<textarea id="sql-input" placeholder="输入 SQL 语句...&#10;&#10;SELECT * FROM users;&#10;INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28);&#10;SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.5.1 在线演示 <textarea id="sql-input" placeholder="输入 SQL 语句...&#10;&#10;SELECT * FROM users;&#10;INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28);&#10;SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.6.0 在线演示
-- 已预置 users / orders / products 表数据 -- 已预置 users / orders / products 表数据
-- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护 -- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护
+5 -4
View File
@@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"> <meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>📖 API 文档 — MetonaSqlark v0.5.1</title> <title>📖 API 文档 — MetonaSqlark v0.6.0</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>"> <link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
<style> <style>
:root { :root {
@@ -752,11 +752,11 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
<table> <table>
<tr><th>引擎</th><th>模式</th><th>持久化</th><th>索引</th><th>事务</th><th>适用场景</th></tr> <tr><th>引擎</th><th>模式</th><th>持久化</th><th>索引</th><th>事务</th><th>适用场景</th></tr>
<tr><td><code>MemoryEngine</code></td><td>memory</td><td></td><td>哈希</td><td>快照回滚</td><td>临时数据、缓存、测试</td></tr> <tr><td><code>MemoryEngine</code></td><td>memory</td><td></td><td>哈希</td><td>快照回滚</td><td>临时数据、缓存、测试</td></tr>
<tr><td><code>IndexedDBEngine</code></td><td>disk</td><td>IDB</td><td>IDB 索引</td><td>延迟写入</td><td>通用持久化,兼容性最好</td></tr> <tr><td><code>KVStoreEngine</code> 🆕</td><td>disk</td><td>KVStore(OPFS)</td><td>哈希</td><td>原子日志 flush</td><td>标准持久化v0.6.0 替代 IndexedDB</td></tr>
<tr><td><code>OPFSEngine</code></td><td>disk</td><td>✅ OPFS</td><td>哈希</td><td>快照回滚</td><td>现代浏览器,文件级存储</td></tr>
<tr><td><code>HybridEngine</code></td><td>hybrid</td><td>✅ Write-Through</td><td>哈希</td><td>双引擎代理</td><td>生产推荐,读写均走内存</td></tr> <tr><td><code>HybridEngine</code></td><td>hybrid</td><td>✅ Write-Through</td><td>哈希</td><td>双引擎代理</td><td>生产推荐,读写均走内存</td></tr>
<tr style="border-top:2px solid var(--primary);"><td><code style="color:#ec4899;font-weight:700;">AriaEngine</code></td><td>aria</td><td>✅ WAL + SSTable</td><td>LSM-Tree</td><td>MVCC 快照隔离</td><td>自研引擎:大表、高并发、需崩溃恢复</td></tr> <tr style="border-top:2px solid var(--primary);"><td><code style="color:#ec4899;font-weight:700;">AriaEngine</code></td><td>aria</td><td>✅ WAL + SSTable</td><td>LSM-Tree</td><td>MVCC 快照隔离</td><td>自研引擎:大表、高并发、需崩溃恢复</td></tr>
</table> </table>
<p><strong>v0.6.0: IndexedDB 已完全移除</strong> — disk 模式改用自研 KVStore 引擎(多 key 原子写 + 快照/日志崩溃恢复),旧库可经 <code>migrateFromIndexedDB()</code> 一键迁移。</p>
<h2 id="aria-engine">🌲 AriaEngine 自研存储引擎</h2> <h2 id="aria-engine">🌲 AriaEngine 自研存储引擎</h2>
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。<br> <p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。<br>
@@ -766,7 +766,8 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
<strong>v0.4.3 关闭时序与后台任务加固</strong> — 后台 flush/compaction 不再使用 setTimeout 延迟(close 排空全部任务后才关闭存储,杜绝"backend 关闭后写存储/重开污染")· 后台失败在 `flush()`/`close()` 显式报告(`ARIA_BACKGROUND_ERROR`,不静默吞错)· 预加载等待链稳定(修复 compaction 竞态跳块丢数据)· 事务提交先落 WAL 再合并快照(崩溃一致)· OPFS 写操作串行队列 + close 等待。<br> <strong>v0.4.3 关闭时序与后台任务加固</strong> — 后台 flush/compaction 不再使用 setTimeout 延迟(close 排空全部任务后才关闭存储,杜绝"backend 关闭后写存储/重开污染")· 后台失败在 `flush()`/`close()` 显式报告(`ARIA_BACKGROUND_ERROR`,不静默吞错)· 预加载等待链稳定(修复 compaction 竞态跳块丢数据)· 事务提交先落 WAL 再合并快照(崩溃一致)· OPFS 写操作串行队列 + close 等待。<br>
<strong>v0.4.4 SSTable 编码修复</strong> — 大段中文内容(如 300KB 笔记)写入 AriaEngine 不再崩溃:块大小估算改 UTF-8 字节精确计算(修复中文 3 字节 vs 1 码元导致的缓冲区低估越界)· 长度字段 u16 → u32(修复 >64KB value 截断)· 大 value 独立成块 · 格式 v2"SSTC")与 v1"SSTB")双格式兼容(旧库数据不丢)· 中文主键 / 大内容索引列同步支持。<br> <strong>v0.4.4 SSTable 编码修复</strong> — 大段中文内容(如 300KB 笔记)写入 AriaEngine 不再崩溃:块大小估算改 UTF-8 字节精确计算(修复中文 3 字节 vs 1 码元导致的缓冲区低估越界)· 长度字段 u16 → u32(修复 >64KB value 截断)· 大 value 独立成块 · 格式 v2"SSTC")与 v1"SSTB")双格式兼容(旧库数据不丢)· 中文主键 / 大内容索引列同步支持。<br>
<strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e7 用例)· DatabaseConfig.aria 配置透传。<br> <strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e7 用例)· DatabaseConfig.aria 配置透传。<br>
<strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)· 1009 测试 62 套件 · 87.3% 行覆盖率。</p> <strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)<br>
<strong>v0.6.0 完全移除 IndexedDB</strong> — 自研 KVStore 事务存储引擎(多 key 原子写 = 单日志记录原子追加 · 快照 checkpoint + 两阶段崩溃恢复 · CRC-32 自愈)· disk 模式切换 KVStoreEngine(替代 IndexedDBEngine + OPFSEngine)· 事务内 DDL / 外键级联 / 二级索引完整持久化 · migrateFromIndexedDB() 旧库一键迁移 · 10 万 key 压力验证 · e2e 崩溃注入(9 用例)· 1021 测试 64 套件 · 89.1% 行覆盖率。</p>
<h3>存储模式对比</h3> <h3>存储模式对比</h3>
<table> <table>
+4 -4
View File
@@ -153,7 +153,7 @@
<!-- Hero --> <!-- Hero -->
<section class="hero"> <section class="hero">
<div class="container"> <div class="container">
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.5.1 存储后端生产级硬化 — 1009测试 62套件 · 全库 AES-GCM 加密 · WAL 分片 · SSTable 页面化 · 多标签页锁 · 真实 Chromium e2e</div> <div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.6.0 完全移除 IndexedDB — 1021测试 64套件 · 自研 KVStore 事务引擎 · 多 key 原子写 · 崩溃恢复 · 旧库一键迁移</div>
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1> <h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p> <p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
<div class="actions"> <div class="actions">
@@ -414,12 +414,12 @@ npm install @metona-team/metona-sqlark
<p>MetonaSqlark 的核心指标</p> <p>MetonaSqlark 的核心指标</p>
</div> </div>
<div class="stats"> <div class="stats">
<div class="stat-card"><div class="num">1009</div><div class="label">测试用例</div></div> <div class="stat-card"><div class="num">1021</div><div class="label">测试用例</div></div>
<div class="stat-card"><div class="num">87.3%</div><div class="label">行覆盖率</div></div> <div class="stat-card"><div class="num">89.1%</div><div class="label">行覆盖率</div></div>
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div> <div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div> <div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
<div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div> <div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div>
<div class="stat-card"><div class="num">62</div><div class="label">测试套件</div></div> <div class="stat-card"><div class="num">64</div><div class="label">测试套件</div></div>
</div> </div>
</div> </div>
</section> </section>
+4 -4
View File
@@ -10,8 +10,8 @@
/** 存储模式 */ /** 存储模式 */
export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria'; export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
/** 磁盘引擎类型 */ /** 磁盘引擎类型v0.6.0: IndexedDB 已移除,'memory' 供 aria 内存后端) */
export type DiskEngine = 'indexeddb' | 'opfs'; export type DiskEngine = 'opfs' | 'memory';
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 字段类型 // 字段类型
@@ -100,7 +100,7 @@ export interface DatabaseConfig {
export const DB_DEFAULTS: Readonly<Required<Omit<DatabaseConfig, 'plugins' | 'onReady' | 'onError' | 'aria'>>> & { aria: undefined } = Object.freeze({ export const DB_DEFAULTS: Readonly<Required<Omit<DatabaseConfig, 'plugins' | 'onReady' | 'onError' | 'aria'>>> & { aria: undefined } = Object.freeze({
name: 'metona-sqlark', name: 'metona-sqlark',
mode: 'hybrid' as const, mode: 'hybrid' as const,
diskEngine: 'indexeddb' as const, diskEngine: 'opfs' as const,
version: 1, version: 1,
maxRowsPerQuery: 0, // 0 = 不限制 maxRowsPerQuery: 0, // 0 = 不限制
debug: false, debug: false,
@@ -214,4 +214,4 @@ export class DatabaseError extends Error {
// 版本 // 版本
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
export const VERSION = '0.5.1'; export const VERSION = '0.6.0';
+5 -5
View File
@@ -9,8 +9,7 @@ import type { IStorageEngine } from './engine/interface';
import type { DatabaseConfig, ColumnDef } from './constants'; import type { DatabaseConfig, ColumnDef } from './constants';
import { DB_DEFAULTS, DatabaseError } from './constants'; import { DB_DEFAULTS, DatabaseError } from './constants';
import { MemoryEngine } from './engine/memory'; import { MemoryEngine } from './engine/memory';
import { IndexedDBEngine } from './engine/indexeddb'; import { KVStoreEngine } from './engine/kvstore_engine';
import { OPFSEngine } from './engine/opfs';
import { AriaEngine } from './engine/aria/index'; import { AriaEngine } from './engine/aria/index';
import { HybridEngine } from './hybrid/index'; import { HybridEngine } from './hybrid/index';
import { Table } from './table/table'; import { Table } from './table/table';
@@ -548,17 +547,18 @@ export class MetonaSqlark {
private createEngine(): IStorageEngine { private createEngine(): IStorageEngine {
const mode = this.mode; const mode = this.mode;
const diskEngine = this.config.diskEngine ?? 'indexeddb'; const diskEngine = this.config.diskEngine ?? 'opfs';
switch (mode) { switch (mode) {
case 'memory': case 'memory':
return new MemoryEngine(); return new MemoryEngine();
case 'disk': case 'disk':
return diskEngine === 'opfs' ? new OPFSEngine() : new IndexedDBEngine(); // v0.6.0: 自研 KVStoreEngine(完全移除 IndexedDB
return new KVStoreEngine();
case 'aria': case 'aria':
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等) // v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
return new AriaEngine({ return new AriaEngine({
storageBackend: diskEngine === 'opfs' ? 'opfs' : 'indexeddb', storageBackend: diskEngine === 'memory' ? 'memory' : 'opfs',
...(this.config.aria ?? {}), ...(this.config.aria ?? {}),
}); });
case 'hybrid': case 'hybrid':
+3 -5
View File
@@ -21,7 +21,7 @@ import { SegmentedWALStore } from './wal/segmented_store';
import { DatabaseLock } from './locks'; import { DatabaseLock } from './locks';
import { WALRecordType, type WALRecord } from './types'; import { WALRecordType, type WALRecord } from './types';
import { CheckpointManager } from './wal/checkpoint'; import { CheckpointManager } from './wal/checkpoint';
import { IndexedDBBackend, MemoryBackend, type IStorageBackend } from './store/backend'; import { MemoryBackend, type IStorageBackend } from './store/backend';
import { OPFSBackend } from './store/opfs_backend'; import { OPFSBackend } from './store/opfs_backend';
import { EncryptedBackend } from './store/encrypted_backend'; import { EncryptedBackend } from './store/encrypted_backend';
import { PageSSTableStore } from './store/page_sstable_store'; import { PageSSTableStore } from './store/page_sstable_store';
@@ -117,8 +117,6 @@ export class AriaEngine implements IStorageEngine {
let baseBackend: IStorageBackend; let baseBackend: IStorageBackend;
if (this.config.storageBackend === 'opfs') { if (this.config.storageBackend === 'opfs') {
baseBackend = new OPFSBackend(); baseBackend = new OPFSBackend();
} else if (this.config.storageBackend === 'indexeddb') {
baseBackend = new IndexedDBBackend();
} else { } else {
baseBackend = new MemoryBackend(); baseBackend = new MemoryBackend();
} }
@@ -300,10 +298,10 @@ export class AriaEngine implements IStorageEngine {
*/ */
async repair(): Promise<void> { async repair(): Promise<void> {
this.ensureOpen(); this.ensureOpen();
// v0.6.0-fix: 先清页面缓存再校验 — 缓存中的"完好页面"会掩盖磁盘损坏
await this.bufferPool.clear();
// 1. 校验全部 SSTable,移除残缺项(打开时已做一次,此处兜底运行期损坏) // 1. 校验全部 SSTable,移除残缺项(打开时已做一次,此处兜底运行期损坏)
const removed = await this.lsm.validateAll(); const removed = await this.lsm.validateAll();
// v0.4.5: 清空页面缓存(损坏数据可能驻留 BufferPool,重新加载)
await this.bufferPool.clear();
// 2. 将 WAL 残留数据落盘并截断,避免无限重放(含空洞截断落地) // 2. 将 WAL 残留数据落盘并截断,避免无限重放(含空洞截断落地)
await this.lsm.flush(); await this.lsm.flush();
await this.wal.checkpoint(); await this.wal.checkpoint();
+5 -3
View File
@@ -683,12 +683,14 @@ export class LSM {
private async dropInvalidSSTable(meta: SSTableMeta): Promise<void> { private async dropInvalidSSTable(meta: SSTableMeta): Promise<void> {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.warn(`[AriaEngine LSM] Skipping corrupted SSTable id=${meta.id} (level=${meta.level})`); console.warn(`[AriaEngine LSM] Skipping corrupted SSTable id=${meta.id} (level=${meta.level})`);
try { // v0.6.0-fix: 先删数据文件再删 meta — 页面化存储的 delete 依赖 meta.pageIds
await this.sstableStore.deleteMeta(meta.id); // 定位页面文件;先删 meta 会丢失 pageIds 导致孤儿页面残留
} catch { /* 清理失败不阻塞打开 */ }
try { try {
await this.sstableStore.delete(meta.id); await this.sstableStore.delete(meta.id);
} catch { /* 清理失败不阻塞打开 */ } } catch { /* 清理失败不阻塞打开 */ }
try {
await this.sstableStore.deleteMeta(meta.id);
} catch { /* 清理失败不阻塞打开 */ }
} }
private unwrapTombstone(value: Record<string, unknown> | null): Record<string, unknown> | null { private unwrapTombstone(value: Record<string, unknown> | null): Record<string, unknown> | null {
-133
View File
@@ -49,139 +49,6 @@ export interface IStorageBackend {
clear(): Promise<void>; clear(): Promise<void>;
} }
// =======================================================================
// IndexedDB Backend
// =======================================================================
export class IndexedDBBackend implements IStorageBackend {
private db: IDBDatabase | null = null;
private dbName = '';
private storeName = 'data';
async open(name: string): Promise<void> {
this.dbName = `aria-${name}`;
return new Promise((resolve, reject) => {
const request = indexedDB.open(this.dbName, 1);
request.onupgradeneeded = () => {
const db = request.result;
if (!db.objectStoreNames.contains(this.storeName)) {
db.createObjectStore(this.storeName);
}
};
request.onsuccess = () => {
this.db = request.result;
resolve();
};
request.onerror = () => reject(new DatabaseError('Failed to open AriaEngine IndexedDB', 'ARIA_IDB_OPEN_ERROR'));
});
}
async close(): Promise<void> {
if (this.db) {
this.db.close();
this.db = null;
}
}
isOpen(): boolean {
return this.db !== null;
}
async read(key: string): Promise<ArrayBuffer | null> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readonly');
const req = tx.objectStore(this.storeName).get(key);
req.onsuccess = () => resolve(req.result ?? null);
req.onerror = () => reject(new DatabaseError('Failed to read from AriaEngine store', 'ARIA_READ_ERROR'));
});
}
async write(key: string, data: ArrayBuffer): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readwrite');
tx.objectStore(this.storeName).put(data, key);
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError('Failed to write to AriaEngine store', 'ARIA_WRITE_ERROR'));
});
}
/**
* v0.4.2-fix: 批量原子写入 IDB key
* WAL count "记录在、计数丢"
*/
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
const keys = Object.keys(entries);
if (keys.length === 0) return;
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readwrite');
const store = tx.objectStore(this.storeName);
for (const key of keys) {
store.put(entries[key], key);
}
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError('Failed to batch write to AriaEngine store', 'ARIA_WRITE_ERROR'));
});
}
async delete(key: string): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readwrite');
tx.objectStore(this.storeName).delete(key);
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError('Failed to delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
});
}
/** v0.4.2-fix: 批量原子删除 — 单个 IDB 事务内删除多个 key */
async deleteMany(keys: string[]): Promise<void> {
if (keys.length === 0) return;
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readwrite');
const store = tx.objectStore(this.storeName);
for (const key of keys) {
store.delete(key);
}
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError('Failed to batch delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
});
}
async listKeys(): Promise<string[]> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readonly');
const req = tx.objectStore(this.storeName).getAllKeys();
req.onsuccess = () => resolve((req.result ?? []) as string[]);
req.onerror = () => reject(new DatabaseError('Failed to list keys', 'ARIA_LIST_ERROR'));
});
}
async exists(key: string): Promise<boolean> {
const result = await this.read(key);
return result !== null;
}
async clear(): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(this.storeName, 'readwrite');
tx.objectStore(this.storeName).clear();
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError('Failed to clear AriaEngine store', 'ARIA_CLEAR_ERROR'));
});
}
private ensureDB(): IDBDatabase {
if (!this.db) throw new DatabaseError('AriaEngine storage not opened', 'ARIA_DB_NOT_OPEN');
return this.db;
}
}
// ======================================================================= // =======================================================================
// Memory Backend(回退 / 测试用) // Memory Backend(回退 / 测试用)
// ======================================================================= // =======================================================================
+2 -2
View File
@@ -207,7 +207,7 @@ export interface AriaEngineConfig {
/** 是否启用页面压缩(默认 false) */ /** 是否启用页面压缩(默认 false) */
compression?: boolean; compression?: boolean;
/** 存储后端 */ /** 存储后端 */
storageBackend?: 'indexeddb' | 'opfs' | 'memory'; storageBackend?: 'opfs' | 'memory';
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB */ /** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB */
walSizeThreshold?: number; walSizeThreshold?: number;
/** 最大内存预算(MB,默认 64) */ /** 最大内存预算(MB,默认 64) */
@@ -241,7 +241,7 @@ export const DEFAULT_ARIA_CONFIG: Required<Omit<AriaEngineConfig, 'encryption' |
walSyncMode: 'full', walSyncMode: 'full',
checkpointInterval: 1000, checkpointInterval: 1000,
compression: false, compression: false,
storageBackend: 'indexeddb', storageBackend: 'opfs',
walSizeThreshold: 16 * 1024 * 1024, // 16MB walSizeThreshold: 16 * 1024 * 1024, // 16MB
maxMemoryMB: 64, maxMemoryMB: 64,
encryption: undefined, encryption: undefined,
+1 -2
View File
@@ -5,7 +5,6 @@
export type { IStorageEngine } from './interface'; export type { IStorageEngine } from './interface';
export { MemoryEngine } from './memory'; export { MemoryEngine } from './memory';
export { IndexedDBEngine } from './indexeddb'; export { KVStoreEngine } from './kvstore_engine';
export { OPFSEngine } from './opfs';
export { AriaEngine } from './aria/index'; export { AriaEngine } from './aria/index';
export type { AriaEngineConfig } from './aria/types'; export type { AriaEngineConfig } from './aria/types';
-840
View File
@@ -1,840 +0,0 @@
/**
* metona-sqlark IndexedDB Engine IndexedDB
* @module engine/indexeddb
*
* v0.1.13: 支持事务 beginTransaction IDB commit rollback
*/
import type { IStorageEngine } from './interface';
import type { QueryPlan, TableSchema } from '../constants';
import { DatabaseError } from '../constants';
import { MemoryEngine } from './memory';
import { matchWhere, applyOrderBy, projectColumns } from '../query/where-matcher';
export class IndexedDBEngine implements IStorageEngine {
readonly name = 'indexeddb';
private db: IDBDatabase | null = null;
private dbName = '';
private version = 1;
private memoryCache: MemoryEngine = new MemoryEngine();
// ---- 事务状态 ----
private txActive = false;
async open(dbName: string, version: number): Promise<void> {
// v0.4.2-fix (P0-3): version < 1 归一化为 1indexedDB.open(name, 0) 抛原生 TypeError
const normalizedVersion = version >= 1 ? Math.floor(version) : 1;
this.dbName = dbName;
this.version = normalizedVersion;
await this.memoryCache.open(dbName, normalizedVersion);
// v0.4.2-fix (P0-2/P2-8): 版本自适应打开 + blocked 重试
this.db = await this.openDatabaseWithRetry(dbName, normalizedVersion);
this.setupVersionChangeHandler();
// v0.4.2-fix (P2-7): 确保 schema/meta 持久化 store 存在(新库或旧库升级时创建),
// 否则迁移版本等库内元数据无处落盘
await this.ensureSchemaStore();
try {
// v0.3.2: reopen 后从 IDB 重建 schemaschema 此前只存内存缓存,重开连接即丢失)
await this.rebuildSchemaFromIDB();
} catch (error) {
throw new DatabaseError(`Failed to restore schema for "${dbName}"`, 'IDB_SCHEMA_RESTORE_ERROR', error);
}
}
/** v0.4.2-fix: 多标签页冲突处理 — 其他标签页升级版本时自动关闭当前连接 */
private setupVersionChangeHandler(): void {
if (!this.db) return;
this.db.onversionchange = () => {
if (this.db) {
this.db.close();
this.db = null;
// eslint-disable-next-line no-console
console.warn(`[metona-sqlark] Database "${this.dbName}" was upgraded in another tab. Connection closed. Please re-open.`);
}
};
}
/**
* v0.4.2-fix: 打开 IndexedDB
* - P0-2: 请求版本低于库实际版本VersionError
* +1config.version
* - P2-8: onblocked IDB_BLOCKED
*/
private async openDatabaseWithRetry(dbName: string, requestedVersion: number): Promise<IDBDatabase> {
const BLOCKED_RETRIES = 10;
let effectiveVersion = requestedVersion;
for (let attempt = 0; attempt < BLOCKED_RETRIES; attempt++) {
try {
return await this.openRequest(dbName, effectiveVersion, 200 + attempt * 150);
} catch (error) {
const err = error as { name?: string };
if (err && err.name === 'VersionError') {
const currentVersion = await this.resolveCurrentVersion(dbName);
if (currentVersion >= 1 && currentVersion !== effectiveVersion) {
effectiveVersion = currentVersion;
this.version = currentVersion;
continue;
}
throw new DatabaseError(
`Failed to open IndexedDB "${dbName}": version mismatch`,
'IDB_VERSION_ERROR',
error,
);
}
if (err && err.name === 'BlockedError') {
// 另一连接短暂持有 → 等待后重试
await new Promise((resolve) => setTimeout(resolve, 50 * (attempt + 1)));
continue;
}
throw new DatabaseError(`Failed to open IndexedDB "${dbName}"`, 'IDB_OPEN_ERROR', error);
}
}
throw new DatabaseError(`IndexedDB "${dbName}" is blocked`, 'IDB_BLOCKED');
}
/**
* indexedDB.open success/error/blocked
* onblocked success
* "拒绝后连接迟到成功"
*/
private openRequest(dbName: string, version: number, timeoutMs: number): Promise<IDBDatabase> {
return new Promise((resolve, reject) => {
const request = indexedDB.open(dbName, version);
let settled = false;
const timeout = setTimeout(() => {
if (settled) return;
settled = true;
// 兼容无 DOMException 构造环境
const blockedError = typeof DOMException !== 'undefined'
? new DOMException('IndexedDB open is blocked', 'BlockedError')
: Object.assign(new Error('IndexedDB open is blocked'), { name: 'BlockedError' });
reject(blockedError);
}, timeoutMs);
request.onsuccess = () => {
if (settled) {
// 超时判失败后连接迟到成功:立即关闭,避免阻塞后续版本升级
request.result.close();
return;
}
settled = true;
clearTimeout(timeout);
resolve(request.result);
};
request.onerror = () => {
if (settled) return;
settled = true;
clearTimeout(timeout);
reject(request.error ?? new Error('Unknown IndexedDB open error'));
};
request.onblocked = () => {
// 保持等待,不拒绝(阻塞解除后 success 会触发;超时由 timer 兜底)
};
});
}
/** 无版本参数打开库,解析其当前实际版本号(随后立即关闭) */
private resolveCurrentVersion(dbName: string): Promise<number> {
return new Promise((resolve, reject) => {
const request = indexedDB.open(dbName);
request.onsuccess = () => {
const actualVersion = request.result.version;
request.result.close();
resolve(actualVersion);
};
request.onerror = () => {
reject(request.error ?? new Error('Failed to resolve IndexedDB version'));
};
});
}
/**
* v0.4.2-fix (P2-7): __metona_schema store
* store
* 使 getMeta/setMeta
*/
private async ensureSchemaStore(): Promise<void> {
if (!this.db) return;
if (this.db.objectStoreNames.contains('__metona_schema')) return;
const newVersion = this.db.version + 1;
this.db.close();
this.db = await new Promise<IDBDatabase>((resolve, reject) => {
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = () => {
const idb = request.result;
if (!idb.objectStoreNames.contains('__metona_schema')) {
idb.createObjectStore('__metona_schema', { keyPath: 'name' });
}
};
request.onsuccess = () => {
this.db = request.result;
this.setupVersionChangeHandler();
resolve(request.result);
};
request.onerror = () => reject(
new DatabaseError('Failed to create schema store', 'IDB_UPGRADE_ERROR', request.error),
);
});
}
/**
* IDB schema
* 1. schema '__metona_schema' storev0.3.2
* 2. 退 objectStore / /
*/
private async rebuildSchemaFromIDB(): Promise<void> {
const db = this.ensureDB();
// 1. 持久化 schema
if (db.objectStoreNames.contains('__metona_schema')) {
const records: { name: string; schema: string }[] = await new Promise((resolve, reject) => {
const req = db.transaction('__metona_schema', 'readonly').objectStore('__metona_schema').getAll();
req.onsuccess = () => resolve((req.result ?? []) as { name: string; schema: string }[]);
req.onerror = () => reject(req.error);
});
for (const rec of records) {
try {
const schema = JSON.parse(rec.schema) as TableSchema;
if (!(await this.memoryCache.getTableSchema(schema.name))) {
await this.memoryCache.createTable(schema);
}
} catch {
// 损坏的 schema 记录忽略
}
}
}
// 2. 回退:无持久化 schema 的表从 IDB 结构推断
const storeNames = Array.from(db.objectStoreNames).filter((n) => n !== '__metona_schema');
for (const tableName of storeNames) {
// 已有 schema(持久化恢复或连续 open)则跳过
const existing = await this.memoryCache.getTableSchema(tableName);
if (existing) continue;
const columns: Record<string, import('../constants').ColumnDef> = {};
const tx = db.transaction(tableName, 'readonly');
const store = tx.objectStore(tableName);
// 主键列
const pk = store.keyPath as string;
columns[pk] = { type: 'string', primaryKey: true };
// 索引列(idx_ 前缀约定)
for (const idxName of Array.from(store.indexNames)) {
if (idxName.startsWith('idx_')) {
const col = idxName.slice(4);
if (!columns[col]) {
columns[col] = { type: 'string', index: true };
}
}
}
// 从样例数据推断其余列的类型
const rows: Record<string, unknown>[] = await new Promise((resolve, reject) => {
const req = store.getAll();
req.onsuccess = () => resolve((req.result ?? []) as Record<string, unknown>[]);
req.onerror = () => reject(req.error);
});
if (rows.length > 0) {
for (const [key, value] of Object.entries(rows[0])) {
if (!columns[key]) {
columns[key] = { type: inferFieldType(value) };
}
}
}
await this.memoryCache.createTable({ name: tableName, columns });
}
}
async close(): Promise<void> {
// v0.4.3-fix: 活跃事务时先回滚(避免 commit 对已关闭连接报错)
if (this.txActive) {
try {
await this.rollbackTransaction();
} catch { /* 回滚失败不阻塞关闭 */ }
}
if (this.db) {
this.db.onversionchange = null; // 清理监听器
this.db.close();
this.db = null;
}
await this.memoryCache.close();
}
/**
* v0.4.2-fix: 自愈 schema schema /
*
*/
async repair(): Promise<void> {
if (!this.db) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
await this.memoryCache.close();
await this.memoryCache.open(this.dbName, this.version);
await this.rebuildSchemaFromIDB();
}
/**
* v0.4.2-fix: 清空全部数据与表结构 schema
*
*/
async clearAll(): Promise<void> {
const db = this.ensureDB();
// 先清内存缓存
const tableNames = await this.memoryCache.getTableNames();
for (const name of tableNames) {
await this.memoryCache.dropTable(name);
}
// 重置 IDB:删除所有表 store + 清空 schema/meta store
const newVersion = db.version + 1;
db.close();
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = (event) => {
const idb = (event.target as IDBOpenDBRequest).result;
const toDelete = Array.from(idb.objectStoreNames).filter((n) => n !== '__metona_schema');
for (const name of toDelete) {
idb.deleteObjectStore(name);
}
if (!idb.objectStoreNames.contains('__metona_schema')) {
idb.createObjectStore('__metona_schema', { keyPath: 'name' });
} else {
// 保留 store 但清空内容(含持久化 schema 与 meta 记录)
const tx = (event.target as IDBOpenDBRequest).transaction!;
tx.objectStore('__metona_schema').clear();
}
};
request.onsuccess = () => {
this.db = request.result;
this.setupVersionChangeHandler();
resolve();
};
request.onerror = () => reject(new DatabaseError('Failed to clear database', 'IDB_CLEAR_ERROR', request.error));
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${this.dbName}" is blocked`, 'IDB_BLOCKED'));
});
}
// ---- 库内元数据(v0.4.2-fix:迁移版本持久化用,复用 __metona_schema store ----
async getMeta(key: string): Promise<string | null> {
const db = this.ensureDB();
if (!db.objectStoreNames.contains('__metona_schema')) return null;
return new Promise((resolve, reject) => {
const req = db.transaction('__metona_schema', 'readonly')
.objectStore('__metona_schema').get(`__meta:${key}`);
req.onsuccess = () => {
const rec = req.result as { schema?: unknown } | undefined;
resolve(rec && typeof rec.schema === 'string' ? rec.schema : null);
};
req.onerror = () => reject(req.error);
});
}
async setMeta(key: string, value: string): Promise<void> {
const db = this.ensureDB();
if (!db.objectStoreNames.contains('__metona_schema')) return;
await new Promise<void>((resolve, reject) => {
const tx = db.transaction('__metona_schema', 'readwrite');
tx.objectStore('__metona_schema').put({ name: `__meta:${key}`, schema: value });
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Failed to persist meta "${key}"`, 'IDB_META_ERROR', tx.error));
});
}
isOpen(): boolean { return this.db !== null; }
// ---- 表管理 ----
async createTable(schema: TableSchema): Promise<void> {
await this.memoryCache.createTable(schema);
if (this.txActive) return; // 事务中延迟 IDB 操作
await this.idbCreateTable(schema);
}
async dropTable(tableName: string): Promise<void> {
await this.memoryCache.dropTable(tableName);
if (this.txActive) return;
await this.idbDropTable(tableName);
}
async hasTable(tableName: string): Promise<boolean> { return this.ensureDB().objectStoreNames.contains(tableName); }
async getTableNames(): Promise<string[]> { return Array.from(this.ensureDB().objectStoreNames); }
async getTableSchema(tableName: string): Promise<TableSchema | null> { return this.memoryCache.getTableSchema(tableName); }
/**
* v0.4.2-fix: 引擎级 ALTER TABLE schema __metona_schema store
* ALTER 退DROP
*/
async alterTable(
tableName: string,
action: 'ADD' | 'DROP',
column: import('../constants').ColumnDef & { name: string },
): Promise<void> {
await this.memoryCache.alterTable(tableName, action, column);
if (this.txActive) return; // 事务中:commit 时统一 flushToIDB 同步
const schema = await this.memoryCache.getTableSchema(tableName);
if (schema) {
await this.persistSchema(schema);
}
if (action === 'DROP') {
// 重写 IDB 存储行:移除该列键(store.put 经 keyPath 自动覆盖原行)
const db = this.ensureDB();
const rows = await this.idbFind(tableName, { table: tableName });
const rewritten = rows.map((row) => {
if (column.name in row) {
const copy = { ...row };
delete (copy as Record<string, unknown>)[column.name];
return copy;
}
return row;
});
if (rewritten.length > 0) {
await new Promise<void>((resolve, reject) => {
const tx = db.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
for (const row of rewritten) {
store.put(row);
}
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Failed to rewrite "${tableName}" after DROP COLUMN`, 'IDB_TX_ERROR', tx.error));
});
}
}
}
// ---- CRUD ----
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
const pks = await this.memoryCache.insert(tableName, rows);
if (this.txActive) return pks; // 事务中延迟写入
await this.idbInsert(tableName, rows);
return pks;
}
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
// 事务中从内存缓存读取(保证读到未提交的变更),否则走 IDB
if (this.txActive) return this.memoryCache.find(tableName, query);
return this.idbFind(tableName, query);
}
/** v0.4.0: 流式查询 — IDB 批量读入后逐行回调(保持接口一致性) */
async findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number> {
if (this.txActive) {
return this.memoryCache.findStream(tableName, query, onRow);
}
const rows = await this.idbFind(tableName, { ...query, orderBy: undefined, limit: undefined, offset: undefined });
let count = 0;
for (const row of rows) {
onRow(row);
count++;
}
return count;
}
async update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number> {
const count = await this.memoryCache.update(tableName, query, updates);
if (this.txActive) return count;
await this.idbUpdate(tableName, query, updates);
return count;
}
async delete(tableName: string, query: QueryPlan): Promise<number> {
const count = await this.memoryCache.delete(tableName, query);
if (this.txActive) return count;
await this.idbDelete(tableName, query);
return count;
}
async count(tableName: string, query?: QueryPlan): Promise<number> {
if (this.txActive) return this.memoryCache.count(tableName, query);
const results = await this.idbFind(tableName, { table: tableName, where: query?.where ?? {} });
return results.length;
}
async clear(tableName: string): Promise<void> {
await this.memoryCache.clear(tableName);
if (this.txActive) return;
await this.idbClear(tableName);
}
// ---- 动态索引(v0.3.0):通过版本升级创建/删除 IDB 索引 ----
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
await this.memoryCache.createIndex(tableName, column, unique);
if (this.txActive) return;
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const newVersion = db.version + 1; db.close();
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = (event) => {
const idb = (event.target as IDBOpenDBRequest).result;
const tx = idb.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
if (!store.indexNames.contains(`idx_${column}`)) {
store.createIndex(`idx_${column}`, column, { unique: unique ?? false });
}
};
request.onsuccess = () => { this.db = request.result; resolve(); };
request.onerror = () => reject(new DatabaseError(`Failed to create index "${tableName}.${column}"`, 'IDB_UPGRADE_ERROR', request.error));
});
}
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
await this.memoryCache.dropIndex(tableName, column);
if (this.txActive) return;
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const newVersion = db.version + 1; db.close();
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = (event) => {
const idb = (event.target as IDBOpenDBRequest).result;
const tx = idb.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
if (store.indexNames.contains(`idx_${column}`)) {
store.deleteIndex(`idx_${column}`);
}
};
request.onsuccess = () => { this.db = request.result; resolve(); };
request.onerror = () => reject(new DatabaseError(`Failed to drop index "${tableName}.${column}"`, 'IDB_UPGRADE_ERROR', request.error));
});
}
// ---- 事务 ----
async beginTransaction(): Promise<void> {
if (this.txActive) throw new DatabaseError('Transaction already in progress', 'TX_ACTIVE');
this.txActive = true;
// 保存内存快照到 MemoryEngine 内部的 beginTransaction
await this.memoryCache.beginTransaction();
}
async commitTransaction(): Promise<void> {
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
// v0.4.2-fix: 先刷盘后提交内存快照 — 此前先 memoryCache.commitTransaction()
// 再 flushToIDBflush 失败时 snapshot 已丢,回滚报 TX_NONE 且内存数据已确认
await this.flushToIDB();
await this.memoryCache.commitTransaction();
this.txActive = false;
}
async rollbackTransaction(): Promise<void> {
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
// 恢复内存层到快照状态
await this.memoryCache.rollbackTransaction();
this.txActive = false;
}
// ---- IDB 原生操作 ----
private async idbCreateTable(schema: TableSchema): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const newVersion = db.version + 1; db.close();
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = (event) => {
const db = (event.target as IDBOpenDBRequest).result;
const pkColumn = Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0];
const store = db.createObjectStore(schema.name, { keyPath: pkColumn });
for (const [colName, colDef] of Object.entries(schema.columns)) {
if (colDef.index && colName !== pkColumn) {
store.createIndex(`idx_${colName}`, colName, { unique: colDef.unique ?? false });
}
}
// v0.3.2: schema 持久化 store(记录在升级完成后的 onsuccess 写入)
if (!db.objectStoreNames.contains('__metona_schema')) {
db.createObjectStore('__metona_schema', { keyPath: 'name' });
}
};
request.onsuccess = () => {
this.db = request.result;
// v0.3.2: 升级完成后持久化 schema(upgrade 事务内异步写会失败)
const schemaTx = this.db.transaction('__metona_schema', 'readwrite');
schemaTx.objectStore('__metona_schema').put({ name: schema.name, schema: JSON.stringify(schema) });
schemaTx.oncomplete = () => resolve();
schemaTx.onerror = () => reject(new DatabaseError(`Failed to persist schema for "${schema.name}"`, 'IDB_SCHEMA_ERROR', schemaTx.error));
};
request.onerror = () => reject(new DatabaseError(`Failed to create table "${schema.name}"`, 'IDB_UPGRADE_ERROR', request.error));
});
}
private async idbDropTable(tableName: string): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const newVersion = db.version + 1; db.close();
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = (event) => {
const db = (event.target as IDBOpenDBRequest).result;
if (db.objectStoreNames.contains(tableName)) db.deleteObjectStore(tableName);
};
request.onsuccess = () => {
this.db = request.result;
// v0.3.2: 清理持久化 schema 记录(upgrade 后执行,失败不阻塞删除)
if (this.db.objectStoreNames.contains('__metona_schema')) {
const schemaTx = this.db.transaction('__metona_schema', 'readwrite');
schemaTx.objectStore('__metona_schema').delete(tableName);
schemaTx.onerror = () => { /* 忽略:旧库可能无此记录 */ };
}
resolve();
};
request.onerror = () => reject(new DatabaseError(`Failed to drop table "${tableName}"`, 'IDB_UPGRADE_ERROR', request.error));
});
}
private async idbInsert(tableName: string, rows: Record<string, unknown>[]): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
for (const row of rows) store.add(row);
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Insert failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
});
}
private async idbFind(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
const db = this.ensureDB();
// 尝试使用 IDB 索引进行等值查询
if (query.where) {
const indexResult = await this.tryIDBIndexLookup(db, tableName, query);
if (indexResult !== null) return indexResult;
}
// 回退到全量 getAll + 内存过滤
return new Promise((resolve, reject) => {
const tx = db.transaction(tableName, 'readonly');
const req = tx.objectStore(tableName).getAll();
req.onsuccess = () => {
let results: Record<string, unknown>[] = req.result ?? [];
if (query.where && Object.keys(query.where).length > 0) {
results = results.filter((row) => matchWhere(row, query.where!));
}
if (query.orderBy && query.orderBy.length > 0) {
results = applyOrderBy(results, query.orderBy);
}
const offset = query.offset ?? 0;
const limit = query.limit ?? results.length;
results = results.slice(offset, offset + limit);
if (query.columns && query.columns.length > 0 && query.columns[0] !== '*') {
results = results.map((r) => projectColumns(r, query.columns!));
}
resolve(results);
};
req.onerror = () => reject(new DatabaseError(`Find failed for "${tableName}"`, 'IDB_READ_ERROR', req.error));
});
}
/** 尝试使用 IDB 索引进行等值查询,成功返回结果,不适用返回 null */
private async tryIDBIndexLookup(
db: IDBDatabase,
tableName: string,
query: QueryPlan,
): Promise<Record<string, unknown>[] | null> {
if (!query.where) return null;
for (const [col, condition] of Object.entries(query.where)) {
// 跳过逻辑组合符
if (col === '$and' || col === '$or' || col === '$not') continue;
// 只处理等值查询
let targetValue: unknown;
if (typeof condition !== 'object' || condition === null) {
targetValue = condition;
} else if ('$eq' in (condition as Record<string, unknown>)) {
targetValue = (condition as Record<string, unknown>).$eq;
} else {
continue;
}
// 检查是否有对应的 IDB 索引
const indexName = `idx_${col}`;
try {
return await new Promise((resolve, reject) => {
const tx = db.transaction(tableName, 'readonly');
const store = tx.objectStore(tableName);
if (!store.indexNames.contains(indexName)) {
resolve(null); // 没有索引,回退
return;
}
const index = store.index(indexName);
const req = index.getAll(targetValue as IDBValidKey);
req.onsuccess = () => {
let results: Record<string, unknown>[] = req.result ?? [];
// 如果有其他 WHERE 条件,继续过滤
const otherKeys = Object.keys(query.where!).filter(
(k) => k !== col && k !== '$and' && k !== '$or' && k !== '$not',
);
if (otherKeys.length > 0) {
results = results.filter((row) => matchWhere(row, query.where!));
}
if (query.orderBy && query.orderBy.length > 0) {
results = applyOrderBy(results, query.orderBy);
}
const offset = query.offset ?? 0;
const limit = query.limit ?? results.length;
results = results.slice(offset, offset + limit);
if (query.columns && query.columns.length > 0 && query.columns[0] !== '*') {
results = results.map((r) => projectColumns(r, query.columns!));
}
resolve(results);
};
req.onerror = () => reject(new DatabaseError(`Index lookup failed for "${tableName}.${col}"`, 'IDB_READ_ERROR', req.error));
});
} catch {
return null; // 索引不可用,回退
}
}
return null;
}
private async idbUpdate(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
const getAllReq = store.getAll();
getAllReq.onsuccess = () => {
for (const row of getAllReq.result ?? []) {
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
Object.assign(row, updates); store.put(row);
}
}
};
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Update failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
});
}
private async idbDelete(tableName: string, query: QueryPlan): Promise<void> {
const db = this.ensureDB();
const schema = await this.memoryCache.getTableSchema(tableName);
if (!schema) throw new DatabaseError(`Table "${tableName}" not found`, 'TABLE_NOT_FOUND');
const pkColumn = Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0];
return new Promise((resolve, reject) => {
const tx = db.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
const getAllReq = store.getAll();
getAllReq.onsuccess = () => {
for (const row of getAllReq.result ?? []) {
if (!query.where || Object.keys(query.where).length === 0 || matchWhere(row, query.where)) {
store.delete(row[pkColumn] as IDBValidKey);
}
}
};
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Delete failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
});
}
private async idbClear(tableName: string): Promise<void> {
const db = this.ensureDB();
return new Promise((resolve, reject) => {
const tx = db.transaction(tableName, 'readwrite');
tx.objectStore(tableName).clear();
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Clear failed for "${tableName}"`, 'IDB_TX_ERROR', tx.error));
});
}
/** 持久化单个表 schema 到 __metona_schema storev0.4.2-fix: ALTER TABLE 用) */
private async persistSchema(schema: TableSchema): Promise<void> {
const db = this.ensureDB();
if (!db.objectStoreNames.contains('__metona_schema')) return;
await new Promise<void>((resolve, reject) => {
const tx = db.transaction('__metona_schema', 'readwrite');
tx.objectStore('__metona_schema').put({ name: schema.name, schema: JSON.stringify(schema) });
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Failed to persist schema for "${schema.name}"`, 'IDB_SCHEMA_ERROR', tx.error));
});
}
/** 将内存缓存中的所有表数据原子性刷新到 IndexedDB */
private async flushToIDB(): Promise<void> {
const tableNames = await this.memoryCache.getTableNames();
let db = this.ensureDB();
// v0.4.2-fix: 事务内 DDL 只更新内存,commit 时同步 IDB 的 objectStore 结构:
// 缺失的表 store 创建(并持久化 schema)、已删除的 store 移除(防止重启幽灵表)
const idbStores = Array.from(db.objectStoreNames);
const missing = tableNames.filter((t) => !idbStores.includes(t));
const stale = idbStores.filter((s) => s !== '__metona_schema' && !tableNames.includes(s));
if (missing.length > 0 || stale.length > 0) {
// 升级事务内是同步上下文,先异步收集缺失表的 schema(主键列定义)
const schemaMap = new Map<string, TableSchema>();
for (const name of missing) {
const s = await this.memoryCache.getTableSchema(name);
if (s) schemaMap.set(name, s);
}
const newVersion = db.version + 1;
db.close();
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(this.dbName, newVersion);
request.onupgradeneeded = (event) => {
const idb = (event.target as IDBOpenDBRequest).result;
for (const name of stale) {
idb.deleteObjectStore(name);
}
for (const name of missing) {
const schema = schemaMap.get(name);
const pkColumn = schema
? (Object.entries(schema.columns).find(([, c]) => c.primaryKey)?.[0] ?? Object.keys(schema.columns)[0])
: undefined;
idb.createObjectStore(name, { keyPath: pkColumn });
}
// 事务内 drop 的表:同步清理持久化 schema 记录(防止重启后幽灵表恢复)
if (idb.objectStoreNames.contains('__metona_schema') && stale.length > 0) {
const tx = (event.target as IDBOpenDBRequest).transaction!;
const store = tx.objectStore('__metona_schema');
for (const name of stale) {
store.delete(name);
}
}
};
request.onsuccess = () => {
this.db = request.result;
this.setupVersionChangeHandler();
resolve();
};
request.onerror = () => reject(new DatabaseError('Failed to sync stores after transaction', 'IDB_UPGRADE_ERROR', request.error));
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${this.dbName}" is blocked`, 'IDB_BLOCKED'));
});
// 事务内新建表的 schema 一并持久化(此前只建 store 不存 schema → 重启后约束推断丢失)
for (const name of missing) {
const schema = await this.memoryCache.getTableSchema(name);
if (schema) await this.persistSchema(schema);
}
// v0.4.2-fix: DDL 升级会 close 旧连接并重新 open — 重新取 db 引用,
// 否则下方数据 flush 用已关闭的连接抛 InvalidStateError
db = this.ensureDB();
}
// 每个表在一个单独的 IDB 事务中完成 clear+insert,保证原子性
for (const tableName of tableNames) {
const rows = await this.memoryCache.find(tableName, { table: tableName });
await new Promise<void>((resolve, reject) => {
const tx = db.transaction(tableName, 'readwrite');
const store = tx.objectStore(tableName);
store.clear(); // 清空
for (const row of rows) store.add(row); // 批量写入
tx.oncomplete = () => resolve();
tx.onerror = () => reject(new DatabaseError(`Flush failed for "${tableName}"`, 'IDB_FLUSH_ERROR', tx.error));
});
}
}
private ensureDB(): IDBDatabase {
if (!this.db) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
return this.db;
}
}
/** 从存储值推断字段类型(schema 重建用,v0.3.2 */
function inferFieldType(value: unknown): import('../constants').FieldType {
if (typeof value === 'number') return 'number';
if (typeof value === 'boolean') return 'boolean';
if (typeof value === 'object' && value !== null) return 'json';
if (typeof value === 'string') {
return isNaN(Date.parse(value)) ? 'string' : 'string';
}
return 'string';
}
+394
View File
@@ -0,0 +1,394 @@
/**
* KVStore KV IndexedDB
* @module engine/kvstore/index
*
* v0.6.0: 在浏览器文件系统OPFS IndexedDB
* - key putMany/deleteMany COW
* IndexedDB
* - checkpoint+ WAL
* - 退
* - checkpoint = meta meta
*
*
* IStorageBackendOPFSBackend / SharedMemoryBackend
* - OPFSnavigator.storage
* - Node/SharedMemoryBackend
*
*
* - checkpoint
* - CRC-32
* - get O(1)checkpoint
*/
import type { IStorageBackend } from '../aria/store/backend';
import { OPFSBackend } from '../aria/store/opfs_backend';
import { SharedMemoryBackend } from './shared_memory_medium';
import { DatabaseError } from '../../constants';
import { encodeLogRecord, parseLogRecords, KVLogOp } from './log';
import { encodeSnapshot, decodeSnapshot } from './snapshot';
import { crc32 } from '../aria/crc32';
/** 存储键 */
const LOG_KEY = '__kv_log';
const SNAPSHOT_KEY = '__kv_snapshot';
const META_KEY = '__kv_meta';
/** checkpoint 自动触发阈值(日志字节数,0=不自动) */
const DEFAULT_CHECKPOINT_THRESHOLD = 16 * 1024 * 1024;
interface KVStoreMeta {
/** 当前日志水位(快照内嵌;无快照时 0) */
seq: number;
}
function defaultMedium(): IStorageBackend {
const nav = (globalThis as { navigator?: { storage?: { getDirectory?: unknown } } }).navigator;
if (typeof nav !== 'undefined' && nav.storage && typeof nav.storage.getDirectory === 'function') {
return new OPFSBackend();
}
return new SharedMemoryBackend();
}
export class KVStore {
private medium: IStorageBackend;
private dbName = '';
private opened = false;
/** 内存索引(热路径权威视图) */
private index = new Map<string, ArrayBuffer>();
/** 日志水位(最后一条已应用日志记录序号) */
private seq = 0;
/** 日志累计字节数(checkpoint 阈值) */
private logBytes = 0;
/** checkpoint 自动触发阈值(字节) */
private checkpointThreshold: number;
/** 写操作串行队列(checkpoint 与写入无交错窗口) */
private opQueue: Promise<unknown> = Promise.resolve();
/** 最近一次后台操作失败(checkpoint 时报告) */
private lastBackgroundError: unknown = null;
constructor(medium?: IStorageBackend, checkpointThreshold: number = DEFAULT_CHECKPOINT_THRESHOLD) {
this.medium = medium ?? defaultMedium();
this.checkpointThreshold = checkpointThreshold;
}
/** 底层介质(测试/诊断用) */
getMedium(): IStorageBackend {
return this.medium;
}
isOpen(): boolean {
return this.opened;
}
// =======================================================================
// 生命周期
// =======================================================================
/** 打开(加载快照 + 重放日志) */
async open(dbName: string): Promise<void> {
if (this.opened) return;
this.dbName = dbName;
await this.medium.open(dbName);
this.index = new Map();
this.seq = 0;
this.logBytes = 0;
// 1. 读 meta(可能缺失/损坏)
let metaSeq = 0;
const metaRaw = await this.medium.read(META_KEY);
if (metaRaw) {
try {
const meta = JSON.parse(new TextDecoder().decode(metaRaw)) as KVStoreMeta;
metaSeq = Number(meta.seq) || 0;
} catch { /* meta 损坏:回退全量日志 */ }
}
// 2. 加载快照(损坏则全量日志重放)
let snapshotSeq = 0;
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
if (snapshotRaw) {
const snap = decodeSnapshot(new Uint8Array(snapshotRaw));
if (snap) {
this.index = new Map(snap.entries);
this.seq = snap.seq;
snapshotSeq = snap.seq;
} else {
// 快照损坏:从空索引 + 全量日志重放
this.index = new Map();
this.seq = 0;
snapshotSeq = 0;
}
}
// 3. 重放日志(seq > 快照水位的记录)
const logRaw = await this.medium.read(LOG_KEY);
if (logRaw && logRaw.byteLength > 0) {
const log = new Uint8Array(logRaw);
const baseSeq = Math.max(metaSeq, snapshotSeq);
const corruptOffsets: number[] = [];
const applied = parseLogRecords(log, (record) => {
if (record.seq <= baseSeq) return; // 快照已包含,跳过(幂等)
this.applyRecord(record.entries);
this.seq = record.seq;
}, (offset) => {
corruptOffsets.push(offset);
return true; // 记录损坏位置后停止(日志是顺序流,无法跳过继续)
});
if (applied > 0 || corruptOffsets.length > 0) {
this.logBytes = log.byteLength;
}
if (corruptOffsets.length > 0) {
// 损坏尾部:截断日志(丢弃未确认记录),下次 checkpoint 落盘
await this.truncateLog();
}
}
this.opened = true;
}
/**
* v0.6.0: 从介质重新加载/
* KVStore open
*/
async reload(): Promise<void> {
if (!this.opened) return;
try { await this.opQueue; } catch { /* ignore */ }
this.index = new Map();
this.seq = 0;
this.logBytes = 0;
this.opened = false;
await this.open(this.dbName);
}
/** 关闭(不丢弃数据;下次 open 同名库恢复) */
async close(): Promise<void> {
if (!this.opened) return;
// 排空写队列
try { await this.opQueue; } catch { /* 写失败已返回 */ }
await this.medium.close();
this.index.clear();
this.seq = 0;
this.logBytes = 0;
this.opened = false;
}
// =======================================================================
// 读写(内存热路径)
// =======================================================================
async get(key: string): Promise<ArrayBuffer | null> {
return this.index.get(key) ?? null;
}
async getAll(): Promise<[string, ArrayBuffer][]> {
return Array.from(this.index.entries());
}
async listKeys(): Promise<string[]> {
return Array.from(this.index.keys());
}
async exists(key: string): Promise<boolean> {
return this.index.has(key);
}
size(): number {
return this.index.size;
}
getSeq(): number {
return this.seq;
}
// =======================================================================
// 写入(原子事务)
// =======================================================================
/** 单 key 写入(原子) */
async put(key: string, value: ArrayBuffer): Promise<void> {
await this.enqueue(async () => {
await this.appendRecord({ [key]: value }, []);
});
}
/** 多 key 原子写入(单条日志记录,崩溃全有或全无) */
async putMany(entries: Record<string, ArrayBuffer>): Promise<void> {
if (Object.keys(entries).length === 0) return;
await this.enqueue(async () => {
await this.appendRecord(entries, []);
});
}
/** 单 key 删除(原子) */
async delete(key: string): Promise<void> {
await this.enqueue(async () => {
await this.appendRecord({}, [key]);
});
}
/** 多 key 原子删除(单条日志记录) */
async deleteMany(keys: string[]): Promise<void> {
if (keys.length === 0) return;
await this.enqueue(async () => {
await this.appendRecord({}, keys);
});
}
// =======================================================================
// 维护
// =======================================================================
/** checkpoint:快照 → meta → 截断日志(时序保证任何崩溃窗口不丢数据) */
async checkpoint(): Promise<void> {
await this.enqueue(async () => {
// 报告上次后台失败
if (this.lastBackgroundError !== null) {
const error = this.lastBackgroundError;
this.lastBackgroundError = null;
throw new DatabaseError('KVStore background write failed', 'KV_BACKGROUND_ERROR', error);
}
if (this.logBytes === 0 && this.index.size === 0) return;
// 1. 写快照(COW 原子)
const snapBytes = encodeSnapshot(this.seq, this.index);
await this.medium.write(SNAPSHOT_KEY, snapBytes.buffer as ArrayBuffer);
// 2. 写 meta(指向新水位)
const meta: KVStoreMeta = { seq: this.seq };
await this.medium.write(META_KEY, new TextEncoder().encode(JSON.stringify(meta)).buffer);
// 3. 截断日志(meta 已更新 → 截断安全)
await this.truncateLog();
});
}
/** 清空全部数据(保留库本身) */
async clear(): Promise<void> {
await this.enqueue(async () => {
await this.medium.clear();
this.index.clear();
this.seq = 0;
this.logBytes = 0;
// 写空 meta(下次 open 正常初始化)
const meta: KVStoreMeta = { seq: 0 };
await this.medium.write(META_KEY, new TextEncoder().encode(JSON.stringify(meta)).buffer);
});
}
/**
* /
* @returns 0 =
*/
async repair(): Promise<number> {
return this.enqueue(async () => {
let discarded = 0;
// 1. 校验快照:损坏则删除(下次打开全量日志重放)
const snapRaw = await this.medium.read(SNAPSHOT_KEY);
if (snapRaw && !decodeSnapshot(new Uint8Array(snapRaw))) {
await this.medium.delete(SNAPSHOT_KEY);
discarded++;
}
// 2. 校验日志:损坏尾部截断
const logRaw = await this.medium.read(LOG_KEY);
if (logRaw && logRaw.byteLength > 0) {
const log = new Uint8Array(logRaw);
const validBytes = this.findValidLogLength(log);
if (validBytes < log.byteLength) {
discarded += log.byteLength - validBytes;
const truncated = log.subarray(0, validBytes).slice().buffer as ArrayBuffer;
await this.medium.write(LOG_KEY, truncated);
}
}
return discarded;
});
}
// =======================================================================
// 内部
// =======================================================================
private enqueue<T>(fn: () => Promise<T>): Promise<T> {
const run = this.opQueue.then(fn, fn);
this.opQueue = run.then(() => undefined, () => undefined);
return run;
}
/** 追加一条日志记录并更新内存索引(队列内调用,无并发) */
private async appendRecord(puts: Record<string, ArrayBuffer>, deletes: string[]): Promise<void> {
this.seq++;
const record = encodeLogRecord(this.seq, puts, deletes);
try {
// 日志追加:介质 append(真追加)或回退读+拼+写
const data = record.buffer.slice(record.byteOffset, record.byteOffset + record.byteLength) as ArrayBuffer;
if (typeof this.medium.append === 'function') {
await this.medium.append(LOG_KEY, data);
} else {
const existing = await this.medium.read(LOG_KEY);
if (existing) {
const combined = new Uint8Array(existing.byteLength + data.byteLength);
combined.set(new Uint8Array(existing), 0);
combined.set(new Uint8Array(data), existing.byteLength);
await this.medium.write(LOG_KEY, combined.buffer as ArrayBuffer);
} else {
await this.medium.write(LOG_KEY, data);
}
}
} catch (error) {
// 记录写入失败:内存索引不更新(原子性),记录后台错误
this.seq--; // 回滚水位
this.lastBackgroundError = error;
throw new DatabaseError('KVStore log append failed', 'KV_LOG_ERROR', error);
}
// 日志成功:更新内存索引(原子语义)
for (const [key, value] of Object.entries(puts)) {
this.index.set(key, value);
}
for (const key of deletes) {
this.index.delete(key);
}
this.logBytes += record.byteLength;
// 自动 checkpoint(日志超阈值)
if (this.checkpointThreshold > 0 && this.logBytes >= this.checkpointThreshold) {
await this.medium.write(SNAPSHOT_KEY, encodeSnapshot(this.seq, this.index).buffer as ArrayBuffer);
const meta: KVStoreMeta = { seq: this.seq };
await this.medium.write(META_KEY, new TextEncoder().encode(JSON.stringify(meta)).buffer);
await this.truncateLog();
}
}
/** 截断日志(清空文件) */
private async truncateLog(): Promise<void> {
try {
await this.medium.write(LOG_KEY, new ArrayBuffer(0));
} catch { /* 截断失败:下次 checkpoint 重试 */ }
this.logBytes = 0;
}
/** 应用记录条目到内存索引 */
private applyRecord(entries: { op: KVLogOp; key: string; value: ArrayBuffer }[]): void {
for (const e of entries) {
if (e.op === KVLogOp.PUT) {
this.index.set(e.key, e.value);
} else {
this.index.delete(e.key);
}
}
}
/** 确定日志中有效字节长度(从 0 开始连续解析到第一条损坏/残缺记录) */
private findValidLogLength(log: Uint8Array): number {
let offset = 0;
const view = new DataView(log.buffer, log.byteOffset, log.byteLength);
while (offset + 4 <= log.byteLength) {
const recordLen = view.getUint32(offset, false);
if (recordLen < 12 || offset + 4 + recordLen > log.byteLength) break;
const raw = log.subarray(offset, offset + 4 + recordLen);
const storedCrc = view.getUint32(offset + recordLen, false);
if (crc32(raw.subarray(0, recordLen)) !== storedCrc) break;
offset += 4 + recordLen;
}
return offset;
}
}
+156
View File
@@ -0,0 +1,156 @@
/**
* KVStore Log
* @module engine/kvstore/log
*
* v0.6.0: 自研 KV
* = putMany / deleteMany
* appendCOW
*
* :
* [recordLen u32] CRC
* [seq u32]
* [entryCount u32]
* entry:
* [op u8] 1=PUT, 2=DELETE
* [keyLen u32][key bytes]
* [valueLen u32][value bytes] DELETE valueLen=0
* [crc u32] CRC CRC-32
*/
import { crc32 } from '../aria/crc32';
/** 日志操作类型 */
export const enum KVLogOp {
PUT = 1,
DELETE = 2,
}
/** 解析后的日志记录 */
export interface KVLogRecord {
/** 日志序号 */
seq: number;
/** 条目列表(op, key, value */
entries: { op: KVLogOp; key: string; value: ArrayBuffer }[];
/** 记录原始字节(CRC 校验用) */
raw: Uint8Array;
}
/**
*
* @param seq
* @param puts key value
* @param deletes key
*/
export function encodeLogRecord(
seq: number,
puts: Record<string, ArrayBuffer>,
deletes: string[] = [],
): Uint8Array {
const encoder = new TextEncoder();
const entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] = [];
for (const [key, value] of Object.entries(puts)) {
entries.push({ op: KVLogOp.PUT, key, value });
}
for (const key of deletes) {
entries.push({ op: KVLogOp.DELETE, key, value: new ArrayBuffer(0) });
}
// 预编码 key 字节,计算总长度
const entryBytes: { op: KVLogOp; key: Uint8Array; value: Uint8Array }[] = [];
let total = 4 + 4 + 4; // recordLen + seq + entryCount
for (const e of entries) {
const kb = encoder.encode(e.key);
const vb = new Uint8Array(e.value);
entryBytes.push({ op: e.op, key: kb, value: vb });
total += 1 + 4 + kb.byteLength + 4 + vb.byteLength;
}
total += 4; // crc
const buf = new Uint8Array(total);
const view = new DataView(buf.buffer);
let offset = 0;
view.setUint32(offset, total - 4, false); offset += 4; // recordLen(不含 CRC
view.setUint32(offset, seq, false); offset += 4;
view.setUint32(offset, entryBytes.length, false); offset += 4;
for (const e of entryBytes) {
view.setUint8(offset, e.op); offset += 1;
view.setUint32(offset, e.key.byteLength, false); offset += 4;
buf.set(e.key, offset); offset += e.key.byteLength;
view.setUint32(offset, e.value.byteLength, false); offset += 4;
buf.set(e.value, offset); offset += e.value.byteLength;
}
const crc = crc32(buf.subarray(0, total - 4));
view.setUint32(total - 4, crc, false);
return buf;
}
/**
*
* @param data
* @param onRecord CRC
* @param onCorrupt false
* @returns
*/
export function parseLogRecords(
data: Uint8Array,
onRecord: (record: KVLogRecord) => void,
onCorrupt?: (offset: number) => boolean,
): number {
let offset = 0;
let count = 0;
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
const decoder = new TextDecoder();
while (offset + 4 <= data.byteLength) {
const recordLen = view.getUint32(offset, false);
if (recordLen < 12 || offset + 4 + recordLen > data.byteLength) {
// 尾部残缺记录(最后一批写入被截断):损坏
if (onCorrupt) {
if (!onCorrupt(offset)) break;
}
break;
}
const recordStart = offset;
const recordEnd = offset + 4 + recordLen;
const raw = data.subarray(recordStart, recordEnd);
const recView = new DataView(data.buffer, data.byteOffset + recordStart, recordLen + 4);
const storedCrc = recView.getUint32(recordLen, false);
const computedCrc = crc32(raw.subarray(0, recordLen));
if (storedCrc !== computedCrc) {
if (onCorrupt) {
if (!onCorrupt(recordStart)) break;
}
break;
}
// 解析条目
let p = 4;
const seq = recView.getUint32(p, false); p += 4;
const entryCount = recView.getUint32(p, false); p += 4;
const entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] = [];
let valid = true;
for (let i = 0; i < entryCount; i++) {
if (p + 1 + 4 > recordLen + 4) { valid = false; break; }
const op = recView.getUint8(p) as KVLogOp; p += 1;
const keyLen = recView.getUint32(p, false); p += 4;
if (p + keyLen + 4 > recordLen + 4) { valid = false; break; }
const key = decoder.decode(raw.subarray(p, p + keyLen)); p += keyLen;
const valueLen = recView.getUint32(p, false); p += 4;
if (p + valueLen > recordLen + 4) { valid = false; break; }
const value = raw.slice(p, p + valueLen).buffer as ArrayBuffer; p += valueLen;
entries.push({ op, key, value });
}
if (!valid) {
if (onCorrupt) {
if (!onCorrupt(recordStart)) break;
}
break;
}
onRecord({ seq, entries, raw });
count++;
offset = recordEnd;
}
return count;
}
@@ -0,0 +1,99 @@
/**
* KVStore SharedMemory Medium
* @module engine/kvstore/shared_memory_medium
*
* v0.6.0: 替代 fake-indexeddb /Node
* MemoryBackend close()
* "磁盘持久化" open
*
* Node 使 OPFS KVStore
*/
import type { IStorageBackend } from '../aria/store/backend';
/** 全局注册表:dbName → key → ArrayBuffer(跨实例共享,模拟持久化) */
const registry = new Map<string, Map<string, ArrayBuffer>>();
export class SharedMemoryBackend implements IStorageBackend {
private dbName = '';
private store: Map<string, ArrayBuffer> | null = null;
/** 清空全局注册表(测试隔离用) */
static clearRegistry(): void {
registry.clear();
}
/** 注册表中的库数量(测试诊断用) */
static registrySize(): number {
return registry.size;
}
async open(name: string): Promise<void> {
this.dbName = name;
if (!registry.has(name)) {
registry.set(name, new Map());
}
this.store = registry.get(name)!;
}
/** close 不清除数据(持久化语义:重开同名库数据仍在) */
async close(): Promise<void> {
this.store = null;
}
isOpen(): boolean {
return this.store !== null;
}
async read(key: string): Promise<ArrayBuffer | null> {
return this.store?.get(key) ?? null;
}
async write(key: string, data: ArrayBuffer): Promise<void> {
this.store?.set(key, data);
}
async append(key: string, data: ArrayBuffer): Promise<void> {
if (!this.store) return;
const existing = this.store.get(key);
if (existing) {
const combined = new Uint8Array(existing.byteLength + data.byteLength);
combined.set(new Uint8Array(existing), 0);
combined.set(new Uint8Array(data), existing.byteLength);
this.store.set(key, combined.buffer as ArrayBuffer);
} else {
this.store.set(key, data);
}
}
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
if (!this.store) return;
// 同步批量写入 = 原子(JS 单线程,无中间 await 点)
for (const [key, data] of Object.entries(entries)) {
this.store.set(key, data);
}
}
async delete(key: string): Promise<void> {
this.store?.delete(key);
}
async deleteMany(keys: string[]): Promise<void> {
if (!this.store) return;
for (const key of keys) {
this.store.delete(key);
}
}
async listKeys(): Promise<string[]> {
return this.store ? Array.from(this.store.keys()) : [];
}
async exists(key: string): Promise<boolean> {
return this.store?.has(key) ?? false;
}
async clear(): Promise<void> {
this.store?.clear();
}
}
+92
View File
@@ -0,0 +1,92 @@
/**
* KVStore Snapshot /
* @module engine/kvstore/snapshot
*
* v0.6.0: checkpoint key-value COW
* "日志水位 seq" seq >
*
* :
* [magic u32] 0x4B56534E ("KVSN")
* [seq u32]
* [entryCount u32]
* : [keyLen u32][key bytes][valueLen u32][value bytes]
* [crc u32] CRC CRC-32
*/
import { crc32 } from '../aria/crc32';
const SNAPSHOT_MAGIC = 0x4b56534e; // "KVSN"
/** 快照内容 */
export interface KVSsnapshot {
/** 日志水位 */
seq: number;
/** key → value */
entries: Map<string, ArrayBuffer>;
}
/** 序列化快照 */
export function encodeSnapshot(seq: number, entries: Map<string, ArrayBuffer>): Uint8Array {
const encoder = new TextEncoder();
const keys = Array.from(entries.keys());
// 预编码
const encoded: { key: Uint8Array; value: Uint8Array }[] = [];
let total = 4 + 4 + 4; // magic + seq + entryCount
for (const key of keys) {
const kb = encoder.encode(key);
const vb = new Uint8Array(entries.get(key)!);
encoded.push({ key: kb, value: vb });
total += 4 + kb.byteLength + 4 + vb.byteLength;
}
total += 4; // crc
const buf = new Uint8Array(total);
const view = new DataView(buf.buffer);
let offset = 0;
view.setUint32(offset, SNAPSHOT_MAGIC, false); offset += 4;
view.setUint32(offset, seq, false); offset += 4;
view.setUint32(offset, encoded.length, false); offset += 4;
for (const e of encoded) {
view.setUint32(offset, e.key.byteLength, false); offset += 4;
buf.set(e.key, offset); offset += e.key.byteLength;
view.setUint32(offset, e.value.byteLength, false); offset += 4;
buf.set(e.value, offset); offset += e.value.byteLength;
}
const crc = crc32(buf.subarray(0, total - 4));
view.setUint32(total - 4, crc, false);
return buf;
}
/**
*
* @returns magic /CRC / null
*/
export function decodeSnapshot(data: Uint8Array): KVSsnapshot | null {
if (data.byteLength < 16) return null;
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
if (view.getUint32(0, false) !== SNAPSHOT_MAGIC) return null;
const storedCrc = view.getUint32(data.byteLength - 4, false);
const computedCrc = crc32(data.subarray(0, data.byteLength - 4));
if (storedCrc !== computedCrc) return null;
const decoder = new TextDecoder();
const entries = new Map<string, ArrayBuffer>();
let p = 4;
const seq = view.getUint32(p, false); p += 4;
const entryCount = view.getUint32(p, false); p += 4;
for (let i = 0; i < entryCount; i++) {
if (p + 4 > data.byteLength - 4) return null;
const keyLen = view.getUint32(p, false); p += 4;
if (p + keyLen + 4 > data.byteLength - 4) return null;
const key = decoder.decode(data.subarray(p, p + keyLen)); p += keyLen;
const valueLen = view.getUint32(p, false); p += 4;
if (p + valueLen > data.byteLength - 4) return null;
const value = data.slice(p, p + valueLen).buffer as ArrayBuffer; p += valueLen;
entries.set(key, value);
}
return { seq, entries };
}
+492
View File
@@ -0,0 +1,492 @@
/**
* KVStoreEngine KVStore IndexedDBEngine / OPFSEngine
* @module engine/kvstore_engine
*
* v0.6.0: 完全移除 IndexedDB disk
*
* MemoryEngine + + KVStore +
* - KVStore
* - + KVStore insert putManyupdate/delete
* / diff
* - + commit flushputMany =
* IndexedDBEngine IDB
*
* KVStore keys:
* `__schema` JSON { tableName: TableSchema }
* `__meta:{key}`
* `t:{table}:{pk}` JSON
*/
import type { IStorageEngine } from './interface';
import type { QueryPlan, TableSchema } from '../constants';
import { DatabaseError } from '../constants';
import { MemoryEngine } from './memory';
import { KVStore } from './kvstore/index';
import { SharedMemoryBackend } from './kvstore/shared_memory_medium';
import type { IStorageBackend } from './aria/store/backend';
const SCHEMA_KEY = '__schema';
const ROW_PREFIX = 't:';
const enc = (s: string) => new TextEncoder().encode(s).buffer as ArrayBuffer;
const dec = (b: ArrayBuffer) => new TextDecoder().decode(b);
export class KVStoreEngine implements IStorageEngine {
readonly name = 'kv';
private kv!: KVStore;
private memory: MemoryEngine = new MemoryEngine();
private dbName = '';
private version = 1;
private opened = false;
/** 活跃事务标记 */
private txActive = false;
/** 事务中写过的表(commit 时只 flush 这些表) */
private txDirtyTables: Set<string> = new Set();
constructor(medium?: IStorageBackend, checkpointThreshold?: number) {
this.kv = new KVStore(medium, checkpointThreshold);
}
// ---- 行 key 编解码 ----
private rowKey(table: string, pk: string): string {
return `${ROW_PREFIX}${table}:${pk}`;
}
private rowPrefix(table: string): string {
return `${ROW_PREFIX}${table}:`;
}
// ---- 生命周期 ----
async open(dbName: string, version: number): Promise<void> {
if (this.opened) return;
this.dbName = dbName;
this.version = version;
await this.kv.open(dbName);
await this.memory.open(dbName, version);
// 恢复 schema
const schemaRaw = await this.kv.get(SCHEMA_KEY);
if (schemaRaw) {
try {
const schemas = JSON.parse(dec(schemaRaw)) as Record<string, TableSchema>;
for (const schema of Object.values(schemas)) {
await this.memory.createTable(schema);
}
} catch {
throw new DatabaseError('Corrupted schema in KVStore', 'KV_SCHEMA_ERROR');
}
}
// 恢复行数据 + 重建索引
const all = await this.kv.getAll();
for (const [key, value] of all) {
if (!key.startsWith(ROW_PREFIX)) continue;
const sep = key.indexOf(':', ROW_PREFIX.length);
if (sep < 0) continue;
const table = key.slice(ROW_PREFIX.length, sep);
if (!(await this.memory.hasTable(table))) continue;
try {
const row = JSON.parse(dec(value));
await this.memory.insert(table, [row]);
} catch {
// 单行损坏跳过(repair 可清理)
}
}
// 重建二级索引(schema 标记的索引列)
const tables = await this.memory.getTableNames();
for (const table of tables) {
const schema = await this.memory.getTableSchema(table);
if (!schema) continue;
for (const [col, colDef] of Object.entries(schema.columns)) {
if (colDef.index || colDef.unique) {
await this.memory.createIndex(table, col, colDef.unique);
}
}
}
this.opened = true;
}
async close(): Promise<void> {
if (!this.opened) return;
// 活跃事务先回滚
if (this.txActive) {
try { await this.rollbackTransaction(); } catch { /* ignore */ }
}
await this.kv.close();
await this.memory.close();
this.opened = false;
}
isOpen(): boolean {
return this.opened;
}
/**
* v0.6.0: KVStore
* Hybrid reloadMemoryFromDisk "读穿透"
* KVStoreEngine reload
*/
async reload(): Promise<void> {
if (!this.opened) return;
// 1. KVStore 重新从介质加载(外部写入可见)
await this.kv.reload();
// 2. 内存缓存重载
await this.memory.close();
await this.memory.open(this.dbName, this.version);
this.opened = false;
await this.open(this.dbName, this.version);
}
/** v0.4.2-fix: 自愈 — 校验 KVStore 日志/快照完整性并重建内存 */
async repair(): Promise<void> {
this.ensureOpen();
await this.kv.repair();
await this.memory.close();
await this.memory.open(this.dbName, this.version);
// 重新恢复(复用 open 的恢复逻辑)
this.opened = false;
await this.open(this.dbName, this.version);
}
async clearAll(): Promise<void> {
this.ensureOpen();
await this.kv.clear();
await this.memory.clearAll();
}
async getMeta(key: string): Promise<string | null> {
const raw = await this.kv.get(`__meta:${key}`);
return raw ? dec(raw) : null;
}
async setMeta(key: string, value: string): Promise<void> {
await this.kv.put(`__meta:${key}`, enc(value));
}
// ---- 表管理 ----
async createTable(schema: TableSchema): Promise<void> {
this.ensureOpen();
await this.memory.createTable(schema);
if (this.txActive) {
this.txDirtyTables.add(schema.name);
return;
}
await this.persistSchema();
}
async dropTable(tableName: string): Promise<void> {
this.ensureOpen();
await this.memory.dropTable(tableName);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return;
}
await this.persistSchema();
// 删除该表全部行(KV 中残留清理)
await this.flushTable(tableName);
}
async hasTable(tableName: string): Promise<boolean> {
return this.memory.hasTable(tableName);
}
async getTableNames(): Promise<string[]> {
return this.memory.getTableNames();
}
async getTableSchema(tableName: string): Promise<TableSchema | null> {
return this.memory.getTableSchema(tableName);
}
async alterTable(
tableName: string,
action: 'ADD' | 'DROP',
column: import('../constants').ColumnDef & { name: string },
): Promise<void> {
this.ensureOpen();
await this.memory.alterTable(tableName, action, column);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return;
}
await this.persistSchema();
if (action === 'DROP') {
// 重写存储行(移除该列)
await this.flushTable(tableName);
}
}
// ---- CRUD ----
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
this.ensureOpen();
const pks = await this.memory.insert(tableName, rows);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return pks;
}
// 增量持久化(原子 putMany
const schema = await this.memory.getTableSchema(tableName);
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema);
const puts: Record<string, ArrayBuffer> = {};
rows.forEach((row, i) => {
puts[this.rowKey(tableName, String(pks[i] ?? row[pkCol]))] = enc(JSON.stringify(row));
});
await this.kv.putMany(puts);
return pks;
}
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
this.ensureOpen();
return this.memory.find(tableName, query);
}
async findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number> {
this.ensureOpen();
return this.memory.findStream(tableName, query, onRow);
}
async update(
tableName: string,
query: QueryPlan,
updates: Record<string, unknown>,
): Promise<number> {
this.ensureOpen();
const schema = await this.memory.getTableSchema(tableName);
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema);
const pkChanged = pkCol in updates;
// 收集受影响旧主键(内存匹配)
const affected = pkChanged ? [] : await this.collectMatchingPks(tableName, query);
const count = await this.memory.update(tableName, query, updates);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return count;
}
if (pkChanged) {
// 主键变更:相关表整表 diff(罕见操作,可靠性优先)
for (const t of await this.affectedTables(tableName)) {
await this.flushTable(t);
}
} else {
// 增量重写受影响行
const puts: Record<string, ArrayBuffer> = {};
const deletes: string[] = [];
for (const pk of affected) {
const row = await this.memory.find(tableName, { table: tableName, where: { [pkCol]: pk } });
if (row.length > 0) {
puts[this.rowKey(tableName, pk)] = enc(JSON.stringify(row[0]));
} else {
deletes.push(this.rowKey(tableName, pk));
}
}
if (Object.keys(puts).length > 0) await this.kv.putMany(puts);
if (deletes.length > 0) await this.kv.deleteMany(deletes);
// 级联影响表(SET NULL/CASCADE 外键)整表 diff
for (const t of await this.affectedTables(tableName)) {
if (t !== tableName) await this.flushTable(t);
}
}
return count;
}
async delete(tableName: string, query: QueryPlan): Promise<number> {
this.ensureOpen();
// 收集受影响主键(内存匹配)
const pks = await this.collectMatchingPks(tableName, query);
const count = await this.memory.delete(tableName, query);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return count;
}
const deletes = pks.map((pk) => this.rowKey(tableName, pk));
if (deletes.length > 0) await this.kv.deleteMany(deletes);
// 级联影响表整表 diff
for (const t of await this.affectedTables(tableName)) {
if (t !== tableName) await this.flushTable(t);
}
return count;
}
async count(tableName: string, query?: QueryPlan): Promise<number> {
this.ensureOpen();
return this.memory.count(tableName, query);
}
async clear(tableName: string): Promise<void> {
this.ensureOpen();
await this.memory.clear(tableName);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return;
}
await this.flushTable(tableName);
}
// ---- 动态索引 ----
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
this.ensureOpen();
await this.memory.createIndex(tableName, column, unique);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return;
}
await this.persistSchema();
}
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
this.ensureOpen();
await this.memory.dropIndex(tableName, column, indexName);
if (this.txActive) {
this.txDirtyTables.add(tableName);
return;
}
await this.persistSchema();
}
// ---- 事务(原子 flush ----
async beginTransaction(): Promise<void> {
this.ensureOpen();
await this.memory.beginTransaction();
this.txActive = true;
this.txDirtyTables = new Set();
}
async commitTransaction(): Promise<void> {
this.ensureOpen();
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
// 先持久化(原子),再提交内存快照(失败可回滚)
for (const table of this.txDirtyTables) {
if (await this.memory.hasTable(table)) {
await this.flushTable(table);
} else {
// 事务内 drop 的表:清理 KV 残留行
const all = await this.kv.getAll();
const prefix = this.rowPrefix(table);
const deletes = all.filter(([key]) => key.startsWith(prefix)).map(([key]) => key);
if (deletes.length > 0) await this.kv.deleteMany(deletes);
}
}
// v0.6.0-fix: 事务内 DDLcreate/drop/alter)的 schema 一并持久化
await this.persistSchema();
await this.kv.checkpoint();
await this.memory.commitTransaction();
this.txActive = false;
this.txDirtyTables = new Set();
}
async rollbackTransaction(): Promise<void> {
this.ensureOpen();
if (!this.txActive) throw new DatabaseError('No active transaction', 'TX_NONE');
await this.memory.rollbackTransaction();
this.txActive = false;
this.txDirtyTables = new Set();
}
// ---- 内部 ----
private ensureOpen(): void {
if (!this.opened) throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
}
private getPK(schema: TableSchema): string {
for (const [name, col] of Object.entries(schema.columns)) {
if (col.primaryKey) return name;
}
return Object.keys(schema.columns)[0];
}
/** 收集匹配查询的内存行主键(持久化差异计算用) */
private async collectMatchingPks(tableName: string, query: QueryPlan): Promise<string[]> {
const schema = await this.memory.getTableSchema(tableName);
if (!schema) throw new DatabaseError(`Table "${tableName}" does not exist`, 'TABLE_NOT_FOUND');
const pkCol = this.getPK(schema);
const rows = await this.memory.find(tableName, query);
return rows.map((r) => String(r[pkCol]));
}
/**
* A B B C {A, B, C}
* delete/update
*/
private async affectedTables(tableName: string): Promise<Set<string>> {
const set = new Set<string>([tableName]);
let changed = true;
while (changed) {
changed = false;
for (const table of await this.memory.getTableNames()) {
if (set.has(table)) continue;
const schema = await this.memory.getTableSchema(table);
if (!schema) continue;
for (const col of Object.values(schema.columns)) {
if (col.references) {
const ref = col.references.split('.')[0];
if (set.has(ref)) {
set.add(table);
changed = true;
break;
}
}
}
}
}
return set;
}
/** 持久化 schema(全部表) */
private async persistSchema(): Promise<void> {
const schemas: Record<string, TableSchema> = {};
for (const table of await this.memory.getTableNames()) {
const schema = await this.memory.getTableSchema(table);
if (schema) schemas[table] = schema;
}
await this.kv.put(SCHEMA_KEY, enc(JSON.stringify(schemas)));
}
/**
* diff put + KV putMany + deleteMany
* / / dropTable / clear / alterTable DROP / commit
*/
private async flushTable(tableName: string): Promise<void> {
const prefix = this.rowPrefix(tableName);
// 表已删除:仅清理 KV 残留行
const schema = await this.memory.getTableSchema(tableName);
if (!schema) {
const all = await this.kv.getAll();
const deletes = all.filter(([key]) => key.startsWith(prefix)).map(([key]) => key);
if (deletes.length > 0) await this.kv.deleteMany(deletes);
return;
}
const pkCol = this.getPK(schema);
const rows = await this.memory.find(tableName, { table: tableName });
const puts: Record<string, ArrayBuffer> = {};
const current = new Set<string>();
for (const row of rows) {
const key = this.rowKey(tableName, String(row[pkCol]));
current.add(key);
puts[key] = enc(JSON.stringify(row));
}
// KV 残留行(内存中已不存在)删除
const all = await this.kv.getAll();
const deletes: string[] = [];
for (const [key] of all) {
if (key.startsWith(prefix) && !current.has(key)) {
deletes.push(key);
}
}
if (Object.keys(puts).length > 0) await this.kv.putMany(puts);
if (deletes.length > 0) await this.kv.deleteMany(deletes);
}
}
-363
View File
@@ -1,363 +0,0 @@
/**
* metona-sqlark OPFS Engine Origin Private File System
* @module engine/opfs
*
* 使 JSON-per-table
* {dbName}/tables/{tableName}.json
*/
import type { IStorageEngine } from './interface';
import type { QueryPlan, TableSchema } from '../constants';
import { DatabaseError } from '../constants';
import { MemoryEngine } from './memory';
// ---------------------------------------------------------------------------
// OPFSEngine
// ---------------------------------------------------------------------------
export class OPFSEngine implements IStorageEngine {
readonly name = 'opfs';
private root: FileSystemDirectoryHandle | null = null;
private tablesDir: FileSystemDirectoryHandle | null = null;
private dbName = '';
// 运行时内存缓存(OPFS 文件读写有延迟)
private memoryCache: MemoryEngine = new MemoryEngine();
/**
* v0.4.3-fix: 写操作串行队列 + +
* close() close /
*
*/
private opQueue: Promise<unknown> = Promise.resolve();
/** 将写操作加入串行队列(快照在队列内取,始终最新) */
private enqueueOp<T>(fn: () => Promise<T>): Promise<T> {
const run = this.opQueue.then(fn, fn);
this.opQueue = run.then(() => undefined, () => undefined);
return run;
}
// ---- 生命周期 ----
async open(dbName: string, version: number): Promise<void> {
this.dbName = dbName;
await this.memoryCache.open(dbName, version);
// 获取 OPFS 根目录
this.root = await navigator.storage.getDirectory();
// 创建/打开数据库目录
this.tablesDir = await this.root.getDirectoryHandle(dbName, { create: true });
// 从 OPFS 恢复已有表数据到内存缓存
await this.loadExistingTables();
}
async close(): Promise<void> {
// v0.4.3-fix: 等待所有挂起写操作完成(否则 close 后写仍在进行 → 重启读旧数据)
try {
await this.opQueue;
} catch { /* 写失败已返回给调用方 */ }
this.root = null;
this.tablesDir = null;
await this.memoryCache.close();
}
isOpen(): boolean {
return this.tablesDir !== null;
}
// ---- v0.4.2-fix: 自愈 / 重置 / 元数据 ----
/** 自愈:重置内存缓存后从 OPFS 重新加载(单文件损坏不影响其他表) */
async repair(): Promise<void> {
// v0.4.3-fix: 先等写队列排空(避免与挂起写竞态)
try { await this.opQueue; } catch { /* ignore */ }
await this.memoryCache.close();
await this.memoryCache.open(this.dbName, 1);
await this.loadExistingTables();
}
/** 清空全部数据与表结构(删除目录内全部文件) */
async clearAll(): Promise<void> {
// v0.4.3-fix: 先等写队列排空
try { await this.opQueue; } catch { /* ignore */ }
await this.memoryCache.close();
await this.memoryCache.open(this.dbName, 1);
if (this.tablesDir) {
const dir = this.tablesDir as any;
for await (const [name] of dir.entries()) {
try { await this.tablesDir!.removeEntry(name); } catch { /* ignore */ }
}
}
}
async getMeta(key: string): Promise<string | null> {
if (!this.tablesDir) return null;
try {
const fh = await this.tablesDir.getFileHandle(`__metona_${key}.meta`);
const file = await fh.getFile();
return await file.text();
} catch {
return null;
}
}
async setMeta(key: string, value: string): Promise<void> {
if (!this.tablesDir) return;
const fh = await this.tablesDir.getFileHandle(`__metona_${key}.meta`, { create: true });
const writable = await fh.createWritable();
await writable.write(value);
await writable.close();
}
// ---- 表管理 ----
async createTable(schema: TableSchema): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.createTable(schema);
// v0.4.2-fix: schema 持久化(此前仅写空数据文件 → 空表重启后消失、索引标记丢失)
await this.setMeta(`schema_${schema.name}`, JSON.stringify(schema));
// OPFS 中表以空 JSON 数组文件形式存在
await this.writeTableData(schema.name, []);
});
}
async dropTable(tableName: string): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.dropTable(tableName);
// v0.4.2-fix: 清理 schema meta(否则重启恢复幽灵表)
if (this.tablesDir) {
try {
await this.tablesDir.removeEntry(`__metona_schema_${tableName}.meta`);
} catch {
// 文件不存在则忽略
}
try {
await this.tablesDir.removeEntry(`${tableName}.json`);
} catch {
// 文件不存在则忽略
}
}
});
}
async hasTable(tableName: string): Promise<boolean> {
if (!this.tablesDir) return false;
try {
await this.tablesDir.getFileHandle(`${tableName}.json`);
return true;
} catch {
return false;
}
}
async getTableNames(): Promise<string[]> {
if (!this.tablesDir) return [];
const names: string[] = [];
for await (const [name] of (this.tablesDir as any).entries()) {
if (name.endsWith('.json')) {
names.push(name.replace('.json', ''));
}
}
return names;
}
async getTableSchema(tableName: string): Promise<TableSchema | null> {
return this.memoryCache.getTableSchema(tableName);
}
/** v0.4.2-fix: 引擎级 ALTER TABLE — 内存 + schema 持久化 + 整表文件重写 */
async alterTable(
tableName: string,
action: 'ADD' | 'DROP',
column: import('../constants').ColumnDef & { name: string },
): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.alterTable(tableName, action, column);
const schema = await this.memoryCache.getTableSchema(tableName);
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
const rows = await this.memoryCache.find(tableName, { table: tableName });
await this.writeTableData(tableName, rows);
});
}
// ---- CRUD ----
async insert(tableName: string, rows: Record<string, unknown>[]): Promise<string[]> {
return this.enqueueOp(async () => {
const pks = await this.memoryCache.insert(tableName, rows);
// 持久化到 OPFS(快照在队列内取,始终最新)
const allRows = await this.memoryCache.find(tableName, { table: tableName });
await this.writeTableData(tableName, allRows);
return pks;
});
}
async find(tableName: string, query: QueryPlan): Promise<Record<string, unknown>[]> {
return this.memoryCache.find(tableName, query);
}
/** v0.4.0: 流式查询(委托内存缓存) */
async findStream(tableName: string, query: QueryPlan, onRow: (row: Record<string, unknown>) => void): Promise<number> {
return this.memoryCache.findStream(tableName, query, onRow);
}
async update(tableName: string, query: QueryPlan, updates: Record<string, unknown>): Promise<number> {
return this.enqueueOp(async () => {
const count = await this.memoryCache.update(tableName, query, updates);
const allRows = await this.memoryCache.find(tableName, { table: tableName });
await this.writeTableData(tableName, allRows);
return count;
});
}
async delete(tableName: string, query: QueryPlan): Promise<number> {
return this.enqueueOp(async () => {
const count = await this.memoryCache.delete(tableName, query);
const allRows = await this.memoryCache.find(tableName, { table: tableName });
await this.writeTableData(tableName, allRows);
return count;
});
}
async count(tableName: string, query?: QueryPlan): Promise<number> {
return this.memoryCache.count(tableName, query);
}
async clear(tableName: string): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.clear(tableName);
await this.writeTableData(tableName, []);
});
}
// ---- 动态索引(v0.3.0 ----
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.createIndex(tableName, column, unique);
// v0.4.2-fix: 索引标记持久化(重启后索引结构恢复)
const schema = await this.memoryCache.getTableSchema(tableName);
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
});
}
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.dropIndex(tableName, column, indexName);
const schema = await this.memoryCache.getTableSchema(tableName);
if (schema) await this.setMeta(`schema_${tableName}`, JSON.stringify(schema));
});
}
// ---- 事务 ----
async beginTransaction(): Promise<void> {
await this.memoryCache.beginTransaction();
}
async commitTransaction(): Promise<void> {
return this.enqueueOp(async () => {
await this.memoryCache.commitTransaction();
// 将内存数据刷到 OPFS
const tableNames = await this.memoryCache.getTableNames();
for (const tableName of tableNames) {
const rows = await this.memoryCache.find(tableName, { table: tableName });
await this.writeTableData(tableName, rows);
}
});
}
async rollbackTransaction(): Promise<void> {
await this.memoryCache.rollbackTransaction();
}
// ---- 内部辅助 ----
private ensureDir(): FileSystemDirectoryHandle {
if (!this.tablesDir) {
throw new DatabaseError('Database not opened', 'DB_NOT_OPEN');
}
return this.tablesDir;
}
private async writeTableData(tableName: string, data: Record<string, unknown>[]): Promise<void> {
// 由 enqueueOp 串行化调用,此处直接写文件
const dir = this.ensureDir();
const fileName = `${tableName}.json`;
const fileHandle = await dir.getFileHandle(fileName, { create: true });
const writable = await fileHandle.createWritable();
await writable.write(JSON.stringify(data));
await writable.close();
}
private async readTableData(tableName: string): Promise<Record<string, unknown>[]> {
const dir = this.ensureDir();
const fileName = `${tableName}.json`;
try {
const fileHandle = await dir.getFileHandle(fileName);
const file = await fileHandle.getFile();
const text = await file.text();
return JSON.parse(text);
} catch {
return [];
}
}
/**
* OPFS
* v0.4.2-fix: 优先从持久化 schema__metona_schema_*.meta
* // schema
*/
private async loadExistingTables(): Promise<void> {
if (!this.tablesDir) return;
const dir = this.tablesDir as any;
const fileNames: string[] = [];
for await (const [name] of dir.entries()) {
if (name.endsWith('.json')) fileNames.push(name);
}
for (const fileName of fileNames) {
const tableName = fileName.replace('.json', '');
try {
// 1. 优先:持久化 schema
const schemaRaw = await this.getMeta(`schema_${tableName}`);
if (schemaRaw) {
const schema = JSON.parse(schemaRaw) as TableSchema;
await this.memoryCache.createTable(schema);
const rows = await this.readTableData(tableName);
for (const row of rows) {
try {
await this.memoryCache.insert(tableName, [row]);
} catch {
// 单行损坏不影响整表恢复
}
}
continue;
}
// 2. 兼容旧库:从数据推断 schema(空表且无 schema 记录 → 跳过)
const data = await this.readTableData(tableName);
if (data.length > 0) {
const firstRow = data[0];
const columns: Record<string, any> = {};
for (const key of Object.keys(firstRow)) {
const val = firstRow[key];
const type = typeof val === 'number' ? 'number' :
typeof val === 'boolean' ? 'boolean' :
typeof val === 'object' ? 'json' : 'string';
columns[key] = { type, primaryKey: key === 'id' };
}
await this.memoryCache.createTable({ name: tableName, columns });
for (const row of data) {
await this.memoryCache.insert(tableName, [row]);
}
}
} catch {
// 单个文件损坏不影响其他表
}
}
}
}
+10 -4
View File
@@ -12,8 +12,7 @@ import type { IStorageEngine } from '../engine/interface';
import type { QueryPlan, TableSchema, DiskEngine } from '../constants'; import type { QueryPlan, TableSchema, DiskEngine } from '../constants';
import { DatabaseError } from '../constants'; import { DatabaseError } from '../constants';
import { MemoryEngine } from '../engine/memory'; import { MemoryEngine } from '../engine/memory';
import { IndexedDBEngine } from '../engine/indexeddb'; import { KVStoreEngine } from '../engine/kvstore_engine';
import { OPFSEngine } from '../engine/opfs';
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// HybridEngine // HybridEngine
@@ -28,10 +27,11 @@ export class HybridEngine implements IStorageEngine {
private dbName = ''; private dbName = '';
private version = 1; private version = 1;
constructor(diskEngine: DiskEngine = 'indexeddb') { constructor(diskEngine: DiskEngine = 'opfs') {
this.memoryEngine = new MemoryEngine(); this.memoryEngine = new MemoryEngine();
this.diskEngineType = diskEngine; this.diskEngineType = diskEngine;
this.diskEngine = diskEngine === 'opfs' ? new OPFSEngine() : new IndexedDBEngine(); // v0.6.0: 磁盘层统一为自研 KVStoreEngine
this.diskEngine = new KVStoreEngine();
} }
// ---- 生命周期 ---- // ---- 生命周期 ----
@@ -57,6 +57,12 @@ export class HybridEngine implements IStorageEngine {
await this.memoryEngine.close(); await this.memoryEngine.close();
await this.memoryEngine.open(this.dbName, this.version); await this.memoryEngine.open(this.dbName, this.version);
// v0.6.0: KVStoreEngine 读内存 → 先重载磁盘最新数据
const disk = this.diskEngine as IStorageEngine & { reload?: () => Promise<void> };
if (typeof disk.reload === 'function') {
await disk.reload();
}
const tableNames = await this.diskEngine.getTableNames(); const tableNames = await this.diskEngine.getTableNames();
for (const tableName of tableNames) { for (const tableName of tableNames) {
const schema = await this.diskEngine.getTableSchema(tableName); const schema = await this.diskEngine.getTableSchema(tableName);
+1 -2
View File
@@ -78,8 +78,7 @@ export type { DatabaseConfig, TableSchema, ColumnDef, FieldType, StorageMode, Di
export type { IStorageEngine } from './engine/interface'; export type { IStorageEngine } from './engine/interface';
export type { Statement, SelectStatement, InsertStatement, UpdateStatement, DeleteStatement } from './query/ast'; export type { Statement, SelectStatement, InsertStatement, UpdateStatement, DeleteStatement } from './query/ast';
export { MemoryEngine } from './engine/memory'; export { MemoryEngine } from './engine/memory';
export { IndexedDBEngine } from './engine/indexeddb'; export { KVStoreEngine } from './engine/kvstore_engine';
export { OPFSEngine } from './engine/opfs';
export { AriaEngine } from './engine/aria/index'; export { AriaEngine } from './engine/aria/index';
export { HybridEngine } from './hybrid/index'; export { HybridEngine } from './hybrid/index';
export { Table } from './table/table'; export { Table } from './table/table';
+170
View File
@@ -0,0 +1,170 @@
/**
* migrateFromIndexedDB IndexedDB KV
* @module migration/index
*
* v0.6.0: IndexedDB
* KVStoreEngine disk / AriaEngine
*
*
* - disk IndexedDBEngine = dbName
* - aria IndexedDBBackend = `aria-${dbName}`
*
* IndexedDB
*/
import type { MetonaSqlark } from '../core';
import type { TableSchema, ColumnDef, FieldType } from '../constants';
export interface MigrationOptions {
/** 旧库名(业务名,不含 aria- 前缀) */
dbName: string;
/**
* diskIndexedDBEngine objectStore
* aria IndexedDBBackendSSTable/WAL
*/
engine: 'disk';
/** 目标数据库实例(已初始化,新引擎) */
target: MetonaSqlark;
/** 进度回调 */
onProgress?: (done: number, total: number, table?: string) => void;
}
export interface MigrationResult {
/** 已迁移的表 */
migratedTables: string[];
/** 迁移的行总数 */
rowCount: number;
/** 跳过(无 schema 且无数据)的表 */
skippedTables: string[];
}
/** 旧库中持久化 schema 的 store 名(IndexedDBEngine v0.3.2+ */
const SCHEMA_STORE = '__metona_schema';
function inferFieldType(value: unknown): FieldType {
if (typeof value === 'number') return 'number';
if (typeof value === 'boolean') return 'boolean';
if (typeof value === 'object' && value !== null) return 'json';
return 'string';
}
/** 从样例行推断 schema(旧库无持久化 schema 时回退) */
function inferSchema(tableName: string, rows: Record<string, unknown>[]): TableSchema {
const columns: Record<string, ColumnDef> = {};
if (rows.length === 0) return { name: tableName, columns };
const first = rows[0];
for (const key of Object.keys(first)) {
columns[key] = {
type: inferFieldType(first[key]),
primaryKey: key === 'id',
};
}
return { name: tableName, columns };
}
/** 打开旧 IndexedDB 库(只读) */
function openLegacyDB(idbName: string): Promise<IDBDatabase> {
return new Promise((resolve, reject) => {
const request = indexedDB.open(idbName);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error ?? new Error(`Failed to open legacy IndexedDB "${idbName}"`));
});
}
/** 读取 object store 全部记录 */
function readAllRecords(store: IDBObjectStore): Promise<Record<string, unknown>[]> {
return new Promise((resolve, reject) => {
const req = store.getAll();
req.onsuccess = () => resolve((req.result ?? []) as Record<string, unknown>[]);
req.onerror = () => reject(req.error);
});
}
/** 读取持久化 schema 记录 */
function readSchemas(db: IDBDatabase): Promise<Record<string, TableSchema>> {
if (!db.objectStoreNames.contains(SCHEMA_STORE)) {
return Promise.resolve({});
}
return new Promise((resolve, reject) => {
const req = db.transaction(SCHEMA_STORE, 'readonly').objectStore(SCHEMA_STORE).getAll();
req.onsuccess = () => {
const result: Record<string, TableSchema> = {};
for (const rec of (req.result ?? []) as { name: string; schema?: string }[]) {
if (!rec.schema) continue;
try {
const schema = JSON.parse(rec.schema) as TableSchema;
result[schema.name] = schema;
} catch { /* 损坏记录跳过 */ }
}
resolve(result);
};
req.onerror = () => reject(req.error);
});
}
/**
* IndexedDB
* @returns /
*/
export async function migrateFromIndexedDB(opts: MigrationOptions): Promise<MigrationResult> {
// v0.6.0: aria 旧库为引擎私有格式(SSTable/WAL),不支持按行迁移(运行时防御)
if ((opts.engine as string) === 'aria') {
throw new Error(
'Migration from AriaEngine IndexedDB backend is not supported ' +
'(data is stored in engine-private SSTable/WAL format). ' +
'Only disk-mode IndexedDBEngine databases can be migrated.',
);
}
const idbName = opts.dbName;
let db: IDBDatabase | null = null;
try {
db = await openLegacyDB(idbName);
} catch (error) {
throw new Error(
`Legacy IndexedDB database "${idbName}" not found or unreadable: ${(error as Error).message}`,
);
}
const result: MigrationResult = { migratedTables: [], rowCount: 0, skippedTables: [] };
const schemas = await readSchemas(db);
try {
const storeNames = Array.from(db.objectStoreNames).filter((n) => n !== SCHEMA_STORE);
for (let i = 0; i < storeNames.length; i++) {
const tableName = storeNames[i];
opts.onProgress?.(i, storeNames.length, tableName);
const rows = await readAllRecords(db.transaction(tableName, 'readonly').objectStore(tableName));
// 表已存在于目标库 → 跳过(避免覆盖)
const names = await opts.target.getTableNames();
if (names.includes(tableName)) {
result.skippedTables.push(tableName);
continue;
}
// schema:持久化优先,否则从数据推断(空表且无 schema → 跳过)
let schema = schemas[tableName];
if (!schema) {
if (rows.length === 0) {
result.skippedTables.push(tableName);
continue;
}
schema = inferSchema(tableName, rows);
}
// 写入目标引擎
await opts.target.defineTable(tableName, schema.columns);
if (rows.length > 0) {
await opts.target.table(tableName).insertMany(rows);
}
result.migratedTables.push(tableName);
result.rowCount += rows.length;
}
} finally {
db.close();
}
return result;
}
+11 -8
View File
@@ -2,11 +2,14 @@
* v0.4.1 AriaEngine + clearAll * v0.4.1 AriaEngine + clearAll
*/ */
import 'fake-indexeddb/auto';
import { AriaEngine } from '../src/engine/aria/index'; import { AriaEngine } from '../src/engine/aria/index';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
const mkEngine = async (name: string): Promise<AriaEngine> => { const mkEngine = async (name: string): Promise<AriaEngine> => {
const e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(`cascade-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, 1); await e.open(`cascade-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, 1);
return e; return e;
}; };
@@ -126,7 +129,7 @@ describe('[v0.4.1] AriaEngine clearAll', () => {
test('clearAll 后重启(模拟刷新)无残留数据', async () => { test('clearAll 后重启(模拟刷新)无残留数据', async () => {
const name = `clr2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; const name = `clr2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
let e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); let e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
await e.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } }); await e.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
await e.insert('t', [{ id: '1' }]); await e.insert('t', [{ id: '1' }]);
@@ -134,7 +137,7 @@ describe('[v0.4.1] AriaEngine clearAll', () => {
await e.close(); await e.close();
// 重新打开(模拟页面刷新):不应有残留表 // 重新打开(模拟页面刷新):不应有残留表
e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
expect(await e.getTableNames()).toHaveLength(0); expect(await e.getTableNames()).toHaveLength(0);
await e.close(); await e.close();
@@ -159,7 +162,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
test('ALTER 持久化:重启后 schema 与行一致', async () => { test('ALTER 持久化:重启后 schema 与行一致', async () => {
const name = `alt2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; const name = `alt2-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
let e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); let e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } }); await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
await e.insert('users', [{ id: '1', name: 'Alice' }]); await e.insert('users', [{ id: '1', name: 'Alice' }]);
@@ -168,7 +171,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' }); await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' });
await e.close(); await e.close();
e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
const schema = await e.getTableSchema('users'); const schema = await e.getTableSchema('users');
expect(Object.keys(schema!.columns)).not.toContain('phone'); expect(Object.keys(schema!.columns)).not.toContain('phone');
@@ -179,7 +182,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
test('ALTER 模拟崩溃(不 close)重启:schema 与行一致', async () => { test('ALTER 模拟崩溃(不 close)重启:schema 与行一致', async () => {
const name = `alt3-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; const name = `alt3-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
let e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); let e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } }); await e.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
await e.insert('users', [{ id: '1', name: 'Alice' }]); await e.insert('users', [{ id: '1', name: 'Alice' }]);
@@ -188,7 +191,7 @@ describe('[v0.4.1] AriaEngine ALTER TABLE', () => {
await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' }); await e.alterTable('users', 'DROP', { name: 'phone', type: 'string' });
// 不 close,模拟崩溃 // 不 close,模拟崩溃
e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
const schema = await e.getTableSchema('users'); const schema = await e.getTableSchema('users');
expect(Object.keys(schema!.columns)).not.toContain('phone'); expect(Object.keys(schema!.columns)).not.toContain('phone');
+2 -5
View File
@@ -3,7 +3,6 @@
*/ */
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import 'fake-indexeddb/auto';
let diskCounter = 0; let diskCounter = 0;
@@ -215,13 +214,12 @@ describe('MetonaSqlark Disk 模式', () => {
beforeEach(async () => { beforeEach(async () => {
dbName = `test-disk-${++diskCounter}`; dbName = `test-disk-${++diskCounter}`;
db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' }); db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
await db.init(); await db.init();
}); });
afterEach(async () => { afterEach(async () => {
await db.close(); await db.close();
try { indexedDB.deleteDatabase(dbName); } catch {}
}); });
it('创建 disk 模式数据库', () => { it('创建 disk 模式数据库', () => {
@@ -250,13 +248,12 @@ describe('MetonaSqlark Hybrid 模式', () => {
beforeEach(async () => { beforeEach(async () => {
dbName = `test-hybrid-core-${++diskCounter}`; dbName = `test-hybrid-core-${++diskCounter}`;
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
}); });
afterEach(async () => { afterEach(async () => {
await db.close(); await db.close();
try { indexedDB.deleteDatabase(dbName); } catch {}
}); });
it('创建 hybrid 模式数据库', () => { it('创建 hybrid 模式数据库', () => {
+68
View File
@@ -119,6 +119,74 @@ test.describe('OPFS 真实环境', () => {
await page2.close(); await page2.close();
}); });
test('写入进行中崩溃(不等待完成)→ 重开数据不损坏且可恢复已确认写入', async ({ page }) => {
await openPage(page);
await run(page, 'open', {
name: 'e2e-crash-mid', diskEngine: 'opfs', checkpointInterval: 999999999,
});
await run(page, 'createTable', {
table: 't', columns: [{ name: 'id', type: 'string', primaryKey: true }],
});
// 前 20 条确认完成
for (let i = 0; i < 20; i++) {
await run(page, 'insert', { table: 't', rows: [{ id: `c-${i}` }] });
}
// 发起一批写入后立即崩溃(不等 Promise 完成)
const pending = page.evaluate(async () => {
const ms = (window as unknown as { __ms: { insert: (o: unknown) => Promise<unknown> } }).__ms;
const rows = [];
for (let i = 0; i < 30; i++) rows.push({ id: `p-${i}` });
await ms.insert({ table: 't', rows });
});
await new Promise((r) => setTimeout(r, 50));
await crashPage(page);
await pending.catch(() => {});
// 重开:库可打开,已确认 20 条完整;写入中数据要么全部可见要么部分(日志尾部截断)
const page2 = await page.context().newPage();
await openPage(page2);
await run(page2, 'open', {
name: 'e2e-crash-mid', diskEngine: 'opfs', checkpointInterval: 999999999,
});
const res = await run<{ count: number }>(page2, 'count', { table: 't' });
expect(res.count).toBeGreaterThanOrEqual(20);
// 已确认的 20 条必须完整
for (let i = 0; i < 20; i++) {
const found = await run<{ rows: Record<string, unknown>[] }>(page2, 'find', {
table: 't', where: { id: `c-${i}` },
});
expect(found.rows).toHaveLength(1);
}
await run(page2, 'close');
await page2.close();
});
test('checkpoint 前后崩溃 → 快照/日志双路径恢复一致', async ({ page }) => {
await openPage(page);
await run(page, 'open', { name: 'e2e-crash-cp', diskEngine: 'opfs', checkpointInterval: 999999999 });
await run(page, 'createTable', {
table: 't', columns: [{ name: 'id', type: 'string', primaryKey: true }],
});
for (let i = 0; i < 10; i++) {
await run(page, 'insert', { table: 't', rows: [{ id: `a-${i}` }] });
}
// 触发 checkpoint(强制落盘)
await run(page, 'repair'); // repair 内含 flush+checkpoint
for (let i = 0; i < 10; i++) {
await run(page, 'insert', { table: 't', rows: [{ id: `b-${i}` }] });
}
// 崩溃(checkpoint 后数据在快照,后续在日志)
await crashPage(page);
const page2 = await page.context().newPage();
await openPage(page2);
await run(page2, 'open', { name: 'e2e-crash-cp', diskEngine: 'opfs', checkpointInterval: 999999999 });
const res = await run<{ count: number }>(page2, 'count', { table: 't' });
expect(res.count).toBe(20);
await run(page2, 'close');
await page2.close();
});
test('多标签页锁:第二个标签页打开同一库 → ARIA_LOCKED', async ({ page }) => { test('多标签页锁:第二个标签页打开同一库 → ARIA_LOCKED', async ({ page }) => {
await openPage(page); await openPage(page);
await run(page, 'open', { name: 'e2e-lock', diskEngine: 'opfs' }); await run(page, 'open', { name: 'e2e-lock', diskEngine: 'opfs' });
+4
View File
@@ -10,6 +10,10 @@ import { BufferPool } from '../../src/engine/aria/buffer/pool';
import { MemoryBackend } from '../../src/engine/aria/store/backend'; import { MemoryBackend } from '../../src/engine/aria/store/backend';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — Bloom + WAL + BufferPool', () => { describe('AriaEngine — Bloom + WAL + BufferPool', () => {
// ---- BloomFilter 完整测试 ---- // ---- BloomFilter 完整测试 ----
describe('BloomFilter', () => { describe('BloomFilter', () => {
+4
View File
@@ -5,6 +5,10 @@ import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { MemoryBackend } from '../../src/engine/aria/store/backend'; import { MemoryBackend } from '../../src/engine/aria/store/backend';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — 批量扩展测试', () => { describe('AriaEngine — 批量扩展测试', () => {
let engine: AriaEngine; let engine: AriaEngine;
beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('bat', 1); }); beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('bat', 1); });
+4
View File
@@ -10,6 +10,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
/** 构造小缓存 + 小 MemTable 阈值的引擎,快速产生多个 SSTable */ /** 构造小缓存 + 小 MemTable 阈值的引擎,快速产生多个 SSTable */
function createSmallCacheEngine(bufferPoolPages = 2) { function createSmallCacheEngine(bufferPoolPages = 2) {
return new AriaEngine({ return new AriaEngine({
+37 -23
View File
@@ -8,7 +8,10 @@
*/ */
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import 'fake-indexeddb/auto';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -21,7 +24,7 @@ const SCHEMA = () => createSchema('users', {
age: { type: 'number' }, age: { type: 'number' },
}); });
/** 篡改 MemoryBackend 中指定 key 的一个字节 */ /** 篡改指定 pg_ 页面文件的一个字节 */
async function corruptKey(engine: AriaEngine, key: string, byteOffset: number): Promise<void> { async function corruptKey(engine: AriaEngine, key: string, byteOffset: number): Promise<void> {
const backend = (engine as any).backend; const backend = (engine as any).backend;
const raw = await backend.read(key); const raw = await backend.read(key);
@@ -31,16 +34,24 @@ async function corruptKey(engine: AriaEngine, key: string, byteOffset: number):
await backend.write(key, buf.buffer as ArrayBuffer); await backend.write(key, buf.buffer as ArrayBuffer);
} }
/** 列出 backend 中所有 sst_ 前缀 key */ /** 列出全部 SSTable 的页面文件 key(页面化存储:pg_ 前缀) */
async function listSSTKeys(engine: AriaEngine): Promise<string[]> { async function listSSTKeys(engine: AriaEngine): Promise<string[]> {
const backend = (engine as any).backend; const backend = (engine as any).backend;
const keys = await backend.listKeys(); const keys = (await backend.listKeys()) as string[];
return (keys as string[]).filter((k) => k.startsWith('sst_')); return keys.filter((k) => k.startsWith('pg_'));
}
/** 读取主 LSM 的 SSTable meta 列表 */
async function listSSTMetas(engine: AriaEngine): Promise<{ id: number; pageIds?: number[] }[]> {
const backend = (engine as any).backend;
const raw = await backend.read('__aria_lsm_meta');
if (!raw) return [];
return JSON.parse(new TextDecoder().decode(raw)) as { id: number; pageIds?: number[] }[];
} }
describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => { describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
it('打开时 CRC 损坏的 SSTable 被清理,其余数据可读', async () => { it('打开时 CRC 损坏的 SSTable 被清理,其余数据可读', async () => {
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
const dbName = uniqueDB(); const dbName = uniqueDB();
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
@@ -57,7 +68,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
await engine.close(); await engine.close();
// 重新打开,篡改第一个 SSTable 文件的数据区 // 重新打开,篡改第一个 SSTable 文件的数据区
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
const sstKeys = await listSSTKeys(engine2); const sstKeys = await listSSTKeys(engine2);
expect(sstKeys.length).toBeGreaterThanOrEqual(2); expect(sstKeys.length).toBeGreaterThanOrEqual(2);
@@ -65,7 +76,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
// 再次打开:损坏文件应被跳过并清理,打开不抛错 // 再次打开:损坏文件应被跳过并清理,打开不抛错
await engine2.close(); await engine2.close();
const engine3 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine3 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine3.open(dbName, 1); await engine3.open(dbName, 1);
// 剩余未损坏文件的数据应可查询 // 剩余未损坏文件的数据应可查询
@@ -73,18 +84,20 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
expect(remaining.length).toBeGreaterThan(0); expect(remaining.length).toBeGreaterThan(0);
expect(remaining.length).toBeLessThan(200); expect(remaining.length).toBeLessThan(200);
// 损坏文件已被清理(meta 移除) // 损坏文件已被清理(页面 + meta 移除)
const afterKeys = await listSSTKeys(engine3); const afterKeys = await listSSTKeys(engine3);
expect(afterKeys).not.toContain(sstKeys[0]); expect(afterKeys).not.toContain(sstKeys[0]);
const lsm = (engine3 as any).lsm; const metas = await listSSTMetas(engine3);
const metas = await lsm.sstableStore.listMeta(); // 被篡改页面所属的 SSTable meta 应被移除
expect(metas.some((m: { id: number }) => m.id === Number(sstKeys[0].replace('sst_', '')))).toBe(false); const victimMetas = await listSSTMetas(engine2);
const victimMeta = victimMetas.find((m) => m.pageIds?.includes(Number(sstKeys[0].slice(3))));
expect(metas.some((m) => m.id === victimMeta?.id)).toBe(false);
await engine3.close(); await engine3.close();
}); });
it('打开时损坏全部 SSTable → 库仍可打开,数据为空但不崩溃', async () => { it('打开时损坏全部 SSTable → 库仍可打开,数据为空但不崩溃', async () => {
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
const dbName = uniqueDB(); const dbName = uniqueDB();
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
@@ -98,14 +111,14 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
} }
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
for (const key of await listSSTKeys(engine2)) { for (const key of await listSSTKeys(engine2)) {
await corruptKey(engine2, key, 16); await corruptKey(engine2, key, 16);
} }
await engine2.close(); await engine2.close();
const engine3 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine3 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
// 不应抛 ARIA_OPEN_ERROR // 不应抛 ARIA_OPEN_ERROR
await engine3.open(dbName, 1); await engine3.open(dbName, 1);
const rows3 = await engine3.find('users', { table: 'users' }); const rows3 = await engine3.find('users', { table: 'users' });
@@ -114,7 +127,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
}); });
it('运行期 CRC 损坏 → 预加载不缓存损坏文件并清理(自愈)', async () => { it('运行期 CRC 损坏 → 预加载不缓存损坏文件并清理(自愈)', async () => {
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
const dbName = uniqueDB(); const dbName = uniqueDB();
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
@@ -133,22 +146,23 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
const victim = sstKeys[sstKeys.length - 1]; // 篡改最新(memtable 已 flush 后的文件) const victim = sstKeys[sstKeys.length - 1]; // 篡改最新(memtable 已 flush 后的文件)
await corruptKey(engine, victim, 100); await corruptKey(engine, victim, 100);
// 缓存里已有该文件(flush 时缓存)→ 清缓存模拟运行期重载 // 缓存里已有该文件(flush 时缓存)→ 清 LSM 层与 BufferPool 页面缓存模拟运行期磁盘损坏
const lsm = (engine as any).lsm; const lsm = (engine as any).lsm;
lsm.sstableCache.delete(Number(victim.replace('sst_', ''))); lsm.sstableCache.clear();
await (engine as any).bufferPool.clear();
// 查询触发 prefetchRange → preloadSSTable 发现 CRC 失败 → 清理 + 不缓存 // 查询触发 prefetchRange → preloadSSTable 发现 CRC 失败 → 清理 + 不缓存
const remaining = await engine.find('users', { table: 'users' }); const remaining = await engine.find('users', { table: 'users' });
expect(remaining.length).toBeLessThan(100); expect(remaining.length).toBeLessThan(100);
expect(lsm.sstableCache.has(Number(victim.replace('sst_', '')))).toBe(false); expect(lsm.sstableCache.has(Number(victim.slice(3)))).toBe(false);
const metas = await lsm.sstableStore.listMeta(); const metas = await lsm.sstableStore.listMeta();
expect(metas.some((m: { id: number }) => m.id === Number(victim.replace('sst_', '')))).toBe(false); expect(metas.some((m: { id: number }) => m.id === Number(victim.slice(3)))).toBe(false);
await engine.close(); await engine.close();
}); });
it('旧版无校验文件(checksum=0)在 LSM 中正常加载', async () => { it('旧版无校验文件(checksum=0)在 LSM 中正常加载', async () => {
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
const dbName = uniqueDB(); const dbName = uniqueDB();
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
@@ -159,7 +173,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
await (engine as any).lsm.flush(); await (engine as any).lsm.flush();
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
const sstKeys = await listSSTKeys(engine2); const sstKeys = await listSSTKeys(engine2);
expect(sstKeys.length).toBe(1); expect(sstKeys.length).toBe(1);
@@ -171,7 +185,7 @@ describe('AriaEngine — SSTable CRC 损坏检测(集成)', () => {
await engine2.close(); await engine2.close();
// 重开:checksum=0 跳过校验,数据完整可读 // 重开:checksum=0 跳过校验,数据完整可读
const engine3 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 64 * 1024 * 1024 }); const engine3 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine3.open(dbName, 1); await engine3.open(dbName, 1);
const rows3 = await engine3.find('users', { table: 'users' }); const rows3 = await engine3.find('users', { table: 'users' });
expect(rows3.length).toBe(2); expect(rows3.length).toBe(2);
-1
View File
@@ -4,7 +4,6 @@
* *
* v0.2.6 * v0.2.6
*/ */
import 'fake-indexeddb/auto';
import { CryptoManager } from '../../src/engine/aria/crypto'; import { CryptoManager } from '../../src/engine/aria/crypto';
function toBytes(data: ArrayBuffer): number[] { function toBytes(data: ArrayBuffer): number[] {
+4
View File
@@ -5,6 +5,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — 扩展边缘测试', () => { describe('AriaEngine — 扩展边缘测试', () => {
let engine: AriaEngine; let engine: AriaEngine;
+20 -17
View File
@@ -14,7 +14,10 @@ import { AriaEngine } from '../../src/engine/aria/index';
import { EncryptedBackend } from '../../src/engine/aria/store/encrypted_backend'; import { EncryptedBackend } from '../../src/engine/aria/store/encrypted_backend';
import { MemoryBackend } from '../../src/engine/aria/store/backend'; import { MemoryBackend } from '../../src/engine/aria/store/backend';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import 'fake-indexeddb/auto';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -182,7 +185,7 @@ describe('AriaEngine — EncryptedBackend 单元', () => {
describe('AriaEngine — 全库加密(集成)', () => { describe('AriaEngine — 全库加密(集成)', () => {
it('加密库:写入 → close → 同密码重开数据完整', async () => { it('加密库:写入 → close → 同密码重开数据完整', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'master-pass' } }); const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'master-pass' } });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.insert('users', [ await engine.insert('users', [
@@ -193,7 +196,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
await engine.close(); await engine.close();
// 重开(同密码) // 重开(同密码)
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'master-pass' } }); const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'master-pass' } });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
const rows = await engine2.find('users', { table: 'users' }); const rows = await engine2.find('users', { table: 'users' });
expect(rows).toHaveLength(2); expect(rows).toHaveLength(2);
@@ -206,42 +209,42 @@ describe('AriaEngine — 全库加密(集成)', () => {
it('加密库:错误密码重开 → ARIA_DECRYPT_ERROR', async () => { it('加密库:错误密码重开 → ARIA_DECRYPT_ERROR', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'right-pass' } }); const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'right-pass' } });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]); await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]);
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'wrong-pass' } }); const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'wrong-pass' } });
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' }); await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
}); });
it('加密库:无密码打开 → ARIA_ENCRYPT_REQUIRED', async () => { it('加密库:无密码打开 → ARIA_ENCRYPT_REQUIRED', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } }); const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' }); const engine2 = new AriaEngine({ storageBackend: 'opfs' });
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_REQUIRED' }); await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_REQUIRED' });
}); });
it('明文库:加密配置打开 → ARIA_ENCRYPT_CONFIG_ERROR', async () => { it('明文库:加密配置打开 → ARIA_ENCRYPT_CONFIG_ERROR', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb' }); const engine = new AriaEngine({ storageBackend: 'opfs' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.insert('users', [{ id: '1', name: 'A', secret: 'plain' }]); await engine.insert('users', [{ id: '1', name: 'A', secret: 'plain' }]);
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } }); const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_CONFIG_ERROR' }); await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_ENCRYPT_CONFIG_ERROR' });
}); });
it('加密库:底层存储全部为密文(SSTable/Schema/WAL 均不可见明文)', async () => { it('加密库:底层存储全部为密文(SSTable/Schema/WAL 均不可见明文)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } }); const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.insert('users', [{ id: '1', name: 'Alice', secret: 'needle-in-cipher' }]); await engine.insert('users', [{ id: '1', name: 'Alice', secret: 'needle-in-cipher' }]);
@@ -267,7 +270,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
it('加密 + 压缩组合:往返完整(压缩层先压,加密层后加密)', async () => { it('加密 + 压缩组合:往返完整(压缩层先压,加密层后加密)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
compression: true, compression: true,
encryption: { password: 'pw-compress' }, encryption: { password: 'pw-compress' },
memtableSizeThreshold: 64 * 1024 * 1024, memtableSizeThreshold: 64 * 1024 * 1024,
@@ -283,7 +286,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ const engine2 = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
compression: true, compression: true,
encryption: { password: 'pw-compress' }, encryption: { password: 'pw-compress' },
}); });
@@ -296,7 +299,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
it('加密库:clearAll 保留密钥(库身份),同密码重开重建,新密码被拒', async () => { it('加密库:clearAll 保留密钥(库身份),同密码重开重建,新密码被拒', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'old-pw' } }); const engine = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'old-pw' } });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]); await engine.insert('users', [{ id: '1', name: 'A', secret: 'x' }]);
@@ -304,11 +307,11 @@ describe('AriaEngine — 全库加密(集成)', () => {
await engine.close(); await engine.close();
// clearAll 保留 keymeta → 换新密码会被拒绝 // clearAll 保留 keymeta → 换新密码会被拒绝
const engineWrong = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'new-pw' } }); const engineWrong = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'new-pw' } });
await expect(engineWrong.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' }); await expect(engineWrong.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
// 同密码重开可重建 // 同密码重开可重建
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'old-pw' } }); const engine2 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'old-pw' } });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
await engine2.createTable(SCHEMA()); await engine2.createTable(SCHEMA());
await engine2.insert('users', [{ id: '1', name: 'A', secret: 'new' }]); await engine2.insert('users', [{ id: '1', name: 'A', secret: 'new' }]);
@@ -319,7 +322,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
it('加密库:WAL 崩溃恢复路径可用(含加密 WAL)', async () => { it('加密库:WAL 崩溃恢复路径可用(含加密 WAL)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
encryption: { password: 'pw' }, encryption: { password: 'pw' },
walSyncMode: 'full', walSyncMode: 'full',
checkpointInterval: 100000, checkpointInterval: 100000,
@@ -336,7 +339,7 @@ describe('AriaEngine — 全库加密(集成)', () => {
// 重开:WAL 重放(加密 WAL 解密后解析) // 重开:WAL 重放(加密 WAL 解密后解析)
const engine2 = new AriaEngine({ const engine2 = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
encryption: { password: 'pw' }, encryption: { password: 'pw' },
walSyncMode: 'full', walSyncMode: 'full',
checkpointInterval: 100000, checkpointInterval: 100000,
+4
View File
@@ -4,6 +4,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — 补充测试', () => { describe('AriaEngine — 补充测试', () => {
let engine: AriaEngine; let engine: AriaEngine;
beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('sup', 1); }); beforeEach(async () => { engine = new AriaEngine({ storageBackend: 'memory' }); await engine.open('sup', 1); });
+4
View File
@@ -7,6 +7,10 @@ import { createSchema } from '../../src/table/schema';
import { MemoryBackend } from '../../src/engine/aria/store/backend'; import { MemoryBackend } from '../../src/engine/aria/store/backend';
import { checkFieldType } from '../../src/table/schema'; import { checkFieldType } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — 最终扩展测试', () => { describe('AriaEngine — 最终扩展测试', () => {
let engine: AriaEngine; let engine: AriaEngine;
+4
View File
@@ -5,6 +5,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — 二级索引查询', () => { describe('AriaEngine — 二级索引查询', () => {
let engine: AriaEngine; let engine: AriaEngine;
+18 -9
View File
@@ -11,7 +11,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { DatabaseLock, lockName } from '../../src/engine/aria/locks'; import { DatabaseLock, lockName } from '../../src/engine/aria/locks';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import 'fake-indexeddb/auto';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -25,8 +28,12 @@ let lockHolders: Map<string, true>;
function installLocksMock() { function installLocksMock() {
lockHolders = new Map(); lockHolders = new Map();
// 保留 storageOPFS mock 设置过),与 locks 共存
const existingNav = (globalThis as { navigator?: { storage?: unknown } }).navigator;
const storage = existingNav?.storage;
Object.defineProperty(globalThis, 'navigator', { Object.defineProperty(globalThis, 'navigator', {
value: { value: {
storage,
locks: { locks: {
request: async (_name: string, opts: { ifAvailable?: boolean }, cb: (lock: { name: string } | null) => Promise<void> | void) => { request: async (_name: string, opts: { ifAvailable?: boolean }, cb: (lock: { name: string } | null) => Promise<void> | void) => {
const name = _name; const name = _name;
@@ -53,8 +60,10 @@ function installLocksMock() {
} }
function removeLocksMock() { function removeLocksMock() {
// 只移除 locks,保留 storageOPFS mock 需要)
const existingNav = (globalThis as { navigator?: { storage?: unknown } }).navigator;
Object.defineProperty(globalThis, 'navigator', { Object.defineProperty(globalThis, 'navigator', {
value: {}, value: { storage: existingNav?.storage },
configurable: true, configurable: true,
writable: true, writable: true,
}); });
@@ -120,11 +129,11 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
it('第二个标签页打开同一库 → ARIA_LOCKED', async () => { it('第二个标签页打开同一库 → ARIA_LOCKED', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine1 = new AriaEngine({ storageBackend: 'indexeddb' }); const engine1 = new AriaEngine({ storageBackend: 'opfs' });
await engine1.open(dbName, 1); await engine1.open(dbName, 1);
await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } })); await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' }); const engine2 = new AriaEngine({ storageBackend: 'opfs' });
await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_LOCKED' }); await expect(engine2.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_LOCKED' });
// engine2 未持锁、未 opened // engine2 未持锁、未 opened
expect((engine2 as any).opened).toBe(false); expect((engine2 as any).opened).toBe(false);
@@ -132,7 +141,7 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
await engine1.close(); await engine1.close();
// 释放后可打开 // 释放后可打开
const engine3 = new AriaEngine({ storageBackend: 'indexeddb' }); const engine3 = new AriaEngine({ storageBackend: 'opfs' });
await engine3.open(dbName, 1); await engine3.open(dbName, 1);
expect(await engine3.count('t')).toBe(0); expect(await engine3.count('t')).toBe(0);
await engine3.close(); await engine3.close();
@@ -140,19 +149,19 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
it('open 失败(错误密码)→ 锁释放,其他标签页可打开', async () => { it('open 失败(错误密码)→ 锁释放,其他标签页可打开', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine1 = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } }); const engine1 = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
await engine1.open(dbName, 1); await engine1.open(dbName, 1);
await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } })); await engine1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
await engine1.close(); await engine1.close();
// 错误密码 → open 失败 // 错误密码 → open 失败
const engineBad = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'wrong' } }); const engineBad = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'wrong' } });
await expect(engineBad.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' }); await expect(engineBad.open(dbName, 1)).rejects.toMatchObject({ code: 'ARIA_DECRYPT_ERROR' });
// 失败后锁已释放(未泄漏) // 失败后锁已释放(未泄漏)
expect(lockHolders.has(lockName(dbName))).toBe(false); expect(lockHolders.has(lockName(dbName))).toBe(false);
// 正确密码可打开 // 正确密码可打开
const engineOk = new AriaEngine({ storageBackend: 'indexeddb', encryption: { password: 'pw' } }); const engineOk = new AriaEngine({ storageBackend: 'opfs', encryption: { password: 'pw' } });
await engineOk.open(dbName, 1); await engineOk.open(dbName, 1);
await engineOk.close(); await engineOk.close();
}); });
@@ -160,7 +169,7 @@ describe('AriaEngine — 多标签页锁(集成)', () => {
it('无 Web Locks 环境 → 降级打开(不抛错)', async () => { it('无 Web Locks 环境 → 降级打开(不抛错)', async () => {
removeLocksMock(); removeLocksMock();
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb' }); const engine = new AriaEngine({ storageBackend: 'opfs' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } })); await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
await engine.insert('t', [{ id: '1' }]); await engine.insert('t', [{ id: '1' }]);
+4
View File
@@ -7,6 +7,10 @@ import { createSchema } from '../../src/table/schema';
import { QueryExecutor } from '../../src/query/executor'; import { QueryExecutor } from '../../src/query/executor';
import { parse } from '../../src/sql/parser'; import { parse } from '../../src/sql/parser';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — ANALYZE/VACUUM/REINDEX + EXPLAIN', () => { describe('AriaEngine — ANALYZE/VACUUM/REINDEX + EXPLAIN', () => {
let engine: AriaEngine; let engine: AriaEngine;
+4
View File
@@ -5,6 +5,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
describe('AriaEngine — MVCC 事务 + Savepoint', () => { describe('AriaEngine — MVCC 事务 + Savepoint', () => {
let engine: AriaEngine; let engine: AriaEngine;
+7 -6
View File
@@ -12,7 +12,8 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { installOPFSMock } from '../helpers/opfs-mock'; import { installOPFSMock } from '../helpers/opfs-mock';
import 'fake-indexeddb/auto';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -32,7 +33,7 @@ describe('AriaEngine — repair 自愈增强', () => {
it('清理孤儿页面(meta 未引用的 pg_ 文件)', async () => { it('清理孤儿页面(meta 未引用的 pg_ 文件)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
pageStorage: true, // 强制页面化(IDB 上也启用,便于断言页面文件) pageStorage: true, // 强制页面化(IDB 上也启用,便于断言页面文件)
memtableSizeThreshold: 64 * 1024 * 1024, memtableSizeThreshold: 64 * 1024 * 1024,
checkpointInterval: 100000, checkpointInterval: 100000,
@@ -86,7 +87,7 @@ describe('AriaEngine — repair 自愈增强', () => {
it('WAL 空洞 + repair → 截断清空(不再重放错位数据)', async () => { it('WAL 空洞 + repair → 截断清空(不再重放错位数据)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
// 数据落盘 + checkpoint 清 WAL // 数据落盘 + checkpoint 清 WAL
@@ -116,7 +117,7 @@ describe('AriaEngine — repair 自愈增强', () => {
it('repair 幂等(连续调用无副作用)', async () => { it('repair 幂等(连续调用无副作用)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(SCHEMA()); await engine.createTable(SCHEMA());
await engine.insert('items', [{ id: 'a', val: 1, tag: 'x' }]); await engine.insert('items', [{ id: 'a', val: 1, tag: 'x' }]);
@@ -135,7 +136,7 @@ describe('AriaEngine — 随机操作压力 + 模拟崩溃', () => {
it('500 随机操作(insert/update/delete)→ 模拟崩溃 → 重开验证全部已确认写入', async () => { it('500 随机操作(insert/update/delete)→ 模拟崩溃 → 重开验证全部已确认写入', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
memtableSizeThreshold: 8 * 1024, memtableSizeThreshold: 8 * 1024,
checkpointInterval: 50, checkpointInterval: 50,
walSyncMode: 'full', walSyncMode: 'full',
@@ -178,7 +179,7 @@ describe('AriaEngine — 随机操作压力 + 模拟崩溃', () => {
(engine as any).opened = false; (engine as any).opened = false;
// 重开:WAL 重放 + SSTable 加载 // 重开:WAL 重放 + SSTable 加载
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 50, walSyncMode: 'full' }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 50, walSyncMode: 'full' });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
const rows = await engine2.find('items', { table: 'items' }); const rows = await engine2.find('items', { table: 'items' });
const byId = new Map(rows.map((r) => [r.id, r])); const byId = new Map(rows.map((r) => [r.id, r]));
+11 -8
View File
@@ -13,7 +13,10 @@ import { SegmentedWALStore } from '../../src/engine/aria/wal/segmented_store';
import { MemoryBackend } from '../../src/engine/aria/store/backend'; import { MemoryBackend } from '../../src/engine/aria/store/backend';
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import 'fake-indexeddb/auto';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -153,7 +156,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
it('高频写入(多条 WAL 记录)→ close → reopen 数据完整', async () => { it('高频写入(多条 WAL 记录)→ close → reopen 数据完整', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 100000, checkpointInterval: 100000,
walSyncMode: 'full', walSyncMode: 'full',
}); });
@@ -171,7 +174,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
(engine as any).opened = false; (engine as any).opened = false;
const engine2 = new AriaEngine({ const engine2 = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 100000, checkpointInterval: 100000,
walSyncMode: 'full', walSyncMode: 'full',
}); });
@@ -185,7 +188,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
it('分片结构落盘验证(backend 中是分片文件而非旧单记录键)', async () => { it('分片结构落盘验证(backend 中是分片文件而非旧单记录键)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 100000, checkpointInterval: 100000,
walSyncMode: 'full', walSyncMode: 'full',
}); });
@@ -208,7 +211,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
it('WAL 分片部分残留(空洞)→ 重开恢复已落盘数据,不丢已 checkpoint 数据', async () => { it('WAL 分片部分残留(空洞)→ 重开恢复已落盘数据,不丢已 checkpoint 数据', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 100000, checkpointInterval: 100000,
walSyncMode: 'full', walSyncMode: 'full',
}); });
@@ -228,7 +231,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
await backend.delete('__wal_000000.bin'); await backend.delete('__wal_000000.bin');
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
// 已落盘的 a/b 必须保留;c/d 在 WAL 中且文件被删 → 恢复不到(可接受的保守丢弃) // 已落盘的 a/b 必须保留;c/d 在 WAL 中且文件被删 → 恢复不到(可接受的保守丢弃)
const rows = await engine2.find('t', { table: 't' }); const rows = await engine2.find('t', { table: 't' });
@@ -241,7 +244,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
// 手工构造旧格式 WAL 库:直接写旧键(模拟 v0.4.4 库崩溃现场) // 手工构造旧格式 WAL 库:直接写旧键(模拟 v0.4.4 库崩溃现场)
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 100000, checkpointInterval: 100000,
walSyncMode: 'full', walSyncMode: 'full',
}); });
@@ -262,7 +265,7 @@ describe('AriaEngine — 分片 WAL 集成', () => {
await (engine as any).backend.close(); await (engine as any).backend.close();
(engine as any).opened = false; (engine as any).opened = false;
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
const rows = await engine2.find('t', { table: 't' }); const rows = await engine2.find('t', { table: 't' });
expect(rows).toHaveLength(1); expect(rows).toHaveLength(1);
+5 -2
View File
@@ -8,7 +8,10 @@
import { AriaEngine } from '../../src/engine/aria/index'; import { AriaEngine } from '../../src/engine/aria/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import { MetonaSqlark } from '../../src/core'; import { MetonaSqlark } from '../../src/core';
import 'fake-indexeddb/auto';
import { installOPFSMock } from '../helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
// =================================================================== // ===================================================================
// AriaEngine 引擎级测试 (Memory Backend) // AriaEngine 引擎级测试 (Memory Backend)
@@ -549,7 +552,7 @@ describe('MetonaSqlark with mode=aria (Memory)', () => {
let db: MetonaSqlark; let db: MetonaSqlark;
beforeEach(async () => { beforeEach(async () => {
db = new MetonaSqlark({ name: `ms-aria-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'indexeddb' }); db = new MetonaSqlark({ name: `ms-aria-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'opfs' });
await db.init(); await db.init();
}); });
-323
View File
@@ -1,323 +0,0 @@
/**
* IndexedDBEngine 使 fake-indexeddb
* 使
*/
import 'fake-indexeddb/auto';
import { IndexedDBEngine } from '../../src/engine/indexeddb';
import { createSchema } from '../../src/table/schema';
let dbCounter = 0;
describe('IndexedDBEngine', () => {
let engine: IndexedDBEngine;
let dbName: string;
const userSchema = createSchema('users', {
id: { type: 'string', primaryKey: true },
name: { type: 'string', required: true },
age: { type: 'number', default: 0 },
email: { type: 'string', unique: true },
});
beforeEach(async () => {
dbName = `test-idb-${++dbCounter}`;
engine = new IndexedDBEngine();
await engine.open(dbName, 1);
});
afterEach(async () => {
if (engine.isOpen()) {
await engine.close();
}
// 清理:删除 IndexedDB 数据库
try {
indexedDB.deleteDatabase(dbName);
} catch {}
});
// ---- 生命周期 ----
describe('生命周期', () => {
it('打开和关闭', async () => {
expect(engine.isOpen()).toBe(true);
await engine.close();
expect(engine.isOpen()).toBe(false);
});
it('未打开时操作抛出错误', async () => {
await engine.close();
await expect(engine.count('users')).rejects.toThrow('not opened');
});
it('可以重新打开', async () => {
await engine.close();
await engine.open(dbName, 2);
expect(engine.isOpen()).toBe(true);
});
});
// ---- 表管理 ----
describe('表管理', () => {
it('创建和删除表', async () => {
await engine.createTable(userSchema);
expect(await engine.hasTable('users')).toBe(true);
await engine.dropTable('users');
expect(await engine.hasTable('users')).toBe(false);
});
it('表名列表', async () => {
await engine.createTable(userSchema);
const names = await engine.getTableNames();
expect(names).toContain('users');
});
it('获取表结构', async () => {
await engine.createTable(userSchema);
const schema = await engine.getTableSchema('users');
expect(schema).not.toBeNull();
expect(schema!.name).toBe('users');
});
it('获取不存在的表结构返回 null', async () => {
const schema = await engine.getTableSchema('nonexistent');
expect(schema).toBeNull();
});
});
// ---- 插入 ----
describe('插入', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
});
it('插入单行', async () => {
const pks = await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'alice@test.com' },
]);
expect(pks).toEqual(['1']);
expect(await engine.count('users')).toBe(1);
});
it('插入多行', async () => {
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
]);
expect(await engine.count('users')).toBe(2);
});
it('必填字段缺失抛出错误', async () => {
await expect(
engine.insert('users', [{ id: '1', email: 'a@t.com' }]),
).rejects.toThrow('required');
});
});
// ---- 查询 ----
describe('查询', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'alice@test.com' },
{ id: '2', name: 'Bob', age: 25, email: 'bob@test.com' },
{ id: '3', name: 'Charlie', age: 35, email: 'charlie@test.com' },
]);
});
it('查询所有', async () => {
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(3);
});
it('WHERE 条件', async () => {
const rows = await engine.find('users', {
table: 'users',
where: { age: { $gt: 28 } },
});
expect(rows).toHaveLength(2);
});
it('ORDER BY + LIMIT', async () => {
const rows = await engine.find('users', {
table: 'users',
orderBy: [{ column: 'age', direction: 'desc' }],
limit: 2,
});
expect(rows.map(r => r.age)).toEqual([35, 30]);
});
it('列选择', async () => {
const rows = await engine.find('users', {
table: 'users',
columns: ['id'],
where: { id: '1' },
});
expect(Object.keys(rows[0])).toEqual(['id']);
});
});
// ---- 更新 ----
describe('更新', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
]);
});
it('更新匹配行', async () => {
const count = await engine.update(
'users',
{ table: 'users', where: { id: '1' } },
{ age: 31 },
);
expect(count).toBe(1);
const rows = await engine.find('users', { table: 'users', where: { id: '1' } });
expect(rows[0].age).toBe(31);
});
it('更新所有行', async () => {
const count = await engine.update(
'users',
{ table: 'users' },
{ age: 100 },
);
expect(count).toBe(2);
});
});
// ---- 删除 ----
describe('删除', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
]);
});
it('条件删除', async () => {
const count = await engine.delete('users', {
table: 'users',
where: { id: '1' },
});
expect(count).toBe(1);
expect(await engine.count('users')).toBe(1);
});
it('清空表', async () => {
await engine.clear('users');
expect(await engine.count('users')).toBe(0);
// 表结构仍存在
expect(await engine.hasTable('users')).toBe(true);
});
it('删除不存在的表抛出错误', async () => {
await expect(
engine.delete('nonexistent', { table: 'nonexistent' }),
).rejects.toThrow();
});
});
// ---- Count ----
describe('Count', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
]);
});
it('count 全部', async () => {
expect(await engine.count('users')).toBe(2);
});
it('count with where', async () => {
expect(await engine.count('users', {
table: 'users',
where: { age: { $gt: 28 } },
})).toBe(1);
});
});
// ---- 复杂 Where 条件 ----
describe('复杂 Where', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
{ id: '2', name: 'Bob', age: 25, email: 'b@t.com' },
{ id: '3', name: 'Charlie', age: 35, email: 'c@t.com' },
]);
});
it('$like 模糊匹配', async () => {
const rows = await engine.find('users', {
table: 'users',
where: { name: { $like: 'A%' } },
});
expect(rows).toHaveLength(1);
});
it('$in 列表', async () => {
const rows = await engine.find('users', {
table: 'users',
where: { id: { $in: ['1', '3'] } },
});
expect(rows).toHaveLength(2);
});
it('$ne 不等于', async () => {
const rows = await engine.find('users', {
table: 'users',
where: { age: { $ne: 25 } },
});
expect(rows).toHaveLength(2);
});
it('$gte $lte 范围', async () => {
const rows = await engine.find('users', {
table: 'users',
where: { age: { $gte: 25, $lte: 30 } },
});
expect(rows).toHaveLength(2);
});
});
// ---- 并发/多次操作 ----
describe('多次操作', () => {
it('连续创建删除表', async () => {
await engine.createTable(userSchema);
expect(await engine.hasTable('users')).toBe(true);
await engine.dropTable('users');
expect(await engine.hasTable('users')).toBe(false);
await engine.createTable(userSchema);
expect(await engine.hasTable('users')).toBe(true);
});
it('插入后查询再更新再查询', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30, email: 'a@t.com' },
]);
let rows = await engine.find('users', { table: 'users' });
expect(rows[0].age).toBe(30);
await engine.update('users', { table: 'users' }, { age: 31 });
rows = await engine.find('users', { table: 'users' });
expect(rows[0].age).toBe(31);
});
});
});
+508
View File
@@ -0,0 +1,508 @@
/**
* SharedMemoryBackend
*/
import { KVStoreEngine } from '../../src/engine/kvstore_engine';
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
import { createSchema } from '../../src/table/schema';
let dbCounter = 0;
describe('KVStoreEngine', () => {
let engine: KVStoreEngine;
let dbName: string;
const userSchema = createSchema('users', {
id: { type: 'string', primaryKey: true },
name: { type: 'string', required: true },
age: { type: 'number', default: 0 },
email: { type: 'string', unique: true },
});
beforeEach(async () => {
dbName = `test-kv-${++dbCounter}-${Math.random().toString(36).slice(2, 6)}`;
engine = new KVStoreEngine(new SharedMemoryBackend());
await engine.open(dbName, 1);
});
afterEach(async () => {
if (engine.isOpen()) {
await engine.close();
}
});
// ---- 生命周期 ----
describe('生命周期', () => {
it('打开和关闭', async () => {
expect(engine.isOpen()).toBe(true);
await engine.close();
expect(engine.isOpen()).toBe(false);
});
it('未打开时操作抛出错误', async () => {
await engine.close();
await expect(engine.count('users')).rejects.toThrow('not opened');
});
it('可以重新打开', async () => {
await engine.close();
await engine.open(dbName, 2);
expect(engine.isOpen()).toBe(true);
});
});
// ---- 表管理 ----
describe('表管理', () => {
it('创建表', async () => {
await engine.createTable(userSchema);
expect(await engine.hasTable('users')).toBe(true);
const names = await engine.getTableNames();
expect(names).toContain('users');
});
it('重复创建表抛错', async () => {
await engine.createTable(userSchema);
await expect(engine.createTable(userSchema)).rejects.toThrow('already exists');
});
it('删除表', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.dropTable('users');
expect(await engine.hasTable('users')).toBe(false);
await expect(engine.count('users')).rejects.toThrow('does not exist');
});
it('获取表结构', async () => {
await engine.createTable(userSchema);
const schema = await engine.getTableSchema('users');
expect(schema).not.toBeNull();
expect(schema!.columns.id.primaryKey).toBe(true);
});
it('schema 持久化:重启后表结构保留', async () => {
await engine.createTable(userSchema);
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.hasTable('users')).toBe(true);
const schema = await engine2.getTableSchema('users');
expect(schema!.columns.email.unique).toBe(true);
await engine2.close();
});
});
// ---- 插入 ----
describe('插入', () => {
beforeEach(async () => { await engine.createTable(userSchema); });
it('插入单行并返回主键', async () => {
const pks = await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
expect(pks).toEqual(['1']);
expect(await engine.count('users')).toBe(1);
});
it('默认值生效', async () => {
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
const rows = await engine.find('users', { table: 'users' });
expect(rows[0].age).toBe(0);
});
it('重复主键抛错', async () => {
await engine.insert('users', [{ id: '1', name: 'A' }]);
await expect(engine.insert('users', [{ id: '1', name: 'B' }])).rejects.toThrow('Duplicate primary key');
});
it('唯一约束冲突抛错', async () => {
await engine.insert('users', [{ id: '1', name: 'A', email: 'a@x.com' }]);
await expect(engine.insert('users', [{ id: '2', name: 'B', email: 'a@x.com' }]))
.rejects.toThrow('Unique constraint violation');
});
it('必填字段校验', async () => {
await expect(engine.insert('users', [{ id: '1' }])).rejects.toThrow('is required');
});
it('批量插入', async () => {
const pks = await engine.insert('users', [
{ id: '1', name: 'A' }, { id: '2', name: 'B' }, { id: '3', name: 'C' },
]);
expect(pks).toEqual(['1', '2', '3']);
});
});
// ---- 查询 ----
describe('查询', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30 },
{ id: '2', name: 'Bob', age: 25 },
{ id: '3', name: 'Carol', age: 35 },
]);
});
it('全表查询', async () => {
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(3);
});
it('WHERE 等值过滤', async () => {
const rows = await engine.find('users', { table: 'users', where: { name: 'Bob' } });
expect(rows).toHaveLength(1);
expect(rows[0].id).toBe('2');
});
it('WHERE 操作符', async () => {
const rows = await engine.find('users', { table: 'users', where: { age: { $gt: 26 } } });
expect(rows.map((r) => r.id).sort()).toEqual(['1', '3']);
});
it('ORDER BY', async () => {
const rows = await engine.find('users', { table: 'users', orderBy: [{ column: 'age', direction: 'desc' }] });
expect(rows[0].id).toBe('3');
expect(rows[2].id).toBe('2');
});
it('LIMIT / OFFSET', async () => {
const rows = await engine.find('users', { table: 'users', limit: 2, offset: 1 });
expect(rows).toHaveLength(2);
});
it('列投影', async () => {
const rows = await engine.find('users', { table: 'users', columns: ['name'] });
expect(Object.keys(rows[0])).toEqual(['name']);
});
});
// ---- 更新 ----
describe('更新', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30 },
{ id: '2', name: 'Bob', age: 25 },
]);
});
it('按条件更新', async () => {
const count = await engine.update('users', { table: 'users', where: { id: '1' } }, { age: 31 });
expect(count).toBe(1);
const rows = await engine.find('users', { table: 'users', where: { id: '1' } });
expect(rows[0].age).toBe(31);
});
it('无匹配行影响 0 行', async () => {
const count = await engine.update('users', { table: 'users', where: { id: 'nope' } }, { age: 1 });
expect(count).toBe(0);
});
it('更新主键(旧键删除 + 新键写入)', async () => {
const count = await engine.update('users', { table: 'users', where: { id: '1' } }, { id: '10', age: 99 });
expect(count).toBe(1);
expect(await engine.count('users')).toBe(2);
const rows = await engine.find('users', { table: 'users', where: { id: '10' } });
expect(rows[0].age).toBe(99);
expect(await engine.find('users', { table: 'users', where: { id: '1' } })).toHaveLength(0);
});
it('更新持久化:重启后保留', async () => {
await engine.update('users', { table: 'users', where: { id: '1' } }, { age: 99 });
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
const rows = await engine2.find('users', { table: 'users', where: { id: '1' } });
expect(rows[0].age).toBe(99);
await engine2.close();
});
});
// ---- 删除 ----
describe('删除', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }, { id: '3', name: 'Carol' },
]);
});
it('按条件删除', async () => {
const count = await engine.delete('users', { table: 'users', where: { id: '1' } });
expect(count).toBe(1);
expect(await engine.count('users')).toBe(2);
});
it('删除持久化:重启后保留', async () => {
await engine.delete('users', { table: 'users', where: { id: '1' } });
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('users')).toBe(2);
await engine2.close();
});
it('级联删除(CASCADE', async () => {
const ordersSchema = createSchema('orders', {
id: { type: 'string', primaryKey: true },
user_id: { type: 'string', references: 'users.id', onDelete: 'CASCADE' },
});
await engine.createTable(ordersSchema);
await engine.insert('orders', [{ id: 'o1', user_id: '1' }, { id: 'o2', user_id: '2' }]);
await engine.delete('users', { table: 'users', where: { id: '1' } });
// 级联:orders 中 user_id=1 的行被删
expect(await engine.count('orders')).toBe(1);
// 重启后级联结果持久化
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('users')).toBe(2);
expect(await engine2.count('orders')).toBe(1);
await engine2.close();
});
});
// ---- Count ----
describe('Count', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'A', age: 10 }, { id: '2', name: 'B', age: 20 }, { id: '3', name: 'C', age: 30 },
]);
});
it('总数', async () => {
expect(await engine.count('users')).toBe(3);
});
it('条件计数', async () => {
expect(await engine.count('users', { table: 'users', where: { age: { $gte: 20 } } })).toBe(2);
});
});
// ---- 事务 ----
describe('事务', () => {
beforeEach(async () => { await engine.createTable(userSchema); });
it('commit 后数据可见且持久化', async () => {
await engine.beginTransaction();
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.commitTransaction();
expect(await engine.count('users')).toBe(1);
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('users')).toBe(1);
await engine2.close();
});
it('rollback 后数据消失', async () => {
await engine.beginTransaction();
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.rollbackTransaction();
expect(await engine.count('users')).toBe(0);
});
it('事务内读到自己写入的行', async () => {
await engine.beginTransaction();
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(1);
await engine.rollbackTransaction();
});
it('commit 多表原子(全部生效或全部不生效)', async () => {
await engine.createTable(createSchema('orders', {
id: { type: 'string', primaryKey: true },
user_id: { type: 'string' },
}));
await engine.beginTransaction();
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.insert('orders', [{ id: 'o1', user_id: '1' }]);
await engine.commitTransaction();
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('users')).toBe(1);
expect(await engine2.count('orders')).toBe(1);
await engine2.close();
});
it('事务中 DDL 支持(create/drop 随 commit 生效)', async () => {
const txSchema = createSchema('tx_users', {
id: { type: 'string', primaryKey: true },
name: { type: 'string' },
});
await engine.beginTransaction();
await engine.createTable(txSchema);
await engine.insert('tx_users', [{ id: '1', name: 'A' }]);
await engine.commitTransaction();
expect(await engine.hasTable('tx_users')).toBe(true);
expect(await engine.count('tx_users')).toBe(1);
// 事务中 drop 表
await engine.beginTransaction();
await engine.dropTable('tx_users');
await engine.commitTransaction();
expect(await engine.hasTable('tx_users')).toBe(false);
// 重启后 drop 生效(KV 残留清理)
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.hasTable('tx_users')).toBe(false);
await engine2.close();
});
});
// ---- 持久化与崩溃恢复 ----
describe('持久化与崩溃恢复', () => {
it('未 checkpoint 数据重启后经日志重放恢复', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30 },
{ id: '2', name: 'Bob', age: 25 },
]);
await engine.close(); // 模拟"崩溃后重开"(未 checkpoint,数据在 KVStore 日志)
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('users')).toBe(2);
const rows = await engine2.find('users', { table: 'users' });
expect(rows.map((r) => r.id).sort()).toEqual(['1', '2']);
await engine2.close();
});
it('checkpoint 后重启(快照恢复)', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await (engine as any).kv.checkpoint();
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('users')).toBe(2);
await engine2.close();
});
it('索引列跨重启恢复(查询走索引)', async () => {
const idxSchema = createSchema('items', {
id: { type: 'string', primaryKey: true },
tag: { type: 'string', index: true },
});
await engine.createTable(idxSchema);
await engine.insert('items', [
{ id: '1', tag: 'a' }, { id: '2', tag: 'b' }, { id: '3', tag: 'a' },
]);
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
const rows = await engine2.find('items', { table: 'items', where: { tag: 'a' } });
expect(rows).toHaveLength(2);
await engine2.close();
});
it('CREATE INDEX 持久化(重启后索引保留)', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
await engine.createIndex('users', 'name');
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
const schema = await engine2.getTableSchema('users');
expect(schema!.columns.name.index).toBe(true);
const rows = await engine2.find('users', { table: 'users', where: { name: 'Alice' } });
expect(rows).toHaveLength(1);
await engine2.close();
});
it('ALTER TABLE ADD/DROP 持久化', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
await engine.alterTable('users', 'ADD', { name: 'phone', type: 'string' });
await engine.alterTable('users', 'DROP', { name: 'age', type: 'number' });
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
const schema = await engine2.getTableSchema('users');
expect(schema!.columns.phone).toBeDefined();
expect(schema!.columns.age).toBeUndefined();
const rows = await engine2.find('users', { table: 'users' });
expect(rows[0].phone).toBeUndefined();
expect(rows[0].age).toBeUndefined();
await engine2.close();
});
it('元数据 getMeta/setMeta 持久化', async () => {
await engine.setMeta('__metona_version', '3');
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.getMeta('__metona_version')).toBe('3');
await engine2.close();
});
it('数据损坏自愈:日志损坏 → open 不崩,repair 截断', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.close();
// 篡改日志尾部 CRC(模拟最后一条记录写入中断)
const medium = new SharedMemoryBackend();
await medium.open(dbName);
const log = await medium.read('__kv_log');
expect(log).not.toBeNull();
const corrupted = new Uint8Array(log as ArrayBuffer);
corrupted[corrupted.byteLength - 5] ^= 0xff;
await medium.write('__kv_log', corrupted.buffer as ArrayBuffer);
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1); // 损坏尾部截断,打开不崩
expect(await engine2.hasTable('users')).toBe(true);
await engine2.repair();
expect(await engine2.hasTable('users')).toBe(true);
// repair 后新写入正常
await engine2.insert('users', [{ id: '2', name: 'Bob' }]);
expect(await engine2.count('users')).toBeGreaterThanOrEqual(1);
await engine2.close();
});
});
// ---- 流式查询 ----
describe('流式查询', () => {
beforeEach(async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'A' }, { id: '2', name: 'B' }, { id: '3', name: 'C' },
]);
});
it('findStream 逐行回调', async () => {
const rows: Record<string, unknown>[] = [];
const count = await engine.findStream!('users', { table: 'users' }, (r) => rows.push(r));
expect(count).toBe(3);
expect(rows.map((r) => r.id).sort()).toEqual(['1', '2', '3']);
});
});
});
+139
View File
@@ -0,0 +1,139 @@
/**
* KVStoreEngine 10
*
*
* 1. 10 key + checkpoint +
* 2. insert/update/delete5 + checkpoint
* 3. checkpoint
*/
import { KVStoreEngine } from '../../src/engine/kvstore_engine';
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
import { createSchema } from '../../src/table/schema';
let counter = 0;
function uniqueDB(): string {
return `stress-${Date.now()}-${++counter}-${Math.random().toString(36).slice(2, 6)}`;
}
beforeEach(() => {
SharedMemoryBackend.clearRegistry();
});
describe('KVStoreEngine — 10 万级压力', () => {
it('10 万 key 写入 + checkpoint + 重开全量验证', async () => {
const dbName = uniqueDB();
const engine = new KVStoreEngine(new SharedMemoryBackend());
await engine.open(dbName, 1);
await engine.createTable(createSchema('big', {
id: { type: 'string', primaryKey: true },
val: { type: 'number' },
name: { type: 'string' },
}));
const TOTAL = 100000;
// 分批写入(每批 1000
for (let batch = 0; batch < TOTAL / 1000; batch++) {
const rows = [] as Record<string, unknown>[];
for (let i = 0; i < 1000; i++) {
const idx = batch * 1000 + i;
rows.push({ id: `k${idx}`, val: idx, name: `User${idx}` });
}
await engine.insert('big', rows);
// 每 20 批 checkpoint 一次
if ((batch + 1) % 20 === 0) {
await (engine as any).kv.checkpoint();
}
}
expect(await engine.count('big')).toBe(TOTAL);
await engine.close();
// 重开:全量验证
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('big')).toBe(TOTAL);
// 抽样验证
for (const id of ['k0', 'k49999', 'k99999', 'k12345']) {
const rows = await engine2.find('big', { table: 'big', where: { id } });
expect(rows).toHaveLength(1);
expect(Number(rows[0].val)).toBe(Number(id.slice(1)));
}
await engine2.close();
}, 120000);
it('5 万混合操作 + 崩溃模拟(不 checkpoint)→ 重开全部已确认写入可见', async () => {
const dbName = uniqueDB();
const engine = new KVStoreEngine(new SharedMemoryBackend());
await engine.open(dbName, 1);
await engine.createTable(createSchema('ops', {
id: { type: 'string', primaryKey: true },
val: { type: 'number' },
}));
let seed = 42;
const rand = () => { seed = (seed * 1103515245 + 12345) & 0x7fffffff; return seed / 0x7fffffff; };
const confirmed = new Map<string, number>();
const TOTAL = 50000;
for (let i = 0; i < TOTAL; i++) {
const r = rand();
const id = `k${Math.floor(rand() * 20000)}`;
if (r < 0.6) {
const val = Math.floor(rand() * 1000000);
try {
await engine.insert('ops', [{ id, val }]);
confirmed.set(id, val);
} catch (e) {
if ((e as { code?: string }).code !== 'DUPLICATE_KEY') throw e;
}
} else if (r < 0.8) {
const val = Math.floor(rand() * 1000000);
await engine.update('ops', { table: 'ops', where: { id } }, { val });
if (confirmed.has(id)) confirmed.set(id, val);
} else {
await engine.delete('ops', { table: 'ops', where: { id } });
confirmed.delete(id);
}
// 每 5000 次 checkpoint
if ((i + 1) % 5000 === 0) {
await (engine as any).kv.checkpoint();
}
}
// 崩溃模拟:直接断开(不 close → 数据在 KVStore 日志/快照)
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend());
await engine2.open(dbName, 1);
expect(await engine2.count('ops')).toBe(confirmed.size);
// 抽样验证值
let sampled = 0;
for (const [id, val] of confirmed) {
if (sampled++ > 1000) break;
const rows = await engine2.find('ops', { table: 'ops', where: { id } });
expect(rows[0].val).toBe(val);
}
await engine2.close();
}, 120000);
it('多次 checkpoint 循环(500 次)数据不丢', async () => {
const dbName = uniqueDB();
const engine = new KVStoreEngine(new SharedMemoryBackend(), 0);
await engine.open(dbName, 1);
await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true },
v: { type: 'number' },
}));
for (let i = 0; i < 500; i++) {
await engine.insert('t', [{ id: `k${i}`, v: i }]);
await (engine as any).kv.checkpoint();
}
await engine.close();
const engine2 = new KVStoreEngine(new SharedMemoryBackend(), 0);
await engine2.open(dbName, 1);
expect(await engine2.count('t')).toBe(500);
const last = await engine2.find('t', { table: 't', where: { id: 'k499' } });
expect(last[0].v).toBe(499);
await engine2.close();
}, 60000);
});
+342
View File
@@ -0,0 +1,342 @@
/**
* KVStore KV
*
*
* 1. put/get/delete/exists/listKeys
* 2. putMany/deleteMany key
* 3. SharedMemory close
* 4. checkpoint
* 5. checkpoint +
* 6.
* 7.
* 8.
* 9. clear / repair
* 10. checkpoint
*/
import { KVStore } from '../../src/engine/kvstore/index';
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
import { encodeLogRecord, parseLogRecords, KVLogOp } from '../../src/engine/kvstore/log';
import { encodeSnapshot, decodeSnapshot } from '../../src/engine/kvstore/snapshot';
let dbCounter = 0;
function uniqueDB(): string {
return `kv-${Date.now()}-${++dbCounter}-${Math.random().toString(36).slice(2, 8)}`;
}
const enc = (s: string) => new TextEncoder().encode(s).buffer as ArrayBuffer;
const dec = (b: ArrayBuffer | null) => (b ? new TextDecoder().decode(b) : null);
beforeEach(() => {
SharedMemoryBackend.clearRegistry();
});
describe('KVStore — 基本读写', () => {
it('put/get/delete/exists/listKeys/size', async () => {
const kv = new KVStore(new SharedMemoryBackend());
await kv.open(uniqueDB());
await kv.put('a', enc('AAA'));
await kv.put('b', enc('BBB'));
expect(dec(await kv.get('a'))).toBe('AAA');
expect(await kv.exists('b')).toBe(true);
expect(await kv.exists('nope')).toBe(false);
expect((await kv.listKeys()).sort()).toEqual(['a', 'b']);
expect(kv.size()).toBe(2);
await kv.delete('a');
expect(await kv.exists('a')).toBe(false);
expect(kv.size()).toBe(1);
await kv.close();
});
it('putMany 多 key 原子写入', async () => {
const kv = new KVStore(new SharedMemoryBackend());
await kv.open(uniqueDB());
await kv.putMany({ a: enc('1'), b: enc('2'), c: enc('3') });
expect(dec(await kv.get('a'))).toBe('1');
expect(dec(await kv.get('c'))).toBe('3');
await kv.deleteMany(['a', 'c']);
expect(await kv.exists('a')).toBe(false);
expect(await kv.exists('c')).toBe(false);
expect(await kv.exists('b')).toBe(true);
await kv.close();
});
});
describe('KVStore — 持久化与崩溃恢复', () => {
it('未 checkpoint 的数据:close 后重开经日志重放恢复', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium);
await kv1.open(dbName);
await kv1.putMany({ a: enc('AAA'), b: enc('BBB') });
await kv1.put('c', enc('CCC'));
await kv1.delete('b');
await kv1.close();
// 模拟"崩溃后重开"(新实例,共享介质)
const kv2 = new KVStore(medium);
await kv2.open(dbName);
expect(dec(await kv2.get('a'))).toBe('AAA');
expect(await kv2.exists('b')).toBe(false);
expect(dec(await kv2.get('c'))).toBe('CCC');
await kv2.close();
});
it('checkpoint 后重开:快照加载 + 日志水位跳过(无重复/无丢失)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium, 0); // 关闭自动 checkpoint
await kv1.open(dbName);
await kv1.putMany({ a: enc('A1'), b: enc('B1') });
await kv1.checkpoint();
// checkpoint 后新写入(进日志)
await kv1.put('c', enc('C1'));
await kv1.put('a', enc('A2'));
await kv1.close();
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
expect(dec(await kv2.get('a'))).toBe('A2'); // 日志重放覆盖快照值
expect(dec(await kv2.get('b'))).toBe('B1');
expect(dec(await kv2.get('c'))).toBe('C1');
await kv2.close();
});
it('多次 checkpoint + 截断日志后重开正确', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium, 0);
await kv1.open(dbName);
for (let i = 0; i < 10; i++) {
await kv1.put(`k${i}`, enc(`v${i}`));
await kv1.checkpoint();
}
await kv1.put('last', enc('L'));
await kv1.close();
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
expect(kv2.size()).toBe(11);
expect(dec(await kv2.get('k9'))).toBe('v9');
expect(dec(await kv2.get('last'))).toBe('L');
await kv2.close();
});
it('快照损坏 → 全量日志重放自愈(数据不丢)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium, 0);
await kv1.open(dbName);
await kv1.putMany({ a: enc('AAA'), b: enc('BBB') });
await kv1.checkpoint();
await kv1.put('c', enc('CCC'));
await kv1.close();
// 篡改快照(模拟损坏)
const snap = await medium.read('__kv_snapshot');
const corrupted = new Uint8Array(snap as ArrayBuffer);
corrupted[20] ^= 0xff;
await medium.write('__kv_snapshot', corrupted.buffer as ArrayBuffer);
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
// 快照损坏 → 全量日志重放
expect(dec(await kv2.get('a'))).toBe('AAA');
expect(dec(await kv2.get('b'))).toBe('BBB');
expect(dec(await kv2.get('c'))).toBe('CCC');
await kv2.close();
});
it('日志损坏尾部 → 截断至损坏处(丢弃未确认尾部,已确认数据保留)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium, 0);
await kv1.open(dbName);
await kv1.put('confirmed', enc('KEEP'));
await kv1.put('confirmed2', enc('KEEP2'));
await kv1.close();
// 追加一条损坏记录(模拟写入中断:半写记录)
const bad = encodeLogRecord(999, { damaged: enc('X') });
const corrupted = new Uint8Array(bad);
corrupted[20] ^= 0xff; // 破坏 CRC
// close 后介质实例不可用,重新打开访问"磁盘"
const disk = new SharedMemoryBackend();
await disk.open(dbName);
const log = await disk.read('__kv_log');
const combined = new Uint8Array((log as ArrayBuffer).byteLength + corrupted.byteLength);
combined.set(new Uint8Array(log as ArrayBuffer), 0);
combined.set(corrupted, (log as ArrayBuffer).byteLength);
await disk.write('__kv_log', combined.buffer as ArrayBuffer);
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
expect(dec(await kv2.get('confirmed'))).toBe('KEEP');
expect(dec(await kv2.get('confirmed2'))).toBe('KEEP2');
expect(await kv2.exists('damaged')).toBe(false);
// 损坏日志已被截断(重开后再写正常)
await kv2.put('after', enc('OK'));
await kv2.close();
const kv3 = new KVStore(medium, 0);
await kv3.open(dbName);
expect(dec(await kv3.get('after'))).toBe('OK');
await kv3.close();
});
it('写入失败 → 内存索引不更新(原子性)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv = new KVStore(medium, 0);
await kv.open(dbName);
// 注入日志追加失败
const origAppend = medium.append!.bind(medium);
medium.append = async () => { throw new Error('disk full'); };
await expect(kv.put('x', enc('X'))).rejects.toMatchObject({ code: 'KV_LOG_ERROR' });
expect(await kv.exists('x')).toBe(false);
medium.append = origAppend;
await kv.put('y', enc('Y'));
expect(dec(await kv.get('y'))).toBe('Y');
await kv.close();
});
});
describe('KVStore — 维护与并发', () => {
it('clear 清空全部(保留库),重开为空', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium, 0);
await kv1.open(dbName);
await kv1.putMany({ a: enc('1'), b: enc('2') });
await kv1.checkpoint();
await kv1.put('c', enc('3'));
await kv1.clear();
expect(kv1.size()).toBe(0);
await kv1.close();
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
expect(kv2.size()).toBe(0);
await kv2.close();
});
it('repair 清理损坏快照与日志(损坏快照数据无法恢复,日志数据保留)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv1 = new KVStore(medium, 0);
await kv1.open(dbName);
await kv1.put('a', enc('A'));
await kv1.checkpoint(); // 快照成为 a 的唯一副本(日志已截断)
await kv1.put('b', enc('B')); // 日志中(seq=2
await kv1.close();
// 快照损坏 + 日志尾部损坏
const disk = new SharedMemoryBackend();
await disk.open(dbName);
const snap = new Uint8Array(await disk.read('__kv_snapshot') as ArrayBuffer);
snap[10] ^= 0xff;
await disk.write('__kv_snapshot', snap.buffer as ArrayBuffer);
const bad = encodeLogRecord(500, { junk: enc('J') });
bad[15] ^= 0xff;
const log = new Uint8Array(await disk.read('__kv_log') as ArrayBuffer);
const combined = new Uint8Array(log.byteLength + bad.byteLength);
combined.set(log, 0); combined.set(bad, log.byteLength);
await disk.write('__kv_log', combined.buffer as ArrayBuffer);
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
const discarded = await kv2.repair();
expect(discarded).toBeGreaterThan(0);
// 快照损坏(a 的唯一副本丢失)+ 损坏日志尾部被截断 → 日志数据 b 完整
expect(await kv2.exists('a')).toBe(false);
expect(dec(await kv2.get('b'))).toBe('B');
// repair 后介质上的损坏快照已清理
expect(await medium.exists('__kv_snapshot')).toBe(false);
await kv2.close();
});
it('并发写 + checkpoint 串行(无交错丢数据)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv = new KVStore(medium, 0);
await kv.open(dbName);
// 并发发起大量写入 + 中途 checkpoint
const writes = [] as Promise<void>[];
for (let i = 0; i < 200; i++) {
writes.push(kv.put(`k${i}`, enc(`v${i}`)));
if (i === 100) writes.push(kv.checkpoint());
}
await Promise.all(writes);
expect(kv.size()).toBe(200);
await kv.close();
// 重开验证全部恢复
const kv2 = new KVStore(medium, 0);
await kv2.open(dbName);
expect(kv2.size()).toBe(200);
expect(dec(await kv2.get('k150'))).toBe('v150');
await kv2.close();
});
it('自动 checkpoint 阈值触发(日志不无限增长)', async () => {
const dbName = uniqueDB();
const medium = new SharedMemoryBackend();
const kv = new KVStore(medium, 256); // 小阈值
await kv.open(dbName);
for (let i = 0; i < 50; i++) {
await kv.put(`k${i}`, enc(`value-${i}-`.repeat(10)));
}
// 自动 checkpoint 后日志应被截断
const log = await medium.read('__kv_log');
expect((log as ArrayBuffer).byteLength).toBeLessThan(4096);
await kv.close();
const kv2 = new KVStore(medium, 256);
await kv2.open(dbName);
expect(kv2.size()).toBe(50);
await kv2.close();
});
});
describe('KVStore — 编解码单元', () => {
it('encodeLogRecord / parseLogRecords 往返', () => {
const rec = encodeLogRecord(1, { a: enc('A'), b: enc('BB') }, ['del']);
const records: { seq: number; entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] }[] = [];
parseLogRecords(rec, (r) => records.push(r));
expect(records).toHaveLength(1);
expect(records[0].seq).toBe(1);
expect(records[0].entries).toHaveLength(3);
expect(dec(records[0].entries[0].value)).toBe('A');
expect(records[0].entries[2].op).toBe(KVLogOp.DELETE);
});
it('多记录日志顺序解析', () => {
const r1 = encodeLogRecord(1, { a: enc('1') });
const r2 = encodeLogRecord(2, { b: enc('2') });
const combined = new Uint8Array(r1.byteLength + r2.byteLength);
combined.set(r1, 0); combined.set(r2, r1.byteLength);
const seqs: number[] = [];
parseLogRecords(combined, (r) => seqs.push(r.seq));
expect(seqs).toEqual([1, 2]);
});
it('encodeSnapshot / decodeSnapshot 往返', () => {
const entries = new Map<string, ArrayBuffer>([['a', enc('A')], ['b', enc('BB')]]);
const bytes = encodeSnapshot(42, entries);
const snap = decodeSnapshot(bytes);
expect(snap).not.toBeNull();
expect(snap!.seq).toBe(42);
expect(dec(snap!.entries.get('a'))).toBe('A');
expect(dec(snap!.entries.get('b'))).toBe('BB');
});
it('损坏快照 decode 返回 null', () => {
const entries = new Map<string, ArrayBuffer>([['a', enc('A')]]);
const bytes = encodeSnapshot(1, entries);
bytes[10] ^= 0xff;
expect(decodeSnapshot(bytes)).toBeNull();
});
});
-158
View File
@@ -1,158 +0,0 @@
/**
* OPFSEngine mock FileSystemDirectoryHandle
* jsdom OPFS API使 mock
*/
import { OPFSEngine } from '../../src/engine/opfs';
import { createSchema } from '../../src/table/schema';
// Mock navigator.storage.getDirectory + FileSystemDirectoryHandle
function mockOPFS() {
const files = new Map<string, string>();
const dirMock = {
getDirectoryHandle: async (_name: string, _opts?: any) => dirMock as any,
getFileHandle: async (name: string, opts?: any) => {
if (opts?.create) {
return { createWritable: async () => ({ write: async (d: string) => { files.set(name, d); }, close: async () => {} }) };
}
if (!files.has(name)) throw new Error('Not found');
return { getFile: async () => ({ text: async () => files.get(name)!, arrayBuffer: async () => new ArrayBuffer(0) }) };
},
removeEntry: async (name: string) => { files.delete(name); },
};
// 为 entries() 添加可迭代接口
(dirMock as any).entries = () => ({
[Symbol.asyncIterator]: async function* () {
for (const [k] of files) yield [k];
},
});
// jsdom 环境下 navigator 可能已存在,直接设置 storage 属性
const nav = (globalThis as any).navigator || {};
nav.storage = { getDirectory: async () => dirMock };
(globalThis as any).navigator = nav;
return { files };
}
const userSchema = createSchema('users', {
id: { type: 'string', primaryKey: true },
name: { type: 'string', required: true },
age: { type: 'number', default: 0 },
});
describe('OPFSEngine', () => {
let engine: OPFSEngine;
let mocks: ReturnType<typeof mockOPFS>;
beforeEach(async () => {
mocks = mockOPFS();
engine = new OPFSEngine();
await engine.open('test-opfs', 1);
});
afterEach(async () => {
await engine.close();
});
// ---- 生命周期 ----
it('打开后 isOpen 返回 true', () => {
expect(engine.isOpen()).toBe(true);
});
it('关闭后 isOpen 返回 false', async () => {
await engine.close();
expect(engine.isOpen()).toBe(false);
});
// ---- 表管理 ----
it('创建表后 hasTable 返回 true', async () => {
await engine.createTable(userSchema);
expect(await engine.hasTable('users')).toBe(true);
});
it('获取表名列表', async () => {
await engine.createTable(userSchema);
const names = await engine.getTableNames();
expect(names).toContain('users');
});
it('删除表后 hasTable 返回 false', async () => {
await engine.createTable(userSchema);
await engine.dropTable('users');
expect(await engine.hasTable('users')).toBe(false);
});
// ---- CRUD ----
it('插入后立即可查询', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30 },
{ id: '2', name: 'Bob', age: 25 },
]);
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(2);
});
it('count 返回正确行数', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
expect(await engine.count('users')).toBe(1);
});
it('更新后数据变化', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice', age: 30 }]);
await engine.update('users', { table: 'users', where: { id: '1' } }, { age: 31 });
const rows = await engine.find('users', { table: 'users', where: { id: '1' } });
expect(rows[0].age).toBe(31);
});
it('删除后 count 减少', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }]);
await engine.delete('users', { table: 'users', where: { id: '1' } });
expect(await engine.count('users')).toBe(1);
});
// ---- 数据持久化(模拟重启) ----
it('close + reopen 后数据恢复', async () => {
// 写入数据
await engine.createTable(userSchema);
await engine.insert('users', [
{ id: '1', name: 'Alice', age: 30 },
{ id: '2', name: 'Bob', age: 25 },
]);
const beforeRows = await engine.find('users', { table: 'users' });
expect(beforeRows).toHaveLength(2);
// 关闭后重新打开(模拟重启)
await engine.close();
engine = new OPFSEngine();
await engine.open('test-opfs', 1);
// 数据应恢复
const afterRows = await engine.find('users', { table: 'users' });
expect(afterRows).toHaveLength(2);
expect(afterRows.map((r: any) => r.name).sort()).toEqual(['Alice', 'Bob']);
});
// ---- 事务 ----
it('事务 commit 后数据持久化', async () => {
await engine.createTable(userSchema);
await engine.beginTransaction();
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.commitTransaction();
expect(await engine.count('users')).toBe(1);
});
it('事务 rollback 后数据恢复', async () => {
await engine.createTable(userSchema);
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.beginTransaction();
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
await engine.rollbackTransaction();
expect(await engine.count('users')).toBe(1);
});
});
-1
View File
@@ -6,7 +6,6 @@
* Table API SQL query 14 * Table API SQL query 14
*/ */
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import 'fake-indexeddb/auto';
async function createDb(name: string): Promise<MetonaSqlark> { async function createDb(name: string): Promise<MetonaSqlark> {
const db = new MetonaSqlark({ name: `hooks-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`, mode: 'memory' }); const db = new MetonaSqlark({ name: `hooks-${name}-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`, mode: 'memory' });
-2
View File
@@ -6,7 +6,6 @@
import { HybridEngine } from '../../src/hybrid/index'; import { HybridEngine } from '../../src/hybrid/index';
import { createSchema } from '../../src/table/schema'; import { createSchema } from '../../src/table/schema';
import 'fake-indexeddb/auto';
let hybridCounter = 0; let hybridCounter = 0;
@@ -30,7 +29,6 @@ describe('HybridEngine', () => {
if (engine.isOpen()) { if (engine.isOpen()) {
await engine.close(); await engine.close();
} }
try { indexedDB.deleteDatabase(dbName); } catch {}
}); });
it('引擎名称', () => { it('引擎名称', () => {
-1
View File
@@ -9,7 +9,6 @@
* - refresh * - refresh
* - SQL * - SQL
*/ */
import 'fake-indexeddb/auto';
// ---- 最小 Vue mock(自包含:jest.mock 工厂不能引用外部变量) ---- // ---- 最小 Vue mock(自包含:jest.mock 工厂不能引用外部变量) ----
jest.mock('vue', () => { jest.mock('vue', () => {
+5 -2
View File
@@ -7,7 +7,10 @@
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import { parse } from '../src/sql/parser'; import { parse } from '../src/sql/parser';
import { createSchema } from '../src/table/schema'; import { createSchema } from '../src/table/schema';
import 'fake-indexeddb/auto';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let dbCounter = 0; let dbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -53,7 +56,7 @@ describe('v0.5.1 — 维护语句执行(Aria 引擎)', () => {
const db = new MetonaSqlark({ const db = new MetonaSqlark({
name: uniqueDB(), name: uniqueDB(),
mode: 'aria', mode: 'aria',
diskEngine: 'indexeddb', diskEngine: 'opfs',
aria: { checkpointInterval: 100000, walSyncMode: 'full' }, aria: { checkpointInterval: 100000, walSyncMode: 'full' },
}); });
await db.init(); await db.init();
+187
View File
@@ -0,0 +1,187 @@
/**
* migrateFromIndexedDB
*
* fake-indexeddb v0.5.x IndexedDB
* KV KVStoreEngine/schema/
*/
import 'fake-indexeddb/auto';
import { MetonaSqlark } from '../src/core';
import { migrateFromIndexedDB } from '../src/migration/index';
let counter = 0;
function uniqueDB(): string {
return `mig-${Date.now()}-${++counter}-${Math.random().toString(36).slice(2, 6)}`;
}
/** 构造旧 disk 模式库(IndexedDBEngine 布局:每表一个 objectStore + __metona_schema */
async function createLegacyDiskDB(dbName: string): Promise<void> {
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(dbName, 2);
request.onupgradeneeded = () => {
const db = request.result;
const users = db.createObjectStore('users', { keyPath: 'id' });
users.createIndex('idx_email', 'email', { unique: true });
db.createObjectStore('orders', { keyPath: 'id' });
db.createObjectStore('__metona_schema', { keyPath: 'name' });
};
request.onsuccess = () => {
const db = request.result;
const tx = db.transaction(['users', 'orders', '__metona_schema'], 'readwrite');
const users = tx.objectStore('users');
users.add({ id: '1', name: 'Alice', email: 'alice@x.com', age: 30 });
users.add({ id: '2', name: 'Bob', email: 'bob@x.com', age: 25 });
const orders = tx.objectStore('orders');
orders.add({ id: 'o1', user_id: '1', amount: 100 });
// 持久化 schemaIndexedDBEngine v0.3.2+ 布局)
tx.objectStore('__metona_schema').put({
name: 'users',
schema: JSON.stringify({
name: 'users',
columns: {
id: { type: 'string', primaryKey: true },
name: { type: 'string', required: true },
email: { type: 'string', unique: true, index: true },
age: { type: 'number', default: 0 },
},
}),
});
tx.objectStore('__metona_schema').put({
name: 'orders',
schema: JSON.stringify({
name: 'orders',
columns: {
id: { type: 'string', primaryKey: true },
user_id: { type: 'string' },
amount: { type: 'number' },
},
}),
});
tx.oncomplete = () => { db.close(); resolve(); };
tx.onerror = () => reject(tx.error);
};
request.onerror = () => reject(request.error);
});
}
/** 构造旧 aria 模式库(IndexedDBBackend 布局:单 data store + aria- 前缀) */
async function createLegacyAriaDB(dbName: string): Promise<void> {
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(`aria-${dbName}`, 1);
request.onupgradeneeded = () => {
request.result.createObjectStore('data');
};
request.onsuccess = () => {
const db = request.result;
const tx = db.transaction('data', 'readwrite');
const store = tx.objectStore('data');
store.put(new TextEncoder().encode('legacy').buffer, '__aria_keymeta_test');
tx.oncomplete = () => { db.close(); resolve(); };
tx.onerror = () => reject(tx.error);
};
request.onerror = () => reject(request.error);
});
}
describe('migrateFromIndexedDB', () => {
it('disk 模式旧库:schema + 数据完整迁移(含索引标记)', async () => {
const legacyName = uniqueDB();
await createLegacyDiskDB(legacyName);
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
expect(result.migratedTables.sort()).toEqual(['orders', 'users']);
expect(result.rowCount).toBe(3);
expect(await target.table('users').count()).toBe(2);
expect(await target.table('orders').count()).toBe(1);
// schema 保留(索引/唯一标记)
const schema = await target.getEngine().getTableSchema('users');
expect(schema!.columns.email.unique).toBe(true);
expect(schema!.columns.email.index).toBe(true);
// 数据正确
const alice = await target.table('users').select().where({ id: '1' }).execute();
expect(alice[0].email).toBe('alice@x.com');
// 重启后(持久化)数据保留
await target.close();
const target2 = new MetonaSqlark({ name: target.name, mode: 'disk' });
await target2.init();
expect(await target2.table('users').count()).toBe(2);
await target2.close();
});
it('无持久化 schema 的旧库:从数据推断', async () => {
const legacyName = uniqueDB();
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(legacyName, 1);
request.onupgradeneeded = () => {
request.result.createObjectStore('legacy_tbl', { keyPath: 'id' });
};
request.onsuccess = () => {
const db = request.result;
const tx = db.transaction('legacy_tbl', 'readwrite');
tx.objectStore('legacy_tbl').add({ id: 'a', name: 'X', age: 10 });
tx.oncomplete = () => { db.close(); resolve(); };
tx.onerror = () => reject(tx.error);
};
request.onerror = () => reject(request.error);
});
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
expect(result.migratedTables).toEqual(['legacy_tbl']);
expect(result.rowCount).toBe(1);
const rows = await target.table('legacy_tbl').select().execute();
expect(rows[0].name).toBe('X');
await target.close();
});
it('目标库已存在的表跳过(不覆盖)', async () => {
const legacyName = uniqueDB();
await createLegacyDiskDB(legacyName);
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
await target.defineTable('users', { id: { type: 'string', primaryKey: true } });
await target.table('users').insert({ id: 'keep' });
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
expect(result.skippedTables).toContain('users');
// 已存在的表未被覆盖
expect(await target.table('users').count()).toBe(1);
await target.close();
});
it('空旧库(无业务 store)→ 返回空迁移结果', async () => {
const legacyName = uniqueDB();
await new Promise<void>((resolve, reject) => {
const request = indexedDB.open(legacyName, 1);
request.onsuccess = () => { request.result.close(); resolve(); };
request.onerror = () => reject(request.error);
});
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
const result = await migrateFromIndexedDB({ dbName: legacyName, engine: 'disk', target });
expect(result.migratedTables).toEqual([]);
expect(result.rowCount).toBe(0);
await target.close();
});
it('aria 模式旧库(引擎私有格式)→ 明确抛错说明不支持', async () => {
const legacyName = uniqueDB();
await createLegacyAriaDB(legacyName);
const target = new MetonaSqlark({ name: uniqueDB(), mode: 'disk' });
await target.init();
await expect(migrateFromIndexedDB({ dbName: legacyName, engine: 'aria' as never, target }))
.rejects.toThrow('not supported');
await target.close();
});
});
+4 -111
View File
@@ -4,9 +4,7 @@
*/ */
import { MemoryEngine } from '../src/engine/memory'; import { MemoryEngine } from '../src/engine/memory';
import { IndexedDBEngine } from '../src/engine/indexeddb';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import 'fake-indexeddb/auto';
let idbCounter = 0; let idbCounter = 0;
@@ -51,75 +49,6 @@ describe('v0.1.14 幂等 init — MemoryEngine', () => {
}); });
// =================================================================== // ===================================================================
// IndexedDB flushToIDB 原子性
// ===================================================================
describe('v0.1.14 原子性 flush — IndexedDBEngine', () => {
let engine: IndexedDBEngine;
let dbName: string;
beforeEach(async () => {
dbName = uniqueDB();
engine = new IndexedDBEngine();
await engine.open(dbName, 1);
});
afterEach(async () => {
try { await engine.close(); } catch {}
try { indexedDB.deleteDatabase(dbName); } catch {}
});
it('事务 commit 后内存数据完整', async () => {
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
await engine.beginTransaction();
await engine.insert('users', [{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }]);
await engine.commitTransaction();
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(2);
});
it('事务 commit 后混合操作数据完整', async () => {
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.beginTransaction();
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
await engine.update('users', { table: 'users', where: { id: '1' } }, { name: 'Alice2' });
await engine.delete('users', { table: 'users', where: { id: '2' } });
await engine.commitTransaction();
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('Alice2');
});
it('事务回滚后数据完全恢复', async () => {
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.beginTransaction();
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
await engine.rollbackTransaction();
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('Alice');
});
it('事务内的 find 能读到未提交的变更', async () => {
await engine.createTable({ name: 'users', columns: { id: { type: 'string', primaryKey: true }, name: { type: 'string' } } });
await engine.insert('users', [{ id: '1', name: 'Alice' }]);
await engine.beginTransaction();
await engine.insert('users', [{ id: '2', name: 'Bob' }]);
// 事务内的 find 应能读到事务写入的数据
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(2);
await engine.rollbackTransaction();
});
});
// =================================================================== // ===================================================================
// Hybrid 引擎事务持久化 // Hybrid 引擎事务持久化
// =================================================================== // ===================================================================
@@ -130,7 +59,7 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
beforeEach(async () => { beforeEach(async () => {
dbName = uniqueDB(); dbName = uniqueDB();
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -140,7 +69,6 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
afterEach(async () => { afterEach(async () => {
await db.close(); await db.close();
try { indexedDB.deleteDatabase(dbName); } catch {}
}); });
it('Hybrid 事务 commit 后内存和磁盘一致', async () => { it('Hybrid 事务 commit 后内存和磁盘一致', async () => {
@@ -153,16 +81,11 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
const rows = await db.table('users').select().execute(); const rows = await db.table('users').select().execute();
expect(rows).toHaveLength(2); expect(rows).toHaveLength(2);
// 重新连接同一数据库(先关闭再创建新实例,但要先 deleteDatabase 让 fake-idb 干净) // 重新打开同一库:Hybrid 从磁盘(KVStore)恢复已提交事务数据
await db.close(); await db.close();
try { indexedDB.deleteDatabase(dbName); } catch {} const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
// 用新实例打开(验证内存+磁盘一致性:Hybrid 应该从磁盘恢复)
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' });
await db2.init(); await db2.init();
// 新数据库是空的(因为删了 IDB),Hybrid 从磁盘加载也是空的 expect(await db2.table('users').count()).toBe(2);
const names2 = await db2.getTableNames();
expect(names2).toHaveLength(0);
await db2.close(); await db2.close();
}); });
@@ -181,33 +104,3 @@ describe('v0.1.14 Hybrid 引擎事务持久化', () => {
expect(rows[0].id).toBe('1'); expect(rows[0].id).toBe('1');
}); });
}); });
// ===================================================================
// onversionchange 监听
// ===================================================================
describe('v0.1.14 onversionchange — IndexedDBEngine', () => {
it('open 后 onversionchange 回调已绑定', async () => {
const dbName = uniqueDB();
const engine = new IndexedDBEngine();
await engine.open(dbName, 1);
const rawDb = (engine as any).db;
expect(rawDb).toBeTruthy();
expect(typeof rawDb.onversionchange).toBe('function');
await engine.close();
try { indexedDB.deleteDatabase(dbName); } catch {}
});
it('close 后 db 引用和监听器均被清理', async () => {
const dbName = uniqueDB();
const engine = new IndexedDBEngine();
await engine.open(dbName, 1);
await engine.close();
const rawDb = (engine as any).db;
expect(rawDb).toBeNull();
try { indexedDB.deleteDatabase(dbName); } catch {}
});
});
+5 -2
View File
@@ -4,13 +4,16 @@
* *
* / / INSERT...SELECT / UNION / CREATE INDEX / EXISTS * / / INSERT...SELECT / UNION / CREATE INDEX / EXISTS
*/ */
import 'fake-indexeddb/auto';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import '../src/connection-manager'; import '../src/connection-manager';
import { parse, parseAll } from '../src/sql/parser'; import { parse, parseAll } from '../src/sql/parser';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
async function createDb(mode: 'memory' | 'disk' | 'aria' = 'memory') { async function createDb(mode: 'memory' | 'disk' | 'aria' = 'memory') {
const db = new MetonaSqlark({ name: `sql-ext-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: `sql-ext-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
+5 -2
View File
@@ -4,14 +4,17 @@
* *
* CASE WHEN / JOIN + / WAL * CASE WHEN / JOIN + / WAL
*/ */
import 'fake-indexeddb/auto';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import { parse } from '../src/sql/parser'; import { parse } from '../src/sql/parser';
import { WAL, type WALStore } from '../src/engine/aria/wal/log'; import { WAL, type WALStore } from '../src/engine/aria/wal/log';
import { WALRecordType } from '../src/engine/aria/types'; import { WALRecordType } from '../src/engine/aria/types';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
async function createDb(mode: 'memory' | 'aria' = 'memory') { async function createDb(mode: 'memory' | 'aria' = 'memory') {
const db = new MetonaSqlark({ name: `sql-ext2-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: `sql-ext2-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
+17 -18
View File
@@ -4,14 +4,13 @@
* *
* CASE WHEN WHERE/ / JOIN / * CASE WHEN WHERE/ / JOIN /
*/ */
import 'fake-indexeddb/auto';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
async function createDb(mode: 'memory' | 'hybrid' = 'memory', extra: Record<string, unknown> = {}) { async function createDb(mode: 'memory' | 'hybrid' = 'memory', extra: Record<string, unknown> = {}) {
const db = new MetonaSqlark({ const db = new MetonaSqlark({
name: `sql-ext3-${mode}-${Date.now()}-${Math.random()}`, name: `sql-ext3-${mode}-${Date.now()}-${Math.random()}`,
mode, mode,
diskEngine: 'indexeddb', diskEngine: 'opfs',
...extra, ...extra,
}); });
await db.init(); await db.init();
@@ -249,13 +248,13 @@ describe('[v0.3.2] 多标签页同步', () => {
test('SQL 写语句广播表变更,其他标签页订阅收到 external 事件', async () => { test('SQL 写语句广播表变更,其他标签页订阅收到 external 事件', async () => {
// 先建表(DDL 版本升级会触发其他标签页 onversionchange 关闭连接,故先建表再开第二连接) // 先建表(DDL 版本升级会触发其他标签页 onversionchange 关闭连接,故先建表再开第二连接)
const setup = new MetonaSqlark({ name: 'mt-a', mode: 'hybrid', diskEngine: 'indexeddb' }); const setup = new MetonaSqlark({ name: 'mt-a', mode: 'hybrid', diskEngine: 'opfs' });
await setup.init(); await setup.init();
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } }); await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
await setup.close(); await setup.close();
const dbA = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbA = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
const dbB = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbB = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
await dbA.init(); await dbA.init();
await dbB.init(); await dbB.init();
@@ -274,13 +273,13 @@ describe('[v0.3.2] 多标签页同步', () => {
}); });
test('Hybrid 标签页收到广播后内存重载(读到其他标签页的新数据)', async () => { test('Hybrid 标签页收到广播后内存重载(读到其他标签页的新数据)', async () => {
const setup = new MetonaSqlark({ name: 'mt-b', mode: 'hybrid', diskEngine: 'indexeddb' }); const setup = new MetonaSqlark({ name: 'mt-b', mode: 'hybrid', diskEngine: 'opfs' });
await setup.init(); await setup.init();
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } }); await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
await setup.close(); await setup.close();
const dbA = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbA = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
const dbB = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbB = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
await dbA.init(); await dbA.init();
await dbB.init(); await dbB.init();
@@ -305,13 +304,13 @@ describe('[v0.3.2] 多标签页同步', () => {
}); });
test('未启用 multiTabSync 不广播', async () => { test('未启用 multiTabSync 不广播', async () => {
const setup = new MetonaSqlark({ name: 'mt-c', mode: 'hybrid', diskEngine: 'indexeddb' }); const setup = new MetonaSqlark({ name: 'mt-c', mode: 'hybrid', diskEngine: 'opfs' });
await setup.init(); await setup.init();
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } }); await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
await setup.close(); await setup.close();
const dbA = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' }); const dbA = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'opfs' });
const dbB = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbB = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
await dbA.init(); await dbA.init();
await dbB.init(); await dbB.init();
@@ -327,13 +326,13 @@ describe('[v0.3.2] 多标签页同步', () => {
}); });
test('Table API 写入也广播', async () => { test('Table API 写入也广播', async () => {
const setup = new MetonaSqlark({ name: 'mt-d', mode: 'hybrid', diskEngine: 'indexeddb' }); const setup = new MetonaSqlark({ name: 'mt-d', mode: 'hybrid', diskEngine: 'opfs' });
await setup.init(); await setup.init();
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } }); await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
await setup.close(); await setup.close();
const dbA = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbA = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
const dbB = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true }); const dbB = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'opfs', multiTabSync: true });
await dbA.init(); await dbA.init();
await dbB.init(); await dbB.init();
@@ -354,7 +353,7 @@ describe('[v0.3.2] 多标签页同步', () => {
describe('[v0.3.2] IndexedDB reopen schema 持久化', () => { describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
test('close 后重新 open 表结构与数据完整', async () => { test('close 后重新 open 表结构与数据完整', async () => {
const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'indexeddb' }); const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'opfs' });
await setup.init(); await setup.init();
await setup.defineTable('t', { await setup.defineTable('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -365,7 +364,7 @@ describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
await setup.close(); await setup.close();
// 重新打开(模拟页面刷新) // 重新打开(模拟页面刷新)
const db = new MetonaSqlark({ name: 'reopen-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: 'reopen-a', version: 2, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
const schema = await db.getEngine().getTableSchema('t'); const schema = await db.getEngine().getTableSchema('t');
@@ -383,7 +382,7 @@ describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
}); });
test('reopen 后 UPDATE 全列生效', async () => { test('reopen 后 UPDATE 全列生效', async () => {
const setup = new MetonaSqlark({ name: 'reopen-b', mode: 'hybrid', diskEngine: 'indexeddb' }); const setup = new MetonaSqlark({ name: 'reopen-b', mode: 'hybrid', diskEngine: 'opfs' });
await setup.init(); await setup.init();
await setup.defineTable('t', { await setup.defineTable('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -393,7 +392,7 @@ describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
await setup.query(`INSERT INTO t VALUES ('1', 'Alice', 42)`); await setup.query(`INSERT INTO t VALUES ('1', 'Alice', 42)`);
await setup.close(); await setup.close();
const db = new MetonaSqlark({ name: 'reopen-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: 'reopen-b', version: 2, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.query(`UPDATE t SET name = 'Renamed', v = 99 WHERE id = '1'`); await db.query(`UPDATE t SET name = 'Renamed', v = 99 WHERE id = '1'`);
+2 -5
View File
@@ -4,7 +4,6 @@
*/ */
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import 'fake-indexeddb/auto';
let idbCounter = 0; let idbCounter = 0;
@@ -128,7 +127,7 @@ describe('v0.1.13 事务回滚 — Disk 模式 (IndexedDB)', () => {
beforeEach(async () => { beforeEach(async () => {
dbName = `tx-disk-${++idbCounter}`; dbName = `tx-disk-${++idbCounter}`;
db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' }); db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -138,7 +137,6 @@ describe('v0.1.13 事务回滚 — Disk 模式 (IndexedDB)', () => {
afterEach(async () => { afterEach(async () => {
await db.close(); await db.close();
try { indexedDB.deleteDatabase(dbName); } catch {}
}); });
it('commit: 事务成功数据持久化', async () => { it('commit: 事务成功数据持久化', async () => {
@@ -175,7 +173,7 @@ describe('v0.1.13 事务回滚 — Hybrid 模式', () => {
beforeEach(async () => { beforeEach(async () => {
dbName = `tx-hybrid-${++idbCounter}`; dbName = `tx-hybrid-${++idbCounter}`;
db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -185,7 +183,6 @@ describe('v0.1.13 事务回滚 — Hybrid 模式', () => {
afterEach(async () => { afterEach(async () => {
await db.close(); await db.close();
try { indexedDB.deleteDatabase(dbName); } catch {}
}); });
it('commit: 事务成功', async () => { it('commit: 事务成功', async () => {
+7 -3
View File
@@ -18,13 +18,17 @@ import type { SSTableMeta } from '../src/engine/aria/types';
import type { MetonaPlugin } from '../src/constants'; import type { MetonaPlugin } from '../src/constants';
import { createSchema } from '../src/table/schema'; import { createSchema } from '../src/table/schema';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// P0-1: 版本号统一 // P0-1: 版本号统一
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
describe('[v0.2.5] P0-1: 版本号统一', () => { describe('[v0.2.5] P0-1: 版本号统一', () => {
test('VERSION 常量为当前版本(0.5.1', () => { test('VERSION 常量为当前版本(0.6.0', () => {
expect(VERSION).toBe('0.5.1'); expect(VERSION).toBe('0.6.0');
}); });
}); });
@@ -41,7 +45,7 @@ describe('[v0.2.5] P0-2: AriaEngine OPFS 后端映射', () => {
}); });
test('mode=aria + diskEngine=indexeddb 时应使用 indexeddb 后端', () => { test('mode=aria + diskEngine=indexeddb 时应使用 indexeddb 后端', () => {
const db = new MetonaSqlark({ name: `test-idb-map-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: `test-idb-map-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'aria', diskEngine: 'opfs' });
expect(db).toBeDefined(); expect(db).toBeDefined();
}); });
}); });
+6 -3
View File
@@ -12,7 +12,6 @@
* - P1-9: Savepoint MVCC + rollback * - P1-9: Savepoint MVCC + rollback
*/ */
import 'fake-indexeddb/auto';
import { VERSION } from '../src/constants'; import { VERSION } from '../src/constants';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import { AriaEngine } from '../src/engine/aria/index'; import { AriaEngine } from '../src/engine/aria/index';
@@ -20,13 +19,17 @@ import { MemoryEngine } from '../src/engine/memory';
import { WAL } from '../src/engine/aria/wal/log'; import { WAL } from '../src/engine/aria/wal/log';
import { tokenize } from '../src/sql/lexer'; import { tokenize } from '../src/sql/lexer';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// P0-1: Aria WAL DROP_TABLE 崩溃恢复 // P0-1: Aria WAL DROP_TABLE 崩溃恢复
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
describe('[v0.3.3] P0-1: WAL DROP_TABLE 崩溃恢复', () => { describe('[v0.3.3] P0-1: WAL DROP_TABLE 崩溃恢复', () => {
const mkEngine = async (name: string): Promise<AriaEngine> => { const mkEngine = async (name: string): Promise<AriaEngine> => {
const e = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const e = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await e.open(name, 1); await e.open(name, 1);
return e; return e;
}; };
@@ -400,7 +403,7 @@ describe('[v0.3.3] P1-9: Savepoint + MVCC 一致性', () => {
describe('[v0.3.3] 端到端', () => { describe('[v0.3.3] 端到端', () => {
test('全部修复点可共存于 MetonaSqlark API', async () => { test('全部修复点可共存于 MetonaSqlark API', async () => {
expect(VERSION).toBe('0.5.1'); expect(VERSION).toBe('0.6.0');
const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' }); const db = new MetonaSqlark({ name: `e2e-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`, mode: 'memory' });
await db.init(); await db.init();
await db.defineTable('users', { await db.defineTable('users', {
+5 -2
View File
@@ -6,10 +6,13 @@
* - B-4: COUNT(DISTINCT) + NULLS FIRST/LAST * - B-4: COUNT(DISTINCT) + NULLS FIRST/LAST
*/ */
import 'fake-indexeddb/auto';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import { AriaEngine } from '../src/engine/aria/index'; import { AriaEngine } from '../src/engine/aria/index';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
const uniqueName = (prefix: string): string => const uniqueName = (prefix: string): string =>
`${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
@@ -366,7 +369,7 @@ describe('[v0.4.0] B-4: COUNT(DISTINCT) + NULLS FIRST/LAST', () => {
}); });
test('Aria $in 查询去重(IN 子查询含重复值不返回重复行)', async () => { test('Aria $in 查询去重(IN 子查询含重复值不返回重复行)', async () => {
const db = new MetonaSqlark({ name: uniqueName('indup'), mode: 'aria', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: uniqueName('indup'), mode: 'aria', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.getEngine().clearAll(); await db.getEngine().clearAll();
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } }); await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
+80 -117
View File
@@ -15,11 +15,15 @@ import { AriaEngine } from '../src/engine/aria/index';
import { createSchema } from '../src/table/schema'; import { createSchema } from '../src/table/schema';
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder'; import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
import { SSTableReader } from '../src/engine/aria/index/sstable'; import { SSTableReader } from '../src/engine/aria/index/sstable';
import { IndexedDBEngine } from '../src/engine/indexeddb';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import { IndexedDBBackend } from '../src/engine/aria/store/backend'; import { OPFSBackend } from '../src/engine/aria/store/opfs_backend';
import { KVStoreEngine } from '../src/engine/kvstore_engine';
import { SharedMemoryBackend } from '../src/engine/kvstore/shared_memory_medium';
import type { SSTableMeta } from '../src/engine/aria/types'; import type { SSTableMeta } from '../src/engine/aria/types';
import 'fake-indexeddb/auto';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -133,7 +137,7 @@ describe('P0-1a — SSTableReader 残缺数据防御', () => {
describe('P0-1b — AriaEngine 打开时完整性校验', () => { describe('P0-1b — AriaEngine 打开时完整性校验', () => {
it('meta 引用残缺文件 → 打开跳过损坏 SSTable 不崩溃,库可用', async () => { it('meta 引用残缺文件 → 打开跳过损坏 SSTable 不崩溃,库可用', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('users', { await engine.createTable(createSchema('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -145,17 +149,18 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
await (engine as any).lsm.flush(); await (engine as any).lsm.flush();
await engine.close(); await engine.close();
// 篡改存储:把第一个 sst_ 文件写成残缺内容(meta 仍引用它) // 篡改存储:把第一个 SSTable 的第一个页面写成残缺内容(meta 仍引用它)
const backend = new IndexedDBBackend(); const backend = new OPFSBackend();
await backend.open(dbName); await backend.open(dbName);
const sstKeys = (await backend.listKeys()) const metas = JSON.parse(new TextDecoder().decode(
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_')); await backend.read('__aria_lsm_meta') as ArrayBuffer)) as { id: number; pageIds: number[] }[];
expect(sstKeys.length).toBeGreaterThan(0); expect(metas.length).toBeGreaterThan(0);
await backend.write(sstKeys[0], new TextEncoder().encode('truncated-garbage').buffer); const pageId = metas[0].pageIds[0];
await backend.write(`pg_${pageId}`, new TextEncoder().encode('truncated-garbage').buffer);
await backend.close(); await backend.close();
// 重开:不得崩溃(此前抛 RangeError 打不开库) // 重开:不得崩溃(此前抛 RangeError 打不开库)
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined(); await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
expect(engine2.isOpen()).toBe(true); expect(engine2.isOpen()).toBe(true);
// schema 完整,损坏的 SSTable 已被清理 // schema 完整,损坏的 SSTable 已被清理
@@ -165,19 +170,21 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
await engine2.close(); await engine2.close();
// 清理 meta 已验证:损坏文件被删除 // 清理 meta 已验证:损坏文件被删除
const backend2 = new IndexedDBBackend(); const backend2 = new OPFSBackend();
await backend2.open(dbName); await backend2.open(dbName);
const remaining = (await backend2.listKeys()) const remaining = (await backend2.listKeys()).filter((k) => k.startsWith('pg_'));
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_'));
const metaRaw = await backend2.read('__aria_lsm_meta'); const metaRaw = await backend2.read('__aria_lsm_meta');
const metaList = JSON.parse(new TextDecoder().decode(metaRaw ?? new Uint8Array())) as { id: number }[]; const metaList = JSON.parse(new TextDecoder().decode(metaRaw ?? new Uint8Array())) as { pageIds?: number[] }[];
expect(remaining.length).toBe(metaList.length); // 无孤儿文件 / 无悬空 meta const livePages = new Set<number>();
for (const m of metaList) if (m.pageIds) for (const pid of m.pageIds) livePages.add(pid);
const orphan = remaining.filter((k) => !livePages.has(Number(k.slice(3))));
expect(orphan.length).toBe(0); // 无孤儿页面
await backend2.close(); await backend2.close();
}); });
it('meta 引用缺失文件 → 打开清理 meta 不崩溃', async () => { it('meta 引用缺失页面 → 打开清理 meta 不崩溃', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -187,15 +194,14 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
await (engine as any).lsm.flush(); await (engine as any).lsm.flush();
await engine.close(); await engine.close();
// 删除全部 sst_ 文件(保留 meta → 悬空引用) // 删除全部 pg_ 页面(保留 meta → 悬空引用)
const backend = new IndexedDBBackend(); const backend = new OPFSBackend();
await backend.open(dbName); await backend.open(dbName);
const sstKeys = (await backend.listKeys()) const pgKeys = (await backend.listKeys()).filter((k) => k.startsWith('pg_'));
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_')); await backend.deleteMany(pgKeys);
await backend.deleteMany(sstKeys);
await backend.close(); await backend.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 1024 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined(); await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
expect(engine2.isOpen()).toBe(true); expect(engine2.isOpen()).toBe(true);
expect(await engine2.getTableNames()).toEqual(['t']); expect(await engine2.getTableNames()).toEqual(['t']);
@@ -210,7 +216,7 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
describe('P0-1c / P1-4 — WAL 原子性与恢复', () => { describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
it('IndexedDBBackend.writeMany/deleteMany 单事务原子批量操作', async () => { it('IndexedDBBackend.writeMany/deleteMany 单事务原子批量操作', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const be = new IndexedDBBackend(); const be = new OPFSBackend();
await be.open(dbName); await be.open(dbName);
await be.writeMany({ await be.writeMany({
a: new TextEncoder().encode('1').buffer, a: new TextEncoder().encode('1').buffer,
@@ -226,7 +232,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
it('P1-4: 连续快速写入 200 条 → close+重开 数据完整(实测曾丢 4 条)', async () => { it('P1-4: 连续快速写入 200 条 → close+重开 数据完整(实测曾丢 4 条)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const engine = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -237,7 +243,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
} }
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const engine2 = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
expect(await engine2.count('t')).toBe(200); expect(await engine2.count('t')).toBe(200);
await engine2.close(); await engine2.close();
@@ -245,7 +251,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
it('P1-4: count 键丢失(模拟崩溃竞态)→ 按 key 扫描恢复不丢记录', async () => { it('P1-4: count 键丢失(模拟崩溃竞态)→ 按 key 扫描恢复不丢记录', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const engine = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -256,12 +262,12 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
} }
// 模拟异常退出(不 close):直接删掉 count 键,制造 count 与记录不一致 // 模拟异常退出(不 close):直接删掉 count 键,制造 count 与记录不一致
const backend = new IndexedDBBackend(); const backend = new OPFSBackend();
await backend.open(dbName); await backend.open(dbName);
await backend.delete('__wal_count'); await backend.delete('__wal_count');
await backend.close(); await backend.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const engine2 = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
expect(await engine2.count('t')).toBe(20); expect(await engine2.count('t')).toBe(20);
await engine2.close(); await engine2.close();
@@ -275,7 +281,7 @@ describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
describe('P1-5 — AriaEngine.close 截断 WAL', () => { describe('P1-5 — AriaEngine.close 截断 WAL', () => {
it('close 后 WAL 记录键全部清空(不无限重放)', async () => { it('close 后 WAL 记录键全部清空(不无限重放)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', walSyncMode: 'full' }); const engine = new AriaEngine({ storageBackend: 'opfs', walSyncMode: 'full' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -284,7 +290,7 @@ describe('P1-5 — AriaEngine.close 截断 WAL', () => {
await engine.insert('t', [{ id: '1', v: 1 }, { id: '2', v: 2 }]); await engine.insert('t', [{ id: '1', v: 1 }, { id: '2', v: 2 }]);
await engine.close(); await engine.close();
const backend = new IndexedDBBackend(); const backend = new OPFSBackend();
await backend.open(dbName); await backend.open(dbName);
// __wal_count 计数键合法保留(作为 append 序号分配器),WAL 记录键必须清空 // __wal_count 计数键合法保留(作为 append 序号分配器),WAL 记录键必须清空
const walKeys = (await backend.listKeys()) const walKeys = (await backend.listKeys())
@@ -294,65 +300,6 @@ describe('P1-5 — AriaEngine.close 截断 WAL', () => {
}); });
}); });
// ===================================================================
// P0-2 / P0-3 / P2-8: IndexedDB 版本管理与参数校验
// ===================================================================
describe('P0-2 / P0-3 — IndexedDB 版本管理', () => {
it('P0-2: 建表提升版本后以旧 version 重开成功(VersionError 自适应)', async () => {
const dbName = uniqueDB();
const engine = new IndexedDBEngine();
await engine.open(dbName, 1);
await engine.createTable({ name: 'a', columns: { id: { type: 'string', primaryKey: true } } });
await engine.createTable({ name: 'b', columns: { id: { type: 'string', primaryKey: true } } });
await engine.createTable({ name: 'c', columns: { id: { type: 'string', primaryKey: true } } });
await engine.close();
// 用旧版本 1 重开:此前必报 IDB_OPEN_ERROR(VersionError)
const engine2 = new IndexedDBEngine();
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
const names = await engine2.getTableNames();
expect(names).toContain('a');
expect(names).toContain('b');
expect(names).toContain('c');
await engine2.close();
});
it('P0-2: MetonaSqlark hybrid 二次启动不报 VersionErrorMarkLite 场景)', async () => {
const dbName = uniqueDB();
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 1 });
await db1.init();
await db1.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
await db1.table('users').insert({ id: '1', name: 'Alice' });
await db1.close();
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 1 });
await expect(db2.init()).resolves.toBeUndefined();
expect(await db2.table('users').count()).toBe(1);
await db2.close();
});
it('P0-3: IndexedDBEngine open(version=0) 归一化为 1,不抛 TypeError', async () => {
const dbName = uniqueDB();
const engine = new IndexedDBEngine();
await expect(engine.open(dbName, 0)).resolves.toBeUndefined();
expect(engine.isOpen()).toBe(true);
await engine.createTable({ name: 't', columns: { id: { type: 'string', primaryKey: true } } });
await engine.insert('t', [{ id: '1' }]);
expect(await engine.find('t', { table: 't' })).toHaveLength(1);
await engine.close();
});
it('P0-3: MetonaSqlark version=0 初始化正常', async () => {
const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 });
await expect(db.init()).resolves.toBeUndefined();
await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
await db.table('t').insert({ id: '1' });
await db.close();
});
});
// =================================================================== // ===================================================================
// P1-6: 引擎错误包装 DatabaseError // P1-6: 引擎错误包装 DatabaseError
// =================================================================== // ===================================================================
@@ -362,24 +309,36 @@ describe('P1-6 — 引擎内部错误统一包装 DatabaseError', () => {
// 构造一个无法打开的 backend 场景:直接调用 openInternal 模拟底层异常不可行, // 构造一个无法打开的 backend 场景:直接调用 openInternal 模拟底层异常不可行,
// 这里验证损坏库重开时抛的是 DatabaseError 而非原生错误(不崩溃路径已由 P0-1b 覆盖) // 这里验证损坏库重开时抛的是 DatabaseError 而非原生错误(不崩溃路径已由 P0-1b 覆盖)
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb' }); const engine = new AriaEngine({ storageBackend: 'opfs' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } })); await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
await engine.insert('t', [{ id: '1' }]); await engine.insert('t', [{ id: '1' }]);
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb' }); const engine2 = new AriaEngine({ storageBackend: 'opfs' });
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined(); await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
expect(engine2.isOpen()).toBe(true); expect(engine2.isOpen()).toBe(true);
await engine2.close(); await engine2.close();
}); });
it('open 未初始化 IndexedDB 环境时抛 DatabaseError 而非原生错误', async () => { it('底层介质故障时抛 DatabaseError 而非原生错误', async () => {
const dbName = uniqueDB(); const failing = {
const engine = new IndexedDBEngine(); open: async () => { throw new Error('disk failure'); },
await engine.open(dbName, 1); close: async () => {},
expect(engine.isOpen()).toBe(true); isOpen: () => false,
await engine.close(); read: async () => null,
write: async () => { throw new Error('disk failure'); },
append: async () => {},
writeMany: async () => {},
delete: async () => {},
deleteMany: async () => {},
listKeys: async () => [],
exists: async () => false,
clear: async () => {},
};
const engine = new KVStoreEngine(failing as never);
// 介质故障如实传播(不吞错),AriaEngine 路径由 ARIA_OPEN_ERROR 包装(上方测试覆盖)
await expect(engine.open(uniqueDB(), 1)).rejects.toThrow('disk failure');
}); });
}); });
@@ -393,7 +352,7 @@ describe('P2-7 — 迁移版本持久化', () => {
const runs: number[] = []; const runs: number[] = [];
// version: 0 表示"无 schema 起点"migration 1 可执行 // version: 0 表示"无 schema 起点"migration 1 可执行
const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 }); const db1 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs', version: 0 });
await db1.init(); await db1.init();
db1.addMigration(1, async () => { runs.push(1); }); db1.addMigration(1, async () => { runs.push(1); });
db1.addMigration(2, async () => { runs.push(2); }); db1.addMigration(2, async () => { runs.push(2); });
@@ -402,7 +361,7 @@ describe('P2-7 — 迁移版本持久化', () => {
await db1.close(); await db1.close();
// 重启:version 又重置为 config.version=0(此前会重跑 migration 1/2 // 重启:version 又重置为 config.version=0(此前会重跑 migration 1/2
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb', version: 0 }); const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs', version: 0 });
await db2.init(); await db2.init();
db2.addMigration(1, async () => { runs.push(1); }); db2.addMigration(1, async () => { runs.push(1); });
db2.addMigration(2, async () => { runs.push(2); }); db2.addMigration(2, async () => { runs.push(2); });
@@ -440,35 +399,39 @@ describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
it('AriaEngine.repair 清理损坏 SSTable 后引擎可用', async () => { it('AriaEngine.repair 清理损坏 SSTable 后引擎可用', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', memtableSizeThreshold: 4096 }); const engine = new AriaEngine({ storageBackend: 'opfs', memtableSizeThreshold: 64 * 1024 * 1024 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
v: { type: 'number' }, v: { type: 'number' },
})); }));
for (let i = 0; i < 100; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]); for (let i = 0; i < 50; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
await (engine as any).lsm.flush();
for (let i = 50; i < 100; i++) await engine.insert('t', [{ id: `k${i}`, v: i }]);
await (engine as any).lsm.flush(); await (engine as any).lsm.flush();
// 篡改一个 sst 文件 // 篡改一个 SSTable 的页面文件
const backend = new IndexedDBBackend(); const backend = new OPFSBackend();
await backend.open(dbName); await backend.open(dbName);
const sstKeys = (await backend.listKeys()) const metas = JSON.parse(new TextDecoder().decode(
.filter((k) => k.startsWith('sst_') && !k.startsWith('sst_idx_')); await backend.read('__aria_lsm_meta') as ArrayBuffer)) as { id: number; pageIds: number[] }[];
expect(sstKeys.length).toBeGreaterThan(0); expect(metas.length).toBeGreaterThan(0);
await backend.write(sstKeys[0], new TextEncoder().encode('garbage').buffer); const victimPage = metas[0].pageIds[0];
const raw = new Uint8Array(await backend.read(`pg_${victimPage}`) as ArrayBuffer);
raw[50] ^= 0xff;
await backend.write(`pg_${victimPage}`, raw.buffer as ArrayBuffer);
await backend.close(); await backend.close();
await expect(engine.repair()).resolves.toBeUndefined(); // repair:校验清理损坏 SSTable,其余数据可用
const rows = await engine.find('t', { table: 't' }); await (engine as any).repair();
expect(Array.isArray(rows)).toBe(true); expect(await engine.count('t')).toBeGreaterThan(0);
expect(rows.length).toBeLessThanOrEqual(100); expect(await engine.count('t')).toBeLessThan(100);
expect(engine.isOpen()).toBe(true);
await engine.close(); await engine.close();
}); });
it('MetonaSqlark.clearAll 统一接口(hybrid', async () => { it('MetonaSqlark.clearAll 统一接口(hybrid', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('t', { id: { type: 'string', primaryKey: true } }); await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
await db.table('t').insert({ id: '1' }); await db.table('t').insert({ id: '1' });
@@ -479,7 +442,7 @@ describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
it('MetonaSqlark.repair 统一接口(aria', async () => { it('MetonaSqlark.repair 统一接口(aria', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'aria', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'aria', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } }); await db.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
await db.table('t').insert({ id: '1', v: 42 }); await db.table('t').insert({ id: '1', v: 42 });
@@ -490,7 +453,7 @@ describe('P2-9 — 统一自愈接口 repair / clearAll', () => {
it('MetonaSqlark.repair 统一接口(hybrid', async () => { it('MetonaSqlark.repair 统一接口(hybrid', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('t', { id: { type: 'string', primaryKey: true } }); await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
await db.table('t').insert({ id: '1' }); await db.table('t').insert({ id: '1' });
+24 -21
View File
@@ -13,7 +13,10 @@ import { createSchema } from '../src/table/schema';
import { MemoryEngine } from '../src/engine/memory'; import { MemoryEngine } from '../src/engine/memory';
import { HybridEngine } from '../src/hybrid/index'; import { HybridEngine } from '../src/hybrid/index';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import 'fake-indexeddb/auto';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -53,7 +56,7 @@ describe('P0-A — Compaction 缓存独立性', () => {
it('多层级 compaction 后数据仍完整且可重开', async () => { it('多层级 compaction 后数据仍完整且可重开', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
bufferPoolPages: 8, // 32KB 缓存 bufferPoolPages: 8, // 32KB 缓存
memtableSizeThreshold: 16 * 1024, memtableSizeThreshold: 16 * 1024,
checkpointInterval: 100000, checkpointInterval: 100000,
@@ -74,7 +77,7 @@ describe('P0-A — Compaction 缓存独立性', () => {
// 重开:数据完整(meta/文件未被 compaction 破坏) // 重开:数据完整(meta/文件未被 compaction 破坏)
const engine2 = new AriaEngine({ const engine2 = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
bufferPoolPages: 8, bufferPoolPages: 8,
memtableSizeThreshold: 16 * 1024, memtableSizeThreshold: 16 * 1024,
checkpointInterval: 100000, checkpointInterval: 100000,
@@ -92,7 +95,7 @@ describe('P0-A — Compaction 缓存独立性', () => {
describe('P1-A — 二级索引恢复', () => { describe('P1-A — 二级索引恢复', () => {
it('Aria 重开后二级索引可用(索引 LSM 持久化恢复)', async () => { it('Aria 重开后二级索引可用(索引 LSM 持久化恢复)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('users', { await engine.createTable(createSchema('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -107,7 +110,7 @@ describe('P1-A — 二级索引恢复', () => {
await engine.close(); await engine.close();
// 重开:二级索引 LSM 应自动恢复(修复前为空 → 索引查询回退全表,createIndex 也静默跳过) // 重开:二级索引 LSM 应自动恢复(修复前为空 → 索引查询回退全表,createIndex 也静默跳过)
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
// 强断言:索引 LSM 真实存在且数据完整(防止"索引缺失静默回退全表扫描"的假通过) // 强断言:索引 LSM 真实存在且数据完整(防止"索引缺失静默回退全表扫描"的假通过)
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:email'); const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:email');
@@ -124,7 +127,7 @@ describe('P1-A — 二级索引恢复', () => {
it('WAL 恢复后二级索引与主数据一致(崩溃前索引未更新)', async () => { it('WAL 恢复后二级索引与主数据一致(崩溃前索引未更新)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('users', { await engine.createTable(createSchema('users', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -138,7 +141,7 @@ describe('P1-A — 二级索引恢复', () => {
await (engine as any).backend.close(); await (engine as any).backend.close();
(engine as any).opened = false; (engine as any).opened = false;
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
// 强断言:索引 LSM 已恢复且包含 WAL 回放的行(崩溃前索引未更新,恢复后必须重建) // 强断言:索引 LSM 已恢复且包含 WAL 回放的行(崩溃前索引未更新,恢复后必须重建)
const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:city'); const idxLsm = (engine2 as any).secondaryIndexes.get('users:idx:city');
@@ -154,7 +157,7 @@ describe('P1-A — 二级索引恢复', () => {
it('createIndex 重开后仍可新建(schema 标记恢复后不阻塞)', async () => { it('createIndex 重开后仍可新建(schema 标记恢复后不阻塞)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -163,7 +166,7 @@ describe('P1-A — 二级索引恢复', () => {
await engine.insert('t', [{ id: '1', name: 'A' }]); await engine.insert('t', [{ id: '1', name: 'A' }]);
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
await engine2.createIndex('t', 'name'); // 修复前 schema 无标记时正常;此处验证无标记场景 await engine2.createIndex('t', 'name'); // 修复前 schema 无标记时正常;此处验证无标记场景
const byName = await engine2.find('t', { table: 't', where: { name: 'A' } }); const byName = await engine2.find('t', { table: 't', where: { name: 'A' } });
@@ -177,9 +180,9 @@ describe('P1-A — 二级索引恢复', () => {
// =================================================================== // ===================================================================
describe('P1-B — ALTER TABLE 持久化', () => { describe('P1-B — ALTER TABLE 持久化', () => {
it('IndexedDBEngine DROP COLUMN 后重启不复活', async () => { it('KVStoreEngine DROP COLUMN 后重启不复活', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('t', { await db.defineTable('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -194,7 +197,7 @@ describe('P1-B — ALTER TABLE 持久化', () => {
await db.close(); await db.close();
// 重启:schema 不复活,列定义已持久化 // 重启:schema 不复活,列定义已持久化
const db2 = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'indexeddb' }); const db2 = new MetonaSqlark({ name: dbName, mode: 'disk', diskEngine: 'opfs' });
await db2.init(); await db2.init();
const schema2 = await db2.getEngine().getTableSchema('t'); const schema2 = await db2.getEngine().getTableSchema('t');
expect(schema2!.columns.old_col).toBeUndefined(); expect(schema2!.columns.old_col).toBeUndefined();
@@ -204,13 +207,13 @@ describe('P1-B — ALTER TABLE 持久化', () => {
it('HybridEngine ADD COLUMN 后重启保留', async () => { it('HybridEngine ADD COLUMN 后重启保留', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('t', { id: { type: 'string', primaryKey: true } }); await db.defineTable('t', { id: { type: 'string', primaryKey: true } });
await db.query('ALTER TABLE t ADD COLUMN phone STRING'); await db.query('ALTER TABLE t ADD COLUMN phone STRING');
await db.close(); await db.close();
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db2.init(); await db2.init();
const schema2 = await db2.getEngine().getTableSchema('t'); const schema2 = await db2.getEngine().getTableSchema('t');
expect(schema2!.columns.phone).toBeDefined(); expect(schema2!.columns.phone).toBeDefined();
@@ -219,13 +222,13 @@ describe('P1-B — ALTER TABLE 持久化', () => {
}); });
// =================================================================== // ===================================================================
// P1-C: IndexedDB 事务内 DDL // P1-C: KVStore 事务内 DDL
// =================================================================== // ===================================================================
describe('P1-C — IndexedDB 事务内 DDL', () => { describe('P1-C — KVStore 事务内 DDL', () => {
it('事务内建表 → commit 后磁盘一致(重启表存在且可查)', async () => { it('事务内建表 → commit 后磁盘一致(重启表存在且可查)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('base', { id: { type: 'string', primaryKey: true } }); await db.defineTable('base', { id: { type: 'string', primaryKey: true } });
await db.table('base').insert({ id: '1' }); await db.table('base').insert({ id: '1' });
@@ -240,7 +243,7 @@ describe('P1-C — IndexedDB 事务内 DDL', () => {
expect(await db.table('created_in_tx').count()).toBe(1); expect(await db.table('created_in_tx').count()).toBe(1);
await db.close(); await db.close();
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db2.init(); await db2.init();
expect(await db2.table('created_in_tx').count()).toBe(1); expect(await db2.table('created_in_tx').count()).toBe(1);
expect(await db2.table('base').count()).toBe(2); expect(await db2.table('base').count()).toBe(2);
@@ -249,7 +252,7 @@ describe('P1-C — IndexedDB 事务内 DDL', () => {
it('事务内删表 → commit 后磁盘一致(重启无幽灵表)', async () => { it('事务内删表 → commit 后磁盘一致(重启无幽灵表)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db.init(); await db.init();
await db.defineTable('ghost', { id: { type: 'string', primaryKey: true } }); await db.defineTable('ghost', { id: { type: 'string', primaryKey: true } });
await db.table('ghost').insert({ id: '1' }); await db.table('ghost').insert({ id: '1' });
@@ -259,7 +262,7 @@ describe('P1-C — IndexedDB 事务内 DDL', () => {
}); });
await db.close(); await db.close();
const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'indexeddb' }); const db2 = new MetonaSqlark({ name: dbName, mode: 'hybrid', diskEngine: 'opfs' });
await db2.init(); await db2.init();
const names = await db2.getTableNames(); const names = await db2.getTableNames();
expect(names).not.toContain('ghost'); expect(names).not.toContain('ghost');
@@ -300,7 +303,7 @@ describe('全模式抽查 — 生命周期与元数据', () => {
it('AriaEngine repair 幂等', async () => { it('AriaEngine repair 幂等', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb' }); const engine = new AriaEngine({ storageBackend: 'opfs' });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } })); await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
await engine.insert('t', [{ id: '1' }]); await engine.insert('t', [{ id: '1' }]);
+10 -90
View File
@@ -9,10 +9,12 @@
import { AriaEngine } from '../src/engine/aria/index'; import { AriaEngine } from '../src/engine/aria/index';
import { createSchema } from '../src/table/schema'; import { createSchema } from '../src/table/schema';
import { OPFSEngine } from '../src/engine/opfs';
import { MemTable } from '../src/engine/aria/index/memtable'; import { MemTable } from '../src/engine/aria/index/memtable';
import { MetonaSqlark } from '../src/core'; import { MetonaSqlark } from '../src/core';
import 'fake-indexeddb/auto';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -73,7 +75,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
it('事务中触发 checkpoint → 崩溃恢复不丢事务数据', async () => { it('事务中触发 checkpoint → 崩溃恢复不丢事务数据', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 2, // 每 2 次操作即触发 checkpoint(事务中途) checkpointInterval: 2, // 每 2 次操作即触发 checkpoint(事务中途)
}); });
await engine.open(dbName, 1); await engine.open(dbName, 1);
@@ -91,7 +93,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
await (engine as any).backend.close(); await (engine as any).backend.close();
(engine as any).opened = false; (engine as any).opened = false;
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
// 修复前:checkpoint 截断了事务的 WAL 记录 → 恢复后数据丢失 // 修复前:checkpoint 截断了事务的 WAL 记录 → 恢复后数据丢失
expect(await engine2.count('t')).toBe(2); expect(await engine2.count('t')).toBe(2);
@@ -101,7 +103,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
it('事务回滚后再 checkpoint 正常截断', async () => { it('事务回滚后再 checkpoint 正常截断', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
checkpointInterval: 1, checkpointInterval: 1,
}); });
await engine.open(dbName, 1); await engine.open(dbName, 1);
@@ -115,7 +117,7 @@ describe('P0 — 事务进行中 checkpoint 不得截断 WAL', () => {
await (engine as any).backend.close(); await (engine as any).backend.close();
(engine as any).opened = false; (engine as any).opened = false;
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
expect(await engine2.count('t')).toBe(1); expect(await engine2.count('t')).toBe(1);
await engine2.close(); await engine2.close();
@@ -168,7 +170,7 @@ describe('P1 — Aria DDL 索引清理', () => {
it('DROP_TABLE 崩溃恢复同样清理索引', async () => { it('DROP_TABLE 崩溃恢复同样清理索引', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('t', { await engine.createTable(createSchema('t', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -180,7 +182,7 @@ describe('P1 — Aria DDL 索引清理', () => {
await (engine as any).backend.close(); await (engine as any).backend.close();
(engine as any).opened = false; (engine as any).opened = false;
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
// 表不存在,且无索引残留 // 表不存在,且无索引残留
expect(await engine2.hasTable('t')).toBe(false); expect(await engine2.hasTable('t')).toBe(false);
@@ -189,88 +191,6 @@ describe('P1 — Aria DDL 索引清理', () => {
}); });
}); });
// ===================================================================
// P1: OPFS 持久化与并发
// ===================================================================
describe('P1 — OPFS 生产加固', () => {
it('并发写不丢数据(内存快照总是最新,多写内容一致)', async () => {
// 模拟真实 I/O 延迟:内存写同步、持久化异步 → 并发写内容均基于最新内存快照
const files = mockOPFS(undefined, {
ioDelay: 10,
writeDelayFor: (data) => (data.length < 40 ? 30 : 5),
});
const engine = new OPFSEngine();
await engine.open('opfs-race', 1);
await engine.createTable(createSchema('users', {
id: { type: 'string', primaryKey: true },
name: { type: 'string' },
}));
// 两个写并发:内存同步写保证两个快照一致 → 无论完成顺序文件内容完整
const p1 = engine.insert('users', [{ id: '1', name: 'A' }]);
const p2 = engine.insert('users', [{ id: '2', name: 'B' }]);
await Promise.all([p1, p2]);
const rows = await engine.find('users', { table: 'users' });
expect(rows).toHaveLength(2);
// 文件内容完整(重启不丢)
expect(files.get('users.json')).toContain('"B"');
await engine.close();
});
it('空表重启后保留(schema 持久化)', async () => {
mockOPFS();
const e1 = new OPFSEngine();
await e1.open('opfs-schema', 1);
await e1.createTable(createSchema('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', index: true },
}));
await e1.close();
// 重启:空表也应恢复(修复前空表消失)+ 索引标记恢复
const e2 = new OPFSEngine();
await e2.open('opfs-schema', 1);
expect(await e2.hasTable('users')).toBe(true);
const schema = await e2.getTableSchema('users');
expect(schema!.columns.email.index).toBe(true);
await e2.close();
});
it('有数据重启后索引查询可用', async () => {
mockOPFS();
const e1 = new OPFSEngine();
await e1.open('opfs-idx', 1);
await e1.createTable(createSchema('users', {
id: { type: 'string', primaryKey: true },
email: { type: 'string', index: true },
}));
await e1.insert('users', [{ id: '1', email: 'a@x.com' }]);
await e1.close();
const e2 = new OPFSEngine();
await e2.open('opfs-idx', 1);
const byEmail = await e2.find('users', { table: 'users', where: { email: 'a@x.com' } });
expect(byEmail).toHaveLength(1);
await e2.close();
});
it('dropTable 后重启无幽灵表', async () => {
mockOPFS();
const e1 = new OPFSEngine();
await e1.open('opfs-drop', 1);
await e1.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
await e1.insert('t', [{ id: '1' }]);
await e1.dropTable('t');
await e1.close();
const e2 = new OPFSEngine();
await e2.open('opfs-drop', 1);
expect(await e2.hasTable('t')).toBe(false);
expect(await e2.getTableNames()).toEqual([]);
await e2.close();
});
});
// =================================================================== // ===================================================================
// P2: Aria 内部状态 // P2: Aria 内部状态
// =================================================================== // ===================================================================
+5 -46
View File
@@ -10,8 +10,10 @@
import { AriaEngine } from '../src/engine/aria/index'; import { AriaEngine } from '../src/engine/aria/index';
import { createSchema } from '../src/table/schema'; import { createSchema } from '../src/table/schema';
import { LSM } from '../src/engine/aria/index/lsm'; import { LSM } from '../src/engine/aria/index/lsm';
import { OPFSEngine } from '../src/engine/opfs';
import 'fake-indexeddb/auto'; import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -56,7 +58,7 @@ describe('P0 — close 后无残留后台任务', () => {
it('close 后立即 reopen 不被旧后台任务污染', async () => { it('close 后立即 reopen 不被旧后台任务污染', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
bufferPoolPages: 4, bufferPoolPages: 4,
memtableSizeThreshold: 32 * 1024, memtableSizeThreshold: 32 * 1024,
checkpointInterval: 100000, checkpointInterval: 100000,
@@ -162,47 +164,4 @@ describe('P1 — 提交顺序与 close 等待', () => {
await engine.close(); await engine.close();
}); });
it('OPFS close 等待挂起写完成(文件完整)', async () => {
// mock OPFS(跨实例共享)
const files = new Map<string, string>();
const dirMock = {
getDirectoryHandle: async (_n: string, _o?: any) => dirMock as any,
getFileHandle: async (name: string, opts?: any) => {
if (opts?.create) {
return {
createWritable: async () => ({
write: async (d: string) => {
// 模拟慢 I/O
await new Promise((r) => setTimeout(r, 30));
files.set(name, d);
},
close: async () => {},
}),
};
}
if (!files.has(name)) throw new Error('Not found');
return { getFile: async () => ({ text: async () => files.get(name)!, arrayBuffer: async () => new ArrayBuffer(0) }) };
},
removeEntry: async (n: string) => { files.delete(n); },
};
(dirMock as any).entries = () => ({
[Symbol.asyncIterator]: async function* () {
for (const [k] of files) yield [k];
},
}); });
const nav = (globalThis as any).navigator || {};
nav.storage = { getDirectory: async () => dirMock };
(globalThis as any).navigator = nav;
const engine = new OPFSEngine();
await engine.open('opfs-close', 1);
await engine.createTable(createSchema('t', { id: { type: 'string', primaryKey: true } }));
// 写入不等待(写 I/O 30ms 挂起)
const p = engine.insert('t', [{ id: '1' }]);
// 立即 close:必须等待挂起写完成
await engine.close();
await p;
// 文件完整(修复前 close 不等写 → 可能读到旧/空文件)
expect(files.get('t.json')).toContain('"1"');
});
});
+8 -5
View File
@@ -13,7 +13,10 @@ import { createSchema } from '../src/table/schema';
import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder'; import { SSTableBuilder } from '../src/engine/aria/index/sstable_builder';
import { SSTableReader } from '../src/engine/aria/index/sstable'; import { SSTableReader } from '../src/engine/aria/index/sstable';
import type { SSTableMeta } from '../src/engine/aria/types'; import type { SSTableMeta } from '../src/engine/aria/types';
import 'fake-indexeddb/auto';
import { installOPFSMock } from './helpers/opfs-mock';
beforeEach(() => { installOPFSMock(new Map()); });
let idbCounter = 0; let idbCounter = 0;
function uniqueDB(): string { function uniqueDB(): string {
@@ -164,7 +167,7 @@ describe('P1 — v1 旧格式兼容', () => {
describe('P0 — 大内容端到端(aria 模式)', () => { describe('P0 — 大内容端到端(aria 模式)', () => {
it('300KB 中文写入 → close → reopen 数据完整(修复前打开必崩)', async () => { it('300KB 中文写入 → close → reopen 数据完整(修复前打开必崩)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine.open(dbName, 1); await engine.open(dbName, 1);
await engine.createTable(createSchema('notes', { await engine.createTable(createSchema('notes', {
id: { type: 'string', primaryKey: true }, id: { type: 'string', primaryKey: true },
@@ -182,7 +185,7 @@ describe('P0 — 大内容端到端(aria 模式)', () => {
await engine.close(); await engine.close();
// 修复前:打开解析半写文件 → ARIA_OPEN_ERROR // 修复前:打开解析半写文件 → ARIA_OPEN_ERROR
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await expect(engine2.open(dbName, 1)).resolves.toBeUndefined(); await expect(engine2.open(dbName, 1)).resolves.toBeUndefined();
const rows = await engine2.find('notes', { table: 'notes' }); const rows = await engine2.find('notes', { table: 'notes' });
expect(rows).toHaveLength(2); expect(rows).toHaveLength(2);
@@ -196,7 +199,7 @@ describe('P0 — 大内容端到端(aria 模式)', () => {
it('多行 >64KB 内容批量写入重开完整(含索引列)', async () => { it('多行 >64KB 内容批量写入重开完整(含索引列)', async () => {
const dbName = uniqueDB(); const dbName = uniqueDB();
const engine = new AriaEngine({ const engine = new AriaEngine({
storageBackend: 'indexeddb', storageBackend: 'opfs',
memtableSizeThreshold: 32 * 1024, memtableSizeThreshold: 32 * 1024,
checkpointInterval: 100000, checkpointInterval: 100000,
}); });
@@ -214,7 +217,7 @@ describe('P0 — 大内容端到端(aria 模式)', () => {
} }
await engine.close(); await engine.close();
const engine2 = new AriaEngine({ storageBackend: 'indexeddb', checkpointInterval: 100000 }); const engine2 = new AriaEngine({ storageBackend: 'opfs', checkpointInterval: 100000 });
await engine2.open(dbName, 1); await engine2.open(dbName, 1);
expect(await engine2.count('docs')).toBe(10); expect(await engine2.count('docs')).toBe(10);
const evens = await engine2.find('docs', { table: 'docs', where: { tag: 'even' } }); const evens = await engine2.find('docs', { table: 'docs', where: { tag: 'even' } });