feat: v0.6.1 — AriaEngine 可选自研 KVStore 后端(storageBackend: 'kv')+ KVStore APPEND 日志类型 + 10 个 aria+kv 集成测试 + 文档全量同步
This commit is contained in:
@@ -29,6 +29,12 @@ All notable changes to MetonaSqlark will be documented in this file.
|
||||
|
||||
### Added
|
||||
|
||||
- **AriaEngine 可选自研 KVStore 后端**(`storageBackend: 'kv'` / `diskEngine: 'kv'`):
|
||||
- aria 完全跑在自研存储栈上(KVStore 日志+快照),不再依赖浏览器 OPFS API(Node/测试用 SharedMemory 介质)
|
||||
- KVStore 新增 APPEND 日志操作类型:aria WAL 分片追加 O(chunk) 高效,恢复按序拼接,checkpoint 快照含最终值
|
||||
- writeMany/deleteMany 经 KVStore 单日志记录真原子(此前 OPFS 逐文件写)
|
||||
- aria 崩溃恢复 = KVStore 双层恢复 + WAL 重放;二级索引/页面化兼容(kv 后端 SSTable 整 value 存储)
|
||||
- 10 个 aria+kv 集成测试(后端单元/CRUD 持久化/WAL 追加恢复/checkpoint 混合/索引/大数据崩溃/高层 API 事务级联)
|
||||
- 27 个生产异常场景测试(`tests/production-abnormal.test.ts`):
|
||||
- KVStore 边界:空 value/空 key、1MB 大 value、中文/emoji/冒号/超长 key、未 open、
|
||||
重复 open/close 幂等、clear 后写、1 万小 key、写失败后继续可用
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
<p align="center">
|
||||
<img src="https://img.shields.io/badge/version-0.6.1-blue?style=flat-square" alt="version">
|
||||
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
||||
<img src="https://img.shields.io/badge/coverage-89.3%25-brightgreen?style=flat-square" alt="coverage">
|
||||
<img src="https://img.shields.io/badge/tests-1049%20passed-success?style=flat-square" alt="tests">
|
||||
<img src="https://img.shields.io/badge/coverage-89.4%25-brightgreen?style=flat-square" alt="coverage">
|
||||
<img src="https://img.shields.io/badge/tests-1064%20passed-success?style=flat-square" alt="tests">
|
||||
</p>
|
||||
|
||||
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。
|
||||
@@ -20,7 +20,7 @@
|
||||
- 🛠 **维护语句 SQL 入口** — `EXPLAIN`/`ANALYZE`/`REINDEX`/`VACUUM`/`SAVEPOINT` 原生 SQL 支持(v0.5.1 补齐)
|
||||
- 🛡 **输入校验全覆盖** — `maxLength`/`min`/`max` 约束、类型检查、必填验证
|
||||
- 💾 **多引擎架构** — Memory / **KVStore**(自研 KV 引擎)/ OPFS / Hybrid(write-through) / Aria 五种模式(v0.6.0: IndexedDB 完全移除)
|
||||
- 💾 **KVStore 自研 KV 引擎** — 日志结构化事务存储:多 key 原子写(putMany/deleteMany 单记录原子追加)、快照 checkpoint、崩溃两阶段恢复、CRC-32 自愈;替代 IndexedDB(v0.6.0)
|
||||
- 💾 **KVStore 自研 KV 引擎** — 日志结构化事务存储:多 key 原子写(putMany/deleteMany 单记录原子追加)、快照 checkpoint、崩溃两阶段恢复、CRC-32 自愈、APPEND 追加写入(aria WAL 分片用);替代 IndexedDB(v0.6.0)且可作 AriaEngine 后端(v0.6.1)
|
||||
- 🔄 **旧库一键迁移** — `migrateFromIndexedDB()` 把旧 IndexedDB 数据(schema/索引/行)导入新引擎(v0.6.0)
|
||||
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTS(v0.3.0)+ CASE WHEN/哈希连接/组提交(v0.3.1)+ 多标签页同步(v0.3.2)
|
||||
- 🚰 **流式查询** — `queryStream`/`stream()` 逐行回调,Aria LSM 惰性扫描不物化结果集(v0.4.0)
|
||||
@@ -36,7 +36,7 @@
|
||||
- 🛡 **级联环路保护** — A→B→A 循环引用 + CASCADE 不无限递归(visited 保护,全引擎)(v0.6.1)
|
||||
- ⚛️ **多表事务真原子** — 事务 commit / 级联更新合并单条日志记录(崩溃无部分提交)(v0.6.1)
|
||||
- 🌐 **浏览器兼容** — Chrome 102+ / Firefox 111+ / Safari 15.2+ / Edge 102+ / Node.js 16+
|
||||
- 🧪 **1049 测试 · 89.3% 覆盖率** — 65 套件 + 12 个 Playwright 真实 Chromium e2e(含崩溃注入),生产级质量保证
|
||||
- 🧪 **1064 测试 · 89.4% 覆盖率** — 66 套件 + 12 个 Playwright 真实 Chromium e2e(含崩溃注入),生产级质量保证
|
||||
|
||||
---
|
||||
|
||||
@@ -111,7 +111,7 @@ import { MetonaSqlark } from '@metona-team/metona-sqlark';
|
||||
const db = await MetonaSqlark.create({
|
||||
name: 'my-app',
|
||||
mode: 'hybrid', // 'memory' | 'disk' | 'hybrid' | 'aria'
|
||||
diskEngine: 'opfs', // 'opfs' | 'memory'(v0.6.0: IndexedDB 已移除)
|
||||
diskEngine: 'opfs', // 'opfs' | 'kv'(自研 KVStore 后端)| 'memory'(v0.6.0: IndexedDB 已移除)
|
||||
});
|
||||
|
||||
// 定义表 — 支持外键级联
|
||||
@@ -218,7 +218,7 @@ await db2.disconnect(); // 引用计数 -1
|
||||
|------|------|------|------|
|
||||
| `name` | `string` | `'metona-sqlark'` | 数据库名称 |
|
||||
| `mode` | `'memory' \| 'disk' \| 'hybrid' \| 'aria'` | `'hybrid'` | 存储模式 🆕 aria |
|
||||
| `diskEngine` | `'opfs' \| 'memory'` | `'opfs'` | 磁盘引擎(aria 模式下为存储后端;v0.6.0: IndexedDB 已移除) |
|
||||
| `diskEngine` | `'opfs' \| 'memory' \| 'kv'` | `'opfs'` | 磁盘引擎(aria 模式下为存储后端;'kv' = 自研 KVStore 后端 v0.6.1) |
|
||||
| `version` | `number` | `1` | 版本号 |
|
||||
| `maxRowsPerQuery` | `number` | `0` | 查询结果行数上限(0=不限制)✅ v0.2.5 生效 |
|
||||
| `debug` | `boolean` | `false` | 调试模式,输出详细日志 🆕 |
|
||||
@@ -363,7 +363,7 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
||||
- **适用**: 需要内存速度 + 磁盘持久化的混合场景
|
||||
|
||||
### Aria 模式
|
||||
- **环境**: 所有浏览器(后端可选 OPFS / Memory)
|
||||
- **环境**: 所有浏览器(后端可选 OPFS / **KVStore**(自研,v0.6.1)/ Memory)
|
||||
- **限制**: Memory 后端重启丢失;OPFS 后端需 OPFS 支持(Chrome 102+ / Firefox 111+ / Safari 15.2+);同一库仅允许一个标签页打开(Web Locks 独占锁,第二个标签页抛 `ARIA_LOCKED`)
|
||||
- **能力**: LSM-Tree 存储引擎、SSTable 4KB 页面化物理存储(OPFS 后端默认启用,BufferPool LRU 缓存)、二级索引(跨重启恢复)、MVCC 事务、WAL 分片文件原子写入崩溃恢复(真追加 + 空洞检测)、ON UPDATE/DELETE 外键级联、Bloom Filter、**全库 AES-GCM 透明加密**(v0.5.0)、LZ4 压缩、Savepoint、EXPLAIN、ANALYZE、REINDEX、VACUUM、`repair()` 自愈(含孤儿页面清理)
|
||||
- **适用**: 大规模数据分析、需要自研引擎可控性的高级场景
|
||||
@@ -379,7 +379,7 @@ AriaEngine 是内置的页面式存储引擎,对标 SQLite 的设计理念:
|
||||
const db = await MetonaSqlark.create({
|
||||
name: 'my-app',
|
||||
mode: 'aria', // 🆕 自研引擎模式
|
||||
diskEngine: 'opfs', // 'opfs' | 'memory'
|
||||
diskEngine: 'opfs', // 'opfs' | 'kv'(自研 KVStore 后端)| 'memory'
|
||||
aria: { // 🆕 AriaEngine 专属配置透传(v0.5.0)
|
||||
walSyncMode: 'full', // WAL 同步模式
|
||||
encryption: { password: 'my-password' }, // 全库 AES-GCM 加密
|
||||
@@ -453,9 +453,9 @@ npm run typecheck # 类型检查
|
||||
|
||||
| 指标 | 数值 |
|
||||
|------|------|
|
||||
| 测试用例 | 1049 |
|
||||
| 测试套件 | 65(+12 Playwright e2e) |
|
||||
| 行覆盖率 | 89.3% |
|
||||
| 测试用例 | 1064 |
|
||||
| 测试套件 | 66(+12 Playwright e2e) |
|
||||
| 行覆盖率 | 89.4% |
|
||||
| SQL 关键字 | 72 |
|
||||
| 存储引擎 | 5(Memory / **KVStore** / OPFS / Hybrid / **Aria**) |
|
||||
|
||||
|
||||
Vendored
+113
-4
@@ -1124,14 +1124,17 @@ var KVLogOp;
|
||||
(function (KVLogOp) {
|
||||
KVLogOp[KVLogOp["PUT"] = 1] = "PUT";
|
||||
KVLogOp[KVLogOp["DELETE"] = 2] = "DELETE";
|
||||
/** v0.6.1: 追加写入(value 拼接语义,恢复时按序 concat;aria WAL 分片用) */
|
||||
KVLogOp[KVLogOp["APPEND"] = 3] = "APPEND";
|
||||
})(KVLogOp || (KVLogOp = {}));
|
||||
/**
|
||||
* 编码一条日志记录。
|
||||
* @param seq 日志序号
|
||||
* @param puts key → value 写入条目
|
||||
* @param deletes 删除 key 列表
|
||||
* @param appends key → 追加块列表(APPEND 语义,恢复时拼接)
|
||||
*/
|
||||
function encodeLogRecord(seq, puts, deletes = []) {
|
||||
function encodeLogRecord(seq, puts, deletes = [], appends = {}) {
|
||||
const encoder = new TextEncoder();
|
||||
const entries = [];
|
||||
for (const [key, value] of Object.entries(puts)) {
|
||||
@@ -1140,6 +1143,9 @@ function encodeLogRecord(seq, puts, deletes = []) {
|
||||
for (const key of deletes) {
|
||||
entries.push({ op: KVLogOp.DELETE, key, value: new ArrayBuffer(0) });
|
||||
}
|
||||
for (const [key, value] of Object.entries(appends)) {
|
||||
entries.push({ op: KVLogOp.APPEND, key, value });
|
||||
}
|
||||
// 预编码 key 字节,计算总长度
|
||||
const entryBytes = [];
|
||||
let total = 4 + 4 + 4; // recordLen + seq + entryCount
|
||||
@@ -1545,6 +1551,18 @@ class KVStore {
|
||||
await this.appendRecord({}, keys);
|
||||
});
|
||||
}
|
||||
/**
|
||||
* v0.6.1: 追加写入(value 拼接语义)— aria WAL 分片等追加型数据用。
|
||||
* 日志记录 APPEND 类型(O(chunk) 高效),恢复时按 seq 顺序拼接,
|
||||
* checkpoint 后快照含最终值。崩溃时该次追加全有或全无(单记录原子)。
|
||||
*/
|
||||
async appendValue(key, chunk) {
|
||||
if (chunk.byteLength === 0)
|
||||
return;
|
||||
await this.enqueue(async () => {
|
||||
await this.appendRecord({}, [], { [key]: chunk });
|
||||
});
|
||||
}
|
||||
// =======================================================================
|
||||
// 维护
|
||||
// =======================================================================
|
||||
@@ -1617,9 +1635,9 @@ class KVStore {
|
||||
return run;
|
||||
}
|
||||
/** 追加一条日志记录并更新内存索引(队列内调用,无并发) */
|
||||
async appendRecord(puts, deletes) {
|
||||
async appendRecord(puts, deletes, appends = {}) {
|
||||
this.seq++;
|
||||
const record = encodeLogRecord(this.seq, puts, deletes);
|
||||
const record = encodeLogRecord(this.seq, puts, deletes, appends);
|
||||
try {
|
||||
// 日志追加:介质 append(真追加)或回退读+拼+写
|
||||
const data = record.buffer.slice(record.byteOffset, record.byteOffset + record.byteLength);
|
||||
@@ -1652,6 +1670,18 @@ class KVStore {
|
||||
for (const key of deletes) {
|
||||
this.index.delete(key);
|
||||
}
|
||||
for (const [key, chunk] of Object.entries(appends)) {
|
||||
const existing = this.index.get(key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + chunk.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(chunk), existing.byteLength);
|
||||
this.index.set(key, combined.buffer);
|
||||
}
|
||||
else {
|
||||
this.index.set(key, chunk);
|
||||
}
|
||||
}
|
||||
this.logBytes += record.byteLength;
|
||||
// 自动 checkpoint(日志超阈值)
|
||||
if (this.checkpointThreshold > 0 && this.logBytes >= this.checkpointThreshold) {
|
||||
@@ -1675,6 +1705,18 @@ class KVStore {
|
||||
if (e.op === KVLogOp.PUT) {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
else if (e.op === KVLogOp.APPEND) {
|
||||
const existing = this.index.get(e.key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + e.value.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(e.value), existing.byteLength);
|
||||
this.index.set(e.key, combined.buffer);
|
||||
}
|
||||
else {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
}
|
||||
else {
|
||||
this.index.delete(e.key);
|
||||
}
|
||||
@@ -4822,6 +4864,68 @@ class MemoryBackend {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* KVStoreBackend — 基于自研 KVStore 的 AriaEngine 存储后端
|
||||
* @module engine/aria/store/kvstore_backend
|
||||
*
|
||||
* v0.6.1: AriaEngine 可选后端(storageBackend: 'kv'),完全跑在自研 KVStore 上:
|
||||
* - write/read/delete/listKeys/exists/clear → KVStore 直接映射
|
||||
* - append → KVStore.appendValue(日志 APPEND 类型,O(chunk) 高效,aria WAL 分片用)
|
||||
* - writeMany/deleteMany → KVStore.putMany/deleteMany(单日志记录真原子,
|
||||
* 此前 OPFS 逐文件写靠空洞检测兜底,KV 后端原生原子)
|
||||
* - 崩溃恢复:KVStore 快照+日志恢复 → aria 打开重放自己的 WAL(双层恢复)
|
||||
*
|
||||
* 介质:浏览器 OPFS(KVStore 默认)或 Node SharedMemory——aria 不再依赖浏览器 OPFS API。
|
||||
*/
|
||||
class KVStoreBackend {
|
||||
constructor(medium, checkpointThreshold) {
|
||||
this.kv = new KVStore(medium, checkpointThreshold);
|
||||
}
|
||||
/** 底层 KVStore(测试/诊断用) */
|
||||
getKV() {
|
||||
return this.kv;
|
||||
}
|
||||
async open(name) {
|
||||
await this.kv.open(name);
|
||||
}
|
||||
async close() {
|
||||
await this.kv.close();
|
||||
}
|
||||
isOpen() {
|
||||
return this.kv.isOpen();
|
||||
}
|
||||
async read(key) {
|
||||
return this.kv.get(key);
|
||||
}
|
||||
async write(key, data) {
|
||||
await this.kv.put(key, data);
|
||||
}
|
||||
/** 追加写入(KVStore APPEND 日志,O(chunk)) */
|
||||
async append(key, data) {
|
||||
await this.kv.appendValue(key, data);
|
||||
}
|
||||
/** 多 key 原子写入(单日志记录) */
|
||||
async writeMany(entries) {
|
||||
await this.kv.putMany(entries);
|
||||
}
|
||||
async delete(key) {
|
||||
await this.kv.delete(key);
|
||||
}
|
||||
/** 多 key 原子删除(单日志记录) */
|
||||
async deleteMany(keys) {
|
||||
await this.kv.deleteMany(keys);
|
||||
}
|
||||
async listKeys() {
|
||||
return this.kv.listKeys();
|
||||
}
|
||||
async exists(key) {
|
||||
return this.kv.exists(key);
|
||||
}
|
||||
async clear() {
|
||||
await this.kv.clear();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* AriaEngine Crypto — 页面级 AES-GCM 加密
|
||||
* @module engine/aria/crypto
|
||||
@@ -5957,6 +6061,10 @@ class AriaEngine {
|
||||
if (this.config.storageBackend === 'opfs') {
|
||||
baseBackend = new OPFSBackend();
|
||||
}
|
||||
else if (this.config.storageBackend === 'kv') {
|
||||
// v0.6.1: 自研 KVStore 后端(aria 完全跑在自研存储栈上,不依赖浏览器 OPFS)
|
||||
baseBackend = new KVStoreBackend();
|
||||
}
|
||||
else {
|
||||
baseBackend = new MemoryBackend();
|
||||
}
|
||||
@@ -11594,8 +11702,9 @@ class MetonaSqlark {
|
||||
return new KVStoreEngine();
|
||||
case 'aria':
|
||||
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
|
||||
// v0.6.1: diskEngine 'kv' → 自研 KVStore 后端
|
||||
return new AriaEngine({
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : 'opfs',
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : diskEngine === 'kv' ? 'kv' : 'opfs',
|
||||
...(this.config.aria ?? {}),
|
||||
});
|
||||
case 'hybrid':
|
||||
|
||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+3
-3
@@ -18,7 +18,7 @@ interface AriaEngineConfig {
|
||||
/** 是否启用页面压缩(默认 false) */
|
||||
compression?: boolean;
|
||||
/** 存储后端 */
|
||||
storageBackend?: 'opfs' | 'memory';
|
||||
storageBackend?: 'opfs' | 'memory' | 'kv';
|
||||
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB) */
|
||||
walSizeThreshold?: number;
|
||||
/** 最大内存预算(MB,默认 64) */
|
||||
@@ -45,8 +45,8 @@ interface AriaEngineConfig {
|
||||
*/
|
||||
/** 存储模式 */
|
||||
type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
||||
/** 磁盘引擎类型(v0.6.0: IndexedDB 已移除,'memory' 供 aria 内存后端) */
|
||||
type DiskEngine = 'opfs' | 'memory';
|
||||
/** 磁盘引擎类型(v0.6.0: IndexedDB 已移除;'memory' 供 aria 内存后端;'kv' 供 aria 自研 KVStore 后端) */
|
||||
type DiskEngine = 'opfs' | 'memory' | 'kv';
|
||||
/** 字段数据类型 */
|
||||
type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json';
|
||||
/** 列定义 */
|
||||
|
||||
Vendored
+113
-4
@@ -1120,14 +1120,17 @@ var KVLogOp;
|
||||
(function (KVLogOp) {
|
||||
KVLogOp[KVLogOp["PUT"] = 1] = "PUT";
|
||||
KVLogOp[KVLogOp["DELETE"] = 2] = "DELETE";
|
||||
/** v0.6.1: 追加写入(value 拼接语义,恢复时按序 concat;aria WAL 分片用) */
|
||||
KVLogOp[KVLogOp["APPEND"] = 3] = "APPEND";
|
||||
})(KVLogOp || (KVLogOp = {}));
|
||||
/**
|
||||
* 编码一条日志记录。
|
||||
* @param seq 日志序号
|
||||
* @param puts key → value 写入条目
|
||||
* @param deletes 删除 key 列表
|
||||
* @param appends key → 追加块列表(APPEND 语义,恢复时拼接)
|
||||
*/
|
||||
function encodeLogRecord(seq, puts, deletes = []) {
|
||||
function encodeLogRecord(seq, puts, deletes = [], appends = {}) {
|
||||
const encoder = new TextEncoder();
|
||||
const entries = [];
|
||||
for (const [key, value] of Object.entries(puts)) {
|
||||
@@ -1136,6 +1139,9 @@ function encodeLogRecord(seq, puts, deletes = []) {
|
||||
for (const key of deletes) {
|
||||
entries.push({ op: KVLogOp.DELETE, key, value: new ArrayBuffer(0) });
|
||||
}
|
||||
for (const [key, value] of Object.entries(appends)) {
|
||||
entries.push({ op: KVLogOp.APPEND, key, value });
|
||||
}
|
||||
// 预编码 key 字节,计算总长度
|
||||
const entryBytes = [];
|
||||
let total = 4 + 4 + 4; // recordLen + seq + entryCount
|
||||
@@ -1541,6 +1547,18 @@ class KVStore {
|
||||
await this.appendRecord({}, keys);
|
||||
});
|
||||
}
|
||||
/**
|
||||
* v0.6.1: 追加写入(value 拼接语义)— aria WAL 分片等追加型数据用。
|
||||
* 日志记录 APPEND 类型(O(chunk) 高效),恢复时按 seq 顺序拼接,
|
||||
* checkpoint 后快照含最终值。崩溃时该次追加全有或全无(单记录原子)。
|
||||
*/
|
||||
async appendValue(key, chunk) {
|
||||
if (chunk.byteLength === 0)
|
||||
return;
|
||||
await this.enqueue(async () => {
|
||||
await this.appendRecord({}, [], { [key]: chunk });
|
||||
});
|
||||
}
|
||||
// =======================================================================
|
||||
// 维护
|
||||
// =======================================================================
|
||||
@@ -1613,9 +1631,9 @@ class KVStore {
|
||||
return run;
|
||||
}
|
||||
/** 追加一条日志记录并更新内存索引(队列内调用,无并发) */
|
||||
async appendRecord(puts, deletes) {
|
||||
async appendRecord(puts, deletes, appends = {}) {
|
||||
this.seq++;
|
||||
const record = encodeLogRecord(this.seq, puts, deletes);
|
||||
const record = encodeLogRecord(this.seq, puts, deletes, appends);
|
||||
try {
|
||||
// 日志追加:介质 append(真追加)或回退读+拼+写
|
||||
const data = record.buffer.slice(record.byteOffset, record.byteOffset + record.byteLength);
|
||||
@@ -1648,6 +1666,18 @@ class KVStore {
|
||||
for (const key of deletes) {
|
||||
this.index.delete(key);
|
||||
}
|
||||
for (const [key, chunk] of Object.entries(appends)) {
|
||||
const existing = this.index.get(key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + chunk.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(chunk), existing.byteLength);
|
||||
this.index.set(key, combined.buffer);
|
||||
}
|
||||
else {
|
||||
this.index.set(key, chunk);
|
||||
}
|
||||
}
|
||||
this.logBytes += record.byteLength;
|
||||
// 自动 checkpoint(日志超阈值)
|
||||
if (this.checkpointThreshold > 0 && this.logBytes >= this.checkpointThreshold) {
|
||||
@@ -1671,6 +1701,18 @@ class KVStore {
|
||||
if (e.op === KVLogOp.PUT) {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
else if (e.op === KVLogOp.APPEND) {
|
||||
const existing = this.index.get(e.key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + e.value.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(e.value), existing.byteLength);
|
||||
this.index.set(e.key, combined.buffer);
|
||||
}
|
||||
else {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
}
|
||||
else {
|
||||
this.index.delete(e.key);
|
||||
}
|
||||
@@ -4818,6 +4860,68 @@ class MemoryBackend {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* KVStoreBackend — 基于自研 KVStore 的 AriaEngine 存储后端
|
||||
* @module engine/aria/store/kvstore_backend
|
||||
*
|
||||
* v0.6.1: AriaEngine 可选后端(storageBackend: 'kv'),完全跑在自研 KVStore 上:
|
||||
* - write/read/delete/listKeys/exists/clear → KVStore 直接映射
|
||||
* - append → KVStore.appendValue(日志 APPEND 类型,O(chunk) 高效,aria WAL 分片用)
|
||||
* - writeMany/deleteMany → KVStore.putMany/deleteMany(单日志记录真原子,
|
||||
* 此前 OPFS 逐文件写靠空洞检测兜底,KV 后端原生原子)
|
||||
* - 崩溃恢复:KVStore 快照+日志恢复 → aria 打开重放自己的 WAL(双层恢复)
|
||||
*
|
||||
* 介质:浏览器 OPFS(KVStore 默认)或 Node SharedMemory——aria 不再依赖浏览器 OPFS API。
|
||||
*/
|
||||
class KVStoreBackend {
|
||||
constructor(medium, checkpointThreshold) {
|
||||
this.kv = new KVStore(medium, checkpointThreshold);
|
||||
}
|
||||
/** 底层 KVStore(测试/诊断用) */
|
||||
getKV() {
|
||||
return this.kv;
|
||||
}
|
||||
async open(name) {
|
||||
await this.kv.open(name);
|
||||
}
|
||||
async close() {
|
||||
await this.kv.close();
|
||||
}
|
||||
isOpen() {
|
||||
return this.kv.isOpen();
|
||||
}
|
||||
async read(key) {
|
||||
return this.kv.get(key);
|
||||
}
|
||||
async write(key, data) {
|
||||
await this.kv.put(key, data);
|
||||
}
|
||||
/** 追加写入(KVStore APPEND 日志,O(chunk)) */
|
||||
async append(key, data) {
|
||||
await this.kv.appendValue(key, data);
|
||||
}
|
||||
/** 多 key 原子写入(单日志记录) */
|
||||
async writeMany(entries) {
|
||||
await this.kv.putMany(entries);
|
||||
}
|
||||
async delete(key) {
|
||||
await this.kv.delete(key);
|
||||
}
|
||||
/** 多 key 原子删除(单日志记录) */
|
||||
async deleteMany(keys) {
|
||||
await this.kv.deleteMany(keys);
|
||||
}
|
||||
async listKeys() {
|
||||
return this.kv.listKeys();
|
||||
}
|
||||
async exists(key) {
|
||||
return this.kv.exists(key);
|
||||
}
|
||||
async clear() {
|
||||
await this.kv.clear();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* AriaEngine Crypto — 页面级 AES-GCM 加密
|
||||
* @module engine/aria/crypto
|
||||
@@ -5953,6 +6057,10 @@ class AriaEngine {
|
||||
if (this.config.storageBackend === 'opfs') {
|
||||
baseBackend = new OPFSBackend();
|
||||
}
|
||||
else if (this.config.storageBackend === 'kv') {
|
||||
// v0.6.1: 自研 KVStore 后端(aria 完全跑在自研存储栈上,不依赖浏览器 OPFS)
|
||||
baseBackend = new KVStoreBackend();
|
||||
}
|
||||
else {
|
||||
baseBackend = new MemoryBackend();
|
||||
}
|
||||
@@ -11590,8 +11698,9 @@ class MetonaSqlark {
|
||||
return new KVStoreEngine();
|
||||
case 'aria':
|
||||
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
|
||||
// v0.6.1: diskEngine 'kv' → 自研 KVStore 后端
|
||||
return new AriaEngine({
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : 'opfs',
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : diskEngine === 'kv' ? 'kv' : 'opfs',
|
||||
...(this.config.aria ?? {}),
|
||||
});
|
||||
case 'hybrid':
|
||||
|
||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+113
-4
@@ -1126,14 +1126,17 @@
|
||||
(function (KVLogOp) {
|
||||
KVLogOp[KVLogOp["PUT"] = 1] = "PUT";
|
||||
KVLogOp[KVLogOp["DELETE"] = 2] = "DELETE";
|
||||
/** v0.6.1: 追加写入(value 拼接语义,恢复时按序 concat;aria WAL 分片用) */
|
||||
KVLogOp[KVLogOp["APPEND"] = 3] = "APPEND";
|
||||
})(KVLogOp || (KVLogOp = {}));
|
||||
/**
|
||||
* 编码一条日志记录。
|
||||
* @param seq 日志序号
|
||||
* @param puts key → value 写入条目
|
||||
* @param deletes 删除 key 列表
|
||||
* @param appends key → 追加块列表(APPEND 语义,恢复时拼接)
|
||||
*/
|
||||
function encodeLogRecord(seq, puts, deletes = []) {
|
||||
function encodeLogRecord(seq, puts, deletes = [], appends = {}) {
|
||||
const encoder = new TextEncoder();
|
||||
const entries = [];
|
||||
for (const [key, value] of Object.entries(puts)) {
|
||||
@@ -1142,6 +1145,9 @@
|
||||
for (const key of deletes) {
|
||||
entries.push({ op: KVLogOp.DELETE, key, value: new ArrayBuffer(0) });
|
||||
}
|
||||
for (const [key, value] of Object.entries(appends)) {
|
||||
entries.push({ op: KVLogOp.APPEND, key, value });
|
||||
}
|
||||
// 预编码 key 字节,计算总长度
|
||||
const entryBytes = [];
|
||||
let total = 4 + 4 + 4; // recordLen + seq + entryCount
|
||||
@@ -1547,6 +1553,18 @@
|
||||
await this.appendRecord({}, keys);
|
||||
});
|
||||
}
|
||||
/**
|
||||
* v0.6.1: 追加写入(value 拼接语义)— aria WAL 分片等追加型数据用。
|
||||
* 日志记录 APPEND 类型(O(chunk) 高效),恢复时按 seq 顺序拼接,
|
||||
* checkpoint 后快照含最终值。崩溃时该次追加全有或全无(单记录原子)。
|
||||
*/
|
||||
async appendValue(key, chunk) {
|
||||
if (chunk.byteLength === 0)
|
||||
return;
|
||||
await this.enqueue(async () => {
|
||||
await this.appendRecord({}, [], { [key]: chunk });
|
||||
});
|
||||
}
|
||||
// =======================================================================
|
||||
// 维护
|
||||
// =======================================================================
|
||||
@@ -1619,9 +1637,9 @@
|
||||
return run;
|
||||
}
|
||||
/** 追加一条日志记录并更新内存索引(队列内调用,无并发) */
|
||||
async appendRecord(puts, deletes) {
|
||||
async appendRecord(puts, deletes, appends = {}) {
|
||||
this.seq++;
|
||||
const record = encodeLogRecord(this.seq, puts, deletes);
|
||||
const record = encodeLogRecord(this.seq, puts, deletes, appends);
|
||||
try {
|
||||
// 日志追加:介质 append(真追加)或回退读+拼+写
|
||||
const data = record.buffer.slice(record.byteOffset, record.byteOffset + record.byteLength);
|
||||
@@ -1654,6 +1672,18 @@
|
||||
for (const key of deletes) {
|
||||
this.index.delete(key);
|
||||
}
|
||||
for (const [key, chunk] of Object.entries(appends)) {
|
||||
const existing = this.index.get(key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + chunk.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(chunk), existing.byteLength);
|
||||
this.index.set(key, combined.buffer);
|
||||
}
|
||||
else {
|
||||
this.index.set(key, chunk);
|
||||
}
|
||||
}
|
||||
this.logBytes += record.byteLength;
|
||||
// 自动 checkpoint(日志超阈值)
|
||||
if (this.checkpointThreshold > 0 && this.logBytes >= this.checkpointThreshold) {
|
||||
@@ -1677,6 +1707,18 @@
|
||||
if (e.op === KVLogOp.PUT) {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
else if (e.op === KVLogOp.APPEND) {
|
||||
const existing = this.index.get(e.key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + e.value.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(e.value), existing.byteLength);
|
||||
this.index.set(e.key, combined.buffer);
|
||||
}
|
||||
else {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
}
|
||||
else {
|
||||
this.index.delete(e.key);
|
||||
}
|
||||
@@ -4824,6 +4866,68 @@
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* KVStoreBackend — 基于自研 KVStore 的 AriaEngine 存储后端
|
||||
* @module engine/aria/store/kvstore_backend
|
||||
*
|
||||
* v0.6.1: AriaEngine 可选后端(storageBackend: 'kv'),完全跑在自研 KVStore 上:
|
||||
* - write/read/delete/listKeys/exists/clear → KVStore 直接映射
|
||||
* - append → KVStore.appendValue(日志 APPEND 类型,O(chunk) 高效,aria WAL 分片用)
|
||||
* - writeMany/deleteMany → KVStore.putMany/deleteMany(单日志记录真原子,
|
||||
* 此前 OPFS 逐文件写靠空洞检测兜底,KV 后端原生原子)
|
||||
* - 崩溃恢复:KVStore 快照+日志恢复 → aria 打开重放自己的 WAL(双层恢复)
|
||||
*
|
||||
* 介质:浏览器 OPFS(KVStore 默认)或 Node SharedMemory——aria 不再依赖浏览器 OPFS API。
|
||||
*/
|
||||
class KVStoreBackend {
|
||||
constructor(medium, checkpointThreshold) {
|
||||
this.kv = new KVStore(medium, checkpointThreshold);
|
||||
}
|
||||
/** 底层 KVStore(测试/诊断用) */
|
||||
getKV() {
|
||||
return this.kv;
|
||||
}
|
||||
async open(name) {
|
||||
await this.kv.open(name);
|
||||
}
|
||||
async close() {
|
||||
await this.kv.close();
|
||||
}
|
||||
isOpen() {
|
||||
return this.kv.isOpen();
|
||||
}
|
||||
async read(key) {
|
||||
return this.kv.get(key);
|
||||
}
|
||||
async write(key, data) {
|
||||
await this.kv.put(key, data);
|
||||
}
|
||||
/** 追加写入(KVStore APPEND 日志,O(chunk)) */
|
||||
async append(key, data) {
|
||||
await this.kv.appendValue(key, data);
|
||||
}
|
||||
/** 多 key 原子写入(单日志记录) */
|
||||
async writeMany(entries) {
|
||||
await this.kv.putMany(entries);
|
||||
}
|
||||
async delete(key) {
|
||||
await this.kv.delete(key);
|
||||
}
|
||||
/** 多 key 原子删除(单日志记录) */
|
||||
async deleteMany(keys) {
|
||||
await this.kv.deleteMany(keys);
|
||||
}
|
||||
async listKeys() {
|
||||
return this.kv.listKeys();
|
||||
}
|
||||
async exists(key) {
|
||||
return this.kv.exists(key);
|
||||
}
|
||||
async clear() {
|
||||
await this.kv.clear();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* AriaEngine Crypto — 页面级 AES-GCM 加密
|
||||
* @module engine/aria/crypto
|
||||
@@ -5959,6 +6063,10 @@
|
||||
if (this.config.storageBackend === 'opfs') {
|
||||
baseBackend = new OPFSBackend();
|
||||
}
|
||||
else if (this.config.storageBackend === 'kv') {
|
||||
// v0.6.1: 自研 KVStore 后端(aria 完全跑在自研存储栈上,不依赖浏览器 OPFS)
|
||||
baseBackend = new KVStoreBackend();
|
||||
}
|
||||
else {
|
||||
baseBackend = new MemoryBackend();
|
||||
}
|
||||
@@ -11596,8 +11704,9 @@
|
||||
return new KVStoreEngine();
|
||||
case 'aria':
|
||||
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
|
||||
// v0.6.1: diskEngine 'kv' → 自研 KVStore 后端
|
||||
return new AriaEngine({
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : 'opfs',
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : diskEngine === 'kv' ? 'kv' : 'opfs',
|
||||
...(this.config.aria ?? {}),
|
||||
});
|
||||
case 'hybrid':
|
||||
|
||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1
-1
File diff suppressed because one or more lines are too long
+2
-2
@@ -768,7 +768,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
||||
<strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e(7 用例)· DatabaseConfig.aria 配置透传。<br>
|
||||
<strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)。<br>
|
||||
<strong>v0.6.0 完全移除 IndexedDB</strong> — 自研 KVStore 事务存储引擎(多 key 原子写 = 单日志记录原子追加 · 快照 checkpoint + 两阶段崩溃恢复 · CRC-32 自愈)· disk 模式切换 KVStoreEngine(替代 IndexedDBEngine + OPFSEngine)· 事务内 DDL / 外键级联 / 二级索引完整持久化 · migrateFromIndexedDB() 旧库一键迁移 · 10 万 key 压力验证 · e2e 崩溃注入 + KVStoreEngine 真实环境(12 用例)。<br>
|
||||
<strong>v0.6.1 生产可用性深度审查</strong> — MemoryEngine 级联环(A→B→A)无限递归修复(visited 保护,与 Aria 对齐)· KVStore 快照损坏水位 bug 修复(metaSeq 误跳日志)· 多表事务 / 级联写入合并单条日志记录真原子(崩溃无部分提交)· 未 open 防护统一 · 27 个异常场景测试(空值/大 value/特殊字符 key/写失败/事务故障/级联环/批量删除等)· 1049 测试 65 套件 · 89.3% 行覆盖率。</p>
|
||||
<strong>v0.6.1 生产可用性深度审查</strong> — MemoryEngine 级联环(A→B→A)无限递归修复(visited 保护,与 Aria 对齐)· KVStore 快照损坏水位 bug 修复(metaSeq 误跳日志)· 多表事务 / 级联写入合并单条日志记录真原子(崩溃无部分提交)· 未 open 防护统一 · <strong>AriaEngine 可选自研 KVStore 后端</strong>(storageBackend: 'kv':KVStore APPEND 日志支持 WAL 追加、writeMany 真原子、不再依赖浏览器 OPFS)· 27 个异常场景测试 + 10 个 aria+kv 集成测试 · 1064 测试 66 套件 · 89.4% 行覆盖率。</p>
|
||||
|
||||
<h3>存储模式对比</h3>
|
||||
<table>
|
||||
@@ -829,7 +829,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
||||
<tr><td><code>checkpointInterval</code></td><td><code>number</code></td><td><code>1000</code></td><td>Checkpoint 触发间隔(操作数)</td></tr>
|
||||
<tr><td><code>walSizeThreshold</code></td><td><code>number</code></td><td><code>16777216</code></td><td>WAL 大小阈值(字节),超阈值触发 checkpoint ✅ v0.2.5</td></tr>
|
||||
<tr><td><code>compression</code></td><td><code>boolean</code></td><td><code>false</code></td><td>是否启用页面压缩</td></tr>
|
||||
<tr><td><code>storageBackend</code></td><td><code>'opfs'|'memory'</code></td><td><code>'opfs'</code></td><td>存储后端类型(v0.6.0: IndexedDB 已移除)</td></tr>
|
||||
<tr><td><code>storageBackend</code></td><td><code>'opfs'|'memory'|'kv'</code></td><td><code>'opfs'</code></td><td>存储后端类型('kv' = 自研 KVStore 后端 🆕 v0.6.1)</td></tr>
|
||||
<tr><td><code>encryption</code></td><td><code>{ password: string }</code></td><td>-</td><td>全库 AES-256-GCM 透明加密(PBKDF2 派生 + salt 持久化 + 密码验证)🆕 v0.5.0</td></tr>
|
||||
<tr><td><code>pageStorage</code></td><td><code>boolean</code></td><td>OPFS 自动启用</td><td>SSTable 4KB 页面化物理存储(BufferPool 缓存)🆕 v0.5.0</td></tr>
|
||||
</table>
|
||||
|
||||
+4
-4
@@ -153,7 +153,7 @@
|
||||
<!-- Hero -->
|
||||
<section class="hero">
|
||||
<div class="container">
|
||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.6.1 生产可用性硬化 — 1049测试 65套件 · 自研 KVStore 事务引擎 · 多 key 原子写 · 级联环保护 · 崩溃恢复 · 旧库一键迁移</div>
|
||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.6.1 生产可用性硬化 — 1064测试 66套件 · 自研 KVStore 事务引擎 · 多 key 原子写 · 级联环保护 · 崩溃恢复 · 旧库一键迁移</div>
|
||||
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
||||
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
||||
<div class="actions">
|
||||
@@ -414,12 +414,12 @@ npm install @metona-team/metona-sqlark
|
||||
<p>MetonaSqlark 的核心指标</p>
|
||||
</div>
|
||||
<div class="stats">
|
||||
<div class="stat-card"><div class="num">1049</div><div class="label">测试用例</div></div>
|
||||
<div class="stat-card"><div class="num">89.3%</div><div class="label">行覆盖率</div></div>
|
||||
<div class="stat-card"><div class="num">1064</div><div class="label">测试用例</div></div>
|
||||
<div class="stat-card"><div class="num">89.4%</div><div class="label">行覆盖率</div></div>
|
||||
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
||||
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
||||
<div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div>
|
||||
<div class="stat-card"><div class="num">65</div><div class="label">测试套件</div></div>
|
||||
<div class="stat-card"><div class="num">66</div><div class="label">测试套件</div></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
+2
-2
@@ -10,8 +10,8 @@
|
||||
/** 存储模式 */
|
||||
export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
||||
|
||||
/** 磁盘引擎类型(v0.6.0: IndexedDB 已移除,'memory' 供 aria 内存后端) */
|
||||
export type DiskEngine = 'opfs' | 'memory';
|
||||
/** 磁盘引擎类型(v0.6.0: IndexedDB 已移除;'memory' 供 aria 内存后端;'kv' 供 aria 自研 KVStore 后端) */
|
||||
export type DiskEngine = 'opfs' | 'memory' | 'kv';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 字段类型
|
||||
|
||||
+2
-1
@@ -557,8 +557,9 @@ export class MetonaSqlark {
|
||||
return new KVStoreEngine();
|
||||
case 'aria':
|
||||
// v0.4.5: 透传 AriaEngine 专属配置(walSyncMode/checkpointInterval/encryption/pageStorage 等)
|
||||
// v0.6.1: diskEngine 'kv' → 自研 KVStore 后端
|
||||
return new AriaEngine({
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : 'opfs',
|
||||
storageBackend: diskEngine === 'memory' ? 'memory' : diskEngine === 'kv' ? 'kv' : 'opfs',
|
||||
...(this.config.aria ?? {}),
|
||||
});
|
||||
case 'hybrid':
|
||||
|
||||
@@ -22,6 +22,7 @@ import { DatabaseLock } from './locks';
|
||||
import { WALRecordType, type WALRecord } from './types';
|
||||
import { CheckpointManager } from './wal/checkpoint';
|
||||
import { MemoryBackend, type IStorageBackend } from './store/backend';
|
||||
import { KVStoreBackend } from './store/kvstore_backend';
|
||||
import { OPFSBackend } from './store/opfs_backend';
|
||||
import { EncryptedBackend } from './store/encrypted_backend';
|
||||
import { PageSSTableStore } from './store/page_sstable_store';
|
||||
@@ -117,6 +118,9 @@ export class AriaEngine implements IStorageEngine {
|
||||
let baseBackend: IStorageBackend;
|
||||
if (this.config.storageBackend === 'opfs') {
|
||||
baseBackend = new OPFSBackend();
|
||||
} else if (this.config.storageBackend === 'kv') {
|
||||
// v0.6.1: 自研 KVStore 后端(aria 完全跑在自研存储栈上,不依赖浏览器 OPFS)
|
||||
baseBackend = new KVStoreBackend();
|
||||
} else {
|
||||
baseBackend = new MemoryBackend();
|
||||
}
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* KVStoreBackend — 基于自研 KVStore 的 AriaEngine 存储后端
|
||||
* @module engine/aria/store/kvstore_backend
|
||||
*
|
||||
* v0.6.1: AriaEngine 可选后端(storageBackend: 'kv'),完全跑在自研 KVStore 上:
|
||||
* - write/read/delete/listKeys/exists/clear → KVStore 直接映射
|
||||
* - append → KVStore.appendValue(日志 APPEND 类型,O(chunk) 高效,aria WAL 分片用)
|
||||
* - writeMany/deleteMany → KVStore.putMany/deleteMany(单日志记录真原子,
|
||||
* 此前 OPFS 逐文件写靠空洞检测兜底,KV 后端原生原子)
|
||||
* - 崩溃恢复:KVStore 快照+日志恢复 → aria 打开重放自己的 WAL(双层恢复)
|
||||
*
|
||||
* 介质:浏览器 OPFS(KVStore 默认)或 Node SharedMemory——aria 不再依赖浏览器 OPFS API。
|
||||
*/
|
||||
|
||||
import type { IStorageBackend } from './backend';
|
||||
import { KVStore } from '../../kvstore/index';
|
||||
|
||||
export class KVStoreBackend implements IStorageBackend {
|
||||
private kv: KVStore;
|
||||
|
||||
constructor(medium?: IStorageBackend, checkpointThreshold?: number) {
|
||||
this.kv = new KVStore(medium, checkpointThreshold);
|
||||
}
|
||||
|
||||
/** 底层 KVStore(测试/诊断用) */
|
||||
getKV(): KVStore {
|
||||
return this.kv;
|
||||
}
|
||||
|
||||
async open(name: string): Promise<void> {
|
||||
await this.kv.open(name);
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
await this.kv.close();
|
||||
}
|
||||
|
||||
isOpen(): boolean {
|
||||
return this.kv.isOpen();
|
||||
}
|
||||
|
||||
async read(key: string): Promise<ArrayBuffer | null> {
|
||||
return this.kv.get(key);
|
||||
}
|
||||
|
||||
async write(key: string, data: ArrayBuffer): Promise<void> {
|
||||
await this.kv.put(key, data);
|
||||
}
|
||||
|
||||
/** 追加写入(KVStore APPEND 日志,O(chunk)) */
|
||||
async append(key: string, data: ArrayBuffer): Promise<void> {
|
||||
await this.kv.appendValue(key, data);
|
||||
}
|
||||
|
||||
/** 多 key 原子写入(单日志记录) */
|
||||
async writeMany(entries: Record<string, ArrayBuffer>): Promise<void> {
|
||||
await this.kv.putMany(entries);
|
||||
}
|
||||
|
||||
async delete(key: string): Promise<void> {
|
||||
await this.kv.delete(key);
|
||||
}
|
||||
|
||||
/** 多 key 原子删除(单日志记录) */
|
||||
async deleteMany(keys: string[]): Promise<void> {
|
||||
await this.kv.deleteMany(keys);
|
||||
}
|
||||
|
||||
async listKeys(): Promise<string[]> {
|
||||
return this.kv.listKeys();
|
||||
}
|
||||
|
||||
async exists(key: string): Promise<boolean> {
|
||||
return this.kv.exists(key);
|
||||
}
|
||||
|
||||
async clear(): Promise<void> {
|
||||
await this.kv.clear();
|
||||
}
|
||||
}
|
||||
@@ -207,7 +207,7 @@ export interface AriaEngineConfig {
|
||||
/** 是否启用页面压缩(默认 false) */
|
||||
compression?: boolean;
|
||||
/** 存储后端 */
|
||||
storageBackend?: 'opfs' | 'memory';
|
||||
storageBackend?: 'opfs' | 'memory' | 'kv';
|
||||
/** WAL 大小阈值(字节,超过则强制 checkpoint,默认 16MB) */
|
||||
walSizeThreshold?: number;
|
||||
/** 最大内存预算(MB,默认 64) */
|
||||
|
||||
@@ -223,6 +223,18 @@ export class KVStore {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.6.1: 追加写入(value 拼接语义)— aria WAL 分片等追加型数据用。
|
||||
* 日志记录 APPEND 类型(O(chunk) 高效),恢复时按 seq 顺序拼接,
|
||||
* checkpoint 后快照含最终值。崩溃时该次追加全有或全无(单记录原子)。
|
||||
*/
|
||||
async appendValue(key: string, chunk: ArrayBuffer): Promise<void> {
|
||||
if (chunk.byteLength === 0) return;
|
||||
await this.enqueue(async () => {
|
||||
await this.appendRecord({}, [], { [key]: chunk });
|
||||
});
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 维护
|
||||
// =======================================================================
|
||||
@@ -301,9 +313,13 @@ export class KVStore {
|
||||
}
|
||||
|
||||
/** 追加一条日志记录并更新内存索引(队列内调用,无并发) */
|
||||
private async appendRecord(puts: Record<string, ArrayBuffer>, deletes: string[]): Promise<void> {
|
||||
private async appendRecord(
|
||||
puts: Record<string, ArrayBuffer>,
|
||||
deletes: string[],
|
||||
appends: Record<string, ArrayBuffer> = {},
|
||||
): Promise<void> {
|
||||
this.seq++;
|
||||
const record = encodeLogRecord(this.seq, puts, deletes);
|
||||
const record = encodeLogRecord(this.seq, puts, deletes, appends);
|
||||
try {
|
||||
// 日志追加:介质 append(真追加)或回退读+拼+写
|
||||
const data = record.buffer.slice(record.byteOffset, record.byteOffset + record.byteLength) as ArrayBuffer;
|
||||
@@ -334,6 +350,17 @@ export class KVStore {
|
||||
for (const key of deletes) {
|
||||
this.index.delete(key);
|
||||
}
|
||||
for (const [key, chunk] of Object.entries(appends)) {
|
||||
const existing = this.index.get(key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + chunk.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(chunk), existing.byteLength);
|
||||
this.index.set(key, combined.buffer as ArrayBuffer);
|
||||
} else {
|
||||
this.index.set(key, chunk);
|
||||
}
|
||||
}
|
||||
this.logBytes += record.byteLength;
|
||||
|
||||
// 自动 checkpoint(日志超阈值)
|
||||
@@ -358,6 +385,16 @@ export class KVStore {
|
||||
for (const e of entries) {
|
||||
if (e.op === KVLogOp.PUT) {
|
||||
this.index.set(e.key, e.value);
|
||||
} else if (e.op === KVLogOp.APPEND) {
|
||||
const existing = this.index.get(e.key);
|
||||
if (existing) {
|
||||
const combined = new Uint8Array(existing.byteLength + e.value.byteLength);
|
||||
combined.set(new Uint8Array(existing), 0);
|
||||
combined.set(new Uint8Array(e.value), existing.byteLength);
|
||||
this.index.set(e.key, combined.buffer as ArrayBuffer);
|
||||
} else {
|
||||
this.index.set(e.key, e.value);
|
||||
}
|
||||
} else {
|
||||
this.index.delete(e.key);
|
||||
}
|
||||
|
||||
@@ -23,6 +23,8 @@ import { crc32 } from '../aria/crc32';
|
||||
export const enum KVLogOp {
|
||||
PUT = 1,
|
||||
DELETE = 2,
|
||||
/** v0.6.1: 追加写入(value 拼接语义,恢复时按序 concat;aria WAL 分片用) */
|
||||
APPEND = 3,
|
||||
}
|
||||
|
||||
/** 解析后的日志记录 */
|
||||
@@ -40,11 +42,13 @@ export interface KVLogRecord {
|
||||
* @param seq 日志序号
|
||||
* @param puts key → value 写入条目
|
||||
* @param deletes 删除 key 列表
|
||||
* @param appends key → 追加块列表(APPEND 语义,恢复时拼接)
|
||||
*/
|
||||
export function encodeLogRecord(
|
||||
seq: number,
|
||||
puts: Record<string, ArrayBuffer>,
|
||||
deletes: string[] = [],
|
||||
appends: Record<string, ArrayBuffer> = {},
|
||||
): Uint8Array {
|
||||
const encoder = new TextEncoder();
|
||||
const entries: { op: KVLogOp; key: string; value: ArrayBuffer }[] = [];
|
||||
@@ -54,6 +58,9 @@ export function encodeLogRecord(
|
||||
for (const key of deletes) {
|
||||
entries.push({ op: KVLogOp.DELETE, key, value: new ArrayBuffer(0) });
|
||||
}
|
||||
for (const [key, value] of Object.entries(appends)) {
|
||||
entries.push({ op: KVLogOp.APPEND, key, value });
|
||||
}
|
||||
|
||||
// 预编码 key 字节,计算总长度
|
||||
const entryBytes: { op: KVLogOp; key: Uint8Array; value: Uint8Array }[] = [];
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
/**
|
||||
* KVStoreBackend + AriaEngine(kv 后端) 集成测试
|
||||
*
|
||||
* 覆盖:
|
||||
* 1. KVStoreBackend 单元:读写/追加/批量/删除/clear/close
|
||||
* 2. AriaEngine + kv 后端:CRUD 持久化(跨实例恢复)
|
||||
* 3. WAL 分片追加(backend.append 走 KVStore APPEND)
|
||||
* 4. 崩溃恢复:kv 后端下 WAL 重放
|
||||
* 5. 二级索引跨重启
|
||||
* 6. 页面化存储(SSTable → KVStore key)
|
||||
* 7. writeMany 原子性(aria 已不依赖,验证接口)
|
||||
*/
|
||||
import { KVStoreBackend } from '../../src/engine/aria/store/kvstore_backend';
|
||||
import { SharedMemoryBackend } from '../../src/engine/kvstore/shared_memory_medium';
|
||||
import { AriaEngine } from '../../src/engine/aria/index';
|
||||
import { createSchema } from '../../src/table/schema';
|
||||
|
||||
let counter = 0;
|
||||
function uniqueDB(): string {
|
||||
return `kvbe-${Date.now()}-${++counter}-${Math.random().toString(36).slice(2, 6)}`;
|
||||
}
|
||||
|
||||
const enc = (s: string) => new TextEncoder().encode(s).buffer as ArrayBuffer;
|
||||
const dec = (b: ArrayBuffer | null) => (b ? new TextDecoder().decode(b) : null);
|
||||
|
||||
beforeEach(() => {
|
||||
SharedMemoryBackend.clearRegistry();
|
||||
});
|
||||
|
||||
describe('KVStoreBackend — 单元', () => {
|
||||
it('读写/追加/批量/删除/clear 全接口', async () => {
|
||||
const backend = new KVStoreBackend(new SharedMemoryBackend());
|
||||
await backend.open(uniqueDB());
|
||||
await backend.write('k1', enc('V1'));
|
||||
expect(dec(await backend.read('k1'))).toBe('V1');
|
||||
expect(await backend.exists('k1')).toBe(true);
|
||||
|
||||
// 追加
|
||||
await backend.append('wal', enc('A'));
|
||||
await backend.append('wal', enc('B'));
|
||||
expect(dec(await backend.read('wal'))).toBe('AB');
|
||||
|
||||
// 批量原子
|
||||
await backend.writeMany({ a: enc('1'), b: enc('2') });
|
||||
expect(dec(await backend.read('b'))).toBe('2');
|
||||
await backend.deleteMany(['a']);
|
||||
expect(await backend.exists('a')).toBe(false);
|
||||
|
||||
// listKeys / delete / clear
|
||||
const keys = await backend.listKeys();
|
||||
expect(keys).toContain('k1');
|
||||
expect(keys).toContain('wal');
|
||||
await backend.delete('k1');
|
||||
expect(await backend.exists('k1')).toBe(false);
|
||||
await backend.clear();
|
||||
expect(await backend.listKeys()).toEqual([]);
|
||||
await backend.close();
|
||||
expect(backend.isOpen()).toBe(false);
|
||||
});
|
||||
|
||||
it('跨实例持久化(SharedMemory 语义)', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const b1 = new KVStoreBackend(new SharedMemoryBackend());
|
||||
await b1.open(dbName);
|
||||
await b1.write('k', enc('PERSIST'));
|
||||
await b1.append('wal', enc('LOG'));
|
||||
await b1.close();
|
||||
|
||||
const b2 = new KVStoreBackend(new SharedMemoryBackend());
|
||||
await b2.open(dbName);
|
||||
expect(dec(await b2.read('k'))).toBe('PERSIST');
|
||||
expect(dec(await b2.read('wal'))).toBe('LOG');
|
||||
await b2.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe('AriaEngine + kv 后端(storageBackend: kv)', () => {
|
||||
function createEngine(): AriaEngine {
|
||||
const engine = new AriaEngine({
|
||||
storageBackend: 'kv',
|
||||
checkpointInterval: 100000,
|
||||
walSyncMode: 'full',
|
||||
memtableSizeThreshold: 64 * 1024 * 1024,
|
||||
});
|
||||
return engine;
|
||||
}
|
||||
|
||||
it('CRUD 持久化:写 → close → 重开数据完整(KVStore 快照/日志)', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const e1 = createEngine();
|
||||
await e1.open(dbName, 1);
|
||||
await e1.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string', required: true },
|
||||
age: { type: 'number', default: 0 },
|
||||
}));
|
||||
await e1.insert('users', [
|
||||
{ id: '1', name: 'Alice', age: 30 },
|
||||
{ id: '2', name: 'Bob', age: 25 },
|
||||
]);
|
||||
await e1.close();
|
||||
|
||||
const e2 = createEngine();
|
||||
await e2.open(dbName, 1);
|
||||
expect(await e2.count('users')).toBe(2);
|
||||
const rows = await e2.find('users', { table: 'users', where: { name: 'Alice' } });
|
||||
expect(rows[0].age).toBe(30);
|
||||
await e2.close();
|
||||
});
|
||||
|
||||
it('WAL 分片经 KVStore APPEND 追加:崩溃恢复重放', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const e1 = createEngine();
|
||||
await e1.open(dbName, 1);
|
||||
await e1.createTable(createSchema('logs', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
msg: { type: 'string' },
|
||||
}));
|
||||
// 不 flush 不 checkpoint:数据全在 WAL(KVStore APPEND 记录)
|
||||
for (let i = 0; i < 50; i++) {
|
||||
await e1.insert('logs', [{ id: `l-${i}`, msg: `msg-${i}` }]);
|
||||
}
|
||||
// 模拟崩溃:直接断开(不 close)
|
||||
await (e1 as any).backend.close();
|
||||
(e1 as any).opened = false;
|
||||
|
||||
const e2 = createEngine();
|
||||
await e2.open(dbName, 1);
|
||||
expect(await e2.count('logs')).toBe(50);
|
||||
await e2.close();
|
||||
});
|
||||
|
||||
it('checkpoint 后崩溃:快照 + WAL 混合恢复', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const e1 = createEngine();
|
||||
await e1.open(dbName, 1);
|
||||
await e1.createTable(createSchema('t', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
v: { type: 'number' },
|
||||
}));
|
||||
await e1.insert('t', [{ id: 'a', v: 1 }]);
|
||||
// 完整 checkpoint:flush LSM(a 落盘为 SSTable)+ 截断 WAL
|
||||
await (e1 as any).checkpointManager.checkpoint();
|
||||
await e1.insert('t', [{ id: 'b', v: 2 }]); // b 只在 WAL
|
||||
await (e1 as any).backend.close();
|
||||
(e1 as any).opened = false;
|
||||
|
||||
const e2 = createEngine();
|
||||
await e2.open(dbName, 1);
|
||||
expect(await e2.count('t')).toBe(2);
|
||||
await e2.close();
|
||||
});
|
||||
|
||||
it('二级索引跨重启恢复', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const e1 = createEngine();
|
||||
await e1.open(dbName, 1);
|
||||
await e1.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
tag: { type: 'string', index: true },
|
||||
}));
|
||||
await e1.insert('items', [
|
||||
{ id: '1', tag: 'a' }, { id: '2', tag: 'b' }, { id: '3', tag: 'a' },
|
||||
]);
|
||||
await e1.close();
|
||||
|
||||
const e2 = createEngine();
|
||||
await e2.open(dbName, 1);
|
||||
const rows = await e2.find('items', { table: 'items', where: { tag: 'a' } });
|
||||
expect(rows).toHaveLength(2);
|
||||
await e2.close();
|
||||
});
|
||||
|
||||
it('页面化存储:SSTable 页面经 KVStore 读写', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const e1 = createEngine();
|
||||
await e1.open(dbName, 1);
|
||||
await e1.createTable(createSchema('docs', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
body: { type: 'string' },
|
||||
}));
|
||||
const rows = [] as Record<string, unknown>[];
|
||||
for (let i = 0; i < 30; i++) {
|
||||
rows.push({ id: `d-${i}`, body: '页面化内容'.repeat(40) });
|
||||
}
|
||||
await e1.insert('docs', rows);
|
||||
await (e1 as any).lsm.flush();
|
||||
// kv 后端:SSTable 整 value 存储(KVStore 日志型天然适合大块,无需拆 4KB 页面)
|
||||
const kv = (e1 as any).backend.getKV();
|
||||
const keys = await kv.listKeys();
|
||||
expect(keys.some((k: string) => k.startsWith('sst_'))).toBe(true);
|
||||
await e1.close();
|
||||
|
||||
const e2 = createEngine();
|
||||
await e2.open(dbName, 1);
|
||||
expect(await e2.count('docs')).toBe(30);
|
||||
await e2.close();
|
||||
});
|
||||
|
||||
it('大数据量 + 崩溃模拟:KV 后端零丢失', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const e1 = createEngine();
|
||||
await e1.open(dbName, 1);
|
||||
await e1.createTable(createSchema('big', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
v: { type: 'number' },
|
||||
}));
|
||||
for (let batch = 0; batch < 10; batch++) {
|
||||
const rows = [] as Record<string, unknown>[];
|
||||
for (let i = 0; i < 100; i++) rows.push({ id: `k${batch}-${i}`, v: i });
|
||||
await e1.insert('big', rows);
|
||||
if ((batch + 1) % 3 === 0) await (e1 as any).kvCheckpoint?.();
|
||||
}
|
||||
await (e1 as any).backend.close();
|
||||
(e1 as any).opened = false;
|
||||
|
||||
const e2 = createEngine();
|
||||
await e2.open(dbName, 1);
|
||||
expect(await e2.count('big')).toBe(1000);
|
||||
await e2.close();
|
||||
}, 60000);
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// 高层 API:MetonaSqlark + diskEngine 'kv'
|
||||
// ===================================================================
|
||||
describe('MetonaSqlark + diskEngine: kv(高层 API)', () => {
|
||||
it('创建 → 建表 → CRUD → SQL → 持久化重开', async () => {
|
||||
const { MetonaSqlark } = require('../../src/core');
|
||||
const dbName = uniqueDB();
|
||||
const db = new MetonaSqlark({
|
||||
name: dbName,
|
||||
mode: 'aria',
|
||||
diskEngine: 'kv',
|
||||
aria: { walSyncMode: 'full', checkpointInterval: 100000, memtableSizeThreshold: 64 * 1024 * 1024 },
|
||||
});
|
||||
await db.init();
|
||||
|
||||
await db.defineTable('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string', required: true },
|
||||
age: { type: 'number', default: 0 },
|
||||
});
|
||||
await db.query("INSERT INTO users VALUES ('1', 'Alice', 30)");
|
||||
await db.query("INSERT INTO users VALUES ('2', 'Bob', 25)");
|
||||
const rows = await db.query("SELECT * FROM users WHERE age > 26") as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].name).toBe('Alice');
|
||||
await db.close();
|
||||
|
||||
// 重开:数据完整
|
||||
const db2 = new MetonaSqlark({
|
||||
name: dbName,
|
||||
mode: 'aria',
|
||||
diskEngine: 'kv',
|
||||
aria: { walSyncMode: 'full', checkpointInterval: 100000, memtableSizeThreshold: 64 * 1024 * 1024 },
|
||||
});
|
||||
await db2.init();
|
||||
expect(await db2.table('users').count()).toBe(2);
|
||||
await db2.close();
|
||||
});
|
||||
|
||||
it('事务回滚 + 外键级联在 kv 后端工作', async () => {
|
||||
const { MetonaSqlark } = require('../../src/core');
|
||||
const db = new MetonaSqlark({
|
||||
name: uniqueDB(), mode: 'aria', diskEngine: 'kv',
|
||||
aria: { walSyncMode: 'full', checkpointInterval: 100000 },
|
||||
});
|
||||
await db.init();
|
||||
|
||||
await db.defineTable('users', { id: { type: 'string', primaryKey: true } });
|
||||
await db.defineTable('orders', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
user_id: { type: 'string', references: 'users.id', onDelete: 'CASCADE' },
|
||||
});
|
||||
|
||||
// 事务回滚
|
||||
await expect(db.transaction(async (trx) => {
|
||||
await trx.table('users').insert({ id: '1' });
|
||||
throw new Error('boom');
|
||||
})).rejects.toThrow('boom');
|
||||
expect(await db.table('users').count()).toBe(0);
|
||||
|
||||
// 级联删除
|
||||
await db.table('users').insert({ id: '1' });
|
||||
await db.table('orders').insert({ id: 'o1', user_id: '1' });
|
||||
await db.table('users').delete().where({ id: '1' }).execute();
|
||||
expect(await db.table('orders').count()).toBe(0);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
@@ -366,3 +366,103 @@ describe('KVStore — 编解码单元', () => {
|
||||
expect(decodeSnapshot(bytes)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// APPEND 追加写入(v0.6.1)
|
||||
// ===================================================================
|
||||
describe('KVStore — APPEND 追加写入', () => {
|
||||
it('appendValue 拼接 + 恢复完整(跨实例)', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const medium = new SharedMemoryBackend();
|
||||
const kv = new KVStore(medium, 0);
|
||||
await kv.open(dbName);
|
||||
await kv.appendValue('log', enc('AAA'));
|
||||
await kv.appendValue('log', enc('BBB'));
|
||||
await kv.appendValue('log', enc('CCC'));
|
||||
expect(dec(await kv.get('log'))).toBe('AAABBBCCC');
|
||||
await kv.close();
|
||||
|
||||
const kv2 = new KVStore(medium, 0);
|
||||
await kv2.open(dbName);
|
||||
expect(dec(await kv2.get('log'))).toBe('AAABBBCCC');
|
||||
await kv2.close();
|
||||
});
|
||||
|
||||
it('append 与 put/delete 混合 + checkpoint 后恢复', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const medium = new SharedMemoryBackend();
|
||||
const kv = new KVStore(medium, 0);
|
||||
await kv.open(dbName);
|
||||
await kv.appendValue('log', enc('A'));
|
||||
await kv.put('meta', enc('M'));
|
||||
await kv.appendValue('log', enc('B'));
|
||||
await kv.checkpoint();
|
||||
await kv.appendValue('log', enc('C'));
|
||||
await kv.put('meta2', enc('M2'));
|
||||
await kv.close();
|
||||
|
||||
const kv2 = new KVStore(medium, 0);
|
||||
await kv2.open(dbName);
|
||||
expect(dec(await kv2.get('log'))).toBe('ABC');
|
||||
expect(dec(await kv2.get('meta'))).toBe('M');
|
||||
expect(dec(await kv2.get('meta2'))).toBe('M2');
|
||||
await kv2.close();
|
||||
});
|
||||
|
||||
it('append 覆盖语义:先 put 后 append 拼接', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const medium = new SharedMemoryBackend();
|
||||
const kv = new KVStore(medium, 0);
|
||||
await kv.open(dbName);
|
||||
await kv.put('f', enc('HEAD-'));
|
||||
await kv.appendValue('f', enc('BODY'));
|
||||
expect(dec(await kv.get('f'))).toBe('HEAD-BODY');
|
||||
await kv.close();
|
||||
|
||||
const kv2 = new KVStore(medium, 0);
|
||||
await kv2.open(dbName);
|
||||
expect(dec(await kv2.get('f'))).toBe('HEAD-BODY');
|
||||
await kv2.close();
|
||||
});
|
||||
|
||||
it('append 后 delete → 删除生效(索引与恢复一致)', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const medium = new SharedMemoryBackend();
|
||||
const kv = new KVStore(medium, 0);
|
||||
await kv.open(dbName);
|
||||
await kv.appendValue('log', enc('X'));
|
||||
await kv.delete('log');
|
||||
expect(await kv.exists('log')).toBe(false);
|
||||
await kv.close();
|
||||
|
||||
const kv2 = new KVStore(medium, 0);
|
||||
await kv2.open(dbName);
|
||||
expect(await kv2.exists('log')).toBe(false);
|
||||
await kv2.close();
|
||||
});
|
||||
|
||||
it('日志损坏截断后已确认的 append 数据保留', async () => {
|
||||
const dbName = uniqueDB();
|
||||
const medium = new SharedMemoryBackend();
|
||||
const kv = new KVStore(medium, 0);
|
||||
await kv.open(dbName);
|
||||
await kv.appendValue('log', enc('KEEP'));
|
||||
await kv.close();
|
||||
|
||||
// 追加损坏记录
|
||||
const disk = new SharedMemoryBackend();
|
||||
await disk.open(dbName);
|
||||
const log = await disk.read('__kv_log');
|
||||
const bad = new Uint8Array(encodeLogRecord(999, { x: enc('X') }));
|
||||
bad[15] ^= 0xff;
|
||||
const combined = new Uint8Array((log as ArrayBuffer).byteLength + bad.byteLength);
|
||||
combined.set(new Uint8Array(log as ArrayBuffer), 0);
|
||||
combined.set(bad, (log as ArrayBuffer).byteLength);
|
||||
await disk.write('__kv_log', combined.buffer as ArrayBuffer);
|
||||
|
||||
const kv2 = new KVStore(medium, 0);
|
||||
await kv2.open(dbName);
|
||||
expect(dec(await kv2.get('log'))).toBe('KEEP');
|
||||
await kv2.close();
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user