fix: v0.6.0 复查修正 — KVStore 快照损坏水位 bug(metaSeq 误跳日志)+ 快照损坏测试盲区修复(此前篡改未生效)+ metaSeq 回归测试 + KVStoreEngine 真实浏览器 e2e(3 用例)+ 版本/配置文档校准
This commit is contained in:
+2
-2
@@ -29,7 +29,7 @@ All notable changes to MetonaSqlark will be documented in this file.
|
|||||||
- **可靠性强化**:
|
- **可靠性强化**:
|
||||||
- 10 万 key 写入+checkpoint+重开全量验证(928ms)
|
- 10 万 key 写入+checkpoint+重开全量验证(928ms)
|
||||||
- 5 万混合操作+崩溃模拟零丢失
|
- 5 万混合操作+崩溃模拟零丢失
|
||||||
- e2e 新增写入中途崩溃 / checkpoint 前后崩溃注入(9 用例)
|
- e2e 新增写入中途崩溃 / checkpoint 前后崩溃注入 / KVStoreEngine 真实环境持久化-索引-事务(12 用例)
|
||||||
- **`MetonaSqlark` 配置**:`diskEngine` 类型收敛为 `'opfs' | 'memory'`(默认 'opfs')
|
- **`MetonaSqlark` 配置**:`diskEngine` 类型收敛为 `'opfs' | 'memory'`(默认 'opfs')
|
||||||
|
|
||||||
### Removed
|
### Removed
|
||||||
@@ -48,7 +48,7 @@ All notable changes to MetonaSqlark will be documented in this file.
|
|||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
- 测试 1009 → **1021**(64 套件;+58 新增 kvstore/kvstore-engine/压力/迁移测试,-46 删除/改造 IDB 测试)
|
- 测试 1009 → **1022**(64 套件;+58 新增 kvstore/kvstore-engine/压力/迁移测试,-46 删除/改造 IDB 测试)
|
||||||
- 行覆盖率 87.25% → **89.13%**
|
- 行覆盖率 87.25% → **89.13%**
|
||||||
- 浏览器支持收敛到 OPFS 矩阵(Chrome/Edge 102+、Firefox 111+、Safari 15.2+);Node 用内存介质
|
- 浏览器支持收敛到 OPFS 矩阵(Chrome/Edge 102+、Firefox 111+、Safari 15.2+);Node 用内存介质
|
||||||
- fake-indexeddb 保留为 devDependency(仅迁移工具测试用)
|
- fake-indexeddb 保留为 devDependency(仅迁移工具测试用)
|
||||||
|
|||||||
@@ -3,8 +3,8 @@
|
|||||||
<p align="center">
|
<p align="center">
|
||||||
<img src="https://img.shields.io/badge/version-0.6.0-blue?style=flat-square" alt="version">
|
<img src="https://img.shields.io/badge/version-0.6.0-blue?style=flat-square" alt="version">
|
||||||
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
||||||
<img src="https://img.shields.io/badge/coverage-89.1%25-brightgreen?style=flat-square" alt="coverage">
|
<img src="https://img.shields.io/badge/coverage-89.15%25-brightgreen?style=flat-square" alt="coverage">
|
||||||
<img src="https://img.shields.io/badge/tests-1021%20passed-success?style=flat-square" alt="tests">
|
<img src="https://img.shields.io/badge/tests-1022%20passed-success?style=flat-square" alt="tests">
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。
|
> 基于 TypeScript 的**前端关系型数据库**,支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研存储引擎**。
|
||||||
@@ -34,7 +34,7 @@
|
|||||||
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
|
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复,O(log n) 保证
|
||||||
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
||||||
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
||||||
- 🧪 **1021 测试 · 89.1% 覆盖率** — 64 套件 + 9 个 Playwright 真实 Chromium e2e(含崩溃注入),生产级质量保证
|
- 🧪 **1022 测试 · 89.15% 覆盖率** — 64 套件 + 12 个 Playwright 真实 Chromium e2e(含崩溃注入),生产级质量保证
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -109,7 +109,7 @@ import { MetonaSqlark } from '@metona-team/metona-sqlark';
|
|||||||
const db = await MetonaSqlark.create({
|
const db = await MetonaSqlark.create({
|
||||||
name: 'my-app',
|
name: 'my-app',
|
||||||
mode: 'hybrid', // 'memory' | 'disk' | 'hybrid' | 'aria'
|
mode: 'hybrid', // 'memory' | 'disk' | 'hybrid' | 'aria'
|
||||||
diskEngine: 'indexeddb', // 'indexeddb' | 'opfs'
|
diskEngine: 'opfs', // 'opfs' | 'memory'(v0.6.0: IndexedDB 已移除)
|
||||||
});
|
});
|
||||||
|
|
||||||
// 定义表 — 支持外键级联
|
// 定义表 — 支持外键级联
|
||||||
@@ -216,7 +216,7 @@ await db2.disconnect(); // 引用计数 -1
|
|||||||
|------|------|------|------|
|
|------|------|------|------|
|
||||||
| `name` | `string` | `'metona-sqlark'` | 数据库名称 |
|
| `name` | `string` | `'metona-sqlark'` | 数据库名称 |
|
||||||
| `mode` | `'memory' \| 'disk' \| 'hybrid' \| 'aria'` | `'hybrid'` | 存储模式 🆕 aria |
|
| `mode` | `'memory' \| 'disk' \| 'hybrid' \| 'aria'` | `'hybrid'` | 存储模式 🆕 aria |
|
||||||
| `diskEngine` | `'indexeddb' \| 'opfs'` | `'indexeddb'` | 磁盘引擎(aria 模式下为存储后端) |
|
| `diskEngine` | `'opfs' \| 'memory'` | `'opfs'` | 磁盘引擎(aria 模式下为存储后端;v0.6.0: IndexedDB 已移除) |
|
||||||
| `version` | `number` | `1` | 版本号 |
|
| `version` | `number` | `1` | 版本号 |
|
||||||
| `maxRowsPerQuery` | `number` | `0` | 查询结果行数上限(0=不限制)✅ v0.2.5 生效 |
|
| `maxRowsPerQuery` | `number` | `0` | 查询结果行数上限(0=不限制)✅ v0.2.5 生效 |
|
||||||
| `debug` | `boolean` | `false` | 调试模式,输出详细日志 🆕 |
|
| `debug` | `boolean` | `false` | 调试模式,输出详细日志 🆕 |
|
||||||
@@ -383,7 +383,7 @@ AriaEngine 是内置的页面式存储引擎,对标 SQLite 的设计理念:
|
|||||||
const db = await MetonaSqlark.create({
|
const db = await MetonaSqlark.create({
|
||||||
name: 'my-app',
|
name: 'my-app',
|
||||||
mode: 'aria', // 🆕 自研引擎模式
|
mode: 'aria', // 🆕 自研引擎模式
|
||||||
diskEngine: 'opfs', // 'indexeddb' | 'opfs' | 'memory'
|
diskEngine: 'opfs', // 'opfs' | 'memory'
|
||||||
aria: { // 🆕 AriaEngine 专属配置透传(v0.5.0)
|
aria: { // 🆕 AriaEngine 专属配置透传(v0.5.0)
|
||||||
walSyncMode: 'full', // WAL 同步模式
|
walSyncMode: 'full', // WAL 同步模式
|
||||||
encryption: { password: 'my-password' }, // 全库 AES-GCM 加密
|
encryption: { password: 'my-password' }, // 全库 AES-GCM 加密
|
||||||
@@ -403,7 +403,7 @@ const rows = await db.query('SELECT * FROM users');
|
|||||||
|
|
||||||
```
|
```
|
||||||
┌──────────────────────────────────────────┐
|
┌──────────────────────────────────────────┐
|
||||||
│ AriaEngine v0.5.1 │
|
│ AriaEngine v0.6.0 │
|
||||||
│ (implements IStorageEngine) │
|
│ (implements IStorageEngine) │
|
||||||
├──────────────────────────────────────────┤
|
├──────────────────────────────────────────┤
|
||||||
│ LSM-Tree │ Buffer Pool │ WAL │
|
│ LSM-Tree │ Buffer Pool │ WAL │
|
||||||
@@ -457,9 +457,9 @@ npm run typecheck # 类型检查
|
|||||||
|
|
||||||
| 指标 | 数值 |
|
| 指标 | 数值 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| 测试用例 | 1021 |
|
| 测试用例 | 1022 |
|
||||||
| 测试套件 | 64(+9 Playwright e2e) |
|
| 测试套件 | 64(+12 Playwright e2e) |
|
||||||
| 行覆盖率 | 89.1% |
|
| 行覆盖率 | 89.15% |
|
||||||
| SQL 关键字 | 72 |
|
| SQL 关键字 | 72 |
|
||||||
| 存储引擎 | 5(Memory / **KVStore** / OPFS / Hybrid / **Aria**) |
|
| 存储引擎 | 5(Memory / **KVStore** / OPFS / Hybrid / **Aria**) |
|
||||||
|
|
||||||
|
|||||||
Vendored
+8
-23
@@ -984,10 +984,6 @@ class SharedMemoryBackend {
|
|||||||
static clearRegistry() {
|
static clearRegistry() {
|
||||||
registry.clear();
|
registry.clear();
|
||||||
}
|
}
|
||||||
/** 注册表中的库数量(测试诊断用) */
|
|
||||||
static registrySize() {
|
|
||||||
return registry.size;
|
|
||||||
}
|
|
||||||
async open(name) {
|
async open(name) {
|
||||||
this.dbName = name;
|
this.dbName = name;
|
||||||
if (!registry.has(name)) {
|
if (!registry.has(name)) {
|
||||||
@@ -1400,10 +1396,6 @@ class KVStore {
|
|||||||
this.medium = medium ?? defaultMedium();
|
this.medium = medium ?? defaultMedium();
|
||||||
this.checkpointThreshold = checkpointThreshold;
|
this.checkpointThreshold = checkpointThreshold;
|
||||||
}
|
}
|
||||||
/** 底层介质(测试/诊断用) */
|
|
||||||
getMedium() {
|
|
||||||
return this.medium;
|
|
||||||
}
|
|
||||||
isOpen() {
|
isOpen() {
|
||||||
return this.opened;
|
return this.opened;
|
||||||
}
|
}
|
||||||
@@ -1419,17 +1411,8 @@ class KVStore {
|
|||||||
this.index = new Map();
|
this.index = new Map();
|
||||||
this.seq = 0;
|
this.seq = 0;
|
||||||
this.logBytes = 0;
|
this.logBytes = 0;
|
||||||
// 1. 读 meta(可能缺失/损坏)
|
// 1. 加载快照(损坏则全量日志重放;快照内嵌 seq 为日志水位权威)
|
||||||
let metaSeq = 0;
|
// meta(checkpoint 写入)仅标记库存在,不参与水位判断
|
||||||
const metaRaw = await this.medium.read(META_KEY);
|
|
||||||
if (metaRaw) {
|
|
||||||
try {
|
|
||||||
const meta = JSON.parse(new TextDecoder().decode(metaRaw));
|
|
||||||
metaSeq = Number(meta.seq) || 0;
|
|
||||||
}
|
|
||||||
catch { /* meta 损坏:回退全量日志 */ }
|
|
||||||
}
|
|
||||||
// 2. 加载快照(损坏则全量日志重放)
|
|
||||||
let snapshotSeq = 0;
|
let snapshotSeq = 0;
|
||||||
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
||||||
if (snapshotRaw) {
|
if (snapshotRaw) {
|
||||||
@@ -1447,10 +1430,15 @@ class KVStore {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
// 3. 重放日志(seq > 快照水位的记录)
|
// 3. 重放日志(seq > 快照水位的记录)
|
||||||
|
// v0.6.1-fix: 快照水位只信任快照内嵌 seq(snapshotSeq)。
|
||||||
|
// 此前 baseSeq = max(metaSeq, snapshotSeq):快照损坏回退全量重放时,
|
||||||
|
// meta.seq(最后一次 checkpoint 水位)会错误跳过日志中 checkpoint 后
|
||||||
|
// 的有效记录(日志被截断过,metaSeq 不代表日志内容水位)→ 丢数据。
|
||||||
|
// meta 仅在库存在性标记,不参与水位判断。
|
||||||
const logRaw = await this.medium.read(LOG_KEY);
|
const logRaw = await this.medium.read(LOG_KEY);
|
||||||
if (logRaw && logRaw.byteLength > 0) {
|
if (logRaw && logRaw.byteLength > 0) {
|
||||||
const log = new Uint8Array(logRaw);
|
const log = new Uint8Array(logRaw);
|
||||||
const baseSeq = Math.max(metaSeq, snapshotSeq);
|
const baseSeq = snapshotSeq;
|
||||||
const corruptOffsets = [];
|
const corruptOffsets = [];
|
||||||
const applied = parseLogRecords(log, (record) => {
|
const applied = parseLogRecords(log, (record) => {
|
||||||
if (record.seq <= baseSeq)
|
if (record.seq <= baseSeq)
|
||||||
@@ -1521,9 +1509,6 @@ class KVStore {
|
|||||||
size() {
|
size() {
|
||||||
return this.index.size;
|
return this.index.size;
|
||||||
}
|
}
|
||||||
getSeq() {
|
|
||||||
return this.seq;
|
|
||||||
}
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// 写入(原子事务)
|
// 写入(原子事务)
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|||||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+8
-23
@@ -980,10 +980,6 @@ class SharedMemoryBackend {
|
|||||||
static clearRegistry() {
|
static clearRegistry() {
|
||||||
registry.clear();
|
registry.clear();
|
||||||
}
|
}
|
||||||
/** 注册表中的库数量(测试诊断用) */
|
|
||||||
static registrySize() {
|
|
||||||
return registry.size;
|
|
||||||
}
|
|
||||||
async open(name) {
|
async open(name) {
|
||||||
this.dbName = name;
|
this.dbName = name;
|
||||||
if (!registry.has(name)) {
|
if (!registry.has(name)) {
|
||||||
@@ -1396,10 +1392,6 @@ class KVStore {
|
|||||||
this.medium = medium ?? defaultMedium();
|
this.medium = medium ?? defaultMedium();
|
||||||
this.checkpointThreshold = checkpointThreshold;
|
this.checkpointThreshold = checkpointThreshold;
|
||||||
}
|
}
|
||||||
/** 底层介质(测试/诊断用) */
|
|
||||||
getMedium() {
|
|
||||||
return this.medium;
|
|
||||||
}
|
|
||||||
isOpen() {
|
isOpen() {
|
||||||
return this.opened;
|
return this.opened;
|
||||||
}
|
}
|
||||||
@@ -1415,17 +1407,8 @@ class KVStore {
|
|||||||
this.index = new Map();
|
this.index = new Map();
|
||||||
this.seq = 0;
|
this.seq = 0;
|
||||||
this.logBytes = 0;
|
this.logBytes = 0;
|
||||||
// 1. 读 meta(可能缺失/损坏)
|
// 1. 加载快照(损坏则全量日志重放;快照内嵌 seq 为日志水位权威)
|
||||||
let metaSeq = 0;
|
// meta(checkpoint 写入)仅标记库存在,不参与水位判断
|
||||||
const metaRaw = await this.medium.read(META_KEY);
|
|
||||||
if (metaRaw) {
|
|
||||||
try {
|
|
||||||
const meta = JSON.parse(new TextDecoder().decode(metaRaw));
|
|
||||||
metaSeq = Number(meta.seq) || 0;
|
|
||||||
}
|
|
||||||
catch { /* meta 损坏:回退全量日志 */ }
|
|
||||||
}
|
|
||||||
// 2. 加载快照(损坏则全量日志重放)
|
|
||||||
let snapshotSeq = 0;
|
let snapshotSeq = 0;
|
||||||
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
||||||
if (snapshotRaw) {
|
if (snapshotRaw) {
|
||||||
@@ -1443,10 +1426,15 @@ class KVStore {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
// 3. 重放日志(seq > 快照水位的记录)
|
// 3. 重放日志(seq > 快照水位的记录)
|
||||||
|
// v0.6.1-fix: 快照水位只信任快照内嵌 seq(snapshotSeq)。
|
||||||
|
// 此前 baseSeq = max(metaSeq, snapshotSeq):快照损坏回退全量重放时,
|
||||||
|
// meta.seq(最后一次 checkpoint 水位)会错误跳过日志中 checkpoint 后
|
||||||
|
// 的有效记录(日志被截断过,metaSeq 不代表日志内容水位)→ 丢数据。
|
||||||
|
// meta 仅在库存在性标记,不参与水位判断。
|
||||||
const logRaw = await this.medium.read(LOG_KEY);
|
const logRaw = await this.medium.read(LOG_KEY);
|
||||||
if (logRaw && logRaw.byteLength > 0) {
|
if (logRaw && logRaw.byteLength > 0) {
|
||||||
const log = new Uint8Array(logRaw);
|
const log = new Uint8Array(logRaw);
|
||||||
const baseSeq = Math.max(metaSeq, snapshotSeq);
|
const baseSeq = snapshotSeq;
|
||||||
const corruptOffsets = [];
|
const corruptOffsets = [];
|
||||||
const applied = parseLogRecords(log, (record) => {
|
const applied = parseLogRecords(log, (record) => {
|
||||||
if (record.seq <= baseSeq)
|
if (record.seq <= baseSeq)
|
||||||
@@ -1517,9 +1505,6 @@ class KVStore {
|
|||||||
size() {
|
size() {
|
||||||
return this.index.size;
|
return this.index.size;
|
||||||
}
|
}
|
||||||
getSeq() {
|
|
||||||
return this.seq;
|
|
||||||
}
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// 写入(原子事务)
|
// 写入(原子事务)
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|||||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+8
-23
@@ -986,10 +986,6 @@
|
|||||||
static clearRegistry() {
|
static clearRegistry() {
|
||||||
registry.clear();
|
registry.clear();
|
||||||
}
|
}
|
||||||
/** 注册表中的库数量(测试诊断用) */
|
|
||||||
static registrySize() {
|
|
||||||
return registry.size;
|
|
||||||
}
|
|
||||||
async open(name) {
|
async open(name) {
|
||||||
this.dbName = name;
|
this.dbName = name;
|
||||||
if (!registry.has(name)) {
|
if (!registry.has(name)) {
|
||||||
@@ -1402,10 +1398,6 @@
|
|||||||
this.medium = medium ?? defaultMedium();
|
this.medium = medium ?? defaultMedium();
|
||||||
this.checkpointThreshold = checkpointThreshold;
|
this.checkpointThreshold = checkpointThreshold;
|
||||||
}
|
}
|
||||||
/** 底层介质(测试/诊断用) */
|
|
||||||
getMedium() {
|
|
||||||
return this.medium;
|
|
||||||
}
|
|
||||||
isOpen() {
|
isOpen() {
|
||||||
return this.opened;
|
return this.opened;
|
||||||
}
|
}
|
||||||
@@ -1421,17 +1413,8 @@
|
|||||||
this.index = new Map();
|
this.index = new Map();
|
||||||
this.seq = 0;
|
this.seq = 0;
|
||||||
this.logBytes = 0;
|
this.logBytes = 0;
|
||||||
// 1. 读 meta(可能缺失/损坏)
|
// 1. 加载快照(损坏则全量日志重放;快照内嵌 seq 为日志水位权威)
|
||||||
let metaSeq = 0;
|
// meta(checkpoint 写入)仅标记库存在,不参与水位判断
|
||||||
const metaRaw = await this.medium.read(META_KEY);
|
|
||||||
if (metaRaw) {
|
|
||||||
try {
|
|
||||||
const meta = JSON.parse(new TextDecoder().decode(metaRaw));
|
|
||||||
metaSeq = Number(meta.seq) || 0;
|
|
||||||
}
|
|
||||||
catch { /* meta 损坏:回退全量日志 */ }
|
|
||||||
}
|
|
||||||
// 2. 加载快照(损坏则全量日志重放)
|
|
||||||
let snapshotSeq = 0;
|
let snapshotSeq = 0;
|
||||||
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
||||||
if (snapshotRaw) {
|
if (snapshotRaw) {
|
||||||
@@ -1449,10 +1432,15 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
// 3. 重放日志(seq > 快照水位的记录)
|
// 3. 重放日志(seq > 快照水位的记录)
|
||||||
|
// v0.6.1-fix: 快照水位只信任快照内嵌 seq(snapshotSeq)。
|
||||||
|
// 此前 baseSeq = max(metaSeq, snapshotSeq):快照损坏回退全量重放时,
|
||||||
|
// meta.seq(最后一次 checkpoint 水位)会错误跳过日志中 checkpoint 后
|
||||||
|
// 的有效记录(日志被截断过,metaSeq 不代表日志内容水位)→ 丢数据。
|
||||||
|
// meta 仅在库存在性标记,不参与水位判断。
|
||||||
const logRaw = await this.medium.read(LOG_KEY);
|
const logRaw = await this.medium.read(LOG_KEY);
|
||||||
if (logRaw && logRaw.byteLength > 0) {
|
if (logRaw && logRaw.byteLength > 0) {
|
||||||
const log = new Uint8Array(logRaw);
|
const log = new Uint8Array(logRaw);
|
||||||
const baseSeq = Math.max(metaSeq, snapshotSeq);
|
const baseSeq = snapshotSeq;
|
||||||
const corruptOffsets = [];
|
const corruptOffsets = [];
|
||||||
const applied = parseLogRecords(log, (record) => {
|
const applied = parseLogRecords(log, (record) => {
|
||||||
if (record.seq <= baseSeq)
|
if (record.seq <= baseSeq)
|
||||||
@@ -1523,9 +1511,6 @@
|
|||||||
size() {
|
size() {
|
||||||
return this.index.size;
|
return this.index.size;
|
||||||
}
|
}
|
||||||
getSeq() {
|
|
||||||
return this.seq;
|
|
||||||
}
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// 写入(原子事务)
|
// 写入(原子事务)
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|||||||
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
@@ -555,7 +555,7 @@ SELECT COUNT(*) as total FROM temp_logs;
|
|||||||
|
|
||||||
-- 清理
|
-- 清理
|
||||||
DROP TABLE temp_logs;`,
|
DROP TABLE temp_logs;`,
|
||||||
aria: `-- 🌲 AriaEngine 演示 (v0.5.1)
|
aria: `-- 🌲 AriaEngine 演示 (v0.6.0)
|
||||||
-- 点击右上角「🌲 Aria」按钮切换数据库引擎到 AriaEngine
|
-- 点击右上角「🌲 Aria」按钮切换数据库引擎到 AriaEngine
|
||||||
-- 当前数据库即运行在 Aria 引擎上(LSM-Tree · WAL 崩溃恢复 · MVCC · BloomFilter)
|
-- 当前数据库即运行在 Aria 引擎上(LSM-Tree · WAL 崩溃恢复 · MVCC · BloomFilter)
|
||||||
-- 基础 CRUD 与 Memory 引擎完全兼容
|
-- 基础 CRUD 与 Memory 引擎完全兼容
|
||||||
|
|||||||
+6
-6
@@ -152,7 +152,7 @@ npm install @metona-team/metona-sqlark</pre>
|
|||||||
<pre><span class="k">const</span> db = <span class="k">await</span> <span class="f">MetonaSqlark.create</span>({
|
<pre><span class="k">const</span> db = <span class="k">await</span> <span class="f">MetonaSqlark.create</span>({
|
||||||
<span class="s">name</span>: <span class="s">'my-app'</span>,
|
<span class="s">name</span>: <span class="s">'my-app'</span>,
|
||||||
<span class="s">mode</span>: <span class="s">'hybrid'</span>, <span class="c">// 'memory' | 'disk' | 'hybrid' | 'aria' 🆕</span>
|
<span class="s">mode</span>: <span class="s">'hybrid'</span>, <span class="c">// 'memory' | 'disk' | 'hybrid' | 'aria' 🆕</span>
|
||||||
<span class="s">diskEngine</span>: <span class="s">'indexeddb'</span>, <span class="c">// 'indexeddb' | 'opfs'(仅 disk/hybrid 生效)</span>
|
<span class="s">diskEngine</span>: <span class="s">'opfs'</span>, <span class="c">// 'opfs' | 'memory'(v0.6.0: IndexedDB 已移除)</span>
|
||||||
<span class="s">version</span>: <span class="n">1</span>,
|
<span class="s">version</span>: <span class="n">1</span>,
|
||||||
<span class="s">plugins</span>: [], <span class="c">// MetonaPlugin[]</span>
|
<span class="s">plugins</span>: [], <span class="c">// MetonaPlugin[]</span>
|
||||||
<span class="s">onReady</span>: (db) => {}, <span class="c">// 就绪回调</span>
|
<span class="s">onReady</span>: (db) => {}, <span class="c">// 就绪回调</span>
|
||||||
@@ -737,7 +737,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<tr><th>属性</th><th>类型</th><th>默认值</th><th>说明</th></tr>
|
<tr><th>属性</th><th>类型</th><th>默认值</th><th>说明</th></tr>
|
||||||
<tr><td><code>name</code></td><td><code>string</code></td><td><code>'metona-sqlark'</code></td><td>数据库名称(必填)</td></tr>
|
<tr><td><code>name</code></td><td><code>string</code></td><td><code>'metona-sqlark'</code></td><td>数据库名称(必填)</td></tr>
|
||||||
<tr><td><code>mode</code></td><td><code>'memory'|'disk'|'hybrid'|'aria'</code></td><td><code>'hybrid'</code></td><td>存储模式 🆕 aria</td></tr>
|
<tr><td><code>mode</code></td><td><code>'memory'|'disk'|'hybrid'|'aria'</code></td><td><code>'hybrid'</code></td><td>存储模式 🆕 aria</td></tr>
|
||||||
<tr><td><code>diskEngine</code></td><td><code>'indexeddb'|'opfs'</code></td><td><code>'indexeddb'</code></td><td>磁盘引擎类型</td></tr>
|
<tr><td><code>diskEngine</code></td><td><code>'opfs'|'memory'</code></td><td><code>'opfs'</code></td><td>磁盘引擎类型(v0.6.0: IndexedDB 已移除)</td></tr>
|
||||||
<tr><td><code>version</code></td><td><code>number</code></td><td><code>1</code></td><td>数据库版本号</td></tr>
|
<tr><td><code>version</code></td><td><code>number</code></td><td><code>1</code></td><td>数据库版本号</td></tr>
|
||||||
<tr><td><code>plugins</code></td><td><code>MetonaPlugin[]</code></td><td><code>[]</code></td><td>初始插件列表</td></tr>
|
<tr><td><code>plugins</code></td><td><code>MetonaPlugin[]</code></td><td><code>[]</code></td><td>初始插件列表</td></tr>
|
||||||
<tr><td><code>onReady</code></td><td><code>(db) => void</code></td><td>-</td><td>初始化完成回调</td></tr>
|
<tr><td><code>onReady</code></td><td><code>(db) => void</code></td><td>-</td><td>初始化完成回调</td></tr>
|
||||||
@@ -767,7 +767,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<strong>v0.4.4 SSTable 编码修复</strong> — 大段中文内容(如 300KB 笔记)写入 AriaEngine 不再崩溃:块大小估算改 UTF-8 字节精确计算(修复中文 3 字节 vs 1 码元导致的缓冲区低估越界)· 长度字段 u16 → u32(修复 >64KB value 截断)· 大 value 独立成块 · 格式 v2("SSTC")与 v1("SSTB")双格式兼容(旧库数据不丢)· 中文主键 / 大内容索引列同步支持。<br>
|
<strong>v0.4.4 SSTable 编码修复</strong> — 大段中文内容(如 300KB 笔记)写入 AriaEngine 不再崩溃:块大小估算改 UTF-8 字节精确计算(修复中文 3 字节 vs 1 码元导致的缓冲区低估越界)· 长度字段 u16 → u32(修复 >64KB value 截断)· 大 value 独立成块 · 格式 v2("SSTC")与 v1("SSTB")双格式兼容(旧库数据不丢)· 中文主键 / 大内容索引列同步支持。<br>
|
||||||
<strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e(7 用例)· DatabaseConfig.aria 配置透传。<br>
|
<strong>v0.5.0 存储后端生产级硬化</strong> — 真实 CRC-32 完整性校验(SSTable 整文件 + WAL 记录,旧文件兼容)· 全库 AES-256-GCM 透明加密(EncryptedBackend + PBKDF2 密钥派生 + 密码验证)· WAL 分片文件重构(真追加 + 空洞检测 + 旧格式迁移)· SSTable 4KB 页面化物理存储(BufferPool/FileManager 真实接入,meta 存 pageIds 兼容旧数据)· OPFS 后端 v2(append 真追加 / 写队列健壮性 / 残留清理)· Web Locks 多标签页独占锁(ARIA_LOCKED)· LZ4 v2 原始大小头 · Playwright 真实 Chromium e2e(7 用例)· DatabaseConfig.aria 配置透传。<br>
|
||||||
<strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)。<br>
|
<strong>v0.5.1 深度审查修复</strong> — 14 个生命周期钩子全部真实接线(此前 6 个 CRUD 钩子从未触发)· EXPLAIN / ANALYZE / REINDEX / VACUUM / SAVEPOINT SQL 入口补齐(此前仅有引擎方法无法触发)· db.backup() 公共方法 · 删除全部死代码(utils.ts 整文件 / MVCC 读侧 / estimateQueryCost 未接线优化器 / 40+ 统计辅助方法)。<br>
|
||||||
<strong>v0.6.0 完全移除 IndexedDB</strong> — 自研 KVStore 事务存储引擎(多 key 原子写 = 单日志记录原子追加 · 快照 checkpoint + 两阶段崩溃恢复 · CRC-32 自愈)· disk 模式切换 KVStoreEngine(替代 IndexedDBEngine + OPFSEngine)· 事务内 DDL / 外键级联 / 二级索引完整持久化 · migrateFromIndexedDB() 旧库一键迁移 · 10 万 key 压力验证 · e2e 崩溃注入(9 用例)· 1021 测试 64 套件 · 89.1% 行覆盖率。</p>
|
<strong>v0.6.0 完全移除 IndexedDB</strong> — 自研 KVStore 事务存储引擎(多 key 原子写 = 单日志记录原子追加 · 快照 checkpoint + 两阶段崩溃恢复 · CRC-32 自愈)· disk 模式切换 KVStoreEngine(替代 IndexedDBEngine + OPFSEngine)· 事务内 DDL / 外键级联 / 二级索引完整持久化 · migrateFromIndexedDB() 旧库一键迁移 · 10 万 key 压力验证 · e2e 崩溃注入 + KVStoreEngine 真实环境(12 用例)· 1022 测试 64 套件 · 89.15% 行覆盖率。</p>
|
||||||
|
|
||||||
<h3>存储模式对比</h3>
|
<h3>存储模式对比</h3>
|
||||||
<table>
|
<table>
|
||||||
@@ -798,7 +798,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<span class="k">const</span> db = <span class="k">await</span> <span class="f">MetonaSqlark.create</span>({
|
<span class="k">const</span> db = <span class="k">await</span> <span class="f">MetonaSqlark.create</span>({
|
||||||
<span class="s">name</span>: <span class="s">'my-app'</span>,
|
<span class="s">name</span>: <span class="s">'my-app'</span>,
|
||||||
<span class="s">mode</span>: <span class="s">'aria'</span>, <span class="c">// 🆕 AriaEngine 模式</span>
|
<span class="s">mode</span>: <span class="s">'aria'</span>, <span class="c">// 🆕 AriaEngine 模式</span>
|
||||||
<span class="s">diskEngine</span>: <span class="s">'opfs'</span>, <span class="c">// 底层存储后端(indexeddb | opfs | memory)</span>
|
<span class="s">diskEngine</span>: <span class="s">'opfs'</span>, <span class="c">// 底层存储后端(opfs | memory)</span>
|
||||||
<span class="s">aria</span>: { <span class="c">// AriaEngine 专属配置透传 🆕 v0.5.0</span>
|
<span class="s">aria</span>: { <span class="c">// AriaEngine 专属配置透传 🆕 v0.5.0</span>
|
||||||
<span class="s">walSyncMode</span>: <span class="s">'full'</span>, <span class="c">// WAL 同步模式</span>
|
<span class="s">walSyncMode</span>: <span class="s">'full'</span>, <span class="c">// WAL 同步模式</span>
|
||||||
<span class="s">encryption</span>: { <span class="s">password</span>: <span class="s">'my-password'</span> }, <span class="c">// 全库 AES-GCM 加密 🆕 v0.5.0</span>
|
<span class="s">encryption</span>: { <span class="s">password</span>: <span class="s">'my-password'</span> }, <span class="c">// 全库 AES-GCM 加密 🆕 v0.5.0</span>
|
||||||
@@ -811,7 +811,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<span class="s">bufferPoolPages</span>: <span class="n">256</span>, <span class="c">// Buffer Pool 页面数量</span>
|
<span class="s">bufferPoolPages</span>: <span class="n">256</span>, <span class="c">// Buffer Pool 页面数量</span>
|
||||||
<span class="s">memtableSizeThreshold</span>: <span class="n">4194304</span>, <span class="c">// MemTable 刷盘阈值 4MB</span>
|
<span class="s">memtableSizeThreshold</span>: <span class="n">4194304</span>, <span class="c">// MemTable 刷盘阈值 4MB</span>
|
||||||
<span class="s">walSyncMode</span>: <span class="s">'full'</span>, <span class="c">// 'full' | 'batch' | 'none'(默认 full)</span>
|
<span class="s">walSyncMode</span>: <span class="s">'full'</span>, <span class="c">// 'full' | 'batch' | 'none'(默认 full)</span>
|
||||||
<span class="s">storageBackend</span>: <span class="s">'indexeddb'</span>, <span class="c">// 存储后端</span>
|
<span class="s">storageBackend</span>: <span class="s">'opfs'</span>, <span class="c">// 存储后端(opfs | memory)</span>
|
||||||
<span class="s">encryption</span>: { <span class="s">password</span>: <span class="s">'my-password'</span> }, <span class="c">// 全库加密</span>
|
<span class="s">encryption</span>: { <span class="s">password</span>: <span class="s">'my-password'</span> }, <span class="c">// 全库加密</span>
|
||||||
});</pre>
|
});</pre>
|
||||||
|
|
||||||
@@ -828,7 +828,7 @@ db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
|||||||
<tr><td><code>checkpointInterval</code></td><td><code>number</code></td><td><code>1000</code></td><td>Checkpoint 触发间隔(操作数)</td></tr>
|
<tr><td><code>checkpointInterval</code></td><td><code>number</code></td><td><code>1000</code></td><td>Checkpoint 触发间隔(操作数)</td></tr>
|
||||||
<tr><td><code>walSizeThreshold</code></td><td><code>number</code></td><td><code>16777216</code></td><td>WAL 大小阈值(字节),超阈值触发 checkpoint ✅ v0.2.5</td></tr>
|
<tr><td><code>walSizeThreshold</code></td><td><code>number</code></td><td><code>16777216</code></td><td>WAL 大小阈值(字节),超阈值触发 checkpoint ✅ v0.2.5</td></tr>
|
||||||
<tr><td><code>compression</code></td><td><code>boolean</code></td><td><code>false</code></td><td>是否启用页面压缩</td></tr>
|
<tr><td><code>compression</code></td><td><code>boolean</code></td><td><code>false</code></td><td>是否启用页面压缩</td></tr>
|
||||||
<tr><td><code>storageBackend</code></td><td><code>'indexeddb'|'opfs'|'memory'</code></td><td><code>'indexeddb'</code></td><td>存储后端类型</td></tr>
|
<tr><td><code>storageBackend</code></td><td><code>'opfs'|'memory'</code></td><td><code>'opfs'</code></td><td>存储后端类型(v0.6.0: IndexedDB 已移除)</td></tr>
|
||||||
<tr><td><code>encryption</code></td><td><code>{ password: string }</code></td><td>-</td><td>全库 AES-256-GCM 透明加密(PBKDF2 派生 + salt 持久化 + 密码验证)🆕 v0.5.0</td></tr>
|
<tr><td><code>encryption</code></td><td><code>{ password: string }</code></td><td>-</td><td>全库 AES-256-GCM 透明加密(PBKDF2 派生 + salt 持久化 + 密码验证)🆕 v0.5.0</td></tr>
|
||||||
<tr><td><code>pageStorage</code></td><td><code>boolean</code></td><td>OPFS 自动启用</td><td>SSTable 4KB 页面化物理存储(BufferPool 缓存)🆕 v0.5.0</td></tr>
|
<tr><td><code>pageStorage</code></td><td><code>boolean</code></td><td>OPFS 自动启用</td><td>SSTable 4KB 页面化物理存储(BufferPool 缓存)🆕 v0.5.0</td></tr>
|
||||||
</table>
|
</table>
|
||||||
|
|||||||
+4
-4
@@ -153,7 +153,7 @@
|
|||||||
<!-- Hero -->
|
<!-- Hero -->
|
||||||
<section class="hero">
|
<section class="hero">
|
||||||
<div class="container">
|
<div class="container">
|
||||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.6.0 完全移除 IndexedDB — 1021测试 64套件 · 自研 KVStore 事务引擎 · 多 key 原子写 · 崩溃恢复 · 旧库一键迁移</div>
|
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.6.0 完全移除 IndexedDB — 1022测试 64套件 · 自研 KVStore 事务引擎 · 多 key 原子写 · 崩溃恢复 · 旧库一键迁移</div>
|
||||||
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
||||||
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
<p>TypeScript 原生构建,5 种存储引擎,支持完整 SQL 查询。<br>零运行时依赖,开箱即用。AriaEngine 自研引擎:LSM-Tree + WAL 同步 + MVCC。</p>
|
||||||
<div class="actions">
|
<div class="actions">
|
||||||
@@ -243,7 +243,7 @@ npm install @metona-team/metona-sqlark
|
|||||||
</div>
|
</div>
|
||||||
<div class="feature-card">
|
<div class="feature-card">
|
||||||
<div class="icon">🌲</div>
|
<div class="icon">🌲</div>
|
||||||
<h3>AriaEngine <span style="font-size:0.65rem;color:var(--accent);vertical-align:super;">v0.5.1</span></h3>
|
<h3>AriaEngine <span style="font-size:0.65rem;color:var(--accent);vertical-align:super;">v0.6.0</span></h3>
|
||||||
<p>自研 LSM-Tree 存储引擎:SSTable 4KB 页面化物理存储(BufferPool LRU)、WAL 分片文件原子写入、全库 AES-GCM 透明加密、标准 CRC-32 完整性校验、MVCC 快照隔离、二级索引跨重启恢复、ON UPDATE/DELETE 外键级联、崩溃恢复自愈。</p>
|
<p>自研 LSM-Tree 存储引擎:SSTable 4KB 页面化物理存储(BufferPool LRU)、WAL 分片文件原子写入、全库 AES-GCM 透明加密、标准 CRC-32 完整性校验、MVCC 快照隔离、二级索引跨重启恢复、ON UPDATE/DELETE 外键级联、崩溃恢复自愈。</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="feature-card">
|
<div class="feature-card">
|
||||||
@@ -414,8 +414,8 @@ npm install @metona-team/metona-sqlark
|
|||||||
<p>MetonaSqlark 的核心指标</p>
|
<p>MetonaSqlark 的核心指标</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="stats">
|
<div class="stats">
|
||||||
<div class="stat-card"><div class="num">1021</div><div class="label">测试用例</div></div>
|
<div class="stat-card"><div class="num">1022</div><div class="label">测试用例</div></div>
|
||||||
<div class="stat-card"><div class="num">89.1%</div><div class="label">行覆盖率</div></div>
|
<div class="stat-card"><div class="num">89.15%</div><div class="label">行覆盖率</div></div>
|
||||||
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
||||||
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
||||||
<div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div>
|
<div class="stat-card"><div class="num">72</div><div class="label">SQL 关键字</div></div>
|
||||||
|
|||||||
@@ -73,11 +73,6 @@ export class KVStore {
|
|||||||
this.checkpointThreshold = checkpointThreshold;
|
this.checkpointThreshold = checkpointThreshold;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 底层介质(测试/诊断用) */
|
|
||||||
getMedium(): IStorageBackend {
|
|
||||||
return this.medium;
|
|
||||||
}
|
|
||||||
|
|
||||||
isOpen(): boolean {
|
isOpen(): boolean {
|
||||||
return this.opened;
|
return this.opened;
|
||||||
}
|
}
|
||||||
@@ -95,17 +90,8 @@ export class KVStore {
|
|||||||
this.seq = 0;
|
this.seq = 0;
|
||||||
this.logBytes = 0;
|
this.logBytes = 0;
|
||||||
|
|
||||||
// 1. 读 meta(可能缺失/损坏)
|
// 1. 加载快照(损坏则全量日志重放;快照内嵌 seq 为日志水位权威)
|
||||||
let metaSeq = 0;
|
// meta(checkpoint 写入)仅标记库存在,不参与水位判断
|
||||||
const metaRaw = await this.medium.read(META_KEY);
|
|
||||||
if (metaRaw) {
|
|
||||||
try {
|
|
||||||
const meta = JSON.parse(new TextDecoder().decode(metaRaw)) as KVStoreMeta;
|
|
||||||
metaSeq = Number(meta.seq) || 0;
|
|
||||||
} catch { /* meta 损坏:回退全量日志 */ }
|
|
||||||
}
|
|
||||||
|
|
||||||
// 2. 加载快照(损坏则全量日志重放)
|
|
||||||
let snapshotSeq = 0;
|
let snapshotSeq = 0;
|
||||||
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
const snapshotRaw = await this.medium.read(SNAPSHOT_KEY);
|
||||||
if (snapshotRaw) {
|
if (snapshotRaw) {
|
||||||
@@ -123,10 +109,15 @@ export class KVStore {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 3. 重放日志(seq > 快照水位的记录)
|
// 3. 重放日志(seq > 快照水位的记录)
|
||||||
|
// v0.6.1-fix: 快照水位只信任快照内嵌 seq(snapshotSeq)。
|
||||||
|
// 此前 baseSeq = max(metaSeq, snapshotSeq):快照损坏回退全量重放时,
|
||||||
|
// meta.seq(最后一次 checkpoint 水位)会错误跳过日志中 checkpoint 后
|
||||||
|
// 的有效记录(日志被截断过,metaSeq 不代表日志内容水位)→ 丢数据。
|
||||||
|
// meta 仅在库存在性标记,不参与水位判断。
|
||||||
const logRaw = await this.medium.read(LOG_KEY);
|
const logRaw = await this.medium.read(LOG_KEY);
|
||||||
if (logRaw && logRaw.byteLength > 0) {
|
if (logRaw && logRaw.byteLength > 0) {
|
||||||
const log = new Uint8Array(logRaw);
|
const log = new Uint8Array(logRaw);
|
||||||
const baseSeq = Math.max(metaSeq, snapshotSeq);
|
const baseSeq = snapshotSeq;
|
||||||
const corruptOffsets: number[] = [];
|
const corruptOffsets: number[] = [];
|
||||||
const applied = parseLogRecords(log, (record) => {
|
const applied = parseLogRecords(log, (record) => {
|
||||||
if (record.seq <= baseSeq) return; // 快照已包含,跳过(幂等)
|
if (record.seq <= baseSeq) return; // 快照已包含,跳过(幂等)
|
||||||
@@ -198,10 +189,6 @@ export class KVStore {
|
|||||||
return this.index.size;
|
return this.index.size;
|
||||||
}
|
}
|
||||||
|
|
||||||
getSeq(): number {
|
|
||||||
return this.seq;
|
|
||||||
}
|
|
||||||
|
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
// 写入(原子事务)
|
// 写入(原子事务)
|
||||||
// =======================================================================
|
// =======================================================================
|
||||||
|
|||||||
@@ -23,11 +23,6 @@ export class SharedMemoryBackend implements IStorageBackend {
|
|||||||
registry.clear();
|
registry.clear();
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 注册表中的库数量(测试诊断用) */
|
|
||||||
static registrySize(): number {
|
|
||||||
return registry.size;
|
|
||||||
}
|
|
||||||
|
|
||||||
async open(name: string): Promise<void> {
|
async open(name: string): Promise<void> {
|
||||||
this.dbName = name;
|
this.dbName = name;
|
||||||
if (!registry.has(name)) {
|
if (!registry.has(name)) {
|
||||||
|
|||||||
+10
-6
@@ -17,23 +17,27 @@
|
|||||||
/**
|
/**
|
||||||
* 创建并打开 Aria 库。
|
* 创建并打开 Aria 库。
|
||||||
* opts: {
|
* opts: {
|
||||||
* name, diskEngine ('opfs'|'indexeddb'|'memory'),
|
* name, diskEngine ('opfs'|'memory'),
|
||||||
* walSyncMode, checkpointInterval,
|
* walSyncMode, checkpointInterval,
|
||||||
* encryptionPassword, memtableSizeThreshold
|
* encryptionPassword, memtableSizeThreshold
|
||||||
* }
|
* }
|
||||||
*/
|
*/
|
||||||
open: async (opts) => {
|
open: async (opts) => {
|
||||||
const db = await window.MetonaSqlark.create({
|
// opts.mode: 'aria'(默认)| 'disk'(KVStoreEngine)
|
||||||
|
const config = {
|
||||||
name: opts.name,
|
name: opts.name,
|
||||||
mode: 'aria',
|
mode: opts.mode || 'aria',
|
||||||
diskEngine: opts.diskEngine || 'opfs',
|
diskEngine: opts.diskEngine || 'opfs',
|
||||||
aria: {
|
};
|
||||||
|
if (opts.mode !== 'disk') {
|
||||||
|
config.aria = {
|
||||||
walSyncMode: opts.walSyncMode || 'full',
|
walSyncMode: opts.walSyncMode || 'full',
|
||||||
checkpointInterval: opts.checkpointInterval || 100000,
|
checkpointInterval: opts.checkpointInterval || 100000,
|
||||||
memtableSizeThreshold: opts.memtableSizeThreshold || 64 * 1024 * 1024,
|
memtableSizeThreshold: opts.memtableSizeThreshold || 64 * 1024 * 1024,
|
||||||
...(opts.encryptionPassword ? { encryption: { password: opts.encryptionPassword } } : {}),
|
...(opts.encryptionPassword ? { encryption: { password: opts.encryptionPassword } } : {}),
|
||||||
},
|
};
|
||||||
});
|
}
|
||||||
|
const db = await window.MetonaSqlark.create(config);
|
||||||
window.__ms.__db = db;
|
window.__ms.__db = db;
|
||||||
return { ok: true, engineName: db.getEngine().name };
|
return { ok: true, engineName: db.getEngine().name };
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -327,3 +327,82 @@ test.describe('OPFS 真实环境', () => {
|
|||||||
await run(page, 'close');
|
await run(page, 'close');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test.describe('KVStoreEngine(disk 模式)真实环境', () => {
|
||||||
|
test('写入 → 刷新页面 → 数据保留(KVStore 持久化)', async ({ page }) => {
|
||||||
|
await openPage(page);
|
||||||
|
await run(page, 'open', { name: 'e2e-kv-persist', mode: 'disk', diskEngine: 'opfs' });
|
||||||
|
await run(page, 'createTable', {
|
||||||
|
table: 'kvusers',
|
||||||
|
columns: [
|
||||||
|
{ name: 'id', type: 'string', primaryKey: true },
|
||||||
|
{ name: 'name', type: 'string' },
|
||||||
|
{ name: 'tag', type: 'string', index: true },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
for (let i = 0; i < 30; i++) {
|
||||||
|
await run(page, 'insert', {
|
||||||
|
table: 'kvusers',
|
||||||
|
rows: [{ id: `u-${i}`, name: `User${i}`, tag: i % 2 === 0 ? 'even' : 'odd' }],
|
||||||
|
});
|
||||||
|
}
|
||||||
|
expect(await run<{ count: number }>(page, 'count', { table: 'kvusers' })).toEqual({ count: 30 });
|
||||||
|
|
||||||
|
// 刷新:KVStore 快照/日志恢复
|
||||||
|
await page.reload();
|
||||||
|
await expect(page.locator('#status')).toHaveText('ready');
|
||||||
|
await run(page, 'open', { name: 'e2e-kv-persist', mode: 'disk', diskEngine: 'opfs' });
|
||||||
|
const res = await run<{ count: number }>(page, 'count', { table: 'kvusers' });
|
||||||
|
expect(res.count).toBe(30);
|
||||||
|
// 索引查询(重启重建)
|
||||||
|
const evens = await run<{ rows: Record<string, unknown>[] }>(page, 'find', {
|
||||||
|
table: 'kvusers', where: { tag: 'even' },
|
||||||
|
});
|
||||||
|
expect(evens.rows.length).toBe(15);
|
||||||
|
await run(page, 'close');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('崩溃恢复:写入后强制终止 → 重开日志重放数据完整', async ({ page }) => {
|
||||||
|
await openPage(page);
|
||||||
|
await run(page, 'open', { name: 'e2e-kv-crash', mode: 'disk', diskEngine: 'opfs' });
|
||||||
|
await run(page, 'createTable', {
|
||||||
|
table: 'logs', columns: [{ name: 'id', type: 'string', primaryKey: true }],
|
||||||
|
});
|
||||||
|
const total = 40;
|
||||||
|
for (let i = 0; i < total; i++) {
|
||||||
|
await run(page, 'insert', { table: 'logs', rows: [{ id: `log-${i}` }] });
|
||||||
|
}
|
||||||
|
// 模拟崩溃:不 close
|
||||||
|
await crashPage(page);
|
||||||
|
|
||||||
|
const page2 = await page.context().newPage();
|
||||||
|
await openPage(page2);
|
||||||
|
await run(page2, 'open', { name: 'e2e-kv-crash', mode: 'disk', diskEngine: 'opfs' });
|
||||||
|
const res = await run<{ count: number }>(page2, 'count', { table: 'logs' });
|
||||||
|
expect(res.count).toBe(total);
|
||||||
|
await run(page2, 'close');
|
||||||
|
await page2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('事务 commit 原子性:多表写入后重启一致', async ({ page }) => {
|
||||||
|
await openPage(page);
|
||||||
|
await run(page, 'open', { name: 'e2e-kv-tx', mode: 'disk', diskEngine: 'opfs' });
|
||||||
|
await run(page, 'createTable', {
|
||||||
|
table: 't1', columns: [{ name: 'id', type: 'string', primaryKey: true }],
|
||||||
|
});
|
||||||
|
await run(page, 'createTable', {
|
||||||
|
table: 't2', columns: [{ name: 'id', type: 'string', primaryKey: true }],
|
||||||
|
});
|
||||||
|
// 用 SQL 多语句触发原子写(事务)
|
||||||
|
await run(page, 'query', { sql: "BEGIN; INSERT INTO t1 VALUES ('a'); INSERT INTO t2 VALUES ('b'); COMMIT;" });
|
||||||
|
await crashPage(page);
|
||||||
|
|
||||||
|
const page2 = await page.context().newPage();
|
||||||
|
await openPage(page2);
|
||||||
|
await run(page2, 'open', { name: 'e2e-kv-tx', mode: 'disk', diskEngine: 'opfs' });
|
||||||
|
expect(await run<{ count: number }>(page2, 'count', { table: 't1' })).toEqual({ count: 1 });
|
||||||
|
expect(await run<{ count: number }>(page2, 'count', { table: 't2' })).toEqual({ count: 1 });
|
||||||
|
await run(page2, 'close');
|
||||||
|
await page2.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -122,28 +122,54 @@ describe('KVStore — 持久化与崩溃恢复', () => {
|
|||||||
await kv2.close();
|
await kv2.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('快照损坏 → 全量日志重放自愈(数据不丢)', async () => {
|
it('快照损坏 → 全量日志重放:日志数据保留,快照唯一副本丢失(自愈不崩)', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const medium = new SharedMemoryBackend();
|
const medium = new SharedMemoryBackend();
|
||||||
const kv1 = new KVStore(medium, 0);
|
const kv1 = new KVStore(medium, 0);
|
||||||
await kv1.open(dbName);
|
await kv1.open(dbName);
|
||||||
await kv1.putMany({ a: enc('AAA'), b: enc('BBB') });
|
await kv1.putMany({ a: enc('AAA'), b: enc('BBB') });
|
||||||
await kv1.checkpoint();
|
await kv1.checkpoint(); // 快照成为 a/b 的唯一副本(日志已截断)
|
||||||
await kv1.put('c', enc('CCC'));
|
await kv1.put('c', enc('CCC')); // 日志中(seq=2)
|
||||||
await kv1.close();
|
await kv1.close();
|
||||||
|
|
||||||
// 篡改快照(模拟损坏)
|
// 篡改快照(close 后介质不可用,重新打开访问"磁盘")
|
||||||
const snap = await medium.read('__kv_snapshot');
|
const disk = new SharedMemoryBackend();
|
||||||
|
await disk.open(dbName);
|
||||||
|
const snap = await disk.read('__kv_snapshot');
|
||||||
|
expect(snap).not.toBeNull();
|
||||||
const corrupted = new Uint8Array(snap as ArrayBuffer);
|
const corrupted = new Uint8Array(snap as ArrayBuffer);
|
||||||
corrupted[20] ^= 0xff;
|
corrupted[20] ^= 0xff; // 破坏 entry 长度字段 → CRC/边界校验失败
|
||||||
await medium.write('__kv_snapshot', corrupted.buffer as ArrayBuffer);
|
await disk.write('__kv_snapshot', corrupted.buffer as ArrayBuffer);
|
||||||
|
|
||||||
const kv2 = new KVStore(medium, 0);
|
const kv2 = new KVStore(medium, 0);
|
||||||
await kv2.open(dbName);
|
await kv2.open(dbName);
|
||||||
// 快照损坏 → 全量日志重放
|
// 快照损坏 → 全量日志重放:c(日志中)保留;a/b(快照唯一副本)丢失
|
||||||
|
expect(await kv2.exists('a')).toBe(false);
|
||||||
|
expect(await kv2.exists('b')).toBe(false);
|
||||||
|
expect(dec(await kv2.get('c'))).toBe('CCC');
|
||||||
|
await kv2.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('回归:快照完好但 meta.seq 超前 → 日志不被 metaSeq 跳过(水位只信任快照)', async () => {
|
||||||
|
const dbName = uniqueDB();
|
||||||
|
const medium = new SharedMemoryBackend();
|
||||||
|
const kv1 = new KVStore(medium, 0);
|
||||||
|
await kv1.open(dbName);
|
||||||
|
await kv1.put('a', enc('AAA'));
|
||||||
|
await kv1.checkpoint(); // 快照 seq=1,meta seq=1,日志截断
|
||||||
|
await kv1.put('b', enc('BBB')); // 日志 seq=2
|
||||||
|
await kv1.close();
|
||||||
|
|
||||||
|
// 手工把 meta.seq 改大(模拟异常状态:meta 比日志内容超前)
|
||||||
|
const disk = new SharedMemoryBackend();
|
||||||
|
await disk.open(dbName);
|
||||||
|
await disk.write('__kv_meta', new TextEncoder().encode(JSON.stringify({ seq: 999 })).buffer);
|
||||||
|
|
||||||
|
const kv2 = new KVStore(medium, 0);
|
||||||
|
await kv2.open(dbName);
|
||||||
|
// 快照完好(seq=1)→ 水位=1 → 日志 seq2 应用;meta.seq=999 不参与水位判断
|
||||||
expect(dec(await kv2.get('a'))).toBe('AAA');
|
expect(dec(await kv2.get('a'))).toBe('AAA');
|
||||||
expect(dec(await kv2.get('b'))).toBe('BBB');
|
expect(dec(await kv2.get('b'))).toBe('BBB');
|
||||||
expect(dec(await kv2.get('c'))).toBe('CCC');
|
|
||||||
await kv2.close();
|
await kv2.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -351,7 +351,7 @@ describe('[v0.3.2] 多标签页同步', () => {
|
|||||||
// 回归:IndexedDB reopen 后 schema 持久化(v0.3.2 修复)
|
// 回归:IndexedDB reopen 后 schema 持久化(v0.3.2 修复)
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
|
|
||||||
describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
describe('[v0.3.2] KVStore reopen schema 持久化', () => {
|
||||||
test('close 后重新 open 表结构与数据完整', async () => {
|
test('close 后重新 open 表结构与数据完整', async () => {
|
||||||
const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'opfs' });
|
const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'opfs' });
|
||||||
await setup.init();
|
await setup.init();
|
||||||
|
|||||||
@@ -121,7 +121,7 @@ describe('v0.1.13 事务回滚 — Memory 模式', () => {
|
|||||||
// IndexedDB 引擎事务回滚
|
// IndexedDB 引擎事务回滚
|
||||||
// ===================================================================
|
// ===================================================================
|
||||||
|
|
||||||
describe('v0.1.13 事务回滚 — Disk 模式 (IndexedDB)', () => {
|
describe('v0.1.13 事务回滚 — Disk 模式 (KVStore)', () => {
|
||||||
let db: MetonaSqlark;
|
let db: MetonaSqlark;
|
||||||
let dbName: string;
|
let dbName: string;
|
||||||
|
|
||||||
|
|||||||
@@ -214,7 +214,7 @@ describe('P0-1b — AriaEngine 打开时完整性校验', () => {
|
|||||||
// ===================================================================
|
// ===================================================================
|
||||||
|
|
||||||
describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
describe('P0-1c / P1-4 — WAL 原子性与恢复', () => {
|
||||||
it('IndexedDBBackend.writeMany/deleteMany 单事务原子批量操作', async () => {
|
it('OPFSBackend.writeMany/deleteMany 批量原子操作', async () => {
|
||||||
const dbName = uniqueDB();
|
const dbName = uniqueDB();
|
||||||
const be = new OPFSBackend();
|
const be = new OPFSBackend();
|
||||||
await be.open(dbName);
|
await be.open(dbName);
|
||||||
|
|||||||
Reference in New Issue
Block a user