release: v0.3.2 — 质量加固 + SQL扩展 + 表达式 + 并发同步
v0.2.6 质量加固: - 修复 AriaEngine 二级索引 SSTable 互相覆盖(命名空间隔离) - 修复 LSM 多版本读取顺序错误 + MergeIterator 取最新来源 - 重写 LZ4 压缩器(往返一致性 + 缓冲区溢出) - sstableCache LRU 上限 + 预加载兜底(BufferPool 配置生效) - 修复 React/Vue 集成 import type 运行时 bug + exports 子路径 - 新增 38 个测试(LZ4往返/Crypto/集成), 删除伪测试 v0.3.0 SQL 功能扩展: - 多语句 parseAll + 事务语句 BEGIN/COMMIT/ROLLBACK - INSERT INTO ... SELECT + UNION/UNION ALL + EXISTS 关联子查询 - CREATE/DROP INDEX 五引擎实现 + 别名 WHERE 修复 - benchmark 页面 + 36 个新测试 v0.3.1 表达式与性能: - CASE WHEN 表达式(SELECT 列/WHERE/聚合) - JOIN + 关联子查询逐行绑定 - WAL 批量组提交(写放大 O(N)→O(1)) - 修复 pending frozen 可见性 + flush 缓存竞争 v0.3.2 并发: - CASE WHEN 用于 WHERE/聚合 + JOIN 哈希连接 - 多标签页同步(multiTabSync + BroadcastChannel) - IndexedDB schema 持久化(reopen 后表结构恢复) - 修复 where-matcher 顶层 $not - 修复 CJS 产物 .js 被 ESM 解析(exports 空) — .cjs 后缀 + exports 修正 - 836 测试 / 44 套件 / 81.0% 覆盖率
This commit is contained in:
thzxx
+336
-241
@@ -1,241 +1,336 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to MetonaSqlark will be documented in this file.
|
||||
|
||||
## [0.2.5] - 2026-07-29
|
||||
|
||||
### Fixed
|
||||
- **版本号统一** — `constants.ts` VERSION 从 `'0.2.0'` 更新为 `'0.2.5'`，修正 `index.ts`/`utils.ts`/`CONTRIBUTING.md` 中过时的版本注释和数据
|
||||
- **AriaEngine OPFS 后端映射** — `core.ts` 中 `mode: 'aria'` + `diskEngine: 'opfs'` 时实际使用 Memory 后端的 bug 已修复
|
||||
- **`_onError` 接入执行路径** — `query()`/`defineTable()`/`dropTable()`/`transaction()`/`importTable()` 的 catch 路径现在调用 `_onError` 全局错误回调
|
||||
- **`maxRowsPerQuery` 生效** — `QueryExecutor` 构造时接收 `maxRowsPerQuery` 参数，SELECT 结果在返回前截断
|
||||
- **WAL full 模式真正同步** — `WAL.append()` 改为 `async`，`full` 模式下 `await this.store.append()` 真正等待写入完成，不再 fire-and-forget
|
||||
- **PluginManager.install 传 db 实例** — `register()` 增加可选 `db` 参数，`core.ts` 初始化时传入 `this`，插件可获取 db 引用
|
||||
|
||||
### Changed
|
||||
- **SSTableReader 二分查找统一** — `locateBlockGE`/`locateBlockLE` 从线性扫描改为二分查找，rangeScan 性能在大型 SSTable 下不再退化
|
||||
- **crypto 实例化** — 全局状态改为 `CryptoManager` 类，每个 AriaEngine 实例可拥有独立加密配置，保留全局函数向后兼容
|
||||
- **compactLevelSync 接口公开化** — LSM 新增 public `compactLevel()` 方法，`vacuum()` 不再使用 `as any` 绕过 private 访问
|
||||
- **WAL 大小阈值接入 checkpoint** — `CheckpointManager` 接收 `walSizeThreshold` 参数，WAL 缓冲超阈值时自动触发 checkpoint
|
||||
|
||||
### Added
|
||||
- **ALTER TABLE 语法** — 支持 `ALTER TABLE ... ADD COLUMN` / `DROP COLUMN`（含可选 COLUMN 关键字）
|
||||
- **TRUNCATE TABLE 语法** — 支持 `TRUNCATE TABLE name` 快速清空表数据
|
||||
- **MVCC 接入读写路径** — 事务内 insert/update/delete 调用 `mvcc.writeVersion`/`mvcc.deleteVersion`，版本链作为 undo log
|
||||
- **IndexedDB 索引利用** — `IndexedDBEngine.find` 等值查询时优先使用 IDB 索引（`idx_col` 命名约定），避免全量 getAll
|
||||
- **SQL 注入防护** — React `useTable` / Vue `useSqlarkTable` 增加表名合法性校验（`/^[a-zA-Z_][a-zA-Z0-9_]*$/`）
|
||||
|
||||
---
|
||||
|
||||
## [0.2.4] - 2026-07-27
|
||||
|
||||
### Added
|
||||
- **二级索引** — 每列可独立维护 LSM Tree 索引，`$eq`/`$in`/`$gt`/`$lt` 走索引 O(log n)
|
||||
- **MVCC 接入引擎** — MVCCManager 正式投产，替换 ad-hoc txnSnapshot
|
||||
- **MVCC 自动 GC** — 每 10 次 checkpoint 自动回收过旧版本（保留最新 100 个）
|
||||
- **Bloom Filter 序列化** — 写入 SSTable footer + 读取时加载 + 查询时 probe 快速否定
|
||||
- **WAL 大小阈值** — `walSizeThreshold` 配置（默认 16MB），超阈值强制 checkpoint
|
||||
- **内存预算** — `maxMemoryMB` 配置（默认 64MB）
|
||||
|
||||
### Changed
|
||||
- **WAL 默认同步模式** — `walSyncMode` 从 `'batch'` 改为 `'full'`，消除 crash 丢数据风险
|
||||
- **查询优化** — `tryIndexLookup` 扩展支持非 PK 列索引查找
|
||||
|
||||
---
|
||||
|
||||
## [0.2.3] - 2026-07-27
|
||||
|
||||
### Fixed
|
||||
- **RB-Tree fixDelete 完整实现** — 补全标准红黑树删除修复，保证 O(log n)
|
||||
- **LSM SSTable 缓存预热** — `init()` 预加载所有 SSTable，消除 cache miss
|
||||
- **OPFS 数据恢复** — `open()` 自动从 OPFS 文件加载已有表数据到内存
|
||||
- **Aria WAL 事务恢复** — 两阶段恢复：仅回放已提交事务，未提交数据不回放
|
||||
- **LZ4 格式修复** — 重写 token 格式，消除中间字面量 token 歧义
|
||||
|
||||
---
|
||||
|
||||
## [0.2.2] - 2026-07-27
|
||||
|
||||
### Added
|
||||
- **OPFS 自研存储后端** — AriaEngine 新增 `OPFSBackend`，纯浏览器文件系统 API，零 IndexedDB 依赖
|
||||
- 每个 key 对应一个二进制文件，存储在 `navigator.storage.getDirectory()` 下
|
||||
- 支持 read/write/delete/list/exists/clear 完整接口
|
||||
- 页面文件（4KB）、WAL 日志、Schema、SSTable 全部存储为独立文件
|
||||
- `{ dbName }/pg_1`, `{ dbName }/__wal_0`, `{ dbName }/__aria_schemas` ...
|
||||
|
||||
### Changed
|
||||
- `AriaEngineConfig.storageBackend` 已支持 `'opfs'`，`AriaEngine.open()` 自动选择 OPFSBackend
|
||||
|
||||
---
|
||||
|
||||
## [0.2.1] - 2026-07-27
|
||||
|
||||
### Fixed
|
||||
- **WAL CRC 校验验证** — 恢复时计算并验证 CRC32，损坏记录自动跳过并告警
|
||||
- **Hybrid 引擎提交顺序** — commit 先写磁盘再写内存，磁盘失败回滚内存，消除数据不一致风险
|
||||
- **RESTRICT 外键行为修正** — 检测到引用行时抛出 `FOREIGN_KEY_VIOLATION`，不再静默保留孤儿
|
||||
- **SSTable rangeScan 边界保护** — 空索引或 startBlockIdx > endBlockIdx 时安全返回
|
||||
|
||||
### Added
|
||||
- **ColumnDef 约束激活** — `maxLength`/`min`/`max` 约束在 `checkFieldType` 中正式生效
|
||||
- **查询结果上限** — `DatabaseConfig.maxRowsPerQuery`（默认 0 不限制），防止超大结果集 OOM
|
||||
- **onError 全局回调** — 配置中的 `onError` 回调实际接入 CRUD 异常路径
|
||||
- **debug 调试模式** — `DatabaseConfig.debug: true` 输出 `[MetonaSqlark:name]` 前缀的结构化日志
|
||||
- **浏览器兼容性声明** — README 增加 Chrome 80+/Firefox 80+/Safari 14+/Edge 80+ 支持矩阵
|
||||
|
||||
### Changed
|
||||
- MemoryEngine RESTRICT 外键行为：从"不级联保留孤儿"改为"禁止删除抛异常"
|
||||
- Hybrid 引擎 commitTransaction 磁盘优先于内存
|
||||
- DB_DEFAULTS 新增 `maxRowsPerQuery: 0` 和 `debug: false`
|
||||
|
||||
---
|
||||
|
||||
## [0.2.0] - 2026-07-27
|
||||
|
||||
### Added
|
||||
- **AriaEngine 自研存储引擎** — 基于 LSM-Tree 的页面式存储引擎，19 个新模块，~3500 行 TypeScript
|
||||
- **页面格式层** (`page/`): 4KB Slotted Page 编解码、Tuple 二进制序列化、CRC32 校验
|
||||
- **Buffer Pool** (`buffer/`): LRU 页面缓存 + 驱逐策略，可控内存占用
|
||||
- **LSM-Tree 索引** (`index/`): MemTable (红黑树) + 多级 SSTable + Leveled Compaction
|
||||
- **Bloom Filter** (`index/bloom.ts`): FNV-1a + Murmur 双哈希，快速键否定判定
|
||||
- **SSTable 构建器/读取器** (`index/sstable_builder.ts`, `sstable.ts`): 二分查找 + 范围扫描
|
||||
- **Merge Iterator** (`index/merge_iterator.ts`): 最小堆多路归并，去重保留最新值
|
||||
- **WAL** (`wal/`): 二进制日志格式 (LSN/type/txnId/table/key/json/CRC) + Checkpoint 管理
|
||||
- **MVCC** (`transaction/mvcc.ts`): 版本链 + 快照隔离 + GC
|
||||
- **存储后端** (`store/`): IndexedDB / Memory 双后端抽象
|
||||
- **LZ4 压缩** (`compression/lz4.ts`): 简易页面级压缩
|
||||
- **`mode: 'aria'`** — 新增存储模式，可通过 `MetonaSqlark.create({ mode: 'aria' })` 激活
|
||||
- **Schema 持久化** — 表结构自动保存到 `__aria_schemas`，重启自动恢复
|
||||
- **SSTable 元数据管理** — SSTable 索引信息持久化，启动时自动扫描加载
|
||||
- **事务感知 CRUD** — insert/update/delete 在事务中缓冲到 snapshot，commit 批量写入 LSM
|
||||
- **244 个 AriaEngine 专项测试** — 覆盖生命周期/表管理/CRUD/事务/持久化/SQL 集成/页面格式/LSM/压缩
|
||||
|
||||
### Changed
|
||||
- `StorageMode` 类型新增 `'aria'`
|
||||
- `STORAGE_MODES` 数组新增 `'aria'`
|
||||
- `createEngine()` 支持 `mode: 'aria'` 分支
|
||||
- 测试从 318 → **526**，套件从 20 → **27**
|
||||
- 新增 7 个模块级测试文件：`aria-page`、`aria-index`、`aria-sstable`、`aria-buffer`、`aria-wal-mvcc`、`aria-compress`、`aria`
|
||||
- LRUList 修复 size 追踪 bug
|
||||
- WAL 存储改为按记录独立 key（避免拼接缓冲区越界）
|
||||
- CheckpointManager 解耦 BufferPool 依赖
|
||||
|
||||
### Fixed
|
||||
- **LZ4 压缩无限循环** — 字面量分支在发现匹配后回退导致 litLen=0 死循环，CI 卡死根因
|
||||
- **SSTableReader.get() 自比较 bug** — 参数 key 被循环变量遮蔽导致永远返回第一条
|
||||
- **CheckpointManager 测试 null 引用** — 改为 Mock 对象避免 TypeError 导致进程无法退出
|
||||
- **IndexedDB 持久化测试** — 替换为 Memory Backend 验证，消除 fake-indexeddb timer 堆积
|
||||
- **LSM.flush() 不必要 setTimeout** — 替换为 Promise.resolve()，消除额外 timer 延迟
|
||||
|
||||
---
|
||||
|
||||
## [0.1.14] - 2026-07-26
|
||||
|
||||
### Fixed
|
||||
- **IndexedDB 事务原子性**: `flushToIDB` 改为单 IDB 事务包裹 clear+insert，消除崩溃丢数据风险
|
||||
- **多标签页冲突**: `open()` 添加 `onversionchange` 监听，其他标签页升级版本时自动关闭过期连接
|
||||
- **Memory 引擎幂等**: `open()` 重复调用安全无副作用
|
||||
- 关闭时清理 `onversionchange` 监听器，防止内存泄漏
|
||||
|
||||
---
|
||||
|
||||
## [0.1.13] - 2026-07-26
|
||||
|
||||
### Added
|
||||
- **事务回滚机制**: `IStorageEngine` 新增 `beginTransaction/commitTransaction/rollbackTransaction` 接口
|
||||
- MemoryEngine: 快照式回滚（深拷贝 tables/schemas/indexes）
|
||||
- IndexedDBEngine: 延迟写入策略（事务中仅写内存，commit 批量刷 IDB）
|
||||
- OPFSEngine: 快照式回滚（commit 批量写文件）
|
||||
- HybridEngine: 同时代理内存+磁盘引擎事务
|
||||
- **子查询支持**: SQL Parser + Executor 支持 `IN (SELECT ...)` 和 `op (SELECT ...)` 子查询
|
||||
- AST 新增 `SubqueryExpression` 类型
|
||||
- Parser 在 IN 和比较运算符后检测子查询
|
||||
- Executor 新增 `resolveSubqueries()` 递归解析，自动执行子查询并替换为具体值
|
||||
- **外键级联操作**: ColumnDef 新增 `onDelete/onUpdate` 选项
|
||||
- 支持 `CASCADE`（递归级联删除）、`SET NULL`、`RESTRICT`
|
||||
- SQL Parser 解析 `REFERENCES table(col) ON DELETE CASCADE ON UPDATE CASCADE`
|
||||
- MemoryEngine 内置 `cascadeDelete()` 递归级联逻辑
|
||||
- **连接池管理**: `MetonaSqlark.connect()` 静态方法
|
||||
- 同名数据库复用已打开实例，引用计数管理
|
||||
- `db.disconnect()` 释放连接，归零自动 close
|
||||
- `MetonaSqlark.disconnectAll()` 强制关闭所有连接
|
||||
|
||||
### Changed
|
||||
- `TransactionManager.execute()` 调用引擎层 begin/commit/rollback 实现真正原子性
|
||||
- IndexedDBEngine CRUD 事务感知：活跃事务中延迟 IDB 写入
|
||||
- ASTColumnDef / astColumnToColumnDef 支持 references/onDelete/onUpdate 透传
|
||||
|
||||
### Fixed
|
||||
- IndexedDBEngine 事务中 find/count 从内存缓存读取（保证读到未提交变更）
|
||||
- SQL Parser parseColumnDef 循环条件扩展，避免 REFERENCES 语法解析中断
|
||||
|
||||
---
|
||||
|
||||
## [0.1.12] - 2026-07-26
|
||||
|
||||
### Added
|
||||
- `MeSqlark` 别名导出，与 `MetonaSqlark` 完全等价
|
||||
- `metona-sqlark.cjs.js` CommonJS 构建产物
|
||||
- 完善的 `where-matcher.ts` 消除 220+ 行重复代码
|
||||
- `$col` 列引用支持 JOIN ON 条件
|
||||
- `$and/$or` 嵌套支持（字段级 + 顶层）
|
||||
- `$not` 逻辑非操作符
|
||||
- `$like` 正则缓存加速
|
||||
- `projectColumns` 列投影支持 `table.column` 格式
|
||||
- `DISTINCT` 去重支持（O(n) 时间，列值拼接优化）
|
||||
- 测试覆盖率提升至 93.46%（264 测试/15 套件）
|
||||
|
||||
### Changed
|
||||
- `where-matcher` 统一 MemoryEngine / IndexedDBEngine / Executor 的 WHERE 逻辑
|
||||
- JOIN ON 匹配支持 `$col` 语法
|
||||
- LIKE 正则改为缓存式编译
|
||||
- Executor DISTINCT 用列值拼接代替 JSON.stringify
|
||||
- JOIN 嵌套循环连接优化：避免 ON 时对象扩散
|
||||
|
||||
### Fixed
|
||||
- IndexedDBEngine `update/delete` 方法正确同步内存缓存
|
||||
- OPFSEngine `getTableNames` 兼容 `entries()` 返回值
|
||||
- SQL Parser 正确处理字符串转义字符
|
||||
- SQL Parser 正确处理表别名（无 AS 关键字）
|
||||
- SQL Parser 解析 `IS NULL` / `IS NOT NULL` / `NOT LIKE`
|
||||
- `peekTokenIs` 方法正确暴露给语法分析
|
||||
- 多处边界条件空值/假值处理
|
||||
|
||||
---
|
||||
|
||||
## [0.1.11] - 2026-07-25
|
||||
|
||||
### Added
|
||||
- React 集成 hooks: `useQuery`, `useTable`, `useDatabase`
|
||||
- Vue 集成 composables: `useSqlarkQuery`, `useSqlarkTable`, `useSqlarkDatabase`
|
||||
- 发布订阅系统: `subscribe()`, `emit()`
|
||||
- 数据迁移系统: `addMigration()`, `migrateTo()`
|
||||
- 导入导出: `exportTable()`, `exportAll()`, `importTable()`
|
||||
- 完整 SQL 解析器: `Lexer` + `Parser`（递归下降）
|
||||
- SQL 支持: SELECT, INSERT, UPDATE, DELETE, CREATE TABLE, DROP TABLE
|
||||
- JOIN 支持: INNER, LEFT, RIGHT, CROSS
|
||||
- GROUP BY + HAVING + 聚合函数 (COUNT, SUM, AVG, MIN, MAX)
|
||||
- DISTINCT, ORDER BY, LIMIT, OFFSET
|
||||
- WHERE 条件: AND, OR, NOT, IN, LIKE, IS NULL
|
||||
- Query Builder 链式 API: `select().where().orderBy().limit().execute()`
|
||||
- 插件系统: 14 种生命周期钩子 + PluginManager
|
||||
- Metadata 系统: ColumnDef 完整约束 (type/primaryKey/required/unique/index/default/references/maxLength/min/max)
|
||||
|
||||
### Changed
|
||||
- 架构重构为分层设计：Engine → QueryExecutor → Table/QueryBuilder → SQL Parser → Public API
|
||||
- AST 作为统一中间表示，SQL 和 QueryBuilder 行为完全一致
|
||||
- 测试框架完善：core / edge / groupby / join / query-system / sql / table / engine / transaction / plugin / hybrid 全覆盖
|
||||
|
||||
---
|
||||
|
||||
## [0.0.1] - 2026-07-24
|
||||
|
||||
### Added
|
||||
- 初始项目骨架
|
||||
- TypeScript 配置（严格模式）
|
||||
- Rollup 构建（UMD / ESM / CJS / min / .d.ts）
|
||||
- Jest + jsdom 测试环境
|
||||
- ESLint + @typescript-eslint
|
||||
- Gitea Actions CI 工作流
|
||||
- 示例站点（index / demo / docs）
|
||||
- 构建脚本 build.sh
|
||||
# Changelog
|
||||
|
||||
All notable changes to MetonaSqlark will be documented in this file.
|
||||
|
||||
## [0.3.2] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **CASE WHEN 用于 WHERE** — `WHERE CASE WHEN cond THEN a ELSE b END = 'x'` 逐行求值（含 NOT/AND/OR 组合）
|
||||
- **CASE WHEN 用于聚合** — `SUM(CASE WHEN age > 18 THEN 1 ELSE 0 END)`、`COUNT/AVG/MIN/MAX(CASE...)`、`GROUP BY + CASE` 非聚合列
|
||||
- **JOIN 哈希连接** — 等值 ON + 右表主键/索引列时，收集左表连接值 → 一次 `$in` 查询 → 哈希映射匹配，替代嵌套循环（INNER/LEFT JOIN，O(N+M)）；不适用自动回退
|
||||
- **多标签页同步** — `multiTabSync: true` 配置 + BroadcastChannel：
|
||||
- SQL 写语句（INSERT/UPDATE/DELETE/DDL）自动广播表变更
|
||||
- Table API 写入（insert/insertMany/update/delete/clear/drop）自动广播
|
||||
- 其他标签页收到 `external` 变更事件（订阅者可见）+ Hybrid 内存自动重载
|
||||
- `db.broadcastChange(table)` 手动广播
|
||||
- **IndexedDB schema 持久化** — `__metona_schema` store 保存完整列定义，reopen（页面刷新/重连）后恢复；旧数据回退为"主键 + 索引 + 样例推断"
|
||||
|
||||
### Fixed
|
||||
- **IndexedDB reopen 后 schema 丢失（严重）** — schema 仅存内存缓存，close 后重开连接表结构即丢失（INSERT 列映射截断、UPDATE 全列失效）。v0.3.2 持久化 + 重建
|
||||
- **where-matcher 顶层 $not 不生效** — `NOT (expr)` 解析为顶层 `{ $not }` 但被当作字段匹配，始终返回 true
|
||||
- **CASE 剥离后空 `$not` 恒假** — 引擎层执行 `NOT(true)` 过滤掉所有行
|
||||
|
||||
### Changed
|
||||
- 版本号升至 v0.3.2
|
||||
|
||||
---
|
||||
|
||||
## [0.3.1] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **CASE WHEN 表达式（SELECT 列）** — `CASE WHEN cond THEN value [WHEN ...] [ELSE value] END [AS alias]`
|
||||
- 多 WHEN 按顺序匹配、ELSE 缺省返回 null
|
||||
- THEN/ELSE 值支持字面量（字符串/数字/布尔/null）与列引用
|
||||
- 条件支持 AND/OR/比较运算符/IS NULL 等完整 WHERE 语法
|
||||
- 引擎层返回原始行供求值（避免投影丢失条件列），与普通列混合投影
|
||||
- **JOIN + 关联子查询** — JOIN 结果的 WHERE 支持 `EXISTS (SELECT ... WHERE o2.x = u.y)` 与 `$col` 引用外层行，逐行绑定求值
|
||||
- **WAL 批量组提交** — `WAL.appendBatch()` 合并多条记录为一次底层写入；引擎 insert/update/delete 批量记录后一次落盘（写放大从 O(N) 降为 O(1)）
|
||||
- **`parseWhereCondition()` 导出** — 独立条件表达式解析（CASE WHEN 求值内部使用）
|
||||
|
||||
### Fixed
|
||||
- **LSM pending frozen 可见性** — 异步 flush 链期间，中间冻结的 MemTable 仅存在于闭包中，读取路径不可见导致数据缺失。新增 `frozenMemtables` 列表，get/rangeScan/getAllEntries 遍历所有 pending frozen（新→旧）
|
||||
- **flush 缓存裁剪与预加载竞争** — flush 完成时的 LRU 裁剪会驱逐 prefetch 刚加载的 SSTable，导致扫描静默丢数据。prefetchRange/prefetchKeys 先等待 flush 链完成再加载
|
||||
- **CASE 值解析顺序** — `true`/`false`/`null` 字面量被误判为列引用
|
||||
|
||||
### Changed
|
||||
- 版本号升至 v0.3.1
|
||||
|
||||
---
|
||||
|
||||
## [0.3.0] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **SQL 多语句支持** — `parseAll()` 解析分号分隔的多条语句，`db.query()` 顺序执行并返回最后一条结果（`parse()` 保持单语句兼容）
|
||||
- **事务语句（TCL）** — `BEGIN` / `COMMIT` / `ROLLBACK`（支持可选 TRANSACTION 关键字），直接驱动引擎层事务，五引擎可用
|
||||
- **INSERT INTO ... SELECT** — 支持 `INSERT INTO t (cols) SELECT ...`（含 WHERE 过滤、列映射）
|
||||
- **UNION / UNION ALL** — 支持去重合并与链式组合（`A UNION B UNION C`），右侧结果按位置投影到左侧列结构
|
||||
- **CREATE INDEX / DROP INDEX** — 动态二级索引：
|
||||
- `CREATE [UNIQUE] INDEX idx ON table (col)` / `DROP INDEX idx ON table (col)`
|
||||
- 五引擎实现：Memory 哈希索引 / IndexedDB 版本升级建 `idx_col` / OPFS 委托 / Hybrid 双写 / Aria 动态 LSM 索引
|
||||
- Aria 主键索引受保护（不可 DROP）；CREATE INDEX 对已有数据立即建索引（回填），后续写入自动同步
|
||||
- **EXISTS / NOT EXISTS** — 支持关联子查询（`WHERE EXISTS (SELECT 1 FROM o WHERE o.user_id = u.id)`），逐行绑定外层上下文求值
|
||||
- **别名 WHERE 修复** — `WHERE u.age > 20`（主表别名前缀）此前不工作，v0.3.0 字段名规范化（JOIN 路径不受影响）
|
||||
- **`SELECT 1` 常量列** — 数字字面量可作为 SELECT 列（EXISTS 子查询常见用法）
|
||||
- **benchmark 页面** — `site/benchmark.html` 浏览器内实测 Memory / Aria 引擎 × 1K/10K/50K 行的 INSERT/PK 查询/索引查询/UPDATE/DELETE 吞吐（ops/sec）
|
||||
- **`parseAll` / 新 AST 类型导出** — `SelectUnionStatement` / `CreateIndexStatement` / `DropIndexStatement` / `BeginTransactionStatement` 等
|
||||
|
||||
### Changed
|
||||
- 版本号升至 v0.3.0
|
||||
|
||||
### Fixed
|
||||
- `NOT EXISTS` 解析丢 EXISTS token 导致 parse 失败
|
||||
|
||||
---
|
||||
|
||||
## [0.2.6] - 2026-08-08
|
||||
|
||||
### Fixed
|
||||
- **AriaEngine 二级索引 SSTable 互相覆盖（严重）** — 主 LSM 与二级索引 LSM 此前共享同一 `sstableStore`（id 空间 + meta 列表），`allocateId()` 返回 `Date.now()` 且各 LSM 从同一 meta 恢复 `nextSSTableId`，高频写入时 id 冲突导致索引数据文件覆盖主数据、meta 与数据错配。v0.2.6 按命名空间隔离（文件前缀 + meta key + 独立 id 序列），索引 LSM 各持独立 store
|
||||
- **LSM 多版本读取顺序错误（严重）** — `levels[]` 数组按 `push` 追加（新文件在尾部），但 `get()`/`rangeScan` 从头遍历（旧文件优先），同一 key 跨多次 flush 更新后读到旧值；`MergeIterator` 去重只取"先出堆者"，不保证最新来源。v0.2.6 改为"数组头部即最新"（`unshift` + 按 id 降序加载），MergeIterator 显式取 sourceIndex 最小者
|
||||
- **LZ4 往返不一致** — ① 匹配长度超过 19 字节被截断但 `si` 跳过全部匹配，解压丢数据；② matchField=0 的组合 token 与纯字面量 token 格式歧义；③ 字面量累积超过 15 字节后组合 token 永远无法输出；④ 输出缓冲区 `maxOut` 不足导致越界写入；⑤ 压缩失败时返回原样 input，解压端无法区分。v0.2.6 重写压缩器（长匹配分段、纯字面量 token 语义明确、动态上限）
|
||||
- **React/Vue 集成运行时 ReferenceError** — `useDatabase`/`useSqlarkDatabase` 用 `import type` 导入 `MetonaSqlark` 但运行时 `new MetonaSqlark()`，`@ts-nocheck` 掩盖此 bug，实际使用必炸。改为 value import
|
||||
- **sstableCache 无限增长** — SSTable 缓存无内存上限且 `init()` 预加载全部数据，大库 OOM。v0.2.6 引入 LRU 容量上限（`bufferPoolPages` × `pageSize`），查询前异步预加载兜底（修复缓存未命中静默返回 null 丢数据的隐患）
|
||||
- **integrations 子路径缺失** — README 宣称 `metona-sqlark/react`、`metona-sqlark/vue` 可用但 `exports` 未声明，现已补齐（react/vue 为 peer dependency，rollup external）
|
||||
|
||||
### Changed
|
||||
- **Buffer Pool 配置真正生效** — `bufferPoolPages` 从无效配置变为 SSTable 缓存容量上限（默认 256 页 ≈ 1MB 可控内存）
|
||||
- **LZ4 格式变更** — 旧版本压缩数据与新解压器不兼容，使用 `compression: true` 的用户需重新导入数据
|
||||
- **SSTable 持久化串行化** — flush/compaction 改为串行链（`flushChain`），id 分配与持久化顺序一致，消除 fire-and-forget 竞态
|
||||
- 文档数据修正 — 测试 761（41 套件）、覆盖率 81.1%（此前 91.0% 为排除 Aria 模块的陈旧数据）、体积 ~105KB / gzip ~27KB
|
||||
|
||||
### Added
|
||||
- **AriaEngine 缓存内存上限测试** — 9 个测试：LRU 上限约束、驱逐后全表/PK/索引查询完整性、UPDATE/DELETE 全量作用、写路径上限、命名空间隔离回归、多版本回归、tombstone 回归
|
||||
- **LZ4 往返一致性测试** — 6 个往返测试（重复/文本/随机/边界长度/15 字节边界/长匹配）
|
||||
- **CryptoManager 加解密测试** — 7 个测试（往返/错误密码/不同 salt/实例隔离/全局兼容层/4KB 页面）
|
||||
- **React/Vue hooks 集成测试** — 16 个测试（jest.mock 零依赖：mount 执行、错误路径、refresh、注入防护、数据库初始化）
|
||||
- **jest.setup.js WebCrypto polyfill** — jsdom 环境补齐 `crypto.subtle` 支持 AES-GCM
|
||||
|
||||
---
|
||||
|
||||
## [0.2.5] - 2026-07-29
|
||||
|
||||
### Fixed
|
||||
- **版本号统一** — `constants.ts` VERSION 从 `'0.2.0'` 更新为 `'0.2.5'`，修正 `index.ts`/`utils.ts`/`CONTRIBUTING.md` 中过时的版本注释和数据
|
||||
- **AriaEngine OPFS 后端映射** — `core.ts` 中 `mode: 'aria'` + `diskEngine: 'opfs'` 时实际使用 Memory 后端的 bug 已修复
|
||||
- **`_onError` 接入执行路径** — `query()`/`defineTable()`/`dropTable()`/`transaction()`/`importTable()` 的 catch 路径现在调用 `_onError` 全局错误回调
|
||||
- **`maxRowsPerQuery` 生效** — `QueryExecutor` 构造时接收 `maxRowsPerQuery` 参数，SELECT 结果在返回前截断
|
||||
- **WAL full 模式真正同步** — `WAL.append()` 改为 `async`，`full` 模式下 `await this.store.append()` 真正等待写入完成，不再 fire-and-forget
|
||||
- **PluginManager.install 传 db 实例** — `register()` 增加可选 `db` 参数，`core.ts` 初始化时传入 `this`，插件可获取 db 引用
|
||||
|
||||
### Changed
|
||||
- **SSTableReader 二分查找统一** — `locateBlockGE`/`locateBlockLE` 从线性扫描改为二分查找，rangeScan 性能在大型 SSTable 下不再退化
|
||||
- **crypto 实例化** — 全局状态改为 `CryptoManager` 类，每个 AriaEngine 实例可拥有独立加密配置，保留全局函数向后兼容
|
||||
- **compactLevelSync 接口公开化** — LSM 新增 public `compactLevel()` 方法，`vacuum()` 不再使用 `as any` 绕过 private 访问
|
||||
- **WAL 大小阈值接入 checkpoint** — `CheckpointManager` 接收 `walSizeThreshold` 参数，WAL 缓冲超阈值时自动触发 checkpoint
|
||||
|
||||
### Added
|
||||
- **ALTER TABLE 语法** — 支持 `ALTER TABLE ... ADD COLUMN` / `DROP COLUMN`（含可选 COLUMN 关键字）
|
||||
- **TRUNCATE TABLE 语法** — 支持 `TRUNCATE TABLE name` 快速清空表数据
|
||||
- **MVCC 接入读写路径** — 事务内 insert/update/delete 调用 `mvcc.writeVersion`/`mvcc.deleteVersion`，版本链作为 undo log
|
||||
- **IndexedDB 索引利用** — `IndexedDBEngine.find` 等值查询时优先使用 IDB 索引（`idx_col` 命名约定），避免全量 getAll
|
||||
- **SQL 注入防护** — React `useTable` / Vue `useSqlarkTable` 增加表名合法性校验（`/^[a-zA-Z_][a-zA-Z0-9_]*$/`）
|
||||
|
||||
---
|
||||
|
||||
## [0.2.4] - 2026-07-27
|
||||
|
||||
### Added
|
||||
- **二级索引** — 每列可独立维护 LSM Tree 索引，`$eq`/`$in`/`$gt`/`$lt` 走索引 O(log n)
|
||||
- **MVCC 接入引擎** — MVCCManager 正式投产，替换 ad-hoc txnSnapshot
|
||||
- **MVCC 自动 GC** — 每 10 次 checkpoint 自动回收过旧版本（保留最新 100 个）
|
||||
- **Bloom Filter 序列化** — 写入 SSTable footer + 读取时加载 + 查询时 probe 快速否定
|
||||
- **WAL 大小阈值** — `walSizeThreshold` 配置（默认 16MB），超阈值强制 checkpoint
|
||||
- **内存预算** — `maxMemoryMB` 配置（默认 64MB）
|
||||
|
||||
### Changed
|
||||
- **WAL 默认同步模式** — `walSyncMode` 从 `'batch'` 改为 `'full'`，消除 crash 丢数据风险
|
||||
- **查询优化** — `tryIndexLookup` 扩展支持非 PK 列索引查找
|
||||
|
||||
---
|
||||
|
||||
## [0.2.3] - 2026-07-27
|
||||
|
||||
### Fixed
|
||||
- **RB-Tree fixDelete 完整实现** — 补全标准红黑树删除修复，保证 O(log n)
|
||||
- **LSM SSTable 缓存预热** — `init()` 预加载所有 SSTable，消除 cache miss
|
||||
- **OPFS 数据恢复** — `open()` 自动从 OPFS 文件加载已有表数据到内存
|
||||
- **Aria WAL 事务恢复** — 两阶段恢复：仅回放已提交事务，未提交数据不回放
|
||||
- **LZ4 格式修复** — 重写 token 格式，消除中间字面量 token 歧义
|
||||
|
||||
---
|
||||
|
||||
## [0.2.2] - 2026-07-27
|
||||
|
||||
### Added
|
||||
- **OPFS 自研存储后端** — AriaEngine 新增 `OPFSBackend`，纯浏览器文件系统 API，零 IndexedDB 依赖
|
||||
- 每个 key 对应一个二进制文件，存储在 `navigator.storage.getDirectory()` 下
|
||||
- 支持 read/write/delete/list/exists/clear 完整接口
|
||||
- 页面文件（4KB）、WAL 日志、Schema、SSTable 全部存储为独立文件
|
||||
- `{ dbName }/pg_1`, `{ dbName }/__wal_0`, `{ dbName }/__aria_schemas` ...
|
||||
|
||||
### Changed
|
||||
- `AriaEngineConfig.storageBackend` 已支持 `'opfs'`，`AriaEngine.open()` 自动选择 OPFSBackend
|
||||
|
||||
---
|
||||
|
||||
## [0.2.1] - 2026-07-27
|
||||
|
||||
### Fixed
|
||||
- **WAL CRC 校验验证** — 恢复时计算并验证 CRC32，损坏记录自动跳过并告警
|
||||
- **Hybrid 引擎提交顺序** — commit 先写磁盘再写内存，磁盘失败回滚内存，消除数据不一致风险
|
||||
- **RESTRICT 外键行为修正** — 检测到引用行时抛出 `FOREIGN_KEY_VIOLATION`，不再静默保留孤儿
|
||||
- **SSTable rangeScan 边界保护** — 空索引或 startBlockIdx > endBlockIdx 时安全返回
|
||||
|
||||
### Added
|
||||
- **ColumnDef 约束激活** — `maxLength`/`min`/`max` 约束在 `checkFieldType` 中正式生效
|
||||
- **查询结果上限** — `DatabaseConfig.maxRowsPerQuery`（默认 0 不限制），防止超大结果集 OOM
|
||||
- **onError 全局回调** — 配置中的 `onError` 回调实际接入 CRUD 异常路径
|
||||
- **debug 调试模式** — `DatabaseConfig.debug: true` 输出 `[MetonaSqlark:name]` 前缀的结构化日志
|
||||
- **浏览器兼容性声明** — README 增加 Chrome 80+/Firefox 80+/Safari 14+/Edge 80+ 支持矩阵
|
||||
|
||||
### Changed
|
||||
- MemoryEngine RESTRICT 外键行为：从"不级联保留孤儿"改为"禁止删除抛异常"
|
||||
- Hybrid 引擎 commitTransaction 磁盘优先于内存
|
||||
- DB_DEFAULTS 新增 `maxRowsPerQuery: 0` 和 `debug: false`
|
||||
|
||||
---
|
||||
|
||||
## [0.2.0] - 2026-07-27
|
||||
|
||||
### Added
|
||||
- **AriaEngine 自研存储引擎** — 基于 LSM-Tree 的页面式存储引擎，19 个新模块，~3500 行 TypeScript
|
||||
- **页面格式层** (`page/`): 4KB Slotted Page 编解码、Tuple 二进制序列化、CRC32 校验
|
||||
- **Buffer Pool** (`buffer/`): LRU 页面缓存 + 驱逐策略，可控内存占用
|
||||
- **LSM-Tree 索引** (`index/`): MemTable (红黑树) + 多级 SSTable + Leveled Compaction
|
||||
- **Bloom Filter** (`index/bloom.ts`): FNV-1a + Murmur 双哈希，快速键否定判定
|
||||
- **SSTable 构建器/读取器** (`index/sstable_builder.ts`, `sstable.ts`): 二分查找 + 范围扫描
|
||||
- **Merge Iterator** (`index/merge_iterator.ts`): 最小堆多路归并，去重保留最新值
|
||||
- **WAL** (`wal/`): 二进制日志格式 (LSN/type/txnId/table/key/json/CRC) + Checkpoint 管理
|
||||
- **MVCC** (`transaction/mvcc.ts`): 版本链 + 快照隔离 + GC
|
||||
- **存储后端** (`store/`): IndexedDB / Memory 双后端抽象
|
||||
- **LZ4 压缩** (`compression/lz4.ts`): 简易页面级压缩
|
||||
- **`mode: 'aria'`** — 新增存储模式，可通过 `MetonaSqlark.create({ mode: 'aria' })` 激活
|
||||
- **Schema 持久化** — 表结构自动保存到 `__aria_schemas`，重启自动恢复
|
||||
- **SSTable 元数据管理** — SSTable 索引信息持久化，启动时自动扫描加载
|
||||
- **事务感知 CRUD** — insert/update/delete 在事务中缓冲到 snapshot，commit 批量写入 LSM
|
||||
- **244 个 AriaEngine 专项测试** — 覆盖生命周期/表管理/CRUD/事务/持久化/SQL 集成/页面格式/LSM/压缩
|
||||
|
||||
### Changed
|
||||
- `StorageMode` 类型新增 `'aria'`
|
||||
- `STORAGE_MODES` 数组新增 `'aria'`
|
||||
- `createEngine()` 支持 `mode: 'aria'` 分支
|
||||
- 测试从 318 → **526**，套件从 20 → **27**
|
||||
- 新增 7 个模块级测试文件：`aria-page`、`aria-index`、`aria-sstable`、`aria-buffer`、`aria-wal-mvcc`、`aria-compress`、`aria`
|
||||
- LRUList 修复 size 追踪 bug
|
||||
- WAL 存储改为按记录独立 key（避免拼接缓冲区越界）
|
||||
- CheckpointManager 解耦 BufferPool 依赖
|
||||
|
||||
### Fixed
|
||||
- **LZ4 压缩无限循环** — 字面量分支在发现匹配后回退导致 litLen=0 死循环，CI 卡死根因
|
||||
- **SSTableReader.get() 自比较 bug** — 参数 key 被循环变量遮蔽导致永远返回第一条
|
||||
- **CheckpointManager 测试 null 引用** — 改为 Mock 对象避免 TypeError 导致进程无法退出
|
||||
- **IndexedDB 持久化测试** — 替换为 Memory Backend 验证，消除 fake-indexeddb timer 堆积
|
||||
- **LSM.flush() 不必要 setTimeout** — 替换为 Promise.resolve()，消除额外 timer 延迟
|
||||
|
||||
---
|
||||
|
||||
## [0.1.14] - 2026-07-26
|
||||
|
||||
### Fixed
|
||||
- **IndexedDB 事务原子性**: `flushToIDB` 改为单 IDB 事务包裹 clear+insert，消除崩溃丢数据风险
|
||||
- **多标签页冲突**: `open()` 添加 `onversionchange` 监听，其他标签页升级版本时自动关闭过期连接
|
||||
- **Memory 引擎幂等**: `open()` 重复调用安全无副作用
|
||||
- 关闭时清理 `onversionchange` 监听器，防止内存泄漏
|
||||
|
||||
---
|
||||
|
||||
## [0.1.13] - 2026-07-26
|
||||
|
||||
### Added
|
||||
- **事务回滚机制**: `IStorageEngine` 新增 `beginTransaction/commitTransaction/rollbackTransaction` 接口
|
||||
- MemoryEngine: 快照式回滚（深拷贝 tables/schemas/indexes）
|
||||
- IndexedDBEngine: 延迟写入策略（事务中仅写内存，commit 批量刷 IDB）
|
||||
- OPFSEngine: 快照式回滚（commit 批量写文件）
|
||||
- HybridEngine: 同时代理内存+磁盘引擎事务
|
||||
- **子查询支持**: SQL Parser + Executor 支持 `IN (SELECT ...)` 和 `op (SELECT ...)` 子查询
|
||||
- AST 新增 `SubqueryExpression` 类型
|
||||
- Parser 在 IN 和比较运算符后检测子查询
|
||||
- Executor 新增 `resolveSubqueries()` 递归解析，自动执行子查询并替换为具体值
|
||||
- **外键级联操作**: ColumnDef 新增 `onDelete/onUpdate` 选项
|
||||
- 支持 `CASCADE`（递归级联删除）、`SET NULL`、`RESTRICT`
|
||||
- SQL Parser 解析 `REFERENCES table(col) ON DELETE CASCADE ON UPDATE CASCADE`
|
||||
- MemoryEngine 内置 `cascadeDelete()` 递归级联逻辑
|
||||
- **连接池管理**: `MetonaSqlark.connect()` 静态方法
|
||||
- 同名数据库复用已打开实例，引用计数管理
|
||||
- `db.disconnect()` 释放连接，归零自动 close
|
||||
- `MetonaSqlark.disconnectAll()` 强制关闭所有连接
|
||||
|
||||
### Changed
|
||||
- `TransactionManager.execute()` 调用引擎层 begin/commit/rollback 实现真正原子性
|
||||
- IndexedDBEngine CRUD 事务感知：活跃事务中延迟 IDB 写入
|
||||
- ASTColumnDef / astColumnToColumnDef 支持 references/onDelete/onUpdate 透传
|
||||
|
||||
### Fixed
|
||||
- IndexedDBEngine 事务中 find/count 从内存缓存读取（保证读到未提交变更）
|
||||
- SQL Parser parseColumnDef 循环条件扩展，避免 REFERENCES 语法解析中断
|
||||
|
||||
---
|
||||
|
||||
## [0.1.12] - 2026-07-26
|
||||
|
||||
### Added
|
||||
- `MeSqlark` 别名导出，与 `MetonaSqlark` 完全等价
|
||||
- `metona-sqlark.cjs.js` CommonJS 构建产物
|
||||
- 完善的 `where-matcher.ts` 消除 220+ 行重复代码
|
||||
- `$col` 列引用支持 JOIN ON 条件
|
||||
- `$and/$or` 嵌套支持（字段级 + 顶层）
|
||||
- `$not` 逻辑非操作符
|
||||
- `$like` 正则缓存加速
|
||||
- `projectColumns` 列投影支持 `table.column` 格式
|
||||
- `DISTINCT` 去重支持（O(n) 时间，列值拼接优化）
|
||||
- 测试覆盖率提升至 93.46%（264 测试/15 套件）
|
||||
|
||||
### Changed
|
||||
- `where-matcher` 统一 MemoryEngine / IndexedDBEngine / Executor 的 WHERE 逻辑
|
||||
- JOIN ON 匹配支持 `$col` 语法
|
||||
- LIKE 正则改为缓存式编译
|
||||
- Executor DISTINCT 用列值拼接代替 JSON.stringify
|
||||
- JOIN 嵌套循环连接优化：避免 ON 时对象扩散
|
||||
|
||||
### Fixed
|
||||
- IndexedDBEngine `update/delete` 方法正确同步内存缓存
|
||||
- OPFSEngine `getTableNames` 兼容 `entries()` 返回值
|
||||
- SQL Parser 正确处理字符串转义字符
|
||||
- SQL Parser 正确处理表别名（无 AS 关键字）
|
||||
- SQL Parser 解析 `IS NULL` / `IS NOT NULL` / `NOT LIKE`
|
||||
- `peekTokenIs` 方法正确暴露给语法分析
|
||||
- 多处边界条件空值/假值处理
|
||||
|
||||
---
|
||||
|
||||
## [0.1.11] - 2026-07-25
|
||||
|
||||
### Added
|
||||
- React 集成 hooks: `useQuery`, `useTable`, `useDatabase`
|
||||
- Vue 集成 composables: `useSqlarkQuery`, `useSqlarkTable`, `useSqlarkDatabase`
|
||||
- 发布订阅系统: `subscribe()`, `emit()`
|
||||
- 数据迁移系统: `addMigration()`, `migrateTo()`
|
||||
- 导入导出: `exportTable()`, `exportAll()`, `importTable()`
|
||||
- 完整 SQL 解析器: `Lexer` + `Parser`（递归下降）
|
||||
- SQL 支持: SELECT, INSERT, UPDATE, DELETE, CREATE TABLE, DROP TABLE
|
||||
- JOIN 支持: INNER, LEFT, RIGHT, CROSS
|
||||
- GROUP BY + HAVING + 聚合函数 (COUNT, SUM, AVG, MIN, MAX)
|
||||
- DISTINCT, ORDER BY, LIMIT, OFFSET
|
||||
- WHERE 条件: AND, OR, NOT, IN, LIKE, IS NULL
|
||||
- Query Builder 链式 API: `select().where().orderBy().limit().execute()`
|
||||
- 插件系统: 14 种生命周期钩子 + PluginManager
|
||||
- Metadata 系统: ColumnDef 完整约束 (type/primaryKey/required/unique/index/default/references/maxLength/min/max)
|
||||
|
||||
### Changed
|
||||
- 架构重构为分层设计：Engine → QueryExecutor → Table/QueryBuilder → SQL Parser → Public API
|
||||
- AST 作为统一中间表示，SQL 和 QueryBuilder 行为完全一致
|
||||
- 测试框架完善：core / edge / groupby / join / query-system / sql / table / engine / transaction / plugin / hybrid 全覆盖
|
||||
|
||||
---
|
||||
|
||||
## [0.0.1] - 2026-07-24
|
||||
|
||||
### Added
|
||||
- 初始项目骨架
|
||||
- TypeScript 配置（严格模式）
|
||||
- Rollup 构建（UMD / ESM / CJS / min / .d.ts）
|
||||
- Jest + jsdom 测试环境
|
||||
- ESLint + @typescript-eslint
|
||||
- Gitea Actions CI 工作流
|
||||
- 示例站点（index / demo / docs）
|
||||
- 构建脚本 build.sh
|
||||
|
||||
+13
-4
@@ -43,11 +43,20 @@ src/
|
||||
├── core.ts # MetonaSqlark main class
|
||||
├── constants.ts # Types, defaults, enums, errors
|
||||
├── utils.ts # Utility functions
|
||||
├── connection-manager.ts # Connection pool (connect/disconnect)
|
||||
├── engine/ # Storage engines
|
||||
│ ├── interface.ts # IStorageEngine interface
|
||||
│ ├── memory.ts # MemoryEngine (Map-based)
|
||||
│ ├── indexeddb.ts # IndexedDBEngine (browser persistence)
|
||||
│ └── opfs.ts # OPFSEngine (Origin Private File System)
|
||||
│ ├── opfs.ts # OPFSEngine (Origin Private File System)
|
||||
│ └── aria/ # AriaEngine (LSM-Tree page storage engine)
|
||||
│ ├── index/ # LSM / MemTable / SSTable / Bloom / MergeIterator
|
||||
│ ├── page/ # 4KB slotted page format
|
||||
│ ├── buffer/ # Buffer Pool (LRU eviction)
|
||||
│ ├── wal/ # Write-Ahead Log + Checkpoint
|
||||
│ ├── transaction/ # MVCC manager
|
||||
│ ├── store/ # Backends (IndexedDB / OPFS / Memory)
|
||||
│ └── compression/ # LZ4
|
||||
├── hybrid/ # HybridEngine (write-through)
|
||||
├── table/ # Table management & Schema validation
|
||||
├── query/ # Query system
|
||||
@@ -64,7 +73,7 @@ src/
|
||||
├── plugin/ # Plugin system (14 lifecycle hooks)
|
||||
└── integrations/ # React & Vue hooks
|
||||
|
||||
tests/ # Test suite (701+ test cases, 32 test suites)
|
||||
tests/ # Test suite (761+ test cases, 41 test suites)
|
||||
site/ # Documentation site (index / docs / demo)
|
||||
```
|
||||
|
||||
@@ -133,9 +142,9 @@ npm run build
|
||||
|
||||
# Output in dist/
|
||||
# ├── metona-sqlark.js UMD
|
||||
# ├── metona-sqlark.min.js UMD minified (~42KB / ~10KB gzip)
|
||||
# ├── metona-sqlark.min.js UMD minified (~105KB / ~27KB gzip)
|
||||
# ├── metona-sqlark.esm.js ES Module
|
||||
# ├── metona-sqlark.cjs.js CommonJS
|
||||
# ├── metona-sqlark.cjs CommonJS
|
||||
# └── metona-sqlark.d.ts TypeScript declarations
|
||||
```
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# MetonaSqlark
|
||||
|
||||
<p align="center">
|
||||
<img src="https://img.shields.io/badge/version-0.2.5-blue?style=flat-square" alt="version">
|
||||
<img src="https://img.shields.io/badge/version-0.3.2-blue?style=flat-square" alt="version">
|
||||
<img src="https://img.shields.io/badge/license-MIT-green?style=flat-square" alt="license">
|
||||
<img src="https://img.shields.io/badge/coverage-91.0%25-brightgreen?style=flat-square" alt="coverage">
|
||||
<img src="https://img.shields.io/badge/tests-721%20passed-success?style=flat-square" alt="tests">
|
||||
<img src="https://img.shields.io/badge/coverage-81.0%25-brightgreen?style=flat-square" alt="coverage">
|
||||
<img src="https://img.shields.io/badge/tests-761%20passed-success?style=flat-square" alt="tests">
|
||||
</p>
|
||||
|
||||
> 基于 TypeScript 的**前端关系型数据库**，支持完整 SQL 查询、Query Builder 链式 API、与 **AriaEngine 自研页面式存储引擎**。
|
||||
@@ -18,13 +18,13 @@
|
||||
- 🔒 **生产级数据安全** — WAL CRC 完整性校验、`RESTRICT` 外键约束、Hybrid 提交原子性、SQL 注入防护
|
||||
- 🛡 **输入校验全覆盖** — `maxLength`/`min`/`max` 约束、类型检查、必填验证
|
||||
- 💾 **多引擎架构** — Memory / IndexedDB / OPFS / Hybrid(write-through) / Aria 五种模式
|
||||
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE
|
||||
- 📝 **完整 SQL 支持** — SELECT/JOIN/子查询/GROUP BY/HAVING/ORDER BY/LIMIT/BETWEEN/IF NOT EXISTS/ALTER TABLE/TRUNCATE TABLE/UNION/INSERT...SELECT/事务语句/CREATE INDEX/EXISTS（v0.3.0）
|
||||
- 🔗 **Query Builder API** — 链式 `.select().where().orderBy().limit().execute()`
|
||||
- 🔄 **事务回滚** — Memory/IndexedDB/Hybrid/Aria 四引擎事务原子性，自动回滚，MVCC 版本链接入读写路径
|
||||
- 🌲 **RB-Tree 完整实现** — 标准红黑树插入+删除修复，O(log n) 保证
|
||||
- ⚡ **性能优化** — SSTableReader 二分查找统一、IndexedDB 索引利用、crypto 实例化避免全局状态
|
||||
- 🌐 **浏览器兼容** — Chrome 80+ / Firefox 80+ / Safari 14+ / Edge 80+ / Node.js 16+
|
||||
- 🧪 **721 测试 · 91.0% 覆盖率** — 37 套件，生产级质量保证
|
||||
- 🧪 **761 测试 · 81.0% 覆盖率** — 41 套件，生产级质量保证
|
||||
|
||||
---
|
||||
|
||||
@@ -48,9 +48,9 @@ npm install @metona-team/metona-sqlark
|
||||
|
||||
或从 [`dist/`](./dist/) 目录下载：
|
||||
- `metona-sqlark.js` — UMD 开发版（含 sourcemap）
|
||||
- `metona-sqlark.min.js` — UMD 压缩版（~42KB）
|
||||
- `metona-sqlark.min.js` — UMD 压缩版（~105KB，gzip ~27KB）
|
||||
- `metona-sqlark.esm.js` — ES Module
|
||||
- `metona-sqlark.cjs.js` — CommonJS
|
||||
- `metona-sqlark.cjs` — CommonJS
|
||||
- `metona-sqlark.d.ts` — TypeScript 类型声明
|
||||
|
||||
---
|
||||
@@ -98,7 +98,7 @@ import { MetonaSqlark } from '@metona-team/metona-sqlark';
|
||||
|
||||
const db = await MetonaSqlark.create({
|
||||
name: 'my-app',
|
||||
mode: 'hybrid', // 'memory' | 'disk' | 'hybrid'
|
||||
mode: 'hybrid', // 'memory' | 'disk' | 'hybrid' | 'aria'
|
||||
diskEngine: 'indexeddb', // 'indexeddb' | 'opfs'
|
||||
});
|
||||
|
||||
@@ -144,6 +144,27 @@ await db.query('ALTER TABLE users DROP COLUMN phone');
|
||||
// TRUNCATE TABLE — 快速清空表 v0.2.5
|
||||
await db.query('TRUNCATE TABLE old_logs');
|
||||
|
||||
// v0.3.0 — SQL 功能扩展
|
||||
// 多语句（分号分隔）
|
||||
await db.query("CREATE TABLE t (id STRING PRIMARY KEY); INSERT INTO t VALUES ('1'); INSERT INTO t VALUES ('2')");
|
||||
// 事务语句
|
||||
await db.query('BEGIN');
|
||||
await db.query("INSERT INTO t VALUES ('3')");
|
||||
await db.query('ROLLBACK'); // 回滚
|
||||
// INSERT INTO ... SELECT
|
||||
await db.query('INSERT INTO t SELECT id FROM t2 WHERE x > 1');
|
||||
// UNION / UNION ALL
|
||||
const rows = await db.query('SELECT name FROM users WHERE city = \'Beijing\' UNION SELECT name FROM users WHERE age < 30');
|
||||
// 动态索引
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)');
|
||||
await db.query('DROP INDEX idx_users_city ON users (city)');
|
||||
// EXISTS 关联子查询
|
||||
const hasOrders = await db.query('SELECT * FROM users u WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)');
|
||||
|
||||
// v0.3.1 — CASE WHEN / JOIN 关联子查询 / 组提交
|
||||
const labeled = await db.query("SELECT name, CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users");
|
||||
const joinExists = await db.query('SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 150)');
|
||||
|
||||
// 事务 — 自动回滚 v0.1.13
|
||||
await db.transaction(async (trx) => {
|
||||
await trx.table('users').insert({ id: '3', name: 'Charlie' });
|
||||
@@ -278,7 +299,7 @@ const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
||||
|
||||
---
|
||||
|
||||
## 🌲 AriaEngine — 自研存储引擎 (v0.2.5)
|
||||
## 🌲 AriaEngine — 自研存储引擎
|
||||
|
||||
AriaEngine 是内置的页面式存储引擎，对标 SQLite 的设计理念：
|
||||
|
||||
@@ -326,7 +347,7 @@ const rows = await db.query('SELECT * FROM users');
|
||||
| **LSM-Tree** | MemTable (红黑树) → SSTable 多级索引，异步 Compaction，写背压 |
|
||||
| **WAL** | Write-Ahead Log 二进制格式，CRC 校验，full/batch/none 三种模式（full 模式真正同步 ✅ v0.2.5），16MB 阈值自动 checkpoint |
|
||||
| **MVCC** | 版本链 + 快照隔离，事务读写不互斥，自动 GC（每10次检查点），读写路径接入版本链 ✅ v0.2.5 |
|
||||
| **Buffer Pool** | FileManager + LRU 页面缓存，256 页 ≈ 1MB 可控内存 |
|
||||
| **Buffer Pool** | SSTable 缓存 LRU 上限（`bufferPoolPages` × `pageSize`，默认 256 页 ≈ 1MB 可控内存）✅ v0.2.6 生效，查询前异步预加载兜底，缓存驱逐不丢数据 |
|
||||
| **Bloom Filter** | FNV-1a + Murmur 双哈希，SSTable footer 序列化，查询时 probe |
|
||||
| **二级索引** | 每列独立 LSM Tree，支持 $eq/$in/$gt/$lt 范围扫描，SSTableReader 二分查找统一 ✅ v0.2.5 |
|
||||
| **AES-GCM** | PBKDF2 密钥派生 + AES-256-GCM 页面级加密，CryptoManager 实例化 ✅ v0.2.5 |
|
||||
@@ -352,9 +373,9 @@ npm run typecheck # 类型检查
|
||||
|
||||
| 指标 | 数值 |
|
||||
|------|------|
|
||||
| 测试用例 | 721 |
|
||||
| 测试套件 | 37 |
|
||||
| 行覆盖率 | 91.0% |
|
||||
| 测试用例 | 761 |
|
||||
| 测试套件 | 41 |
|
||||
| 行覆盖率 | 81.0% |
|
||||
| SQL 关键字 | 36 |
|
||||
| 存储引擎 | 5（Memory / IndexedDB / OPFS / Hybrid / **Aria**） |
|
||||
|
||||
|
||||
+9
-9
@@ -1,9 +1,9 @@
|
||||
module.exports = {
|
||||
presets: [
|
||||
['@babel/preset-env', { targets: { node: 'current' }, modules: 'commonjs' }],
|
||||
'@babel/preset-typescript',
|
||||
],
|
||||
plugins: [
|
||||
'@babel/plugin-transform-modules-commonjs',
|
||||
],
|
||||
};
|
||||
module.exports = {
|
||||
presets: [
|
||||
['@babel/preset-env', { targets: { node: 'current' }, modules: 'commonjs' }],
|
||||
'@babel/preset-typescript',
|
||||
],
|
||||
plugins: [
|
||||
'@babel/plugin-transform-modules-commonjs',
|
||||
],
|
||||
};
|
||||
|
||||
+2056
-591
File diff suppressed because it is too large
Load Diff
Vendored
-1
File diff suppressed because one or more lines are too long
Vendored
+1
File diff suppressed because one or more lines are too long
Vendored
+152
-7
@@ -62,6 +62,8 @@ interface DatabaseConfig {
|
||||
maxRowsPerQuery?: number;
|
||||
/** 调试模式（启用后输出详细操作日志） */
|
||||
debug?: boolean;
|
||||
/** 多标签页同步（v0.3.2）：BroadcastChannel 广播表变更，其他标签页自动刷新 */
|
||||
multiTabSync?: boolean;
|
||||
}
|
||||
/** Where 条件操作符 */
|
||||
type WhereOperator = '$eq' | '$ne' | '$gt' | '$gte' | '$lt' | '$lte' | '$in' | '$nin' | '$like' | '$and' | '$or' | '$not';
|
||||
@@ -114,7 +116,7 @@ interface MetonaPlugin {
|
||||
/** 销毁 */
|
||||
destroy(): void;
|
||||
}
|
||||
declare const VERSION = "0.2.5";
|
||||
declare const VERSION = "0.3.2";
|
||||
|
||||
/**
|
||||
* metona-sqlark Plugin — 插件系统
|
||||
@@ -179,6 +181,10 @@ interface IStorageEngine {
|
||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||
/** 清空表数据（保留结构） */
|
||||
clear(tableName: string): Promise<void>;
|
||||
/** 创建二级索引（CREATE INDEX） */
|
||||
createIndex?(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
/** 删除二级索引（DROP INDEX） */
|
||||
dropIndex?(tableName: string, column: string, indexName?: string): Promise<void>;
|
||||
/** 开始事务 */
|
||||
beginTransaction(): Promise<void>;
|
||||
/** 提交事务 */
|
||||
@@ -254,7 +260,10 @@ interface InsertStatement {
|
||||
type: 'INSERT';
|
||||
into: string;
|
||||
columns?: string[];
|
||||
values: unknown[][];
|
||||
/** VALUES 字面量 */
|
||||
values?: unknown[][];
|
||||
/** INSERT INTO ... SELECT ...（v0.3.0） */
|
||||
select?: SelectStatement | SelectUnionStatement;
|
||||
}
|
||||
interface UpdateStatement {
|
||||
type: 'UPDATE';
|
||||
@@ -285,6 +294,15 @@ interface SelectStatement {
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
interface SelectUnionStatement {
|
||||
type: 'SELECT_UNION';
|
||||
/** 左操作数（可以是 SELECT 或嵌套 UNION） */
|
||||
left: SelectStatement | SelectUnionStatement;
|
||||
/** 右操作数 */
|
||||
right: SelectStatement | SelectUnionStatement;
|
||||
/** UNION ALL 不去重 */
|
||||
all?: boolean;
|
||||
}
|
||||
interface AlterTableStatement {
|
||||
type: 'ALTER_TABLE';
|
||||
name: string;
|
||||
@@ -295,7 +313,31 @@ interface TruncateTableStatement {
|
||||
type: 'TRUNCATE_TABLE';
|
||||
name: string;
|
||||
}
|
||||
type Statement = SelectStatement | ExplainStatement | InsertStatement | UpdateStatement | DeleteStatement | CreateTableStatement | DropTableStatement | AlterTableStatement | TruncateTableStatement;
|
||||
interface CreateIndexStatement {
|
||||
type: 'CREATE_INDEX';
|
||||
/** 索引名（语法占位） */
|
||||
name: string;
|
||||
table: string;
|
||||
column: string;
|
||||
/** UNIQUE 索引 */
|
||||
unique?: boolean;
|
||||
}
|
||||
interface DropIndexStatement {
|
||||
type: 'DROP_INDEX';
|
||||
name: string;
|
||||
table: string;
|
||||
column: string;
|
||||
}
|
||||
interface BeginTransactionStatement {
|
||||
type: 'BEGIN';
|
||||
}
|
||||
interface CommitTransactionStatement {
|
||||
type: 'COMMIT';
|
||||
}
|
||||
interface RollbackTransactionStatement {
|
||||
type: 'ROLLBACK';
|
||||
}
|
||||
type Statement = SelectStatement | SelectUnionStatement | ExplainStatement | InsertStatement | UpdateStatement | DeleteStatement | CreateTableStatement | DropTableStatement | AlterTableStatement | TruncateTableStatement | CreateIndexStatement | DropIndexStatement | BeginTransactionStatement | CommitTransactionStatement | RollbackTransactionStatement;
|
||||
|
||||
/**
|
||||
* metona-sqlark Query Executor — AST 执行器
|
||||
@@ -311,11 +353,23 @@ declare class QueryExecutor {
|
||||
/** 设置查询结果行数上限 */
|
||||
setMaxRowsPerQuery(max: number): void;
|
||||
execute(stmt: Statement): Promise<unknown>;
|
||||
/** 递归执行 UNION / UNION ALL，返回合并结果 */
|
||||
private executeSelectUnion;
|
||||
private executeSelectPart;
|
||||
/** 将 UNION 右侧行投影为左侧列结构（按位置取值） */
|
||||
private projectUnionRow;
|
||||
/** EXPLAIN: 输出查询计划 */
|
||||
private executeExplain;
|
||||
private executeSelect;
|
||||
private executeJoinSelect;
|
||||
private prefixRow;
|
||||
/**
|
||||
* 哈希连接（v0.3.2）：ON 为单一等值条件且右表列为索引/主键时，
|
||||
* 收集左表连接值 → 一次 $in 查询右表 → 哈希映射匹配。
|
||||
* 替代嵌套循环，大表 INNER/LEFT JOIN 复杂度 O(N + M)。
|
||||
* 不适用时返回 null（回退嵌套循环）。
|
||||
*/
|
||||
private tryHashJoin;
|
||||
/** 嵌套循环连接（优化：避免 ON 时对象扩散） */
|
||||
private joinRows;
|
||||
private executeGroupBy;
|
||||
@@ -328,14 +382,47 @@ declare class QueryExecutor {
|
||||
private executeDropTable;
|
||||
private executeAlterTable;
|
||||
private executeTruncateTable;
|
||||
private executeCreateIndex;
|
||||
private executeDropIndex;
|
||||
private executeBegin;
|
||||
private executeCommit;
|
||||
private executeRollback;
|
||||
/** 列列表是否包含 CASE WHEN 表达式 */
|
||||
private hasCaseColumn;
|
||||
/** WHERE 是否包含 CASE WHEN 表达式键 */
|
||||
private whereHasCase;
|
||||
getEngine(): IStorageEngine;
|
||||
/**
|
||||
* 列投影（v0.3.1）：普通列走 projectColumns，CASE WHEN 表达式逐行求值
|
||||
*/
|
||||
private projectRow;
|
||||
/** 检查 SELECT 列列表中是否包含聚合函数 */
|
||||
private _hasAggregateColumn;
|
||||
/** 计算单行聚合结果（无 GROUP BY） */
|
||||
private computeSingleAggregate;
|
||||
/** 剥离主表别名前缀：'u.id' → 'id'（键与 $col 值均处理，支持多层别名） */
|
||||
private normalizeWhereColumns;
|
||||
private normalizeExistsValue;
|
||||
private normalizeFieldValue;
|
||||
private stripAlias;
|
||||
/** WHERE 是否含关联引用（$col 或关联 EXISTS）或 CASE WHEN 表达式键 */
|
||||
private hasCorrelatedRefs;
|
||||
private fieldHasColRef;
|
||||
/** 移除关联 EXISTS 标记（引擎层先执行无 EXISTS 条件的查询） */
|
||||
private stripCorrelatedExists;
|
||||
/** 逐行绑定外层行上下文，求值关联 EXISTS、$col 引用与 CASE WHEN 键 */
|
||||
private filterCorrelated;
|
||||
/** 将 WHERE 中的 CASE WHEN 表达式键求值为布尔条件（$caseResult） */
|
||||
private resolveCaseKeys;
|
||||
/** CASE 求值结果与操作符条件比较 */
|
||||
private caseConditionMatches;
|
||||
/** 将 where 中的 $col 引用替换为上下文行值 */
|
||||
private bindColumnRefs;
|
||||
private bindWhereRefs;
|
||||
/**
|
||||
* 递归扫描 WHERE 条件，找到 $subquery 标记并执行子查询，
|
||||
* 将结果替换为具体值。
|
||||
* @param contextRow 关联子查询的外层行上下文（用于绑定 $col 引用）
|
||||
*/
|
||||
private resolveSubqueries;
|
||||
/**
|
||||
@@ -395,7 +482,8 @@ declare class UpdateQueryBuilder {
|
||||
private tableName;
|
||||
private _updates;
|
||||
private _where;
|
||||
constructor(engine: IStorageEngine, tableName: string, _updates: Record<string, unknown>);
|
||||
private onWrite?;
|
||||
constructor(engine: IStorageEngine, tableName: string, _updates: Record<string, unknown>, onWrite?: (table: string) => void);
|
||||
where(condition: WhereCondition): this;
|
||||
execute(): Promise<number>;
|
||||
toAST(): UpdateStatement;
|
||||
@@ -404,7 +492,8 @@ declare class DeleteQueryBuilder {
|
||||
private engine;
|
||||
private tableName;
|
||||
private _where;
|
||||
constructor(engine: IStorageEngine, tableName: string);
|
||||
private onWrite?;
|
||||
constructor(engine: IStorageEngine, tableName: string, onWrite?: (table: string) => void);
|
||||
where(condition: WhereCondition): this;
|
||||
execute(): Promise<number>;
|
||||
toAST(): DeleteStatement;
|
||||
@@ -420,7 +509,9 @@ declare class Table<T = Record<string, unknown>> {
|
||||
private engine;
|
||||
private schema;
|
||||
private executor;
|
||||
constructor(engine: IStorageEngine, tableName: string, executor?: QueryExecutor);
|
||||
/** 写入回调（多标签页广播，v0.3.2） */
|
||||
private onWrite?;
|
||||
constructor(engine: IStorageEngine, tableName: string, executor?: QueryExecutor, onWrite?: (table: string) => void);
|
||||
getSchema(): Promise<TableSchema>;
|
||||
insert(row: T & Record<string, unknown>): Promise<string>;
|
||||
insertMany(rows: (T & Record<string, unknown>)[]): Promise<string[]>;
|
||||
@@ -472,6 +563,8 @@ declare class MetonaSqlark {
|
||||
get maxRowsPerQuery(): number;
|
||||
/** 调试模式 */
|
||||
get debug(): boolean;
|
||||
/** 多标签页同步通道（v0.3.2） */
|
||||
private channel;
|
||||
constructor(config: DatabaseConfig);
|
||||
/** 初始化数据库（创建引擎、打开连接） */
|
||||
init(): Promise<void>;
|
||||
@@ -500,12 +593,18 @@ declare class MetonaSqlark {
|
||||
subscribe(tableName: string, callback: (event: {
|
||||
type: string;
|
||||
row?: unknown;
|
||||
table?: string;
|
||||
}) => void): () => void;
|
||||
/** 触发变更事件 */
|
||||
emit(tableName: string, event: {
|
||||
type: string;
|
||||
row?: unknown;
|
||||
table?: string;
|
||||
}): void;
|
||||
/** 广播表变更到其他标签页（多标签页同步） */
|
||||
broadcastChange(tableName: string): void;
|
||||
/** 写语句对应的表名（多标签页广播用） */
|
||||
private writeStatementTable;
|
||||
private migrations;
|
||||
/** 注册迁移 */
|
||||
addMigration(version: number, up: (db: MetonaSqlark) => Promise<void>): void;
|
||||
@@ -553,6 +652,8 @@ declare class MemoryEngine implements IStorageEngine {
|
||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||
clear(tableName: string): Promise<void>;
|
||||
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
dropIndex(tableName: string, column: string, _indexName?: string): Promise<void>;
|
||||
beginTransaction(): Promise<void>;
|
||||
commitTransaction(): Promise<void>;
|
||||
rollbackTransaction(): Promise<void>;
|
||||
@@ -590,6 +691,12 @@ declare class IndexedDBEngine implements IStorageEngine {
|
||||
private memoryCache;
|
||||
private txActive;
|
||||
open(dbName: string, version: number): Promise<void>;
|
||||
/**
|
||||
* 从 IDB 恢复内存 schema：
|
||||
* 1. 优先读取持久化的 schema 记录（'__metona_schema' store，v0.3.2）
|
||||
* 2. 旧数据回退：从 objectStore 主键 / 索引 / 样例数据推断
|
||||
*/
|
||||
private rebuildSchemaFromIDB;
|
||||
close(): Promise<void>;
|
||||
isOpen(): boolean;
|
||||
createTable(schema: TableSchema): Promise<void>;
|
||||
@@ -603,6 +710,8 @@ declare class IndexedDBEngine implements IStorageEngine {
|
||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||
clear(tableName: string): Promise<void>;
|
||||
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
dropIndex(tableName: string, column: string, _indexName?: string): Promise<void>;
|
||||
beginTransaction(): Promise<void>;
|
||||
commitTransaction(): Promise<void>;
|
||||
rollbackTransaction(): Promise<void>;
|
||||
@@ -648,6 +757,8 @@ declare class OPFSEngine implements IStorageEngine {
|
||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||
clear(tableName: string): Promise<void>;
|
||||
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
dropIndex(tableName: string, column: string, indexName?: string): Promise<void>;
|
||||
beginTransaction(): Promise<void>;
|
||||
commitTransaction(): Promise<void>;
|
||||
rollbackTransaction(): Promise<void>;
|
||||
@@ -727,6 +838,8 @@ declare class AriaEngine implements IStorageEngine {
|
||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||
clear(tableName: string): Promise<void>;
|
||||
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
dropIndex(tableName: string, column: string, _indexName?: string): Promise<void>;
|
||||
beginTransaction(): Promise<void>;
|
||||
commitTransaction(): Promise<void>;
|
||||
rollbackTransaction(): Promise<void>;
|
||||
@@ -741,6 +854,14 @@ declare class AriaEngine implements IStorageEngine {
|
||||
private checkType;
|
||||
private persistSchemas;
|
||||
private loadSchemas;
|
||||
/**
|
||||
* 创建命名空间隔离的 SSTableStore。
|
||||
*
|
||||
* 主 LSM 与每个二级索引 LSM 各持有独立实例：
|
||||
* - 文件 key 前缀隔离（sst_ / sst_idx_${table}_${col}_）
|
||||
* - 元数据 key 隔离（__aria_lsm_meta / __aria_lsm_meta_${ns}）
|
||||
* - id 序列独立（避免 v0.2.4 共享 id 空间导致的文件互相覆盖）
|
||||
*/
|
||||
private createSSTableStore;
|
||||
private applyWALRecord;
|
||||
/** 更新行的二级索引条目 */
|
||||
@@ -751,6 +872,8 @@ declare class AriaEngine implements IStorageEngine {
|
||||
private indexScanToRows;
|
||||
/** 每 10 次 gc 计数器触发一次 MVCC 垃圾回收 */
|
||||
private tryGC;
|
||||
/** 回收主 LSM 与所有二级索引 LSM 的临时缓存超限 */
|
||||
private trimAllCaches;
|
||||
/** 检查内存预算，超出时强制 flush + GC */
|
||||
private checkMemoryBudget;
|
||||
/** 估算 WAL 大小（字节） */
|
||||
@@ -801,8 +924,15 @@ declare class HybridEngine implements IStorageEngine {
|
||||
private memoryEngine;
|
||||
private diskEngine;
|
||||
private diskEngineType;
|
||||
private dbName;
|
||||
private version;
|
||||
constructor(diskEngine?: DiskEngine);
|
||||
open(dbName: string, version: number): Promise<void>;
|
||||
/**
|
||||
* 从磁盘重载内存缓存（v0.3.2：多标签页同步）。
|
||||
* 其他标签页写入磁盘后调用，使本标签页读到最新数据。
|
||||
*/
|
||||
reloadMemoryFromDisk(): Promise<void>;
|
||||
close(): Promise<void>;
|
||||
isOpen(): boolean;
|
||||
createTable(schema: TableSchema): Promise<void>;
|
||||
@@ -816,6 +946,8 @@ declare class HybridEngine implements IStorageEngine {
|
||||
delete(tableName: string, query: QueryPlan): Promise<number>;
|
||||
count(tableName: string, query?: QueryPlan): Promise<number>;
|
||||
clear(tableName: string): Promise<void>;
|
||||
createIndex(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
dropIndex(tableName: string, column: string, indexName?: string): Promise<void>;
|
||||
beginTransaction(): Promise<void>;
|
||||
commitTransaction(): Promise<void>;
|
||||
rollbackTransaction(): Promise<void>;
|
||||
@@ -835,6 +967,8 @@ declare class HybridEngine implements IStorageEngine {
|
||||
|
||||
/** 解析 SQL 字符串为 AST Statement */
|
||||
declare function parse(sql: string): Statement;
|
||||
/** 解析 SQL 字符串为 AST Statement 数组（分号分隔的多语句支持，v0.3.0） */
|
||||
declare function parseAll(sql: string): Statement[];
|
||||
|
||||
/**
|
||||
* metona-sqlark SQL Token Types — 词法单元定义
|
||||
@@ -895,6 +1029,17 @@ declare enum TokenType {
|
||||
MIN = "MIN",
|
||||
MAX = "MAX",
|
||||
DISTINCT = "DISTINCT",
|
||||
BEGIN = "BEGIN",
|
||||
COMMIT = "COMMIT",
|
||||
ROLLBACK = "ROLLBACK",
|
||||
UNION = "UNION",
|
||||
ALL = "ALL",
|
||||
INDEX = "INDEX",
|
||||
CASE = "CASE",
|
||||
WHEN = "WHEN",
|
||||
THEN = "THEN",
|
||||
ELSE = "ELSE",
|
||||
END = "END",
|
||||
IDENTIFIER = "IDENTIFIER",
|
||||
STRING = "STRING",
|
||||
NUMBER = "NUMBER",
|
||||
@@ -1028,4 +1173,4 @@ declare global {
|
||||
|
||||
declare const MeSqlark: typeof MetonaSqlark;
|
||||
|
||||
export { AriaEngine, AriaEngineConfig, ColumnDef, DatabaseConfig, DeleteStatement, DiskEngine, FieldType, HybridEngine, IStorageEngine, IndexedDBEngine, InsertStatement, MeSqlark, MemoryEngine, MetonaSqlark, OPFSBackend, OPFSEngine, SelectStatement, Statement, StorageMode, Table, TableSchema, UpdateStatement, VERSION, api, create, api as default, parse, tokenize };
|
||||
export { AriaEngine, AriaEngineConfig, ColumnDef, DatabaseConfig, DeleteStatement, DiskEngine, FieldType, HybridEngine, IStorageEngine, IndexedDBEngine, InsertStatement, MeSqlark, MemoryEngine, MetonaSqlark, OPFSBackend, OPFSEngine, SelectStatement, Statement, StorageMode, Table, TableSchema, UpdateStatement, VERSION, api, create, api as default, parse, parseAll, tokenize };
|
||||
|
||||
Vendored
+2055
-591
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+2055
-590
File diff suppressed because it is too large
Load Diff
Vendored
+1
-1
File diff suppressed because one or more lines are too long
Vendored
+1
-1
File diff suppressed because one or more lines are too long
+26
-26
@@ -1,26 +1,26 @@
|
||||
module.exports = {
|
||||
testEnvironment: 'jsdom',
|
||||
setupFiles: ['./jest.setup.js'],
|
||||
transform: {
|
||||
'^.+\\.ts$': 'babel-jest',
|
||||
},
|
||||
transformIgnorePatterns: [
|
||||
'/node_modules/(?!(@rollup)/)',
|
||||
],
|
||||
moduleFileExtensions: ['ts', 'js', 'json'],
|
||||
collectCoverageFrom: [
|
||||
'src/**/*.ts',
|
||||
'!src/index.ts',
|
||||
'!src/**/index.ts',
|
||||
'!src/query/ast.ts',
|
||||
'!src/engine/interface.ts',
|
||||
'!src/engine/opfs.ts',
|
||||
'!src/integrations/**',
|
||||
],
|
||||
coverageDirectory: 'coverage',
|
||||
coverageReporters: ['text', 'lcov'],
|
||||
verbose: true,
|
||||
// Must have: fake-indexeddb + debounce timers keep event loop alive
|
||||
forceExit: true,
|
||||
testTimeout: 15000,
|
||||
};
|
||||
module.exports = {
|
||||
testEnvironment: 'jsdom',
|
||||
setupFiles: ['./jest.setup.js'],
|
||||
transform: {
|
||||
'^.+\\.ts$': 'babel-jest',
|
||||
},
|
||||
transformIgnorePatterns: [
|
||||
'/node_modules/(?!(@rollup)/)',
|
||||
],
|
||||
moduleFileExtensions: ['ts', 'js', 'json'],
|
||||
collectCoverageFrom: [
|
||||
'src/**/*.ts',
|
||||
'!src/index.ts',
|
||||
'!src/**/index.ts',
|
||||
'!src/query/ast.ts',
|
||||
'!src/engine/interface.ts',
|
||||
'!src/engine/opfs.ts',
|
||||
'!src/integrations/**',
|
||||
],
|
||||
coverageDirectory: 'coverage',
|
||||
coverageReporters: ['text', 'lcov'],
|
||||
verbose: true,
|
||||
// Must have: fake-indexeddb + debounce timers keep event loop alive
|
||||
forceExit: true,
|
||||
testTimeout: 15000,
|
||||
};
|
||||
|
||||
+27
-15
@@ -1,15 +1,27 @@
|
||||
// jest setup: polyfill structuredClone for fake-indexeddb
|
||||
if (typeof globalThis.structuredClone !== 'function') {
|
||||
globalThis.structuredClone = (obj) => JSON.parse(JSON.stringify(obj));
|
||||
}
|
||||
|
||||
// polyfill TextEncoder/TextDecoder for jsdom environment
|
||||
if (typeof globalThis.TextEncoder === 'undefined') {
|
||||
const { TextEncoder: TE, TextDecoder: TD } = require('util');
|
||||
globalThis.TextEncoder = TE;
|
||||
globalThis.TextDecoder = TD;
|
||||
}
|
||||
if (typeof globalThis.TextDecoder === 'undefined') {
|
||||
const { TextDecoder: TD } = require('util');
|
||||
globalThis.TextDecoder = TD;
|
||||
}
|
||||
// jest setup: polyfill structuredClone for fake-indexeddb
|
||||
if (typeof globalThis.structuredClone !== 'function') {
|
||||
globalThis.structuredClone = (obj) => JSON.parse(JSON.stringify(obj));
|
||||
}
|
||||
|
||||
// polyfill TextEncoder/TextDecoder for jsdom environment
|
||||
if (typeof globalThis.TextEncoder === 'undefined') {
|
||||
const { TextEncoder: TE, TextDecoder: TD } = require('util');
|
||||
globalThis.TextEncoder = TE;
|
||||
globalThis.TextDecoder = TD;
|
||||
}
|
||||
if (typeof globalThis.TextDecoder === 'undefined') {
|
||||
const { TextDecoder: TD } = require('util');
|
||||
globalThis.TextDecoder = TD;
|
||||
}
|
||||
|
||||
// polyfill WebCrypto (crypto.subtle) — jsdom 仅提供 getRandomValues
|
||||
if (typeof globalThis.crypto === 'undefined' || typeof globalThis.crypto.subtle === 'undefined') {
|
||||
const { webcrypto } = require('crypto');
|
||||
globalThis.crypto = webcrypto;
|
||||
// jsdom 环境暴露 getRandomValues 的旧引用可能被覆盖，统一替换
|
||||
Object.defineProperty(globalThis, 'crypto', {
|
||||
value: webcrypto,
|
||||
writable: true,
|
||||
configurable: true,
|
||||
});
|
||||
}
|
||||
|
||||
Generated
+8111
-8111
File diff suppressed because it is too large
Load Diff
+13
-5
@@ -1,17 +1,25 @@
|
||||
{
|
||||
"name": "@metona-team/metona-sqlark",
|
||||
"version": "0.2.5",
|
||||
"version": "0.3.2",
|
||||
"description": "Frontend SQL database with in-memory and disk dual-mode storage",
|
||||
"type": "module",
|
||||
"main": "dist/metona-sqlark.js",
|
||||
"module": "src/index.ts",
|
||||
"main": "dist/metona-sqlark.cjs",
|
||||
"module": "dist/metona-sqlark.esm.js",
|
||||
"unpkg": "dist/metona-sqlark.min.js",
|
||||
"jsdelivr": "dist/metona-sqlark.min.js",
|
||||
"types": "dist/metona-sqlark.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"import": "./src/index.ts",
|
||||
"require": "./dist/metona-sqlark.js",
|
||||
"import": "./dist/metona-sqlark.esm.js",
|
||||
"require": "./dist/metona-sqlark.cjs",
|
||||
"types": "./dist/metona-sqlark.d.ts"
|
||||
},
|
||||
"./react": {
|
||||
"import": "./src/integrations/react.ts",
|
||||
"types": "./dist/metona-sqlark.d.ts"
|
||||
},
|
||||
"./vue": {
|
||||
"import": "./src/integrations/vue.ts",
|
||||
"types": "./dist/metona-sqlark.d.ts"
|
||||
}
|
||||
},
|
||||
|
||||
+7
-2
@@ -40,6 +40,8 @@ export default [
|
||||
exports: 'named',
|
||||
sourcemap: true,
|
||||
},
|
||||
// react/vue 为 peer dependency（integrations 子路径），不打包
|
||||
external: ['react', 'vue'],
|
||||
plugins: [...basePlugins, ...devPlugins],
|
||||
},
|
||||
// Only build all formats in production
|
||||
@@ -53,17 +55,19 @@ export default [
|
||||
exports: 'named',
|
||||
sourcemap: true,
|
||||
},
|
||||
external: ['react', 'vue'],
|
||||
plugins: basePlugins,
|
||||
},
|
||||
// CommonJS
|
||||
// CommonJS（.cjs 后缀：package.json 为 type:module，.js 会被 Node 按 ESM 解析）
|
||||
{
|
||||
input: 'src/index.ts',
|
||||
output: {
|
||||
file: 'dist/metona-sqlark.cjs.js',
|
||||
file: 'dist/metona-sqlark.cjs',
|
||||
format: 'cjs',
|
||||
exports: 'named',
|
||||
sourcemap: true,
|
||||
},
|
||||
external: ['react', 'vue'],
|
||||
plugins: basePlugins,
|
||||
},
|
||||
// UMD minified
|
||||
@@ -76,6 +80,7 @@ export default [
|
||||
exports: 'named',
|
||||
sourcemap: false,
|
||||
},
|
||||
external: ['react', 'vue'],
|
||||
plugins: [
|
||||
...basePlugins,
|
||||
terser({
|
||||
|
||||
@@ -0,0 +1,266 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>⚡ 性能基准 — MetonaSqlark v0.3.2</title>
|
||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||
<style>
|
||||
:root {
|
||||
--bg:#0a0a0f; --surface:#0d1117; --surface2:#161b22; --border:#30363d;
|
||||
--primary:#6366f1; --accent:#06b6d4; --accent2:#ec4899; --green:#22c55e;
|
||||
--text:#e2e8f0; --text2:#8b949e; --radius:12px;
|
||||
--gradient:linear-gradient(135deg,#6366f1,#06b6d4,#ec4899);
|
||||
}
|
||||
* { margin:0; padding:0; box-sizing:border-box; }
|
||||
body { font-family:'Inter',-apple-system,system-ui,sans-serif; background:var(--bg); color:var(--text); min-height:100vh; }
|
||||
header {
|
||||
height:56px; display:flex; align-items:center; justify-content:space-between; padding:0 24px;
|
||||
background:var(--surface); border-bottom:1px solid var(--border); position:sticky; top:0; z-index:10;
|
||||
}
|
||||
.logo { font-weight:800; font-size:1.1rem; display:flex; align-items:center; gap:10px; }
|
||||
.logo .icon { width:28px; height:28px; border-radius:7px; background:var(--gradient); display:flex; align-items:center; justify-content:center; font-size:0.85rem; font-weight:900; color:#fff; }
|
||||
.logo span { background:var(--gradient); -webkit-background-clip:text; -webkit-text-fill-color:transparent; }
|
||||
nav { display:flex; gap:20px; }
|
||||
nav a { color:var(--text2); text-decoration:none; font-size:0.88rem; transition:.2s; }
|
||||
nav a:hover,.nav-active { color:var(--text); }
|
||||
.container { max-width:1000px; margin:0 auto; padding:32px 24px 60px; }
|
||||
.hero { margin-bottom:28px; }
|
||||
.hero h1 { font-size:1.6rem; margin-bottom:8px; }
|
||||
.hero p { color:var(--text2); font-size:0.92rem; line-height:1.7; }
|
||||
.hero p code { background:var(--surface2); padding:2px 6px; border-radius:5px; font-size:0.85em; }
|
||||
.controls { display:flex; gap:12px; align-items:center; margin-bottom:24px; flex-wrap:wrap; }
|
||||
.btn {
|
||||
display:inline-flex; align-items:center; gap:6px; padding:10px 22px; border-radius:8px;
|
||||
font-weight:600; font-size:0.85rem; cursor:pointer; border:none; transition:.25s; font-family:inherit;
|
||||
}
|
||||
.btn-run { background:var(--gradient); color:#fff; box-shadow:0 4px 15px rgba(99,102,241,0.35); }
|
||||
.btn-run:hover { transform:translateY(-1px); box-shadow:0 6px 25px rgba(99,102,241,0.5); }
|
||||
.btn-run:active { transform:scale(0.97); }
|
||||
.btn-run:disabled { opacity:0.5; cursor:not-allowed; transform:none; }
|
||||
.btn-small { background:var(--surface2); color:var(--text2); border:1px solid var(--border); font-size:0.8rem; padding:8px 16px; }
|
||||
.btn-small:hover { color:var(--text); border-color:var(--text2); }
|
||||
select {
|
||||
background:var(--surface2); color:var(--text); border:1px solid var(--border); border-radius:8px;
|
||||
padding:9px 12px; font-size:0.85rem; font-family:inherit; outline:none;
|
||||
}
|
||||
.status { font-size:0.85rem; color:var(--text2); }
|
||||
.status .spinner { display:inline-block; width:14px; height:14px; border:2px solid var(--border); border-top-color:var(--accent); border-radius:50%; animation:spin 0.8s linear infinite; vertical-align:-2px; }
|
||||
@keyframes spin { to { transform:rotate(360deg); } }
|
||||
table { width:100%; border-collapse:collapse; background:var(--surface); border-radius:var(--radius); overflow:hidden; border:1px solid var(--border); }
|
||||
th,td { padding:12px 16px; text-align:left; font-size:0.88rem; border-bottom:1px solid var(--border); }
|
||||
th { background:var(--surface2); color:var(--text2); font-size:0.78rem; text-transform:uppercase; letter-spacing:0.8px; white-space:nowrap; }
|
||||
tr:last-child td { border-bottom:none; }
|
||||
td.num { font-family:'JetBrains Mono',monospace; text-align:right; }
|
||||
td.best { color:var(--green); font-weight:700; }
|
||||
.group-header td { background:rgba(99,102,241,0.08); font-weight:700; color:var(--accent); }
|
||||
.section { margin-bottom:32px; }
|
||||
.section h2 { font-size:1.05rem; margin-bottom:14px; color:var(--text); display:flex; align-items:center; gap:8px; }
|
||||
.section h2 .tag { font-size:0.68rem; padding:3px 8px; border-radius:999px; background:var(--surface2); border:1px solid var(--border); color:var(--text2); font-weight:600; }
|
||||
.note { font-size:0.8rem; color:var(--text2); margin-top:14px; line-height:1.7; }
|
||||
.note code { background:var(--surface2); padding:2px 5px; border-radius:4px; }
|
||||
.summary-cards { display:grid; grid-template-columns:repeat(auto-fit,minmax(180px,1fr)); gap:14px; margin-bottom:28px; }
|
||||
.card { background:var(--surface); border:1px solid var(--border); border-radius:var(--radius); padding:18px; }
|
||||
.card .val { font-size:1.5rem; font-weight:800; background:var(--gradient); -webkit-background-clip:text; -webkit-text-fill-color:transparent; }
|
||||
.card .lbl { font-size:0.78rem; color:var(--text2); margin-top:4px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<div class="logo"><div class="icon">◈</div><span>MetonaSqlark</span></div>
|
||||
<nav>
|
||||
<a href="index.html">首页</a>
|
||||
<a href="docs.html">文档</a>
|
||||
<a href="demo.html">演示</a>
|
||||
<a href="benchmark.html" class="nav-active">基准</a>
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<div class="container">
|
||||
<div class="hero">
|
||||
<h1>⚡ 性能基准测试</h1>
|
||||
<p>在浏览器中实测 <code>Memory</code> 与 <code>Aria</code>（LSM-Tree 自研引擎）两种模式的真实性能。
|
||||
数据量分三档（1K / 10K / 50K 行），测量 <code>INSERT</code> / 主键查询 / 索引查询 / <code>UPDATE</code> / <code>DELETE</code> 的吞吐（ops/sec）。
|
||||
<strong>测试结果取决于你的设备与浏览器</strong>，仅供参考。</p>
|
||||
</div>
|
||||
|
||||
<div class="controls">
|
||||
<button class="btn btn-run" id="run-btn" onclick="runAll()">▶ 运行完整基准</button>
|
||||
<button class="btn btn-small" onclick="runAll(true)">⚡ 快速模式（1K/10K）</button>
|
||||
<button class="btn btn-small" onclick="window.location.reload()">↻ 重置</button>
|
||||
<span class="status" id="status"></span>
|
||||
</div>
|
||||
|
||||
<div class="summary-cards" id="summary"></div>
|
||||
|
||||
<div class="section" id="result-section" style="display:none">
|
||||
<h2>测试结果 <span class="tag">ops/sec</span></h2>
|
||||
<table id="result-table">
|
||||
<thead><tr><th>引擎</th><th>数据量</th><th>INSERT</th><th>主键查询</th><th>索引查询</th><th>UPDATE</th><th>DELETE</th></tr></thead>
|
||||
<tbody></tbody>
|
||||
</table>
|
||||
<p class="note">注：Memory 模式为纯内存 Map 存储；Aria 模式为 LSM-Tree + MemTable 存储（<code>storageBackend: 'memory'</code>，无磁盘 I/O）。
|
||||
INSERT 为单行批量写入（含 WAL 记录）；UPDATE/DELETE 按主键条件执行。</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="../dist/metona-sqlark.js"></script>
|
||||
<script>
|
||||
const M = window.MetonaSqlark;
|
||||
const SIZES = [1000, 10000, 50000];
|
||||
const SIZES_FAST = [1000, 10000];
|
||||
|
||||
let cancelled = false;
|
||||
|
||||
function setStatus(text, spin) {
|
||||
const el = document.getElementById('status');
|
||||
el.innerHTML = (spin ? '<span class="spinner"></span> ' : '') + text;
|
||||
}
|
||||
function setRunning(on) {
|
||||
document.getElementById('run-btn').disabled = on;
|
||||
}
|
||||
|
||||
function makeRows(n) {
|
||||
const rows = [];
|
||||
for (let i = 0; i < n; i++) {
|
||||
rows.push({ id: 'k' + i, name: 'User' + i, age: 18 + (i % 60), city: 'City' + (i % 100) });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
async function createDb(mode) {
|
||||
const db = new M.MetonaSqlark({
|
||||
name: 'bench-' + mode + '-' + Date.now(),
|
||||
mode,
|
||||
diskEngine: 'memory',
|
||||
});
|
||||
await db.init();
|
||||
await db.defineTable('bench', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number', index: true },
|
||||
city: { type: 'string' },
|
||||
});
|
||||
return db;
|
||||
}
|
||||
|
||||
async function timeIt(fn) {
|
||||
const start = performance.now();
|
||||
await fn();
|
||||
return performance.now() - start;
|
||||
}
|
||||
|
||||
async function runBench(mode, size) {
|
||||
const db = await createDb(mode);
|
||||
const rows = makeRows(size);
|
||||
const result = { mode, size, insert: 0, pk: 0, idx: 0, update: 0, del: 0 };
|
||||
|
||||
try {
|
||||
// INSERT（批量）
|
||||
let ms = await timeIt(() => db.table('bench').insertMany(rows));
|
||||
result.insert = Math.round(size / (ms / 1000));
|
||||
|
||||
// 主键查询 × 500
|
||||
ms = await timeIt(async () => {
|
||||
for (let i = 0; i < 500; i++) await db.query(`SELECT * FROM bench WHERE id = 'k${(i * 37) % size}'`);
|
||||
});
|
||||
result.pk = Math.round(500 / (ms / 1000));
|
||||
|
||||
// 索引查询 × 500
|
||||
ms = await timeIt(async () => {
|
||||
for (let i = 0; i < 500; i++) await db.query(`SELECT * FROM bench WHERE age = ${18 + ((i * 17) % 60)}`);
|
||||
});
|
||||
result.idx = Math.round(500 / (ms / 1000));
|
||||
|
||||
// UPDATE × 200（按主键）
|
||||
ms = await timeIt(async () => {
|
||||
for (let i = 0; i < 200; i++) await db.query(`UPDATE bench SET name = 'X${i}' WHERE id = 'k${(i * 53) % size}'`);
|
||||
});
|
||||
result.update = Math.round(200 / (ms / 1000));
|
||||
|
||||
// DELETE × 200（按主键）
|
||||
ms = await timeIt(async () => {
|
||||
for (let i = 0; i < 200; i++) await db.query(`DELETE FROM bench WHERE id = 'k${(i * 71) % size}'`);
|
||||
});
|
||||
result.del = Math.round(200 / (ms / 1000));
|
||||
} finally {
|
||||
await db.close();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function runAll(quick) {
|
||||
if (cancelled) { cancelled = false; }
|
||||
const sizes = quick ? SIZES_FAST : SIZES;
|
||||
const btn = document.getElementById('run-btn');
|
||||
btn.textContent = '⏳ 测试中…';
|
||||
setRunning(true);
|
||||
document.getElementById('result-section').style.display = 'none';
|
||||
document.getElementById('summary').innerHTML = '';
|
||||
|
||||
const results = [];
|
||||
for (const mode of ['memory', 'aria']) {
|
||||
for (const size of sizes) {
|
||||
setStatus(`正在测试 ${mode === 'memory' ? 'Memory' : 'Aria'} 引擎 · ${size.toLocaleString()} 行…`, true);
|
||||
// 让 UI 刷新
|
||||
await new Promise((r) => setTimeout(r, 30));
|
||||
try {
|
||||
const r = await runBench(mode, size);
|
||||
results.push(r);
|
||||
} catch (e) {
|
||||
setStatus(`✗ ${mode}/${size} 失败: ${e.message}`, false);
|
||||
console.error(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
renderResults(results);
|
||||
setStatus('✓ 测试完成', false);
|
||||
btn.textContent = '▶ 运行完整基准';
|
||||
setRunning(false);
|
||||
}
|
||||
|
||||
function renderResults(results) {
|
||||
const tbody = document.querySelector('#result-table tbody');
|
||||
tbody.innerHTML = '';
|
||||
|
||||
// 汇总卡片
|
||||
const summary = document.getElementById('summary');
|
||||
let bestInsert = 0, bestInsertMode = '', bestPk = 0, bestPkMode = '', totalTests = 0;
|
||||
for (const r of results) {
|
||||
totalTests++;
|
||||
if (r.insert > bestInsert) { bestInsert = r.insert; bestInsertMode = r.mode; }
|
||||
if (r.pk > bestPk) { bestPk = r.pk; bestPkMode = r.mode; }
|
||||
}
|
||||
summary.innerHTML = `
|
||||
<div class="card"><div class="val">${results.length} 组</div><div class="lbl">测试组数</div></div>
|
||||
<div class="card"><div class="val">${bestInsert.toLocaleString()} ops/s</div><div class="lbl">INSERT 峰值（${bestInsertMode}）</div></div>
|
||||
<div class="card"><div class="val">${bestPk.toLocaleString()} ops/s</div><div class="lbl">主键查询峰值（${bestPkMode}）</div></div>
|
||||
`;
|
||||
|
||||
// 表格（按引擎分组）
|
||||
let lastMode = '';
|
||||
for (const r of results) {
|
||||
if (r.mode !== lastMode) {
|
||||
lastMode = r.mode;
|
||||
const row = document.createElement('tr');
|
||||
row.className = 'group-header';
|
||||
row.innerHTML = `<td colspan="6">${r.mode === 'memory' ? '🚀 Memory 引擎' : '🌲 Aria 引擎（LSM-Tree）'}</td>`;
|
||||
tbody.appendChild(row);
|
||||
}
|
||||
const row = document.createElement('tr');
|
||||
const fmt = (v) => `<td class="num">${v ? v.toLocaleString() : '—'}</td>`;
|
||||
row.innerHTML = `
|
||||
<td>${r.mode}</td>
|
||||
<td>${r.size.toLocaleString()} 行</td>
|
||||
${fmt(r.insert)}${fmt(r.pk)}${fmt(r.idx)}${fmt(r.update)}${fmt(r.del)}
|
||||
`;
|
||||
tbody.appendChild(row);
|
||||
}
|
||||
|
||||
document.getElementById('result-section').style.display = 'block';
|
||||
document.getElementById('result-section').scrollIntoView({ behavior: 'smooth' });
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
+131
-9
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>🧪 在线演示 — MetonaSqlark v0.2.5</title>
|
||||
<title>🧪 在线演示 — MetonaSqlark v0.3.2</title>
|
||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||
<style>
|
||||
:root {
|
||||
@@ -82,14 +82,15 @@
|
||||
<a href="index.html">首页</a>
|
||||
<a href="docs.html">文档</a>
|
||||
<a href="demo.html" class="nav-active">演示</a>
|
||||
<a href="benchmark.html">基准</a>
|
||||
</nav>
|
||||
<div class="status"><span class="dot"></span> Memory 模式 — v0.2.5</div>
|
||||
<div class="status"><span class="dot"></span> Memory 模式 — v0.3.2</div>
|
||||
</header>
|
||||
|
||||
<div class="main">
|
||||
<div class="editor-panel">
|
||||
<div class="editor-area">
|
||||
<textarea id="sql-input" placeholder="输入 SQL 语句... SELECT * FROM users; INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28); SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.2.5 在线演示
|
||||
<textarea id="sql-input" placeholder="输入 SQL 语句... SELECT * FROM users; INSERT INTO users VALUES ('4', 'Diana', 'diana@test.com', 28); SELECT u.name, o.amount FROM users u INNER JOIN orders o ON u.id = o.user_id;">-- 🚀 MetonaSqlark v0.3.2 在线演示
|
||||
-- 已预置 users / orders / products 表数据
|
||||
-- 新特性: ALTER TABLE · TRUNCATE TABLE · WAL同步 · MVCC · SQL注入防护
|
||||
|
||||
@@ -121,6 +122,12 @@
|
||||
<button class="btn btn-preset" onclick="loadPreset('adv')">🧪 高级</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('alter')">🏗 ALTER</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('truncate')">🗑 TRUNCATE</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('casewhen')">🎯 CASE WHEN</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('union')">🔀 UNION</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('insertselect')">📥 INSERT SELECT</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('exists')">🔍 EXISTS</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('index')">🗂 索引</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('multistmt')">📜 多语句/事务</button>
|
||||
<button class="btn btn-preset" onclick="loadPreset('aria')" style="color:#ec4899;border-color:#ec4899;">🌲 Aria</button>
|
||||
</div>
|
||||
</div>
|
||||
@@ -511,9 +518,9 @@ SELECT COUNT(*) as total FROM temp_logs;
|
||||
|
||||
-- 清理
|
||||
DROP TABLE temp_logs;`,
|
||||
aria: `-- 🌲 AriaEngine 演示 (v0.2.5)
|
||||
aria: `-- 🌲 AriaEngine 演示 (v0.2.5 → v0.3.2)
|
||||
-- AriaEngine: LSM-Tree 自研存储引擎
|
||||
-- 支持 LSM-Tree · WAL CRC同步 · MVCC版本链 · BloomFilter · 二级索引 · AES-GCM · 721测试
|
||||
-- 支持 LSM-Tree · WAL CRC同步 · MVCC版本链 · BloomFilter · 二级索引 · AES-GCM · 836测试
|
||||
|
||||
-- 基础 CRUD 完全兼容
|
||||
CREATE TABLE IF NOT EXISTS tasks (
|
||||
@@ -533,18 +540,133 @@ DROP TABLE temp_logs;`,
|
||||
-- 聚合统计
|
||||
SELECT done, COUNT(*) as cnt FROM tasks GROUP BY done;
|
||||
|
||||
-- CASE WHEN 表达式
|
||||
SELECT title,
|
||||
CASE WHEN done THEN '✅ done' ELSE '⏳ pending' END AS status
|
||||
FROM tasks;
|
||||
|
||||
-- EXISTS 关联子查询
|
||||
SELECT u.name FROM users u
|
||||
WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id);
|
||||
|
||||
-- AriaEngine 特性:
|
||||
-- • LSM-Tree: MemTable (红黑树) → SSTable 多级索引
|
||||
-- • WAL: Write-Ahead Log 保证崩溃恢复
|
||||
-- • WAL: Write-Ahead Log 保证崩溃恢复 + 批量组提交
|
||||
-- • MVCC: 版本链 + 快照隔离
|
||||
-- • Buffer Pool: LRU 页面缓存 (256页 ~ 1MB)
|
||||
-- • Buffer Pool: SSTable LRU 缓存 (256页 ~ 1MB) ✅ 已生效
|
||||
-- • Bloom Filter: FNV-1a + Murmur 双哈希
|
||||
-- • Slotted Page: 4KB 页面 + Tuple 二进制编码
|
||||
-- • 二级索引: 每列独立 LSM + 动态 CREATE INDEX
|
||||
|
||||
-- 生产环境: mode: 'aria' 激活自研引擎
|
||||
-- const db = await MetonaSqlark.create({
|
||||
-- name: 'my-app', mode: 'aria'
|
||||
-- });`,
|
||||
casewhen: `-- 🎯 CASE WHEN 条件表达式 (v0.3.1 / v0.3.2)
|
||||
|
||||
-- SELECT 列：多 WHEN + ELSE
|
||||
SELECT name, age,
|
||||
CASE WHEN age >= 30 THEN 'senior'
|
||||
WHEN age >= 25 THEN 'mid'
|
||||
ELSE 'junior' END AS age_group
|
||||
FROM users ORDER BY age;
|
||||
|
||||
-- WHERE 条件中的 CASE WHEN
|
||||
SELECT name FROM users
|
||||
WHERE CASE WHEN age >= 25 THEN 'adult' ELSE 'young' END = 'adult';
|
||||
|
||||
-- 聚合中的 CASE WHEN
|
||||
SELECT
|
||||
SUM(CASE WHEN age >= 30 THEN 1 ELSE 0 END) AS seniors,
|
||||
SUM(CASE WHEN age < 30 THEN 1 ELSE 0 END) AS juniors,
|
||||
AVG(CASE WHEN age >= 25 THEN age END) AS avg_adult_age
|
||||
FROM users;`,
|
||||
union: `-- 🔀 UNION / UNION ALL (v0.3.0)
|
||||
|
||||
-- UNION 去重合并两表数据
|
||||
SELECT name FROM users WHERE age >= 30
|
||||
UNION
|
||||
SELECT name FROM users WHERE age < 30;
|
||||
|
||||
-- UNION 对重复行去重（同查询两次 → 结果去重）
|
||||
SELECT category FROM products
|
||||
UNION
|
||||
SELECT category FROM products;
|
||||
|
||||
-- UNION ALL 保留重复
|
||||
SELECT category FROM products WHERE price >= 100
|
||||
UNION ALL
|
||||
SELECT category FROM products WHERE price >= 50;`,
|
||||
insertselect: `-- 📥 INSERT INTO ... SELECT (v0.3.0)
|
||||
|
||||
-- 建备份表并复制全量数据
|
||||
DROP TABLE IF EXISTS users_backup;
|
||||
CREATE TABLE users_backup (
|
||||
id STRING PRIMARY KEY, name STRING, email STRING, age NUMBER
|
||||
);
|
||||
INSERT INTO users_backup SELECT * FROM users;
|
||||
|
||||
-- 确认复制
|
||||
SELECT COUNT(*) as backed_up FROM users_backup;
|
||||
|
||||
-- INSERT SELECT 带 WHERE 过滤
|
||||
DROP TABLE IF EXISTS big_spenders;
|
||||
CREATE TABLE big_spenders (
|
||||
id STRING PRIMARY KEY, product STRING, amount NUMBER
|
||||
);
|
||||
INSERT INTO big_spenders (id, product, amount)
|
||||
SELECT o.id, o.product, o.amount FROM orders o WHERE o.amount >= 150;
|
||||
|
||||
-- 确认过滤结果
|
||||
SELECT * FROM big_spenders;`,
|
||||
exists: `-- 🔍 EXISTS 关联子查询 (v0.3.0 / v0.3.1)
|
||||
|
||||
-- 有订单的用户（关联子查询）
|
||||
SELECT u.name FROM users u
|
||||
WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id);
|
||||
|
||||
-- 无订单的用户（NOT EXISTS）
|
||||
SELECT u.name FROM users u
|
||||
WHERE NOT EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id);
|
||||
|
||||
-- EXISTS + JOIN 组合
|
||||
SELECT u.name, o.product, o.amount FROM users u
|
||||
JOIN orders o ON u.id = o.user_id
|
||||
WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 100);`,
|
||||
index: `-- 🗂 动态索引 CREATE / DROP INDEX (v0.3.0)
|
||||
|
||||
-- 为 orders.user_id 创建索引
|
||||
CREATE INDEX idx_orders_user ON orders (user_id);
|
||||
|
||||
-- 索引查找（走二级索引）
|
||||
SELECT u.name, o.product, o.amount
|
||||
FROM orders o JOIN users u ON u.id = o.user_id
|
||||
WHERE o.user_id = '1';
|
||||
|
||||
-- 删除索引
|
||||
DROP INDEX idx_orders_user ON orders (user_id);
|
||||
|
||||
-- 删除后回退全表扫描（结果不变）
|
||||
SELECT * FROM orders WHERE user_id = '3';`,
|
||||
multistmt: `-- 📜 多语句 + 事务语句 (v0.3.0)
|
||||
|
||||
-- 分号分隔的多语句一次执行
|
||||
DROP TABLE IF EXISTS audit;
|
||||
CREATE TABLE audit (id STRING PRIMARY KEY, msg STRING);
|
||||
INSERT INTO audit VALUES ('a1', 'first');
|
||||
INSERT INTO audit VALUES ('a2', 'second');
|
||||
SELECT * FROM audit ORDER BY id;
|
||||
|
||||
-- 事务语句：BEGIN → ROLLBACK（a3 不会生效）
|
||||
BEGIN;
|
||||
INSERT INTO audit VALUES ('a3', 'will be rolled back');
|
||||
ROLLBACK;
|
||||
SELECT * FROM audit ORDER BY id;
|
||||
|
||||
-- 事务语句：BEGIN → COMMIT（a4 生效）
|
||||
BEGIN;
|
||||
INSERT INTO audit VALUES ('a4', 'will be committed');
|
||||
COMMIT;
|
||||
SELECT COUNT(*) as total FROM audit;`,
|
||||
};
|
||||
|
||||
function loadPreset(name) {
|
||||
@@ -564,7 +686,7 @@ document.addEventListener('keydown', e => {
|
||||
|
||||
// Boot
|
||||
initDB().then(() => {
|
||||
console.log('✅ MetonaSqlark v0.2.5 demo ready');
|
||||
console.log('✅ MetonaSqlark v0.3.2 demo ready');
|
||||
setTimeout(runQuery, 300);
|
||||
}).catch(err => {
|
||||
renderError('初始化失败: ' + err.message);
|
||||
|
||||
+58
-6
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>📖 API 文档 — MetonaSqlark v0.2.5</title>
|
||||
<title>📖 API 文档 — MetonaSqlark v0.3.2</title>
|
||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
|
||||
<style>
|
||||
:root {
|
||||
@@ -69,6 +69,7 @@
|
||||
<a href="index.html">首页</a>
|
||||
<a href="docs.html" class="nav-active">文档</a>
|
||||
<a href="demo.html">演示</a>
|
||||
<a href="benchmark.html">基准</a>
|
||||
</nav>
|
||||
</div>
|
||||
</header>
|
||||
@@ -121,7 +122,7 @@ npm install @metona-team/metona-sqlark</pre>
|
||||
|
||||
<h3>CDN / UMD</h3>
|
||||
<pre><span class="c"><!-- UMD 格式，暴露 window.MetonaSqlark 和 window.MeSqlark --></span>
|
||||
<script src=<span class="s">"https://git.metona.cn/.../metona-sqlark.min.js"</span>></script>
|
||||
<script src=<span class="s">"https://git.metona.cn/MetonaTeam/MetonaSqlark/raw/branch/master/dist/metona-sqlark.min.js"</span>></script>
|
||||
<script>
|
||||
<span class="k">const</span> db = <span class="k">await</span> window.<span class="f">MetonaSqlark</span>.create({...});
|
||||
<span class="c">// 或 window.MeSqlark.create(...) — 完全等价</span>
|
||||
@@ -138,9 +139,9 @@ npm install @metona-team/metona-sqlark</pre>
|
||||
<table>
|
||||
<tr><th>文件</th><th>格式</th><th>用途</th></tr>
|
||||
<tr><td><code>metona-sqlark.js</code></td><td>UMD</td><td>浏览器开发版（含 sourcemap）</td></tr>
|
||||
<tr><td><code>metona-sqlark.min.js</code></td><td>UMD (minified)</td><td>生产环境（~42KB / ~10KB gzip）</td></tr>
|
||||
<tr><td><code>metona-sqlark.min.js</code></td><td>UMD (minified)</td><td>生产环境（~105KB / ~27KB gzip）</td></tr>
|
||||
<tr><td><code>metona-sqlark.esm.js</code></td><td>ES Module</td><td>现代打包工具 / 浏览器 ESM</td></tr>
|
||||
<tr><td><code>metona-sqlark.cjs.js</code></td><td>CommonJS</td><td>Node.js require()</td></tr>
|
||||
<tr><td><code>metona-sqlark.cjs</code></td><td>CommonJS</td><td>Node.js require()</td></tr>
|
||||
<tr><td><code>metona-sqlark.d.ts</code></td><td>TypeScript 声明</td><td>类型提示</td></tr>
|
||||
</table>
|
||||
|
||||
@@ -237,6 +238,37 @@ db.<span class="f">isReady</span>(); <span class="c">// true</span>
|
||||
<span class="c">-- TRUNCATE TABLE — 快速清空表数据 (v0.2.5)</span>
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">'TRUNCATE TABLE old_logs'</span>);</pre>
|
||||
|
||||
<h3>SQL 扩展 (v0.3.0+)</h3>
|
||||
<pre><span class="c">// 多语句 — 分号分隔一次执行（返回最后一条结果）</span>
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">`CREATE TABLE t (id STRING PRIMARY KEY);
|
||||
INSERT INTO t VALUES ('1'); INSERT INTO t VALUES ('2')`</span>);
|
||||
|
||||
<span class="c">// 事务语句 — BEGIN / COMMIT / ROLLBACK</span>
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">'BEGIN'</span>);
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">"INSERT INTO t VALUES ('3')"</span>);
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">'ROLLBACK'</span>); <span class="c">// 回滚</span>
|
||||
|
||||
<span class="c">// INSERT INTO ... SELECT — 查询结果写入</span>
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">'INSERT INTO t SELECT id FROM t2 WHERE x > 1'</span>);
|
||||
|
||||
<span class="c">// UNION / UNION ALL — 合并查询</span>
|
||||
<span class="k">const</span> merged = <span class="k">await</span> db.<span class="f">query</span>(
|
||||
<span class="s">`SELECT name FROM users WHERE city = 'Beijing'
|
||||
UNION SELECT name FROM users WHERE age < 30`</span>);
|
||||
|
||||
<span class="c">// EXISTS / NOT EXISTS — 关联子查询</span>
|
||||
<span class="k">const</span> hasOrders = <span class="k">await</span> db.<span class="f">query</span>(<span class="s">`SELECT * FROM users u
|
||||
WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)`</span>);
|
||||
|
||||
<span class="c">// CASE WHEN — SELECT 列 / WHERE / 聚合 (v0.3.1 / v0.3.2)</span>
|
||||
<span class="k">const</span> labeled = <span class="k">await</span> db.<span class="f">query</span>(<span class="s">`SELECT name,
|
||||
CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users`</span>);
|
||||
<span class="k">const</span> adults = <span class="k">await</span> db.<span class="f">query</span>(<span class="s">`SELECT SUM(CASE WHEN age >= 18 THEN 1 ELSE 0 END) FROM users`</span>);
|
||||
|
||||
<span class="c">// CREATE / DROP INDEX — 动态二级索引</span>
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">'CREATE INDEX idx_users_city ON users (city)'</span>);
|
||||
<span class="k">await</span> db.<span class="f">query</span>(<span class="s">'DROP INDEX idx_users_city ON users (city)'</span>);</pre>
|
||||
|
||||
<h3>条件表达式</h3>
|
||||
<pre><span class="c">// 比较运算符</span>
|
||||
<span class="s">`WHERE age > 18 AND name LIKE 'A%'`</span>
|
||||
@@ -578,6 +610,25 @@ db.<span class="f">emit</span>(<span class="s">'users'</span>, { <span class="s"
|
||||
<span class="c">// 取消订阅</span>
|
||||
<span class="f">unsubscribe</span>();</pre>
|
||||
|
||||
<h3>多标签页同步 (v0.3.2)</h3>
|
||||
<pre><span class="c">// 启用 multiTabSync 后，其他标签页的写操作会广播到此标签页</span>
|
||||
<span class="k">const</span> db = <span class="k">await</span> MetonaSqlark.<span class="f">create</span>({
|
||||
<span class="s">name</span>: <span class="s">'my-app'</span>,
|
||||
<span class="s">mode</span>: <span class="s">'hybrid'</span>,
|
||||
<span class="s">multiTabSync</span>: <span class="k">true</span>,
|
||||
});
|
||||
|
||||
<span class="c">// 订阅其他标签页的变更（event.type === 'external'）</span>
|
||||
db.<span class="f">subscribe</span>(<span class="s">'users'</span>, (event) => {
|
||||
<span class="k">if</span> (event.<span class="s">type</span> === <span class="s">'external'</span>) {
|
||||
<span class="c">// Hybrid 模式已自动从磁盘重载，此处可刷新 UI</span>
|
||||
<span class="f">refreshList</span>();
|
||||
}
|
||||
});
|
||||
|
||||
<span class="c">// 手动广播（Table API 已自动广播；自定义写入可调用）</span>
|
||||
db.<span class="f">broadcastChange</span>(<span class="s">'users'</span>);</pre>
|
||||
|
||||
<h2 id="react">⚛️ React 集成</h2>
|
||||
<pre><span class="k">import</span> { useQuery, useTable, useDatabase } <span class="k">from</span> <span class="s">'@metona-team/metona-sqlark/react'</span>;
|
||||
<span class="k">import</span> { db } <span class="k">from</span> <span class="s">'./db'</span>;
|
||||
@@ -663,6 +714,7 @@ db.<span class="f">emit</span>(<span class="s">'users'</span>, { <span class="s"
|
||||
<tr><td><code>onReady</code></td><td><code>(db) => void</code></td><td>-</td><td>初始化完成回调</td></tr>
|
||||
<tr><td><code>onError</code></td><td><code>(err) => void</code></td><td>-</td><td>错误回调（v0.2.5 接入执行路径）</td></tr>
|
||||
<tr><td><code>maxRowsPerQuery</code></td><td><code>number</code></td><td><code>0</code></td><td>查询结果行数上限（0=不限制）✅ v0.2.5 生效</td></tr>
|
||||
<tr><td><code>multiTabSync</code></td><td><code>boolean</code></td><td><code>false</code></td><td>多标签页同步：BroadcastChannel 广播表变更，其他标签页自动刷新 🆕 v0.3.2</td></tr>
|
||||
</table>
|
||||
|
||||
<h2 id="engine">💾 存储引擎</h2>
|
||||
@@ -678,8 +730,8 @@ db.<span class="f">emit</span>(<span class="s">'users'</span>, { <span class="s"
|
||||
|
||||
<h2 id="aria-engine">🌲 AriaEngine 自研存储引擎</h2>
|
||||
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎，对标 SQLite 设计理念。<br>
|
||||
<strong>v0.2.4 生产级</strong> — 二级索引 · MVCC · BloomFilter · WAL CRC全同步 · AES-GCM加密 · Savepoint · EXPLAIN · ANALYZE · REINDEX · VACUUM · BufferPool · 701测试 · 零死代码。<br>
|
||||
<strong>v0.2.5 质量加固</strong> — WAL full模式真正同步 · MVCC接入读写路径 · SSTableReader二分查找统一 · crypto实例化 · IndexedDB索引利用 · compactLevel public接口 · WAL大小阈值自动checkpoint · SQL注入防护 · ALTER TABLE · TRUNCATE TABLE · 721测试 37套件。</p>
|
||||
<strong>v0.2.4 生产级</strong> — 二级索引 · MVCC · BloomFilter · WAL CRC全同步 · AES-GCM加密 · Savepoint · EXPLAIN · ANALYZE · REINDEX · VACUUM · BufferPool · 761测试 · 零死代码。<br>
|
||||
<strong>v0.3.2 表达式与并发</strong> — WAL full模式真正同步 · MVCC接入读写路径 · SSTableReader二分查找统一 · crypto实例化 · IndexedDB索引利用 · compactLevel public接口 · WAL大小阈值自动checkpoint · SQL注入防护 · ALTER TABLE · TRUNCATE TABLE · 761测试 41套件。</p>
|
||||
|
||||
<h3>存储模式对比</h3>
|
||||
<table>
|
||||
|
||||
+7
-6
@@ -143,6 +143,7 @@
|
||||
<a href="index.html">首页</a>
|
||||
<a href="docs.html">文档</a>
|
||||
<a href="demo.html">演示</a>
|
||||
<a href="benchmark.html">基准</a>
|
||||
<a href="https://git.metona.cn/MetonaTeam/MetonaSqlark" target="_blank">Gitea</a>
|
||||
</nav>
|
||||
<a href="demo.html" class="btn btn-primary">立即体验</a>
|
||||
@@ -152,7 +153,7 @@
|
||||
<!-- Hero -->
|
||||
<section class="hero">
|
||||
<div class="container">
|
||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.2.5 质量加固 — 721测试 37套件 · 五模式全覆盖 · WAL同步修复 · ALTER/TRUNCATE · SQL注入防护 · 零回归</div>
|
||||
<div class="badge" style="margin-bottom:24px;"><span class="dot"></span> v0.3.2 表达式与并发 — 761测试 41套件 · 五模式全覆盖 · WAL同步修复 · ALTER/TRUNCATE · SQL注入防护 · 零回归</div>
|
||||
<h1>前端的 <span class="gradient-text">SQL 数据库</span></h1>
|
||||
<p>TypeScript 原生构建，5 种存储引擎，支持完整 SQL 查询。<br>零运行时依赖，开箱即用。AriaEngine 自研引擎：LSM-Tree + WAL 同步 + MVCC。</p>
|
||||
<div class="actions">
|
||||
@@ -242,7 +243,7 @@ npm install @metona-team/metona-sqlark
|
||||
</div>
|
||||
<div class="feature-card">
|
||||
<div class="icon">🌲</div>
|
||||
<h3>AriaEngine <span style="font-size:0.65rem;color:var(--accent);vertical-align:super;">v0.2.5</span></h3>
|
||||
<h3>AriaEngine <span style="font-size:0.65rem;color:var(--accent);vertical-align:super;">v0.3.2</span></h3>
|
||||
<p>自研 LSM-Tree 页面式存储引擎。MemTable 红黑树 + 多级 SSTable、Bloom Filter 快速判存、WAL full模式真正同步、MVCC 版本链接入读写路径。</p>
|
||||
</div>
|
||||
<div class="feature-card">
|
||||
@@ -273,7 +274,7 @@ npm install @metona-team/metona-sqlark
|
||||
<div class="feature-card">
|
||||
<div class="icon">📦</div>
|
||||
<h3>零运行时依赖</h3>
|
||||
<p>纯 TypeScript 实现，不依赖任何第三方库。Tree-shakable，UMD/ESM/CJS 多格式输出，~10KB gzip。</p>
|
||||
<p>纯 TypeScript 实现，不依赖任何第三方库。Tree-shakable，UMD/ESM/CJS 多格式输出，~27KB gzip。</p>
|
||||
</div>
|
||||
<div class="feature-card">
|
||||
<div class="icon">📊</div>
|
||||
@@ -393,9 +394,9 @@ npm install @metona-team/metona-sqlark
|
||||
<p>MetonaSqlark 的核心指标</p>
|
||||
</div>
|
||||
<div class="stats">
|
||||
<div class="stat-card"><div class="num">721</div><div class="label">测试用例</div></div>
|
||||
<div class="stat-card"><div class="num">91.0%</div><div class="label">行覆盖率</div></div>
|
||||
<div class="stat-card"><div class="num">~10KB</div><div class="label">gzip 体积</div></div>
|
||||
<div class="stat-card"><div class="num">761</div><div class="label">测试用例</div></div>
|
||||
<div class="stat-card"><div class="num">81.0%</div><div class="label">行覆盖率</div></div>
|
||||
<div class="stat-card"><div class="num">~27KB</div><div class="label">gzip 体积</div></div>
|
||||
<div class="stat-card"><div class="num">5</div><div class="label">存储引擎</div></div>
|
||||
<div class="stat-card"><div class="num">36</div><div class="label">SQL 关键字</div></div>
|
||||
<div class="stat-card"><div class="num">37</div><div class="label">测试套件</div></div>
|
||||
|
||||
+215
-212
@@ -1,212 +1,215 @@
|
||||
/**
|
||||
* metona-sqlark Constants — 类型定义 / 默认配置 / 枚举
|
||||
* @module constants
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 存储模式
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 存储模式 */
|
||||
export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
||||
|
||||
/** 磁盘引擎类型 */
|
||||
export type DiskEngine = 'indexeddb' | 'opfs';
|
||||
|
||||
/** 所有存储模式 */
|
||||
export const STORAGE_MODES: StorageMode[] = ['memory', 'disk', 'hybrid', 'aria'];
|
||||
|
||||
/** 所有磁盘引擎 */
|
||||
export const DISK_ENGINES: DiskEngine[] = ['indexeddb', 'opfs'];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 字段类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 字段数据类型 */
|
||||
export type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json';
|
||||
|
||||
/** 所有字段类型 */
|
||||
export const FIELD_TYPES: FieldType[] = ['string', 'number', 'boolean', 'date', 'json'];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 列定义 & 表结构
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 列定义 */
|
||||
export interface ColumnDef {
|
||||
/** 字段类型 */
|
||||
type: FieldType;
|
||||
/** 是否主键 */
|
||||
primaryKey?: boolean;
|
||||
/** 是否必填 */
|
||||
required?: boolean;
|
||||
/** 是否唯一 */
|
||||
unique?: boolean;
|
||||
/** 默认值 */
|
||||
default?: unknown;
|
||||
/** 是否创建索引 */
|
||||
index?: boolean;
|
||||
/** 外键引用: 'table.column' */
|
||||
references?: string;
|
||||
/** 删除级联: 'CASCADE' | 'SET NULL' | 'RESTRICT' */
|
||||
onDelete?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
/** 更新级联: 'CASCADE' | 'SET NULL' | 'RESTRICT' */
|
||||
onUpdate?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
/** 字符串最大长度 */
|
||||
maxLength?: number;
|
||||
/** 数字最小值 */
|
||||
min?: number;
|
||||
/** 数字最大值 */
|
||||
max?: number;
|
||||
}
|
||||
|
||||
/** 表结构定义 */
|
||||
export interface TableSchema {
|
||||
/** 表名 */
|
||||
name: string;
|
||||
/** 列定义映射 */
|
||||
columns: Record<string, ColumnDef>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 数据库配置
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 数据库配置 */
|
||||
export interface DatabaseConfig {
|
||||
/** 数据库名称 */
|
||||
name: string;
|
||||
/** 存储模式 */
|
||||
mode?: StorageMode;
|
||||
/** 磁盘引擎（仅 mode='disk'|'hybrid' 时生效） */
|
||||
diskEngine?: DiskEngine;
|
||||
/** 版本号 */
|
||||
version?: number;
|
||||
/** 插件列表 */
|
||||
plugins?: MetonaPlugin[];
|
||||
/** 数据库就绪回调 */
|
||||
onReady?: (db: unknown) => void;
|
||||
/** 错误回调 */
|
||||
onError?: (error: Error) => void;
|
||||
/** 查询结果行数上限（默认 0，0 表示不限制） */
|
||||
maxRowsPerQuery?: number;
|
||||
/** 调试模式（启用后输出详细操作日志） */
|
||||
debug?: boolean;
|
||||
}
|
||||
|
||||
/** 数据库默认配置 */
|
||||
export const DB_DEFAULTS: Readonly<Required<Omit<DatabaseConfig, 'plugins' | 'onReady' | 'onError'>>> = Object.freeze({
|
||||
name: 'metona-sqlark',
|
||||
mode: 'hybrid' as const,
|
||||
diskEngine: 'indexeddb' as const,
|
||||
version: 1,
|
||||
maxRowsPerQuery: 0, // 0 = 不限制
|
||||
debug: false,
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Where 操作符
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Where 条件操作符 */
|
||||
export type WhereOperator = '$eq' | '$ne' | '$gt' | '$gte' | '$lt' | '$lte' | '$in' | '$nin' | '$like' | '$and' | '$or' | '$not';
|
||||
|
||||
/** 简单条件值：直接相等 */
|
||||
export type SimpleCondition = unknown;
|
||||
|
||||
/** 操作符条件 */
|
||||
export type OperatorCondition = Partial<Record<WhereOperator, unknown>>;
|
||||
|
||||
/** 字段条件：简单值 | 操作符对象 */
|
||||
export type FieldCondition = SimpleCondition | OperatorCondition;
|
||||
|
||||
/** Where 条件对象 */
|
||||
export type WhereCondition = Record<string, FieldCondition>;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 排序
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 排序方向 */
|
||||
export type SortDirection = 'asc' | 'desc';
|
||||
|
||||
/** 排序定义 */
|
||||
export interface OrderBy {
|
||||
/** 列名 */
|
||||
column: string;
|
||||
/** 排序方向 */
|
||||
direction: SortDirection;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 查询计划（引擎层使用）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 查询计划 — 由 Executor 编译 AST 后生成 */
|
||||
export interface QueryPlan {
|
||||
/** 表名 */
|
||||
table: string;
|
||||
/** 要返回的列（undefined = 全部，['*'] = 全部） */
|
||||
columns?: string[];
|
||||
/** 过滤条件 */
|
||||
where?: WhereCondition;
|
||||
/** 排序 */
|
||||
orderBy?: OrderBy[];
|
||||
/** 限制条数 */
|
||||
limit?: number;
|
||||
/** 偏移量 */
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 插件接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 钩子名称 */
|
||||
export type HookName =
|
||||
| 'beforeCreateTable' | 'afterCreateTable'
|
||||
| 'beforeDropTable' | 'afterDropTable'
|
||||
| 'beforeInsert' | 'afterInsert'
|
||||
| 'beforeUpdate' | 'afterUpdate'
|
||||
| 'beforeDelete' | 'afterDelete'
|
||||
| 'beforeQuery' | 'afterQuery'
|
||||
| 'beforeTransaction' | 'afterTransaction';
|
||||
|
||||
/** 插件定义 */
|
||||
export interface MetonaPlugin {
|
||||
/** 插件名称 */
|
||||
name: string;
|
||||
/** 插件版本 */
|
||||
version: string;
|
||||
/** 描述 */
|
||||
description?: string;
|
||||
/** 优先级，越大越先执行 */
|
||||
priority?: number;
|
||||
/** 安装 */
|
||||
install(db: unknown): void;
|
||||
/** 销毁 */
|
||||
destroy(): void;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 错误类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 数据库错误 */
|
||||
export class DatabaseError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public code: string,
|
||||
public details?: unknown,
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'DatabaseError';
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 版本
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const VERSION = '0.2.5';
|
||||
/**
|
||||
* metona-sqlark Constants — 类型定义 / 默认配置 / 枚举
|
||||
* @module constants
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 存储模式
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 存储模式 */
|
||||
export type StorageMode = 'memory' | 'disk' | 'hybrid' | 'aria';
|
||||
|
||||
/** 磁盘引擎类型 */
|
||||
export type DiskEngine = 'indexeddb' | 'opfs';
|
||||
|
||||
/** 所有存储模式 */
|
||||
export const STORAGE_MODES: StorageMode[] = ['memory', 'disk', 'hybrid', 'aria'];
|
||||
|
||||
/** 所有磁盘引擎 */
|
||||
export const DISK_ENGINES: DiskEngine[] = ['indexeddb', 'opfs'];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 字段类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 字段数据类型 */
|
||||
export type FieldType = 'string' | 'number' | 'boolean' | 'date' | 'json';
|
||||
|
||||
/** 所有字段类型 */
|
||||
export const FIELD_TYPES: FieldType[] = ['string', 'number', 'boolean', 'date', 'json'];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 列定义 & 表结构
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 列定义 */
|
||||
export interface ColumnDef {
|
||||
/** 字段类型 */
|
||||
type: FieldType;
|
||||
/** 是否主键 */
|
||||
primaryKey?: boolean;
|
||||
/** 是否必填 */
|
||||
required?: boolean;
|
||||
/** 是否唯一 */
|
||||
unique?: boolean;
|
||||
/** 默认值 */
|
||||
default?: unknown;
|
||||
/** 是否创建索引 */
|
||||
index?: boolean;
|
||||
/** 外键引用: 'table.column' */
|
||||
references?: string;
|
||||
/** 删除级联: 'CASCADE' | 'SET NULL' | 'RESTRICT' */
|
||||
onDelete?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
/** 更新级联: 'CASCADE' | 'SET NULL' | 'RESTRICT' */
|
||||
onUpdate?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
/** 字符串最大长度 */
|
||||
maxLength?: number;
|
||||
/** 数字最小值 */
|
||||
min?: number;
|
||||
/** 数字最大值 */
|
||||
max?: number;
|
||||
}
|
||||
|
||||
/** 表结构定义 */
|
||||
export interface TableSchema {
|
||||
/** 表名 */
|
||||
name: string;
|
||||
/** 列定义映射 */
|
||||
columns: Record<string, ColumnDef>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 数据库配置
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 数据库配置 */
|
||||
export interface DatabaseConfig {
|
||||
/** 数据库名称 */
|
||||
name: string;
|
||||
/** 存储模式 */
|
||||
mode?: StorageMode;
|
||||
/** 磁盘引擎（仅 mode='disk'|'hybrid' 时生效） */
|
||||
diskEngine?: DiskEngine;
|
||||
/** 版本号 */
|
||||
version?: number;
|
||||
/** 插件列表 */
|
||||
plugins?: MetonaPlugin[];
|
||||
/** 数据库就绪回调 */
|
||||
onReady?: (db: unknown) => void;
|
||||
/** 错误回调 */
|
||||
onError?: (error: Error) => void;
|
||||
/** 查询结果行数上限（默认 0，0 表示不限制） */
|
||||
maxRowsPerQuery?: number;
|
||||
/** 调试模式（启用后输出详细操作日志） */
|
||||
debug?: boolean;
|
||||
/** 多标签页同步（v0.3.2）：BroadcastChannel 广播表变更，其他标签页自动刷新 */
|
||||
multiTabSync?: boolean;
|
||||
}
|
||||
|
||||
/** 数据库默认配置 */
|
||||
export const DB_DEFAULTS: Readonly<Required<Omit<DatabaseConfig, 'plugins' | 'onReady' | 'onError'>>> = Object.freeze({
|
||||
name: 'metona-sqlark',
|
||||
mode: 'hybrid' as const,
|
||||
diskEngine: 'indexeddb' as const,
|
||||
version: 1,
|
||||
maxRowsPerQuery: 0, // 0 = 不限制
|
||||
debug: false,
|
||||
multiTabSync: false,
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Where 操作符
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Where 条件操作符 */
|
||||
export type WhereOperator = '$eq' | '$ne' | '$gt' | '$gte' | '$lt' | '$lte' | '$in' | '$nin' | '$like' | '$and' | '$or' | '$not';
|
||||
|
||||
/** 简单条件值：直接相等 */
|
||||
export type SimpleCondition = unknown;
|
||||
|
||||
/** 操作符条件 */
|
||||
export type OperatorCondition = Partial<Record<WhereOperator, unknown>>;
|
||||
|
||||
/** 字段条件：简单值 | 操作符对象 */
|
||||
export type FieldCondition = SimpleCondition | OperatorCondition;
|
||||
|
||||
/** Where 条件对象 */
|
||||
export type WhereCondition = Record<string, FieldCondition>;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 排序
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 排序方向 */
|
||||
export type SortDirection = 'asc' | 'desc';
|
||||
|
||||
/** 排序定义 */
|
||||
export interface OrderBy {
|
||||
/** 列名 */
|
||||
column: string;
|
||||
/** 排序方向 */
|
||||
direction: SortDirection;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 查询计划（引擎层使用）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 查询计划 — 由 Executor 编译 AST 后生成 */
|
||||
export interface QueryPlan {
|
||||
/** 表名 */
|
||||
table: string;
|
||||
/** 要返回的列（undefined = 全部，['*'] = 全部） */
|
||||
columns?: string[];
|
||||
/** 过滤条件 */
|
||||
where?: WhereCondition;
|
||||
/** 排序 */
|
||||
orderBy?: OrderBy[];
|
||||
/** 限制条数 */
|
||||
limit?: number;
|
||||
/** 偏移量 */
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 插件接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 钩子名称 */
|
||||
export type HookName =
|
||||
| 'beforeCreateTable' | 'afterCreateTable'
|
||||
| 'beforeDropTable' | 'afterDropTable'
|
||||
| 'beforeInsert' | 'afterInsert'
|
||||
| 'beforeUpdate' | 'afterUpdate'
|
||||
| 'beforeDelete' | 'afterDelete'
|
||||
| 'beforeQuery' | 'afterQuery'
|
||||
| 'beforeTransaction' | 'afterTransaction';
|
||||
|
||||
/** 插件定义 */
|
||||
export interface MetonaPlugin {
|
||||
/** 插件名称 */
|
||||
name: string;
|
||||
/** 插件版本 */
|
||||
version: string;
|
||||
/** 描述 */
|
||||
description?: string;
|
||||
/** 优先级，越大越先执行 */
|
||||
priority?: number;
|
||||
/** 安装 */
|
||||
install(db: unknown): void;
|
||||
/** 销毁 */
|
||||
destroy(): void;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 错误类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 数据库错误 */
|
||||
export class DatabaseError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public code: string,
|
||||
public details?: unknown,
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'DatabaseError';
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 版本
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const VERSION = '0.3.2';
|
||||
|
||||
+391
-329
@@ -1,329 +1,391 @@
|
||||
/**
|
||||
* metona-sqlark Core — 数据库主类
|
||||
* @module core
|
||||
*
|
||||
* 管理数据库生命周期、引擎调度、表操作、SQL 查询、事务和插件。
|
||||
*/
|
||||
|
||||
import type { IStorageEngine } from './engine/interface';
|
||||
import type { DatabaseConfig, ColumnDef } from './constants';
|
||||
import { DB_DEFAULTS, DatabaseError } from './constants';
|
||||
import { MemoryEngine } from './engine/memory';
|
||||
import { IndexedDBEngine } from './engine/indexeddb';
|
||||
import { OPFSEngine } from './engine/opfs';
|
||||
import { AriaEngine } from './engine/aria/index';
|
||||
import { HybridEngine } from './hybrid/index';
|
||||
import { Table } from './table/table';
|
||||
import { createSchema } from './table/schema';
|
||||
import { QueryExecutor } from './query/executor';
|
||||
import { parse } from './sql/parser';
|
||||
import { TransactionManager } from './transaction/index';
|
||||
import { PluginManager } from './plugin/index';
|
||||
import type { Statement } from './query/ast';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MetonaSqlark
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class MetonaSqlark {
|
||||
/** 数据库名称 */
|
||||
readonly name: string;
|
||||
|
||||
/** 存储模式 */
|
||||
readonly mode: string;
|
||||
|
||||
/** 版本号 */
|
||||
private _version: number;
|
||||
|
||||
/** 获取版本号 */
|
||||
get version(): number { return this._version; }
|
||||
|
||||
private engine!: IStorageEngine;
|
||||
private executor!: QueryExecutor;
|
||||
private transactionManager!: TransactionManager;
|
||||
private pluginManager: PluginManager;
|
||||
private config: DatabaseConfig;
|
||||
private ready = false;
|
||||
|
||||
private tableCache: Map<string, Table> = new Map();
|
||||
|
||||
/** 查询结果行数上限 */
|
||||
get maxRowsPerQuery(): number { return this.config.maxRowsPerQuery ?? 0; }
|
||||
|
||||
/** 调试模式 */
|
||||
get debug(): boolean { return this.config.debug ?? false; }
|
||||
|
||||
constructor(config: DatabaseConfig) {
|
||||
this.config = config;
|
||||
this.name = config.name ?? DB_DEFAULTS.name;
|
||||
this.mode = config.mode ?? DB_DEFAULTS.mode;
|
||||
this._version = config.version ?? DB_DEFAULTS.version;
|
||||
this.pluginManager = new PluginManager();
|
||||
}
|
||||
|
||||
// ---- 初始化 ----
|
||||
|
||||
/** 初始化数据库（创建引擎、打开连接） */
|
||||
async init(): Promise<void> {
|
||||
// 创建引擎
|
||||
this.engine = this.createEngine();
|
||||
|
||||
// 打开连接
|
||||
await this.engine.open(this.name, this.version);
|
||||
|
||||
// 初始化执行器和事务管理器
|
||||
this.executor = new QueryExecutor(this.engine, this.maxRowsPerQuery);
|
||||
this.transactionManager = new TransactionManager(this.engine);
|
||||
|
||||
// 注册插件
|
||||
if (this.config.plugins) {
|
||||
for (const plugin of this.config.plugins) {
|
||||
this.pluginManager.register(plugin, this);
|
||||
}
|
||||
}
|
||||
|
||||
this.ready = true;
|
||||
|
||||
// 回调
|
||||
if (this.config.onReady) {
|
||||
this.config.onReady(this);
|
||||
}
|
||||
}
|
||||
|
||||
/** 检查是否就绪 */
|
||||
isReady(): boolean {
|
||||
return this.ready;
|
||||
}
|
||||
|
||||
// ---- 表管理 ----
|
||||
|
||||
/** 创建表 */
|
||||
async defineTable(name: string, columns: Record<string, ColumnDef>): Promise<void> {
|
||||
this.ensureReady();
|
||||
const schema = createSchema(name, columns);
|
||||
|
||||
try {
|
||||
await this.pluginManager.trigger('beforeCreateTable', schema);
|
||||
await this.engine.createTable(schema);
|
||||
await this.pluginManager.trigger('afterCreateTable', schema);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
// 清除缓存
|
||||
this.tableCache.delete(name);
|
||||
}
|
||||
|
||||
/** 获取表操作对象 */
|
||||
table(name: string): Table {
|
||||
this.ensureReady();
|
||||
|
||||
let t = this.tableCache.get(name);
|
||||
if (!t) {
|
||||
t = new Table(this.engine, name, this.executor);
|
||||
this.tableCache.set(name, t);
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
/** 删除表 */
|
||||
async dropTable(name: string): Promise<void> {
|
||||
this.ensureReady();
|
||||
try {
|
||||
await this.pluginManager.trigger('beforeDropTable', name);
|
||||
await this.engine.dropTable(name);
|
||||
await this.pluginManager.trigger('afterDropTable', name);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
this.tableCache.delete(name);
|
||||
}
|
||||
|
||||
/** 获取所有表名 */
|
||||
async getTableNames(): Promise<string[]> {
|
||||
this.ensureReady();
|
||||
return this.engine.getTableNames();
|
||||
}
|
||||
|
||||
// ---- SQL 查询 ----
|
||||
|
||||
/** 执行 SQL 字符串查询 */
|
||||
async query(sql: string): Promise<unknown> {
|
||||
this.ensureReady();
|
||||
const startTime = this.debug ? Date.now() : 0;
|
||||
|
||||
await this.pluginManager.trigger('beforeQuery', sql);
|
||||
|
||||
let result: unknown;
|
||||
try {
|
||||
const stmt: Statement = parse(sql);
|
||||
result = await this.executor.execute(stmt);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
await this.pluginManager.trigger('afterQuery', sql, result);
|
||||
|
||||
if (this.debug) {
|
||||
const elapsed = Date.now() - startTime;
|
||||
const rows = Array.isArray(result) ? (result as any[]).length : 0;
|
||||
this._debug(`query [${elapsed}ms] ${rows} rows: ${sql.slice(0, 100)}`);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
/** 执行事务 */
|
||||
async transaction<T>(fn: (trx: import('./transaction/index').Transaction) => Promise<T>): Promise<T> {
|
||||
this.ensureReady();
|
||||
await this.pluginManager.trigger('beforeTransaction');
|
||||
try {
|
||||
const result = await this.transactionManager.execute(fn);
|
||||
await this.pluginManager.trigger('afterTransaction');
|
||||
return result;
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 导入导出 ----
|
||||
|
||||
/** 导出表数据为 JSON */
|
||||
async exportTable(tableName: string): Promise<Record<string, unknown>[]> {
|
||||
this.ensureReady();
|
||||
return this.engine.find(tableName, { table: tableName });
|
||||
}
|
||||
|
||||
/** 导入 JSON 数据到表 */
|
||||
async importTable(tableName: string, data: Record<string, unknown>[]): Promise<string[]> {
|
||||
this.ensureReady();
|
||||
try {
|
||||
return await this.engine.insert(tableName, data);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/** 导出整个数据库为 JSON */
|
||||
async exportAll(): Promise<Record<string, Record<string, unknown>[]>> {
|
||||
this.ensureReady();
|
||||
const result: Record<string, Record<string, unknown>[]> = {};
|
||||
const names = await this.engine.getTableNames();
|
||||
for (const name of names) {
|
||||
result[name] = await this.engine.find(name, { table: name });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// ---- 发布订阅 ----
|
||||
|
||||
private listeners: Map<string, Set<(data: unknown) => void>> = new Map();
|
||||
|
||||
/** 订阅表变更 */
|
||||
subscribe(tableName: string, callback: (event: { type: string; row?: unknown }) => void): () => void {
|
||||
const key = `change:${tableName}`;
|
||||
if (!this.listeners.has(key)) this.listeners.set(key, new Set());
|
||||
this.listeners.get(key)!.add(callback as (data: unknown) => void);
|
||||
return () => this.listeners.get(key)?.delete(callback as (data: unknown) => void);
|
||||
}
|
||||
|
||||
/** 触发变更事件 */
|
||||
emit(tableName: string, event: { type: string; row?: unknown }): void {
|
||||
const key = `change:${tableName}`;
|
||||
this.listeners.get(key)?.forEach((cb) => cb(event));
|
||||
}
|
||||
|
||||
// ---- 迁移 ----
|
||||
|
||||
private migrations: Map<number, (db: MetonaSqlark) => Promise<void>> = new Map();
|
||||
|
||||
/** 注册迁移 */
|
||||
addMigration(version: number, up: (db: MetonaSqlark) => Promise<void>): void {
|
||||
this.migrations.set(version, up);
|
||||
}
|
||||
|
||||
/** 执行迁移到指定版本 */
|
||||
async migrateTo(targetVersion: number): Promise<void> {
|
||||
this.ensureReady();
|
||||
for (const [version, up] of [...this.migrations.entries()].sort((a, b) => a[0] - b[0])) {
|
||||
if (version <= targetVersion && version > this.version) {
|
||||
await up(this);
|
||||
this._version = version;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 插件 ----
|
||||
|
||||
/** 获取插件管理器 */
|
||||
getPluginManager(): PluginManager {
|
||||
return this.pluginManager;
|
||||
}
|
||||
|
||||
/** 注册钩子 */
|
||||
on(hook: import('./constants').HookName, callback: import('./plugin/index').HookCallback): void {
|
||||
this.pluginManager.on(hook, callback);
|
||||
}
|
||||
|
||||
// ---- 生命周期 ----
|
||||
|
||||
/** 关闭数据库 */
|
||||
async close(): Promise<void> {
|
||||
this.pluginManager.destroy();
|
||||
await this.engine.close();
|
||||
this.tableCache.clear();
|
||||
this.ready = false;
|
||||
}
|
||||
|
||||
/** 获取底层引擎 */
|
||||
getEngine(): IStorageEngine {
|
||||
return this.engine;
|
||||
}
|
||||
|
||||
// ---- 内部 ----
|
||||
|
||||
private createEngine(): IStorageEngine {
|
||||
const mode = this.mode;
|
||||
const diskEngine = this.config.diskEngine ?? 'indexeddb';
|
||||
|
||||
switch (mode) {
|
||||
case 'memory':
|
||||
return new MemoryEngine();
|
||||
case 'disk':
|
||||
return diskEngine === 'opfs' ? new OPFSEngine() : new IndexedDBEngine();
|
||||
case 'aria':
|
||||
return new AriaEngine({ storageBackend: diskEngine === 'opfs' ? 'opfs' : 'indexeddb' });
|
||||
case 'hybrid':
|
||||
return new HybridEngine(diskEngine);
|
||||
default:
|
||||
throw new DatabaseError(`Unknown storage mode: ${mode}`, 'CONFIG_ERROR');
|
||||
}
|
||||
}
|
||||
|
||||
private ensureReady(): void {
|
||||
if (!this.ready) {
|
||||
throw new DatabaseError('Database not initialized. Call await db.init() first.', 'DB_NOT_READY');
|
||||
}
|
||||
}
|
||||
|
||||
/** 错误回调分发 */
|
||||
private _onError(error: Error): void {
|
||||
if (this.config.onError) {
|
||||
try { this.config.onError(error); } catch { /* 避免回调自身异常影响主流程 */ }
|
||||
}
|
||||
}
|
||||
|
||||
/** 调试日志 */
|
||||
private _debug(msg: string, ...args: unknown[]): void {
|
||||
if (this.debug) {
|
||||
console.debug(`[MetonaSqlark:${this.name}] ${msg}`, ...args);
|
||||
}
|
||||
}
|
||||
}
|
||||
/**
|
||||
* metona-sqlark Core — 数据库主类
|
||||
* @module core
|
||||
*
|
||||
* 管理数据库生命周期、引擎调度、表操作、SQL 查询、事务和插件。
|
||||
*/
|
||||
|
||||
import type { IStorageEngine } from './engine/interface';
|
||||
import type { DatabaseConfig, ColumnDef } from './constants';
|
||||
import { DB_DEFAULTS, DatabaseError } from './constants';
|
||||
import { MemoryEngine } from './engine/memory';
|
||||
import { IndexedDBEngine } from './engine/indexeddb';
|
||||
import { OPFSEngine } from './engine/opfs';
|
||||
import { AriaEngine } from './engine/aria/index';
|
||||
import { HybridEngine } from './hybrid/index';
|
||||
import { Table } from './table/table';
|
||||
import { createSchema } from './table/schema';
|
||||
import { QueryExecutor } from './query/executor';
|
||||
import { parseAll } from './sql/parser';
|
||||
import { TransactionManager } from './transaction/index';
|
||||
import { PluginManager } from './plugin/index';
|
||||
import type { Statement } from './query/ast';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MetonaSqlark
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class MetonaSqlark {
|
||||
/** 数据库名称 */
|
||||
readonly name: string;
|
||||
|
||||
/** 存储模式 */
|
||||
readonly mode: string;
|
||||
|
||||
/** 版本号 */
|
||||
private _version: number;
|
||||
|
||||
/** 获取版本号 */
|
||||
get version(): number { return this._version; }
|
||||
|
||||
private engine!: IStorageEngine;
|
||||
private executor!: QueryExecutor;
|
||||
private transactionManager!: TransactionManager;
|
||||
private pluginManager: PluginManager;
|
||||
private config: DatabaseConfig;
|
||||
private ready = false;
|
||||
|
||||
private tableCache: Map<string, Table> = new Map();
|
||||
|
||||
/** 查询结果行数上限 */
|
||||
get maxRowsPerQuery(): number { return this.config.maxRowsPerQuery ?? 0; }
|
||||
|
||||
/** 调试模式 */
|
||||
get debug(): boolean { return this.config.debug ?? false; }
|
||||
|
||||
/** 多标签页同步通道（v0.3.2） */
|
||||
private channel: BroadcastChannel | null = null;
|
||||
|
||||
constructor(config: DatabaseConfig) {
|
||||
this.config = config;
|
||||
this.name = config.name ?? DB_DEFAULTS.name;
|
||||
this.mode = config.mode ?? DB_DEFAULTS.mode;
|
||||
this._version = config.version ?? DB_DEFAULTS.version;
|
||||
this.pluginManager = new PluginManager();
|
||||
|
||||
// v0.3.2: 多标签页同步 — BroadcastChannel 广播表变更
|
||||
if (config.multiTabSync && typeof BroadcastChannel !== 'undefined') {
|
||||
this.channel = new BroadcastChannel(`metona-sqlark:${this.name}`);
|
||||
this.channel.onmessage = (event) => {
|
||||
const msg = event.data as { type?: string; table?: string } | null;
|
||||
if (!msg || msg.type !== 'change') return;
|
||||
this.emit(msg.table ?? '', { type: 'external', table: msg.table ?? '' });
|
||||
// Hybrid 引擎：从磁盘重载内存，保证读到其他标签页的最新数据
|
||||
if (this.engine instanceof HybridEngine) {
|
||||
(this.engine as HybridEngine).reloadMemoryFromDisk().catch(() => {
|
||||
// 重载失败不影响主流程（下次读可能短暂过期）
|
||||
});
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 初始化 ----
|
||||
|
||||
/** 初始化数据库（创建引擎、打开连接） */
|
||||
async init(): Promise<void> {
|
||||
// 创建引擎
|
||||
this.engine = this.createEngine();
|
||||
|
||||
// 打开连接
|
||||
await this.engine.open(this.name, this.version);
|
||||
|
||||
// 初始化执行器和事务管理器
|
||||
this.executor = new QueryExecutor(this.engine, this.maxRowsPerQuery);
|
||||
this.transactionManager = new TransactionManager(this.engine);
|
||||
|
||||
// 注册插件
|
||||
if (this.config.plugins) {
|
||||
for (const plugin of this.config.plugins) {
|
||||
this.pluginManager.register(plugin, this);
|
||||
}
|
||||
}
|
||||
|
||||
this.ready = true;
|
||||
|
||||
// 回调
|
||||
if (this.config.onReady) {
|
||||
this.config.onReady(this);
|
||||
}
|
||||
}
|
||||
|
||||
/** 检查是否就绪 */
|
||||
isReady(): boolean {
|
||||
return this.ready;
|
||||
}
|
||||
|
||||
// ---- 表管理 ----
|
||||
|
||||
/** 创建表 */
|
||||
async defineTable(name: string, columns: Record<string, ColumnDef>): Promise<void> {
|
||||
this.ensureReady();
|
||||
const schema = createSchema(name, columns);
|
||||
|
||||
try {
|
||||
await this.pluginManager.trigger('beforeCreateTable', schema);
|
||||
await this.engine.createTable(schema);
|
||||
await this.pluginManager.trigger('afterCreateTable', schema);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
// 清除缓存
|
||||
this.tableCache.delete(name);
|
||||
}
|
||||
|
||||
/** 获取表操作对象 */
|
||||
table(name: string): Table {
|
||||
this.ensureReady();
|
||||
|
||||
let t = this.tableCache.get(name);
|
||||
if (!t) {
|
||||
// v0.3.2: 表操作写入后广播变更（多标签页同步）
|
||||
t = new Table(this.engine, name, this.executor, (tableName) => this.broadcastChange(tableName));
|
||||
this.tableCache.set(name, t);
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
/** 删除表 */
|
||||
async dropTable(name: string): Promise<void> {
|
||||
this.ensureReady();
|
||||
try {
|
||||
await this.pluginManager.trigger('beforeDropTable', name);
|
||||
await this.engine.dropTable(name);
|
||||
await this.pluginManager.trigger('afterDropTable', name);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
this.tableCache.delete(name);
|
||||
}
|
||||
|
||||
/** 获取所有表名 */
|
||||
async getTableNames(): Promise<string[]> {
|
||||
this.ensureReady();
|
||||
return this.engine.getTableNames();
|
||||
}
|
||||
|
||||
// ---- SQL 查询 ----
|
||||
|
||||
/** 执行 SQL 字符串查询 */
|
||||
async query(sql: string): Promise<unknown> {
|
||||
this.ensureReady();
|
||||
const startTime = this.debug ? Date.now() : 0;
|
||||
|
||||
await this.pluginManager.trigger('beforeQuery', sql);
|
||||
|
||||
let result: unknown;
|
||||
try {
|
||||
// v0.3.0: 支持分号分隔的多语句，逐条顺序执行，返回最后一条的结果
|
||||
const statements: Statement[] = parseAll(sql);
|
||||
for (const stmt of statements) {
|
||||
result = await this.executor.execute(stmt);
|
||||
// v0.3.2: 写语句广播表变更（多标签页同步）
|
||||
const table = this.writeStatementTable(stmt);
|
||||
if (table) this.broadcastChange(table);
|
||||
}
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
await this.pluginManager.trigger('afterQuery', sql, result);
|
||||
|
||||
if (this.debug) {
|
||||
const elapsed = Date.now() - startTime;
|
||||
const rows = Array.isArray(result) ? (result as any[]).length : 0;
|
||||
this._debug(`query [${elapsed}ms] ${rows} rows: ${sql.slice(0, 100)}`);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
/** 执行事务 */
|
||||
async transaction<T>(fn: (trx: import('./transaction/index').Transaction) => Promise<T>): Promise<T> {
|
||||
this.ensureReady();
|
||||
await this.pluginManager.trigger('beforeTransaction');
|
||||
try {
|
||||
const result = await this.transactionManager.execute(fn);
|
||||
await this.pluginManager.trigger('afterTransaction');
|
||||
return result;
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 导入导出 ----
|
||||
|
||||
/** 导出表数据为 JSON */
|
||||
async exportTable(tableName: string): Promise<Record<string, unknown>[]> {
|
||||
this.ensureReady();
|
||||
return this.engine.find(tableName, { table: tableName });
|
||||
}
|
||||
|
||||
/** 导入 JSON 数据到表 */
|
||||
async importTable(tableName: string, data: Record<string, unknown>[]): Promise<string[]> {
|
||||
this.ensureReady();
|
||||
try {
|
||||
return await this.engine.insert(tableName, data);
|
||||
} catch (error) {
|
||||
this._onError(error as Error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/** 导出整个数据库为 JSON */
|
||||
async exportAll(): Promise<Record<string, Record<string, unknown>[]>> {
|
||||
this.ensureReady();
|
||||
const result: Record<string, Record<string, unknown>[]> = {};
|
||||
const names = await this.engine.getTableNames();
|
||||
for (const name of names) {
|
||||
result[name] = await this.engine.find(name, { table: name });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// ---- 发布订阅 ----
|
||||
|
||||
private listeners: Map<string, Set<(data: unknown) => void>> = new Map();
|
||||
|
||||
/** 订阅表变更 */
|
||||
subscribe(tableName: string, callback: (event: { type: string; row?: unknown; table?: string }) => void): () => void {
|
||||
const key = `change:${tableName}`;
|
||||
if (!this.listeners.has(key)) this.listeners.set(key, new Set());
|
||||
this.listeners.get(key)!.add(callback as (data: unknown) => void);
|
||||
return () => this.listeners.get(key)?.delete(callback as (data: unknown) => void);
|
||||
}
|
||||
|
||||
/** 触发变更事件 */
|
||||
emit(tableName: string, event: { type: string; row?: unknown; table?: string }): void {
|
||||
const key = `change:${tableName}`;
|
||||
this.listeners.get(key)?.forEach((cb) => cb(event));
|
||||
}
|
||||
|
||||
// ---- 多标签页同步（v0.3.2） ----
|
||||
|
||||
/** 广播表变更到其他标签页（多标签页同步） */
|
||||
broadcastChange(tableName: string): void {
|
||||
if (!this.channel) return;
|
||||
try {
|
||||
this.channel.postMessage({ type: 'change', table: tableName });
|
||||
} catch {
|
||||
// 广播失败不影响主流程
|
||||
}
|
||||
}
|
||||
|
||||
/** 写语句对应的表名（多标签页广播用） */
|
||||
private writeStatementTable(stmt: Statement): string | null {
|
||||
switch (stmt.type) {
|
||||
case 'INSERT': return stmt.into;
|
||||
case 'UPDATE': return stmt.table;
|
||||
case 'DELETE': return stmt.from;
|
||||
case 'CREATE_TABLE':
|
||||
case 'DROP_TABLE':
|
||||
case 'TRUNCATE_TABLE':
|
||||
return stmt.name;
|
||||
case 'ALTER_TABLE': return stmt.name;
|
||||
case 'CREATE_INDEX':
|
||||
case 'DROP_INDEX':
|
||||
return stmt.table;
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 迁移 ----
|
||||
|
||||
private migrations: Map<number, (db: MetonaSqlark) => Promise<void>> = new Map();
|
||||
|
||||
/** 注册迁移 */
|
||||
addMigration(version: number, up: (db: MetonaSqlark) => Promise<void>): void {
|
||||
this.migrations.set(version, up);
|
||||
}
|
||||
|
||||
/** 执行迁移到指定版本 */
|
||||
async migrateTo(targetVersion: number): Promise<void> {
|
||||
this.ensureReady();
|
||||
for (const [version, up] of [...this.migrations.entries()].sort((a, b) => a[0] - b[0])) {
|
||||
if (version <= targetVersion && version > this.version) {
|
||||
await up(this);
|
||||
this._version = version;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 插件 ----
|
||||
|
||||
/** 获取插件管理器 */
|
||||
getPluginManager(): PluginManager {
|
||||
return this.pluginManager;
|
||||
}
|
||||
|
||||
/** 注册钩子 */
|
||||
on(hook: import('./constants').HookName, callback: import('./plugin/index').HookCallback): void {
|
||||
this.pluginManager.on(hook, callback);
|
||||
}
|
||||
|
||||
// ---- 生命周期 ----
|
||||
|
||||
/** 关闭数据库 */
|
||||
async close(): Promise<void> {
|
||||
if (this.channel) {
|
||||
this.channel.close();
|
||||
this.channel = null;
|
||||
}
|
||||
this.pluginManager.destroy();
|
||||
await this.engine.close();
|
||||
this.tableCache.clear();
|
||||
this.ready = false;
|
||||
}
|
||||
|
||||
/** 获取底层引擎 */
|
||||
getEngine(): IStorageEngine {
|
||||
return this.engine;
|
||||
}
|
||||
|
||||
// ---- 内部 ----
|
||||
|
||||
private createEngine(): IStorageEngine {
|
||||
const mode = this.mode;
|
||||
const diskEngine = this.config.diskEngine ?? 'indexeddb';
|
||||
|
||||
switch (mode) {
|
||||
case 'memory':
|
||||
return new MemoryEngine();
|
||||
case 'disk':
|
||||
return diskEngine === 'opfs' ? new OPFSEngine() : new IndexedDBEngine();
|
||||
case 'aria':
|
||||
return new AriaEngine({ storageBackend: diskEngine === 'opfs' ? 'opfs' : 'indexeddb' });
|
||||
case 'hybrid':
|
||||
return new HybridEngine(diskEngine);
|
||||
default:
|
||||
throw new DatabaseError(`Unknown storage mode: ${mode}`, 'CONFIG_ERROR');
|
||||
}
|
||||
}
|
||||
|
||||
private ensureReady(): void {
|
||||
if (!this.ready) {
|
||||
throw new DatabaseError('Database not initialized. Call await db.init() first.', 'DB_NOT_READY');
|
||||
}
|
||||
}
|
||||
|
||||
/** 错误回调分发 */
|
||||
private _onError(error: Error): void {
|
||||
if (this.config.onError) {
|
||||
try { this.config.onError(error); } catch { /* 避免回调自身异常影响主流程 */ }
|
||||
}
|
||||
}
|
||||
|
||||
/** 调试日志 */
|
||||
private _debug(msg: string, ...args: unknown[]): void {
|
||||
if (this.debug) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.debug(`[MetonaSqlark:${this.name}] ${msg}`, ...args);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+206
-206
@@ -1,206 +1,206 @@
|
||||
/**
|
||||
* AriaEngine Buffer Pool Eviction — LRU 驱逐策略
|
||||
* @module engine/aria/buffer/eviction
|
||||
*/
|
||||
|
||||
import type { PageHandle } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// LRU 双向链表
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* LRU 链表管理器 — 双向链表，头部是 most recently used，尾部是 least recently used。
|
||||
*/
|
||||
export class LRUList {
|
||||
private head: PageHandle | null = null;
|
||||
private tail: PageHandle | null = null;
|
||||
private _size = 0;
|
||||
|
||||
get size(): number {
|
||||
return this._size;
|
||||
}
|
||||
|
||||
/** 将页面移到链表头部。如果是新页面则插入，已存在则移动。 */
|
||||
moveToHead(page: PageHandle): void {
|
||||
// 如果已经在头部，无需操作
|
||||
if (this.head === page) return;
|
||||
|
||||
// 检测是否在链表中
|
||||
const inList = page.prev !== null || page.next !== null || this.head === page || this.tail === page;
|
||||
|
||||
if (inList) {
|
||||
// 先从当前位置移除
|
||||
this.detach(page);
|
||||
} else {
|
||||
this._size++;
|
||||
}
|
||||
|
||||
// 插入头部
|
||||
page.prev = null;
|
||||
page.next = this.head;
|
||||
if (this.head) {
|
||||
this.head.prev = page;
|
||||
}
|
||||
this.head = page;
|
||||
if (!this.tail) {
|
||||
this.tail = page;
|
||||
}
|
||||
}
|
||||
|
||||
/** 从链表中移除页面 */
|
||||
remove(page: PageHandle): void {
|
||||
const inList = page.prev !== null || page.next !== null || this.head === page || this.tail === page;
|
||||
if (!inList) return;
|
||||
|
||||
this.detach(page);
|
||||
this._size = Math.max(0, this._size - 1);
|
||||
}
|
||||
|
||||
/** 内部：只调整指针，不修改 _size */
|
||||
private detach(page: PageHandle): void {
|
||||
if (page.prev) {
|
||||
page.prev.next = page.next;
|
||||
} else if (this.head === page) {
|
||||
this.head = page.next;
|
||||
}
|
||||
|
||||
if (page.next) {
|
||||
page.next.prev = page.prev;
|
||||
} else if (this.tail === page) {
|
||||
this.tail = page.prev;
|
||||
}
|
||||
|
||||
page.prev = null;
|
||||
page.next = null;
|
||||
}
|
||||
|
||||
/** 获取 LRU 尾部（最久未使用的页面） */
|
||||
getLRU(): PageHandle | null {
|
||||
return this.tail;
|
||||
}
|
||||
|
||||
/** 弹出 LRU 尾部 */
|
||||
popLRU(): PageHandle | null {
|
||||
const lru = this.tail;
|
||||
if (lru) {
|
||||
this.remove(lru);
|
||||
}
|
||||
return lru;
|
||||
}
|
||||
|
||||
/** 清空链表 */
|
||||
clear(): void {
|
||||
this.head = null;
|
||||
this.tail = null;
|
||||
this._size = 0;
|
||||
}
|
||||
|
||||
/** 获取所有页面（用于迭代） */
|
||||
getAllPages(): PageHandle[] {
|
||||
const pages: PageHandle[] = [];
|
||||
let current = this.head;
|
||||
while (current) {
|
||||
pages.push(current);
|
||||
current = current.next;
|
||||
}
|
||||
return pages;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Eviction 策略
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type EvictionCallback = (page: PageHandle) => Promise<void>;
|
||||
|
||||
/**
|
||||
* 驱逐管理器 — 当 Buffer Pool 满时驱逐页面。
|
||||
*/
|
||||
export class EvictionManager {
|
||||
private lru: LRUList;
|
||||
private onEvict: EvictionCallback;
|
||||
private capacity: number;
|
||||
|
||||
constructor(capacity: number, onEvict: EvictionCallback) {
|
||||
this.lru = new LRUList();
|
||||
this.capacity = capacity;
|
||||
this.onEvict = onEvict;
|
||||
}
|
||||
|
||||
/** 访问页面，更新 LRU */
|
||||
access(page: PageHandle): void {
|
||||
page.lastAccess = Date.now();
|
||||
this.lru.moveToHead(page);
|
||||
}
|
||||
|
||||
/** 添加新页面到池中 */
|
||||
add(page: PageHandle): void {
|
||||
this.access(page);
|
||||
}
|
||||
|
||||
/** 移除指定页面 */
|
||||
remove(page: PageHandle): void {
|
||||
this.lru.remove(page);
|
||||
}
|
||||
|
||||
/**
|
||||
* 驱逐页面直到池中有足够空间。
|
||||
* 只驱逐未 pin 的干净页面（dirty=false）。
|
||||
* 如果没有干净页面可驱逐，尝试刷脏页。
|
||||
*/
|
||||
async evictIfNeeded(count: number): Promise<number> {
|
||||
let evicted = 0;
|
||||
|
||||
while (this.lru.size + count > this.capacity && this.lru.size > 0) {
|
||||
// 找到可驱逐的页面
|
||||
const victim = this.findEvictionCandidate();
|
||||
if (!victim) break;
|
||||
|
||||
// 脏页先刷盘
|
||||
if (victim.dirty) {
|
||||
await this.onEvict(victim);
|
||||
victim.dirty = false;
|
||||
}
|
||||
|
||||
this.lru.remove(victim);
|
||||
evicted++;
|
||||
}
|
||||
|
||||
return evicted;
|
||||
}
|
||||
|
||||
/** 查找驱逐候选（优先干净页面，然后最久未用的脏页） */
|
||||
private findEvictionCandidate(): PageHandle | null {
|
||||
// 先从尾部找未 pin 的干净页面
|
||||
let current = this.lru.getLRU();
|
||||
while (current) {
|
||||
if (current.pins === 0 && !current.dirty) return current;
|
||||
current = current.prev;
|
||||
}
|
||||
|
||||
// 没有干净页，找未 pin 的脏页
|
||||
current = this.lru.getLRU();
|
||||
while (current) {
|
||||
if (current.pins === 0) return current;
|
||||
current = current.prev;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 获取当前大小 */
|
||||
getSize(): number {
|
||||
return this.lru.size;
|
||||
}
|
||||
|
||||
/** 获取容量 */
|
||||
getCapacity(): number {
|
||||
return this.capacity;
|
||||
}
|
||||
|
||||
/** 清空 */
|
||||
clear(): void {
|
||||
this.lru.clear();
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine Buffer Pool Eviction — LRU 驱逐策略
|
||||
* @module engine/aria/buffer/eviction
|
||||
*/
|
||||
|
||||
import type { PageHandle } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// LRU 双向链表
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* LRU 链表管理器 — 双向链表，头部是 most recently used，尾部是 least recently used。
|
||||
*/
|
||||
export class LRUList {
|
||||
private head: PageHandle | null = null;
|
||||
private tail: PageHandle | null = null;
|
||||
private _size = 0;
|
||||
|
||||
get size(): number {
|
||||
return this._size;
|
||||
}
|
||||
|
||||
/** 将页面移到链表头部。如果是新页面则插入，已存在则移动。 */
|
||||
moveToHead(page: PageHandle): void {
|
||||
// 如果已经在头部，无需操作
|
||||
if (this.head === page) return;
|
||||
|
||||
// 检测是否在链表中
|
||||
const inList = page.prev !== null || page.next !== null || this.head === page || this.tail === page;
|
||||
|
||||
if (inList) {
|
||||
// 先从当前位置移除
|
||||
this.detach(page);
|
||||
} else {
|
||||
this._size++;
|
||||
}
|
||||
|
||||
// 插入头部
|
||||
page.prev = null;
|
||||
page.next = this.head;
|
||||
if (this.head) {
|
||||
this.head.prev = page;
|
||||
}
|
||||
this.head = page;
|
||||
if (!this.tail) {
|
||||
this.tail = page;
|
||||
}
|
||||
}
|
||||
|
||||
/** 从链表中移除页面 */
|
||||
remove(page: PageHandle): void {
|
||||
const inList = page.prev !== null || page.next !== null || this.head === page || this.tail === page;
|
||||
if (!inList) return;
|
||||
|
||||
this.detach(page);
|
||||
this._size = Math.max(0, this._size - 1);
|
||||
}
|
||||
|
||||
/** 内部：只调整指针，不修改 _size */
|
||||
private detach(page: PageHandle): void {
|
||||
if (page.prev) {
|
||||
page.prev.next = page.next;
|
||||
} else if (this.head === page) {
|
||||
this.head = page.next;
|
||||
}
|
||||
|
||||
if (page.next) {
|
||||
page.next.prev = page.prev;
|
||||
} else if (this.tail === page) {
|
||||
this.tail = page.prev;
|
||||
}
|
||||
|
||||
page.prev = null;
|
||||
page.next = null;
|
||||
}
|
||||
|
||||
/** 获取 LRU 尾部（最久未使用的页面） */
|
||||
getLRU(): PageHandle | null {
|
||||
return this.tail;
|
||||
}
|
||||
|
||||
/** 弹出 LRU 尾部 */
|
||||
popLRU(): PageHandle | null {
|
||||
const lru = this.tail;
|
||||
if (lru) {
|
||||
this.remove(lru);
|
||||
}
|
||||
return lru;
|
||||
}
|
||||
|
||||
/** 清空链表 */
|
||||
clear(): void {
|
||||
this.head = null;
|
||||
this.tail = null;
|
||||
this._size = 0;
|
||||
}
|
||||
|
||||
/** 获取所有页面（用于迭代） */
|
||||
getAllPages(): PageHandle[] {
|
||||
const pages: PageHandle[] = [];
|
||||
let current = this.head;
|
||||
while (current) {
|
||||
pages.push(current);
|
||||
current = current.next;
|
||||
}
|
||||
return pages;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Eviction 策略
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type EvictionCallback = (page: PageHandle) => Promise<void>;
|
||||
|
||||
/**
|
||||
* 驱逐管理器 — 当 Buffer Pool 满时驱逐页面。
|
||||
*/
|
||||
export class EvictionManager {
|
||||
private lru: LRUList;
|
||||
private onEvict: EvictionCallback;
|
||||
private capacity: number;
|
||||
|
||||
constructor(capacity: number, onEvict: EvictionCallback) {
|
||||
this.lru = new LRUList();
|
||||
this.capacity = capacity;
|
||||
this.onEvict = onEvict;
|
||||
}
|
||||
|
||||
/** 访问页面，更新 LRU */
|
||||
access(page: PageHandle): void {
|
||||
page.lastAccess = Date.now();
|
||||
this.lru.moveToHead(page);
|
||||
}
|
||||
|
||||
/** 添加新页面到池中 */
|
||||
add(page: PageHandle): void {
|
||||
this.access(page);
|
||||
}
|
||||
|
||||
/** 移除指定页面 */
|
||||
remove(page: PageHandle): void {
|
||||
this.lru.remove(page);
|
||||
}
|
||||
|
||||
/**
|
||||
* 驱逐页面直到池中有足够空间。
|
||||
* 只驱逐未 pin 的干净页面（dirty=false）。
|
||||
* 如果没有干净页面可驱逐，尝试刷脏页。
|
||||
*/
|
||||
async evictIfNeeded(count: number): Promise<number> {
|
||||
let evicted = 0;
|
||||
|
||||
while (this.lru.size + count > this.capacity && this.lru.size > 0) {
|
||||
// 找到可驱逐的页面
|
||||
const victim = this.findEvictionCandidate();
|
||||
if (!victim) break;
|
||||
|
||||
// 脏页先刷盘
|
||||
if (victim.dirty) {
|
||||
await this.onEvict(victim);
|
||||
victim.dirty = false;
|
||||
}
|
||||
|
||||
this.lru.remove(victim);
|
||||
evicted++;
|
||||
}
|
||||
|
||||
return evicted;
|
||||
}
|
||||
|
||||
/** 查找驱逐候选（优先干净页面，然后最久未用的脏页） */
|
||||
private findEvictionCandidate(): PageHandle | null {
|
||||
// 先从尾部找未 pin 的干净页面
|
||||
let current = this.lru.getLRU();
|
||||
while (current) {
|
||||
if (current.pins === 0 && !current.dirty) return current;
|
||||
current = current.prev;
|
||||
}
|
||||
|
||||
// 没有干净页，找未 pin 的脏页
|
||||
current = this.lru.getLRU();
|
||||
while (current) {
|
||||
if (current.pins === 0) return current;
|
||||
current = current.prev;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 获取当前大小 */
|
||||
getSize(): number {
|
||||
return this.lru.size;
|
||||
}
|
||||
|
||||
/** 获取容量 */
|
||||
getCapacity(): number {
|
||||
return this.capacity;
|
||||
}
|
||||
|
||||
/** 清空 */
|
||||
clear(): void {
|
||||
this.lru.clear();
|
||||
}
|
||||
}
|
||||
|
||||
+185
-185
@@ -1,185 +1,185 @@
|
||||
/**
|
||||
* AriaEngine Buffer Pool — 页面缓存池
|
||||
* @module engine/aria/buffer/pool
|
||||
*
|
||||
* 管理固定数量页面的 LRU 缓存，减少对底层储存的访问。
|
||||
*/
|
||||
|
||||
import type { PageHandle } from '../types';
|
||||
import { PageType, DEFAULT_BUFFER_POOL_PAGES } from '../types';
|
||||
import { createPage } from '../page/format';
|
||||
import { EvictionManager } from './eviction';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Page Read / Write 回调
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PageIO {
|
||||
/** 从存储后端读取页面 */
|
||||
readPage(pageId: number): Promise<ArrayBuffer | null>;
|
||||
/** 将页面写入存储后端 */
|
||||
writePage(pageId: number, data: ArrayBuffer): Promise<void>;
|
||||
/** 分配新页面 ID */
|
||||
allocatePageId(): Promise<number>;
|
||||
/** 释放页面 ID */
|
||||
freePageId(pageId: number): Promise<void>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Buffer Pool
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class BufferPool {
|
||||
private pages: Map<number, PageHandle> = new Map();
|
||||
private eviction: EvictionManager;
|
||||
private pageIO: PageIO;
|
||||
private nextPageId = 0;
|
||||
|
||||
constructor(pageIO: PageIO, capacity: number = DEFAULT_BUFFER_POOL_PAGES) {
|
||||
this.pageIO = pageIO;
|
||||
this.eviction = new EvictionManager(capacity, async (page) => {
|
||||
if (page.dirty) {
|
||||
await this.pageIO.writePage(page.pageId, page.data);
|
||||
page.dirty = false;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 页面获取
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 获取页面（必要时从磁盘读取）。
|
||||
* 返回 pin 的页面，使用完成后必须调用 unpin()。
|
||||
*/
|
||||
async getPage(pageId: number): Promise<PageHandle | null> {
|
||||
// 已在池中
|
||||
let page = this.pages.get(pageId);
|
||||
if (page) {
|
||||
this.eviction.access(page);
|
||||
page.pins++;
|
||||
return page;
|
||||
}
|
||||
|
||||
// 需要从磁盘加载
|
||||
const buffer = await this.pageIO.readPage(pageId);
|
||||
if (!buffer) return null;
|
||||
|
||||
// 确保有空间
|
||||
await this.eviction.evictIfNeeded(1);
|
||||
|
||||
const type = new DataView(buffer).getUint8(4) as PageType;
|
||||
page = {
|
||||
pageId,
|
||||
type,
|
||||
data: buffer,
|
||||
dirty: false,
|
||||
pins: 1,
|
||||
prev: null,
|
||||
next: null,
|
||||
lastAccess: Date.now(),
|
||||
};
|
||||
|
||||
this.pages.set(pageId, page);
|
||||
this.eviction.add(page);
|
||||
return page;
|
||||
}
|
||||
|
||||
/**
|
||||
* 创建新页面。
|
||||
*/
|
||||
async newPage(type: PageType = PageType.DATA): Promise<PageHandle> {
|
||||
const pageId = await this.pageIO.allocatePageId();
|
||||
|
||||
await this.eviction.evictIfNeeded(1);
|
||||
|
||||
const page = createPage(pageId, type);
|
||||
page.pins = 1;
|
||||
this.pages.set(pageId, page);
|
||||
this.eviction.add(page);
|
||||
return page;
|
||||
}
|
||||
|
||||
/**
|
||||
* 释放页面的 pin。
|
||||
*/
|
||||
unpin(page: PageHandle): void {
|
||||
if (page.pins > 0) {
|
||||
page.pins--;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 标记页面为脏（需要写回）。
|
||||
*/
|
||||
markDirty(page: PageHandle): void {
|
||||
page.dirty = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* 将脏页面刷新到磁盘。
|
||||
*/
|
||||
async flushPage(pageId: number): Promise<void> {
|
||||
const page = this.pages.get(pageId);
|
||||
if (page && page.dirty) {
|
||||
await this.pageIO.writePage(pageId, page.data);
|
||||
page.dirty = false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 刷新所有脏页面。
|
||||
*/
|
||||
async flushAll(): Promise<void> {
|
||||
for (const [, page] of this.pages) {
|
||||
if (page.dirty) {
|
||||
await this.pageIO.writePage(page.pageId, page.data);
|
||||
page.dirty = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 从缓存中删除指定页面（不刷盘）。
|
||||
*/
|
||||
removePage(pageId: number): void {
|
||||
const page = this.pages.get(pageId);
|
||||
if (page) {
|
||||
this.eviction.remove(page);
|
||||
this.pages.delete(pageId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 清空缓存池（先刷脏页）。
|
||||
*/
|
||||
async clear(): Promise<void> {
|
||||
await this.flushAll();
|
||||
this.pages.clear();
|
||||
this.eviction.clear();
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 统计
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/** 获取当前缓存页面数 */
|
||||
getCachedPageCount(): number {
|
||||
return this.pages.size;
|
||||
}
|
||||
|
||||
/** 获取缓存容量 */
|
||||
getCapacity(): number {
|
||||
return this.eviction.getCapacity();
|
||||
}
|
||||
|
||||
/** 获取脏页面数 */
|
||||
getDirtyPageCount(): number {
|
||||
let count = 0;
|
||||
for (const [, page] of this.pages) {
|
||||
if (page.dirty) count++;
|
||||
}
|
||||
return count;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine Buffer Pool — 页面缓存池
|
||||
* @module engine/aria/buffer/pool
|
||||
*
|
||||
* 管理固定数量页面的 LRU 缓存，减少对底层储存的访问。
|
||||
*/
|
||||
|
||||
import type { PageHandle } from '../types';
|
||||
import { PageType, DEFAULT_BUFFER_POOL_PAGES } from '../types';
|
||||
import { createPage } from '../page/format';
|
||||
import { EvictionManager } from './eviction';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Page Read / Write 回调
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PageIO {
|
||||
/** 从存储后端读取页面 */
|
||||
readPage(pageId: number): Promise<ArrayBuffer | null>;
|
||||
/** 将页面写入存储后端 */
|
||||
writePage(pageId: number, data: ArrayBuffer): Promise<void>;
|
||||
/** 分配新页面 ID */
|
||||
allocatePageId(): Promise<number>;
|
||||
/** 释放页面 ID */
|
||||
freePageId(pageId: number): Promise<void>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Buffer Pool
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class BufferPool {
|
||||
private pages: Map<number, PageHandle> = new Map();
|
||||
private eviction: EvictionManager;
|
||||
private pageIO: PageIO;
|
||||
private nextPageId = 0;
|
||||
|
||||
constructor(pageIO: PageIO, capacity: number = DEFAULT_BUFFER_POOL_PAGES) {
|
||||
this.pageIO = pageIO;
|
||||
this.eviction = new EvictionManager(capacity, async (page) => {
|
||||
if (page.dirty) {
|
||||
await this.pageIO.writePage(page.pageId, page.data);
|
||||
page.dirty = false;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 页面获取
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 获取页面（必要时从磁盘读取）。
|
||||
* 返回 pin 的页面，使用完成后必须调用 unpin()。
|
||||
*/
|
||||
async getPage(pageId: number): Promise<PageHandle | null> {
|
||||
// 已在池中
|
||||
let page = this.pages.get(pageId);
|
||||
if (page) {
|
||||
this.eviction.access(page);
|
||||
page.pins++;
|
||||
return page;
|
||||
}
|
||||
|
||||
// 需要从磁盘加载
|
||||
const buffer = await this.pageIO.readPage(pageId);
|
||||
if (!buffer) return null;
|
||||
|
||||
// 确保有空间
|
||||
await this.eviction.evictIfNeeded(1);
|
||||
|
||||
const type = new DataView(buffer).getUint8(4) as PageType;
|
||||
page = {
|
||||
pageId,
|
||||
type,
|
||||
data: buffer,
|
||||
dirty: false,
|
||||
pins: 1,
|
||||
prev: null,
|
||||
next: null,
|
||||
lastAccess: Date.now(),
|
||||
};
|
||||
|
||||
this.pages.set(pageId, page);
|
||||
this.eviction.add(page);
|
||||
return page;
|
||||
}
|
||||
|
||||
/**
|
||||
* 创建新页面。
|
||||
*/
|
||||
async newPage(type: PageType = PageType.DATA): Promise<PageHandle> {
|
||||
const pageId = await this.pageIO.allocatePageId();
|
||||
|
||||
await this.eviction.evictIfNeeded(1);
|
||||
|
||||
const page = createPage(pageId, type);
|
||||
page.pins = 1;
|
||||
this.pages.set(pageId, page);
|
||||
this.eviction.add(page);
|
||||
return page;
|
||||
}
|
||||
|
||||
/**
|
||||
* 释放页面的 pin。
|
||||
*/
|
||||
unpin(page: PageHandle): void {
|
||||
if (page.pins > 0) {
|
||||
page.pins--;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 标记页面为脏（需要写回）。
|
||||
*/
|
||||
markDirty(page: PageHandle): void {
|
||||
page.dirty = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* 将脏页面刷新到磁盘。
|
||||
*/
|
||||
async flushPage(pageId: number): Promise<void> {
|
||||
const page = this.pages.get(pageId);
|
||||
if (page && page.dirty) {
|
||||
await this.pageIO.writePage(pageId, page.data);
|
||||
page.dirty = false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 刷新所有脏页面。
|
||||
*/
|
||||
async flushAll(): Promise<void> {
|
||||
for (const [, page] of this.pages) {
|
||||
if (page.dirty) {
|
||||
await this.pageIO.writePage(page.pageId, page.data);
|
||||
page.dirty = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 从缓存中删除指定页面（不刷盘）。
|
||||
*/
|
||||
removePage(pageId: number): void {
|
||||
const page = this.pages.get(pageId);
|
||||
if (page) {
|
||||
this.eviction.remove(page);
|
||||
this.pages.delete(pageId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 清空缓存池（先刷脏页）。
|
||||
*/
|
||||
async clear(): Promise<void> {
|
||||
await this.flushAll();
|
||||
this.pages.clear();
|
||||
this.eviction.clear();
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 统计
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/** 获取当前缓存页面数 */
|
||||
getCachedPageCount(): number {
|
||||
return this.pages.size;
|
||||
}
|
||||
|
||||
/** 获取缓存容量 */
|
||||
getCapacity(): number {
|
||||
return this.eviction.getCapacity();
|
||||
}
|
||||
|
||||
/** 获取脏页面数 */
|
||||
getDirtyPageCount(): number {
|
||||
let count = 0;
|
||||
for (const [, page] of this.pages) {
|
||||
if (page.dirty) count++;
|
||||
}
|
||||
return count;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,39 +2,48 @@
|
||||
* AriaEngine LZ4 Compression — 简化 LZ4 压缩/解压
|
||||
* @module engine/aria/compression/lz4
|
||||
*
|
||||
* v0.2.6: 修复往返一致性
|
||||
* - 匹配长度截断到 19 字节（matchField 上限 15 + MIN_MATCH），长匹配分段输出
|
||||
* - 组合 token 的 matchField ∈ [1,15]；matchField=0 且 lo=0 表示末尾纯字面量（无 offset）
|
||||
* - 消除"matchField=0 的组合 token 与纯字面量 token 歧义"
|
||||
*
|
||||
* Token 格式（1 字节）:
|
||||
* hi 4bit = litLen (0-15)
|
||||
* lo 4bit = matchField (0-15, 实际匹配 = field+4)
|
||||
* lo 4bit = matchField (1-15, 实际匹配 = field+4)
|
||||
*
|
||||
* 字面量-匹配序列: [token] [litLen bytes] [2B LE offset]
|
||||
* 末尾纯字面量: [token with lo=0] [litLen bytes] ← 仅在流末尾出现
|
||||
*/
|
||||
|
||||
const MIN_MATCH = 4;
|
||||
const MAX_MATCH = MIN_MATCH + 15; // 19，匹配长度上限
|
||||
|
||||
export function compressLZ4(input: Uint8Array): Uint8Array {
|
||||
if (input.byteLength < MIN_MATCH) return input;
|
||||
// 空输入直接返回（无 token 可输出）
|
||||
if (input.byteLength === 0) return input;
|
||||
|
||||
const maxOut = input.byteLength + (input.byteLength >> 8) + 32;
|
||||
// 最坏情况：纯字面量分块输出 len/15 个 token + 末尾 token
|
||||
// 上限：len + ceil(len/15) + 8（组合 token 的 offset 开销已包含在内）
|
||||
const maxOut = input.byteLength + Math.ceil(input.byteLength / 15) + 8;
|
||||
const out = new Uint8Array(maxOut);
|
||||
let si = 0, di = 0;
|
||||
let litStart = 0;
|
||||
|
||||
while (si < input.byteLength) {
|
||||
// 搜索最长 backward match
|
||||
// 搜索最长 backward match（截断到 MAX_MATCH，避免 token 字段溢出）
|
||||
let bestLen = 0, bestOff = 0;
|
||||
const searchStart = Math.max(0, si - 65535);
|
||||
for (let p = searchStart; p < si; p++) {
|
||||
let ml = 0;
|
||||
while (si + ml < input.byteLength && p + ml < si &&
|
||||
input[p + ml] === input[si + ml] && ml < 255) ml++;
|
||||
input[p + ml] === input[si + ml] && ml < MAX_MATCH) ml++;
|
||||
if (ml >= MIN_MATCH && ml > bestLen) { bestLen = ml; bestOff = si - p; }
|
||||
}
|
||||
|
||||
if (bestLen >= MIN_MATCH && (si - litStart) <= 15) {
|
||||
// 有匹配 → 输出组合 token（字面量+匹配）
|
||||
// 仅当匹配完整可编码（field 1-15）且字面量不超过 15 时才输出组合 token
|
||||
if (bestLen > MIN_MATCH && (si - litStart) <= 15) {
|
||||
const litLen = si - litStart;
|
||||
const matchField = Math.min(bestLen - MIN_MATCH, 15);
|
||||
const matchField = bestLen - MIN_MATCH; // 1..15
|
||||
out[di++] = ((litLen & 0x0F) << 4) | (matchField & 0x0F);
|
||||
for (let j = 0; j < litLen; j++) out[di++] = input[litStart + j];
|
||||
out[di++] = bestOff & 0xFF;
|
||||
@@ -42,8 +51,15 @@ export function compressLZ4(input: Uint8Array): Uint8Array {
|
||||
si += bestLen;
|
||||
litStart = si;
|
||||
} else {
|
||||
// 无匹配或字面量已满 15 → 继续累积（不单独输出，等下个匹配合并）
|
||||
// 无匹配 / 匹配长度 4（field=0 有歧义）→ 继续累积字面量
|
||||
si++;
|
||||
// 字面量达到 15 字节上限：结清为纯字面量 token（lo=0），
|
||||
// 否则后续组合 token 的字面量长度会超过 token 字段上限
|
||||
if (si - litStart >= 15) {
|
||||
out[di++] = (15 & 0x0F) << 4; // lo=0 无匹配
|
||||
for (let j = 0; j < 15; j++) out[di++] = input[litStart + j];
|
||||
litStart = si;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,7 +73,10 @@ export function compressLZ4(input: Uint8Array): Uint8Array {
|
||||
litStart += chunk;
|
||||
}
|
||||
|
||||
return di >= input.byteLength ? input : out.slice(0, di);
|
||||
// 始终输出压缩流（即使比原数据略大）。
|
||||
// 注意：不能返回原样 input —— 解压端无法区分"压缩流"与"原始数据"，
|
||||
// 原样返回会导致解压器将原始字节误解析为 token（v0.2.6 修复）
|
||||
return out.slice(0, di);
|
||||
}
|
||||
|
||||
export function decompressLZ4(input: Uint8Array, originalSize: number): Uint8Array {
|
||||
@@ -74,16 +93,17 @@ export function decompressLZ4(input: Uint8Array, originalSize: number): Uint8Arr
|
||||
out[di++] = input[si++];
|
||||
}
|
||||
|
||||
if (di >= originalSize || si >= input.byteLength) break;
|
||||
// matchField=0：纯字面量 token（无 offset 无匹配）。
|
||||
// 可能出现在流中任意位置（超长字面量分块输出），不能 break
|
||||
if (matchField === 0) continue;
|
||||
|
||||
// 非末尾 → 必有 offset + 匹配（即使 matchField==0 也复制 MIN_MATCH 字节）
|
||||
if (si + 1 < input.byteLength) {
|
||||
const offset = input[si++] | (input[si++] << 8);
|
||||
const matchLen = matchField + MIN_MATCH;
|
||||
for (let i = 0; i < matchLen && di < originalSize; i++) {
|
||||
out[di] = out[di - offset];
|
||||
di++;
|
||||
}
|
||||
// 组合 token：读取 offset + 复制匹配（可能自重叠）
|
||||
if (si + 1 >= input.byteLength) break;
|
||||
const offset = input[si++] | (input[si++] << 8);
|
||||
const matchLen = matchField + MIN_MATCH;
|
||||
for (let i = 0; i < matchLen && di < originalSize; i++) {
|
||||
out[di] = out[di - offset];
|
||||
di++;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+1195
-1080
File diff suppressed because it is too large
Load Diff
+123
-123
@@ -1,123 +1,123 @@
|
||||
/**
|
||||
* AriaEngine Bloom Filter — 快速判定 key 是否可能存在
|
||||
* @module engine/aria/index/bloom
|
||||
*
|
||||
* 使用双哈希函数 + Kirsch-Mitzenmacher 优化生成 k 个哈希值。
|
||||
*/
|
||||
|
||||
import { DEFAULT_BLOOM_BITS_PER_KEY } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// BloomFilter
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class BloomFilter {
|
||||
private bits: Uint8Array;
|
||||
private numHashes: number;
|
||||
private _inserted = 0;
|
||||
|
||||
/**
|
||||
* @param numKeys 预期插入的 key 数量
|
||||
* @param bitsPerKey 每个 key 的位数（默认 10，误报率约 1%）
|
||||
*/
|
||||
constructor(numKeys: number, bitsPerKey: number = DEFAULT_BLOOM_BITS_PER_KEY) {
|
||||
// ceil(numKeys * bitsPerKey / 8)，最少 64 位
|
||||
const numBits = Math.max(64, numKeys * bitsPerKey);
|
||||
const numBytes = Math.ceil(numBits / 8);
|
||||
this.bits = new Uint8Array(numBytes);
|
||||
|
||||
// k = bitsPerKey * ln(2) ≈ bitsPerKey * 0.69
|
||||
this.numHashes = Math.max(1, Math.floor(bitsPerKey * 0.69));
|
||||
}
|
||||
|
||||
/** 从现有数据恢复 */
|
||||
static fromData(data: Uint8Array, numHashes: number): BloomFilter {
|
||||
const bf = new BloomFilter(1); // dummy
|
||||
bf.bits = data;
|
||||
bf.numHashes = numHashes;
|
||||
return bf;
|
||||
}
|
||||
|
||||
/** 插入 key */
|
||||
insert(key: string): void {
|
||||
const hashes = this.getHashes(key);
|
||||
for (const h of hashes) {
|
||||
const byteIdx = Math.floor(h / 8);
|
||||
const bitIdx = h % 8;
|
||||
this.bits[byteIdx] |= (1 << bitIdx);
|
||||
}
|
||||
this._inserted++;
|
||||
}
|
||||
|
||||
/** 检查 key 可能存在（false positive 可能，false negative 不可能） */
|
||||
mayContain(key: string): boolean {
|
||||
const hashes = this.getHashes(key);
|
||||
for (const h of hashes) {
|
||||
const byteIdx = Math.floor(h / 8);
|
||||
const bitIdx = h % 8;
|
||||
if ((this.bits[byteIdx] & (1 << bitIdx)) === 0) {
|
||||
return false; // 确定不存在
|
||||
}
|
||||
}
|
||||
return true; // 可能存在
|
||||
}
|
||||
|
||||
/** 获取序列化数据 */
|
||||
serialize(): Uint8Array {
|
||||
return this.bits;
|
||||
}
|
||||
|
||||
/** bit 数组大小 */
|
||||
getBitSize(): number {
|
||||
return this.bits.byteLength * 8;
|
||||
}
|
||||
|
||||
/** 已插入 key 数量 */
|
||||
getInsertedCount(): number {
|
||||
return this._inserted;
|
||||
}
|
||||
|
||||
/** hash 函数数量 */
|
||||
getHashCount(): number {
|
||||
return this.numHashes;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 哈希
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private getHashes(key: string): number[] {
|
||||
const bits = this.bits.byteLength * 8;
|
||||
const h1 = this.fnv1a(key);
|
||||
const h2 = this.murmurSimple(key);
|
||||
|
||||
const hashes: number[] = [];
|
||||
for (let i = 0; i < this.numHashes; i++) {
|
||||
// Kirsch-Mitzenmacher: h_i = h1 + i * h2
|
||||
const h = Math.abs((h1 + i * h2) % bits);
|
||||
hashes.push(h);
|
||||
}
|
||||
return hashes;
|
||||
}
|
||||
|
||||
/** FNV-1a 哈希 */
|
||||
private fnv1a(str: string): number {
|
||||
let hash = 0x811c9dc5;
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
hash ^= str.charCodeAt(i);
|
||||
hash = (hash * 0x01000193) >>> 0;
|
||||
}
|
||||
return hash;
|
||||
}
|
||||
|
||||
/** 简化的 Murmur-like 哈希 */
|
||||
private murmurSimple(str: string): number {
|
||||
let hash = 0;
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
const ch = str.charCodeAt(i);
|
||||
hash = ((hash << 5) - hash + ch) | 0;
|
||||
hash = (hash ^ (hash >>> 16)) >>> 0;
|
||||
}
|
||||
return Math.abs(hash);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine Bloom Filter — 快速判定 key 是否可能存在
|
||||
* @module engine/aria/index/bloom
|
||||
*
|
||||
* 使用双哈希函数 + Kirsch-Mitzenmacher 优化生成 k 个哈希值。
|
||||
*/
|
||||
|
||||
import { DEFAULT_BLOOM_BITS_PER_KEY } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// BloomFilter
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class BloomFilter {
|
||||
private bits: Uint8Array;
|
||||
private numHashes: number;
|
||||
private _inserted = 0;
|
||||
|
||||
/**
|
||||
* @param numKeys 预期插入的 key 数量
|
||||
* @param bitsPerKey 每个 key 的位数（默认 10，误报率约 1%）
|
||||
*/
|
||||
constructor(numKeys: number, bitsPerKey: number = DEFAULT_BLOOM_BITS_PER_KEY) {
|
||||
// ceil(numKeys * bitsPerKey / 8)，最少 64 位
|
||||
const numBits = Math.max(64, numKeys * bitsPerKey);
|
||||
const numBytes = Math.ceil(numBits / 8);
|
||||
this.bits = new Uint8Array(numBytes);
|
||||
|
||||
// k = bitsPerKey * ln(2) ≈ bitsPerKey * 0.69
|
||||
this.numHashes = Math.max(1, Math.floor(bitsPerKey * 0.69));
|
||||
}
|
||||
|
||||
/** 从现有数据恢复 */
|
||||
static fromData(data: Uint8Array, numHashes: number): BloomFilter {
|
||||
const bf = new BloomFilter(1); // dummy
|
||||
bf.bits = data;
|
||||
bf.numHashes = numHashes;
|
||||
return bf;
|
||||
}
|
||||
|
||||
/** 插入 key */
|
||||
insert(key: string): void {
|
||||
const hashes = this.getHashes(key);
|
||||
for (const h of hashes) {
|
||||
const byteIdx = Math.floor(h / 8);
|
||||
const bitIdx = h % 8;
|
||||
this.bits[byteIdx] |= (1 << bitIdx);
|
||||
}
|
||||
this._inserted++;
|
||||
}
|
||||
|
||||
/** 检查 key 可能存在（false positive 可能，false negative 不可能） */
|
||||
mayContain(key: string): boolean {
|
||||
const hashes = this.getHashes(key);
|
||||
for (const h of hashes) {
|
||||
const byteIdx = Math.floor(h / 8);
|
||||
const bitIdx = h % 8;
|
||||
if ((this.bits[byteIdx] & (1 << bitIdx)) === 0) {
|
||||
return false; // 确定不存在
|
||||
}
|
||||
}
|
||||
return true; // 可能存在
|
||||
}
|
||||
|
||||
/** 获取序列化数据 */
|
||||
serialize(): Uint8Array {
|
||||
return this.bits;
|
||||
}
|
||||
|
||||
/** bit 数组大小 */
|
||||
getBitSize(): number {
|
||||
return this.bits.byteLength * 8;
|
||||
}
|
||||
|
||||
/** 已插入 key 数量 */
|
||||
getInsertedCount(): number {
|
||||
return this._inserted;
|
||||
}
|
||||
|
||||
/** hash 函数数量 */
|
||||
getHashCount(): number {
|
||||
return this.numHashes;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 哈希
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private getHashes(key: string): number[] {
|
||||
const bits = this.bits.byteLength * 8;
|
||||
const h1 = this.fnv1a(key);
|
||||
const h2 = this.murmurSimple(key);
|
||||
|
||||
const hashes: number[] = [];
|
||||
for (let i = 0; i < this.numHashes; i++) {
|
||||
// Kirsch-Mitzenmacher: h_i = h1 + i * h2
|
||||
const h = Math.abs((h1 + i * h2) % bits);
|
||||
hashes.push(h);
|
||||
}
|
||||
return hashes;
|
||||
}
|
||||
|
||||
/** FNV-1a 哈希 */
|
||||
private fnv1a(str: string): number {
|
||||
let hash = 0x811c9dc5;
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
hash ^= str.charCodeAt(i);
|
||||
hash = (hash * 0x01000193) >>> 0;
|
||||
}
|
||||
return hash;
|
||||
}
|
||||
|
||||
/** 简化的 Murmur-like 哈希 */
|
||||
private murmurSimple(str: string): number {
|
||||
let hash = 0;
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
const ch = str.charCodeAt(i);
|
||||
hash = ((hash << 5) - hash + ch) | 0;
|
||||
hash = (hash ^ (hash >>> 16)) >>> 0;
|
||||
}
|
||||
return Math.abs(hash);
|
||||
}
|
||||
}
|
||||
|
||||
+579
-466
File diff suppressed because it is too large
Load Diff
+467
-467
@@ -1,467 +1,467 @@
|
||||
/**
|
||||
* AriaEngine MemTable — 基于红黑树的内存表
|
||||
* @module engine/aria/index/memtable
|
||||
*
|
||||
* 写操作先进入 MemTable，达到阈值后冻结并 flush 成 SSTable。
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// RB-Tree Node
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
enum Color { RED, BLACK }
|
||||
|
||||
class RBNode<K, V> {
|
||||
key: K;
|
||||
value: V;
|
||||
color: Color = Color.RED;
|
||||
left: RBNode<K, V> | null = null;
|
||||
right: RBNode<K, V> | null = null;
|
||||
parent: RBNode<K, V> | null = null;
|
||||
|
||||
constructor(key: K, value: V) {
|
||||
this.key = key;
|
||||
this.value = value;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Red-Black Tree
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
class RedBlackTree<K, V> {
|
||||
private root: RBNode<K, V> | null = null;
|
||||
private _size = 0;
|
||||
|
||||
get size(): number { return this._size; }
|
||||
|
||||
// ---- 插入 ----
|
||||
insert(key: K, value: V): void {
|
||||
const node = new RBNode(key, value);
|
||||
|
||||
if (!this.root) {
|
||||
this.root = node;
|
||||
node.color = Color.BLACK;
|
||||
this._size++;
|
||||
return;
|
||||
}
|
||||
|
||||
let parent: RBNode<K, V> | null = null;
|
||||
let current: RBNode<K, V> | null = this.root;
|
||||
|
||||
while (current) {
|
||||
parent = current;
|
||||
if (key < current.key) {
|
||||
current = current.left;
|
||||
} else if (key > current.key) {
|
||||
current = current.right;
|
||||
} else {
|
||||
// 更新已存在的 key
|
||||
current.value = value;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
node.parent = parent;
|
||||
if (key < parent!.key) {
|
||||
parent!.left = node;
|
||||
} else {
|
||||
parent!.right = node;
|
||||
}
|
||||
|
||||
this._size++;
|
||||
this.fixInsert(node);
|
||||
}
|
||||
|
||||
// ---- 查找 ----
|
||||
find(key: K): V | null {
|
||||
let current = this.root;
|
||||
while (current) {
|
||||
if (key < current.key) {
|
||||
current = current.left;
|
||||
} else if (key > current.key) {
|
||||
current = current.right;
|
||||
} else {
|
||||
return current.value;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---- 删除 ----
|
||||
delete(key: K): boolean {
|
||||
// 简化实现：标记删除（实际改为找到并调整树）
|
||||
const node = this.findNode(key);
|
||||
if (!node) return false;
|
||||
|
||||
this.deleteNode(node);
|
||||
this._size--;
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---- 遍历 ----
|
||||
/** 中序遍历（有序） */
|
||||
inorder(callback: (key: K, value: V) => void): void {
|
||||
this._inorder(this.root, callback);
|
||||
}
|
||||
|
||||
/** 范围遍历 */
|
||||
rangeScan(
|
||||
startKey: K,
|
||||
endKey: K,
|
||||
callback: (key: K, value: V) => void,
|
||||
): void {
|
||||
this._rangeScan(this.root, startKey, endKey, callback);
|
||||
}
|
||||
|
||||
/** 获取所有条目 */
|
||||
getAllEntries(): [K, V][] {
|
||||
const entries: [K, V][] = [];
|
||||
this.inorder((k, v) => entries.push([k, v]));
|
||||
return entries;
|
||||
}
|
||||
|
||||
/** 清空 */
|
||||
clear(): void {
|
||||
this.root = null;
|
||||
this._size = 0;
|
||||
}
|
||||
|
||||
// ---- 内部方法 ----
|
||||
|
||||
private findNode(key: K): RBNode<K, V> | null {
|
||||
let current = this.root;
|
||||
while (current) {
|
||||
if (key < current.key) {
|
||||
current = current.left;
|
||||
} else if (key > current.key) {
|
||||
current = current.right;
|
||||
} else {
|
||||
return current;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private deleteNode(node: RBNode<K, V>): void {
|
||||
// 简化：用左子树最大或右子树最小替换
|
||||
// 完整实现较复杂，这里采用简化策略
|
||||
if (!node.left && !node.right) {
|
||||
this.transplant(node, null);
|
||||
if (node.color === Color.BLACK) this.fixDelete(null, node.parent);
|
||||
} else if (!node.left) {
|
||||
this.transplant(node, node.right);
|
||||
if (node.color === Color.BLACK) this.fixDelete(node.right, node.right!.parent);
|
||||
} else if (!node.right) {
|
||||
this.transplant(node, node.left);
|
||||
if (node.color === Color.BLACK) this.fixDelete(node.left, node.left!.parent);
|
||||
} else {
|
||||
const successor = this.minimum(node.right);
|
||||
if (successor!.parent !== node) {
|
||||
this.transplant(successor!, successor!.right);
|
||||
successor!.right = node.right;
|
||||
successor!.right!.parent = successor;
|
||||
}
|
||||
this.transplant(node, successor);
|
||||
successor!.left = node.left;
|
||||
successor!.left!.parent = successor;
|
||||
const origColor = successor!.color;
|
||||
successor!.color = node.color;
|
||||
if (origColor === Color.BLACK) this.fixDelete(successor!.right, successor!.right?.parent ?? null);
|
||||
}
|
||||
}
|
||||
|
||||
private transplant(u: RBNode<K, V> | null, v: RBNode<K, V> | null): void {
|
||||
if (!u!.parent) {
|
||||
this.root = v;
|
||||
} else if (u === u!.parent.left) {
|
||||
u!.parent.left = v;
|
||||
} else {
|
||||
u!.parent.right = v;
|
||||
}
|
||||
if (v) v.parent = u!.parent;
|
||||
}
|
||||
|
||||
private minimum(node: RBNode<K, V>): RBNode<K, V> {
|
||||
while (node.left) node = node.left;
|
||||
return node;
|
||||
}
|
||||
|
||||
private fixInsert(node: RBNode<K, V>): void {
|
||||
while (node.parent && node.parent.color === Color.RED) {
|
||||
const parent = node.parent;
|
||||
const grandparent = parent.parent;
|
||||
if (!grandparent) break;
|
||||
|
||||
if (parent === grandparent.left) {
|
||||
const uncle = grandparent.right;
|
||||
if (uncle && uncle.color === Color.RED) {
|
||||
parent.color = Color.BLACK;
|
||||
uncle.color = Color.BLACK;
|
||||
grandparent.color = Color.RED;
|
||||
node = grandparent;
|
||||
} else {
|
||||
if (node === parent.right) {
|
||||
node = parent;
|
||||
this.rotateLeft(node);
|
||||
}
|
||||
if (node.parent) node.parent.color = Color.BLACK;
|
||||
if (node.parent?.parent) node.parent.parent.color = Color.RED;
|
||||
if (node.parent?.parent) this.rotateRight(node.parent.parent);
|
||||
}
|
||||
} else {
|
||||
const uncle = grandparent.left;
|
||||
if (uncle && uncle.color === Color.RED) {
|
||||
parent.color = Color.BLACK;
|
||||
uncle.color = Color.BLACK;
|
||||
grandparent.color = Color.RED;
|
||||
node = grandparent;
|
||||
} else {
|
||||
if (node === parent.left) {
|
||||
node = parent;
|
||||
this.rotateRight(node);
|
||||
}
|
||||
if (node.parent) node.parent.color = Color.BLACK;
|
||||
if (node.parent?.parent) node.parent.parent.color = Color.RED;
|
||||
if (node.parent?.parent) this.rotateLeft(node.parent.parent);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (this.root) this.root.color = Color.BLACK;
|
||||
}
|
||||
|
||||
private fixDelete(x: RBNode<K, V> | null, parent: RBNode<K, V> | null): void {
|
||||
// 标准 RB-Tree 删除修复（修复"双黑"问题）
|
||||
let node = x;
|
||||
let nodeParent = parent;
|
||||
|
||||
while ((!node || node.color === Color.BLACK) && node !== this.root) {
|
||||
if (!nodeParent) break;
|
||||
|
||||
if (node === nodeParent.left) {
|
||||
let sibling = nodeParent.right;
|
||||
if (!sibling) break;
|
||||
|
||||
// Case 1: 兄弟是红色
|
||||
if (sibling.color === Color.RED) {
|
||||
sibling.color = Color.BLACK;
|
||||
nodeParent.color = Color.RED;
|
||||
this.rotateLeft(nodeParent);
|
||||
sibling = nodeParent.right;
|
||||
if (!sibling) break;
|
||||
}
|
||||
|
||||
// Case 2: 兄弟的两个子节点都是黑色
|
||||
const sibLeft = sibling.left;
|
||||
const sibRight = sibling.right;
|
||||
if ((!sibLeft || sibLeft.color === Color.BLACK) &&
|
||||
(!sibRight || sibRight.color === Color.BLACK)) {
|
||||
sibling.color = Color.RED;
|
||||
node = nodeParent;
|
||||
nodeParent = node.parent;
|
||||
} else {
|
||||
// Case 3: 兄弟右子黑色（左子红色）
|
||||
if (!sibRight || sibRight.color === Color.BLACK) {
|
||||
if (sibLeft) sibLeft.color = Color.BLACK;
|
||||
sibling.color = Color.RED;
|
||||
this.rotateRight(sibling);
|
||||
sibling = nodeParent.right;
|
||||
if (!sibling) break;
|
||||
}
|
||||
// Case 4: 兄弟右子红色
|
||||
sibling.color = nodeParent.color;
|
||||
nodeParent.color = Color.BLACK;
|
||||
if (sibling.right) sibling.right.color = Color.BLACK;
|
||||
this.rotateLeft(nodeParent);
|
||||
node = this.root;
|
||||
}
|
||||
} else {
|
||||
// 镜像：node 是父节点的右子
|
||||
let sibling = nodeParent.left;
|
||||
if (!sibling) break;
|
||||
|
||||
if (sibling.color === Color.RED) {
|
||||
sibling.color = Color.BLACK;
|
||||
nodeParent.color = Color.RED;
|
||||
this.rotateRight(nodeParent);
|
||||
sibling = nodeParent.left;
|
||||
if (!sibling) break;
|
||||
}
|
||||
|
||||
const sibLeft = sibling.left;
|
||||
const sibRight = sibling.right;
|
||||
if ((!sibLeft || sibLeft.color === Color.BLACK) &&
|
||||
(!sibRight || sibRight.color === Color.BLACK)) {
|
||||
sibling.color = Color.RED;
|
||||
node = nodeParent;
|
||||
nodeParent = node.parent;
|
||||
} else {
|
||||
if (!sibLeft || sibLeft.color === Color.BLACK) {
|
||||
if (sibRight) sibRight.color = Color.BLACK;
|
||||
sibling.color = Color.RED;
|
||||
this.rotateLeft(sibling);
|
||||
sibling = nodeParent.left;
|
||||
if (!sibling) break;
|
||||
}
|
||||
sibling.color = nodeParent.color;
|
||||
nodeParent.color = Color.BLACK;
|
||||
if (sibling.left) sibling.left.color = Color.BLACK;
|
||||
this.rotateRight(nodeParent);
|
||||
node = this.root;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (node) node.color = Color.BLACK;
|
||||
}
|
||||
|
||||
private rotateLeft(x: RBNode<K, V>): void {
|
||||
const y = x.right;
|
||||
if (!y) return;
|
||||
x.right = y.left;
|
||||
if (y.left) y.left.parent = x;
|
||||
y.parent = x.parent;
|
||||
if (!x.parent) {
|
||||
this.root = y;
|
||||
} else if (x === x.parent.left) {
|
||||
x.parent.left = y;
|
||||
} else {
|
||||
x.parent.right = y;
|
||||
}
|
||||
y.left = x;
|
||||
x.parent = y;
|
||||
}
|
||||
|
||||
private rotateRight(x: RBNode<K, V>): void {
|
||||
const y = x.left;
|
||||
if (!y) return;
|
||||
x.left = y.right;
|
||||
if (y.right) y.right.parent = x;
|
||||
y.parent = x.parent;
|
||||
if (!x.parent) {
|
||||
this.root = y;
|
||||
} else if (x === x.parent.right) {
|
||||
x.parent.right = y;
|
||||
} else {
|
||||
x.parent.left = y;
|
||||
}
|
||||
y.right = x;
|
||||
x.parent = y;
|
||||
}
|
||||
|
||||
private _inorder(node: RBNode<K, V> | null, cb: (k: K, v: V) => void): void {
|
||||
if (!node) return;
|
||||
this._inorder(node.left, cb);
|
||||
cb(node.key, node.value);
|
||||
this._inorder(node.right, cb);
|
||||
}
|
||||
|
||||
private _rangeScan(
|
||||
node: RBNode<K, V> | null,
|
||||
start: K,
|
||||
end: K,
|
||||
cb: (k: K, v: V) => void,
|
||||
): void {
|
||||
if (!node) return;
|
||||
if (node.key > start) this._rangeScan(node.left, start, end, cb);
|
||||
if (node.key >= start && node.key <= end) cb(node.key, node.value);
|
||||
if (node.key < end) this._rangeScan(node.right, start, end, cb);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MemTable
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class MemTable {
|
||||
private tree: RedBlackTree<string, Record<string, unknown>>;
|
||||
private _estimatedSize = 0;
|
||||
private maxSize: number;
|
||||
|
||||
constructor(maxSize: number = 4 * 1024 * 1024) {
|
||||
this.tree = new RedBlackTree();
|
||||
this.maxSize = maxSize;
|
||||
}
|
||||
|
||||
/** 插入或更新 */
|
||||
put(key: string, value: Record<string, unknown>): void {
|
||||
const oldSize = this.estimateEntrySize(key, this.tree.find(key));
|
||||
const newSize = this.estimateEntrySize(key, value);
|
||||
this.tree.insert(key, value);
|
||||
this._estimatedSize += newSize - oldSize;
|
||||
}
|
||||
|
||||
/** 获取 */
|
||||
get(key: string): Record<string, unknown> | null {
|
||||
return this.tree.find(key);
|
||||
}
|
||||
|
||||
/** 删除 */
|
||||
delete(key: string): boolean {
|
||||
const oldVal = this.tree.find(key);
|
||||
if (oldVal) {
|
||||
this._estimatedSize -= this.estimateEntrySize(key, oldVal);
|
||||
}
|
||||
return this.tree.delete(key);
|
||||
}
|
||||
|
||||
/** 是否应刷盘 */
|
||||
shouldFlush(): boolean {
|
||||
return this._estimatedSize >= this.maxSize;
|
||||
}
|
||||
|
||||
/** 获取所有有序条目 */
|
||||
getAllEntries(): [string, Record<string, unknown>][] {
|
||||
return this.tree.getAllEntries();
|
||||
}
|
||||
|
||||
/** 范围扫描 */
|
||||
rangeScan(
|
||||
startKey: string,
|
||||
endKey: string,
|
||||
): [string, Record<string, unknown>][] {
|
||||
const entries: [string, Record<string, unknown>][] = [];
|
||||
this.tree.rangeScan(startKey, endKey, (k, v) => entries.push([k, v]));
|
||||
return entries;
|
||||
}
|
||||
|
||||
/** 条目数 */
|
||||
getEntryCount(): number {
|
||||
return this.tree.size;
|
||||
}
|
||||
|
||||
/** 估计大小（字节） */
|
||||
getEstimatedSize(): number {
|
||||
return this._estimatedSize;
|
||||
}
|
||||
|
||||
/** 清空 */
|
||||
clear(): void {
|
||||
this.tree.clear();
|
||||
this._estimatedSize = 0;
|
||||
}
|
||||
|
||||
/** 检查 key 是否存在 */
|
||||
contains(key: string): boolean {
|
||||
return this.tree.find(key) !== null;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 内部
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private estimateEntrySize(key: string, value: Record<string, unknown> | null): number {
|
||||
if (!value) return 0;
|
||||
let size = key.length * 2; // UTF-16
|
||||
for (const entry of Object.entries(value)) {
|
||||
size += entry[0].length * 2;
|
||||
const v = entry[1];
|
||||
if (typeof v === 'string') size += v.length * 2;
|
||||
else if (typeof v === 'number') size += 8;
|
||||
else if (typeof v === 'boolean') size += 1;
|
||||
else if (v === null || v === undefined) size += 1;
|
||||
else size += 16; // rough estimate
|
||||
}
|
||||
return size;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine MemTable — 基于红黑树的内存表
|
||||
* @module engine/aria/index/memtable
|
||||
*
|
||||
* 写操作先进入 MemTable，达到阈值后冻结并 flush 成 SSTable。
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// RB-Tree Node
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
enum Color { RED, BLACK }
|
||||
|
||||
class RBNode<K, V> {
|
||||
key: K;
|
||||
value: V;
|
||||
color: Color = Color.RED;
|
||||
left: RBNode<K, V> | null = null;
|
||||
right: RBNode<K, V> | null = null;
|
||||
parent: RBNode<K, V> | null = null;
|
||||
|
||||
constructor(key: K, value: V) {
|
||||
this.key = key;
|
||||
this.value = value;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Red-Black Tree
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
class RedBlackTree<K, V> {
|
||||
private root: RBNode<K, V> | null = null;
|
||||
private _size = 0;
|
||||
|
||||
get size(): number { return this._size; }
|
||||
|
||||
// ---- 插入 ----
|
||||
insert(key: K, value: V): void {
|
||||
const node = new RBNode(key, value);
|
||||
|
||||
if (!this.root) {
|
||||
this.root = node;
|
||||
node.color = Color.BLACK;
|
||||
this._size++;
|
||||
return;
|
||||
}
|
||||
|
||||
let parent: RBNode<K, V> | null = null;
|
||||
let current: RBNode<K, V> | null = this.root;
|
||||
|
||||
while (current) {
|
||||
parent = current;
|
||||
if (key < current.key) {
|
||||
current = current.left;
|
||||
} else if (key > current.key) {
|
||||
current = current.right;
|
||||
} else {
|
||||
// 更新已存在的 key
|
||||
current.value = value;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
node.parent = parent;
|
||||
if (key < parent!.key) {
|
||||
parent!.left = node;
|
||||
} else {
|
||||
parent!.right = node;
|
||||
}
|
||||
|
||||
this._size++;
|
||||
this.fixInsert(node);
|
||||
}
|
||||
|
||||
// ---- 查找 ----
|
||||
find(key: K): V | null {
|
||||
let current = this.root;
|
||||
while (current) {
|
||||
if (key < current.key) {
|
||||
current = current.left;
|
||||
} else if (key > current.key) {
|
||||
current = current.right;
|
||||
} else {
|
||||
return current.value;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---- 删除 ----
|
||||
delete(key: K): boolean {
|
||||
// 简化实现：标记删除（实际改为找到并调整树）
|
||||
const node = this.findNode(key);
|
||||
if (!node) return false;
|
||||
|
||||
this.deleteNode(node);
|
||||
this._size--;
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---- 遍历 ----
|
||||
/** 中序遍历（有序） */
|
||||
inorder(callback: (key: K, value: V) => void): void {
|
||||
this._inorder(this.root, callback);
|
||||
}
|
||||
|
||||
/** 范围遍历 */
|
||||
rangeScan(
|
||||
startKey: K,
|
||||
endKey: K,
|
||||
callback: (key: K, value: V) => void,
|
||||
): void {
|
||||
this._rangeScan(this.root, startKey, endKey, callback);
|
||||
}
|
||||
|
||||
/** 获取所有条目 */
|
||||
getAllEntries(): [K, V][] {
|
||||
const entries: [K, V][] = [];
|
||||
this.inorder((k, v) => entries.push([k, v]));
|
||||
return entries;
|
||||
}
|
||||
|
||||
/** 清空 */
|
||||
clear(): void {
|
||||
this.root = null;
|
||||
this._size = 0;
|
||||
}
|
||||
|
||||
// ---- 内部方法 ----
|
||||
|
||||
private findNode(key: K): RBNode<K, V> | null {
|
||||
let current = this.root;
|
||||
while (current) {
|
||||
if (key < current.key) {
|
||||
current = current.left;
|
||||
} else if (key > current.key) {
|
||||
current = current.right;
|
||||
} else {
|
||||
return current;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private deleteNode(node: RBNode<K, V>): void {
|
||||
// 简化：用左子树最大或右子树最小替换
|
||||
// 完整实现较复杂，这里采用简化策略
|
||||
if (!node.left && !node.right) {
|
||||
this.transplant(node, null);
|
||||
if (node.color === Color.BLACK) this.fixDelete(null, node.parent);
|
||||
} else if (!node.left) {
|
||||
this.transplant(node, node.right);
|
||||
if (node.color === Color.BLACK) this.fixDelete(node.right, node.right!.parent);
|
||||
} else if (!node.right) {
|
||||
this.transplant(node, node.left);
|
||||
if (node.color === Color.BLACK) this.fixDelete(node.left, node.left!.parent);
|
||||
} else {
|
||||
const successor = this.minimum(node.right);
|
||||
if (successor!.parent !== node) {
|
||||
this.transplant(successor!, successor!.right);
|
||||
successor!.right = node.right;
|
||||
successor!.right!.parent = successor;
|
||||
}
|
||||
this.transplant(node, successor);
|
||||
successor!.left = node.left;
|
||||
successor!.left!.parent = successor;
|
||||
const origColor = successor!.color;
|
||||
successor!.color = node.color;
|
||||
if (origColor === Color.BLACK) this.fixDelete(successor!.right, successor!.right?.parent ?? null);
|
||||
}
|
||||
}
|
||||
|
||||
private transplant(u: RBNode<K, V> | null, v: RBNode<K, V> | null): void {
|
||||
if (!u!.parent) {
|
||||
this.root = v;
|
||||
} else if (u === u!.parent.left) {
|
||||
u!.parent.left = v;
|
||||
} else {
|
||||
u!.parent.right = v;
|
||||
}
|
||||
if (v) v.parent = u!.parent;
|
||||
}
|
||||
|
||||
private minimum(node: RBNode<K, V>): RBNode<K, V> {
|
||||
while (node.left) node = node.left;
|
||||
return node;
|
||||
}
|
||||
|
||||
private fixInsert(node: RBNode<K, V>): void {
|
||||
while (node.parent && node.parent.color === Color.RED) {
|
||||
const parent = node.parent;
|
||||
const grandparent = parent.parent;
|
||||
if (!grandparent) break;
|
||||
|
||||
if (parent === grandparent.left) {
|
||||
const uncle = grandparent.right;
|
||||
if (uncle && uncle.color === Color.RED) {
|
||||
parent.color = Color.BLACK;
|
||||
uncle.color = Color.BLACK;
|
||||
grandparent.color = Color.RED;
|
||||
node = grandparent;
|
||||
} else {
|
||||
if (node === parent.right) {
|
||||
node = parent;
|
||||
this.rotateLeft(node);
|
||||
}
|
||||
if (node.parent) node.parent.color = Color.BLACK;
|
||||
if (node.parent?.parent) node.parent.parent.color = Color.RED;
|
||||
if (node.parent?.parent) this.rotateRight(node.parent.parent);
|
||||
}
|
||||
} else {
|
||||
const uncle = grandparent.left;
|
||||
if (uncle && uncle.color === Color.RED) {
|
||||
parent.color = Color.BLACK;
|
||||
uncle.color = Color.BLACK;
|
||||
grandparent.color = Color.RED;
|
||||
node = grandparent;
|
||||
} else {
|
||||
if (node === parent.left) {
|
||||
node = parent;
|
||||
this.rotateRight(node);
|
||||
}
|
||||
if (node.parent) node.parent.color = Color.BLACK;
|
||||
if (node.parent?.parent) node.parent.parent.color = Color.RED;
|
||||
if (node.parent?.parent) this.rotateLeft(node.parent.parent);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (this.root) this.root.color = Color.BLACK;
|
||||
}
|
||||
|
||||
private fixDelete(x: RBNode<K, V> | null, parent: RBNode<K, V> | null): void {
|
||||
// 标准 RB-Tree 删除修复（修复"双黑"问题）
|
||||
let node = x;
|
||||
let nodeParent = parent;
|
||||
|
||||
while ((!node || node.color === Color.BLACK) && node !== this.root) {
|
||||
if (!nodeParent) break;
|
||||
|
||||
if (node === nodeParent.left) {
|
||||
let sibling = nodeParent.right;
|
||||
if (!sibling) break;
|
||||
|
||||
// Case 1: 兄弟是红色
|
||||
if (sibling.color === Color.RED) {
|
||||
sibling.color = Color.BLACK;
|
||||
nodeParent.color = Color.RED;
|
||||
this.rotateLeft(nodeParent);
|
||||
sibling = nodeParent.right;
|
||||
if (!sibling) break;
|
||||
}
|
||||
|
||||
// Case 2: 兄弟的两个子节点都是黑色
|
||||
const sibLeft = sibling.left;
|
||||
const sibRight = sibling.right;
|
||||
if ((!sibLeft || sibLeft.color === Color.BLACK) &&
|
||||
(!sibRight || sibRight.color === Color.BLACK)) {
|
||||
sibling.color = Color.RED;
|
||||
node = nodeParent;
|
||||
nodeParent = node.parent;
|
||||
} else {
|
||||
// Case 3: 兄弟右子黑色（左子红色）
|
||||
if (!sibRight || sibRight.color === Color.BLACK) {
|
||||
if (sibLeft) sibLeft.color = Color.BLACK;
|
||||
sibling.color = Color.RED;
|
||||
this.rotateRight(sibling);
|
||||
sibling = nodeParent.right;
|
||||
if (!sibling) break;
|
||||
}
|
||||
// Case 4: 兄弟右子红色
|
||||
sibling.color = nodeParent.color;
|
||||
nodeParent.color = Color.BLACK;
|
||||
if (sibling.right) sibling.right.color = Color.BLACK;
|
||||
this.rotateLeft(nodeParent);
|
||||
node = this.root;
|
||||
}
|
||||
} else {
|
||||
// 镜像：node 是父节点的右子
|
||||
let sibling = nodeParent.left;
|
||||
if (!sibling) break;
|
||||
|
||||
if (sibling.color === Color.RED) {
|
||||
sibling.color = Color.BLACK;
|
||||
nodeParent.color = Color.RED;
|
||||
this.rotateRight(nodeParent);
|
||||
sibling = nodeParent.left;
|
||||
if (!sibling) break;
|
||||
}
|
||||
|
||||
const sibLeft = sibling.left;
|
||||
const sibRight = sibling.right;
|
||||
if ((!sibLeft || sibLeft.color === Color.BLACK) &&
|
||||
(!sibRight || sibRight.color === Color.BLACK)) {
|
||||
sibling.color = Color.RED;
|
||||
node = nodeParent;
|
||||
nodeParent = node.parent;
|
||||
} else {
|
||||
if (!sibLeft || sibLeft.color === Color.BLACK) {
|
||||
if (sibRight) sibRight.color = Color.BLACK;
|
||||
sibling.color = Color.RED;
|
||||
this.rotateLeft(sibling);
|
||||
sibling = nodeParent.left;
|
||||
if (!sibling) break;
|
||||
}
|
||||
sibling.color = nodeParent.color;
|
||||
nodeParent.color = Color.BLACK;
|
||||
if (sibling.left) sibling.left.color = Color.BLACK;
|
||||
this.rotateRight(nodeParent);
|
||||
node = this.root;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (node) node.color = Color.BLACK;
|
||||
}
|
||||
|
||||
private rotateLeft(x: RBNode<K, V>): void {
|
||||
const y = x.right;
|
||||
if (!y) return;
|
||||
x.right = y.left;
|
||||
if (y.left) y.left.parent = x;
|
||||
y.parent = x.parent;
|
||||
if (!x.parent) {
|
||||
this.root = y;
|
||||
} else if (x === x.parent.left) {
|
||||
x.parent.left = y;
|
||||
} else {
|
||||
x.parent.right = y;
|
||||
}
|
||||
y.left = x;
|
||||
x.parent = y;
|
||||
}
|
||||
|
||||
private rotateRight(x: RBNode<K, V>): void {
|
||||
const y = x.left;
|
||||
if (!y) return;
|
||||
x.left = y.right;
|
||||
if (y.right) y.right.parent = x;
|
||||
y.parent = x.parent;
|
||||
if (!x.parent) {
|
||||
this.root = y;
|
||||
} else if (x === x.parent.right) {
|
||||
x.parent.right = y;
|
||||
} else {
|
||||
x.parent.left = y;
|
||||
}
|
||||
y.right = x;
|
||||
x.parent = y;
|
||||
}
|
||||
|
||||
private _inorder(node: RBNode<K, V> | null, cb: (k: K, v: V) => void): void {
|
||||
if (!node) return;
|
||||
this._inorder(node.left, cb);
|
||||
cb(node.key, node.value);
|
||||
this._inorder(node.right, cb);
|
||||
}
|
||||
|
||||
private _rangeScan(
|
||||
node: RBNode<K, V> | null,
|
||||
start: K,
|
||||
end: K,
|
||||
cb: (k: K, v: V) => void,
|
||||
): void {
|
||||
if (!node) return;
|
||||
if (node.key > start) this._rangeScan(node.left, start, end, cb);
|
||||
if (node.key >= start && node.key <= end) cb(node.key, node.value);
|
||||
if (node.key < end) this._rangeScan(node.right, start, end, cb);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MemTable
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class MemTable {
|
||||
private tree: RedBlackTree<string, Record<string, unknown>>;
|
||||
private _estimatedSize = 0;
|
||||
private maxSize: number;
|
||||
|
||||
constructor(maxSize: number = 4 * 1024 * 1024) {
|
||||
this.tree = new RedBlackTree();
|
||||
this.maxSize = maxSize;
|
||||
}
|
||||
|
||||
/** 插入或更新 */
|
||||
put(key: string, value: Record<string, unknown>): void {
|
||||
const oldSize = this.estimateEntrySize(key, this.tree.find(key));
|
||||
const newSize = this.estimateEntrySize(key, value);
|
||||
this.tree.insert(key, value);
|
||||
this._estimatedSize += newSize - oldSize;
|
||||
}
|
||||
|
||||
/** 获取 */
|
||||
get(key: string): Record<string, unknown> | null {
|
||||
return this.tree.find(key);
|
||||
}
|
||||
|
||||
/** 删除 */
|
||||
delete(key: string): boolean {
|
||||
const oldVal = this.tree.find(key);
|
||||
if (oldVal) {
|
||||
this._estimatedSize -= this.estimateEntrySize(key, oldVal);
|
||||
}
|
||||
return this.tree.delete(key);
|
||||
}
|
||||
|
||||
/** 是否应刷盘 */
|
||||
shouldFlush(): boolean {
|
||||
return this._estimatedSize >= this.maxSize;
|
||||
}
|
||||
|
||||
/** 获取所有有序条目 */
|
||||
getAllEntries(): [string, Record<string, unknown>][] {
|
||||
return this.tree.getAllEntries();
|
||||
}
|
||||
|
||||
/** 范围扫描 */
|
||||
rangeScan(
|
||||
startKey: string,
|
||||
endKey: string,
|
||||
): [string, Record<string, unknown>][] {
|
||||
const entries: [string, Record<string, unknown>][] = [];
|
||||
this.tree.rangeScan(startKey, endKey, (k, v) => entries.push([k, v]));
|
||||
return entries;
|
||||
}
|
||||
|
||||
/** 条目数 */
|
||||
getEntryCount(): number {
|
||||
return this.tree.size;
|
||||
}
|
||||
|
||||
/** 估计大小（字节） */
|
||||
getEstimatedSize(): number {
|
||||
return this._estimatedSize;
|
||||
}
|
||||
|
||||
/** 清空 */
|
||||
clear(): void {
|
||||
this.tree.clear();
|
||||
this._estimatedSize = 0;
|
||||
}
|
||||
|
||||
/** 检查 key 是否存在 */
|
||||
contains(key: string): boolean {
|
||||
return this.tree.find(key) !== null;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 内部
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private estimateEntrySize(key: string, value: Record<string, unknown> | null): number {
|
||||
if (!value) return 0;
|
||||
let size = key.length * 2; // UTF-16
|
||||
for (const entry of Object.entries(value)) {
|
||||
size += entry[0].length * 2;
|
||||
const v = entry[1];
|
||||
if (typeof v === 'string') size += v.length * 2;
|
||||
else if (typeof v === 'number') size += 8;
|
||||
else if (typeof v === 'boolean') size += 1;
|
||||
else if (v === null || v === undefined) size += 1;
|
||||
else size += 16; // rough estimate
|
||||
}
|
||||
return size;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,189 +1,192 @@
|
||||
/**
|
||||
* AriaEngine Merge Iterator — 多路归并迭代器
|
||||
* @module engine/aria/index/merge_iterator
|
||||
*
|
||||
* 对多个有序 SSTable 或 MemTable 的结果进行归并去重（保留最新值）。
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MergeIterator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface EntrySource {
|
||||
/** 获取下一个条目，无更多时返回 null */
|
||||
next(): [string, Record<string, unknown>] | null;
|
||||
/** 重置迭代器 */
|
||||
reset(): void;
|
||||
}
|
||||
|
||||
/** 数组数据源的迭代器 */
|
||||
export class ArrayEntrySource implements EntrySource {
|
||||
private entries: [string, Record<string, unknown>][];
|
||||
private index = 0;
|
||||
|
||||
constructor(entries: [string, Record<string, unknown>][]) {
|
||||
this.entries = entries;
|
||||
}
|
||||
|
||||
next(): [string, Record<string, unknown>] | null {
|
||||
if (this.index >= this.entries.length) return null;
|
||||
return this.entries[this.index++];
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.index = 0;
|
||||
}
|
||||
}
|
||||
|
||||
/** 回调数据源的迭代器 */
|
||||
export class CallbackEntrySource implements EntrySource {
|
||||
private items: [string, Record<string, unknown>][] = [];
|
||||
private index = 0;
|
||||
private consumed = false;
|
||||
|
||||
/**
|
||||
* @param producer 产生所有条目的回调
|
||||
*/
|
||||
constructor(producer: (cb: (key: string, value: Record<string, unknown>) => void) => void) {
|
||||
producer((key, value) => this.items.push([key, value]));
|
||||
}
|
||||
|
||||
next(): [string, Record<string, unknown>] | null {
|
||||
if (this.index >= this.items.length) return null;
|
||||
return this.items[this.index++];
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.index = 0;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Heap 节点（用于多路归并）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface HeapNode {
|
||||
key: string;
|
||||
value: Record<string, unknown>;
|
||||
sourceIndex: number;
|
||||
}
|
||||
|
||||
/** 最小堆 */
|
||||
class MinHeap {
|
||||
private heap: HeapNode[] = [];
|
||||
|
||||
push(node: HeapNode): void {
|
||||
this.heap.push(node);
|
||||
this.bubbleUp(this.heap.length - 1);
|
||||
}
|
||||
|
||||
pop(): HeapNode | null {
|
||||
if (this.heap.length === 0) return null;
|
||||
if (this.heap.length === 1) return this.heap.pop()!;
|
||||
|
||||
const result = this.heap[0];
|
||||
this.heap[0] = this.heap.pop()!;
|
||||
this.bubbleDown(0);
|
||||
return result;
|
||||
}
|
||||
|
||||
peek(): HeapNode | null {
|
||||
return this.heap.length > 0 ? this.heap[0] : null;
|
||||
}
|
||||
|
||||
get size(): number {
|
||||
return this.heap.length;
|
||||
}
|
||||
|
||||
private bubbleUp(idx: number): void {
|
||||
while (idx > 0) {
|
||||
const parent = Math.floor((idx - 1) / 2);
|
||||
if (this.heap[idx].key >= this.heap[parent].key) break;
|
||||
[this.heap[idx], this.heap[parent]] = [this.heap[parent], this.heap[idx]];
|
||||
idx = parent;
|
||||
}
|
||||
}
|
||||
|
||||
private bubbleDown(idx: number): void {
|
||||
const n = this.heap.length;
|
||||
while (true) {
|
||||
let smallest = idx;
|
||||
const left = 2 * idx + 1;
|
||||
const right = 2 * idx + 2;
|
||||
|
||||
if (left < n && this.heap[left].key < this.heap[smallest].key) smallest = left;
|
||||
if (right < n && this.heap[right].key < this.heap[smallest].key) smallest = right;
|
||||
|
||||
if (smallest === idx) break;
|
||||
|
||||
[this.heap[idx], this.heap[smallest]] = [this.heap[smallest], this.heap[idx]];
|
||||
idx = smallest;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MergeIterator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 对多个有序数据源进行归并，重复 key 保留最新（后出现的）。
|
||||
* 数据源按新鲜度排序：越新的数据源在下标越小（如 MemTable 在 SSTable 之前）。
|
||||
*/
|
||||
export class MergeIterator {
|
||||
private sources: EntrySource[];
|
||||
private heap: MinHeap;
|
||||
|
||||
constructor() {
|
||||
this.sources = [];
|
||||
this.heap = new MinHeap();
|
||||
}
|
||||
|
||||
/** 添加数据源 */
|
||||
addSource(source: EntrySource): void {
|
||||
this.sources.push(source);
|
||||
this.seedFromSource(this.sources.length - 1);
|
||||
}
|
||||
|
||||
/** 获取下一个归并后的条目 */
|
||||
next(): [string, Record<string, unknown>] | null {
|
||||
if (this.heap.size === 0) return null;
|
||||
|
||||
const node = this.heap.pop()!;
|
||||
const key = node.key;
|
||||
const value = node.value;
|
||||
|
||||
// 刷新此来源的下一个值
|
||||
this.seedFromSource(node.sourceIndex);
|
||||
|
||||
// 跳过重复 key：取最新的（堆顶的即是最新的，因为来源下标越小越新）
|
||||
while (this.heap.peek() && this.heap.peek()!.key === key) {
|
||||
const dup = this.heap.pop()!;
|
||||
this.seedFromSource(dup.sourceIndex);
|
||||
}
|
||||
|
||||
return [key, value];
|
||||
}
|
||||
|
||||
/** 耗尽管道，返回所有归并结果 */
|
||||
drain(): [string, Record<string, unknown>][] {
|
||||
const result: [string, Record<string, unknown>][] = [];
|
||||
let entry = this.next();
|
||||
while (entry) {
|
||||
result.push(entry);
|
||||
entry = this.next();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private seedFromSource(sourceIndex: number): void {
|
||||
const entry = this.sources[sourceIndex].next();
|
||||
if (entry) {
|
||||
this.heap.push({
|
||||
key: entry[0],
|
||||
value: entry[1],
|
||||
sourceIndex,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine Merge Iterator — 多路归并迭代器
|
||||
* @module engine/aria/index/merge_iterator
|
||||
*
|
||||
* 对多个有序 SSTable 或 MemTable 的结果进行归并去重（保留最新值）。
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MergeIterator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface EntrySource {
|
||||
/** 获取下一个条目，无更多时返回 null */
|
||||
next(): [string, Record<string, unknown>] | null;
|
||||
/** 重置迭代器 */
|
||||
reset(): void;
|
||||
}
|
||||
|
||||
/** 数组数据源的迭代器 */
|
||||
export class ArrayEntrySource implements EntrySource {
|
||||
private entries: [string, Record<string, unknown>][];
|
||||
private index = 0;
|
||||
|
||||
constructor(entries: [string, Record<string, unknown>][]) {
|
||||
this.entries = entries;
|
||||
}
|
||||
|
||||
next(): [string, Record<string, unknown>] | null {
|
||||
if (this.index >= this.entries.length) return null;
|
||||
return this.entries[this.index++];
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.index = 0;
|
||||
}
|
||||
}
|
||||
|
||||
/** 回调数据源的迭代器 */
|
||||
export class CallbackEntrySource implements EntrySource {
|
||||
private items: [string, Record<string, unknown>][] = [];
|
||||
private index = 0;
|
||||
private consumed = false;
|
||||
|
||||
/**
|
||||
* @param producer 产生所有条目的回调
|
||||
*/
|
||||
constructor(producer: (cb: (key: string, value: Record<string, unknown>) => void) => void) {
|
||||
producer((key, value) => this.items.push([key, value]));
|
||||
}
|
||||
|
||||
next(): [string, Record<string, unknown>] | null {
|
||||
if (this.index >= this.items.length) return null;
|
||||
return this.items[this.index++];
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.index = 0;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Heap 节点（用于多路归并）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface HeapNode {
|
||||
key: string;
|
||||
value: Record<string, unknown>;
|
||||
sourceIndex: number;
|
||||
}
|
||||
|
||||
/** 最小堆 */
|
||||
class MinHeap {
|
||||
private heap: HeapNode[] = [];
|
||||
|
||||
push(node: HeapNode): void {
|
||||
this.heap.push(node);
|
||||
this.bubbleUp(this.heap.length - 1);
|
||||
}
|
||||
|
||||
pop(): HeapNode | null {
|
||||
if (this.heap.length === 0) return null;
|
||||
if (this.heap.length === 1) return this.heap.pop()!;
|
||||
|
||||
const result = this.heap[0];
|
||||
this.heap[0] = this.heap.pop()!;
|
||||
this.bubbleDown(0);
|
||||
return result;
|
||||
}
|
||||
|
||||
peek(): HeapNode | null {
|
||||
return this.heap.length > 0 ? this.heap[0] : null;
|
||||
}
|
||||
|
||||
get size(): number {
|
||||
return this.heap.length;
|
||||
}
|
||||
|
||||
private bubbleUp(idx: number): void {
|
||||
while (idx > 0) {
|
||||
const parent = Math.floor((idx - 1) / 2);
|
||||
if (this.heap[idx].key >= this.heap[parent].key) break;
|
||||
[this.heap[idx], this.heap[parent]] = [this.heap[parent], this.heap[idx]];
|
||||
idx = parent;
|
||||
}
|
||||
}
|
||||
|
||||
private bubbleDown(idx: number): void {
|
||||
const n = this.heap.length;
|
||||
while (true) {
|
||||
let smallest = idx;
|
||||
const left = 2 * idx + 1;
|
||||
const right = 2 * idx + 2;
|
||||
|
||||
if (left < n && this.heap[left].key < this.heap[smallest].key) smallest = left;
|
||||
if (right < n && this.heap[right].key < this.heap[smallest].key) smallest = right;
|
||||
|
||||
if (smallest === idx) break;
|
||||
|
||||
[this.heap[idx], this.heap[smallest]] = [this.heap[smallest], this.heap[idx]];
|
||||
idx = smallest;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MergeIterator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 对多个有序数据源进行归并，重复 key 保留最新（后出现的）。
|
||||
* 数据源按新鲜度排序：越新的数据源在下标越小（如 MemTable 在 SSTable 之前）。
|
||||
*/
|
||||
export class MergeIterator {
|
||||
private sources: EntrySource[];
|
||||
private heap: MinHeap;
|
||||
|
||||
constructor() {
|
||||
this.sources = [];
|
||||
this.heap = new MinHeap();
|
||||
}
|
||||
|
||||
/** 添加数据源 */
|
||||
addSource(source: EntrySource): void {
|
||||
this.sources.push(source);
|
||||
this.seedFromSource(this.sources.length - 1);
|
||||
}
|
||||
|
||||
/** 获取下一个归并后的条目 */
|
||||
next(): [string, Record<string, unknown>] | null {
|
||||
if (this.heap.size === 0) return null;
|
||||
|
||||
const first = this.heap.pop()!;
|
||||
const key = first.key;
|
||||
let best = first;
|
||||
|
||||
// 刷新 first 来源的下一个值
|
||||
this.seedFromSource(first.sourceIndex);
|
||||
|
||||
// 跳过重复 key：在多个来源中保留 sourceIndex 最小（最新）的条目
|
||||
while (this.heap.peek() && this.heap.peek()!.key === key) {
|
||||
const dup = this.heap.pop()!;
|
||||
this.seedFromSource(dup.sourceIndex);
|
||||
if (dup.sourceIndex < best.sourceIndex) {
|
||||
best = dup;
|
||||
}
|
||||
}
|
||||
|
||||
return [best.key, best.value];
|
||||
}
|
||||
|
||||
/** 耗尽管道，返回所有归并结果 */
|
||||
drain(): [string, Record<string, unknown>][] {
|
||||
const result: [string, Record<string, unknown>][] = [];
|
||||
let entry = this.next();
|
||||
while (entry) {
|
||||
result.push(entry);
|
||||
entry = this.next();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private seedFromSource(sourceIndex: number): void {
|
||||
const entry = this.sources[sourceIndex].next();
|
||||
if (entry) {
|
||||
this.heap.push({
|
||||
key: entry[0],
|
||||
value: entry[1],
|
||||
sourceIndex,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+261
-261
@@ -1,261 +1,261 @@
|
||||
/**
|
||||
* AriaEngine SSTable Reader — 从 SSTable 二进制数据中读取
|
||||
* @module engine/aria/index/sstable
|
||||
*/
|
||||
|
||||
import type { IndexEntry, SSTableMeta } from '../types';
|
||||
import { BloomFilter } from './bloom';
|
||||
|
||||
const SSTABLE_MAGIC = 0x53535442; // "SSTB"
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SSTableReader
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class SSTableReader {
|
||||
private data: Uint8Array;
|
||||
private view: DataView;
|
||||
private indexEntries: IndexEntry[] = [];
|
||||
private entryCount = 0;
|
||||
private meta: SSTableMeta;
|
||||
private bloomFilter: BloomFilter | null = null;
|
||||
|
||||
constructor(data: Uint8Array, meta: SSTableMeta) {
|
||||
this.data = data;
|
||||
this.view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
||||
this.meta = meta;
|
||||
this.parseFooter();
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 查询
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/** 精确查找 key */
|
||||
get(targetKey: string): Record<string, unknown> | null {
|
||||
// Bloom Filter 快速否定
|
||||
if (this.bloomFilter && !this.bloomFilter.mayContain(targetKey)) return null;
|
||||
|
||||
const blockIdx = this.locateBlock(targetKey);
|
||||
if (blockIdx < 0) return null;
|
||||
|
||||
const entry = this.indexEntries[blockIdx];
|
||||
const blockData = new Uint8Array(
|
||||
this.data.buffer,
|
||||
this.data.byteOffset + entry.blockOffset,
|
||||
entry.blockSize,
|
||||
);
|
||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||
|
||||
const entryCount = blockView.getUint32(0, false);
|
||||
let offset = 4;
|
||||
|
||||
// 顺序扫描 block 内的条目（生产中应二分查找）
|
||||
for (let i = 0; i < entryCount; i++) {
|
||||
const keyLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const valLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const valBytes = blockData.slice(offset, offset + valLen);
|
||||
offset += valLen;
|
||||
|
||||
if (key === targetKey) {
|
||||
try {
|
||||
return JSON.parse(new TextDecoder().decode(valBytes));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 范围扫描 */
|
||||
rangeScan(
|
||||
startKey: string,
|
||||
endKey: string,
|
||||
callback: (key: string, value: Record<string, unknown>) => void,
|
||||
): void {
|
||||
if (this.indexEntries.length === 0) return;
|
||||
const startBlockIdx = Math.max(0, this.locateBlockGE(startKey));
|
||||
const endBlockIdx = Math.min(this.indexEntries.length - 1, this.locateBlockLE(endKey));
|
||||
if (startBlockIdx < 0 || endBlockIdx < 0 || startBlockIdx > endBlockIdx) return;
|
||||
|
||||
for (let bi = startBlockIdx; bi <= endBlockIdx && bi >= 0; bi++) {
|
||||
const entry = this.indexEntries[bi];
|
||||
const blockData = new Uint8Array(
|
||||
this.data.buffer,
|
||||
this.data.byteOffset + entry.blockOffset,
|
||||
entry.blockSize,
|
||||
);
|
||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||
|
||||
const blockEntryCount = blockView.getUint32(0, false);
|
||||
let offset = 4;
|
||||
|
||||
for (let i = 0; i < blockEntryCount; i++) {
|
||||
const keyLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const valLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const valBytes = blockData.slice(offset, offset + valLen);
|
||||
offset += valLen;
|
||||
|
||||
if (key >= startKey && key <= endKey) {
|
||||
try {
|
||||
const value = JSON.parse(new TextDecoder().decode(valBytes));
|
||||
callback(key, value);
|
||||
} catch {
|
||||
// skip corrupted entry
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** 扫描所有条目 */
|
||||
scanAll(callback: (key: string, value: Record<string, unknown>) => void): void {
|
||||
for (const entry of this.indexEntries) {
|
||||
const blockData = new Uint8Array(
|
||||
this.data.buffer,
|
||||
this.data.byteOffset + entry.blockOffset,
|
||||
entry.blockSize,
|
||||
);
|
||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||
|
||||
const blockEntryCount = blockView.getUint32(0, false);
|
||||
let offset = 4;
|
||||
|
||||
for (let i = 0; i < blockEntryCount; i++) {
|
||||
const keyLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const valLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const valBytes = blockData.slice(offset, offset + valLen);
|
||||
offset += valLen;
|
||||
|
||||
try {
|
||||
const value = JSON.parse(new TextDecoder().decode(valBytes));
|
||||
callback(key, value);
|
||||
} catch {
|
||||
// skip corrupted entry
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** 获取元数据 */
|
||||
getMeta(): SSTableMeta {
|
||||
return this.meta;
|
||||
}
|
||||
|
||||
/** 获取索引条目数 */
|
||||
getIndexBlockCount(): number {
|
||||
return this.indexEntries.length;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 内部
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private parseFooter(): void {
|
||||
if (this.data.byteLength < 32) {
|
||||
throw new Error('SSTable too small: missing footer');
|
||||
}
|
||||
|
||||
const footerOffset = this.data.byteLength - 32;
|
||||
|
||||
// 验证魔数
|
||||
const magic = this.view.getUint32(footerOffset + 24, false);
|
||||
if (magic !== SSTABLE_MAGIC) {
|
||||
throw new Error(`Invalid SSTable magic: expected ${SSTABLE_MAGIC}, got ${magic}`);
|
||||
}
|
||||
|
||||
const indexOffset = this.view.getUint32(footerOffset, false);
|
||||
const indexSize = this.view.getUint32(footerOffset + 4, false);
|
||||
const bloomOffset = this.view.getUint32(footerOffset + 8, false);
|
||||
const bloomSize = this.view.getUint32(footerOffset + 12, false);
|
||||
const bloomHashCount = this.view.getUint32(footerOffset + 16, false);
|
||||
this.entryCount = this.view.getUint32(footerOffset + 20, false);
|
||||
|
||||
// 解析索引块
|
||||
this.parseIndexBlock(indexOffset, indexSize);
|
||||
|
||||
// 加载 Bloom Filter
|
||||
if (bloomOffset > 0 && bloomSize > 0 && bloomOffset + bloomSize <= this.data.byteLength) {
|
||||
try {
|
||||
const bloomBytes = this.data.slice(bloomOffset, bloomOffset + bloomSize);
|
||||
this.bloomFilter = BloomFilter.fromData(bloomBytes, bloomHashCount || 10);
|
||||
} catch {
|
||||
// 损坏的 bloom filter 不影响读取（仅跳过快速否定优化）
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private parseIndexBlock(offset: number, _size: number): void {
|
||||
const entryCount = this.view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
|
||||
for (let i = 0; i < entryCount; i++) {
|
||||
const keyLen = this.view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(this.data.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const blockOffset = this.view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
const blockSize = this.view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
|
||||
this.indexEntries.push({ key, blockOffset, blockSize });
|
||||
}
|
||||
}
|
||||
|
||||
/** 二分查找某 key 所在的 block 索引 */
|
||||
private locateBlock(key: string): number {
|
||||
let lo = 0;
|
||||
let hi = this.indexEntries.length - 1;
|
||||
|
||||
while (lo <= hi) {
|
||||
const mid = Math.floor((lo + hi) / 2);
|
||||
const entry = this.indexEntries[mid];
|
||||
|
||||
if (key <= entry.key) {
|
||||
// 检查是否在此 block 范围内
|
||||
const firstKey = mid === 0 ? '' : this.indexEntries[mid - 1].key;
|
||||
if (key > firstKey) return mid;
|
||||
hi = mid - 1;
|
||||
} else {
|
||||
lo = mid + 1;
|
||||
}
|
||||
}
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
private locateBlockGE(key: string): number {
|
||||
let lo = 0, hi = this.indexEntries.length;
|
||||
while (lo < hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
if (this.indexEntries[mid].key < key) lo = mid + 1;
|
||||
else hi = mid;
|
||||
}
|
||||
return lo < this.indexEntries.length ? lo : this.indexEntries.length - 1;
|
||||
}
|
||||
|
||||
private locateBlockLE(key: string): number {
|
||||
let lo = 0, hi = this.indexEntries.length;
|
||||
while (lo < hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
if (this.indexEntries[mid].key <= key) lo = mid + 1;
|
||||
else hi = mid;
|
||||
}
|
||||
return lo > 0 ? lo - 1 : 0;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine SSTable Reader — 从 SSTable 二进制数据中读取
|
||||
* @module engine/aria/index/sstable
|
||||
*/
|
||||
|
||||
import type { IndexEntry, SSTableMeta } from '../types';
|
||||
import { BloomFilter } from './bloom';
|
||||
|
||||
const SSTABLE_MAGIC = 0x53535442; // "SSTB"
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SSTableReader
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class SSTableReader {
|
||||
private data: Uint8Array;
|
||||
private view: DataView;
|
||||
private indexEntries: IndexEntry[] = [];
|
||||
private entryCount = 0;
|
||||
private meta: SSTableMeta;
|
||||
private bloomFilter: BloomFilter | null = null;
|
||||
|
||||
constructor(data: Uint8Array, meta: SSTableMeta) {
|
||||
this.data = data;
|
||||
this.view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
||||
this.meta = meta;
|
||||
this.parseFooter();
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 查询
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/** 精确查找 key */
|
||||
get(targetKey: string): Record<string, unknown> | null {
|
||||
// Bloom Filter 快速否定
|
||||
if (this.bloomFilter && !this.bloomFilter.mayContain(targetKey)) return null;
|
||||
|
||||
const blockIdx = this.locateBlock(targetKey);
|
||||
if (blockIdx < 0) return null;
|
||||
|
||||
const entry = this.indexEntries[blockIdx];
|
||||
const blockData = new Uint8Array(
|
||||
this.data.buffer,
|
||||
this.data.byteOffset + entry.blockOffset,
|
||||
entry.blockSize,
|
||||
);
|
||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||
|
||||
const entryCount = blockView.getUint32(0, false);
|
||||
let offset = 4;
|
||||
|
||||
// 顺序扫描 block 内的条目（生产中应二分查找）
|
||||
for (let i = 0; i < entryCount; i++) {
|
||||
const keyLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const valLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const valBytes = blockData.slice(offset, offset + valLen);
|
||||
offset += valLen;
|
||||
|
||||
if (key === targetKey) {
|
||||
try {
|
||||
return JSON.parse(new TextDecoder().decode(valBytes));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 范围扫描 */
|
||||
rangeScan(
|
||||
startKey: string,
|
||||
endKey: string,
|
||||
callback: (key: string, value: Record<string, unknown>) => void,
|
||||
): void {
|
||||
if (this.indexEntries.length === 0) return;
|
||||
const startBlockIdx = Math.max(0, this.locateBlockGE(startKey));
|
||||
const endBlockIdx = Math.min(this.indexEntries.length - 1, this.locateBlockLE(endKey));
|
||||
if (startBlockIdx < 0 || endBlockIdx < 0 || startBlockIdx > endBlockIdx) return;
|
||||
|
||||
for (let bi = startBlockIdx; bi <= endBlockIdx && bi >= 0; bi++) {
|
||||
const entry = this.indexEntries[bi];
|
||||
const blockData = new Uint8Array(
|
||||
this.data.buffer,
|
||||
this.data.byteOffset + entry.blockOffset,
|
||||
entry.blockSize,
|
||||
);
|
||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||
|
||||
const blockEntryCount = blockView.getUint32(0, false);
|
||||
let offset = 4;
|
||||
|
||||
for (let i = 0; i < blockEntryCount; i++) {
|
||||
const keyLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const valLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const valBytes = blockData.slice(offset, offset + valLen);
|
||||
offset += valLen;
|
||||
|
||||
if (key >= startKey && key <= endKey) {
|
||||
try {
|
||||
const value = JSON.parse(new TextDecoder().decode(valBytes));
|
||||
callback(key, value);
|
||||
} catch {
|
||||
// skip corrupted entry
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** 扫描所有条目 */
|
||||
scanAll(callback: (key: string, value: Record<string, unknown>) => void): void {
|
||||
for (const entry of this.indexEntries) {
|
||||
const blockData = new Uint8Array(
|
||||
this.data.buffer,
|
||||
this.data.byteOffset + entry.blockOffset,
|
||||
entry.blockSize,
|
||||
);
|
||||
const blockView = new DataView(blockData.buffer, blockData.byteOffset, blockData.byteLength);
|
||||
|
||||
const blockEntryCount = blockView.getUint32(0, false);
|
||||
let offset = 4;
|
||||
|
||||
for (let i = 0; i < blockEntryCount; i++) {
|
||||
const keyLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(blockData.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const valLen = blockView.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const valBytes = blockData.slice(offset, offset + valLen);
|
||||
offset += valLen;
|
||||
|
||||
try {
|
||||
const value = JSON.parse(new TextDecoder().decode(valBytes));
|
||||
callback(key, value);
|
||||
} catch {
|
||||
// skip corrupted entry
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** 获取元数据 */
|
||||
getMeta(): SSTableMeta {
|
||||
return this.meta;
|
||||
}
|
||||
|
||||
/** 获取索引条目数 */
|
||||
getIndexBlockCount(): number {
|
||||
return this.indexEntries.length;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 内部
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private parseFooter(): void {
|
||||
if (this.data.byteLength < 32) {
|
||||
throw new Error('SSTable too small: missing footer');
|
||||
}
|
||||
|
||||
const footerOffset = this.data.byteLength - 32;
|
||||
|
||||
// 验证魔数
|
||||
const magic = this.view.getUint32(footerOffset + 24, false);
|
||||
if (magic !== SSTABLE_MAGIC) {
|
||||
throw new Error(`Invalid SSTable magic: expected ${SSTABLE_MAGIC}, got ${magic}`);
|
||||
}
|
||||
|
||||
const indexOffset = this.view.getUint32(footerOffset, false);
|
||||
const indexSize = this.view.getUint32(footerOffset + 4, false);
|
||||
const bloomOffset = this.view.getUint32(footerOffset + 8, false);
|
||||
const bloomSize = this.view.getUint32(footerOffset + 12, false);
|
||||
const bloomHashCount = this.view.getUint32(footerOffset + 16, false);
|
||||
this.entryCount = this.view.getUint32(footerOffset + 20, false);
|
||||
|
||||
// 解析索引块
|
||||
this.parseIndexBlock(indexOffset, indexSize);
|
||||
|
||||
// 加载 Bloom Filter
|
||||
if (bloomOffset > 0 && bloomSize > 0 && bloomOffset + bloomSize <= this.data.byteLength) {
|
||||
try {
|
||||
const bloomBytes = this.data.slice(bloomOffset, bloomOffset + bloomSize);
|
||||
this.bloomFilter = BloomFilter.fromData(bloomBytes, bloomHashCount || 10);
|
||||
} catch {
|
||||
// 损坏的 bloom filter 不影响读取（仅跳过快速否定优化）
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private parseIndexBlock(offset: number, _size: number): void {
|
||||
const entryCount = this.view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
|
||||
for (let i = 0; i < entryCount; i++) {
|
||||
const keyLen = this.view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
const key = new TextDecoder().decode(this.data.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
const blockOffset = this.view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
const blockSize = this.view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
|
||||
this.indexEntries.push({ key, blockOffset, blockSize });
|
||||
}
|
||||
}
|
||||
|
||||
/** 二分查找某 key 所在的 block 索引 */
|
||||
private locateBlock(key: string): number {
|
||||
let lo = 0;
|
||||
let hi = this.indexEntries.length - 1;
|
||||
|
||||
while (lo <= hi) {
|
||||
const mid = Math.floor((lo + hi) / 2);
|
||||
const entry = this.indexEntries[mid];
|
||||
|
||||
if (key <= entry.key) {
|
||||
// 检查是否在此 block 范围内
|
||||
const firstKey = mid === 0 ? '' : this.indexEntries[mid - 1].key;
|
||||
if (key > firstKey) return mid;
|
||||
hi = mid - 1;
|
||||
} else {
|
||||
lo = mid + 1;
|
||||
}
|
||||
}
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
private locateBlockGE(key: string): number {
|
||||
let lo = 0, hi = this.indexEntries.length;
|
||||
while (lo < hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
if (this.indexEntries[mid].key < key) lo = mid + 1;
|
||||
else hi = mid;
|
||||
}
|
||||
return lo < this.indexEntries.length ? lo : this.indexEntries.length - 1;
|
||||
}
|
||||
|
||||
private locateBlockLE(key: string): number {
|
||||
let lo = 0, hi = this.indexEntries.length;
|
||||
while (lo < hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
if (this.indexEntries[mid].key <= key) lo = mid + 1;
|
||||
else hi = mid;
|
||||
}
|
||||
return lo > 0 ? lo - 1 : 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,249 +1,247 @@
|
||||
/**
|
||||
* AriaEngine SSTable Builder — 构建有序字符串表
|
||||
* @module engine/aria/index/sstable_builder
|
||||
*
|
||||
* 将排序后的 key-value 数据写入 SSTable 格式。
|
||||
*
|
||||
* SSTable 文件布局:
|
||||
* ┌──────────────────────────────────────────────┐
|
||||
* │ Data Block 0 │
|
||||
* │ Data Block 1 │
|
||||
* │ ... │
|
||||
* │ Index Block (block offset → key range) │
|
||||
* │ Bloom Filter │
|
||||
* │ Footer (32 bytes) │
|
||||
* │ - index_offset (u32) │
|
||||
* │ - index_size (u32) │
|
||||
* │ - bloom_offset (u32) │
|
||||
* │ - bloom_size (u32) │
|
||||
* │ - bloom_hash_count (u32) │
|
||||
* │ - entry_count (u32) │
|
||||
* │ - magic_number (u32, 0x53535442 ="SSTB")│
|
||||
* │ - checksum (u32) │
|
||||
* └──────────────────────────────────────────────┘
|
||||
*/
|
||||
|
||||
import { BloomFilter } from './bloom';
|
||||
import type { IndexEntry, DataBlock } from '../types';
|
||||
|
||||
const SSTABLE_MAGIC = 0x53535442; // "SSTB"
|
||||
const SSTABLE_FOOTER_SIZE = 32;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SSTableBuilder
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class SSTableBuilder {
|
||||
private entries: [string, Record<string, unknown>][] = [];
|
||||
private currentBlock: [string, Record<string, unknown>][] = [];
|
||||
private currentBlockStartKey = '';
|
||||
private blockSizeLimit: number;
|
||||
|
||||
constructor(blockSizeLimit: number = 4096) {
|
||||
this.blockSizeLimit = blockSizeLimit;
|
||||
}
|
||||
|
||||
/** 添加一个 key-value 条目（必须按键排序添加） */
|
||||
add(key: string, value: Record<string, unknown>): void {
|
||||
if (this.currentBlock.length === 0) {
|
||||
this.currentBlockStartKey = key;
|
||||
}
|
||||
|
||||
this.currentBlock.push([key, value]);
|
||||
this.entries.push([key, value]);
|
||||
|
||||
// 如果当前 Block 达到大小限制，切割
|
||||
const estimated = this.estimateBlockSize();
|
||||
if (estimated >= this.blockSizeLimit && this.currentBlock.length > 1) {
|
||||
// 当前 block 结束（不在这里切割，在 build 时统一处理）
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建 SSTable 文件的二进制数据。
|
||||
* 返回 { data: Uint8Array, indexEntries: IndexEntry[], bloomFilter: BloomFilter }
|
||||
*/
|
||||
build(): { sstableData: Uint8Array; indexEntries: IndexEntry[] } {
|
||||
const blocks = this.splitIntoBlocks();
|
||||
const bloomFilter = new BloomFilter(this.entries.length);
|
||||
|
||||
// 预计算总大小
|
||||
let totalSize = 0;
|
||||
const blockOffsets: number[] = [];
|
||||
|
||||
for (const block of blocks) {
|
||||
blockOffsets.push(totalSize);
|
||||
const blockSize = this.computeBlockSize(block);
|
||||
totalSize += blockSize;
|
||||
}
|
||||
|
||||
// 索引块
|
||||
const indexEntries: IndexEntry[] = [];
|
||||
for (let i = 0; i < blocks.length; i++) {
|
||||
const block = blocks[i];
|
||||
const lastKey = block[block.length - 1][0];
|
||||
const blockSize = this.computeBlockSize(block);
|
||||
indexEntries.push({
|
||||
key: lastKey,
|
||||
blockOffset: blockOffsets[i],
|
||||
blockSize,
|
||||
});
|
||||
}
|
||||
|
||||
const indexBlockSize = this.estimateIndexBlockSize(indexEntries);
|
||||
|
||||
// 序列化 bloom filter 以获取其大小
|
||||
const bloomData = bloomFilter.serialize();
|
||||
const bloomSize = bloomData.byteLength;
|
||||
|
||||
// 写入到 buffer（包含 bloom block）
|
||||
const finalSize = totalSize + indexBlockSize + bloomSize + SSTABLE_FOOTER_SIZE;
|
||||
const buf = new ArrayBuffer(finalSize);
|
||||
const view = new DataView(buf);
|
||||
|
||||
let offset = 0;
|
||||
|
||||
// ---- Data Blocks ----
|
||||
for (const block of blocks) {
|
||||
offset = this.writeDataBlock(view, offset, block, bloomFilter);
|
||||
}
|
||||
|
||||
// ---- Index Block ----
|
||||
const indexOffset = offset;
|
||||
offset = this.writeIndexBlock(view, offset, indexEntries);
|
||||
|
||||
// ---- Bloom Filter Block ----
|
||||
const bloomOffset = offset;
|
||||
new Uint8Array(view.buffer).set(bloomData, offset);
|
||||
offset += bloomSize;
|
||||
|
||||
// ---- Footer ----
|
||||
const footerOffset = offset;
|
||||
view.setUint32(footerOffset, indexOffset, false); // index_offset
|
||||
view.setUint32(footerOffset + 4, indexBlockSize, false); // index_size
|
||||
view.setUint32(footerOffset + 8, bloomOffset, false); // bloom_offset
|
||||
view.setUint32(footerOffset + 12, bloomSize, false); // bloom_size
|
||||
view.setUint32(footerOffset + 16, bloomFilter.getHashCount(), false);
|
||||
view.setUint32(footerOffset + 20, this.entries.length, false);
|
||||
view.setUint32(footerOffset + 24, SSTABLE_MAGIC, false);
|
||||
view.setUint32(footerOffset + 28, 0, false); // checksum (simplified: 0)
|
||||
|
||||
return {
|
||||
sstableData: new Uint8Array(buf),
|
||||
indexEntries,
|
||||
};
|
||||
}
|
||||
|
||||
/** 获取条目数 */
|
||||
getEntryCount(): number {
|
||||
return this.entries.length;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 内部
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private splitIntoBlocks(): [string, Record<string, unknown>][][] {
|
||||
const blocks: [string, Record<string, unknown>][][] = [];
|
||||
let current: [string, Record<string, unknown>][] = [];
|
||||
|
||||
for (const entry of this.entries) {
|
||||
current.push(entry);
|
||||
if (this.estimateBlockSizeFromEntries(current) >= this.blockSizeLimit && current.length > 1) {
|
||||
blocks.push(current.slice(0, -1));
|
||||
current = [entry];
|
||||
}
|
||||
}
|
||||
if (current.length > 0) blocks.push(current);
|
||||
|
||||
return blocks;
|
||||
}
|
||||
|
||||
private estimateBlockSize(): number {
|
||||
return this.estimateBlockSizeFromEntries(this.currentBlock);
|
||||
}
|
||||
|
||||
private estimateBlockSizeFromEntries(entries: [string, unknown][]): number {
|
||||
let size = 0;
|
||||
for (const [key, value] of entries) {
|
||||
size += 4 + key.length + JSON.stringify(value).length;
|
||||
}
|
||||
return size;
|
||||
}
|
||||
|
||||
private computeBlockSize(block: [string, unknown][]): number {
|
||||
// entryCount (u32) + 每对: keyLen(u16) + key + valueLen(u16) + value json
|
||||
let size = 4;
|
||||
for (const [key, value] of block) {
|
||||
const json = JSON.stringify(value);
|
||||
size += 2 + key.length + 2 + json.length;
|
||||
}
|
||||
return size;
|
||||
}
|
||||
|
||||
private writeDataBlock(
|
||||
view: DataView,
|
||||
offset: number,
|
||||
block: [string, Record<string, unknown>][],
|
||||
bloomFilter: BloomFilter,
|
||||
): number {
|
||||
const start = offset;
|
||||
|
||||
// entry count
|
||||
view.setUint32(offset, block.length, false);
|
||||
offset += 4;
|
||||
|
||||
for (const [key, value] of block) {
|
||||
const encoder = new TextEncoder();
|
||||
const keyBytes = encoder.encode(key);
|
||||
const valueBytes = encoder.encode(JSON.stringify(value));
|
||||
|
||||
// key length
|
||||
view.setUint16(offset, keyBytes.length, false);
|
||||
offset += 2;
|
||||
// key
|
||||
new Uint8Array(view.buffer).set(keyBytes, offset);
|
||||
offset += keyBytes.length;
|
||||
// value length
|
||||
view.setUint16(offset, valueBytes.length, false);
|
||||
offset += 2;
|
||||
// value
|
||||
new Uint8Array(view.buffer).set(valueBytes, offset);
|
||||
offset += valueBytes.length;
|
||||
|
||||
// 插入 bloom filter
|
||||
bloomFilter.insert(key);
|
||||
}
|
||||
|
||||
return offset;
|
||||
}
|
||||
|
||||
private estimateIndexBlockSize(entries: IndexEntry[]): number {
|
||||
// entryCount(u32) + each: keyLen(u16)+key+blockOffset(u32)+blockSize(u32)
|
||||
let size = 4;
|
||||
for (const entry of entries) {
|
||||
size += 2 + entry.key.length + 8;
|
||||
}
|
||||
return size;
|
||||
}
|
||||
|
||||
private writeIndexBlock(view: DataView, offset: number, entries: IndexEntry[]): number {
|
||||
view.setUint32(offset, entries.length, false);
|
||||
offset += 4;
|
||||
|
||||
for (const entry of entries) {
|
||||
const encoder = new TextEncoder();
|
||||
const keyBytes = encoder.encode(entry.key);
|
||||
view.setUint16(offset, keyBytes.length, false);
|
||||
offset += 2;
|
||||
new Uint8Array(view.buffer).set(keyBytes, offset);
|
||||
offset += keyBytes.length;
|
||||
view.setUint32(offset, entry.blockOffset, false);
|
||||
offset += 4;
|
||||
view.setUint32(offset, entry.blockSize, false);
|
||||
offset += 4;
|
||||
}
|
||||
|
||||
return offset;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine SSTable Builder — 构建有序字符串表
|
||||
* @module engine/aria/index/sstable_builder
|
||||
*
|
||||
* 将排序后的 key-value 数据写入 SSTable 格式。
|
||||
*
|
||||
* SSTable 文件布局:
|
||||
* ┌──────────────────────────────────────────────┐
|
||||
* │ Data Block 0 │
|
||||
* │ Data Block 1 │
|
||||
* │ ... │
|
||||
* │ Index Block (block offset → key range) │
|
||||
* │ Bloom Filter │
|
||||
* │ Footer (32 bytes) │
|
||||
* │ - index_offset (u32) │
|
||||
* │ - index_size (u32) │
|
||||
* │ - bloom_offset (u32) │
|
||||
* │ - bloom_size (u32) │
|
||||
* │ - bloom_hash_count (u32) │
|
||||
* │ - entry_count (u32) │
|
||||
* │ - magic_number (u32, 0x53535442 ="SSTB")│
|
||||
* │ - checksum (u32) │
|
||||
* └──────────────────────────────────────────────┘
|
||||
*/
|
||||
|
||||
import { BloomFilter } from './bloom';
|
||||
import type { IndexEntry } from '../types';
|
||||
|
||||
const SSTABLE_MAGIC = 0x53535442; // "SSTB"
|
||||
const SSTABLE_FOOTER_SIZE = 32;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SSTableBuilder
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class SSTableBuilder {
|
||||
private entries: [string, Record<string, unknown>][] = [];
|
||||
private currentBlock: [string, Record<string, unknown>][] = [];
|
||||
private currentBlockStartKey = '';
|
||||
private blockSizeLimit: number;
|
||||
|
||||
constructor(blockSizeLimit: number = 4096) {
|
||||
this.blockSizeLimit = blockSizeLimit;
|
||||
}
|
||||
|
||||
/** 添加一个 key-value 条目（必须按键排序添加） */
|
||||
add(key: string, value: Record<string, unknown>): void {
|
||||
if (this.currentBlock.length === 0) {
|
||||
this.currentBlockStartKey = key;
|
||||
}
|
||||
|
||||
this.currentBlock.push([key, value]);
|
||||
this.entries.push([key, value]);
|
||||
|
||||
// 如果当前 Block 达到大小限制，切割
|
||||
const estimated = this.estimateBlockSize();
|
||||
if (estimated >= this.blockSizeLimit && this.currentBlock.length > 1) {
|
||||
// 当前 block 结束（不在这里切割，在 build 时统一处理）
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建 SSTable 文件的二进制数据。
|
||||
* 返回 { data: Uint8Array, indexEntries: IndexEntry[], bloomFilter: BloomFilter }
|
||||
*/
|
||||
build(): { sstableData: Uint8Array; indexEntries: IndexEntry[] } {
|
||||
const blocks = this.splitIntoBlocks();
|
||||
const bloomFilter = new BloomFilter(this.entries.length);
|
||||
|
||||
// 预计算总大小
|
||||
let totalSize = 0;
|
||||
const blockOffsets: number[] = [];
|
||||
|
||||
for (const block of blocks) {
|
||||
blockOffsets.push(totalSize);
|
||||
const blockSize = this.computeBlockSize(block);
|
||||
totalSize += blockSize;
|
||||
}
|
||||
|
||||
// 索引块
|
||||
const indexEntries: IndexEntry[] = [];
|
||||
for (let i = 0; i < blocks.length; i++) {
|
||||
const block = blocks[i];
|
||||
const lastKey = block[block.length - 1][0];
|
||||
const blockSize = this.computeBlockSize(block);
|
||||
indexEntries.push({
|
||||
key: lastKey,
|
||||
blockOffset: blockOffsets[i],
|
||||
blockSize,
|
||||
});
|
||||
}
|
||||
|
||||
const indexBlockSize = this.estimateIndexBlockSize(indexEntries);
|
||||
|
||||
// 序列化 bloom filter 以获取其大小
|
||||
const bloomData = bloomFilter.serialize();
|
||||
const bloomSize = bloomData.byteLength;
|
||||
|
||||
// 写入到 buffer（包含 bloom block）
|
||||
const finalSize = totalSize + indexBlockSize + bloomSize + SSTABLE_FOOTER_SIZE;
|
||||
const buf = new ArrayBuffer(finalSize);
|
||||
const view = new DataView(buf);
|
||||
|
||||
let offset = 0;
|
||||
|
||||
// ---- Data Blocks ----
|
||||
for (const block of blocks) {
|
||||
offset = this.writeDataBlock(view, offset, block, bloomFilter);
|
||||
}
|
||||
|
||||
// ---- Index Block ----
|
||||
const indexOffset = offset;
|
||||
offset = this.writeIndexBlock(view, offset, indexEntries);
|
||||
|
||||
// ---- Bloom Filter Block ----
|
||||
const bloomOffset = offset;
|
||||
new Uint8Array(view.buffer).set(bloomData, offset);
|
||||
offset += bloomSize;
|
||||
|
||||
// ---- Footer ----
|
||||
const footerOffset = offset;
|
||||
view.setUint32(footerOffset, indexOffset, false); // index_offset
|
||||
view.setUint32(footerOffset + 4, indexBlockSize, false); // index_size
|
||||
view.setUint32(footerOffset + 8, bloomOffset, false); // bloom_offset
|
||||
view.setUint32(footerOffset + 12, bloomSize, false); // bloom_size
|
||||
view.setUint32(footerOffset + 16, bloomFilter.getHashCount(), false);
|
||||
view.setUint32(footerOffset + 20, this.entries.length, false);
|
||||
view.setUint32(footerOffset + 24, SSTABLE_MAGIC, false);
|
||||
view.setUint32(footerOffset + 28, 0, false); // checksum (simplified: 0)
|
||||
|
||||
return {
|
||||
sstableData: new Uint8Array(buf),
|
||||
indexEntries,
|
||||
};
|
||||
}
|
||||
|
||||
/** 获取条目数 */
|
||||
getEntryCount(): number {
|
||||
return this.entries.length;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 内部
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private splitIntoBlocks(): [string, Record<string, unknown>][][] {
|
||||
const blocks: [string, Record<string, unknown>][][] = [];
|
||||
let current: [string, Record<string, unknown>][] = [];
|
||||
|
||||
for (const entry of this.entries) {
|
||||
current.push(entry);
|
||||
if (this.estimateBlockSizeFromEntries(current) >= this.blockSizeLimit && current.length > 1) {
|
||||
blocks.push(current.slice(0, -1));
|
||||
current = [entry];
|
||||
}
|
||||
}
|
||||
if (current.length > 0) blocks.push(current);
|
||||
|
||||
return blocks;
|
||||
}
|
||||
|
||||
private estimateBlockSize(): number {
|
||||
return this.estimateBlockSizeFromEntries(this.currentBlock);
|
||||
}
|
||||
|
||||
private estimateBlockSizeFromEntries(entries: [string, unknown][]): number {
|
||||
let size = 0;
|
||||
for (const [key, value] of entries) {
|
||||
size += 4 + key.length + JSON.stringify(value).length;
|
||||
}
|
||||
return size;
|
||||
}
|
||||
|
||||
private computeBlockSize(block: [string, unknown][]): number {
|
||||
// entryCount (u32) + 每对: keyLen(u16) + key + valueLen(u16) + value json
|
||||
let size = 4;
|
||||
for (const [key, value] of block) {
|
||||
const json = JSON.stringify(value);
|
||||
size += 2 + key.length + 2 + json.length;
|
||||
}
|
||||
return size;
|
||||
}
|
||||
|
||||
private writeDataBlock(
|
||||
view: DataView,
|
||||
offset: number,
|
||||
block: [string, Record<string, unknown>][],
|
||||
bloomFilter: BloomFilter,
|
||||
): number {
|
||||
// entry count
|
||||
view.setUint32(offset, block.length, false);
|
||||
offset += 4;
|
||||
|
||||
for (const [key, value] of block) {
|
||||
const encoder = new TextEncoder();
|
||||
const keyBytes = encoder.encode(key);
|
||||
const valueBytes = encoder.encode(JSON.stringify(value));
|
||||
|
||||
// key length
|
||||
view.setUint16(offset, keyBytes.length, false);
|
||||
offset += 2;
|
||||
// key
|
||||
new Uint8Array(view.buffer).set(keyBytes, offset);
|
||||
offset += keyBytes.length;
|
||||
// value length
|
||||
view.setUint16(offset, valueBytes.length, false);
|
||||
offset += 2;
|
||||
// value
|
||||
new Uint8Array(view.buffer).set(valueBytes, offset);
|
||||
offset += valueBytes.length;
|
||||
|
||||
// 插入 bloom filter
|
||||
bloomFilter.insert(key);
|
||||
}
|
||||
|
||||
return offset;
|
||||
}
|
||||
|
||||
private estimateIndexBlockSize(entries: IndexEntry[]): number {
|
||||
// entryCount(u32) + each: keyLen(u16)+key+blockOffset(u32)+blockSize(u32)
|
||||
let size = 4;
|
||||
for (const entry of entries) {
|
||||
size += 2 + entry.key.length + 8;
|
||||
}
|
||||
return size;
|
||||
}
|
||||
|
||||
private writeIndexBlock(view: DataView, offset: number, entries: IndexEntry[]): number {
|
||||
view.setUint32(offset, entries.length, false);
|
||||
offset += 4;
|
||||
|
||||
for (const entry of entries) {
|
||||
const encoder = new TextEncoder();
|
||||
const keyBytes = encoder.encode(entry.key);
|
||||
view.setUint16(offset, keyBytes.length, false);
|
||||
offset += 2;
|
||||
new Uint8Array(view.buffer).set(keyBytes, offset);
|
||||
offset += keyBytes.length;
|
||||
view.setUint32(offset, entry.blockOffset, false);
|
||||
offset += 4;
|
||||
view.setUint32(offset, entry.blockSize, false);
|
||||
offset += 4;
|
||||
}
|
||||
|
||||
return offset;
|
||||
}
|
||||
}
|
||||
|
||||
+164
-167
@@ -1,167 +1,164 @@
|
||||
/**
|
||||
* AriaEngine Page Format — 页面格式整合层
|
||||
* @module engine/aria/page/format
|
||||
*
|
||||
* 组合 Header / Slot / Tuple 操作，提供统一的页面管理接口。
|
||||
*/
|
||||
|
||||
import { PAGE_SIZE, PageType, type PageHandle } from '../types';
|
||||
import {
|
||||
initPageHeader,
|
||||
decodePageHeader,
|
||||
encodePageHeader,
|
||||
getSlotCount,
|
||||
getPageId,
|
||||
} from './header';
|
||||
import { allocateSlot, freeSlot, readSlotData, getAllSlots } from './slot';
|
||||
import { encodeTuple, decodeTuple } from './tuple';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 页面创建
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 创建一个新的空页面 */
|
||||
export function createPage(pageId: number, type: PageType): PageHandle {
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, pageId, type);
|
||||
return {
|
||||
pageId,
|
||||
type,
|
||||
data,
|
||||
dirty: true,
|
||||
pins: 0,
|
||||
prev: null,
|
||||
next: null,
|
||||
lastAccess: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
/** 从 ArrayBuffer 恢复页面句柄 */
|
||||
export function pageFromBuffer(
|
||||
pageId: number,
|
||||
buffer: ArrayBuffer,
|
||||
): PageHandle {
|
||||
return {
|
||||
pageId,
|
||||
type: new DataView(buffer).getUint8(4) as PageType,
|
||||
data: buffer,
|
||||
dirty: false,
|
||||
pins: 0,
|
||||
prev: null,
|
||||
next: null,
|
||||
lastAccess: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 行操作（页面级）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 插入一行到页面，返回 slot 索引，空间不足返回 -1。
|
||||
*/
|
||||
export function pageInsertRow(
|
||||
page: PageHandle,
|
||||
row: Record<string, unknown>,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): number {
|
||||
const encoded = encodeTuple(row, columnOrder, columnTypes);
|
||||
const slotIdx = allocateSlot(page.data, encoded);
|
||||
if (slotIdx >= 0) {
|
||||
page.dirty = true;
|
||||
page.lastAccess = Date.now();
|
||||
}
|
||||
return slotIdx;
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取指定 slot 的行数据并解码。
|
||||
*/
|
||||
export function pageReadRow(
|
||||
page: PageHandle,
|
||||
slotIndex: number,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Record<string, unknown> | null {
|
||||
const slotData = readSlotData(page.data, slotIndex);
|
||||
if (!slotData) return null;
|
||||
return decodeTuple(slotData, columnOrder, columnTypes);
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取页面中的所有行。
|
||||
*/
|
||||
export function pageReadAllRows(
|
||||
page: PageHandle,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Record<string, unknown>[] {
|
||||
const rows: Record<string, unknown>[] = [];
|
||||
const slotCount = getSlotCount(page.data);
|
||||
for (let i = 0; i < slotCount; i++) {
|
||||
const row = pageReadRow(page, i, columnOrder, columnTypes);
|
||||
if (row) rows.push(row);
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
/**
|
||||
* 标记 slot 为已删除。
|
||||
*/
|
||||
export function pageDeleteRow(page: PageHandle, slotIndex: number): void {
|
||||
freeSlot(page.data, slotIndex);
|
||||
page.dirty = true;
|
||||
page.lastAccess = Date.now();
|
||||
}
|
||||
|
||||
/**
|
||||
* 重写指定 slot 的行数据。
|
||||
*/
|
||||
export function pageUpdateRow(
|
||||
page: PageHandle,
|
||||
slotIndex: number,
|
||||
row: Record<string, unknown>,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): void {
|
||||
// 先标记旧 slot 为删除
|
||||
pageDeleteRow(page, slotIndex);
|
||||
// 分配新 slot，可能会在不同位置
|
||||
const newSlot = pageInsertRow(page, row, columnOrder, columnTypes);
|
||||
// 注意：调用者需要自行维护 slot index → pk 的映射
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 校验和
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 简单 CRC32（使用预先计算的查找表简化） */
|
||||
export function computeChecksum(data: ArrayBuffer): number {
|
||||
const view = new Uint8Array(data);
|
||||
let hash = 0;
|
||||
for (let i = 0; i < view.byteLength; i++) {
|
||||
hash = ((hash << 5) - hash + view[i]) | 0;
|
||||
}
|
||||
return hash >>> 0;
|
||||
}
|
||||
|
||||
/** 更新页面的校验和字段 */
|
||||
export function updateChecksum(page: PageHandle): void {
|
||||
// 先清零校验和字段
|
||||
const view = new DataView(page.data);
|
||||
view.setUint32(11, 0, false);
|
||||
// 计算校验和
|
||||
const cksum = computeChecksum(page.data);
|
||||
view.setUint32(11, cksum, false);
|
||||
}
|
||||
|
||||
/** 验证页面校验和 */
|
||||
export function verifyChecksum(page: PageHandle): boolean {
|
||||
const stored = new DataView(page.data).getUint32(11, false);
|
||||
// 临时清零
|
||||
new DataView(page.data).setUint32(11, 0, false);
|
||||
const computed = computeChecksum(page.data);
|
||||
new DataView(page.data).setUint32(11, stored, false);
|
||||
return stored === computed;
|
||||
}
|
||||
/**
|
||||
* AriaEngine Page Format — 页面格式整合层
|
||||
* @module engine/aria/page/format
|
||||
*
|
||||
* 组合 Header / Slot / Tuple 操作，提供统一的页面管理接口。
|
||||
*/
|
||||
|
||||
import { PAGE_SIZE, PageType, type PageHandle } from '../types';
|
||||
import {
|
||||
initPageHeader,
|
||||
getSlotCount,
|
||||
} from './header';
|
||||
import { allocateSlot, freeSlot, readSlotData } from './slot';
|
||||
import { encodeTuple, decodeTuple } from './tuple';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 页面创建
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 创建一个新的空页面 */
|
||||
export function createPage(pageId: number, type: PageType): PageHandle {
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, pageId, type);
|
||||
return {
|
||||
pageId,
|
||||
type,
|
||||
data,
|
||||
dirty: true,
|
||||
pins: 0,
|
||||
prev: null,
|
||||
next: null,
|
||||
lastAccess: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
/** 从 ArrayBuffer 恢复页面句柄 */
|
||||
export function pageFromBuffer(
|
||||
pageId: number,
|
||||
buffer: ArrayBuffer,
|
||||
): PageHandle {
|
||||
return {
|
||||
pageId,
|
||||
type: new DataView(buffer).getUint8(4) as PageType,
|
||||
data: buffer,
|
||||
dirty: false,
|
||||
pins: 0,
|
||||
prev: null,
|
||||
next: null,
|
||||
lastAccess: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 行操作（页面级）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 插入一行到页面，返回 slot 索引，空间不足返回 -1。
|
||||
*/
|
||||
export function pageInsertRow(
|
||||
page: PageHandle,
|
||||
row: Record<string, unknown>,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): number {
|
||||
const encoded = encodeTuple(row, columnOrder, columnTypes);
|
||||
const slotIdx = allocateSlot(page.data, encoded);
|
||||
if (slotIdx >= 0) {
|
||||
page.dirty = true;
|
||||
page.lastAccess = Date.now();
|
||||
}
|
||||
return slotIdx;
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取指定 slot 的行数据并解码。
|
||||
*/
|
||||
export function pageReadRow(
|
||||
page: PageHandle,
|
||||
slotIndex: number,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Record<string, unknown> | null {
|
||||
const slotData = readSlotData(page.data, slotIndex);
|
||||
if (!slotData) return null;
|
||||
return decodeTuple(slotData, columnOrder, columnTypes);
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取页面中的所有行。
|
||||
*/
|
||||
export function pageReadAllRows(
|
||||
page: PageHandle,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Record<string, unknown>[] {
|
||||
const rows: Record<string, unknown>[] = [];
|
||||
const slotCount = getSlotCount(page.data);
|
||||
for (let i = 0; i < slotCount; i++) {
|
||||
const row = pageReadRow(page, i, columnOrder, columnTypes);
|
||||
if (row) rows.push(row);
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
/**
|
||||
* 标记 slot 为已删除。
|
||||
*/
|
||||
export function pageDeleteRow(page: PageHandle, slotIndex: number): void {
|
||||
freeSlot(page.data, slotIndex);
|
||||
page.dirty = true;
|
||||
page.lastAccess = Date.now();
|
||||
}
|
||||
|
||||
/**
|
||||
* 重写指定 slot 的行数据。
|
||||
*/
|
||||
export function pageUpdateRow(
|
||||
page: PageHandle,
|
||||
slotIndex: number,
|
||||
row: Record<string, unknown>,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): void {
|
||||
// 先标记旧 slot 为删除
|
||||
pageDeleteRow(page, slotIndex);
|
||||
// 分配新 slot，可能会在不同位置
|
||||
pageInsertRow(page, row, columnOrder, columnTypes);
|
||||
// 注意：调用者需要自行维护 slot index → pk 的映射
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 校验和
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 简单 CRC32（使用预先计算的查找表简化） */
|
||||
export function computeChecksum(data: ArrayBuffer): number {
|
||||
const view = new Uint8Array(data);
|
||||
let hash = 0;
|
||||
for (let i = 0; i < view.byteLength; i++) {
|
||||
hash = ((hash << 5) - hash + view[i]) | 0;
|
||||
}
|
||||
return hash >>> 0;
|
||||
}
|
||||
|
||||
/** 更新页面的校验和字段 */
|
||||
export function updateChecksum(page: PageHandle): void {
|
||||
// 先清零校验和字段
|
||||
const view = new DataView(page.data);
|
||||
view.setUint32(11, 0, false);
|
||||
// 计算校验和
|
||||
const cksum = computeChecksum(page.data);
|
||||
view.setUint32(11, cksum, false);
|
||||
}
|
||||
|
||||
/** 验证页面校验和 */
|
||||
export function verifyChecksum(page: PageHandle): boolean {
|
||||
const stored = new DataView(page.data).getUint32(11, false);
|
||||
// 临时清零
|
||||
new DataView(page.data).setUint32(11, 0, false);
|
||||
const computed = computeChecksum(page.data);
|
||||
new DataView(page.data).setUint32(11, stored, false);
|
||||
return stored === computed;
|
||||
}
|
||||
|
||||
@@ -1,97 +1,97 @@
|
||||
/**
|
||||
* AriaEngine Page Header — 页面头部编解码
|
||||
* @module engine/aria/page/header
|
||||
*/
|
||||
|
||||
import { PAGE_HEADER_SIZE, PageType, type PageHeader } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 编码
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 将 PageHeader 编码写入 ArrayBuffer 的前 16 字节。
|
||||
* 布局（大端序）:
|
||||
* [0-3] page_id u32
|
||||
* [4] type u8
|
||||
* [5-6] free_start u16
|
||||
* [7-8] free_end u16
|
||||
* [9-10] slot_count u16
|
||||
* [11-14] checksum u32
|
||||
* [15] reserved u8
|
||||
*/
|
||||
export function encodePageHeader(header: PageHeader, buf: ArrayBuffer): void {
|
||||
const view = new DataView(buf);
|
||||
view.setUint32(0, header.pageId, false);
|
||||
view.setUint8(4, header.type);
|
||||
view.setUint16(5, header.freeStart, false);
|
||||
view.setUint16(7, header.freeEnd, false);
|
||||
view.setUint16(9, header.slotCount, false);
|
||||
view.setUint32(11, header.checksum, false);
|
||||
view.setUint8(15, 0); // reserved
|
||||
}
|
||||
|
||||
/**
|
||||
* 从 ArrayBuffer 解码 PageHeader。
|
||||
*/
|
||||
export function decodePageHeader(buf: ArrayBuffer): PageHeader {
|
||||
const view = new DataView(buf);
|
||||
return {
|
||||
pageId: view.getUint32(0, false),
|
||||
type: view.getUint8(4) as PageType,
|
||||
freeStart: view.getUint16(5, false),
|
||||
freeEnd: view.getUint16(7, false),
|
||||
slotCount: view.getUint16(9, false),
|
||||
checksum: view.getUint32(11, false),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 初始化新页面的 Header。
|
||||
*/
|
||||
export function initPageHeader(
|
||||
buf: ArrayBuffer,
|
||||
pageId: number,
|
||||
type: PageType,
|
||||
): void {
|
||||
const view = new DataView(buf);
|
||||
view.setUint32(0, pageId, false);
|
||||
view.setUint8(4, type);
|
||||
view.setUint16(5, PAGE_HEADER_SIZE, false); // freeStart = header 之后
|
||||
view.setUint16(7, buf.byteLength, false); // freeEnd = 页面末尾
|
||||
view.setUint16(9, 0, false); // slotCount = 0
|
||||
view.setUint32(11, 0, false); // checksum = 0
|
||||
view.setUint8(15, 0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 从 Buffer 中提取 Header 字段的辅助函数
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function getPageType(buf: ArrayBuffer): PageType {
|
||||
return new DataView(buf).getUint8(4) as PageType;
|
||||
}
|
||||
|
||||
export function getPageId(buf: ArrayBuffer): number {
|
||||
return new DataView(buf).getUint32(0, false);
|
||||
}
|
||||
|
||||
export function getSlotCount(buf: ArrayBuffer): number {
|
||||
return new DataView(buf).getUint16(9, false);
|
||||
}
|
||||
|
||||
export function getFreeStart(buf: ArrayBuffer): number {
|
||||
return new DataView(buf).getUint16(5, false);
|
||||
}
|
||||
|
||||
export function setFreeStart(buf: ArrayBuffer, val: number): void {
|
||||
new DataView(buf).setUint16(5, val, false);
|
||||
}
|
||||
|
||||
export function setFreeEnd(buf: ArrayBuffer, val: number): void {
|
||||
new DataView(buf).setUint16(7, val, false);
|
||||
}
|
||||
|
||||
export function setSlotCount(buf: ArrayBuffer, val: number): void {
|
||||
new DataView(buf).setUint16(9, val, false);
|
||||
}
|
||||
/**
|
||||
* AriaEngine Page Header — 页面头部编解码
|
||||
* @module engine/aria/page/header
|
||||
*/
|
||||
|
||||
import { PAGE_HEADER_SIZE, PageType, type PageHeader } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 编码
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 将 PageHeader 编码写入 ArrayBuffer 的前 16 字节。
|
||||
* 布局（大端序）:
|
||||
* [0-3] page_id u32
|
||||
* [4] type u8
|
||||
* [5-6] free_start u16
|
||||
* [7-8] free_end u16
|
||||
* [9-10] slot_count u16
|
||||
* [11-14] checksum u32
|
||||
* [15] reserved u8
|
||||
*/
|
||||
export function encodePageHeader(header: PageHeader, buf: ArrayBuffer): void {
|
||||
const view = new DataView(buf);
|
||||
view.setUint32(0, header.pageId, false);
|
||||
view.setUint8(4, header.type);
|
||||
view.setUint16(5, header.freeStart, false);
|
||||
view.setUint16(7, header.freeEnd, false);
|
||||
view.setUint16(9, header.slotCount, false);
|
||||
view.setUint32(11, header.checksum, false);
|
||||
view.setUint8(15, 0); // reserved
|
||||
}
|
||||
|
||||
/**
|
||||
* 从 ArrayBuffer 解码 PageHeader。
|
||||
*/
|
||||
export function decodePageHeader(buf: ArrayBuffer): PageHeader {
|
||||
const view = new DataView(buf);
|
||||
return {
|
||||
pageId: view.getUint32(0, false),
|
||||
type: view.getUint8(4) as PageType,
|
||||
freeStart: view.getUint16(5, false),
|
||||
freeEnd: view.getUint16(7, false),
|
||||
slotCount: view.getUint16(9, false),
|
||||
checksum: view.getUint32(11, false),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 初始化新页面的 Header。
|
||||
*/
|
||||
export function initPageHeader(
|
||||
buf: ArrayBuffer,
|
||||
pageId: number,
|
||||
type: PageType,
|
||||
): void {
|
||||
const view = new DataView(buf);
|
||||
view.setUint32(0, pageId, false);
|
||||
view.setUint8(4, type);
|
||||
view.setUint16(5, PAGE_HEADER_SIZE, false); // freeStart = header 之后
|
||||
view.setUint16(7, buf.byteLength, false); // freeEnd = 页面末尾
|
||||
view.setUint16(9, 0, false); // slotCount = 0
|
||||
view.setUint32(11, 0, false); // checksum = 0
|
||||
view.setUint8(15, 0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 从 Buffer 中提取 Header 字段的辅助函数
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function getPageType(buf: ArrayBuffer): PageType {
|
||||
return new DataView(buf).getUint8(4) as PageType;
|
||||
}
|
||||
|
||||
export function getPageId(buf: ArrayBuffer): number {
|
||||
return new DataView(buf).getUint32(0, false);
|
||||
}
|
||||
|
||||
export function getSlotCount(buf: ArrayBuffer): number {
|
||||
return new DataView(buf).getUint16(9, false);
|
||||
}
|
||||
|
||||
export function getFreeStart(buf: ArrayBuffer): number {
|
||||
return new DataView(buf).getUint16(5, false);
|
||||
}
|
||||
|
||||
export function setFreeStart(buf: ArrayBuffer, val: number): void {
|
||||
new DataView(buf).setUint16(5, val, false);
|
||||
}
|
||||
|
||||
export function setFreeEnd(buf: ArrayBuffer, val: number): void {
|
||||
new DataView(buf).setUint16(7, val, false);
|
||||
}
|
||||
|
||||
export function setSlotCount(buf: ArrayBuffer, val: number): void {
|
||||
new DataView(buf).setUint16(9, val, false);
|
||||
}
|
||||
|
||||
+184
-184
@@ -1,184 +1,184 @@
|
||||
/**
|
||||
* AriaEngine Slot Directory — 页面内行槽位管理
|
||||
* @module engine/aria/page/slot
|
||||
*
|
||||
* Slot 目录从页面 Header 之后向下增长，每条记录 4 字节。
|
||||
*/
|
||||
|
||||
import {
|
||||
PAGE_HEADER_SIZE,
|
||||
SLOT_ENTRY_SIZE,
|
||||
PAGE_SIZE,
|
||||
type SlotEntry,
|
||||
} from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Slot 读写
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 读取 slot 号对应的 SlotEntry。
|
||||
*/
|
||||
export function getSlotEntry(buf: ArrayBuffer, slotIndex: number): SlotEntry {
|
||||
const offset = PAGE_HEADER_SIZE + slotIndex * SLOT_ENTRY_SIZE;
|
||||
const view = new DataView(buf);
|
||||
return {
|
||||
offset: view.getUint16(offset, false),
|
||||
length: view.getUint16(offset + 2, false),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 写入 slot 号对应的 SlotEntry。
|
||||
*/
|
||||
export function setSlotEntry(
|
||||
buf: ArrayBuffer,
|
||||
slotIndex: number,
|
||||
entry: SlotEntry,
|
||||
): void {
|
||||
const offset = PAGE_HEADER_SIZE + slotIndex * SLOT_ENTRY_SIZE;
|
||||
const view = new DataView(buf);
|
||||
view.setUint16(offset, entry.offset, false);
|
||||
view.setUint16(offset + 2, entry.length, false);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取页面中所有 slot 条目。
|
||||
*/
|
||||
export function getAllSlots(
|
||||
buf: ArrayBuffer,
|
||||
slotCount: number,
|
||||
): SlotEntry[] {
|
||||
const entries: SlotEntry[] = [];
|
||||
for (let i = 0; i < slotCount; i++) {
|
||||
entries.push(getSlotEntry(buf, i));
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Slot 空间计算
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 获取 slot 目录占用的总字节数 */
|
||||
export function getSlotDirectorySize(slotCount: number): number {
|
||||
return slotCount * SLOT_ENTRY_SIZE;
|
||||
}
|
||||
|
||||
/** 获取可用空闲空间（字节） */
|
||||
export function getFreeSpace(buf: ArrayBuffer): number {
|
||||
const view = new DataView(buf);
|
||||
const freeStart = view.getUint16(5, false); // slot 区之后
|
||||
const freeEnd = view.getUint16(7, false); // 数据区之前
|
||||
return freeEnd - freeStart;
|
||||
}
|
||||
|
||||
/** 检查是否有足够空间存放长度为 len 的行 */
|
||||
export function hasEnoughSpace(buf: ArrayBuffer, len: number): boolean {
|
||||
const slotCount = new DataView(buf).getUint16(9, false);
|
||||
const neededSlotSize = (slotCount + 1) * SLOT_ENTRY_SIZE;
|
||||
const freeStart = PAGE_HEADER_SIZE + neededSlotSize;
|
||||
const freeEnd = new DataView(buf).getUint16(7, false);
|
||||
return freeEnd - freeStart >= len;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 插入 / 删除 slot
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 在页面中分配一个 slot 并写入行数据。
|
||||
* 返回分配的 slot 索引，失败返回 -1。
|
||||
*/
|
||||
export function allocateSlot(
|
||||
buf: ArrayBuffer,
|
||||
rowData: Uint8Array,
|
||||
): number {
|
||||
const view = new DataView(buf);
|
||||
const slotCount = view.getUint16(9, false);
|
||||
const neededSlotSpace = (slotCount + 1) * SLOT_ENTRY_SIZE;
|
||||
const freeStart = PAGE_HEADER_SIZE + neededSlotSpace;
|
||||
const freeEnd = view.getUint16(7, false);
|
||||
|
||||
if (freeEnd - freeStart < rowData.byteLength) {
|
||||
return -1; // 空间不足
|
||||
}
|
||||
|
||||
// 将数据放入页面底部
|
||||
const dataOffset = freeEnd - rowData.byteLength;
|
||||
const dest = new Uint8Array(buf, dataOffset, rowData.byteLength);
|
||||
dest.set(rowData);
|
||||
|
||||
// 写入 slot 条目
|
||||
setSlotEntry(buf, slotCount, { offset: dataOffset, length: rowData.byteLength });
|
||||
|
||||
// 更新 header
|
||||
view.setUint16(5, freeStart, false); // freeStart
|
||||
view.setUint16(7, dataOffset, false); // freeEnd
|
||||
view.setUint16(9, slotCount + 1, false); // slotCount
|
||||
|
||||
return slotCount;
|
||||
}
|
||||
|
||||
/**
|
||||
* 从页面中删除指定 slot 的数据（标记为无效，offset 置 0）。
|
||||
* 注：简化实现，不做 slot 压缩。
|
||||
*/
|
||||
export function freeSlot(buf: ArrayBuffer, slotIndex: number): void {
|
||||
setSlotEntry(buf, slotIndex, { offset: 0, length: 0 });
|
||||
}
|
||||
|
||||
/**
|
||||
* 压缩页面槽位：移除已删除 slot，整理碎片空间。
|
||||
* 将有效数据紧凑排列，释放空洞。
|
||||
*/
|
||||
export function compactSlots(buf: ArrayBuffer): number {
|
||||
const view = new DataView(buf);
|
||||
const slotCount = view.getUint16(9, false);
|
||||
if (slotCount === 0) return 0;
|
||||
|
||||
// 收集有效 slot（offset>0 的）
|
||||
const validSlots: { index: number; offset: number; length: number; data: Uint8Array }[] = [];
|
||||
for (let i = 0; i < slotCount; i++) {
|
||||
const entry = getSlotEntry(buf, i);
|
||||
if (entry.offset > 0 && entry.length > 0) {
|
||||
const data = new Uint8Array(buf, entry.offset, entry.length);
|
||||
validSlots.push({ index: i, offset: entry.offset, length: entry.length, data: new Uint8Array(data) });
|
||||
}
|
||||
}
|
||||
|
||||
if (validSlots.length === slotCount) return 0; // 无碎片
|
||||
|
||||
// 从页面底部重新紧凑排列
|
||||
let dataEnd = PAGE_SIZE;
|
||||
const newSlots: { offset: number; length: number }[] = [];
|
||||
|
||||
for (let i = validSlots.length - 1; i >= 0; i--) {
|
||||
const s = validSlots[i];
|
||||
dataEnd -= s.length;
|
||||
new Uint8Array(buf).set(s.data, dataEnd);
|
||||
newSlots.unshift({ offset: dataEnd, length: s.length });
|
||||
}
|
||||
|
||||
// 重写 slot directory
|
||||
view.setUint16(9, validSlots.length, false); // slotCount
|
||||
view.setUint16(7, dataEnd, false); // freeEnd
|
||||
for (let i = 0; i < validSlots.length; i++) {
|
||||
setSlotEntry(buf, i, newSlots[i]);
|
||||
}
|
||||
// 清除剩余 slot 条目
|
||||
for (let i = validSlots.length; i < slotCount; i++) {
|
||||
setSlotEntry(buf, i, { offset: 0, length: 0 });
|
||||
}
|
||||
|
||||
return slotCount - validSlots.length; // 回收的 slot 数量
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取指定 slot 的行数据。
|
||||
*/
|
||||
export function readSlotData(buf: ArrayBuffer, slotIndex: number): Uint8Array | null {
|
||||
const entry = getSlotEntry(buf, slotIndex);
|
||||
if (entry.offset === 0 || entry.length === 0) return null;
|
||||
return new Uint8Array(buf, entry.offset, entry.length);
|
||||
}
|
||||
/**
|
||||
* AriaEngine Slot Directory — 页面内行槽位管理
|
||||
* @module engine/aria/page/slot
|
||||
*
|
||||
* Slot 目录从页面 Header 之后向下增长，每条记录 4 字节。
|
||||
*/
|
||||
|
||||
import {
|
||||
PAGE_HEADER_SIZE,
|
||||
SLOT_ENTRY_SIZE,
|
||||
PAGE_SIZE,
|
||||
type SlotEntry,
|
||||
} from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Slot 读写
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 读取 slot 号对应的 SlotEntry。
|
||||
*/
|
||||
export function getSlotEntry(buf: ArrayBuffer, slotIndex: number): SlotEntry {
|
||||
const offset = PAGE_HEADER_SIZE + slotIndex * SLOT_ENTRY_SIZE;
|
||||
const view = new DataView(buf);
|
||||
return {
|
||||
offset: view.getUint16(offset, false),
|
||||
length: view.getUint16(offset + 2, false),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 写入 slot 号对应的 SlotEntry。
|
||||
*/
|
||||
export function setSlotEntry(
|
||||
buf: ArrayBuffer,
|
||||
slotIndex: number,
|
||||
entry: SlotEntry,
|
||||
): void {
|
||||
const offset = PAGE_HEADER_SIZE + slotIndex * SLOT_ENTRY_SIZE;
|
||||
const view = new DataView(buf);
|
||||
view.setUint16(offset, entry.offset, false);
|
||||
view.setUint16(offset + 2, entry.length, false);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取页面中所有 slot 条目。
|
||||
*/
|
||||
export function getAllSlots(
|
||||
buf: ArrayBuffer,
|
||||
slotCount: number,
|
||||
): SlotEntry[] {
|
||||
const entries: SlotEntry[] = [];
|
||||
for (let i = 0; i < slotCount; i++) {
|
||||
entries.push(getSlotEntry(buf, i));
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Slot 空间计算
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 获取 slot 目录占用的总字节数 */
|
||||
export function getSlotDirectorySize(slotCount: number): number {
|
||||
return slotCount * SLOT_ENTRY_SIZE;
|
||||
}
|
||||
|
||||
/** 获取可用空闲空间（字节） */
|
||||
export function getFreeSpace(buf: ArrayBuffer): number {
|
||||
const view = new DataView(buf);
|
||||
const freeStart = view.getUint16(5, false); // slot 区之后
|
||||
const freeEnd = view.getUint16(7, false); // 数据区之前
|
||||
return freeEnd - freeStart;
|
||||
}
|
||||
|
||||
/** 检查是否有足够空间存放长度为 len 的行 */
|
||||
export function hasEnoughSpace(buf: ArrayBuffer, len: number): boolean {
|
||||
const slotCount = new DataView(buf).getUint16(9, false);
|
||||
const neededSlotSize = (slotCount + 1) * SLOT_ENTRY_SIZE;
|
||||
const freeStart = PAGE_HEADER_SIZE + neededSlotSize;
|
||||
const freeEnd = new DataView(buf).getUint16(7, false);
|
||||
return freeEnd - freeStart >= len;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 插入 / 删除 slot
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 在页面中分配一个 slot 并写入行数据。
|
||||
* 返回分配的 slot 索引，失败返回 -1。
|
||||
*/
|
||||
export function allocateSlot(
|
||||
buf: ArrayBuffer,
|
||||
rowData: Uint8Array,
|
||||
): number {
|
||||
const view = new DataView(buf);
|
||||
const slotCount = view.getUint16(9, false);
|
||||
const neededSlotSpace = (slotCount + 1) * SLOT_ENTRY_SIZE;
|
||||
const freeStart = PAGE_HEADER_SIZE + neededSlotSpace;
|
||||
const freeEnd = view.getUint16(7, false);
|
||||
|
||||
if (freeEnd - freeStart < rowData.byteLength) {
|
||||
return -1; // 空间不足
|
||||
}
|
||||
|
||||
// 将数据放入页面底部
|
||||
const dataOffset = freeEnd - rowData.byteLength;
|
||||
const dest = new Uint8Array(buf, dataOffset, rowData.byteLength);
|
||||
dest.set(rowData);
|
||||
|
||||
// 写入 slot 条目
|
||||
setSlotEntry(buf, slotCount, { offset: dataOffset, length: rowData.byteLength });
|
||||
|
||||
// 更新 header
|
||||
view.setUint16(5, freeStart, false); // freeStart
|
||||
view.setUint16(7, dataOffset, false); // freeEnd
|
||||
view.setUint16(9, slotCount + 1, false); // slotCount
|
||||
|
||||
return slotCount;
|
||||
}
|
||||
|
||||
/**
|
||||
* 从页面中删除指定 slot 的数据（标记为无效，offset 置 0）。
|
||||
* 注：简化实现，不做 slot 压缩。
|
||||
*/
|
||||
export function freeSlot(buf: ArrayBuffer, slotIndex: number): void {
|
||||
setSlotEntry(buf, slotIndex, { offset: 0, length: 0 });
|
||||
}
|
||||
|
||||
/**
|
||||
* 压缩页面槽位：移除已删除 slot，整理碎片空间。
|
||||
* 将有效数据紧凑排列，释放空洞。
|
||||
*/
|
||||
export function compactSlots(buf: ArrayBuffer): number {
|
||||
const view = new DataView(buf);
|
||||
const slotCount = view.getUint16(9, false);
|
||||
if (slotCount === 0) return 0;
|
||||
|
||||
// 收集有效 slot（offset>0 的）
|
||||
const validSlots: { index: number; offset: number; length: number; data: Uint8Array }[] = [];
|
||||
for (let i = 0; i < slotCount; i++) {
|
||||
const entry = getSlotEntry(buf, i);
|
||||
if (entry.offset > 0 && entry.length > 0) {
|
||||
const data = new Uint8Array(buf, entry.offset, entry.length);
|
||||
validSlots.push({ index: i, offset: entry.offset, length: entry.length, data: new Uint8Array(data) });
|
||||
}
|
||||
}
|
||||
|
||||
if (validSlots.length === slotCount) return 0; // 无碎片
|
||||
|
||||
// 从页面底部重新紧凑排列
|
||||
let dataEnd = PAGE_SIZE;
|
||||
const newSlots: { offset: number; length: number }[] = [];
|
||||
|
||||
for (let i = validSlots.length - 1; i >= 0; i--) {
|
||||
const s = validSlots[i];
|
||||
dataEnd -= s.length;
|
||||
new Uint8Array(buf).set(s.data, dataEnd);
|
||||
newSlots.unshift({ offset: dataEnd, length: s.length });
|
||||
}
|
||||
|
||||
// 重写 slot directory
|
||||
view.setUint16(9, validSlots.length, false); // slotCount
|
||||
view.setUint16(7, dataEnd, false); // freeEnd
|
||||
for (let i = 0; i < validSlots.length; i++) {
|
||||
setSlotEntry(buf, i, newSlots[i]);
|
||||
}
|
||||
// 清除剩余 slot 条目
|
||||
for (let i = validSlots.length; i < slotCount; i++) {
|
||||
setSlotEntry(buf, i, { offset: 0, length: 0 });
|
||||
}
|
||||
|
||||
return slotCount - validSlots.length; // 回收的 slot 数量
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取指定 slot 的行数据。
|
||||
*/
|
||||
export function readSlotData(buf: ArrayBuffer, slotIndex: number): Uint8Array | null {
|
||||
const entry = getSlotEntry(buf, slotIndex);
|
||||
if (entry.offset === 0 || entry.length === 0) return null;
|
||||
return new Uint8Array(buf, entry.offset, entry.length);
|
||||
}
|
||||
|
||||
+251
-252
@@ -1,252 +1,251 @@
|
||||
/**
|
||||
* AriaEngine Tuple Codec — 行数据的二进制编解码
|
||||
* @module engine/aria/page/tuple
|
||||
*
|
||||
* 将 Record<string, unknown> 编码为紧凑的二进制格式。
|
||||
*
|
||||
* 格式:
|
||||
* [null bitmap: ceil(colCount/8) bytes]
|
||||
* [column 1 data]
|
||||
* [column 2 data]
|
||||
* ...
|
||||
*
|
||||
* 每列:
|
||||
* type tag (u8) + data
|
||||
* - STRING: [len: u16][UTF-8 bytes]
|
||||
* - NUMBER: [f64: 8 bytes]
|
||||
* - BOOLEAN: [u8: 1 byte]
|
||||
* - DATE: [f64: 8 bytes] (epoch ms)
|
||||
* - JSON: [len: u16][UTF-8 bytes]
|
||||
* - NULL: (no data, just the tag)
|
||||
*/
|
||||
|
||||
import { ColumnEncoding } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 编码
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 将行数据编码为二进制字节数组。
|
||||
* @param row 行数据
|
||||
* @param columnOrder 列名顺序列表（决定编码顺序）
|
||||
* @param columnTypes 列名 → FieldType 映射
|
||||
*/
|
||||
export function encodeTuple(
|
||||
row: Record<string, unknown>,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Uint8Array {
|
||||
// 先计算总大小
|
||||
let size = 0;
|
||||
const nullBitmapBytes = Math.ceil(columnOrder.length / 8);
|
||||
size += nullBitmapBytes;
|
||||
|
||||
// 预计算每列编码后的字节
|
||||
const colData: (Uint8Array | null)[] = [];
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
const col = columnOrder[i];
|
||||
const val = row[col];
|
||||
const encoded = encodeColumn(val, columnTypes[col] ?? 'string');
|
||||
colData.push(encoded);
|
||||
if (encoded) {
|
||||
size += 1 + encoded.byteLength; // tag + data
|
||||
} else {
|
||||
size += 1; // just the NULL tag
|
||||
}
|
||||
}
|
||||
|
||||
const buf = new Uint8Array(size);
|
||||
const view = new DataView(buf.buffer, buf.byteOffset, buf.byteLength);
|
||||
let offset = 0;
|
||||
|
||||
// Null bitmap
|
||||
const nullBitmap = new Uint8Array(nullBitmapBytes);
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
if (colData[i] === null) {
|
||||
nullBitmap[Math.floor(i / 8)] |= (1 << (i % 8));
|
||||
}
|
||||
}
|
||||
buf.set(nullBitmap, offset);
|
||||
offset += nullBitmapBytes;
|
||||
|
||||
// Column data
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
const encoded = colData[i];
|
||||
if (encoded === null) {
|
||||
view.setUint8(offset, ColumnEncoding.NULL);
|
||||
offset += 1;
|
||||
} else {
|
||||
const tag = getEncodingTag(columnTypes[columnOrder[i]] ?? 'string');
|
||||
view.setUint8(offset, tag);
|
||||
offset += 1;
|
||||
buf.set(encoded, offset);
|
||||
offset += encoded.byteLength;
|
||||
}
|
||||
}
|
||||
|
||||
return buf;
|
||||
}
|
||||
|
||||
/**
|
||||
* 编码单个列的值。
|
||||
*/
|
||||
function encodeColumn(value: unknown, fieldType: string): Uint8Array | null {
|
||||
if (value === null || value === undefined) return null;
|
||||
|
||||
switch (fieldType) {
|
||||
case 'string': {
|
||||
const str = String(value);
|
||||
const encoder = new TextEncoder();
|
||||
const bytes = encoder.encode(str);
|
||||
const buf = new Uint8Array(2 + bytes.byteLength);
|
||||
new DataView(buf.buffer).setUint16(0, bytes.byteLength, false);
|
||||
buf.set(bytes, 2);
|
||||
return buf;
|
||||
}
|
||||
case 'number': {
|
||||
const buf = new ArrayBuffer(8);
|
||||
new DataView(buf).setFloat64(0, Number(value), false);
|
||||
return new Uint8Array(buf);
|
||||
}
|
||||
case 'boolean': {
|
||||
return new Uint8Array([value ? 1 : 0]);
|
||||
}
|
||||
case 'date': {
|
||||
const ts = value instanceof Date ? value.getTime() : new Date(String(value)).getTime();
|
||||
const buf = new ArrayBuffer(8);
|
||||
new DataView(buf).setFloat64(0, ts, false);
|
||||
return new Uint8Array(buf);
|
||||
}
|
||||
case 'json': {
|
||||
const str = JSON.stringify(value);
|
||||
const encoder = new TextEncoder();
|
||||
const bytes = encoder.encode(str);
|
||||
const buf = new Uint8Array(2 + bytes.byteLength);
|
||||
new DataView(buf.buffer).setUint16(0, bytes.byteLength, false);
|
||||
buf.set(bytes, 2);
|
||||
return buf;
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 解码
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 从二进制字节数组解码行数据。
|
||||
* @returns 行数据，如果格式错误返回 null
|
||||
*/
|
||||
export function decodeTuple(
|
||||
bytes: Uint8Array,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Record<string, unknown> | null {
|
||||
try {
|
||||
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
let offset = 0;
|
||||
|
||||
const nullBitmapBytes = Math.ceil(columnOrder.length / 8);
|
||||
if (offset + nullBitmapBytes > bytes.byteLength) return null;
|
||||
|
||||
const nullBitmap = bytes.slice(offset, offset + nullBitmapBytes);
|
||||
offset += nullBitmapBytes;
|
||||
|
||||
const row: Record<string, unknown> = {};
|
||||
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
if (offset >= bytes.byteLength) break;
|
||||
|
||||
const tag = view.getUint8(offset);
|
||||
offset += 1;
|
||||
|
||||
if (tag === ColumnEncoding.NULL) {
|
||||
row[columnOrder[i]] = null;
|
||||
continue;
|
||||
}
|
||||
|
||||
const col = columnOrder[i];
|
||||
const fType = columnTypes[col] ?? 'string';
|
||||
|
||||
const result = decodeColumnValue(bytes, offset, tag, fType);
|
||||
if (result === null) return null;
|
||||
row[col] = result.value;
|
||||
offset = result.nextOffset;
|
||||
}
|
||||
|
||||
return row;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function decodeColumnValue(
|
||||
bytes: Uint8Array,
|
||||
offset: number,
|
||||
tag: number,
|
||||
fieldType: string,
|
||||
): { value: unknown; nextOffset: number } | null {
|
||||
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
|
||||
switch (tag) {
|
||||
case ColumnEncoding.STRING:
|
||||
case ColumnEncoding.JSON: {
|
||||
if (offset + 2 > bytes.byteLength) return null;
|
||||
const len = view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
if (offset + len > bytes.byteLength) return null;
|
||||
const decoder = new TextDecoder();
|
||||
const str = decoder.decode(bytes.slice(offset, offset + len));
|
||||
return {
|
||||
value: tag === ColumnEncoding.JSON ? JSON.parse(str) : str,
|
||||
nextOffset: offset + len,
|
||||
};
|
||||
}
|
||||
case ColumnEncoding.NUMBER: {
|
||||
if (offset + 8 > bytes.byteLength) return null;
|
||||
const val = view.getFloat64(offset, false);
|
||||
return { value: val, nextOffset: offset + 8 };
|
||||
}
|
||||
case ColumnEncoding.BOOLEAN: {
|
||||
if (offset >= bytes.byteLength) return null;
|
||||
return { value: view.getUint8(offset) !== 0, nextOffset: offset + 1 };
|
||||
}
|
||||
case ColumnEncoding.DATE: {
|
||||
if (offset + 8 > bytes.byteLength) return null;
|
||||
const ts = view.getFloat64(offset, false);
|
||||
return { value: new Date(ts).toISOString(), nextOffset: offset + 8 };
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 辅助
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function getEncodingTag(fieldType: string): ColumnEncoding {
|
||||
switch (fieldType) {
|
||||
case 'string': return ColumnEncoding.STRING;
|
||||
case 'number': return ColumnEncoding.NUMBER;
|
||||
case 'boolean': return ColumnEncoding.BOOLEAN;
|
||||
case 'date': return ColumnEncoding.DATE;
|
||||
case 'json': return ColumnEncoding.JSON;
|
||||
default: return ColumnEncoding.NULL;
|
||||
}
|
||||
}
|
||||
|
||||
/** 获取列类型到编码标签的映射 */
|
||||
export function getColumnEncodingMap(
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Map<string, ColumnEncoding> {
|
||||
const map = new Map<string, ColumnEncoding>();
|
||||
for (const col of columnOrder) {
|
||||
map.set(col, getEncodingTag(columnTypes[col] ?? 'string'));
|
||||
}
|
||||
return map;
|
||||
}
|
||||
/**
|
||||
* AriaEngine Tuple Codec — 行数据的二进制编解码
|
||||
* @module engine/aria/page/tuple
|
||||
*
|
||||
* 将 Record<string, unknown> 编码为紧凑的二进制格式。
|
||||
*
|
||||
* 格式:
|
||||
* [null bitmap: ceil(colCount/8) bytes]
|
||||
* [column 1 data]
|
||||
* [column 2 data]
|
||||
* ...
|
||||
*
|
||||
* 每列:
|
||||
* type tag (u8) + data
|
||||
* - STRING: [len: u16][UTF-8 bytes]
|
||||
* - NUMBER: [f64: 8 bytes]
|
||||
* - BOOLEAN: [u8: 1 byte]
|
||||
* - DATE: [f64: 8 bytes] (epoch ms)
|
||||
* - JSON: [len: u16][UTF-8 bytes]
|
||||
* - NULL: (no data, just the tag)
|
||||
*/
|
||||
|
||||
import { ColumnEncoding } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 编码
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 将行数据编码为二进制字节数组。
|
||||
* @param row 行数据
|
||||
* @param columnOrder 列名顺序列表（决定编码顺序）
|
||||
* @param columnTypes 列名 → FieldType 映射
|
||||
*/
|
||||
export function encodeTuple(
|
||||
row: Record<string, unknown>,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Uint8Array {
|
||||
// 先计算总大小
|
||||
let size = 0;
|
||||
const nullBitmapBytes = Math.ceil(columnOrder.length / 8);
|
||||
size += nullBitmapBytes;
|
||||
|
||||
// 预计算每列编码后的字节
|
||||
const colData: (Uint8Array | null)[] = [];
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
const col = columnOrder[i];
|
||||
const val = row[col];
|
||||
const encoded = encodeColumn(val, columnTypes[col] ?? 'string');
|
||||
colData.push(encoded);
|
||||
if (encoded) {
|
||||
size += 1 + encoded.byteLength; // tag + data
|
||||
} else {
|
||||
size += 1; // just the NULL tag
|
||||
}
|
||||
}
|
||||
|
||||
const buf = new Uint8Array(size);
|
||||
const view = new DataView(buf.buffer, buf.byteOffset, buf.byteLength);
|
||||
let offset = 0;
|
||||
|
||||
// Null bitmap
|
||||
const nullBitmap = new Uint8Array(nullBitmapBytes);
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
if (colData[i] === null) {
|
||||
nullBitmap[Math.floor(i / 8)] |= (1 << (i % 8));
|
||||
}
|
||||
}
|
||||
buf.set(nullBitmap, offset);
|
||||
offset += nullBitmapBytes;
|
||||
|
||||
// Column data
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
const encoded = colData[i];
|
||||
if (encoded === null) {
|
||||
view.setUint8(offset, ColumnEncoding.NULL);
|
||||
offset += 1;
|
||||
} else {
|
||||
const tag = getEncodingTag(columnTypes[columnOrder[i]] ?? 'string');
|
||||
view.setUint8(offset, tag);
|
||||
offset += 1;
|
||||
buf.set(encoded, offset);
|
||||
offset += encoded.byteLength;
|
||||
}
|
||||
}
|
||||
|
||||
return buf;
|
||||
}
|
||||
|
||||
/**
|
||||
* 编码单个列的值。
|
||||
*/
|
||||
function encodeColumn(value: unknown, fieldType: string): Uint8Array | null {
|
||||
if (value === null || value === undefined) return null;
|
||||
|
||||
switch (fieldType) {
|
||||
case 'string': {
|
||||
const str = String(value);
|
||||
const encoder = new TextEncoder();
|
||||
const bytes = encoder.encode(str);
|
||||
const buf = new Uint8Array(2 + bytes.byteLength);
|
||||
new DataView(buf.buffer).setUint16(0, bytes.byteLength, false);
|
||||
buf.set(bytes, 2);
|
||||
return buf;
|
||||
}
|
||||
case 'number': {
|
||||
const buf = new ArrayBuffer(8);
|
||||
new DataView(buf).setFloat64(0, Number(value), false);
|
||||
return new Uint8Array(buf);
|
||||
}
|
||||
case 'boolean': {
|
||||
return new Uint8Array([value ? 1 : 0]);
|
||||
}
|
||||
case 'date': {
|
||||
const ts = value instanceof Date ? value.getTime() : new Date(String(value)).getTime();
|
||||
const buf = new ArrayBuffer(8);
|
||||
new DataView(buf).setFloat64(0, ts, false);
|
||||
return new Uint8Array(buf);
|
||||
}
|
||||
case 'json': {
|
||||
const str = JSON.stringify(value);
|
||||
const encoder = new TextEncoder();
|
||||
const bytes = encoder.encode(str);
|
||||
const buf = new Uint8Array(2 + bytes.byteLength);
|
||||
new DataView(buf.buffer).setUint16(0, bytes.byteLength, false);
|
||||
buf.set(bytes, 2);
|
||||
return buf;
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 解码
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 从二进制字节数组解码行数据。
|
||||
* @returns 行数据，如果格式错误返回 null
|
||||
*/
|
||||
export function decodeTuple(
|
||||
bytes: Uint8Array,
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Record<string, unknown> | null {
|
||||
try {
|
||||
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
let offset = 0;
|
||||
|
||||
const nullBitmapBytes = Math.ceil(columnOrder.length / 8);
|
||||
if (offset + nullBitmapBytes > bytes.byteLength) return null;
|
||||
|
||||
offset += nullBitmapBytes;
|
||||
|
||||
const row: Record<string, unknown> = {};
|
||||
|
||||
for (let i = 0; i < columnOrder.length; i++) {
|
||||
if (offset >= bytes.byteLength) break;
|
||||
|
||||
const tag = view.getUint8(offset);
|
||||
offset += 1;
|
||||
|
||||
if (tag === ColumnEncoding.NULL) {
|
||||
row[columnOrder[i]] = null;
|
||||
continue;
|
||||
}
|
||||
|
||||
const col = columnOrder[i];
|
||||
const fType = columnTypes[col] ?? 'string';
|
||||
|
||||
const result = decodeColumnValue(bytes, offset, tag, fType);
|
||||
if (result === null) return null;
|
||||
row[col] = result.value;
|
||||
offset = result.nextOffset;
|
||||
}
|
||||
|
||||
return row;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function decodeColumnValue(
|
||||
bytes: Uint8Array,
|
||||
offset: number,
|
||||
tag: number,
|
||||
_fieldType: string,
|
||||
): { value: unknown; nextOffset: number } | null {
|
||||
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
|
||||
switch (tag) {
|
||||
case ColumnEncoding.STRING:
|
||||
case ColumnEncoding.JSON: {
|
||||
if (offset + 2 > bytes.byteLength) return null;
|
||||
const len = view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
if (offset + len > bytes.byteLength) return null;
|
||||
const decoder = new TextDecoder();
|
||||
const str = decoder.decode(bytes.slice(offset, offset + len));
|
||||
return {
|
||||
value: tag === ColumnEncoding.JSON ? JSON.parse(str) : str,
|
||||
nextOffset: offset + len,
|
||||
};
|
||||
}
|
||||
case ColumnEncoding.NUMBER: {
|
||||
if (offset + 8 > bytes.byteLength) return null;
|
||||
const val = view.getFloat64(offset, false);
|
||||
return { value: val, nextOffset: offset + 8 };
|
||||
}
|
||||
case ColumnEncoding.BOOLEAN: {
|
||||
if (offset >= bytes.byteLength) return null;
|
||||
return { value: view.getUint8(offset) !== 0, nextOffset: offset + 1 };
|
||||
}
|
||||
case ColumnEncoding.DATE: {
|
||||
if (offset + 8 > bytes.byteLength) return null;
|
||||
const ts = view.getFloat64(offset, false);
|
||||
return { value: new Date(ts).toISOString(), nextOffset: offset + 8 };
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 辅助
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function getEncodingTag(fieldType: string): ColumnEncoding {
|
||||
switch (fieldType) {
|
||||
case 'string': return ColumnEncoding.STRING;
|
||||
case 'number': return ColumnEncoding.NUMBER;
|
||||
case 'boolean': return ColumnEncoding.BOOLEAN;
|
||||
case 'date': return ColumnEncoding.DATE;
|
||||
case 'json': return ColumnEncoding.JSON;
|
||||
default: return ColumnEncoding.NULL;
|
||||
}
|
||||
}
|
||||
|
||||
/** 获取列类型到编码标签的映射 */
|
||||
export function getColumnEncodingMap(
|
||||
columnOrder: string[],
|
||||
columnTypes: Record<string, string>,
|
||||
): Map<string, ColumnEncoding> {
|
||||
const map = new Map<string, ColumnEncoding>();
|
||||
for (const col of columnOrder) {
|
||||
map.set(col, getEncodingTag(columnTypes[col] ?? 'string'));
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
+179
-179
@@ -1,179 +1,179 @@
|
||||
/**
|
||||
* AriaEngine Storage Backend — 存储后端抽象层
|
||||
* @module engine/aria/store/backend
|
||||
*
|
||||
* 封装底层浏览器存储 API（IndexedDB / OPFS / Memory 回退），
|
||||
* 供 Buffer Pool 的 PageIO 和 WAL 的 WALStore 使用。
|
||||
*/
|
||||
|
||||
import { DatabaseError } from '../../../constants';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// StorageBackend 接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface IStorageBackend {
|
||||
/** 打开存储 */
|
||||
open(name: string): Promise<void>;
|
||||
/** 关闭存储 */
|
||||
close(): Promise<void>;
|
||||
/** 是否已打开 */
|
||||
isOpen(): boolean;
|
||||
/** 读取数据块 */
|
||||
read(key: string): Promise<ArrayBuffer | null>;
|
||||
/** 写入数据块 */
|
||||
write(key: string, data: ArrayBuffer): Promise<void>;
|
||||
/** 删除数据块 */
|
||||
delete(key: string): Promise<void>;
|
||||
/** 列出所有 key */
|
||||
listKeys(): Promise<string[]>;
|
||||
/** 检查 key 是否存在 */
|
||||
exists(key: string): Promise<boolean>;
|
||||
/** 清空所有数据 */
|
||||
clear(): Promise<void>;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// IndexedDB Backend
|
||||
// =======================================================================
|
||||
|
||||
export class IndexedDBBackend implements IStorageBackend {
|
||||
private db: IDBDatabase | null = null;
|
||||
private dbName = '';
|
||||
private storeName = 'data';
|
||||
|
||||
async open(name: string): Promise<void> {
|
||||
this.dbName = `aria-${name}`;
|
||||
return new Promise((resolve, reject) => {
|
||||
const request = indexedDB.open(this.dbName, 1);
|
||||
request.onupgradeneeded = () => {
|
||||
const db = request.result;
|
||||
if (!db.objectStoreNames.contains(this.storeName)) {
|
||||
db.createObjectStore(this.storeName);
|
||||
}
|
||||
};
|
||||
request.onsuccess = () => {
|
||||
this.db = request.result;
|
||||
resolve();
|
||||
};
|
||||
request.onerror = () => reject(new DatabaseError('Failed to open AriaEngine IndexedDB', 'ARIA_IDB_OPEN_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
if (this.db) {
|
||||
this.db.close();
|
||||
this.db = null;
|
||||
}
|
||||
}
|
||||
|
||||
isOpen(): boolean {
|
||||
return this.db !== null;
|
||||
}
|
||||
|
||||
async read(key: string): Promise<ArrayBuffer | null> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readonly');
|
||||
const req = tx.objectStore(this.storeName).get(key);
|
||||
req.onsuccess = () => resolve(req.result ?? null);
|
||||
req.onerror = () => reject(new DatabaseError('Failed to read from AriaEngine store', 'ARIA_READ_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async write(key: string, data: ArrayBuffer): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readwrite');
|
||||
tx.objectStore(this.storeName).put(data, key);
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(new DatabaseError('Failed to write to AriaEngine store', 'ARIA_WRITE_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async delete(key: string): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readwrite');
|
||||
tx.objectStore(this.storeName).delete(key);
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(new DatabaseError('Failed to delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async listKeys(): Promise<string[]> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readonly');
|
||||
const req = tx.objectStore(this.storeName).getAllKeys();
|
||||
req.onsuccess = () => resolve((req.result ?? []) as string[]);
|
||||
req.onerror = () => reject(new DatabaseError('Failed to list keys', 'ARIA_LIST_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async exists(key: string): Promise<boolean> {
|
||||
const result = await this.read(key);
|
||||
return result !== null;
|
||||
}
|
||||
|
||||
async clear(): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readwrite');
|
||||
tx.objectStore(this.storeName).clear();
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(new DatabaseError('Failed to clear AriaEngine store', 'ARIA_CLEAR_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
private ensureDB(): IDBDatabase {
|
||||
if (!this.db) throw new DatabaseError('AriaEngine storage not opened', 'ARIA_DB_NOT_OPEN');
|
||||
return this.db;
|
||||
}
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// Memory Backend（回退 / 测试用）
|
||||
// =======================================================================
|
||||
|
||||
export class MemoryBackend implements IStorageBackend {
|
||||
private store: Map<string, ArrayBuffer> = new Map();
|
||||
private opened = false;
|
||||
|
||||
async open(_name: string): Promise<void> {
|
||||
this.opened = true;
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
this.store.clear();
|
||||
this.opened = false;
|
||||
}
|
||||
|
||||
isOpen(): boolean {
|
||||
return this.opened;
|
||||
}
|
||||
|
||||
async read(key: string): Promise<ArrayBuffer | null> {
|
||||
return this.store.get(key) ?? null;
|
||||
}
|
||||
|
||||
async write(key: string, data: ArrayBuffer): Promise<void> {
|
||||
this.store.set(key, data);
|
||||
}
|
||||
|
||||
async delete(key: string): Promise<void> {
|
||||
this.store.delete(key);
|
||||
}
|
||||
|
||||
async listKeys(): Promise<string[]> {
|
||||
return Array.from(this.store.keys());
|
||||
}
|
||||
|
||||
async exists(key: string): Promise<boolean> {
|
||||
return this.store.has(key);
|
||||
}
|
||||
|
||||
async clear(): Promise<void> {
|
||||
this.store.clear();
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine Storage Backend — 存储后端抽象层
|
||||
* @module engine/aria/store/backend
|
||||
*
|
||||
* 封装底层浏览器存储 API（IndexedDB / OPFS / Memory 回退），
|
||||
* 供 Buffer Pool 的 PageIO 和 WAL 的 WALStore 使用。
|
||||
*/
|
||||
|
||||
import { DatabaseError } from '../../../constants';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// StorageBackend 接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface IStorageBackend {
|
||||
/** 打开存储 */
|
||||
open(name: string): Promise<void>;
|
||||
/** 关闭存储 */
|
||||
close(): Promise<void>;
|
||||
/** 是否已打开 */
|
||||
isOpen(): boolean;
|
||||
/** 读取数据块 */
|
||||
read(key: string): Promise<ArrayBuffer | null>;
|
||||
/** 写入数据块 */
|
||||
write(key: string, data: ArrayBuffer): Promise<void>;
|
||||
/** 删除数据块 */
|
||||
delete(key: string): Promise<void>;
|
||||
/** 列出所有 key */
|
||||
listKeys(): Promise<string[]>;
|
||||
/** 检查 key 是否存在 */
|
||||
exists(key: string): Promise<boolean>;
|
||||
/** 清空所有数据 */
|
||||
clear(): Promise<void>;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// IndexedDB Backend
|
||||
// =======================================================================
|
||||
|
||||
export class IndexedDBBackend implements IStorageBackend {
|
||||
private db: IDBDatabase | null = null;
|
||||
private dbName = '';
|
||||
private storeName = 'data';
|
||||
|
||||
async open(name: string): Promise<void> {
|
||||
this.dbName = `aria-${name}`;
|
||||
return new Promise((resolve, reject) => {
|
||||
const request = indexedDB.open(this.dbName, 1);
|
||||
request.onupgradeneeded = () => {
|
||||
const db = request.result;
|
||||
if (!db.objectStoreNames.contains(this.storeName)) {
|
||||
db.createObjectStore(this.storeName);
|
||||
}
|
||||
};
|
||||
request.onsuccess = () => {
|
||||
this.db = request.result;
|
||||
resolve();
|
||||
};
|
||||
request.onerror = () => reject(new DatabaseError('Failed to open AriaEngine IndexedDB', 'ARIA_IDB_OPEN_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
if (this.db) {
|
||||
this.db.close();
|
||||
this.db = null;
|
||||
}
|
||||
}
|
||||
|
||||
isOpen(): boolean {
|
||||
return this.db !== null;
|
||||
}
|
||||
|
||||
async read(key: string): Promise<ArrayBuffer | null> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readonly');
|
||||
const req = tx.objectStore(this.storeName).get(key);
|
||||
req.onsuccess = () => resolve(req.result ?? null);
|
||||
req.onerror = () => reject(new DatabaseError('Failed to read from AriaEngine store', 'ARIA_READ_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async write(key: string, data: ArrayBuffer): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readwrite');
|
||||
tx.objectStore(this.storeName).put(data, key);
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(new DatabaseError('Failed to write to AriaEngine store', 'ARIA_WRITE_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async delete(key: string): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readwrite');
|
||||
tx.objectStore(this.storeName).delete(key);
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(new DatabaseError('Failed to delete from AriaEngine store', 'ARIA_DELETE_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async listKeys(): Promise<string[]> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readonly');
|
||||
const req = tx.objectStore(this.storeName).getAllKeys();
|
||||
req.onsuccess = () => resolve((req.result ?? []) as string[]);
|
||||
req.onerror = () => reject(new DatabaseError('Failed to list keys', 'ARIA_LIST_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
async exists(key: string): Promise<boolean> {
|
||||
const result = await this.read(key);
|
||||
return result !== null;
|
||||
}
|
||||
|
||||
async clear(): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const tx = db.transaction(this.storeName, 'readwrite');
|
||||
tx.objectStore(this.storeName).clear();
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(new DatabaseError('Failed to clear AriaEngine store', 'ARIA_CLEAR_ERROR'));
|
||||
});
|
||||
}
|
||||
|
||||
private ensureDB(): IDBDatabase {
|
||||
if (!this.db) throw new DatabaseError('AriaEngine storage not opened', 'ARIA_DB_NOT_OPEN');
|
||||
return this.db;
|
||||
}
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// Memory Backend（回退 / 测试用）
|
||||
// =======================================================================
|
||||
|
||||
export class MemoryBackend implements IStorageBackend {
|
||||
private store: Map<string, ArrayBuffer> = new Map();
|
||||
private opened = false;
|
||||
|
||||
async open(_name: string): Promise<void> {
|
||||
this.opened = true;
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
this.store.clear();
|
||||
this.opened = false;
|
||||
}
|
||||
|
||||
isOpen(): boolean {
|
||||
return this.opened;
|
||||
}
|
||||
|
||||
async read(key: string): Promise<ArrayBuffer | null> {
|
||||
return this.store.get(key) ?? null;
|
||||
}
|
||||
|
||||
async write(key: string, data: ArrayBuffer): Promise<void> {
|
||||
this.store.set(key, data);
|
||||
}
|
||||
|
||||
async delete(key: string): Promise<void> {
|
||||
this.store.delete(key);
|
||||
}
|
||||
|
||||
async listKeys(): Promise<string[]> {
|
||||
return Array.from(this.store.keys());
|
||||
}
|
||||
|
||||
async exists(key: string): Promise<boolean> {
|
||||
return this.store.has(key);
|
||||
}
|
||||
|
||||
async clear(): Promise<void> {
|
||||
this.store.clear();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,111 +1,111 @@
|
||||
/**
|
||||
* AriaEngine File Manager — 页面文件管理 + PageIO 实现
|
||||
* @module engine/aria/store/file_manager
|
||||
*
|
||||
* 负责管理页面文件的生命周期：分配/释放页面 ID，读写页面。
|
||||
*/
|
||||
|
||||
import type { IStorageBackend } from './backend';
|
||||
import type { PageIO } from '../buffer/pool';
|
||||
import { PAGE_SIZE, PageType } from '../types';
|
||||
import { initPageHeader } from '../page/header';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// FileManager (implements PageIO)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class FileManager implements PageIO {
|
||||
private backend: IStorageBackend;
|
||||
private nextPageId = 0;
|
||||
private metaLoaded = false;
|
||||
private dbName = '';
|
||||
|
||||
constructor(backend: IStorageBackend) {
|
||||
this.backend = backend;
|
||||
}
|
||||
|
||||
/** 初始化：从存储中读取元数据 */
|
||||
async init(dbName: string): Promise<void> {
|
||||
this.dbName = dbName;
|
||||
const meta = await this.backend.read('__aria_meta');
|
||||
if (meta && meta instanceof ArrayBuffer && meta.byteLength >= 4) {
|
||||
const view = new DataView(meta);
|
||||
this.nextPageId = view.getUint32(0, false);
|
||||
} else {
|
||||
this.nextPageId = 1;
|
||||
await this.saveMeta();
|
||||
}
|
||||
this.metaLoaded = true;
|
||||
}
|
||||
|
||||
// ---- PageIO ----
|
||||
|
||||
async readPage(pageId: number): Promise<ArrayBuffer | null> {
|
||||
const key = `pg_${pageId}`;
|
||||
const data = await this.backend.read(key);
|
||||
if (!data) {
|
||||
// 第一次访问：创建新页面
|
||||
return this.createEmptyPage(pageId, PageType.DATA);
|
||||
}
|
||||
|
||||
// 确保大小正确
|
||||
if (data.byteLength < PAGE_SIZE) {
|
||||
const padded = new ArrayBuffer(PAGE_SIZE);
|
||||
new Uint8Array(padded).set(new Uint8Array(data));
|
||||
return padded;
|
||||
}
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
async writePage(pageId: number, data: ArrayBuffer): Promise<void> {
|
||||
const key = `pg_${pageId}`;
|
||||
await this.backend.write(key, data);
|
||||
}
|
||||
|
||||
async allocatePageId(): Promise<number> {
|
||||
const id = this.nextPageId++;
|
||||
await this.saveMeta();
|
||||
return id;
|
||||
}
|
||||
|
||||
async freePageId(_pageId: number): Promise<void> {
|
||||
// 简化实现：不回收 pageId
|
||||
const key = `pg_${_pageId}`;
|
||||
await this.backend.delete(key);
|
||||
}
|
||||
|
||||
// ---- 表页面分配 ----
|
||||
|
||||
/**
|
||||
* 分配一个新的表元数据页面。
|
||||
*/
|
||||
async allocateTableRootPage(): Promise<number> {
|
||||
const pageId = await this.allocatePageId();
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, pageId, PageType.META);
|
||||
await this.writePage(pageId, data);
|
||||
return pageId;
|
||||
}
|
||||
|
||||
// ---- 辅助 ----
|
||||
|
||||
private async saveMeta(): Promise<void> {
|
||||
const buf = new ArrayBuffer(8);
|
||||
new DataView(buf).setUint32(0, this.nextPageId, false);
|
||||
await this.backend.write('__aria_meta', buf);
|
||||
}
|
||||
|
||||
private createEmptyPage(pageId: number, type: PageType): ArrayBuffer {
|
||||
const buf = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(buf, pageId, type);
|
||||
return buf;
|
||||
}
|
||||
|
||||
/** 清空所有数据 */
|
||||
async clearAll(): Promise<void> {
|
||||
await this.backend.clear();
|
||||
this.nextPageId = 1;
|
||||
await this.saveMeta();
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine File Manager — 页面文件管理 + PageIO 实现
|
||||
* @module engine/aria/store/file_manager
|
||||
*
|
||||
* 负责管理页面文件的生命周期：分配/释放页面 ID，读写页面。
|
||||
*/
|
||||
|
||||
import type { IStorageBackend } from './backend';
|
||||
import type { PageIO } from '../buffer/pool';
|
||||
import { PAGE_SIZE, PageType } from '../types';
|
||||
import { initPageHeader } from '../page/header';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// FileManager (implements PageIO)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class FileManager implements PageIO {
|
||||
private backend: IStorageBackend;
|
||||
private nextPageId = 0;
|
||||
private metaLoaded = false;
|
||||
private dbName = '';
|
||||
|
||||
constructor(backend: IStorageBackend) {
|
||||
this.backend = backend;
|
||||
}
|
||||
|
||||
/** 初始化：从存储中读取元数据 */
|
||||
async init(dbName: string): Promise<void> {
|
||||
this.dbName = dbName;
|
||||
const meta = await this.backend.read('__aria_meta');
|
||||
if (meta && meta instanceof ArrayBuffer && meta.byteLength >= 4) {
|
||||
const view = new DataView(meta);
|
||||
this.nextPageId = view.getUint32(0, false);
|
||||
} else {
|
||||
this.nextPageId = 1;
|
||||
await this.saveMeta();
|
||||
}
|
||||
this.metaLoaded = true;
|
||||
}
|
||||
|
||||
// ---- PageIO ----
|
||||
|
||||
async readPage(pageId: number): Promise<ArrayBuffer | null> {
|
||||
const key = `pg_${pageId}`;
|
||||
const data = await this.backend.read(key);
|
||||
if (!data) {
|
||||
// 第一次访问：创建新页面
|
||||
return this.createEmptyPage(pageId, PageType.DATA);
|
||||
}
|
||||
|
||||
// 确保大小正确
|
||||
if (data.byteLength < PAGE_SIZE) {
|
||||
const padded = new ArrayBuffer(PAGE_SIZE);
|
||||
new Uint8Array(padded).set(new Uint8Array(data));
|
||||
return padded;
|
||||
}
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
async writePage(pageId: number, data: ArrayBuffer): Promise<void> {
|
||||
const key = `pg_${pageId}`;
|
||||
await this.backend.write(key, data);
|
||||
}
|
||||
|
||||
async allocatePageId(): Promise<number> {
|
||||
const id = this.nextPageId++;
|
||||
await this.saveMeta();
|
||||
return id;
|
||||
}
|
||||
|
||||
async freePageId(_pageId: number): Promise<void> {
|
||||
// 简化实现：不回收 pageId
|
||||
const key = `pg_${_pageId}`;
|
||||
await this.backend.delete(key);
|
||||
}
|
||||
|
||||
// ---- 表页面分配 ----
|
||||
|
||||
/**
|
||||
* 分配一个新的表元数据页面。
|
||||
*/
|
||||
async allocateTableRootPage(): Promise<number> {
|
||||
const pageId = await this.allocatePageId();
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, pageId, PageType.META);
|
||||
await this.writePage(pageId, data);
|
||||
return pageId;
|
||||
}
|
||||
|
||||
// ---- 辅助 ----
|
||||
|
||||
private async saveMeta(): Promise<void> {
|
||||
const buf = new ArrayBuffer(8);
|
||||
new DataView(buf).setUint32(0, this.nextPageId, false);
|
||||
await this.backend.write('__aria_meta', buf);
|
||||
}
|
||||
|
||||
private createEmptyPage(pageId: number, type: PageType): ArrayBuffer {
|
||||
const buf = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(buf, pageId, type);
|
||||
return buf;
|
||||
}
|
||||
|
||||
/** 清空所有数据 */
|
||||
async clearAll(): Promise<void> {
|
||||
await this.backend.clear();
|
||||
this.nextPageId = 1;
|
||||
await this.saveMeta();
|
||||
}
|
||||
}
|
||||
|
||||
+243
-243
@@ -1,243 +1,243 @@
|
||||
/**
|
||||
* AriaEngine MVCC — 多版本并发控制
|
||||
* @module engine/aria/transaction/mvcc
|
||||
*
|
||||
* 实现快照隔离 (Snapshot Isolation)。
|
||||
* 每个事务看到数据库在事务开始时的快照。
|
||||
*/
|
||||
|
||||
import type { RowVersion, TxnEntry } from '../types';
|
||||
import { TransactionState } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MVCCManager
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class MVCCManager {
|
||||
/** 所有行版本的存储：tableName.key → 版本链 */
|
||||
private versionStore: Map<string, RowVersion[]> = new Map();
|
||||
|
||||
/** 活跃事务表：txnId → TxnEntry */
|
||||
private activeTxns: Map<number, TxnEntry> = new Map();
|
||||
|
||||
/** 事务 ID 计数器 */
|
||||
private nextTxnId = 1;
|
||||
|
||||
/** 全局提交序列号（用于可见性判断） */
|
||||
private globalCommitLsn = 0;
|
||||
|
||||
// =======================================================================
|
||||
// 事务管理
|
||||
// =======================================================================
|
||||
|
||||
/** 开始一个事务，返回事务 ID */
|
||||
beginTransaction(): number {
|
||||
const txnId = this.nextTxnId++;
|
||||
this.activeTxns.set(txnId, {
|
||||
txnId,
|
||||
state: TransactionState.ACTIVE,
|
||||
snapshotLsn: this.globalCommitLsn,
|
||||
startTime: Date.now(),
|
||||
});
|
||||
return txnId;
|
||||
}
|
||||
|
||||
/** 提交事务 */
|
||||
commitTransaction(txnId: number): void {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
if (!txn) throw new Error(`Transaction ${txnId} not found`);
|
||||
|
||||
txn.state = TransactionState.COMMITTED;
|
||||
this.globalCommitLsn++;
|
||||
|
||||
// 标记此事务写入的所有版本为已提交
|
||||
for (const [, versions] of this.versionStore) {
|
||||
for (const version of versions) {
|
||||
if (version.txnId === txnId) {
|
||||
version.committed = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 清理已提交事务的记录
|
||||
this.activeTxns.delete(txnId);
|
||||
}
|
||||
|
||||
/** 回滚事务 */
|
||||
rollbackTransaction(txnId: number): void {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
if (!txn) throw new Error(`Transaction ${txnId} not found`);
|
||||
|
||||
txn.state = TransactionState.ABORTED;
|
||||
|
||||
// 移除此事务写入的所有版本
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
const filtered = versions.filter((v) => v.txnId !== txnId);
|
||||
if (filtered.length === 0) {
|
||||
this.versionStore.delete(tableKey);
|
||||
} else {
|
||||
this.versionStore.set(tableKey, filtered);
|
||||
}
|
||||
}
|
||||
|
||||
this.activeTxns.delete(txnId);
|
||||
}
|
||||
|
||||
/** 检查事务是否活跃 */
|
||||
isActive(txnId: number): boolean {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
return txn !== undefined && txn.state === TransactionState.ACTIVE;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 版本读写
|
||||
// =======================================================================
|
||||
|
||||
/**
|
||||
* 写入一行（创建新版本）。
|
||||
*/
|
||||
writeVersion(
|
||||
tableName: string,
|
||||
key: string,
|
||||
data: Record<string, unknown>,
|
||||
txnId: number,
|
||||
): void {
|
||||
const tableKey = `${tableName}.${key}`;
|
||||
const versions = this.versionStore.get(tableKey) ?? [];
|
||||
|
||||
const newVersion: RowVersion = {
|
||||
txnId,
|
||||
data,
|
||||
prevVersion: versions.length > 0 ? versions[versions.length - 1] : null,
|
||||
committed: false,
|
||||
};
|
||||
|
||||
versions.push(newVersion);
|
||||
this.versionStore.set(tableKey, versions);
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取一行（对指定事务可见的最新版本）。
|
||||
*/
|
||||
readVersion(
|
||||
tableName: string,
|
||||
key: string,
|
||||
txnId: number,
|
||||
): Record<string, unknown> | null {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
if (!txn) return null;
|
||||
|
||||
const tableKey = `${tableName}.${key}`;
|
||||
const versions = this.versionStore.get(tableKey);
|
||||
if (!versions || versions.length === 0) return null;
|
||||
|
||||
// 从最新版本向前遍历
|
||||
for (let i = versions.length - 1; i >= 0; i--) {
|
||||
const version = versions[i];
|
||||
|
||||
// 1. 如果是当前事务写入的（未提交），可见
|
||||
if (version.txnId === txnId) {
|
||||
return version.data;
|
||||
}
|
||||
|
||||
// 2. 如果是已提交的版本，且在快照 LSN 之前提交，可见
|
||||
if (version.committed) {
|
||||
// 简化：所有已提交版本都可见
|
||||
return version.data;
|
||||
}
|
||||
|
||||
// 3. 其他事务的未提交版本，不可见，继续找更早的版本
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* 删除一行（创建墓碑版本）。
|
||||
*/
|
||||
deleteVersion(tableName: string, key: string, txnId: number): void {
|
||||
this.writeVersion(tableName, key, { __mvcc_tombstone: true } as unknown as Record<string, unknown>, txnId);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取所有行的最新已提交版本（用于非事务读取）。
|
||||
*/
|
||||
getLatestCommittedVersions(
|
||||
tableName: string,
|
||||
): Record<string, Record<string, unknown>> {
|
||||
const result: Record<string, Record<string, unknown>> = {};
|
||||
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
if (!tableKey.startsWith(`${tableName}.`)) continue;
|
||||
const key = tableKey.slice(tableName.length + 1);
|
||||
|
||||
for (let i = versions.length - 1; i >= 0; i--) {
|
||||
const version = versions[i];
|
||||
if (version.committed && !(version.data as unknown as Record<string, unknown>).__mvcc_tombstone) {
|
||||
result[key] = version.data;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* 清理过旧版本（GC）。
|
||||
* 保留每个 key 的最新 N 个已提交版本。
|
||||
*/
|
||||
gc(maxVersionsPerKey: number = 100): void {
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
if (versions.length <= maxVersionsPerKey) continue;
|
||||
|
||||
// 保留最新的 maxVersionsPerKey 个版本
|
||||
const pruned = versions.slice(versions.length - maxVersionsPerKey);
|
||||
this.versionStore.set(tableKey, pruned);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取所有未提交事务中的 key 列表。
|
||||
*/
|
||||
getActiveWriteKeys(tableName: string, txnId: number): Set<string> {
|
||||
const keys = new Set<string>();
|
||||
const prefix = `${tableName}.`;
|
||||
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
if (!tableKey.startsWith(prefix)) continue;
|
||||
const latestVersion = versions[versions.length - 1];
|
||||
if (latestVersion.txnId === txnId && !latestVersion.committed) {
|
||||
keys.add(tableKey.slice(prefix.length));
|
||||
}
|
||||
}
|
||||
|
||||
return keys;
|
||||
}
|
||||
|
||||
/**
|
||||
* 清理指定表的所有版本。
|
||||
*/
|
||||
clearTable(tableName: string): void {
|
||||
const prefix = `${tableName}.`;
|
||||
for (const [tableKey] of this.versionStore) {
|
||||
if (tableKey.startsWith(prefix)) {
|
||||
this.versionStore.delete(tableKey);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取活跃事务数。
|
||||
*/
|
||||
getActiveTxnCount(): number {
|
||||
return this.activeTxns.size;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取全局 LSN。
|
||||
*/
|
||||
getGlobalLSN(): number {
|
||||
return this.globalCommitLsn;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine MVCC — 多版本并发控制
|
||||
* @module engine/aria/transaction/mvcc
|
||||
*
|
||||
* 实现快照隔离 (Snapshot Isolation)。
|
||||
* 每个事务看到数据库在事务开始时的快照。
|
||||
*/
|
||||
|
||||
import type { RowVersion, TxnEntry } from '../types';
|
||||
import { TransactionState } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MVCCManager
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class MVCCManager {
|
||||
/** 所有行版本的存储：tableName.key → 版本链 */
|
||||
private versionStore: Map<string, RowVersion[]> = new Map();
|
||||
|
||||
/** 活跃事务表：txnId → TxnEntry */
|
||||
private activeTxns: Map<number, TxnEntry> = new Map();
|
||||
|
||||
/** 事务 ID 计数器 */
|
||||
private nextTxnId = 1;
|
||||
|
||||
/** 全局提交序列号（用于可见性判断） */
|
||||
private globalCommitLsn = 0;
|
||||
|
||||
// =======================================================================
|
||||
// 事务管理
|
||||
// =======================================================================
|
||||
|
||||
/** 开始一个事务，返回事务 ID */
|
||||
beginTransaction(): number {
|
||||
const txnId = this.nextTxnId++;
|
||||
this.activeTxns.set(txnId, {
|
||||
txnId,
|
||||
state: TransactionState.ACTIVE,
|
||||
snapshotLsn: this.globalCommitLsn,
|
||||
startTime: Date.now(),
|
||||
});
|
||||
return txnId;
|
||||
}
|
||||
|
||||
/** 提交事务 */
|
||||
commitTransaction(txnId: number): void {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
if (!txn) throw new Error(`Transaction ${txnId} not found`);
|
||||
|
||||
txn.state = TransactionState.COMMITTED;
|
||||
this.globalCommitLsn++;
|
||||
|
||||
// 标记此事务写入的所有版本为已提交
|
||||
for (const [, versions] of this.versionStore) {
|
||||
for (const version of versions) {
|
||||
if (version.txnId === txnId) {
|
||||
version.committed = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 清理已提交事务的记录
|
||||
this.activeTxns.delete(txnId);
|
||||
}
|
||||
|
||||
/** 回滚事务 */
|
||||
rollbackTransaction(txnId: number): void {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
if (!txn) throw new Error(`Transaction ${txnId} not found`);
|
||||
|
||||
txn.state = TransactionState.ABORTED;
|
||||
|
||||
// 移除此事务写入的所有版本
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
const filtered = versions.filter((v) => v.txnId !== txnId);
|
||||
if (filtered.length === 0) {
|
||||
this.versionStore.delete(tableKey);
|
||||
} else {
|
||||
this.versionStore.set(tableKey, filtered);
|
||||
}
|
||||
}
|
||||
|
||||
this.activeTxns.delete(txnId);
|
||||
}
|
||||
|
||||
/** 检查事务是否活跃 */
|
||||
isActive(txnId: number): boolean {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
return txn !== undefined && txn.state === TransactionState.ACTIVE;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 版本读写
|
||||
// =======================================================================
|
||||
|
||||
/**
|
||||
* 写入一行（创建新版本）。
|
||||
*/
|
||||
writeVersion(
|
||||
tableName: string,
|
||||
key: string,
|
||||
data: Record<string, unknown>,
|
||||
txnId: number,
|
||||
): void {
|
||||
const tableKey = `${tableName}.${key}`;
|
||||
const versions = this.versionStore.get(tableKey) ?? [];
|
||||
|
||||
const newVersion: RowVersion = {
|
||||
txnId,
|
||||
data,
|
||||
prevVersion: versions.length > 0 ? versions[versions.length - 1] : null,
|
||||
committed: false,
|
||||
};
|
||||
|
||||
versions.push(newVersion);
|
||||
this.versionStore.set(tableKey, versions);
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取一行（对指定事务可见的最新版本）。
|
||||
*/
|
||||
readVersion(
|
||||
tableName: string,
|
||||
key: string,
|
||||
txnId: number,
|
||||
): Record<string, unknown> | null {
|
||||
const txn = this.activeTxns.get(txnId);
|
||||
if (!txn) return null;
|
||||
|
||||
const tableKey = `${tableName}.${key}`;
|
||||
const versions = this.versionStore.get(tableKey);
|
||||
if (!versions || versions.length === 0) return null;
|
||||
|
||||
// 从最新版本向前遍历
|
||||
for (let i = versions.length - 1; i >= 0; i--) {
|
||||
const version = versions[i];
|
||||
|
||||
// 1. 如果是当前事务写入的（未提交），可见
|
||||
if (version.txnId === txnId) {
|
||||
return version.data;
|
||||
}
|
||||
|
||||
// 2. 如果是已提交的版本，且在快照 LSN 之前提交，可见
|
||||
if (version.committed) {
|
||||
// 简化：所有已提交版本都可见
|
||||
return version.data;
|
||||
}
|
||||
|
||||
// 3. 其他事务的未提交版本，不可见，继续找更早的版本
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* 删除一行（创建墓碑版本）。
|
||||
*/
|
||||
deleteVersion(tableName: string, key: string, txnId: number): void {
|
||||
this.writeVersion(tableName, key, { __mvcc_tombstone: true } as unknown as Record<string, unknown>, txnId);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取所有行的最新已提交版本（用于非事务读取）。
|
||||
*/
|
||||
getLatestCommittedVersions(
|
||||
tableName: string,
|
||||
): Record<string, Record<string, unknown>> {
|
||||
const result: Record<string, Record<string, unknown>> = {};
|
||||
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
if (!tableKey.startsWith(`${tableName}.`)) continue;
|
||||
const key = tableKey.slice(tableName.length + 1);
|
||||
|
||||
for (let i = versions.length - 1; i >= 0; i--) {
|
||||
const version = versions[i];
|
||||
if (version.committed && !(version.data as unknown as Record<string, unknown>).__mvcc_tombstone) {
|
||||
result[key] = version.data;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* 清理过旧版本（GC）。
|
||||
* 保留每个 key 的最新 N 个已提交版本。
|
||||
*/
|
||||
gc(maxVersionsPerKey: number = 100): void {
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
if (versions.length <= maxVersionsPerKey) continue;
|
||||
|
||||
// 保留最新的 maxVersionsPerKey 个版本
|
||||
const pruned = versions.slice(versions.length - maxVersionsPerKey);
|
||||
this.versionStore.set(tableKey, pruned);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取所有未提交事务中的 key 列表。
|
||||
*/
|
||||
getActiveWriteKeys(tableName: string, txnId: number): Set<string> {
|
||||
const keys = new Set<string>();
|
||||
const prefix = `${tableName}.`;
|
||||
|
||||
for (const [tableKey, versions] of this.versionStore) {
|
||||
if (!tableKey.startsWith(prefix)) continue;
|
||||
const latestVersion = versions[versions.length - 1];
|
||||
if (latestVersion.txnId === txnId && !latestVersion.committed) {
|
||||
keys.add(tableKey.slice(prefix.length));
|
||||
}
|
||||
}
|
||||
|
||||
return keys;
|
||||
}
|
||||
|
||||
/**
|
||||
* 清理指定表的所有版本。
|
||||
*/
|
||||
clearTable(tableName: string): void {
|
||||
const prefix = `${tableName}.`;
|
||||
for (const [tableKey] of this.versionStore) {
|
||||
if (tableKey.startsWith(prefix)) {
|
||||
this.versionStore.delete(tableKey);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取活跃事务数。
|
||||
*/
|
||||
getActiveTxnCount(): number {
|
||||
return this.activeTxns.size;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取全局 LSN。
|
||||
*/
|
||||
getGlobalLSN(): number {
|
||||
return this.globalCommitLsn;
|
||||
}
|
||||
}
|
||||
|
||||
+302
-302
@@ -1,302 +1,302 @@
|
||||
/**
|
||||
* AriaEngine Types — 内部类型定义
|
||||
* @module engine/aria/types
|
||||
*
|
||||
* 页面式存储引擎的所有内部枚举、接口和常量。
|
||||
*/
|
||||
|
||||
// =============================================================================
|
||||
// 页面常量
|
||||
// =============================================================================
|
||||
|
||||
/** 页面大小：4KB */
|
||||
export const PAGE_SIZE = 4096;
|
||||
|
||||
/** 页面头大小：16 字节 */
|
||||
export const PAGE_HEADER_SIZE = 16;
|
||||
|
||||
/** 每个 Slot 目录项大小：4 字节 (offset: u16 + len: u16) */
|
||||
export const SLOT_ENTRY_SIZE = 4;
|
||||
|
||||
/** 页面数据区起始偏移（头部之后） */
|
||||
export const PAGE_DATA_START = PAGE_HEADER_SIZE;
|
||||
|
||||
/** 无效页面 ID */
|
||||
export const INVALID_PAGE_ID = 0xFFFFFFFF;
|
||||
|
||||
// =============================================================================
|
||||
// 页面类型
|
||||
// =============================================================================
|
||||
|
||||
export enum PageType {
|
||||
/** 数据页面：存储行数据 */
|
||||
DATA = 1,
|
||||
/** 索引页面：存储索引节点 */
|
||||
INDEX = 2,
|
||||
/** 溢出页面：存储大字段 */
|
||||
OVERFLOW = 3,
|
||||
/** 元数据页面：存储表/库元信息 */
|
||||
META = 4,
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// 页面头部（16 字节）
|
||||
// =============================================================================
|
||||
|
||||
export interface PageHeader {
|
||||
/** 页面 ID（全局唯一） */
|
||||
pageId: number;
|
||||
/** 页面类型 */
|
||||
type: PageType;
|
||||
/** 空闲空间起始偏移（slot 区结束位置） */
|
||||
freeStart: number;
|
||||
/** 数据区结束偏移（从页面底部向上增长） */
|
||||
freeEnd: number;
|
||||
/** 当前 slot 数量 */
|
||||
slotCount: number;
|
||||
/** CRC32 校验和 */
|
||||
checksum: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Slot 目录项（4 字节）
|
||||
// =============================================================================
|
||||
|
||||
export interface SlotEntry {
|
||||
/** 行数据在页面内的偏移 */
|
||||
offset: number;
|
||||
/** 行数据长度（字节） */
|
||||
length: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// 页面句柄（Buffer Pool 中的页面）
|
||||
// =============================================================================
|
||||
|
||||
export interface PageHandle {
|
||||
/** 页面 ID */
|
||||
pageId: number;
|
||||
/** 页面类型 */
|
||||
type: PageType;
|
||||
/** 页面数据缓冲区（4KB ArrayBuffer） */
|
||||
data: ArrayBuffer;
|
||||
/** 是否被修改（脏页） */
|
||||
dirty: boolean;
|
||||
/** 引用计数（pin count） */
|
||||
pins: number;
|
||||
/** LRU 链表前驱 */
|
||||
prev: PageHandle | null;
|
||||
/** LRU 链表后继 */
|
||||
next: PageHandle | null;
|
||||
/** 最后访问时间戳 */
|
||||
lastAccess: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// 行编解码
|
||||
// =============================================================================
|
||||
|
||||
/** 行/元组的二进制表示 */
|
||||
export interface SerializedTuple {
|
||||
/** 序列化后的字节数组 */
|
||||
bytes: Uint8Array;
|
||||
/** 该行中 null 列的位图 */
|
||||
nullBitmap: Uint8Array;
|
||||
}
|
||||
|
||||
/** 列类型（内部二进制编码用） */
|
||||
export enum ColumnEncoding {
|
||||
STRING = 1,
|
||||
NUMBER = 2,
|
||||
BOOLEAN = 3,
|
||||
DATE = 4,
|
||||
JSON = 5,
|
||||
NULL = 6,
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// LSM-Tree
|
||||
// =============================================================================
|
||||
|
||||
/** MemTable 最大大小（默认 4MB） */
|
||||
export const DEFAULT_MEMTABLE_SIZE = 4 * 1024 * 1024;
|
||||
|
||||
/** SSTable 中每个 Data Block 的默认大小 */
|
||||
export const DEFAULT_BLOCK_SIZE = 4096;
|
||||
|
||||
/** Bloom Filter 每 key 的默认位数 */
|
||||
export const DEFAULT_BLOOM_BITS_PER_KEY = 10;
|
||||
|
||||
/** SSTable 最大层级 */
|
||||
export const MAX_LSM_LEVELS = 7;
|
||||
|
||||
/** 每层之间的大小倍数 */
|
||||
export const DEFAULT_LEVEL_SIZE_MULTIPLIER = 10;
|
||||
|
||||
/** SSTable 元数据 */
|
||||
export interface SSTableMeta {
|
||||
/** SSTable 文件 ID */
|
||||
id: number;
|
||||
/** 所在层级 */
|
||||
level: number;
|
||||
/** 最小 key */
|
||||
minKey: string;
|
||||
/** 最大 key */
|
||||
maxKey: string;
|
||||
/** 数据块数量 */
|
||||
blockCount: number;
|
||||
/** 总大小（字节） */
|
||||
totalSize: number;
|
||||
/** Bloom Filter 序列化数据 */
|
||||
bloomData: Uint8Array | null;
|
||||
}
|
||||
|
||||
/** SSTable 内部的 Data Block */
|
||||
export interface DataBlock {
|
||||
/** 该块内的 key-value 条目数 */
|
||||
entryCount: number;
|
||||
/** 该块数据区 */
|
||||
data: Uint8Array;
|
||||
/** 该块起始 key */
|
||||
startKey: string;
|
||||
/** 该块结束 key */
|
||||
endKey: string;
|
||||
}
|
||||
|
||||
/** 索引块条目：key → data block offset */
|
||||
export interface IndexEntry {
|
||||
/** 到此 block 的最后一个 key */
|
||||
key: string;
|
||||
/** data block 在 SSTable 文件中的偏移 */
|
||||
blockOffset: number;
|
||||
/** data block 大小 */
|
||||
blockSize: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// WAL (Write-Ahead Log)
|
||||
// =============================================================================
|
||||
|
||||
/** WAL 记录类型 */
|
||||
export enum WALRecordType {
|
||||
INSERT = 1,
|
||||
UPDATE = 2,
|
||||
DELETE = 3,
|
||||
BEGIN = 4,
|
||||
COMMIT = 5,
|
||||
ROLLBACK = 6,
|
||||
CREATE_TABLE = 7,
|
||||
DROP_TABLE = 8,
|
||||
}
|
||||
|
||||
/** 单条 WAL 记录 */
|
||||
export interface WALRecord {
|
||||
/** 日志序列号 */
|
||||
lsn: number;
|
||||
/** 记录类型 */
|
||||
type: WALRecordType;
|
||||
/** 事务 ID */
|
||||
txnId: number;
|
||||
/** 表名 */
|
||||
tableName: string;
|
||||
/** 主键值 */
|
||||
key: string;
|
||||
/** 操作数据（INSERT/UPDATE 时有效） */
|
||||
data?: Record<string, unknown>;
|
||||
/** 校验和 */
|
||||
checksum: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// MVCC
|
||||
// =============================================================================
|
||||
|
||||
/** 事务隔离级别 */
|
||||
export enum IsolationLevel {
|
||||
READ_COMMITTED = 1,
|
||||
SNAPSHOT = 2,
|
||||
}
|
||||
|
||||
/** 事务状态 */
|
||||
export enum TransactionState {
|
||||
ACTIVE = 1,
|
||||
COMMITTED = 2,
|
||||
ABORTED = 3,
|
||||
}
|
||||
|
||||
/** 行版本 */
|
||||
export interface RowVersion {
|
||||
/** 事务 ID（创建此版本的事务） */
|
||||
txnId: number;
|
||||
/** 版本数据 */
|
||||
data: Record<string, unknown>;
|
||||
/** 指向上一版本的指针（undo 链） */
|
||||
prevVersion: RowVersion | null;
|
||||
/** 该版本是否已提交 */
|
||||
committed: boolean;
|
||||
}
|
||||
|
||||
/** 活跃事务表项 */
|
||||
export interface TxnEntry {
|
||||
/** 事务 ID */
|
||||
txnId: number;
|
||||
/** 事务状态 */
|
||||
state: TransactionState;
|
||||
/** 快照序列号（用于可见性判断） */
|
||||
snapshotLsn: number;
|
||||
/** 事务开始时间 */
|
||||
startTime: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Buffer Pool
|
||||
// =============================================================================
|
||||
|
||||
/** Buffer Pool 默认容量：256 页 ≈ 1MB */
|
||||
export const DEFAULT_BUFFER_POOL_PAGES = 256;
|
||||
|
||||
// =============================================================================
|
||||
// AriaEngine 配置
|
||||
// =============================================================================
|
||||
|
||||
export interface AriaEngineConfig {
|
||||
/** 页面大小（默认 4096） */
|
||||
pageSize?: number;
|
||||
/** Buffer Pool 页面数量（默认 256） */
|
||||
bufferPoolPages?: number;
|
||||
/** MemTable 刷盘阈值（默认 4MB） */
|
||||
memtableSizeThreshold?: number;
|
||||
/** LSM 层级之间的容量倍数（默认 10） */
|
||||
levelSizeMultiplier?: number;
|
||||
/** Bloom Filter 每 key 位数（默认 10） */
|
||||
bloomFilterBitsPerKey?: number;
|
||||
/** 是否启用 WAL（默认 true） */
|
||||
walEnabled?: boolean;
|
||||
/** WAL 同步模式 */
|
||||
walSyncMode?: 'full' | 'batch' | 'none';
|
||||
/** Checkpoint 间隔（操作数，默认 1000） */
|
||||
checkpointInterval?: number;
|
||||
/** 是否启用页面压缩（默认 false） */
|
||||
compression?: boolean;
|
||||
/** 存储后端 */
|
||||
storageBackend?: 'indexeddb' | 'opfs' | 'memory';
|
||||
/** WAL 大小阈值（字节，超过则强制 checkpoint，默认 16MB） */
|
||||
walSizeThreshold?: number;
|
||||
/** 最大内存预算（MB，默认 64） */
|
||||
maxMemoryMB?: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_ARIA_CONFIG: Required<AriaEngineConfig> = {
|
||||
pageSize: PAGE_SIZE,
|
||||
bufferPoolPages: DEFAULT_BUFFER_POOL_PAGES,
|
||||
memtableSizeThreshold: DEFAULT_MEMTABLE_SIZE,
|
||||
levelSizeMultiplier: DEFAULT_LEVEL_SIZE_MULTIPLIER,
|
||||
bloomFilterBitsPerKey: DEFAULT_BLOOM_BITS_PER_KEY,
|
||||
walEnabled: true,
|
||||
walSyncMode: 'full',
|
||||
checkpointInterval: 1000,
|
||||
compression: false,
|
||||
storageBackend: 'indexeddb',
|
||||
walSizeThreshold: 16 * 1024 * 1024, // 16MB
|
||||
maxMemoryMB: 64,
|
||||
};
|
||||
/**
|
||||
* AriaEngine Types — 内部类型定义
|
||||
* @module engine/aria/types
|
||||
*
|
||||
* 页面式存储引擎的所有内部枚举、接口和常量。
|
||||
*/
|
||||
|
||||
// =============================================================================
|
||||
// 页面常量
|
||||
// =============================================================================
|
||||
|
||||
/** 页面大小：4KB */
|
||||
export const PAGE_SIZE = 4096;
|
||||
|
||||
/** 页面头大小：16 字节 */
|
||||
export const PAGE_HEADER_SIZE = 16;
|
||||
|
||||
/** 每个 Slot 目录项大小：4 字节 (offset: u16 + len: u16) */
|
||||
export const SLOT_ENTRY_SIZE = 4;
|
||||
|
||||
/** 页面数据区起始偏移（头部之后） */
|
||||
export const PAGE_DATA_START = PAGE_HEADER_SIZE;
|
||||
|
||||
/** 无效页面 ID */
|
||||
export const INVALID_PAGE_ID = 0xFFFFFFFF;
|
||||
|
||||
// =============================================================================
|
||||
// 页面类型
|
||||
// =============================================================================
|
||||
|
||||
export enum PageType {
|
||||
/** 数据页面：存储行数据 */
|
||||
DATA = 1,
|
||||
/** 索引页面：存储索引节点 */
|
||||
INDEX = 2,
|
||||
/** 溢出页面：存储大字段 */
|
||||
OVERFLOW = 3,
|
||||
/** 元数据页面：存储表/库元信息 */
|
||||
META = 4,
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// 页面头部（16 字节）
|
||||
// =============================================================================
|
||||
|
||||
export interface PageHeader {
|
||||
/** 页面 ID（全局唯一） */
|
||||
pageId: number;
|
||||
/** 页面类型 */
|
||||
type: PageType;
|
||||
/** 空闲空间起始偏移（slot 区结束位置） */
|
||||
freeStart: number;
|
||||
/** 数据区结束偏移（从页面底部向上增长） */
|
||||
freeEnd: number;
|
||||
/** 当前 slot 数量 */
|
||||
slotCount: number;
|
||||
/** CRC32 校验和 */
|
||||
checksum: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Slot 目录项（4 字节）
|
||||
// =============================================================================
|
||||
|
||||
export interface SlotEntry {
|
||||
/** 行数据在页面内的偏移 */
|
||||
offset: number;
|
||||
/** 行数据长度（字节） */
|
||||
length: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// 页面句柄（Buffer Pool 中的页面）
|
||||
// =============================================================================
|
||||
|
||||
export interface PageHandle {
|
||||
/** 页面 ID */
|
||||
pageId: number;
|
||||
/** 页面类型 */
|
||||
type: PageType;
|
||||
/** 页面数据缓冲区（4KB ArrayBuffer） */
|
||||
data: ArrayBuffer;
|
||||
/** 是否被修改（脏页） */
|
||||
dirty: boolean;
|
||||
/** 引用计数（pin count） */
|
||||
pins: number;
|
||||
/** LRU 链表前驱 */
|
||||
prev: PageHandle | null;
|
||||
/** LRU 链表后继 */
|
||||
next: PageHandle | null;
|
||||
/** 最后访问时间戳 */
|
||||
lastAccess: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// 行编解码
|
||||
// =============================================================================
|
||||
|
||||
/** 行/元组的二进制表示 */
|
||||
export interface SerializedTuple {
|
||||
/** 序列化后的字节数组 */
|
||||
bytes: Uint8Array;
|
||||
/** 该行中 null 列的位图 */
|
||||
nullBitmap: Uint8Array;
|
||||
}
|
||||
|
||||
/** 列类型（内部二进制编码用） */
|
||||
export enum ColumnEncoding {
|
||||
STRING = 1,
|
||||
NUMBER = 2,
|
||||
BOOLEAN = 3,
|
||||
DATE = 4,
|
||||
JSON = 5,
|
||||
NULL = 6,
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// LSM-Tree
|
||||
// =============================================================================
|
||||
|
||||
/** MemTable 最大大小（默认 4MB） */
|
||||
export const DEFAULT_MEMTABLE_SIZE = 4 * 1024 * 1024;
|
||||
|
||||
/** SSTable 中每个 Data Block 的默认大小 */
|
||||
export const DEFAULT_BLOCK_SIZE = 4096;
|
||||
|
||||
/** Bloom Filter 每 key 的默认位数 */
|
||||
export const DEFAULT_BLOOM_BITS_PER_KEY = 10;
|
||||
|
||||
/** SSTable 最大层级 */
|
||||
export const MAX_LSM_LEVELS = 7;
|
||||
|
||||
/** 每层之间的大小倍数 */
|
||||
export const DEFAULT_LEVEL_SIZE_MULTIPLIER = 10;
|
||||
|
||||
/** SSTable 元数据 */
|
||||
export interface SSTableMeta {
|
||||
/** SSTable 文件 ID */
|
||||
id: number;
|
||||
/** 所在层级 */
|
||||
level: number;
|
||||
/** 最小 key */
|
||||
minKey: string;
|
||||
/** 最大 key */
|
||||
maxKey: string;
|
||||
/** 数据块数量 */
|
||||
blockCount: number;
|
||||
/** 总大小（字节） */
|
||||
totalSize: number;
|
||||
/** Bloom Filter 序列化数据 */
|
||||
bloomData: Uint8Array | null;
|
||||
}
|
||||
|
||||
/** SSTable 内部的 Data Block */
|
||||
export interface DataBlock {
|
||||
/** 该块内的 key-value 条目数 */
|
||||
entryCount: number;
|
||||
/** 该块数据区 */
|
||||
data: Uint8Array;
|
||||
/** 该块起始 key */
|
||||
startKey: string;
|
||||
/** 该块结束 key */
|
||||
endKey: string;
|
||||
}
|
||||
|
||||
/** 索引块条目：key → data block offset */
|
||||
export interface IndexEntry {
|
||||
/** 到此 block 的最后一个 key */
|
||||
key: string;
|
||||
/** data block 在 SSTable 文件中的偏移 */
|
||||
blockOffset: number;
|
||||
/** data block 大小 */
|
||||
blockSize: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// WAL (Write-Ahead Log)
|
||||
// =============================================================================
|
||||
|
||||
/** WAL 记录类型 */
|
||||
export enum WALRecordType {
|
||||
INSERT = 1,
|
||||
UPDATE = 2,
|
||||
DELETE = 3,
|
||||
BEGIN = 4,
|
||||
COMMIT = 5,
|
||||
ROLLBACK = 6,
|
||||
CREATE_TABLE = 7,
|
||||
DROP_TABLE = 8,
|
||||
}
|
||||
|
||||
/** 单条 WAL 记录 */
|
||||
export interface WALRecord {
|
||||
/** 日志序列号 */
|
||||
lsn: number;
|
||||
/** 记录类型 */
|
||||
type: WALRecordType;
|
||||
/** 事务 ID */
|
||||
txnId: number;
|
||||
/** 表名 */
|
||||
tableName: string;
|
||||
/** 主键值 */
|
||||
key: string;
|
||||
/** 操作数据（INSERT/UPDATE 时有效） */
|
||||
data?: Record<string, unknown>;
|
||||
/** 校验和 */
|
||||
checksum: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// MVCC
|
||||
// =============================================================================
|
||||
|
||||
/** 事务隔离级别 */
|
||||
export enum IsolationLevel {
|
||||
READ_COMMITTED = 1,
|
||||
SNAPSHOT = 2,
|
||||
}
|
||||
|
||||
/** 事务状态 */
|
||||
export enum TransactionState {
|
||||
ACTIVE = 1,
|
||||
COMMITTED = 2,
|
||||
ABORTED = 3,
|
||||
}
|
||||
|
||||
/** 行版本 */
|
||||
export interface RowVersion {
|
||||
/** 事务 ID（创建此版本的事务） */
|
||||
txnId: number;
|
||||
/** 版本数据 */
|
||||
data: Record<string, unknown>;
|
||||
/** 指向上一版本的指针（undo 链） */
|
||||
prevVersion: RowVersion | null;
|
||||
/** 该版本是否已提交 */
|
||||
committed: boolean;
|
||||
}
|
||||
|
||||
/** 活跃事务表项 */
|
||||
export interface TxnEntry {
|
||||
/** 事务 ID */
|
||||
txnId: number;
|
||||
/** 事务状态 */
|
||||
state: TransactionState;
|
||||
/** 快照序列号（用于可见性判断） */
|
||||
snapshotLsn: number;
|
||||
/** 事务开始时间 */
|
||||
startTime: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Buffer Pool
|
||||
// =============================================================================
|
||||
|
||||
/** Buffer Pool 默认容量：256 页 ≈ 1MB */
|
||||
export const DEFAULT_BUFFER_POOL_PAGES = 256;
|
||||
|
||||
// =============================================================================
|
||||
// AriaEngine 配置
|
||||
// =============================================================================
|
||||
|
||||
export interface AriaEngineConfig {
|
||||
/** 页面大小（默认 4096） */
|
||||
pageSize?: number;
|
||||
/** Buffer Pool 页面数量（默认 256） */
|
||||
bufferPoolPages?: number;
|
||||
/** MemTable 刷盘阈值（默认 4MB） */
|
||||
memtableSizeThreshold?: number;
|
||||
/** LSM 层级之间的容量倍数（默认 10） */
|
||||
levelSizeMultiplier?: number;
|
||||
/** Bloom Filter 每 key 位数（默认 10） */
|
||||
bloomFilterBitsPerKey?: number;
|
||||
/** 是否启用 WAL（默认 true） */
|
||||
walEnabled?: boolean;
|
||||
/** WAL 同步模式 */
|
||||
walSyncMode?: 'full' | 'batch' | 'none';
|
||||
/** Checkpoint 间隔（操作数，默认 1000） */
|
||||
checkpointInterval?: number;
|
||||
/** 是否启用页面压缩（默认 false） */
|
||||
compression?: boolean;
|
||||
/** 存储后端 */
|
||||
storageBackend?: 'indexeddb' | 'opfs' | 'memory';
|
||||
/** WAL 大小阈值（字节，超过则强制 checkpoint，默认 16MB） */
|
||||
walSizeThreshold?: number;
|
||||
/** 最大内存预算（MB，默认 64） */
|
||||
maxMemoryMB?: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_ARIA_CONFIG: Required<AriaEngineConfig> = {
|
||||
pageSize: PAGE_SIZE,
|
||||
bufferPoolPages: DEFAULT_BUFFER_POOL_PAGES,
|
||||
memtableSizeThreshold: DEFAULT_MEMTABLE_SIZE,
|
||||
levelSizeMultiplier: DEFAULT_LEVEL_SIZE_MULTIPLIER,
|
||||
bloomFilterBitsPerKey: DEFAULT_BLOOM_BITS_PER_KEY,
|
||||
walEnabled: true,
|
||||
walSyncMode: 'full',
|
||||
checkpointInterval: 1000,
|
||||
compression: false,
|
||||
storageBackend: 'indexeddb',
|
||||
walSizeThreshold: 16 * 1024 * 1024, // 16MB
|
||||
maxMemoryMB: 64,
|
||||
};
|
||||
|
||||
@@ -1,77 +1,77 @@
|
||||
/**
|
||||
* AriaEngine Checkpoint — 检查点机制
|
||||
* @module engine/aria/wal/checkpoint
|
||||
*/
|
||||
|
||||
import type { LSM } from '../index/lsm';
|
||||
import type { WAL } from './log';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 简化的 flush 接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface Flushable {
|
||||
flushAll(): Promise<void>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CheckpointManager
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class CheckpointManager {
|
||||
private lsm: LSM;
|
||||
private wal: WAL;
|
||||
private flushable: Flushable | null;
|
||||
private interval: number;
|
||||
private opCount = 0;
|
||||
private walSizeThreshold: number;
|
||||
|
||||
constructor(
|
||||
lsm: LSM,
|
||||
wal: WAL,
|
||||
flushable: Flushable | null = null,
|
||||
interval: number = 1000,
|
||||
walSizeThreshold: number = 16 * 1024 * 1024,
|
||||
) {
|
||||
this.lsm = lsm;
|
||||
this.wal = wal;
|
||||
this.flushable = flushable;
|
||||
this.interval = interval;
|
||||
this.walSizeThreshold = walSizeThreshold;
|
||||
}
|
||||
|
||||
async tick(): Promise<void> {
|
||||
this.opCount++;
|
||||
// 检查操作计数或 WAL 大小是否超阈值
|
||||
if (this.opCount >= this.interval || this.getWALEstimatedSize() >= this.walSizeThreshold) {
|
||||
await this.checkpoint();
|
||||
}
|
||||
}
|
||||
|
||||
/** 估算 WAL 大小 */
|
||||
private getWALEstimatedSize(): number {
|
||||
const count = typeof this.wal.getBufferedCount === 'function' ? this.wal.getBufferedCount() : 0;
|
||||
return count * 200;
|
||||
}
|
||||
|
||||
async checkpoint(): Promise<void> {
|
||||
await this.lsm.flush();
|
||||
if (this.flushable) {
|
||||
await this.flushable.flushAll();
|
||||
}
|
||||
await this.wal.checkpoint();
|
||||
this.opCount = 0;
|
||||
}
|
||||
|
||||
async forceCheckpoint(): Promise<void> {
|
||||
await this.checkpoint();
|
||||
}
|
||||
|
||||
setInterval(ops: number): void {
|
||||
this.interval = ops;
|
||||
}
|
||||
|
||||
getOpCount(): number {
|
||||
return this.opCount;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine Checkpoint — 检查点机制
|
||||
* @module engine/aria/wal/checkpoint
|
||||
*/
|
||||
|
||||
import type { LSM } from '../index/lsm';
|
||||
import type { WAL } from './log';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 简化的 flush 接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface Flushable {
|
||||
flushAll(): Promise<void>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CheckpointManager
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class CheckpointManager {
|
||||
private lsm: LSM;
|
||||
private wal: WAL;
|
||||
private flushable: Flushable | null;
|
||||
private interval: number;
|
||||
private opCount = 0;
|
||||
private walSizeThreshold: number;
|
||||
|
||||
constructor(
|
||||
lsm: LSM,
|
||||
wal: WAL,
|
||||
flushable: Flushable | null = null,
|
||||
interval: number = 1000,
|
||||
walSizeThreshold: number = 16 * 1024 * 1024,
|
||||
) {
|
||||
this.lsm = lsm;
|
||||
this.wal = wal;
|
||||
this.flushable = flushable;
|
||||
this.interval = interval;
|
||||
this.walSizeThreshold = walSizeThreshold;
|
||||
}
|
||||
|
||||
async tick(): Promise<void> {
|
||||
this.opCount++;
|
||||
// 检查操作计数或 WAL 大小是否超阈值
|
||||
if (this.opCount >= this.interval || this.getWALEstimatedSize() >= this.walSizeThreshold) {
|
||||
await this.checkpoint();
|
||||
}
|
||||
}
|
||||
|
||||
/** 估算 WAL 大小 */
|
||||
private getWALEstimatedSize(): number {
|
||||
const count = typeof this.wal.getBufferedCount === 'function' ? this.wal.getBufferedCount() : 0;
|
||||
return count * 200;
|
||||
}
|
||||
|
||||
async checkpoint(): Promise<void> {
|
||||
await this.lsm.flush();
|
||||
if (this.flushable) {
|
||||
await this.flushable.flushAll();
|
||||
}
|
||||
await this.wal.checkpoint();
|
||||
this.opCount = 0;
|
||||
}
|
||||
|
||||
async forceCheckpoint(): Promise<void> {
|
||||
await this.checkpoint();
|
||||
}
|
||||
|
||||
setInterval(ops: number): void {
|
||||
this.interval = ops;
|
||||
}
|
||||
|
||||
getOpCount(): number {
|
||||
return this.opCount;
|
||||
}
|
||||
}
|
||||
|
||||
+313
-283
@@ -1,283 +1,313 @@
|
||||
/**
|
||||
* AriaEngine WAL — Write-Ahead Log
|
||||
* @module engine/aria/wal/log
|
||||
*
|
||||
* 崩溃恢复前的写操作持久化日志。
|
||||
*
|
||||
* WAL 文件格式:
|
||||
* ┌──────────┬──────────────┬──────────┐
|
||||
* │ Record 1│ Record 2 │ ... │
|
||||
* │ 4B LSN │ │ │
|
||||
* │ 1B type │ │ │
|
||||
* │ 4B txnId│ │ │
|
||||
* │ 2B tblLen│ │ │
|
||||
* │ N table│ │ │
|
||||
* │ 2B keyLen│ │ │
|
||||
* │ N key │ │ │
|
||||
* │ 4B jsonLen│ │ │
|
||||
* │ N json │ │ │
|
||||
* │ 4B CRC │ │ │
|
||||
* └──────────┴──────────────┴──────────┘
|
||||
*/
|
||||
|
||||
import { WALRecordType, type WALRecord } from '../types';
|
||||
import type { BufferPool } from '../buffer/pool';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// WAL 存储接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface WALStore {
|
||||
/** 追加 WAL 记录 */
|
||||
append(data: Uint8Array): Promise<void>;
|
||||
/** 读取所有 WAL 记录 */
|
||||
readAll(): Promise<Uint8Array>;
|
||||
/** 截断 WAL（checkpoint 后清理） */
|
||||
truncate(): Promise<void>;
|
||||
/** 检查 WAL 是否存在 */
|
||||
exists(): Promise<boolean>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// WAL
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class WAL {
|
||||
private lsn = 0;
|
||||
private store: WALStore;
|
||||
private enabled: boolean;
|
||||
private buffer: Uint8Array[] = [];
|
||||
private syncMode: 'full' | 'batch' | 'none';
|
||||
|
||||
constructor(store: WALStore, enabled: boolean = true, syncMode: 'full' | 'batch' | 'none' = 'batch') {
|
||||
this.store = store;
|
||||
this.enabled = enabled;
|
||||
this.syncMode = syncMode;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 写入
|
||||
// =======================================================================
|
||||
|
||||
/** 追加一条 WAL 记录（full 模式同步等待写入完成） */
|
||||
async append(record: Omit<WALRecord, 'lsn' | 'checksum'>): Promise<void> {
|
||||
if (!this.enabled) return;
|
||||
|
||||
this.lsn++;
|
||||
const fullRecord: WALRecord = {
|
||||
...record,
|
||||
lsn: this.lsn,
|
||||
checksum: 0, // 稍后计算
|
||||
};
|
||||
|
||||
const bytes = this.encodeRecord(fullRecord);
|
||||
|
||||
if (this.syncMode === 'full') {
|
||||
try {
|
||||
await this.store.append(bytes);
|
||||
} catch {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('[AriaEngine WAL] Failed to append record');
|
||||
}
|
||||
} else if (this.syncMode === 'batch') {
|
||||
this.buffer.push(bytes);
|
||||
}
|
||||
// 'none' mode: 不写 WAL
|
||||
}
|
||||
|
||||
/** 批量刷新缓冲的 WAL 记录 */
|
||||
async flush(): Promise<void> {
|
||||
if (!this.enabled || this.buffer.length === 0) return;
|
||||
|
||||
const totalLen = this.buffer.reduce((sum, b) => sum + b.byteLength, 0);
|
||||
const combined = new Uint8Array(totalLen);
|
||||
let offset = 0;
|
||||
for (const buf of this.buffer) {
|
||||
combined.set(buf, offset);
|
||||
offset += buf.byteLength;
|
||||
}
|
||||
|
||||
await this.store.append(combined);
|
||||
this.buffer = [];
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 恢复
|
||||
// =======================================================================
|
||||
|
||||
/** 从 WAL 恢复未提交的事务数据 */
|
||||
async recover(
|
||||
applyRecord: (record: WALRecord) => void,
|
||||
): Promise<number> {
|
||||
if (!this.enabled) return 0;
|
||||
|
||||
const exists = await this.store.exists();
|
||||
if (!exists) return 0;
|
||||
|
||||
const data = await this.store.readAll();
|
||||
if (data.byteLength === 0) return 0;
|
||||
|
||||
const records = this.decodeAllRecords(data);
|
||||
for (const record of records) {
|
||||
applyRecord(record);
|
||||
}
|
||||
|
||||
this.lsn = records.length > 0 ? records[records.length - 1].lsn : 0;
|
||||
return records.length;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// Checkpoint
|
||||
// =======================================================================
|
||||
|
||||
/** Checkpoint 后清空 WAL */
|
||||
async checkpoint(): Promise<void> {
|
||||
if (!this.enabled) return;
|
||||
await this.flush();
|
||||
await this.store.truncate();
|
||||
this.lsn = 0;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 统计
|
||||
// =======================================================================
|
||||
|
||||
isEnabled(): boolean {
|
||||
return this.enabled;
|
||||
}
|
||||
|
||||
getLSN(): number {
|
||||
return this.lsn;
|
||||
}
|
||||
|
||||
getBufferedCount(): number {
|
||||
return this.buffer.length;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 编解码
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private encodeRecord(record: WALRecord): Uint8Array {
|
||||
const encoder = new TextEncoder();
|
||||
const tableBytes = encoder.encode(record.tableName);
|
||||
const keyBytes = encoder.encode(record.key);
|
||||
const jsonStr = record.data ? JSON.stringify(record.data) : '';
|
||||
const jsonBytes = encoder.encode(jsonStr);
|
||||
|
||||
const size =
|
||||
4 + // LSN
|
||||
1 + // type
|
||||
4 + // txnId
|
||||
2 + tableBytes.length + // table
|
||||
2 + keyBytes.length + // key
|
||||
4 + jsonBytes.length + // json
|
||||
4; // CRC
|
||||
|
||||
const buf = new ArrayBuffer(size);
|
||||
const view = new DataView(buf);
|
||||
let offset = 0;
|
||||
|
||||
view.setUint32(offset, record.lsn, false);
|
||||
offset += 4;
|
||||
view.setUint8(offset, record.type);
|
||||
offset += 1;
|
||||
view.setUint32(offset, record.txnId, false);
|
||||
offset += 4;
|
||||
|
||||
view.setUint16(offset, tableBytes.length, false);
|
||||
offset += 2;
|
||||
new Uint8Array(buf).set(tableBytes, offset);
|
||||
offset += tableBytes.length;
|
||||
|
||||
view.setUint16(offset, keyBytes.length, false);
|
||||
offset += 2;
|
||||
new Uint8Array(buf).set(keyBytes, offset);
|
||||
offset += keyBytes.length;
|
||||
|
||||
view.setUint32(offset, jsonBytes.length, false);
|
||||
offset += 4;
|
||||
new Uint8Array(buf).set(jsonBytes, offset);
|
||||
offset += jsonBytes.length;
|
||||
|
||||
// 简单 CRC
|
||||
let crc = 0;
|
||||
const u8 = new Uint8Array(buf, 0, offset);
|
||||
for (let i = 0; i < u8.length; i++) {
|
||||
crc = ((crc << 5) - crc + u8[i]) | 0;
|
||||
}
|
||||
view.setUint32(offset, crc >>> 0, false);
|
||||
|
||||
return new Uint8Array(buf);
|
||||
}
|
||||
|
||||
private decodeAllRecords(data: Uint8Array): WALRecord[] {
|
||||
const records: WALRecord[] = [];
|
||||
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
||||
let offset = 0;
|
||||
|
||||
while (offset + 15 <= data.byteLength) {
|
||||
try {
|
||||
const recordStart = offset;
|
||||
const lsn = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
const type = view.getUint8(offset) as WALRecordType;
|
||||
offset += 1;
|
||||
const txnId = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
|
||||
const tableLen = view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
if (offset + tableLen > data.byteLength) break;
|
||||
const tableName = new TextDecoder().decode(data.slice(offset, offset + tableLen));
|
||||
offset += tableLen;
|
||||
|
||||
const keyLen = view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
if (offset + keyLen > data.byteLength) break;
|
||||
const key = new TextDecoder().decode(data.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
|
||||
const jsonLen = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
if (offset + jsonLen > data.byteLength) break;
|
||||
let recordData: Record<string, unknown> | undefined;
|
||||
if (jsonLen > 0) {
|
||||
const json = new TextDecoder().decode(data.slice(offset, offset + jsonLen));
|
||||
try {
|
||||
recordData = JSON.parse(json);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
offset += jsonLen;
|
||||
|
||||
// 验证 CRC（跨记录数据计算，不含 CRC 自身）
|
||||
const storedCrc = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
const recordBytes = data.slice(recordStart, offset - 4);
|
||||
let computedCrc = 0;
|
||||
for (let i = 0; i < recordBytes.length; i++) {
|
||||
computedCrc = ((computedCrc << 5) - computedCrc + recordBytes[i]) | 0;
|
||||
}
|
||||
if ((computedCrc >>> 0) !== storedCrc) {
|
||||
// CRC 不匹配，跳过此损坏记录
|
||||
console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`);
|
||||
continue;
|
||||
}
|
||||
|
||||
records.push({
|
||||
lsn,
|
||||
type,
|
||||
txnId,
|
||||
tableName,
|
||||
key,
|
||||
data: recordData,
|
||||
checksum: storedCrc,
|
||||
});
|
||||
} catch {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return records;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* AriaEngine WAL — Write-Ahead Log
|
||||
* @module engine/aria/wal/log
|
||||
*
|
||||
* 崩溃恢复前的写操作持久化日志。
|
||||
*
|
||||
* WAL 文件格式:
|
||||
* ┌──────────┬──────────────┬──────────┐
|
||||
* │ Record 1│ Record 2 │ ... │
|
||||
* │ 4B LSN │ │ │
|
||||
* │ 1B type │ │ │
|
||||
* │ 4B txnId│ │ │
|
||||
* │ 2B tblLen│ │ │
|
||||
* │ N table│ │ │
|
||||
* │ 2B keyLen│ │ │
|
||||
* │ N key │ │ │
|
||||
* │ 4B jsonLen│ │ │
|
||||
* │ N json │ │ │
|
||||
* │ 4B CRC │ │ │
|
||||
* └──────────┴──────────────┴──────────┘
|
||||
*/
|
||||
|
||||
import { WALRecordType, type WALRecord } from '../types';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// WAL 存储接口
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface WALStore {
|
||||
/** 追加 WAL 记录 */
|
||||
append(data: Uint8Array): Promise<void>;
|
||||
/** 读取所有 WAL 记录 */
|
||||
readAll(): Promise<Uint8Array>;
|
||||
/** 截断 WAL（checkpoint 后清理） */
|
||||
truncate(): Promise<void>;
|
||||
/** 检查 WAL 是否存在 */
|
||||
exists(): Promise<boolean>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// WAL
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class WAL {
|
||||
private lsn = 0;
|
||||
private store: WALStore;
|
||||
private enabled: boolean;
|
||||
private buffer: Uint8Array[] = [];
|
||||
private syncMode: 'full' | 'batch' | 'none';
|
||||
|
||||
constructor(store: WALStore, enabled: boolean = true, syncMode: 'full' | 'batch' | 'none' = 'batch') {
|
||||
this.store = store;
|
||||
this.enabled = enabled;
|
||||
this.syncMode = syncMode;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 写入
|
||||
// =======================================================================
|
||||
|
||||
/** 追加一条 WAL 记录（full 模式同步等待写入完成） */
|
||||
async append(record: Omit<WALRecord, 'lsn' | 'checksum'>): Promise<void> {
|
||||
if (!this.enabled) return;
|
||||
|
||||
this.lsn++;
|
||||
const fullRecord: WALRecord = {
|
||||
...record,
|
||||
lsn: this.lsn,
|
||||
checksum: 0, // 稍后计算
|
||||
};
|
||||
|
||||
const bytes = this.encodeRecord(fullRecord);
|
||||
|
||||
if (this.syncMode === 'full') {
|
||||
try {
|
||||
await this.store.append(bytes);
|
||||
} catch {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('[AriaEngine WAL] Failed to append record');
|
||||
}
|
||||
} else if (this.syncMode === 'batch') {
|
||||
this.buffer.push(bytes);
|
||||
}
|
||||
// 'none' mode: 不写 WAL
|
||||
}
|
||||
|
||||
/** 批量追加多条 WAL 记录（组提交：合并为一次底层写入，v0.3.1） */
|
||||
async appendBatch(records: Omit<WALRecord, 'lsn' | 'checksum'>[]): Promise<void> {
|
||||
if (!this.enabled || records.length === 0) return;
|
||||
|
||||
const chunks: Uint8Array[] = [];
|
||||
for (const record of records) {
|
||||
this.lsn++;
|
||||
chunks.push(this.encodeRecord({ ...record, lsn: this.lsn, checksum: 0 }));
|
||||
}
|
||||
const combined = this.mergeChunks(chunks);
|
||||
|
||||
if (this.syncMode === 'full') {
|
||||
try {
|
||||
await this.store.append(combined);
|
||||
} catch {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('[AriaEngine WAL] Failed to append batch record');
|
||||
}
|
||||
} else if (this.syncMode === 'batch') {
|
||||
this.buffer.push(combined);
|
||||
}
|
||||
// 'none' mode: 不写 WAL
|
||||
}
|
||||
|
||||
/** 批量刷新缓冲的 WAL 记录 */
|
||||
async flush(): Promise<void> {
|
||||
if (!this.enabled || this.buffer.length === 0) return;
|
||||
|
||||
const combined = this.mergeChunks(this.buffer);
|
||||
await this.store.append(combined);
|
||||
this.buffer = [];
|
||||
}
|
||||
|
||||
/** 合并多个字节块为一个连续缓冲区 */
|
||||
private mergeChunks(chunks: Uint8Array[]): Uint8Array {
|
||||
if (chunks.length === 1) return chunks[0];
|
||||
const totalLen = chunks.reduce((sum, b) => sum + b.byteLength, 0);
|
||||
const combined = new Uint8Array(totalLen);
|
||||
let offset = 0;
|
||||
for (const buf of chunks) {
|
||||
combined.set(buf, offset);
|
||||
offset += buf.byteLength;
|
||||
}
|
||||
return combined;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 恢复
|
||||
// =======================================================================
|
||||
|
||||
/** 从 WAL 恢复未提交的事务数据 */
|
||||
async recover(
|
||||
applyRecord: (record: WALRecord) => void,
|
||||
): Promise<number> {
|
||||
if (!this.enabled) return 0;
|
||||
|
||||
const exists = await this.store.exists();
|
||||
if (!exists) return 0;
|
||||
|
||||
const data = await this.store.readAll();
|
||||
if (data.byteLength === 0) return 0;
|
||||
|
||||
const records = this.decodeAllRecords(data);
|
||||
for (const record of records) {
|
||||
applyRecord(record);
|
||||
}
|
||||
|
||||
this.lsn = records.length > 0 ? records[records.length - 1].lsn : 0;
|
||||
return records.length;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// Checkpoint
|
||||
// =======================================================================
|
||||
|
||||
/** Checkpoint 后清空 WAL */
|
||||
async checkpoint(): Promise<void> {
|
||||
if (!this.enabled) return;
|
||||
await this.flush();
|
||||
await this.store.truncate();
|
||||
this.lsn = 0;
|
||||
}
|
||||
|
||||
// =======================================================================
|
||||
// 统计
|
||||
// =======================================================================
|
||||
|
||||
isEnabled(): boolean {
|
||||
return this.enabled;
|
||||
}
|
||||
|
||||
getLSN(): number {
|
||||
return this.lsn;
|
||||
}
|
||||
|
||||
getBufferedCount(): number {
|
||||
return this.buffer.length;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// 编解码
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private encodeRecord(record: WALRecord): Uint8Array {
|
||||
const encoder = new TextEncoder();
|
||||
const tableBytes = encoder.encode(record.tableName);
|
||||
const keyBytes = encoder.encode(record.key);
|
||||
const jsonStr = record.data ? JSON.stringify(record.data) : '';
|
||||
const jsonBytes = encoder.encode(jsonStr);
|
||||
|
||||
const size =
|
||||
4 + // LSN
|
||||
1 + // type
|
||||
4 + // txnId
|
||||
2 + tableBytes.length + // table
|
||||
2 + keyBytes.length + // key
|
||||
4 + jsonBytes.length + // json
|
||||
4; // CRC
|
||||
|
||||
const buf = new ArrayBuffer(size);
|
||||
const view = new DataView(buf);
|
||||
let offset = 0;
|
||||
|
||||
view.setUint32(offset, record.lsn, false);
|
||||
offset += 4;
|
||||
view.setUint8(offset, record.type);
|
||||
offset += 1;
|
||||
view.setUint32(offset, record.txnId, false);
|
||||
offset += 4;
|
||||
|
||||
view.setUint16(offset, tableBytes.length, false);
|
||||
offset += 2;
|
||||
new Uint8Array(buf).set(tableBytes, offset);
|
||||
offset += tableBytes.length;
|
||||
|
||||
view.setUint16(offset, keyBytes.length, false);
|
||||
offset += 2;
|
||||
new Uint8Array(buf).set(keyBytes, offset);
|
||||
offset += keyBytes.length;
|
||||
|
||||
view.setUint32(offset, jsonBytes.length, false);
|
||||
offset += 4;
|
||||
new Uint8Array(buf).set(jsonBytes, offset);
|
||||
offset += jsonBytes.length;
|
||||
|
||||
// 简单 CRC
|
||||
let crc = 0;
|
||||
const u8 = new Uint8Array(buf, 0, offset);
|
||||
for (let i = 0; i < u8.length; i++) {
|
||||
crc = ((crc << 5) - crc + u8[i]) | 0;
|
||||
}
|
||||
view.setUint32(offset, crc >>> 0, false);
|
||||
|
||||
return new Uint8Array(buf);
|
||||
}
|
||||
|
||||
private decodeAllRecords(data: Uint8Array): WALRecord[] {
|
||||
const records: WALRecord[] = [];
|
||||
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
||||
let offset = 0;
|
||||
|
||||
while (offset + 15 <= data.byteLength) {
|
||||
try {
|
||||
const recordStart = offset;
|
||||
const lsn = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
const type = view.getUint8(offset) as WALRecordType;
|
||||
offset += 1;
|
||||
const txnId = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
|
||||
const tableLen = view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
if (offset + tableLen > data.byteLength) break;
|
||||
const tableName = new TextDecoder().decode(data.slice(offset, offset + tableLen));
|
||||
offset += tableLen;
|
||||
|
||||
const keyLen = view.getUint16(offset, false);
|
||||
offset += 2;
|
||||
if (offset + keyLen > data.byteLength) break;
|
||||
const key = new TextDecoder().decode(data.slice(offset, offset + keyLen));
|
||||
offset += keyLen;
|
||||
|
||||
const jsonLen = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
if (offset + jsonLen > data.byteLength) break;
|
||||
let recordData: Record<string, unknown> | undefined;
|
||||
if (jsonLen > 0) {
|
||||
const json = new TextDecoder().decode(data.slice(offset, offset + jsonLen));
|
||||
try {
|
||||
recordData = JSON.parse(json);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
offset += jsonLen;
|
||||
|
||||
// 验证 CRC（跨记录数据计算，不含 CRC 自身）
|
||||
const storedCrc = view.getUint32(offset, false);
|
||||
offset += 4;
|
||||
const recordBytes = data.slice(recordStart, offset - 4);
|
||||
let computedCrc = 0;
|
||||
for (let i = 0; i < recordBytes.length; i++) {
|
||||
computedCrc = ((computedCrc << 5) - computedCrc + recordBytes[i]) | 0;
|
||||
}
|
||||
if ((computedCrc >>> 0) !== storedCrc) {
|
||||
// CRC 不匹配，跳过此损坏记录
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(`[AriaEngine WAL] CRC mismatch at record LSN=${lsn}, skipping`);
|
||||
continue;
|
||||
}
|
||||
|
||||
records.push({
|
||||
lsn,
|
||||
type,
|
||||
txnId,
|
||||
tableName,
|
||||
key,
|
||||
data: recordData,
|
||||
checksum: storedCrc,
|
||||
});
|
||||
} catch {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return records;
|
||||
}
|
||||
}
|
||||
|
||||
+11
-11
@@ -1,11 +1,11 @@
|
||||
/**
|
||||
* metona-sqlark Engine — 存储引擎层
|
||||
* @module engine
|
||||
*/
|
||||
|
||||
export type { IStorageEngine } from './interface';
|
||||
export { MemoryEngine } from './memory';
|
||||
export { IndexedDBEngine } from './indexeddb';
|
||||
export { OPFSEngine } from './opfs';
|
||||
export { AriaEngine } from './aria/index';
|
||||
export type { AriaEngineConfig } from './aria/types';
|
||||
/**
|
||||
* metona-sqlark Engine — 存储引擎层
|
||||
* @module engine
|
||||
*/
|
||||
|
||||
export type { IStorageEngine } from './interface';
|
||||
export { MemoryEngine } from './memory';
|
||||
export { IndexedDBEngine } from './indexeddb';
|
||||
export { OPFSEngine } from './opfs';
|
||||
export { AriaEngine } from './aria/index';
|
||||
export type { AriaEngineConfig } from './aria/types';
|
||||
|
||||
+153
-4
@@ -27,7 +27,7 @@ export class IndexedDBEngine implements IStorageEngine {
|
||||
await this.memoryCache.open(dbName, version);
|
||||
return new Promise((resolve, reject) => {
|
||||
const request = indexedDB.open(dbName, version);
|
||||
request.onsuccess = () => {
|
||||
request.onsuccess = async () => {
|
||||
this.db = request.result;
|
||||
// 多标签页冲突处理：其他标签页升级版本时自动关闭当前连接
|
||||
this.db.onversionchange = () => {
|
||||
@@ -38,13 +38,89 @@ export class IndexedDBEngine implements IStorageEngine {
|
||||
console.warn(`[metona-sqlark] Database "${dbName}" was upgraded in another tab. Connection closed. Please re-open.`);
|
||||
}
|
||||
};
|
||||
resolve();
|
||||
try {
|
||||
// v0.3.2: reopen 后从 IDB 重建 schema（schema 此前只存内存缓存，重开连接即丢失）
|
||||
await this.rebuildSchemaFromIDB();
|
||||
resolve();
|
||||
} catch (error) {
|
||||
reject(new DatabaseError(`Failed to restore schema for "${dbName}"`, 'IDB_SCHEMA_RESTORE_ERROR', error));
|
||||
}
|
||||
};
|
||||
request.onerror = () => reject(new DatabaseError(`Failed to open IndexedDB "${dbName}"`, 'IDB_OPEN_ERROR', request.error));
|
||||
request.onblocked = () => reject(new DatabaseError(`IndexedDB "${dbName}" is blocked`, 'IDB_BLOCKED'));
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 从 IDB 恢复内存 schema：
|
||||
* 1. 优先读取持久化的 schema 记录（'__metona_schema' store，v0.3.2）
|
||||
* 2. 旧数据回退：从 objectStore 主键 / 索引 / 样例数据推断
|
||||
*/
|
||||
private async rebuildSchemaFromIDB(): Promise<void> {
|
||||
const db = this.ensureDB();
|
||||
|
||||
// 1. 持久化 schema
|
||||
if (db.objectStoreNames.contains('__metona_schema')) {
|
||||
const records: { name: string; schema: string }[] = await new Promise((resolve, reject) => {
|
||||
const req = db.transaction('__metona_schema', 'readonly').objectStore('__metona_schema').getAll();
|
||||
req.onsuccess = () => resolve((req.result ?? []) as { name: string; schema: string }[]);
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
for (const rec of records) {
|
||||
try {
|
||||
const schema = JSON.parse(rec.schema) as TableSchema;
|
||||
if (!(await this.memoryCache.getTableSchema(schema.name))) {
|
||||
await this.memoryCache.createTable(schema);
|
||||
}
|
||||
} catch {
|
||||
// 损坏的 schema 记录忽略
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 2. 回退：无持久化 schema 的表从 IDB 结构推断
|
||||
const storeNames = Array.from(db.objectStoreNames).filter((n) => n !== '__metona_schema');
|
||||
for (const tableName of storeNames) {
|
||||
// 已有 schema（持久化恢复或连续 open）则跳过
|
||||
const existing = await this.memoryCache.getTableSchema(tableName);
|
||||
if (existing) continue;
|
||||
|
||||
const columns: Record<string, import('../constants').ColumnDef> = {};
|
||||
const tx = db.transaction(tableName, 'readonly');
|
||||
const store = tx.objectStore(tableName);
|
||||
|
||||
// 主键列
|
||||
const pk = store.keyPath as string;
|
||||
columns[pk] = { type: 'string', primaryKey: true };
|
||||
|
||||
// 索引列（idx_ 前缀约定）
|
||||
for (const idxName of Array.from(store.indexNames)) {
|
||||
if (idxName.startsWith('idx_')) {
|
||||
const col = idxName.slice(4);
|
||||
if (!columns[col]) {
|
||||
columns[col] = { type: 'string', index: true };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 从样例数据推断其余列的类型
|
||||
const rows: Record<string, unknown>[] = await new Promise((resolve, reject) => {
|
||||
const req = store.getAll();
|
||||
req.onsuccess = () => resolve((req.result ?? []) as Record<string, unknown>[]);
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
if (rows.length > 0) {
|
||||
for (const [key, value] of Object.entries(rows[0])) {
|
||||
if (!columns[key]) {
|
||||
columns[key] = { type: inferFieldType(value) };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
await this.memoryCache.createTable({ name: tableName, columns });
|
||||
}
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
if (this.db) {
|
||||
this.db.onversionchange = null; // 清理监听器
|
||||
@@ -113,6 +189,48 @@ export class IndexedDBEngine implements IStorageEngine {
|
||||
await this.idbClear(tableName);
|
||||
}
|
||||
|
||||
// ---- 动态索引（v0.3.0）：通过版本升级创建/删除 IDB 索引 ----
|
||||
|
||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||
await this.memoryCache.createIndex(tableName, column, unique);
|
||||
if (this.txActive) return;
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const newVersion = db.version + 1; db.close();
|
||||
const request = indexedDB.open(this.dbName, newVersion);
|
||||
request.onupgradeneeded = (event) => {
|
||||
const idb = (event.target as IDBOpenDBRequest).result;
|
||||
const tx = idb.transaction(tableName, 'readwrite');
|
||||
const store = tx.objectStore(tableName);
|
||||
if (!store.indexNames.contains(`idx_${column}`)) {
|
||||
store.createIndex(`idx_${column}`, column, { unique: unique ?? false });
|
||||
}
|
||||
};
|
||||
request.onsuccess = () => { this.db = request.result; resolve(); };
|
||||
request.onerror = () => reject(new DatabaseError(`Failed to create index "${tableName}.${column}"`, 'IDB_UPGRADE_ERROR', request.error));
|
||||
});
|
||||
}
|
||||
|
||||
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
|
||||
await this.memoryCache.dropIndex(tableName, column);
|
||||
if (this.txActive) return;
|
||||
const db = this.ensureDB();
|
||||
return new Promise((resolve, reject) => {
|
||||
const newVersion = db.version + 1; db.close();
|
||||
const request = indexedDB.open(this.dbName, newVersion);
|
||||
request.onupgradeneeded = (event) => {
|
||||
const idb = (event.target as IDBOpenDBRequest).result;
|
||||
const tx = idb.transaction(tableName, 'readwrite');
|
||||
const store = tx.objectStore(tableName);
|
||||
if (store.indexNames.contains(`idx_${column}`)) {
|
||||
store.deleteIndex(`idx_${column}`);
|
||||
}
|
||||
};
|
||||
request.onsuccess = () => { this.db = request.result; resolve(); };
|
||||
request.onerror = () => reject(new DatabaseError(`Failed to drop index "${tableName}.${column}"`, 'IDB_UPGRADE_ERROR', request.error));
|
||||
});
|
||||
}
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
async beginTransaction(): Promise<void> {
|
||||
@@ -154,8 +272,19 @@ export class IndexedDBEngine implements IStorageEngine {
|
||||
store.createIndex(`idx_${colName}`, colName, { unique: colDef.unique ?? false });
|
||||
}
|
||||
}
|
||||
// v0.3.2: schema 持久化 store（记录在升级完成后的 onsuccess 写入）
|
||||
if (!db.objectStoreNames.contains('__metona_schema')) {
|
||||
db.createObjectStore('__metona_schema', { keyPath: 'name' });
|
||||
}
|
||||
};
|
||||
request.onsuccess = () => {
|
||||
this.db = request.result;
|
||||
// v0.3.2: 升级完成后持久化 schema（upgrade 事务内异步写会失败）
|
||||
const schemaTx = this.db.transaction('__metona_schema', 'readwrite');
|
||||
schemaTx.objectStore('__metona_schema').put({ name: schema.name, schema: JSON.stringify(schema) });
|
||||
schemaTx.oncomplete = () => resolve();
|
||||
schemaTx.onerror = () => reject(new DatabaseError(`Failed to persist schema for "${schema.name}"`, 'IDB_SCHEMA_ERROR', schemaTx.error));
|
||||
};
|
||||
request.onsuccess = () => { this.db = request.result; resolve(); };
|
||||
request.onerror = () => reject(new DatabaseError(`Failed to create table "${schema.name}"`, 'IDB_UPGRADE_ERROR', request.error));
|
||||
});
|
||||
}
|
||||
@@ -169,7 +298,16 @@ export class IndexedDBEngine implements IStorageEngine {
|
||||
const db = (event.target as IDBOpenDBRequest).result;
|
||||
if (db.objectStoreNames.contains(tableName)) db.deleteObjectStore(tableName);
|
||||
};
|
||||
request.onsuccess = () => { this.db = request.result; resolve(); };
|
||||
request.onsuccess = () => {
|
||||
this.db = request.result;
|
||||
// v0.3.2: 清理持久化 schema 记录（upgrade 后执行，失败不阻塞删除）
|
||||
if (this.db.objectStoreNames.contains('__metona_schema')) {
|
||||
const schemaTx = this.db.transaction('__metona_schema', 'readwrite');
|
||||
schemaTx.objectStore('__metona_schema').delete(tableName);
|
||||
schemaTx.onerror = () => { /* 忽略：旧库可能无此记录 */ };
|
||||
}
|
||||
resolve();
|
||||
};
|
||||
request.onerror = () => reject(new DatabaseError(`Failed to drop table "${tableName}"`, 'IDB_UPGRADE_ERROR', request.error));
|
||||
});
|
||||
}
|
||||
@@ -354,3 +492,14 @@ export class IndexedDBEngine implements IStorageEngine {
|
||||
return this.db;
|
||||
}
|
||||
}
|
||||
|
||||
/** 从存储值推断字段类型（schema 重建用，v0.3.2） */
|
||||
function inferFieldType(value: unknown): import('../constants').FieldType {
|
||||
if (typeof value === 'number') return 'number';
|
||||
if (typeof value === 'boolean') return 'boolean';
|
||||
if (typeof value === 'object' && value !== null) return 'json';
|
||||
if (typeof value === 'string') {
|
||||
return isNaN(Date.parse(value)) ? 'string' : 'string';
|
||||
}
|
||||
return 'string';
|
||||
}
|
||||
|
||||
@@ -55,6 +55,14 @@ export interface IStorageEngine {
|
||||
/** 清空表数据（保留结构） */
|
||||
clear(tableName: string): Promise<void>;
|
||||
|
||||
// ---- 动态索引（可选，v0.3.0） ----
|
||||
|
||||
/** 创建二级索引（CREATE INDEX） */
|
||||
createIndex?(tableName: string, column: string, unique?: boolean): Promise<void>;
|
||||
|
||||
/** 删除二级索引（DROP INDEX） */
|
||||
dropIndex?(tableName: string, column: string, indexName?: string): Promise<void>;
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
/** 开始事务 */
|
||||
|
||||
+36
-2
@@ -148,6 +148,41 @@ export class MemoryEngine implements IStorageEngine {
|
||||
if (tableIndexes) for (const colIndex of tableIndexes.values()) colIndex.clear();
|
||||
}
|
||||
|
||||
// ---- 动态索引（v0.3.0） ----
|
||||
|
||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||
this.ensureTable(tableName);
|
||||
const schema = this.schemas.get(tableName)!;
|
||||
const colDef = schema.columns[column];
|
||||
if (!colDef) throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
||||
if (colDef.index || colDef.unique) return; // 已存在
|
||||
colDef.index = true;
|
||||
if (unique) colDef.unique = true;
|
||||
|
||||
const tableIndexes = this.indexes.get(tableName)!;
|
||||
if (!tableIndexes.has(column)) tableIndexes.set(column, new Map());
|
||||
const colIndex = tableIndexes.get(column)!;
|
||||
const table = this.tables.get(tableName)!;
|
||||
for (const [pk, row] of table) {
|
||||
const value = row[column];
|
||||
if (value !== undefined && value !== null) {
|
||||
if (!colIndex.has(value)) colIndex.set(value, new Set());
|
||||
colIndex.get(value)!.add(pk);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async dropIndex(tableName: string, column: string, _indexName?: string): Promise<void> {
|
||||
this.ensureTable(tableName);
|
||||
const schema = this.schemas.get(tableName)!;
|
||||
const colDef = schema.columns[column];
|
||||
if (!colDef) throw new DatabaseError(`Column "${column}" does not exist in table "${tableName}"`, 'COLUMN_NOT_FOUND');
|
||||
colDef.index = false;
|
||||
colDef.unique = false;
|
||||
const tableIndexes = this.indexes.get(tableName);
|
||||
if (tableIndexes) tableIndexes.delete(column);
|
||||
}
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
async beginTransaction(): Promise<void> {
|
||||
@@ -297,10 +332,9 @@ export class MemoryEngine implements IStorageEngine {
|
||||
for (const [colName, colDef] of Object.entries(refSchema.columns)) {
|
||||
if (!colDef.references || !colDef.onDelete) continue;
|
||||
|
||||
const [refTable, refCol] = colDef.references.split('.');
|
||||
const [refTable] = colDef.references.split('.');
|
||||
if (refTable !== tableName) continue;
|
||||
|
||||
const refPkCol = refCol;
|
||||
const refTableData = this.tables.get(refTableName);
|
||||
if (!refTableData) continue;
|
||||
|
||||
|
||||
@@ -132,6 +132,16 @@ export class OPFSEngine implements IStorageEngine {
|
||||
await this.writeTableData(tableName, []);
|
||||
}
|
||||
|
||||
// ---- 动态索引（v0.3.0） ----
|
||||
|
||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||
return this.memoryCache.createIndex(tableName, column, unique);
|
||||
}
|
||||
|
||||
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
|
||||
return this.memoryCache.dropIndex(tableName, column, indexName);
|
||||
}
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
async beginTransaction(): Promise<void> {
|
||||
|
||||
+37
-6
@@ -25,6 +25,8 @@ export class HybridEngine implements IStorageEngine {
|
||||
private memoryEngine: MemoryEngine;
|
||||
private diskEngine: IStorageEngine;
|
||||
private diskEngineType: DiskEngine;
|
||||
private dbName = '';
|
||||
private version = 1;
|
||||
|
||||
constructor(diskEngine: DiskEngine = 'indexeddb') {
|
||||
this.memoryEngine = new MemoryEngine();
|
||||
@@ -35,6 +37,8 @@ export class HybridEngine implements IStorageEngine {
|
||||
// ---- 生命周期 ----
|
||||
|
||||
async open(dbName: string, version: number): Promise<void> {
|
||||
this.dbName = dbName;
|
||||
this.version = version;
|
||||
// 先打开磁盘引擎
|
||||
await this.diskEngine.open(dbName, version);
|
||||
|
||||
@@ -42,6 +46,17 @@ export class HybridEngine implements IStorageEngine {
|
||||
await this.memoryEngine.open(dbName, version);
|
||||
|
||||
// 从磁盘加载现存表
|
||||
await this.reloadMemoryFromDisk();
|
||||
}
|
||||
|
||||
/**
|
||||
* 从磁盘重载内存缓存（v0.3.2：多标签页同步）。
|
||||
* 其他标签页写入磁盘后调用，使本标签页读到最新数据。
|
||||
*/
|
||||
async reloadMemoryFromDisk(): Promise<void> {
|
||||
await this.memoryEngine.close();
|
||||
await this.memoryEngine.open(this.dbName, this.version);
|
||||
|
||||
const tableNames = await this.diskEngine.getTableNames();
|
||||
for (const tableName of tableNames) {
|
||||
const schema = await this.diskEngine.getTableSchema(tableName);
|
||||
@@ -53,12 +68,12 @@ export class HybridEngine implements IStorageEngine {
|
||||
// 从磁盘加载数据到内存
|
||||
const rows = await this.diskEngine.find(tableName, { table: tableName });
|
||||
if (rows.length > 0) {
|
||||
try {
|
||||
await this.memoryEngine.insert(tableName, rows);
|
||||
} catch (e) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(`[metona-sqlark] Failed to load table "${tableName}" data from disk:`, e);
|
||||
}
|
||||
try {
|
||||
await this.memoryEngine.insert(tableName, rows);
|
||||
} catch (e) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(`[metona-sqlark] Failed to load table "${tableName}" data from disk:`, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -133,6 +148,22 @@ export class HybridEngine implements IStorageEngine {
|
||||
await this.diskEngine.clear(tableName);
|
||||
}
|
||||
|
||||
// ---- 动态索引（v0.3.0） ----
|
||||
|
||||
async createIndex(tableName: string, column: string, unique?: boolean): Promise<void> {
|
||||
await this.memoryEngine.createIndex(tableName, column, unique);
|
||||
if (typeof this.diskEngine.createIndex === 'function') {
|
||||
await this.diskEngine.createIndex(tableName, column, unique);
|
||||
}
|
||||
}
|
||||
|
||||
async dropIndex(tableName: string, column: string, indexName?: string): Promise<void> {
|
||||
await this.memoryEngine.dropIndex(tableName, column, indexName);
|
||||
if (typeof this.diskEngine.dropIndex === 'function') {
|
||||
await this.diskEngine.dropIndex(tableName, column, indexName);
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 事务 ----
|
||||
|
||||
async beginTransaction(): Promise<void> {
|
||||
|
||||
+91
-91
@@ -1,91 +1,91 @@
|
||||
/**
|
||||
* metona-sqlark — 入口文件
|
||||
* @module metona-sqlark
|
||||
* @version 0.2.5
|
||||
*
|
||||
* 前端关系型数据库，内存与磁盘双模式。
|
||||
* 支持 Query Builder 链式 API 和 SQL 字符串查询。
|
||||
*/
|
||||
|
||||
import { MetonaSqlark } from './core';
|
||||
import type { DatabaseConfig } from './constants';
|
||||
import { VERSION } from './constants';
|
||||
|
||||
// 连接池管理器（side-effect: 注入 MetonaSqlark.connect / disconnect 等静态方法）
|
||||
import './connection-manager';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 工厂函数
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 创建数据库实例并初始化
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const db = await MetonaSqlark.create({
|
||||
* name: 'my-app',
|
||||
* mode: 'hybrid',
|
||||
* });
|
||||
*
|
||||
* await db.defineTable('users', {
|
||||
* id: { type: 'string', primaryKey: true },
|
||||
* name: { type: 'string', required: true },
|
||||
* });
|
||||
*
|
||||
* await db.table('users').insert({ id: '1', name: 'Alice' });
|
||||
* const results = await db.query('SELECT * FROM users');
|
||||
* ```
|
||||
*/
|
||||
async function create(config: DatabaseConfig): Promise<MetonaSqlark> {
|
||||
const db = new MetonaSqlark(config);
|
||||
await db.init();
|
||||
return db;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 全局 API
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const api = {
|
||||
VERSION,
|
||||
version: VERSION,
|
||||
create,
|
||||
MetonaSqlark,
|
||||
MeSqlark: MetonaSqlark,
|
||||
};
|
||||
|
||||
// 浏览器全局挂载
|
||||
declare global { interface Window { MetonaSqlark: typeof api; MeSqlark: typeof api; } }
|
||||
if (typeof window !== 'undefined') {
|
||||
window.MetonaSqlark = api;
|
||||
window.MeSqlark = api;
|
||||
}
|
||||
|
||||
export default api;
|
||||
export {
|
||||
api,
|
||||
VERSION,
|
||||
create,
|
||||
MetonaSqlark,
|
||||
};
|
||||
|
||||
// 别名
|
||||
export const MeSqlark = MetonaSqlark;
|
||||
|
||||
// 类型导出
|
||||
export type { DatabaseConfig, TableSchema, ColumnDef, FieldType, StorageMode, DiskEngine } from './constants';
|
||||
export type { IStorageEngine } from './engine/interface';
|
||||
export type { Statement, SelectStatement, InsertStatement, UpdateStatement, DeleteStatement } from './query/ast';
|
||||
export { MemoryEngine } from './engine/memory';
|
||||
export { IndexedDBEngine } from './engine/indexeddb';
|
||||
export { OPFSEngine } from './engine/opfs';
|
||||
export { AriaEngine } from './engine/aria/index';
|
||||
export { HybridEngine } from './hybrid/index';
|
||||
export { Table } from './table/table';
|
||||
export { parse } from './sql/parser';
|
||||
export { tokenize } from './sql/lexer';
|
||||
|
||||
// AriaEngine 类型 & 后端
|
||||
export type { AriaEngineConfig } from './engine/aria/types';
|
||||
export { OPFSBackend } from './engine/aria/store/opfs_backend';
|
||||
/**
|
||||
* metona-sqlark — 入口文件
|
||||
* @module metona-sqlark
|
||||
* @version 0.2.5
|
||||
*
|
||||
* 前端关系型数据库，内存与磁盘双模式。
|
||||
* 支持 Query Builder 链式 API 和 SQL 字符串查询。
|
||||
*/
|
||||
|
||||
import { MetonaSqlark } from './core';
|
||||
import type { DatabaseConfig } from './constants';
|
||||
import { VERSION } from './constants';
|
||||
|
||||
// 连接池管理器（side-effect: 注入 MetonaSqlark.connect / disconnect 等静态方法）
|
||||
import './connection-manager';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 工厂函数
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* 创建数据库实例并初始化
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const db = await MetonaSqlark.create({
|
||||
* name: 'my-app',
|
||||
* mode: 'hybrid',
|
||||
* });
|
||||
*
|
||||
* await db.defineTable('users', {
|
||||
* id: { type: 'string', primaryKey: true },
|
||||
* name: { type: 'string', required: true },
|
||||
* });
|
||||
*
|
||||
* await db.table('users').insert({ id: '1', name: 'Alice' });
|
||||
* const results = await db.query('SELECT * FROM users');
|
||||
* ```
|
||||
*/
|
||||
async function create(config: DatabaseConfig): Promise<MetonaSqlark> {
|
||||
const db = new MetonaSqlark(config);
|
||||
await db.init();
|
||||
return db;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 全局 API
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const api = {
|
||||
VERSION,
|
||||
version: VERSION,
|
||||
create,
|
||||
MetonaSqlark,
|
||||
MeSqlark: MetonaSqlark,
|
||||
};
|
||||
|
||||
// 浏览器全局挂载
|
||||
declare global { interface Window { MetonaSqlark: typeof api; MeSqlark: typeof api; } }
|
||||
if (typeof window !== 'undefined') {
|
||||
window.MetonaSqlark = api;
|
||||
window.MeSqlark = api;
|
||||
}
|
||||
|
||||
export default api;
|
||||
export {
|
||||
api,
|
||||
VERSION,
|
||||
create,
|
||||
MetonaSqlark,
|
||||
};
|
||||
|
||||
// 别名
|
||||
export const MeSqlark = MetonaSqlark;
|
||||
|
||||
// 类型导出
|
||||
export type { DatabaseConfig, TableSchema, ColumnDef, FieldType, StorageMode, DiskEngine } from './constants';
|
||||
export type { IStorageEngine } from './engine/interface';
|
||||
export type { Statement, SelectStatement, InsertStatement, UpdateStatement, DeleteStatement } from './query/ast';
|
||||
export { MemoryEngine } from './engine/memory';
|
||||
export { IndexedDBEngine } from './engine/indexeddb';
|
||||
export { OPFSEngine } from './engine/opfs';
|
||||
export { AriaEngine } from './engine/aria/index';
|
||||
export { HybridEngine } from './hybrid/index';
|
||||
export { Table } from './table/table';
|
||||
export { parse, parseAll } from './sql/parser';
|
||||
export { tokenize } from './sql/lexer';
|
||||
|
||||
// AriaEngine 类型 & 后端
|
||||
export type { AriaEngineConfig } from './engine/aria/types';
|
||||
export { OPFSBackend } from './engine/aria/store/opfs_backend';
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
* }
|
||||
*/
|
||||
|
||||
import type { MetonaSqlark } from '../core';
|
||||
import { MetonaSqlark } from '../core';
|
||||
import { useState, useEffect, useCallback, useRef } from 'react';
|
||||
|
||||
/** useQuery: 执行 SQL 查询 */
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
* const { data, loading, refresh } = useSqlarkQuery(db, 'SELECT * FROM users');
|
||||
*/
|
||||
|
||||
import type { MetonaSqlark } from '../core';
|
||||
import { MetonaSqlark } from '../core';
|
||||
import { ref, watch, onMounted, type Ref } from 'vue';
|
||||
|
||||
/** useSqlarkQuery: 执行 SQL 查询 */
|
||||
|
||||
+275
-209
@@ -1,209 +1,275 @@
|
||||
/**
|
||||
* metona-sqlark Query AST — 查询抽象语法树类型定义
|
||||
* @module query/ast
|
||||
*
|
||||
* QueryBuilder 和 SQL Parser 统一输出此 AST，
|
||||
* Executor 只认 AST，保证两种查询接口行为一致。
|
||||
*/
|
||||
|
||||
import type { WhereCondition, OrderBy } from '../constants';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AST 语句类型枚举
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type StatementType =
|
||||
| 'SELECT'
|
||||
| 'EXPLAIN'
|
||||
| 'INSERT'
|
||||
| 'UPDATE'
|
||||
| 'DELETE'
|
||||
| 'CREATE_TABLE'
|
||||
| 'DROP_TABLE'
|
||||
| 'ALTER_TABLE'
|
||||
| 'TRUNCATE_TABLE';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 列引用
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 列引用，'*' 表示所有列；支持 'table.column' 格式 */
|
||||
export type ColumnRef = string;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// JOIN
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** JOIN 类型 */
|
||||
export type JoinType = 'INNER' | 'LEFT' | 'RIGHT' | 'CROSS';
|
||||
|
||||
/** JOIN 子句 */
|
||||
export interface JoinClause {
|
||||
type: JoinType;
|
||||
table: string;
|
||||
alias?: string;
|
||||
on: WhereCondition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 聚合函数
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 聚合函数类型 */
|
||||
export type AggregateFunc = 'COUNT' | 'SUM' | 'AVG' | 'MIN' | 'MAX';
|
||||
|
||||
/** 聚合表达式 */
|
||||
export interface AggregateExpression {
|
||||
type: 'AGGREGATE';
|
||||
func: AggregateFunc;
|
||||
column: string; // '*' for COUNT(*)
|
||||
alias?: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 子查询
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 子查询表达式 */
|
||||
export interface SubqueryExpression {
|
||||
type: 'SUBQUERY';
|
||||
statement: SelectStatement;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: CREATE TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ASTColumnDef {
|
||||
name: string;
|
||||
type: string;
|
||||
primaryKey?: boolean;
|
||||
unique?: boolean;
|
||||
required?: boolean;
|
||||
default?: unknown;
|
||||
index?: boolean;
|
||||
maxLength?: number;
|
||||
min?: number;
|
||||
max?: number;
|
||||
/** 外键引用 */
|
||||
references?: string;
|
||||
/** 级联删除 */
|
||||
onDelete?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
/** 级联更新 */
|
||||
onUpdate?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
}
|
||||
|
||||
export interface CreateTableStatement {
|
||||
type: 'CREATE_TABLE';
|
||||
name: string;
|
||||
columns: ASTColumnDef[];
|
||||
/** IF NOT EXISTS — 表已存在时不报错 */
|
||||
ifNotExists?: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: DROP TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface DropTableStatement {
|
||||
type: 'DROP_TABLE';
|
||||
name: string;
|
||||
/** IF EXISTS — 表不存在时不报错 */
|
||||
ifExists?: boolean;
|
||||
}
|
||||
|
||||
/** EXPLAIN 查询计划 */
|
||||
export interface ExplainStatement {
|
||||
type: 'EXPLAIN';
|
||||
query: Statement;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DML: INSERT
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface InsertStatement {
|
||||
type: 'INSERT';
|
||||
into: string;
|
||||
columns?: string[];
|
||||
values: unknown[][];
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DML: UPDATE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface UpdateStatement {
|
||||
type: 'UPDATE';
|
||||
table: string;
|
||||
sets: Record<string, unknown>;
|
||||
where: WhereCondition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DML: DELETE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface DeleteStatement {
|
||||
type: 'DELETE';
|
||||
from: string;
|
||||
where: WhereCondition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DQL: SELECT
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SelectStatement {
|
||||
type: 'SELECT';
|
||||
columns: ColumnRef[];
|
||||
distinct?: boolean;
|
||||
from: string;
|
||||
/** 主表别名 */
|
||||
alias?: string;
|
||||
/** JOIN 子句列表 */
|
||||
joins?: JoinClause[];
|
||||
where: WhereCondition;
|
||||
/** GROUP BY */
|
||||
groupBy?: string[];
|
||||
/** HAVING */
|
||||
having?: WhereCondition;
|
||||
orderBy?: OrderBy[];
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: ALTER TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface AlterTableStatement {
|
||||
type: 'ALTER_TABLE';
|
||||
name: string;
|
||||
action: 'ADD' | 'DROP';
|
||||
column: ASTColumnDef;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: TRUNCATE TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface TruncateTableStatement {
|
||||
type: 'TRUNCATE_TABLE';
|
||||
name: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AST 联合类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type Statement =
|
||||
| SelectStatement
|
||||
| ExplainStatement
|
||||
| InsertStatement
|
||||
| UpdateStatement
|
||||
| DeleteStatement
|
||||
| CreateTableStatement
|
||||
| DropTableStatement
|
||||
| AlterTableStatement
|
||||
| TruncateTableStatement;
|
||||
/**
|
||||
* metona-sqlark Query AST — 查询抽象语法树类型定义
|
||||
* @module query/ast
|
||||
*
|
||||
* QueryBuilder 和 SQL Parser 统一输出此 AST，
|
||||
* Executor 只认 AST，保证两种查询接口行为一致。
|
||||
*/
|
||||
|
||||
import type { WhereCondition, OrderBy } from '../constants';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AST 语句类型枚举
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type StatementType =
|
||||
| 'SELECT'
|
||||
| 'SELECT_UNION'
|
||||
| 'EXPLAIN'
|
||||
| 'INSERT'
|
||||
| 'UPDATE'
|
||||
| 'DELETE'
|
||||
| 'CREATE_TABLE'
|
||||
| 'DROP_TABLE'
|
||||
| 'ALTER_TABLE'
|
||||
| 'TRUNCATE_TABLE'
|
||||
| 'CREATE_INDEX'
|
||||
| 'DROP_INDEX'
|
||||
| 'BEGIN'
|
||||
| 'COMMIT'
|
||||
| 'ROLLBACK';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 列引用
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 列引用，'*' 表示所有列；支持 'table.column' 格式 */
|
||||
export type ColumnRef = string;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// JOIN
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** JOIN 类型 */
|
||||
export type JoinType = 'INNER' | 'LEFT' | 'RIGHT' | 'CROSS';
|
||||
|
||||
/** JOIN 子句 */
|
||||
export interface JoinClause {
|
||||
type: JoinType;
|
||||
table: string;
|
||||
alias?: string;
|
||||
on: WhereCondition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 聚合函数
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 聚合函数类型 */
|
||||
export type AggregateFunc = 'COUNT' | 'SUM' | 'AVG' | 'MIN' | 'MAX';
|
||||
|
||||
/** 聚合表达式 */
|
||||
export interface AggregateExpression {
|
||||
type: 'AGGREGATE';
|
||||
func: AggregateFunc;
|
||||
column: string; // '*' for COUNT(*)
|
||||
alias?: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 子查询
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** 子查询表达式 */
|
||||
export interface SubqueryExpression {
|
||||
type: 'SUBQUERY';
|
||||
statement: SelectStatement | SelectUnionStatement;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: CREATE TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ASTColumnDef {
|
||||
name: string;
|
||||
type: string;
|
||||
primaryKey?: boolean;
|
||||
unique?: boolean;
|
||||
required?: boolean;
|
||||
default?: unknown;
|
||||
index?: boolean;
|
||||
maxLength?: number;
|
||||
min?: number;
|
||||
max?: number;
|
||||
/** 外键引用 */
|
||||
references?: string;
|
||||
/** 级联删除 */
|
||||
onDelete?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
/** 级联更新 */
|
||||
onUpdate?: 'CASCADE' | 'SET NULL' | 'RESTRICT';
|
||||
}
|
||||
|
||||
export interface CreateTableStatement {
|
||||
type: 'CREATE_TABLE';
|
||||
name: string;
|
||||
columns: ASTColumnDef[];
|
||||
/** IF NOT EXISTS — 表已存在时不报错 */
|
||||
ifNotExists?: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: DROP TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface DropTableStatement {
|
||||
type: 'DROP_TABLE';
|
||||
name: string;
|
||||
/** IF EXISTS — 表不存在时不报错 */
|
||||
ifExists?: boolean;
|
||||
}
|
||||
|
||||
/** EXPLAIN 查询计划 */
|
||||
export interface ExplainStatement {
|
||||
type: 'EXPLAIN';
|
||||
query: Statement;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DML: INSERT
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface InsertStatement {
|
||||
type: 'INSERT';
|
||||
into: string;
|
||||
columns?: string[];
|
||||
/** VALUES 字面量 */
|
||||
values?: unknown[][];
|
||||
/** INSERT INTO ... SELECT ...（v0.3.0） */
|
||||
select?: SelectStatement | SelectUnionStatement;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DML: UPDATE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface UpdateStatement {
|
||||
type: 'UPDATE';
|
||||
table: string;
|
||||
sets: Record<string, unknown>;
|
||||
where: WhereCondition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DML: DELETE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface DeleteStatement {
|
||||
type: 'DELETE';
|
||||
from: string;
|
||||
where: WhereCondition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DQL: SELECT
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SelectStatement {
|
||||
type: 'SELECT';
|
||||
columns: ColumnRef[];
|
||||
distinct?: boolean;
|
||||
from: string;
|
||||
/** 主表别名 */
|
||||
alias?: string;
|
||||
/** JOIN 子句列表 */
|
||||
joins?: JoinClause[];
|
||||
where: WhereCondition;
|
||||
/** GROUP BY */
|
||||
groupBy?: string[];
|
||||
/** HAVING */
|
||||
having?: WhereCondition;
|
||||
orderBy?: OrderBy[];
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DQL: UNION
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SelectUnionStatement {
|
||||
type: 'SELECT_UNION';
|
||||
/** 左操作数（可以是 SELECT 或嵌套 UNION） */
|
||||
left: SelectStatement | SelectUnionStatement;
|
||||
/** 右操作数 */
|
||||
right: SelectStatement | SelectUnionStatement;
|
||||
/** UNION ALL 不去重 */
|
||||
all?: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: ALTER TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface AlterTableStatement {
|
||||
type: 'ALTER_TABLE';
|
||||
name: string;
|
||||
action: 'ADD' | 'DROP';
|
||||
column: ASTColumnDef;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: TRUNCATE TABLE
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface TruncateTableStatement {
|
||||
type: 'TRUNCATE_TABLE';
|
||||
name: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DDL: CREATE INDEX / DROP INDEX（v0.3.0）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface CreateIndexStatement {
|
||||
type: 'CREATE_INDEX';
|
||||
/** 索引名（语法占位） */
|
||||
name: string;
|
||||
table: string;
|
||||
column: string;
|
||||
/** UNIQUE 索引 */
|
||||
unique?: boolean;
|
||||
}
|
||||
|
||||
export interface DropIndexStatement {
|
||||
type: 'DROP_INDEX';
|
||||
name: string;
|
||||
table: string;
|
||||
column: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// TCL: 事务语句（v0.3.0）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface BeginTransactionStatement {
|
||||
type: 'BEGIN';
|
||||
}
|
||||
|
||||
export interface CommitTransactionStatement {
|
||||
type: 'COMMIT';
|
||||
}
|
||||
|
||||
export interface RollbackTransactionStatement {
|
||||
type: 'ROLLBACK';
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AST 联合类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type Statement =
|
||||
| SelectStatement
|
||||
| SelectUnionStatement
|
||||
| ExplainStatement
|
||||
| InsertStatement
|
||||
| UpdateStatement
|
||||
| DeleteStatement
|
||||
| CreateTableStatement
|
||||
| DropTableStatement
|
||||
| AlterTableStatement
|
||||
| TruncateTableStatement
|
||||
| CreateIndexStatement
|
||||
| DropIndexStatement
|
||||
| BeginTransactionStatement
|
||||
| CommitTransactionStatement
|
||||
| RollbackTransactionStatement;
|
||||
|
||||
+16
-4
@@ -134,12 +134,16 @@ export class SelectQueryBuilder {
|
||||
|
||||
export class UpdateQueryBuilder {
|
||||
private _where: WhereCondition = {};
|
||||
private onWrite?: (table: string) => void;
|
||||
|
||||
constructor(
|
||||
private engine: IStorageEngine,
|
||||
private tableName: string,
|
||||
private _updates: Record<string, unknown>,
|
||||
) {}
|
||||
onWrite?: (table: string) => void,
|
||||
) {
|
||||
this.onWrite = onWrite;
|
||||
}
|
||||
|
||||
where(condition: WhereCondition): this {
|
||||
this._where = { ...this._where, ...condition };
|
||||
@@ -147,7 +151,9 @@ export class UpdateQueryBuilder {
|
||||
}
|
||||
|
||||
async execute(): Promise<number> {
|
||||
return this.engine.update(this.tableName, { table: this.tableName, where: this._where }, this._updates);
|
||||
const count = await this.engine.update(this.tableName, { table: this.tableName, where: this._where }, this._updates);
|
||||
this.onWrite?.(this.tableName);
|
||||
return count;
|
||||
}
|
||||
|
||||
toAST(): UpdateStatement {
|
||||
@@ -161,11 +167,15 @@ export class UpdateQueryBuilder {
|
||||
|
||||
export class DeleteQueryBuilder {
|
||||
private _where: WhereCondition = {};
|
||||
private onWrite?: (table: string) => void;
|
||||
|
||||
constructor(
|
||||
private engine: IStorageEngine,
|
||||
private tableName: string,
|
||||
) {}
|
||||
onWrite?: (table: string) => void,
|
||||
) {
|
||||
this.onWrite = onWrite;
|
||||
}
|
||||
|
||||
where(condition: WhereCondition): this {
|
||||
this._where = { ...this._where, ...condition };
|
||||
@@ -173,7 +183,9 @@ export class DeleteQueryBuilder {
|
||||
}
|
||||
|
||||
async execute(): Promise<number> {
|
||||
return this.engine.delete(this.tableName, { table: this.tableName, where: this._where });
|
||||
const count = await this.engine.delete(this.tableName, { table: this.tableName, where: this._where });
|
||||
this.onWrite?.(this.tableName);
|
||||
return count;
|
||||
}
|
||||
|
||||
toAST(): DeleteStatement {
|
||||
|
||||
+1061
-448
File diff suppressed because it is too large
Load Diff
@@ -42,6 +42,16 @@ export function matchWhere(
|
||||
options: { $col?: boolean } = {},
|
||||
): boolean {
|
||||
for (const [field, condition] of Object.entries(where)) {
|
||||
// 顶层 $caseResult（v0.3.2）：由 Executor 对 CASE WHEN 表达式逐行求值后产生
|
||||
if (field === '$caseResult') {
|
||||
if (condition !== true) return false;
|
||||
continue;
|
||||
}
|
||||
// 顶层 $exists（v0.3.0）：由 Executor.resolveSubqueries 解析为 boolean
|
||||
if (field === '$exists') {
|
||||
if (condition !== true) return false;
|
||||
continue;
|
||||
}
|
||||
// 顶层 $and
|
||||
if (field === '$and') {
|
||||
const subs = condition as WhereCondition[];
|
||||
@@ -54,6 +64,11 @@ export function matchWhere(
|
||||
if (!subs.some((sub) => matchWhere(row, sub, options))) return false;
|
||||
continue;
|
||||
}
|
||||
// 顶层 $not（v0.3.2 修复：NOT (expr) 生成的 { $not: inner }）
|
||||
if (field === '$not') {
|
||||
if (matchWhere(row, condition as WhereCondition, options)) return false;
|
||||
continue;
|
||||
}
|
||||
if (!matchField(row[field], condition, row, options)) return false;
|
||||
}
|
||||
return true;
|
||||
|
||||
+1164
-901
File diff suppressed because it is too large
Load Diff
@@ -68,6 +68,21 @@ export enum TokenType {
|
||||
MAX = 'MAX',
|
||||
DISTINCT = 'DISTINCT',
|
||||
|
||||
// v0.3.0: 事务 / UNION / EXISTS / 动态索引
|
||||
BEGIN = 'BEGIN',
|
||||
COMMIT = 'COMMIT',
|
||||
ROLLBACK = 'ROLLBACK',
|
||||
UNION = 'UNION',
|
||||
ALL = 'ALL',
|
||||
INDEX = 'INDEX',
|
||||
|
||||
// v0.3.1: CASE WHEN 表达式
|
||||
CASE = 'CASE',
|
||||
WHEN = 'WHEN',
|
||||
THEN = 'THEN',
|
||||
ELSE = 'ELSE',
|
||||
END = 'END',
|
||||
|
||||
// 标识符 & 字面量
|
||||
IDENTIFIER = 'IDENTIFIER',
|
||||
STRING = 'STRING',
|
||||
@@ -165,4 +180,19 @@ export const KEYWORDS: Record<string, TokenType> = {
|
||||
'MIN': TokenType.MIN,
|
||||
'MAX': TokenType.MAX,
|
||||
'DISTINCT': TokenType.DISTINCT,
|
||||
|
||||
// v0.3.0
|
||||
'BEGIN': TokenType.BEGIN,
|
||||
'COMMIT': TokenType.COMMIT,
|
||||
'ROLLBACK': TokenType.ROLLBACK,
|
||||
'UNION': TokenType.UNION,
|
||||
'ALL': TokenType.ALL,
|
||||
'INDEX': TokenType.INDEX,
|
||||
|
||||
// v0.3.1
|
||||
'CASE': TokenType.CASE,
|
||||
'WHEN': TokenType.WHEN,
|
||||
'THEN': TokenType.THEN,
|
||||
'ELSE': TokenType.ELSE,
|
||||
'END': TokenType.END,
|
||||
};
|
||||
|
||||
+14
-6
@@ -18,11 +18,14 @@ export class Table<T = Record<string, unknown>> {
|
||||
private engine: IStorageEngine;
|
||||
private schema: TableSchema | null = null;
|
||||
private executor: QueryExecutor | undefined;
|
||||
/** 写入回调（多标签页广播，v0.3.2） */
|
||||
private onWrite?: (table: string) => void;
|
||||
|
||||
constructor(engine: IStorageEngine, tableName: string, executor?: QueryExecutor) {
|
||||
constructor(engine: IStorageEngine, tableName: string, executor?: QueryExecutor, onWrite?: (table: string) => void) {
|
||||
this.engine = engine;
|
||||
this.name = tableName;
|
||||
this.executor = executor;
|
||||
this.onWrite = onWrite;
|
||||
}
|
||||
|
||||
// ---- Schema ----
|
||||
@@ -40,11 +43,14 @@ export class Table<T = Record<string, unknown>> {
|
||||
|
||||
async insert(row: T & Record<string, unknown>): Promise<string> {
|
||||
const pks = await this.engine.insert(this.name, [row as Record<string, unknown>]);
|
||||
this.onWrite?.(this.name);
|
||||
return pks[0];
|
||||
}
|
||||
|
||||
async insertMany(rows: (T & Record<string, unknown>)[]): Promise<string[]> {
|
||||
return this.engine.insert(this.name, rows as Record<string, unknown>[]);
|
||||
const pks = await this.engine.insert(this.name, rows as Record<string, unknown>[]);
|
||||
this.onWrite?.(this.name);
|
||||
return pks;
|
||||
}
|
||||
|
||||
// ---- 查询 ----
|
||||
@@ -56,13 +62,13 @@ export class Table<T = Record<string, unknown>> {
|
||||
// ---- 更新 ----
|
||||
|
||||
update(updates: Partial<T> & Record<string, unknown>): UpdateQueryBuilder {
|
||||
return new UpdateQueryBuilder(this.engine, this.name, updates);
|
||||
return new UpdateQueryBuilder(this.engine, this.name, updates, this.onWrite);
|
||||
}
|
||||
|
||||
// ---- 删除 ----
|
||||
|
||||
delete(): DeleteQueryBuilder {
|
||||
return new DeleteQueryBuilder(this.engine, this.name);
|
||||
return new DeleteQueryBuilder(this.engine, this.name, this.onWrite);
|
||||
}
|
||||
|
||||
// ---- 聚合 ----
|
||||
@@ -74,10 +80,12 @@ export class Table<T = Record<string, unknown>> {
|
||||
// ---- 管理 ----
|
||||
|
||||
async clear(): Promise<void> {
|
||||
return this.engine.clear(this.name);
|
||||
await this.engine.clear(this.name);
|
||||
this.onWrite?.(this.name);
|
||||
}
|
||||
|
||||
async drop(): Promise<void> {
|
||||
return this.engine.dropTable(this.name);
|
||||
await this.engine.dropTable(this.name);
|
||||
this.onWrite?.(this.name);
|
||||
}
|
||||
}
|
||||
|
||||
+273
-273
@@ -1,273 +1,273 @@
|
||||
/**
|
||||
* AriaEngine Buffer Pool + Eviction 单元测试
|
||||
*/
|
||||
import { BufferPool, type PageIO } from '../../src/engine/aria/buffer/pool';
|
||||
import { LRUList, EvictionManager } from '../../src/engine/aria/buffer/eviction';
|
||||
import { PageType, PAGE_SIZE } from '../../src/engine/aria/types';
|
||||
import { initPageHeader } from '../../src/engine/aria/page/header';
|
||||
|
||||
// ===================================================================
|
||||
// LRUList
|
||||
// ===================================================================
|
||||
describe('AriaEngine — LRUList', () => {
|
||||
function makePage(id: number) {
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, id, PageType.DATA);
|
||||
return { pageId: id, type: PageType.DATA, data, dirty: false, pins: 0, prev: null, next: null, lastAccess: Date.now() };
|
||||
}
|
||||
|
||||
it('moveToHead — 单元素', () => {
|
||||
const list = new LRUList();
|
||||
const p = makePage(1);
|
||||
list.moveToHead(p);
|
||||
expect(list.size).toBe(1);
|
||||
});
|
||||
|
||||
it('moveToHead — 多元素保持 MRU 顺序', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
const c = makePage(3);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
list.moveToHead(c);
|
||||
expect(list.size).toBe(3);
|
||||
// c 是最新的
|
||||
});
|
||||
|
||||
it('getLRU 返回最久未使用', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
expect(list.getLRU()!.pageId).toBe(1);
|
||||
});
|
||||
|
||||
it('popLRU 移除并返回最久未使用', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
const popped = list.popLRU();
|
||||
expect(popped!.pageId).toBe(1);
|
||||
expect(list.size).toBe(1);
|
||||
});
|
||||
|
||||
it('remove — 从中间移除', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
const c = makePage(3);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
list.moveToHead(c);
|
||||
list.remove(b);
|
||||
expect(list.size).toBe(2);
|
||||
});
|
||||
|
||||
it('clear 清空', () => {
|
||||
const list = new LRUList();
|
||||
list.moveToHead(makePage(1));
|
||||
list.moveToHead(makePage(2));
|
||||
list.clear();
|
||||
expect(list.size).toBe(0);
|
||||
});
|
||||
|
||||
it('getAllPages 返回所有页面', () => {
|
||||
const list = new LRUList();
|
||||
list.moveToHead(makePage(1));
|
||||
list.moveToHead(makePage(2));
|
||||
expect(list.getAllPages()).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('空 LRU getLRU 返回 null', () => {
|
||||
const list = new LRUList();
|
||||
expect(list.getLRU()).toBeNull();
|
||||
});
|
||||
|
||||
it('空 LRU popLRU 返回 null', () => {
|
||||
const list = new LRUList();
|
||||
expect(list.popLRU()).toBeNull();
|
||||
});
|
||||
|
||||
it('moveToHead 同元素不移重复', () => {
|
||||
const list = new LRUList();
|
||||
const p = makePage(1);
|
||||
list.moveToHead(p);
|
||||
list.moveToHead(p);
|
||||
list.moveToHead(p);
|
||||
expect(list.size).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// EvictionManager
|
||||
// ===================================================================
|
||||
describe('AriaEngine — EvictionManager', () => {
|
||||
function makePage(id: number, dirty = false) {
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, id, PageType.DATA);
|
||||
return { pageId: id, type: PageType.DATA, data, dirty, pins: 0, prev: null, next: null, lastAccess: Date.now() };
|
||||
}
|
||||
|
||||
it('access 更新 LRU', async () => {
|
||||
let evicted = -1;
|
||||
const em = new EvictionManager(3, async (p) => { evicted = p.pageId; });
|
||||
const p = makePage(1);
|
||||
em.add(p);
|
||||
em.access(p);
|
||||
expect(em.getSize()).toBe(1);
|
||||
});
|
||||
|
||||
it('add 不超过容量不触发驱逐', async () => {
|
||||
const evictedPages: number[] = [];
|
||||
const em = new EvictionManager(4, async (p) => { evictedPages.push(p.pageId); });
|
||||
em.add(makePage(1));
|
||||
em.add(makePage(2));
|
||||
em.add(makePage(3));
|
||||
await em.evictIfNeeded(1);
|
||||
expect(evictedPages).toHaveLength(0);
|
||||
expect(em.getSize()).toBe(3);
|
||||
});
|
||||
|
||||
it('evictIfNeeded 超容量触发驱逐', async () => {
|
||||
const evictedPages: number[] = [];
|
||||
const em = new EvictionManager(2, async (p) => { evictedPages.push(p.pageId); });
|
||||
em.add(makePage(1));
|
||||
em.add(makePage(2));
|
||||
em.add(makePage(3)); // 超容量
|
||||
await em.evictIfNeeded(0);
|
||||
// 驱逐后 size 应 <= 2
|
||||
expect(em.getSize()).toBeLessThanOrEqual(2);
|
||||
});
|
||||
|
||||
it('remove 减少 size', () => {
|
||||
const em = new EvictionManager(4, async () => {});
|
||||
const p = makePage(1);
|
||||
em.add(p);
|
||||
em.add(makePage(2));
|
||||
em.remove(p);
|
||||
expect(em.getSize()).toBe(1);
|
||||
});
|
||||
|
||||
it('getCapacity 返回配置容量', () => {
|
||||
const em = new EvictionManager(128, async () => {});
|
||||
expect(em.getCapacity()).toBe(128);
|
||||
});
|
||||
|
||||
it('clear 清空', () => {
|
||||
const em = new EvictionManager(4, async () => {});
|
||||
em.add(makePage(1));
|
||||
em.add(makePage(2));
|
||||
em.clear();
|
||||
expect(em.getSize()).toBe(0);
|
||||
});
|
||||
|
||||
it('脏页驱逐前调用 onEvict 回调', async () => {
|
||||
let flushed = 0;
|
||||
const em = new EvictionManager(2, async (_p) => { flushed++; });
|
||||
em.add(makePage(1, true)); // dirty page
|
||||
em.add(makePage(2));
|
||||
await em.evictIfNeeded(1); // need space → evict page 1
|
||||
expect(flushed).toBeGreaterThanOrEqual(0); // May or may not evict
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// BufferPool (with Mock PageIO)
|
||||
// ===================================================================
|
||||
describe('AriaEngine — BufferPool', () => {
|
||||
class MockPageIO implements PageIO {
|
||||
store = new Map<number, ArrayBuffer>();
|
||||
nextId = 1;
|
||||
reads = 0;
|
||||
writes = 0;
|
||||
|
||||
async readPage(pageId: number) { this.reads++; return this.store.get(pageId) ?? null; }
|
||||
async writePage(pageId: number, data: ArrayBuffer) { this.writes++; this.store.set(pageId, data); }
|
||||
async allocatePageId() { return this.nextId++; }
|
||||
async freePageId(_pageId: number) {}
|
||||
}
|
||||
|
||||
it('newPage 创建页面并 pin', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const page = await pool.newPage();
|
||||
expect(page.pageId).toBe(1);
|
||||
expect(page.pins).toBe(1);
|
||||
expect(page.type).toBe(PageType.DATA);
|
||||
pool.unpin(page);
|
||||
});
|
||||
|
||||
it('getPage — 池中已存在则 pin++', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const p1 = await pool.newPage();
|
||||
pool.unpin(p1);
|
||||
|
||||
const p2 = await pool.getPage(p1.pageId);
|
||||
expect(p2!.pageId).toBe(p1.pageId);
|
||||
expect(p2!.pins).toBe(1);
|
||||
pool.unpin(p2!);
|
||||
});
|
||||
|
||||
it('markDirty + flushPage 写回', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const page = await pool.newPage();
|
||||
new Uint8Array(page.data)[100] = 42;
|
||||
pool.markDirty(page);
|
||||
pool.unpin(page);
|
||||
|
||||
await pool.flushPage(page.pageId);
|
||||
expect(io.writes).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('flushAll 刷新所有脏页', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 8);
|
||||
const p1 = await pool.newPage();
|
||||
const p2 = await pool.newPage();
|
||||
pool.markDirty(p1);
|
||||
pool.markDirty(p2);
|
||||
pool.unpin(p1);
|
||||
pool.unpin(p2);
|
||||
|
||||
await pool.flushAll();
|
||||
expect(io.writes).toBe(2);
|
||||
});
|
||||
|
||||
it('getCachedPageCount 返回缓存数', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
await pool.newPage();
|
||||
await pool.newPage();
|
||||
expect(pool.getCachedPageCount()).toBe(2);
|
||||
});
|
||||
|
||||
it('getDirtyPageCount 返回脏页数', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const p = await pool.newPage();
|
||||
pool.markDirty(p);
|
||||
pool.unpin(p);
|
||||
expect(pool.getDirtyPageCount()).toBe(1);
|
||||
});
|
||||
|
||||
it('getCapacity 返回容量', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 64);
|
||||
expect(pool.getCapacity()).toBe(64);
|
||||
});
|
||||
|
||||
it('removePage 移除缓存', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const p = await pool.newPage();
|
||||
pool.unpin(p);
|
||||
pool.removePage(p.pageId);
|
||||
expect(pool.getCachedPageCount()).toBe(0);
|
||||
});
|
||||
});
|
||||
/**
|
||||
* AriaEngine Buffer Pool + Eviction 单元测试
|
||||
*/
|
||||
import { BufferPool, type PageIO } from '../../src/engine/aria/buffer/pool';
|
||||
import { LRUList, EvictionManager } from '../../src/engine/aria/buffer/eviction';
|
||||
import { PageType, PAGE_SIZE } from '../../src/engine/aria/types';
|
||||
import { initPageHeader } from '../../src/engine/aria/page/header';
|
||||
|
||||
// ===================================================================
|
||||
// LRUList
|
||||
// ===================================================================
|
||||
describe('AriaEngine — LRUList', () => {
|
||||
function makePage(id: number) {
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, id, PageType.DATA);
|
||||
return { pageId: id, type: PageType.DATA, data, dirty: false, pins: 0, prev: null, next: null, lastAccess: Date.now() };
|
||||
}
|
||||
|
||||
it('moveToHead — 单元素', () => {
|
||||
const list = new LRUList();
|
||||
const p = makePage(1);
|
||||
list.moveToHead(p);
|
||||
expect(list.size).toBe(1);
|
||||
});
|
||||
|
||||
it('moveToHead — 多元素保持 MRU 顺序', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
const c = makePage(3);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
list.moveToHead(c);
|
||||
expect(list.size).toBe(3);
|
||||
// c 是最新的
|
||||
});
|
||||
|
||||
it('getLRU 返回最久未使用', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
expect(list.getLRU()!.pageId).toBe(1);
|
||||
});
|
||||
|
||||
it('popLRU 移除并返回最久未使用', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
const popped = list.popLRU();
|
||||
expect(popped!.pageId).toBe(1);
|
||||
expect(list.size).toBe(1);
|
||||
});
|
||||
|
||||
it('remove — 从中间移除', () => {
|
||||
const list = new LRUList();
|
||||
const a = makePage(1);
|
||||
const b = makePage(2);
|
||||
const c = makePage(3);
|
||||
list.moveToHead(a);
|
||||
list.moveToHead(b);
|
||||
list.moveToHead(c);
|
||||
list.remove(b);
|
||||
expect(list.size).toBe(2);
|
||||
});
|
||||
|
||||
it('clear 清空', () => {
|
||||
const list = new LRUList();
|
||||
list.moveToHead(makePage(1));
|
||||
list.moveToHead(makePage(2));
|
||||
list.clear();
|
||||
expect(list.size).toBe(0);
|
||||
});
|
||||
|
||||
it('getAllPages 返回所有页面', () => {
|
||||
const list = new LRUList();
|
||||
list.moveToHead(makePage(1));
|
||||
list.moveToHead(makePage(2));
|
||||
expect(list.getAllPages()).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('空 LRU getLRU 返回 null', () => {
|
||||
const list = new LRUList();
|
||||
expect(list.getLRU()).toBeNull();
|
||||
});
|
||||
|
||||
it('空 LRU popLRU 返回 null', () => {
|
||||
const list = new LRUList();
|
||||
expect(list.popLRU()).toBeNull();
|
||||
});
|
||||
|
||||
it('moveToHead 同元素不移重复', () => {
|
||||
const list = new LRUList();
|
||||
const p = makePage(1);
|
||||
list.moveToHead(p);
|
||||
list.moveToHead(p);
|
||||
list.moveToHead(p);
|
||||
expect(list.size).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// EvictionManager
|
||||
// ===================================================================
|
||||
describe('AriaEngine — EvictionManager', () => {
|
||||
function makePage(id: number, dirty = false) {
|
||||
const data = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(data, id, PageType.DATA);
|
||||
return { pageId: id, type: PageType.DATA, data, dirty, pins: 0, prev: null, next: null, lastAccess: Date.now() };
|
||||
}
|
||||
|
||||
it('access 更新 LRU', async () => {
|
||||
let evicted = -1;
|
||||
const em = new EvictionManager(3, async (p) => { evicted = p.pageId; });
|
||||
const p = makePage(1);
|
||||
em.add(p);
|
||||
em.access(p);
|
||||
expect(em.getSize()).toBe(1);
|
||||
});
|
||||
|
||||
it('add 不超过容量不触发驱逐', async () => {
|
||||
const evictedPages: number[] = [];
|
||||
const em = new EvictionManager(4, async (p) => { evictedPages.push(p.pageId); });
|
||||
em.add(makePage(1));
|
||||
em.add(makePage(2));
|
||||
em.add(makePage(3));
|
||||
await em.evictIfNeeded(1);
|
||||
expect(evictedPages).toHaveLength(0);
|
||||
expect(em.getSize()).toBe(3);
|
||||
});
|
||||
|
||||
it('evictIfNeeded 超容量触发驱逐', async () => {
|
||||
const evictedPages: number[] = [];
|
||||
const em = new EvictionManager(2, async (p) => { evictedPages.push(p.pageId); });
|
||||
em.add(makePage(1));
|
||||
em.add(makePage(2));
|
||||
em.add(makePage(3)); // 超容量
|
||||
await em.evictIfNeeded(0);
|
||||
// 驱逐后 size 应 <= 2
|
||||
expect(em.getSize()).toBeLessThanOrEqual(2);
|
||||
});
|
||||
|
||||
it('remove 减少 size', () => {
|
||||
const em = new EvictionManager(4, async () => {});
|
||||
const p = makePage(1);
|
||||
em.add(p);
|
||||
em.add(makePage(2));
|
||||
em.remove(p);
|
||||
expect(em.getSize()).toBe(1);
|
||||
});
|
||||
|
||||
it('getCapacity 返回配置容量', () => {
|
||||
const em = new EvictionManager(128, async () => {});
|
||||
expect(em.getCapacity()).toBe(128);
|
||||
});
|
||||
|
||||
it('clear 清空', () => {
|
||||
const em = new EvictionManager(4, async () => {});
|
||||
em.add(makePage(1));
|
||||
em.add(makePage(2));
|
||||
em.clear();
|
||||
expect(em.getSize()).toBe(0);
|
||||
});
|
||||
|
||||
it('脏页驱逐前调用 onEvict 回调', async () => {
|
||||
let flushed = 0;
|
||||
const em = new EvictionManager(2, async (_p) => { flushed++; });
|
||||
em.add(makePage(1, true)); // dirty page
|
||||
em.add(makePage(2));
|
||||
await em.evictIfNeeded(1); // need space → evict page 1
|
||||
expect(flushed).toBeGreaterThanOrEqual(0); // May or may not evict
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// BufferPool (with Mock PageIO)
|
||||
// ===================================================================
|
||||
describe('AriaEngine — BufferPool', () => {
|
||||
class MockPageIO implements PageIO {
|
||||
store = new Map<number, ArrayBuffer>();
|
||||
nextId = 1;
|
||||
reads = 0;
|
||||
writes = 0;
|
||||
|
||||
async readPage(pageId: number) { this.reads++; return this.store.get(pageId) ?? null; }
|
||||
async writePage(pageId: number, data: ArrayBuffer) { this.writes++; this.store.set(pageId, data); }
|
||||
async allocatePageId() { return this.nextId++; }
|
||||
async freePageId(_pageId: number) {}
|
||||
}
|
||||
|
||||
it('newPage 创建页面并 pin', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const page = await pool.newPage();
|
||||
expect(page.pageId).toBe(1);
|
||||
expect(page.pins).toBe(1);
|
||||
expect(page.type).toBe(PageType.DATA);
|
||||
pool.unpin(page);
|
||||
});
|
||||
|
||||
it('getPage — 池中已存在则 pin++', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const p1 = await pool.newPage();
|
||||
pool.unpin(p1);
|
||||
|
||||
const p2 = await pool.getPage(p1.pageId);
|
||||
expect(p2!.pageId).toBe(p1.pageId);
|
||||
expect(p2!.pins).toBe(1);
|
||||
pool.unpin(p2!);
|
||||
});
|
||||
|
||||
it('markDirty + flushPage 写回', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const page = await pool.newPage();
|
||||
new Uint8Array(page.data)[100] = 42;
|
||||
pool.markDirty(page);
|
||||
pool.unpin(page);
|
||||
|
||||
await pool.flushPage(page.pageId);
|
||||
expect(io.writes).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('flushAll 刷新所有脏页', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 8);
|
||||
const p1 = await pool.newPage();
|
||||
const p2 = await pool.newPage();
|
||||
pool.markDirty(p1);
|
||||
pool.markDirty(p2);
|
||||
pool.unpin(p1);
|
||||
pool.unpin(p2);
|
||||
|
||||
await pool.flushAll();
|
||||
expect(io.writes).toBe(2);
|
||||
});
|
||||
|
||||
it('getCachedPageCount 返回缓存数', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
await pool.newPage();
|
||||
await pool.newPage();
|
||||
expect(pool.getCachedPageCount()).toBe(2);
|
||||
});
|
||||
|
||||
it('getDirtyPageCount 返回脏页数', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const p = await pool.newPage();
|
||||
pool.markDirty(p);
|
||||
pool.unpin(p);
|
||||
expect(pool.getDirtyPageCount()).toBe(1);
|
||||
});
|
||||
|
||||
it('getCapacity 返回容量', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 64);
|
||||
expect(pool.getCapacity()).toBe(64);
|
||||
});
|
||||
|
||||
it('removePage 移除缓存', async () => {
|
||||
const io = new MockPageIO();
|
||||
const pool = new BufferPool(io, 4);
|
||||
const p = await pool.newPage();
|
||||
pool.unpin(p);
|
||||
pool.removePage(p.pageId);
|
||||
expect(pool.getCachedPageCount()).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
/**
|
||||
* AriaEngine SSTable 缓存内存上限测试
|
||||
* @module tests/engine/aria-cache
|
||||
*
|
||||
* 验证 v0.2.6 修复：
|
||||
* 1. SSTable 缓存受 cacheLimitBytes 上限约束（LRU 裁剪）
|
||||
* 2. 缓存驱逐后所有读取路径（全表/范围/PK/索引）仍返回完整数据（prefetch 兜底）
|
||||
* 3. 写入路径不会导致缓存无限增长
|
||||
*/
|
||||
import { AriaEngine } from '../../src/engine/aria/index';
|
||||
import { createSchema } from '../../src/table/schema';
|
||||
|
||||
/** 构造小缓存 + 小 MemTable 阈值的引擎，快速产生多个 SSTable */
|
||||
function createSmallCacheEngine(bufferPoolPages = 2) {
|
||||
return new AriaEngine({
|
||||
storageBackend: 'memory',
|
||||
memtableSizeThreshold: 2048, // ~2KB 阈值 → 300 行会产生多个 SSTable
|
||||
bufferPoolPages,
|
||||
checkpointInterval: 100000, // 关闭自动 checkpoint，避免干扰
|
||||
walSyncMode: 'none',
|
||||
} as any);
|
||||
}
|
||||
|
||||
function makeRows(count: number): Record<string, unknown>[] {
|
||||
const rows: Record<string, unknown>[] = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
rows.push({ id: `u${i}`, name: `User${i}`, age: 20 + (i % 30) });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
describe('AriaEngine SSTable 缓存内存上限', () => {
|
||||
test('缓存大小受 cacheLimitBytes 约束', async () => {
|
||||
const engine = createSmallCacheEngine(2); // 2 * 4096 = 8KB 上限
|
||||
await engine.open('cache-limit-test', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number', index: true },
|
||||
}));
|
||||
|
||||
await engine.insert('users', makeRows(300));
|
||||
const lsm = (engine as any).lsm as {
|
||||
getCacheSize(): number;
|
||||
getCacheLimit(): number;
|
||||
getStats(): { sstableCount: number };
|
||||
};
|
||||
const stats = lsm.getStats();
|
||||
// 300 行 / 2KB 阈值 → 应产生多个 SSTable
|
||||
expect(stats.sstableCount).toBeGreaterThan(1);
|
||||
|
||||
// 多轮查询后缓存仍受上限约束
|
||||
for (let round = 0; round < 5; round++) {
|
||||
const rows = await engine.find('users', { table: 'users', where: { age: 25 } });
|
||||
expect(rows.length).toBe(10);
|
||||
expect(lsm.getCacheSize()).toBeLessThanOrEqual(lsm.getCacheLimit());
|
||||
}
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('缓存驱逐后全表扫描仍返回完整数据（prefetch 兜底）', async () => {
|
||||
const engine = createSmallCacheEngine(1); // 4KB 上限，必然触发驱逐
|
||||
await engine.open('cache-evict-fullscan', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
}));
|
||||
|
||||
const rows = makeRows(300);
|
||||
await engine.insert('users', rows);
|
||||
|
||||
const all = await engine.find('users', { table: 'users' });
|
||||
expect(all.length).toBe(300);
|
||||
|
||||
const lsm = (engine as any).lsm;
|
||||
expect(lsm.getCacheSize()).toBeLessThanOrEqual(lsm.getCacheLimit());
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('缓存驱逐后 PK 等值查询仍正确（prefetchKeys 兜底）', async () => {
|
||||
const engine = createSmallCacheEngine(1);
|
||||
await engine.open('cache-evict-pk', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
}));
|
||||
|
||||
const rows = makeRows(300);
|
||||
await engine.insert('users', rows);
|
||||
|
||||
// 分散查询多个 PK，每次都会经历 驱逐+重新加载
|
||||
for (let i = 0; i < 300; i += 11) {
|
||||
const found = await engine.find('users', { table: 'users', where: { id: `u${i}` } });
|
||||
expect(found.length).toBe(1);
|
||||
expect(found[0].name).toBe(`User${i}`);
|
||||
}
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('缓存驱逐后二级索引查询仍正确', async () => {
|
||||
const engine = createSmallCacheEngine(1);
|
||||
await engine.open('cache-evict-idx', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number', index: true },
|
||||
}));
|
||||
|
||||
await engine.insert('users', makeRows(300));
|
||||
|
||||
// 索引等值 + 范围查询
|
||||
const eq = await engine.find('users', { table: 'users', where: { age: 25 } });
|
||||
expect(eq.length).toBe(10);
|
||||
|
||||
// age 范围 20-49，每个值 10 行
|
||||
const range = await engine.find('users', { table: 'users', where: { age: { $gte: 40 } } });
|
||||
expect(range.length).toBe(100);
|
||||
|
||||
const range2 = await engine.find('users', { table: 'users', where: { age: { $gt: 45 } } });
|
||||
expect(range2.length).toBe(40);
|
||||
|
||||
const inQuery = await engine.find('users', { table: 'users', where: { age: { $in: [21, 22] } } });
|
||||
expect(inQuery.length).toBe(20);
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('UPDATE/DELETE 在缓存驱逐后仍作用于全部行', async () => {
|
||||
const engine = createSmallCacheEngine(1);
|
||||
await engine.open('cache-evict-mutate', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number' },
|
||||
}));
|
||||
|
||||
await engine.insert('users', makeRows(300));
|
||||
|
||||
// 无条件更新 → 全表更新
|
||||
const updated = await engine.update('users', { table: 'users' }, { name: 'Renamed' });
|
||||
expect(updated).toBe(300);
|
||||
|
||||
// age 20-49 每个值 10 行；$lt 25 → age 20-24 → 50 行
|
||||
const deleted = await engine.delete('users', { table: 'users', where: { age: { $lt: 25 } } });
|
||||
expect(deleted).toBe(50);
|
||||
|
||||
const remaining = await engine.find('users', { table: 'users' });
|
||||
expect(remaining.length).toBe(250);
|
||||
expect(remaining.every((r) => r.name === 'Renamed')).toBe(true);
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('写入路径不突破缓存上限（flush 后立即裁剪）', async () => {
|
||||
const engine = createSmallCacheEngine(2);
|
||||
await engine.open('cache-write-bound', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
}));
|
||||
|
||||
// 分批写入，每批都触发多次 flush
|
||||
for (let batch = 0; batch < 10; batch++) {
|
||||
await engine.insert('users', makeRows(30).map((r, i) => ({ ...r, id: `b${batch}_u${i}` })));
|
||||
const lsm = (engine as any).lsm;
|
||||
expect(lsm.getCacheSize()).toBeLessThanOrEqual(lsm.getCacheLimit());
|
||||
}
|
||||
|
||||
const all = await engine.find('users', { table: 'users' });
|
||||
expect(all.length).toBe(300);
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('回归：主 LSM 与二级索引 LSM 的 SSTable 不互相覆盖（命名空间隔离）', async () => {
|
||||
const engine = createSmallCacheEngine(4);
|
||||
await engine.open('regression-ns', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string', index: true },
|
||||
age: { type: 'number', index: true },
|
||||
}));
|
||||
|
||||
// 小阈值下 insert/update 会同时触发主 LSM 与两个索引 LSM 的多次 flush
|
||||
await engine.insert('users', makeRows(120));
|
||||
await engine.update('users', { table: 'users', where: { age: { $gte: 30 } } }, { name: 'Senior' });
|
||||
|
||||
// 主数据完整且为最新值（age 20-49 每个值出现 4 次；$gte 30 → 20 个值 × 4 = 80 行）
|
||||
const all = await engine.find('users', { table: 'users' });
|
||||
expect(all.length).toBe(120);
|
||||
expect(all.filter((r) => r.name === 'Senior').length).toBe(80);
|
||||
|
||||
// 二级索引等值查找仍正确（索引 LSM 数据未被覆盖）
|
||||
const byName = await engine.find('users', { table: 'users', where: { name: 'Senior' } });
|
||||
expect(byName.length).toBe(80);
|
||||
const byAge = await engine.find('users', { table: 'users', where: { age: 25 } });
|
||||
expect(byAge.length).toBe(4);
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('回归：同 key 跨多次 flush 更新后读到最新值（多版本语义）', async () => {
|
||||
const engine = createSmallCacheEngine(4);
|
||||
await engine.open('regression-versions', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
value: { type: 'number' },
|
||||
}));
|
||||
|
||||
await engine.insert('users', [{ id: 'a', value: 1 }]);
|
||||
|
||||
// 连续更新同一行 20 次，每次更新都经历 flush
|
||||
for (let v = 2; v <= 20; v++) {
|
||||
await engine.update('users', { table: 'users', where: { id: 'a' } }, { value: v });
|
||||
}
|
||||
|
||||
const rows = await engine.find('users', { table: 'users', where: { id: 'a' } });
|
||||
expect(rows.length).toBe(1);
|
||||
expect(rows[0].value).toBe(20);
|
||||
|
||||
// 全表扫描也应返回最新值
|
||||
const all = await engine.find('users', { table: 'users' });
|
||||
expect(all.length).toBe(1);
|
||||
expect(all[0].value).toBe(20);
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('回归：删除后 tombstone 跨 flush 仍生效（不残留旧数据）', async () => {
|
||||
const engine = createSmallCacheEngine(4);
|
||||
await engine.open('regression-tombstone', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
age: { type: 'number', index: true },
|
||||
}));
|
||||
|
||||
await engine.insert('users', makeRows(120));
|
||||
|
||||
// 分批删除，触发多次 flush
|
||||
for (let batch = 0; batch < 4; batch++) {
|
||||
const deleted = await engine.delete('users', { table: 'users', where: { age: { $gte: 20 + batch * 5, $lt: 25 + batch * 5 } } });
|
||||
expect(deleted).toBe(20);
|
||||
}
|
||||
|
||||
const remaining = await engine.find('users', { table: 'users' });
|
||||
expect(remaining.length).toBe(40);
|
||||
|
||||
// 索引查找也不应返回已删除行
|
||||
const ghost = await engine.find('users', { table: 'users', where: { age: 22 } });
|
||||
expect(ghost.length).toBe(0);
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
+218
-127
@@ -1,127 +1,218 @@
|
||||
/**
|
||||
* AriaEngine LZ4 压缩 + LSM Merge Iterator 单元测试
|
||||
* 注：LZ4 为简化演示实现（默认 compression:false），测试聚焦于「不卡死」
|
||||
*/
|
||||
import { compressLZ4, decompressLZ4 } from '../../src/engine/aria/compression/lz4';
|
||||
import { MergeIterator, ArrayEntrySource } from '../../src/engine/aria/index/merge_iterator';
|
||||
|
||||
// ===================================================================
|
||||
// LZ4 压缩 — 安全烟雾测试（不卡死）
|
||||
// ===================================================================
|
||||
describe('AriaEngine — LZ4 Compression', () => {
|
||||
it('短于 4 字节时原样返回', () => {
|
||||
const input = new Uint8Array([1, 2]);
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBe(input);
|
||||
});
|
||||
|
||||
it('简单文本压缩不抛出异常且产生输出', () => {
|
||||
const input = new TextEncoder().encode('hello world hello world hello world');
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBeInstanceOf(Uint8Array);
|
||||
expect(compressed.byteLength).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('重复数据有压缩效果', () => {
|
||||
const pattern = 'ABCD';
|
||||
const repeated = pattern.repeat(100);
|
||||
const input = new TextEncoder().encode(repeated);
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed.byteLength).toBeLessThan(input.byteLength);
|
||||
});
|
||||
|
||||
it('随机不可压缩数据不卡死', () => {
|
||||
const input = new Uint8Array(256);
|
||||
for (let i = 0; i < 256; i++) input[i] = Math.floor(Math.random() * 256);
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBeInstanceOf(Uint8Array);
|
||||
expect(compressed.byteLength).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('长文本压缩不卡死', () => {
|
||||
const input = new TextEncoder().encode('The quick brown fox jumps over the lazy dog. '.repeat(10));
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBeTruthy();
|
||||
expect(compressed.byteLength).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('多种长度输入均不卡死', () => {
|
||||
for (const size of [10, 50, 100, 200, 500]) {
|
||||
const input = new Uint8Array(size);
|
||||
for (let i = 0; i < size; i++) input[i] = i % 256;
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed.byteLength).toBeLessThanOrEqual(input.byteLength + 16);
|
||||
}
|
||||
});
|
||||
|
||||
it('解压不抛出异常', () => {
|
||||
const input = new TextEncoder().encode('test data for decompression smoke test');
|
||||
const compressed = compressLZ4(input);
|
||||
expect(() => decompressLZ4(compressed, input.byteLength)).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// MergeIterator — 归并迭代器单元测试
|
||||
// ===================================================================
|
||||
describe('AriaEngine — MergeIterator', () => {
|
||||
it('单数据源归并', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([
|
||||
['a', { v: 1 }],
|
||||
['b', { v: 2 }],
|
||||
['c', { v: 3 }],
|
||||
]));
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(3);
|
||||
expect(result.map(([k]) => k)).toEqual(['a', 'b', 'c']);
|
||||
});
|
||||
|
||||
it('多数据源归并去重（保留最新）', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'new' }], ['c', { v: 3 }]]));
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'old' }], ['b', { v: 2 }]]));
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(3);
|
||||
expect(result[0][0]).toBe('a');
|
||||
expect(result[0][1].v).toBe('new');
|
||||
expect(result[1][0]).toBe('b');
|
||||
expect(result[2][0]).toBe('c');
|
||||
});
|
||||
|
||||
it('空数据源归并', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([]));
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('大数量归并', () => {
|
||||
const mi = new MergeIterator();
|
||||
for (let s = 0; s < 5; s++) {
|
||||
const entries: [string, Record<string, unknown>][] = [];
|
||||
for (let i = 0; i < 100; i++) {
|
||||
entries.push([`src${s}-key-${String(i).padStart(3, '0')}`, { src: s, idx: i }]);
|
||||
}
|
||||
mi.addSource(new ArrayEntrySource(entries));
|
||||
}
|
||||
const result = mi.drain();
|
||||
// 5 sources × 100 unique keys = 500 total (keys are unique per source)
|
||||
expect(result).toHaveLength(500);
|
||||
});
|
||||
|
||||
it('ArrayEntrySource — 迭代器用完返回 null', () => {
|
||||
const src = new ArrayEntrySource([['k', { v: 1 }]]);
|
||||
expect(src.next()).not.toBeNull();
|
||||
expect(src.next()).toBeNull();
|
||||
expect(src.next()).toBeNull();
|
||||
});
|
||||
|
||||
it('ArrayEntrySource — reset 重置', () => {
|
||||
const src = new ArrayEntrySource([['k1', { v: 1 }], ['k2', { v: 2 }]]);
|
||||
src.next();
|
||||
src.reset();
|
||||
const val = src.next();
|
||||
expect(val![0]).toBe('k1');
|
||||
});
|
||||
});
|
||||
/**
|
||||
* AriaEngine LZ4 压缩 + LSM Merge Iterator 单元测试
|
||||
* 注：LZ4 为简化演示实现（默认 compression:false），测试聚焦于「不卡死」
|
||||
*/
|
||||
import { compressLZ4, decompressLZ4 } from '../../src/engine/aria/compression/lz4';
|
||||
import { MergeIterator, ArrayEntrySource } from '../../src/engine/aria/index/merge_iterator';
|
||||
|
||||
// ===================================================================
|
||||
// LZ4 压缩 — 安全烟雾测试（不卡死）
|
||||
// ===================================================================
|
||||
describe('AriaEngine — LZ4 Compression', () => {
|
||||
it('短于 4 字节时压缩为纯字面量 token 且往返一致', () => {
|
||||
const input = new Uint8Array([1, 2]);
|
||||
const compressed = compressLZ4(input);
|
||||
// token(lo=0) + 2 字节字面量
|
||||
expect(compressed.byteLength).toBe(3);
|
||||
expect(compressed[0]).toBe(0x20); // litLen=2, matchField=0
|
||||
const restored = decompressLZ4(compressed, 2);
|
||||
expect(Array.from(restored)).toEqual([1, 2]);
|
||||
});
|
||||
|
||||
it('简单文本压缩不抛出异常且产生输出', () => {
|
||||
const input = new TextEncoder().encode('hello world hello world hello world');
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBeInstanceOf(Uint8Array);
|
||||
expect(compressed.byteLength).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('重复数据有压缩效果', () => {
|
||||
const pattern = 'ABCD';
|
||||
const repeated = pattern.repeat(100);
|
||||
const input = new TextEncoder().encode(repeated);
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed.byteLength).toBeLessThan(input.byteLength);
|
||||
});
|
||||
|
||||
it('随机不可压缩数据不卡死', () => {
|
||||
const input = new Uint8Array(256);
|
||||
for (let i = 0; i < 256; i++) input[i] = Math.floor(Math.random() * 256);
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBeInstanceOf(Uint8Array);
|
||||
expect(compressed.byteLength).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('长文本压缩不卡死', () => {
|
||||
const input = new TextEncoder().encode('The quick brown fox jumps over the lazy dog. '.repeat(10));
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed).toBeTruthy();
|
||||
expect(compressed.byteLength).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('多种长度输入均不卡死', () => {
|
||||
for (const size of [10, 50, 100, 200, 500]) {
|
||||
const input = new Uint8Array(size);
|
||||
for (let i = 0; i < size; i++) input[i] = i % 256;
|
||||
const compressed = compressLZ4(input);
|
||||
expect(compressed.byteLength).toBeLessThanOrEqual(input.byteLength + 16);
|
||||
}
|
||||
});
|
||||
|
||||
it('解压不抛出异常', () => {
|
||||
const input = new TextEncoder().encode('test data for decompression smoke test');
|
||||
const compressed = compressLZ4(input);
|
||||
expect(() => decompressLZ4(compressed, input.byteLength)).not.toThrow();
|
||||
});
|
||||
|
||||
// ---- v0.2.6 补强：压缩 → 解压 往返一致性 ----
|
||||
|
||||
it('往返一致：重复模式数据', () => {
|
||||
const input = new TextEncoder().encode('ABCD'.repeat(100));
|
||||
const compressed = compressLZ4(input);
|
||||
const restored = decompressLZ4(compressed, input.byteLength);
|
||||
expect(Array.from(restored)).toEqual(Array.from(input));
|
||||
});
|
||||
|
||||
it('往返一致：自然文本数据', () => {
|
||||
const input = new TextEncoder().encode(
|
||||
'The quick brown fox jumps over the lazy dog. '.repeat(10),
|
||||
);
|
||||
const compressed = compressLZ4(input);
|
||||
const restored = decompressLZ4(compressed, input.byteLength);
|
||||
expect(Array.from(restored)).toEqual(Array.from(input));
|
||||
});
|
||||
|
||||
it('往返一致：随机不可压缩数据', () => {
|
||||
const input = new Uint8Array(512);
|
||||
for (let i = 0; i < 512; i++) input[i] = Math.floor(Math.random() * 256);
|
||||
const compressed = compressLZ4(input);
|
||||
const restored = decompressLZ4(compressed, input.byteLength);
|
||||
expect(Array.from(restored)).toEqual(Array.from(input));
|
||||
});
|
||||
|
||||
it('往返一致：多种长度与字节模式', () => {
|
||||
for (const size of [4, 5, 15, 16, 17, 50, 100, 300, 1000]) {
|
||||
const input = new Uint8Array(size);
|
||||
for (let i = 0; i < size; i++) input[i] = i % 7 === 0 ? i % 256 : 0x41;
|
||||
const compressed = compressLZ4(input);
|
||||
const restored = decompressLZ4(compressed, input.byteLength);
|
||||
expect(Array.from(restored)).toEqual(Array.from(input));
|
||||
}
|
||||
});
|
||||
|
||||
it('往返一致：恰好 15 字节字面量边界', () => {
|
||||
// 字面量长度恰好 15（token 上限）时不应丢字节
|
||||
const input = new Uint8Array(15);
|
||||
for (let i = 0; i < 15; i++) input[i] = i;
|
||||
const compressed = compressLZ4(input);
|
||||
const restored = decompressLZ4(compressed, input.byteLength);
|
||||
expect(Array.from(restored)).toEqual(Array.from(input));
|
||||
});
|
||||
|
||||
it('往返一致：超过 15 字节的连续匹配', () => {
|
||||
const input = new TextEncoder().encode('X'.repeat(200) + 'Y' + 'X'.repeat(60));
|
||||
const compressed = compressLZ4(input);
|
||||
const restored = decompressLZ4(compressed, input.byteLength);
|
||||
expect(Array.from(restored)).toEqual(Array.from(input));
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// MergeIterator — 归并迭代器单元测试
|
||||
// ===================================================================
|
||||
describe('AriaEngine — MergeIterator', () => {
|
||||
it('单数据源归并', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([
|
||||
['a', { v: 1 }],
|
||||
['b', { v: 2 }],
|
||||
['c', { v: 3 }],
|
||||
]));
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(3);
|
||||
expect(result.map(([k]) => k)).toEqual(['a', 'b', 'c']);
|
||||
});
|
||||
|
||||
it('多数据源归并去重（保留最新）', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'new' }], ['c', { v: 3 }]]));
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'old' }], ['b', { v: 2 }]]));
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(3);
|
||||
expect(result[0][0]).toBe('a');
|
||||
expect(result[0][1].v).toBe('new');
|
||||
expect(result[1][0]).toBe('b');
|
||||
expect(result[2][0]).toBe('c');
|
||||
});
|
||||
|
||||
// ---- v0.2.6 回归：同 key 多来源时保留 sourceIndex 最小（最新）的条目 ----
|
||||
it('回归：同 key 出现在多个来源时返回 sourceIndex 最小（最新来源）的值', () => {
|
||||
const mi = new MergeIterator();
|
||||
// 语义：sourceIndex 越小越新（memtable=0 < immutable=1 < sstable=2+）
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'source0' }]])); // 最新来源
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'source1' }]]));
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'source2' }]])); // 最旧来源
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(1);
|
||||
// 取的是 sourceIndex 最小（最新来源）的条目，而非堆序决定的任意条目
|
||||
expect(result[0][1].v).toBe('source0');
|
||||
});
|
||||
|
||||
it('回归：最新来源的值位于中间 sourceIndex 时仍取 sourceIndex 最小者', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'middle' }]])); // index 0 = 最新来源
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'newest' }]])); // index 1
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 'oldest' }]])); // index 2
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0][1].v).toBe('middle'); // index 0 的条目胜出
|
||||
});
|
||||
|
||||
it('回归：多个同 key 来源 + 其他独立 key 混合', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 1 }], ['b', { v: 10 }]]));
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 2 }]]));
|
||||
mi.addSource(new ArrayEntrySource([['a', { v: 3 }], ['c', { v: 30 }]]));
|
||||
const result = mi.drain();
|
||||
expect(result.map(([k]) => k)).toEqual(['a', 'b', 'c']);
|
||||
expect(result[0][1].v).toBe(1); // sourceIndex 0 = 最新
|
||||
expect(result[1][1].v).toBe(10);
|
||||
expect(result[2][1].v).toBe(30);
|
||||
});
|
||||
|
||||
it('空数据源归并', () => {
|
||||
const mi = new MergeIterator();
|
||||
mi.addSource(new ArrayEntrySource([]));
|
||||
const result = mi.drain();
|
||||
expect(result).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('大数量归并', () => {
|
||||
const mi = new MergeIterator();
|
||||
for (let s = 0; s < 5; s++) {
|
||||
const entries: [string, Record<string, unknown>][] = [];
|
||||
for (let i = 0; i < 100; i++) {
|
||||
entries.push([`src${s}-key-${String(i).padStart(3, '0')}`, { src: s, idx: i }]);
|
||||
}
|
||||
mi.addSource(new ArrayEntrySource(entries));
|
||||
}
|
||||
const result = mi.drain();
|
||||
// 5 sources × 100 unique keys = 500 total (keys are unique per source)
|
||||
expect(result).toHaveLength(500);
|
||||
});
|
||||
|
||||
it('ArrayEntrySource — 迭代器用完返回 null', () => {
|
||||
const src = new ArrayEntrySource([['k', { v: 1 }]]);
|
||||
expect(src.next()).not.toBeNull();
|
||||
expect(src.next()).toBeNull();
|
||||
expect(src.next()).toBeNull();
|
||||
});
|
||||
|
||||
it('ArrayEntrySource — reset 重置', () => {
|
||||
const src = new ArrayEntrySource([['k1', { v: 1 }], ['k2', { v: 2 }]]);
|
||||
src.next();
|
||||
src.reset();
|
||||
const val = src.next();
|
||||
expect(val![0]).toBe('k1');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
/**
|
||||
* AriaEngine CryptoManager 加解密测试
|
||||
* @module tests/engine/aria-crypto
|
||||
*
|
||||
* v0.2.6 补强：此前仅验证实例化，现在验证真实的加解密往返一致性。
|
||||
*/
|
||||
import {
|
||||
CryptoManager,
|
||||
initCrypto,
|
||||
encryptPage,
|
||||
decryptPage,
|
||||
closeCrypto,
|
||||
} from '../../src/engine/aria/crypto';
|
||||
|
||||
function toBytes(data: ArrayBuffer): number[] {
|
||||
return Array.from(new Uint8Array(data));
|
||||
}
|
||||
|
||||
describe('AriaEngine — CryptoManager', () => {
|
||||
test('加解密往返一致', async () => {
|
||||
const cm = new CryptoManager();
|
||||
await cm.init('test-password');
|
||||
expect(cm.enabled).toBe(true);
|
||||
|
||||
const original = new TextEncoder().encode('sensitive row data').buffer;
|
||||
const { iv, data } = await cm.encryptPage(original);
|
||||
// 密文应为乱码（与原文不同）
|
||||
expect(toBytes(data)).not.toEqual(toBytes(original));
|
||||
|
||||
const decrypted = await cm.decryptPage(iv, data);
|
||||
expect(toBytes(decrypted)).toEqual(toBytes(original));
|
||||
cm.close();
|
||||
expect(cm.enabled).toBe(false);
|
||||
});
|
||||
|
||||
test('错误密码解密失败（密钥不同）', async () => {
|
||||
const cm1 = new CryptoManager();
|
||||
await cm1.init('correct-password');
|
||||
const original = new TextEncoder().encode('top secret').buffer;
|
||||
const { iv, data } = await cm1.encryptPage(original);
|
||||
|
||||
const cm2 = new CryptoManager();
|
||||
await cm2.init('wrong-password');
|
||||
await expect(cm2.decryptPage(iv, data)).rejects.toThrow();
|
||||
|
||||
cm1.close();
|
||||
cm2.close();
|
||||
});
|
||||
|
||||
test('不同 salt 派生不同密钥，解密互相失败', async () => {
|
||||
const cm1 = new CryptoManager();
|
||||
await cm1.init('pwd', new Uint8Array(16).fill(1));
|
||||
const cm2 = new CryptoManager();
|
||||
await cm2.init('pwd', new Uint8Array(16).fill(2));
|
||||
|
||||
const original = new TextEncoder().encode('salt matters').buffer;
|
||||
const { iv, data } = await cm1.encryptPage(original);
|
||||
await expect(cm2.decryptPage(iv, data)).rejects.toThrow();
|
||||
|
||||
cm1.close();
|
||||
cm2.close();
|
||||
});
|
||||
|
||||
test('未初始化时加密抛错', async () => {
|
||||
const cm = new CryptoManager();
|
||||
expect(cm.enabled).toBe(false);
|
||||
const data = new TextEncoder().encode('x').buffer;
|
||||
await expect(cm.encryptPage(data)).rejects.toThrow(/not initialized/);
|
||||
});
|
||||
|
||||
test('不同实例互不影响（独立密钥状态）', async () => {
|
||||
const cm1 = new CryptoManager();
|
||||
await cm1.init('pwd-a');
|
||||
const cm2 = new CryptoManager();
|
||||
await cm2.init('pwd-b');
|
||||
|
||||
const original = new TextEncoder().encode('instance isolation').buffer;
|
||||
const { iv, data } = await cm1.encryptPage(original);
|
||||
await expect(cm2.decryptPage(iv, data)).rejects.toThrow();
|
||||
|
||||
// 各自解密自己的数据
|
||||
const dec2orig = new TextEncoder().encode('two').buffer;
|
||||
const enc2 = await cm2.encryptPage(dec2orig);
|
||||
const dec2 = await cm2.decryptPage(enc2.iv, enc2.data);
|
||||
expect(toBytes(dec2)).toEqual(toBytes(dec2orig));
|
||||
|
||||
cm1.close();
|
||||
cm2.close();
|
||||
});
|
||||
|
||||
test('全局兼容层往返一致', async () => {
|
||||
await initCrypto('global-password');
|
||||
const original = new TextEncoder().encode('global compat layer').buffer;
|
||||
const { iv, data } = await encryptPage(original);
|
||||
const decrypted = await decryptPage(iv, data);
|
||||
expect(toBytes(decrypted)).toEqual(toBytes(original));
|
||||
closeCrypto();
|
||||
});
|
||||
|
||||
test('大块数据（接近页面大小）往返一致', async () => {
|
||||
const cm = new CryptoManager();
|
||||
await cm.init('page-size-test');
|
||||
// 4KB 页面数据
|
||||
const original = new Uint8Array(4096);
|
||||
for (let i = 0; i < 4096; i++) original[i] = i % 251;
|
||||
const { iv, data } = await cm.encryptPage(original.buffer);
|
||||
const decrypted = await cm.decryptPage(iv, data);
|
||||
expect(toBytes(decrypted)).toEqual(toBytes(original.buffer));
|
||||
cm.close();
|
||||
});
|
||||
});
|
||||
+216
-216
@@ -1,216 +1,216 @@
|
||||
/**
|
||||
* AriaEngine Bloom Filter + MemTable 单元测试
|
||||
*/
|
||||
import { BloomFilter } from '../../src/engine/aria/index/bloom';
|
||||
import { MemTable } from '../../src/engine/aria/index/memtable';
|
||||
|
||||
// ===================================================================
|
||||
// BloomFilter
|
||||
// ===================================================================
|
||||
describe('AriaEngine — BloomFilter', () => {
|
||||
it('插入后 mayContain 返回 true', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
bf.insert('hello');
|
||||
expect(bf.mayContain('hello')).toBe(true);
|
||||
});
|
||||
|
||||
it('未插入的 key mayContain 返回 false', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
bf.insert('hello');
|
||||
expect(bf.mayContain('world')).toBe(false);
|
||||
});
|
||||
|
||||
it('批量插入后所有 key 都判定存在', () => {
|
||||
const bf = new BloomFilter(500);
|
||||
const keys: string[] = [];
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const k = `key-${i}`;
|
||||
keys.push(k);
|
||||
bf.insert(k);
|
||||
}
|
||||
for (const k of keys) {
|
||||
expect(bf.mayContain(k)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('False positive 率可控', () => {
|
||||
const n = 500;
|
||||
const bf = new BloomFilter(n, 10);
|
||||
for (let i = 0; i < n; i++) {
|
||||
bf.insert(`present-${i}`);
|
||||
}
|
||||
let fp = 0;
|
||||
for (let i = 0; i < 500; i++) {
|
||||
if (bf.mayContain(`absent-${i}`)) fp++;
|
||||
}
|
||||
expect(fp).toBeLessThan(25);
|
||||
});
|
||||
|
||||
it('getBitSize 返回正确位数', () => {
|
||||
const bf = new BloomFilter(100, 10);
|
||||
expect(bf.getBitSize()).toBeGreaterThanOrEqual(64);
|
||||
});
|
||||
|
||||
it('getInsertedCount 追踪插入数', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
bf.insert('a');
|
||||
bf.insert('b');
|
||||
bf.insert('c');
|
||||
expect(bf.getInsertedCount()).toBe(3);
|
||||
});
|
||||
|
||||
it('getHashCount 返回哈希函数数量', () => {
|
||||
const bf = new BloomFilter(1000, 10);
|
||||
expect(bf.getHashCount()).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('serialize + fromData 往返', () => {
|
||||
const bf1 = new BloomFilter(100);
|
||||
bf1.insert('a');
|
||||
bf1.insert('b');
|
||||
const data = bf1.serialize();
|
||||
|
||||
const bf2 = BloomFilter.fromData(data, bf1.getHashCount());
|
||||
expect(bf2.mayContain('a')).toBe(true);
|
||||
expect(bf2.mayContain('b')).toBe(true);
|
||||
expect(bf2.mayContain('c')).toBe(false);
|
||||
});
|
||||
|
||||
it('空过滤器 mayContain 返回 false', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
expect(bf.mayContain('anything')).toBe(false);
|
||||
});
|
||||
|
||||
it('最少 1 个哈希函数', () => {
|
||||
const bf = new BloomFilter(10, 1);
|
||||
expect(bf.getHashCount()).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// MemTable
|
||||
// ===================================================================
|
||||
describe('AriaEngine — MemTable', () => {
|
||||
let mt: MemTable;
|
||||
|
||||
beforeEach(() => {
|
||||
mt = new MemTable(4 * 1024 * 1024);
|
||||
});
|
||||
|
||||
it('put + get 往返', () => {
|
||||
mt.put('key1', { name: 'Alice', age: 30 });
|
||||
const val = mt.get('key1');
|
||||
expect(val).not.toBeNull();
|
||||
expect(val!.name).toBe('Alice');
|
||||
});
|
||||
|
||||
it('get — 不存在的 key 返回 null', () => {
|
||||
expect(mt.get('nonexistent')).toBeNull();
|
||||
});
|
||||
|
||||
it('put 更新已存在的 key', () => {
|
||||
mt.put('k', { v: 1 });
|
||||
mt.put('k', { v: 2 });
|
||||
expect(mt.get('k')!.v).toBe(2);
|
||||
});
|
||||
|
||||
it('delete 删除成功', () => {
|
||||
mt.put('k', { v: 1 });
|
||||
expect(mt.delete('k')).toBe(true);
|
||||
expect(mt.get('k')).toBeNull();
|
||||
});
|
||||
|
||||
it('delete — 不存在的 key 返回 false', () => {
|
||||
expect(mt.delete('ghost')).toBe(false);
|
||||
});
|
||||
|
||||
it('getAllEntries 返回所有条目（有序）', () => {
|
||||
mt.put('c', { v: 3 });
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
const entries = mt.getAllEntries();
|
||||
expect(entries).toHaveLength(3);
|
||||
expect(entries[0][0]).toBe('a');
|
||||
expect(entries[1][0]).toBe('b');
|
||||
expect(entries[2][0]).toBe('c');
|
||||
});
|
||||
|
||||
it('rangeScan — 范围查询', () => {
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
mt.put('c', { v: 3 });
|
||||
mt.put('d', { v: 4 });
|
||||
const results = mt.rangeScan('b', 'c');
|
||||
expect(results).toHaveLength(2);
|
||||
expect(results[0][0]).toBe('b');
|
||||
expect(results[1][0]).toBe('c');
|
||||
});
|
||||
|
||||
it('rangeScan — 空结果', () => {
|
||||
mt.put('a', { v: 1 });
|
||||
const results = mt.rangeScan('z', 'zz');
|
||||
expect(results).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('getEntryCount 正确计数', () => {
|
||||
expect(mt.getEntryCount()).toBe(0);
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
expect(mt.getEntryCount()).toBe(2);
|
||||
mt.delete('a');
|
||||
expect(mt.getEntryCount()).toBe(1);
|
||||
});
|
||||
|
||||
it('contains 检查存在性', () => {
|
||||
mt.put('x', { v: 1 });
|
||||
expect(mt.contains('x')).toBe(true);
|
||||
expect(mt.contains('y')).toBe(false);
|
||||
});
|
||||
|
||||
it('shouldFlush — 未达阈值返回 false', () => {
|
||||
expect(mt.shouldFlush()).toBe(false);
|
||||
});
|
||||
|
||||
it('clear 清空所有数据', () => {
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
mt.clear();
|
||||
expect(mt.getEntryCount()).toBe(0);
|
||||
expect(mt.get('a')).toBeNull();
|
||||
});
|
||||
|
||||
it('getEstimatedSize 返回合理估计值', () => {
|
||||
expect(mt.getEstimatedSize()).toBe(0);
|
||||
mt.put('hello', { name: 'world', count: 42 });
|
||||
expect(mt.getEstimatedSize()).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('大量数据插入保持有序', () => {
|
||||
const count = 100;
|
||||
for (let i = count - 1; i >= 0; i--) {
|
||||
mt.put(`key-${String(i).padStart(3, '0')}`, { idx: i });
|
||||
}
|
||||
const entries = mt.getAllEntries();
|
||||
expect(entries).toHaveLength(count);
|
||||
for (let i = 0; i < count; i++) {
|
||||
expect(entries[i][1].idx).toBe(i);
|
||||
}
|
||||
});
|
||||
|
||||
it('删除后重新插入', () => {
|
||||
mt.put('k', { v: 1 });
|
||||
mt.delete('k');
|
||||
mt.put('k', { v: 2 });
|
||||
expect(mt.get('k')!.v).toBe(2);
|
||||
});
|
||||
|
||||
it('范围扫描包含边界', () => {
|
||||
mt.put('aa', { v: 1 });
|
||||
mt.put('ab', { v: 2 });
|
||||
mt.put('ac', { v: 3 });
|
||||
const results = mt.rangeScan('aa', 'ab');
|
||||
expect(results).toHaveLength(2);
|
||||
expect(results[0][0]).toBe('aa');
|
||||
expect(results[1][0]).toBe('ab');
|
||||
});
|
||||
});
|
||||
/**
|
||||
* AriaEngine Bloom Filter + MemTable 单元测试
|
||||
*/
|
||||
import { BloomFilter } from '../../src/engine/aria/index/bloom';
|
||||
import { MemTable } from '../../src/engine/aria/index/memtable';
|
||||
|
||||
// ===================================================================
|
||||
// BloomFilter
|
||||
// ===================================================================
|
||||
describe('AriaEngine — BloomFilter', () => {
|
||||
it('插入后 mayContain 返回 true', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
bf.insert('hello');
|
||||
expect(bf.mayContain('hello')).toBe(true);
|
||||
});
|
||||
|
||||
it('未插入的 key mayContain 返回 false', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
bf.insert('hello');
|
||||
expect(bf.mayContain('world')).toBe(false);
|
||||
});
|
||||
|
||||
it('批量插入后所有 key 都判定存在', () => {
|
||||
const bf = new BloomFilter(500);
|
||||
const keys: string[] = [];
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const k = `key-${i}`;
|
||||
keys.push(k);
|
||||
bf.insert(k);
|
||||
}
|
||||
for (const k of keys) {
|
||||
expect(bf.mayContain(k)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('False positive 率可控', () => {
|
||||
const n = 500;
|
||||
const bf = new BloomFilter(n, 10);
|
||||
for (let i = 0; i < n; i++) {
|
||||
bf.insert(`present-${i}`);
|
||||
}
|
||||
let fp = 0;
|
||||
for (let i = 0; i < 500; i++) {
|
||||
if (bf.mayContain(`absent-${i}`)) fp++;
|
||||
}
|
||||
expect(fp).toBeLessThan(25);
|
||||
});
|
||||
|
||||
it('getBitSize 返回正确位数', () => {
|
||||
const bf = new BloomFilter(100, 10);
|
||||
expect(bf.getBitSize()).toBeGreaterThanOrEqual(64);
|
||||
});
|
||||
|
||||
it('getInsertedCount 追踪插入数', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
bf.insert('a');
|
||||
bf.insert('b');
|
||||
bf.insert('c');
|
||||
expect(bf.getInsertedCount()).toBe(3);
|
||||
});
|
||||
|
||||
it('getHashCount 返回哈希函数数量', () => {
|
||||
const bf = new BloomFilter(1000, 10);
|
||||
expect(bf.getHashCount()).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('serialize + fromData 往返', () => {
|
||||
const bf1 = new BloomFilter(100);
|
||||
bf1.insert('a');
|
||||
bf1.insert('b');
|
||||
const data = bf1.serialize();
|
||||
|
||||
const bf2 = BloomFilter.fromData(data, bf1.getHashCount());
|
||||
expect(bf2.mayContain('a')).toBe(true);
|
||||
expect(bf2.mayContain('b')).toBe(true);
|
||||
expect(bf2.mayContain('c')).toBe(false);
|
||||
});
|
||||
|
||||
it('空过滤器 mayContain 返回 false', () => {
|
||||
const bf = new BloomFilter(100);
|
||||
expect(bf.mayContain('anything')).toBe(false);
|
||||
});
|
||||
|
||||
it('最少 1 个哈希函数', () => {
|
||||
const bf = new BloomFilter(10, 1);
|
||||
expect(bf.getHashCount()).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// MemTable
|
||||
// ===================================================================
|
||||
describe('AriaEngine — MemTable', () => {
|
||||
let mt: MemTable;
|
||||
|
||||
beforeEach(() => {
|
||||
mt = new MemTable(4 * 1024 * 1024);
|
||||
});
|
||||
|
||||
it('put + get 往返', () => {
|
||||
mt.put('key1', { name: 'Alice', age: 30 });
|
||||
const val = mt.get('key1');
|
||||
expect(val).not.toBeNull();
|
||||
expect(val!.name).toBe('Alice');
|
||||
});
|
||||
|
||||
it('get — 不存在的 key 返回 null', () => {
|
||||
expect(mt.get('nonexistent')).toBeNull();
|
||||
});
|
||||
|
||||
it('put 更新已存在的 key', () => {
|
||||
mt.put('k', { v: 1 });
|
||||
mt.put('k', { v: 2 });
|
||||
expect(mt.get('k')!.v).toBe(2);
|
||||
});
|
||||
|
||||
it('delete 删除成功', () => {
|
||||
mt.put('k', { v: 1 });
|
||||
expect(mt.delete('k')).toBe(true);
|
||||
expect(mt.get('k')).toBeNull();
|
||||
});
|
||||
|
||||
it('delete — 不存在的 key 返回 false', () => {
|
||||
expect(mt.delete('ghost')).toBe(false);
|
||||
});
|
||||
|
||||
it('getAllEntries 返回所有条目（有序）', () => {
|
||||
mt.put('c', { v: 3 });
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
const entries = mt.getAllEntries();
|
||||
expect(entries).toHaveLength(3);
|
||||
expect(entries[0][0]).toBe('a');
|
||||
expect(entries[1][0]).toBe('b');
|
||||
expect(entries[2][0]).toBe('c');
|
||||
});
|
||||
|
||||
it('rangeScan — 范围查询', () => {
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
mt.put('c', { v: 3 });
|
||||
mt.put('d', { v: 4 });
|
||||
const results = mt.rangeScan('b', 'c');
|
||||
expect(results).toHaveLength(2);
|
||||
expect(results[0][0]).toBe('b');
|
||||
expect(results[1][0]).toBe('c');
|
||||
});
|
||||
|
||||
it('rangeScan — 空结果', () => {
|
||||
mt.put('a', { v: 1 });
|
||||
const results = mt.rangeScan('z', 'zz');
|
||||
expect(results).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('getEntryCount 正确计数', () => {
|
||||
expect(mt.getEntryCount()).toBe(0);
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
expect(mt.getEntryCount()).toBe(2);
|
||||
mt.delete('a');
|
||||
expect(mt.getEntryCount()).toBe(1);
|
||||
});
|
||||
|
||||
it('contains 检查存在性', () => {
|
||||
mt.put('x', { v: 1 });
|
||||
expect(mt.contains('x')).toBe(true);
|
||||
expect(mt.contains('y')).toBe(false);
|
||||
});
|
||||
|
||||
it('shouldFlush — 未达阈值返回 false', () => {
|
||||
expect(mt.shouldFlush()).toBe(false);
|
||||
});
|
||||
|
||||
it('clear 清空所有数据', () => {
|
||||
mt.put('a', { v: 1 });
|
||||
mt.put('b', { v: 2 });
|
||||
mt.clear();
|
||||
expect(mt.getEntryCount()).toBe(0);
|
||||
expect(mt.get('a')).toBeNull();
|
||||
});
|
||||
|
||||
it('getEstimatedSize 返回合理估计值', () => {
|
||||
expect(mt.getEstimatedSize()).toBe(0);
|
||||
mt.put('hello', { name: 'world', count: 42 });
|
||||
expect(mt.getEstimatedSize()).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('大量数据插入保持有序', () => {
|
||||
const count = 100;
|
||||
for (let i = count - 1; i >= 0; i--) {
|
||||
mt.put(`key-${String(i).padStart(3, '0')}`, { idx: i });
|
||||
}
|
||||
const entries = mt.getAllEntries();
|
||||
expect(entries).toHaveLength(count);
|
||||
for (let i = 0; i < count; i++) {
|
||||
expect(entries[i][1].idx).toBe(i);
|
||||
}
|
||||
});
|
||||
|
||||
it('删除后重新插入', () => {
|
||||
mt.put('k', { v: 1 });
|
||||
mt.delete('k');
|
||||
mt.put('k', { v: 2 });
|
||||
expect(mt.get('k')!.v).toBe(2);
|
||||
});
|
||||
|
||||
it('范围扫描包含边界', () => {
|
||||
mt.put('aa', { v: 1 });
|
||||
mt.put('ab', { v: 2 });
|
||||
mt.put('ac', { v: 3 });
|
||||
const results = mt.rangeScan('aa', 'ab');
|
||||
expect(results).toHaveLength(2);
|
||||
expect(results[0][0]).toBe('aa');
|
||||
expect(results[1][0]).toBe('ab');
|
||||
});
|
||||
});
|
||||
|
||||
+337
-337
@@ -1,337 +1,337 @@
|
||||
/**
|
||||
* AriaEngine Page 格式单元测试
|
||||
* 覆盖: PageHeader / Slot / Tuple 编解码 + PageFormat 整合
|
||||
*/
|
||||
import {
|
||||
PAGE_SIZE, PageType, PAGE_HEADER_SIZE, SLOT_ENTRY_SIZE,
|
||||
} from '../../src/engine/aria/types';
|
||||
import {
|
||||
encodePageHeader, decodePageHeader, initPageHeader, getPageType,
|
||||
getSlotCount, getFreeStart, setFreeStart, setFreeEnd,
|
||||
} from '../../src/engine/aria/page/header';
|
||||
import {
|
||||
getSlotEntry, setSlotEntry, getSlotDirectorySize,
|
||||
getFreeSpace, hasEnoughSpace, allocateSlot, readSlotData, freeSlot,
|
||||
} from '../../src/engine/aria/page/slot';
|
||||
import {
|
||||
encodeTuple, decodeTuple, getColumnEncodingMap,
|
||||
} from '../../src/engine/aria/page/tuple';
|
||||
import { ColumnEncoding } from '../../src/engine/aria/types';
|
||||
import {
|
||||
createPage, pageFromBuffer, pageInsertRow, pageReadRow,
|
||||
pageDeleteRow, pageUpdateRow, computeChecksum, verifyChecksum, updateChecksum,
|
||||
} from '../../src/engine/aria/page/format';
|
||||
|
||||
// ===================================================================
|
||||
// PageHeader
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Header', () => {
|
||||
let buf: ArrayBuffer;
|
||||
|
||||
beforeEach(() => {
|
||||
buf = new ArrayBuffer(PAGE_SIZE);
|
||||
});
|
||||
|
||||
it('initPageHeader 初始化头部字段', () => {
|
||||
initPageHeader(buf, 42, PageType.DATA);
|
||||
const h = decodePageHeader(buf);
|
||||
expect(h.pageId).toBe(42);
|
||||
expect(h.type).toBe(PageType.DATA);
|
||||
expect(h.slotCount).toBe(0);
|
||||
expect(h.freeStart).toBe(PAGE_HEADER_SIZE);
|
||||
expect(h.freeEnd).toBe(PAGE_SIZE);
|
||||
});
|
||||
|
||||
it('initPageHeader — INDEX 类型页面', () => {
|
||||
initPageHeader(buf, 99, PageType.INDEX);
|
||||
expect(getPageType(buf)).toBe(PageType.INDEX);
|
||||
});
|
||||
|
||||
it('encodePageHeader + decodePageHeader 往返一致', () => {
|
||||
const header = { pageId: 7, type: PageType.META, freeStart: 32, freeEnd: 4000, slotCount: 5, checksum: 0xdeadbeef };
|
||||
encodePageHeader(header, buf);
|
||||
const decoded = decodePageHeader(buf);
|
||||
expect(decoded.pageId).toBe(7);
|
||||
expect(decoded.type).toBe(PageType.META);
|
||||
expect(decoded.freeStart).toBe(32);
|
||||
expect(decoded.freeEnd).toBe(4000);
|
||||
expect(decoded.slotCount).toBe(5);
|
||||
});
|
||||
|
||||
it('不同 pageId 正确编解码', () => {
|
||||
for (const id of [0, 1, 255, 65535, 0xffffffff]) {
|
||||
initPageHeader(buf, id, PageType.DATA);
|
||||
expect(decodePageHeader(buf).pageId).toBe(id >>> 0);
|
||||
}
|
||||
});
|
||||
|
||||
it('setFreeStart / setFreeEnd 修改字段', () => {
|
||||
initPageHeader(buf, 1, PageType.DATA);
|
||||
setFreeStart(buf, 100);
|
||||
setFreeEnd(buf, 3000);
|
||||
expect(getFreeStart(buf)).toBe(100);
|
||||
expect(decodePageHeader(buf).freeEnd).toBe(3000);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Slot Directory
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Slot', () => {
|
||||
let buf: ArrayBuffer;
|
||||
|
||||
beforeEach(() => {
|
||||
buf = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(buf, 1, PageType.DATA);
|
||||
});
|
||||
|
||||
it('getSlotEntry — 空页面 slotCount 为 0', () => {
|
||||
expect(getSlotCount(buf)).toBe(0);
|
||||
});
|
||||
|
||||
it('setSlotEntry + getSlotEntry 往返', () => {
|
||||
// 手动写一个 slot（不通过 allocateSlot）
|
||||
new DataView(buf).setUint16(9, 1, false); // slotCount = 1
|
||||
setSlotEntry(buf, 0, { offset: 1000, length: 50 });
|
||||
const entry = getSlotEntry(buf, 0);
|
||||
expect(entry.offset).toBe(1000);
|
||||
expect(entry.length).toBe(50);
|
||||
});
|
||||
|
||||
it('getSlotDirectorySize 计算正确', () => {
|
||||
expect(getSlotDirectorySize(0)).toBe(0);
|
||||
expect(getSlotDirectorySize(1)).toBe(SLOT_ENTRY_SIZE);
|
||||
expect(getSlotDirectorySize(10)).toBe(10 * SLOT_ENTRY_SIZE);
|
||||
});
|
||||
|
||||
it('getFreeSpace — 空页面有最大空闲空间', () => {
|
||||
const free = getFreeSpace(buf);
|
||||
expect(free).toBe(PAGE_SIZE - PAGE_HEADER_SIZE);
|
||||
});
|
||||
|
||||
it('hasEnoughSpace — 小数据返回 true', () => {
|
||||
expect(hasEnoughSpace(buf, 100)).toBe(true);
|
||||
});
|
||||
|
||||
it('hasEnoughSpace — 超大数据返回 false', () => {
|
||||
expect(hasEnoughSpace(buf, PAGE_SIZE * 2)).toBe(false);
|
||||
});
|
||||
|
||||
it('allocateSlot 分配并写入数据', () => {
|
||||
const data = new Uint8Array([1, 2, 3, 4, 5]);
|
||||
const idx = allocateSlot(buf, data);
|
||||
expect(idx).toBe(0);
|
||||
expect(getSlotCount(buf)).toBe(1);
|
||||
|
||||
const readBack = readSlotData(buf, 0);
|
||||
expect(readBack).not.toBeNull();
|
||||
expect(Array.from(readBack!)).toEqual([1, 2, 3, 4, 5]);
|
||||
});
|
||||
|
||||
it('allocateSlot 多次分配', () => {
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const data = new Uint8Array([i, i + 1]);
|
||||
const idx = allocateSlot(buf, data);
|
||||
expect(idx).toBe(i);
|
||||
}
|
||||
expect(getSlotCount(buf)).toBe(10);
|
||||
|
||||
// 验证每个 slot 数据正确
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const data = readSlotData(buf, i);
|
||||
expect(Array.from(data!)).toEqual([i, i + 1]);
|
||||
}
|
||||
});
|
||||
|
||||
it('freeSlot 标记为删除', () => {
|
||||
const data = new Uint8Array([10, 20, 30]);
|
||||
allocateSlot(buf, data);
|
||||
freeSlot(buf, 0);
|
||||
const entry = getSlotEntry(buf, 0);
|
||||
expect(entry.offset).toBe(0);
|
||||
expect(entry.length).toBe(0);
|
||||
});
|
||||
|
||||
it('readSlotData — 已删除 slot 返回 null', () => {
|
||||
allocateSlot(buf, new Uint8Array([1]));
|
||||
freeSlot(buf, 0);
|
||||
expect(readSlotData(buf, 0)).toBeNull();
|
||||
});
|
||||
|
||||
it('allocateSlot — 空间不足返回 -1', () => {
|
||||
// 填满页面
|
||||
const big = new Uint8Array(PAGE_SIZE - PAGE_HEADER_SIZE - SLOT_ENTRY_SIZE);
|
||||
const idx1 = allocateSlot(buf, big);
|
||||
expect(idx1).toBe(0);
|
||||
const idx2 = allocateSlot(buf, new Uint8Array([1]));
|
||||
expect(idx2).toBe(-1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Tuple Codec
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Tuple', () => {
|
||||
const colOrder = ['id', 'name', 'age', 'active', 'data'];
|
||||
const colTypes: Record<string, string> = {
|
||||
id: 'string', name: 'string', age: 'number', active: 'boolean', data: 'json',
|
||||
};
|
||||
|
||||
it('encodeTuple + decodeTuple 完整往返', () => {
|
||||
const row = { id: '1', name: 'Alice', age: 30, active: true, data: { x: 1 } };
|
||||
const encoded = encodeTuple(row, colOrder, colTypes);
|
||||
expect(encoded.byteLength).toBeGreaterThan(0);
|
||||
|
||||
const decoded = decodeTuple(encoded, colOrder, colTypes);
|
||||
expect(decoded).not.toBeNull();
|
||||
expect(decoded!.id).toBe('1');
|
||||
expect(decoded!.name).toBe('Alice');
|
||||
expect(decoded!.age).toBe(30);
|
||||
expect(decoded!.active).toBe(true);
|
||||
expect(decoded!.data).toEqual({ x: 1 });
|
||||
});
|
||||
|
||||
it('encodeTuple — null 值正确处理', () => {
|
||||
const row = { id: '2', name: null, age: 25, active: null, data: null };
|
||||
const encoded = encodeTuple(row, colOrder, colTypes);
|
||||
const decoded = decodeTuple(encoded, colOrder, colTypes);
|
||||
expect(decoded!.name).toBeNull();
|
||||
expect(decoded!.active).toBeNull();
|
||||
expect(decoded!.data).toBeNull();
|
||||
});
|
||||
|
||||
it('encodeTuple — undefined 值按 null 处理', () => {
|
||||
const row = { id: '3', age: 30 } as any;
|
||||
const encoded = encodeTuple(row, colOrder, colTypes);
|
||||
const decoded = decodeTuple(encoded, colOrder, colTypes);
|
||||
expect(decoded!.id).toBe('3');
|
||||
expect(decoded!.name).toBeNull();
|
||||
});
|
||||
|
||||
it('encodeTuple — date 类型', () => {
|
||||
const order = ['ts'];
|
||||
const types = { ts: 'date' };
|
||||
const row = { ts: '2024-01-15T00:00:00.000Z' };
|
||||
const encoded = encodeTuple(row, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.ts).toBe('2024-01-15T00:00:00.000Z');
|
||||
});
|
||||
|
||||
it('encodeTuple — boolean false', () => {
|
||||
const order = ['flag'];
|
||||
const types = { flag: 'boolean' };
|
||||
const encoded = encodeTuple({ flag: false }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.flag).toBe(false);
|
||||
});
|
||||
|
||||
it('encodeTuple — 负数', () => {
|
||||
const order = ['val'];
|
||||
const types = { val: 'number' };
|
||||
const encoded = encodeTuple({ val: -42.5 }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.val).toBe(-42.5);
|
||||
});
|
||||
|
||||
it('encodeTuple — 空字符串', () => {
|
||||
const order = ['s'];
|
||||
const types = { s: 'string' };
|
||||
const encoded = encodeTuple({ s: '' }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.s).toBe('');
|
||||
});
|
||||
|
||||
it('encodeTuple — 长字符串', () => {
|
||||
const order = ['s'];
|
||||
const types = { s: 'string' };
|
||||
const long = 'x'.repeat(10000);
|
||||
const encoded = encodeTuple({ s: long }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.s).toBe(long);
|
||||
});
|
||||
|
||||
it('getColumnEncodingMap 返回正确映射', () => {
|
||||
const map = getColumnEncodingMap(colOrder, colTypes);
|
||||
expect(map.get('id')).toBe(ColumnEncoding.STRING);
|
||||
expect(map.get('age')).toBe(ColumnEncoding.NUMBER);
|
||||
expect(map.get('active')).toBe(ColumnEncoding.BOOLEAN);
|
||||
expect(map.get('data')).toBe(ColumnEncoding.JSON);
|
||||
});
|
||||
|
||||
it('decodeTuple — 损坏数据返回 null', () => {
|
||||
const broken = new Uint8Array([0xff, 0xff, 0xff]);
|
||||
expect(decodeTuple(broken, colOrder, colTypes)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Page Format 整合
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Format', () => {
|
||||
const colOrder = ['id', 'name'];
|
||||
const colTypes: Record<string, string> = { id: 'string', name: 'string' };
|
||||
|
||||
it('createPage 创建合法页面', () => {
|
||||
const page = createPage(100, PageType.DATA);
|
||||
expect(page.pageId).toBe(100);
|
||||
expect(page.type).toBe(PageType.DATA);
|
||||
expect(page.pins).toBe(0);
|
||||
expect(page.dirty).toBe(true);
|
||||
});
|
||||
|
||||
it('pageInsertRow + pageReadRow 往返', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
const row = { id: 'u1', name: 'Test' };
|
||||
const idx = pageInsertRow(page, row, colOrder, colTypes);
|
||||
expect(idx).toBe(0);
|
||||
|
||||
const read = pageReadRow(page, 0, colOrder, colTypes);
|
||||
expect(read).not.toBeNull();
|
||||
expect(read!.id).toBe('u1');
|
||||
expect(read!.name).toBe('Test');
|
||||
});
|
||||
|
||||
it('pageInsertRow — 多次插入', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
for (let i = 0; i < 50; i++) {
|
||||
const idx = pageInsertRow(page, { id: `${i}`, name: `User${i}` }, colOrder, colTypes);
|
||||
expect(idx).toBe(i);
|
||||
}
|
||||
for (let i = 0; i < 50; i++) {
|
||||
const row = pageReadRow(page, i, colOrder, colTypes);
|
||||
expect(row!.id).toBe(`${i}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('pageDeleteRow 标记删除', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
pageInsertRow(page, { id: '1', name: 'A' }, colOrder, colTypes);
|
||||
pageInsertRow(page, { id: '2', name: 'B' }, colOrder, colTypes);
|
||||
pageDeleteRow(page, 0);
|
||||
expect(page.dirty).toBe(true);
|
||||
// 已删除的行读取失败
|
||||
const row = pageReadRow(page, 0, colOrder, colTypes);
|
||||
expect(row).toBeNull();
|
||||
// 未删除的行仍然可读
|
||||
const row2 = pageReadRow(page, 1, colOrder, colTypes);
|
||||
expect(row2!.id).toBe('2');
|
||||
});
|
||||
|
||||
it('pageFromBuffer 从 ArrayBuffer 恢复', () => {
|
||||
const page = createPage(5, PageType.INDEX);
|
||||
const restored = pageFromBuffer(5, page.data);
|
||||
expect(restored.pageId).toBe(5);
|
||||
expect(restored.type).toBe(PageType.INDEX);
|
||||
expect(restored.dirty).toBe(false);
|
||||
});
|
||||
|
||||
it('computeChecksum + verifyChecksum', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
updateChecksum(page);
|
||||
expect(verifyChecksum(page)).toBe(true);
|
||||
|
||||
// 修改页面 → 校验和失效
|
||||
new Uint8Array(page.data)[100] = 0xff;
|
||||
expect(verifyChecksum(page)).toBe(false);
|
||||
});
|
||||
});
|
||||
/**
|
||||
* AriaEngine Page 格式单元测试
|
||||
* 覆盖: PageHeader / Slot / Tuple 编解码 + PageFormat 整合
|
||||
*/
|
||||
import {
|
||||
PAGE_SIZE, PageType, PAGE_HEADER_SIZE, SLOT_ENTRY_SIZE,
|
||||
} from '../../src/engine/aria/types';
|
||||
import {
|
||||
encodePageHeader, decodePageHeader, initPageHeader, getPageType,
|
||||
getSlotCount, getFreeStart, setFreeStart, setFreeEnd,
|
||||
} from '../../src/engine/aria/page/header';
|
||||
import {
|
||||
getSlotEntry, setSlotEntry, getSlotDirectorySize,
|
||||
getFreeSpace, hasEnoughSpace, allocateSlot, readSlotData, freeSlot,
|
||||
} from '../../src/engine/aria/page/slot';
|
||||
import {
|
||||
encodeTuple, decodeTuple, getColumnEncodingMap,
|
||||
} from '../../src/engine/aria/page/tuple';
|
||||
import { ColumnEncoding } from '../../src/engine/aria/types';
|
||||
import {
|
||||
createPage, pageFromBuffer, pageInsertRow, pageReadRow,
|
||||
pageDeleteRow, pageUpdateRow, computeChecksum, verifyChecksum, updateChecksum,
|
||||
} from '../../src/engine/aria/page/format';
|
||||
|
||||
// ===================================================================
|
||||
// PageHeader
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Header', () => {
|
||||
let buf: ArrayBuffer;
|
||||
|
||||
beforeEach(() => {
|
||||
buf = new ArrayBuffer(PAGE_SIZE);
|
||||
});
|
||||
|
||||
it('initPageHeader 初始化头部字段', () => {
|
||||
initPageHeader(buf, 42, PageType.DATA);
|
||||
const h = decodePageHeader(buf);
|
||||
expect(h.pageId).toBe(42);
|
||||
expect(h.type).toBe(PageType.DATA);
|
||||
expect(h.slotCount).toBe(0);
|
||||
expect(h.freeStart).toBe(PAGE_HEADER_SIZE);
|
||||
expect(h.freeEnd).toBe(PAGE_SIZE);
|
||||
});
|
||||
|
||||
it('initPageHeader — INDEX 类型页面', () => {
|
||||
initPageHeader(buf, 99, PageType.INDEX);
|
||||
expect(getPageType(buf)).toBe(PageType.INDEX);
|
||||
});
|
||||
|
||||
it('encodePageHeader + decodePageHeader 往返一致', () => {
|
||||
const header = { pageId: 7, type: PageType.META, freeStart: 32, freeEnd: 4000, slotCount: 5, checksum: 0xdeadbeef };
|
||||
encodePageHeader(header, buf);
|
||||
const decoded = decodePageHeader(buf);
|
||||
expect(decoded.pageId).toBe(7);
|
||||
expect(decoded.type).toBe(PageType.META);
|
||||
expect(decoded.freeStart).toBe(32);
|
||||
expect(decoded.freeEnd).toBe(4000);
|
||||
expect(decoded.slotCount).toBe(5);
|
||||
});
|
||||
|
||||
it('不同 pageId 正确编解码', () => {
|
||||
for (const id of [0, 1, 255, 65535, 0xffffffff]) {
|
||||
initPageHeader(buf, id, PageType.DATA);
|
||||
expect(decodePageHeader(buf).pageId).toBe(id >>> 0);
|
||||
}
|
||||
});
|
||||
|
||||
it('setFreeStart / setFreeEnd 修改字段', () => {
|
||||
initPageHeader(buf, 1, PageType.DATA);
|
||||
setFreeStart(buf, 100);
|
||||
setFreeEnd(buf, 3000);
|
||||
expect(getFreeStart(buf)).toBe(100);
|
||||
expect(decodePageHeader(buf).freeEnd).toBe(3000);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Slot Directory
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Slot', () => {
|
||||
let buf: ArrayBuffer;
|
||||
|
||||
beforeEach(() => {
|
||||
buf = new ArrayBuffer(PAGE_SIZE);
|
||||
initPageHeader(buf, 1, PageType.DATA);
|
||||
});
|
||||
|
||||
it('getSlotEntry — 空页面 slotCount 为 0', () => {
|
||||
expect(getSlotCount(buf)).toBe(0);
|
||||
});
|
||||
|
||||
it('setSlotEntry + getSlotEntry 往返', () => {
|
||||
// 手动写一个 slot（不通过 allocateSlot）
|
||||
new DataView(buf).setUint16(9, 1, false); // slotCount = 1
|
||||
setSlotEntry(buf, 0, { offset: 1000, length: 50 });
|
||||
const entry = getSlotEntry(buf, 0);
|
||||
expect(entry.offset).toBe(1000);
|
||||
expect(entry.length).toBe(50);
|
||||
});
|
||||
|
||||
it('getSlotDirectorySize 计算正确', () => {
|
||||
expect(getSlotDirectorySize(0)).toBe(0);
|
||||
expect(getSlotDirectorySize(1)).toBe(SLOT_ENTRY_SIZE);
|
||||
expect(getSlotDirectorySize(10)).toBe(10 * SLOT_ENTRY_SIZE);
|
||||
});
|
||||
|
||||
it('getFreeSpace — 空页面有最大空闲空间', () => {
|
||||
const free = getFreeSpace(buf);
|
||||
expect(free).toBe(PAGE_SIZE - PAGE_HEADER_SIZE);
|
||||
});
|
||||
|
||||
it('hasEnoughSpace — 小数据返回 true', () => {
|
||||
expect(hasEnoughSpace(buf, 100)).toBe(true);
|
||||
});
|
||||
|
||||
it('hasEnoughSpace — 超大数据返回 false', () => {
|
||||
expect(hasEnoughSpace(buf, PAGE_SIZE * 2)).toBe(false);
|
||||
});
|
||||
|
||||
it('allocateSlot 分配并写入数据', () => {
|
||||
const data = new Uint8Array([1, 2, 3, 4, 5]);
|
||||
const idx = allocateSlot(buf, data);
|
||||
expect(idx).toBe(0);
|
||||
expect(getSlotCount(buf)).toBe(1);
|
||||
|
||||
const readBack = readSlotData(buf, 0);
|
||||
expect(readBack).not.toBeNull();
|
||||
expect(Array.from(readBack!)).toEqual([1, 2, 3, 4, 5]);
|
||||
});
|
||||
|
||||
it('allocateSlot 多次分配', () => {
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const data = new Uint8Array([i, i + 1]);
|
||||
const idx = allocateSlot(buf, data);
|
||||
expect(idx).toBe(i);
|
||||
}
|
||||
expect(getSlotCount(buf)).toBe(10);
|
||||
|
||||
// 验证每个 slot 数据正确
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const data = readSlotData(buf, i);
|
||||
expect(Array.from(data!)).toEqual([i, i + 1]);
|
||||
}
|
||||
});
|
||||
|
||||
it('freeSlot 标记为删除', () => {
|
||||
const data = new Uint8Array([10, 20, 30]);
|
||||
allocateSlot(buf, data);
|
||||
freeSlot(buf, 0);
|
||||
const entry = getSlotEntry(buf, 0);
|
||||
expect(entry.offset).toBe(0);
|
||||
expect(entry.length).toBe(0);
|
||||
});
|
||||
|
||||
it('readSlotData — 已删除 slot 返回 null', () => {
|
||||
allocateSlot(buf, new Uint8Array([1]));
|
||||
freeSlot(buf, 0);
|
||||
expect(readSlotData(buf, 0)).toBeNull();
|
||||
});
|
||||
|
||||
it('allocateSlot — 空间不足返回 -1', () => {
|
||||
// 填满页面
|
||||
const big = new Uint8Array(PAGE_SIZE - PAGE_HEADER_SIZE - SLOT_ENTRY_SIZE);
|
||||
const idx1 = allocateSlot(buf, big);
|
||||
expect(idx1).toBe(0);
|
||||
const idx2 = allocateSlot(buf, new Uint8Array([1]));
|
||||
expect(idx2).toBe(-1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Tuple Codec
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Tuple', () => {
|
||||
const colOrder = ['id', 'name', 'age', 'active', 'data'];
|
||||
const colTypes: Record<string, string> = {
|
||||
id: 'string', name: 'string', age: 'number', active: 'boolean', data: 'json',
|
||||
};
|
||||
|
||||
it('encodeTuple + decodeTuple 完整往返', () => {
|
||||
const row = { id: '1', name: 'Alice', age: 30, active: true, data: { x: 1 } };
|
||||
const encoded = encodeTuple(row, colOrder, colTypes);
|
||||
expect(encoded.byteLength).toBeGreaterThan(0);
|
||||
|
||||
const decoded = decodeTuple(encoded, colOrder, colTypes);
|
||||
expect(decoded).not.toBeNull();
|
||||
expect(decoded!.id).toBe('1');
|
||||
expect(decoded!.name).toBe('Alice');
|
||||
expect(decoded!.age).toBe(30);
|
||||
expect(decoded!.active).toBe(true);
|
||||
expect(decoded!.data).toEqual({ x: 1 });
|
||||
});
|
||||
|
||||
it('encodeTuple — null 值正确处理', () => {
|
||||
const row = { id: '2', name: null, age: 25, active: null, data: null };
|
||||
const encoded = encodeTuple(row, colOrder, colTypes);
|
||||
const decoded = decodeTuple(encoded, colOrder, colTypes);
|
||||
expect(decoded!.name).toBeNull();
|
||||
expect(decoded!.active).toBeNull();
|
||||
expect(decoded!.data).toBeNull();
|
||||
});
|
||||
|
||||
it('encodeTuple — undefined 值按 null 处理', () => {
|
||||
const row = { id: '3', age: 30 } as any;
|
||||
const encoded = encodeTuple(row, colOrder, colTypes);
|
||||
const decoded = decodeTuple(encoded, colOrder, colTypes);
|
||||
expect(decoded!.id).toBe('3');
|
||||
expect(decoded!.name).toBeNull();
|
||||
});
|
||||
|
||||
it('encodeTuple — date 类型', () => {
|
||||
const order = ['ts'];
|
||||
const types = { ts: 'date' };
|
||||
const row = { ts: '2024-01-15T00:00:00.000Z' };
|
||||
const encoded = encodeTuple(row, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.ts).toBe('2024-01-15T00:00:00.000Z');
|
||||
});
|
||||
|
||||
it('encodeTuple — boolean false', () => {
|
||||
const order = ['flag'];
|
||||
const types = { flag: 'boolean' };
|
||||
const encoded = encodeTuple({ flag: false }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.flag).toBe(false);
|
||||
});
|
||||
|
||||
it('encodeTuple — 负数', () => {
|
||||
const order = ['val'];
|
||||
const types = { val: 'number' };
|
||||
const encoded = encodeTuple({ val: -42.5 }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.val).toBe(-42.5);
|
||||
});
|
||||
|
||||
it('encodeTuple — 空字符串', () => {
|
||||
const order = ['s'];
|
||||
const types = { s: 'string' };
|
||||
const encoded = encodeTuple({ s: '' }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.s).toBe('');
|
||||
});
|
||||
|
||||
it('encodeTuple — 长字符串', () => {
|
||||
const order = ['s'];
|
||||
const types = { s: 'string' };
|
||||
const long = 'x'.repeat(10000);
|
||||
const encoded = encodeTuple({ s: long }, order, types);
|
||||
const decoded = decodeTuple(encoded, order, types);
|
||||
expect(decoded!.s).toBe(long);
|
||||
});
|
||||
|
||||
it('getColumnEncodingMap 返回正确映射', () => {
|
||||
const map = getColumnEncodingMap(colOrder, colTypes);
|
||||
expect(map.get('id')).toBe(ColumnEncoding.STRING);
|
||||
expect(map.get('age')).toBe(ColumnEncoding.NUMBER);
|
||||
expect(map.get('active')).toBe(ColumnEncoding.BOOLEAN);
|
||||
expect(map.get('data')).toBe(ColumnEncoding.JSON);
|
||||
});
|
||||
|
||||
it('decodeTuple — 损坏数据返回 null', () => {
|
||||
const broken = new Uint8Array([0xff, 0xff, 0xff]);
|
||||
expect(decodeTuple(broken, colOrder, colTypes)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Page Format 整合
|
||||
// ===================================================================
|
||||
describe('AriaEngine Page — Format', () => {
|
||||
const colOrder = ['id', 'name'];
|
||||
const colTypes: Record<string, string> = { id: 'string', name: 'string' };
|
||||
|
||||
it('createPage 创建合法页面', () => {
|
||||
const page = createPage(100, PageType.DATA);
|
||||
expect(page.pageId).toBe(100);
|
||||
expect(page.type).toBe(PageType.DATA);
|
||||
expect(page.pins).toBe(0);
|
||||
expect(page.dirty).toBe(true);
|
||||
});
|
||||
|
||||
it('pageInsertRow + pageReadRow 往返', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
const row = { id: 'u1', name: 'Test' };
|
||||
const idx = pageInsertRow(page, row, colOrder, colTypes);
|
||||
expect(idx).toBe(0);
|
||||
|
||||
const read = pageReadRow(page, 0, colOrder, colTypes);
|
||||
expect(read).not.toBeNull();
|
||||
expect(read!.id).toBe('u1');
|
||||
expect(read!.name).toBe('Test');
|
||||
});
|
||||
|
||||
it('pageInsertRow — 多次插入', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
for (let i = 0; i < 50; i++) {
|
||||
const idx = pageInsertRow(page, { id: `${i}`, name: `User${i}` }, colOrder, colTypes);
|
||||
expect(idx).toBe(i);
|
||||
}
|
||||
for (let i = 0; i < 50; i++) {
|
||||
const row = pageReadRow(page, i, colOrder, colTypes);
|
||||
expect(row!.id).toBe(`${i}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('pageDeleteRow 标记删除', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
pageInsertRow(page, { id: '1', name: 'A' }, colOrder, colTypes);
|
||||
pageInsertRow(page, { id: '2', name: 'B' }, colOrder, colTypes);
|
||||
pageDeleteRow(page, 0);
|
||||
expect(page.dirty).toBe(true);
|
||||
// 已删除的行读取失败
|
||||
const row = pageReadRow(page, 0, colOrder, colTypes);
|
||||
expect(row).toBeNull();
|
||||
// 未删除的行仍然可读
|
||||
const row2 = pageReadRow(page, 1, colOrder, colTypes);
|
||||
expect(row2!.id).toBe('2');
|
||||
});
|
||||
|
||||
it('pageFromBuffer 从 ArrayBuffer 恢复', () => {
|
||||
const page = createPage(5, PageType.INDEX);
|
||||
const restored = pageFromBuffer(5, page.data);
|
||||
expect(restored.pageId).toBe(5);
|
||||
expect(restored.type).toBe(PageType.INDEX);
|
||||
expect(restored.dirty).toBe(false);
|
||||
});
|
||||
|
||||
it('computeChecksum + verifyChecksum', () => {
|
||||
const page = createPage(1, PageType.DATA);
|
||||
updateChecksum(page);
|
||||
expect(verifyChecksum(page)).toBe(true);
|
||||
|
||||
// 修改页面 → 校验和失效
|
||||
new Uint8Array(page.data)[100] = 0xff;
|
||||
expect(verifyChecksum(page)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
+129
-129
@@ -1,129 +1,129 @@
|
||||
/**
|
||||
* AriaEngine SSTable Builder + Reader 单元测试
|
||||
*/
|
||||
import { SSTableBuilder } from '../../src/engine/aria/index/sstable_builder';
|
||||
import { SSTableReader } from '../../src/engine/aria/index/sstable';
|
||||
import type { SSTableMeta } from '../../src/engine/aria/types';
|
||||
|
||||
// ===================================================================
|
||||
// SSTable Builder + Reader
|
||||
// ===================================================================
|
||||
describe('AriaEngine — SSTable Builder + Reader', () => {
|
||||
const makeMeta = (data: Uint8Array): SSTableMeta => ({
|
||||
id: 1, level: 0, minKey: '', maxKey: '\uffff',
|
||||
blockCount: 1, totalSize: data.byteLength, bloomData: null,
|
||||
});
|
||||
|
||||
it('构建单条目 SSTable 并精确读取', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
builder.add('key1', { name: 'Alice', age: 30 });
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
const result = reader.get('key1');
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.name).toBe('Alice');
|
||||
expect(result!.age).toBe(30);
|
||||
});
|
||||
|
||||
it('构建多条 SSTable 并全部读取', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
const items: [string, Record<string, unknown>][] = [];
|
||||
for (let i = 0; i < 100; i++) {
|
||||
const key = `user-${String(i).padStart(3, '0')}`;
|
||||
const value = { idx: i, name: `User${i}` };
|
||||
items.push([key, value]);
|
||||
builder.add(key, value);
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
for (const [key, value] of items) {
|
||||
const result = reader.get(key);
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.idx).toBe(value.idx);
|
||||
}
|
||||
});
|
||||
|
||||
it('get — 不存在的 key 返回 null', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
builder.add('a', { v: 1 });
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.get('nonexistent')).toBeNull();
|
||||
});
|
||||
|
||||
it('rangeScan — 范围查询', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
for (let i = 0; i < 20; i++) {
|
||||
builder.add(`k-${String(i).padStart(2, '0')}`, { v: i });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
const results: [string, Record<string, unknown>][] = [];
|
||||
reader.rangeScan('k-05', 'k-10', (k, v) => results.push([k, v]));
|
||||
expect(results).toHaveLength(6);
|
||||
expect(results[0][0]).toBe('k-05');
|
||||
expect(results[results.length - 1][0]).toBe('k-10');
|
||||
});
|
||||
|
||||
it('scanAll — 遍历所有条目', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
const count = 50;
|
||||
for (let i = 0; i < count; i++) {
|
||||
builder.add(`item-${i}`, { idx: i });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
const items: [string, Record<string, unknown>][] = [];
|
||||
reader.scanAll((k, v) => items.push([k, v]));
|
||||
expect(items).toHaveLength(count);
|
||||
});
|
||||
|
||||
it('getIndexBlockCount 返回索引块数', () => {
|
||||
const builder = new SSTableBuilder(256); // 小 block size 触发多个 block
|
||||
for (let i = 0; i < 100; i++) {
|
||||
builder.add(`k-${i}`, { data: 'x'.repeat(50) });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.getIndexBlockCount()).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
it('边界 — 空 SSTable 不抛异常', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.get('any')).toBeNull();
|
||||
const results: [string, Record<string, unknown>][] = [];
|
||||
reader.scanAll((k, v) => results.push([k, v]));
|
||||
expect(results).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('带特殊字符的 key', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
// 必须按键排序添加（按 ASCII 排序：空格 < 短横 < 点号）
|
||||
builder.add('key with space', { v: 3 });
|
||||
builder.add('key-with-dash', { v: 1 });
|
||||
builder.add('key.with.dot', { v: 2 });
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.get('key-with-dash')!.v).toBe(1);
|
||||
expect(reader.get('key.with.dot')!.v).toBe(2);
|
||||
expect(reader.get('key with space')!.v).toBe(3);
|
||||
});
|
||||
|
||||
it('getEntryCount 返回正确条目数', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
builder.add('a', { v: 1 });
|
||||
builder.add('b', { v: 2 });
|
||||
builder.add('c', { v: 3 });
|
||||
expect(builder.getEntryCount()).toBe(3);
|
||||
});
|
||||
});
|
||||
/**
|
||||
* AriaEngine SSTable Builder + Reader 单元测试
|
||||
*/
|
||||
import { SSTableBuilder } from '../../src/engine/aria/index/sstable_builder';
|
||||
import { SSTableReader } from '../../src/engine/aria/index/sstable';
|
||||
import type { SSTableMeta } from '../../src/engine/aria/types';
|
||||
|
||||
// ===================================================================
|
||||
// SSTable Builder + Reader
|
||||
// ===================================================================
|
||||
describe('AriaEngine — SSTable Builder + Reader', () => {
|
||||
const makeMeta = (data: Uint8Array): SSTableMeta => ({
|
||||
id: 1, level: 0, minKey: '', maxKey: '\uffff',
|
||||
blockCount: 1, totalSize: data.byteLength, bloomData: null,
|
||||
});
|
||||
|
||||
it('构建单条目 SSTable 并精确读取', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
builder.add('key1', { name: 'Alice', age: 30 });
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
const result = reader.get('key1');
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.name).toBe('Alice');
|
||||
expect(result!.age).toBe(30);
|
||||
});
|
||||
|
||||
it('构建多条 SSTable 并全部读取', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
const items: [string, Record<string, unknown>][] = [];
|
||||
for (let i = 0; i < 100; i++) {
|
||||
const key = `user-${String(i).padStart(3, '0')}`;
|
||||
const value = { idx: i, name: `User${i}` };
|
||||
items.push([key, value]);
|
||||
builder.add(key, value);
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
for (const [key, value] of items) {
|
||||
const result = reader.get(key);
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.idx).toBe(value.idx);
|
||||
}
|
||||
});
|
||||
|
||||
it('get — 不存在的 key 返回 null', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
builder.add('a', { v: 1 });
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.get('nonexistent')).toBeNull();
|
||||
});
|
||||
|
||||
it('rangeScan — 范围查询', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
for (let i = 0; i < 20; i++) {
|
||||
builder.add(`k-${String(i).padStart(2, '0')}`, { v: i });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
const results: [string, Record<string, unknown>][] = [];
|
||||
reader.rangeScan('k-05', 'k-10', (k, v) => results.push([k, v]));
|
||||
expect(results).toHaveLength(6);
|
||||
expect(results[0][0]).toBe('k-05');
|
||||
expect(results[results.length - 1][0]).toBe('k-10');
|
||||
});
|
||||
|
||||
it('scanAll — 遍历所有条目', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
const count = 50;
|
||||
for (let i = 0; i < count; i++) {
|
||||
builder.add(`item-${i}`, { idx: i });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
const items: [string, Record<string, unknown>][] = [];
|
||||
reader.scanAll((k, v) => items.push([k, v]));
|
||||
expect(items).toHaveLength(count);
|
||||
});
|
||||
|
||||
it('getIndexBlockCount 返回索引块数', () => {
|
||||
const builder = new SSTableBuilder(256); // 小 block size 触发多个 block
|
||||
for (let i = 0; i < 100; i++) {
|
||||
builder.add(`k-${i}`, { data: 'x'.repeat(50) });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.getIndexBlockCount()).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
it('边界 — 空 SSTable 不抛异常', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.get('any')).toBeNull();
|
||||
const results: [string, Record<string, unknown>][] = [];
|
||||
reader.scanAll((k, v) => results.push([k, v]));
|
||||
expect(results).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('带特殊字符的 key', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
// 必须按键排序添加（按 ASCII 排序：空格 < 短横 < 点号）
|
||||
builder.add('key with space', { v: 3 });
|
||||
builder.add('key-with-dash', { v: 1 });
|
||||
builder.add('key.with.dot', { v: 2 });
|
||||
const { sstableData } = builder.build();
|
||||
|
||||
const reader = new SSTableReader(sstableData, makeMeta(sstableData));
|
||||
expect(reader.get('key-with-dash')!.v).toBe(1);
|
||||
expect(reader.get('key.with.dot')!.v).toBe(2);
|
||||
expect(reader.get('key with space')!.v).toBe(3);
|
||||
});
|
||||
|
||||
it('getEntryCount 返回正确条目数', () => {
|
||||
const builder = new SSTableBuilder(4096);
|
||||
builder.add('a', { v: 1 });
|
||||
builder.add('b', { v: 2 });
|
||||
builder.add('c', { v: 3 });
|
||||
expect(builder.getEntryCount()).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
+289
-289
@@ -1,289 +1,289 @@
|
||||
/**
|
||||
* AriaEngine WAL + MVCC 单元测试
|
||||
*/
|
||||
import { WAL, type WALStore } from '../../src/engine/aria/wal/log';
|
||||
import { WALRecordType, type WALRecord } from '../../src/engine/aria/types';
|
||||
import { CheckpointManager, type Flushable } from '../../src/engine/aria/wal/checkpoint';
|
||||
import { MVCCManager } from '../../src/engine/aria/transaction/mvcc';
|
||||
|
||||
// ===================================================================
|
||||
// WAL 存储 Mock
|
||||
// ===================================================================
|
||||
class MockWALStore implements WALStore {
|
||||
chunks: Uint8Array[] = [];
|
||||
async append(data: Uint8Array) { this.chunks.push(data); }
|
||||
async readAll(): Promise<Uint8Array> {
|
||||
const total = this.chunks.reduce((s, c) => s + c.byteLength, 0);
|
||||
const combined = new Uint8Array(total);
|
||||
let off = 0;
|
||||
for (const c of this.chunks) { combined.set(c, off); off += c.byteLength; }
|
||||
return combined;
|
||||
}
|
||||
async truncate() { this.chunks = []; }
|
||||
async exists() { return this.chunks.length > 0; }
|
||||
}
|
||||
|
||||
// ===================================================================
|
||||
// WAL 测试
|
||||
// ===================================================================
|
||||
describe('AriaEngine — WAL', () => {
|
||||
it('append 记录后可恢复', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 1, tableName: 'users', key: '1', data: { name: 'Alice' } });
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 1, tableName: 'users', key: '2', data: { name: 'Bob' } });
|
||||
|
||||
const records: WALRecord[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(2);
|
||||
expect(records[0].tableName).toBe('users');
|
||||
expect(records[0].key).toBe('1');
|
||||
});
|
||||
|
||||
it('batch 模式缓冲后 flush', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'batch');
|
||||
|
||||
wal.append({ type: WALRecordType.UPDATE, txnId: 2, tableName: 'items', key: 'a', data: { v: 1 } });
|
||||
wal.append({ type: WALRecordType.DELETE, txnId: 2, tableName: 'items', key: 'b' });
|
||||
|
||||
// 未 flush 前无法恢复
|
||||
let records: WALRecord[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(0);
|
||||
|
||||
await wal.flush();
|
||||
records = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('none 模式不记录', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, false, 'none');
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 3, tableName: 'x', key: 'y', data: {} });
|
||||
expect(store.chunks).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('checkpoint 清空 WAL', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
wal.append({ type: WALRecordType.CREATE_TABLE, txnId: 0, tableName: 't', key: '' });
|
||||
expect(await store.exists()).toBe(true);
|
||||
|
||||
await wal.checkpoint();
|
||||
expect(await store.exists()).toBe(false);
|
||||
});
|
||||
|
||||
it('getLSN 跟踪序列号', () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
expect(wal.getLSN()).toBe(0);
|
||||
wal.append({ type: WALRecordType.BEGIN, txnId: 10, tableName: '', key: '' });
|
||||
expect(wal.getLSN()).toBe(1);
|
||||
wal.append({ type: WALRecordType.COMMIT, txnId: 10, tableName: '', key: '' });
|
||||
expect(wal.getLSN()).toBe(2);
|
||||
});
|
||||
|
||||
it('isEnabled 反映配置', () => {
|
||||
expect(new WAL(new MockWALStore(), true).isEnabled()).toBe(true);
|
||||
expect(new WAL(new MockWALStore(), false).isEnabled()).toBe(false);
|
||||
});
|
||||
|
||||
it('多种记录类型编解码', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
wal.append({ type: WALRecordType.BEGIN, txnId: 100, tableName: '', key: '' });
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 100, tableName: 'users', key: '1', data: { x: 'hello' } });
|
||||
wal.append({ type: WALRecordType.UPDATE, txnId: 100, tableName: 'users', key: '1', data: { x: 'world' } });
|
||||
wal.append({ type: WALRecordType.COMMIT, txnId: 100, tableName: '', key: '' });
|
||||
|
||||
const records: WALRecord[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(4);
|
||||
expect(records[0].type).toBe(WALRecordType.BEGIN);
|
||||
expect(records[1].type).toBe(WALRecordType.INSERT);
|
||||
expect(records[2].type).toBe(WALRecordType.UPDATE);
|
||||
expect(records[3].type).toBe(WALRecordType.COMMIT);
|
||||
});
|
||||
|
||||
it('getBufferedCount 返回缓冲数', () => {
|
||||
const wal = new WAL(new MockWALStore(), true, 'batch');
|
||||
expect(wal.getBufferedCount()).toBe(0);
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: 'k' });
|
||||
expect(wal.getBufferedCount()).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Checkpoint 测试（使用安全 Mock，避免 null 引用导致 CI 卡死）
|
||||
// ===================================================================
|
||||
describe('AriaEngine — CheckpointManager', () => {
|
||||
class MockFlushable implements Flushable { flushed = false; async flushAll() { this.flushed = true; } }
|
||||
class MockLSM { flushed = false; async flush() { this.flushed = true; } }
|
||||
class MockWAL { checkpointed = false; async checkpoint() { this.checkpointed = true; } async flush() {} }
|
||||
|
||||
it('tick 未达间隔不触发 checkpoint', async () => {
|
||||
const lsm = new MockLSM();
|
||||
const wal = new MockWAL();
|
||||
const flushable = new MockFlushable();
|
||||
const cm = new CheckpointManager(lsm as any, wal as any, flushable, 100);
|
||||
await cm.tick();
|
||||
await cm.tick();
|
||||
expect(cm.getOpCount()).toBe(2);
|
||||
expect(flushable.flushed).toBe(false);
|
||||
expect(lsm.flushed).toBe(false);
|
||||
});
|
||||
|
||||
it('setInterval 修改间隔后 tick 触发 checkpoint', async () => {
|
||||
const lsm = new MockLSM();
|
||||
const wal = new MockWAL();
|
||||
const cm = new CheckpointManager(lsm as any, wal as any, null, 1000);
|
||||
cm.setInterval(2);
|
||||
await cm.tick();
|
||||
await cm.tick();
|
||||
expect(cm.getOpCount()).toBe(0); // reset after checkpoint
|
||||
expect(lsm.flushed).toBe(true);
|
||||
expect(wal.checkpointed).toBe(true);
|
||||
});
|
||||
|
||||
it('forceCheckpoint 强制触发', async () => {
|
||||
const lsm = new MockLSM();
|
||||
const wal = new MockWAL();
|
||||
const cm = new CheckpointManager(lsm as any, wal as any, null, 100);
|
||||
await cm.forceCheckpoint();
|
||||
expect(cm.getOpCount()).toBe(0);
|
||||
expect(lsm.flushed).toBe(true);
|
||||
expect(wal.checkpointed).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// MVCC 测试
|
||||
// ===================================================================
|
||||
describe('AriaEngine — MVCC', () => {
|
||||
let mvcc: MVCCManager;
|
||||
|
||||
beforeEach(() => { mvcc = new MVCCManager(); });
|
||||
|
||||
it('beginTransaction 分配唯一 ID', () => {
|
||||
const id1 = mvcc.beginTransaction();
|
||||
const id2 = mvcc.beginTransaction();
|
||||
expect(id1).not.toBe(id2);
|
||||
expect(mvcc.isActive(id1)).toBe(true);
|
||||
expect(mvcc.isActive(id2)).toBe(true);
|
||||
});
|
||||
|
||||
it('commit 后 isActive 返回 false', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.commitTransaction(txnId);
|
||||
expect(mvcc.isActive(txnId)).toBe(false);
|
||||
});
|
||||
|
||||
it('rollback 后 isActive 返回 false', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.rollbackTransaction(txnId);
|
||||
expect(mvcc.isActive(txnId)).toBe(false);
|
||||
});
|
||||
|
||||
it('writeVersion + readVersion 往返', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice', age: 30 }, txnId);
|
||||
const val = mvcc.readVersion('users', '1', txnId);
|
||||
expect(val).not.toBeNull();
|
||||
expect(val!.name).toBe('Alice');
|
||||
});
|
||||
|
||||
it('未提交版本对其他事务不可见', () => {
|
||||
const txn1 = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txn1);
|
||||
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txn2);
|
||||
expect(val).toBeNull(); // txn1's write not yet committed
|
||||
});
|
||||
|
||||
it('commit 后新事务可见', () => {
|
||||
const txn1 = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txn1);
|
||||
mvcc.commitTransaction(txn1);
|
||||
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txn2);
|
||||
expect(val).not.toBeNull();
|
||||
expect(val!.name).toBe('Alice');
|
||||
});
|
||||
|
||||
it('rollback 移除写入的版本', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Temp' }, txnId);
|
||||
mvcc.rollbackTransaction(txnId);
|
||||
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
expect(mvcc.readVersion('users', '1', txn2)).toBeNull();
|
||||
});
|
||||
|
||||
it('deleteVersion 创建墓碑', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
// delete
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
mvcc.deleteVersion('users', '1', txn2);
|
||||
mvcc.commitTransaction(txn2);
|
||||
// 删除后读取
|
||||
const txn3 = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txn3);
|
||||
expect(val).not.toBeNull();
|
||||
expect((val! as any).__mvcc_tombstone).toBe(true);
|
||||
});
|
||||
|
||||
it('getLatestCommittedVersions 返回最新已提交', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txnId);
|
||||
mvcc.writeVersion('users', '2', { name: 'Bob' }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
|
||||
const result = mvcc.getLatestCommittedVersions('users');
|
||||
expect(result['1'].name).toBe('Alice');
|
||||
expect(result['2'].name).toBe('Bob');
|
||||
});
|
||||
|
||||
it('clearTable 清理指定表', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'A' }, txnId);
|
||||
mvcc.writeVersion('users', '2', { name: 'B' }, txnId);
|
||||
mvcc.writeVersion('orders', 'o1', { amt: 100 }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
|
||||
mvcc.clearTable('users');
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
expect(mvcc.readVersion('users', '1', txn2)).toBeNull();
|
||||
expect(mvcc.readVersion('users', '2', txn2)).toBeNull();
|
||||
expect(mvcc.readVersion('orders', 'o1', txn2)).not.toBeNull();
|
||||
});
|
||||
|
||||
it('getActiveTxnCount 返回活跃事务数', () => {
|
||||
expect(mvcc.getActiveTxnCount()).toBe(0);
|
||||
mvcc.beginTransaction();
|
||||
mvcc.beginTransaction();
|
||||
expect(mvcc.getActiveTxnCount()).toBe(2);
|
||||
});
|
||||
|
||||
it('gc 清理过旧版本', () => {
|
||||
// 创建很多版本后 gc
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { ver: i }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
}
|
||||
mvcc.gc(100);
|
||||
// gc 后应可继续操作
|
||||
const txnId = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txnId);
|
||||
expect(val).not.toBeNull();
|
||||
});
|
||||
});
|
||||
/**
|
||||
* AriaEngine WAL + MVCC 单元测试
|
||||
*/
|
||||
import { WAL, type WALStore } from '../../src/engine/aria/wal/log';
|
||||
import { WALRecordType, type WALRecord } from '../../src/engine/aria/types';
|
||||
import { CheckpointManager, type Flushable } from '../../src/engine/aria/wal/checkpoint';
|
||||
import { MVCCManager } from '../../src/engine/aria/transaction/mvcc';
|
||||
|
||||
// ===================================================================
|
||||
// WAL 存储 Mock
|
||||
// ===================================================================
|
||||
class MockWALStore implements WALStore {
|
||||
chunks: Uint8Array[] = [];
|
||||
async append(data: Uint8Array) { this.chunks.push(data); }
|
||||
async readAll(): Promise<Uint8Array> {
|
||||
const total = this.chunks.reduce((s, c) => s + c.byteLength, 0);
|
||||
const combined = new Uint8Array(total);
|
||||
let off = 0;
|
||||
for (const c of this.chunks) { combined.set(c, off); off += c.byteLength; }
|
||||
return combined;
|
||||
}
|
||||
async truncate() { this.chunks = []; }
|
||||
async exists() { return this.chunks.length > 0; }
|
||||
}
|
||||
|
||||
// ===================================================================
|
||||
// WAL 测试
|
||||
// ===================================================================
|
||||
describe('AriaEngine — WAL', () => {
|
||||
it('append 记录后可恢复', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 1, tableName: 'users', key: '1', data: { name: 'Alice' } });
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 1, tableName: 'users', key: '2', data: { name: 'Bob' } });
|
||||
|
||||
const records: WALRecord[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(2);
|
||||
expect(records[0].tableName).toBe('users');
|
||||
expect(records[0].key).toBe('1');
|
||||
});
|
||||
|
||||
it('batch 模式缓冲后 flush', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'batch');
|
||||
|
||||
wal.append({ type: WALRecordType.UPDATE, txnId: 2, tableName: 'items', key: 'a', data: { v: 1 } });
|
||||
wal.append({ type: WALRecordType.DELETE, txnId: 2, tableName: 'items', key: 'b' });
|
||||
|
||||
// 未 flush 前无法恢复
|
||||
let records: WALRecord[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(0);
|
||||
|
||||
await wal.flush();
|
||||
records = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('none 模式不记录', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, false, 'none');
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 3, tableName: 'x', key: 'y', data: {} });
|
||||
expect(store.chunks).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('checkpoint 清空 WAL', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
wal.append({ type: WALRecordType.CREATE_TABLE, txnId: 0, tableName: 't', key: '' });
|
||||
expect(await store.exists()).toBe(true);
|
||||
|
||||
await wal.checkpoint();
|
||||
expect(await store.exists()).toBe(false);
|
||||
});
|
||||
|
||||
it('getLSN 跟踪序列号', () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
expect(wal.getLSN()).toBe(0);
|
||||
wal.append({ type: WALRecordType.BEGIN, txnId: 10, tableName: '', key: '' });
|
||||
expect(wal.getLSN()).toBe(1);
|
||||
wal.append({ type: WALRecordType.COMMIT, txnId: 10, tableName: '', key: '' });
|
||||
expect(wal.getLSN()).toBe(2);
|
||||
});
|
||||
|
||||
it('isEnabled 反映配置', () => {
|
||||
expect(new WAL(new MockWALStore(), true).isEnabled()).toBe(true);
|
||||
expect(new WAL(new MockWALStore(), false).isEnabled()).toBe(false);
|
||||
});
|
||||
|
||||
it('多种记录类型编解码', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
wal.append({ type: WALRecordType.BEGIN, txnId: 100, tableName: '', key: '' });
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 100, tableName: 'users', key: '1', data: { x: 'hello' } });
|
||||
wal.append({ type: WALRecordType.UPDATE, txnId: 100, tableName: 'users', key: '1', data: { x: 'world' } });
|
||||
wal.append({ type: WALRecordType.COMMIT, txnId: 100, tableName: '', key: '' });
|
||||
|
||||
const records: WALRecord[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(4);
|
||||
expect(records[0].type).toBe(WALRecordType.BEGIN);
|
||||
expect(records[1].type).toBe(WALRecordType.INSERT);
|
||||
expect(records[2].type).toBe(WALRecordType.UPDATE);
|
||||
expect(records[3].type).toBe(WALRecordType.COMMIT);
|
||||
});
|
||||
|
||||
it('getBufferedCount 返回缓冲数', () => {
|
||||
const wal = new WAL(new MockWALStore(), true, 'batch');
|
||||
expect(wal.getBufferedCount()).toBe(0);
|
||||
wal.append({ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: 'k' });
|
||||
expect(wal.getBufferedCount()).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// Checkpoint 测试（使用安全 Mock，避免 null 引用导致 CI 卡死）
|
||||
// ===================================================================
|
||||
describe('AriaEngine — CheckpointManager', () => {
|
||||
class MockFlushable implements Flushable { flushed = false; async flushAll() { this.flushed = true; } }
|
||||
class MockLSM { flushed = false; async flush() { this.flushed = true; } }
|
||||
class MockWAL { checkpointed = false; async checkpoint() { this.checkpointed = true; } async flush() {} }
|
||||
|
||||
it('tick 未达间隔不触发 checkpoint', async () => {
|
||||
const lsm = new MockLSM();
|
||||
const wal = new MockWAL();
|
||||
const flushable = new MockFlushable();
|
||||
const cm = new CheckpointManager(lsm as any, wal as any, flushable, 100);
|
||||
await cm.tick();
|
||||
await cm.tick();
|
||||
expect(cm.getOpCount()).toBe(2);
|
||||
expect(flushable.flushed).toBe(false);
|
||||
expect(lsm.flushed).toBe(false);
|
||||
});
|
||||
|
||||
it('setInterval 修改间隔后 tick 触发 checkpoint', async () => {
|
||||
const lsm = new MockLSM();
|
||||
const wal = new MockWAL();
|
||||
const cm = new CheckpointManager(lsm as any, wal as any, null, 1000);
|
||||
cm.setInterval(2);
|
||||
await cm.tick();
|
||||
await cm.tick();
|
||||
expect(cm.getOpCount()).toBe(0); // reset after checkpoint
|
||||
expect(lsm.flushed).toBe(true);
|
||||
expect(wal.checkpointed).toBe(true);
|
||||
});
|
||||
|
||||
it('forceCheckpoint 强制触发', async () => {
|
||||
const lsm = new MockLSM();
|
||||
const wal = new MockWAL();
|
||||
const cm = new CheckpointManager(lsm as any, wal as any, null, 100);
|
||||
await cm.forceCheckpoint();
|
||||
expect(cm.getOpCount()).toBe(0);
|
||||
expect(lsm.flushed).toBe(true);
|
||||
expect(wal.checkpointed).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// MVCC 测试
|
||||
// ===================================================================
|
||||
describe('AriaEngine — MVCC', () => {
|
||||
let mvcc: MVCCManager;
|
||||
|
||||
beforeEach(() => { mvcc = new MVCCManager(); });
|
||||
|
||||
it('beginTransaction 分配唯一 ID', () => {
|
||||
const id1 = mvcc.beginTransaction();
|
||||
const id2 = mvcc.beginTransaction();
|
||||
expect(id1).not.toBe(id2);
|
||||
expect(mvcc.isActive(id1)).toBe(true);
|
||||
expect(mvcc.isActive(id2)).toBe(true);
|
||||
});
|
||||
|
||||
it('commit 后 isActive 返回 false', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.commitTransaction(txnId);
|
||||
expect(mvcc.isActive(txnId)).toBe(false);
|
||||
});
|
||||
|
||||
it('rollback 后 isActive 返回 false', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.rollbackTransaction(txnId);
|
||||
expect(mvcc.isActive(txnId)).toBe(false);
|
||||
});
|
||||
|
||||
it('writeVersion + readVersion 往返', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice', age: 30 }, txnId);
|
||||
const val = mvcc.readVersion('users', '1', txnId);
|
||||
expect(val).not.toBeNull();
|
||||
expect(val!.name).toBe('Alice');
|
||||
});
|
||||
|
||||
it('未提交版本对其他事务不可见', () => {
|
||||
const txn1 = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txn1);
|
||||
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txn2);
|
||||
expect(val).toBeNull(); // txn1's write not yet committed
|
||||
});
|
||||
|
||||
it('commit 后新事务可见', () => {
|
||||
const txn1 = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txn1);
|
||||
mvcc.commitTransaction(txn1);
|
||||
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txn2);
|
||||
expect(val).not.toBeNull();
|
||||
expect(val!.name).toBe('Alice');
|
||||
});
|
||||
|
||||
it('rollback 移除写入的版本', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Temp' }, txnId);
|
||||
mvcc.rollbackTransaction(txnId);
|
||||
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
expect(mvcc.readVersion('users', '1', txn2)).toBeNull();
|
||||
});
|
||||
|
||||
it('deleteVersion 创建墓碑', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
// delete
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
mvcc.deleteVersion('users', '1', txn2);
|
||||
mvcc.commitTransaction(txn2);
|
||||
// 删除后读取
|
||||
const txn3 = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txn3);
|
||||
expect(val).not.toBeNull();
|
||||
expect((val! as any).__mvcc_tombstone).toBe(true);
|
||||
});
|
||||
|
||||
it('getLatestCommittedVersions 返回最新已提交', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'Alice' }, txnId);
|
||||
mvcc.writeVersion('users', '2', { name: 'Bob' }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
|
||||
const result = mvcc.getLatestCommittedVersions('users');
|
||||
expect(result['1'].name).toBe('Alice');
|
||||
expect(result['2'].name).toBe('Bob');
|
||||
});
|
||||
|
||||
it('clearTable 清理指定表', () => {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { name: 'A' }, txnId);
|
||||
mvcc.writeVersion('users', '2', { name: 'B' }, txnId);
|
||||
mvcc.writeVersion('orders', 'o1', { amt: 100 }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
|
||||
mvcc.clearTable('users');
|
||||
const txn2 = mvcc.beginTransaction();
|
||||
expect(mvcc.readVersion('users', '1', txn2)).toBeNull();
|
||||
expect(mvcc.readVersion('users', '2', txn2)).toBeNull();
|
||||
expect(mvcc.readVersion('orders', 'o1', txn2)).not.toBeNull();
|
||||
});
|
||||
|
||||
it('getActiveTxnCount 返回活跃事务数', () => {
|
||||
expect(mvcc.getActiveTxnCount()).toBe(0);
|
||||
mvcc.beginTransaction();
|
||||
mvcc.beginTransaction();
|
||||
expect(mvcc.getActiveTxnCount()).toBe(2);
|
||||
});
|
||||
|
||||
it('gc 清理过旧版本', () => {
|
||||
// 创建很多版本后 gc
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const txnId = mvcc.beginTransaction();
|
||||
mvcc.writeVersion('users', '1', { ver: i }, txnId);
|
||||
mvcc.commitTransaction(txnId);
|
||||
}
|
||||
mvcc.gc(100);
|
||||
// gc 后应可继续操作
|
||||
const txnId = mvcc.beginTransaction();
|
||||
const val = mvcc.readVersion('users', '1', txnId);
|
||||
expect(val).not.toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
+932
-932
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,200 @@
|
||||
/**
|
||||
* React 集成 hooks 测试（v0.2.6 补强）
|
||||
* @module tests/integrations/react
|
||||
*
|
||||
* 项目零运行时依赖（react 为 peer dependency），
|
||||
* 使用最小 React mock 验证 hooks 的真实逻辑：
|
||||
* - mount 时执行查询并更新状态
|
||||
* - 错误路径
|
||||
* - refresh 重新执行
|
||||
* - 表名校验（SQL 注入防护）
|
||||
*/
|
||||
import { MetonaSqlark } from '../../src/core';
|
||||
|
||||
// ---- 最小 React mock（自包含：jest.mock 工厂不能引用外部变量） ----
|
||||
jest.mock('react', () => {
|
||||
const stateStore: any[] = [];
|
||||
const registeredEffects = new Set<number>();
|
||||
const effectQueue: Array<() => void | Promise<void>> = [];
|
||||
let cursor = 0;
|
||||
return {
|
||||
useState: (init: any) => {
|
||||
const idx = cursor++;
|
||||
if (!(idx in stateStore)) stateStore[idx] = init;
|
||||
return [
|
||||
stateStore[idx],
|
||||
(v: any) => {
|
||||
stateStore[idx] = typeof v === 'function' ? v(stateStore[idx]) : v;
|
||||
},
|
||||
];
|
||||
},
|
||||
// 按 hook 调用位置去重：同一位置的 effect 只在首次 render 注册
|
||||
useEffect: (fn: any, _deps: any[]) => {
|
||||
const idx = cursor;
|
||||
if (!registeredEffects.has(idx)) {
|
||||
registeredEffects.add(idx);
|
||||
effectQueue.push(fn);
|
||||
}
|
||||
},
|
||||
useCallback: (fn: any) => fn,
|
||||
useRef: (init: any) => ({ current: init }),
|
||||
/** 模拟一次组件渲染：cursor 归零后执行 hook 函数 */
|
||||
__mockRender: (fn: () => any): any => {
|
||||
cursor = 0;
|
||||
return fn();
|
||||
},
|
||||
__mockEffects: effectQueue,
|
||||
__mockReset: () => {
|
||||
stateStore.length = 0;
|
||||
cursor = 0;
|
||||
effectQueue.length = 0;
|
||||
registeredEffects.clear();
|
||||
},
|
||||
};
|
||||
}, { virtual: true });
|
||||
|
||||
import { useQuery, useTable, useDatabase } from '../../src/integrations/react';
|
||||
|
||||
/** mock 模块内的 effect 队列与渲染控制（自包含作用域） */
|
||||
const reactMock = jest.requireMock('react') as {
|
||||
__mockEffects: Array<() => void | Promise<void>>;
|
||||
__mockRender: (fn: () => any) => any;
|
||||
__mockReset: () => void;
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
reactMock.__mockReset();
|
||||
});
|
||||
|
||||
/** 模拟组件挂载：执行 useEffect 中注册的回调 */
|
||||
async function flushEffects(): Promise<void> {
|
||||
const fns = reactMock.__mockEffects.splice(0);
|
||||
for (const fn of fns) {
|
||||
await fn();
|
||||
}
|
||||
}
|
||||
|
||||
/** 模拟组件渲染（cursor 归零，读取最新状态） */
|
||||
function render<T>(fn: () => T): T {
|
||||
return reactMock.__mockRender(fn);
|
||||
}
|
||||
|
||||
describe('useQuery', () => {
|
||||
test('mount 时执行 SQL 查询并更新 data/loading', async () => {
|
||||
const db = { query: jest.fn().mockResolvedValue([{ id: '1', name: 'Alice' }]) } as any;
|
||||
const mount = () => useQuery(db, 'SELECT * FROM users');
|
||||
const hook = render(mount);
|
||||
|
||||
expect(hook.loading).toBe(true);
|
||||
expect(db.query).not.toHaveBeenCalled();
|
||||
|
||||
await flushEffects();
|
||||
const after = render(mount); // 模拟重渲染读取最新状态
|
||||
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM users');
|
||||
expect(after.loading).toBe(false);
|
||||
expect(after.data).toEqual([{ id: '1', name: 'Alice' }]);
|
||||
expect(after.error).toBeNull();
|
||||
});
|
||||
|
||||
test('查询失败时设置 error 且 data 保持空', async () => {
|
||||
const db = { query: jest.fn().mockRejectedValue(new Error('query boom')) } as any;
|
||||
const mount = () => useQuery(db, 'SELECT * FROM users');
|
||||
render(mount);
|
||||
|
||||
await flushEffects();
|
||||
const after = render(mount);
|
||||
|
||||
expect(after.error).toBeInstanceOf(Error);
|
||||
expect((after.error as Error).message).toBe('query boom');
|
||||
expect(after.data).toEqual([]);
|
||||
expect(after.loading).toBe(false);
|
||||
});
|
||||
|
||||
test('refresh 可重新执行查询', async () => {
|
||||
let call = 0;
|
||||
const db = { query: jest.fn().mockImplementation(async () => [{ n: ++call }]) } as any;
|
||||
const mount = () => useQuery(db, 'SELECT * FROM users');
|
||||
const hook = render(mount);
|
||||
|
||||
await flushEffects();
|
||||
expect(render(mount).data).toEqual([{ n: 1 }]);
|
||||
|
||||
await hook.refresh();
|
||||
await flushEffects();
|
||||
expect(db.query).toHaveBeenCalledTimes(2);
|
||||
expect(render(mount).data).toEqual([{ n: 2 }]);
|
||||
});
|
||||
|
||||
test('不同 SQL 使用各自独立的 hook 状态', async () => {
|
||||
const db = { query: jest.fn().mockResolvedValue([]) } as any;
|
||||
const hook1 = useQuery(db, 'SELECT * FROM a');
|
||||
const hook2 = useQuery(db, 'SELECT * FROM b');
|
||||
await flushEffects();
|
||||
|
||||
expect(hook1).not.toBe(hook2);
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM a');
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM b');
|
||||
});
|
||||
});
|
||||
|
||||
describe('useTable', () => {
|
||||
test('合法表名执行全表查询', async () => {
|
||||
const db = { query: jest.fn().mockResolvedValue([{ id: 1 }]) } as any;
|
||||
const mount = () => useTable(db, 'users');
|
||||
render(mount);
|
||||
|
||||
await flushEffects();
|
||||
const after = render(mount);
|
||||
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM users');
|
||||
expect(after.data).toEqual([{ id: 1 }]);
|
||||
expect(after.loading).toBe(false);
|
||||
});
|
||||
|
||||
test('非法表名抛出校验错误（SQL 注入防护）', () => {
|
||||
const db = { query: jest.fn() } as any;
|
||||
expect(() => useTable(db, 'users; DROP TABLE orders')).toThrow(/Invalid table name/);
|
||||
expect(() => useTable(db, "users' OR '1'='1")).toThrow(/Invalid table name/);
|
||||
expect(() => useTable(db, '1users')).toThrow(/Invalid table name/);
|
||||
expect(db.query).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('useDatabase', () => {
|
||||
test('创建数据库实例并初始化', async () => {
|
||||
const initSpy = jest.spyOn(MetonaSqlark.prototype, 'init').mockResolvedValue();
|
||||
const closeSpy = jest.spyOn(MetonaSqlark.prototype, 'close').mockResolvedValue();
|
||||
|
||||
const mount = () => useDatabase({ name: 'hook-test' });
|
||||
const hook = render(mount);
|
||||
expect(hook.ready).toBe(false);
|
||||
|
||||
await flushEffects();
|
||||
const after = render(mount);
|
||||
|
||||
expect(initSpy).toHaveBeenCalled();
|
||||
expect(after.db).toBeInstanceOf(MetonaSqlark);
|
||||
expect(after.ready).toBe(true);
|
||||
expect(after.error).toBeNull();
|
||||
|
||||
initSpy.mockRestore();
|
||||
closeSpy.mockRestore();
|
||||
});
|
||||
|
||||
test('初始化失败时设置 error', async () => {
|
||||
const initSpy = jest.spyOn(MetonaSqlark.prototype, 'init').mockRejectedValue(new Error('init fail'));
|
||||
|
||||
const mount = () => useDatabase({ name: 'hook-fail' });
|
||||
render(mount);
|
||||
await flushEffects();
|
||||
const after = render(mount);
|
||||
|
||||
expect(after.db).toBeNull();
|
||||
expect(after.ready).toBe(false);
|
||||
expect(after.error).toBeInstanceOf(Error);
|
||||
expect((after.error as Error).message).toBe('init fail');
|
||||
|
||||
initSpy.mockRestore();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,167 @@
|
||||
/**
|
||||
* Vue 集成 composables 测试（v0.2.6 补强）
|
||||
* @module tests/integrations/vue
|
||||
*
|
||||
* 项目零运行时依赖（vue 为 peer dependency），
|
||||
* 使用最小 Vue mock 验证 composables 的真实逻辑：
|
||||
* - onMounted 时执行查询
|
||||
* - watch sql/deps 变化重新执行
|
||||
* - refresh 手动刷新
|
||||
* - 表名校验（SQL 注入防护）
|
||||
*/
|
||||
import 'fake-indexeddb/auto';
|
||||
|
||||
// ---- 最小 Vue mock（自包含：jest.mock 工厂不能引用外部变量） ----
|
||||
jest.mock('vue', () => {
|
||||
const mountQueue: Array<() => void | Promise<void>> = [];
|
||||
const watchList: Array<{ sources: any[]; cb: () => void | Promise<void> }> = [];
|
||||
return {
|
||||
ref: (init: any) => {
|
||||
const box: { value: any } = { value: init };
|
||||
return box;
|
||||
},
|
||||
watch: (sources: any[], cb: any) => {
|
||||
watchList.push({ sources, cb });
|
||||
},
|
||||
onMounted: (fn: any) => {
|
||||
mountQueue.push(fn);
|
||||
},
|
||||
__mockMounted: mountQueue,
|
||||
__mockWatch: watchList,
|
||||
__mockReset: () => {
|
||||
mountQueue.length = 0;
|
||||
watchList.length = 0;
|
||||
},
|
||||
};
|
||||
}, { virtual: true });
|
||||
|
||||
import { useSqlarkQuery, useSqlarkTable, useSqlarkDatabase } from '../../src/integrations/vue';
|
||||
|
||||
/** mock 模块内的挂载/监听队列（自包含作用域） */
|
||||
const vueMock = jest.requireMock('vue') as {
|
||||
__mockMounted: Array<() => void | Promise<void>>;
|
||||
__mockWatch: Array<{ sources: any[]; cb: () => void | Promise<void> }>;
|
||||
__mockReset: () => void;
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vueMock.__mockReset();
|
||||
});
|
||||
|
||||
/** 模拟组件挂载：执行 onMounted 注册的回调 */
|
||||
async function flushMounted(): Promise<void> {
|
||||
const fns = vueMock.__mockMounted.splice(0);
|
||||
for (const fn of fns) {
|
||||
await fn();
|
||||
}
|
||||
}
|
||||
|
||||
/** 触发 watch 回调 */
|
||||
async function flushWatch(): Promise<void> {
|
||||
const pairs = vueMock.__mockWatch.splice(0);
|
||||
for (const { cb } of pairs) {
|
||||
await cb();
|
||||
}
|
||||
}
|
||||
|
||||
/** 构造最小响应式引用（与 vue.mock 的 ref 等价） */
|
||||
function makeRef<T>(init: T): { value: T } {
|
||||
return { value: init };
|
||||
}
|
||||
|
||||
describe('useSqlarkQuery', () => {
|
||||
test('onMounted 时执行 SQL 查询并更新响应式 data', async () => {
|
||||
const db = { query: jest.fn().mockResolvedValue([{ id: '1' }]) } as any;
|
||||
const hook = useSqlarkQuery(db, 'SELECT * FROM users');
|
||||
|
||||
expect(hook.loading.value).toBe(true);
|
||||
expect(db.query).not.toHaveBeenCalled();
|
||||
|
||||
await flushMounted();
|
||||
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM users');
|
||||
expect(hook.loading.value).toBe(false);
|
||||
expect(hook.data.value).toEqual([{ id: '1' }]);
|
||||
expect(hook.error.value).toBeNull();
|
||||
});
|
||||
|
||||
test('查询失败时设置 error', async () => {
|
||||
const db = { query: jest.fn().mockRejectedValue(new Error('vue boom')) } as any;
|
||||
const hook = useSqlarkQuery(db, 'SELECT * FROM users');
|
||||
|
||||
await flushMounted();
|
||||
|
||||
expect(hook.error.value).toBeInstanceOf(Error);
|
||||
expect((hook.error.value as Error).message).toBe('vue boom');
|
||||
expect(hook.data.value).toEqual([]);
|
||||
});
|
||||
|
||||
test('refresh 重新执行查询', async () => {
|
||||
let call = 0;
|
||||
const db = { query: jest.fn().mockImplementation(async () => [{ n: ++call }]) } as any;
|
||||
const hook = useSqlarkQuery(db, 'SELECT * FROM users');
|
||||
|
||||
await flushMounted();
|
||||
expect(hook.data.value).toEqual([{ n: 1 }]);
|
||||
|
||||
await hook.refresh();
|
||||
expect(db.query).toHaveBeenCalledTimes(2);
|
||||
expect(hook.data.value).toEqual([{ n: 2 }]);
|
||||
});
|
||||
|
||||
test('watch 注册在 sql 与 deps 上', async () => {
|
||||
const db = { query: jest.fn().mockResolvedValue([]) } as any;
|
||||
const sqlRef = makeRef('SELECT * FROM users');
|
||||
useSqlarkQuery(db, sqlRef.value, [sqlRef]);
|
||||
|
||||
expect(vueMock.__mockWatch).toHaveLength(1);
|
||||
expect(vueMock.__mockWatch[0].sources).toHaveLength(2); // [() => sql, ...deps]
|
||||
|
||||
// 模拟依赖变化触发 watch
|
||||
sqlRef.value = 'SELECT * FROM orders';
|
||||
await flushWatch();
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM users');
|
||||
});
|
||||
});
|
||||
|
||||
describe('useSqlarkTable', () => {
|
||||
test('合法表名执行全表查询', async () => {
|
||||
const db = { query: jest.fn().mockResolvedValue([{ id: 1 }]) } as any;
|
||||
const hook = useSqlarkTable(db, 'users');
|
||||
|
||||
await flushMounted();
|
||||
|
||||
expect(db.query).toHaveBeenCalledWith('SELECT * FROM users');
|
||||
expect(hook.data.value).toEqual([{ id: 1 }]);
|
||||
});
|
||||
|
||||
test('非法表名抛出校验错误（SQL 注入防护）', () => {
|
||||
const db = { query: jest.fn() } as any;
|
||||
expect(() => useSqlarkTable(db, 'users; DELETE FROM orders')).toThrow(/Invalid table name/);
|
||||
expect(() => useSqlarkTable(db, 'users--')).toThrow(/Invalid table name/);
|
||||
expect(db.query).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('useSqlarkDatabase', () => {
|
||||
test('onMounted 创建并初始化数据库实例', async () => {
|
||||
const hook = useSqlarkDatabase({ name: 'vue-hook' });
|
||||
expect(hook.ready.value).toBe(false);
|
||||
|
||||
await flushMounted();
|
||||
|
||||
expect(hook.db.value).not.toBeNull();
|
||||
expect(hook.ready.value).toBe(true);
|
||||
expect(hook.error.value).toBeNull();
|
||||
});
|
||||
|
||||
test('初始化失败时设置 error', async () => {
|
||||
// 使用非法配置触发初始化错误（mode 未知）
|
||||
const hook = useSqlarkDatabase({ name: 'vue-fail', mode: 'unknown-mode' as any });
|
||||
await flushMounted();
|
||||
|
||||
expect(hook.db.value).toBeNull();
|
||||
expect(hook.ready.value).toBe(false);
|
||||
expect(hook.error.value).toBeInstanceOf(Error);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,372 @@
|
||||
/**
|
||||
* v0.3.0 SQL 功能扩展测试
|
||||
* @module tests/sql-ext
|
||||
*
|
||||
* 覆盖：多语句 / 事务语句 / INSERT...SELECT / UNION / CREATE INDEX / EXISTS
|
||||
*/
|
||||
import 'fake-indexeddb/auto';
|
||||
import { MetonaSqlark } from '../src/core';
|
||||
import '../src/connection-manager';
|
||||
import { parse, parseAll } from '../src/sql/parser';
|
||||
|
||||
async function createDb(mode: 'memory' | 'disk' | 'aria' = 'memory') {
|
||||
const db = new MetonaSqlark({ name: `sql-ext-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'indexeddb' });
|
||||
await db.init();
|
||||
await db.defineTable('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number' },
|
||||
city: { type: 'string' },
|
||||
});
|
||||
await db.defineTable('orders', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
user_id: { type: 'string' },
|
||||
amount: { type: 'number' },
|
||||
});
|
||||
await db.query(`INSERT INTO users VALUES ('u1', 'Alice', 30, 'Beijing')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u2', 'Bob', 25, 'Shanghai')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u3', 'Carol', 35, 'Beijing')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u4', 'Dave', 28, 'Shenzhen')`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o1', 'u1', 100)`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o2', 'u1', 200)`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o3', 'u2', 50)`);
|
||||
return db;
|
||||
}
|
||||
|
||||
// ===================================================================
|
||||
// 多语句 parseAll
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.0] 多语句支持', () => {
|
||||
test('parseAll 解析分号分隔的多条语句', () => {
|
||||
const stmts = parseAll('SELECT * FROM a; INSERT INTO b VALUES (1); DELETE FROM c WHERE id = 1');
|
||||
expect(stmts).toHaveLength(3);
|
||||
expect(stmts[0].type).toBe('SELECT');
|
||||
expect(stmts[1].type).toBe('INSERT');
|
||||
expect(stmts[2].type).toBe('DELETE');
|
||||
});
|
||||
|
||||
test('parseAll 忽略多余分号与尾部空语句', () => {
|
||||
const stmts = parseAll(';;SELECT * FROM a;;;');
|
||||
expect(stmts).toHaveLength(1);
|
||||
expect(stmts[0].type).toBe('SELECT');
|
||||
});
|
||||
|
||||
test('parseAll 支持事务语句', () => {
|
||||
const stmts = parseAll('BEGIN; INSERT INTO a VALUES (1); COMMIT');
|
||||
expect(stmts.map((s) => s.type)).toEqual(['BEGIN', 'INSERT', 'COMMIT']);
|
||||
});
|
||||
|
||||
test('parse 保持单语句兼容', () => {
|
||||
expect(parse('SELECT * FROM a').type).toBe('SELECT');
|
||||
});
|
||||
|
||||
test('parseAll 语句间缺分号报错', () => {
|
||||
expect(() => parseAll('SELECT * FROM a SELECT * FROM b')).toThrow(/Expected ';'/);
|
||||
});
|
||||
|
||||
test('core.query 顺序执行多语句', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('CREATE TABLE logs (id STRING PRIMARY KEY, msg STRING); INSERT INTO logs VALUES (\'l1\', \'hello\'); INSERT INTO logs VALUES (\'l2\', \'world\')');
|
||||
const rows = await db.query('SELECT * FROM logs') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// 事务语句 BEGIN / COMMIT / ROLLBACK
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.0] 事务语句', () => {
|
||||
test('BEGIN + INSERT + COMMIT 持久化', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('BEGIN');
|
||||
await db.query(`INSERT INTO users VALUES ('u5', 'Eve', 22, 'Beijing')`);
|
||||
const visible = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||
expect(visible).toHaveLength(5); // 事务内可见
|
||||
await db.query('COMMIT');
|
||||
const after = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||
expect(after).toHaveLength(5);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('BEGIN + INSERT + ROLLBACK 回滚', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('BEGIN');
|
||||
await db.query(`INSERT INTO users VALUES ('u5', 'Eve', 22, 'Beijing')`);
|
||||
await db.query('ROLLBACK');
|
||||
const after = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||
expect(after).toHaveLength(4);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('BEGIN 嵌套报错', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('BEGIN');
|
||||
await expect(db.query('BEGIN')).rejects.toThrow(/TX_ACTIVE|Transaction already/);
|
||||
await db.query('ROLLBACK');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('无事务时 COMMIT 报错', async () => {
|
||||
const db = await createDb();
|
||||
await expect(db.query('COMMIT')).rejects.toThrow(/TX_NONE|No active transaction/);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('事务语句在 Aria 引擎上可用', async () => {
|
||||
const db = await createDb('aria');
|
||||
await db.query('BEGIN');
|
||||
await db.query(`INSERT INTO users VALUES ('u5', 'Eve', 22, 'Beijing')`);
|
||||
await db.query('ROLLBACK');
|
||||
const after = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||
expect(after).toHaveLength(4);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// INSERT INTO ... SELECT
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.0] INSERT INTO ... SELECT', () => {
|
||||
test('INSERT SELECT 全列复制', async () => {
|
||||
const db = await createDb();
|
||||
await db.defineTable('users_backup', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number' },
|
||||
city: { type: 'string' },
|
||||
});
|
||||
await db.query('INSERT INTO users_backup SELECT * FROM users');
|
||||
const rows = await db.query('SELECT * FROM users_backup') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(4);
|
||||
expect(rows[0].name).toBe('Alice');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('INSERT SELECT 带 WHERE 过滤', async () => {
|
||||
const db = await createDb();
|
||||
await db.defineTable('beijing_users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number' },
|
||||
city: { type: 'string' },
|
||||
});
|
||||
await db.query(`INSERT INTO beijing_users SELECT * FROM users WHERE city = 'Beijing'`);
|
||||
const rows = await db.query('SELECT * FROM beijing_users') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('INSERT SELECT 指定列映射', async () => {
|
||||
const db = await createDb();
|
||||
await db.defineTable('names', { id: { type: 'string', primaryKey: true }, n: { type: 'string' } });
|
||||
await db.query('INSERT INTO names (id, n) SELECT id, name FROM users');
|
||||
const rows = await db.query('SELECT * FROM names') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(4);
|
||||
expect(rows[0].n).toBe('Alice');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('INSERT SELECT 语法解析', () => {
|
||||
const stmt = parse('INSERT INTO a (x, y) SELECT p, q FROM b WHERE r > 1') as any;
|
||||
expect(stmt.type).toBe('INSERT');
|
||||
expect(stmt.select).toBeDefined();
|
||||
expect(stmt.select.type).toBe('SELECT');
|
||||
expect(stmt.columns).toEqual(['x', 'y']);
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// UNION / UNION ALL
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.0] UNION / UNION ALL', () => {
|
||||
test('UNION 去重合并', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(`SELECT name FROM users WHERE city = 'Beijing' UNION SELECT name FROM users WHERE age < 30`) as Record<string, unknown>[];
|
||||
// Beijing: Alice, Carol（Alice 同时 age30 不重复）；age<30: Bob, Dave
|
||||
expect(rows).toHaveLength(4);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('UNION ALL 不去重', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(`SELECT city FROM users WHERE city = 'Beijing' UNION ALL SELECT city FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(4); // 2 + 2
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('UNION 对相同行去重', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(`SELECT city FROM users WHERE city = 'Beijing' UNION SELECT city FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
// 左右各 2 行同值 → 去重后 1 行
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].city).toBe('Beijing');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('UNION 链式（三表合并）', async () => {
|
||||
const db = await createDb();
|
||||
await db.query(`INSERT INTO users VALUES ('u5', 'Eve', 22, 'Beijing')`);
|
||||
const rows = await db.query(`SELECT city FROM users WHERE city = 'Beijing' UNION SELECT city FROM users WHERE city = 'Shanghai' UNION SELECT city FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
// Beijing(3) + Shanghai(1) + Beijing(3 重复去重) → 2 个城市
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('UNION 语法解析', () => {
|
||||
const stmt = parse('SELECT a FROM t1 UNION SELECT b FROM t2') as any;
|
||||
expect(stmt.type).toBe('SELECT_UNION');
|
||||
expect(stmt.all).toBeUndefined();
|
||||
const stmtAll = parse('SELECT a FROM t1 UNION ALL SELECT b FROM t2') as any;
|
||||
expect(stmtAll.type).toBe('SELECT_UNION');
|
||||
expect(stmtAll.all).toBe(true);
|
||||
});
|
||||
|
||||
test('UNION 在 Aria 引擎上可用', async () => {
|
||||
const db = await createDb('aria');
|
||||
const rows = await db.query(`SELECT city FROM users WHERE city = 'Beijing' UNION SELECT city FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(1);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// CREATE INDEX / DROP INDEX
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.0] CREATE INDEX / DROP INDEX', () => {
|
||||
test('CREATE INDEX 后索引查询可用（Memory）', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)');
|
||||
const rows = await db.query(`SELECT * FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('CREATE INDEX 后索引查询可用（Aria）', async () => {
|
||||
const db = await createDb('aria');
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)');
|
||||
const rows = await db.query(`SELECT * FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('CREATE INDEX 对已有数据立即生效（索引填充）', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('INSERT INTO users VALUES (\'u5\', \'Eve\', 22, \'Beijing\')');
|
||||
await db.query('CREATE INDEX idx_users_age ON users (age)');
|
||||
const rows = await db.query(`SELECT * FROM users WHERE age = 22`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].id).toBe('u5');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('DROP INDEX 后查询回退全表扫描', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)');
|
||||
await db.query('DROP INDEX idx_users_city ON users (city)');
|
||||
const rows = await db.query(`SELECT * FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('CREATE INDEX 重复创建幂等', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)');
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)'); // 不抛错
|
||||
const rows = await db.query(`SELECT * FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('CREATE INDEX 不存在列报错', async () => {
|
||||
const db = await createDb();
|
||||
const err = await db.query('CREATE INDEX idx_bad ON users (nonexistent)').catch((e: any) => e);
|
||||
expect(err).toBeDefined();
|
||||
expect(err.code).toBe('COLUMN_NOT_FOUND');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('CREATE INDEX 后插入新数据索引同步更新', async () => {
|
||||
const db = await createDb();
|
||||
await db.query('CREATE INDEX idx_users_city ON users (city)');
|
||||
await db.query('INSERT INTO users VALUES (\'u5\', \'Eve\', 22, \'Beijing\')');
|
||||
const rows = await db.query(`SELECT * FROM users WHERE city = 'Beijing'`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(3);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('DROP INDEX Aria 主键索引受保护', async () => {
|
||||
const db = await createDb('aria');
|
||||
await expect(db.query('DROP INDEX pk ON users (id)')).rejects.toThrow(/Cannot drop primary key/);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('CREATE UNIQUE INDEX 语法解析', () => {
|
||||
const stmt = parse('CREATE UNIQUE INDEX idx_u ON users (email)') as any;
|
||||
expect(stmt.type).toBe('CREATE_INDEX');
|
||||
expect(stmt.unique).toBe(true);
|
||||
expect(stmt.column).toBe('email');
|
||||
});
|
||||
|
||||
test('CREATE INDEX / DROP INDEX 语法解析', () => {
|
||||
const stmt = parse('CREATE INDEX idx_c ON users (city)') as any;
|
||||
expect(stmt).toMatchObject({ type: 'CREATE_INDEX', name: 'idx_c', table: 'users', column: 'city' });
|
||||
const drop = parse('DROP INDEX idx_c ON users (city)') as any;
|
||||
expect(drop).toMatchObject({ type: 'DROP_INDEX', name: 'idx_c', table: 'users', column: 'city' });
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// EXISTS / NOT EXISTS
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.0] EXISTS / NOT EXISTS', () => {
|
||||
test('EXISTS 子查询为真返回行', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(`SELECT * FROM users u WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)`) as Record<string, unknown>[];
|
||||
// 有订单的用户：u1, u2
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows.map((r) => r.id).sort()).toEqual(['u1', 'u2']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('NOT EXISTS 返回无关联行', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(`SELECT * FROM users u WHERE NOT EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)`) as Record<string, unknown>[];
|
||||
// 无订单用户：u3, u4
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows.map((r) => r.id).sort()).toEqual(['u3', 'u4']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('EXISTS 与 AND 组合', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(`SELECT * FROM users u WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id AND o.amount > 150) AND u.city = 'Beijing'`) as Record<string, unknown>[];
|
||||
// o2 金额 200 > 150 属于 u1（Beijing）
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].id).toBe('u1');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('EXISTS 语法解析', () => {
|
||||
const stmt = parse(`SELECT * FROM users WHERE EXISTS (SELECT 1 FROM orders)`) as any;
|
||||
expect(stmt.where.$exists).toBeDefined();
|
||||
expect(stmt.where.$exists.$subquery.type).toBe('SELECT');
|
||||
|
||||
const notStmt = parse(`SELECT * FROM users WHERE NOT EXISTS (SELECT 1 FROM orders)`) as any;
|
||||
expect(notStmt.where.$exists).toBeDefined();
|
||||
expect(notStmt.where.$exists.$negate).toBe(true);
|
||||
});
|
||||
|
||||
test('EXISTS 在 Aria 引擎上可用', async () => {
|
||||
const db = await createDb('aria');
|
||||
const rows = await db.query(`SELECT * FROM users u WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)`) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,313 @@
|
||||
/**
|
||||
* v0.3.1 功能测试
|
||||
* @module tests/sql-ext2
|
||||
*
|
||||
* 覆盖：CASE WHEN 表达式 / JOIN + 关联子查询 / WAL 批量组提交
|
||||
*/
|
||||
import 'fake-indexeddb/auto';
|
||||
import { MetonaSqlark } from '../src/core';
|
||||
import { parse } from '../src/sql/parser';
|
||||
import { WAL, type WALStore } from '../src/engine/aria/wal/log';
|
||||
import { WALRecordType } from '../src/engine/aria/types';
|
||||
|
||||
async function createDb(mode: 'memory' | 'aria' = 'memory') {
|
||||
const db = new MetonaSqlark({ name: `sql-ext2-${mode}-${Date.now()}-${Math.random()}`, mode, diskEngine: 'indexeddb' });
|
||||
await db.init();
|
||||
await db.defineTable('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number' },
|
||||
city: { type: 'string' },
|
||||
});
|
||||
await db.defineTable('orders', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
user_id: { type: 'string' },
|
||||
amount: { type: 'number' },
|
||||
});
|
||||
await db.query(`INSERT INTO users VALUES ('u1', 'Alice', 30, 'Beijing')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u2', 'Bob', 17, 'Shanghai')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u3', 'Carol', 42, 'Beijing')`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o1', 'u1', 100)`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o2', 'u1', 200)`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o3', 'u2', 50)`);
|
||||
return db;
|
||||
}
|
||||
|
||||
// ===================================================================
|
||||
// CASE WHEN
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.1] CASE WHEN', () => {
|
||||
test('基本 CASE WHEN（单条件 + ELSE）', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name, CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(3);
|
||||
const byName = Object.fromEntries(rows.map((r) => [r.name, r]));
|
||||
expect(byName['Alice'].status).toBe('adult');
|
||||
expect(byName['Bob'].status).toBe('minor');
|
||||
expect(byName['Carol'].status).toBe('adult');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('多 WHEN 分支按顺序匹配', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name, CASE WHEN age < 18 THEN 'teen' WHEN age < 40 THEN 'adult' ELSE 'senior' END AS age_group FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
const byName = Object.fromEntries(rows.map((r) => [r.name, r]));
|
||||
expect(byName['Alice'].age_group).toBe('adult');
|
||||
expect(byName['Bob'].age_group).toBe('teen');
|
||||
expect(byName['Carol'].age_group).toBe('senior');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('THEN 值为列引用', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT CASE WHEN age >= 18 THEN city ELSE 'underage' END AS location FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
const cities = rows.map((r) => r.location);
|
||||
expect(cities).toContain('Beijing');
|
||||
expect(cities).toContain('underage');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('无 ELSE 时返回 null', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name, CASE WHEN age >= 40 THEN 'senior' END AS tag FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
const byName = Object.fromEntries(rows.map((r) => [r.name, r]));
|
||||
expect(byName['Carol'].tag).toBe('senior');
|
||||
expect(byName['Alice'].tag).toBeNull();
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('字面量：数字 / 布尔 / 字符串', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name, CASE WHEN age > 20 THEN 1 ELSE 0 END AS flag, CASE WHEN city = 'Beijing' THEN true ELSE false END AS is_bj FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
const byName = Object.fromEntries(rows.map((r) => [r.name, r]));
|
||||
expect(byName['Alice'].flag).toBe(1);
|
||||
expect(byName['Bob'].flag).toBe(0);
|
||||
expect(byName['Alice'].is_bj).toBe(true);
|
||||
expect(byName['Bob'].is_bj).toBe(false);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('多条件组合（AND/OR）', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name, CASE WHEN age >= 18 AND city = 'Beijing' THEN 'local adult' ELSE 'other' END AS label FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
const byName = Object.fromEntries(rows.map((r) => [r.name, r]));
|
||||
expect(byName['Alice'].label).toBe('local adult');
|
||||
expect(byName['Carol'].label).toBe('local adult');
|
||||
expect(byName['Bob'].label).toBe('other');
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('语法解析：CASE 列被解析为原文', () => {
|
||||
const stmt = parse(`SELECT name, CASE WHEN age > 18 THEN 'x' ELSE 'y' END AS s FROM users`) as any;
|
||||
expect(stmt.columns[1]).toMatch(/^CASE WHEN age > 18 THEN 'x' ELSE 'y' END AS s$/);
|
||||
});
|
||||
|
||||
test('与普通列混合投影', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name, age, CASE WHEN age >= 18 THEN 'ok' ELSE 'no' END AS adult FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows[0].name).toBeDefined();
|
||||
expect(rows[0].age).toBeDefined();
|
||||
expect(rows[0].adult).toBeDefined();
|
||||
expect(Object.keys(rows[0])).toEqual(expect.arrayContaining(['name', 'age', 'adult']));
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('Aria 引擎可用', async () => {
|
||||
const db = await createDb('aria');
|
||||
const rows = await db.query(
|
||||
`SELECT name, CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(3);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// JOIN + 关联子查询
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.1] JOIN + 关联子查询', () => {
|
||||
test('JOIN 结果上执行 EXISTS 关联过滤', async () => {
|
||||
const db = await createDb();
|
||||
// 有高额订单（>150）的用户
|
||||
const rows = await db.query(
|
||||
`SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 150)`,
|
||||
) as Record<string, unknown>[];
|
||||
// 只有 u1 有 200 元订单
|
||||
expect(rows).toHaveLength(2); // JOIN 展开 2 行（u1 有 2 个订单）
|
||||
expect(rows.every((r) => r['u.name'] === 'Alice')).toBe(true);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('JOIN + NOT EXISTS 排除关联行', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT DISTINCT u.name FROM users u LEFT JOIN orders o ON u.id = o.user_id WHERE NOT EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id)`,
|
||||
) as Record<string, unknown>[];
|
||||
// 无订单用户：u3 (Carol)（JOIN 列键带表名前缀）
|
||||
expect(rows.map((r) => r['u.name'])).toEqual(['Carol']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('JOIN + 关联子查询 + 普通条件组合', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id) AND o.amount > 50`,
|
||||
) as Record<string, unknown>[];
|
||||
// 有订单且订单 >50：u1 的 o1(100), o2(200) + u2 的 o3(50 不满足)
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows.every((r) => r['u.name'] === 'Alice')).toBe(true);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('JOIN 子查询中的 $col 引用绑定外层行', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT o.id, o.amount FROM orders o JOIN users u ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.amount > o.amount)`,
|
||||
) as Record<string, unknown>[];
|
||||
// 存在比自身金额更大的订单：o1(100) < o2(200) → o1 满足；o2 无更大；o3(50) < o1/o2 → 满足
|
||||
const ids = rows.map((r) => r['o.id']).sort();
|
||||
expect(ids).toEqual(['o1', 'o3']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('Aria 引擎 JOIN + EXISTS 可用', async () => {
|
||||
const db = await createDb('aria');
|
||||
const rows = await db.query(
|
||||
`SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE EXISTS (SELECT 1 FROM orders o2 WHERE o2.user_id = u.id AND o2.amount > 150)`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(2);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// WAL 批量组提交
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.1] WAL 批量组提交', () => {
|
||||
class MockWALStore implements WALStore {
|
||||
chunks: Uint8Array[] = [];
|
||||
appendCount = 0;
|
||||
async append(data: Uint8Array) { this.chunks.push(data); this.appendCount++; }
|
||||
async readAll(): Promise<Uint8Array> {
|
||||
const total = this.chunks.reduce((s, c) => s + c.byteLength, 0);
|
||||
const combined = new Uint8Array(total);
|
||||
let off = 0;
|
||||
for (const c of this.chunks) { combined.set(c, off); off += c.byteLength; }
|
||||
return combined;
|
||||
}
|
||||
async truncate() { this.chunks = []; this.appendCount = 0; }
|
||||
async exists() { return this.chunks.length > 0; }
|
||||
}
|
||||
|
||||
test('appendBatch 合并为一次底层写入', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
await wal.appendBatch([
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: '1', data: { v: 1 } },
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: '2', data: { v: 2 } },
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: '3', data: { v: 3 } },
|
||||
]);
|
||||
|
||||
expect(store.appendCount).toBe(1); // 3 条记录 1 次写入
|
||||
});
|
||||
|
||||
test('appendBatch 记录可恢复', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
await wal.appendBatch([
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 'users', key: 'a', data: { n: 1 } },
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 'users', key: 'b', data: { n: 2 } },
|
||||
]);
|
||||
|
||||
const records: { tableName: string; key: string }[] = [];
|
||||
await wal.recover((r) => records.push(r));
|
||||
expect(records).toHaveLength(2);
|
||||
expect(records[0].key).toBe('a');
|
||||
expect(records[1].key).toBe('b');
|
||||
});
|
||||
|
||||
test('batch 模式 appendBatch 缓冲后 flush', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'batch');
|
||||
|
||||
await wal.appendBatch([
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: '1', data: {} },
|
||||
{ type: WALRecordType.INSERT, txnId: 0, tableName: 't', key: '2', data: {} },
|
||||
]);
|
||||
expect(store.appendCount).toBe(0); // 缓冲未落盘
|
||||
|
||||
await wal.flush();
|
||||
expect(store.appendCount).toBe(1);
|
||||
});
|
||||
|
||||
test('append 与 appendBatch 共存', async () => {
|
||||
const store = new MockWALStore();
|
||||
const wal = new WAL(store, true, 'full');
|
||||
|
||||
await wal.append({ type: WALRecordType.BEGIN, txnId: 7, tableName: '', key: '' });
|
||||
await wal.appendBatch([
|
||||
{ type: WALRecordType.INSERT, txnId: 7, tableName: 't', key: '1', data: {} },
|
||||
{ type: WALRecordType.INSERT, txnId: 7, tableName: 't', key: '2', data: {} },
|
||||
]);
|
||||
await wal.append({ type: WALRecordType.COMMIT, txnId: 7, tableName: '', key: '' });
|
||||
|
||||
expect(store.appendCount).toBe(3); // BEGIN + 批量(1) + COMMIT
|
||||
const records: number[] = [];
|
||||
await wal.recover((r) => records.push(r.type));
|
||||
expect(records).toEqual([
|
||||
WALRecordType.BEGIN,
|
||||
WALRecordType.INSERT,
|
||||
WALRecordType.INSERT,
|
||||
WALRecordType.COMMIT,
|
||||
]);
|
||||
});
|
||||
|
||||
test('引擎 insert 批量写入只触发一次 WAL 落盘', async () => {
|
||||
const db = new MetonaSqlark({ name: `wal-batch-${Date.now()}`, mode: 'aria', diskEngine: 'memory' });
|
||||
await db.init();
|
||||
await db.defineTable('users', { id: { type: 'string', primaryKey: true }, name: { type: 'string' } });
|
||||
|
||||
const engine = db.getEngine() as any;
|
||||
let before = 0;
|
||||
const origRead = engine.backend.read.bind(engine.backend);
|
||||
// 统计 __wal_ 写入次数
|
||||
const origWrite = engine.backend.write.bind(engine.backend);
|
||||
let walWrites = 0;
|
||||
engine.backend.write = async (key: string, data: ArrayBuffer) => {
|
||||
if (key.startsWith('__wal_') && !key.startsWith('__wal_count')) walWrites++;
|
||||
return origWrite(key, data);
|
||||
};
|
||||
void before; void origRead;
|
||||
|
||||
await db.table('users').insertMany([
|
||||
{ id: '1', name: 'A' },
|
||||
{ id: '2', name: 'B' },
|
||||
{ id: '3', name: 'C' },
|
||||
{ id: '4', name: 'D' },
|
||||
]);
|
||||
|
||||
expect(walWrites).toBe(1); // 4 行 1 次 WAL 写入
|
||||
const rows = await db.query('SELECT * FROM users') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(4);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,406 @@
|
||||
/**
|
||||
* v0.3.2 功能测试
|
||||
* @module tests/sql-ext3
|
||||
*
|
||||
* 覆盖：CASE WHEN 用于 WHERE/聚合 / JOIN 哈希连接 / 多标签页同步
|
||||
*/
|
||||
import 'fake-indexeddb/auto';
|
||||
import { MetonaSqlark } from '../src/core';
|
||||
|
||||
async function createDb(mode: 'memory' | 'hybrid' = 'memory', extra: Record<string, unknown> = {}) {
|
||||
const db = new MetonaSqlark({
|
||||
name: `sql-ext3-${mode}-${Date.now()}-${Math.random()}`,
|
||||
mode,
|
||||
diskEngine: 'indexeddb',
|
||||
...extra,
|
||||
});
|
||||
await db.init();
|
||||
await db.defineTable('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
age: { type: 'number' },
|
||||
city: { type: 'string' },
|
||||
});
|
||||
await db.defineTable('orders', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
user_id: { type: 'string' },
|
||||
amount: { type: 'number' },
|
||||
});
|
||||
await db.query(`INSERT INTO users VALUES ('u1', 'Alice', 30, 'Beijing')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u2', 'Bob', 17, 'Shanghai')`);
|
||||
await db.query(`INSERT INTO users VALUES ('u3', 'Carol', 42, 'Beijing')`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o1', 'u1', 100)`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o2', 'u1', 200)`);
|
||||
await db.query(`INSERT INTO orders VALUES ('o3', 'u2', 50)`);
|
||||
return db;
|
||||
}
|
||||
|
||||
// ===================================================================
|
||||
// CASE WHEN 用于 WHERE
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.2] CASE WHEN 用于 WHERE', () => {
|
||||
test('WHERE CASE 等值比较', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name FROM users WHERE CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END = 'adult'`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows.map((r) => r.name).sort()).toEqual(['Alice', 'Carol']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('WHERE CASE 与 AND 组合', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name FROM users WHERE CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END = 'adult' AND city = 'Beijing'`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows.map((r) => r.name).sort()).toEqual(['Alice', 'Carol']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('WHERE CASE 数字比较', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name FROM users WHERE CASE WHEN city = 'Beijing' THEN 1 ELSE 0 END = 1`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows.map((r) => r.name).sort()).toEqual(['Alice', 'Carol']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('WHERE NOT CASE 组合', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT name FROM users WHERE NOT (CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END = 'adult')`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows.map((r) => r.name)).toEqual(['Bob']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('Aria 引擎 WHERE CASE 可用', async () => {
|
||||
const db = await createDb('hybrid');
|
||||
const rows = await db.query(
|
||||
`SELECT name FROM users WHERE CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END = 'minor'`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows.map((r) => r.name)).toEqual(['Bob']);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// CASE WHEN 用于聚合
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.2] CASE WHEN 用于聚合', () => {
|
||||
test('SUM(CASE WHEN...) 条件计数', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT SUM(CASE WHEN age >= 18 THEN 1 ELSE 0 END) AS adults FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows[0].adults).toBe(2);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('COUNT(CASE WHEN...) 与 AVG', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT COUNT(CASE WHEN city = 'Beijing' THEN 1 END) AS bj_count, AVG(CASE WHEN age >= 18 THEN age END) AS adult_avg FROM users`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows[0].bj_count).toBe(2);
|
||||
expect(rows[0].adult_avg).toBe(36); // (30 + 42) / 2
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('GROUP BY + SUM(CASE WHEN...)', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT city, SUM(CASE WHEN age >= 18 THEN 1 ELSE 0 END) AS adults FROM users GROUP BY city`,
|
||||
) as Record<string, unknown>[];
|
||||
const byCity = Object.fromEntries(rows.map((r) => [r.city, r.adults]));
|
||||
expect(byCity['Beijing']).toBe(2); // Alice + Carol
|
||||
expect(byCity['Shanghai']).toBe(0); // Bob 17 岁
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('GROUP BY + CASE 非聚合列', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT city, CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END AS status FROM users GROUP BY city`,
|
||||
) as Record<string, unknown>[];
|
||||
const byCity = Object.fromEntries(rows.map((r) => [r.city, r.status]));
|
||||
expect(byCity['Beijing']).toBe('adult'); // 组内第一行 Alice
|
||||
expect(byCity['Shanghai']).toBe('minor');
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// JOIN 哈希连接
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.2] JOIN 哈希连接', () => {
|
||||
test('INNER JOIN 主键等值走哈希连接（结果正确）', async () => {
|
||||
const db = await createDb();
|
||||
const rows = await db.query(
|
||||
`SELECT o.id FROM orders o INNER JOIN users u ON u.id = o.user_id`,
|
||||
) as Record<string, unknown>[];
|
||||
// orders 全有匹配用户
|
||||
expect(rows.map((r) => r['o.id']).sort()).toEqual(['o1', 'o2', 'o3']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('LEFT JOIN 哈希连接保留未匹配行（null 填充）', async () => {
|
||||
const db = await createDb();
|
||||
await db.query(`INSERT INTO users VALUES ('u9', 'Zoe', 20, 'Guangzhou')`);
|
||||
const rows = await db.query(
|
||||
`SELECT u.name, o.id FROM users u LEFT JOIN orders o ON o.user_id = u.id`,
|
||||
) as Record<string, unknown>[];
|
||||
// Zoe 无订单 → 保留（null 填充）；Alice 有 2 个订单 → 展开 2 行（LEFT JOIN 语义）
|
||||
expect(rows.map((r) => r['u.name']).sort()).toEqual(['Alice', 'Alice', 'Bob', 'Carol', 'Zoe']);
|
||||
const zoe = rows.find((r) => r['u.name'] === 'Zoe');
|
||||
expect(zoe).toBeDefined();
|
||||
expect(zoe!['o.id']).toBeNull();
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('INNER JOIN 哈希连接过滤无匹配行', async () => {
|
||||
const db = await createDb();
|
||||
await db.query(`INSERT INTO users VALUES ('u9', 'Zoe', 20, 'Guangzhou')`);
|
||||
const rows = await db.query(
|
||||
`SELECT u.name FROM users u INNER JOIN orders o ON o.user_id = u.id`,
|
||||
) as Record<string, unknown>[];
|
||||
// Alice 2 个订单 → 2 行；Bob 1 行；Carol 无订单被过滤
|
||||
expect(rows.map((r) => r['u.name']).sort()).toEqual(['Alice', 'Alice', 'Bob']);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('哈希连接一次 $in 查询（不再全表拉取）', async () => {
|
||||
const db = await createDb();
|
||||
const engine = db.getEngine() as any;
|
||||
let rightTableFinds = 0;
|
||||
const origFind = engine.find.bind(engine);
|
||||
engine.find = async (table: string, query: any) => {
|
||||
if (table === 'orders') {
|
||||
rightTableFinds++;
|
||||
if (query.where?.user_id?.$in) {
|
||||
// 哈希连接：$in 一次查询
|
||||
expect(query.where.user_id.$in).toEqual(expect.arrayContaining(['u1', 'u2']));
|
||||
}
|
||||
}
|
||||
return origFind(table, query);
|
||||
};
|
||||
await db.query(`SELECT u.name FROM users u INNER JOIN orders o ON o.user_id = u.id`);
|
||||
// orders.user_id 无索引 → 哈希回退；仍应恰有一次右表查询
|
||||
expect(rightTableFinds).toBe(1);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('哈希连接不适用于非索引右列（回退嵌套循环）', async () => {
|
||||
const db = await createDb();
|
||||
// orders.user_id 无索引 → 回退；结果仍正确
|
||||
const rows = await db.query(
|
||||
`SELECT o.id FROM orders o INNER JOIN users u ON u.id = o.user_id`,
|
||||
) as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(3);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// 多标签页同步
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.2] 多标签页同步', () => {
|
||||
// BroadcastChannel mock：模拟同源标签页间消息传递
|
||||
class MockBroadcastChannel {
|
||||
static instances: MockBroadcastChannel[] = [];
|
||||
name: string;
|
||||
onmessage: ((event: { data: unknown }) => void) | null = null;
|
||||
closed = false;
|
||||
constructor(name: string) {
|
||||
this.name = name;
|
||||
MockBroadcastChannel.instances.push(this);
|
||||
}
|
||||
postMessage(data: unknown): void {
|
||||
if (this.closed) return;
|
||||
for (const other of MockBroadcastChannel.instances) {
|
||||
if (other !== this && other.name === this.name && !other.closed && other.onmessage) {
|
||||
other.onmessage({ data });
|
||||
}
|
||||
}
|
||||
}
|
||||
close(): void {
|
||||
this.closed = true;
|
||||
}
|
||||
static reset(): void {
|
||||
MockBroadcastChannel.instances = [];
|
||||
}
|
||||
}
|
||||
|
||||
const origBC = (globalThis as any).BroadcastChannel;
|
||||
beforeAll(() => {
|
||||
(globalThis as any).BroadcastChannel = MockBroadcastChannel;
|
||||
});
|
||||
afterAll(() => {
|
||||
(globalThis as any).BroadcastChannel = origBC;
|
||||
});
|
||||
beforeEach(() => {
|
||||
MockBroadcastChannel.reset();
|
||||
});
|
||||
|
||||
test('SQL 写语句广播表变更，其他标签页订阅收到 external 事件', async () => {
|
||||
// 先建表（DDL 版本升级会触发其他标签页 onversionchange 关闭连接，故先建表再开第二连接）
|
||||
const setup = new MetonaSqlark({ name: 'mt-a', mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await setup.init();
|
||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||
await setup.close();
|
||||
|
||||
const dbA = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
const dbB = new MetonaSqlark({ name: 'mt-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
await dbA.init();
|
||||
await dbB.init();
|
||||
|
||||
const events: { type: string; table?: string }[] = [];
|
||||
dbB.subscribe('t', (e) => events.push(e));
|
||||
|
||||
await dbA.query(`INSERT INTO t VALUES ('1', 10)`);
|
||||
|
||||
// 等待广播送达（同步 mock 已即时）
|
||||
expect(events.length).toBeGreaterThan(0);
|
||||
expect(events[0].type).toBe('external');
|
||||
expect(events[0].table).toBe('t');
|
||||
|
||||
await dbA.close();
|
||||
await dbB.close();
|
||||
});
|
||||
|
||||
test('Hybrid 标签页收到广播后内存重载（读到其他标签页的新数据）', async () => {
|
||||
const setup = new MetonaSqlark({ name: 'mt-b', mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await setup.init();
|
||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||
await setup.close();
|
||||
|
||||
const dbA = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
const dbB = new MetonaSqlark({ name: 'mt-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
await dbA.init();
|
||||
await dbB.init();
|
||||
|
||||
// B 订阅外部变更后等待 reload 完成
|
||||
let reloadDone: Promise<void> = Promise.resolve();
|
||||
dbB.subscribe('t', async () => {
|
||||
reloadDone = reloadDone.then(async () => {
|
||||
// Hybrid reload 由 onmessage 触发（异步），订阅回调后再等一拍
|
||||
await new Promise((r) => setTimeout(r, 30));
|
||||
});
|
||||
});
|
||||
|
||||
await dbA.query(`INSERT INTO t VALUES ('1', 100)`);
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
|
||||
const rows = await dbB.query('SELECT * FROM t') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].v).toBe(100);
|
||||
|
||||
await dbA.close();
|
||||
await dbB.close();
|
||||
});
|
||||
|
||||
test('未启用 multiTabSync 不广播', async () => {
|
||||
const setup = new MetonaSqlark({ name: 'mt-c', mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await setup.init();
|
||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||
await setup.close();
|
||||
|
||||
const dbA = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
const dbB = new MetonaSqlark({ name: 'mt-c', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
await dbA.init();
|
||||
await dbB.init();
|
||||
|
||||
const events: unknown[] = [];
|
||||
dbB.subscribe('t', (e) => events.push(e));
|
||||
|
||||
await dbA.query(`INSERT INTO t VALUES ('1', 10)`);
|
||||
await new Promise((r) => setTimeout(r, 30));
|
||||
expect(events).toHaveLength(0); // dbA 未启用 → 无广播
|
||||
|
||||
await dbA.close();
|
||||
await dbB.close();
|
||||
});
|
||||
|
||||
test('Table API 写入也广播', async () => {
|
||||
const setup = new MetonaSqlark({ name: 'mt-d', mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await setup.init();
|
||||
await setup.defineTable('t', { id: { type: 'string', primaryKey: true }, v: { type: 'number' } });
|
||||
await setup.close();
|
||||
|
||||
const dbA = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
const dbB = new MetonaSqlark({ name: 'mt-d', version: 2, mode: 'hybrid', diskEngine: 'indexeddb', multiTabSync: true });
|
||||
await dbA.init();
|
||||
await dbB.init();
|
||||
|
||||
const events: unknown[] = [];
|
||||
dbB.subscribe('t', (e) => events.push(e));
|
||||
|
||||
await dbA.table('t').insert({ id: '1', v: 10 });
|
||||
expect(events.length).toBeGreaterThan(0);
|
||||
|
||||
await dbA.close();
|
||||
await dbB.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ===================================================================
|
||||
// 回归：IndexedDB reopen 后 schema 持久化（v0.3.2 修复）
|
||||
// ===================================================================
|
||||
|
||||
describe('[v0.3.2] IndexedDB reopen schema 持久化', () => {
|
||||
test('close 后重新 open 表结构与数据完整', async () => {
|
||||
const setup = new MetonaSqlark({ name: 'reopen-a', mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await setup.init();
|
||||
await setup.defineTable('t', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
v: { type: 'number' },
|
||||
});
|
||||
await setup.query(`INSERT INTO t VALUES ('1', 'Alice', 42)`);
|
||||
await setup.close();
|
||||
|
||||
// 重新打开（模拟页面刷新）
|
||||
const db = new MetonaSqlark({ name: 'reopen-a', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await db.init();
|
||||
|
||||
const schema = await db.getEngine().getTableSchema('t');
|
||||
expect(schema?.columns.v).toBeDefined(); // 持久化 schema 保留完整列
|
||||
|
||||
const rows = await db.query('SELECT * FROM t') as Record<string, unknown>[];
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].name).toBe('Alice');
|
||||
expect(rows[0].v).toBe(42);
|
||||
|
||||
// 重新打开后仍可写入并校验类型
|
||||
await db.query(`INSERT INTO t VALUES ('2', 'Bob', 30)`);
|
||||
await expect(db.query(`INSERT INTO t VALUES ('3', 'Bad', 'not-a-number')`)).rejects.toBeDefined();
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('reopen 后 UPDATE 全列生效', async () => {
|
||||
const setup = new MetonaSqlark({ name: 'reopen-b', mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await setup.init();
|
||||
await setup.defineTable('t', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
v: { type: 'number' },
|
||||
});
|
||||
await setup.query(`INSERT INTO t VALUES ('1', 'Alice', 42)`);
|
||||
await setup.close();
|
||||
|
||||
const db = new MetonaSqlark({ name: 'reopen-b', version: 2, mode: 'hybrid', diskEngine: 'indexeddb' });
|
||||
await db.init();
|
||||
await db.query(`UPDATE t SET name = 'Renamed', v = 99 WHERE id = '1'`);
|
||||
|
||||
const rows = await db.query('SELECT * FROM t') as Record<string, unknown>[];
|
||||
expect(rows[0].name).toBe('Renamed');
|
||||
expect(rows[0].v).toBe(99);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
});
|
||||
+99
-99
@@ -1,99 +1,99 @@
|
||||
/**
|
||||
* utils.ts 单元测试模板
|
||||
*/
|
||||
|
||||
import {
|
||||
generateId,
|
||||
escapeHTML,
|
||||
debounce,
|
||||
throttle,
|
||||
deepMerge,
|
||||
isBrowser,
|
||||
} from '../src/utils';
|
||||
|
||||
describe('generateId', () => {
|
||||
test('返回字符串', () => {
|
||||
expect(typeof generateId()).toBe('string');
|
||||
});
|
||||
|
||||
test('多次调用产生不同值', () => {
|
||||
const ids = new Set<string>();
|
||||
for (let i = 0; i < 100; i++) ids.add(generateId());
|
||||
expect(ids.size).toBe(100);
|
||||
});
|
||||
});
|
||||
|
||||
describe('escapeHTML', () => {
|
||||
test('转义 < > &', () => {
|
||||
const out = escapeHTML('<a>&b</a>');
|
||||
expect(out).toContain('<');
|
||||
expect(out).toContain('>');
|
||||
expect(out).toContain('&');
|
||||
});
|
||||
|
||||
test('null/undefined 返回空字符串', () => {
|
||||
expect(escapeHTML(null)).toBe('');
|
||||
expect(escapeHTML(undefined)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('debounce', () => {
|
||||
test('延迟执行', (done) => {
|
||||
let called = 0;
|
||||
const fn = debounce(() => { called++; }, 20);
|
||||
fn();
|
||||
expect(called).toBe(0);
|
||||
setTimeout(() => {
|
||||
expect(called).toBe(1);
|
||||
done();
|
||||
}, 50);
|
||||
});
|
||||
|
||||
test('多次调用只执行最后一次', (done) => {
|
||||
let result = 0;
|
||||
const fn = debounce((v: number) => { result = v; }, 20);
|
||||
fn(1);
|
||||
fn(2);
|
||||
fn(3);
|
||||
setTimeout(() => {
|
||||
expect(result).toBe(3);
|
||||
done();
|
||||
}, 50);
|
||||
});
|
||||
});
|
||||
|
||||
describe('throttle', () => {
|
||||
test('首次立即执行', () => {
|
||||
let called = 0;
|
||||
const fn = throttle(() => { called++; }, 50);
|
||||
fn();
|
||||
expect(called).toBe(1);
|
||||
});
|
||||
|
||||
test('限流期内不重复执行', () => {
|
||||
let called = 0;
|
||||
const fn = throttle(() => { called++; }, 50);
|
||||
fn();
|
||||
fn();
|
||||
fn();
|
||||
expect(called).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('deepMerge', () => {
|
||||
test('浅层合并', () => {
|
||||
const out = deepMerge({ a: 1, b: 2 }, { b: 3, c: 4 });
|
||||
expect(out).toEqual({ a: 1, b: 3, c: 4 });
|
||||
});
|
||||
|
||||
test('深层对象合并', () => {
|
||||
const out = deepMerge({ obj: { x: 1 } }, { obj: { y: 2 } });
|
||||
expect(out).toEqual({ obj: { x: 1, y: 2 } });
|
||||
});
|
||||
});
|
||||
|
||||
describe('isBrowser', () => {
|
||||
test('jsdom 环境下返回 true', () => {
|
||||
expect(isBrowser()).toBe(true);
|
||||
});
|
||||
});
|
||||
/**
|
||||
* utils.ts 单元测试模板
|
||||
*/
|
||||
|
||||
import {
|
||||
generateId,
|
||||
escapeHTML,
|
||||
debounce,
|
||||
throttle,
|
||||
deepMerge,
|
||||
isBrowser,
|
||||
} from '../src/utils';
|
||||
|
||||
describe('generateId', () => {
|
||||
test('返回字符串', () => {
|
||||
expect(typeof generateId()).toBe('string');
|
||||
});
|
||||
|
||||
test('多次调用产生不同值', () => {
|
||||
const ids = new Set<string>();
|
||||
for (let i = 0; i < 100; i++) ids.add(generateId());
|
||||
expect(ids.size).toBe(100);
|
||||
});
|
||||
});
|
||||
|
||||
describe('escapeHTML', () => {
|
||||
test('转义 < > &', () => {
|
||||
const out = escapeHTML('<a>&b</a>');
|
||||
expect(out).toContain('<');
|
||||
expect(out).toContain('>');
|
||||
expect(out).toContain('&');
|
||||
});
|
||||
|
||||
test('null/undefined 返回空字符串', () => {
|
||||
expect(escapeHTML(null)).toBe('');
|
||||
expect(escapeHTML(undefined)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('debounce', () => {
|
||||
test('延迟执行', (done) => {
|
||||
let called = 0;
|
||||
const fn = debounce(() => { called++; }, 20);
|
||||
fn();
|
||||
expect(called).toBe(0);
|
||||
setTimeout(() => {
|
||||
expect(called).toBe(1);
|
||||
done();
|
||||
}, 50);
|
||||
});
|
||||
|
||||
test('多次调用只执行最后一次', (done) => {
|
||||
let result = 0;
|
||||
const fn = debounce((v: number) => { result = v; }, 20);
|
||||
fn(1);
|
||||
fn(2);
|
||||
fn(3);
|
||||
setTimeout(() => {
|
||||
expect(result).toBe(3);
|
||||
done();
|
||||
}, 50);
|
||||
});
|
||||
});
|
||||
|
||||
describe('throttle', () => {
|
||||
test('首次立即执行', () => {
|
||||
let called = 0;
|
||||
const fn = throttle(() => { called++; }, 50);
|
||||
fn();
|
||||
expect(called).toBe(1);
|
||||
});
|
||||
|
||||
test('限流期内不重复执行', () => {
|
||||
let called = 0;
|
||||
const fn = throttle(() => { called++; }, 50);
|
||||
fn();
|
||||
fn();
|
||||
fn();
|
||||
expect(called).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('deepMerge', () => {
|
||||
test('浅层合并', () => {
|
||||
const out = deepMerge({ a: 1, b: 2 }, { b: 3, c: 4 });
|
||||
expect(out).toEqual({ a: 1, b: 3, c: 4 });
|
||||
});
|
||||
|
||||
test('深层对象合并', () => {
|
||||
const out = deepMerge({ obj: { x: 1 } }, { obj: { y: 2 } });
|
||||
expect(out).toEqual({ obj: { x: 1, y: 2 } });
|
||||
});
|
||||
});
|
||||
|
||||
describe('isBrowser', () => {
|
||||
test('jsdom 环境下返回 true', () => {
|
||||
expect(isBrowser()).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
+361
-374
@@ -1,374 +1,361 @@
|
||||
/**
|
||||
* v0.2.5 修复验证测试
|
||||
* 验证所有 P0/P1/P2 修复点
|
||||
*/
|
||||
|
||||
import { VERSION } from '../src/constants';
|
||||
import { MetonaSqlark } from '../src/core';
|
||||
import { AriaEngine } from '../src/engine/aria/index';
|
||||
import { MemoryEngine } from '../src/engine/memory';
|
||||
import { parse } from '../src/sql/parser';
|
||||
import { QueryExecutor } from '../src/query/executor';
|
||||
import { WAL } from '../src/engine/aria/wal/log';
|
||||
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
||||
import { BloomFilter } from '../src/engine/aria/index/bloom';
|
||||
import { CryptoManager } from '../src/engine/aria/crypto';
|
||||
import { PluginManager } from '../src/plugin/index';
|
||||
import type { SSTableMeta } from '../src/engine/aria/types';
|
||||
import type { MetonaPlugin } from '../src/constants';
|
||||
import { createSchema } from '../src/table/schema';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-1: 版本号统一
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
||||
test('VERSION 常量为 0.2.5', () => {
|
||||
expect(VERSION).toBe('0.2.5');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-2: AriaEngine OPFS 后端映射修复
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-2: AriaEngine OPFS 后端映射', () => {
|
||||
test('mode=aria + diskEngine=opfs 时应使用 opfs 后端', () => {
|
||||
const db = new MetonaSqlark({ name: 'test-opfs-map', mode: 'aria', diskEngine: 'opfs' });
|
||||
// 不实际 open（需要浏览器环境），只验证 createEngine 逻辑
|
||||
// 通过 getEngine 在 init 后检查
|
||||
expect(db).toBeDefined();
|
||||
});
|
||||
|
||||
test('mode=aria + diskEngine=indexeddb 时应使用 indexeddb 后端', () => {
|
||||
const db = new MetonaSqlark({ name: 'test-idb-map', mode: 'aria', diskEngine: 'indexeddb' });
|
||||
expect(db).toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-3: _onError 接入执行路径
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-3: _onError 接入执行路径', () => {
|
||||
test('query 失败时调用 onError 回调', async () => {
|
||||
const errors: Error[] = [];
|
||||
const db = new MetonaSqlark({
|
||||
name: 'test-onerror',
|
||||
mode: 'memory',
|
||||
onError: (e) => errors.push(e),
|
||||
});
|
||||
await db.init();
|
||||
await db.defineTable('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
});
|
||||
|
||||
// 故意执行不存在的表查询
|
||||
await expect(db.query('SELECT * FROM nonexistent')).rejects.toThrow();
|
||||
expect(errors.length).toBeGreaterThan(0);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('defineTable 失败时调用 onError', async () => {
|
||||
const errors: Error[] = [];
|
||||
const db = new MetonaSqlark({
|
||||
name: 'test-onerror2',
|
||||
mode: 'memory',
|
||||
onError: (e) => errors.push(e),
|
||||
});
|
||||
await db.init();
|
||||
await db.defineTable('dup', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
});
|
||||
// 重复创建
|
||||
await expect(db.defineTable('dup', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
})).rejects.toThrow();
|
||||
expect(errors.length).toBeGreaterThan(0);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-4: maxRowsPerQuery 生效
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-4: maxRowsPerQuery 生效', () => {
|
||||
test('结果集被截断为 maxRowsPerQuery', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-maxrows', 1);
|
||||
await engine.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
val: { type: 'number' },
|
||||
}));
|
||||
|
||||
// 插入 100 行
|
||||
for (let i = 0; i < 100; i++) {
|
||||
await engine.insert('items', [{ id: `item${i}`, val: i }]);
|
||||
}
|
||||
|
||||
const executor = new QueryExecutor(engine, 10); // maxRowsPerQuery=10
|
||||
const stmt = parse('SELECT * FROM items');
|
||||
const result = await executor.execute(stmt) as Record<string, unknown>[];
|
||||
expect(result.length).toBe(10); // 截断为 10 行
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('maxRowsPerQuery=0 表示不限制', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-nolimit', 1);
|
||||
await engine.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
}));
|
||||
for (let i = 0; i < 50; i++) {
|
||||
await engine.insert('items', [{ id: `i${i}` }]);
|
||||
}
|
||||
const executor = new QueryExecutor(engine, 0);
|
||||
const stmt = parse('SELECT * FROM items');
|
||||
const result = await executor.execute(stmt) as Record<string, unknown>[];
|
||||
expect(result.length).toBe(50);
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-5: WAL full 模式真正同步
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-5: WAL full 模式同步', () => {
|
||||
test('append 在 full 模式下是 async 且可 await', async () => {
|
||||
let appendCount = 0;
|
||||
const wal = new WAL({
|
||||
append: async (_data: Uint8Array) => { appendCount++; },
|
||||
readAll: async () => new Uint8Array(0),
|
||||
truncate: async () => {},
|
||||
exists: async () => false,
|
||||
}, true, 'full');
|
||||
|
||||
// append 现在返回 Promise
|
||||
await wal.append({
|
||||
type: 1, // INSERT
|
||||
txnId: 0,
|
||||
tableName: 'test',
|
||||
key: 'k1',
|
||||
data: { v: 1 },
|
||||
} as any);
|
||||
|
||||
expect(appendCount).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-6: PluginManager.install 传 db 实例
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-6: PluginManager.install 传 db 实例', () => {
|
||||
test('register 传 db 给 install', () => {
|
||||
let receivedDb: unknown = null;
|
||||
const plugin: MetonaPlugin = {
|
||||
name: 'test-plugin',
|
||||
install: (db) => { receivedDb = db; },
|
||||
destroy: () => {},
|
||||
};
|
||||
const pm = new PluginManager();
|
||||
const fakeDb = { name: 'fake' };
|
||||
pm.register(plugin, fakeDb);
|
||||
expect(receivedDb).toBe(fakeDb);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P1-7: SSTableReader 二分查找统一
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P1-7: SSTableReader 二分查找', () => {
|
||||
test('rangeScan 使用二分查找正确定位', () => {
|
||||
// 构建一个 SSTable 手工
|
||||
const { SSTableBuilder } = require('../src/engine/aria/index/sstable_builder');
|
||||
const builder = new SSTableBuilder(4096);
|
||||
// 添加足够多的条目以形成多个 block
|
||||
for (let i = 0; i < 100; i++) {
|
||||
const key = `key${String(i).padStart(5, '0')}`;
|
||||
builder.add(key, { data: `value${i}` });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
const meta: SSTableMeta = {
|
||||
id: 1,
|
||||
level: 0,
|
||||
minKey: 'key00000',
|
||||
maxKey: 'key00099',
|
||||
blockCount: 1,
|
||||
totalSize: sstableData.byteLength,
|
||||
bloomData: null,
|
||||
};
|
||||
const reader = new SSTableReader(sstableData, meta);
|
||||
|
||||
// 精确查找
|
||||
const result = reader.get('key00050');
|
||||
expect(result).not.toBeNull();
|
||||
expect((result as any).data).toBe('value50');
|
||||
|
||||
// 范围扫描
|
||||
const collected: string[] = [];
|
||||
reader.rangeScan('key00010', 'key00020', (k) => collected.push(k));
|
||||
expect(collected.length).toBeGreaterThan(0);
|
||||
expect(collected[0]).toBe('key00010');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P1-8: SQL 注入防护 — 表名校验
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P1-8: SQL 注入防护', () => {
|
||||
test('表名校验正则表达式正确', () => {
|
||||
// 验证正则逻辑本身
|
||||
const validName = /^[a-zA-Z_][a-zA-Z0-9_]*$/;
|
||||
expect(validName.test('users')).toBe(true);
|
||||
expect(validName.test('user_table')).toBe(true);
|
||||
expect(validName.test('_private')).toBe(true);
|
||||
expect(validName.test('Table1')).toBe(true);
|
||||
// 非法表名
|
||||
expect(validName.test('users; DROP TABLE')).toBe(false);
|
||||
expect(validName.test('1table')).toBe(false);
|
||||
expect(validName.test('user.name')).toBe(false);
|
||||
expect(validName.test('user name')).toBe(false);
|
||||
expect(validName.test("'; DROP TABLE users; --")).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P1-9: crypto 实例化
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P1-9: CryptoManager 实例化', () => {
|
||||
test('CryptoManager 可以独立实例化', () => {
|
||||
const cm1 = new CryptoManager();
|
||||
const cm2 = new CryptoManager();
|
||||
expect(cm1.enabled).toBe(false);
|
||||
expect(cm2.enabled).toBe(false);
|
||||
// 两个实例互不影响
|
||||
expect(cm1).not.toBe(cm2);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-14: ALTER TABLE 语法
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-14: ALTER TABLE', () => {
|
||||
test('解析 ALTER TABLE ADD COLUMN', () => {
|
||||
const stmt = parse('ALTER TABLE users ADD COLUMN email VARCHAR(255) UNIQUE');
|
||||
expect(stmt.type).toBe('ALTER_TABLE');
|
||||
expect((stmt as any).name).toBe('users');
|
||||
expect((stmt as any).action).toBe('ADD');
|
||||
expect((stmt as any).column.name).toBe('email');
|
||||
});
|
||||
|
||||
test('解析 ALTER TABLE DROP COLUMN', () => {
|
||||
const stmt = parse('ALTER TABLE users DROP COLUMN email');
|
||||
expect(stmt.type).toBe('ALTER_TABLE');
|
||||
expect((stmt as any).action).toBe('DROP');
|
||||
expect((stmt as any).column.name).toBe('email');
|
||||
});
|
||||
|
||||
test('执行 ALTER TABLE ADD COLUMN', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-alter', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
}));
|
||||
|
||||
const executor = new QueryExecutor(engine);
|
||||
const stmt = parse('ALTER TABLE users ADD COLUMN email VARCHAR(255)');
|
||||
await executor.execute(stmt);
|
||||
|
||||
const schema = await engine.getTableSchema('users');
|
||||
expect(schema!.columns.email).toBeDefined();
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('执行 ALTER TABLE DROP COLUMN', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-alter-drop', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
email: { type: 'string' },
|
||||
}));
|
||||
|
||||
const executor = new QueryExecutor(engine);
|
||||
const stmt = parse('ALTER TABLE users DROP COLUMN email');
|
||||
await executor.execute(stmt);
|
||||
|
||||
const schema = await engine.getTableSchema('users');
|
||||
expect(schema!.columns.email).toBeUndefined();
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-15: TRUNCATE TABLE 语法
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-15: TRUNCATE TABLE', () => {
|
||||
test('解析 TRUNCATE TABLE', () => {
|
||||
const stmt = parse('TRUNCATE TABLE users');
|
||||
expect(stmt.type).toBe('TRUNCATE_TABLE');
|
||||
expect((stmt as any).name).toBe('users');
|
||||
});
|
||||
|
||||
test('执行 TRUNCATE TABLE 清空数据', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-truncate', 1);
|
||||
await engine.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
}));
|
||||
await engine.insert('items', [
|
||||
{ id: 'a' }, { id: 'b' }, { id: 'c' },
|
||||
]);
|
||||
|
||||
const executor = new QueryExecutor(engine);
|
||||
const stmt = parse('TRUNCATE TABLE items');
|
||||
await executor.execute(stmt);
|
||||
|
||||
const rows = await engine.find('items', { table: 'items' });
|
||||
expect(rows.length).toBe(0);
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-12: WAL 大小阈值接入 checkpoint
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-12: WAL 大小阈值', () => {
|
||||
test('CheckpointManager 接收 walSizeThreshold 参数', () => {
|
||||
const { CheckpointManager } = require('../src/engine/aria/wal/checkpoint');
|
||||
const fakeLsm = { flush: async () => {} };
|
||||
const fakeWal = { flush: async () => {}, checkpoint: async () => {}, getBufferedCount: () => 0 };
|
||||
const cm = new CheckpointManager(fakeLsm, fakeWal, null, 1000, 1024);
|
||||
expect(cm).toBeDefined();
|
||||
expect(cm.getOpCount()).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-13: compactLevelSync 接口公开化
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-13: compactLevel public', () => {
|
||||
test('LSM.compactLevel 是 public 方法', () => {
|
||||
const { LSM } = require('../src/engine/aria/index/lsm');
|
||||
const lsm = new LSM({
|
||||
sstableStore: {
|
||||
save: async () => {}, load: async () => null, delete: async () => {},
|
||||
allocateId: async () => 1, listMeta: async () => [], saveMeta: async () => {}, deleteMeta: async () => {},
|
||||
},
|
||||
});
|
||||
expect(typeof lsm.compactLevel).toBe('function');
|
||||
});
|
||||
});
|
||||
/**
|
||||
* v0.2.5 修复验证测试
|
||||
* 验证所有 P0/P1/P2 修复点
|
||||
*/
|
||||
|
||||
import { VERSION } from '../src/constants';
|
||||
import { MetonaSqlark } from '../src/core';
|
||||
import { AriaEngine } from '../src/engine/aria/index';
|
||||
import { MemoryEngine } from '../src/engine/memory';
|
||||
import { parse } from '../src/sql/parser';
|
||||
import { QueryExecutor } from '../src/query/executor';
|
||||
import { WAL } from '../src/engine/aria/wal/log';
|
||||
import { SSTableReader } from '../src/engine/aria/index/sstable';
|
||||
import { BloomFilter } from '../src/engine/aria/index/bloom';
|
||||
import { CryptoManager } from '../src/engine/aria/crypto';
|
||||
import { PluginManager } from '../src/plugin/index';
|
||||
import type { SSTableMeta } from '../src/engine/aria/types';
|
||||
import type { MetonaPlugin } from '../src/constants';
|
||||
import { createSchema } from '../src/table/schema';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-1: 版本号统一
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-1: 版本号统一', () => {
|
||||
test('VERSION 常量为当前版本（0.3.2）', () => {
|
||||
expect(VERSION).toBe('0.3.2');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-2: AriaEngine OPFS 后端映射修复
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-2: AriaEngine OPFS 后端映射', () => {
|
||||
test('mode=aria + diskEngine=opfs 时应使用 opfs 后端', () => {
|
||||
const db = new MetonaSqlark({ name: 'test-opfs-map', mode: 'aria', diskEngine: 'opfs' });
|
||||
// 不实际 open（需要浏览器环境），只验证 createEngine 逻辑
|
||||
// 通过 getEngine 在 init 后检查
|
||||
expect(db).toBeDefined();
|
||||
});
|
||||
|
||||
test('mode=aria + diskEngine=indexeddb 时应使用 indexeddb 后端', () => {
|
||||
const db = new MetonaSqlark({ name: 'test-idb-map', mode: 'aria', diskEngine: 'indexeddb' });
|
||||
expect(db).toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-3: _onError 接入执行路径
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-3: _onError 接入执行路径', () => {
|
||||
test('query 失败时调用 onError 回调', async () => {
|
||||
const errors: Error[] = [];
|
||||
const db = new MetonaSqlark({
|
||||
name: 'test-onerror',
|
||||
mode: 'memory',
|
||||
onError: (e) => errors.push(e),
|
||||
});
|
||||
await db.init();
|
||||
await db.defineTable('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
});
|
||||
|
||||
// 故意执行不存在的表查询
|
||||
await expect(db.query('SELECT * FROM nonexistent')).rejects.toThrow();
|
||||
expect(errors.length).toBeGreaterThan(0);
|
||||
await db.close();
|
||||
});
|
||||
|
||||
test('defineTable 失败时调用 onError', async () => {
|
||||
const errors: Error[] = [];
|
||||
const db = new MetonaSqlark({
|
||||
name: 'test-onerror2',
|
||||
mode: 'memory',
|
||||
onError: (e) => errors.push(e),
|
||||
});
|
||||
await db.init();
|
||||
await db.defineTable('dup', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
});
|
||||
// 重复创建
|
||||
await expect(db.defineTable('dup', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
})).rejects.toThrow();
|
||||
expect(errors.length).toBeGreaterThan(0);
|
||||
await db.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-4: maxRowsPerQuery 生效
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-4: maxRowsPerQuery 生效', () => {
|
||||
test('结果集被截断为 maxRowsPerQuery', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-maxrows', 1);
|
||||
await engine.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
val: { type: 'number' },
|
||||
}));
|
||||
|
||||
// 插入 100 行
|
||||
for (let i = 0; i < 100; i++) {
|
||||
await engine.insert('items', [{ id: `item${i}`, val: i }]);
|
||||
}
|
||||
|
||||
const executor = new QueryExecutor(engine, 10); // maxRowsPerQuery=10
|
||||
const stmt = parse('SELECT * FROM items');
|
||||
const result = await executor.execute(stmt) as Record<string, unknown>[];
|
||||
expect(result.length).toBe(10); // 截断为 10 行
|
||||
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('maxRowsPerQuery=0 表示不限制', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-nolimit', 1);
|
||||
await engine.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
}));
|
||||
for (let i = 0; i < 50; i++) {
|
||||
await engine.insert('items', [{ id: `i${i}` }]);
|
||||
}
|
||||
const executor = new QueryExecutor(engine, 0);
|
||||
const stmt = parse('SELECT * FROM items');
|
||||
const result = await executor.execute(stmt) as Record<string, unknown>[];
|
||||
expect(result.length).toBe(50);
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-5: WAL full 模式真正同步
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-5: WAL full 模式同步', () => {
|
||||
test('append 在 full 模式下是 async 且可 await', async () => {
|
||||
let appendCount = 0;
|
||||
const wal = new WAL({
|
||||
append: async (_data: Uint8Array) => { appendCount++; },
|
||||
readAll: async () => new Uint8Array(0),
|
||||
truncate: async () => {},
|
||||
exists: async () => false,
|
||||
}, true, 'full');
|
||||
|
||||
// append 现在返回 Promise
|
||||
await wal.append({
|
||||
type: 1, // INSERT
|
||||
txnId: 0,
|
||||
tableName: 'test',
|
||||
key: 'k1',
|
||||
data: { v: 1 },
|
||||
} as any);
|
||||
|
||||
expect(appendCount).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P0-6: PluginManager.install 传 db 实例
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P0-6: PluginManager.install 传 db 实例', () => {
|
||||
test('register 传 db 给 install', () => {
|
||||
let receivedDb: unknown = null;
|
||||
const plugin: MetonaPlugin = {
|
||||
name: 'test-plugin',
|
||||
install: (db) => { receivedDb = db; },
|
||||
destroy: () => {},
|
||||
};
|
||||
const pm = new PluginManager();
|
||||
const fakeDb = { name: 'fake' };
|
||||
pm.register(plugin, fakeDb);
|
||||
expect(receivedDb).toBe(fakeDb);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P1-7: SSTableReader 二分查找统一
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P1-7: SSTableReader 二分查找', () => {
|
||||
test('rangeScan 使用二分查找正确定位', () => {
|
||||
// 构建一个 SSTable 手工
|
||||
const { SSTableBuilder } = require('../src/engine/aria/index/sstable_builder');
|
||||
const builder = new SSTableBuilder(4096);
|
||||
// 添加足够多的条目以形成多个 block
|
||||
for (let i = 0; i < 100; i++) {
|
||||
const key = `key${String(i).padStart(5, '0')}`;
|
||||
builder.add(key, { data: `value${i}` });
|
||||
}
|
||||
const { sstableData } = builder.build();
|
||||
const meta: SSTableMeta = {
|
||||
id: 1,
|
||||
level: 0,
|
||||
minKey: 'key00000',
|
||||
maxKey: 'key00099',
|
||||
blockCount: 1,
|
||||
totalSize: sstableData.byteLength,
|
||||
bloomData: null,
|
||||
};
|
||||
const reader = new SSTableReader(sstableData, meta);
|
||||
|
||||
// 精确查找
|
||||
const result = reader.get('key00050');
|
||||
expect(result).not.toBeNull();
|
||||
expect((result as any).data).toBe('value50');
|
||||
|
||||
// 范围扫描
|
||||
const collected: string[] = [];
|
||||
reader.rangeScan('key00010', 'key00020', (k) => collected.push(k));
|
||||
expect(collected.length).toBeGreaterThan(0);
|
||||
expect(collected[0]).toBe('key00010');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P1-8: SQL 注入防护 — 表名校验
|
||||
// 真实覆盖位置：tests/integrations/react.test.ts（useTable）
|
||||
// tests/integrations/vue.test.ts（useSqlarkTable）
|
||||
// 通过真实 hooks 调用触发 validateTableName，验证非法表名抛错。
|
||||
// 注：v0.2.6 移除此处"复制正则自测"的伪测试（未触达真实代码）。
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P1-9: crypto 实例化
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P1-9: CryptoManager 实例化', () => {
|
||||
test('CryptoManager 可以独立实例化', () => {
|
||||
const cm1 = new CryptoManager();
|
||||
const cm2 = new CryptoManager();
|
||||
expect(cm1.enabled).toBe(false);
|
||||
expect(cm2.enabled).toBe(false);
|
||||
// 两个实例互不影响
|
||||
expect(cm1).not.toBe(cm2);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-14: ALTER TABLE 语法
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-14: ALTER TABLE', () => {
|
||||
test('解析 ALTER TABLE ADD COLUMN', () => {
|
||||
const stmt = parse('ALTER TABLE users ADD COLUMN email VARCHAR(255) UNIQUE');
|
||||
expect(stmt.type).toBe('ALTER_TABLE');
|
||||
expect((stmt as any).name).toBe('users');
|
||||
expect((stmt as any).action).toBe('ADD');
|
||||
expect((stmt as any).column.name).toBe('email');
|
||||
});
|
||||
|
||||
test('解析 ALTER TABLE DROP COLUMN', () => {
|
||||
const stmt = parse('ALTER TABLE users DROP COLUMN email');
|
||||
expect(stmt.type).toBe('ALTER_TABLE');
|
||||
expect((stmt as any).action).toBe('DROP');
|
||||
expect((stmt as any).column.name).toBe('email');
|
||||
});
|
||||
|
||||
test('执行 ALTER TABLE ADD COLUMN', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-alter', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
}));
|
||||
|
||||
const executor = new QueryExecutor(engine);
|
||||
const stmt = parse('ALTER TABLE users ADD COLUMN email VARCHAR(255)');
|
||||
await executor.execute(stmt);
|
||||
|
||||
const schema = await engine.getTableSchema('users');
|
||||
expect(schema!.columns.email).toBeDefined();
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test('执行 ALTER TABLE DROP COLUMN', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-alter-drop', 1);
|
||||
await engine.createTable(createSchema('users', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
name: { type: 'string' },
|
||||
email: { type: 'string' },
|
||||
}));
|
||||
|
||||
const executor = new QueryExecutor(engine);
|
||||
const stmt = parse('ALTER TABLE users DROP COLUMN email');
|
||||
await executor.execute(stmt);
|
||||
|
||||
const schema = await engine.getTableSchema('users');
|
||||
expect(schema!.columns.email).toBeUndefined();
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-15: TRUNCATE TABLE 语法
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-15: TRUNCATE TABLE', () => {
|
||||
test('解析 TRUNCATE TABLE', () => {
|
||||
const stmt = parse('TRUNCATE TABLE users');
|
||||
expect(stmt.type).toBe('TRUNCATE_TABLE');
|
||||
expect((stmt as any).name).toBe('users');
|
||||
});
|
||||
|
||||
test('执行 TRUNCATE TABLE 清空数据', async () => {
|
||||
const engine = new MemoryEngine();
|
||||
await engine.open('test-truncate', 1);
|
||||
await engine.createTable(createSchema('items', {
|
||||
id: { type: 'string', primaryKey: true },
|
||||
}));
|
||||
await engine.insert('items', [
|
||||
{ id: 'a' }, { id: 'b' }, { id: 'c' },
|
||||
]);
|
||||
|
||||
const executor = new QueryExecutor(engine);
|
||||
const stmt = parse('TRUNCATE TABLE items');
|
||||
await executor.execute(stmt);
|
||||
|
||||
const rows = await engine.find('items', { table: 'items' });
|
||||
expect(rows.length).toBe(0);
|
||||
await engine.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-12: WAL 大小阈值接入 checkpoint
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-12: WAL 大小阈值', () => {
|
||||
test('CheckpointManager 接收 walSizeThreshold 参数', () => {
|
||||
const { CheckpointManager } = require('../src/engine/aria/wal/checkpoint');
|
||||
const fakeLsm = { flush: async () => {} };
|
||||
const fakeWal = { flush: async () => {}, checkpoint: async () => {}, getBufferedCount: () => 0 };
|
||||
const cm = new CheckpointManager(fakeLsm, fakeWal, null, 1000, 1024);
|
||||
expect(cm).toBeDefined();
|
||||
expect(cm.getOpCount()).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2-13: compactLevelSync 接口公开化
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('[v0.2.5] P2-13: compactLevel public', () => {
|
||||
test('LSM.compactLevel 是 public 方法', () => {
|
||||
const { LSM } = require('../src/engine/aria/index/lsm');
|
||||
const lsm = new LSM({
|
||||
sstableStore: {
|
||||
save: async () => {}, load: async () => null, delete: async () => {},
|
||||
allocateId: async () => 1, listMeta: async () => [], saveMeta: async () => {}, deleteMeta: async () => {},
|
||||
},
|
||||
});
|
||||
expect(typeof lsm.compactLevel).toBe('function');
|
||||
});
|
||||
});
|
||||
|
||||
+24
-24
@@ -1,24 +1,24 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2020",
|
||||
"module": "ES2020",
|
||||
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
||||
"moduleResolution": "node",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"noUnusedParameters": false,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"skipLibCheck": true,
|
||||
"declaration": false,
|
||||
"declarationMap": false,
|
||||
"sourceMap": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"baseUrl": ".",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["node_modules", "dist", "tests"]
|
||||
}
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2020",
|
||||
"module": "ES2020",
|
||||
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
||||
"moduleResolution": "node",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"noUnusedParameters": false,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"skipLibCheck": true,
|
||||
"declaration": false,
|
||||
"declarationMap": false,
|
||||
"sourceMap": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"baseUrl": ".",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["node_modules", "dist", "tests"]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user