feat: v0.2.0 AriaEngine 自研存储引擎
CI / test (20.x) (push) Canceled after 0s
CI / test (22.x) (push) Canceled after 0s
CI / test (24.x) (push) Canceled after 0s
CI / test (18.x) (push) Canceled after 1h26m17s

- 新增 AriaEngine: LSM-Tree 页面式存储引擎,19 个模块,~3500 行 TS
  - page/: Slotted Page 格式 (header/slot/tuple/format) + CRC32
  - buffer/: Buffer Pool (LRU 缓存 + 驱逐策略)
  - index/: LSM-Tree (MemTable 红黑树 + SSTable + Bloom Filter + Merge Iterator)
  - wal/: WAL 日志 (二进制格式) + Checkpoint 管理
  - transaction/: MVCC 版本链 + 快照隔离
  - store/: IndexedDB / Memory 双后端抽象
  - compression/: LZ4 页面压缩

- 完整持久化: Schema 自动保存、SSTable 元数据管理、WAL 恢复
- 事务感知 CRUD: insert/update/delete 在事务中缓冲到 snapshot
- mode: 'aria' 激活自研引擎

- 新增 7 个测试文件,测试数 318 → 524,套件 20 → 27
  - aria-page.test.ts (32 tests): Page 格式单元测试
  - aria-index.test.ts (26 tests): Bloom Filter + MemTable
  - aria-sstable.test.ts (9 tests): SSTable Builder + Reader
  - aria-buffer.test.ts (25 tests): LRU + Eviction + Buffer Pool
  - aria-wal-mvcc.test.ts (22 tests): WAL 编解码 + MVCC 事务
  - aria-compress.test.ts (11 tests): LZ4 + Merge Iterator
  - aria.test.ts (80 tests): AriaEngine 集成 + 边界测试

- Bug 修复: LRUList size 跟踪、WAL 缓冲区越界、ColumnEncoding 导入
- 全面更新 README.md + site/ 站点文件 (index/docs/demo)
This commit is contained in:
2026-07-27 16:40:29 +08:00
parent c00738aea0
commit f84673e519
47 changed files with 14553 additions and 108 deletions
+61 -12
View File
@@ -3,7 +3,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>📖 API 文档 — MetonaSqlark v0.1.14</title>
<title>📖 API 文档 — MetonaSqlark v0.2.0</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='8' fill='%236366f1'/><text x='16' y='22' text-anchor='middle' font-size='20' fill='white'>◈</text></svg>">
<style>
:root {
@@ -86,7 +86,8 @@
<a href="#groupby">GROUP BY & 聚合</a>
<a href="#where-ops">WHERE 操作符</a>
<h4>高级特性</h4>
<a href="#transaction">事务 & 回滚</a>
<a href="#aria-engine">AriaEngine 🆕</a>
<a href="#transaction">事务 & 回滚</a>
<a href="#subquery">子查询</a>
<a href="#foreign-key">外键级联</a>
<a href="#connection-pool">连接池</a>
@@ -146,7 +147,7 @@ npm install @metona-team/metona-sqlark</pre>
<pre><span class="k">const</span> db = <span class="k">await</span> <span class="f">MetonaSqlark.create</span>({
<span class="s">name</span>: <span class="s">'my-app'</span>,
<span class="s">mode</span>: <span class="s">'hybrid'</span>, <span class="c">// 'memory' | 'disk' | 'hybrid'</span>
<span class="s">mode</span>: <span class="s">'hybrid'</span>, <span class="c">// 'memory' | 'disk' | 'hybrid' | 'aria' 🆕</span>
<span class="s">diskEngine</span>: <span class="s">'indexeddb'</span>, <span class="c">// 'indexeddb' | 'opfs'(仅 disk/hybrid 生效)</span>
<span class="s">version</span>: <span class="n">1</span>,
<span class="s">plugins</span>: [], <span class="c">// MetonaPlugin[]</span>
@@ -607,7 +608,7 @@ db.<span class="f">emit</span>(<span class="s">'users'</span>, { <span class="s"
<table>
<tr><th>属性</th><th>类型</th><th>默认值</th><th>说明</th></tr>
<tr><td><code>name</code></td><td><code>string</code></td><td><code>'metona-sqlark'</code></td><td>数据库名称(必填)</td></tr>
<tr><td><code>mode</code></td><td><code>'memory'|'disk'|'hybrid'</code></td><td><code>'hybrid'</code></td><td>存储模式</td></tr>
<tr><td><code>mode</code></td><td><code>'memory'|'disk'|'hybrid'|'aria'</code></td><td><code>'hybrid'</code></td><td>存储模式 🆕 aria</td></tr>
<tr><td><code>diskEngine</code></td><td><code>'indexeddb'|'opfs'</code></td><td><code>'indexeddb'</code></td><td>磁盘引擎类型</td></tr>
<tr><td><code>version</code></td><td><code>number</code></td><td><code>1</code></td><td>数据库版本号</td></tr>
<tr><td><code>plugins</code></td><td><code>MetonaPlugin[]</code></td><td><code>[]</code></td><td>初始插件列表</td></tr>
@@ -617,15 +618,63 @@ db.<span class="f">emit</span>(<span class="s">'users'</span>, { <span class="s"
<h2 id="engine">💾 存储引擎</h2>
<table>
<tr><th>引擎</th><th>模式</th><th>持久化</th><th>性能</th><th>适用场景</th></tr>
<tr><td><code>MemoryEngine</code></td><td>memory</td><td></td><td>⚡ 极快</td><td>临时数据、缓存、测试</td></tr>
<tr><td><code>IndexedDBEngine</code></td><td>disk</td><td></td><td>🚀 快</td><td>通用持久化,兼容性最好</td></tr>
<tr><td><code>OPFSEngine</code></td><td>disk</td><td></td><td>🚀 快</td><td>现代浏览器,文件级存储</td></tr>
<tr><td><code>HybridEngine</code></td><td>hybrid</td><td></td><td>⚡ 极快</td><td>生产推荐,读写均走内存</td></tr>
</table>
<table>
<tr><th>引擎</th><th>模式</th><th>持久化</th><th>索引</th><th>事务</th><th>适用场景</th></tr>
<tr><td><code>MemoryEngine</code></td><td>memory</td><td></td><td>哈希</td><td>快照回滚</td><td>临时数据、缓存、测试</td></tr>
<tr><td><code>IndexedDBEngine</code></td><td>disk</td><td>IDB</td><td>IDB 索引</td><td>延迟写入</td><td>通用持久化,兼容性最好</td></tr>
<tr><td><code>OPFSEngine</code></td><td>disk</td><td>OPFS</td><td>哈希</td><td>快照回滚</td><td>现代浏览器,文件级存储</td></tr>
<tr><td><code>HybridEngine</code></td><td>hybrid</td><td>Write-Through</td><td>哈希</td><td>双引擎代理</td><td>生产推荐,读写均走内存</td></tr>
<tr style="border-top:2px solid var(--primary);"><td><code style="color:#ec4899;font-weight:700;">AriaEngine 🆕</code></td><td>aria</td><td>✅ WAL + SSTable</td><td>LSM-Tree</td><td>MVCC 快照隔离</td><td>自研引擎:大表、高并发、需崩溃恢复</td></tr>
</table>
<p>Hybrid 引擎采用 <strong>write-through</strong> 策略:所有写操作同时写入内存和磁盘,所有读操作直接从内存返回,启动时从磁盘加载数据到内存。</p>
<h2 id="aria-engine">🌲 AriaEngine 自研存储引擎</h2>
<p><strong>v0.2.0 新增</strong> — AriaEngine 是专为 MetonaSqlark 设计的页面式存储引擎,对标 SQLite 设计理念。</p>
<h3>核心特性</h3>
<table>
<tr><th>特性</th><th>说明</th></tr>
<tr><td><strong>LSM-Tree 索引</strong></td><td>MemTable (红黑树) + 多级 SSTable,写优化,支持范围扫描</td></tr>
<tr><td><strong>Slotted Page 格式</strong></td><td>4KB 固定页面,Slot Directory + Tuple 二进制序列化</td></tr>
<tr><td><strong>Buffer Pool</strong></td><td>LRU 页面缓存,可控内存占用(默认 256 页 ≈ 1MB)</td></tr>
<tr><td><strong>WAL 日志</strong></td><td>Write-Ahead Log 保证崩溃恢复,支持 full/batch/none 三种同步模式</td></tr>
<tr><td><strong>MVCC 事务</strong></td><td>快照隔离 (Snapshot Isolation),读写不互斥,版本链 + GC</td></tr>
<tr><td><strong>Bloom Filter</strong></td><td>快速判定 key 不存在,减少无效磁盘 I/O</td></tr>
<tr><td><strong>LZ4 压缩</strong></td><td>可选页面级压缩,空间效率提升</td></tr>
</table>
<h3>使用方式</h3>
<pre><span class="c">// 激活 AriaEngine</span>
<span class="k">const</span> db = <span class="k">await</span> <span class="f">MetonaSqlark.create</span>({
<span class="s">name</span>: <span class="s">'my-app'</span>,
<span class="s">mode</span>: <span class="s">'aria'</span>, <span class="c">// 🆕 AriaEngine 模式</span>
<span class="s">diskEngine</span>: <span class="s">'indexeddb'</span>, <span class="c">// 底层存储后端(indexeddb | opfs | memory</span>
});
<span class="c">// 或直接实例化 — 支持细粒度配置</span>
<span class="k">import</span> { <span class="t">AriaEngine</span> } <span class="k">from</span> <span class="s">'@metona-team/metona-sqlark'</span>;
<span class="k">const</span> engine = <span class="k">new</span> <span class="f">AriaEngine</span>({
<span class="s">pageSize</span>: <span class="n">4096</span>, <span class="c">// 页面大小</span>
<span class="s">bufferPoolPages</span>: <span class="n">256</span>, <span class="c">// 缓存页数</span>
<span class="s">memtableSizeThreshold</span>: <span class="n">4194304</span>, <span class="c">// MemTable 刷盘阈值 4MB</span>
<span class="s">walEnabled</span>: <span class="k">true</span>, <span class="c">// 启用 WAL</span>
<span class="s">walSyncMode</span>: <span class="s">'batch'</span>, <span class="c">// 'full' | 'batch' | 'none'</span>
<span class="s">storageBackend</span>: <span class="s">'indexeddb'</span>, <span class="c">// 存储后端</span>
});</pre>
<h3>AriaEngine 配置项</h3>
<table>
<tr><th>属性</th><th>类型</th><th>默认值</th><th>说明</th></tr>
<tr><td><code>pageSize</code></td><td><code>number</code></td><td><code>4096</code></td><td>页面大小(字节)</td></tr>
<tr><td><code>bufferPoolPages</code></td><td><code>number</code></td><td><code>256</code></td><td>Buffer Pool 页面数量</td></tr>
<tr><td><code>memtableSizeThreshold</code></td><td><code>number</code></td><td><code>4194304</code></td><td>MemTable 刷盘阈值(字节)</td></tr>
<tr><td><code>levelSizeMultiplier</code></td><td><code>number</code></td><td><code>10</code></td><td>LSM 层级容量倍数</td></tr>
<tr><td><code>bloomFilterBitsPerKey</code></td><td><code>number</code></td><td><code>10</code></td><td>Bloom Filter 每 key 位数</td></tr>
<tr><td><code>walEnabled</code></td><td><code>boolean</code></td><td><code>true</code></td><td>是否启用 WAL</td></tr>
<tr><td><code>walSyncMode</code></td><td><code>'full'|'batch'|'none'</code></td><td><code>'batch'</code></td><td>WAL 同步策略</td></tr>
<tr><td><code>checkpointInterval</code></td><td><code>number</code></td><td><code>1000</code></td><td>Checkpoint 触发间隔(操作数)</td></tr>
<tr><td><code>compression</code></td><td><code>boolean</code></td><td><code>false</code></td><td>是否启用页面压缩</td></tr>
<tr><td><code>storageBackend</code></td><td><code>'indexeddb'|'opfs'|'memory'</code></td><td><code>'indexeddb'</code></td><td>存储后端类型</td></tr>
</table>
<h2 id="errors">⚠️ 错误处理</h2>
<p>所有错误抛出 <code>DatabaseError</code> 实例。</p>