From 9f09f9c9022bb1383ac328a8576b5541b3041cb0 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:11:11 +0800 Subject: [PATCH 01/17] =?UTF-8?q?feat:=20=E5=B0=86=E4=B9=A6=E7=B1=8D?= =?UTF-8?q?=E6=95=B0=E6=8D=AE=E8=BF=81=E5=85=A5=20SQLite=20=E5=B9=B6?= =?UTF-8?q?=E4=BF=9D=E7=95=99=E5=A4=87=E4=BB=BD=E4=B8=8E=E5=90=8C=E6=AD=A5?= =?UTF-8?q?=E5=85=BC=E5=AE=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agents/skills/readerx-data/SKILL.md | 2 +- .../readerx-data/references/contracts.md | 8 +- .agents/skills/readerx-source-engine/SKILL.md | 1 + .github/workflows/test-data.yml | 36 + THIRD-PARTY-NOTICES.md | 8 + docs/annotations.md | 4 +- docs/backup.md | 4 +- docs/book-storage.md | 63 + docs/sync.md | 14 +- src-tauri/Cargo.lock | 85 ++ src-tauri/Cargo.toml | 2 + src-tauri/src/book_store.rs | 1284 +++++++++-------- src-tauri/src/book_store/sqlite.rs | 964 +++++++++++++ src-tauri/src/commands.rs | 6 +- src-tauri/src/data_transfer/archive.rs | 2 +- src-tauri/src/data_transfer/export.rs | 37 +- src-tauri/src/data_transfer/import.rs | 80 +- src-tauri/src/data_transfer/mod.rs | 6 +- src-tauri/src/data_transfer/plan.rs | 2 +- src-tauri/src/data_transfer/round_trip.rs | 64 +- src-tauri/src/lib.rs | 2 +- src-tauri/src/storage.rs | 2 +- src-tauri/src/sync/book_ids.rs | 49 +- src-tauri/src/sync/bridge.rs | 10 +- src-tauri/src/sync/content.rs | 11 +- src-tauri/src/sync/data_ids.rs | 88 +- src-tauri/src/sync/service.rs | 2 +- src-tauri/tests/support/book_json.rs | 161 +++ src-tauri/tests/sync_bridge.rs | 66 +- 29 files changed, 2259 insertions(+), 804 deletions(-) create mode 100644 .github/workflows/test-data.yml create mode 100644 docs/book-storage.md create mode 100644 src-tauri/src/book_store/sqlite.rs create mode 100644 src-tauri/tests/support/book_json.rs diff --git a/.agents/skills/readerx-data/SKILL.md b/.agents/skills/readerx-data/SKILL.md index cda633c..67bfa6d 100644 --- a/.agents/skills/readerx-data/SKILL.md +++ b/.agents/skills/readerx-data/SKILL.md @@ -11,7 +11,7 @@ description: 修改 ReaderX 本地书库存储、稳定身份与旧数据迁移 | 功能 | 实现入口 | 按需资料 | | --- | --- | --- | -| 本地书库存储、迁移、稳定 ID | src-tauri/src/book_store.rs、storage.rs、models.rs、sync/identity.rs、sync/book_ids.rs、sync/data_ids.rs；src/lib/books.ts、booksTypes.ts、dataIds.ts | [存储与身份约束](references/contracts.md#书库与稳定身份) | +| 本地书库存储、迁移、稳定 ID | src-tauri/src/book_store.rs、book_store/sqlite.rs、storage.rs、models.rs、sync/identity.rs、sync/book_ids.rs、sync/data_ids.rs；src/lib/books.ts、booksTypes.ts、dataIds.ts | [存储与身份约束](references/contracts.md#书库与稳定身份) | | 备份与恢复 | src-tauri/src/data_transfer/；src/lib/backup.ts、src/pages/DataBackup.tsx | [备份约束](references/contracts.md#备份与恢复)、[备份文档](../../../docs/backup.md) | | 同步协议、合并与传输 | src-tauri/crates/readerx-sync/ | [同步约束](references/contracts.md#局域网同步)、[同步文档](../../../docs/sync.md) 对应章节 | | 同步 App 接线与落库 | src-tauri/src/sync/；src/lib/sync.ts、src/components/SyncProgress.tsx | 同步文档 App 接线、实时落库与停止部分 | diff --git a/.agents/skills/readerx-data/references/contracts.md b/.agents/skills/readerx-data/references/contracts.md index dc986df..456a26a 100644 --- a/.agents/skills/readerx-data/references/contracts.md +++ b/.agents/skills/readerx-data/references/contracts.md @@ -4,11 +4,11 @@ ## 书库与稳定身份 -- book_store.rs 按 books// 拆分 bookdetail.json、content.json、bookmarks.json、annotations.json；digest.json 是派生指纹缓存。元信息补丁不读写整本正文，逐章回写与书签各走对应文件。新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts，保留 schemaVersion 信封及字段往返测试。 -- 注释走 readerx_annotations_get/put，与正文/书签文件独立，信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移随整个书目录移动；删除/恢复失效缓存并阻止在飞读取回填。备份包含文件并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 +- book_store.rs / book_store/sqlite.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、状态 JSON 和图片字节不入库。数据库 user_version 与便携 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 +- 注释走 readerx_annotations_get/put，与正文/书签独立存表，便携信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含文件并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 - book_store 的读写入口在进程级书库事务锁内完成迁移、读取、修改、落盘及指纹更新；持锁时不调用同步引擎。单章补丁以 chapter.cid 定位，index 仅兼容旧载荷；先验证整批 cid 存在且唯一，再只改正文，保留当前目录标题/地址。 - 普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 -- 旧 books/.json 布局迁移成功后才删旧文件，失败仍可回退读取；全库旧书签完成迁移后留 .migrated 备份。不要通过删旧数据让新布局检查通过。 +- 旧整书 / 目录 JSON 逐本在事务内导入并保存迁移回执，成功才改名 .migrated 留底；失败保留读回退和重试，不影响其他书。归档中断不能重复导入旧数据覆盖新编辑。数据库身份改动和状态 / TTS 目录迁移保持可重跑。 - 书库与同步共用稳定 ID；已有规范 ID 沿用，不能在每次编辑后按新名字/地址重新计算。身份算法以 sync/identity.rs 为准：在线书首次按书源地址与详情页地址派生，导入书按归一文件名与字节数派生。分组与书源首次按名称/地址派生后编辑沿用 ID；规则仍按内容派生。前端 dataIds.ts 与 Rust 种子、大小写和归一化口径保持一致。 - 迁移顺序与启动入口检查 src-tauri/src/lib.rs：书籍迁移在 UI/同步访问前完成，data_ids 再处理分组、书源、规则及引用。迁移待办先落盘，每步可重跑；目录或身份冲突保留原数据并返回失败，不能覆盖目标。 - 改 ID 时检查书架进度、书签 bookId、按书规则、阅读时长及 tts-audio 目录。现有书籍迁移保留图片名，因为正文持有旧引用；不能只改目录或强行按新 ID 重命名图片。 @@ -16,7 +16,7 @@ ## 备份与恢复 -- 归档入口按 data_transfer/archive.rs 分类与校验条目名，拒绝越界路径；不绕过 is_safe_entry 直接解压。plan.rs 只读规划，import.rs 写盘；书籍导入先落 bookdetail.json，复用 write_bytes 创建父目录，再处理书签/注释，不单独重复建目录或依赖条目字母顺序；流式处理正文/图片，避免为整库复制字节到内存。 +- 归档入口按 data_transfer/archive.rs 分类与校验条目名，拒绝越界路径；不绕过 is_safe_entry 直接解压。plan.rs 只读规划，import.rs 写盘；书籍导入先经 import_book_json 在事务内写元信息和正文，再处理书签/注释；导出从 SQLite 按章生成原 JSON 归档，不直接复制数据库文件；图片流式处理。 - 合并与覆盖语义不同：合并保留本机额外数据并按身份映射引用，偏好已有值保留；覆盖先写后删并清空归档缺失的内容类状态。新增 state 键时明确分类、合并规则、缺失语义与 ID 换算，不能直接全量 JSON 覆盖。 - 书源会话与 WebDAV 密码默认过滤，显式包含才导出；sync/、tts-audio/、logs/ 和本机 reading-time-origin 不作为换机数据搬走。阅读时长 contributions 应保留，恢复后本机新增时间使用独立来源。 - 覆盖删除书或书源前先调用同步删除钩子（定位身份仍需要旧数据），批量导入后调用 on_data_imported 全量对账。前端 import 完成后按 backup.ts 依赖顺序失效书签并重载分组、书库、进度、统计、规则、书源。 diff --git a/.agents/skills/readerx-source-engine/SKILL.md b/.agents/skills/readerx-source-engine/SKILL.md index c310241..c6cbcc4 100644 --- a/.agents/skills/readerx-source-engine/SKILL.md +++ b/.agents/skills/readerx-source-engine/SKILL.md @@ -23,6 +23,7 @@ description: 修改 ReaderX 书源引擎、Boa 宿主 API、浏览器认证后 - 桌面登录看 src-tauri/plugins/tauri-plugin-webview-login；Linux 用 WebKitGTK CookieManager 读取含 httpOnly 的 Cookie；页面存储能力以 src/desktop.rs 的实测表为准，不用宿主 eval 读隔离的页面 localStorage。探针生成逻辑在引擎 storage.rs，后端不解析其内部结构。桌面正常收尾与 Android 一样不要求 Cookie 非空，仍采集并透传存储探针；等待标记按求值字符串解码后的完整值比较，不匹配快照子串。 - 数据根目录由宿主 store::init_data_root 指定。修改格式前检查 store.rs 布局和旧数据，保持 App/CLI 共用数据兼容，明确迁移方式。 - 书源稳定 ID 与迁移查看 src-tauri/crates/readerx-source/src/id_migration.rs；涉及在线书引用、分组或同步身份时同时读 [数据技能](../readerx-data/SKILL.md)，检查 src-tauri/src/sync/data_ids.rs 与 src/lib/dataIds.ts，不因编辑书源地址重建已有稳定 ID。 +- App 保存书源通过 sync/data_ids.rs 的 save_source；文件仍由共享书源 crate 写入，数据库中的 bookSourceId 引用与迁移映射由 App 维护。旧元信息损坏时保留映射并延后引用迁移。 - 新 App command 同步 src-tauri/src/lib.rs 注册与 capability；书源变动同步相应 docs/。 ## 验证与交付 diff --git a/.github/workflows/test-data.yml b/.github/workflows/test-data.yml new file mode 100644 index 0000000..205c157 --- /dev/null +++ b/.github/workflows/test-data.yml @@ -0,0 +1,36 @@ +name: Test book storage and data compatibility + +on: + pull_request: + paths: + - 'src-tauri/**' + - '.github/workflows/test-data.yml' + - '.github/actions/setup-build/**' + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: test-data-${{ github.ref }} + cancel-in-progress: true + +jobs: + data-tests: + runs-on: ubuntu-latest + timeout-minutes: 45 + steps: + - uses: actions/checkout@v5 + - uses: ./.github/actions/setup-build + - name: Prepare frontend assets + run: pnpm build + - name: Book storage, migrations, backups and sync integration + working-directory: src-tauri + run: | + cargo test --locked -p readerx --lib book_store + cargo test --locked -p readerx --lib sync::book_ids + cargo test --locked -p readerx --lib sync::data_ids + cargo test --locked -p readerx --lib data_transfer + cargo test --locked -p readerx --test sync_bridge + cargo test --locked -p readerx-sync + cargo test --locked -p readerx-source diff --git a/THIRD-PARTY-NOTICES.md b/THIRD-PARTY-NOTICES.md index 9962c3f..e933fad 100644 --- a/THIRD-PARTY-NOTICES.md +++ b/THIRD-PARTY-NOTICES.md @@ -43,6 +43,14 @@ tauri-plugin-tts-api（MIT） 【后端 / Rust】 +rusqlite（MIT） + 书籍元信息、章节、书签与注释的 SQLite 存储和事务 + https://github.com/rusqlite/rusqlite + +SQLite（Public Domain） + 随应用编译的数据库引擎（rusqlite bundled） + https://sqlite.org/copyright.html + mobi（MIT） 解析 MOBI 元信息 https://github.com/vv9k/mobi-rs diff --git a/docs/annotations.md b/docs/annotations.md index af22771..55fa958 100644 --- a/docs/annotations.md +++ b/docs/annotations.md @@ -1,5 +1,7 @@ # 阅读注释 +书籍在应用目录使用 SQLite，备份仍为便携 JSON 条目；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 + 在同一段落内选取文字，点选区菜单的「注释」即可写下一条文本注释。保存后， 该段最后一个字符右侧出现 SVG 图标；点击图标可查看该段的全部注释、继续添加， 或通过每条右侧的编辑按钮改写已有内容。单条注释最多 2000 个 UTF-16 单元， @@ -15,7 +17,7 @@ `src/lib/annotations.ts` 负责段落锚点与状态，`AnnotationSheet.tsx` 负责抽屉， `Reader.tsx` 接通选区和正文图标。读写通过 `readerx_annotations_get/put`，由 Rust -在书库事务锁内原子保存到 `books//annotations.json`，与正文和书签文件分开。 +在书库事务锁内保存到 `books.sqlite3` 的 `annotations` 表，与章节和书签分别存表。 纯浏览器开发模式只存内存，刷新页面后不保留。 文件信封为 `schemaVersion: 1` 与 `annotations` 数组。每条记录包含独立 `id`、 diff --git a/docs/backup.md b/docs/backup.md index 946dafb..ab69172 100644 --- a/docs/backup.md +++ b/docs/backup.md @@ -1,5 +1,7 @@ # 数据备份与恢复（导入 / 导出） +书籍在应用目录使用 SQLite，备份仍为便携 JSON 条目；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 + > 状态：**已接入 App**（设置 → 数据 → 数据备份）。本文件说明备份里有什么、归档长什么样、 > 导入的两种语义，以及与「局域网同步」的边界。 @@ -126,7 +128,7 @@ source_sessions/.json 书源登录态（**仅当导出时勾选「包含 - 导入结束后调一次 `SyncService::on_data_imported()` —— 同步启用过就做一次全量对账 （把导入的内容发布进引擎），否则导入的东西在下次同步前对其它设备不可见； -- 覆盖恢复删除本地书 / 书源**之前**调用同步的删除钩子（删除要读 `bookdetail.json` 定位身份）， +- 覆盖恢复删除本地书 / 书源**之前**调用同步的删除钩子（删除要读书籍元信息定位身份）， 让引擎里留下墓碑，避免对端一改那本书就把它「复活」回本机； - 同步从未启用过（没有引擎）时这些都是空操作。 diff --git a/docs/book-storage.md b/docs/book-storage.md new file mode 100644 index 0000000..0c9be71 --- /dev/null +++ b/docs/book-storage.md @@ -0,0 +1,63 @@ +# 书籍数据存储 + +书籍数据保存在 `storage::data_root` 指定的应用数据根目录下的 `books.sqlite3`。 +前端仍通过原有 `readerx_book_*`、`readerx_bookmarks_*`、`readerx_annotations_*` +command 访问，数据形状和同步协议不变。 + +## 数据边界 + +| 数据 | 存储位置 | +| --- | --- | +| 元信息（含封面字段） | SQLite `books`，按书籍 ID 一行 | +| 章节正文、目录、字数、同步指纹和图片引用名 | SQLite `chapters`，按书籍 ID 和章节顺序一行，cid 建索引 | +| 书签、段落注释 | SQLite `bookmarks` / `annotations`，记录原样保存 | +| 偏好、书架进度、分组、规则等状态 | 原有 `state/.json` | +| 书源与登录态 | 原有 `book_sources/`、`source_sessions/` | +| TTS 缓存 | 原有 `tts-audio//` | +| 章节插图 / PDF 页面图的字节 | 原有 `images/`；数据库只存引用名 | + +元信息和章内结构以 JSON 列保留现有字段形状，但数据库按书 / 章分别存行。 +列表通过索引连接查询元信息和预计算的章节头，不扫描正文。 +按 cid 读取或回写只操作命中的章节，目录标题 / 地址保持当前值；旧 index 仅为兼容提示。 +历史空或重复 cid 可以迁入，存在多个命中时正文补丁明确失败，避免错写。 + +## 事务与迁移 + +`book_store` 保留进程级书库锁，不持锁调用同步引擎。SQLite 启用外键和 +`FULL` 同步，整书替换、正文批次与其指纹 / 图片引用、目录修改均由事务提交。 +删书通过外键级联删除章节、书签和注释；图片继续检查其他书的引用后再清理。 +数据库由每次操作按当前数据根打开，不把测试或同步夹具的另一份数据根混入本机。 +数据库格式使用 `PRAGMA user_version=1`，拒绝更高版本。 + +首次访问逐本导入旧 `books//` 和 `books/.json`，同时处理旧全库书签。 +每本书先完整读取并校验，再在一个事务中提交元信息、正文、书签、注释及迁移回执。 +旧 `digest.json` 不作为权威数据，章节指纹按正文重算。 +成功后书籍 JSON 改名为 `.json.migrated` 留底；失败保留原文件，读路径仍可回退， +其他书照常迁移，后续访问可重试。文件归档中断时，事务内保存的回执阻止旧数据覆盖 +数据库中的新编辑。已有同名留底文件不被覆盖。 + +旧身份迁移待办继续使用可重跑日志；数据库书籍改 ID 在事务内级联更新， +书签中的 bookId 随之更新。书架、规则、阅读时长及 TTS 目录仍使用原迁移流程。 +分组和书源引用同时更新数据库元信息；书源映射在重命名文件前单独留待办，避免中断丢映射。 +损坏的旧元信息使后续引用迁移延后，保留原分组 / 书源及映射，已迁入书籍仍可正常使用。 +书源保存继续由共享书源 crate 写文件，App 同步维护数据库中的书源引用。 + +## 备份与同步 + +备份格式不变：数据库书籍通过 `book_store::export_books` 转成原来的 +`books//{bookdetail,content,bookmarks,annotations}.json` 归档条目。 +正文按章节行写入 zip，不为导出复制整库正文；数据库文件及事务日志不进入归档。 +恢复通过 `book_store::import_book_json` 写入数据库，不在应用目录重建书籍 JSON。 +合并 / 覆盖恢复、缺失注释清空、书签合并和凭据过滤保持原语义。 +有尚未完成迁移的旧书籍时拒绝导出不完整备份。 + +同步元信息、目录、正文指纹、按需取章、封面与插图引用全部经 `book_store`。 +资源字节仍从文件读取，线上消息与协议版本不变。 + +## 验证 + +在 `src-tauri` 运行 `cargo test -p readerx --lib book_store`、 +`cargo test -p readerx --lib sync::book_ids`、`cargo test -p readerx --lib sync::data_ids`、 +`cargo test -p readerx --lib data_transfer`、`cargo test -p readerx --test sync_bridge` 和 +`cargo test -p readerx-sync`。PR 的 Test book storage and data compatibility 工作流执行这些检查。 +跨 Android / Windows 的原生构建和实际双设备同步仍需对应环境验证。 diff --git a/docs/sync.md b/docs/sync.md index 690fecd..d804bf4 100644 --- a/docs/sync.md +++ b/docs/sync.md @@ -1,5 +1,7 @@ # 局域网同步（readerx-sync） +书籍在应用目录使用 SQLite，备份仍为便携 JSON 条目；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 + > 状态：**已接入 App**。本文件说明它解决什么问题、怎么用、边界在哪， > 以及 App 侧的接线方式（`src-tauri/src/sync/` 与 `src/lib/sync.ts`）。 @@ -298,7 +300,7 @@ src/pages/SyncConflicts.tsx 冲突裁决 | 分章规则 | ✅ | 用户自定义的章节标题正则，以**名称 + 正则**为身份 | | 书籍结构 | ✅ | 每本书的章节目录（cid / 标题 / 地址），一本书一份、整份 LWW，见第 7.8 节 | | 书籍正文 | ✅ | 按章指纹对账 + 按需搬运，**不进操作日志**，见第 7.9 节 | -| 封面 | ✅ | 字节走**资源通道**（不进操作日志），本机仍是 `bookdetail.json` 里的一段 data URL，见第 7.10 节 | +| 封面 | ✅ | 字节走**资源通道**（不进操作日志），本机仍是书籍元信息中的一段 data URL，见第 7.10 节 | | 章节插图 | ✅ | 同上；插图还要设备无关的文件名，见第 7.11 节 | | 未下载的插图 | ✅ | 正文里的引用会同步过去，字节由资源通道按引用补齐（两轮到位） | | 导入时间 / 色相 | ❌ | 每台设备各自生成的，同步过去只会制造无意义的差异 | @@ -330,7 +332,7 @@ App 书库、书源、书架分组、书源分组及规则与同步使用同一 原样保留，已有同步实体、操作历史及墓碑不重建。迁移待办先写盘，中断后继续执行。 遇到重复同步身份或状态键冲突时保留原数据并返回错误，避免覆盖用户数据。 -同一启动阶段还迁移书源、两类分组及规则 ID，更新 `bookdetail.json` 的 +同一启动阶段还迁移书源、两类分组及规则 ID，更新书籍元信息的 `bookSourceId` / `groupId` 和书源的 `groupId`。书源 ID 同时用于源文件、同步实体与 在线书关联；新同步载荷携带 `book_source_id`，旧载荷保留按地址查找的兼容路径。 登录态 `source_sessions/` 与 CLI 登录配置 `profiles/` 跟随书源 ID 搬迁。 @@ -410,7 +412,7 @@ cargo test -p readerx-sync # 引擎与协议（含 App 场景回 cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock AppHandle ``` -`tests/sync_bridge.rs` 覆盖的是单元测试够不到的那一层：真实 `bookdetail.json` / +`tests/sync_bridge.rs` 覆盖的是单元测试够不到的那一层：真实 SQLite 书籍数据 / `bookmarks.json` / `readerx.shelf.json` / 书源文件与引擎实体之间的双向搬运 （首次对账、落地、重新导入、对端删书、冲突留档、规则类数据、目录落地时按 cid 保留正文、 对端正文落地时新建本地书、新设备主动拉取 TXT / EPUB / MOBI / PDF 与扫描 PDF 页面图），以及 App 删除设备后对方真的连不进来。 @@ -466,7 +468,7 @@ cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock 要搬的东西。 ```text -本机：ContentSource（App 读 books//content.json 的章节指纹缓存 digest.json） +本机：ContentSource（App 读 SQLite chapters 表中的章节指纹） + 暂存区（<同步目录>/content/<书实体 id>/，对端刚发来、还没落地） 一次会话：逐本比总览 → 不一样的书比逐章指纹 → 推「对端没有的」、拉「本机没有的」 落地：宿主在下一次 materialize 里把暂存正文写进书库（必要时按元信息新建这本书） @@ -484,7 +486,7 @@ cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock 对端没有正文来源（旧版本 / CLI）时直接跳过这条通道 —— 既不向它发未知请求 （旧版本会当协议错误断连接），也不会把正文搬到它那里去。 - **落地即建书**：本机还没有这本书时按引擎里的元信息 + 目录 + 暂存正文新建一本 - （`bookdetail.json` + `content.json`）。两种情况**暂不建书**、正文继续留在暂存区： + （SQLite 元信息与章节行）。两种情况**暂不建书**、正文继续留在暂存区： 元信息还没同步过来、在线书的本机没有对应书源（书身份含书源地址，缺了会建成同步不到的 孤儿）。PDF 与其他导入书一样，根据解析后的文字块 / 页面图片建书。封面与插图走资源通道 （第 7.10 节）单独搬，不阻塞建书；封面还没到时底色由书实体 id 派生。 @@ -529,7 +531,7 @@ cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock - **暂存区的粒度是「书的实体 id + 类别 + 名字」**：`assets/<书实体 id>/<类别:名字 的 sha256>.json`， 内容就是一份序列化后的资源（字节编 base64）。落地成功才删暂存文件，写失败留着下次再试； 对端删书时，它还没落地的资源一起丢掉。 -- **封面就是一段 data URL**：同步不改变本机的存储形态 —— 本机 `bookdetail.json` 里的 +- **封面就是一段 data URL**：同步不改变本机的存储形态 —— 本机书籍元信息里的 `cover` 还是 data URL，界面那一侧一个字节都不用动。因此封面资源会额外带上**原文** （`dataUrl`），落地时原样写回；只用「字节 + MIME」拼回来，只要差一个字符就会被当成 「封面又变了」而反复写盘。封面不参与实体（操作日志）里的 `cover` 字段，那个字段仍然 diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index adf8daf..2d7489f 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -1577,6 +1577,18 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "fallible-iterator" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2acce4a10f12dc2fb14a218589d4f1f62ef011b2d0cc4b3cb1bba8e94da14649" + +[[package]] +name = "fallible-streaming-iterator" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a" + [[package]] name = "fast-float2" version = "0.2.4" @@ -2123,6 +2135,15 @@ version = "0.14.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +[[package]] +name = "hashbrown" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" +dependencies = [ + "foldhash", +] + [[package]] name = "hashbrown" version = "0.17.1" @@ -2134,6 +2155,15 @@ dependencies = [ "foldhash", ] +[[package]] +name = "hashlink" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824e001ac4f3012dd16a264bec811403a67ca9deb6c102fc5049b32c4574b35f" +dependencies = [ + "hashbrown 0.16.1", +] + [[package]] name = "heck" version = "0.4.1" @@ -2768,6 +2798,17 @@ dependencies = [ "libc", ] +[[package]] +name = "libsqlite3-sys" +version = "0.36.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "95b4103cffefa72eb8428cb6b47d6627161e51c2739fc5e3b734584157bc642a" +dependencies = [ + "cc", + "pkg-config", + "vcpkg", +] + [[package]] name = "linux-raw-sys" version = "0.12.1" @@ -3882,6 +3923,7 @@ dependencies = [ "readerx-source", "readerx-sync", "reqwest 0.12.28", + "rusqlite", "semver", "serde", "serde_json", @@ -4146,6 +4188,31 @@ dependencies = [ "windows-sys 0.52.0", ] +[[package]] +name = "rsqlite-vfs" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c51c9ae4df8a7fba42103df5c621fa3c37eccf3a3c650879e90fc48b11cc192c" +dependencies = [ + "hashbrown 0.16.1", + "thiserror 2.0.20", +] + +[[package]] +name = "rusqlite" +version = "0.38.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1c93dd1c9683b438c392c492109cb702b8090b2bfc8fed6f6e4eb4523f17af3" +dependencies = [ + "bitflags 2.13.1", + "fallible-iterator", + "fallible-streaming-iterator", + "hashlink", + "libsqlite3-sys", + "smallvec", + "sqlite-wasm-rs", +] + [[package]] name = "rustc-hash" version = "2.1.3" @@ -4682,6 +4749,18 @@ dependencies = [ "bindgen", ] +[[package]] +name = "sqlite-wasm-rs" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc3efc0da82635d7e1ced0053bbbfa8c7ab9645d0bf36ceb4f7127bb85315d75" +dependencies = [ + "cc", + "js-sys", + "rsqlite-vfs", + "wasm-bindgen", +] + [[package]] name = "stable_deref_trait" version = "1.2.1" @@ -5893,6 +5972,12 @@ dependencies = [ "wasm-bindgen", ] +[[package]] +name = "vcpkg" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" + [[package]] name = "version-compare" version = "0.2.1" diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index b1f6882..eb5af36 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -55,6 +55,8 @@ readerx-log = { path = "crates/readerx-log" } log = "0.4" serde = { version = "1", features = ["derive"] } serde_json = "1" +# 仅书籍数据入库；bundled 为 Android / Linux / Windows 编译 SQLite。 +rusqlite = { version = "0.38", features = ["bundled"] } base64 = "0.22" # 跨设备书身份用 sha256 派生（见 src/sync/identity.rs）：同一个文件在两台设备上 # 要算出同一个 id，不能用进程内随机种子的哈希。 diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index 59d6073..6babc93 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -1,29 +1,7 @@ -//! 本地书存储：`books//` 目录布局（元信息 / 正文 / 书签分文件）。 -//! -//! ```text -//! <应用数据目录>/books// -//! bookdetail.json 元信息（书名、作者、封面、分组、标签…，不含正文） -//! content.json 章节正文 -//! bookmarks.json 该书书签 -//! annotations.json 按段落聚合的注释 -//! digest.json 章节正文指纹缓存（同步对账用；派生数据，删了会自动重建） -//! ``` -//! -//! **为什么拆开**：原先整本只有一个 `books/.json`，改一个分组名也要把几百 MB 的 -//! 正文读回来再整本写一遍；书签更是全库挤在一个 `state/readerx.bookmarks.json` 里， -//! 删一本书的书签得把所有书的书签重写一遍。拆开后每类数据各写各的文件，互不牵连： -//! 元信息补丁只动 `bookdetail.json`，逐章回写只动 `content.json`，书签只动 `bookmarks.json`。 -//! -//! 三个文件都带 `schemaVersion` 信封，便于以后再改格式时识别版本； -//! 书签记录的结构由前端定义，这里只做信封与存取，不解析记录内容（免得以后加字段被后端丢弃）。 -//! -//! **旧布局迁移**（[`migrate_legacy_layout`]，进程内只跑一次、幂等）： -//! - `books/.json`（整本）→ `books//{bookdetail,content}.json`，成功后才删旧文件； -//! - `state/readerx.bookmarks.json`（全库书签一处）→ 各书 `bookmarks.json`， -//! 全部迁完后旧文件改名为 `.migrated` 留底。 -//! -//! 迁移没成功的书仍可按旧布局读取（[`get_book`] / [`list_book_meta`] 留有回退）， -//! 不会因为一次写盘失败就从书架上凭空消失。 +//! 本地书籍数据存储在应用数据根的 `books.sqlite3`。 +//! 元信息、逐章正文与派生指纹、书签和注释经 SQLite 事务读写。 +//! 旧目录 / 整书 JSON 逐本迁移，成功后改名 `.migrated` 留底，失败保留读回退。 +//! TTS、书源、状态 JSON 与图片字节保持文件存储；备份仍采用原 JSON 归档格式。 use crate::models::{ BookChapterPatch, BookMeta, BookMetaPatch, ChapterBlock, ChapterHead, LocalBook, @@ -35,14 +13,19 @@ use serde::{Deserialize, Serialize}; use serde_json::Value; use std::collections::{HashMap, HashSet}; use std::fs; -use std::io::{BufReader, BufWriter}; +use std::io::BufReader; +#[cfg(test)] +use std::io::BufWriter; use std::path::{Path, PathBuf}; -use std::sync::{Mutex, OnceLock}; +use std::sync::Mutex; use tauri::AppHandle; -/// 元信息文件 +#[path = "book_store/sqlite.rs"] +mod sqlite; + +/// 旧布局 / 便携备份的元信息文件名 const BOOKDETAIL_FILE: &str = "bookdetail.json"; -/// 正文文件 +/// 旧布局 / 便携备份的正文文件名 const CONTENT_FILE: &str = "content.json"; /// 段落注释文件 const ANNOTATIONS_FILE: &str = "annotations.json"; @@ -217,6 +200,7 @@ fn read_annotations_file(path: &Path) -> Result, String> { } Ok(file.annotations) } +#[cfg(test)] fn write_annotations_file(path: &Path, annotations: &[Value]) -> Result<(), String> { write_json_atomic( path, @@ -338,54 +322,8 @@ fn scan_chapter_head(chapter: ChapterScan) -> ChapterHead { // 路径与落盘 // --------------------------------------------------------------------------- -/// `books/` 根目录（不存在则创建） -fn books_root(app: &AppHandle) -> Result { - let dir = crate::storage::data_root(app)?.join("books"); - crate::storage::ensure_dir(&dir)?; - Ok(dir) -} - -/// 单本书的目录路径。**不创建目录**：读路径不该给不存在的书留下空目录。 -fn book_dir(app: &AppHandle, id: &str) -> Result { - book_dir_at(app, None, id) -} - -/// 同 [`book_dir`]，但允许指定数据根（见 `storage::data_root_at`）。 -fn book_dir_at( - app: &AppHandle, - root: Option<&Path>, - id: &str, -) -> Result { - if !crate::storage::valid_component(id) { - return Err("非法的书籍 id".to_string()); - } - Ok(crate::storage::data_root_at(app, root)?.join("books").join(id)) -} - -/// 旧布局的整书文件路径：`books/.json` -fn legacy_book_file(app: &AppHandle, id: &str) -> Result { - if !crate::storage::valid_component(id) { - return Err("非法的书籍 id".to_string()); - } - Ok(crate::storage::data_root(app)?.join("books").join(format!("{id}.json"))) -} - -/// 写路径用的书籍目录：不存在则建；调用方已持事务锁并完成旧布局迁移。 -fn ensure_book_dir(app: &AppHandle, id: &str) -> Result { - let dir = book_dir(app, id)?; - if dir.is_dir() { - return Ok(dir); - } - if legacy_book_file(app, id)?.is_file() { - // 旧文件还在说明这本没迁移成功（畸形文件 / 写盘失败）：不能当它不存在， - // 否则会凭空造出一个没有正文的空书目录，把旧数据挡在外面 - return Err("书籍数据迁移失败，请查看应用日志".to_string()); - } - crate::storage::ensure_dir(&dir)?; - Ok(dir) -} - /// 原子写 JSON：先写临时文件再替换（中途失败 / 进程被杀不会留下半截文件） +#[cfg(test)] fn write_json_atomic(path: &Path, value: &T, what: &str) -> Result<(), String> { let tmp = path.with_extension("json.tmp"); let _temporary = crate::temporary_file::TemporaryFile(tmp.clone()); @@ -394,7 +332,10 @@ fn write_json_atomic(path: &Path, value: &T, what: &str) -> Result let mut writer = BufWriter::new(file); serde_json::to_writer(&mut writer, value).map_err(|e| format!("序列化{what}失败: {e}"))?; std::io::Write::flush(&mut writer).map_err(|e| format!("写入{what}失败: {e}"))?; - writer.get_ref().sync_all().map_err(|e| format!("写入{what}失败: {e}"))?; + writer + .get_ref() + .sync_all() + .map_err(|e| format!("写入{what}失败: {e}"))?; } fs::rename(&tmp, path).map_err(|e| format!("写入{what}失败: {e}")) } @@ -424,32 +365,6 @@ fn scan_json_file(path: &Path) -> Result { } } -/// 书籍文件较大时先做一次「流式图片迁移」：把旧数据里以 data URL 形式塞进章节的图片 -/// 抽成本地文件（内存占用只与单张图片同级）。这样后续「读回整本再解析」不会为了 -/// 几百 MB 的 base64 把内存打满 —— 旧版本正是因此在大图片书籍上直接闪退。 -/// 迁移失败（畸形文件等）只记日志、不影响原路径读取。 -fn stream_migrate_if_large(app: &AppHandle, id: &str, path: &Path) { - let Ok(metadata) = fs::metadata(path) else { - return; - }; - if metadata.len() < crate::book_images::STREAM_MIGRATE_MIN_BYTES { - return; - } - // 没有内嵌图片（纯文本巨书）就不必整本重写一遍 - if !crate::book_images::file_has_data_image(path) { - return; - } - let Ok(root) = crate::book_images::images_root(app) else { - log::warn!("书籍图片迁移无法定位图片目录（{id}），本次跳过"); - return; - }; - match crate::book_images::migrate_book_file(&root, id, path) { - Ok(true) => log::info!("书籍图片迁移完成 id={id}（data URL 已抽成本地文件）"), - Ok(false) => {} - Err(error) => log::warn!("书籍图片迁移跳过（{id}）: {error}"), - } -} - // --------------------------------------------------------------------------- // 目录布局的读写（纯路径，便于单测） // --------------------------------------------------------------------------- @@ -459,6 +374,7 @@ fn stream_migrate_if_large(app: &AppHandle, id: &str, path /// **先写正文再写元信息**：书名等信息是「书已在书架上」的标志（列表按 bookdetail.json /// 判定），正文先落盘可以保证「列表里能看到的书一定有正文」；反过来写就会出现 /// 一本点开是空白的书。 +#[cfg(test)] fn write_book_files(dir: &Path, book: LocalBook) -> Result<(), String> { let detail = BookDetail::from_book(&book); let content = BookContent { @@ -471,12 +387,14 @@ fn write_book_files(dir: &Path, book: LocalBook) -> Result<(), String> { } /// 只回写正文（逐章回写用） +#[cfg(test)] fn write_book_content(dir: &Path, chapters: &[LocalBookChapter]) -> Result<(), String> { write_book_content_with_digests(dir, chapters, None) } /// 写正文 + 章节指纹缓存：`digests` 为 `Some` 时直接用调用方算好的（逐章回写路径 /// 只重算改动的那几章），`None` 时整本重算。 +#[cfg(test)] fn write_book_content_with_digests( dir: &Path, chapters: &[LocalBookChapter], @@ -493,7 +411,9 @@ fn write_book_content_with_digests( let mut assets = read_digest_assets(dir, chapters.len()); for (index, chapter) in chapters.iter().enumerate() { if let Some(slot) = assets.get_mut(index) { - *slot = ChapterAssets { locals: chapter_asset_locals(chapter) }; + *slot = ChapterAssets { + locals: chapter_asset_locals(chapter), + }; } } write_digest_file_with(dir, digests, assets) @@ -529,11 +449,13 @@ fn read_meta_from_dir(dir: &Path) -> Result, String> { let detail: BookDetail = read_json_file(&detail_path, "书籍元信息")?; let content_path = dir.join(CONTENT_FILE); let chapters = if content_path.is_file() { - scan_json_file::(&content_path)? - .chapters - .into_iter() - .map(scan_chapter_head) - .collect() + match scan_json_file::(&content_path) { + Ok(scan) => scan.chapters.into_iter().map(scan_chapter_head).collect(), + Err(error) => { + log::warn!("旧书籍正文损坏，保留书架元信息：{error}"); + Vec::new() + } + } } else { Vec::new() }; @@ -542,7 +464,12 @@ fn read_meta_from_dir(dir: &Path) -> Result, String> { /// 旧布局整书文件 → 目录布局（幂等）：先写齐两个新文件，成功后才删旧文件。 /// 任何一步失败都保留旧文件，调用方下次仍能按旧布局读出来。 -fn convert_legacy_book(dir: &Path, legacy: &Path, images_root: Option<&Path>) -> Result<(), String> { +#[cfg(test)] +fn convert_legacy_book( + dir: &Path, + legacy: &Path, + images_root: Option<&Path>, +) -> Result<(), String> { let mut book: LocalBook = read_json_file(legacy, "书籍")?; // 旧数据里的 data URL 图片：迁移机会只有这一次，顺手抽成文件 if let Some(root) = images_root { @@ -567,6 +494,7 @@ fn read_bookmarks_file(path: &Path) -> Result, String> { Ok(file.bookmarks) } +#[cfg(test)] fn write_bookmarks_file(path: &Path, bookmarks: &[Value]) -> Result<(), String> { let file = BookmarkFile { schema_version: SCHEMA_VERSION, @@ -579,19 +507,6 @@ fn write_bookmarks_file(path: &Path, bookmarks: &[Value]) -> Result<(), String> // 旧布局迁移 // --------------------------------------------------------------------------- -/// 进程内迁移账本：串行化迁移，并记住这次已经试过的书 -/// （迁移失败的书仍按旧布局读，但不反复重读同一个坏文件）。 -#[derive(Default)] -struct LegacyMigration { - done: bool, - attempted: HashSet, -} - -fn legacy_migration() -> &'static Mutex { - static STATE: OnceLock> = OnceLock::new(); - STATE.get_or_init(|| Mutex::new(LegacyMigration::default())) -} - /// 所有书库入口共用事务锁；不持锁调用同步引擎，避免引擎与磁盘锁顺序反转。 /// 读操作也参与，保证元信息、正文和派生缓存来自同一次完整写入。 fn library_transaction() -> std::sync::MutexGuard<'static, ()> { @@ -599,78 +514,10 @@ fn library_transaction() -> std::sync::MutexGuard<'static, ()> { LOCK.lock().unwrap_or_else(|error| error.into_inner()) } -/// 旧布局迁移（幂等）。每次运行最多走一遍：单本书失败只记日志并把旧文件留着 -/// （该书的列表与阅读都留有旧布局回退），不在这次运行里反复重试 —— -/// 失败的书已记进账本，下次启动（新的账本）才会再试。 -/// 各书籍 / 书签读写入口都会先调用它兜底，因此不必在别处手动触发。 -fn migrate_legacy_layout(app: &AppHandle) { - let Ok(mut state) = legacy_migration().lock() else { - // 锁中毒：别的线程在迁移中 panic 了，本次跳过（下次启动再来） - return; - }; - if state.done { - return; - } - state.done = true; - if let Err(error) = migrate_books(app, &mut state) { - log::warn!("旧书籍布局迁移未完成（下次启动继续）: {error}"); - } else if let Err(error) = migrate_legacy_bookmarks(app) { - log::warn!("旧书签迁移未完成（下次启动继续）: {error}"); - } -} - -/// `books/.json` → `books//{bookdetail,content}.json` -fn migrate_books( - app: &AppHandle, - state: &mut LegacyMigration, -) -> Result<(), String> { - let dir = books_root(app)?; - let images = crate::book_images::images_root(app).ok(); - for entry in fs::read_dir(&dir).map_err(|e| format!("读取书库失败: {e}"))?.flatten() { - let path = entry.path(); - if path.extension().and_then(|s| s.to_str()) != Some("json") { - continue; - } - let Some(id) = path.file_stem().and_then(|s| s.to_str()) else { - continue; - }; - if state.attempted.contains(id) { - continue; - } - state.attempted.insert(id.to_string()); - // 大文件先流式抽图，避免整本 base64 进内存 - stream_migrate_if_large(app, id, &path); - if let Err(error) = convert_legacy_book(&dir.join(id), &path, images.as_deref()) { - log::warn!("书籍旧布局迁移失败（{id}），保留原文件: {error}"); - continue; - } - log::info!("书籍已迁移到目录布局 id={id}"); - } - Ok(()) -} - -/// `state/readerx.bookmarks.json`（全库书签一处）→ 各书 `books//bookmarks.json`。 -/// 全部能迁的都迁完之后，旧文件改名为 `.migrated` 留底（不再参与读取）。 -fn migrate_legacy_bookmarks(app: &AppHandle) -> Result { - let legacy = crate::storage::state_dir(app)?.join("readerx.bookmarks.json"); - if !legacy.is_file() { - return Ok(0); - } - let books = crate::storage::data_root(app)?.join("books"); - let (migrated, pending) = split_legacy_bookmarks(&legacy, &books)?; - if pending > 0 { - // 有书还没迁过来（迁移失败）：整份旧文件留着，下次继续 - return Err(format!("还有 {pending} 本书未完成迁移")); - } - let backup = legacy.with_extension("json.migrated"); - fs::rename(&legacy, &backup).map_err(|e| format!("归档旧书签文件失败: {e}"))?; - log::info!("书签已迁移为每本书独立文件 books={migrated}"); - Ok(migrated) -} - /// 把「全库一份」的旧书签按书拆开写进各自的 `books//bookmarks.json`（幂等）。 /// 返回（已写入的书数，因书籍目录还不存在而暂时跳过的书数）——调用方据此决定 /// 能否归档旧文件：只要还有书没迁过来，旧文件就不能动。 +#[cfg(test)] fn split_legacy_bookmarks(legacy: &Path, books_dir: &Path) -> Result<(u64, u64), String> { let text = fs::read_to_string(legacy).map_err(|e| format!("读取旧书签失败: {e}"))?; let map: HashMap> = @@ -708,85 +555,73 @@ fn split_legacy_bookmarks(legacy: &Path, books_dir: &Path) -> Result<(u64, u64), // 对外读写 // --------------------------------------------------------------------------- -/// 读整本书。目录布局优先；迁移没成功的旧文件仍可读（内容不丢）。 +/// 读整本书。数据库优先；迁移失败时保留旧 JSON 读回退。 pub(crate) fn get_book( - app: &AppHandle, id: &str, + app: &AppHandle, + id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - if let Some(mut book) = read_book_from_dir(&dir)? { - // 图片自愈：本地副本不在了就清引用、旧数据里的 data URL 抽成文件（有改动才回写） - if let Ok(root) = crate::book_images::images_root(app) { - if crate::book_images::migrate_chapters(&root, id, &mut book.chapters) { - log::info!("书籍图片旧数据迁移完成 id={id}"); - write_book_content(&dir, &book.chapters)?; + sqlite::valid_id(id)?; + let root = crate::storage::data_root(app)?; + match sqlite::open(&root) { + Ok(mut db) => { + if let Some(mut book) = sqlite::get(&db, id)? { + if crate::book_images::migrate_chapters( + &root.join("images"), + id, + &mut book.chapters, + ) { + let tx = sqlite::transaction(&mut db)?; + sqlite::put(&tx, &book)?; + sqlite::commit(tx)?; + } + return Ok(Some(book)); } } - return Ok(Some(book)); - } - let legacy = legacy_book_file(app, id)?; - if !legacy.is_file() { - return Ok(None); + Err(error) => { + if !root.join("books").join(id).join(BOOKDETAIL_FILE).is_file() + && !root.join("books").join(format!("{id}.json")).is_file() + { + return Err(error); + } + log::warn!("数据库读取失败，使用未迁移书籍 id={id}"); + } } - stream_migrate_if_large(app, id, &legacy); - let book = read_json_file(&legacy, "书籍")?; - log::debug!("按旧布局读取书籍 id={id}"); - Ok(Some(book)) + legacy_book_at(&root, id) } /// 整本写入（导入 / 在线书整本替换）。 pub(crate) fn put_book( - app: &AppHandle, mut book: LocalBook, + app: &AppHandle, + mut book: LocalBook, ) -> Result<(), String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = ensure_book_dir(app, &book.id)?; - if let Ok(root) = crate::book_images::images_root(app) { - crate::book_images::migrate_book(&root, &mut book); + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + sqlite::valid_id(&book.id)?; + if sqlite::detail(&db, &book.id)?.is_none() && legacy_book_at(&root, &book.id)?.is_some() { + return Err("书籍数据迁移失败，请查看应用日志".into()); } - write_book_files(&dir, book) + crate::book_images::migrate_book(&root.join("images"), &mut book); + let tx = sqlite::transaction(&mut db)?; + sqlite::put(&tx, &book)?; + sqlite::commit(tx) } -/// 只回写一本书的若干章节（在线书逐批下载正文用）： -/// 只读改 `content.json`，元信息一个字节都不碰（原先要把整本读回来再整本写）。 +/// 按 cid 增量回写章节正文；批次及字数 / 指纹 / 图片引用在同一事务内提交。 pub(crate) fn put_book_chapters( app: &AppHandle, id: &str, updates: &[BookChapterPatch], ) -> Result<(), String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - let path = dir.join(CONTENT_FILE); - if !path.is_file() { - return Err(missing_book_error(app, id)?); - } - stream_migrate_if_large(app, id, &path); - let mut content: BookContent = read_json_file(&path, "书籍正文")?; - // 章节指纹缓存：这里只更新被回写的这几章（整本重算 = 把几百 MB 正文重新哈希一遍， - // 在线书逐批下载会一批调用一次）；缓存对不上号时退回整本重算。 - let mut digests = read_digest_file(&dir).ok().filter(|digests| { - digests.len() == content.chapters.len() - && digests - .iter() - .zip(content.chapters.iter()) - .all(|(digest, chapter)| digest.cid == chapter.cid) - }); - let indices = patch_chapters(&mut content.chapters, updates)?; - // 迁移结果只在内存里，写盘时自然落定；写盘失败也不影响本次章节回写的数据 - if let Ok(root) = crate::book_images::images_root(app) { - crate::book_images::migrate_chapters(&root, id, &mut content.chapters); - } - if let Some(digests) = digests.as_mut() { - for index in indices { - digests[index] = chapter_digest(&content.chapters[index]); - } - } - write_book_content_with_digests(&dir, &content.chapters, digests) + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + sqlite::patch(&mut db, &root, id, updates) } /// 下标仅是旧客户端的提示；身份必须以 cid 为准，目录变更不能改变正文归属。 +#[cfg(test)] fn patch_chapters( chapters: &mut [LocalBookChapter], updates: &[BookChapterPatch], @@ -817,7 +652,7 @@ fn patch_chapters( Ok(indices) } -/// 只改元信息（分组 / 书名 / 封面 / 标签…）：只读写 `bookdetail.json`， +/// 只改元信息（分组 / 书名 / 封面 / 标签…）：只读写 books 行， /// 几百 MB 的正文不必读回来，也不经过 IPC。 pub(crate) fn patch_book_meta( app: &AppHandle, @@ -825,37 +660,51 @@ pub(crate) fn patch_book_meta( patch: &BookMetaPatch, ) -> Result<(), String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let path = book_dir(app, id)?.join(BOOKDETAIL_FILE); - if !path.is_file() { - return Err(missing_book_error(app, id)?); - } - let detail: BookDetail = read_json_file(&path, "书籍元信息")?; - // 借用 LocalBook 的补丁语义，保证与旧实现逐字段一致（正文不参与） - let mut book = detail.into_book(Vec::new()); - if !patch.apply_to(&mut book) { - return Ok(()); + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + sqlite::require_book(&db, &root, id)?; + let tx = sqlite::transaction(&mut db)?; + let mut book = sqlite::detail(&tx, id)? + .ok_or("书籍不存在")? + .into_book(Vec::new()); + if patch.apply_to(&mut book) { + sqlite::save_detail(&tx, &BookDetail::from_book(&book))?; } - write_json_atomic(&path, &BookDetail::from_book(&book), "书籍元信息") -} - -/// 「书籍不存在」的具体原因：旧文件还在说明是迁移没成功（畸形文件 / 写盘失败）， -/// 直接说「不存在」会让用户以为是书被删了。 -fn missing_book_error(app: &AppHandle, id: &str) -> Result { - if legacy_book_file(app, id)?.is_file() { - return Ok("书籍数据迁移失败，请查看应用日志".to_string()); - } - Ok("书籍不存在".to_string()) + sqlite::commit(tx) } /// 读取某本书的书签（书不存在 / 没有书签都返回空列表） pub(crate) fn get_bookmarks( - app: &AppHandle, id: &str, + app: &AppHandle, + id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let path = book_dir(app, id)?.join(BOOKMARKS_FILE); - read_bookmarks_file(&path) + sqlite::valid_id(id)?; + let root = crate::storage::data_root(app)?; + match sqlite::open(&root) { + Ok(db) if sqlite::detail(&db, id)?.is_some() => sqlite::records(&db, "bookmarks", id), + Ok(_) => legacy_bookmarks_at(&root, id), + Err(error) => { + let path = root.join("books").join(id).join(BOOKMARKS_FILE); + if !path.is_file() && !root.join("state/readerx.bookmarks.json").is_file() { + return Err(error); + } + legacy_bookmarks_at(&root, id) + } + } +} + +fn legacy_bookmarks_at(root: &Path, id: &str) -> Result, String> { + let path = root.join("books").join(id).join(BOOKMARKS_FILE); + if path.is_file() { + return read_bookmarks_file(&path); + } + let global = root.join("state/readerx.bookmarks.json"); + if global.is_file() { + let mut records: HashMap> = read_json_file(&global, "旧书签")?; + return Ok(records.remove(id).unwrap_or_default()); + } + Ok(Vec::new()) } /// 覆盖式写入某本书的书签 @@ -865,14 +714,17 @@ pub(crate) fn put_bookmarks( bookmarks: &[Value], ) -> Result<(), String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - if !dir.is_dir() { - // 书已经不在了（删书与写书签撞上）：不落盘，也不留下一个只有书签的孤儿目录 - log::debug!("书籍不存在，书签未写入 id={id}"); + let root = crate::storage::data_root(app)?; + let db = sqlite::open(&root)?; + if sqlite::detail(&db, id)?.is_none() { + if root.join("books").join(id).join(BOOKDETAIL_FILE).is_file() + || root.join("books").join(format!("{id}.json")).is_file() + { + sqlite::require_book(&db, &root, id)?; + } return Ok(()); } - write_bookmarks_file(&dir.join(BOOKMARKS_FILE), bookmarks) + sqlite::put_records(&db, "bookmarks", id, bookmarks) } pub(crate) fn get_annotations( @@ -880,8 +732,19 @@ pub(crate) fn get_annotations( id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - read_annotations_file(&book_dir(app, id)?.join(ANNOTATIONS_FILE)) + sqlite::valid_id(id)?; + let root = crate::storage::data_root(app)?; + match sqlite::open(&root) { + Ok(db) if sqlite::detail(&db, id)?.is_some() => sqlite::records(&db, "annotations", id), + Ok(_) => read_annotations_file(&root.join("books").join(id).join(ANNOTATIONS_FILE)), + Err(error) => { + let path = root.join("books").join(id).join(ANNOTATIONS_FILE); + if !path.is_file() { + return Err(error); + } + read_annotations_file(&path) + } + } } pub(crate) fn put_annotations( app: &AppHandle, @@ -889,44 +752,87 @@ pub(crate) fn put_annotations( annotations: &[Value], ) -> Result<(), String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - if !dir.is_dir() { - return Err("书籍不存在".into()); - } - write_annotations_file(&dir.join(ANNOTATIONS_FILE), annotations) + let root = crate::storage::data_root(app)?; + let db = sqlite::open(&root)?; + sqlite::require_book(&db, &root, id)?; + sqlite::put_records(&db, "annotations", id, annotations) } /// 前端快照只用于算差集，读改写在同一书库事务内保留刚落地的远端记录。 pub(crate) fn patch_annotations( - app: &AppHandle, id: &str, previous: &[Value], next: &[Value], + app: &AppHandle, + id: &str, + previous: &[Value], + next: &[Value], ) -> Result, String> { let changed = crate::annotations::changed(previous, next)?; let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - if !dir.is_dir() { return Err("书籍不存在".into()); } - let path = dir.join(ANNOTATIONS_FILE); - let mut current = read_annotations_file(&path)?; + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + sqlite::require_book(&db, &root, id)?; + let tx = sqlite::transaction(&mut db)?; + let mut current = sqlite::records(&tx, "annotations", id)?; let existing = crate::annotations::flatten(¤t)?; - for note in existing.values() { crate::annotations::upsert(&mut current, note); } - for note in changed.values() { crate::annotations::upsert(&mut current, note); } - write_annotations_file(&path, ¤t)?; + for note in existing.values() { + crate::annotations::upsert(&mut current, note); + } + for note in changed.values() { + crate::annotations::upsert(&mut current, note); + } + sqlite::put_records(&tx, "annotations", id, ¤t)?; + sqlite::commit(tx)?; Ok(current) } -/// 删除一本书：整个书籍目录（含书签）连同听书缓存、章节插图一起清掉。 +/// 删除一本书：数据库级联清理，文件缓存和独占图片仍在文件层清理。 pub(crate) fn delete_book(app: &AppHandle, id: &str) -> Result<(), String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - if !crate::storage::valid_component(id) { - return Err("非法的书籍 id".to_string()); + let root = crate::storage::data_root(app)?; + let db = sqlite::open(&root)?; + sqlite::valid_id(id)?; + let mut locals: Vec<_> = sqlite::assets(&db, id)? + .into_iter() + .map(|(_, local)| local) + .collect(); + // Remove failed legacy inputs before the row, so a failed cleanup cannot resurrect the book. + let books = root.join("books"); + for path in [ + books.join(id).join(CONTENT_FILE), + books.join(format!("{id}.json")), + ] { + if let Ok(names) = image_locals_from_file(&path) { + locals.extend(names); + } + } + for path in [books.join(id), books.join(format!("{id}.json"))] { + if path.is_dir() { + fs::remove_dir_all(path).map_err(|e| e.to_string())?; + } else if path.is_file() { + fs::remove_file(path).map_err(|e| e.to_string())?; + } + } + sqlite::delete(&db, id)?; + let references = (|| { + let mut references = remaining_image_locals(&books)?; + for detail in sqlite::details(&db)? { + references.extend( + sqlite::assets(&db, &detail.id)? + .into_iter() + .map(|(_, local)| local), + ); + } + Ok::<_, String>(references) + })(); + match references { + Ok(references) => { + crate::book_images::remove_book_images(&root.join("images"), &locals, |name| { + !references.contains(name) + }); + } + Err(error) => log::warn!("删书后插图引用检查失败，保留图片：{error}"), } - let books = books_root(app)?; - let images = crate::book_images::images_root(app).ok(); - delete_book_files(&books, images.as_deref(), id)?; crate::storage::remove_book_tts_cache(app, id)?; - log::info!("书籍已删除 id={id}（含书签、听书缓存与章节插图）"); + log::info!("书籍已删除 id={id}（含书签、注释与听书缓存）"); Ok(()) } @@ -942,24 +848,22 @@ pub(crate) fn normalize_book_images( id: &str, ) -> Result { let _transaction = library_transaction(); - let dir = book_dir_at(app, root, id)?; - if !dir.join(BOOKDETAIL_FILE).is_file() { - return Ok(false); - } - let content_path = dir.join(CONTENT_FILE); - if !content_path.is_file() { + let root = crate::storage::data_root_at(app, root)?; + let mut db = sqlite::open(&root)?; + let Some(mut book) = sqlite::get(&db, id)? else { return Ok(false); - } - let mut content: BookContent = read_json_file(&content_path, "书籍正文")?; - let images_root = crate::book_images::images_root_at(app, root)?; - if !crate::book_images::normalize_image_refs(&images_root, &mut content.chapters) { + }; + if !crate::book_images::normalize_image_refs(&root.join("images"), &mut book.chapters) { return Ok(false); } - write_book_content(&dir, &content.chapters)?; + let tx = sqlite::transaction(&mut db)?; + sqlite::put(&tx, &book)?; + sqlite::commit(tx)?; Ok(true) } /// 先保存待删书的插图引用，再删书籍文件；只清理其余书籍不再引用的文件。 +#[cfg(test)] fn delete_book_files(books: &Path, images: Option<&Path>, id: &str) -> Result<(), String> { let dir = books.join(id); let legacy = books.join(format!("{id}.json")); @@ -1049,6 +953,9 @@ fn image_locals_from_file(path: &Path) -> Result, String> { /// 删除待删书之后扫描剩余引用，两种布局都认；旧文件仍存在时也保留它的引用。 fn remaining_image_locals(books: &Path) -> Result, String> { + if !books.exists() { + return Ok(HashSet::new()); + } let entries = fs::read_dir(books).map_err(|e| format!("读取书库失败: {e}"))?; let mut referenced = HashSet::new(); for entry in entries { @@ -1071,13 +978,31 @@ fn remaining_image_locals(books: &Path) -> Result, String> { /// 书库元数据列表（不含任何章节正文）：应用启动 / 书架只拉这一份， /// 避免把每本书的全文经 IPC 搬到 WebView（含在线书内嵌的 data URL 大图）。 -/// 这里流式扫描正文、跳过图片载荷，内存占用与书库总字节数无关。 +/// 数据库查询预计算的章节头；仅未迁移的旧书走 JSON 扫描。 pub(crate) fn list_book_meta( app: &AppHandle, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - scan_books_dir(&books_root(app)?) + let root = crate::storage::data_root(app)?; + let mut metas = match sqlite::open(&root) { + Ok(db) => sqlite::metas(&db)?, + Err(error) => { + if !root.join("books").is_dir() { + return Err(error); + } + log::warn!("数据库读取失败，列出未迁移书籍"); + Vec::new() + } + }; + if root.join("books").is_dir() { + for meta in scan_books_dir(&root.join("books"))? { + if !metas.iter().any(|existing| existing.id == meta.id) { + metas.push(meta); + } + } + } + metas.sort_by_key(|book| std::cmp::Reverse(book.imported_at)); + Ok(metas) } /// 扫描书库目录，**两种布局都认**： @@ -1150,11 +1075,10 @@ fn scan_books_dir(dir: &Path) -> Result, String> { // 同步桥接视图：只读元信息（正文一个字节都不碰） // --------------------------------------------------------------------------- -/// 可同步的书籍元信息（`bookdetail.json` 里参与同步的那些字段）。 +/// 可同步的书籍元信息（数据库 detail 中参与同步的那些字段）。 /// /// **为什么不直接用 [`LocalBook`]**：同步对账在每次启动、每次改书标签时都要跑， -/// 而读整本要把 `content.json`（可能是几百 MB 的正文）解析一遍。同步只关心元信息， -/// 因此单独开一个只读 `bookdetail.json` 的视图。 +/// 而读整本会解析所有章节正文。同步只关心元信息，因此单独读取数据库 detail。 /// /// 刻意不在这里的字段（不同步）：`cover`（data URL，会把操作日志撑爆）、 /// `imported_at` / `hue`（导入时间与封面色相）。书源 ID 与同步身份一致。 @@ -1216,24 +1140,24 @@ impl BookSyncMeta { } } -/// 读取一本书的可同步元信息（只读 `bookdetail.json`；旧布局回退同样支持）。 +/// 读取一本书的可同步元信息（只读数据库 detail；旧布局回退同样支持）。 pub(crate) fn get_sync_meta( - app: &AppHandle, id: &str, + app: &AppHandle, + id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - if let Some(meta) = sync_meta_from_dir(&book_dir(app, id)?)? { - return Ok(Some(meta)); + let root = crate::storage::data_root(app)?; + let db = sqlite::open(&root)?; + if let Some(detail) = sqlite::detail(&db, id)? { + return Ok(Some(BookSyncMeta::from_detail(&detail))); } - let legacy = legacy_book_file(app, id)?; - if !legacy.is_file() { - return Ok(None); - } - let scan: BookScan = scan_json_file(&legacy)?; - Ok(Some(BookSyncMeta::from_scan(&scan))) + Ok(legacy_book_at(&root, id)? + .as_ref() + .map(|book| BookSyncMeta::from_detail(&BookDetail::from_book(book)))) } /// 目录布局的可同步元信息（纯路径，便于单测）。 +#[cfg(test)] fn sync_meta_from_dir(dir: &Path) -> Result, String> { let path = dir.join(BOOKDETAIL_FILE); if !path.is_file() { @@ -1245,8 +1169,7 @@ fn sync_meta_from_dir(dir: &Path) -> Result, String> { /// 列出全部本地书的可同步元信息（两种布局都认，口径与书架列表一致）。 /// -/// 与 [`scan_books_dir`] 的区别只有一个：**不读 `content.json`**，因此不会为了 -/// 「这本书有几章」把整库正文解析一遍。 +/// 只读数据库 detail；迁移失败的旧书籍仍使用元信息回退。 pub(crate) fn list_sync_meta( app: &AppHandle, ) -> Result, String> { @@ -1259,44 +1182,27 @@ pub(crate) fn list_sync_meta_at( root: Option<&Path>, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = match root { - Some(root) => root.join("books"), - None => books_root(app)?, - }; - let mut books = Vec::new(); - for entry in fs::read_dir(&dir).map_err(|e| format!("读取书库失败: {e}"))?.flatten() { - let path = entry.path(); - if path.is_dir() { - let detail_path = path.join(BOOKDETAIL_FILE); - if !detail_path.is_file() { - continue; - } - match read_json_file::(&detail_path, "书籍元信息") { - Ok(detail) => books.push(BookSyncMeta::from_detail(&detail)), - Err(error) => log::warn!("同步对账跳过无法解析的书籍 {}：{error}", path.display()), + let root = crate::storage::data_root_at(app, root)?; + let db = sqlite::open(&root)?; + let mut out: Vec<_> = sqlite::details(&db)? + .iter() + .map(BookSyncMeta::from_detail) + .collect(); + if root.join("books").is_dir() { + for meta in scan_books_dir(&root.join("books"))? { + if !out.iter().any(|existing| existing.id == meta.id) { + // Failed migrations remain visible without reading all chapter bodies. + let id = &meta.id; + if let Some(detail) = legacy_detail_at(&root, id)? { + out.push(BookSyncMeta::from_detail(&detail)); + } } - continue; - } - if path.extension().and_then(|s| s.to_str()) != Some("json") { - continue; - } - let Some(id) = path.file_stem().and_then(|s| s.to_str()) else { - continue; - }; - // 目录布局已经存在时不再看旧文件（与书架列表同一口径：同一本书不出现两次） - if dir.join(id).join(BOOKDETAIL_FILE).is_file() { - continue; - } - match scan_json_file::(&path) { - Ok(scan) => books.push(BookSyncMeta::from_scan(&scan)), - Err(error) => log::warn!("同步对账跳过无法解析的书籍文件 {}：{error}", path.display()), } } - Ok(books) + Ok(out) } -/// 把同步合并后的元信息写回 `bookdetail.json`（只动 `want` 里的字段， +/// 把同步合并后的元信息写回数据库 detail（只动 `want` 里的字段， /// 封面 / 导入时间 / 色相 / 书源 id 原样保留）。返回是否真的改动了磁盘。 pub(crate) fn apply_sync_meta( app: &AppHandle, @@ -1304,11 +1210,43 @@ pub(crate) fn apply_sync_meta( want: &BookSyncMeta, ) -> Result { let _transaction = library_transaction(); - migrate_legacy_layout(app); - apply_sync_meta_in_dir(&book_dir(app, id)?, want) + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + let tx = sqlite::transaction(&mut db)?; + let Some(mut detail) = sqlite::detail(&tx, id)? else { + return Ok(false); + }; + if BookSyncMeta::from_detail(&detail) == *want { + return Ok(false); + } + detail.title = want.title.clone(); + detail.author = want.author.clone(); + detail.intro = want.intro.clone(); + detail.format = want.format.clone(); + detail.file_name = want.file_name.clone(); + detail.size = want.size; + detail.split_desc = want.split_desc.clone(); + detail.source = want.source.clone(); + detail.book_url = want.book_url.clone(); + detail.book_source_id = want.book_source_id.clone(); + detail.group_id = want.group_id.clone(); + detail.tags = if want.tags.is_empty() { + None + } else { + Some(want.tags.clone()) + }; + detail.source_tags = if want.source_tags.is_empty() { + None + } else { + Some(want.source_tags.clone()) + }; + sqlite::save_detail(&tx, &detail)?; + sqlite::commit(tx)?; + Ok(true) } /// [`apply_sync_meta`] 的纯路径版本（便于单测）。 +#[cfg(test)] fn apply_sync_meta_in_dir(dir: &Path, want: &BookSyncMeta) -> Result { let path = dir.join(BOOKDETAIL_FILE); if !path.is_file() { @@ -1330,7 +1268,11 @@ fn apply_sync_meta_in_dir(dir: &Path, want: &BookSyncMeta) -> Result Vec { /// `content.json` 的目录扫描视图：**只取章节头**，段落与结构化块一律不解析 /// （serde 对未声明字段走忽略语义：字符串只扫描、不分配），因此扫一本几百 MB 的书 /// 也不会把正文读进内存 —— 对账要为每本书算一份目录，不能按「读整本」的代价来。 +#[cfg(test)] #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] struct ChapterRefScan { @@ -1386,6 +1329,7 @@ struct ChapterRefScan { url: Option, } +#[cfg(test)] #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] struct ContentRefScan { @@ -1393,6 +1337,7 @@ struct ContentRefScan { chapters: Vec, } +#[cfg(test)] fn scan_chapter_refs(path: &Path) -> Result, String> { if !path.is_file() { return Ok(Vec::new()); @@ -1411,85 +1356,94 @@ fn scan_chapter_refs(path: &Path) -> Result, String> { /// 列出全部本地书的目录（同步对账用；两种布局都认）。 /// -/// 只读 `content.json` 的章节头，**不解析正文**，也不统计字数：这是「启动时把书库 +/// 只读数据库章节头，**不解析正文**，也不统计字数：这是「启动时把书库 /// 目录灌进引擎」的那一步，不能按读整本的代价来。 pub(crate) fn list_sync_structures( app: &AppHandle, ) -> Result)>, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = books_root(app)?; - let mut books = Vec::new(); - for entry in fs::read_dir(&dir).map_err(|e| format!("读取书库失败: {e}"))?.flatten() { - let path = entry.path(); - if path.is_dir() { - if !path.join(BOOKDETAIL_FILE).is_file() { + let root = crate::storage::data_root(app)?; + let db = sqlite::open(&root)?; + let mut out = Vec::new(); + for detail in sqlite::details(&db)? { + let refs = sqlite::refs(&db, &detail.id)?; + if !refs.is_empty() { + out.push((detail.id, refs)); + } + } + if root.join("books").is_dir() { + for meta in scan_books_dir(&root.join("books"))? { + if sqlite::detail(&db, &meta.id)?.is_some() { continue; } - let Some(id) = path.file_name().and_then(|s| s.to_str()) else { - continue; - }; - match scan_chapter_refs(&path.join(CONTENT_FILE)) { - Ok(chapters) if !chapters.is_empty() => books.push((id.to_string(), chapters)), - Ok(_) => {} - Err(error) => log::warn!("同步对账跳过无法解析的目录 {}：{error}", path.display()), + if !meta.chapters.is_empty() { + out.push(( + meta.id, + meta.chapters + .into_iter() + .map(|h| ChapterRef { + cid: h.cid, + title: h.title, + url: h.url, + }) + .collect(), + )); } - continue; - } - // 旧布局（迁移没成功）：整书文件里直接带 chapters - if path.extension().and_then(|s| s.to_str()) != Some("json") { - continue; - } - let Some(id) = path.file_stem().and_then(|s| s.to_str()) else { - continue; - }; - if dir.join(id).join(BOOKDETAIL_FILE).is_file() { - continue; - } - let scan: Result = scan_json_file(&path); - match scan { - Ok(scan) => { - let chapters: Vec = scan - .chapters - .into_iter() - .map(|chapter| ChapterRef { - cid: chapter.cid, - title: chapter.title, - url: chapter.url, - }) - .collect(); - if !chapters.is_empty() { - books.push((scan.id, chapters)); - } - } - Err(error) => log::warn!("同步对账跳过无法解析的目录 {}：{error}", path.display()), } } - Ok(books) + Ok(out) } -/// 把同步来的目录落到本地 `content.json`：**按 cid 复用原有正文**，只更新标题 / 地址、 +/// 把同步来的目录落到数据库：**按 cid 复用原有正文**，只更新标题 / 地址、 /// 增删章节并重排顺序。正文本身由内容通道单独搬运（见 docs/sync.md），这里一个字节 /// 都不改，因此「目录变了」不会顺带把谁读了一半的正文弄丢。 /// /// 返回是否真的改动了磁盘；目录（cid / 标题 / 地址）本来就一致时直接返回 false， -/// 连整本 `content.json` 都不解析。 +/// 不读取章节正文。 pub(crate) fn apply_sync_structure( app: &AppHandle, id: &str, want: &[ChapterRef], ) -> Result { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - if !dir.join(BOOKDETAIL_FILE).is_file() { - // 书已经不在本机了：目录跟着书的删除一起没了，不是错误 + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + let tx = sqlite::transaction(&mut db)?; + if sqlite::detail(&tx, id)?.is_none() { + return Ok(false); + } + if sqlite::refs(&tx, id)? == want { return Ok(false); } - apply_sync_structure_in_dir(&dir, want) + let mut by_cid: HashMap<_, _> = sqlite::chapters(&tx, id)? + .into_iter() + .map(|c| (c.cid.clone(), c)) + .collect(); + let mut book = sqlite::detail(&tx, id)? + .ok_or("书籍不存在")? + .into_book(Vec::new()); + book.chapters = want + .iter() + .map(|entry| { + let mut chapter = by_cid.remove(&entry.cid).unwrap_or(LocalBookChapter { + cid: entry.cid.clone(), + title: String::new(), + url: None, + paragraphs: Vec::new(), + blocks: None, + }); + chapter.title = entry.title.clone(); + chapter.url = entry.url.clone(); + chapter + }) + .collect(); + sqlite::put(&tx, &book)?; + sqlite::commit(tx)?; + Ok(true) } /// [`apply_sync_structure`] 的纯路径版本（便于单测）。 +#[cfg(test)] fn apply_sync_structure_in_dir(dir: &Path, want: &[ChapterRef]) -> Result { let content_path = dir.join(CONTENT_FILE); let current = scan_chapter_refs(&content_path)?; @@ -1499,7 +1453,10 @@ fn apply_sync_structure_in_dir(dir: &Path, want: &[ChapterRef]) -> Result = content .chapters @@ -1532,15 +1489,11 @@ fn apply_sync_structure_in_dir(dir: &Path, want: &[ChapterRef]) -> Result/digest.json`）。 -/// -/// **派生缓存**：内容完全由 `content.json` 决定，删掉会自动重建（也能被旧版本忽略）。 -/// 为什么要有它：正文同步每次会话都要问「这本书本机有哪些章、指纹是什么」， -/// 每次都把整本 `content.json` 解析一遍（几百 MB）不可接受 —— 写正文时顺手把指纹 -/// 落在这个小文件里，读的时候只读它。 +/// 旧章节指纹缓存文件名；升级时归档。生产指纹与正文同事务存储在章节行内。 const DIGEST_FILE: &str = "digest.json"; /// 指纹文件格式：章节顺序与 `content.json` 一一对应（`hash` 为空串 = 该章没有正文）。 +#[cfg(test)] #[derive(Debug, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] struct BookDigestFile { @@ -1563,6 +1516,7 @@ struct BookDigestFile { } /// 一章正文里引用到的插图（见 [`BookDigestFile::assets`]）。 +#[cfg(test)] #[derive(Debug, Clone, Default, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] struct ChapterAssets { @@ -1588,6 +1542,7 @@ fn chapter_asset_locals(chapter: &LocalBookChapter) -> Vec { locals } +#[cfg(test)] fn content_stamp(path: &Path) -> (u64, u64) { let Ok(metadata) = fs::metadata(path) else { return (0, 0); @@ -1607,7 +1562,10 @@ fn chapter_digest(chapter: &LocalBookChapter) -> ChapterDigest { .blocks .as_ref() .and_then(|blocks| serde_json::to_value(blocks).ok()); - let hash = if chapter.paragraphs.iter().any(|text| !text.trim().is_empty()) + let hash = if chapter + .paragraphs + .iter() + .any(|text| !text.trim().is_empty()) || blocks.is_some() { readerx_sync::content::body_fingerprint(&chapter.paragraphs, blocks.as_ref()) @@ -1615,10 +1573,14 @@ fn chapter_digest(chapter: &LocalBookChapter) -> ChapterDigest { // 没有正文的章节：占位（保持与 content.json 的下标一致），同步时不参与搬运 String::new() }; - ChapterDigest { cid: chapter.cid.clone(), hash } + ChapterDigest { + cid: chapter.cid.clone(), + hash, + } } /// 写指纹缓存（正文写完之后调用；写失败只记日志，不影响正文本身）。 +#[cfg(test)] fn write_digest_file(dir: &Path, chapters: &[LocalBookChapter]) -> Result<(), String> { write_digest_file_with(dir, chapter_digests(chapters), chapter_assets(chapters)) } @@ -1629,14 +1591,18 @@ fn chapter_digests(chapters: &[LocalBookChapter]) -> Vec { } /// 见 [`chapter_digests`]。 +#[cfg(test)] fn chapter_assets(chapters: &[LocalBookChapter]) -> Vec { chapters .iter() - .map(|chapter| ChapterAssets { locals: chapter_asset_locals(chapter) }) + .map(|chapter| ChapterAssets { + locals: chapter_asset_locals(chapter), + }) .collect() } /// 写指纹缓存（调用方已经算好两份清单，见 [`write_book_content_with_digests`] 的逐章路径）。 +#[cfg(test)] fn write_digest_file_with( dir: &Path, chapters: Vec, @@ -1657,6 +1623,7 @@ fn write_digest_file_with( /// /// 缓存缺失 / 过期 / 长度对不上时返回等长的空清单：那几章的引用会在下一次整本重算时补上， /// 资源通道据此把它们当成「还没落地」—— 宁可不搬，也不要把错的引用发出去。 +#[cfg(test)] fn read_digest_assets(dir: &Path, chapters: usize) -> Vec { let cached = read_json_file::(&dir.join(DIGEST_FILE), "章节指纹") .ok() @@ -1669,6 +1636,7 @@ fn read_digest_assets(dir: &Path, chapters: usize) -> Vec { } /// 读指纹缓存；缓存缺失 / 过期时**从正文重建**（一次整本解析，之后都走缓存）。 +#[cfg(test)] fn read_digest_file(dir: &Path) -> Result, String> { let content_path = dir.join(CONTENT_FILE); let (mtime, size) = content_stamp(&content_path); @@ -1701,18 +1669,22 @@ pub(crate) fn read_sync_digests_at( id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir_at(app, root, id)?; - if !dir.join(BOOKDETAIL_FILE).is_file() { - return Ok(Vec::new()); + let root = crate::storage::data_root_at(app, root)?; + let db = sqlite::open(&root)?; + if sqlite::detail(&db, id)?.is_none() { + return Ok(legacy_book_at(&root, id)? + .map(|b| { + chapter_digests(&b.chapters) + .into_iter() + .filter(|d| !d.hash.is_empty()) + .collect() + }) + .unwrap_or_default()); } - Ok(read_digest_file(&dir)? - .into_iter() - .filter(|digest| !digest.hash.is_empty()) - .collect()) + sqlite::digests(&db, id) } -/// 一本书的封面（`bookdetail.json` 里的 data URL）。只读元信息，不碰正文。 +/// 一本书的封面（数据库元信息里的 data URL）。不碰正文。 /// /// 封面同步走资源通道（见 `docs/sync.md` 第 7.10 节）：引擎要的是「这段 data URL 的 /// 字节与指纹」，因此这里只提供只读入口，**不改存储形态** —— 封面仍然是元信息里的 @@ -1723,19 +1695,12 @@ pub(crate) fn get_cover_at( id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let path = book_dir_at(app, root, id)?.join(BOOKDETAIL_FILE); - if path.is_file() { - let detail: BookDetail = read_json_file(&path, "书籍元信息")?; + let root = crate::storage::data_root_at(app, root)?; + let db = sqlite::open(&root)?; + if let Some(detail) = sqlite::detail(&db, id)? { return Ok(detail.cover); } - // 旧布局回退：与 `get_sync_meta` 同一口径（迁移没成功的书仍可读） - let legacy = legacy_book_file(app, id)?; - if !legacy.is_file() { - return Ok(None); - } - let scan: BookScan = scan_json_file(&legacy)?; - Ok(scan.cover) + Ok(legacy_detail_at(&root, id)?.and_then(|d| d.cover)) } /// 写一本书的封面（`Some` 写值，`None` / 空串清除）；返回是否真的改动了磁盘。 @@ -1749,19 +1714,19 @@ pub(crate) fn set_cover_at( cover: Option<&str>, ) -> Result { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let path = book_dir_at(app, root, id)?.join(BOOKDETAIL_FILE); - if !path.is_file() { - // 书已经不在了（对端刚同步来、本地却没这个文件）：不是错误，跳过即可 + let root = crate::storage::data_root_at(app, root)?; + let mut db = sqlite::open(&root)?; + let tx = sqlite::transaction(&mut db)?; + let Some(mut detail) = sqlite::detail(&tx, id)? else { return Ok(false); - } - let mut detail: BookDetail = read_json_file(&path, "书籍元信息")?; - let want = cover.filter(|value| !value.is_empty()).map(str::to_string); + }; + let want = cover.filter(|v| !v.is_empty()).map(str::to_string); if detail.cover == want { return Ok(false); } detail.cover = want; - write_json_atomic(&path, &detail, "书籍元信息")?; + sqlite::save_detail(&tx, &detail)?; + sqlite::commit(tx)?; Ok(true) } @@ -1776,32 +1741,12 @@ pub(crate) fn read_sync_asset_refs_at( id: &str, ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir_at(app, root, id)?; - if !dir.join(BOOKDETAIL_FILE).is_file() { - return Ok(Vec::new()); - } - // 先让缓存/正文对齐一次（`read_digest_file` 负责过期重建），再取引用清单 - read_digest_file(&dir)?; - let file: BookDigestFile = read_json_file(&dir.join(DIGEST_FILE), "章节指纹")?; - let mut refs: Vec<(String, String)> = Vec::new(); - for chapter in file.assets { - for local in chapter.locals { - if let Some(name) = crate::book_images::asset_name(&local) { - refs.push((name, local)); - } - } - } - refs.sort(); - refs.dedup(); - Ok(refs) + let root = crate::storage::data_root_at(app, root)?; + let db = sqlite::open(&root)?; + sqlite::assets(&db, id) } -/// 按 cid 取章节正文（对端要哪几章就取哪几章）。 -/// -/// 实现是**顺序扫一遍** `content.json`，只留下命中的章节：内存占用与单章同级， -/// 不会因为对端只要一章就把整本正文读进内存。代价是要扫完整份文件（图片载荷字段 -/// 会被完整解析，但只保留命中的那一章）。 +/// 按 cid 索引查询指定章节；正文内存占用与请求的章节同级。 pub(crate) fn read_chapters_by_cid_at( app: &AppHandle, root: Option<&Path>, @@ -1809,175 +1754,84 @@ pub(crate) fn read_chapters_by_cid_at( cids: &[String], ) -> Result, String> { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let path = book_dir_at(app, root, id)?.join(CONTENT_FILE); - if !path.is_file() || cids.is_empty() { - return Ok(Vec::new()); - } - let wanted: HashSet = cids.iter().cloned().collect(); - let file = fs::File::open(&path).map_err(|e| format!("读取书籍正文失败: {e}"))?; - let mut deserializer = serde_json::Deserializer::from_reader(BufReader::new(file)); - let picked = serde::de::DeserializeSeed::deserialize( - PickChapters { wanted: &wanted }, - &mut deserializer, - ) - .map_err(|e| format!("解析书籍正文失败: {e}"))?; - Ok(picked) -} - -/// 只挑出指定 cid 的章节（`chapters` 数组逐项反序列化，未命中的读完即丢）。 -struct PickChapters<'a> { - wanted: &'a HashSet, -} - -impl<'de, 'a> serde::de::DeserializeSeed<'de> for PickChapters<'a> { - type Value = Vec; - - fn deserialize(self, deserializer: D) -> Result - where - D: serde::Deserializer<'de>, - { - deserializer.deserialize_map(PickTop { wanted: self.wanted }) - } -} - -/// `content.json` 的顶层对象：只关心 `chapters`，其余字段跳过。 -struct PickTop<'a> { - wanted: &'a HashSet, -} - -impl<'de, 'a> serde::de::Visitor<'de> for PickTop<'a> { - type Value = Vec; - - fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("书籍正文对象") - } - - fn visit_map(self, mut map: A) -> Result - where - A: serde::de::MapAccess<'de>, - { - let mut picked = Vec::new(); - while let Some(key) = map.next_key::()? { - if key == "chapters" { - picked = map.next_value_seed(PickList { wanted: self.wanted })?; - } else { - map.next_value::()?; - } - } - Ok(picked) - } -} - -/// `chapters` 数组：逐项反序列化成章节，未命中 cid 的直接丢掉。 -struct PickList<'a> { - wanted: &'a HashSet, -} - -impl<'de, 'a> serde::de::DeserializeSeed<'de> for PickList<'a> { - type Value = Vec; - - fn deserialize(self, deserializer: D) -> Result - where - D: serde::Deserializer<'de>, - { - deserializer.deserialize_seq(PickListVisitor { wanted: self.wanted }) - } -} - -struct PickListVisitor<'a> { - wanted: &'a HashSet, -} - -impl<'de, 'a> serde::de::Visitor<'de> for PickListVisitor<'a> { - type Value = Vec; - - fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("章节数组") - } - - fn visit_seq(self, mut seq: A) -> Result - where - A: serde::de::SeqAccess<'de>, - { - let mut out = Vec::new(); - while let Some(chapter) = seq.next_element::()? { - if self.wanted.contains(&chapter.cid) { - out.push(chapter); - } - } - Ok(out) + let root = crate::storage::data_root_at(app, root)?; + let db = sqlite::open(&root)?; + if sqlite::detail(&db, id)?.is_none() { + let wanted: HashSet<_> = cids.iter().collect(); + return Ok(legacy_book_at(&root, id)? + .map(|b| { + b.chapters + .into_iter() + .filter(|c| wanted.contains(&c.cid)) + .collect() + }) + .unwrap_or_default()); } + Ok(sqlite::picked(&db, id, cids)? + .into_iter() + .map(|(_, c)| c) + .collect()) } -/// 把同步来的正文写进某本书（按 cid 合并：更新已有章的正文，缺的补在后面）。 -/// -/// 标题 / 地址只在本地缺失时采用对端那份：目录（结构）实体才是它们的权威来源， -/// 这里不该用一份可能过时的正文载荷把刚同步好的目录改回去。 pub(crate) fn apply_sync_chapters( app: &AppHandle, id: &str, bodies: &[ChapterContent], ) -> Result { let _transaction = library_transaction(); - migrate_legacy_layout(app); - let dir = book_dir(app, id)?; - if !dir.join(BOOKDETAIL_FILE).is_file() { - return Err("书籍不在本机".to_string()); - } - let content_path = dir.join(CONTENT_FILE); - let mut chapters = if content_path.is_file() { - read_json_file::(&content_path, "书籍正文")?.chapters - } else { - Vec::new() - }; - let mut applied = 0usize; + let root = crate::storage::data_root(app)?; + let mut db = sqlite::open(&root)?; + sqlite::require_book(&db, &root, id)?; + let tx = sqlite::transaction(&mut db)?; + let mut next = sqlite::refs(&tx, id)?.len(); + let mut applied = 0; for body in bodies { if !has_body(body) { continue; } - let paragraphs = body.paragraphs.clone(); + let mut matches = sqlite::picked(&tx, id, &[body.cid.clone()])?; + if matches.len() > 1 { + return Err("章节 cid 重复，无法同步正文".into()); + } + let (position, mut chapter) = matches.pop().unwrap_or_else(|| { + let p = next; + next += 1; + ( + p, + LocalBookChapter { + cid: body.cid.clone(), + title: body.title.clone(), + url: body.url.clone(), + paragraphs: Vec::new(), + blocks: None, + }, + ) + }); let blocks = body .blocks .as_ref() - .and_then(|value| serde_json::from_value::>(value.clone()).ok()); - match chapters.iter_mut().find(|chapter| chapter.cid == body.cid) { - Some(chapter) => { - // 正文完全一样就不动它（同步会话可能重复推到同一章） - let same_blocks = match (&chapter.blocks, &body.blocks) { - (None, None) => true, - (Some(existing), Some(incoming)) => { - serde_json::to_value(existing).ok().as_ref() == Some(incoming) - } - _ => false, - }; - if chapter.paragraphs == paragraphs && same_blocks { - continue; - } - chapter.paragraphs = paragraphs; - chapter.blocks = blocks; - if chapter.title.is_empty() { - chapter.title = body.title.clone(); - } - if chapter.url.is_none() { - chapter.url = body.url.clone(); - } - } - None => chapters.push(LocalBookChapter { - cid: body.cid.clone(), - title: body.title.clone(), - paragraphs, - blocks, - url: body.url.clone(), - }), + .map(|v| { + serde_json::from_value::>(v.clone()) + .map_err(|e| format!("同步章节块无效: {e}")) + }) + .transpose()?; + if chapter.paragraphs == body.paragraphs + && serde_json::to_value(&chapter.blocks).ok() == serde_json::to_value(&blocks).ok() + { + continue; + } + chapter.paragraphs = body.paragraphs.clone(); + chapter.blocks = blocks; + if chapter.title.is_empty() { + chapter.title = body.title.clone(); + } + if chapter.url.is_none() { + chapter.url = body.url.clone(); } + sqlite::save_chapter(&tx, id, position, &chapter)?; applied += 1; } - if applied == 0 { - return Ok(0); - } - write_book_content(&dir, &chapters)?; - log::debug!("同步正文已写入 id={id} 章节={applied}"); + sqlite::commit(tx)?; Ok(applied) } @@ -1991,6 +1845,154 @@ fn has_body(body: &ChapterContent) -> bool { .is_some_and(|items| !items.is_empty()) } +fn legacy_book_at(root: &Path, id: &str) -> Result, String> { + sqlite::valid_id(id)?; + if let Some(book) = read_book_from_dir(&root.join("books").join(id))? { + return Ok(Some(book)); + } + let flat = root.join("books").join(format!("{id}.json")); + if flat.is_file() { + return read_json_file(&flat, "旧书籍").map(Some); + } + Ok(None) +} +fn legacy_detail_at(root: &Path, id: &str) -> Result, String> { + sqlite::valid_id(id)?; + let path = root.join("books").join(id).join(BOOKDETAIL_FILE); + if path.is_file() { + return read_json_file(&path, "旧书籍元信息").map(Some); + } + Ok(legacy_book_at(root, id)? + .as_ref() + .map(BookDetail::from_book)) +} + +pub(crate) fn contains_at( + app: &AppHandle, + root: Option<&Path>, + id: &str, +) -> Result { + let _transaction = library_transaction(); + let root = crate::storage::data_root_at(app, root)?; + Ok(sqlite::detail(&sqlite::open(&root)?, id)?.is_some()) +} +pub(crate) fn export_books( + app: &AppHandle, + zip: &mut zip::ZipWriter, + report: &mut dyn FnMut(&str, u64, u64), +) -> Result { + let _transaction = library_transaction(); + let root = crate::storage::data_root(app)?; + let db = sqlite::open(&root)?; + // Refuse a partial backup when any readable legacy book could not be imported. + if root.join("books").is_dir() && !scan_books_dir(&root.join("books"))?.is_empty() { + return Err("书籍迁移尚未完成，暂不能导出完整备份".into()); + } + sqlite::export(&db, zip, report) +} +/// Backup JSON is a portable representation, not the on-disk database schema. +pub(crate) fn import_book_json( + app: &AppHandle, + id: &str, + detail: &[u8], + content: Option<&[u8]>, +) -> Result<(), String> { + let mut detail: BookDetail = + serde_json::from_slice(detail).map_err(|e| format!("书籍元信息无法解析: {e}"))?; + detail.id = id.to_string(); + if detail.schema_version != SCHEMA_VERSION { + return Err("书籍格式版本不受支持".into()); + } + let chapters = match content { + Some(bytes) => { + let content: BookContent = + serde_json::from_slice(bytes).map_err(|e| format!("书籍正文无法解析: {e}"))?; + if content.schema_version != SCHEMA_VERSION { + return Err("书籍正文格式版本不受支持".into()); + } + content.chapters + } + None => Vec::new(), + }; + put_book(app, detail.into_book(chapters)) +} +/// Called by the journalled identity migration; old/new collision checks precede all writes. +pub(crate) fn rename_id_at(root: &Path, old: &str, new: &str) -> Result<(), String> { + let _transaction = library_transaction(); + sqlite::valid_id(old)?; + sqlite::valid_id(new)?; + let mut db = sqlite::open(root)?; + let tx = sqlite::transaction(&mut db)?; + let from = sqlite::detail(&tx, old)?; + let to = sqlite::detail(&tx, new)?; + if from.is_some() && to.is_some() { + return Err("书籍 ID 迁移目标已存在".into()); + } + if let Some(mut detail) = from { + detail.id = new.to_string(); + tx.execute( + "UPDATE books SET id=?1,detail=?2 WHERE id=?3", + rusqlite::params![ + new, + serde_json::to_string(&detail).map_err(|e| e.to_string())?, + old + ], + ) + .map_err(|e| e.to_string())?; + let mut records = sqlite::records(&tx, "bookmarks", new)?; + fn remap(value: &mut Value, old: &str, new: &str) { + match value { + Value::Object(fields) => { + for (key, value) in fields { + if key == "bookId" && value.as_str() == Some(old) { + *value = Value::String(new.into()); + } else { + remap(value, old, new); + } + } + } + Value::Array(values) => { + for value in values { + remap(value, old, new); + } + } + _ => {} + } + } + for record in &mut records { + remap(record, old, new); + } + sqlite::put_records(&tx, "bookmarks", new, &records)?; + } else if to.is_none() { + return Err("书籍 ID 迁移缺少原书籍".into()); + } + sqlite::commit(tx) +} +pub(crate) fn remap_metadata_at( + root: &Path, + field: &str, + remap: &std::collections::BTreeMap, +) -> Result<(), String> { + if remap.is_empty() || !root.join(sqlite::DATABASE).is_file() { + return Ok(()); + } + let _transaction = library_transaction(); + let mut db = sqlite::connect(root)?; + let tx = sqlite::transaction(&mut db)?; + for mut detail in sqlite::details(&tx)? { + let value = match field { + "groupId" => &mut detail.group_id, + "bookSourceId" => &mut detail.book_source_id, + _ => return Err("未知元信息引用".into()), + }; + if let Some(new) = value.as_ref().and_then(|id| remap.get(id)) { + *value = Some(new.clone()); + sqlite::save_detail(&tx, &detail)?; + } + } + sqlite::commit(tx) +} + #[cfg(test)] mod tests { use super::*; @@ -2093,7 +2095,8 @@ mod tests { scope.spawn(|| { start.wait(); let _transaction = library_transaction(); - let mut content: BookContent = read_json_file(&dir.join(CONTENT_FILE), "正文").unwrap(); + let mut content: BookContent = + read_json_file(&dir.join(CONTENT_FILE), "正文").unwrap(); patch_chapters( &mut content.chapters, &[BookChapterPatch { @@ -2389,7 +2392,10 @@ mod tests { let book = read_book_from_dir(&paths).unwrap().unwrap(); assert_eq!(book.chapters.len(), 2); assert_eq!(book.chapters[0].title, "第一章（改名）"); - assert_eq!(book.chapters[0].url.as_deref(), Some("https://example.com/c1")); + assert_eq!( + book.chapters[0].url.as_deref(), + Some("https://example.com/c1") + ); assert_eq!(book.chapters[0].paragraphs, vec!["正文".to_string()]); assert!(book.chapters[1].paragraphs.is_empty()); @@ -2491,11 +2497,7 @@ mod tests { // 旧布局（迁移没成功留下的） let mut legacy = sample_book("b2"); legacy.imported_at += 10; - fs::write( - dir.join("b2.json"), - serde_json::to_string(&legacy).unwrap(), - ) - .unwrap(); + fs::write(dir.join("b2.json"), serde_json::to_string(&legacy).unwrap()).unwrap(); // 两种布局同时存在：只认目录，不能列出两本 b1 let mut both = sample_book("b3"); both.imported_at += 20; diff --git a/src-tauri/src/book_store/sqlite.rs b/src-tauri/src/book_store/sqlite.rs new file mode 100644 index 0000000..2ab1331 --- /dev/null +++ b/src-tauri/src/book_store/sqlite.rs @@ -0,0 +1,964 @@ +//! SQLite implementation. Callers hold the book-store lock; no sync-engine calls here. +use super::*; +use rusqlite::{params, Connection, OptionalExtension}; +use std::io::Write; + +pub(super) const DATABASE: &str = "books.sqlite3"; +const DATABASE_VERSION: i64 = 1; + +fn sql(error: rusqlite::Error) -> String { + format!("书库数据库操作失败: {error}") +} +fn encode(value: &T) -> Result { + serde_json::to_string(value).map_err(|e| format!("序列化书籍数据失败: {e}")) +} +fn decode(value: &str) -> Result { + serde_json::from_str(value).map_err(|e| format!("解析书籍数据失败: {e}")) +} +pub(super) fn valid_id(id: &str) -> Result<(), String> { + if crate::storage::valid_component(id) { + Ok(()) + } else { + Err("非法的书籍 id".into()) + } +} + +pub(super) fn connect(root: &Path) -> Result { + crate::storage::ensure_dir(root)?; + let mut db = Connection::open(root.join(DATABASE)).map_err(sql)?; + db.busy_timeout(std::time::Duration::from_secs(5)) + .map_err(sql)?; + db.execute_batch("PRAGMA foreign_keys=ON; PRAGMA synchronous=FULL;") + .map_err(sql)?; + let version: i64 = db + .query_row("PRAGMA user_version", [], |row| row.get(0)) + .map_err(sql)?; + if version > DATABASE_VERSION { + return Err("书库数据库格式版本不受支持".into()); + } + if version == 0 { + let tx = db.transaction().map_err(sql)?; + tx.execute_batch( + "CREATE TABLE books (id TEXT PRIMARY KEY, detail TEXT NOT NULL); + CREATE TABLE chapters ( + book_id TEXT NOT NULL REFERENCES books(id) ON DELETE CASCADE ON UPDATE CASCADE, + position INTEGER NOT NULL, cid TEXT NOT NULL, + chapter TEXT NOT NULL, head TEXT NOT NULL, digest TEXT NOT NULL, assets TEXT NOT NULL, + PRIMARY KEY(book_id, position)); + CREATE INDEX chapters_by_cid ON chapters(book_id, cid); + CREATE TABLE bookmarks (book_id TEXT PRIMARY KEY REFERENCES books(id) + ON DELETE CASCADE ON UPDATE CASCADE, records TEXT NOT NULL); + CREATE TABLE annotations (book_id TEXT PRIMARY KEY REFERENCES books(id) + ON DELETE CASCADE ON UPDATE CASCADE, records TEXT NOT NULL); + CREATE TABLE migrated_books (id TEXT PRIMARY KEY); + PRAGMA user_version=1;" + ).map_err(sql)?; + tx.commit().map_err(sql)?; + } + Ok(db) +} + +pub(super) fn open(root: &Path) -> Result { + // Complete pre-SQLite file journals before importing their metadata snapshots. + for journal in [ + "migrations/readerx.sourceSaveMigration.json", + "migrations/readerx.sourceIdMigration.json", + "migrations/readerx.dataIdMigration.json", + ] { + readerx_source::id_migration::FilePlan::resume(root, journal)?; + } + let mut db = connect(root)?; + migrate(&mut db, root)?; + Ok(db) +} + +/// One transaction per book includes records and a durable migration receipt. +/// Receipts prevent a crash during file cleanup from replaying stale JSON over later edits. +fn migrate(db: &mut Connection, root: &Path) -> Result<(), String> { + let books = root.join("books"); + let global = root.join("state/readerx.bookmarks.json"); + let old_marks: Result>, String> = if global.is_file() { + read_json_file(&global, "旧书签") + } else { + Ok(HashMap::new()) + }; + let mut pending = old_marks.is_err(); + if books.is_dir() { + for entry in fs::read_dir(&books).map_err(|e| format!("读取旧书库失败: {e}"))? { + let entry = entry.map_err(|e| e.to_string())?; + let kind = entry.file_type().map_err(|e| e.to_string())?; + if !kind.is_file() && !kind.is_dir() { + continue; + } + let path = entry.path(); + let id = if kind.is_dir() { + let id = entry.file_name().to_string_lossy().into_owned(); + if !path.join(BOOKDETAIL_FILE).is_file() { + let receipt: bool = db + .query_row( + "SELECT EXISTS(SELECT 1 FROM migrated_books WHERE id=?1)", + [&id], + |r| r.get(0), + ) + .map_err(sql)?; + if !receipt { + continue; + } + } + id + } else { + if path.extension().and_then(|s| s.to_str()) != Some("json") { + continue; + } + path.file_stem() + .unwrap_or_default() + .to_string_lossy() + .into_owned() + }; + let result = (|| { + valid_id(&id)?; + let migrated: bool = db + .query_row( + "SELECT EXISTS(SELECT 1 FROM migrated_books WHERE id=?1)", + [&id], + |r| r.get(0), + ) + .map_err(sql)?; + if !migrated { + // Do not hide a failed upgrade under a newly-created empty book. + if detail(db, &id)?.is_some() { + return Err("旧书籍与数据库书籍冲突，原文件已保留".into()); + } + let split = books.join(&id); + let is_split = split.join(BOOKDETAIL_FILE).is_file(); + let mut book = if is_split { + let content = split.join(CONTENT_FILE); + if content.is_file() + && fs::metadata(&content).map_err(|e| e.to_string())?.len() + >= crate::book_images::STREAM_MIGRATE_MIN_BYTES + && crate::book_images::file_has_data_image(&content) + { + crate::book_images::migrate_book_file( + &root.join("images"), + &id, + &content, + )?; + } + read_book_from_dir(&split)?.ok_or("旧书籍元信息缺失")? + } else { + if fs::metadata(&path).map_err(|e| e.to_string())?.len() + >= crate::book_images::STREAM_MIGRATE_MIN_BYTES + && crate::book_images::file_has_data_image(&path) + { + crate::book_images::migrate_book_file( + &root.join("images"), + &id, + &path, + )?; + } + read_json_file::(&path, "旧书籍")? + }; + if is_split { + let detail: BookDetail = + read_json_file(&split.join(BOOKDETAIL_FILE), "旧书籍元信息")?; + if detail.schema_version != SCHEMA_VERSION { + return Err("旧书籍格式版本不受支持".into()); + } + if split.join(CONTENT_FILE).is_file() { + let content: BookContent = + read_json_file(&split.join(CONTENT_FILE), "旧书籍正文")?; + if content.schema_version != SCHEMA_VERSION { + return Err("旧书籍正文格式版本不受支持".into()); + } + } + } + if book.id != id { + return Err("旧书籍 ID 与路径不一致，原文件已保留".into()); + } + let dir = books.join(&id); + let marks = if dir.join(BOOKMARKS_FILE).is_file() { + read_bookmarks_file(&dir.join(BOOKMARKS_FILE))? + } else { + old_marks + .as_ref() + .map_err(Clone::clone)? + .get(&id) + .cloned() + .unwrap_or_default() + }; + let notes = read_annotations_file(&dir.join(ANNOTATIONS_FILE))?; + crate::book_images::migrate_book(&root.join("images"), &mut book); + let tx = db.transaction().map_err(sql)?; + put(&tx, &book)?; + put_records(&tx, "bookmarks", &id, &marks)?; + put_records(&tx, "annotations", &id, ¬es)?; + tx.execute("INSERT INTO migrated_books(id) VALUES(?1)", [&id]) + .map_err(sql)?; + tx.commit().map_err(sql)?; + log::info!("书籍已迁移到 SQLite id={id}"); + } + // Only book data files are archived. Unknown files / images / caches stay outside SQLite. + for name in [ + BOOKDETAIL_FILE, + CONTENT_FILE, + BOOKMARKS_FILE, + ANNOTATIONS_FILE, + DIGEST_FILE, + ] { + let file = books.join(&id).join(name); + if file.is_file() { + archive_legacy(&file)?; + } + } + let flat = books.join(format!("{id}.json")); + if flat.is_file() { + archive_legacy(&flat)?; + } + Ok::<_, String>(()) + })(); + if let Err(error) = result { + pending = true; + log::warn!( + "书籍 SQLite 迁移未完成 id={id}：{}", + readerx_log::redact::urls_in_text(&error) + ); + } + } + } + // Previously split books may already be in the database when only the global file remains. + if let Ok(marks) = &old_marks { + for (id, records) in marks { + if valid_id(id).is_err() { + log::warn!("旧全库书签中有非法书籍 ID，已跳过"); + continue; + } + if records.is_empty() { + continue; + } + if detail(db, id)?.is_some() { + db.execute( + "INSERT OR IGNORE INTO bookmarks(book_id, records) VALUES(?1, ?2)", + params![id, encode(records)?], + ) + .map_err(sql)?; + } else if books.join(id).join(BOOKDETAIL_FILE).is_file() + || books.join(format!("{id}.json")).is_file() + { + pending = true; + } + } + } + if global.is_file() && !pending { + archive_legacy(&global)?; + } + Ok(()) +} +fn archive_legacy(path: &Path) -> Result<(), String> { + let backup = path.with_extension("json.migrated"); + // A pre-existing backup must never be overwritten by a later import or interrupted upgrade. + if backup.exists() { + return Err("旧书籍备份已存在，保留待处理文件".into()); + } + fs::rename(path, backup).map_err(|e| format!("归档旧书籍文件失败: {e}")) +} + +pub(super) fn detail(db: &Connection, id: &str) -> Result, String> { + valid_id(id)?; + db.query_row("SELECT detail FROM books WHERE id=?1", [id], |r| { + r.get::<_, String>(0) + }) + .optional() + .map_err(sql)? + .map(|s| decode(&s)) + .transpose() +} +pub(super) fn details(db: &Connection) -> Result, String> { + strings(db, "SELECT detail FROM books ORDER BY id", [])? + .iter() + .map(|s| decode(s)) + .collect() +} +fn strings( + db: &Connection, + query: &str, + params: P, +) -> Result, String> { + let mut statement = db.prepare(query).map_err(sql)?; + let rows = statement + .query_map(params, |r| r.get::<_, String>(0)) + .map_err(sql)?; + rows.collect::, _>>().map_err(sql) +} +pub(super) fn chapters(db: &Connection, id: &str) -> Result, String> { + valid_id(id)?; + strings( + db, + "SELECT chapter FROM chapters WHERE book_id=?1 ORDER BY position", + [id], + )? + .iter() + .map(|s| decode(s)) + .collect() +} +pub(super) fn get(db: &Connection, id: &str) -> Result, String> { + detail(db, id)? + .map(|d| Ok(d.into_book(chapters(db, id)?))) + .transpose() +} +pub(super) fn save_detail(db: &Connection, detail: &BookDetail) -> Result<(), String> { + valid_id(&detail.id)?; + db.execute("INSERT INTO books(id,detail) VALUES(?1,?2) ON CONFLICT(id) DO UPDATE SET detail=excluded.detail", + params![detail.id, encode(detail)?]).map_err(sql)?; + Ok(()) +} +fn head(chapter: &LocalBookChapter) -> Result { + Ok(scan_chapter_head( + serde_json::from_value(serde_json::to_value(chapter).map_err(|e| e.to_string())?) + .map_err(|e| e.to_string())?, + )) +} +pub(super) fn save_chapter( + db: &Connection, + id: &str, + position: usize, + chapter: &LocalBookChapter, +) -> Result<(), String> { + db.execute("INSERT INTO chapters(book_id,position,cid,chapter,head,digest,assets) VALUES(?1,?2,?3,?4,?5,?6,?7) + ON CONFLICT(book_id,position) DO UPDATE SET cid=excluded.cid,chapter=excluded.chapter,head=excluded.head,digest=excluded.digest,assets=excluded.assets", + params![id, position as i64, chapter.cid, encode(chapter)?, encode(&head(chapter)?)?, encode(&chapter_digest(chapter))?, encode(&chapter_asset_locals(chapter))?]).map_err(sql)?; + Ok(()) +} +pub(super) fn put(db: &Connection, book: &LocalBook) -> Result<(), String> { + save_detail(db, &BookDetail::from_book(book))?; + db.execute("DELETE FROM chapters WHERE book_id=?1", [&book.id]) + .map_err(sql)?; + for (position, chapter) in book.chapters.iter().enumerate() { + save_chapter(db, &book.id, position, chapter)?; + } + Ok(()) +} +pub(super) fn records(db: &Connection, table: &str, id: &str) -> Result, String> { + valid_id(id)?; + let query = match table { + "bookmarks" => "SELECT records FROM bookmarks WHERE book_id=?1", + "annotations" => "SELECT records FROM annotations WHERE book_id=?1", + _ => return Err("未知的书籍记录类型".into()), + }; + match db + .query_row(query, [id], |r| r.get::<_, String>(0)) + .optional() + .map_err(sql)? + { + Some(s) => decode(&s), + None => Ok(Vec::new()), + } +} +pub(super) fn put_records( + db: &Connection, + table: &str, + id: &str, + records: &[Value], +) -> Result<(), String> { + valid_id(id)?; + let query = match table { + "bookmarks" => "INSERT INTO bookmarks(book_id,records) VALUES(?1,?2) ON CONFLICT(book_id) DO UPDATE SET records=excluded.records", + "annotations" => "INSERT INTO annotations(book_id,records) VALUES(?1,?2) ON CONFLICT(book_id) DO UPDATE SET records=excluded.records", + _ => return Err("未知的书籍记录类型".into()), + }; + db.execute(query, params![id, encode(&records)?]) + .map_err(sql)?; + Ok(()) +} + +pub(super) fn metas(db: &Connection) -> Result, String> { + // One indexed join reads only metadata and precomputed chapter heads, never chapter bodies. + let mut statement = db.prepare("SELECT b.detail,c.head FROM books b LEFT JOIN chapters c ON c.book_id=b.id ORDER BY b.id,c.position").map_err(sql)?; + let mut rows = statement.query([]).map_err(sql)?; + let mut out: Vec = Vec::new(); + while let Some(row) = rows.next().map_err(sql)? { + let detail: BookDetail = decode(&row.get::<_, String>(0).map_err(sql)?)?; + if out.last().is_none_or(|last| last.id != detail.id) { + out.push(detail.into_meta(Vec::new())); + } + if let Some(s) = row.get::<_, Option>(1).map_err(sql)? { + out.last_mut().unwrap().chapters.push(decode(&s)?); + } + } + Ok(out) +} +pub(super) fn refs(db: &Connection, id: &str) -> Result, String> { + valid_id(id)?; + strings( + db, + "SELECT head FROM chapters WHERE book_id=?1 ORDER BY position", + [id], + )? + .iter() + .map(|s| { + let head: ChapterHead = decode(s)?; + Ok(ChapterRef { + cid: head.cid, + title: head.title, + url: head.url, + }) + }) + .collect() +} +pub(super) fn digests(db: &Connection, id: &str) -> Result, String> { + valid_id(id)?; + let all: Vec = strings( + db, + "SELECT digest FROM chapters WHERE book_id=?1 ORDER BY position", + [id], + )? + .iter() + .map(|s| decode(s)) + .collect::>()?; + Ok(all.into_iter().filter(|d| !d.hash.is_empty()).collect()) +} +pub(super) fn assets(db: &Connection, id: &str) -> Result, String> { + valid_id(id)?; + let mut out = Vec::new(); + for s in strings( + db, + "SELECT assets FROM chapters WHERE book_id=?1 ORDER BY position", + [id], + )? { + for local in decode::>(&s)? { + if let Some(name) = crate::book_images::asset_name(&local) { + out.push((name, local)); + } + } + } + out.sort(); + out.dedup(); + Ok(out) +} +pub(super) fn picked( + db: &Connection, + id: &str, + cids: &[String], +) -> Result, String> { + valid_id(id)?; + let mut statement = db + .prepare( + "SELECT position,chapter FROM chapters WHERE book_id=?1 AND cid=?2 ORDER BY position", + ) + .map_err(sql)?; + let mut out = Vec::new(); + for cid in cids.iter().collect::>() { + let rows = statement + .query_map(params![id, cid], |r| { + Ok((r.get::<_, i64>(0)?, r.get::<_, String>(1)?)) + }) + .map_err(sql)?; + for row in rows { + let (p, s) = row.map_err(sql)?; + out.push((p as usize, decode(&s)?)); + } + } + out.sort_by_key(|(p, _)| *p); + Ok(out) +} +pub(super) fn patch( + db: &mut Connection, + root: &Path, + id: &str, + updates: &[BookChapterPatch], +) -> Result<(), String> { + require_book(db, root, id)?; + let tx = db.transaction().map_err(sql)?; + let mut changed = Vec::new(); + for update in updates { + if update.chapter.cid.is_empty() { + return Err("章节补丁缺少 cid".into()); + } + let mut matches = picked(&tx, id, &[update.chapter.cid.clone()])?; + if matches.len() != 1 { + return Err("章节目录已变化或 cid 重复，无法定位补丁".into()); + } + let (p, mut chapter) = matches.remove(0); + chapter.paragraphs = update.chapter.paragraphs.clone(); + chapter.blocks = update.chapter.blocks.clone(); + crate::book_images::migrate_chapters( + &root.join("images"), + id, + std::slice::from_mut(&mut chapter), + ); + changed.push((p, chapter)); + } + for (p, chapter) in changed { + save_chapter(&tx, id, p, &chapter)?; + } + tx.commit().map_err(sql) +} +pub(super) fn require_book(db: &Connection, root: &Path, id: &str) -> Result<(), String> { + if detail(db, id)?.is_some() { + return Ok(()); + } + if root.join("books").join(id).join(BOOKDETAIL_FILE).is_file() + || root.join("books").join(format!("{id}.json")).is_file() + { + Err("书籍数据迁移失败，请查看应用日志".into()) + } else { + Err("书籍不存在".into()) + } +} +pub(super) fn commit(db: rusqlite::Transaction<'_>) -> Result<(), String> { + db.commit().map_err(sql) +} +pub(super) fn transaction(db: &mut Connection) -> Result, String> { + db.transaction().map_err(sql) +} +pub(super) fn delete(db: &Connection, id: &str) -> Result<(), String> { + valid_id(id)?; + db.execute("DELETE FROM books WHERE id=?1", [id]) + .map_err(sql)?; + Ok(()) +} + +/// Archives stay JSON based and version-compatible. Stream chapters one row at a time. +pub(super) fn export( + db: &Connection, + zip: &mut zip::ZipWriter, + report: &mut dyn FnMut(&str, u64, u64), +) -> Result { + let details = details(db)?; + let total = details.len() as u64; + report("books", 0, total); + for (index, d) in details.iter().enumerate() { + let prefix = format!("books/{}/", d.id); + zip.start_file( + format!("{prefix}{BOOKDETAIL_FILE}"), + crate::data_transfer::archive::text_options(None), + ) + .map_err(|e| e.to_string())?; + serde_json::to_writer(&mut *zip, d).map_err(|e| e.to_string())?; + zip.start_file( + format!("{prefix}{CONTENT_FILE}"), + crate::data_transfer::archive::text_options(None), + ) + .map_err(|e| e.to_string())?; + zip.write_all(b"{\"schemaVersion\":1,\"chapters\":[") + .map_err(|e| e.to_string())?; + let mut statement = db + .prepare("SELECT chapter FROM chapters WHERE book_id=?1 ORDER BY position") + .map_err(sql)?; + let rows = statement + .query_map([&d.id], |r| r.get::<_, String>(0)) + .map_err(sql)?; + for (i, row) in rows.enumerate() { + if i > 0 { + zip.write_all(b",").map_err(|e| e.to_string())?; + } + zip.write_all(row.map_err(sql)?.as_bytes()) + .map_err(|e| e.to_string())?; + } + zip.write_all(b"]}").map_err(|e| e.to_string())?; + for (file, table, field) in [ + (BOOKMARKS_FILE, "bookmarks", "bookmarks"), + (ANNOTATIONS_FILE, "annotations", "annotations"), + ] { + zip.start_file( + format!("{prefix}{file}"), + crate::data_transfer::archive::text_options(None), + ) + .map_err(|e| e.to_string())?; + let value = serde_json::json!({"schemaVersion":1,(field):records(db, table, &d.id)?}); + serde_json::to_writer(&mut *zip, &value).map_err(|e| e.to_string())?; + } + report("books", index as u64 + 1, total); + } + Ok(total) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + fn root(name: &str) -> PathBuf { + let root = + std::env::temp_dir().join(format!("readerx-sqlite-{name}-{}", readerx_sync::new_id())); + fs::create_dir_all(&root).unwrap(); + root + } + fn book(id: &str) -> LocalBook { + serde_json::from_value(json!({"id":id,"title":"测试书","author":"作者","format":"epub", + "fileName":"book.epub","size":123,"importedAt":4,"hue":7,"splitDesc":"分章", + "tags":["科幻"],"sourceTags":["标签"],"intro":"简介","bookSourceId":"源", + "bookUrl":"https://example.com/book","groupId":"组","source":"webdav","cover":"data:image/png;base64,AAAA", + "chapters":[{"cid":"c1","title":"标题","url":"https://example.com/c1","paragraphs":["原文😀"]}, + {"cid":"c2","title":"第二章","paragraphs":["保留正文"]}]})).unwrap() + } + fn legacy(root: &Path, book: &LocalBook) { + let dir = root.join("books").join(&book.id); + fs::create_dir_all(&dir).unwrap(); + write_json_atomic( + &dir.join(BOOKDETAIL_FILE), + &BookDetail::from_book(book), + "夹具", + ) + .unwrap(); + write_json_atomic( + &dir.join(CONTENT_FILE), + &BookContent { + schema_version: 1, + chapters: book.chapters.clone(), + }, + "夹具", + ) + .unwrap(); + } + #[test] + fn sqlite_round_trip_preserves_fields_records_and_derived_data() { + let root = root("roundtrip"); + let mut db = open(&root).unwrap(); + let book = book("b1"); + let tx = transaction(&mut db).unwrap(); + put(&tx, &book).unwrap(); + commit(tx).unwrap(); + let records = vec![json!({"id":"mark","unknown":{"color":"red"}})]; + put_records(&db, "bookmarks", "b1", &records).unwrap(); + put_records(&db, "annotations", "b1", &records).unwrap(); + assert_eq!( + serde_json::to_value(get(&db, "b1").unwrap().unwrap()).unwrap(), + serde_json::to_value(&book).unwrap() + ); + assert_eq!(metas(&db).unwrap()[0].chapters[0].chars, 4); + assert_eq!(records, super::records(&db, "bookmarks", "b1").unwrap()); + assert_eq!( + digests(&db, "b1").unwrap()[0].hash, + chapter_digest(&book.chapters[0]).hash + ); + drop(db); + assert!(!root.join("books").exists()); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn chapter_patch_is_incremental_and_batch_failure_rolls_back() { + let root = root("patch"); + let mut db = open(&root).unwrap(); + put(&db, &book("b1")).unwrap(); + let before: String = db + .query_row( + "SELECT chapter FROM chapters WHERE book_id='b1' AND cid='c2'", + [], + |r| r.get(0), + ) + .unwrap(); + let mut changed = book("b1").chapters.remove(0); + changed.title = "过期标题".into(); + changed.paragraphs = vec!["新正文".into()]; + let update = BookChapterPatch { + index: 1, + chapter: changed, + }; + let mut invalid = update.clone(); + invalid.chapter.cid = "missing".into(); + assert!(patch(&mut db, &root, "b1", &[update.clone(), invalid]).is_err()); + assert_eq!( + get(&db, "b1").unwrap().unwrap().chapters[0].paragraphs, + vec!["原文😀"] + ); + patch(&mut db, &root, "b1", &[update]).unwrap(); + let after: String = db + .query_row( + "SELECT chapter FROM chapters WHERE book_id='b1' AND cid='c2'", + [], + |r| r.get(0), + ) + .unwrap(); + assert_eq!(before, after); + assert_eq!(get(&db, "b1").unwrap().unwrap().chapters[0].title, "标题"); + assert_eq!(picked(&db, "b1", &["c1".into()]).unwrap().len(), 1); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn migration_keeps_failures_and_retries_without_affecting_other_books() { + let root = root("migrate"); + legacy(&root, &book("good")); + legacy(&root, &book("bad")); + let bad = root.join("books/bad/annotations.json"); + fs::write(&bad, "broken").unwrap(); + let db = open(&root).unwrap(); + assert!(detail(&db, "good").unwrap().is_some()); + assert!(detail(&db, "bad").unwrap().is_none()); + assert!(root.join("books/good/bookdetail.json.migrated").is_file()); + assert!(root.join("books/bad/bookdetail.json").is_file()); + assert!(require_book(&db, &root, "bad").is_err()); + fs::write(&bad, r#"{"schemaVersion":1,"annotations":[]}"#).unwrap(); + drop(db); + let db = open(&root).unwrap(); + assert!(detail(&db, "bad").unwrap().is_some()); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn interrupted_cleanup_never_replays_json_over_new_database_edits() { + let root = root("receipt"); + legacy(&root, &book("b1")); + // A filesystem obstacle causes cleanup to stop after the committed transaction. + let obstacle = root.join("books/b1/bookdetail.json.migrated"); + fs::create_dir(&obstacle).unwrap(); + let db = open(&root).unwrap(); + let mut detail = detail(&db, "b1").unwrap().unwrap(); + detail.title = "数据库新书名".into(); + save_detail(&db, &detail).unwrap(); + drop(db); + fs::remove_dir(&obstacle).unwrap(); + let db = open(&root).unwrap(); + assert_eq!( + super::detail(&db, "b1").unwrap().unwrap().title, + "数据库新书名" + ); + assert!(!root.join("books/b1/bookdetail.json").exists()); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn old_flat_books_global_bookmarks_and_excluded_files_survive_upgrade() { + let root = root("flat"); + fs::create_dir_all(root.join("books")).unwrap(); + fs::create_dir_all(root.join("state")).unwrap(); + fs::write( + root.join("books/b1.json"), + serde_json::to_vec(&book("b1")).unwrap(), + ) + .unwrap(); + fs::write( + root.join("state/readerx.bookmarks.json"), + json!({"b1":[{"id":"mark","bookId":"b1"}]}).to_string(), + ) + .unwrap(); + for name in [ + "state/preferences.json", + "book_sources/s1.json", + "source_sessions/s1.json", + "images/image.png", + "tts-audio/b1/audio.mp3", + "books/other.json.tmp", + ] { + let path = root.join(name); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, b"unchanged").unwrap(); + } + let db = open(&root).unwrap(); + assert_eq!(records(&db, "bookmarks", "b1").unwrap()[0]["id"], "mark"); + assert!(root.join("books/b1.json.migrated").is_file()); + assert!(root.join("state/readerx.bookmarks.json.migrated").is_file()); + for name in [ + "state/preferences.json", + "book_sources/s1.json", + "source_sessions/s1.json", + "images/image.png", + "tts-audio/b1/audio.mp3", + "books/other.json.tmp", + ] { + assert_eq!(fs::read(root.join(name)).unwrap(), b"unchanged"); + } + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn deletion_cascades_and_data_roots_are_isolated() { + let first = root("first"); + let second = root("second"); + let db = open(&first).unwrap(); + let peer = open(&second).unwrap(); + put(&db, &book("b1")).unwrap(); + put_records(&db, "bookmarks", "b1", &[json!({"id":"m"})]).unwrap(); + put_records(&db, "annotations", "b1", &[json!({"id":"n"})]).unwrap(); + assert!(detail(&peer, "b1").unwrap().is_none()); + delete(&db, "b1").unwrap(); + for table in ["chapters", "bookmarks", "annotations"] { + let count: i64 = db + .query_row(&format!("SELECT count(*) FROM {table}"), [], |r| r.get(0)) + .unwrap(); + assert_eq!(count, 0); + } + drop(db); + drop(peer); + fs::remove_dir_all(first).unwrap(); + fs::remove_dir_all(second).unwrap(); + } + #[test] + fn failed_sql_write_leaves_metadata_body_and_fingerprint_together() { + let root = root("rollback"); + let mut db = open(&root).unwrap(); + put(&db, &book("b1")).unwrap(); + db.execute_batch("CREATE TRIGGER fail_second BEFORE INSERT ON chapters WHEN NEW.cid='c2' BEGIN SELECT RAISE(ABORT,'injected'); END").unwrap(); + let before = serde_json::to_value(get(&db, "b1").unwrap()).unwrap(); + let hashes = digests(&db, "b1").unwrap(); + { + let tx = transaction(&mut db).unwrap(); + let mut changed = book("b1"); + changed.title = "失败的更改".into(); + changed.chapters[0].paragraphs = vec!["未提交".into()]; + assert!(put(&tx, &changed).is_err()); + } + assert_eq!( + before, + serde_json::to_value(get(&db, "b1").unwrap()).unwrap() + ); + assert_eq!(hashes[0].hash, digests(&db, "b1").unwrap()[0].hash); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn export_uses_portable_json_and_omits_database_bytes() { + let root = root("export"); + let db = open(&root).unwrap(); + let original = book("b1"); + put(&db, &original).unwrap(); + let file = fs::File::create(root.join("backup.zip")).unwrap(); + let mut zip = zip::ZipWriter::new(file); + assert_eq!(export(&db, &mut zip, &mut |_, _, _| {}).unwrap(), 1); + drop(zip.finish().unwrap()); + let file = fs::File::open(root.join("backup.zip")).unwrap(); + let mut archive = zip::ZipArchive::new(file).unwrap(); + assert!(archive.by_name(DATABASE).is_err()); + let detail: BookDetail = + serde_json::from_reader(archive.by_name("books/b1/bookdetail.json").unwrap()).unwrap(); + let content: BookContent = + serde_json::from_reader(archive.by_name("books/b1/content.json").unwrap()).unwrap(); + assert_eq!( + serde_json::to_value(detail.into_book(content.chapters)).unwrap(), + serde_json::to_value(original).unwrap() + ); + let records: Value = + serde_json::from_reader(archive.by_name("books/b1/annotations.json").unwrap()).unwrap(); + assert_eq!(records["annotations"], json!([])); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn sqlite_identity_migration_is_idempotent_and_updates_bookmark_references() { + let root = root("ids"); + let db = open(&root).unwrap(); + put(&db, &book("old")).unwrap(); + put_records( + &db, + "bookmarks", + "old", + &[json!({"id":"mark","bookId":"old"})], + ) + .unwrap(); + let hash = digests(&db, "old").unwrap()[0].hash.clone(); + drop(db); + let new = "b-0123456789abcdef"; + super::super::rename_id_at(&root, "old", new).unwrap(); + super::super::rename_id_at(&root, "old", new).unwrap(); + let db = open(&root).unwrap(); + assert!(detail(&db, "old").unwrap().is_none()); + assert_eq!(records(&db, "bookmarks", new).unwrap()[0]["bookId"], new); + assert_eq!(digests(&db, new).unwrap()[0].hash, hash); + put(&db, &book("collision")).unwrap(); + drop(db); + assert!(super::super::rename_id_at(&root, "collision", new).is_err()); + let db = open(&root).unwrap(); + assert!(detail(&db, "collision").unwrap().is_some()); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn sqlite_metadata_reference_remaps_do_not_touch_chapter_bodies() { + let root = root("refs"); + let db = open(&root).unwrap(); + put(&db, &book("b1")).unwrap(); + let hash = digests(&db, "b1").unwrap()[0].hash.clone(); + drop(db); + super::super::remap_metadata_at( + &root, + "groupId", + &std::collections::BTreeMap::from([("组".into(), "g-new".into())]), + ) + .unwrap(); + super::super::remap_metadata_at( + &root, + "bookSourceId", + &std::collections::BTreeMap::from([("源".into(), "s-new".into())]), + ) + .unwrap(); + let db = open(&root).unwrap(); + let d = detail(&db, "b1").unwrap().unwrap(); + assert_eq!(d.group_id.as_deref(), Some("g-new")); + assert_eq!(d.book_source_id.as_deref(), Some("s-new")); + assert_eq!(digests(&db, "b1").unwrap()[0].hash, hash); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn bad_legacy_metadata_defers_identity_migration_without_hiding_healthy_books() { + let root = root("bad-detail-ids"); + let db = open(&root).unwrap(); + put(&db, &book("healthy")).unwrap(); + let bad = root.join("books/bad/bookdetail.json"); + fs::create_dir_all(bad.parent().unwrap()).unwrap(); + fs::write(&bad, b"{broken").unwrap(); + readerx_source::id_migration::write( + &root.join("state/readerx.groups.json"), + &json!([{"id":"组","name":"旧分组"}]), + ) + .unwrap(); + crate::sync::data_ids::migrate(&root).unwrap(); + assert_eq!(fs::read(&bad).unwrap(), b"{broken"); + assert_eq!( + detail(&db, "healthy").unwrap().unwrap().group_id.as_deref(), + Some("组") + ); + fs::remove_file(bad).unwrap(); + crate::sync::data_ids::migrate(&root).unwrap(); + assert_eq!( + detail(&db, "healthy").unwrap().unwrap().group_id.as_deref(), + Some(crate::sync::identity::group_uid("旧分组").as_str()) + ); + drop(db); + fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn source_save_and_interrupted_alias_journal_update_sqlite_references() { + let root = root("source-save"); + let db = open(&root).unwrap(); + let mut original = book("b1"); + original.book_source_id = Some("old-source".into()); + put(&db, &original).unwrap(); + let source = serde_json::from_value(json!({"schemaVersion":1,"id":"s-0123456789abcdef", + "name":"书源","bookSourceUrl":"https://example.com","js":""})) + .unwrap(); + crate::sync::data_ids::save_source(&root, &source, "old-source").unwrap(); + assert_eq!( + detail(&db, "b1") + .unwrap() + .unwrap() + .book_source_id + .as_deref(), + Some("s-0123456789abcdef") + ); + // Simulate a restart after source files changed but before DB reference updates. + let mut d = detail(&db, "b1").unwrap().unwrap(); + d.book_source_id = Some("interrupted".into()); + save_detail(&db, &d).unwrap(); + readerx_source::id_migration::write( + &root.join("migrations/readerx.sqliteSourceRefs.json"), + &json!({"interrupted":"s-0123456789abcdef"}), + ) + .unwrap(); + crate::sync::data_ids::migrate_sources(&root, &Default::default()).unwrap(); + assert_eq!( + detail(&db, "b1") + .unwrap() + .unwrap() + .book_source_id + .as_deref(), + Some("s-0123456789abcdef") + ); + assert!(!root + .join("migrations/readerx.sqliteSourceRefs.json") + .exists()); + drop(db); + fs::remove_dir_all(root).unwrap(); + } +} diff --git a/src-tauri/src/commands.rs b/src-tauri/src/commands.rs index ca12a1e..193d70a 100644 --- a/src-tauri/src/commands.rs +++ b/src-tauri/src/commands.rs @@ -150,7 +150,7 @@ pub async fn readerx_book_get(app: AppHandle, id: String) -> Result/bookmarks.json）；没有书签返回空列表。 +/// 读取某本书的数据库书签；没有书签返回空列表。 /// 书签按书分文件，读写都不必碰正文，也不必碰其它书的书签。 #[tauri::command] pub async fn readerx_bookmarks_get(app: AppHandle, book_id: String) -> Result, String> { @@ -437,7 +437,7 @@ pub async fn readerx_source_put(app: AppHandle, mut source: BookSource) -> Resul validate_source(&source)?; let old = source.id.clone(); source.id = sync::identity::entity_id(&old, "s-", sync::identity::source_uid(&source.book_source_url)); - readerx_source::id_migration::save_source(&storage::data_root(&app)?, &source, &old)?; + sync::data_ids::save_source(&storage::data_root(&app)?, &source, &old)?; sync::service_hook(&app).on_source_changed(&source); Ok(source) }) diff --git a/src-tauri/src/data_transfer/archive.rs b/src-tauri/src/data_transfer/archive.rs index 122f626..96d46af 100644 --- a/src-tauri/src/data_transfer/archive.rs +++ b/src-tauri/src/data_transfer/archive.rs @@ -125,7 +125,7 @@ pub(super) fn writer(file: File) -> ZipWriter { } /// 文本类条目（JSON）的压缩参数 -pub(super) fn text_options(source: Option<&Path>) -> SimpleFileOptions { +pub(crate) fn text_options(source: Option<&Path>) -> SimpleFileOptions { file_options(source, CompressionMethod::Deflated) } diff --git a/src-tauri/src/data_transfer/export.rs b/src-tauri/src/data_transfer/export.rs index a9284e5..0a85ef2 100644 --- a/src-tauri/src/data_transfer/export.rs +++ b/src-tauri/src/data_transfer/export.rs @@ -3,12 +3,11 @@ //! 采集口径与 `data_transfer/mod.rs` 的文件头一致：书籍（元信息 / 正文 / 书签）、 //! 状态文件、插图、书源，外加**按需**的书源登录态。听书缓存、日志、同步目录不进来。 //! -//! **正文以字节流写进归档**（`std::io::copy`，不解析）：一本几百兆的书不会在内存里 -//! 再复制一份，导出内存占用与书库体积无关。 +//! 数据库正文按章节行流式写入归档，文件资源用 `std::io::copy`。 +//! 不为导出复制整本或整库正文。 use super::{archive, ExportOptions, ExportSummary, Manifest, BACKUP_FORMAT}; -use super::{BOOKS_DIR, IMAGES_DIR, SESSIONS_DIR, SOURCES_DIR, STATE_DIR}; -use std::collections::HashSet; +use super::{IMAGES_DIR, SESSIONS_DIR, SOURCES_DIR, STATE_DIR}; use std::fs::{self, File}; use std::io::Write; use std::path::{Path, PathBuf}; @@ -37,8 +36,7 @@ pub(super) fn export_to( let root = crate::storage::data_root(app)?; let mut state = Vec::new(); walk(&root.join(STATE_DIR), STATE_DIR, &is_state_file, &mut state)?; - let mut books = Vec::new(); - walk(&root.join(BOOKS_DIR), BOOKS_DIR, &is_data_file, &mut books)?; + let book_count = crate::book_store::list_book_meta(app)?.len() as u64; let mut images = Vec::new(); walk( &root.join(IMAGES_DIR), @@ -62,17 +60,10 @@ pub(super) fn export_to( &mut sessions, )?; } - for list in [ - &mut state, - &mut books, - &mut images, - &mut sources, - &mut sessions, - ] { + for list in [&mut state, &mut images, &mut sources, &mut sessions] { list.sort_by(|a, b| a.name.cmp(&b.name)); } - let book_count = distinct_books(&books); let manifest = Manifest { format: BACKUP_FORMAT.to_string(), app_version: app.package_info().version.to_string(), @@ -96,7 +87,6 @@ pub(super) fn export_to( for (category, entries) in [ ("state", &state), - ("books", &books), ("images", &images), ("sources", &sources), ("sessions", &sessions), @@ -104,6 +94,8 @@ pub(super) fn export_to( write_entries(&mut zip, entries, category, report)?; } + crate::book_store::export_books(app, &mut zip, report)?; + let file = zip.finish().map_err(|e| format!("写入备份文件失败: {e}"))?; let bytes = file.metadata().map(|meta| meta.len()).unwrap_or(0); log::info!( @@ -181,19 +173,6 @@ fn redacted_webdav(path: &Path) -> Result, String> { Ok(text.into_bytes()) } -/// 只看一层：归档里 `books//...` 的书籍本数 -fn distinct_books(entries: &[Entry]) -> u64 { - let mut ids: HashSet<&str> = HashSet::new(); - for entry in entries { - let mut parts = entry.name.splitn(3, '/'); - parts.next(); - if let Some(id) = parts.next() { - ids.insert(id); - } - } - ids.len() as u64 -} - /// 状态文件：`.json`，key 必须合法（与 `readerx_state_get` 同一口径） fn is_state_file(name: &str) -> bool { name.strip_suffix(".json") @@ -208,7 +187,7 @@ fn is_id_file(name: &str) -> bool { .unwrap_or(false) } -/// 书籍目录与插图目录里的普通数据文件（写盘中途留下的临时文件不要） +/// 插图目录里的普通数据文件（写盘中途留下的临时文件不要） fn is_data_file(name: &str) -> bool { !name.starts_with('.') && !name.ends_with(".tmp") && !name.ends_with(".migrated") } diff --git a/src-tauri/src/data_transfer/import.rs b/src-tauri/src/data_transfer/import.rs index 99e6511..4923fde 100644 --- a/src-tauri/src/data_transfer/import.rs +++ b/src-tauri/src/data_transfer/import.rs @@ -80,15 +80,35 @@ pub(super) fn apply( )?; } - let aliases = archive_sources.iter().map(|source| { - let local = if mode == ImportMode::Merge { - local_sources.iter().find(|local| same_source(&local.book_source_url, &source.url)) - } else { None }; - let target = local.map(|local| crate::sync::identity::entity_id(&local.id, "s-", crate::sync::identity::source_uid(&local.book_source_url))) - .unwrap_or_else(|| crate::sync::identity::entity_id(&source.id, "s-", crate::sync::identity::source_uid(&source.url))); - (source.id.clone(), target) - }).collect(); - readerx_source::id_migration::migrate(&storage::data_root(app)?, &aliases)?; + let aliases = archive_sources + .iter() + .map(|source| { + let local = if mode == ImportMode::Merge { + local_sources + .iter() + .find(|local| same_source(&local.book_source_url, &source.url)) + } else { + None + }; + let target = local + .map(|local| { + crate::sync::identity::entity_id( + &local.id, + "s-", + crate::sync::identity::source_uid(&local.book_source_url), + ) + }) + .unwrap_or_else(|| { + crate::sync::identity::entity_id( + &source.id, + "s-", + crate::sync::identity::source_uid(&source.url), + ) + }); + (source.id.clone(), target) + }) + .collect(); + crate::sync::data_ids::migrate_sources(&storage::data_root(app)?, &aliases)?; crate::sync::book_ids::migrate(app)?; crate::sync::data_ids::migrate(&storage::data_root(app)?)?; @@ -122,7 +142,6 @@ fn write_books( summary: &mut ImportSummary, report: &mut dyn FnMut(&str, u64, u64), ) -> Result<(), String> { - let root = storage::data_root(app)?.join(BOOKS_DIR); report("books", 0, plan.len() as u64); for (index, item) in plan.iter().enumerate() { match item.action { @@ -131,32 +150,31 @@ fn write_books( BookAction::Skip => summary.books_skipped += 1, } if item.action != BookAction::Skip { - let dir = root.join(&item.local_id); - // 元信息先落盘，复用 write_bytes 的父目录创建；注释合并不能依赖归档字母顺序。 - let entries = item.entries.iter() - .filter(|name| name.ends_with("/bookdetail.json")) - .chain(item.entries.iter().filter(|name| !name.ends_with("/bookdetail.json"))); - for name in entries { - let Some(file) = name.rsplit('/').next() else { + let prefix = format!("{BOOKS_DIR}/{}/", item.archive_id); + let detail = archive::read_entry(zip, &format!("{prefix}bookdetail.json"))? + .ok_or("备份缺少书籍元信息")?; + let content = archive::read_entry(zip, &format!("{prefix}content.json"))?; + let detail = remap_detail(&detail, remap, mode)?; + book_store::import_book_json(app, &item.local_id, &detail, content.as_deref())?; + for name in &item.entries { + let file = name.rsplit('/').next().unwrap_or_default(); + if !matches!(file, "annotations.json" | "bookmarks.json") { continue; - }; + } let Some(bytes) = archive::read_entry(zip, name)? else { continue; }; - match file { - "bookdetail.json" => { - let value = remap_detail(&bytes, remap, mode)?; - write_bytes(&dir.join(file), &value)?; - } - "annotations.json" => { - merge_annotation_file(app, &item.local_id, &bytes, mode)?; - } - "bookmarks.json" => { - merge_bookmark_file(app, &item.local_id, &bytes, mode)?; - } - _ => write_bytes(&dir.join(file), &bytes)?, + if file == "annotations.json" { + merge_annotation_file(app, &item.local_id, &bytes, mode)?; + } else { + merge_bookmark_file(app, &item.local_id, &bytes, mode)?; } } + if mode == ImportMode::Replace + && !item.entries.iter().any(|n| n.ends_with("/bookmarks.json")) + { + book_store::put_bookmarks(app, &item.local_id, &[])?; + } } // 本机已有同一本书（另一个 id）：归档里的书签仍要并进本机那本，不能丢 if item.action == BookAction::Skip { @@ -474,7 +492,7 @@ fn remove_extras( if keep_books.contains(book.id.as_str()) { continue; } - // 同步删除要读 bookdetail.json 定位身份，必须在删文件之前 + // 同步删除要读书籍元信息定位身份，必须在删书之前 hook.on_book_deleted(&book.id); book_store::delete_book(app, &book.id)?; summary.books_removed += 1; diff --git a/src-tauri/src/data_transfer/mod.rs b/src-tauri/src/data_transfer/mod.rs index 6659b25..a78098d 100644 --- a/src-tauri/src/data_transfer/mod.rs +++ b/src-tauri/src/data_transfer/mod.rs @@ -9,7 +9,7 @@ //! 导入的语义只有「合并」与「覆盖恢复」两种（见 [`ImportMode`]）。 //! //! 归档就是一个 **zip 文件**（后缀 `.zip`，正文体积大必须压缩；同时用户能自己解开检查）， -//! 内部路径与应用数据目录一一对应： +//! 保持原来的可移植 JSON 路径；书籍由 SQLite 转换为这些条目，其他数据按文件导出： //! //! ```text //! readerx-backup.json 清单：格式版本、导出版本、时间、是否含登录信息、各类条目数 @@ -41,7 +41,7 @@ //! 界面与文档都要说清「这份文件里有账号登录信息，别随手分享」。日志里永远只有数量， //! 没有内容（与全项目口径一致）。 -mod archive; +pub(crate) mod archive; mod export; mod import; mod merge; @@ -59,7 +59,7 @@ use serde::{Deserialize, Serialize}; pub(crate) const MANIFEST_NAME: &str = "readerx-backup.json"; /// 归档格式标识；读到更高版本直接报错，而不是按旧语义读坏新数据 pub(crate) const BACKUP_FORMAT: &str = "readerx-backup/1"; -/// 归档内的顶层目录名（与数据目录同名，一一对应） +/// 归档内的顶层分类；books 是数据库书籍转换出的便携 JSON pub(crate) const STATE_DIR: &str = "state"; pub(crate) const BOOKS_DIR: &str = "books"; pub(crate) const IMAGES_DIR: &str = "images"; diff --git a/src-tauri/src/data_transfer/plan.rs b/src-tauri/src/data_transfer/plan.rs index ac26227..be94ebe 100644 --- a/src-tauri/src/data_transfer/plan.rs +++ b/src-tauri/src/data_transfer/plan.rs @@ -60,7 +60,7 @@ pub(super) struct BookPlan { pub(super) entries: Vec, } -/// 本机书籍 id 与身份（读 `bookdetail.json`，不碰正文） +/// 本机书籍 id 与身份（读数据库元信息，不碰正文） pub(super) fn local_books(app: &AppHandle) -> Result, String> { let metas = book_store::list_sync_meta(app)?; // 书源地址只读一次：书籍身份要用它，逐本去列书源会变成 O(书 × 源) diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index 83f415b..ad3d150 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -56,18 +56,18 @@ fn setup() -> (tauri::AppHandle, PathBuf) { ] { let _ = fs::remove_dir_all(data_root.join(sub)); } + fs::create_dir_all(&data_root).unwrap(); + let _ = std::fs::remove_file(data_root.join("books.sqlite3")); (handle, data_root) } +#[path = "../../tests/support/book_json.rs"] +mod book_json; fn write_json(path: &Path, value: &Value) { - if let Some(parent) = path.parent() { - fs::create_dir_all(parent).unwrap(); - } - fs::write(path, serde_json::to_string_pretty(value).unwrap()).unwrap(); + book_json::write(path, value); } - fn read_json(path: &Path) -> Value { - serde_json::from_str(&fs::read_to_string(path).unwrap()).unwrap() + book_json::read(path) } /// 造一本本地书（与 `book_store.rs` 的目录布局一致） @@ -258,6 +258,7 @@ fn export_and_merge_import_restores_everything() { ] { let _ = fs::remove_dir_all(root.join(sub)); } + let _ = fs::remove_file(root.join("books.sqlite3")); let summary = import(&app, &backup, ImportMode::Merge); assert_eq!(summary.books_added, 2); assert_eq!(summary.sources_added, 1); @@ -293,11 +294,17 @@ fn export_and_merge_import_restores_everything() { "卷首" ); assert_eq!( - read_json(&root.join(format!("book_sources/{}.json", crate::sync::identity::source_uid("https://a.example.com"))))["bookSourceUrl"], + read_json(&root.join(format!( + "book_sources/{}.json", + crate::sync::identity::source_uid("https://a.example.com") + )))["bookSourceUrl"], "https://a.example.com" ); assert_eq!( - read_json(&root.join(format!("source_sessions/{}.json", crate::sync::identity::source_uid("https://a.example.com"))))["cookie"], + read_json(&root.join(format!( + "source_sessions/{}.json", + crate::sync::identity::source_uid("https://a.example.com") + )))["cookie"], "sid=secret" ); // 偏好类状态原样回来（换机时是「本机没有」→ 取归档那份） @@ -338,9 +345,17 @@ fn export_without_credentials_leaves_login_state_and_passwords_out() { // 导入后本机的登录态不被删（归档没带它，不等于「用户不要了」） let summary = import(&app, &backup, ImportMode::Merge); assert_eq!(summary.books_updated, 2); - assert!(root.join(format!("source_sessions/{}.json", crate::sync::identity::source_uid("https://a.example.com"))).is_file()); + assert!(root + .join(format!( + "source_sessions/{}.json", + crate::sync::identity::source_uid("https://a.example.com") + )) + .is_file()); assert_eq!( - read_json(&root.join(format!("source_sessions/{}.json", crate::sync::identity::source_uid("https://a.example.com"))))["cookie"], + read_json(&root.join(format!( + "source_sessions/{}.json", + crate::sync::identity::source_uid("https://a.example.com") + )))["cookie"], "sid=secret" ); } @@ -377,7 +392,10 @@ fn merge_keeps_local_extras_and_newer_progress() { assert_eq!(summary.books_removed, 0); // 本机多出来的书一本都不动 - assert!(root.join("books/b-3e75c5b1cc830167/bookdetail.json").is_file()); + assert_eq!( + read_json(&root.join("books/b-3e75c5b1cc830167/bookdetail.json"))["title"], + "超新星纪元" + ); // 进度按 updatedAt 取新的：本机那份更靠后 → 保留本机 let shelf = read_json(&root.join("state/readerx.shelf.json")); assert_eq!(shelf["b-5e20dcb39c59dc92"]["chapter"], 9); @@ -467,6 +485,7 @@ fn merge_matches_the_same_book_from_another_device_by_identity() { ] { let _ = fs::remove_dir_all(root.join(sub)); } + fs::remove_file(root.join("books.sqlite3")).unwrap(); seed_book(&root, "local-other", "三体", "三体.epub", 1024, None); seed_state( &root, @@ -536,6 +555,25 @@ fn archive_read_json(path: &Path, entry: &str) -> Value { .clone() } +fn omit_archive_entry(path: &Path, omitted: &str) { + let mut original = zip::ZipArchive::new(fs::File::open(path).unwrap()).unwrap(); + let temporary = path.with_extension("test.zip"); + let mut rewritten = zip::ZipWriter::new(fs::File::create(&temporary).unwrap()); + for index in 0..original.len() { + let mut entry = original.by_index(index).unwrap(); + if entry.name() == omitted { + continue; + } + rewritten + .start_file(entry.name(), archive::text_options(None)) + .unwrap(); + std::io::copy(&mut entry, &mut rewritten).unwrap(); + } + drop(rewritten.finish().unwrap()); + drop(original); + fs::rename(temporary, path).unwrap(); +} + #[test] fn annotations_survive_archive_merge_replace_and_old_backup() { let _guard = SERIAL.lock().unwrap_or_else(|e| e.into_inner()); @@ -550,6 +588,7 @@ fn annotations_survive_archive_merge_replace_and_old_backup() { ); let backup = export(&app, &root, "annotations", false); fs::remove_dir_all(root.join("books")).unwrap(); + fs::remove_file(root.join("books.sqlite3")).unwrap(); import(&app, &backup, ImportMode::Merge); assert_eq!(read_json(&path)["annotations"][0], paragraph); let mut local = paragraph.clone(); @@ -568,8 +607,9 @@ fn annotations_survive_archive_merge_replace_and_old_backup() { ); import(&app, &backup, ImportMode::Replace); assert_eq!(read_json(&path)["annotations"][0], paragraph); - fs::remove_file(&path).unwrap(); let old_backup = export(&app, &root, "without-annotations", false); + // Reproduce a backup written before annotations existed, rather than merely storing an empty list. + omit_archive_entry(&old_backup, &format!("books/{id}/annotations.json")); write_json(&path, &json!({"schemaVersion":1,"annotations":[paragraph]})); import(&app, &old_backup, ImportMode::Replace); assert!(read_json(&path)["annotations"] diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index 3d2b256..c538644 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -138,7 +138,7 @@ pub fn run() { log::warn!("启动清理存储临时文件失败：{error}"); } // 本地书的旧布局（整本 books/.json、全库一份的 state/readerx.bookmarks.json） - // 先转为目录布局，再统一书籍 ID；必须早于界面与同步线程访问书库。 + // 逐本迁入 SQLite，再统一书籍 ID；必须早于界面与同步线程访问书库。 sync::book_ids::migrate(app.handle()).map_err(std::io::Error::other)?; sync::data_ids::migrate(&storage::data_root(app.handle())?).map_err(std::io::Error::other)?; // 网页登录后端：把「插件（Android 原生浮层 / 桌面独立登录窗口）」注册为引擎的认证实现 diff --git a/src-tauri/src/storage.rs b/src-tauri/src/storage.rs index 695d11f..dbaa790 100644 --- a/src-tauri/src/storage.rs +++ b/src-tauri/src/storage.rs @@ -1,7 +1,7 @@ //! 持久化层（App 侧）：偏好 / 书架 / 分章规则等状态、听书缓存、以及应用数据根目录。 //! //! - 状态按 key 存为 JSON 文件（`state/.json`）； -//! - **本地书籍**按 `books//` 目录存（元信息 / 正文 / 书签分文件，见 `book_store.rs`）； +//! - **本地书籍**存入 `books.sqlite3`（见 `book_store.rs` 和 `book_store/sqlite.rs`）； //! - **书源与书源登录态**由 `readerx-source` crate 实现（见文件末尾的转发段）。 //! //! 全部为同步磁盘 I/O，仅对 `commands` 暴露；WebView 侧只通过 command 访问。 diff --git a/src-tauri/src/sync/book_ids.rs b/src-tauri/src/sync/book_ids.rs index b45cd6d..4de1965 100644 --- a/src-tauri/src/sync/book_ids.rs +++ b/src-tauri/src/sync/book_ids.rs @@ -42,6 +42,10 @@ pub fn migrate(app: &AppHandle) -> Result<(), String> { let mut plan = BTreeMap::new(); let mut targets = BTreeMap::new(); for book in books { + if !book_store::contains_at(app, None, &book.id)? { + log::warn!("书籍尚未迁入数据库，保留旧 ID 等待重试 id={}", book.id); + continue; + } let uid = bridge::book_uid_of(app, &book); if targets.insert(uid.clone(), book.id.clone()).is_some() { return Err("书库存在重复同步身份，请先处理重复书籍；原数据已保留".into()); @@ -54,8 +58,8 @@ pub fn migrate(app: &AppHandle) -> Result<(), String> { return Ok(()); } // 必须在任何修改之前验证，不能用远端身份覆盖无关本地目录。 - for (old, new) in &plan { - if !root.join("books").join(old).is_dir() || root.join("books").join(new).exists() { + for new in plan.values() { + if book_store::contains_at(app, None, new)? { return Err("书籍 ID 迁移目录冲突或旧布局未迁移完成；原数据已保留".into()); } } @@ -74,24 +78,29 @@ fn apply(root: &Path, plan: &BTreeMap) -> Result<(), String> { if !storage::valid_component(old) || !canonical_id(new) { return Err("书籍 ID 迁移待办包含非法 ID".into()); } - let from = root.join("books").join(old); - let to = root.join("books").join(new); - if from.exists() && to.exists() { - return Err("书籍 ID 迁移目标已存在".into()); - } - let dir = if from.exists() { &from } else { &to }; - let detail = dir.join("bookdetail.json"); - let mut value = read(&detail)?; - value["id"] = Value::String(new.clone()); - atomic_write(&detail, &value)?; - let bookmarks = dir.join("bookmarks.json"); - if bookmarks.is_file() { - let mut value = read(&bookmarks)?; - replace_refs(&mut value, plan); - atomic_write(&bookmarks, &value)?; - } - if from.exists() { - fs::rename(from, to).map_err(|e| format!("迁移书籍目录失败: {e}"))?; + if root.join("books.sqlite3").is_file() { + book_store::rename_id_at(root, old, new)?; + } else { + // Pure legacy fixture / interrupted pre-SQLite migration. + let from = root.join("books").join(old); + let to = root.join("books").join(new); + if from.exists() && to.exists() { + return Err("书籍 ID 迁移目标已存在".into()); + } + let dir = if from.exists() { &from } else { &to }; + let detail = dir.join("bookdetail.json"); + let mut value = read(&detail)?; + value["id"] = Value::String(new.clone()); + atomic_write(&detail, &value)?; + let bookmarks = dir.join("bookmarks.json"); + if bookmarks.is_file() { + let mut value = read(&bookmarks)?; + replace_refs(&mut value, plan); + atomic_write(&bookmarks, &value)?; + } + if from.exists() { + fs::rename(from, to).map_err(|e| e.to_string())?; + } } // 图片文件名是正文引用的一部分，原样保留；听书缓存按书籍目录搬迁。 let cache = root.join("tts-audio").join(old); diff --git a/src-tauri/src/sync/bridge.rs b/src-tauri/src/sync/bridge.rs index e18eb8b..8265634 100644 --- a/src-tauri/src/sync/bridge.rs +++ b/src-tauri/src/sync/bridge.rs @@ -408,7 +408,7 @@ fn publish_book_locked( /// 删除一本书：书实体进墓碑（书签按级联规则一起删），并删掉它的进度实体。 /// -/// **必须在本地删文件之前调用**：定位书身份要读 `bookdetail.json`。 +/// **必须在本地删书之前调用**：定位书身份要读取书籍元信息。 pub fn publish_book_delete( app: &AppHandle, engine: &SharedEngine, @@ -921,7 +921,7 @@ pub(crate) fn reconcile_annotations_mode( if let Ok(Some(rules)) = storage::read_state(app, CHAPTER_RULES_KEY) { publish_chapter_rules(app, engine, &rules)?; } - // 目录（结构）：只扫 content.json 的章节头，不解析正文 —— 首次启用同步时 + // 目录（结构）：只读数据库章节头，不解析正文 —— 首次启用同步时 // 要把现有书库的目录一起灌进引擎，不能按「读整本」的代价来 match book_store::list_sync_structures(app) { Ok(structures) => { @@ -1623,7 +1623,7 @@ fn collect_snapshots( // 本地清单（规则条数少，重建比逐条对账更好推理，也不会漏掉删除） "text_replace" => snapshots.push(Snapshot::TextReplace), "chapter_rule" => snapshots.push(Snapshot::ChapterRule), - // 目录：整份落到本地 content.json（按 cid 复用原有正文） + // 目录：整份落到本地数据库（按 cid 复用原有正文） "book_structure" => snapshots.push(Snapshot::Structure { book_uid: book_ref_of(entity), deleted, @@ -1832,7 +1832,7 @@ fn apply_progress( }) } -/// 按引擎里的书签实体重写某本书的 `bookmarks.json`。 +/// 按引擎里的书签实体重写某本书的数据库书签。 fn apply_bookmarks( app: &AppHandle, engine: &SharedEngine, @@ -2161,7 +2161,7 @@ fn apply_source( return Ok(false); } } - readerx_source::id_migration::save_source(&storage::data_root(app)?, &source, found.map(|old| old.id.as_str()).unwrap_or(uid))?; + super::data_ids::save_source(&storage::data_root(app)?, &source, found.map(|old| old.id.as_str()).unwrap_or(uid))?; Ok(true) } diff --git a/src-tauri/src/sync/content.rs b/src-tauri/src/sync/content.rs index f881214..8243ddb 100644 --- a/src-tauri/src/sync/content.rs +++ b/src-tauri/src/sync/content.rs @@ -1,11 +1,11 @@ //! 正文来源：把 App 的书库接到同步引擎的**正文通道**上（见 `readerx_sync::content`）。 //! -//! 引擎只认识「书实体 id + 章节 cid」，不认识 App 的 `books//content.json` 布局， +//! 引擎只认识「书实体 id + 章节 cid」，不认识 App 的 SQLite 书库， //! 也不知道本机书 id 与书实体 id 的对应关系 —— 这一层就干这两件事： //! //! ```text -//! 引擎问：这本书有哪些章？ → 读 books//digest.json（小文件，写正文时顺手更新） -//! 引擎说：把这几章的正文给我 → 顺序扫 content.json，只留命中的章节 +//! 引擎问：这本书有哪些章？ → 查询数据库章节指纹（与正文同事务更新） +//! 引擎说：把这几章的正文给我 → 通过 cid 索引读取命中的章节 //! ``` //! //! 收到对端的正文不在这里落地（那是 [`super::bridge`] 的事）：引擎先存进自己的暂存区， @@ -17,7 +17,7 @@ //! 各有各的存法，这里负责翻译成引擎认识的资源： //! //! ```text -//! 封面 bookdetail.json 里的 data URL → 名字固定 `cover` +//! 封面 数据库元信息里的 data URL → 名字固定 `cover` //! 插图 images/<设备无关的名字>.png → 名字 = 图片地址的哈希（正文块里就是这么引用的） //! ``` //! @@ -91,8 +91,7 @@ impl AppContent { fn local_id(&self, uid: &str) -> Option { // 迁移后的书直接按同步 ID 定位，新增 / 恢复书籍不受旧索引节流影响。 if super::book_ids::canonical_id(uid) - && crate::storage::data_root_at(&self.app, self.root()).ok()? - .join("books").join(uid).join("bookdetail.json").is_file() + && crate::book_store::contains_at(&self.app, self.root(), uid).ok()? { return Some(uid.to_string()); } diff --git a/src-tauri/src/sync/data_ids.rs b/src-tauri/src/sync/data_ids.rs index a98cb08..948f3b3 100644 --- a/src-tauri/src/sync/data_ids.rs +++ b/src-tauri/src/sync/data_ids.rs @@ -7,7 +7,11 @@ use std::{collections::BTreeMap, path::Path}; pub fn migrate(root: &Path) -> Result<(), String> { const JOURNAL: &str = "migrations/readerx.dataIdMigration.json"; FilePlan::resume(root, JOURNAL)?; - id_migration::migrate(root, &BTreeMap::new())?; + if unreadable_book_details(root)? { + // Keep the original groups/sources until their references can be remapped on retry. + return Ok(()); + } + migrate_sources(root, &BTreeMap::new())?; let mut plan = FilePlan::default(); let mut groups = BTreeMap::new(); let mut source_groups = BTreeMap::new(); @@ -101,12 +105,94 @@ pub fn migrate(root: &Path) -> Result<(), String> { plan.writes .retain(|path, value| id_migration::read(&root.join(path)).ok().as_ref() != Some(value)); let changed = plan.writes.len(); + // Apply DB references before the file plan: failed file writes leave the old groups + // available for deriving the same mapping on retry. + crate::book_store::remap_metadata_at(root, "groupId", &groups)?; plan.commit(root, JOURNAL)?; if changed > 0 { log::info!("同步数据 ID 迁移完成 files={changed}"); } Ok(()) } +/// Journal aliases before source files can be renamed by the source crate. +/// A crash between source migration and DB updates must not lose the old ID mapping. +pub(crate) fn migrate_sources( + root: &Path, + aliases: &BTreeMap, +) -> Result<(), String> { + let journal = root.join("migrations/readerx.sqliteSourceRefs.json"); + let mut remap: BTreeMap = if journal.is_file() { + serde_json::from_value(id_migration::read(&journal)?).map_err(|e| e.to_string())? + } else { + BTreeMap::new() + }; + remap.extend(aliases.clone()); + for path in id_migration::json_files(&root.join("book_sources"))? { + let value = id_migration::read(&path)?; + let old = text(&value, "id"); + remap.entry(old.clone()).or_insert_with(|| { + identity::entity_id( + &old, + "s-", + identity::source_uid(&text(&value, "bookSourceUrl")), + ) + }); + } + if !remap.is_empty() { + id_migration::write( + &journal, + &serde_json::to_value(&remap).map_err(|e| e.to_string())?, + )?; + } + if unreadable_book_details(root)? { + return Ok(()); + } + id_migration::migrate(root, &remap)?; + crate::book_store::remap_metadata_at(root, "bookSourceId", &remap)?; + if journal.is_file() { + std::fs::remove_file(journal).map_err(|e| e.to_string())?; + } + Ok(()) +} + +fn unreadable_book_details(root: &Path) -> Result { + let mut pending = false; + for path in id_migration::details(root)? { + if id_migration::read(&path).is_err() { + log::warn!("旧书籍元信息无法读取，保留身份迁移待办，稍后重试"); + pending = true; + } + } + Ok(pending) +} + +/// The shared source crate still owns file writes; the App maintains SQLite references. +pub(crate) fn save_source( + root: &Path, + source: &readerx_source::models::BookSource, + old: &str, +) -> Result<(), String> { + if !readerx_source::store::valid_component(old) + || !readerx_source::store::valid_component(&source.id) + { + return Err("书源 ID 非法".into()); + } + let journal = root.join("migrations/readerx.sqliteSourceRefs.json"); + let mut remap: BTreeMap = if journal.is_file() { + serde_json::from_value(id_migration::read(&journal)?).map_err(|e| e.to_string())? + } else { + BTreeMap::new() + }; + remap.insert(old.to_string(), source.id.clone()); + id_migration::write( + &journal, + &serde_json::to_value(&remap).map_err(|e| e.to_string())?, + )?; + id_migration::save_source(root, source, old)?; + crate::book_store::remap_metadata_at(root, "bookSourceId", &remap)?; + std::fs::remove_file(journal).map_err(|e| e.to_string()) +} + fn text(value: &Value, key: &str) -> String { value .get(key) diff --git a/src-tauri/src/sync/service.rs b/src-tauri/src/sync/service.rs index b0018ff..1fdba63 100644 --- a/src-tauri/src/sync/service.rs +++ b/src-tauri/src/sync/service.rs @@ -1091,7 +1091,7 @@ impl SyncService { } } - /// 书籍被删除（**必须在本地删文件之前调用**：定位书身份要读 `bookdetail.json`）。 + /// 书籍被删除（**必须在本地删书之前调用**：定位书身份要读取书籍元信息）。 pub fn on_book_deleted(&self, book_id: &str) { let Some(engine) = self.engine() else { return; diff --git a/src-tauri/tests/support/book_json.rs b/src-tauri/tests/support/book_json.rs new file mode 100644 index 0000000..e6995ef --- /dev/null +++ b/src-tauri/tests/support/book_json.rs @@ -0,0 +1,161 @@ +//! Test-only JSON view of persisted SQLite rows, while legacy fixtures remain real files. +use rusqlite::{params, Connection, OptionalExtension}; +use serde_json::{json, Value}; +use std::path::{Path, PathBuf}; + +fn location(path: &Path) -> Option<(PathBuf, String, String)> { + let file = path.file_name()?.to_str()?.to_string(); + if !matches!( + file.as_str(), + "bookdetail.json" | "content.json" | "bookmarks.json" | "annotations.json" + ) { + return None; + } + let dir = path.parent()?; + let books = dir.parent()?; + if books.file_name()?.to_str()? != "books" { + return None; + } + Some(( + books.parent()?.join("books.sqlite3"), + dir.file_name()?.to_str()?.into(), + file, + )) +} +fn connection(path: &Path) -> Option<(Connection, String, String)> { + let (path, id, file) = location(path)?; + if !path.is_file() { + return None; + } + let db = Connection::open(path).unwrap(); + db.busy_timeout(std::time::Duration::from_secs(5)).unwrap(); + db.execute_batch("PRAGMA foreign_keys=ON").unwrap(); + let present: bool = db + .query_row( + "SELECT EXISTS(SELECT 1 FROM books WHERE id=?1)", + [&id], + |r| r.get(0), + ) + .unwrap(); + if present { + Some((db, id, file)) + } else { + None + } +} +pub fn read(path: &Path) -> Value { + let Some((db, id, file)) = connection(path) else { + return serde_json::from_slice(&std::fs::read(path).unwrap()).unwrap(); + }; + if file == "bookdetail.json" { + let text: String = db + .query_row("SELECT detail FROM books WHERE id=?1", [&id], |r| r.get(0)) + .unwrap(); + return serde_json::from_str(&text).unwrap(); + } + if file == "content.json" { + let mut query = db + .prepare("SELECT chapter FROM chapters WHERE book_id=?1 ORDER BY position") + .unwrap(); + let rows = query.query_map([&id], |r| r.get::<_, String>(0)).unwrap(); + let chapters: Vec = rows + .map(|r| serde_json::from_str(&r.unwrap()).unwrap()) + .collect(); + return json!({"schemaVersion":1,"chapters":chapters}); + } + let table = if file == "bookmarks.json" { + "bookmarks" + } else { + "annotations" + }; + let text: Option = db + .query_row( + &format!("SELECT records FROM {table} WHERE book_id=?1"), + [&id], + |r| r.get(0), + ) + .optional() + .unwrap(); + json!({"schemaVersion":1,(table):text.map(|s|serde_json::from_str::(&s).unwrap()).unwrap_or(json!([]))}) +} +pub fn write(path: &Path, value: &Value) { + let Some((mut db, id, file)) = connection(path) else { + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(path, serde_json::to_vec_pretty(value).unwrap()).unwrap(); + return; + }; + let tx = db.transaction().unwrap(); + if file == "bookdetail.json" { + tx.execute( + "UPDATE books SET detail=?1 WHERE id=?2", + params![value.to_string(), id], + ) + .unwrap(); + } else if file == "content.json" { + tx.execute("DELETE FROM chapters WHERE book_id=?1", [&id]) + .unwrap(); + for (p, c) in value["chapters"].as_array().unwrap().iter().enumerate() { + let paragraphs: Vec = + serde_json::from_value(c.get("paragraphs").cloned().unwrap_or(json!([]))).unwrap(); + let blocks = c.get("blocks").filter(|v| !v.is_null()); + let chars: usize = match blocks.and_then(Value::as_array).filter(|b| !b.is_empty()) { + Some(blocks) => blocks + .iter() + .filter(|b| matches!(b["kind"].as_str(), Some("p" | "h"))) + .map(|b| { + b["text"] + .as_str() + .unwrap_or_default() + .encode_utf16() + .count() + }) + .sum(), + None => paragraphs.iter().map(|s| s.encode_utf16().count()).sum(), + }; + let cid = c["cid"].as_str().unwrap_or_default(); + let head = json!({"cid":cid,"title":c["title"],"url":c["url"],"chars":chars}); + let hash = if paragraphs.iter().any(|p| !p.trim().is_empty()) || blocks.is_some() { + readerx_sync::content::body_fingerprint(¶graphs, blocks) + } else { + String::new() + }; + let digest = json!({"cid":cid,"hash":hash}); + let mut assets: Vec = Vec::new(); + for b in blocks.and_then(Value::as_array).into_iter().flatten() { + if let Some(s) = b["local"].as_str() { + assets.push(s.into()); + } + for image in b["imgs"].as_array().into_iter().flatten() { + if let Some(s) = image["local"].as_str() { + assets.push(s.into()); + } + } + } + tx.execute("INSERT INTO chapters(book_id,position,cid,chapter,head,digest,assets) VALUES(?1,?2,?3,?4,?5,?6,?7)", + params![id,p as i64,cid,c.to_string(),head.to_string(),digest.to_string(),json!(assets).to_string()]).unwrap(); + } + } else { + let table = if file == "bookmarks.json" { + "bookmarks" + } else { + "annotations" + }; + tx.execute(&format!("INSERT INTO {table}(book_id,records) VALUES(?1,?2) ON CONFLICT(book_id) DO UPDATE SET records=excluded.records"), + params![id,value[table].to_string()]).unwrap(); + } + tx.commit().unwrap(); +} + +pub fn ids(root: &Path) -> Vec { + let path = root.join("books.sqlite3"); + if !path.is_file() { + return Vec::new(); + } + let db = Connection::open(path).unwrap(); + let mut query = db.prepare("SELECT id FROM books ORDER BY id").unwrap(); + query + .query_map([], |r| r.get::<_, String>(0)) + .unwrap() + .map(Result::unwrap) + .collect() +} diff --git a/src-tauri/tests/sync_bridge.rs b/src-tauri/tests/sync_bridge.rs index bbebcd5..9e59c80 100644 --- a/src-tauri/tests/sync_bridge.rs +++ b/src-tauri/tests/sync_bridge.rs @@ -50,6 +50,7 @@ fn setup() -> (tauri::AppHandle, PathBuf) { ] { let _ = std::fs::remove_dir_all(data_root.join(sub)); } + let _ = std::fs::remove_file(data_root.join("books.sqlite3")); (handle, data_root) } @@ -88,18 +89,12 @@ fn shared_app() -> (tauri::AppHandle, PathBuf) { (handle, data_root) } -fn write_json(path: &Path, value: &Value) { - if let Some(parent) = path.parent() { - std::fs::create_dir_all(parent).unwrap(); - } - std::fs::write(path, serde_json::to_string_pretty(value).unwrap()).unwrap(); -} - -fn read_json(path: &Path) -> Value { - serde_json::from_str(&std::fs::read_to_string(path).unwrap()).unwrap() -} +#[path = "support/book_json.rs"] +mod book_json; +fn write_json(path: &Path, value: &Value) { book_json::write(path, value); } +fn read_json(path: &Path) -> Value { book_json::read(path) } -/// 造一本本地书（与 `book_store.rs` 的目录布局一致：书签单独一个文件）。 +/// 造一本升级前的本地书：在 book_store 首次访问时迁入数据库。 fn seed_local_book(data_root: &Path, id: &str, title: &str) { seed_book(&data_root.join("books").join(id), id, title, "三体.epub", true); } @@ -358,6 +353,10 @@ fn sync_once(client: &SharedEngine, server: &SharedEngine) -> readerx_sync::Sync /// 目录下的文件名（不存在时为空）。 fn list_files(dir: &Path) -> Vec { + if dir.file_name().and_then(|n|n.to_str()) == Some("books") { + let root = dir.parent().unwrap(); + if root.join("books.sqlite3").is_file() { return book_json::ids(root); } + } std::fs::read_dir(dir) .map(|items| { items @@ -1188,17 +1187,12 @@ fn synced_content_creates_the_book_locally() { "对端的正文应先落在暂存区" ); - // 落地：本机按引擎里的元信息建书，并把正文写进 content.json + // 落地：本机按引擎里的元信息建书，并把正文写进 SQLite。 let changes = bridge::materialize(&handle, &local, 0, &mut index).unwrap(); assert!(changes.books, "新书落地要刷新书架：{changes:?}"); assert_eq!(changes.chapters.len(), 1, "{changes:?}"); - let book_ids: Vec = std::fs::read_dir(app_data.join("books")) - .unwrap() - .flatten() - .filter(|entry| entry.path().is_dir()) - .map(|entry| entry.file_name().to_string_lossy().to_string()) - .collect(); + let book_ids = book_json::ids(&app_data); assert_eq!(book_ids.len(), 1, "应该正好新建一本：{book_ids:?}"); let book_id = &book_ids[0]; assert_eq!(book_id, &uid, "接收端必须直接使用同步书籍 ID"); @@ -1217,10 +1211,14 @@ fn synced_content_creates_the_book_locally() { assert_eq!(chapters[1]["paragraphs"][0], json!("正文二")); assert!(chapters[2]["paragraphs"].as_array().unwrap().is_empty()); - assert!( - app_data.join("books").join(book_id).join("digest.json").is_file(), - "正文落地要顺手写下章节指纹缓存" - ); + let db = rusqlite::Connection::open(app_data.join("books.sqlite3")).unwrap(); + let mut query = db.prepare("SELECT digest FROM chapters WHERE book_id=?1 ORDER BY position").unwrap(); + let digests: Vec = query.query_map([book_id], |row|row.get::<_, String>(0)).unwrap() + .map(|row|serde_json::from_str(&row.unwrap()).unwrap()).collect(); + assert_eq!(digests.len(), 3); + assert!(!digests[0]["hash"].as_str().unwrap().is_empty()); + assert!(!digests[1]["hash"].as_str().unwrap().is_empty()); + assert_eq!(digests[2]["hash"], "", "未下载章节没有正文指纹"); assert!( lock_engine(&local).staged_bodies(&uid).is_empty(), "落地完成后暂存区要清干净" @@ -1232,12 +1230,7 @@ fn synced_content_creates_the_book_locally() { // 再落地一次不该重复建书 let again = bridge::materialize(&handle, &local, usize::MAX, &mut index).unwrap(); assert!(again.chapters.is_empty() && !again.books, "重复落地应无事发生：{again:?}"); - let still: Vec = std::fs::read_dir(app_data.join("books")) - .unwrap() - .flatten() - .filter(|entry| entry.path().is_dir()) - .map(|entry| entry.file_name().to_string_lossy().to_string()) - .collect(); + let still = book_json::ids(&app_data); assert_eq!(still, book_ids); } @@ -1808,6 +1801,7 @@ fn hidden_group_membership_survives_sync_and_can_be_cleared() { sync_once(&local, &peer.engine); // 将远端暂存作为新设备数据拉回：本地书库清空，验证建书时也保留归属。 std::fs::remove_dir_all(app_data.join("books")).unwrap(); + std::fs::remove_file(app_data.join("books.sqlite3")).unwrap(); drop(local); let fresh = local_engine(&handle, &app_data); sync_once(&fresh, &peer.engine); @@ -1987,12 +1981,12 @@ fn sync_service_streams_large_books_and_saves_before_stop_or_disconnect() { // 落库失败不能推进操作游标；移除写盘障碍后无需重新接收即可恢复。 let cursor = read_json(&app_data.join("sync/settings.json"))["materializedOps"].clone(); - let obstacle = app_data.join("books").join(&ids[0]).join("bookdetail.json.tmp"); - std::fs::create_dir(&obstacle).unwrap(); + let blocker = rusqlite::Connection::open(app_data.join("books.sqlite3")).unwrap(); + blocker.execute_batch("BEGIN IMMEDIATE").unwrap(); lock_engine(&peer.engine).set_field(&uid, "title", json!("同步后的书名")).unwrap(); assert!(service.sync_with_addr_now(&server.local_addr().to_string()).is_err()); assert_eq!(read_json(&app_data.join("sync/settings.json"))["materializedOps"], cursor); - std::fs::remove_dir(&obstacle).unwrap(); + blocker.execute_batch("ROLLBACK").unwrap(); service.sync_with_addr_now(&server.local_addr().to_string()).unwrap(); assert_eq!(read_json(&app_data.join("books").join(&ids[0]).join("bookdetail.json"))["title"], "同步后的书名"); @@ -2236,7 +2230,7 @@ fn annotations_sync_merge_conflict_resolution_restart_and_book_deletion() { let _pin = readerx_lib::pin_data_root(Some(b.data_root.clone())); assert_eq!(bridge::materialize(&app, &b.engine, 0, &mut index_b).unwrap().deleted_books, vec![uid.clone()]); } - assert!(!path_b.exists()); + assert!(!book_json::ids(&b.data_root).contains(&uid)); } #[test] @@ -2257,10 +2251,12 @@ fn annotation_save_with_stale_snapshot_preserves_remote_add_and_edit() { assert_eq!(notes.iter().find(|n| n["id"] == "n2").unwrap()["text"], "远端改写"); assert_eq!(notes.iter().find(|n| n["id"] == "n3").unwrap()["text"], "远端新增"); assert_eq!(notes.iter().find(|n| n["id"] == "n1").unwrap()["text"], "本机修改第一条"); - // 损坏文件必须保留原样、向上返回失败。 - std::fs::write(&path, "broken").unwrap(); + // 损坏数据库记录必须保留原样、向上返回失败。 + let db = rusqlite::Connection::open(root.join("books.sqlite3")).unwrap(); + db.execute("UPDATE annotations SET records='broken' WHERE book_id='local-save'", []).unwrap(); assert!(service.save_annotations("local-save", &previous, &next).is_err()); - assert_eq!(std::fs::read_to_string(path).unwrap(), "broken"); + let raw: String = db.query_row("SELECT records FROM annotations WHERE book_id='local-save'", [], |row|row.get(0)).unwrap(); + assert_eq!(raw, "broken"); } #[test] From 2bb2036e49857ed654ab95a240dc2c230e5fe995 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:14:51 +0800 Subject: [PATCH 02/17] =?UTF-8?q?fix:=20=E9=98=B2=E6=AD=A2=E8=BA=AB?= =?UTF-8?q?=E4=BB=BD=E8=BF=81=E7=A7=BB=E8=A6=86=E7=9B=96=E6=97=A7=E4=B9=A6?= =?UTF-8?q?=E5=B9=B6=E5=87=8F=E5=B0=91=E5=B0=81=E9=9D=A2=E9=87=8D=E5=A4=8D?= =?UTF-8?q?=E8=AF=BB=E5=8F=96?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/book-storage.md | 2 +- src-tauri/src/book_store.rs | 6 +++++ src-tauri/src/book_store/sqlite.rs | 38 +++++++++++++++++++++++------- src-tauri/src/sync/book_ids.rs | 9 ++++++- 4 files changed, 44 insertions(+), 11 deletions(-) diff --git a/docs/book-storage.md b/docs/book-storage.md index 0c9be71..10afd6e 100644 --- a/docs/book-storage.md +++ b/docs/book-storage.md @@ -17,7 +17,7 @@ command 访问，数据形状和同步协议不变。 | 章节插图 / PDF 页面图的字节 | 原有 `images/`；数据库只存引用名 | 元信息和章内结构以 JSON 列保留现有字段形状，但数据库按书 / 章分别存行。 -列表通过索引连接查询元信息和预计算的章节头，不扫描正文。 +列表分别查询元信息和预计算的章节头，每本元信息只读一次，不扫描正文或按章重复读取封面。 按 cid 读取或回写只操作命中的章节，目录标题 / 地址保持当前值；旧 index 仅为兼容提示。 历史空或重复 cid 可以迁入，存在多个命中时正文补丁明确失败，避免错写。 diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index 6babc93..d60e748 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -1925,6 +1925,12 @@ pub(crate) fn rename_id_at(root: &Path, old: &str, new: &str) -> Result<(), Stri let tx = sqlite::transaction(&mut db)?; let from = sqlite::detail(&tx, old)?; let to = sqlite::detail(&tx, new)?; + if from.is_some() + && (root.join("books").join(new).join(BOOKDETAIL_FILE).is_file() + || root.join("books").join(format!("{new}.json")).is_file()) + { + return Err("书籍 ID 迁移目标仍有未迁入的旧书籍，原数据已保留".into()); + } if from.is_some() && to.is_some() { return Err("书籍 ID 迁移目标已存在".into()); } diff --git a/src-tauri/src/book_store/sqlite.rs b/src-tauri/src/book_store/sqlite.rs index 2ab1331..a4d6ea9 100644 --- a/src-tauri/src/book_store/sqlite.rs +++ b/src-tauri/src/book_store/sqlite.rs @@ -371,17 +371,27 @@ pub(super) fn put_records( } pub(super) fn metas(db: &Connection) -> Result, String> { - // One indexed join reads only metadata and precomputed chapter heads, never chapter bodies. - let mut statement = db.prepare("SELECT b.detail,c.head FROM books b LEFT JOIN chapters c ON c.book_id=b.id ORDER BY b.id,c.position").map_err(sql)?; + // Read detail once per book: a join would repeat large covers for every chapter. + // The second query reads only precomputed heads, never chapter bodies. + let mut out: Vec = details(db)? + .into_iter() + .map(|d| d.into_meta(Vec::new())) + .collect(); + let indices: HashMap = out + .iter() + .enumerate() + .map(|(i, book)| (book.id.clone(), i)) + .collect(); + let mut statement = db + .prepare("SELECT book_id,head FROM chapters ORDER BY book_id,position") + .map_err(sql)?; let mut rows = statement.query([]).map_err(sql)?; - let mut out: Vec = Vec::new(); while let Some(row) = rows.next().map_err(sql)? { - let detail: BookDetail = decode(&row.get::<_, String>(0).map_err(sql)?)?; - if out.last().is_none_or(|last| last.id != detail.id) { - out.push(detail.into_meta(Vec::new())); - } - if let Some(s) = row.get::<_, Option>(1).map_err(sql)? { - out.last_mut().unwrap().chapters.push(decode(&s)?); + let id: String = row.get(0).map_err(sql)?; + if let Some(&index) = indices.get(&id) { + out[index] + .chapters + .push(decode(&row.get::<_, String>(1).map_err(sql)?)?); } } Ok(out) @@ -859,6 +869,16 @@ mod tests { let db = open(&root).unwrap(); assert!(detail(&db, "collision").unwrap().is_some()); drop(db); + let blocked = "b-1111111111111111"; + legacy(&root, &book(blocked)); + let notes = root.join("books").join(blocked).join(ANNOTATIONS_FILE); + fs::write(¬es, b"broken").unwrap(); + assert!(super::super::rename_id_at(&root, new, blocked).is_err()); + let db = open(&root).unwrap(); + assert!(detail(&db, new).unwrap().is_some()); + assert!(detail(&db, blocked).unwrap().is_none()); + assert_eq!(fs::read(notes).unwrap(), b"broken"); + drop(db); fs::remove_dir_all(root).unwrap(); } #[test] diff --git a/src-tauri/src/sync/book_ids.rs b/src-tauri/src/sync/book_ids.rs index 4de1965..930fd35 100644 --- a/src-tauri/src/sync/book_ids.rs +++ b/src-tauri/src/sync/book_ids.rs @@ -59,7 +59,14 @@ pub fn migrate(app: &AppHandle) -> Result<(), String> { } // 必须在任何修改之前验证，不能用远端身份覆盖无关本地目录。 for new in plan.values() { - if book_store::contains_at(app, None, new)? { + if book_store::contains_at(app, None, new)? + || root + .join("books") + .join(new) + .join("bookdetail.json") + .is_file() + || root.join("books").join(format!("{new}.json")).is_file() + { return Err("书籍 ID 迁移目录冲突或旧布局未迁移完成；原数据已保留".into()); } } From 89c491c451764bc6c6db974f117e248227d51dfb Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:17:49 +0800 Subject: [PATCH 03/17] =?UTF-8?q?fix:=20=E8=BF=81=E7=A7=BB=E5=A4=B1?= =?UTF-8?q?=E8=B4=A5=E6=97=B6=E4=BF=9D=E7=95=99=E5=90=8C=E6=AD=A5=E5=86=99?= =?UTF-8?q?=E5=85=A5=E5=BE=85=E5=8A=9E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/book-storage.md | 1 + src-tauri/src/book_store.rs | 4 ++++ src-tauri/src/book_store/sqlite.rs | 9 ++++++++- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/book-storage.md b/docs/book-storage.md index 10afd6e..b7df5ef 100644 --- a/docs/book-storage.md +++ b/docs/book-storage.md @@ -35,6 +35,7 @@ command 访问，数据形状和同步协议不变。 成功后书籍 JSON 改名为 `.json.migrated` 留底；失败保留原文件，读路径仍可回退， 其他书照常迁移，后续访问可重试。文件归档中断时，事务内保存的回执阻止旧数据覆盖 数据库中的新编辑。已有同名留底文件不被覆盖。 +尚未迁入的书籍拒绝同步写入并返回失败，让同步保留待落库数据，避免当作已删书跳过更新。 旧身份迁移待办继续使用可重跑日志；数据库书籍改 ID 在事务内级联更新， 书签中的 bookId 随之更新。书架、规则、阅读时长及 TTS 目录仍使用原迁移流程。 diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index d60e748..ff369c9 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -851,6 +851,7 @@ pub(crate) fn normalize_book_images( let root = crate::storage::data_root_at(app, root)?; let mut db = sqlite::open(&root)?; let Some(mut book) = sqlite::get(&db, id)? else { + sqlite::reject_pending_migration(&root, id)?; return Ok(false); }; if !crate::book_images::normalize_image_refs(&root.join("images"), &mut book.chapters) { @@ -1214,6 +1215,7 @@ pub(crate) fn apply_sync_meta( let mut db = sqlite::open(&root)?; let tx = sqlite::transaction(&mut db)?; let Some(mut detail) = sqlite::detail(&tx, id)? else { + sqlite::reject_pending_migration(&root, id)?; return Ok(false); }; if BookSyncMeta::from_detail(&detail) == *want { @@ -1410,6 +1412,7 @@ pub(crate) fn apply_sync_structure( let mut db = sqlite::open(&root)?; let tx = sqlite::transaction(&mut db)?; if sqlite::detail(&tx, id)?.is_none() { + sqlite::reject_pending_migration(&root, id)?; return Ok(false); } if sqlite::refs(&tx, id)? == want { @@ -1718,6 +1721,7 @@ pub(crate) fn set_cover_at( let mut db = sqlite::open(&root)?; let tx = sqlite::transaction(&mut db)?; let Some(mut detail) = sqlite::detail(&tx, id)? else { + sqlite::reject_pending_migration(&root, id)?; return Ok(false); }; let want = cover.filter(|v| !v.is_empty()).map(str::to_string); diff --git a/src-tauri/src/book_store/sqlite.rs b/src-tauri/src/book_store/sqlite.rs index a4d6ea9..45abee2 100644 --- a/src-tauri/src/book_store/sqlite.rs +++ b/src-tauri/src/book_store/sqlite.rs @@ -506,12 +506,17 @@ pub(super) fn require_book(db: &Connection, root: &Path, id: &str) -> Result<(), if detail(db, id)?.is_some() { return Ok(()); } + reject_pending_migration(root, id)?; + Err("书籍不存在".into()) +} +/// A failed migration is not a deleted book: writes must fail so sync can retry. +pub(super) fn reject_pending_migration(root: &Path, id: &str) -> Result<(), String> { if root.join("books").join(id).join(BOOKDETAIL_FILE).is_file() || root.join("books").join(format!("{id}.json")).is_file() { Err("书籍数据迁移失败，请查看应用日志".into()) } else { - Err("书籍不存在".into()) + Ok(()) } } pub(super) fn commit(db: rusqlite::Transaction<'_>) -> Result<(), String> { @@ -697,6 +702,8 @@ mod tests { assert!(root.join("books/good/bookdetail.json.migrated").is_file()); assert!(root.join("books/bad/bookdetail.json").is_file()); assert!(require_book(&db, &root, "bad").is_err()); + assert!(reject_pending_migration(&root, "bad").is_err()); + assert!(reject_pending_migration(&root, "missing").is_ok()); fs::write(&bad, r#"{"schemaVersion":1,"annotations":[]}"#).unwrap(); drop(db); let db = open(&root).unwrap(); From ef0aa3f57958092eba58870eee69beac69cc54af Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:22:55 +0800 Subject: [PATCH 04/17] =?UTF-8?q?fix:=20=E5=A4=87=E4=BB=BD=E5=89=8D?= =?UTF-8?q?=E5=AE=8C=E6=88=90=E6=97=A7=E4=B9=A6=E7=AD=BE=E8=BF=81=E7=A7=BB?= =?UTF-8?q?=E5=B9=B6=E6=8B=92=E7=BB=9D=E9=81=97=E6=BC=8F=E6=8D=9F=E5=9D=8F?= =?UTF-8?q?=E4=B9=A6=E7=B1=8D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src-tauri/src/book_store.rs | 15 +++++++++--- src-tauri/src/data_transfer/export.rs | 3 ++- src-tauri/src/data_transfer/round_trip.rs | 29 +++++++++++++++++++++++ 3 files changed, 43 insertions(+), 4 deletions(-) diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index ff369c9..5e27807 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -1888,9 +1888,18 @@ pub(crate) fn export_books( let _transaction = library_transaction(); let root = crate::storage::data_root(app)?; let db = sqlite::open(&root)?; - // Refuse a partial backup when any readable legacy book could not be imported. - if root.join("books").is_dir() && !scan_books_dir(&root.join("books"))?.is_empty() { - return Err("书籍迁移尚未完成，暂不能导出完整备份".into()); + // Check file presence, not parsed metadata: corrupt books must also block partial backups. + if root.join("books").is_dir() { + for entry in fs::read_dir(root.join("books")).map_err(|e| e.to_string())? { + let entry = entry.map_err(|e| e.to_string())?; + let kind = entry.file_type().map_err(|e| e.to_string())?; + let path = entry.path(); + if (kind.is_dir() && path.join(BOOKDETAIL_FILE).is_file()) + || (kind.is_file() && path.extension().is_some_and(|e| e == "json")) + { + return Err("书籍迁移尚未完成，暂不能导出完整备份".into()); + } + } } sqlite::export(&db, zip, report) } diff --git a/src-tauri/src/data_transfer/export.rs b/src-tauri/src/data_transfer/export.rs index 0a85ef2..c88b6b7 100644 --- a/src-tauri/src/data_transfer/export.rs +++ b/src-tauri/src/data_transfer/export.rs @@ -34,9 +34,10 @@ pub(super) fn export_to( report: &mut dyn FnMut(&str, u64, u64), ) -> Result { let root = crate::storage::data_root(app)?; + // Migration may archive the old global bookmark state; enumerate files afterwards. + let book_count = crate::book_store::list_book_meta(app)?.len() as u64; let mut state = Vec::new(); walk(&root.join(STATE_DIR), STATE_DIR, &is_state_file, &mut state)?; - let book_count = crate::book_store::list_book_meta(app)?.len() as u64; let mut images = Vec::new(); walk( &root.join(IMAGES_DIR), diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index ad3d150..adfabbb 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -233,6 +233,35 @@ fn import( apply(app, &path.to_string_lossy(), mode, &mut |_, _, _| {}).expect("导入应成功") } +#[test] +fn export_migrates_global_bookmarks_before_state_scan_and_rejects_unreadable_books() { + let _guard = SERIAL.lock().unwrap_or_else(|e| e.into_inner()); + let (app, root) = setup(); + seed_book(&root, "legacy", "旧书", "old.epub", 321, None); + fs::remove_file(root.join("books/legacy/bookmarks.json")).unwrap(); + let marks = json!([{"id":"old-mark","bookId":"legacy","unknown":42}]); + seed_state(&root, "readerx.bookmarks", &json!({"legacy":marks})); + let backup = export(&app, &root, "global-marks", false); + let mut zip = zip::ZipArchive::new(fs::File::open(backup).unwrap()).unwrap(); + assert!(zip.by_name("state/readerx.bookmarks.json").is_err()); + let stored = archive::read_entry_json(&mut zip, "books/legacy/bookmarks.json") + .unwrap() + .unwrap(); + assert_eq!(stored["bookmarks"], marks); + let bad = root.join("books/bad/bookdetail.json"); + fs::create_dir_all(bad.parent().unwrap()).unwrap(); + fs::write(&bad, b"{broken").unwrap(); + let destination = fs::File::create(export_path(&root, "must-fail")).unwrap(); + assert!(export_to( + &app, + destination, + ExportOptions::default(), + &mut |_, _, _| {} + ) + .is_err()); + assert_eq!(fs::read(bad).unwrap(), b"{broken"); +} + #[test] fn export_and_merge_import_restores_everything() { let _guard = SERIAL.lock().unwrap_or_else(|e| e.into_inner()); From 6dde42cf12230aafe6f8894cd7465a7fc921f143 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:24:50 +0800 Subject: [PATCH 05/17] =?UTF-8?q?fix:=20=E5=90=8C=E6=AD=A5=E5=8D=87?= =?UTF-8?q?=E7=BA=A7=E5=9B=9E=E5=BD=92=E9=AA=8C=E8=AF=81=E6=95=B0=E6=8D=AE?= =?UTF-8?q?=E5=BA=93=E8=BA=AB=E4=BB=BD=E4=B8=8E=E8=BF=81=E7=A7=BB=E7=95=99?= =?UTF-8?q?=E5=BA=95?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src-tauri/tests/sync_bridge.rs | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src-tauri/tests/sync_bridge.rs b/src-tauri/tests/sync_bridge.rs index 9e59c80..78fe811 100644 --- a/src-tauri/tests/sync_bridge.rs +++ b/src-tauri/tests/sync_bridge.rs @@ -2118,7 +2118,9 @@ fn upgrade_migrates_book_ids_without_recreating_sync_entities() { let uid = bridge::local_uid(&handle, "local-old"); let before = lock_engine(&local).entities_of_kind("book", true)[0].clone(); readerx_lib::sync::book_ids::migrate(&handle).unwrap(); - assert!(!root.join("books/local-old").exists()); + assert!(!root.join("books/local-old/bookdetail.json").exists()); + assert!(root.join("books/local-old/bookdetail.json.migrated").is_file()); + assert_eq!(book_json::ids(&root), vec![uid.clone()], "旧 ID 应从数据库移除，留底不能重建旧书"); assert_eq!(read_json(&root.join("books").join(&uid).join("bookdetail.json"))["id"], uid); assert_eq!(read_json(&root.join("books").join(&uid).join("bookmarks.json"))["bookmarks"][0]["bookId"], uid); let shelf = read_json(&root.join("state/readerx.shelf.json")); From c1bcaf11538ff428bf4170f2ec06aa0a7282c48c Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:36:12 +0800 Subject: [PATCH 06/17] =?UTF-8?q?fix:=20=E4=B8=B2=E8=A1=8C=E8=BF=90?= =?UTF-8?q?=E8=A1=8C=E4=B9=A6=E6=BA=90=E6=B5=8B=E8=AF=95=E5=87=8F=E5=B0=91?= =?UTF-8?q?=E8=AE=A1=E6=97=B6=E7=94=A8=E4=BE=8B=E5=B9=B2=E6=89=B0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/workflows/test-data.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/test-data.yml b/.github/workflows/test-data.yml index 205c157..c12bb8f 100644 --- a/.github/workflows/test-data.yml +++ b/.github/workflows/test-data.yml @@ -33,4 +33,5 @@ jobs: cargo test --locked -p readerx --lib data_transfer cargo test --locked -p readerx --test sync_bridge cargo test --locked -p readerx-sync - cargo test --locked -p readerx-source + # Isolate the existing concurrency timing test from other Boa test workloads. + cargo test --locked -p readerx-source -- --test-threads=1 From 2a55adf430904370a87eae6f42df23556d27cca8 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 05:54:53 +0800 Subject: [PATCH 07/17] =?UTF-8?q?feat:=20=E5=A4=87=E4=BB=BD=E5=AF=BC?= =?UTF-8?q?=E5=87=BA=20SQLite=20=E5=BF=AB=E7=85=A7=E5=B9=B6=E4=BF=9D?= =?UTF-8?q?=E7=95=99=E4=B9=A6=E6=BA=90=20JSON=20=E4=B8=8E=E6=97=A7?= =?UTF-8?q?=E5=BD=92=E6=A1=A3=E5=85=BC=E5=AE=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agents/skills/readerx-data/SKILL.md | 2 +- .../readerx-data/references/contracts.md | 6 +- docs/annotations.md | 2 +- docs/backup.md | 15 +- docs/book-storage.md | 14 +- docs/sync.md | 2 +- src-tauri/src/book_store.rs | 9 +- src-tauri/src/book_store/backup.rs | 156 +++++++++++++++++ src-tauri/src/book_store/sqlite.rs | 100 ++++------- src-tauri/src/data_transfer/archive.rs | 71 ++++++++ src-tauri/src/data_transfer/export.rs | 7 +- src-tauri/src/data_transfer/import.rs | 28 ++- src-tauri/src/data_transfer/mod.rs | 20 ++- src-tauri/src/data_transfer/plan.rs | 12 +- src-tauri/src/data_transfer/round_trip.rs | 162 +++++++++++++++++- 15 files changed, 486 insertions(+), 120 deletions(-) create mode 100644 src-tauri/src/book_store/backup.rs diff --git a/.agents/skills/readerx-data/SKILL.md b/.agents/skills/readerx-data/SKILL.md index 67bfa6d..a9f371a 100644 --- a/.agents/skills/readerx-data/SKILL.md +++ b/.agents/skills/readerx-data/SKILL.md @@ -11,7 +11,7 @@ description: 修改 ReaderX 本地书库存储、稳定身份与旧数据迁移 | 功能 | 实现入口 | 按需资料 | | --- | --- | --- | -| 本地书库存储、迁移、稳定 ID | src-tauri/src/book_store.rs、book_store/sqlite.rs、storage.rs、models.rs、sync/identity.rs、sync/book_ids.rs、sync/data_ids.rs；src/lib/books.ts、booksTypes.ts、dataIds.ts | [存储与身份约束](references/contracts.md#书库与稳定身份) | +| 本地书库存储、迁移、稳定 ID | src-tauri/src/book_store.rs、book_store/sqlite.rs、book_store/backup.rs、storage.rs、models.rs、sync/identity.rs、sync/book_ids.rs、sync/data_ids.rs；src/lib/books.ts、booksTypes.ts、dataIds.ts | [存储与身份约束](references/contracts.md#书库与稳定身份) | | 备份与恢复 | src-tauri/src/data_transfer/；src/lib/backup.ts、src/pages/DataBackup.tsx | [备份约束](references/contracts.md#备份与恢复)、[备份文档](../../../docs/backup.md) | | 同步协议、合并与传输 | src-tauri/crates/readerx-sync/ | [同步约束](references/contracts.md#局域网同步)、[同步文档](../../../docs/sync.md) 对应章节 | | 同步 App 接线与落库 | src-tauri/src/sync/；src/lib/sync.ts、src/components/SyncProgress.tsx | 同步文档 App 接线、实时落库与停止部分 | diff --git a/.agents/skills/readerx-data/references/contracts.md b/.agents/skills/readerx-data/references/contracts.md index 456a26a..2c526f5 100644 --- a/.agents/skills/readerx-data/references/contracts.md +++ b/.agents/skills/readerx-data/references/contracts.md @@ -4,8 +4,8 @@ ## 书库与稳定身份 -- book_store.rs / book_store/sqlite.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、状态 JSON 和图片字节不入库。数据库 user_version 与便携 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 -- 注释走 readerx_annotations_get/put，与正文/书签独立存表，便携信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含文件并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 +- book_store.rs / book_store/sqlite.rs / book_store/backup.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、状态 JSON 和图片字节不入库。数据库 user_version 与便携 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 +- 注释走 readerx_annotations_get/put，与正文/书签独立存表，便携信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含注释表并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 - book_store 的读写入口在进程级书库事务锁内完成迁移、读取、修改、落盘及指纹更新；持锁时不调用同步引擎。单章补丁以 chapter.cid 定位，index 仅兼容旧载荷；先验证整批 cid 存在且唯一，再只改正文，保留当前目录标题/地址。 - 普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 - 旧整书 / 目录 JSON 逐本在事务内导入并保存迁移回执，成功才改名 .migrated 留底；失败保留读回退和重试，不影响其他书。归档中断不能重复导入旧数据覆盖新编辑。数据库身份改动和状态 / TTS 目录迁移保持可重跑。 @@ -16,7 +16,7 @@ ## 备份与恢复 -- 归档入口按 data_transfer/archive.rs 分类与校验条目名，拒绝越界路径；不绕过 is_safe_entry 直接解压。plan.rs 只读规划，import.rs 写盘；书籍导入先经 import_book_json 在事务内写元信息和正文，再处理书签/注释；导出从 SQLite 按章生成原 JSON 归档，不直接复制数据库文件；图片流式处理。 +- 归档入口按 data_transfer/archive.rs 分类与校验条目名，拒绝越界路径；不绕过 is_safe_entry 直接解压。plan.rs 只读规划，import.rs 写盘；书籍导入先经 import_book_json 在事务内写元信息和正文，再处理书签/注释；导出 readerx-backup/2，在书库锁内用 VACUUM INTO 生成一致的 books.sqlite3 快照并流式写 ZIP，书源仍导出 JSON，事务日志不进归档；导入先只读校验数据库，再逐书复用身份规划与合并入口，不整库替换或导入迁移回执；兼容 readerx-backup/1 JSON，图片流式处理。 - 合并与覆盖语义不同：合并保留本机额外数据并按身份映射引用，偏好已有值保留；覆盖先写后删并清空归档缺失的内容类状态。新增 state 键时明确分类、合并规则、缺失语义与 ID 换算，不能直接全量 JSON 覆盖。 - 书源会话与 WebDAV 密码默认过滤，显式包含才导出；sync/、tts-audio/、logs/ 和本机 reading-time-origin 不作为换机数据搬走。阅读时长 contributions 应保留，恢复后本机新增时间使用独立来源。 - 覆盖删除书或书源前先调用同步删除钩子（定位身份仍需要旧数据），批量导入后调用 on_data_imported 全量对账。前端 import 完成后按 backup.ts 依赖顺序失效书签并重载分组、书库、进度、统计、规则、书源。 diff --git a/docs/annotations.md b/docs/annotations.md index 55fa958..48c4d17 100644 --- a/docs/annotations.md +++ b/docs/annotations.md @@ -1,6 +1,6 @@ # 阅读注释 -书籍在应用目录使用 SQLite，备份仍为便携 JSON 条目；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 +书籍在应用目录与备份中使用 SQLite，书源和状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 在同一段落内选取文字，点选区菜单的「注释」即可写下一条文本注释。保存后， 该段最后一个字符右侧出现 SVG 图标；点击图标可查看该段的全部注释、继续添加， diff --git a/docs/backup.md b/docs/backup.md index ab69172..5a96d69 100644 --- a/docs/backup.md +++ b/docs/backup.md @@ -1,6 +1,6 @@ # 数据备份与恢复（导入 / 导出） -书籍在应用目录使用 SQLite，备份仍为便携 JSON 条目；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 +书籍在应用目录与备份中使用 SQLite，书源和状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 > 状态：**已接入 App**（设置 → 数据 → 数据备份）。本文件说明备份里有什么、归档长什么样、 > 导入的两种语义，以及与「局域网同步」的边界。 @@ -27,16 +27,19 @@ ```text readerx-backup.json 清单：格式版本、导出版本、时间、是否含登录信息、各类条目数 state/.json readerx.* 状态（书架进度、分组、书源分组、替换 / 分章规则、偏好…） -books//bookdetail.json 书籍元信息（书名 / 作者 / 封面 / 分组归属…） -books//content.json 章节正文 -books//bookmarks.json 书签 -books//annotations.json 段落注释 +books.sqlite3 书籍元信息、正文、书签与段落注释的一致快照 images/ 章节插图 / PDF 页面图（文件名前缀就是它所属的书籍 id） book_sources/.json 书源定义 source_sessions/.json 书源登录态（**仅当导出时勾选「包含登录信息」**） ``` -清单里的 `format` 是 `readerx-backup/1`：读到**更高**的版本直接报错，而不是按旧语义读坏新数据。 +清单里的 `format` 是 `readerx-backup/2`：书籍导出 SQLite，书源仍导出 `book_sources/.json`。 +导入兼容 `readerx-backup/1` 的 `books//*.json`；更高版本直接报错。旧版 ReaderX 无法读取新备份。 + +导出先在书库锁内通过 `VACUUM INTO` 生成一致快照，再流式写入 ZIP，事务日志不进归档。 +恢复先校验数据库版本、完整性、引用及逐行数据，再逐书执行身份映射与合并 / 覆盖， +不直接替换本机数据库，也不移植归档里的旧 JSON 迁移回执。临时快照用完即删除， +中断遗留文件由下次启动回收。 **不在归档里**（刻意排除）： diff --git a/docs/book-storage.md b/docs/book-storage.md index b7df5ef..901c586 100644 --- a/docs/book-storage.md +++ b/docs/book-storage.md @@ -45,11 +45,15 @@ command 访问，数据形状和同步协议不变。 ## 备份与同步 -备份格式不变：数据库书籍通过 `book_store::export_books` 转成原来的 -`books//{bookdetail,content,bookmarks,annotations}.json` 归档条目。 -正文按章节行写入 zip，不为导出复制整库正文；数据库文件及事务日志不进入归档。 -恢复通过 `book_store::import_book_json` 写入数据库，不在应用目录重建书籍 JSON。 -合并 / 覆盖恢复、缺失注释清空、书签合并和凭据过滤保持原语义。 +新备份为 `readerx-backup/2`：`book_store::backup_books` 在书库锁内通过 +`VACUUM INTO` 生成一致的 `books.sqlite3` 快照，流式压缩进 ZIP。 +不复制正在写入的数据库文件，也不导出 WAL / SHM / journal；书源仍为 +`book_sources/.json`，状态与图片继续按文件导出。 +恢复只读打开归档数据库，先校验版本、完整性、引用与逐行数据，再将单书内容适配到 +现有 `book_store::import_book_json` 和记录合并入口，不整库覆盖本机数据库。 +归档里的迁移回执不导入本机。仍兼容旧 `readerx-backup/1` JSON 备份， +合并 / 覆盖恢复、旧备份缺失注释清空、书签合并和凭据过滤保持原语义。 +快照位于受启动回收管理的 `books/*.tmp`，正常与失败退出均清理。 有尚未完成迁移的旧书籍时拒绝导出不完整备份。 同步元信息、目录、正文指纹、按需取章、封面与插图引用全部经 `book_store`。 diff --git a/docs/sync.md b/docs/sync.md index d804bf4..46283aa 100644 --- a/docs/sync.md +++ b/docs/sync.md @@ -1,6 +1,6 @@ # 局域网同步（readerx-sync） -书籍在应用目录使用 SQLite，备份仍为便携 JSON 条目；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 +书籍在应用目录与备份中使用 SQLite，书源和状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 > 状态：**已接入 App**。本文件说明它解决什么问题、怎么用、边界在哪， > 以及 App 侧的接线方式（`src-tauri/src/sync/` 与 `src/lib/sync.ts`）。 diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index 5e27807..46852fd 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -1880,11 +1880,10 @@ pub(crate) fn contains_at( let root = crate::storage::data_root_at(app, root)?; Ok(sqlite::detail(&sqlite::open(&root)?, id)?.is_some()) } -pub(crate) fn export_books( +pub(crate) use sqlite::BackupDatabase; +pub(crate) fn backup_books( app: &AppHandle, - zip: &mut zip::ZipWriter, - report: &mut dyn FnMut(&str, u64, u64), -) -> Result { +) -> Result { let _transaction = library_transaction(); let root = crate::storage::data_root(app)?; let db = sqlite::open(&root)?; @@ -1901,7 +1900,7 @@ pub(crate) fn export_books( } } } - sqlite::export(&db, zip, report) + sqlite::snapshot(&db, &root) } /// Backup JSON is a portable representation, not the on-disk database schema. pub(crate) fn import_book_json( diff --git a/src-tauri/src/book_store/backup.rs b/src-tauri/src/book_store/backup.rs new file mode 100644 index 0000000..d62bb2a --- /dev/null +++ b/src-tauri/src/book_store/backup.rs @@ -0,0 +1,156 @@ +//! SQLite backup snapshots and read-only archive validation. +use super::*; + +/// A self-contained, read-only backup. Close the connection before removing its file. +pub(crate) struct BackupDatabase { + db: Connection, + temporary: crate::temporary_file::TemporaryFile, +} +impl BackupDatabase { + pub(crate) fn open(temporary: crate::temporary_file::TemporaryFile) -> Result { + let db = + Connection::open_with_flags(&temporary.0, rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY) + .map_err(sql)?; + db.execute_batch("PRAGMA trusted_schema=OFF; PRAGMA query_only=ON;") + .map_err(sql)?; + let version: i64 = db + .query_row("PRAGMA user_version", [], |r| r.get(0)) + .map_err(sql)?; + if version != DATABASE_VERSION { + return Err("备份书库数据库格式版本不受支持".into()); + } + let integrity: String = db + .query_row("PRAGMA integrity_check", [], |r| r.get(0)) + .map_err(sql)?; + if integrity != "ok" { + return Err("备份书库数据库已损坏".into()); + } + // Only ordinary application tables are queried, never archive-supplied views/triggers. + for table in ["books", "chapters", "bookmarks", "annotations"] { + let ordinary: bool = db.query_row( + "SELECT EXISTS(SELECT 1 FROM pragma_table_list WHERE schema='main' AND name=?1 AND type='table')", + [table], |r| r.get(0)).map_err(sql)?; + if !ordinary { + return Err("备份书库数据库缺少有效数据表".into()); + } + } + let foreign_key_error = db + .prepare("PRAGMA foreign_key_check") + .map_err(sql)? + .exists([]) + .map_err(sql)?; + if foreign_key_error { + return Err("备份书库数据库引用无效".into()); + } + let backup = Self { db, temporary }; + // Validate every payload before import can mutate local data, without loading the library. + for id in backup.ids()? { + valid_id(&id)?; + let d = detail(&backup.db, &id)?.ok_or("备份缺少书籍元信息")?; + if d.id != id || d.schema_version != 1 { + return Err("备份书籍元信息无效".into()); + } + let mut query = backup + .db + .prepare("SELECT chapter FROM chapters WHERE book_id=?1") + .map_err(sql)?; + let rows = query + .query_map([&id], |r| r.get::<_, String>(0)) + .map_err(sql)?; + for row in rows { + let _: LocalBookChapter = decode(&row.map_err(sql)?)?; + } + records(&backup.db, "bookmarks", &id)?; + for paragraph in records(&backup.db, "annotations", &id)? { + let notes = paragraph + .get("notes") + .and_then(Value::as_array) + .ok_or("备份注释列表无效")?; + for note in notes { + if note.get("id").and_then(Value::as_str).is_none() { + return Err("备份注释身份无效".into()); + } + } + } + } + Ok(backup) + } + #[cfg(test)] + pub(super) fn path(&self) -> &Path { + &self.temporary.0 + } + pub(crate) fn ids(&self) -> Result, String> { + let mut query = self + .db + .prepare("SELECT id FROM books ORDER BY id") + .map_err(sql)?; + let rows = query + .query_map([], |r| r.get::<_, String>(0)) + .map_err(sql)?; + rows.map(|r| r.map_err(sql)).collect() + } + /// Adapt a single book to the existing restore pipeline; these are not ZIP entries. + pub(crate) fn read(&self, id: &str, file: &str) -> Result>, String> { + let value = match file { + BOOKDETAIL_FILE => detail(&self.db, id)? + .map(|d| serde_json::to_value(d).map_err(|e| e.to_string())) + .transpose()?, + CONTENT_FILE => Some( + serde_json::to_value(BookContent { + schema_version: 1, + chapters: chapters(&self.db, id)?, + }) + .map_err(|e| e.to_string())?, + ), + BOOKMARKS_FILE => Some( + serde_json::json!({"schemaVersion":1,"bookmarks":records(&self.db,"bookmarks",id)?}), + ), + ANNOTATIONS_FILE => Some( + serde_json::json!({"schemaVersion":1,"annotations":records(&self.db,"annotations",id)?}), + ), + _ => None, + }; + value + .map(|v| serde_json::to_vec(&v).map_err(|e| e.to_string())) + .transpose() + } + pub(crate) fn write_zip( + &self, + zip: &mut zip::ZipWriter, + report: &mut dyn FnMut(&str, u64, u64), + ) -> Result<(), String> { + let total = self.ids()?.len() as u64; + report("books", 0, total); + zip.start_file( + DATABASE, + crate::data_transfer::archive::text_options(Some(&self.temporary.0)), + ) + .map_err(|e| e.to_string())?; + let mut file = fs::File::open(&self.temporary.0).map_err(|e| e.to_string())?; + std::io::copy(&mut file, zip).map_err(|e| e.to_string())?; + report("books", total, total); + Ok(()) + } +} + +/// VACUUM INTO includes committed WAL pages and produces a compact consistent snapshot. +pub(crate) fn snapshot(db: &Connection, root: &Path) -> Result { + let dir = root.join("books"); + crate::storage::ensure_dir(&dir)?; + let temporary = crate::temporary_file::TemporaryFile( + dir.join(format!(".backup-{}.tmp", readerx_sync::new_id())), + ); + drop( + fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&temporary.0) + .map_err(|e| e.to_string())?, + ); + db.execute( + "VACUUM INTO ?1", + [temporary.0.to_str().ok_or("备份临时路径无效")?], + ) + .map_err(sql)?; + BackupDatabase::open(temporary) +} diff --git a/src-tauri/src/book_store/sqlite.rs b/src-tauri/src/book_store/sqlite.rs index 45abee2..7a94a93 100644 --- a/src-tauri/src/book_store/sqlite.rs +++ b/src-tauri/src/book_store/sqlite.rs @@ -1,7 +1,6 @@ //! SQLite implementation. Callers hold the book-store lock; no sync-engine calls here. use super::*; use rusqlite::{params, Connection, OptionalExtension}; -use std::io::Write; pub(super) const DATABASE: &str = "books.sqlite3"; const DATABASE_VERSION: i64 = 1; @@ -532,60 +531,10 @@ pub(super) fn delete(db: &Connection, id: &str) -> Result<(), String> { Ok(()) } -/// Archives stay JSON based and version-compatible. Stream chapters one row at a time. -pub(super) fn export( - db: &Connection, - zip: &mut zip::ZipWriter, - report: &mut dyn FnMut(&str, u64, u64), -) -> Result { - let details = details(db)?; - let total = details.len() as u64; - report("books", 0, total); - for (index, d) in details.iter().enumerate() { - let prefix = format!("books/{}/", d.id); - zip.start_file( - format!("{prefix}{BOOKDETAIL_FILE}"), - crate::data_transfer::archive::text_options(None), - ) - .map_err(|e| e.to_string())?; - serde_json::to_writer(&mut *zip, d).map_err(|e| e.to_string())?; - zip.start_file( - format!("{prefix}{CONTENT_FILE}"), - crate::data_transfer::archive::text_options(None), - ) - .map_err(|e| e.to_string())?; - zip.write_all(b"{\"schemaVersion\":1,\"chapters\":[") - .map_err(|e| e.to_string())?; - let mut statement = db - .prepare("SELECT chapter FROM chapters WHERE book_id=?1 ORDER BY position") - .map_err(sql)?; - let rows = statement - .query_map([&d.id], |r| r.get::<_, String>(0)) - .map_err(sql)?; - for (i, row) in rows.enumerate() { - if i > 0 { - zip.write_all(b",").map_err(|e| e.to_string())?; - } - zip.write_all(row.map_err(sql)?.as_bytes()) - .map_err(|e| e.to_string())?; - } - zip.write_all(b"]}").map_err(|e| e.to_string())?; - for (file, table, field) in [ - (BOOKMARKS_FILE, "bookmarks", "bookmarks"), - (ANNOTATIONS_FILE, "annotations", "annotations"), - ] { - zip.start_file( - format!("{prefix}{file}"), - crate::data_transfer::archive::text_options(None), - ) - .map_err(|e| e.to_string())?; - let value = serde_json::json!({"schemaVersion":1,(field):records(db, table, &d.id)?}); - serde_json::to_writer(&mut *zip, &value).map_err(|e| e.to_string())?; - } - report("books", index as u64 + 1, total); - } - Ok(total) -} +#[path = "backup.rs"] +mod backup; +pub(super) use backup::snapshot; +pub(crate) use backup::BackupDatabase; #[cfg(test)] mod tests { @@ -823,29 +772,40 @@ mod tests { fs::remove_dir_all(root).unwrap(); } #[test] - fn export_uses_portable_json_and_omits_database_bytes() { + fn export_is_a_standalone_sqlite_snapshot_including_wal() { let root = root("export"); let db = open(&root).unwrap(); let original = book("b1"); put(&db, &original).unwrap(); - let file = fs::File::create(root.join("backup.zip")).unwrap(); - let mut zip = zip::ZipWriter::new(file); - assert_eq!(export(&db, &mut zip, &mut |_, _, _| {}).unwrap(), 1); + db.execute_batch("PRAGMA journal_mode=WAL;").unwrap(); + put_records(&db, "bookmarks", "b1", &[json!({"id":"m","bookId":"b1"})]).unwrap(); + let backup = snapshot(&db, &root).unwrap(); + let snapshot_path = backup.path().to_path_buf(); + let mut changed = original.clone(); + changed.title = "快照之后的编辑".into(); + put(&db, &changed).unwrap(); + let mut zip = zip::ZipWriter::new(fs::File::create(root.join("backup.zip")).unwrap()); + backup.write_zip(&mut zip, &mut |_, _, _| {}).unwrap(); drop(zip.finish().unwrap()); - let file = fs::File::open(root.join("backup.zip")).unwrap(); - let mut archive = zip::ZipArchive::new(file).unwrap(); - assert!(archive.by_name(DATABASE).is_err()); - let detail: BookDetail = - serde_json::from_reader(archive.by_name("books/b1/bookdetail.json").unwrap()).unwrap(); - let content: BookContent = - serde_json::from_reader(archive.by_name("books/b1/content.json").unwrap()).unwrap(); + let mut archive = + zip::ZipArchive::new(fs::File::open(root.join("backup.zip")).unwrap()).unwrap(); + assert_eq!(archive.len(), 1); + let restored = root.join("restored.sqlite3"); + std::io::copy( + &mut archive.by_name(DATABASE).unwrap(), + &mut fs::File::create(&restored).unwrap(), + ) + .unwrap(); + let restored = Connection::open(restored).unwrap(); assert_eq!( - serde_json::to_value(detail.into_book(content.chapters)).unwrap(), + serde_json::to_value(get(&restored, "b1").unwrap().unwrap()).unwrap(), serde_json::to_value(original).unwrap() ); - let records: Value = - serde_json::from_reader(archive.by_name("books/b1/annotations.json").unwrap()).unwrap(); - assert_eq!(records["annotations"], json!([])); + assert_eq!(records(&restored, "bookmarks", "b1").unwrap()[0]["id"], "m"); + drop(restored); + drop(archive); + drop(backup); + assert!(!snapshot_path.exists()); drop(db); fs::remove_dir_all(root).unwrap(); } diff --git a/src-tauri/src/data_transfer/archive.rs b/src-tauri/src/data_transfer/archive.rs index 96d46af..6573eff 100644 --- a/src-tauri/src/data_transfer/archive.rs +++ b/src-tauri/src/data_transfer/archive.rs @@ -301,6 +301,77 @@ pub(super) fn classify(zip: &mut ZipArchive, manifest: Manifest, bytes: u6 } } +/// Extract only the v2 database to an owned temporary file, then validate it read-only. +/// The live database is never replaced, including in Replace mode. +pub(super) fn open_books( + root: &Path, + zip: &mut ZipArchive, + scan: &mut ArchiveScan, +) -> Result, String> { + if scan.manifest.format != super::BACKUP_FORMAT { + return Ok(None); + } + if scan.entries.iter().any(|name| name.starts_with("books/")) { + return Err("备份同时包含两种书籍布局".into()); + } + let dir = root.join("books"); + crate::storage::ensure_dir(&dir)?; + let temporary = crate::temporary_file::TemporaryFile( + dir.join(format!(".restore-{}.tmp", readerx_sync::new_id())), + ); + { + let mut destination = fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&temporary.0) + .map_err(|e| format!("创建备份临时文件失败: {e}"))?; + let mut entry = zip + .by_name("books.sqlite3") + .map_err(|_| "备份缺少书库数据库")?; + std::io::copy(&mut entry, &mut destination) + .map_err(|e| format!("读取备份书库失败: {e}"))?; + destination.sync_all().map_err(|e| e.to_string())?; + } + let books = crate::book_store::BackupDatabase::open(temporary)?; + scan.books = books.ids()?; + if scan.books.len() as u64 != scan.manifest.books { + return Err("备份书籍数量与清单不符".into()); + } + // Logical names let both archive versions share identity planning and record merging. + for id in &scan.books { + for file in [ + "bookdetail.json", + "content.json", + "bookmarks.json", + "annotations.json", + ] { + scan.entries.push(format!("books/{id}/{file}")); + } + } + Ok(Some(books)) +} + +pub(super) fn read_book_entry( + zip: &mut ZipArchive, + books: Option<&crate::book_store::BackupDatabase>, + name: &str, +) -> Result>, String> { + if let Some(books) = books { + let mut parts = name.split('/'); + if parts.next() != Some(super::BOOKS_DIR) { + return Err("非法的书籍条目".into()); + } + let id = parts.next().ok_or("书籍条目缺少身份")?; + let file = parts.next().ok_or("书籍条目缺少类别")?; + if parts.next().is_some() { + return Err("非法的书籍条目".into()); + } + books.read(id, file) + } else { + read_entry(zip, name) + } +} + /// 读一个条目的全部字节；条目不存在返回 `Ok(None)` pub(super) fn read_entry( zip: &mut ZipArchive, diff --git a/src-tauri/src/data_transfer/export.rs b/src-tauri/src/data_transfer/export.rs index c88b6b7..d7d9ab3 100644 --- a/src-tauri/src/data_transfer/export.rs +++ b/src-tauri/src/data_transfer/export.rs @@ -3,7 +3,7 @@ //! 采集口径与 `data_transfer/mod.rs` 的文件头一致：书籍（元信息 / 正文 / 书签）、 //! 状态文件、插图、书源，外加**按需**的书源登录态。听书缓存、日志、同步目录不进来。 //! -//! 数据库正文按章节行流式写入归档，文件资源用 `std::io::copy`。 +//! 数据库生成一致快照后流式写入归档，文件资源用 `std::io::copy`。 //! 不为导出复制整本或整库正文。 use super::{archive, ExportOptions, ExportSummary, Manifest, BACKUP_FORMAT}; @@ -35,7 +35,8 @@ pub(super) fn export_to( ) -> Result { let root = crate::storage::data_root(app)?; // Migration may archive the old global bookmark state; enumerate files afterwards. - let book_count = crate::book_store::list_book_meta(app)?.len() as u64; + let books = crate::book_store::backup_books(app)?; + let book_count = books.ids()?.len() as u64; let mut state = Vec::new(); walk(&root.join(STATE_DIR), STATE_DIR, &is_state_file, &mut state)?; let mut images = Vec::new(); @@ -95,7 +96,7 @@ pub(super) fn export_to( write_entries(&mut zip, entries, category, report)?; } - crate::book_store::export_books(app, &mut zip, report)?; + books.write_zip(&mut zip, report)?; let file = zip.finish().map_err(|e| format!("写入备份文件失败: {e}"))?; let bytes = file.metadata().map(|meta| meta.len()).unwrap_or(0); diff --git a/src-tauri/src/data_transfer/import.rs b/src-tauri/src/data_transfer/import.rs index 4923fde..7587866 100644 --- a/src-tauri/src/data_transfer/import.rs +++ b/src-tauri/src/data_transfer/import.rs @@ -36,7 +36,8 @@ pub(super) fn apply( let bytes = file.metadata().map(|meta| meta.len()).unwrap_or(0); let mut zip = archive::open_zip_from(file)?; let manifest = archive::read_manifest(&mut zip)?; - let scan = archive::classify(&mut zip, manifest, bytes); + let mut scan = archive::classify(&mut zip, manifest, bytes); + let books = archive::open_books(&storage::data_root(app)?, &mut zip, &mut scan)?; log::info!( "开始导入备份 方式={} 书籍={} 插图={} 书源={} 状态={} 登录信息={}", mode.as_str(), @@ -54,7 +55,8 @@ pub(super) fn apply( let local_books = plan::local_books(app)?; let local_sources = readerx_source::store::list_sources().unwrap_or_default(); let archive_sources = plan::read_archive_sources(&mut zip, &scan)?; - let archive_books = plan::read_archive_books(&mut zip, &scan, &archive_sources, mode)?; + let archive_books = + plan::read_archive_books(&mut zip, books.as_ref(), &scan, &archive_sources, mode)?; // 分组换算表：书与书源的 groupId 都要跟着走 let remap = plan::plan_remap(app, mode, &local_books, &archive_books, &mut zip, &scan)?; @@ -63,7 +65,16 @@ pub(super) fn apply( .map(|book| plan::plan_book(book, &local_books, mode)) .collect(); - write_books(app, &mut zip, &plan, &remap, mode, &mut summary, report)?; + write_books( + app, + &mut zip, + books.as_ref(), + &plan, + &remap, + mode, + &mut summary, + report, + )?; write_images(app, &mut zip, &scan, &plan, mode, &mut summary, report)?; write_sources(&local_sources, &archive_sources, &remap, mode, &mut summary)?; write_sessions(app, &mut zip, &scan, report)?; @@ -136,6 +147,7 @@ pub(super) fn apply( fn write_books( app: &AppHandle, zip: &mut ZipArchive, + books: Option<&book_store::BackupDatabase>, plan: &[BookPlan], remap: &Remap, mode: ImportMode, @@ -151,9 +163,9 @@ fn write_books( } if item.action != BookAction::Skip { let prefix = format!("{BOOKS_DIR}/{}/", item.archive_id); - let detail = archive::read_entry(zip, &format!("{prefix}bookdetail.json"))? + let detail = archive::read_book_entry(zip, books, &format!("{prefix}bookdetail.json"))? .ok_or("备份缺少书籍元信息")?; - let content = archive::read_entry(zip, &format!("{prefix}content.json"))?; + let content = archive::read_book_entry(zip, books, &format!("{prefix}content.json"))?; let detail = remap_detail(&detail, remap, mode)?; book_store::import_book_json(app, &item.local_id, &detail, content.as_deref())?; for name in &item.entries { @@ -161,7 +173,7 @@ fn write_books( if !matches!(file, "annotations.json" | "bookmarks.json") { continue; } - let Some(bytes) = archive::read_entry(zip, name)? else { + let Some(bytes) = archive::read_book_entry(zip, books, name)? else { continue; }; if file == "annotations.json" { @@ -179,13 +191,13 @@ fn write_books( // 本机已有同一本书（另一个 id）：归档里的书签仍要并进本机那本，不能丢 if item.action == BookAction::Skip { let name = format!("{BOOKS_DIR}/{}/bookmarks.json", item.archive_id); - if let Some(bytes) = archive::read_entry(zip, &name)? { + if let Some(bytes) = archive::read_book_entry(zip, books, &name)? { merge_bookmark_file(app, &item.local_id, &bytes, mode)?; } } if item.action == BookAction::Skip { let name = format!("{BOOKS_DIR}/{}/annotations.json", item.archive_id); - if let Some(bytes) = archive::read_entry(zip, &name)? { + if let Some(bytes) = archive::read_book_entry(zip, books, &name)? { merge_annotation_file(app, &item.local_id, &bytes, mode)?; } } else if mode == ImportMode::Replace diff --git a/src-tauri/src/data_transfer/mod.rs b/src-tauri/src/data_transfer/mod.rs index a78098d..6226c4b 100644 --- a/src-tauri/src/data_transfer/mod.rs +++ b/src-tauri/src/data_transfer/mod.rs @@ -9,15 +9,12 @@ //! 导入的语义只有「合并」与「覆盖恢复」两种（见 [`ImportMode`]）。 //! //! 归档就是一个 **zip 文件**（后缀 `.zip`，正文体积大必须压缩；同时用户能自己解开检查）， -//! 保持原来的可移植 JSON 路径；书籍由 SQLite 转换为这些条目，其他数据按文件导出： +//! 书籍导出一致的 SQLite 快照，书源与状态保持 JSON；兼容导入旧 JSON 备份： //! //! ```text //! readerx-backup.json 清单：格式版本、导出版本、时间、是否含登录信息、各类条目数 //! state/.json readerx.* 状态（书架进度、分组、替换 / 分章规则、偏好…） -//! books//bookdetail.json 书籍元信息 -//! books//content.json 章节正文 -//! books//bookmarks.json 书签 -//! books//annotations.json 段落注释 +//! books.sqlite3 书籍元信息、章节正文、书签与注释的一致快照 //! images/ 章节插图 / PDF 页面图（文件名前缀就是它所属的书籍 id） //! book_sources/.json 书源定义 //! source_sessions/.json 书源登录态（**仅当导出时勾选「包含登录信息」**） @@ -58,8 +55,8 @@ use serde::{Deserialize, Serialize}; /// 清单文件名（归档内路径） pub(crate) const MANIFEST_NAME: &str = "readerx-backup.json"; /// 归档格式标识；读到更高版本直接报错，而不是按旧语义读坏新数据 -pub(crate) const BACKUP_FORMAT: &str = "readerx-backup/1"; -/// 归档内的顶层分类；books 是数据库书籍转换出的便携 JSON +pub(crate) const BACKUP_FORMAT: &str = "readerx-backup/2"; +/// 归档内的顶层分类；books 仅用于兼容旧 JSON 备份 pub(crate) const STATE_DIR: &str = "state"; pub(crate) const BOOKS_DIR: &str = "books"; pub(crate) const IMAGES_DIR: &str = "images"; @@ -197,7 +194,10 @@ pub(crate) fn is_safe_entry(name: &str) -> bool { /// 清单里的格式版本是否本程序读得懂 pub(crate) fn format_supported(format: &str) -> bool { match format.strip_prefix("readerx-backup/") { - Some(version) => version.parse::().map(|v| v <= 1).unwrap_or(false), + Some(version) => version + .parse::() + .map(|v| (1..=2).contains(&v)) + .unwrap_or(false), None => false, } } @@ -247,7 +247,9 @@ mod tests { fn only_known_backup_formats_are_accepted() { assert!(format_supported(BACKUP_FORMAT)); assert!(format_supported("readerx-backup/1")); - assert!(!format_supported("readerx-backup/2")); + assert!(format_supported("readerx-backup/2")); + assert!(!format_supported("readerx-backup/0")); + assert!(!format_supported("readerx-backup/3")); assert!(!format_supported("readerx-backup/x")); assert!(!format_supported("其它工具/1")); } diff --git a/src-tauri/src/data_transfer/plan.rs b/src-tauri/src/data_transfer/plan.rs index be94ebe..5f648a4 100644 --- a/src-tauri/src/data_transfer/plan.rs +++ b/src-tauri/src/data_transfer/plan.rs @@ -88,7 +88,9 @@ fn local_source_urls() -> HashMap { /// 书籍身份：与 `readerx_sync::bridge` 同一口径，只是书源地址的来源由调用方给 /// （归档里的书要用**归档自己的**书源地址，否则跨设备对不上） fn book_uid(meta: &BookSyncMeta, source_urls: &HashMap) -> String { - if crate::sync::book_ids::canonical_id(&meta.id) { return meta.id.clone(); } + if crate::sync::book_ids::canonical_id(&meta.id) { + return meta.id.clone(); + } let source_url = meta .book_source_id .as_deref() @@ -127,6 +129,7 @@ pub(super) fn read_archive_sources( pub(super) fn read_archive_books( zip: &mut ZipArchive, + books: Option<&crate::book_store::BackupDatabase>, scan: &ArchiveScan, sources: &[ArchiveSource], mode: ImportMode, @@ -150,8 +153,11 @@ pub(super) fn read_archive_books( .cloned() .collect(); let uid = if mode == ImportMode::Merge { - match archive::read_entry_json(zip, &format!("{prefix}bookdetail.json"))? { - Some(value) => book_uid(&meta_from_detail(&value), &source_urls), + match archive::read_book_entry(zip, books, &format!("{prefix}bookdetail.json"))? { + Some(bytes) => book_uid( + &meta_from_detail(&serde_json::from_slice(&bytes).map_err(|e| e.to_string())?), + &source_urls, + ), None => String::new(), } } else { diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index adfabbb..690bb12 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -242,11 +242,15 @@ fn export_migrates_global_bookmarks_before_state_scan_and_rejects_unreadable_boo let marks = json!([{"id":"old-mark","bookId":"legacy","unknown":42}]); seed_state(&root, "readerx.bookmarks", &json!({"legacy":marks})); let backup = export(&app, &root, "global-marks", false); - let mut zip = zip::ZipArchive::new(fs::File::open(backup).unwrap()).unwrap(); + let mut zip = zip::ZipArchive::new(fs::File::open(&backup).unwrap()).unwrap(); assert!(zip.by_name("state/readerx.bookmarks.json").is_err()); - let stored = archive::read_entry_json(&mut zip, "books/legacy/bookmarks.json") + let manifest = archive::read_manifest(&mut zip).unwrap(); + let mut scan = archive::classify(&mut zip, manifest, 0); + let books = archive::open_books(&root, &mut zip, &mut scan) .unwrap() .unwrap(); + let stored: Value = + serde_json::from_slice(&books.read("legacy", "bookmarks.json").unwrap().unwrap()).unwrap(); assert_eq!(stored["bookmarks"], marks); let bad = root.join("books/bad/bookdetail.json"); fs::create_dir_all(bad.parent().unwrap()).unwrap(); @@ -584,13 +588,22 @@ fn archive_read_json(path: &Path, entry: &str) -> Value { .clone() } -fn omit_archive_entry(path: &Path, omitted: &str) { +/// Build an actual v1 JSON archive to keep old-format import coverage independent of v2. +fn convert_to_v1(root: &Path, path: &Path, omitted: Option<&str>) { + use std::io::Write; let mut original = zip::ZipArchive::new(fs::File::open(path).unwrap()).unwrap(); + let mut manifest = archive::read_manifest(&mut original).unwrap(); + let mut scan = archive::classify(&mut original, manifest.clone(), 0); + let books = archive::open_books(root, &mut original, &mut scan) + .unwrap() + .unwrap(); + manifest.format = "readerx-backup/1".into(); let temporary = path.with_extension("test.zip"); let mut rewritten = zip::ZipWriter::new(fs::File::create(&temporary).unwrap()); + archive::write_manifest(&mut rewritten, &manifest).unwrap(); for index in 0..original.len() { let mut entry = original.by_index(index).unwrap(); - if entry.name() == omitted { + if matches!(entry.name(), "books.sqlite3" | "readerx-backup.json") { continue; } rewritten @@ -598,9 +611,144 @@ fn omit_archive_entry(path: &Path, omitted: &str) { .unwrap(); std::io::copy(&mut entry, &mut rewritten).unwrap(); } + for id in &scan.books { + for file in [ + "bookdetail.json", + "content.json", + "bookmarks.json", + "annotations.json", + ] { + let name = format!("books/{id}/{file}"); + if omitted == Some(name.as_str()) { + continue; + } + rewritten + .start_file(name, archive::text_options(None)) + .unwrap(); + rewritten + .write_all(&books.read(id, file).unwrap().unwrap()) + .unwrap(); + } + } + drop(rewritten.finish().unwrap()); + drop(original); + drop(books); + fs::rename(temporary, path).unwrap(); +} + +#[test] +fn sqlite_export_keeps_sources_json_and_v1_import_restores_books() { + let _guard = SERIAL.lock().unwrap_or_else(|e| e.into_inner()); + let (app, root) = setup(); + seed_device_a(&root); + let backup = export(&app, &root, "layout", false); + let mut zip = archive::open_zip_from(fs::File::open(&backup).unwrap()).unwrap(); + let manifest = archive::read_manifest(&mut zip).unwrap(); + assert_eq!(manifest.format, "readerx-backup/2"); + assert!(zip.by_name("books.sqlite3").is_ok()); + assert!(!zip.file_names().any(|name| name.starts_with("books/") + || name.ends_with("-wal") + || name.ends_with("-shm"))); + let source = zip + .file_names() + .find(|n| n.starts_with("book_sources/")) + .unwrap() + .to_string(); + let value = archive::read_entry_json(&mut zip, &source) + .unwrap() + .unwrap(); + assert!(value["bookSourceUrl"].is_string()); + drop(zip); + // A fresh v2 restore imports book rows, never the old device's migration receipts. + let (_, root) = setup(); + let summary = import(&app, &backup, ImportMode::Replace); + assert_eq!(summary.books_added, 2); + let db = rusqlite::Connection::open(root.join("books.sqlite3")).unwrap(); + let receipts: i64 = db + .query_row("SELECT count(*) FROM migrated_books", [], |r| r.get(0)) + .unwrap(); + assert_eq!(receipts, 0); + drop(db); + convert_to_v1(&root, &backup, None); + let mut zip = archive::open_zip_from(fs::File::open(&backup).unwrap()).unwrap(); + assert_eq!( + archive::read_manifest(&mut zip).unwrap().format, + "readerx-backup/1" + ); + assert!(zip.by_name("books.sqlite3").is_err()); + drop(zip); + let (_, root) = setup(); + let summary = import(&app, &backup, ImportMode::Merge); + assert_eq!(summary.books_added, 2); + let books = crate::book_store::list_book_meta(&app).unwrap(); + assert_eq!(books.len(), 2); + assert!( + !read_json(&root.join("books/b-5e20dcb39c59dc92/content.json"))["chapters"] + .as_array() + .unwrap() + .is_empty() + ); +} + +/// Mutate the SQLite payload without ever touching the local database. +fn rewrite_database(path: &Path, root: &Path, change: Option<&str>) { + let mut original = archive::open_zip_from(fs::File::open(path).unwrap()).unwrap(); + let temporary = path.with_extension("test.zip"); + let database = root.join("tampered.sqlite3"); + { + let mut entry = original.by_name("books.sqlite3").unwrap(); + std::io::copy(&mut entry, &mut fs::File::create(&database).unwrap()).unwrap(); + } + if let Some(sql) = change { + let db = rusqlite::Connection::open(&database).unwrap(); + db.execute_batch(sql).unwrap(); + } + let mut rewritten = zip::ZipWriter::new(fs::File::create(&temporary).unwrap()); + for index in 0..original.len() { + let mut entry = original.by_index(index).unwrap(); + if entry.name() == "books.sqlite3" { + if change.is_some() { + rewritten + .start_file(entry.name(), archive::text_options(None)) + .unwrap(); + std::io::copy(&mut fs::File::open(&database).unwrap(), &mut rewritten).unwrap(); + } + } else { + rewritten + .start_file(entry.name(), archive::text_options(None)) + .unwrap(); + std::io::copy(&mut entry, &mut rewritten).unwrap(); + } + } drop(rewritten.finish().unwrap()); drop(original); fs::rename(temporary, path).unwrap(); + fs::remove_file(database).unwrap(); +} + +#[test] +fn invalid_sqlite_archive_fails_before_local_writes_and_cleans_temporary_files() { + let _guard = SERIAL.lock().unwrap_or_else(|e| e.into_inner()); + let (app, root) = setup(); + seed_device_a(&root); + for (index,change) in [ + None, + Some("PRAGMA user_version=2;"), + Some("UPDATE chapters SET chapter='broken';"), + Some("UPDATE books SET id='../outside';"), + Some("INSERT INTO annotations SELECT id,'{}' FROM books;"), + Some("PRAGMA foreign_keys=OFF; INSERT INTO bookmarks VALUES('missing','[]');"), + Some("DROP TABLE annotations; CREATE VIEW annotations AS SELECT book_id,records FROM bookmarks;"), + ].iter().enumerate() { + let backup = export(&app,&root,&format!("invalid-{index}"),false); + let before = fs::read(root.join("books.sqlite3")).unwrap(); + rewrite_database(&backup,&root,*change); + for mode in [ImportMode::Merge,ImportMode::Replace] { + assert!(apply(&app,&backup.to_string_lossy(),mode,&mut |_,_,_|{}).is_err(),"invalid case {index}"); + assert_eq!(fs::read(root.join("books.sqlite3")).unwrap(),before,"invalid archive must not change local data"); + assert!(!fs::read_dir(root.join("books")).unwrap().flatten().any(|e|e.file_name().to_string_lossy().ends_with(".tmp")),"temporary snapshot must be cleaned"); + } + } } #[test] @@ -638,7 +786,11 @@ fn annotations_survive_archive_merge_replace_and_old_backup() { assert_eq!(read_json(&path)["annotations"][0], paragraph); let old_backup = export(&app, &root, "without-annotations", false); // Reproduce a backup written before annotations existed, rather than merely storing an empty list. - omit_archive_entry(&old_backup, &format!("books/{id}/annotations.json")); + convert_to_v1( + &root, + &old_backup, + Some(&format!("books/{id}/annotations.json")), + ); write_json(&path, &json!({"schemaVersion":1,"annotations":[paragraph]})); import(&app, &old_backup, ImportMode::Replace); assert!(read_json(&path)["annotations"] From 123e30cad8d0fdb7eb516190672faaf4e801c882 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 05:57:51 +0800 Subject: [PATCH 08/17] =?UTF-8?q?fix:=20=E6=A0=A1=E9=AA=8C=E5=A4=87?= =?UTF-8?q?=E4=BB=BD=E7=89=88=E6=9C=AC=E4=B8=8E=E4=B9=A6=E7=B1=8D=E5=B8=83?= =?UTF-8?q?=E5=B1=80=E4=B8=80=E8=87=B4=E9=81=BF=E5=85=8D=E9=94=99=E8=AF=AF?= =?UTF-8?q?=E8=A6=86=E7=9B=96?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src-tauri/src/data_transfer/archive.rs | 41 ++++++++++++++++++++++++++ src-tauri/src/data_transfer/mod.rs | 9 ++---- 2 files changed, 43 insertions(+), 7 deletions(-) diff --git a/src-tauri/src/data_transfer/archive.rs b/src-tauri/src/data_transfer/archive.rs index 6573eff..9ce37b6 100644 --- a/src-tauri/src/data_transfer/archive.rs +++ b/src-tauri/src/data_transfer/archive.rs @@ -309,6 +309,9 @@ pub(super) fn open_books( scan: &mut ArchiveScan, ) -> Result, String> { if scan.manifest.format != super::BACKUP_FORMAT { + if scan.entries.iter().any(|name| name == "books.sqlite3") { + return Err("备份书籍布局与格式版本不符".into()); + } return Ok(None); } if scan.entries.iter().any(|name| name.starts_with("books/")) { @@ -422,6 +425,44 @@ fn valid_image(name: &str) -> bool { mod tests { use super::*; + #[test] + fn archive_version_must_match_book_layout() { + let dir = + std::env::temp_dir().join(format!("readerx-archive-layout-{}", readerx_sync::new_id())); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join("backup.zip"); + for (format, mixed_json) in [("readerx-backup/1", false), ("readerx-backup/2", true)] { + let mut zip = writer(File::create(&path).unwrap()); + let manifest = Manifest { + format: format.into(), + app_version: "test".into(), + created_at: 0, + credentials: false, + books: 0, + images: 0, + sources: 0, + state_keys: 0, + }; + write_manifest(&mut zip, &manifest).unwrap(); + zip.start_file("books.sqlite3", text_options(None)).unwrap(); + zip.write_all(b"not a database").unwrap(); + if mixed_json { + zip.start_file("books/a/bookdetail.json", text_options(None)) + .unwrap(); + zip.write_all(b"{}").unwrap(); + } + drop(zip.finish().unwrap()); + let mut zip = open_zip_from(File::open(&path).unwrap()).unwrap(); + let mut scan = classify(&mut zip, manifest, 0); + assert!(open_books(&dir, &mut zip, &mut scan).is_err()); + assert!( + !dir.join("books").exists(), + "reject layout before extracting" + ); + } + fs::remove_dir_all(dir).unwrap(); + } + #[test] fn entry_allocation_ignores_declared_uncompressed_size() { let dir = std::env::temp_dir().join(format!("readerx-archive-size-{}", std::process::id())); diff --git a/src-tauri/src/data_transfer/mod.rs b/src-tauri/src/data_transfer/mod.rs index 6226c4b..9a2a59f 100644 --- a/src-tauri/src/data_transfer/mod.rs +++ b/src-tauri/src/data_transfer/mod.rs @@ -193,13 +193,7 @@ pub(crate) fn is_safe_entry(name: &str) -> bool { /// 清单里的格式版本是否本程序读得懂 pub(crate) fn format_supported(format: &str) -> bool { - match format.strip_prefix("readerx-backup/") { - Some(version) => version - .parse::() - .map(|v| (1..=2).contains(&v)) - .unwrap_or(false), - None => false, - } + matches!(format, "readerx-backup/1" | "readerx-backup/2") } /// 当前时间（毫秒时间戳） @@ -249,6 +243,7 @@ mod tests { assert!(format_supported("readerx-backup/1")); assert!(format_supported("readerx-backup/2")); assert!(!format_supported("readerx-backup/0")); + assert!(!format_supported("readerx-backup/02")); assert!(!format_supported("readerx-backup/3")); assert!(!format_supported("readerx-backup/x")); assert!(!format_supported("其它工具/1")); From 2052ec52c7166f278063411822efd85fb007200c Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 05:58:08 +0800 Subject: [PATCH 09/17] =?UTF-8?q?fix:=20=E4=BF=AE=E6=AD=A3=E5=BC=82?= =?UTF-8?q?=E5=B8=B8=E5=A4=87=E4=BB=BD=E6=A0=B7=E6=9C=AC=E9=81=BF=E5=85=8D?= =?UTF-8?q?=E6=9E=84=E9=80=A0=E9=98=B6=E6=AE=B5=E4=B8=BB=E9=94=AE=E5=86=B2?= =?UTF-8?q?=E7=AA=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src-tauri/src/data_transfer/round_trip.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index 690bb12..8853306 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -735,8 +735,8 @@ fn invalid_sqlite_archive_fails_before_local_writes_and_cleans_temporary_files() None, Some("PRAGMA user_version=2;"), Some("UPDATE chapters SET chapter='broken';"), - Some("UPDATE books SET id='../outside';"), - Some("INSERT INTO annotations SELECT id,'{}' FROM books;"), + Some("UPDATE books SET id='../outside-' || id;"), + Some("INSERT OR REPLACE INTO annotations SELECT id,'{}' FROM books;"), Some("PRAGMA foreign_keys=OFF; INSERT INTO bookmarks VALUES('missing','[]');"), Some("DROP TABLE annotations; CREATE VIEW annotations AS SELECT book_id,records FROM bookmarks;"), ].iter().enumerate() { From db36727a7c936e37611de1ecc1708f32c53e6f2d Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:00:01 +0800 Subject: [PATCH 10/17] =?UTF-8?q?docs:=20=E5=AF=B9=E9=BD=90=E6=B3=A8?= =?UTF-8?q?=E9=87=8A=E8=A1=A8=E4=B8=8E=20SQLite=20=E5=A4=87=E4=BB=BD?= =?UTF-8?q?=E7=9A=84=E5=AD=98=E5=82=A8=E8=AF=B4=E6=98=8E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../readerx-data/references/contracts.md | 4 ++-- docs/annotations.md | 19 ++++++++++--------- 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/.agents/skills/readerx-data/references/contracts.md b/.agents/skills/readerx-data/references/contracts.md index 2c526f5..3f205cd 100644 --- a/.agents/skills/readerx-data/references/contracts.md +++ b/.agents/skills/readerx-data/references/contracts.md @@ -4,8 +4,8 @@ ## 书库与稳定身份 -- book_store.rs / book_store/sqlite.rs / book_store/backup.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、状态 JSON 和图片字节不入库。数据库 user_version 与便携 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 -- 注释走 readerx_annotations_get/put，与正文/书签独立存表，便携信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含注释表并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 +- book_store.rs / book_store/sqlite.rs / book_store/backup.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、状态 JSON 和图片字节不入库。数据库 user_version 与旧 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 +- 注释走 readerx_annotations_get/put，与正文/书签独立存表，旧 JSON 信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含注释表并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 - book_store 的读写入口在进程级书库事务锁内完成迁移、读取、修改、落盘及指纹更新；持锁时不调用同步引擎。单章补丁以 chapter.cid 定位，index 仅兼容旧载荷；先验证整批 cid 存在且唯一，再只改正文，保留当前目录标题/地址。 - 普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 - 旧整书 / 目录 JSON 逐本在事务内导入并保存迁移回执，成功才改名 .migrated 留底；失败保留读回退和重试，不影响其他书。归档中断不能重复导入旧数据覆盖新编辑。数据库身份改动和状态 / TTS 目录迁移保持可重跑。 diff --git a/docs/annotations.md b/docs/annotations.md index 48c4d17..8f8dfbf 100644 --- a/docs/annotations.md +++ b/docs/annotations.md @@ -20,22 +20,23 @@ 在书库事务锁内保存到 `books.sqlite3` 的 `annotations` 表，与章节和书签分别存表。 纯浏览器开发模式只存内存，刷新页面后不保留。 -文件信封为 `schemaVersion: 1` 与 `annotations` 数组。每条记录包含独立 `id`、 +SQLite 的 `records` 列保存注释数组；旧 JSON 备份信封为 `schemaVersion: 1` 与 +`annotations` 数组。每条记录包含独立 `id`、 `chapterCid`、`unitIndex`、原段落文本的 SHA-256 `fingerprint`、相邻段落指纹 `before/after`，以及该段聚合的 `notes` 数组。每条 note 包含 `id`、`text`、 -`createdAt` 与 `updatedAt`。文件内不重复保存 bookId，所属书籍由目录决定， -因此书籍身份迁移时可随整个目录移动。 +`createdAt` 与 `updatedAt`。记录内不重复保存 bookId，归属由表的 `book_id` 外键决定， +书籍身份迁移时级联更新。 锚点使用原书的段落单元，显示替换、简繁转换和字号变化不改动其身份。 正文重排后采用唯一文本指纹或邻段指纹定位；存在歧义或章节 cid 已变化时， -保留文件里的记录，不将图标挂到无法确认的段落。 +保留数据库里的记录，不将图标挂到无法确认的段落。 首次操作先读取该书的注释，按 note 计算本次保存差集，Rust 在同一书库事务内 -合并磁盘上的最新记录，成功后才更新界面。读取失败不作为空数组写回；保存失败保留草稿供重试。删除书籍时文件随目录删除并清除该书缓存。 +合并磁盘上的最新记录，成功后才更新界面。读取失败不作为空数组写回；保存失败保留草稿供重试。删除书籍时数据库级联删除注释并清除该书缓存。 ## 备份 -全量备份包含注释文件。合并恢复以章节 cid、段落序号和文本指纹聚合记录， +全量备份的 `books.sqlite3` 包含注释表。合并恢复以章节 cid、段落序号和文本指纹聚合记录， 同段 note 按 id 取并集；重复 id 保留本机内容。覆盖恢复采用归档中的记录， 恢复没有注释文件的旧备份时清空该书当前注释。导入后清除缓存，下次读取重新加载。 @@ -53,14 +54,14 @@ 已打开的阅读页和注释抽屉随之更新，正在编辑的草稿保留。 同步协议升级为 `readerx-sync/4`，两端都需升级；旧端缺少注释 schema 和删除级联规则， -在握手阶段拒绝混用。书库注释文件仍是 schemaVersion=1，无需改写已有正文或书签。 +在握手阶段拒绝混用。书库使用 SQLite（user_version=1），旧 JSON 信封仍按 schemaVersion=1 导入。 ## 检查方法 - `node scripts/annotations-test.mjs`：同段多条、编辑、书籍隔离、重复文字重定位、 并发首次保存、读写失败保护及缓存失效。 -- `cargo test -p readerx --lib book_store`：按书文件读写、缺失文件、损坏文件及版本校验。 -- `cargo test -p readerx-sync` 和 `cargo test -p readerx --test sync_bridge`：逐条合并、并发编辑及裁决、第三设备转发、启动幂等、删书级联、旧快照保存、损坏文件失败与备份恢复。 +- `cargo test -p readerx --lib book_store`：按书记录读写、缺失记录、损坏记录及版本校验。 +- `cargo test -p readerx-sync` 和 `cargo test -p readerx --test sync_bridge`：逐条合并、并发编辑及裁决、第三设备转发、启动幂等、删书级联、旧快照保存、损坏记录失败与备份恢复。 - `cargo test -p readerx --lib data_transfer`：真实 zip 与磁盘往返、同段合并、覆盖恢复及旧备份。 - 界面验证：手机单页与桌面双页选区入口、段末点击、添加/编辑、字号变化与滚动模式； Android / Linux / Windows 原生应用和软键盘行为需在对应设备确认。 From 6ff0544709547fd50bc71931b98eaa8033c6fe97 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:22:27 +0800 Subject: [PATCH 11/17] =?UTF-8?q?feat:=20=E5=B0=86=E5=88=86=E7=BB=84?= =?UTF-8?q?=E8=A7=84=E5=88=99=E5=92=8C=E4=B9=A6=E6=9E=B6=E8=BF=9B=E5=BA=A6?= =?UTF-8?q?=E5=AD=98=E5=85=A5=20SQLite?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agents/skills/readerx-data/SKILL.md | 2 +- .../readerx-data/references/contracts.md | 6 +- docs/annotations.md | 2 +- docs/backup.md | 8 +- docs/book-storage.md | 21 +- docs/sync.md | 6 +- src-tauri/Cargo.toml | 2 +- src-tauri/src/book_store.rs | 23 +- src-tauri/src/book_store/backup.rs | 47 ++- src-tauri/src/book_store/sqlite.rs | 20 +- src-tauri/src/book_store/state.rs | 393 ++++++++++++++++++ src-tauri/src/data_transfer/archive.rs | 20 + src-tauri/src/data_transfer/export.rs | 4 +- src-tauri/src/data_transfer/import.rs | 26 +- src-tauri/src/data_transfer/plan.rs | 10 +- src-tauri/src/data_transfer/round_trip.rs | 10 +- src-tauri/src/storage.rs | 16 +- src-tauri/src/sync/book_ids.rs | 25 +- src-tauri/src/sync/data_ids.rs | 44 +- src-tauri/tests/support/book_json.rs | 104 +++++ src-tauri/tests/sync_bridge.rs | 7 +- 21 files changed, 739 insertions(+), 57 deletions(-) create mode 100644 src-tauri/src/book_store/state.rs diff --git a/.agents/skills/readerx-data/SKILL.md b/.agents/skills/readerx-data/SKILL.md index a9f371a..b6fdf90 100644 --- a/.agents/skills/readerx-data/SKILL.md +++ b/.agents/skills/readerx-data/SKILL.md @@ -11,7 +11,7 @@ description: 修改 ReaderX 本地书库存储、稳定身份与旧数据迁移 | 功能 | 实现入口 | 按需资料 | | --- | --- | --- | -| 本地书库存储、迁移、稳定 ID | src-tauri/src/book_store.rs、book_store/sqlite.rs、book_store/backup.rs、storage.rs、models.rs、sync/identity.rs、sync/book_ids.rs、sync/data_ids.rs；src/lib/books.ts、booksTypes.ts、dataIds.ts | [存储与身份约束](references/contracts.md#书库与稳定身份) | +| 本地书库存储、迁移、稳定 ID | src-tauri/src/book_store.rs、book_store/sqlite.rs、book_store/backup.rs、book_store/state.rs、storage.rs、models.rs、sync/identity.rs、sync/book_ids.rs、sync/data_ids.rs；src/lib/books.ts、booksTypes.ts、dataIds.ts | [存储与身份约束](references/contracts.md#书库与稳定身份) | | 备份与恢复 | src-tauri/src/data_transfer/；src/lib/backup.ts、src/pages/DataBackup.tsx | [备份约束](references/contracts.md#备份与恢复)、[备份文档](../../../docs/backup.md) | | 同步协议、合并与传输 | src-tauri/crates/readerx-sync/ | [同步约束](references/contracts.md#局域网同步)、[同步文档](../../../docs/sync.md) 对应章节 | | 同步 App 接线与落库 | src-tauri/src/sync/；src/lib/sync.ts、src/components/SyncProgress.tsx | 同步文档 App 接线、实时落库与停止部分 | diff --git a/.agents/skills/readerx-data/references/contracts.md b/.agents/skills/readerx-data/references/contracts.md index 3f205cd..56986f9 100644 --- a/.agents/skills/readerx-data/references/contracts.md +++ b/.agents/skills/readerx-data/references/contracts.md @@ -4,10 +4,10 @@ ## 书库与稳定身份 -- book_store.rs / book_store/sqlite.rs / book_store/backup.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、状态 JSON 和图片字节不入库。数据库 user_version 与旧 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 +- book_store.rs / book_store/sqlite.rs / book_store/backup.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations、groups、rules；进度直接在 books.detail.progress）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、其他状态 JSON 和图片字节不入库。数据库 user_version 与旧 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 - 注释走 readerx_annotations_get/put，与正文/书签独立存表，旧 JSON 信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含注释表并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 - book_store 的读写入口在进程级书库事务锁内完成迁移、读取、修改、落盘及指纹更新；持锁时不调用同步引擎。单章补丁以 chapter.cid 定位，index 仅兼容旧载荷；先验证整批 cid 存在且唯一，再只改正文，保留当前目录标题/地址。 -- 普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 +- 分组 / 书源分组 / 文本替换 / 分章规则经 book_store/state.rs 存库，书架 API 是 books.detail.progress 的视图；整书及元信息补丁不得清除进度。这些状态与书庫共用事务锁，旧 JSON 和回执同事务迁入，成功才归档；数据库 user_version=2，恢复兼容 v1。其他普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 - 旧整书 / 目录 JSON 逐本在事务内导入并保存迁移回执，成功才改名 .migrated 留底；失败保留读回退和重试，不影响其他书。归档中断不能重复导入旧数据覆盖新编辑。数据库身份改动和状态 / TTS 目录迁移保持可重跑。 - 书库与同步共用稳定 ID；已有规范 ID 沿用，不能在每次编辑后按新名字/地址重新计算。身份算法以 sync/identity.rs 为准：在线书首次按书源地址与详情页地址派生，导入书按归一文件名与字节数派生。分组与书源首次按名称/地址派生后编辑沿用 ID；规则仍按内容派生。前端 dataIds.ts 与 Rust 种子、大小写和归一化口径保持一致。 - 迁移顺序与启动入口检查 src-tauri/src/lib.rs：书籍迁移在 UI/同步访问前完成，data_ids 再处理分组、书源、规则及引用。迁移待办先落盘，每步可重跑；目录或身份冲突保留原数据并返回失败，不能覆盖目标。 @@ -16,7 +16,7 @@ ## 备份与恢复 -- 归档入口按 data_transfer/archive.rs 分类与校验条目名，拒绝越界路径；不绕过 is_safe_entry 直接解压。plan.rs 只读规划，import.rs 写盘；书籍导入先经 import_book_json 在事务内写元信息和正文，再处理书签/注释；导出 readerx-backup/2，在书库锁内用 VACUUM INTO 生成一致的 books.sqlite3 快照并流式写 ZIP，书源仍导出 JSON，事务日志不进归档；导入先只读校验数据库，再逐书复用身份规划与合并入口，不整库替换或导入迁移回执；兼容 readerx-backup/1 JSON，图片流式处理。 +- 归档入口按 data_transfer/archive.rs 分类与校验条目名，拒绝越界路径；不绕过 is_safe_entry 直接解压。plan.rs 只读规划，import.rs 写盘；书籍导入先经 import_book_json 在事务内写元信息和正文，再处理书签/注释；导出 readerx-backup/2，在书库锁内用 VACUUM INTO 生成一致的 books.sqlite3 快照并流式写 ZIP，书源仍导出 JSON，分组、规则与进度只在数据库快照中导出，事务日志不进归档；导入先只读校验数据库，再逐书复用身份规划与合并入口，不整库替换或导入迁移回执；兼容 readerx-backup/1 JSON，图片流式处理。 - 合并与覆盖语义不同：合并保留本机额外数据并按身份映射引用，偏好已有值保留；覆盖先写后删并清空归档缺失的内容类状态。新增 state 键时明确分类、合并规则、缺失语义与 ID 换算，不能直接全量 JSON 覆盖。 - 书源会话与 WebDAV 密码默认过滤，显式包含才导出；sync/、tts-audio/、logs/ 和本机 reading-time-origin 不作为换机数据搬走。阅读时长 contributions 应保留，恢复后本机新增时间使用独立来源。 - 覆盖删除书或书源前先调用同步删除钩子（定位身份仍需要旧数据），批量导入后调用 on_data_imported 全量对账。前端 import 完成后按 backup.ts 依赖顺序失效书签并重载分组、书库、进度、统计、规则、书源。 diff --git a/docs/annotations.md b/docs/annotations.md index 8f8dfbf..7c28f8e 100644 --- a/docs/annotations.md +++ b/docs/annotations.md @@ -1,6 +1,6 @@ # 阅读注释 -书籍在应用目录与备份中使用 SQLite，书源和状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 +书籍、分组、规则与书架进度在应用目录与备份中使用 SQLite，书源和其他状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 在同一段落内选取文字，点选区菜单的「注释」即可写下一条文本注释。保存后， 该段最后一个字符右侧出现 SVG 图标；点击图标可查看该段的全部注释、继续添加， diff --git a/docs/backup.md b/docs/backup.md index 5a96d69..77e6292 100644 --- a/docs/backup.md +++ b/docs/backup.md @@ -1,6 +1,6 @@ # 数据备份与恢复（导入 / 导出） -书籍在应用目录与备份中使用 SQLite，书源和状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 +书籍、分组、规则与书架进度在应用目录与备份中使用 SQLite，书源和其他状态保持 JSON；存储边界与旧数据迁移见 [书籍存储](book-storage.md)。 > 状态：**已接入 App**（设置 → 数据 → 数据备份）。本文件说明备份里有什么、归档长什么样、 > 导入的两种语义，以及与「局域网同步」的边界。 @@ -26,15 +26,15 @@ ```text readerx-backup.json 清单：格式版本、导出版本、时间、是否含登录信息、各类条目数 -state/.json readerx.* 状态（书架进度、分组、书源分组、替换 / 分章规则、偏好…） -books.sqlite3 书籍元信息、正文、书签与段落注释的一致快照 +state/.json 偏好、阅读时长等其他 readerx.* 状态 +books.sqlite3 书籍（含元信息中的进度）、正文、书签、注释、分组与规则的一致快照 images/ 章节插图 / PDF 页面图（文件名前缀就是它所属的书籍 id） book_sources/.json 书源定义 source_sessions/.json 书源登录态（**仅当导出时勾选「包含登录信息」**） ``` 清单里的 `format` 是 `readerx-backup/2`：书籍导出 SQLite，书源仍导出 `book_sources/.json`。 -导入兼容 `readerx-backup/1` 的 `books//*.json`；更高版本直接报错。旧版 ReaderX 无法读取新备份。 +数据库 user_version=2，导入也兼容 v1 数据库与状态 JSON。导入兼容 `readerx-backup/1` 的 `books//*.json`；更高版本直接报错。旧版 ReaderX 无法读取新备份。 导出先在书库锁内通过 `VACUUM INTO` 生成一致快照，再流式写入 ZIP，事务日志不进归档。 恢复先校验数据库版本、完整性、引用及逐行数据，再逐书执行身份映射与合并 / 覆盖， diff --git a/docs/book-storage.md b/docs/book-storage.md index 901c586..0338a71 100644 --- a/docs/book-storage.md +++ b/docs/book-storage.md @@ -11,7 +11,10 @@ command 访问，数据形状和同步协议不变。 | 元信息（含封面字段） | SQLite `books`，按书籍 ID 一行 | | 章节正文、目录、字数、同步指纹和图片引用名 | SQLite `chapters`，按书籍 ID 和章节顺序一行，cid 建索引 | | 书签、段落注释 | SQLite `bookmarks` / `annotations`，记录原样保存 | -| 偏好、书架进度、分组、规则等状态 | 原有 `state/.json` | +| 书架进度 | SQLite `books.detail.progress`，随书籍元信息保存 | +| 书架 / 书源分组 | SQLite `groups`，按集合 key 和顺序逐项保存 | +| 文本替换 / 分章规则 | SQLite `rules`，按集合 key 和顺序逐项保存 | +| 偏好、阅读时长等其他状态 | 原有 `state/.json` | | 书源与登录态 | 原有 `book_sources/`、`source_sessions/` | | TTS 缓存 | 原有 `tts-audio//` | | 章节插图 / PDF 页面图的字节 | 原有 `images/`；数据库只存引用名 | @@ -27,7 +30,7 @@ command 访问，数据形状和同步协议不变。 `FULL` 同步，整书替换、正文批次与其指纹 / 图片引用、目录修改均由事务提交。 删书通过外键级联删除章节、书签和注释；图片继续检查其他书的引用后再清理。 数据库由每次操作按当前数据根打开，不把测试或同步夹具的另一份数据根混入本机。 -数据库格式使用 `PRAGMA user_version=1`，拒绝更高版本。 +数据库格式使用 `PRAGMA user_version=2`（自动升级 v1），拒绝更高版本。 首次访问逐本导入旧 `books//` 和 `books/.json`，同时处理旧全库书签。 每本书先完整读取并校验，再在一个事务中提交元信息、正文、书签、注释及迁移回执。 @@ -37,8 +40,14 @@ command 访问，数据形状和同步协议不变。 数据库中的新编辑。已有同名留底文件不被覆盖。 尚未迁入的书籍拒绝同步写入并返回失败，让同步保留待落库数据，避免当作已删书跳过更新。 +分组、书源分组、文本替换、分章规则和旧书架状态首次通过状态入口或备份访问时迁入数据库， +每个集合与迁移回执在同一事务提交，成功后将原 JSON 改名留底；失败保留原文件并返回错误。 +书架接口仍返回按书籍 ID 索引的进度视图，实际只更新有变化的 `books.detail.progress`。 +整书保存和元信息补丁保留进度；删除书籍同时删除进度。初始化只补缺项，重置在后端更新已有记录。 +`collections` 区分空集合与不存在的集合；`migrated_states` 防止归档中断后重复导入覆盖新编辑。 + 旧身份迁移待办继续使用可重跑日志；数据库书籍改 ID 在事务内级联更新， -书签中的 bookId 随之更新。书架、规则、阅读时长及 TTS 目录仍使用原迁移流程。 +书签中的 bookId 随之更新。进度中的 bookId 同事务更新，规则引用继续通过原身份迁移入口更新；阅读时长及 TTS 目录沿用文件迁移。 分组和书源引用同时更新数据库元信息；书源映射在重命名文件前单独留待办，避免中断丢映射。 损坏的旧元信息使后续引用迁移延后，保留原分组 / 书源及映射，已迁入书籍仍可正常使用。 书源保存继续由共享书源 crate 写文件，App 同步维护数据库中的书源引用。 @@ -48,13 +57,13 @@ command 访问，数据形状和同步协议不变。 新备份为 `readerx-backup/2`：`book_store::backup_books` 在书库锁内通过 `VACUUM INTO` 生成一致的 `books.sqlite3` 快照，流式压缩进 ZIP。 不复制正在写入的数据库文件，也不导出 WAL / SHM / journal；书源仍为 -`book_sources/.json`，状态与图片继续按文件导出。 +`book_sources/.json`，分组、规则与进度包含在同一快照中，其他状态与图片按文件导出。 恢复只读打开归档数据库，先校验版本、完整性、引用与逐行数据，再将单书内容适配到 现有 `book_store::import_book_json` 和记录合并入口，不整库覆盖本机数据库。 -归档里的迁移回执不导入本机。仍兼容旧 `readerx-backup/1` JSON 备份， +归档里的迁移回执不导入本机。恢复兼容数据库 v1（旧状态 JSON）与 v2，按原集合合并 / 覆盖语义还原，进度仍按 updatedAt 选择较新记录。仍兼容旧 `readerx-backup/1` JSON 备份， 合并 / 覆盖恢复、旧备份缺失注释清空、书签合并和凭据过滤保持原语义。 快照位于受启动回收管理的 `books/*.tmp`，正常与失败退出均清理。 -有尚未完成迁移的旧书籍时拒绝导出不完整备份。 +有尚未完成迁移的旧书籍或数据状态时拒绝导出不完整备份。 同步元信息、目录、正文指纹、按需取章、封面与插图引用全部经 `book_store`。 资源字节仍从文件读取，线上消息与协议版本不变。 diff --git a/docs/sync.md b/docs/sync.md index 46283aa..75d5a8f 100644 --- a/docs/sync.md +++ b/docs/sync.md @@ -413,7 +413,7 @@ cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock ``` `tests/sync_bridge.rs` 覆盖的是单元测试够不到的那一层：真实 SQLite 书籍数据 / -`bookmarks.json` / `readerx.shelf.json` / 书源文件与引擎实体之间的双向搬运 +SQLite 书签表 / books 元信息中的进度 / 书源文件与引擎实体之间的双向搬运 （首次对账、落地、重新导入、对端删书、冲突留档、规则类数据、目录落地时按 cid 保留正文、 对端正文落地时新建本地书、新设备主动拉取 TXT / EPUB / MOBI / PDF 与扫描 PDF 页面图），以及 App 删除设备后对方真的连不进来。 `crates/readerx-sync/tests/content_transfer.rs` 走进程内直连验证正文通道本身： @@ -437,7 +437,7 @@ cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock - 本机还没导入的书：它的规则留在引擎里**不落地**（本地清单里看不到）， 书落到本机时再补一次；同理，本机发布规则时**不会**把这类规则当成 「用户删掉了」而入墓碑 —— 否则一次本地发布就会抹掉对端为那本书建的规则。 -- **分章规则**（`readerx.chapterRules`）：内置规则是代码常量、不进状态文件， +- **分章规则**（`readerx.chapterRules`）：内置规则是代码常量、不进 SQLite 规则集合， 参与同步的只有用户自定义的那些；显示顺序按创建时间、其次实体 id， 两台设备看到同一个顺序。 @@ -633,7 +633,7 @@ cargo test -p readerx --test sync_bridge # 桥接：真实磁盘布局 + mock 接收端每个完整数据包经过校验与暂存后立即落库，成功后才确认推送或发送进度。 正文与图片暂存、书库文件采用写入后同步磁盘与原子替换；写盘失败保留暂存和原游标。 -App 的 `book_store` 入口使用同一进程级事务锁，覆盖读改写全过程及迁移、自愈回写、指纹缓存和删书；同步引擎钩子在释放书库锁后调用。下载章节补丁按 `chapter.cid` 匹配当前目录，拒绝缺失或重复身份，保留最新标题/地址。普通状态的读改写使用 `storage::update_state`，与状态写入/删除共用锁；前端阅读进度经 `readerx_shelf_patch` 仅提交本次改动的书，初始化只补缺项，避免旧书架快照覆盖同步进度。大书读写期间，其它书库操作会等待该事务完成。 +App 的 `book_store` 入口使用同一进程级事务锁，覆盖读改写全过程及迁移、自愈回写、指纹缓存和删书；同步引擎钩子在释放书库锁后调用。下载章节补丁按 `chapter.cid` 匹配当前目录，拒绝缺失或重复身份，保留最新标题/地址。状态的读改写使用 `storage::update_state`；分组、规则和进度路由到 SQLite 并与书库共用锁，其他状态与 JSON 写入/删除共用状态锁；前端阅读进度经 `readerx_shelf_patch` 仅提交本次改动的书，初始化只补缺项，避免旧书架快照覆盖同步进度。大书读写期间，其它书库操作会等待该事务完成。 落库串行执行，操作游标仅推进到落库开始时已经存在的操作，避免并发到达的数据被漏掉。 连接或同步错误时，在释放连接之前再执行一次落库。未完整收到或尚未通过校验的报文 diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index eb5af36..c17d38d 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -55,7 +55,7 @@ readerx-log = { path = "crates/readerx-log" } log = "0.4" serde = { version = "1", features = ["derive"] } serde_json = "1" -# 仅书籍数据入库；bundled 为 Android / Linux / Windows 编译 SQLite。 +# 书籍、分组、规则与进度入库；bundled 为 Android / Linux / Windows 编译 SQLite。 rusqlite = { version = "0.38", features = ["bundled"] } base64 = "0.22" # 跨设备书身份用 sha256 派生（见 src/sync/identity.rs）：同一个文件在两台设备上 diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index 46852fd..d429227 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -1,7 +1,7 @@ //! 本地书籍数据存储在应用数据根的 `books.sqlite3`。 //! 元信息、逐章正文与派生指纹、书签和注释经 SQLite 事务读写。 //! 旧目录 / 整书 JSON 逐本迁移，成功后改名 `.migrated` 留底，失败保留读回退。 -//! TTS、书源、状态 JSON 与图片字节保持文件存储；备份仍采用原 JSON 归档格式。 +//! 分组、规则同库保存，书架进度在 books 元信息中；TTS、书源、其他状态及图片字节保持文件存储。 use crate::models::{ BookChapterPatch, BookMeta, BookMetaPatch, ChapterBlock, ChapterHead, LocalBook, @@ -22,6 +22,8 @@ use tauri::AppHandle; #[path = "book_store/sqlite.rs"] mod sqlite; +#[path = "book_store/state.rs"] +pub(crate) mod state; /// 旧布局 / 便携备份的元信息文件名 const BOOKDETAIL_FILE: &str = "bookdetail.json"; @@ -49,6 +51,8 @@ fn schema_version() -> u32 { #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] struct BookDetail { + #[serde(default, skip_serializing_if = "Option::is_none")] + progress: Option, #[serde(default = "schema_version")] schema_version: u32, id: String, @@ -90,6 +94,7 @@ impl BookDetail { /// 从整书取元信息（正文留给 content.json） fn from_book(book: &LocalBook) -> Self { Self { + progress: None, schema_version: SCHEMA_VERSION, id: book.id.clone(), title: book.title.clone(), @@ -664,11 +669,13 @@ pub(crate) fn patch_book_meta( let mut db = sqlite::open(&root)?; sqlite::require_book(&db, &root, id)?; let tx = sqlite::transaction(&mut db)?; - let mut book = sqlite::detail(&tx, id)? - .ok_or("书籍不存在")? - .into_book(Vec::new()); + let detail = sqlite::detail(&tx, id)?.ok_or("书籍不存在")?; + let progress = detail.progress.clone(); + let mut book = detail.into_book(Vec::new()); if patch.apply_to(&mut book) { - sqlite::save_detail(&tx, &BookDetail::from_book(&book))?; + let mut detail = BookDetail::from_book(&book); + detail.progress = progress; + sqlite::save_detail(&tx, &detail)?; } sqlite::commit(tx) } @@ -1900,6 +1907,8 @@ pub(crate) fn backup_books( } } } + let mut db = db; + state::migrate(&mut db, &root)?; sqlite::snapshot(&db, &root) } /// Backup JSON is a portable representation, not the on-disk database schema. @@ -1934,6 +1943,7 @@ pub(crate) fn rename_id_at(root: &Path, old: &str, new: &str) -> Result<(), Stri sqlite::valid_id(old)?; sqlite::valid_id(new)?; let mut db = sqlite::open(root)?; + state::migrate(&mut db, root)?; let tx = sqlite::transaction(&mut db)?; let from = sqlite::detail(&tx, old)?; let to = sqlite::detail(&tx, new)?; @@ -1948,6 +1958,9 @@ pub(crate) fn rename_id_at(root: &Path, old: &str, new: &str) -> Result<(), Stri } if let Some(mut detail) = from { detail.id = new.to_string(); + if let Some(progress) = detail.progress.as_mut().and_then(Value::as_object_mut) { + progress.insert("bookId".into(), Value::String(new.into())); + } tx.execute( "UPDATE books SET id=?1,detail=?2 WHERE id=?3", rusqlite::params![ diff --git a/src-tauri/src/book_store/backup.rs b/src-tauri/src/book_store/backup.rs index d62bb2a..922db69 100644 --- a/src-tauri/src/book_store/backup.rs +++ b/src-tauri/src/book_store/backup.rs @@ -16,7 +16,7 @@ impl BackupDatabase { let version: i64 = db .query_row("PRAGMA user_version", [], |r| r.get(0)) .map_err(sql)?; - if version != DATABASE_VERSION { + if !(1..=DATABASE_VERSION).contains(&version) { return Err("备份书库数据库格式版本不受支持".into()); } let integrity: String = db @@ -42,7 +42,30 @@ impl BackupDatabase { if foreign_key_error { return Err("备份书库数据库引用无效".into()); } + if version >= 2 { + for table in ["collections", "groups", "rules"] { + let ordinary: bool = db.query_row( + "SELECT EXISTS(SELECT 1 FROM pragma_table_list WHERE schema='main' AND name=?1 AND type='table')", + [table], |r|r.get(0)).map_err(sql)?; + if !ordinary { + return Err("备份缺少分组或规则数据表".into()); + } + } + let mut query = db.prepare("SELECT key FROM collections").map_err(sql)?; + for row in query + .query_map([], |r| r.get::<_, String>(0)) + .map_err(sql)? + { + let key = row.map_err(sql)?; + if !super::super::state::handles(&key) || key == "readerx.shelf" { + return Err("备份包含未知数据集合".into()); + } + } + } let backup = Self { db, temporary }; + for key in backup.state_keys()? { + backup.read_state(&key)?; + } // Validate every payload before import can mutate local data, without loading the library. for id in backup.ids()? { valid_id(&id)?; @@ -75,6 +98,28 @@ impl BackupDatabase { } Ok(backup) } + pub(crate) fn state_keys(&self) -> Result, String> { + let version: i64 = self + .db + .query_row("PRAGMA user_version", [], |r| r.get(0)) + .map_err(sql)?; + if version < 2 { + return Ok(Vec::new()); + } + let mut query = self + .db + .prepare("SELECT key FROM collections ORDER BY key") + .map_err(sql)?; + let rows = query + .query_map([], |r| r.get::<_, String>(0)) + .map_err(sql)?; + let mut keys = rows.collect::, _>>().map_err(sql)?; + keys.push("readerx.shelf".into()); + Ok(keys) + } + pub(crate) fn read_state(&self, key: &str) -> Result, String> { + super::super::state::read(&self.db, key) + } #[cfg(test)] pub(super) fn path(&self) -> &Path { &self.temporary.0 diff --git a/src-tauri/src/book_store/sqlite.rs b/src-tauri/src/book_store/sqlite.rs index 7a94a93..b1d48b0 100644 --- a/src-tauri/src/book_store/sqlite.rs +++ b/src-tauri/src/book_store/sqlite.rs @@ -3,7 +3,7 @@ use super::*; use rusqlite::{params, Connection, OptionalExtension}; pub(super) const DATABASE: &str = "books.sqlite3"; -const DATABASE_VERSION: i64 = 1; +const DATABASE_VERSION: i64 = 2; fn sql(error: rusqlite::Error) -> String { format!("书库数据库操作失败: {error}") @@ -54,6 +54,20 @@ pub(super) fn connect(root: &Path) -> Result { ).map_err(sql)?; tx.commit().map_err(sql)?; } + if version < 2 { + let tx = db.transaction().map_err(sql)?; + tx.execute_batch( + "CREATE TABLE collections (key TEXT PRIMARY KEY); + CREATE TABLE groups (key TEXT NOT NULL REFERENCES collections(key) ON DELETE CASCADE, + position INTEGER NOT NULL, record TEXT NOT NULL, PRIMARY KEY(key,position)); + CREATE TABLE rules (key TEXT NOT NULL REFERENCES collections(key) ON DELETE CASCADE, + position INTEGER NOT NULL, record TEXT NOT NULL, PRIMARY KEY(key,position)); + CREATE TABLE migrated_states (key TEXT PRIMARY KEY); + PRAGMA user_version=2;", + ) + .map_err(sql)?; + tx.commit().map_err(sql)?; + } Ok(db) } @@ -328,7 +342,9 @@ pub(super) fn save_chapter( Ok(()) } pub(super) fn put(db: &Connection, book: &LocalBook) -> Result<(), String> { - save_detail(db, &BookDetail::from_book(book))?; + let mut next = BookDetail::from_book(book); + next.progress = detail(db, &book.id)?.and_then(|d| d.progress); + save_detail(db, &next)?; db.execute("DELETE FROM chapters WHERE book_id=?1", [&book.id]) .map_err(sql)?; for (position, chapter) in book.chapters.iter().enumerate() { diff --git a/src-tauri/src/book_store/state.rs b/src-tauri/src/book_store/state.rs new file mode 100644 index 0000000..e03cd23 --- /dev/null +++ b/src-tauri/src/book_store/state.rs @@ -0,0 +1,393 @@ +//! SQLite-backed collections and the legacy shelf API's view of books.detail.progress. +//! All entry points share the library lock; callbacks never re-enter storage or sync. +use super::{library_transaction, sqlite, BookDetail}; +use rusqlite::{params, Connection}; +use serde_json::{Map, Value}; +use std::{fs, path::Path}; + +pub(crate) const KEYS: [&str; 5] = [ + "readerx.groups", + "readerx.sourceGroups", + "readerx.textReplacements", + "readerx.chapterRules", + "readerx.shelf", +]; +pub(crate) fn handles(key: &str) -> bool { + KEYS.contains(&key) +} +fn sql(e: rusqlite::Error) -> String { + format!("数据状态数据库操作失败: {e}") +} +fn table(key: &str) -> Result<&'static str, String> { + match key { + "readerx.groups" | "readerx.sourceGroups" => Ok("groups"), + "readerx.textReplacements" | "readerx.chapterRules" => Ok("rules"), + _ => Err("未知的数据状态类型".into()), + } +} +pub(super) fn read(db: &Connection, key: &str) -> Result, String> { + if key == "readerx.shelf" { + let mut out = Map::new(); + let mut query = db + .prepare("SELECT id,detail FROM books ORDER BY id") + .map_err(sql)?; + let rows = query + .query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))) + .map_err(sql)?; + for row in rows { + let (id, raw) = row.map_err(sql)?; + let detail: BookDetail = serde_json::from_str(&raw).map_err(|e| e.to_string())?; + if let Some(progress) = detail.progress { + validate_progress(&progress)?; + out.insert(id, progress); + } + } + return Ok(Some(Value::Object(out))); + } + let table = table(key)?; + let present: bool = db + .query_row( + "SELECT EXISTS(SELECT 1 FROM collections WHERE key=?1)", + [key], + |r| r.get(0), + ) + .map_err(sql)?; + if !present { + return Ok(None); + } + let mut query = db + .prepare(&format!( + "SELECT record FROM {table} WHERE key=?1 ORDER BY position" + )) + .map_err(sql)?; + let rows = query + .query_map([key], |r| r.get::<_, String>(0)) + .map_err(sql)?; + let values: Result, String> = rows + .map(|row| { + let value: Value = + serde_json::from_str(&row.map_err(sql)?).map_err(|e| e.to_string())?; + if !value.is_object() { + return Err("分组或规则记录格式错误".into()); + } + Ok(value) + }) + .collect(); + Ok(Some(Value::Array(values?))) +} +fn validate_progress(value: &Value) -> Result<(), String> { + if !value.is_object() { + return Err("书架进度格式错误".into()); + } + Ok(()) +} +// Caller owns an SQL transaction; removing a book naturally removes its progress. +fn write(db: &Connection, root: &Path, key: &str, value: Option<&Value>) -> Result<(), String> { + if key == "readerx.shelf" { + let empty = Map::new(); + let entries = match value { + Some(value) => value.as_object().ok_or("书架状态格式错误")?, + None => &empty, + }; + for (id, progress) in entries { + sqlite::valid_id(id)?; + validate_progress(progress)?; + if sqlite::detail(db, id)?.is_none() { + sqlite::reject_pending_migration(root, id)?; + } + } + let ids = { + let mut q = db.prepare("SELECT id FROM books").map_err(sql)?; + let rows = q.query_map([], |r| r.get::<_, String>(0)).map_err(sql)?; + rows.collect::, _>>().map_err(sql)? + }; + for id in ids { + let mut detail = sqlite::detail(db, &id)?.ok_or("书籍元信息缺失")?; + let progress = entries.get(&id).map(|entry| { + let mut entry = entry.clone(); + entry["bookId"] = Value::String(id.clone()); + entry + }); + if detail.progress != progress { + detail.progress = progress; + sqlite::save_detail(db, &detail)?; + } + } + return Ok(()); + } + let table = table(key)?; + let items = value + .map(|v| v.as_array().ok_or("分组或规则列表格式错误")) + .transpose()?; + if items.is_some_and(|items| items.iter().any(|v| !v.is_object())) { + return Err("分组或规则记录格式错误".into()); + } + db.execute("DELETE FROM collections WHERE key=?1", [key]) + .map_err(sql)?; + if let Some(items) = items { + db.execute("INSERT INTO collections(key) VALUES(?1)", [key]) + .map_err(sql)?; + for (position, item) in items.iter().enumerate() { + db.execute( + &format!("INSERT INTO {table}(key,position,record) VALUES(?1,?2,?3)"), + params![key, position as i64, item.to_string()], + ) + .map_err(sql)?; + } + } + Ok(()) +} +/// Commit each legacy state and its receipt together, then archive the original. +/// A failed archive is retried without replaying a stale snapshot over newer edits. +pub(super) fn migrate(db: &mut Connection, root: &Path) -> Result<(), String> { + for key in KEYS { + let path = root.join("state").join(format!("{key}.json")); + if !path.is_file() { + continue; + } + let done: bool = db + .query_row( + "SELECT EXISTS(SELECT 1 FROM migrated_states WHERE key=?1)", + [key], + |r| r.get(0), + ) + .map_err(sql)?; + if !done { + let value: Value = serde_json::from_slice(&fs::read(&path).map_err(|e| e.to_string())?) + .map_err(|e| e.to_string())?; + let tx = db.transaction().map_err(sql)?; + // Existing database collections win on retry/upgrade; shelf adds only missing entries. + if key == "readerx.shelf" { + let current = read(&tx, key)?.unwrap_or_else(|| serde_json::json!({})); + let mut merged = value.as_object().ok_or("旧书架状态格式错误")?.clone(); + merged.extend(current.as_object().ok_or("书架状态格式错误")?.clone()); + write(&tx, root, key, Some(&Value::Object(merged)))?; + } else if read(&tx, key)?.is_none() { + write(&tx, root, key, Some(&value))?; + } + tx.execute("INSERT INTO migrated_states(key) VALUES(?1)", [key]) + .map_err(sql)?; + tx.commit().map_err(sql)?; + } + let backup = path.with_extension("json.migrated"); + if backup.exists() { + return Err("旧状态备份已存在，原文件已保留".into()); + } + fs::rename(&path, backup).map_err(|e| format!("归档旧状态失败: {e}"))?; + } + Ok(()) +} +pub(crate) fn read_at(root: &Path, key: &str) -> Result, String> { + let _lock = library_transaction(); + let mut db = sqlite::open(root)?; + migrate(&mut db, root)?; + read(&db, key) +} +pub(crate) fn update_at( + root: &Path, + key: &str, + update: impl FnOnce(&mut Option) -> Result<(), String>, +) -> Result { + let _lock = library_transaction(); + let mut db = sqlite::open(root)?; + migrate(&mut db, root)?; + let tx = db.transaction().map_err(sql)?; + let current = read(&tx, key)?; + let mut next = current.clone(); + update(&mut next)?; + if current == next { + return Ok(false); + } + write(&tx, root, key, next.as_ref())?; + tx.commit().map_err(sql)?; + Ok(true) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + fn setup() -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("readerx-state-{}", readerx_sync::new_id())); + let db = sqlite::open(&root).unwrap(); + for id in ["b1", "b2"] { + sqlite::save_detail( + &db, + &serde_json::from_value(json!({"id":id,"title":"书"})).unwrap(), + ) + .unwrap(); + } + fs::create_dir_all(root.join("state")).unwrap(); + root + } + fn set(root: &Path, key: &str, value: Value) { + update_at(root, key, |state| { + *state = Some(value); + Ok(()) + }) + .unwrap(); + } + #[test] + fn legacy_collections_and_progress_migrate_into_sqlite_and_do_not_replay() { + let root = setup(); + for key in &KEYS[..4] { + fs::write( + root.join(format!("state/{key}.json")), + json!([{"id":"old","unknown":42}]).to_string(), + ) + .unwrap(); + } + let path = root.join("state/readerx.shelf.json"); + fs::write( + &path, + json!({"b1":{"bookId":"b1","chapter":3,"charOffset":42}}).to_string(), + ) + .unwrap(); + // Archive failure occurs after the migration transaction has committed. + fs::create_dir(path.with_extension("json.migrated")).unwrap(); + assert!(read_at(&root, "readerx.shelf").is_err()); + let db = sqlite::open(&root).unwrap(); + let mut detail = sqlite::detail(&db, "b1").unwrap().unwrap(); + detail.progress.as_mut().unwrap()["chapter"] = json!(8); + sqlite::save_detail(&db, &detail).unwrap(); + drop(db); + fs::remove_dir(path.with_extension("json.migrated")).unwrap(); + assert_eq!( + read_at(&root, "readerx.shelf").unwrap().unwrap()["b1"]["chapter"], + 8 + ); + for key in &KEYS[..4] { + assert!(!root.join(format!("state/{key}.json")).exists()); + assert_eq!(read_at(&root, key).unwrap().unwrap()[0]["unknown"], 42); + } + let db = sqlite::open(&root).unwrap(); + assert_eq!( + sqlite::detail(&db, "b1") + .unwrap() + .unwrap() + .progress + .unwrap()["chapter"], + 8 + ); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn malformed_legacy_state_and_failed_sql_write_keep_previous_data() { + let root = setup(); + let path = root.join("state/readerx.groups.json"); + fs::write(&path, "broken").unwrap(); + assert!(read_at(&root, "readerx.groups").is_err()); + assert_eq!(fs::read_to_string(&path).unwrap(), "broken"); + fs::remove_file(path).unwrap(); + set(&root, "readerx.groups", json!([{"id":"old"}])); + let db = sqlite::open(&root).unwrap(); + db.execute_batch("CREATE TRIGGER fail_group BEFORE INSERT ON groups WHEN NEW.position=1 BEGIN SELECT RAISE(ABORT,'injected'); END").unwrap(); + assert!(update_at(&root, "readerx.groups", |v| { + *v = Some(json!([{"id":"new"},{"id":"bad"}])); + Ok(()) + }) + .is_err()); + assert_eq!( + read_at(&root, "readerx.groups").unwrap().unwrap(), + json!([{"id":"old"}]) + ); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn shelf_ensure_update_reset_metadata_save_and_delete_share_book_rows() { + let root = setup(); + set( + &root, + "readerx.shelf", + json!({"b1":{"chapter":5,"updatedAt":100}}), + ); + update_at(&root, "readerx.shelf", |state| { + crate::storage::patch_shelf( + state, + json!({"b1":{"chapter":0},"b2":{"chapter":2}}) + .as_object() + .unwrap(), + "ensure", + )?; + Ok(()) + }) + .unwrap(); + let shelf = read_at(&root, "readerx.shelf").unwrap().unwrap(); + assert_eq!(shelf["b1"]["chapter"], 5); + assert_eq!(shelf["b2"]["chapter"], 2); + let db = sqlite::open(&root).unwrap(); + let book = sqlite::get(&db, "b1").unwrap().unwrap(); + sqlite::put(&db, &book).unwrap(); + assert_eq!( + sqlite::detail(&db, "b1") + .unwrap() + .unwrap() + .progress + .unwrap()["chapter"], + 5 + ); + sqlite::delete(&db, "b2").unwrap(); + drop(db); + update_at(&root, "readerx.shelf", |state| { + crate::storage::patch_shelf(state, &Map::new(), "reset")?; + Ok(()) + }) + .unwrap(); + let shelf = read_at(&root, "readerx.shelf").unwrap().unwrap(); + assert_eq!(shelf["b1"]["chapter"], 0); + assert!(shelf.get("b2").is_none()); + assert!(!root.join("state/readerx.shelf.json").exists()); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn identity_migration_keeps_progress_and_rule_book_references() { + let root = setup(); + set(&root, "readerx.shelf", json!({"b1":{"chapter":7}})); + set( + &root, + "readerx.textReplacements", + json!([{"id":"r1","bookId":"b1"}]), + ); + super::super::rename_id_at(&root, "b1", "b-0123456789abcdef").unwrap(); + let shelf = read_at(&root, "readerx.shelf").unwrap().unwrap(); + assert_eq!(shelf["b-0123456789abcdef"]["chapter"], 7); + assert_eq!(shelf["b-0123456789abcdef"]["bookId"], "b-0123456789abcdef"); + assert!(shelf.get("b1").is_none()); + fs::remove_dir_all(root).unwrap(); + } + #[test] + fn database_v1_upgrade_and_snapshot_include_collections_and_progress() { + let root = setup(); + let db = sqlite::open(&root).unwrap(); + db.execute_batch("DROP TABLE groups; DROP TABLE rules; DROP TABLE collections; DROP TABLE migrated_states; PRAGMA user_version=1;").unwrap(); + drop(db); + set(&root, "readerx.groups", json!([{"id":"g1"}])); + set( + &root, + "readerx.chapterRules", + json!([{"id":"r1","pattern":"^第"}]), + ); + set(&root, "readerx.shelf", json!({"b1":{"chapter":4}})); + let db = sqlite::open(&root).unwrap(); + let snapshot = sqlite::snapshot(&db, &root).unwrap(); + set(&root, "readerx.shelf", json!({"b1":{"chapter":9}})); + assert_eq!( + snapshot.read_state("readerx.shelf").unwrap().unwrap()["b1"]["chapter"], + 4 + ); + assert_eq!( + snapshot.read_state("readerx.groups").unwrap().unwrap()[0]["id"], + "g1" + ); + assert_eq!( + snapshot + .read_state("readerx.chapterRules") + .unwrap() + .unwrap()[0]["pattern"], + "^第" + ); + drop(snapshot); + drop(db); + fs::remove_dir_all(root).unwrap(); + } +} diff --git a/src-tauri/src/data_transfer/archive.rs b/src-tauri/src/data_transfer/archive.rs index 9ce37b6..23b2833 100644 --- a/src-tauri/src/data_transfer/archive.rs +++ b/src-tauri/src/data_transfer/archive.rs @@ -336,6 +336,13 @@ pub(super) fn open_books( destination.sync_all().map_err(|e| e.to_string())?; } let books = crate::book_store::BackupDatabase::open(temporary)?; + for key in books.state_keys()? { + if scan.state_keys.contains(&key) { + return Err("备份同时包含两种数据状态布局".into()); + } + scan.state_keys.push(key); + } + scan.state_keys.sort(); scan.books = books.ids()?; if scan.books.len() as u64 != scan.manifest.books { return Err("备份书籍数量与清单不符".into()); @@ -354,6 +361,19 @@ pub(super) fn open_books( Ok(Some(books)) } +pub(super) fn read_state_entry( + zip: &mut ZipArchive, + books: Option<&crate::book_store::BackupDatabase>, + key: &str, +) -> Result, String> { + if let Some(books) = books { + if books.state_keys()?.iter().any(|k| k == key) { + return books.read_state(key); + } + } + read_entry_json(zip, &format!("{}/{key}.json", super::STATE_DIR)) +} + pub(super) fn read_book_entry( zip: &mut ZipArchive, books: Option<&crate::book_store::BackupDatabase>, diff --git a/src-tauri/src/data_transfer/export.rs b/src-tauri/src/data_transfer/export.rs index d7d9ab3..f9def9c 100644 --- a/src-tauri/src/data_transfer/export.rs +++ b/src-tauri/src/data_transfer/export.rs @@ -74,7 +74,7 @@ pub(super) fn export_to( books: book_count, images: images.len() as u64, sources: sources.len() as u64, - state_keys: state.len() as u64, + state_keys: (state.len() + books.state_keys()?.len()) as u64, }; let mut zip = archive::writer(destination); @@ -178,7 +178,7 @@ fn redacted_webdav(path: &Path) -> Result, String> { /// 状态文件：`.json`，key 必须合法（与 `readerx_state_get` 同一口径） fn is_state_file(name: &str) -> bool { name.strip_suffix(".json") - .map(crate::storage::valid_state_key) + .map(|key| crate::storage::valid_state_key(key) && !crate::book_store::state::handles(key)) .unwrap_or(false) } diff --git a/src-tauri/src/data_transfer/import.rs b/src-tauri/src/data_transfer/import.rs index 7587866..e0ae5af 100644 --- a/src-tauri/src/data_transfer/import.rs +++ b/src-tauri/src/data_transfer/import.rs @@ -14,7 +14,7 @@ use super::archive::{self, ArchiveScan}; use super::merge::{self, Remap}; use super::plan::{self, ArchiveSource, BookAction, BookPlan, LocalBook}; use super::{ImportMode, ImportSummary}; -use super::{BOOKS_DIR, IMAGES_DIR, SESSIONS_DIR, STATE_DIR}; +use super::{BOOKS_DIR, IMAGES_DIR, SESSIONS_DIR}; use crate::book_store; use crate::models::BookSource; use crate::storage; @@ -59,7 +59,15 @@ pub(super) fn apply( plan::read_archive_books(&mut zip, books.as_ref(), &scan, &archive_sources, mode)?; // 分组换算表：书与书源的 groupId 都要跟着走 - let remap = plan::plan_remap(app, mode, &local_books, &archive_books, &mut zip, &scan)?; + let remap = plan::plan_remap( + app, + mode, + &local_books, + &archive_books, + &mut zip, + &scan, + books.as_ref(), + )?; let plan: Vec = archive_books .iter() .map(|book| plan::plan_book(book, &local_books, mode)) @@ -78,7 +86,15 @@ pub(super) fn apply( write_images(app, &mut zip, &scan, &plan, mode, &mut summary, report)?; write_sources(&local_sources, &archive_sources, &remap, mode, &mut summary)?; write_sessions(app, &mut zip, &scan, report)?; - write_state(app, &mut zip, &scan, &remap, mode, &mut summary)?; + write_state( + app, + &mut zip, + books.as_ref(), + &scan, + &remap, + mode, + &mut summary, + )?; if mode == ImportMode::Replace { remove_extras( @@ -455,14 +471,14 @@ fn write_sessions( fn write_state( app: &AppHandle, zip: &mut ZipArchive, + books: Option<&crate::book_store::BackupDatabase>, scan: &ArchiveScan, remap: &Remap, mode: ImportMode, summary: &mut ImportSummary, ) -> Result<(), String> { for key in &scan.state_keys { - let Some(incoming) = archive::read_entry_json(zip, &format!("{STATE_DIR}/{key}.json"))? - else { + let Some(incoming) = archive::read_state_entry(zip, books, key)? else { continue; }; let changed = storage::update_state(app, key, |local| { diff --git a/src-tauri/src/data_transfer/plan.rs b/src-tauri/src/data_transfer/plan.rs index 5f648a4..0c3bbea 100644 --- a/src-tauri/src/data_transfer/plan.rs +++ b/src-tauri/src/data_transfer/plan.rs @@ -7,7 +7,7 @@ use super::archive::{self, ArchiveScan}; use super::merge::{self, Remap}; use super::ImportMode; -use super::{BOOKS_DIR, SOURCES_DIR, STATE_DIR}; +use super::{BOOKS_DIR, SOURCES_DIR}; use crate::book_store::{self, BookSyncMeta}; use crate::models::BookSource; use crate::storage; @@ -267,6 +267,7 @@ pub(super) fn plan_remap( archive_books: &[ArchiveBook], zip: &mut ZipArchive, scan: &ArchiveScan, + books: Option<&crate::book_store::BackupDatabase>, ) -> Result { let mut remap = Remap::default(); for book in archive_books { @@ -284,8 +285,8 @@ pub(super) fn plan_remap( let local_groups = state_array(app, merge::GROUPS_KEY); let local_source_groups = state_array(app, merge::SOURCE_GROUPS_KEY); - let archive_groups = read_archive_groups(zip, scan, merge::GROUPS_KEY)?; - let archive_source_groups = read_archive_groups(zip, scan, merge::SOURCE_GROUPS_KEY)?; + let archive_groups = read_archive_groups(zip, scan, books, merge::GROUPS_KEY)?; + let archive_source_groups = read_archive_groups(zip, scan, books, merge::SOURCE_GROUPS_KEY)?; let mut seq = 0u64; remap.groups = map_groups(&local_groups, &archive_groups, "grp", &mut seq); remap.source_groups = map_groups(&local_source_groups, &archive_source_groups, "sg", &mut seq); @@ -310,12 +311,13 @@ pub(super) fn plan_remap( fn read_archive_groups( zip: &mut ZipArchive, scan: &ArchiveScan, + books: Option<&crate::book_store::BackupDatabase>, key: &str, ) -> Result, String> { if !scan.state_keys.iter().any(|item| item == key) { return Ok(Vec::new()); } - let Some(value) = archive::read_entry_json(zip, &format!("{STATE_DIR}/{key}.json"))? else { + let Some(value) = archive::read_state_entry(zip, books, key)? else { return Ok(Vec::new()); }; Ok(value.as_array().cloned().unwrap_or_default()) diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index 8853306..18d4d00 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -130,6 +130,14 @@ fn seed_image(root: &Path, book_id: &str, hash: &str) { } fn seed_state(root: &Path, key: &str, value: &Value) { + if root.join("books.sqlite3").is_file() && crate::book_store::state::handles(key) { + crate::book_store::state::update_at(root, key, |state| { + *state = Some(value.clone()); + Ok(()) + }) + .unwrap(); + return; + } write_json(&root.join("state").join(format!("{key}.json")), value); } @@ -733,7 +741,7 @@ fn invalid_sqlite_archive_fails_before_local_writes_and_cleans_temporary_files() seed_device_a(&root); for (index,change) in [ None, - Some("PRAGMA user_version=2;"), + Some("PRAGMA user_version=999;"), Some("UPDATE chapters SET chapter='broken';"), Some("UPDATE books SET id='../outside-' || id;"), Some("INSERT OR REPLACE INTO annotations SELECT id,'{}' FROM books;"), diff --git a/src-tauri/src/storage.rs b/src-tauri/src/storage.rs index dbaa790..09f46d9 100644 --- a/src-tauri/src/storage.rs +++ b/src-tauri/src/storage.rs @@ -1,6 +1,6 @@ //! 持久化层（App 侧）：偏好 / 书架 / 分章规则等状态、听书缓存、以及应用数据根目录。 //! -//! - 状态按 key 存为 JSON 文件（`state/.json`）； +//! - 分组与规则存 SQLite，书架状态是 books 元信息中的进度视图；其他状态按 key 存 JSON； //! - **本地书籍**存入 `books.sqlite3`（见 `book_store.rs` 和 `book_store/sqlite.rs`）； //! - **书源与书源登录态**由 `readerx-source` crate 实现（见文件末尾的转发段）。 //! @@ -166,6 +166,9 @@ pub(crate) fn read_state( app: &AppHandle, key: &str, ) -> Result, String> { + if crate::book_store::state::handles(key) { + return crate::book_store::state::read_at(&data_root(app)?, key); + } let _transaction = state_transaction(); read_state_path(&state_path(app, key)?) } @@ -175,6 +178,10 @@ pub(crate) fn write_state( key: &str, value: &Value, ) -> Result<(), String> { + if crate::book_store::state::handles(key) { + crate::book_store::state::update_at(&data_root(app)?, key, |state| { *state = Some(value.clone()); Ok(()) })?; + return Ok(()); + } let _transaction = state_transaction(); write_state_path(&state_path(app, key)?, value) } @@ -185,6 +192,9 @@ pub(crate) fn update_state( key: &str, update: impl FnOnce(&mut Option) -> Result<(), String>, ) -> Result { + if crate::book_store::state::handles(key) { + return crate::book_store::state::update_at(&data_root(app)?, key, update); + } update_state_path(&state_path(app, key)?, update) } @@ -246,6 +256,10 @@ pub(crate) fn patch_shelf( } pub(crate) fn remove_state(app: &AppHandle, key: &str) -> Result<(), String> { + if crate::book_store::state::handles(key) { + crate::book_store::state::update_at(&data_root(app)?, key, |state| { *state = None; Ok(()) })?; + return Ok(()); + } let _transaction = state_transaction(); let path = state_path(app, key)?; if path.exists() { diff --git a/src-tauri/src/sync/book_ids.rs b/src-tauri/src/sync/book_ids.rs index 930fd35..3d2dfee 100644 --- a/src-tauri/src/sync/book_ids.rs +++ b/src-tauri/src/sync/book_ids.rs @@ -126,10 +126,18 @@ fn apply(root: &Path, plan: &BTreeMap) -> Result<(), String> { "readerx.readingTime", ] { let path = root.join("state").join(format!("{key}.json")); - if !path.is_file() { - continue; - } - let mut value = read(&path)?; + let in_db = root.join("books.sqlite3").is_file() && crate::book_store::state::handles(key); + let mut value = if in_db { + let Some(value) = crate::book_store::state::read_at(root, key)? else { + continue; + }; + value + } else { + if !path.is_file() { + continue; + } + read(&path)? + }; replace_refs(&mut value, plan); if key == "readerx.shelf" || key == "readerx.bookmarks" { replace_keys(&mut value, plan)?; @@ -140,7 +148,14 @@ fn apply(root: &Path, plan: &BTreeMap) -> Result<(), String> { replace_keys(books, plan)?; } } - atomic_write(&path, &value)?; + if in_db { + crate::book_store::state::update_at(root, key, |state| { + *state = Some(value); + Ok(()) + })?; + } else { + atomic_write(&path, &value)?; + } } Ok(()) } diff --git a/src-tauri/src/sync/data_ids.rs b/src-tauri/src/sync/data_ids.rs index 948f3b3..dd8c3af 100644 --- a/src-tauri/src/sync/data_ids.rs +++ b/src-tauri/src/sync/data_ids.rs @@ -13,6 +13,7 @@ pub fn migrate(root: &Path) -> Result<(), String> { } migrate_sources(root, &BTreeMap::new())?; let mut plan = FilePlan::default(); + let mut states = BTreeMap::new(); let mut groups = BTreeMap::new(); let mut source_groups = BTreeMap::new(); for (key, prefix, uid_of, remap) in [ @@ -30,10 +31,9 @@ pub fn migrate(root: &Path) -> Result<(), String> { ), ] { let relative = std::path::PathBuf::from(format!("state/{key}.json")); - if !root.join(&relative).exists() { + let Some(mut value) = read_collection(root, key)? else { continue; - } - let mut value = id_migration::read(&root.join(&relative))?; + }; for item in value.as_array_mut().into_iter().flatten() { let old = text(item, "id"); if old == "__hidden__" { @@ -43,7 +43,11 @@ pub fn migrate(root: &Path) -> Result<(), String> { remap.insert(old, new.clone()); item["id"] = Value::String(new); } - plan.writes.insert(relative, value); + if root.join("books.sqlite3").is_file() { + states.insert(key.to_string(), value); + } else { + plan.writes.insert(relative, value); + } } for path in id_migration::details(root)? { let mut value = id_migration::read(&path)?; @@ -62,10 +66,9 @@ pub fn migrate(root: &Path) -> Result<(), String> { ("readerx.chapterRules", "cr-"), ] { let relative = std::path::PathBuf::from(format!("state/{key}.json")); - if !root.join(&relative).exists() { + let Some(mut value) = read_collection(root, key)? else { continue; - } - let mut value = id_migration::read(&root.join(&relative))?; + }; for item in value.as_array_mut().into_iter().flatten() { if item.get("builtin").and_then(Value::as_bool) == Some(true) { continue; @@ -93,10 +96,14 @@ pub fn migrate(root: &Path) -> Result<(), String> { }; item["id"] = Value::String(fallback); } - plan.writes.insert(relative, value); + if root.join("books.sqlite3").is_file() { + states.insert(key.to_string(), value); + } else { + plan.writes.insert(relative, value); + } } // 相同旧身份收敛为同一个 ID：保留第一项，避免重复出现在界面。 - for value in plan.writes.values_mut() { + for value in plan.writes.values_mut().chain(states.values_mut()) { if let Some(items) = value.as_array_mut() { let mut seen = std::collections::BTreeSet::new(); items.retain(|item| seen.insert(text(item, "id"))); @@ -109,11 +116,30 @@ pub fn migrate(root: &Path) -> Result<(), String> { // available for deriving the same mapping on retry. crate::book_store::remap_metadata_at(root, "groupId", &groups)?; plan.commit(root, JOURNAL)?; + // References are committed first. Old collection IDs remain available after a failure. + for (key, value) in states { + crate::book_store::state::update_at(root, &key, |state| { + *state = Some(value); + Ok(()) + })?; + } if changed > 0 { log::info!("同步数据 ID 迁移完成 files={changed}"); } Ok(()) } +fn read_collection(root: &Path, key: &str) -> Result, String> { + if root.join("books.sqlite3").is_file() { + crate::book_store::state::read_at(root, key) + } else { + let path = root.join("state").join(format!("{key}.json")); + if path.is_file() { + id_migration::read(&path).map(Some) + } else { + Ok(None) + } + } +} /// Journal aliases before source files can be renamed by the source crate. /// A crash between source migration and DB updates must not lose the old ID mapping. pub(crate) fn migrate_sources( diff --git a/src-tauri/tests/support/book_json.rs b/src-tauri/tests/support/book_json.rs index e6995ef..6e5a5e6 100644 --- a/src-tauri/tests/support/book_json.rs +++ b/src-tauri/tests/support/book_json.rs @@ -43,7 +43,107 @@ fn connection(path: &Path) -> Option<(Connection, String, String)> { None } } +fn state_location(path: &Path) -> Option<(Connection, String)> { + let state = path.parent()?; + if state.file_name()?.to_str()? != "state" || path.is_file() { + return None; + } + let key = path.file_stem()?.to_str()?; + if !matches!( + key, + "readerx.groups" + | "readerx.sourceGroups" + | "readerx.chapterRules" + | "readerx.textReplacements" + | "readerx.shelf" + ) { + return None; + } + let db = state.parent()?.join("books.sqlite3"); + if !db.is_file() { + return None; + } + let db = Connection::open(db).unwrap(); + db.execute_batch("PRAGMA foreign_keys=ON").unwrap(); + Some((db, key.into())) +} +fn state_read(db: &Connection, key: &str) -> Value { + if key == "readerx.shelf" { + let mut out = serde_json::Map::new(); + let mut q = db.prepare("SELECT id,detail FROM books").unwrap(); + for row in q + .query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))) + .unwrap() + { + let (id, raw) = row.unwrap(); + let detail: Value = serde_json::from_str(&raw).unwrap(); + if let Some(progress) = detail.get("progress") { + out.insert(id, progress.clone()); + } + } + return Value::Object(out); + } + let table = if key.ends_with("groups") || key.ends_with("sourceGroups") { + "groups" + } else { + "rules" + }; + let mut q = db + .prepare(&format!( + "SELECT record FROM {table} WHERE key=?1 ORDER BY position" + )) + .unwrap(); + Value::Array( + q.query_map([key], |r| r.get::<_, String>(0)) + .unwrap() + .map(|r| serde_json::from_str(&r.unwrap()).unwrap()) + .collect(), + ) +} +fn state_write(db: &mut Connection, key: &str, value: &Value) { + let tx = db.transaction().unwrap(); + if key == "readerx.shelf" { + let details: Vec<(String, String)> = { + let mut q = tx.prepare("SELECT id,detail FROM books").unwrap(); + let rows = q.query_map([], |r| Ok((r.get(0)?, r.get(1)?))).unwrap(); + rows.map(Result::unwrap).collect() + }; + for (id, raw) in details { + let mut detail: Value = serde_json::from_str(&raw).unwrap(); + detail.as_object_mut().unwrap().remove("progress"); + if let Some(progress) = value.get(&id) { + detail["progress"] = progress.clone(); + } + tx.execute( + "UPDATE books SET detail=?1 WHERE id=?2", + params![detail.to_string(), id], + ) + .unwrap(); + } + } else { + let table = if key.ends_with("groups") || key.ends_with("sourceGroups") { + "groups" + } else { + "rules" + }; + tx.execute("DELETE FROM collections WHERE key=?1", [key]) + .unwrap(); + tx.execute("INSERT INTO collections(key) VALUES(?1)", [key]) + .unwrap(); + for (p, item) in value.as_array().unwrap().iter().enumerate() { + tx.execute( + &format!("INSERT INTO {table}(key,position,record) VALUES(?1,?2,?3)"), + params![key, p as i64, item.to_string()], + ) + .unwrap(); + } + } + tx.commit().unwrap(); +} pub fn read(path: &Path) -> Value { + if let Some((db, key)) = state_location(path) { + return state_read(&db, &key); + } let Some((db, id, file)) = connection(path) else { return serde_json::from_slice(&std::fs::read(path).unwrap()).unwrap(); }; @@ -79,6 +179,10 @@ pub fn read(path: &Path) -> Value { json!({"schemaVersion":1,(table):text.map(|s|serde_json::from_str::(&s).unwrap()).unwrap_or(json!([]))}) } pub fn write(path: &Path, value: &Value) { + if let Some((mut db, key)) = state_location(path) { + state_write(&mut db, &key, value); + return; + } let Some((mut db, id, file)) = connection(path) else { std::fs::create_dir_all(path.parent().unwrap()).unwrap(); std::fs::write(path, serde_json::to_vec_pretty(value).unwrap()).unwrap(); diff --git a/src-tauri/tests/sync_bridge.rs b/src-tauri/tests/sync_bridge.rs index 78fe811..bcd8ce5 100644 --- a/src-tauri/tests/sync_bridge.rs +++ b/src-tauri/tests/sync_bridge.rs @@ -695,9 +695,10 @@ fn conflict_resolution_reports_materialization_failures_and_can_retry() { "activated": true, "hiddenGroupMigrated": true, "materializedOps": if pending { 0 } else { count }, })); - // 用目录阻挡状态文件写入，真实触发持久化失败。 + // 只阻断分组的 SQLite 写入，保留前一阶段的书源分组刷新。 let path = app_data.join("state/readerx.groups.json"); - std::fs::create_dir_all(&path).unwrap(); + let db = rusqlite::Connection::open(app_data.join("books.sqlite3")).unwrap(); + db.execute_batch("CREATE TRIGGER fail_groups BEFORE INSERT ON groups WHEN NEW.key='readerx.groups' BEGIN SELECT RAISE(ABORT,'injected'); END").unwrap(); let applied = Arc::new(Mutex::new(Vec::::new())); let captured = applied.clone(); let listener = handle.listen("readerx-sync-applied", move |event| { @@ -716,7 +717,7 @@ fn conflict_resolution_reports_materialization_failures_and_can_retry() { assert!(applied.lock().unwrap().iter().any(|event| event["sourceGroups"] == true), "指定实体落地失败不能吞掉此前已写回的书源分组刷新事件"); } - std::fs::remove_dir(&path).unwrap(); + db.execute_batch("DROP TRIGGER fail_groups").unwrap(); service.resolve_conflict(&id, "remote").expect("恢复写盘后同一裁决可以重试"); let groups = read_json(&path); assert_eq!(groups[0]["name"], "对端分组"); From 5f9d2f9036fe5fe140ffbf03ea6b310f51ec2a18 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:25:26 +0800 Subject: [PATCH 12/17] =?UTF-8?q?fix:=20=E8=A1=A5=E9=BD=90=E5=A4=87?= =?UTF-8?q?=E4=BB=BD=E7=8A=B6=E6=80=81=E9=80=82=E9=85=8D=E4=B8=8E=E8=BF=9B?= =?UTF-8?q?=E5=BA=A6=E5=9B=9E=E5=BD=92=E9=AA=8C=E8=AF=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../readerx-data/references/contracts.md | 2 +- src-tauri/src/book_store.rs | 2 +- src-tauri/src/book_store/state.rs | 6 ++- src-tauri/src/data_transfer/archive.rs | 1 + src-tauri/src/data_transfer/merge.rs | 4 +- src-tauri/src/data_transfer/round_trip.rs | 40 +++++++++++++++++++ 6 files changed, 50 insertions(+), 5 deletions(-) diff --git a/.agents/skills/readerx-data/references/contracts.md b/.agents/skills/readerx-data/references/contracts.md index 56986f9..157c260 100644 --- a/.agents/skills/readerx-data/references/contracts.md +++ b/.agents/skills/readerx-data/references/contracts.md @@ -7,7 +7,7 @@ - book_store.rs / book_store/sqlite.rs / book_store/backup.rs 将书籍数据存入 data_root 下的 books.sqlite3（books、chapters、bookmarks、annotations、groups、rules；进度直接在 books.detail.progress）；正文按章增量更新，章节头、指纹和图片引用同事务提交。TTS、书源、其他状态 JSON 和图片字节不入库。数据库 user_version 与旧 JSON schemaVersion 分别维护，新增字段核对 models.rs、BookDetail 与前端 booksTypes.ts。见 [书籍存储](../../../../docs/book-storage.md)。 - 注释走 readerx_annotations_get/put，与正文/书签独立存表，旧 JSON 信封 schemaVersion=1，记录按段落聚合 notes；只在读取及写入成功后发布内存变化，读失败不得作为空数组覆盖。锚点不含 bookId，身份迁移由外键级联更新归属；删除/恢复失效缓存并阻止在飞读取回填。备份包含注释表并在同段合并 note id，覆盖旧备份的缺失文件清空该书注释；导出/导入/删除前等待注释写队列。同步按 note 独立实体，保存只发布差集且与落库串行；同段新增聚合、正文并发进冲突、删书级联，启动只补缺失实体并恢复引擎已有值。备份导入显式发布改写/删除/恢复，落库通过 AppliedChanges.annotations 按书刷新。契约见 [阅读注释](../../../../docs/annotations.md)，检查为 book_store、data_transfer、readerx-sync 与 sync_bridge 测试。 - book_store 的读写入口在进程级书库事务锁内完成迁移、读取、修改、落盘及指纹更新；持锁时不调用同步引擎。单章补丁以 chapter.cid 定位，index 仅兼容旧载荷；先验证整批 cid 存在且唯一，再只改正文，保留当前目录标题/地址。 -- 分组 / 书源分组 / 文本替换 / 分章规则经 book_store/state.rs 存库，书架 API 是 books.detail.progress 的视图；整书及元信息补丁不得清除进度。这些状态与书庫共用事务锁，旧 JSON 和回执同事务迁入，成功才归档；数据库 user_version=2，恢复兼容 v1。其他普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 +- 分组 / 书源分组 / 文本替换 / 分章规则经 book_store/state.rs 存库，书架 API 是 books.detail.progress 的视图；整书及元信息补丁不得清除进度。这些状态与书库共用事务锁，旧 JSON 和回执同事务迁入，成功才归档；数据库 user_version=2，恢复兼容 v1。其他普通状态写入使用同目录临时文件，先 sync_all 再重命名替换，Unix（含 Android）随后同步父目录；提交前失败保留旧文件并清理临时文件，目录同步失败返回“已替换”错误（不可假定已回滚）。Windows 仅同步文件并替换，不承诺断电后的目录持久性。普通状态读写/删除与 update_state 共用事务锁；读改写必须使用 update_state 闭包，闭包内不重入状态存储或同步引擎。阅读进度经 readerx_shelf_patch 按书提交增量，ensure 仅补缺失项，reset 在后端重置当前全部记录；不要恢复成前端整份旧书架快照覆盖。 - 旧整书 / 目录 JSON 逐本在事务内导入并保存迁移回执，成功才改名 .migrated 留底；失败保留读回退和重试，不影响其他书。归档中断不能重复导入旧数据覆盖新编辑。数据库身份改动和状态 / TTS 目录迁移保持可重跑。 - 书库与同步共用稳定 ID；已有规范 ID 沿用，不能在每次编辑后按新名字/地址重新计算。身份算法以 sync/identity.rs 为准：在线书首次按书源地址与详情页地址派生，导入书按归一文件名与字节数派生。分组与书源首次按名称/地址派生后编辑沿用 ID；规则仍按内容派生。前端 dataIds.ts 与 Rust 种子、大小写和归一化口径保持一致。 - 迁移顺序与启动入口检查 src-tauri/src/lib.rs：书籍迁移在 UI/同步访问前完成，data_ids 再处理分组、书源、规则及引用。迁移待办先落盘，每步可重跑；目录或身份冲突保留原数据并返回失败，不能覆盖目标。 diff --git a/src-tauri/src/book_store.rs b/src-tauri/src/book_store.rs index d429227..f8ecbf4 100644 --- a/src-tauri/src/book_store.rs +++ b/src-tauri/src/book_store.rs @@ -46,7 +46,7 @@ fn schema_version() -> u32 { /// `bookdetail.json`：书籍元信息（不含正文）。 /// -/// 与 [`LocalBook`] 字段同构（只少 `chapters`），两边必须保持一致 —— +/// 对外书籍字段与 [`LocalBook`] 同构（少 `chapters`），另存内部 `progress` —— /// 单元测试 `detail_round_trip_keeps_every_field` 会在加了字段却忘记同步时失败。 #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] diff --git a/src-tauri/src/book_store/state.rs b/src-tauri/src/book_store/state.rs index e03cd23..4b08e64 100644 --- a/src-tauri/src/book_store/state.rs +++ b/src-tauri/src/book_store/state.rs @@ -340,7 +340,7 @@ mod tests { fs::remove_dir_all(root).unwrap(); } #[test] - fn identity_migration_keeps_progress_and_rule_book_references() { + fn identity_migration_keeps_progress() { let root = setup(); set(&root, "readerx.shelf", json!({"b1":{"chapter":7}})); set( @@ -360,6 +360,10 @@ mod tests { let root = setup(); let db = sqlite::open(&root).unwrap(); db.execute_batch("DROP TABLE groups; DROP TABLE rules; DROP TABLE collections; DROP TABLE migrated_states; PRAGMA user_version=1;").unwrap(); + let old_snapshot = sqlite::snapshot(&db, &root).unwrap(); + assert_eq!(old_snapshot.ids().unwrap(), vec!["b1", "b2"]); + assert!(old_snapshot.state_keys().unwrap().is_empty()); + drop(old_snapshot); drop(db); set(&root, "readerx.groups", json!([{"id":"g1"}])); set( diff --git a/src-tauri/src/data_transfer/archive.rs b/src-tauri/src/data_transfer/archive.rs index 23b2833..7480a2a 100644 --- a/src-tauri/src/data_transfer/archive.rs +++ b/src-tauri/src/data_transfer/archive.rs @@ -11,6 +11,7 @@ //! 也能在单元测试里直接用真实文件跑。 use super::{is_safe_entry, Manifest, MANIFEST_NAME}; +use serde_json::Value; use std::fs::{self, File}; use std::io::{Read, Write}; use std::path::Path; diff --git a/src-tauri/src/data_transfer/merge.rs b/src-tauri/src/data_transfer/merge.rs index 90bdeef..7fd71c2 100644 --- a/src-tauri/src/data_transfer/merge.rs +++ b/src-tauri/src/data_transfer/merge.rs @@ -1,10 +1,10 @@ //! 合并规则（导入的「合并」语义）：输入两份 JSON，输出要落盘的那一份。 //! -//! 为什么状态文件不能整份覆盖：`readerx.shelf` 一个文件里装着全库的阅读进度， +//! 状态 API 同时适配 SQLite 与旧 JSON：`readerx.shelf` 是全库阅读进度的逻辑视图， //! `readerx.groups` / `readerx.textReplacements` 装着全部分组与替换规则 —— 整份覆盖 //! 会把本机那份数据抹掉。因此按**身份**取并集： //! -//! | 状态文件 | 身份 | 并发 / 重复时 | +//! | 状态 key | 身份 | 并发 / 重复时 | //! | --- | --- | --- | //! | `readerx.shelf` | 书籍 id | `updatedAt` 较新的一方胜出（与同步的进度口径一致） | //! | `readerx.groups` / `readerx.sourceGroups` | 分组名 | 本机已有同名分组则不动（书 / 书源的归属按名字对上） | diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index 18d4d00..476be01 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -619,6 +619,17 @@ fn convert_to_v1(root: &Path, path: &Path, omitted: Option<&str>) { .unwrap(); std::io::copy(&mut entry, &mut rewritten).unwrap(); } + // Materialize SQLite collections/progress as the real old JSON archive layout. + for key in books.state_keys().unwrap() { + if let Some(value) = books.read_state(&key).unwrap() { + rewritten + .start_file(format!("state/{key}.json"), archive::text_options(None)) + .unwrap(); + rewritten + .write_all(&serde_json::to_vec(&value).unwrap()) + .unwrap(); + } + } for id in &scan.books { for file in [ "bookdetail.json", @@ -654,6 +665,12 @@ fn sqlite_export_keeps_sources_json_and_v1_import_restores_books() { let manifest = archive::read_manifest(&mut zip).unwrap(); assert_eq!(manifest.format, "readerx-backup/2"); assert!(zip.by_name("books.sqlite3").is_ok()); + for key in crate::book_store::state::KEYS { + assert!( + zip.by_name(&format!("state/{key}.json")).is_err(), + "{key} must be inside SQLite" + ); + } assert!(!zip.file_names().any(|name| name.starts_with("books/") || name.ends_with("-wal") || name.ends_with("-shm"))); @@ -676,6 +693,10 @@ fn sqlite_export_keeps_sources_json_and_v1_import_restores_books() { .query_row("SELECT count(*) FROM migrated_books", [], |r| r.get(0)) .unwrap(); assert_eq!(receipts, 0); + let receipts: i64 = db + .query_row("SELECT count(*) FROM migrated_states", [], |r| r.get(0)) + .unwrap(); + assert_eq!(receipts, 0); drop(db); convert_to_v1(&root, &backup, None); let mut zip = archive::open_zip_from(fs::File::open(&backup).unwrap()).unwrap(); @@ -743,6 +764,9 @@ fn invalid_sqlite_archive_fails_before_local_writes_and_cleans_temporary_files() None, Some("PRAGMA user_version=999;"), Some("UPDATE chapters SET chapter='broken';"), + Some("UPDATE groups SET record='broken';"), + Some("UPDATE rules SET record='[]';"), + Some("UPDATE books SET detail=json_set(detail,'$.progress',1);"), Some("UPDATE books SET id='../outside-' || id;"), Some("INSERT OR REPLACE INTO annotations SELECT id,'{}' FROM books;"), Some("PRAGMA foreign_keys=OFF; INSERT INTO bookmarks VALUES('missing','[]');"), @@ -806,3 +830,19 @@ fn annotations_survive_archive_merge_replace_and_old_backup() { .unwrap() .is_empty()); } + +#[test] +fn metadata_patch_keeps_progress_in_the_same_book_row() { + let _guard = SERIAL.lock().unwrap_or_else(|e| e.into_inner()); + let (app, root) = setup(); + seed_device_a(&root); + crate::storage::read_state(&app, "readerx.shelf").unwrap(); + let patch = serde_json::from_value(json!({"title":"新标题","groupId":"new-group"})).unwrap(); + crate::book_store::patch_book_meta(&app, "local-a", &patch).unwrap(); + let detail = read_json(&root.join("books/local-a/bookdetail.json")); + assert_eq!(detail["title"], "新标题"); + assert_eq!(detail["groupId"], "new-group"); + assert_eq!(detail["progress"]["chapter"], 1); + assert_eq!(detail["progress"]["charOffset"], 10); + assert!(!root.join("state/readerx.shelf.json").exists()); +} From 16e878f764956f1f8f6be0364ef0acdc15f7e044 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:29:38 +0800 Subject: [PATCH 13/17] =?UTF-8?q?fix:=20=E6=8C=89=E5=AD=97=E6=AE=B5?= =?UTF-8?q?=E8=AF=BB=E5=86=99=E4=B9=A6=E6=9E=B6=E8=BF=9B=E5=BA=A6=E5=B9=B6?= =?UTF-8?q?=E5=AE=8C=E5=96=84=E5=90=8C=E6=AD=A5=E5=9B=9E=E5=BD=92?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/book-storage.md | 2 +- src-tauri/src/book_store/state.rs | 67 ++++++++++++++++------- src-tauri/src/data_transfer/round_trip.rs | 7 ++- src-tauri/tests/sync_bridge.rs | 3 +- 4 files changed, 55 insertions(+), 24 deletions(-) diff --git a/docs/book-storage.md b/docs/book-storage.md index 0338a71..f4d0a83 100644 --- a/docs/book-storage.md +++ b/docs/book-storage.md @@ -42,7 +42,7 @@ command 访问，数据形状和同步协议不变。 分组、书源分组、文本替换、分章规则和旧书架状态首次通过状态入口或备份访问时迁入数据库， 每个集合与迁移回执在同一事务提交，成功后将原 JSON 改名留底；失败保留原文件并返回错误。 -书架接口仍返回按书籍 ID 索引的进度视图，实际只更新有变化的 `books.detail.progress`。 +书架接口仍返回按书籍 ID 索引的进度视图，实际只查询进度字段，通过 JSON SQL 函数更新有变化的 `books.detail.progress`，不在 Rust 中解析或重序列化封面和其他元信息。 整书保存和元信息补丁保留进度；删除书籍同时删除进度。初始化只补缺项，重置在后端更新已有记录。 `collections` 区分空集合与不存在的集合；`migrated_states` 防止归档中断后重复导入覆盖新编辑。 diff --git a/src-tauri/src/book_store/state.rs b/src-tauri/src/book_store/state.rs index 4b08e64..239433a 100644 --- a/src-tauri/src/book_store/state.rs +++ b/src-tauri/src/book_store/state.rs @@ -1,6 +1,6 @@ //! SQLite-backed collections and the legacy shelf API's view of books.detail.progress. //! All entry points share the library lock; callbacks never re-enter storage or sync. -use super::{library_transaction, sqlite, BookDetail}; +use super::{library_transaction, sqlite}; use rusqlite::{params, Connection}; use serde_json::{Map, Value}; use std::{fs, path::Path}; @@ -29,15 +29,17 @@ pub(super) fn read(db: &Connection, key: &str) -> Result, String> if key == "readerx.shelf" { let mut out = Map::new(); let mut query = db - .prepare("SELECT id,detail FROM books ORDER BY id") + .prepare("SELECT id,json_extract(detail,'$.progress') FROM books ORDER BY id") .map_err(sql)?; let rows = query - .query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))) + .query_map([], |r| { + Ok((r.get::<_, String>(0)?, r.get::<_, Option>(1)?)) + }) .map_err(sql)?; for row in rows { let (id, raw) = row.map_err(sql)?; - let detail: BookDetail = serde_json::from_str(&raw).map_err(|e| e.to_string())?; - if let Some(progress) = detail.progress { + if let Some(raw) = raw { + let progress: Value = serde_json::from_str(&raw).map_err(|e| e.to_string())?; validate_progress(&progress)?; out.insert(id, progress); } @@ -92,25 +94,36 @@ fn write(db: &Connection, root: &Path, key: &str, value: Option<&Value>) -> Resu for (id, progress) in entries { sqlite::valid_id(id)?; validate_progress(progress)?; - if sqlite::detail(db, id)?.is_none() { + let exists: bool = db + .query_row( + "SELECT EXISTS(SELECT 1 FROM books WHERE id=?1)", + [id], + |r| r.get(0), + ) + .map_err(sql)?; + if !exists { sqlite::reject_pending_migration(root, id)?; } } - let ids = { - let mut q = db.prepare("SELECT id FROM books").map_err(sql)?; - let rows = q.query_map([], |r| r.get::<_, String>(0)).map_err(sql)?; - rows.collect::, _>>().map_err(sql)? - }; - for id in ids { - let mut detail = sqlite::detail(db, &id)?.ok_or("书籍元信息缺失")?; - let progress = entries.get(&id).map(|entry| { - let mut entry = entry.clone(); - entry["bookId"] = Value::String(id.clone()); - entry - }); - if detail.progress != progress { - detail.progress = progress; - sqlite::save_detail(db, &detail)?; + // Keep covers and unrelated metadata out of the Rust read/serialize path. + let current = read(db, key)?.ok_or("书架状态缺失")?; + let current = current.as_object().ok_or("书架状态格式错误")?; + for id in current.keys().filter(|id| !entries.contains_key(*id)) { + db.execute( + "UPDATE books SET detail=json_remove(detail,'$.progress') WHERE id=?1", + [id], + ) + .map_err(sql)?; + } + for (id, entry) in entries { + let mut progress = entry.clone(); + progress["bookId"] = Value::String(id.clone()); + if current.get(id) != Some(&progress) { + db.execute( + "UPDATE books SET detail=json_set(detail,'$.progress',json(?1)) WHERE id=?2", + params![progress.to_string(), id], + ) + .map_err(sql)?; } } return Ok(()); @@ -316,6 +329,18 @@ mod tests { assert_eq!(shelf["b1"]["chapter"], 5); assert_eq!(shelf["b2"]["chapter"], 2); let db = sqlite::open(&root).unwrap(); + db.execute_batch("UPDATE books SET detail=json_set(detail,'$.extra',42) WHERE id='b1'; CREATE TRIGGER keep_b2 BEFORE UPDATE ON books WHEN OLD.id='b2' BEGIN SELECT RAISE(ABORT,'unchanged row was written'); END;").unwrap(); + let mut next = shelf.clone(); + next["b1"]["charOffset"] = json!(9); + set(&root, "readerx.shelf", next); + let extra: i64 = db + .query_row( + "SELECT json_extract(detail,'$.extra') FROM books WHERE id='b1'", + [], + |r| r.get(0), + ) + .unwrap(); + assert_eq!(extra, 42); let book = sqlite::get(&db, "b1").unwrap().unwrap(); sqlite::put(&db, &book).unwrap(); assert_eq!( diff --git a/src-tauri/src/data_transfer/round_trip.rs b/src-tauri/src/data_transfer/round_trip.rs index 476be01..a661552 100644 --- a/src-tauri/src/data_transfer/round_trip.rs +++ b/src-tauri/src/data_transfer/round_trip.rs @@ -644,8 +644,13 @@ fn convert_to_v1(root: &Path, path: &Path, omitted: Option<&str>) { rewritten .start_file(name, archive::text_options(None)) .unwrap(); + let mut value: Value = + serde_json::from_slice(&books.read(id, file).unwrap().unwrap()).unwrap(); + if file == "bookdetail.json" { + value.as_object_mut().unwrap().remove("progress"); + } rewritten - .write_all(&books.read(id, file).unwrap().unwrap()) + .write_all(&serde_json::to_vec(&value).unwrap()) .unwrap(); } } diff --git a/src-tauri/tests/sync_bridge.rs b/src-tauri/tests/sync_bridge.rs index bcd8ce5..23ffc77 100644 --- a/src-tauri/tests/sync_bridge.rs +++ b/src-tauri/tests/sync_bridge.rs @@ -697,6 +697,7 @@ fn conflict_resolution_reports_materialization_failures_and_can_retry() { })); // 只阻断分组的 SQLite 写入，保留前一阶段的书源分组刷新。 let path = app_data.join("state/readerx.groups.json"); + readerx_lib::sync::book_ids::migrate(&handle).unwrap(); let db = rusqlite::Connection::open(app_data.join("books.sqlite3")).unwrap(); db.execute_batch("CREATE TRIGGER fail_groups BEFORE INSERT ON groups WHEN NEW.key='readerx.groups' BEGIN SELECT RAISE(ABORT,'injected'); END").unwrap(); let applied = Arc::new(Mutex::new(Vec::::new())); @@ -1812,7 +1813,7 @@ fn hidden_group_membership_survives_sync_and_can_be_cleared() { assert_eq!(ids.len(), 1); let path = app_data.join("books").join(&ids[0]).join("bookdetail.json"); assert_eq!(read_json(&path)["groupId"], "__hidden__"); - assert!(!app_data.join("state/readerx.groups.json").exists(), "不能额外创建普通分组"); + assert_eq!(read_json(&app_data.join("state/readerx.groups.json")), json!([]), "不能额外创建普通分组"); lock_engine(&peer.engine).set_field(&uid, "group", json!(null)).unwrap(); sync_once(&fresh, &peer.engine); bridge::materialize(&handle, &fresh, 0, &mut index).unwrap(); From c95250df99624fbf8023d8c5092fd3021e5256f1 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:31:28 +0800 Subject: [PATCH 14/17] =?UTF-8?q?fix:=20=E6=A0=A1=E9=AA=8C=E8=BF=9B?= =?UTF-8?q?=E5=BA=A6=E6=8F=90=E5=8F=96=E5=90=8E=E7=9A=84=20JSON=20?= =?UTF-8?q?=E7=B1=BB=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src-tauri/src/book_store/state.rs | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/src-tauri/src/book_store/state.rs b/src-tauri/src/book_store/state.rs index 239433a..22b7d7a 100644 --- a/src-tauri/src/book_store/state.rs +++ b/src-tauri/src/book_store/state.rs @@ -29,7 +29,9 @@ pub(super) fn read(db: &Connection, key: &str) -> Result, String> if key == "readerx.shelf" { let mut out = Map::new(); let mut query = db - .prepare("SELECT id,json_extract(detail,'$.progress') FROM books ORDER BY id") + .prepare( + "SELECT id,json_quote(json_extract(detail,'$.progress')) FROM books ORDER BY id", + ) .map_err(sql)?; let rows = query .query_map([], |r| { @@ -40,6 +42,9 @@ pub(super) fn read(db: &Connection, key: &str) -> Result, String> let (id, raw) = row.map_err(sql)?; if let Some(raw) = raw { let progress: Value = serde_json::from_str(&raw).map_err(|e| e.to_string())?; + if progress.is_null() { + continue; + } validate_progress(&progress)?; out.insert(id, progress); } @@ -294,6 +299,14 @@ mod tests { fs::remove_file(path).unwrap(); set(&root, "readerx.groups", json!([{"id":"old"}])); let db = sqlite::open(&root).unwrap(); + db.execute( + "UPDATE books SET detail=json_set(detail,'$.progress',?1) WHERE id='b1'", + ["{\"chapter\":8}"], + ) + .unwrap(); + assert!(read_at(&root, "readerx.shelf").is_err()); + db.execute_batch("UPDATE books SET detail=json_remove(detail,'$.progress') WHERE id='b1'") + .unwrap(); db.execute_batch("CREATE TRIGGER fail_group BEFORE INSERT ON groups WHEN NEW.position=1 BEGIN SELECT RAISE(ABORT,'injected'); END").unwrap(); assert!(update_at(&root, "readerx.groups", |v| { *v = Some(json!([{"id":"new"},{"id":"bad"}])); From f0c51e6b9e2127f6ba931b3a3e9205520779c32a Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 23:35:38 +0800 Subject: [PATCH 15/17] =?UTF-8?q?feat:=20=E5=A2=9E=E5=8A=A0=E6=89=8B?= =?UTF-8?q?=E5=8A=A8=E7=AD=BE=E5=90=8D=20arm64=20APK=20=E6=9E=84=E5=BB=BA?= =?UTF-8?q?=E4=B8=8E=20PR=20=E5=B9=B3=E5=8F=B0=E6=A3=80=E6=9F=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agents/skills/readerx-pr/SKILL.md | 1 + .github/actions/setup-android/action.yml | 46 +++++++++++++++ .github/workflows/build-android-signed.yml | 66 +++++++++++++++++++++ .github/workflows/ci.yml | 57 ++++++++++++++++++ .github/workflows/test-data.yml | 36 +++++++----- README.md | 2 + docs/ci.md | 36 ++++++++++++ scripts/collect-artifacts.mjs | 20 ++++--- scripts/collect-artifacts.test.mjs | 67 ++++++++++++++++++++++ 9 files changed, 310 insertions(+), 21 deletions(-) create mode 100644 .github/actions/setup-android/action.yml create mode 100644 .github/workflows/build-android-signed.yml create mode 100644 .github/workflows/ci.yml create mode 100644 docs/ci.md create mode 100644 scripts/collect-artifacts.test.mjs diff --git a/.agents/skills/readerx-pr/SKILL.md b/.agents/skills/readerx-pr/SKILL.md index cad9076..4c94155 100644 --- a/.agents/skills/readerx-pr/SKILL.md +++ b/.agents/skills/readerx-pr/SKILL.md @@ -19,6 +19,7 @@ description: 按 pr: 请求把 issue 或需求做成 Pull Request：读取需求 - 按 worktree 技能创建 `.worktree/` 与分支 `codex/`，名称取任务关键词；已存在同名目录或分支时先确认，不覆盖已有工作。 - 默认从最新 main 起分支；用户指定其他目标分支时尊重该选择，并在 PR 中使用对应 base。 - 在 worktree 内完成改动，运行适用检查：`pnpm exec tsc --noEmit`、`pnpm run i18n:check`、`pnpm build`；涉及 Rust 时在 src-tauri 运行对应 crate 的检查。 +- PR 自动检查、手动签名 arm64 APK、签名 Secrets 与产物验收见 [CI 与构建](../../../docs/ci.md)；CI 文件变动检查 Actions 语法，产物收集脚本变动运行 `node --test scripts/collect-artifacts.test.mjs`。 - 同步更新受影响的文档、中英文文案与项目 skill；只提交本次任务文件，提交信息用中文 `feat/fix/docs: 内容`，改动较大时按完整改动分批提交。 ## 推送与创建 PR diff --git a/.github/actions/setup-android/action.yml b/.github/actions/setup-android/action.yml new file mode 100644 index 0000000..41351b1 --- /dev/null +++ b/.github/actions/setup-android/action.yml @@ -0,0 +1,46 @@ +name: Setup Android +description: 准备 JDK、Gradle 缓存与 Android SDK / NDK +runs: + using: composite + steps: + - name: Setup JDK 17 + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: '17' + + # Gradle 依赖下载占 Android 构建的大头，缓存下来能省几分钟； + # key 带上 gen/android 下的构建脚本 hash，脚本变了就重新拉依赖。 + - name: Cache Gradle + uses: actions/cache@v5 + with: + path: | + ~/.gradle/caches + ~/.gradle/wrapper + key: gradle-${{ runner.os }}-${{ hashFiles('src-tauri/gen/android/**/*.gradle', 'src-tauri/gen/android/**/*.gradle.kts', 'src-tauri/gen/android/gradle/wrapper/gradle-wrapper.properties') }} + restore-keys: gradle-${{ runner.os }}- + + # GitHub 托管的 runner 预装了 Android SDK/NDK；选最高版本的 NDK 并导出， + # tauri CLI 的 ensure_ndk 会优先读取 NDK_HOME（CI 下不会自行下载 NDK）。 + - name: Resolve Android SDK / NDK + shell: bash + run: | + ANDROID_HOME="${ANDROID_HOME:-/usr/local/lib/android/sdk}" + echo "ANDROID_HOME=${ANDROID_HOME}" >> "$GITHUB_ENV" + echo "ANDROID_SDK_ROOT=${ANDROID_HOME}" >> "$GITHUB_ENV" + NDK_DIR="$(find "${ANDROID_HOME}/ndk" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | sort -V | tail -n 1)" + if [ -z "$NDK_DIR" ]; then + echo "::error::未在 ${ANDROID_HOME}/ndk 下找到 NDK，请先安装 NDK 或在 Actions runner 中配置。" + exit 1 + fi + echo "NDK_HOME=${NDK_DIR}" >> "$GITHUB_ENV" + echo "ANDROID_NDK_HOME=${NDK_DIR}" >> "$GITHUB_ENV" + echo "Using NDK: ${NDK_DIR}" + + # 仓库里 gradle.properties 提交了本机路径 org.gradle.java.home，替换为 runner 的 JDK。 + - name: Point Gradle at runner JDK + shell: bash + run: | + sed -i "s|^org\.gradle\.java\.home=.*|org.gradle.java.home=${JAVA_HOME}|" src-tauri/gen/android/gradle.properties + grep '^org.gradle.java.home' src-tauri/gen/android/gradle.properties + diff --git a/.github/workflows/build-android-signed.yml b/.github/workflows/build-android-signed.yml new file mode 100644 index 0000000..0590f34 --- /dev/null +++ b/.github/workflows/build-android-signed.yml @@ -0,0 +1,66 @@ +name: Build Android (signed arm64, no publish) +on: + workflow_dispatch: +permissions: + contents: read +concurrency: + group: build-android-signed-${{ github.ref }} + cancel-in-progress: true +jobs: + signed-arm64: + name: Signed release APK (arm64-v8a) + runs-on: ubuntu-latest + timeout-minutes: 120 + steps: + - uses: actions/checkout@v5 + - uses: ./.github/actions/setup-build + with: + rust-targets: aarch64-linux-android + - uses: ./.github/actions/setup-android + - name: Configure Android signing (secrets) + env: + KEYSTORE_B64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }} + KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }} + KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }} + KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD }} + run: | + set -euo pipefail + umask 077 + if [ -z "$KEYSTORE_B64" ] || [ -z "$KEYSTORE_PASSWORD" ] || [ -z "$KEY_ALIAS" ] || [ -z "$KEY_PASSWORD" ]; then + echo "::error::缺少签名 secrets。请在仓库 Settings -> Secrets and variables -> Actions 配置" + echo "::error::ANDROID_KEYSTORE_BASE64 / ANDROID_KEYSTORE_PASSWORD / ANDROID_KEY_ALIAS / ANDROID_KEY_PASSWORD。" + exit 1 + fi + echo "$KEYSTORE_B64" | base64 -d > "$RUNNER_TEMP/android-release.keystore" + python3 - <<'PYTHON' + import os + from pathlib import Path + # Encode UTF-16 code units for java.util.Properties, preserving special characters. + def prop(value): + raw = value.encode('utf-16-be') + return ''.join('\\u%04x' % int.from_bytes(raw[i:i+2], 'big') for i in range(0, len(raw), 2)) + values = {'keyAlias': os.environ['KEY_ALIAS'], 'password': os.environ['KEY_PASSWORD'], + 'storePassword': os.environ['KEYSTORE_PASSWORD'], + 'storeFile': os.environ['RUNNER_TEMP'] + '/android-release.keystore'} + Path('src-tauri/gen/android/keystore.properties').write_text( + ''.join(f'{key}={prop(value)}\n' for key, value in values.items())) + PYTHON + + - name: Build signed arm64 APK + run: pnpm tauri android build --apk --target aarch64 --split-per-abi --ci + timeout-minutes: 90 + - name: Collect and verify signed arm64 APK + run: | + node scripts/collect-artifacts.mjs android --abi arm64-v8a --out "$RUNNER_TEMP/artifacts" + BUILD_TOOLS="$(find "$ANDROID_HOME/build-tools" -mindepth 1 -maxdepth 1 -type d | sort -V | tail -n 1)" + "$BUILD_TOOLS/apksigner" verify --verbose "$RUNNER_TEMP"/artifacts/*.apk + "$BUILD_TOOLS/aapt" dump badging "$RUNNER_TEMP"/artifacts/*.apk | grep -Fx "native-code: 'arm64-v8a'" + - name: Remove signing files + if: always() + run: rm -f "$RUNNER_TEMP/android-release.keystore" src-tauri/gen/android/keystore.properties + - uses: actions/upload-artifact@v6 + with: + name: readerx-android-arm64-signed + path: ${{ runner.temp }}/artifacts/*.apk + if-no-files-found: error + retention-days: 14 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..ecbc2aa --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,57 @@ +name: PR CI +on: + pull_request: + types: [opened, synchronize, reopened, ready_for_review] + workflow_dispatch: +permissions: + contents: read +concurrency: + group: pr-ci-${{ github.ref }} + cancel-in-progress: true +jobs: + frontend: + name: Frontend types, i18n and production build + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@v5 + - uses: ./.github/actions/setup-build + with: + system-deps: 'false' + - name: Artifact collection regression + run: node --test scripts/collect-artifacts.test.mjs + - run: pnpm exec tsc --noEmit + - run: pnpm run i18n:check + - run: pnpm build + windows: + name: Windows Rust workspace compilation + runs-on: windows-latest + timeout-minutes: 60 + steps: + - uses: actions/checkout@v5 + - uses: ./.github/actions/setup-build + - run: pnpm build + - name: Check workspace and test targets + working-directory: src-tauri + run: cargo check --locked --workspace --all-targets + android: + name: Android arm64 debug APK (no secrets) + runs-on: ubuntu-latest + timeout-minutes: 120 + steps: + - uses: actions/checkout@v5 + - uses: ./.github/actions/setup-build + with: + rust-targets: aarch64-linux-android + - uses: ./.github/actions/setup-android + - name: Build arm64 debug APK + run: pnpm tauri android build --apk --debug --target aarch64 --split-per-abi --ci + timeout-minutes: 90 + - name: Collect arm64 debug APK + run: node scripts/collect-artifacts.mjs android --debug --abi arm64-v8a --out "${{ runner.temp }}/artifacts" + - uses: actions/upload-artifact@v6 + with: + name: readerx-pr-android-arm64-debug + path: ${{ runner.temp }}/artifacts/*.apk + if-no-files-found: error + retention-days: 7 diff --git a/.github/workflows/test-data.yml b/.github/workflows/test-data.yml index c12bb8f..b4a4c70 100644 --- a/.github/workflows/test-data.yml +++ b/.github/workflows/test-data.yml @@ -2,10 +2,7 @@ name: Test book storage and data compatibility on: pull_request: - paths: - - 'src-tauri/**' - - '.github/workflows/test-data.yml' - - '.github/actions/setup-build/**' + types: [opened, synchronize, reopened, ready_for_review] workflow_dispatch: permissions: @@ -17,6 +14,7 @@ concurrency: jobs: data-tests: + name: Linux storage, backup and sync regression runs-on: ubuntu-latest timeout-minutes: 45 steps: @@ -24,14 +22,24 @@ jobs: - uses: ./.github/actions/setup-build - name: Prepare frontend assets run: pnpm build - - name: Book storage, migrations, backups and sync integration + - name: SQLite book storage and state working-directory: src-tauri - run: | - cargo test --locked -p readerx --lib book_store - cargo test --locked -p readerx --lib sync::book_ids - cargo test --locked -p readerx --lib sync::data_ids - cargo test --locked -p readerx --lib data_transfer - cargo test --locked -p readerx --test sync_bridge - cargo test --locked -p readerx-sync - # Isolate the existing concurrency timing test from other Boa test workloads. - cargo test --locked -p readerx-source -- --test-threads=1 + run: cargo test --locked -p readerx --lib book_store + - name: Book identity migrations + working-directory: src-tauri + run: cargo test --locked -p readerx --lib sync::book_ids + - name: Group and rule identity migrations + working-directory: src-tauri + run: cargo test --locked -p readerx --lib sync::data_ids + - name: Backup export, import and compatibility + working-directory: src-tauri + run: cargo test --locked -p readerx --lib data_transfer + - name: App sync persistence integration + working-directory: src-tauri + run: cargo test --locked -p readerx --test sync_bridge + - name: Sync protocol regression + working-directory: src-tauri + run: cargo test --locked -p readerx-sync + - name: Source engine regression (serial Boa tests) + working-directory: src-tauri + run: cargo test --locked -p readerx-source -- --test-threads=1 diff --git a/README.md b/README.md index 3383cb6..fcc1fe9 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,8 @@ pnpm build 端口 1420 已被占用时，请复用现有开发服务器。Rust 测试需在 `src-tauri/` 中运行。 +Actions 的手动签名 arm64 构建与 PR 检查说明见 [CI 与构建](./docs/ci.md)。 + ## 文档与贡献 | 文档 | 内容 | diff --git a/docs/ci.md b/docs/ci.md new file mode 100644 index 0000000..bda6993 --- /dev/null +++ b/docs/ci.md @@ -0,0 +1,36 @@ +# CI 与手动构建 + +## 手动构建已签名 arm64 APK + +在 Actions 选择 **Build Android (signed arm64, no publish)** → **Run workflow**，选择包含工作流的分支或 tag。PR #13 的分支为 `codex/sqlite-books`。 +此工作流只出 `arm64-v8a` release APK，使用当前仓库版本，不创建 tag 或 GitHub Release。 +成功后下载 `readerx-android-arm64-signed` artifact，文件名为 `readerx-<版本>-android-arm64-v8a.apk`，保留 14 天。 + +首次运行前在 Settings → Secrets and variables → Actions 配置以下仓库 Secrets，与 tag 发布共用同一证书： + +| Secret | 内容 | +| --- | --- | +| `ANDROID_KEYSTORE_BASE64` | keystore 文件的 base64（Linux：`base64 -w0 readerx-release.keystore`） | +| `ANDROID_KEYSTORE_PASSWORD` | keystore 口令 | +| `ANDROID_KEY_ALIAS` | 签名 key 的别名 | +| `ANDROID_KEY_PASSWORD` | key 口令 | + +缺少任何 Secret 会直接失败。工作流验证 APK 签名及唯一原生 ABI，并在成功或失败后清理签名文件；只上传 APK。 +新工作流首次进入默认分支前，Actions 页面可能不显示 Run workflow；可在工作流进入默认分支后选择 PR 分支执行。 +原来的 **Build Android (debug, no publish)** 仍用于手动 universal debug 构建；多 ABI 正式发布使用 `release.yml`。 + +## PR 自动检查 + +所有目标分支的 PR 在 opened / synchronize / reopened / ready_for_review 时触发，不使用路径过滤，避免前端构建依赖变化漏检。重复更新取消旧检查。 + +| 工作流 / Job | 检查范围 | +| --- | --- | +| PR CI / Frontend | 产物收集回归、`tsc --noEmit`、`i18n:check`、生产前端构建 | +| PR CI / Windows | 前端资源构建后，`cargo check --locked --workspace --all-targets` 编译检查 Windows App、workspace 与测试目标；不运行 Windows 测试、不打安装包 | +| PR CI / Android | 构建 arm64 debug APK，覆盖 Android Rust / bundled SQLite、原生插件及 Gradle 打包；无需仓库 Secrets，artifact 保留 7 天 | +| Test book storage and data compatibility / Linux | 前端构建后，分步运行 SQLite 书库 / state、书籍 ID 迁移、分组 / 规则 ID 迁移、备份兼容、App 同步落库、同步协议、书源引擎回归；Boa 测试串行 | + +自动 PR 检查使用 `pull_request` 和只读 contents 权限，不接入 release 签名 Secrets。首次贡献者仍受 GitHub 的工作流审批策略限制。 +这些工作流也支持手动触发。分支保护的 required checks 需在仓库设置中按上述 Job 选择；工作流文件本身不会修改保护规则。 + +产物命名由 `scripts/collect-artifacts.mjs` 统一处理。Android `--abi arm64-v8a` 只收 arm64；省略 `--abi` 时 release 仍要求四个 ABI 齐全，保持 tag 发布的验收条件。 diff --git a/scripts/collect-artifacts.mjs b/scripts/collect-artifacts.mjs index 52c535b..ee60e40 100644 --- a/scripts/collect-artifacts.mjs +++ b/scripts/collect-artifacts.mjs @@ -22,6 +22,7 @@ * --target <三元组> desktop 必填，平台与架构都从三元组解析（如 aarch64-pc-windows-msvc） * --target-dir <目录> desktop 的 target 根目录，默认 src-tauri/target * --out <目录> 产物输出目录，不存在则创建 + * --abi android 只收指定 ABI；未指定时 release 仍要求四个 ABI 齐全 * --debug android 收集 debug 变体（默认 release） * * 收不到任何产物、或 release APK 未签名时以非零码退出：宁可让工作流失败， @@ -60,7 +61,7 @@ function fail(message) { function usage() { console.error("用法:"); console.error(" node scripts/collect-artifacts.mjs desktop --target --out <目录>"); - console.error(" node scripts/collect-artifacts.mjs android [--debug] --out <目录>"); + console.error(" node scripts/collect-artifacts.mjs android [--debug] [--abi ] --out <目录>"); } /** 解析命令行；未知参数直接报错，避免工作流里写错参数却静默少收产物 */ @@ -68,6 +69,7 @@ function parseArgs(argv) { const options = { mode: argv[0], debug: false, + abi: "", target: "", targetDir: join(ROOT, "src-tauri", "target"), out: "", @@ -84,10 +86,11 @@ function parseArgs(argv) { options.debug = true; continue; } - if (arg !== "--target" && arg !== "--target-dir" && arg !== "--out") fail(`未知参数 "${arg}"`); + if (arg !== "--abi" && arg !== "--target" && arg !== "--target-dir" && arg !== "--out") fail(`未知参数 "${arg}"`); const value = argv[++i]; if (!value) fail(`${arg} 缺少取值`); - if (arg === "--target") options.target = value; + if (arg === "--abi") options.abi = value; + else if (arg === "--target") options.target = value; else if (arg === "--target-dir") options.targetDir = value; else options.out = value; } @@ -96,6 +99,8 @@ function parseArgs(argv) { if (options.mode === "desktop" && !options.target) fail("desktop 需要 --target "); if (options.mode === "desktop" && options.debug) fail("--debug 只适用于 android"); if (options.mode === "android" && options.target) fail("--target 只适用于 desktop"); + if (options.abi && (options.mode !== "android" || !Object.values(ANDROID_ABIS).includes(options.abi))) fail("--abi 仅支持 Android 已知 ABI"); + if (options.abi === "universal" && !options.debug) fail("release 不支持 universal ABI"); return options; } @@ -126,11 +131,11 @@ function desktopArtifacts(targetDir, target) { * debug 变体按实际存在的 flavor 收（`--apk` 出 universal，指定单一目标时只出那一个 ABI）； * release 变体要求四个 ABI 齐全，且不接受未签名产物 —— 签名没接上要当场失败。 */ -function androidArtifacts(debug) { +function androidArtifacts(debug, abi) { const apkDir = join(ROOT, "src-tauri", "gen", "android", "app", "build", "outputs", "apk"); if (!existsSync(apkDir)) fail(`未找到 APK 输出目录 ${relative(ROOT, apkDir)}`); - const flavors = debug + const flavors = abi ? [Object.keys(ANDROID_ABIS).find((flavor) => ANDROID_ABIS[flavor] === abi)] : debug ? readdirSync(apkDir) .filter((name) => ANDROID_ABIS[name] && existsSync(join(apkDir, name, "debug"))) .sort() @@ -153,7 +158,8 @@ function androidArtifacts(debug) { const names = readdirSync(variantDir) .filter((name) => extname(name) === ".apk") .sort(); - const name = names.find((candidate) => candidate === `app-${flavor}-${variant}.apk`) ?? names[0]; + const name = names.find((candidate) => candidate === `app-${flavor}-${variant}.apk`) + ?? names.find((candidate) => debug || !candidate.endsWith("-unsigned.apk")); if (!name) fail(`未找到 ${abi} 的 ${variant} APK（期望 ${relative(ROOT, variantDir)}/*.apk）`); found.push({ file: join(variantDir, name), abi }); } @@ -190,7 +196,7 @@ if (options.mode === "desktop") { })); } else { const suffix = options.debug ? "-debug" : ""; - renames = androidArtifacts(options.debug).map(({ file, abi }) => ({ + renames = androidArtifacts(options.debug, options.abi).map(({ file, abi }) => ({ file, name: `${product}-${version}-android-${abi}${suffix}.apk`, })); diff --git a/scripts/collect-artifacts.test.mjs b/scripts/collect-artifacts.test.mjs new file mode 100644 index 0000000..940b696 --- /dev/null +++ b/scripts/collect-artifacts.test.mjs @@ -0,0 +1,67 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, mkdirSync, copyFileSync, writeFileSync, readdirSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { spawnSync } from 'node:child_process'; + +function fixture(t, artifacts) { + const root = mkdtempSync(join(tmpdir(), 'readerx-artifacts-')); + t.after(() => rmSync(root, { recursive: true, force: true })); + mkdirSync(join(root, 'scripts')); + mkdirSync(join(root, 'src-tauri')); + copyFileSync(new URL('./collect-artifacts.mjs', import.meta.url), join(root, 'scripts/collect-artifacts.mjs')); + writeFileSync(join(root, 'src-tauri/tauri.conf.json'), JSON.stringify({ version: '0.5.0', productName: 'readerx' })); + for (const [flavor, variant, name] of artifacts) { + const dir = join(root, 'src-tauri/gen/android/app/build/outputs/apk', flavor, variant); + mkdirSync(dir, { recursive: true }); + writeFileSync(join(dir, name), 'fixture APK bytes'); + } + return (...args) => { + const out = join(root, 'out'); + const result = spawnSync(process.execPath, [join(root, 'scripts/collect-artifacts.mjs'), 'android', ...args, '--out', out], { encoding: 'utf8' }); + return { ...result, files: result.status === 0 ? readdirSync(out) : [] }; + }; +} + +test('single ABI release collects only arm64 even when other outputs exist', t => { + const run = fixture(t, [['arm64', 'release', 'app-arm64-release.apk'], ['x86', 'release', 'app-x86-release.apk']]); + const result = run('--abi', 'arm64-v8a'); + assert.equal(result.status, 0, result.stderr); + assert.deepEqual(result.files, ['readerx-0.5.0-android-arm64-v8a.apk']); +}); + +test('default release still requires all four ABIs', t => { + const result = fixture(t, [['arm64', 'release', 'app-arm64-release.apk']])(); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /armeabi-v7a/); +}); + +test('default release collects all four ABIs', t => { + const run = fixture(t, ['arm64', 'arm', 'x86', 'x86_64'].map(f => [f, 'release', `app-${f}-release.apk`])); + assert.equal(run().files.length, 4); +}); + +test('rejects unsigned APKs including unexpected unsigned names', t => { + for (const name of ['app-arm64-release-unsigned.apk', 'other-unsigned.apk']) { + const result = fixture(t, [['arm64', 'release', name]])('--abi', 'arm64-v8a'); + assert.notEqual(result.status, 0); + } +}); + +test('missing requested ABI fails rather than collecting another ABI', t => { + const result = fixture(t, [['x86', 'release', 'app-x86-release.apk']])('--abi', 'arm64-v8a'); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /arm64-v8a/); +}); + +test('debug single ABI and existing universal debug preserve names', t => { + const run = fixture(t, [['arm64', 'debug', 'app-arm64-debug.apk'], ['universal', 'debug', 'app-universal-debug.apk']]); + assert.deepEqual(run('--debug', '--abi', 'arm64-v8a').files, ['readerx-0.5.0-android-arm64-v8a-debug.apk']); + assert.equal(run('--debug').files.length, 2); +}); + +test('rejects unknown ABI and universal release', t => { + const run = fixture(t, []); + for (const abi of ['aarch64', 'unknown', 'universal']) assert.notEqual(run('--abi', abi).status, 0); +}); From 58a176e74a2062e0ba64599a34ef4e9189595f50 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 23:42:38 +0800 Subject: [PATCH 16/17] =?UTF-8?q?fix:=20PR=20CI=20=E4=BB=85=E6=89=A7?= =?UTF-8?q?=E8=A1=8C=E6=A3=80=E6=9F=A5=E5=B9=B6=E5=B0=86=E7=BC=96=E8=AF=91?= =?UTF-8?q?=E6=B5=8B=E8=AF=95=E6=94=B9=E4=B8=BA=E6=89=8B=E5=8A=A8=E8=A7=A6?= =?UTF-8?q?=E5=8F=91?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/workflows/ci.yml | 53 ++++++++++----------------------- .github/workflows/test-data.yml | 2 -- docs/ci.md | 21 +++++++------ 3 files changed, 28 insertions(+), 48 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ecbc2aa..8b3f127 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -9,8 +9,8 @@ concurrency: group: pr-ci-${{ github.ref }} cancel-in-progress: true jobs: - frontend: - name: Frontend types, i18n and production build + checks: + name: Static checks (no build) runs-on: ubuntu-latest timeout-minutes: 15 steps: @@ -18,40 +18,19 @@ jobs: - uses: ./.github/actions/setup-build with: system-deps: 'false' - - name: Artifact collection regression + - name: TypeScript types (no emit) + run: pnpm exec tsc --noEmit + - name: i18n keys and user-facing text + run: pnpm run i18n:check + - name: Artifact collection regression (fixtures only) run: node --test scripts/collect-artifacts.test.mjs - - run: pnpm exec tsc --noEmit - - run: pnpm run i18n:check - - run: pnpm build - windows: - name: Windows Rust workspace compilation - runs-on: windows-latest - timeout-minutes: 60 - steps: - - uses: actions/checkout@v5 - - uses: ./.github/actions/setup-build - - run: pnpm build - - name: Check workspace and test targets + - name: Rust workspace manifests (no compilation) working-directory: src-tauri - run: cargo check --locked --workspace --all-targets - android: - name: Android arm64 debug APK (no secrets) - runs-on: ubuntu-latest - timeout-minutes: 120 - steps: - - uses: actions/checkout@v5 - - uses: ./.github/actions/setup-build - with: - rust-targets: aarch64-linux-android - - uses: ./.github/actions/setup-android - - name: Build arm64 debug APK - run: pnpm tauri android build --apk --debug --target aarch64 --split-per-abi --ci - timeout-minutes: 90 - - name: Collect arm64 debug APK - run: node scripts/collect-artifacts.mjs android --debug --abi arm64-v8a --out "${{ runner.temp }}/artifacts" - - uses: actions/upload-artifact@v6 - with: - name: readerx-pr-android-arm64-debug - path: ${{ runner.temp }}/artifacts/*.apk - if-no-files-found: error - retention-days: 7 + run: cargo metadata --locked --no-deps --format-version 1 > "$RUNNER_TEMP/cargo-metadata.json" + - name: GitHub Actions syntax + run: | + curl --fail --silent --show-error --location \ + https://github.com/rhysd/actionlint/releases/download/v1.7.12/actionlint_1.7.12_linux_amd64.tar.gz \ + --output "$RUNNER_TEMP/actionlint.tar.gz" + tar -xzf "$RUNNER_TEMP/actionlint.tar.gz" -C "$RUNNER_TEMP" actionlint + "$RUNNER_TEMP/actionlint" -color diff --git a/.github/workflows/test-data.yml b/.github/workflows/test-data.yml index b4a4c70..294192e 100644 --- a/.github/workflows/test-data.yml +++ b/.github/workflows/test-data.yml @@ -1,8 +1,6 @@ name: Test book storage and data compatibility on: - pull_request: - types: [opened, synchronize, reopened, ready_for_review] workflow_dispatch: permissions: diff --git a/docs/ci.md b/docs/ci.md index bda6993..3a1820d 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -21,16 +21,19 @@ ## PR 自动检查 -所有目标分支的 PR 在 opened / synchronize / reopened / ready_for_review 时触发，不使用路径过滤，避免前端构建依赖变化漏检。重复更新取消旧检查。 +所有目标分支的 PR 在 opened / synchronize / reopened / ready_for_review 时触发，不使用路径过滤。重复更新取消旧检查。 +PR CI 只做检查，不运行前端生产构建、Rust 编译 / 测试或 Android / Windows 打包，不上传构建产物。 -| 工作流 / Job | 检查范围 | +| 检查步骤 | 检查范围 | | --- | --- | -| PR CI / Frontend | 产物收集回归、`tsc --noEmit`、`i18n:check`、生产前端构建 | -| PR CI / Windows | 前端资源构建后，`cargo check --locked --workspace --all-targets` 编译检查 Windows App、workspace 与测试目标；不运行 Windows 测试、不打安装包 | -| PR CI / Android | 构建 arm64 debug APK，覆盖 Android Rust / bundled SQLite、原生插件及 Gradle 打包；无需仓库 Secrets，artifact 保留 7 天 | -| Test book storage and data compatibility / Linux | 前端构建后，分步运行 SQLite 书库 / state、书籍 ID 迁移、分组 / 规则 ID 迁移、备份兼容、App 同步落库、同步协议、书源引擎回归；Boa 测试串行 | - -自动 PR 检查使用 `pull_request` 和只读 contents 权限，不接入 release 签名 Secrets。首次贡献者仍受 GitHub 的工作流审批策略限制。 -这些工作流也支持手动触发。分支保护的 required checks 需在仓库设置中按上述 Job 选择；工作流文件本身不会修改保护规则。 +| TypeScript types | `tsc --noEmit`，只做类型检查 | +| i18n keys and user-facing text | `i18n:check`，检查词典与界面文案 | +| Artifact collection regression | Node 测试使用临时模拟 APK 文件，验证收集与命名规则，不构建 APK | +| Rust workspace manifests | `cargo metadata --locked --no-deps`，只解析 workspace 清单，不编译；不验证 Rust 类型或运行回归 | +| GitHub Actions syntax | 固定版本 actionlint 检查工作流语法 | + +自动 PR 检查使用 `pull_request` 和只读 contents 权限，不接入签名 Secrets。首次贡献者仍受 GitHub 的工作流审批策略限制。 +**Test book storage and data compatibility** 改为只支持手动触发；需要验证 SQLite、迁移、备份与同步时再手动运行，它会构建前端并编译执行 Rust 测试。 +Android / Desktop 手动构建和 tag 发布继续独立运行。分支保护的 required checks 需在仓库设置中选择 `Static checks (no build)`；工作流文件本身不会修改保护规则。 产物命名由 `scripts/collect-artifacts.mjs` 统一处理。Android `--abi arm64-v8a` 只收 arm64；省略 `--abi` 时 release 仍要求四个 ABI 齐全，保持 tag 发布的验收条件。 From b2d68c39ee9fc5bb7d4972c6de3f9c6259b7a8e1 Mon Sep 17 00:00:00 2001 From: zilorn <79640587+zilorn@users.noreply.github.com> Date: Fri, 9 Oct 2026 23:45:46 +0800 Subject: [PATCH 17/17] =?UTF-8?q?fix:=20PR=20CI=20=E5=A2=9E=E5=8A=A0=20Rus?= =?UTF-8?q?t=20workspace=20=E4=B8=8E=E6=B5=8B=E8=AF=95=E7=9B=AE=E6=A0=87?= =?UTF-8?q?=E6=A3=80=E6=9F=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/workflows/ci.yml | 15 +++++++++++---- docs/ci.md | 7 ++++--- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8b3f127..cabcd35 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -10,7 +10,7 @@ concurrency: cancel-in-progress: true jobs: checks: - name: Static checks (no build) + name: Frontend and workflow checks runs-on: ubuntu-latest timeout-minutes: 15 steps: @@ -24,9 +24,6 @@ jobs: run: pnpm run i18n:check - name: Artifact collection regression (fixtures only) run: node --test scripts/collect-artifacts.test.mjs - - name: Rust workspace manifests (no compilation) - working-directory: src-tauri - run: cargo metadata --locked --no-deps --format-version 1 > "$RUNNER_TEMP/cargo-metadata.json" - name: GitHub Actions syntax run: | curl --fail --silent --show-error --location \ @@ -34,3 +31,13 @@ jobs: --output "$RUNNER_TEMP/actionlint.tar.gz" tar -xzf "$RUNNER_TEMP/actionlint.tar.gz" -C "$RUNNER_TEMP" actionlint "$RUNNER_TEMP/actionlint" -color + rust: + name: Rust workspace and test target checks + runs-on: ubuntu-latest + timeout-minutes: 45 + steps: + - uses: actions/checkout@v5 + - uses: ./.github/actions/setup-build + - name: Check Rust code and test targets (no application bundle) + working-directory: src-tauri + run: cargo check --locked --workspace --all-targets diff --git a/docs/ci.md b/docs/ci.md index 3a1820d..dd208a6 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -22,18 +22,19 @@ ## PR 自动检查 所有目标分支的 PR 在 opened / synchronize / reopened / ready_for_review 时触发，不使用路径过滤。重复更新取消旧检查。 -PR CI 只做检查，不运行前端生产构建、Rust 编译 / 测试或 Android / Windows 打包，不上传构建产物。 +PR CI 只做检查，不运行前端生产构建、`cargo build`、Rust 测试或 Android / Windows 打包，不上传构建产物。 +Rust 使用 `cargo check` 检查代码与测试目标；Cargo 仍会编译依赖和运行构建脚本，但不会生成应用安装包。Linux Job 安装 Tauri 检查所需系统开发库。 | 检查步骤 | 检查范围 | | --- | --- | | TypeScript types | `tsc --noEmit`，只做类型检查 | | i18n keys and user-facing text | `i18n:check`，检查词典与界面文案 | | Artifact collection regression | Node 测试使用临时模拟 APK 文件，验证收集与命名规则，不构建 APK | -| Rust workspace manifests | `cargo metadata --locked --no-deps`，只解析 workspace 清单，不编译；不验证 Rust 类型或运行回归 | +| Rust workspace and test target checks | `cargo check --locked --workspace --all-targets`，检查 Rust 代码和测试目标，不执行测试、不打包 | | GitHub Actions syntax | 固定版本 actionlint 检查工作流语法 | 自动 PR 检查使用 `pull_request` 和只读 contents 权限，不接入签名 Secrets。首次贡献者仍受 GitHub 的工作流审批策略限制。 **Test book storage and data compatibility** 改为只支持手动触发；需要验证 SQLite、迁移、备份与同步时再手动运行，它会构建前端并编译执行 Rust 测试。 -Android / Desktop 手动构建和 tag 发布继续独立运行。分支保护的 required checks 需在仓库设置中选择 `Static checks (no build)`；工作流文件本身不会修改保护规则。 +Android / Desktop 手动构建和 tag 发布继续独立运行。分支保护的 required checks 需在仓库设置中选择 `Frontend and workflow checks` 与 `Rust workspace and test target checks`；工作流文件本身不会修改保护规则。 产物命名由 `scripts/collect-artifacts.mjs` 统一处理。Android `--abi arm64-v8a` 只收 arm64；省略 `--abi` 时 release 仍要求四个 ABI 齐全，保持 tag 发布的验收条件。