v4.5.0: dark mode, text darkness, X3 polish, stability + bookshelf cleanup

README.md

@@ -59,6 +59,8 @@ Books normally index their chapter structure on first open and re-index when you
 
 The previous Bluetooth-image-cache (`.pxc`) and chapter-text flattening features are still included as part of the optimizer's pre-cache pipeline.
 
+**Hover any pre-baked book's badge in the File Manager** to see a tooltip with the baked-in layout (font, font size, orientation, line spacing, margin). Long-press the badge on-device for the same view. The pre-cache toggle is now named **Pre-Bake (Image, Chapter, Custom Font)** with a bullet-list description so it's clearer what gets baked.
+
 ### Bluetooth remote page-turner
 
 Pairing is done from WITHIN A BOOK ONLY! Click on the "Confirm" button while inside a book to open the reader menu. Navigate to Bluetooth and follow the instructions there to pair a BT HID remote (e.g. an [IINE GameBrick](https://www.amazon.com/dp/B0CK4DNQM4) or Free2) and use it as a wireless page-turner. BLE auto-disables when you exit the book to keep heap pressure off the parser, so you will need to reconnect again when you enter a new book.
@@ -142,6 +144,7 @@ Back from any nested submenu pops one level up; back from the root exits to Home
 - **Auto-retry on chapter-layout abort** — if the parser trips the low-heap floor with BLE consuming its ~58 KB share, CrumBLE silently drops BLE, retries the layout with the recovered headroom, and lets the existing auto-reconnect logic re-pair on your next remote press.
 - **Glyph buffer pre-grown at every BT-enable site** so the font scratch's high-water mark is allocated BEFORE NimBLE eats heap, preventing the mid-page-turn allocation failures that used to drop the BT link on text-heavy chapters.
 - **Large-library + home stability** (v3.0.x) — streaming library index that survives big libraries, crash-proofed series detection, a Lyra Carousel heap-race crash fix, cover-thumbnail revalidation so a single book can't get stuck on a placeholder, and transparent-PNG sleep screens that reliably show the clean last book page.
+- **Additional features** — Dark Reader Mode, Text Darkness, Paragraph Spacing, Retry Failed Covers
 
 For the full changelog, see [CHANGELOG.md](./CHANGELOG.md).
 

lib/EpdFont/EpdFontData.h

@@ -161,8 +161,88 @@ constexpr bool isReplacementFallback(uint32_t cp) { return cp == 0xFFFD; }
 constexpr bool isSpaceFallback(uint32_t cp) {
   return cp == 0x00A0 || cp == 0x1680 || (cp >= 0x2000 && cp <= 0x200A) || cp == 0x202F || cp == 0x205F || cp == 0x3000;
 }
+// Typography fallback table (v4.3 task #28). When a font lacks a fancy
+// punctuation codepoint, alias it to the nearest ASCII equivalent so the
+// renderer's glyph lookup finds a real glyph instead of producing '?'.
+// EpdFontFamily::getFallbackCodepoint runs this on every miss, AND a
+// successful alias still has to be present in the font itself for the
+// renderer to draw it -- but for the SD-font atlas / embedded-subset
+// path the ASCII targets ('. - " ' , * `) appear in almost every section
+// (they are the bread-and-butter glyphs of any prose chapter), so the
+// alias closes the visible '?' gap in practice. Tried-and-failed
+// substitutions still bottom out at REPLACEMENT_GLYPH per the existing
+// getFallbackCodepoint flow.
 constexpr uint32_t aliasCodepoint(uint32_t cp) {
-  return cp == MODIFIER_LETTER_TURNED_COMMA ? LEFT_SINGLE_QUOTATION_MARK : cp;
+  switch (cp) {
+    // Pre-existing alias: the Hawaiian okina / Latin extended turned comma
+    // both look like a left single quote; keep mapping it to the typographic
+    // single quote so any font shipping the typographic glyph wins. The
+    // smart-quote branches below alias that target back to ASCII when the
+    // font lacks it.
+    case MODIFIER_LETTER_TURNED_COMMA:
+      return LEFT_SINGLE_QUOTATION_MARK;
+
+    // Smart single quotes / similar -> ASCII apostrophe (U+0027)
+    case 0x2018:  // LEFT SINGLE QUOTATION MARK
+    case 0x2019:  // RIGHT SINGLE QUOTATION MARK
+    case 0x201A:  // SINGLE LOW-9 QUOTATION MARK
+    case 0x201B:  // SINGLE HIGH-REVERSED-9 QUOTATION MARK
+    case 0x2032:  // PRIME
+      return 0x27;
+
+    // Smart double quotes / similar -> ASCII quotation mark (U+0022)
+    case 0x201C:  // LEFT DOUBLE QUOTATION MARK
+    case 0x201D:  // RIGHT DOUBLE QUOTATION MARK
+    case 0x201E:  // DOUBLE LOW-9 QUOTATION MARK
+    case 0x201F:  // DOUBLE HIGH-REVERSED-9 QUOTATION MARK
+    case 0x2033:  // DOUBLE PRIME
+      return 0x22;
+
+    // Dashes (incl. non-breaking hyphen) -> ASCII hyphen-minus (U+002D).
+    // Loses semantic length distinction (em vs en) -- acceptable trade for
+    // not showing '?' in dialogue dashes and parentheticals.
+    case 0x2010:  // HYPHEN
+    case 0x2011:  // NON-BREAKING HYPHEN
+    case 0x2012:  // FIGURE DASH
+    case 0x2013:  // EN DASH
+    case 0x2014:  // EM DASH
+    case 0x2015:  // HORIZONTAL BAR
+    case 0x2212:  // MINUS SIGN (math)
+      return 0x2D;
+
+    // Bullets, geometric shapes, decoration glyphs -> ASCII asterisk
+    // (U+002A). Covers the scene-break decorator chars publishers use
+    // to separate scenes within a chapter (diamonds, stars, hollow
+    // shapes) -- ChareInk and similarly-trimmed SD fonts often ship
+    // without these specific geometric blocks. The renderer-level
+    // getFallbackCodepoint miss-handler path will try to load the '*'
+    // glyph on demand from the .cpfont's full intervals if the prewarmed
+    // mini set didn't include it.
+    case 0x2022:  // BULLET
+    case 0x2023:  // TRIANGULAR BULLET
+    case 0x25CF:  // BLACK CIRCLE
+    case 0x25E6:  // WHITE BULLET
+    case 0x25C6:  // BLACK DIAMOND
+    case 0x25C7:  // WHITE DIAMOND
+    case 0x25C8:  // WHITE DIAMOND CONTAINING BLACK SMALL DIAMOND
+    case 0x25CA:  // LOZENGE
+    case 0x2666:  // BLACK DIAMOND SUIT
+    case 0x2662:  // WHITE DIAMOND SUIT
+    case 0x2605:  // BLACK STAR
+    case 0x2606:  // WHITE STAR
+    case 0x2731:  // HEAVY ASTERISK
+    case 0x2732:  // OPEN CENTRE ASTERISK
+    case 0x2735:  // EIGHT POINTED PINWHEEL STAR
+      return 0x2A;
+
+    // Horizontal ellipsis -> ASCII period (U+002E). 1:1 alias loses the
+    // three-dot appearance; ASCII '.' is in every chapter so this beats '?'.
+    case 0x2026:
+      return 0x2E;
+
+    default:
+      return cp;
+  }
 }
 
 inline uint16_t solidAdvanceX(const EpdFontData* data, const EpdGlyph* emGlyph) {

lib/EpdFont/EpdFontFamily.cpp

@@ -341,10 +341,43 @@ uint32_t EpdFontFamily::getFallbackCodepoint(const uint32_t cp, const Style styl
   if (findGlyphData(cp, style).glyph) return cp;
   const uint32_t aliasCp = syntheticGlyph::aliasCodepoint(cp);
   if (aliasCp != cp) {
-    return findGlyphData(aliasCp, style).glyph ? aliasCp : REPLACEMENT_GLYPH;
+    if (findGlyphData(aliasCp, style).glyph) return aliasCp;
+    // CrumBLE 4.4 task #28 follow-up: when findGlyphData misses on both
+    // cp and aliasCp, give the font's own miss handler a chance to load
+    // the alias target. The atlas / SD-font miniData only contain
+    // codepoints the chapter actually uses, so a chapter that has '◆'
+    // but no '*' anywhere will see findGlyphData('*') miss even though
+    // the underlying .cpfont ships an asterisk glyph. font->getGlyph
+    // walks the font's intervals AND (for SD-card fonts) calls the
+    // onGlyphMiss handler which lazy-loads from the .cpfont's full
+    // glyph table into the overflow ring buffer -- so a successful
+    // load here means the renderer will draw a real glyph instead of
+    // the synthetic replacement box. Zero cost when the font's intervals
+    // already excluded the alias target (binary-search miss = ~150ns).
+    const EpdFont* f = getFont(style);
+    if (f && f->getGlyph(aliasCp)) return aliasCp;
+    // CrumBLE 4.4 task #26: when bold/italic miss-handler also fails
+    // (font lacks that style entirely), check the regular-style chain.
+    // Mirrors getGlyphData's regular-fallback so drawText's pre-flight
+    // decision matches the actual draw-time outcome.
+    if (style != REGULAR && getGlyphData(aliasCp, REGULAR).glyph) return aliasCp;
+    return REPLACEMENT_GLYPH;
   }
   if (syntheticGlyph::isSpaceFallback(cp)) return cp;
   if (syntheticGlyph::isSolid(cp) || syntheticGlyph::isGreekFallback(cp)) return cp;
+  // CrumBLE 4.4 task #26: chapter titles + emphasis use BOLD / ITALIC.
+  // When the atlas only carries the REGULAR style (the prebake's prewarm
+  // only loads REGULAR by default) AND the SD font lacks the requested
+  // style's glyph (miss handler returns null because the .cpfont ships
+  // regular-only), getGlyphData's existing regular fallback (line ~276)
+  // would salvage the draw -- but only IF the renderer reaches the
+  // renderCharImpl path. Returning REPLACEMENT_GLYPH here short-circuits
+  // that path and forces the synthetic '?' box instead. Probe regular's
+  // chain for `cp`; if it has the glyph, return cp so drawText proceeds
+  // to renderCharImpl which calls getGlyphData and lets the regular
+  // fallback cascade run. The bold/italic visual style is lost but text
+  // stays readable -- preferable to "??? Two" for chapter headers.
+  if (style != REGULAR && getGlyphData(cp, REGULAR).glyph) return cp;
   return REPLACEMENT_GLYPH;
 }
 

lib/EpdFont/SdCardFont.cpp

@@ -146,6 +146,11 @@ void SdCardFont::freeAll() {
   styleCount_ = 0;
   contentHash_ = 0;
   loaded_ = false;
+  // CrumBLE 4.4 task #51: release the persistent glyph file handle on
+  // unload. Re-opened on the next load().
+  if (persistentGlyphFile_.isOpen()) {
+    persistentGlyphFile_.close();
+  }
 }
 
 void SdCardFont::clearOverflow() {
@@ -624,6 +629,36 @@ bool SdCardFont::load(const char* path) {
 
   loaded_ = true;
 
+  // CrumBLE 4.4 task #51: open the persistent glyph-miss file handle
+  // NOW while the heap is still healthy. onGlyphMiss() reuses this
+  // handle (seek + read, no allocation) instead of opening the file
+  // fresh on every miss. Pre-NimBLE heap is comfortable, so the open
+  // will succeed; under post-NimBLE pressure where the per-call open
+  // would fail, this handle is already open and the seek/read
+  // succeeds. If this open fails for some reason, persistentGlyphFile_
+  // stays closed and onGlyphMiss falls back to its original per-call
+  // open path -- graceful degradation, no behavior change vs old.
+  if (!Storage.openFileForRead("SDCF", filePath_, persistentGlyphFile_)) {
+    LOG_ERR("SDCF", "Failed to pre-open persistent glyph file; "
+                    "onGlyphMiss will fall back to per-call open path");
+  }
+
+  // CrumBLE 4.4 task #51: also eagerly load each style's full intervals
+  // table NOW so onGlyphMiss never triggers the lazy
+  // ensureStyleIntervalsLoaded path -- that path opens the file AND
+  // allocates `new EpdUnicodeInterval[intervalCount]`, both of which
+  // fail under post-NimBLE heap pressure. Doing it here costs maybe a
+  // few KB of resident memory per book but happens against a healthy
+  // boot heap. The lazy entry point still exists for other callers
+  // (e.g. prebake tools) and as a fallback if this eager load fails.
+  for (uint8_t i = 0; i < MAX_STYLES; i++) {
+    if (!styles_[i].present) continue;
+    if (!ensureStyleIntervalsLoaded(i)) {
+      LOG_ERR("SDCF", "Failed to pre-load intervals for style %u; "
+                      "onGlyphMiss will retry the lazy load (may fail on tight heap)", i);
+    }
+  }
+
   LOG_DBG("SDCF", "Loaded: %s (v%u, %u styles)", path, CPFONT_VERSION, styleCount_);
   for (uint8_t i = 0; i < MAX_STYLES; i++) {
     if (!styles_[i].present) continue;
@@ -1340,18 +1375,34 @@ const EpdGlyph* SdCardFont::onGlyphMiss(void* ctx, uint32_t codepoint) {
   uint32_t slot = self->overflowNext_;
   bool wasAtCapacity = (self->overflowCount_ == OVERFLOW_CAPACITY);
 
-  // Read glyph metadata into temporary
-  FsFile file;
-  if (!Storage.openFileForRead("SDCF", self->filePath_, file)) {
-    LOG_ERR("SDCF", "Overflow: failed to open .cpfont");
-    return nullptr;
+  // CrumBLE 4.4 task #51: use the persistent file handle when
+  // available. The per-call Storage.openFileForRead path allocates
+  // ~12 bytes for HalFile::Impl and crashes under post-NimBLE heap
+  // pressure. The persistent handle was opened during load() while
+  // the heap was healthy; here we just seek and read on it. Fallback
+  // to the per-call open path if the persistent handle isn't open
+  // (load-time pre-open failed -- graceful degradation).
+  FsFile fallbackFile;
+  HalFile* fileRef = nullptr;
+  if (self->persistentGlyphFile_.isOpen()) {
+    fileRef = &self->persistentGlyphFile_;
+  } else {
+    if (!Storage.openFileForRead("SDCF", self->filePath_, fallbackFile)) {
+      LOG_ERR("SDCF", "Overflow: failed to open .cpfont");
+      return nullptr;
+    }
+    fileRef = &fallbackFile;
   }
+  HalFile& file = *fileRef;
 
   EpdGlyph tempGlyph = {};
   uint32_t glyphFileOff = s.glyphsFileOffset + static_cast<uint32_t>(globalIdx) * sizeof(EpdGlyph);
   if (!file.seekSet(glyphFileOff)) {
     LOG_ERR("SDCF", "Overflow: failed to seek to glyph for U+%04X style %u", codepoint, styleIdx);
-    file.close();
+    // Do NOT close on seek failure -- file is still valid, only the
+    // seek failed. Closing would force every future miss back to the
+    // per-call open path, defeating the fix. fallbackFile (if used)
+    // closes itself on scope exit.
     return nullptr;
   }
   if (file.read(reinterpret_cast<uint8_t*>(&tempGlyph), sizeof(EpdGlyph)) != sizeof(EpdGlyph)) {
@@ -1370,7 +1421,9 @@ const EpdGlyph* SdCardFont::onGlyphMiss(void* ctx, uint32_t codepoint) {
     if (!file.seekSet(s.bitmapFileOffset + tempGlyph.dataOffset)) {
       LOG_ERR("SDCF", "Overflow: failed to seek to bitmap for U+%04X", codepoint);
       delete[] tempBitmap;
-      file.close();
+      // CrumBLE 4.4 task #51: do NOT close the persistent handle on a
+      // bad seek -- the file is still valid. Closing would force every
+      // subsequent miss back to the per-call open path.
       return nullptr;
     }
     if (file.read(tempBitmap, tempGlyph.dataLength) != static_cast<int>(tempGlyph.dataLength)) {

lib/EpdFont/SdCardFont.h

@@ -4,6 +4,8 @@
 #include <string>
 #include <vector>
 
+#include <HalStorage.h>  // CrumBLE 4.4 task #51: HalFile persistentGlyphFile_
+
 #include "EpdFont.h"
 #include "EpdFontData.h"
 
@@ -228,6 +230,18 @@ class SdCardFont {
 
   char filePath_[128] = {};
 
+  // CrumBLE 4.4 task #51: persistent file handle for SdCardFont::onGlyphMiss.
+  // Without this, every glyph miss opens the .cpfont fresh via
+  // Storage.openFileForRead -> make_unique<HalFile::Impl> which allocates
+  // ~12 bytes + FsFile state. Under post-NimBLE heap pressure (MaxAlloc
+  // 500-2000 bytes after BT connect, lower on memory-tight books) that
+  // allocation fails -> bad_alloc -> __terminate -> panic-reboot. The
+  // handle is opened during load() while the heap is healthy and reused
+  // for every glyph miss; onGlyphMiss does only seek + read on it
+  // (no allocations). If the load-time open fails, onGlyphMiss falls
+  // back to the original per-call open path.
+  HalFile persistentGlyphFile_;
+
   // Overflow context: glyphMissHandler needs to know which style it's serving
   struct OverflowContext {
     SdCardFont* self;

lib/Epub/Epub/GlyphAtlas.h

@@ -93,17 +93,29 @@ static_assert(sizeof(BlockHeader) == 12, "BlockHeader must be 12 bytes on the wi
 static_assert(sizeof(StyleHeader) == 12, "StyleHeader must be 12 bytes on the wire");
 static_assert(sizeof(GlyphEntry) == 12, "GlyphEntry must be 12 bytes on the wire");
 
-// Helpers for packed bitmap addressing. The bitmap payload is row-major:
-// for each glyph, height consecutive rows of (width + padBits) / 8 bytes
-// each, where padBits rounds the row up to a byte boundary. This matches
-// the SD-card font format used today, so the existing 1-bit blit code path
-// in EpdFont can be reused unchanged.
+// Helpers for packed bitmap addressing. The bitmap payload is a CONTINUOUS
+// bitstream (no per-row alignment) so a single (y*width + x) pixel index
+// addresses straight into it. This matches GfxRenderer::renderCharImpl's
+// blit loop (see GfxRenderer.cpp, the "pixelPosition" variable: pixel index
+// runs 0..(width*height-1) across rows, and the byte address is
+// pixelPosition >> 3 for 1-bit / >> 2 for 2-bit). The FontDecompressor's
+// compactSingleGlyph likewise produces continuous bitstream output -- so
+// builtin fonts feed the same renderer with the same packing convention.
+// Byte-aligning rows would silently misalign every row past the first for
+// any glyph whose width isn't a multiple of 8 (for 1-bit) or 4 (for 2-bit).
+//
+// rowBytes() is retained only as a per-row helper for callers that walk
+// individual rows; the canonical glyph size is glyphBytes(), which uses
+// continuous packing.
 constexpr uint16_t rowBytes(uint8_t widthPx, uint8_t bitDepth) {
   return static_cast<uint16_t>((widthPx * bitDepth + 7) / 8);
 }
 
 constexpr uint16_t glyphBytes(uint8_t widthPx, uint8_t heightPx, uint8_t bitDepth) {
-  return static_cast<uint16_t>(rowBytes(widthPx, bitDepth)) * heightPx;
+  // Continuous packing: total bits = width * height * bitDepth, rounded up
+  // to the next whole byte for the glyph as a whole (not per-row).
+  return static_cast<uint16_t>(
+      (static_cast<uint32_t>(widthPx) * static_cast<uint32_t>(heightPx) * bitDepth + 7) / 8);
 }
 
 }  // namespace glyphatlas