feat(v4.3-rc1): BT page-render stability baseline

Aggregates the v4.3 BT + SD-font reading work into a single revertable
baseline. After this commit, BT works with image-heavy and built-in
font books; SD-font book BT enable + first-page render works; the
known limitation is page-turning on text-heavy SD-font chapters
(~28+ elements per page) which crashes due to a per-page deserialize
peak exceeding the post-NimBLE contiguous heap budget by a few
hundred bytes. Permanent fix is the page-DOM arena (Phase 1 of
which is the next planned work).

Notable pieces:

- Embedded glyph subset: per-section v39 trailer holds the codepoints
  the chapter actually uses, installed on section load so post-BT
  glyph lookups hit in-RAM instead of routing through SdCardFont's
  miss-handler under tight heap. Survives BT enable (Option 1 -- a
  prior attempt to drop+reload across BT could not reliably re-allocate
  the 5.6 KB block after NimBLE fragmented the heap).

- Page-DOM heap reserve: 18 KB pre-allocated chunk acquired lazily on
  first chapter open, released for BT enable so NimBLE's ~67 KB
  allocation has a clean contiguous slot. Best-effort re-acquire after
  BT disconnect.

- TextBlock typography substitutions: curly quotes / dashes / ellipsis
  / various spaces fold to their ASCII equivalents so embedded subsets
  built from ASCII-only sources still render correctly.

- SdCardFont: static codepoint buffer in prewarm (saves per-render
  2 KB alloc), preflight against 8 KB MaxAlloc floor, lazy non-REGULAR
  styles when BT-bonded controller is present.

- BluetoothHIDManager: deinit(true) on disable so stale NimBLEClient
  objects from the previous session are released. Previously they
  leaked across enable/disable cycles, causing the next connect's
  reconnect-with-existing-client path to fail mid-subscribe with
  BLE_INIT Malloc-failed and a ~22 s recovery.

- NimBLE platformio.ini knobs tuned down (MAX_CONNECTIONS=1, bonds=1,
  MSYS_1_BLOCK_COUNT=4) for the single-gamepad use case.

- Page::elements moved from shared_ptr to unique_ptr (eliminates
  per-element control-block alloc).

- Page heap reserve / NimBLE pre-flight gates moved into
  EpubReaderActivity's BT-toggle path.

The page-deserialize crash on heavy pages is intentionally NOT
papered over here -- a tried escape valve (drop embedded subset
during deserialize) worked mechanically but broke glyph rendering
because the lazy reinstall could not allocate. Documented inline
in EpubReaderActivity render() so the next attempt has the failure
mode written down.

Author: imshentastic

lib/EpdFont/EpdFontFamily.cpp

@@ -262,6 +262,71 @@ EpdFontFamily::GlyphData EpdFontFamily::getGlyphData(const uint32_t cp, const St
     return {font->data, glyph};
   }
 
+  // CrumBLE 4.3: regular-style fallback for SD fonts with missing styles.
+  // When this is a non-regular style and the lookup fully failed (no
+  // embedded subset entry + miss handler returned null -- typically
+  // because the SD font doesn't have this style at all, e.g. Readerly_12
+  // ships only REGULAR), fall back to the regular glyph instead of
+  // REPLACEMENT_GLYPH. The result is italic/bold text rendered in the
+  // regular style -- visually a small downgrade but readable. Previously
+  // these codepoints became '?' which broke ~30% of text in books with
+  // mixed-style italic phrases. We DON'T retry through the regular style's
+  // miss handler -- recurse via getGlyphData(cp, REGULAR) which does the
+  // full embedded + miss-handler lookup on the regular style.
+  if (style != REGULAR) {
+    if (const GlyphData regularData = getGlyphData(cp, REGULAR); regularData.glyph) {
+      return regularData;
+    }
+  }
+
+  // CrumBLE 4.3: typography substitutions. Some SD fonts (Readerly_12,
+  // ChareInk_*) don't ship Unicode punctuation glyphs -- curly quotes,
+  // dashes, ellipsis etc. EPUBs use these heavily (smart-quoted dialogue,
+  // em-dashed asides). Without substitution they render as REPLACEMENT
+  // GLYPH ('?'), which dominates the page on dialogue-heavy chapters.
+  // ASCII equivalents are guaranteed to be in any font (a-z + ASCII
+  // punctuation), so we map the smart character to its straight equivalent
+  // and retry the lookup. Trade-off: typography downgrade (curly quotes
+  // become straight, em dashes become hyphens) but the text is readable.
+  uint32_t substitute = 0;
+  switch (cp) {
+    case 0x2018:  // LEFT SINGLE QUOTATION MARK
+    case 0x2019:  // RIGHT SINGLE QUOTATION MARK
+    case 0x201A:  // SINGLE LOW-9 QUOTATION MARK
+    case 0x201B:  // SINGLE HIGH-REVERSED-9 QUOTATION MARK
+      substitute = 0x0027;  // APOSTROPHE
+      break;
+    case 0x201C:  // LEFT DOUBLE QUOTATION MARK
+    case 0x201D:  // RIGHT DOUBLE QUOTATION MARK
+    case 0x201E:  // DOUBLE LOW-9 QUOTATION MARK
+    case 0x201F:  // DOUBLE HIGH-REVERSED-9 QUOTATION MARK
+      substitute = 0x0022;  // QUOTATION MARK
+      break;
+    case 0x2013:  // EN DASH
+    case 0x2014:  // EM DASH
+    case 0x2015:  // HORIZONTAL BAR
+    case 0x2212:  // MINUS SIGN
+      substitute = 0x002D;  // HYPHEN-MINUS
+      break;
+    case 0x2026:  // HORIZONTAL ELLIPSIS
+      substitute = 0x002E;  // FULL STOP (one dot -- not "..." since the
+                            // page-DOM wordXpos accounts for a single glyph)
+      break;
+    case 0x00A0:  // NO-BREAK SPACE
+    case 0x2009:  // THIN SPACE
+    case 0x200A:  // HAIR SPACE
+    case 0x202F:  // NARROW NO-BREAK SPACE
+      substitute = 0x0020;  // SPACE
+      break;
+    default:
+      break;
+  }
+  if (substitute != 0 && substitute != cp) {
+    if (const GlyphData subData = getGlyphData(substitute, style); subData.glyph) {
+      return subData;
+    }
+  }
+
   if (cp != REPLACEMENT_GLYPH) {
     return getGlyphData(REPLACEMENT_GLYPH, style);
   }

lib/EpdFont/SdCardFont.cpp

@@ -668,12 +668,18 @@ int SdCardFont::prewarm(const char* utf8Text, uint8_t styleMask, bool metadataOn
   // = ~131K comparisons, but in practice pages contain far fewer unique codepoints so the
   // actual cost is much lower. This is dwarfed by SD I/O that follows. Alternatives (hash
   // set, bitmap) exceed the 256-byte stack limit or add template bloat.
-  // Heap-allocated: MAX_PAGE_GLYPHS * 4 = 2048 bytes, too large for stack (limit < 256 bytes)
-  std::unique_ptr<uint32_t[]> codepoints(new (std::nothrow) uint32_t[MAX_PAGE_GLYPHS]);
-  if (!codepoints) {
-    LOG_ERR("SDCF", "Failed to allocate codepoint buffer (%u bytes)", MAX_PAGE_GLYPHS * 4);
-    return -1;
-  }
+  //
+  // CrumBLE 4.3 task #2 follow-up: this buffer is now STATIC (one-time
+  // 2 KB BSS allocation at program start, reused every render). The
+  // previous per-render heap allocation was the operator new[] that
+  // throws bad_alloc post-NimBLE, which terminates the render task on
+  // SD-font + BT. Moving the buffer to BSS removes the per-render heap
+  // pressure entirely. Safety: prewarm() is only called from the render
+  // task (FontCacheManager::PrewarmScope::endScanAndPrewarm); the task
+  // is single-threaded so there's no reentrancy concern. Two SdCardFont
+  // instances cannot prewarm concurrently because the render task is
+  // the sole caller.
+  static uint32_t codepoints[MAX_PAGE_GLYPHS];
   uint32_t cpCount = 0;
 
   const unsigned char* p = reinterpret_cast<const unsigned char*>(utf8Text);
@@ -747,13 +753,13 @@ int SdCardFont::prewarm(const char* utf8Text, uint8_t styleMask, bool metadataOn
   }
 
   // Sort codepoints for ordered interval building
-  std::sort(codepoints.get(), codepoints.get() + cpCount);
+  std::sort(codepoints, codepoints + cpCount);
 
   // Prewarm each requested style
   int totalMissed = 0;
   for (uint8_t si = 0; si < MAX_STYLES; si++) {
     if (!(styleMask & (1 << si)) || !styles_[si].present) continue;
-    totalMissed += prewarmStyle(si, codepoints.get(), cpCount, metadataOnly);
+    totalMissed += prewarmStyle(si, codepoints, cpCount, metadataOnly);
   }
 
   stats_.prewarmTotalMs = millis() - startMs;
@@ -1467,3 +1473,42 @@ bool SdCardFont::miniIs2Bit(uint8_t styleIdx) const {
   if (styleIdx >= kMaxStylesForAccessor) return false;
   return styles_[styleIdx].miniData.is2Bit;
 }
+uint16_t SdCardFont::miniKernLeftEntryCount(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return 0;
+  return styles_[styleIdx].miniKernLeftEntryCount;
+}
+uint16_t SdCardFont::miniKernRightEntryCount(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return 0;
+  return styles_[styleIdx].miniKernRightEntryCount;
+}
+uint8_t SdCardFont::miniKernLeftClassCount(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return 0;
+  return styles_[styleIdx].miniKernLeftClassCount;
+}
+uint8_t SdCardFont::miniKernRightClassCount(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return 0;
+  return styles_[styleIdx].miniKernRightClassCount;
+}
+uint16_t SdCardFont::miniLigaturePairCount(uint8_t styleIdx) const {
+  // CrumBLE 4.3 v2: ligature pairs are FONT-WIDE (header), not per-style.
+  // Returned once for any installed style so the emit code can write the
+  // same ligature table per style slot.
+  if (styleIdx >= kMaxStylesForAccessor) return 0;
+  return styles_[styleIdx].header.ligaturePairCount;
+}
+const EpdKernClassEntry* SdCardFont::miniKernLeftClassesPtr(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return nullptr;
+  return styles_[styleIdx].miniKernLeftClasses;
+}
+const EpdKernClassEntry* SdCardFont::miniKernRightClassesPtr(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return nullptr;
+  return styles_[styleIdx].miniKernRightClasses;
+}
+const int8_t* SdCardFont::miniKernMatrixPtr(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return nullptr;
+  return styles_[styleIdx].miniKernMatrix;
+}
+const EpdLigaturePair* SdCardFont::miniLigaturePairsPtr(uint8_t styleIdx) const {
+  if (styleIdx >= kMaxStylesForAccessor) return nullptr;
+  return styles_[styleIdx].ligaturePairs;
+}

lib/EpdFont/SdCardFont.h

@@ -136,6 +136,18 @@ class SdCardFont {
   int16_t miniAscender(uint8_t styleIdx) const;
   int16_t miniDescender(uint8_t styleIdx) const;
   bool miniIs2Bit(uint8_t styleIdx) const;
+  // CrumBLE 4.3 v2: kerning + ligature accessors for the embedded subset's
+  // v2 per-style payload (prebake CLI consumes these to write the kern
+  // matrix / class tables / ligature pairs alongside the bitmap blob).
+  uint16_t miniKernLeftEntryCount(uint8_t styleIdx) const;
+  uint16_t miniKernRightEntryCount(uint8_t styleIdx) const;
+  uint8_t miniKernLeftClassCount(uint8_t styleIdx) const;
+  uint8_t miniKernRightClassCount(uint8_t styleIdx) const;
+  uint16_t miniLigaturePairCount(uint8_t styleIdx) const;
+  const EpdKernClassEntry* miniKernLeftClassesPtr(uint8_t styleIdx) const;
+  const EpdKernClassEntry* miniKernRightClassesPtr(uint8_t styleIdx) const;
+  const int8_t* miniKernMatrixPtr(uint8_t styleIdx) const;
+  const EpdLigaturePair* miniLigaturePairsPtr(uint8_t styleIdx) const;
 
  private:
   // Per-style metadata (parsed from file header/TOC)

lib/Epub/Epub/EmbeddedGlyphSubset.h

@@ -44,13 +44,16 @@
 // .cpfont's bitmap blob). The prebake CLI rebases them when emitting; the
 // loader doesn't need to translate further.
 //
-// Kerning + ligatures are NOT yet embedded in v1 of this block; they would
-// add ~kernLeftEntries*3 + kernRightEntries*3 + leftClasses*rightClasses +
-// ligaturePairs*8 bytes per style (~5-15 KB on typical fonts). For the
-// vertical-slice scope, kerning falls back to "no kerning applied" when an
-// embedded subset is active. A future v2 of this block can add them
-// without disturbing v1 readers (extend Style header with optional offsets
-// to kern + ligature blobs).
+// v2 (CrumBLE 4.3 second pass): kerning + ligatures are now embedded.
+// Per-style header gained 8 bytes of count fields and the per-style data
+// blob is followed by kernLeftEntries, kernRightEntries, kernMatrix, and
+// ligaturePairs in that order. Adds ~700 bytes-2 KB per style depending
+// on font density. Fixes the "text overshoots viewport / Outside range
+// (480, *)" floods that happened on SD-font + BT books because the
+// runtime was using glyph-only advance widths without the pair-kerning
+// offsets the prebake CLI baked into wordXpos. v1 (kerning-less) sections
+// are rejected by the v2 loader -- bump triggered a full re-bake of all
+// section caches anyway, so callers must regenerate prebake artifacts.
 
 #include <cstdint>
 
@@ -59,7 +62,7 @@
 namespace embeddedGlyphSubset {
 
 constexpr uint32_t BLOCK_MAGIC = 0x4253474C;  // "LGSB" little-endian
-constexpr uint16_t BLOCK_VERSION = 1;
+constexpr uint16_t BLOCK_VERSION = 2;
 constexpr uint8_t MAX_STYLES = 4;
 
 // Per-style flag bits.
@@ -76,8 +79,10 @@ struct BlockHeader {
 } __attribute__((packed));
 static_assert(sizeof(BlockHeader) == 16, "EmbeddedGlyphSubset::BlockHeader must be 16 bytes");
 
-// On-disk per-style entry header (24 bytes). Followed by intervals, glyphs,
-// and bitmaps in that order.
+// On-disk per-style entry header (32 bytes). Followed by intervals, glyphs,
+// bitmaps, kernLeftEntries, kernRightEntries, kernMatrix, ligaturePairs in
+// that order. The four kern* and ligature* count fields drive the size of
+// each blob; install code reads them and resizes the matching slot vectors.
 struct StyleHeader {
   uint8_t styleId;
   uint8_t flags;
@@ -89,7 +94,14 @@ struct StyleHeader {
   int16_t ascender;
   int16_t descender;
   uint16_t reserved2;
+  // v2 additions: kerning + ligature counts. Blobs of corresponding sizes
+  // come after the bitmap, in the listed order.
+  uint16_t kernLeftEntryCount;
+  uint16_t kernRightEntryCount;
+  uint8_t kernLeftClassCount;
+  uint8_t kernRightClassCount;
+  uint16_t ligaturePairCount;
 } __attribute__((packed));
-static_assert(sizeof(StyleHeader) == 24, "EmbeddedGlyphSubset::StyleHeader must be 24 bytes");
+static_assert(sizeof(StyleHeader) == 32, "EmbeddedGlyphSubset::StyleHeader v2 must be 32 bytes");
 
 }  // namespace embeddedGlyphSubset

lib/Epub/Epub/Page.cpp

@@ -15,7 +15,7 @@ static_assert(TableFragmentRow::MAX_SERIALIZED_CELLS == MAX_TABLE_CELLS_PER_ROW)
 static_assert(PageTableFragment::MAX_SERIALIZED_ROWS == MAX_TABLE_ROWS_PER_FRAGMENT);
 
 template <typename Predicate>
-void renderFilteredPageElements(const std::vector<std::shared_ptr<PageElement>>& elements, GfxRenderer& renderer,
+void renderFilteredPageElements(const std::vector<std::unique_ptr<PageElement>>& elements, GfxRenderer& renderer,
                                 const int fontId, const int xOffset, const int yOffset, Predicate&& predicate) {
   for (const auto& element : elements) {
     if (predicate(*element)) {

lib/Epub/Epub/Page.h

@@ -120,8 +120,13 @@ class PageTableFragment final : public PageElement {
 
 class Page {
  public:
-  // the list of block index and line numbers on this page
-  std::vector<std::shared_ptr<PageElement>> elements;
+  // the list of block index and line numbers on this page.
+  // CrumBLE 4.3: unique_ptr (was shared_ptr) to skip the per-element
+  // control-block heap allocation that bad_alloc terminates the device
+  // on SD-font + BT (each PageLine push_back used to allocate ~24 bytes
+  // for the shared_ptr control block; under post-NimBLE heap fragmentation
+  // that allocation can't find a contiguous slot).
+  std::vector<std::unique_ptr<PageElement>> elements;
   std::vector<FootnoteEntry> footnotes;
   static constexpr uint16_t MAX_FOOTNOTES_PER_PAGE = 16;
 
@@ -144,7 +149,7 @@ class Page {
   // Check if page contains any images (used to force full refresh)
   bool hasImages() const {
     return std::any_of(elements.begin(), elements.end(),
-                       [](const std::shared_ptr<PageElement>& el) { return el->getTag() == TAG_PageImage; });
+                       [](const std::unique_ptr<PageElement>& el) { return el->getTag() == TAG_PageImage; });
   }
 
   // Get bounding box of all images on the page (union of image rects)