feat: sortdb detect external blobs

[francesco-stacks]

Description

A missing piece I forgot to add in the marf-squash-engine PR. marf-squash creates all MARFs with external blobs, so this is needed to make a pruned chainstate work on develop

Applicable issues

• fixes #

Additional info (benefits, drawbacks, caveats)

Checklist

• Test coverage for new or modified code paths
• For new Clarity features or consensus changes, add property tests (see docs/property-testing.md)
• Changelog fragment(s) or "no changelog" label added (see changelog.d/README.md)
• Required documentation changes (e.g., rpc/openapi.yaml for RPC endpoints, event-dispatcher.md for new events)
• New clarity functions have corresponding PR in clarity-benchmarking repo

[coveralls]

Coverage Report for CI Build 27534101156

Coverage increased (+0.2%) to 85.899%

Details

• Coverage increased (+0.2%) from the base build.
• Patch coverage: 1 of 1 lines across 1 file are fully covered (100%).
• 7683 coverage regressions across 125 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

7683 previously-covered lines in 125 files lost coverage.

Top 10 Files by Coverage Loss	Lines Losing Coverage	Coverage
stackslib/src/chainstate/nakamoto/mod.rs	406	84.53%
stackslib/src/config/mod.rs	376	68.96%
stackslib/src/net/mod.rs	310	78.12%
stackslib/src/chainstate/stacks/index/storage.rs	277	82.41%
clarity/src/vm/database/clarity_db.rs	268	82.11%
stackslib/src/chainstate/stacks/miner.rs	252	83.51%
stackslib/src/chainstate/stacks/db/transactions.rs	239	97.15%
stackslib/src/net/inv/epoch2x.rs	229	79.03%
stackslib/src/net/chat.rs	200	93.03%
stackslib/src/chainstate/stacks/db/mod.rs	198	86.23%

---

Coverage Stats

Relevant Lines:	225119
Covered Lines:	193376
Line Coverage:	85.9%
Coverage Strength:	18945171.29 hits per line

---

💛 - Coveralls

[cylewitruk-stacks]

lgtm

[benjamin-stacks]

This LGTM, but do you know the original reason for open_opts.external_blobs?

I'm asking because if someone so explicitly forced the sortition DB to have internal blobs, than that fact might be an implicit assumption elsewhere.

[cylewitruk-stacks]

I'm asking because if someone so explicitly forced the sortition DB to have internal blobs, than that fact might be an implicit assumption elsewhere.

I believe the history is that external blobs was a later addition to the MARF, and the SortitionDB was so small (comparatively) at that point and not suffering performance issues, that they didn't see a point in forcing it to external blobs. I'm speculating, but that would make sense to me.

Otherwise, the MARF handles the "am I external or not" question internally, so it's not something that external callers should be depending on (I would hope that our tests would break if that assumption were false...).

[francesco-stacks]

This LGTM, but do you know the original reason for open_opts.external_blobs?

I'm asking because if someone so explicitly forced the sortition DB to have internal blobs, than that fact might be an implicit assumption elsewhere.

I believe it's like @cylewitruk-stacks says. Even the index was previously using internal blobs and only later added a migration to move them to external. The squash is already running them externally and it's running at chaintip allright