Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

The lineage

A passphrase-gated, Matrix-styled static site that follows one word through every episode of a YouTube show. Every point on the page links to the exact second on YouTube. Which show, and which word, are not written anywhere in this repository.

The site lives in docs/ and is served by GitHub Pages. Everything readable is encrypted in the browser's favour: the data, the transcripts and the search index ship only as AES-256-GCM ciphertext under a key derived from the passphrase.

How it works

  1. scripts/fetch_captions.sh reads work/source.env (channel URL, playlist, title keyword, display names), lists the channel, picks the episodes (scripts/select_episodes.py), and downloads English captions with yt-dlp.
  2. scripts/build_data.py scans the captions for a word and writes a payload to work/.
  3. scripts/build_search.py chunks every transcript into timestamped passages and scripts/embed_search.mjs embeds them with MiniLM (transformers.js in Node) into work/search/. This powers the terminal's find command. Needs cd scripts && npm install once.
  4. scripts/encrypt_site.mjs encrypts all of that (PBKDF2 over the passphrase, then AES-256-GCM) into docs/enc/. Only ciphertext is committed.
  5. docs/index.html asks for the passphrase, decrypts in the browser, and renders the charts and the terminal. No build step for the page itself.

work/ is gitignored and holds every secret and every plaintext intermediate: work/passphrases.txt, work/source.env, work/subjects*.json, work/subject_extractor.js, captions, and data.

Refreshing the data

brew install yt-dlp        # or pipx install yt-dlp
./scripts/fetch_captions.sh
git commit -am "Refresh data" && git push

The download is deliberately slow (a few seconds per video) because YouTube rate-limits the caption endpoint. Re-running the script skips captions that are already in work/subs/, so a refresh only fetches new episodes.

The terminal

The header is an interactive shell. help lists commands. find does semantic search in the browser with a small embedding model (about 25 MB, downloaded once): it searches every passage of every episode, with the word's own moments shown first when they fit. The model runs in a Web Worker so the page stays responsive.

Hosting

In the repository settings, under Pages, set the source to Deploy from a branch, branch main, folder /docs.

Caveats

  • Auto-generated captions occasionally mis-hear a word, so a handful of mentions may be missing or spurious.
  • Episode selection is heuristic: the show's playlist, an episode number in the title, or the show's name in the title, longer than 30 minutes.
  • Timestamps come from the caption cue, so a link may land a second or two before the word is spoken.

About

the lineage

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages