PROGRESS.md

Project: Simple OPDS Catalog (SOPDS) - Go Version

Current Version: 1.3.3

Revision 86 - 2026-07-12

TTS: parallel workers for CLI conversion + intra-op thread cap; Apple Silicon stays CPU (decision 003).

Chased down why a 3× smaller book took ~3× longer on the M5 Pro (1h44m) than the bigger book on fedya's GTX 1070 (35 min): mac5 runs Piper CPU-only — CoreML can't run VITS's data-dependent output length (decision 003) — while fedya uses its real GPU, and fb2-to-wav.sh ran a single daemon on a 15-core machine.

main.rs: optional SOPDS_TTS_THREADS env caps the ORT intra-op thread count per session, so several daemons can run in parallel without oversubscribing.
fb2-to-wav.sh: new WORKERS (default 4) runs N resident daemons over round-robin shards, each capped to ~cores/WORKERS threads, with aggregate progress/ETA by polling produced WAVs. (Also fixed a pipefail + ls-on-empty-glob bug that silently aborted the run before the first WAV existed.)
Benchmarked on mac5 (M5 Pro, 5 P + 10 E cores), 40 real ru chunks: 27 s → 14 s at 4 workers, flat at 5/8/10 — ~2× is the ceiling, set by the performance-core count. So the ~50 h book drops from ~1h44m to ~52 min on mac5; fedya (GPU) stays faster for big books.
Decision doc 003 records (sourced) that Apple-Silicon GPU/ANE for Piper is a dead-end — the VITS duration-predictor length dynamism forces CPU fallback, and ANE would be a wash even if it ran.

Files: sopds-tts-rs/src/main.rs, sopds-tts-rs/fb2-to-wav.sh, docs/decisions/003-apple-silicon-piper-stays-cpu.md, README.md.

Revision 85 - 2026-07-12

fb2-to-wav.sh: output format by extension (default MP3), 4 GB-WAV guard, chunk size in chars.

Two issues surfaced converting a ~50 h book on mac5:

The concatenated audio was 8 GB, but WAV's 32-bit size fields cap a file at 4 GB → ffmpeg wrote a broken header. Fix: output format now follows the extension (mp3/m4b/m4a/opus/ogg are encoded at a speech bitrate; wav is copied), the default is .mp3, and a .wav that would exceed 4 GB auto-switches to .mp3 so you never get a broken file. (An already-broken WAV recovers with ffmpeg -ignore_length 1 -i big.wav -c:a libmp3lame -b:a 64k out.mp3.)
MAXBYTES was a misnomer — gawk length() counts characters, so it always limited chars, not bytes (900 → ~1750 bytes for Cyrillic, near the GPU OOM cliff, and the reason the same-title library book split into 12455 byte-based chunks while the script made 1827 char-based ones). Renamed to MAXCHARS (legacy MAXBYTES still honored); default lowered 900 → 600 chars (GPU-safe — char count tracks phoneme count, which drives attention memory). Files: sopds-tts-rs/fb2-to-wav.sh.

Revision 84 - 2026-07-11

fb2-to-wav.sh: live progress % + elapsed + ETA.

The synth loop now prints N/M (pct%) <elapsed> ETA <dur> on one refreshing line (ANSI clear-to-EOL), extrapolating ETA from the average per-chunk rate so far, plus a final done in <dur> summary. Durations format as 1h02m / 3m05s / 12s. Files: sopds-tts-rs/fb2-to-wav.sh.

Revision 83 - 2026-07-11

sopds-tts-rs: a phoneme-less chunk no longer kills the whole book (silence + no crash).

A 12455-chunk Russian book reached chunk 12454/12455, then the job was marked failed and the UI reverted to "Audio Not Generated" — 12454 good WAVs thrown away. Cause: chunk 909's text had no pronounceable phonemes (a "* * *"-style scene break / bare separator), so synth returned no phoneme IDs generated. On the one-shot fallback that error propagated via ?, which unwind-dropped the ORT Session → the CUDA-teardown core dump (Rev 79's exit_ok guarded only the success path). Go saw the non-zero exit and failed the entire job.

Two fixes in main.rs:

synth now returns ~0.25 s of silence for a phoneme-less chunk instead of an error — a scene break becomes a short pause, and one dud chunk can't fail a whole book. (The daemon path already tolerated synth errors; this makes the result correct too.)
The one-shot arm runs synth+write in a match and calls exit_ok() / exit(1) without dropping the Session on either path, so a genuine synth error exits cleanly (exit 1) instead of core-dumping.

Verified on mac5: empty/whitespace text → ok, 5512-sample (0.25 s) silent WAV; real text unchanged; empty stdin → clean exit 1, no core dump. Files: sopds-tts-rs/src/main.rs (synth, one-shot arm).

Follow-up (not yet done): harden processJob to tolerate any single failed chunk — write a silence placeholder and complete the job rather than discarding all completed work.

Revision 82 - 2026-07-11

sopds-tts-rs/fb2-to-wav.sh — one-command FB2 → single WAV (manual/CLI conversion).

A standalone converter for one-off files (the web UI already chunks; this is the manual path Dmitry asked for). ./fb2-to-wav.sh <book.fb2> <model.onnx> [out.wav]:

extracts the main <body> text (skips <body name="notes"> footnotes) via xmllint,
splits it into ~900-byte sentence-grouped chunks (MAXBYTES overridable) — big chunks OOM the GPU (Rev 81),
synthesizes every chunk through one resident daemon (model loaded once, NDJSON protocol),
concatenates the chunk WAVs into one file via ffmpeg -c copy.

Self-bootstraps its tools (espeak-ng, ffmpeg, jq, xmllint, GNU sed/awk) through nix shell when nix is present, so nothing needs preinstalling; degrades to whatever's on PATH otherwise. Tested on mac5: single- and multi-chunk paths, footnote exclusion, Cyrillic; 2-chunk vs 1-chunk output durations match (~13 s). Files: sopds-tts-rs/fb2-to-wav.sh.

Revision 81 - 2026-07-11

TTS: default chunk_size 5000 → 1000 bytes — the old default OOMs the GPU (real cause of "slow").

A live fedya generation (a large Russian book, ru_RU-irina-medium) ran at ~2.3 s/chunk, ETA 1h42m. The log showed why: every chunk failed on the daemon with a CUDA OOM and fell back to the slow one-shot path (fresh process + CUDA init + 114 MB model reload each time): BFCArena … Failed to allocate memory for requested buffer of size 3035899904 (~3 GB).

Root cause: Piper/VITS attention/upsampling is ~O(n²) in text length (the intermediate is roughly text_len × predicted-audio-frames), and chunk_size defaulted to 5000 bytes. Measured threshold on the GTX 1070 (8 GB, 7.85 GB free): chunks of 576 / 1152 bytes synthesize fine (175 / 535 ms), a 2304-byte chunk requests >7.8 GB and OOMs. So it is not GPU contention — a single chunk's tensor simply doesn't fit.

Fix: lower the default chunk_size to 1000 bytes (config.go default + extractor.go fallback). chunk_size is measured in bytes, and Cyrillic is ~2 bytes/char, so 1000 ≈ 500 ru / ~1000 en chars — a few sentences, ~20–40 s of audio. Confirmed on fedya: realistic EN (362 B → 919 ms) and RU (436 B → 484 ms) chunks run on the daemon with no OOM. Also set chunk_size: 1000 in the live /opt/sopds/config.yaml and restarted. Bigger GPUs can raise it; the O(n²) wall is the only limit.

Note: total synth time is bounded by GPU throughput (audio length is fixed), so the win here is (a) the daemon actually engaging instead of the OOM→one-shot fallback, and (b) better prosody from sentence-scale chunks. Files: internal/config/config.go, internal/tts/extractor.go.

Future option: set the CUDA EP arena to a memory limit / kSameAsRequested, or auto-split a chunk that still OOMs, instead of failing that chunk.

Revision 80 - 2026-07-11

Deploy: sopds-tts-rs live on fedya (CUDA GPU); sopds server rebuilt to Rev 79 for daemon mode.

Completed the fedya (dvg-fedya, GTX 1070 / Pascal) TTS deployment that Rev 79 left as a TODO. The TTS binary is CUDA and the sopds service runs as User=sopds with a minimal PATH and no nix.

Recipe (all on dvg-fedya):

Build the CUDA binary via the flake: cd sopds-tts-rs && nix develop -c cargo build --release (builds cudaPkgs.onnxruntime for sm_61 — heavy the first time, cached after).
Copy it where the service user can exec it (home dir is 0700): sudo install -m755 target/release/sopds-tts-rs /opt/sopds/sopds-tts-rs-bin.
Wrapper — install the file getTTSBinaryPath() looks for, next to /opt/sopds/sopds: /opt/sopds/sopds-tts, a sh script that puts espeak-ng (nix store) on PATH and execs the binary. libonnxruntime + cuDNN/cuBLAS resolve via the binary's baked RPATH; libcuda from /run/opengl-driver/lib (NixOS default loader path). No LD_LIBRARY_PATH needed.
Pin those store paths against nix-GC: nix develop sopds-tts-rs --profile ~/.local/state/sopds-tts-devshell -c true (indirect gcroot).
Server — rebuild the Go server from Rev 79 (has the daemon integration) and swap it in. The server needs CGO: nix shell nixpkgs#gcc -c go build -o /tmp/sopds-new ./cmd/sopds, then sudo systemctl stop sopds && sudo cp -a /opt/sopds/sopds /opt/sopds/sopds.bak-pre-rev78 && sudo install -m755 /tmp/sopds-new /opt/sopds/sopds && sudo systemctl start sopds.

Regenerate steps 1–4 together if sopds-tts-rs/flake.lock changes (espeak path in the wrapper + the gcroot). Verified: binary runs on the GPU (nvidia-smi → 368 MiB); one-shot and daemon both exit 0 (no more CUDA-teardown core dump, Rev 79); the wrapper works under sudo -u sopds with a systemd-minimal env; the redeployed server logs TTS: Starting 2 workers on boot. Rollback: /opt/sopds/sopds.bak-pre-rev78. Left for the operator: a UI-triggered generate to confirm the resident daemon end-to-end (TTS routes are session-authed).

Revision 79 - 2026-07-11

sopds-tts-rs: fix the fedya "TTS starts and dies" crash — exit before the ORT Session drops:

Root cause of fedya's TTS dying immediately: the binary synthesized the WAV correctly, then core-dumped (corrupted double-linked list) the instant the Tts struct — holding the ONNX Runtime CUDA Session — was dropped at end of scope. ORT's CUDA-provider static teardown corrupts the heap on this box (onnxruntime-1.22 CUDA build, cuDNN 9.8, GTX 1070 / Pascal). In one-shot mode that non-zero exit (134, core dumped) made sopds-go treat a perfectly good chunk as a failure — hence "starts and dies very quickly".

Fix: added exit_ok() (flush stdout, std::process::exit(0)) and call it while the Session is still alive — at the end of the one-shot arm and at daemon EOF — so the Session's Drop (the crashing CUDA teardown) never runs. The WAV is already finalized on disk and any daemon responses already sent/flushed, so nothing is lost. No-op on the clean-exiting macOS/CPU path.

Verified on fedya (CUDA): one-shot now exits 0 with a valid WAV; daemon mode answers 3 chunks (492 / 215 / 115 ms) and exits 0 on EOF. Files: sopds-tts-rs/src/main.rs (main, new exit_ok, one-shot arm, serve EOF).

Still TODO for fedya (deploy, not code): the sopds service (User=sopds, /opt/sopds/sopds) finds the TTS binary via getTTSBinaryPath() = sopds-tts next to the exe or on PATH — and /opt/sopds/ has no such binary, so spawns fail. Needs a sopds-tts wrapper on that box that provides the ORT/cuDNN/espeak-ng runtime env (the service PATH has no nix).

Revision 78 - 2026-07-11

sopds-go TTS: use the resident daemon (Rev 77) instead of one process per chunk:

Wired the daemon protocol into internal/tts — this is what turns Rev 77 into a UI-visible speedup. New daemon.go: a daemonPool keeps up to Workers live sopds-tts <model> processes per model, reused across chunks and jobs. generateChunk now prefers the pool (NDJSON request → WAV) and transparently falls back to the old one-shot subprocess (generateChunkOneShot) when a daemon can't start — an old binary without daemon mode, or espeak-ng missing. The failure is detected once per model (a broken flag on the model's pool), so there's no per-chunk retry storm; a daemon that dies mid-stream is discarded and respawned. The pool is created in Start() (sized to Workers) and shut down when the workers stop.

Integration test daemon_test.go (skipped unless SOPDS_TTS_BIN/SOPDS_TTS_MODEL are set and espeak-ng is on PATH) drives 4 chunks through the pool in parallel: 0.52 s for 4 chunks with 2 daemons (model loaded once each) vs ~1.6 s as 4 separate one-shot processes. For a full book (dozens of chunks) the two model loads amortize to near-zero, so per-chunk cost approaches the ~20–90 ms measured in Rev 77.

Files: internal/tts/daemon.go (new), internal/tts/daemon_test.go (new), internal/tts/generator.go (pool field; Start/shutdown wiring; generateChunk daemon-first with one-shot fallback, old body renamed generateChunkOneShot).

Revision 77 - 2026-07-11

sopds-tts-rs: daemon mode — load the model once, then ~20–90 ms per chunk (no GPU):

The one-shot contract sopds-tts-rs <model> <output> reloads the ~60 MB ONNX model on every call. Benchmarking showed that reload (~0.34 s) — not synthesis — is the dominant cost: actual inference is ~0.02–0.09 s per sentence on the M5 Pro CPU. So the win isn't the GPU (CoreML can't run this VITS model anyway, see Rev 76); it's a resident model.

Added a backward-compatible daemon mode:

sopds-tts-rs <model> <output> — unchanged one-shot (text on stdin, WAV out).
sopds-tts-rs <model> (no output arg) — loads model + session once, prints ready: … to stderr, then reads NDJSON requests on stdin, one per line: {"text":"…","output":"/p.wav"}. For each it synthesizes + writes the WAV and emits one NDJSON response: {"ok":true,"samples":N,"elapsed_ms":M,"output":"…"} (or {"ok":false,"error":"…"}). EOF → exit.

Refactored load/synth into a Tts struct (load() once → synth(text) -> Vec<i16>), shared by both modes. Measured on mac5, 5 sentences in one daemon session: 86 / 76 / 19 / 60 / 70 ms each, vs ~340 ms per separate one-shot call — sub-0.1 s per chunk, ×4–15 faster, CPU only.

Files: sopds-tts-rs/src/main.rs (Tts struct, serve() daemon, run() dispatch on argc). TODO (next rev): internal/tts/generator.go should keep one daemon per model alive and speak this NDJSON protocol instead of spawning a process per chunk — that's what turns this into a UI-visible speedup.

Revision 76 - 2026-07-11

sopds-tts-rs: build + run on macOS (Apple Silicon) — one repo, auto per platform:

The Rust CUDA TTS port (sopds-tts-rs/) now also builds and runs on macOS. Cargo.toml selects the ONNX Runtime execution provider per target via [target.'cfg(...)'.dependencies] (CUDA on Linux — unchanged, lib via ORT_LIB_LOCATION/run-gpu.sh; download-binaries on macOS), and main.rs picks the provider with cfg(target_os). cargo build --release now Just Works on both: CUDA binary on fedya, CPU binary on mac5.

macOS runs on CPU. CoreML — the only ORT GPU path on Apple Silicon — can't run this Piper/VITS model: its inputs have dynamic (unbounded) dimensions that CoreML rejects (NeuralNetwork format segfaults; MLProgram floods unbounded dimension errors and falls back to CPU anyway). Not worth chasing — Apple-Silicon CPU is ~13× real-time (13.8 s of audio in 1.08 s on an M5 Pro). Re-add the coreml feature only if the model is re-exported with bounded shapes.
No change to the Linux/CUDA path.

Revision 75 - 2026-05-11

v1.3.3 patch — resend-verification + SMTP password_env + docs cleanup:

Bundle of features and fixes since v1.3.2 (Rev 70). New user-visible behavior plus the usual archive-rebuild for any users who pull from GoReleaser/Homebrew/GHCR.

Features (vs. v1.3.2):

Rev 73 — /web/resend-verification flow with 60s per-user cooldown, login-page link, email-enumeration-safe responses.
Rev 74 — smtp.password_env: for secret-store integration (sops/k8s/vault/docker-env).

Docs/fixes (vs. v1.3.2):

Rev 71-72 — docker-compose recipe + Taskfile helpers for the typical ~/dockers/sopds/ deployment pattern.
Migration 013 — verify_token_sent_at column for resend cooldown state.
Various config.yaml.example fixes: SMTP block, JWT secret, auth.users [] placeholder, port aligned to 8081, library.root container-vs-host distinction documented in README.
init.sql removed from compose recipe (POSTGRES_DB env var handles bootstrap; init.sql is for standalone PG only).
PG volume mount path corrected for PG18+ layout (/var/lib/postgresql instead of /.../data).
PostgreSQL bumped 16 → 18 in Homebrew formula dep.

Migration note: Existing v1.3.x installations need sopds migrate (or task migrate in the docker stack) on first start with v1.3.3 to apply migration 013. Users running first-time install just get the column for free at bootstrap.

Auto-triggered on tag push:

GoReleaser builds 4 archives (linux/macos × x86_64/arm64) + checksums.txt.
Docker images republished as ghcr.io/dimgord/sopds-go:1.3.3 and :latest.
Homebrew formula auto-updated in dimgord/homebrew-tap to v1.3.3.

Revision 74 - 2026-05-11

SMTP password_env: env-var indirection — keep secrets out of config files:

smtp.password: in config.yaml lands secrets on disk (and into any committed-or-mounted version of that file). Not OK for production deployments where the SMTP credential should come from sops, k8s secrets, Vault, or docker env. Added generic env-var indirection:

smtp:
  password: ""                              # leave blank
  password_env: "SOPDS_SMTP_PASSWORD"       # name of env var holding the secret

At startup sopds calls SMTPConfig.ResolvedPassword() which checks os.Getenv(c.PasswordEnv) first, falls back to c.Password if the env var is unset/empty. Existing deployments using password: continue to work unchanged.

Files Modified:

internal/config/config.go: new PasswordEnv string field on SMTPConfig; new method ResolvedPassword() string (env-lookup with fallback).
internal/server/email.go: one-line change — e.config.Password → e.config.ResolvedPassword() in the auth setup.
config.yaml.example: SMTP block documents both options (a) literal password and (b) password_env indirection, with the rationale and fallback semantics.
docker-compose.example.yml: added environment: SOPDS_SMTP_PASSWORD: ${SOPDS_SMTP_PASSWORD:-} passthrough on the sopds service, with comment showing the sops exec-env … docker compose up -d deploy pattern.

Deploy pattern with sops-nix secrets:

cd ~/dockers/sopds
sops exec-env ~/.nixfiles/secrets/secrets.yaml \
  'SOPDS_SMTP_PASSWORD="$email_app_password" docker compose up -d'

sops exec-env decrypts secrets.yaml in-memory and exports each key as an env var for the wrapped command; docker compose's ${SOPDS_SMTP_PASSWORD} interpolation then forwards it into the container's env; sopds reads it on startup via os.Getenv. Plaintext secret never touches disk.

Works equally well with .env file (compose auto-loads), k8s secrets: mount → envFrom:, AWS ECS task secrets, etc. — env-var is the lowest common denominator.

Verified: go build ./... clean; go test ./... all 14 packages pass.

Revision 73 - 2026-05-11

Resend-verification-email flow:

SendVerificationEmail was already wired into the signup path (auth.go:378 since Rev ), but there was no way for a user to request a new email if the first one was lost or expired. Added a complete resend flow with per-user rate-limiting and email-enumeration protection.

Domain layer (internal/domain/user/user.go):

New field VerifyTokenSentAt *time.Time on User — tracks when the most recent verification email was sent.
New method RegenerateVerifyToken() error — replaces the existing token with a fresh value (new 24h expiry), invalidating any previous email link.
New method CanResendVerification() (allowed bool, remaining time.Duration) — returns whether the cooldown has elapsed and how long to wait. Returns (true, 0) for users without a sent-at timestamp (pre-migration accounts).
New const VerifyResendCooldown = 60 * time.Second — short enough to be user-friendly when an email is lost, long enough to make burst-spam unattractive.
NewUser now sets VerifyTokenSentAt = &now so the cooldown clock starts at signup.
VerifyEmail clears VerifyTokenSentAt to nil (irrelevant once verified).

Persistence (internal/infrastructure/persistence/user_repository.go):

New column VerifyTokenSentAt *time.Time (GORM tag column:verify_token_sent_at).
Updated userToModel and modelToUser converters to round-trip the new field.

Migration (migrations/013_verify_resend.sql):

ALTER TABLE users ADD COLUMN IF NOT EXISTS verify_token_sent_at TIMESTAMP.
Index idx_users_verify_sent_at (partial, WHERE email_verified = FALSE) for future nag-cron jobs.
Existing users get NULL, treated as "no recent send" → can resend immediately.

Handler (internal/server/auth.go::HandleResendVerification):

GET → renders form asking for email.
POST → looks up user by email; if exists + not verified + cooldown OK → regenerate token, save, send email.
Email-enumeration safe: every outcome below the form-validation level renders the same generic "if an account exists, a new link was sent" message. Doesn't matter whether the email exists, is already verified, or the SMTP send itself failed — no information disclosure.
Per-user cooldown rejection IS shown to the caller (form-error with seconds remaining) — there's no enumeration risk because cooldown only triggers when a real account was found, but the cooldown response only fires AFTER the lookup; we accept that minor info-leak in exchange for a sane UX. (An attacker can still discover this by sending two requests in quick succession to the same email and observing the response delta — but at that point they've also gotten two genuine SMTP sends.)

Routes (internal/server/server.go):

r.Get(WebPrefix+"/resend-verification", HandleResendVerification) — show form.
r.With(RateLimitMiddleware(checkRateLimiter)).Post(WebPrefix+"/resend-verification", HandleResendVerification) — process. The HTTP rate limiter (per-IP) layers on top of the per-user 60s cooldown.

Template (internal/server/auth_templates.go):

New resend-verification template, modelled on the forgot-password form.
Login page now links to it: Forgot password? · Resend verification email next to the existing forgot-password link, so users who hit the "Please verify your email first" error have a one-click path forward.

Files Modified:

internal/domain/user/user.go — new field + 2 methods + 1 const.
internal/infrastructure/persistence/user_repository.go — column + converters.
internal/infrastructure/persistence/migrations/013_verify_resend.sql — new file.
internal/server/auth.go — HandleResendVerification (~70 lines), renderResendVerificationPage, fmt import.
internal/server/auth_templates.go — new template + login-page link.
internal/server/server.go — 2 new route lines.

Verified: go build ./... clean; go test ./... all 14 packages pass.

Revision 72 - 2026-05-11

Taskfile.docker.example.yml — task-runner recipes for the docker-compose stack:

Sister file to docker-compose.example.yml (Rev 71). Bundles common operational commands so end-users don't need to memorize docker compose ... invocations or docker exec -it sopds ... -c /etc/sopds/config.yaml boilerplate.

Task groups:

Stack lifecycle: up, down, nuke (with prompt: confirmation since it deletes the data volume), restart (sopds only — keeps PG warm), restart-all, pull, update (pull + up = rolling upgrade).
Observability: ps, logs (sopds), logs-pg, logs-all.
sopds CLI (via docker exec; image is distroless so calls are direct to /usr/local/bin/sopds rather than through a shell): status, version, scan, migrate.
Postgres: psql (interactive shell into the DB), psql-exec -- "<SQL>" (one-shot), db-size (DB size + top-20 table row counts), shell-pg (debug shell into PG container).
Backup / restore: backup (pg_dump custom-format → ./backups/sopds_<ts>.dump), backup-sql (plain SQL, human-readable), restore -- backups/<file>.dump (pg_restore with --clean --if-exists).

Why ship in the repo as *.example.yml:

Same logic as docker-compose.example.yml — these are starter templates users curl down alongside their stack. Naming with .example.yml keeps it from being confused with the repo's own dev Taskfile.yml (which has go build / go test / etc. for sopds-go development, irrelevant to a runtime deployment).

README docker recipe updated to curl this third file alongside the compose + config + (mv it to Taskfile.yml), with a one-line pointer to the task list.

Files Modified:

Taskfile.docker.example.yml (new, ~100 lines including section comments and prompt'd nuke).
README.md: +3 lines in the Docker recipe block + a paragraph about task discoverability.

Revision 71 - 2026-05-10

docker-compose.example.yml + README compose-stack instructions:

The single-container docker run example in README assumes the user already has PostgreSQL running somewhere — true for some deployments but not the common "spin up a fresh stack in ~/dockers/sopds/" pattern. The README didn't cover the latter explicitly; this Rev fills that gap.

This Rev adds:

docker-compose.example.yml at repo root: two services (sopds + postgres:18-alpine), named volume for PG data, healthcheck-gated depends_on, init.sql mounted into /docker-entrypoint-initdb.d/, configurable PG password via SOPDS_PG_PASSWORD env var. Comments document required edits before first up -d (library path, password sync between compose env and config.yaml).
README Docker section expansion: split into "Single container" (existing curl-like example) and "Compose stack" (new — three-curl bootstrap of compose+config+init.sql, plus the first-scan command). End-user gets a 5-command path from zero to running stack.

Why postgres:18-alpine specifically:

Matches the bumped recommended dep from Rev 69 (Homebrew formula's postgresql@18).
Alpine variant is lean (~80MB vs ~150MB Debian-based) — appropriate since we don't need OS-level tooling, just postgres.
PostgreSQL 18 is current stable as of 2026-05.

Compose semantics:

depends_on.postgres.condition: service_healthy makes sopds wait for pg_isready before starting, eliminating the "first start fails because DB isn't up yet" footgun common in naive sopds+pg stacks.
Named volume sopds-pg-data (not bind-mount) — Postgres data lives in Docker-managed storage; survives compose down, nuked by compose down -v.
restart: unless-stopped for both services — survives host reboot but respects manual docker stop.

Pre-publication checklist update:

Add docker-compose.example.yml to GoReleaser archive bundles? Not done in this Rev — the file is small and easily curl-able from raw.githubusercontent. Could be added later if download-traffic patterns justify it.

Files Modified:

docker-compose.example.yml (new, ~55 lines including comments).
README.md: Docker section expanded ~15 lines.

Revision 70 - 2026-05-10

v1.3.2 patch — push Rev 68/69 fixes through to Homebrew formula:

The Homebrew formula at dimgord/homebrew-tap/Formula/sopds-go.rb is auto-generated by GoReleaser on each v* tag. v1.3.1 was tagged before Rev 69 (PostgreSQL bump 16→18) landed, so users running brew install sopds-go are still pulling postgresql@16 as a dependency.

This Rev re-tags as v1.3.2 to push the formula update through the same release pipeline. No code changes since v1.3.1; the only delta is:

Homebrew dep: postgresql@16 → postgresql@18 (Rev 69)
nix run github:dimgord/sopds-go works for end users now (Rev 68 — the v1.3.1 tag has the broken path:./sopds-tts-rs input). Tagging v1.3.2 makes nix run github:dimgord/sopds-go/v1.3.2 -- start work; the unversioned nix run github:dimgord/sopds-go already worked since Rev 68 went on main.

End-user upgrade path:

brew update && brew upgrade sopds-go    # picks up the new postgresql@18 dep
docker pull ghcr.io/dimgord/sopds-go:1.3.2
nix run github:dimgord/sopds-go/v1.3.2 -- version

Pipeline verification (post-tag):

GoReleaser regenerates Formula/sopds-go.rb with version "1.3.2" and four new download URLs/SHA-256 pairs.
Docker images :1.3.2-amd64, :1.3.2-arm64, :1.3.2, :latest published.
GitHub Release page lists 4 archives + checksums.txt.

Revision 69 - 2026-05-10

PostgreSQL bump 16 → 18 across all configs:

PG 18 was released 2025-09-25 and is the current stable major as of 2026-05. PG 16 is still supported upstream (until Nov 2028) but trails three majors behind; for an OSS project being published today, recommending the current major is the right default. Nixpkgs and Homebrew both have postgresql_18 / postgresql@18 available.

Changes:

flake.nix devShell: postgresql_16 → postgresql_18 (client tools — psql, pg_dump, pg_restore — for migrations/backups). Verified via nix develop --command psql --version → psql (PostgreSQL) 18.3.
.goreleaser.yaml brews block:
- dependencies.postgresql@16 → postgresql@18 (recommended dep installed alongside sopds-go via brew).
- caveats step 1: brew services start postgresql@16 → postgresql@18.
README.md Requirements: PostgreSQL 14+ → PostgreSQL 16+ (raised the lower bound — features used by sopds-go don't strictly require >14, but no point shipping with an unsupported floor); explicit "18 recommended" note pointing at the brew dep.
CLAUDE.md Requirements: Go 1.21+ → Go 1.25+ (catches Rev 62 toolchain bump that was missed in the dev-doc); PostgreSQL 12+ → PostgreSQL 16+ with the same "18 recommended" note.

Compat notes:

Schema in init.sql and migrations works on PG 16+ — the only PG-15-or-newer feature used is "Grant schema permissions" (per the comment in init.sql:13). PG 12-14 would need additional GRANTs which we never tested; raising the floor to 16 makes the support story honest.
No code changes needed in Go — gorm.io/driver/postgres autodetects server version.

For users on existing PG 14/15/16 installs: keep using your current Postgres; no schema migration needed. The bump is just the recommended default for fresh installs.

Files Modified:

flake.nix: 1 line (postgres pkg name).
.goreleaser.yaml: 2 lines (dep name + caveat).
README.md: 1 line (Requirements bullet).
CLAUDE.md: 2 lines (Go + PG bullets).

Revision 68 - 2026-05-10

Drop sopds-tts-rs from root flake.nix inputs — fixes nix run github:dimgord/sopds-go for end users:

Symptom: nix run github:dimgord/sopds-go -- status failed for an end-user invocation with:

error: cannot write modified lock file of flake 'github:dimgord/sopds-go'
       (use '--no-write-lock-file' to ignore)

Root cause: Rev 60 composed sopds-tts-rs into the root flake via a path:./sopds-tts-rs input. path: inputs in flakes lock to a specific narHash + lastModified pair tied to the parent flake's working-tree position. When the root flake is fetched from GitHub (read-only), Nix tries to re-resolve the path: to the new fetch location and discovers the narHash drifts — wants to update the lock, but can't write to a remote read-only flake. Fail.

Fix: drop the input entirely. The tts-rs devShell that re-exported sopds-tts-rs.devShells.${system}.default is removed; users who want the Rust + CUDA workflow now cd sopds-tts-rs && nix develop ./ from a local clone (the sopds-tts-rs flake remains self-contained and works fine — the issue is only with composing it under a different parent flake that's then consumed remotely).

Trade-off: nix develop from repo root no longer gives you the Rust toolchain in one shot. Have to nest. Acceptable — the Rust+CUDA stack is heavy (cuDNN 9.8 pin from Jun-2025 nixpkgs, ~15-30min source build of ORT) and most users of sopds-go don't need it. Those who do are already cd-ing into the subproject.

Verified after fix:

nix flake update removed sopds-tts-rs and its 4 transitive inputs (nixpkgs, nixpkgs-cuda, rust-overlay, rust-overlay/nixpkgs) from flake.lock.
nix build .#sopds --no-link builds clean (only depends on nixpkgs + flake-utils now).
nix run github:dimgord/sopds-go -- version will work for end users once this Rev is on main.

Files Modified:

flake.nix: -25 lines (input declaration + devShells..tts-rs entry + the pkgs.lib.optionalAttrs Linux gate). Updated comment block at top of inputs explaining the rationale so future-me doesn't re-introduce the same trap.
flake.lock: 5 inputs auto-removed by nix flake update.
PROGRESS.md: this entry.

Revision 67 - 2026-05-10

Fix flake.nix vendorHash — nix run github:dimgord/sopds-go now actually works:

Rev 60 introduced the root flake.nix but with vendorHash = "sha256-AAAAA…AAAAA" (lib.fakeHash placeholder), so any nix build would fail until the real Go-modules vendor hash was substituted. Resolved this Rev.

Mechanism:

nix build .#sopds --no-link printed the expected hash:

hash mismatch in fixed-output derivation '…sopds-go-modules.drv':
    specified: sha256-AAAAA…
          got: sha256-/Bws/W3fmXls7i+4GN26S4kJt/+cpf9sKIqLaWhIImA=

Substituted into commonGoArgs.vendorHash. Re-ran nix build .#sopds → built clean.
Smoke test: nix run .#sopds -- version → SOPDS dev-dirty (Go rewrite) (the "dev-dirty" suffix is correct — it's self.dirtyRev because the working tree was uncommitted at build time).

nix run github:dimgord/sopds-go from any user is now functional. The version string in their build will reflect the git rev they're tagging from, e.g. v1.3.1 users get SOPDS v1.3.1 (per the ${version} substitution in commonGoArgs.ldflags).

Caveat — vendorHash drift:

Hash is computed from go.sum + go.mod. Any dep change → must bump this hash again.
CI does not currently catch hash drift; would need a nix flake check step in .github/workflows/ci.yml. Filed as future work — when it surfaces (e.g. dependabot opens a Go-deps PR and nix build breaks for downstream users), add the CI step.
For human bumps: nix build .#sopds 2>&1 | grep got: extracts the new hash; copy it into flake.nix and commit.

Files Modified:

flake.nix: 4-line comment block + 1-line hash. Net +1 line.

Pre-publication checklist update:

Nix flake fully working (vendorHash filled in)

Revision 66 - 2026-05-10

v1.3.1 patch — re-publish release archives with Rev 64+65 fixes baked in:

v1.3.0 release archives (linux/macos × x86_64/arm64 tar.gz) shipped with the Rev-54/55 mis-attribution in the bundled NOTICE.md and an unhelpful sopds status error message. Both fixed in Rev 64+65 on main; this Rev re-tags as v1.3.1 to push corrected docs and the status UX fix into the GitHub Release / Docker image / Homebrew formula.

What's different vs. v1.3.0:

NOTICE.md: correct upstream attribution (Dmitry V. Shelepnev, © 2014, admin@sopds.ru) instead of hallucinated "V.A. Onishchenko"; dead www.sopds.ru and sergey-dryabzhinsky/sopds links removed.
README.md: same attribution fixes + new "Related projects" section linking to fbe-go.
sopds binary: status command distinguishes ESRCH/EPERM and shows PID file path (Rev 65).
No new features. Pure patch-level release.

Auto-triggered on tag push:

GoReleaser builds 4 archives + sha256 checksums.
Docker image republished as ghcr.io/dimgord/sopds-go:1.3.1 and :latest.
Homebrew formula auto-updated in dimgord/homebrew-tap/Formula/sopds-go.rb.

Revision 65 - 2026-05-10

sopds status UX fix — distinguish ESRCH from EPERM, show PID file path:

Symptom: ps -ef | grep sopds on a deployment shows sopds 2260 1 0 … /opt/sopds/sopds start (process running for days). But /opt/sopds/sopds status reports "SOPDS server is not running (process not found)" — even with explicit -c <config.yaml>.

Root causes (two separate issues, both surfacing through the same symptom):

Process is running under a daemon user; status invoked from a different account lacks permission to signal it. os.FindProcess on Linux always succeeds (it just constructs a os.Process struct); the actual liveness check is process.Signal(syscall.Signal(0)), which on Linux returns EPERM (permission denied) when the process exists but is owned by another user, and ESRCH (no such process) when the PID is stale. The pre-Rev-65 code lumped both into "process not found", erasing the diagnostic distinction.
PID file path differs between configs. When sopds status is invoked without -c, the binary loads default config which has empty Scanner.PIDFile → falls back to /tmp/sopds.pid. A production sopds started under a service account with its own config writes its PID file elsewhere (e.g. /opt/sopds/sopds.pid). The two-config situation is correct architecture; the bug is that the error message gives no hint which PID file path was being read, leaving the user to guess.

Fix in cmd/sopds/main.go::runStatus:

Distinguish ESRCH (stale PID) vs EPERM (running but not yours) vs other errors via errors.Is(err, syscall.ESRCH/EPERM).
Print the PID file path in every "not running" branch so the user immediately sees where status was looking.
strings.TrimSpace the PID file content before strconv.Atoi — some pid files have trailing newline.

New status outputs:

Condition	Pre-Rev-65 message	Post-Rev-65 message
PID file absent	`not running (no PID file)`	`not running (no PID file at <path>)`
PID file unparseable	`status unknown (invalid PID file)`	`status unknown (invalid PID file content: %q)`
PID file has stale PID	`not running (process not found)`	`not running (stale PID N in <path>)`
Process exists but owned by other user	`not running (process not found)`	`running (PID: N) — owned by another user; run as that user or root to control it`
Process exists, signaled OK	`running (PID: N)`	`running (PID: N)` (unchanged)
Other syscall error	`not running (process not found)`	`status check failed: <err>`

Files Modified:

cmd/sopds/main.go: imports errors (new); runStatus rewritten as switch on signal(0) error class. ~25 lines net.

Revision 64 - 2026-05-10

Fix incorrect upstream attribution + drop dead links:

In Rev 54 (README) and Rev 55 (NOTICE.md) I attributed the original Simple OPDS Python project to "V.A. Onishchenko" — that was a hallucination. The actual author per the LICENSE header (Russian text, sourced from a local copy of the upstream Python source) is Dmitry V. Shelepnev (Дмитрий Шелепнёв), © 2014, contact admin@sopds.ru, version 0.23.

I also linked to a sergey-dryabzhinsky/sopds GitHub repo as an "active fork" — that link returns 404 (verified with curl). The upstream homepage www.sopds.ru referenced in the LICENSE / README headers also doesn't resolve in 2026. Both removed from sopds-go's docs to avoid promising users dead resources.

License clause noted: original Python SOPDS is under "GPL-3.0 or, at your option, any later version" — the "any later version" clause is what permits an AGPL-3.0 downstream (per GNU's license compatibility matrix). Worth documenting because if it were strict GPL-3.0-only, our AGPL upgrade would be more delicate.

Files Modified:

README.md: opening paragraph attribution corrected (English transliteration Dmitry V. Shelepnev + Cyrillic Дмитрий Шелепнёв + version 0.23 + email). Dead links to sopds.ru and the GitHub fork removed; replaced with "appears to be offline as of 2026" note. License section attribution corrected likewise.
NOTICE.md: "Original SOPDS — Python implementation" section rewritten with correct author block, the "any later version" license-clause note, and a list of upstream conventions carried over (OPDS feed structure, MySQL schema for import-mysql tool, sopds.conf-style ini-to-YAML mapping, CLI shape).
README.md: new "Related projects" section before License — links to fbe-go (sister project, same author, complementary FB2 editor role: edit metadata in fbe-go, serve from sopds-go).
PROGRESS.md: this entry.

No code changes — pure metadata correction. v1.3.0 release artifacts still ship the original (mis-attributed) NOTICE.md; users who already downloaded v1.3.0 archives have the wrong text baked in. If bundled-doc accuracy matters, re-tag v1.3.1 to push corrected archives. Decision deferred — GitHub-Release-page README/NOTICE will reflect the fix immediately as part of the next push.

Revision 63 - 2026-05-10

v1.3.0 milestone — OSS-publication ready:

Bump version 1.2.0 → 1.3.0 marking the first publicly-installable release. Nothing functional changed since v1.2.0 (Rust ports remained); this Rev wraps up the OSS-publication track that was Revs 53-62.

What's new for end-users at v1.3.0:

License clarity — AGPL-3.0 with a README section explaining what it means for self-hosters vs. forks-deployed-as-services (Rev 53).
NOTICE.md attribution — clean third-party-licenses audit, including the AGPL-compatibility verdict for each dep (Rev 55).
Module path — github.com/dimgord/sopds-go matches the repo URL, so go install github.com/dimgord/sopds-go/cmd/sopds@latest actually works (Rev 56).
CI — every push to main runs lint + test + build on Linux & macOS (Rev 57).
GitHub Releases — cross-platform binaries (linux/darwin × amd64/arm64) auto-built via GoReleaser on each v* tag, with SHA-256 checksums and conventional-commits-grouped changelog (Rev 58).
Docker images — ghcr.io/dimgord/sopds-go:latest (multi-arch via manifest list, distroless base, ~75MB) auto-published on each release (Rev 59).
Nix — nix run github:dimgord/sopds-go -- start works directly; nix develop .#tts-rs for the CUDA Rust workflow (Rev 60).
Homebrew — brew tap dimgord/tap && brew install sopds-go after the first release ships the formula auto-update (Rev 61).
Go 1.25 — toolchain bumped to current Go stable (Rev 62).

Install matrix (post-v1.3.0):

# Pre-built binary
curl -LO https://github.com/dimgord/sopds-go/releases/download/v1.3.0/sopds-go_1.3.0_linux_x86_64.tar.gz

# Docker
docker run -d ghcr.io/dimgord/sopds-go:latest

# Nix
nix run github:dimgord/sopds-go -- start

# Homebrew
brew tap dimgord/tap && brew install sopds-go

# Source
go install github.com/dimgord/sopds-go/cmd/sopds@v1.3.0

v1.3.0 release pipeline test plan:

Tag v1.3.0 will trigger .github/workflows/release.yml. First release reveals what's brittle in the pipeline. Known pre-flight risks:

GoReleaser docker step needs amd64 + arm64 cross-compile — should work but no prior validation.
Homebrew formula push — HOMEBREW_TAP_GITHUB_TOKEN set as repo secret; first auto-formula commit goes to dimgord/homebrew-tap.
GHCR multi-arch manifest — first push to ghcr.io/dimgord/sopds-go; package needs to be created (likely auto on first push under dimgord namespace).
Changelog generation — pulls from git commits since previous tag (v1.2.0); should pick up Rev 53-62 commits and group by Conventional Commits prefix.

If any step fails, that's diagnostic data for follow-up Revs. Either re-run with gh workflow run release.yml -f tag=v1.3.0 after the fix, or move forward with v1.3.1.

Files Modified:

PROGRESS.md: header Current Version 1.2.0 → 1.3.0; Rev 63 entry.

Revision 62 - 2026-05-10

Go bump 1.24 → 1.25 + golangci-lint version unpinned (CI hotfix):

First push of Rev 57 (CI workflow) failed in the lint job with:

Error: can't load config: the Go language version (go1.23) used to build
golangci-lint is lower than the targeted Go version (1.24.0)

golangci-lint v1.61 (which I pinned in Rev 57) was itself built with Go 1.23, and golangci-lint's parser refuses to analyze code targeting a Go version newer than its own build version. So pinning the linter while the project tracks current-Go is a self-defeating maintenance trap — every Go bump silently breaks lint until the pin is bumped too.

Fix is twofold:

Bump Go target 1.24 → 1.25.0 in go.mod to align with current stable Go. Also dropped the toolchain go1.24.10 line — Go's auto-downloader will fetch 1.25 on hosts without it.
Switch CI lint version from v1.61 → latest so the action grabs whatever golangci-lint release is current at run time. This eliminates the version-skew failure mode for good. Comment block in ci.yml documents the rationale so this doesn't get re-pinned by reflex on a future PR.

Other files touched to stay in sync:

.github/workflows/ci.yml — GO_VERSION: 1.24.x → 1.25.x; golangci-lint pin → latest.
.github/workflows/release.yml — go-version: 1.24.x → 1.25.x (otherwise GoReleaser builds release binaries with stale Go).
flake.nix — devShell go_1_24 → go_1_25 so nix develop matches CI.
go.mod — direct + indirect deps re-tidied via go mod tidy (added a few transitively-needed packages: go-internal, objx, pretty, mousetrap, randomstring).

Verified locally:

go version reports 1.26.1 (newer than the go.mod floor — fine, Go is forward-compatible).
go build ./... clean.
go test ./... all 14 packages pass.

Files Modified:

go.mod: go 1.24.0 / toolchain go1.24.10 → go 1.25.0; tidy added indirect entries.
.github/workflows/ci.yml: 2 lines.
.github/workflows/release.yml: 1 line.
flake.nix: 1 line.

Revision 61 - 2026-05-10

Homebrew tap — dimgord/homebrew-tap + GoReleaser brews integration (Phase 9):

End-user macOS install path closed: brew tap dimgord/tap && brew install sopds-go. Each v* tag automatically pushes a formula update to dimgord/homebrew-tap via GoReleaser's brews: integration — no manual formula maintenance.

1. New repo: dimgord/homebrew-tap

Created public via gh repo create dimgord/homebrew-tap --public --add-readme. Default branch: master. README explains the auto-update flow and lists available formulae (currently just sopds-go); the Formula/ directory will be populated by GoReleaser on the first release tag — no formula committed manually. Brew's tap-name convention (homebrew- prefix on the repo name, dropped in brew tap user-side) means users get the friendly dimgord/tap shorthand.

2. .goreleaser.yaml — brews: block:

repository.{owner,name,branch} → dimgord/homebrew-tap on master. Auth via HOMEBREW_TAP_GITHUB_TOKEN env var (PAT or fine-grained token with contents: write on the tap repo).
directory: Formula — Homebrew's standard formula location (resolves to Formula/sopds-go.rb).
commit_author — dimgord-bot for tap-side audit clarity (vs. mixing with personal commits).
install block — drops all three binaries to bin/, config.yaml.example and init.sql to pkgshare/ so users have reference materials in $(brew --prefix)/share/sopds-go/.
test block — system "#{bin}/sopds", "version" is the smoke test that brew test sopds-go runs. Confirms the binary links cleanly and the version-injection ldflags from Rev 58 took effect.
dependencies — postgresql@16 as recommended (not required, since users may have a separate Postgres install).
caveats — multi-step setup runbook shown after brew install. Mentions optional runtime deps (calibre, espeak-ng) and points at sopds-tts-rs for CUDA TTS.

3. .github/workflows/release.yml — passes HOMEBREW_TAP_GITHUB_TOKEN to GoReleaser env:

Added HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }} next to existing GITHUB_TOKEN. Comment block in workflow file documents how to set the secret.

Manual step required from the maintainer — set the secret:

The auto-built GITHUB_TOKEN for sopds-go's workflow runs only has access to sopds-go itself; cross-repo writes (to homebrew-tap) need a separate token. Two options:

Classic PAT (simplest): https://github.com/settings/tokens/new — repo scope. Set:

gh secret set HOMEBREW_TAP_GITHUB_TOKEN --repo dimgord/sopds-go
# paste the PAT when prompted

Fine-grained token (recommended for least-privilege): https://github.com/settings/personal-access-tokens/new — Repository access: Only select repositories → dimgord/homebrew-tap. Repository permissions: Contents: Read and write. Same gh secret set command.

Once the secret is set, the next v* tag push will auto-publish the formula. No re-trigger of v1.2.0 is needed; just bump to v1.3.0 (or whatever) when convenient.

Pre-publication checklist update:

Optional: Homebrew tap

Files Modified:

dimgord/homebrew-tap (new repo): polished README pushed (1 commit, 33 lines).
.goreleaser.yaml: +60 lines (brews: block).
.github/workflows/release.yml: +5 lines (HOMEBREW_TAP_GITHUB_TOKEN env + comment).

Revision 60 - 2026-05-10

Root flake.nix — nix run github:dimgord/sopds-go (Phase 8):

Composite Nix flake at repo root that packages all three Go binaries (sopds, sopds-tts, zipdupes) plus re-exports sopds-tts-rs/'s CUDA-aware Rust dev shell. Lets Nix users install/run sopds-go without ever touching go build directly, and gives a single nix develop entry point covering both languages.

Flake structure:

Inputs:
- nixpkgs (nixos-unstable) — Go toolchain + system tools.
- flake-utils — eachDefaultSystem boilerplate for the 4-system matrix (x86/arm × linux/darwin).
- sopds-tts-rs — path:./sopds-tts-rs so the Rust subproject's existing flake (with CUDA + cuDNN 9.8 pinning for Pascal sm_61, see Rev 52) is composed in. inputs.nixpkgs.follows = "nixpkgs" keeps the Go-side and Rust-side nixpkgs identical to avoid double-eval at the cost of using whatever sopds-tts-rs's own pinned cuDNN dictates for cuda-related tools (nixpkgs-cuda is a separate sopds-tts-rs input — that one stays pinned).
packages.<system>.{sopds,sopds-tts,zipdupes,default} — one buildGoModule per CLI, with shared commonGoArgs (vendorHash, ldflags, doCheck = false). default aliases sopds. Version computed from self.rev (short git rev) or "dev"/"dev-dirty" fallback. Linker flags match GoReleaser's tagged-release flags (-s -w -X main.version=${version}).
apps.<system>.{sopds,sopds-tts,zipdupes,default} — nix run github:dimgord/sopds-go [-- args] invokes the corresponding binary's bin/<name> entry. Default = sopds. Lets a curious user spin up the server in one command without cloning.
devShells.<system>.default — Go dev shell: go_1_24, gopls, golangci-lint, delve, go-task, postgresql_16 (client tools — psql / pg_dump / pg_restore for migrations + backups). Shell hook prints versions + quick task references.
devShells.x86_64-linux.tts-rs — re-export of sopds-tts-rs.devShells.x86_64-linux.default. Linux-only because the upstream flake hardcodes x86_64-linux + cudaSupport. Users do nix develop .#tts-rs from repo root for the CUDA + Rust toolchain in one go.

Known caveat — vendorHash is fakeHash:

vendorHash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";

First nix build will fail with a "hash mismatch — expected: AAA…, got: " error. Copy the real hash into commonGoArgs.vendorHash and re-run. Going through the flake.lock + actually running nix build .#sopds once on a Linux box is the only way to know the right hash; CI will do this on first release-tag push, but for now it's a placeholder. Future improvement: gomod2nix derivative tooling to auto-generate hashes from go.mod, eliminating the manual step.

Verified evaluation:

nix flake metadata resolves all four inputs (nixpkgs at 2026-05-05, sopds-tts-rs path-input, flake-utils, transitive rust-overlay + nixpkgs-cuda from Jun 2025).
nix flake show --no-update-lock-file enumerates all 4 systems × 4 apps + 4 packages + 1 devShell (default) + 1 extra devShell on x86_64-linux (tts-rs). No errors.
Actual nix build deferred until a Linux test box and the vendorHash is filled in — the structure is sound.

Files Modified:

flake.nix (new): ~140 lines.
flake.lock (new): generated by nix flake metadata. Committed for reproducibility.

Pre-publication checklist update:

Root flake.nix packaging the binary for nix run github:dimgord/sopds-go

Revision 59 - 2026-05-10

Docker image build + GHCR publish on tag (Phase 7):

Multi-arch (linux/amd64 + linux/arm64) Docker image automatically built and pushed to ghcr.io/dimgord/sopds-go on each v* tag, integrated into the existing GoReleaser pipeline (Rev 58). Users can now docker run -d ghcr.io/dimgord/sopds-go:latest for self-hosting without compiling anything.

1. Dockerfile at repo root, ~30 lines:

Base: gcr.io/distroless/base-debian12:nonroot — minimal Google distroless image with glibc + CA certs + nonroot user, no shell, no package manager. ~25MB before binaries; ~75MB after.
Strategy: GoReleaser builds the binaries outside the image (cross-platform via Go's native cross-compile, way faster than QEMU-emulated go build inside arm64 image), then COPYs them into the runtime image. Dockerfile has no Go toolchain — pure runtime.
COPYs sopds, sopds-tts, zipdupes to /usr/local/bin/. Bundles config.yaml.example to /etc/sopds/config.yaml.example as reference; users mount their actual config over /etc/sopds/config.yaml.
Exposes 8081, runs as nonroot:nonroot (no privileged daemon needed for the OPDS server).
ENTRYPOINT/CMD: sopds start -c /etc/sopds/config.yaml. Override CMD for one-shot ops (migrate, scan, import-mysql).
Comments explicitly call out what's NOT included: PostgreSQL (run separately as a sibling container), Calibre (~500MB, build a derivative if MOBI conversion needed), espeak-ng + Piper voices (same logic — build a derivative for TTS). README will document this for users.

2. .dockerignore — excludes .git/, .github/, .task/, editor files, build artifacts, the Rust subprojects, fb2converters/, fetch/, test files, plus historical noise (*.lst, tags, sopds_backup.dump*). Keeps the build context tight.

3. .goreleaser.yaml extended with two dockers: blocks (one per arch) and docker_manifests: (manifest list aliasing both archs under :{{ .Version }} and :latest):

Image tags: ghcr.io/dimgord/sopds-go:<arch> for arch-specific, plus manifest list at ghcr.io/dimgord/sopds-go:{{ .Version }} and ghcr.io/dimgord/sopds-go:latest.
OCI labels: org.opencontainers.image.{title,description,url,source,version,created,revision,licenses} populated from GoReleaser context — registries that read these (Docker Hub, GHCR's own UI) display them as image metadata.

4. .github/workflows/release.yml extended with three new steps before goreleaser:

docker/setup-qemu-action@v3 — registers binfmt for arm64 emulation on the amd64 GitHub runner. Needed only because GoReleaser still does a docker build (even though the binary is pre-cross-compiled, the FROM base layer needs to be unpacked for the target arch).
docker/setup-buildx-action@v3 — buildx is required for --platform=linux/arm64 flag GoReleaser passes to docker build.
docker/login-action@v3 to ghcr.io using the workflow's auto-provided GITHUB_TOKEN. Permissions extended: packages: write added next to existing contents: write so the token can push to GHCR.

Pre-publication checklist update:

Dockerfile + GHCR publish

Files Modified:

Dockerfile (new): ~30 lines.
.dockerignore (new): ~30 lines.
.goreleaser.yaml: +75 lines (dockers × 2 + docker_manifests).
.github/workflows/release.yml: +14 lines (qemu + buildx + login + permissions: packages:write).

First Docker image will be built and pushed automatically on the next v* tag. No tag needs to be bumped; current v1.2.0 was pre-Docker. Either re-trigger the existing tag via workflow_dispatch (Actions tab → Release → Run workflow → tag: v1.2.0) or wait for the next semver tag (v1.3.0 when Phase 8/9 ship).

Revision 58 - 2026-05-10

Release automation: GoReleaser + .github/workflows/release.yml:

End-to-end release pipeline that fires on v* tag push: cross-compiles all three CLI binaries (sopds, sopds-tts, zipdupes) for linux/darwin × amd64/arm64, packages them with assets, computes SHA-256 checksums, generates a changelog from git commits, and publishes a GitHub Release.

1. .goreleaser.yaml at repo root:

version: 2 — modern GoReleaser config schema.
3 builds entries — one per CLI. All static (CGO_ENABLED=0) so the binaries run on any glibc/musl distro. -trimpath for reproducible builds (no local-path leak); -s -w strips DWARF debug info; -X main.version={{ .Version }} injects the tag into the sopds binary's version command (refactored to use a package var — see code change below).
Archive naming: sopds-go_<version>_<os>_<arch>.tar.gz with macOS aliased to macos and arm64/amd64 normalized to arm64/x86_64 for human readability. Each archive bundles both binaries plus LICENSE, NOTICE.md, README.md, PROGRESS.md, config.yaml.example, init.sql.
Checksums: checksums.txt in SHA-256.
Changelog from GitHub commits: uses Conventional Commits regex grouping — feat: → "Features", fix: / bugfix: → "Bug fixes", everything else → "Other". Filters out ^docs:, ^test:, ^chore:, ^ci:, and typo from the changelog (signal noise).
Release header/footer: header points to README + PROGRESS; footer has a curl-tar-install one-liner for Linux x86_64 + sha256sum verification reminder + AGPL-3.0 disclosure.
prerelease: auto — tags with -rc, -beta, -alpha automatically flagged as pre-release on GitHub.

2. .github/workflows/release.yml — single job, ~50 lines:

Triggered on push of v* tag, plus workflow_dispatch for manual rebuilds of an existing tag.
permissions: contents: write — minimum scope, lets GoReleaser create the Release; no other writes.
fetch-depth: 0 — GoReleaser needs full git history for the changelog.
goreleaser/goreleaser-action@v6 with version: ~> v2.
GITHUB_TOKEN from default action secrets — no PAT needed; default token has the necessary releases: write scope.

3. Code change — cmd/sopds/main.go: replaced the hardcoded fmt.Println("SOPDS v1.2.0 (Go rewrite)") with fmt.Printf("SOPDS %s (Go rewrite)\n", version) where version is a package-level var version = "dev" overridable via -ldflags. So:

go install github.com/dimgord/sopds-go/cmd/sopds@latest → version reports dev
go build -ldflags '-X main.version=v1.2.0' … → reports v1.2.0
GoReleaser-built binaries → reports the tag name from {{ .Version }}
Verified: /tmp/sopds_test version → SOPDS v1.2.0 (Go rewrite). ✓

4. Future-Phase wiring: the GoReleaser config includes hooks for Docker (Phase 7) and Homebrew tap (Phase 9) but they're commented out / not yet enabled — will come in subsequent Revs.

Pre-publication checklist update:

Release automation

Files Modified:

.goreleaser.yaml (new): ~120 lines.
.github/workflows/release.yml (new): ~50 lines.
cmd/sopds/main.go: 4 lines changed (added version package var, switched Println → Printf).

Revision 57 - 2026-05-10

CI workflow — .github/workflows/ci.yml:

Added GitHub Actions CI that runs on push to main/dev and PRs to main. Patterns adapted from fbe-go's CI but trimmed — sopds-go has no frontend, no Wails, no XSD-tag matrix.

Three jobs:

lint (Linux only): golangci-lint v1.61 with default linter set (govet, ineffassign, staticcheck, errcheck, unused). 5-min timeout. No .golangci.yml in repo yet — using action's defaults; can tune later.
test (Linux + macOS matrix): go vet ./... then go test -race -timeout 5m ./.... Hermetic — no PostgreSQL service container needed because internal/database/*_test.go only tests model structs (struct-roundtrip, constants, pagination math), not actual SQL queries. The -race detector catches goroutine misuse early.
build (Linux + macOS matrix): builds all three CLI binaries — sopds, sopds-tts, zipdupes — to confirm they link on each target. Catches build-tag / CGo issues that pure-Go tests don't (e.g. if a future change adds a Linux-only syscall import that breaks macOS).

Why no PostgreSQL service: verified by inspection — find internal -name '*_test.go' -exec grep -l 'sql\|gorm\|database/sql' {} only matches internal/config/config_test.go (uses gorm types but doesn't open a connection), internal/database/models_test.go (tests model structs), and a couple of domain-layer tests (use repository interfaces, no real connection). If actual integration tests are added later, add a services.postgres block to the test job.

Cancel-in-progress is on for the same workflow+ref so successive pushes don't pile up runs. permissions: contents: read — minimum scope, avoids CodeQL warning, follows the principle that CI is read-only (only reads code, doesn't comment back or push).

Files Modified:

.github/workflows/ci.yml (new): 3 jobs, ~85 lines.

Pre-publication checklist update:

CI workflow

Revision 56 - 2026-05-10

Rename Go module path: github.com/sopds/sopds-go → github.com/dimgord/sopds-go:

The original module path declared in go.mod referenced a non-existent GitHub organization (sopds/), while the actual repo lives at github.com/dimgord/sopds-go. This mismatch wouldn't matter for go build but would break go install github.com/dimgord/sopds-go/cmd/sopds@latest for users — Go's module proxy expects the path to match the hosted location.

Why rename instead of creating a sopds/ org:

sopds-go is an application, not a library — nobody is importing it as a Go package, so the rename has zero external blast radius.
Renaming aligns the module path with the actual repo location, making go install work for users who want to build from source without cloning.
Creating an empty sopds org just to host one repo is administrative overhead with no upside.

Mechanism:

find . -type f \( -name '*.go' -o -name 'go.mod' \) ! -path './.git/*' ! -path '*/target/*' \
  -exec sed -i '' 's|github.com/sopds/sopds-go|github.com/dimgord/sopds-go|g' {} +

Replaced 78 occurrences across 38 files (37 Go files + go.mod). PROGRESS.md was deliberately excluded from sed — historical entries that mention the old path remain as factual snapshots.

Verification:

go build ./... — clean (no errors).
go test ./... — all 14 internal packages pass (converter, database, domain/{author,book,catalog,genre,repository,series,user}, i18n, opds, scanner, tts).

Files Modified: 37 .go files + go.mod — all internal imports rewritten consistently.

Revision 55 - 2026-05-10

NOTICE.md — third-party attributions audit:

Added NOTICE.md at the repo root listing every third-party work that ships with sopds-go (bundled), is linked at build time (Go modules / Rust crates), or is invoked as a subprocess at runtime, with license + AGPL-compatibility verified for each.

Sections:

Original SOPDS attribution — V.A. Onishchenko's Python implementation, GPL-licensed, source for sopds-go's architecture / config schema / MySQL DB layout.
Bundled converters (fb2converters/fb2mobi/, fb2converters/fb2epub/) — both MIT-licensed, called as subprocesses (not linked). Originally based on dnkorpushov's fb2conv + eBook .NET respectively.
Go module direct deps — 14 modules: chi (MIT), gorm (MIT), cobra (Apache-2.0), pgx via driver/postgres (MIT), mysql driver (MPL-2.0), golang.org/x/* (BSD-3), yaml.v3 (MIT/Apache-2.0), etc. AGPL-compatibility verified for each. Notable: MPL-2.0 (mysql driver) is AGPL-compatible per FSF list because MPL §3.3 explicitly permits combination.
Rust crates for sopds-tts-rs + zipdupes-rs — ort, serde, serde_json, hound, clap, walkdir. All Apache-2.0/MIT dual or compatible.
CUDA / cuDNN — proprietary NVIDIA libraries linked dynamically by ORT at build time on GPU systems. Not redistributed — sourced from user's NVIDIA installation. Binary remains AGPL-distributable.
External subprocess tools — Calibre (GPL-3.0), espeak-ng (GPL-3.0), Piper TTS (MIT), PostgreSQL (BSD-like). GPL-3.0 tools called via exec.Command do not impose GPL on sopds-go.
Voice models — NOT bundled, user-downloaded from rhasspy/piper-voices. Mostly MIT/CC-BY per model author.
Web UI assets — Font Awesome via CDN (SIL OFL / CC BY / MIT), not bundled.

Why this matters at OSS publication time:

AGPL-3.0 places strict requirements on derivative works and combined works. NOTICE.md documents that each link/bundle/subprocess relationship has been classified correctly (linked vs. invoked) so license obligations transfer cleanly.
Apache-2.0 deps (cobra, hound, ort, serde) require attribution preservation under §4(c); listing them here satisfies that.
Subprocess invocation (Calibre, espeak-ng, ebook-convert) does not propagate GPL to sopds-go — but users need to know these are GPL tools they're installing alongside, especially if they redistribute prebuilt sopds-go bundles with these tools included (Docker image needs this disclaimer).
Voice models are user-provided, no bundling concern, but README and NOTICE both clarify each model has its own license.

Files Modified:

NOTICE.md (new): ~140 lines covering all categories above.

Revision 54 - 2026-05-09

README polish for OSS publication:

Significant rewrite preparing the repo for public release on GitHub. Goals: orient new readers, surface the Rust subprojects, replace placeholder <repository> references with real install paths, sync version + Go-version data with reality.

Changes:

Header: replaced bare **Version: 1.1.0** line with four shields.io badges — latest release, license (AGPL-3.0), Go version (read from go.mod), CI build status. Badges link to relevant pages so a curious visitor can land on any of them in one click.
Description rewrite: opening paragraph now positions sopds-go as a Go rewrite of Onishchenko's Python SOPDS, with concrete value-prop language — "self-hosted home libraries", "scans nested ZIP archives", "even produces audiobooks via local TTS". Replaces the previous one-liner that didn't tell a stranger why they'd care.
Features section grouped into four subsections: Catalog & Browsing / Library Management / Conversion & TTS / Operations. Adds previously-missing items: TTS (Piper), GPU-accelerated TTS (sopds-tts-rs), Docker image, Nix flake, prebuilt binaries.
Requirements bumped: Go 1.21+ → Go 1.24+ (go.mod actually pins 1.24.0). Added optional dependencies for TTS (espeak-ng, Piper) and CUDA. PostgreSQL 12+ → 14+ (more reasonable lower bound).
Installation section restructured — four subsections in priority order:
- Pre-built binaries (curl + tar + install) — for users
- Docker (docker run with library + config volumes) — for self-hosters
- Nix flake (nix run github:dimgord/sopds-go) — for Nix users
- From source (go build) — for hackers Replaces the previous single git clone <repository> placeholder.
New ## Rust subprojects section — explains sopds-tts-rs/ (CUDA-accelerated, Pascal sm_61 supported, ORT 2.0 + cuDNN 9.8 dual-nixpkgs flake) and zipdupes-rs/ (rip-grep-style speed for corpus scans). Both presented as CLI-drop-in replacements; Go originals retained as fallbacks. References Rev 52 for the engineering trail.
Project Structure refreshed to reflect actual directory tree (added internal/tts/, internal/infrastructure/, both Rust subdirs, fb2converters/, Taskfile.yml).
Web Interface section trimmed and renamed ## Web UI (was ## Web Interface Screenshots but had no actual screenshots — false promise).

Files Modified:

README.md: ~120 lines added, ~30 removed; net +90 lines.

Pre-publication checklist (work in progress; not blocking this Rev):

Revision 53 - 2026-05-09

License switch: GPL-3.0 → AGPL-3.0:

Replaced the LICENSE file with the canonical AGPL-3.0 text from gnu.org. README's License section rewritten to explain the rationale and the practical difference for network-deployed instances.

Why AGPL specifically (not staying on GPL):

sopds-go is inherently a network service — its primary interface is the OPDS XML feed served over HTTP, plus the HTML web UI. Most users will deploy it as a public/intranet catalog, not run it as a local CLI.
GPL-3.0 has the well-known "SaaS loophole": a hosting provider can take GPL code, modify it, run it as a public service, and never release the modifications because the software isn't distributed to users in the GPL sense — only its output (XML, HTML) is.
AGPL-3.0 closes this loophole via §13: "if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network ... an opportunity to receive the Corresponding Source".
For an OPDS catalog, this is the correct ethical default: someone running a forked sopds-go for their book community should share their improvements back, not silently keep them.
Self-hosting for personal use remains unrestricted under both GPL and AGPL.

Practical impact for users:

Personal home server: zero change — you already have the source.
Forking + deploying as a community service: must publish your fork's source.
Embedding sopds-go logic in another network application: that application must be AGPL-licensed (strong copyleft).

Practical impact for contributors:

Patches submitted via PR are accepted under AGPL-3.0 (CONTRIBUTING.md will spell this out when added).
Contributors retain copyright; sopds-go does not require CLA assignment.

Files Modified:

LICENSE: replaced GPL-3.0 text with AGPL-3.0 (canonical from https://www.gnu.org/licenses/agpl-3.0.txt; sha1 4c665f87b5dc2e7d26279c4b48968d085e1ace32, 661 lines).
README.md: License section rewritten — explicit AGPL-3.0 link, network-deployment note, attribution to original SOPDS project (V.A. Onishchenko's Python implementation, also GPL-family).

Revision 52 - 2026-03-12

Rust ports of sopds-tts and zipdupes for performance + GPU acceleration:

Two CLI utilities rewritten in Rust as drop-in replacements for the Go versions, primarily to recover GPU acceleration on Pascal-class hardware (sm_61, GTX 1070-era) and squeeze more throughput out of the audiobook-generation pipeline. Both Rust subprojects ship as separate crates under the sopds-go monorepo and remain CLI-compatible with their Go predecessors so callers don't change.

1. sopds-tts-rs/ — CUDA-accelerated TTS (drop-in for sopds-tts):

Same CLI: sopds-tts-rs <model> <output>, text on stdin, WAV on <output>. Identical interface to Go version, so the audiobook generator's subprocess plumbing stays unchanged.
Stack: ort 2.0.0-rc.10 (ONNX Runtime Rust binding, CUDA execution provider) + serde (model metadata) + hound (WAV encoding) + espeak-ng (IPA phonemizer for non-"text" Piper models).
Why Rust: Go's onnxruntime-go has limited CUDA-EP support and trails upstream. ORT-rs binds against the C API directly, gets the full CUDA EP, and the executable is dramatically faster than the Go path for typical chapter-length inputs.
Pascal + Nix bring-up was the hard part — most of the engineering hours on this Rev. Current nixpkgs-unstable bundles cuDNN 9.x but not in a sm_61-compatible build (newer cuDNN dropped Pascal compute capability). Solution: flake.nix pulls TWO nixpkgs inputs:
- nixpkgs (latest) — Rust toolchain (rustc, cargo, pkg-config).
- nixpkgs-pinned (June 2025 snapshot) — cudaPackages.cudnn = 9.8 which still ships sm_61. Linked at build time via LD_LIBRARY_PATH overlay.
ORT itself is built from source (CUDA-EP enabled, sm_61 added explicitly to CMAKE_CUDA_ARCHITECTURES). First nix develop is 15-30 min on cold cache; subsequent builds are trivial.
Files: sopds-tts-rs/src/main.rs (291 lines: ONNX inference + speaker-id + phoneme map + WAV write), sopds-tts-rs/Cargo.toml, sopds-tts-rs/Cargo.lock, sopds-tts-rs/flake.nix, sopds-tts-rs/flake.lock, sopds-tts-rs/run-gpu.sh.

2. zipdupes-rs/ — Rust port of the FB2 archive de-duplicator:

Same CLI as Go zipdupes. Walks a directory tree of .fb2.zip archives, hashes their entries, identifies duplicates across archives.
Why Rust: rip-grep-style speed for I/O-heavy walking + hashing on the 1TB book corpus on /1TB. Go version was acceptable but visibly slow on cold-cache full corpus scans.
Files: zipdupes-rs/src/main.rs (455 lines), zipdupes-rs/Cargo.toml, zipdupes-rs/Cargo.lock.

Status:

Both ports landed in production at this Rev — sopds-tts-rs is the active TTS backend whenever GPU TTS is needed; zipdupes-rs replaces the Go zipdupes for batch corpus scans. Neither has been heavily exercised since (no major book imports / TTS jobs in the recent month), so any subtle bugs are still latent. The Go versions remain in-tree as fallbacks (not deleted).

Files Modified:

sopds-tts-rs/ (new directory, 7 files): Cargo.toml, Cargo.lock, flake.nix, flake.lock, run-gpu.sh, src/main.rs. Total: 1547 lines.
zipdupes-rs/ (new directory, 3 files): Cargo.toml, Cargo.lock, src/main.rs. Total: 682 lines.
go.mod, go.sum: dependency churn from co-landed Go-side cleanups (49 + 322 lines added).
.gitignore: added Rust target/ patterns.
CLAUDE.md (commit 7e0c83d, follow-up): documents the Rust subprojects in the architecture tree and adds a "Rust TTS alternative" section under "Audiobook Support" describing the Nix CUDA build flow.

Commits covered (retroactively documented 2026-05-09 — original landing skipped PROGRESS.md):

9f6e1e5 Add rust versions of zipdupes and sopds-tts.
7e0c83d Add Rust TTS (sopds-tts-rs) documentation to CLAUDE.md

Revision 51 - 2026-02-08

Bugfix: Ukrainian TTS - Multi-Speaker Model & Language Code Matching:

Three issues fixed for Ukrainian book TTS generation:

Multi-speaker ONNX model support (Missing Input: sid error)
- uk_UA-ukrainian_tts-medium has 3 speakers (lada, mykyta, tetiana)
- ONNX model requires sid input tensor but we only sent 3 inputs
- Added sid tensor to runInference() when num_speakers > 1
- Added SpeakerIDMap and speakerID field to Piper struct
- ONNX session now includes "sid" in input names for multi-speaker models
Text phoneme type support (model uses raw text, not IPA)
- uk_UA-ukrainian_tts-medium has phoneme_type: "text"
- Its phoneme_id_map contains Ukrainian letters directly, not IPA symbols
- textToPhonemes() now skips espeak-ng for "text" type models
- Returns lowercase text directly for phoneme ID mapping
Language code normalization (uk-UA not matching config key uk)
- Book 211236 had lang uk-UA in FB2 metadata, config key was uk
- GetVoice() now tries base language code (strips -UA, -US etc.)
- Ukrainian books no longer fall back to English voice

Files Modified:

internal/tts/piper.go:
- Added PhonemeType, SpeakerIDMap to PiperConfig
- Added speakerID field to Piper struct
- NewPiper() configures "sid" input name and default speaker ID for multi-speaker models
- textToPhonemes() returns raw lowercase text for "text" phoneme type
- runInference() adds sid tensor when num_speakers > 1
internal/tts/generator.go:
- GetVoice() tries base language code when exact match fails

Revision 50 - 2026-01-30

Bugfix: Text Chunk Splitting UTF-8 Bug (Root Cause of OOM):

Found root cause: chunk 7 was 16,551 chars instead of 5,000 due to UTF-8 split bug
Bug in findSplitPoint(): when UTF-8 boundary search backed up to 0, returned len(text) (entire remaining text)
Fix: Instead of returning entire text, find first valid rune start after original position
This was causing ONNX to allocate massive memory for oversized chunks, leading to OOM

Files Modified:

internal/tts/extractor.go:
- Fixed findSplitPoint() to not return entire text length on UTF-8 boundary failure
- Now finds next valid rune start instead of returning all remaining text
internal/tts/generator.go:
- Reverted to normal chunk ordering
- Re-enabled parallel processing

Revision 49 - 2026-01-30

TTS Debug Logging and ONNX Isolation:

Added debug logging to generateChunk to trace subprocess execution
Removed ONNX initialization from main sopds process entirely
IsAvailable() now just checks for sopds-tts binary and models dir
Main sopds process has zero ONNX/native TTS code loaded

Files Modified:

internal/tts/generator.go:
- Added log statements in generateChunk for subprocess tracking
- IsAvailable() no longer calls initORT()

Revision 48 - 2026-01-30

TTS Subprocess Architecture (Memory Leak Fix):

Created separate sopds-tts binary for TTS chunk generation
Each chunk runs in its own subprocess, guaranteeing complete memory release
Re-enabled parallel chunk processing (subprocesses are independent)
Supports multiple simultaneous users (each request spawns separate processes)

New Files:

cmd/sopds-tts/main.go: Standalone TTS binary
- Takes model path and output path as args, text via stdin
- Creates Piper, generates audio, exits (releasing all memory)

Files Modified:

internal/tts/generator.go:
- Added getTTSBinaryPath() to locate sopds-tts binary
- generateChunk() now spawns sopds-tts subprocess instead of native Piper
- Re-enabled parallel processing with configurable workers
Taskfile.yml: Build task now builds both sopds and sopds-tts

Revision 47 - 2026-01-30

TTS Memory Leak Investigation:

Reverted parallel chunk processing to single-threaded (numWorkers=1)
Added debug.FreeOSMemory() for aggressive memory release
Added explicit nil assignments in Piper.Close() to help GC
Issue: ONNX runtime (onnxruntime_go) appears to leak native memory
TODO: If memory issues persist, implement subprocess-based TTS worker

Files Modified:

internal/tts/generator.go: Single worker, added debug.FreeOSMemory()
internal/tts/piper.go: Clear all references in Close()

Revision 46 - 2026-01-30

Bugfix: TTS Status Transition Not Updating UI:

Fixed page not updating when TTS status changes from "queued" to "processing"
Added initialStatus variable to track page's initial state
Page now reloads when polled status differs from initial status

Files Modified:

internal/server/web.go: Added status change detection in TTS polling JavaScript

Revision 45 - 2026-01-30

Enhancement: Parallel TTS Chunk Generation:

Chunk generation now runs in parallel using worker pool pattern
Number of parallel workers controlled by tts.workers config (default: 2)
Uses atomic counter for thread-safe progress tracking
Graceful cancellation support - workers check context and exit early
First error stops all workers and reports failure

Files Modified:

internal/tts/generator.go:
- Added sync/atomic import
- Replaced sequential chunk loop with parallel worker pool
- Workers pull from task channel, process independently
- Progress updated atomically as each chunk completes

Revision 44 - 2026-01-30

Bugfix: TTS Memory Leak (OOM Kill):

Fixed massive memory leak causing OOM kill after ~7 chunks (58GB memory usage)
Root cause: Cached ONNX/Piper sessions accumulate memory across inference runs
Solution: Create fresh Piper instance for each chunk, close immediately after use
Added explicit runtime.GC() call after closing each Piper to release ONNX memory
Removed unused Piper caching code (getPiper, closePipers, pipers map)

Files Modified:

internal/tts/generator.go:
- Removed pipers map and pipersMu mutex from Generator struct
- Removed getPiper() and closePipers() methods
- generateChunk() now creates fresh Piper per chunk with Close() + GC after each

Revision 43 - 2026-01-30

Bugfix: TTS Player JavaScript Errors:

Fixed "style is undefined" errors in TTS player page
Root cause: JavaScript runs unconditionally but DOM elements only exist in specific template branches
Fix 1: Added null checks for genProgress/genText in status polling (queued→processing transition)
Fix 2: Wrapped audio event listeners in {{if .IsComplete}} template condition
- timeupdate, loadedmetadata, ended listeners now only register when player exists

Files Modified:

internal/server/web.go:
- Added element existence checks in TTS polling JavaScript
- Wrapped audio event listeners in IsComplete template condition

Revision 42 - 2026-01-30

Enhancement: TTS Player Progress Display:

Added chunk progress display to TTS generation status bar
Shows "X/Y chunks" alongside percentage during generation
Added ETA calculation based on average chunk processing time
ETA displays minutes or hours format depending on remaining time

Files Modified:

internal/server/web.go:
- Added ChunksDone, ChunksTotal fields to TTS player template data
- Updated HTML template to show initial chunk counts
- Enhanced JavaScript polling to update chunks and calculate ETA

Revision 41 - 2026-01-30

Feature: TTS (Text-to-Speech) with Piper:

Added text-to-speech capability for FB2 ebooks using piper TTS engine
Background generation with configurable worker pool
Cached audio files per book for instant playback

New Package: internal/tts/

extractor.go - Extract plain text from FB2 XML for TTS input
- Strips markup, preserves paragraph breaks
- Splits text into chunks at sentence/paragraph boundaries
cache.go - Manage cached audio files per book
- Directory structure: {cache_dir}/{book_id}/chunk_NNN.wav
- Metadata JSON tracks voice, language, chunk count, status
queue.go - Job queue for background generation
- In-memory queue with book lookup
- Status tracking: queued, processing, completed, failed
- Progress tracking per chunk
generator.go - Piper integration and worker pool
- Configurable number of workers
- Language-specific voice selection
- Graceful start/stop

New Routes:

POST /web/book/{id}/tts/generate - Queue TTS generation
GET /web/book/{id}/tts/status - Get generation status (JSON)
GET /web/book/{id}/tts - TTS player page
GET /web/book/{id}/tts/chunk/{idx} - Stream audio chunk

TTS Player Features:

Chapter navigation with clickable list
Play/pause, prev/next track controls
Progress bar with seek
Dark mode toggle
Auto-play next chunk
Status polling during generation

Configuration (config.yaml):

tts:
  enabled: true
  piper_path: "/usr/bin/piper"
  models_dir: "/var/lib/piper/models"
  voices:
    en: "en_US-lessac-medium"
    uk: "uk_UA-lada-x_low"
  default_voice: "en_US-lessac-medium"
  cache_dir: "/var/lib/sopds/tts_cache"
  workers: 2
  chunk_size: 5000

Files Created:

internal/tts/extractor.go - FB2 text extraction
internal/tts/cache.go - Cache management
internal/tts/queue.go - Job queue
internal/tts/generator.go - Piper integration
internal/tts/tts_test.go - Unit tests (15 tests)

Files Modified:

internal/config/config.go - Added TTSConfig struct
internal/server/server.go - TTS generator init, routes, book data callback
internal/server/web.go - TTS handlers and player template
internal/i18n/locales/en.yaml - English TTS translations
internal/i18n/locales/uk.yaml - Ukrainian TTS translations

i18n Keys Added:

tts.listen, tts.generate, tts.generating
tts.queued, tts.queued_desc, tts.waiting
tts.not_generated, tts.not_generated_desc
tts.chapters, tts.chapter, tts.progress
tts.play, tts.pause, tts.download
tts.not_available, tts.failed

Requirements:

piper binary (https://github.com/rhasspy/piper)
Voice model files (.onnx) in models_dir

Revision 40 - 2026-01-28

DRY Refactoring - Server Helpers:

Created internal/server/helpers.go (208 lines) with consolidated helper functions
Eliminated ~150 lines of duplicated code across handlers.go and web.go

New Helper Functions:

BookPathInfo struct - parsed path info (archive/regular file detection)
getBookPath() - full filesystem path for book file
getBookDir() - directory containing the book
parseBookPath() - parses archive info (ZIP/7z detection, internal path)
getTrackPath() - audiobook track path construction
isAudioExtension(), isArchiveExtension() - format detection
readFromArchive() - unified ZIP/7z reading (replaces separate functions)
readFileFromZip(), readFileFrom7z() - low-level archive reading
BookLinks struct - authors, genres, series for a book
getBookLinks() - fetches all book links in one call
getTotalPages() - pagination calculation

Code Consolidated:

20+ filepath.Join(s.config.Library.Root, ...) patterns → helper methods
12 occurrences of GetBookAuthors/Genres/Series triplet → getBookLinks()
8 pagination calculations → getTotalPages()
readFromZip() and readFrom7z() → readFromArchive()
serveFromZip() simplified using parseBookPath()

Bug Fixed:

web.go:2132 was reading from book.Path instead of full path with filename

Revision 39 - 2026-01-28

Comprehensive Test Suite:

Created comprehensive tests before DRY refactoring (99 test functions total)
Tests document actual behavior for regression detection
Baseline saved to test_baseline.txt

New Test Files Created:

internal/config/config_test.go (405 lines, 11 tests)
- DefaultConfig, DSN, Load, Save, validation, struct fields
internal/converter/converter_test.go (464 lines, 17 tests)
- FB2→EPUB conversion, cover handling, BOM support, ZIP structure
internal/converter/reader_test.go (411 lines, 18 tests)
- FB2→ReaderHTML, TOC extraction, nested sections, heading levels
internal/scanner/fb2parser_test.go (251 lines, 8 tests)
- FB2 parsing, authors, genres, series, BOM, partial parsing
internal/scanner/audioparser_test.go (401 lines, 19 tests)
- Audio format detection, MIME types, author extraction, encoding fixes
internal/scanner/audiobookgrouper_test.go (279 lines, 12 tests)
- Folder name parsing, year suffix removal, duration formatting
internal/scanner/duration_test.go (269 lines, 14 tests)
- Duration extraction for MP4, MP3, FLAC, OGG formats
internal/scanner/inxparser_test.go (231 lines, updated)
- Nokia INX parser structs, total duration, file detection

Known Implementation Issues Documented:

FormatDuration() only shows single digits (uses modulo 10)
parseTitleFromFolderName() doesn't handle trailing separator
Tests document actual behavior for safe refactoring

Revision 38 - 2026-01-26

Feature: Nokia AWB Audiobook Support:

Added support for Nokia audiobooks (AWB format from Nokia Audiobook Manager ~2008)
AWB is AMR-WB (Adaptive Multi-Rate Wideband) audio codec
INX index files provide accurate durations (AWB files have no metadata tags)
AWB audiobooks treated as regular folder audiobooks (format="folder")

Files Created:

internal/scanner/inxparser.go - Nokia INX index file parser
- NokiaAudiobook struct with name, tracks, chapters, version, codec info
- NokiaTrack struct with filename and duration (seconds)
- ParseINX() parses UTF-16 LE/BE INX files with BOM detection
- FindINXFile() finds INX file in a directory
internal/scanner/inxparser_test.go - Unit tests for INX parser

Files Modified:

internal/scanner/scanner.go
- Added .awb to audioExtSet for audio file detection
- Modified processAudioGroup() to check for INX files and read durations
- Modified countFiles() to count audio files from audioExtSet
internal/scanner/audioparser.go
- Added AWB format handling - returns basic metadata (no tags to parse)
internal/server/web.go
- Added serveTrackFromAWB() for AWB→MP3 streaming conversion via ffmpeg
- Modified folder audiobook handling to detect AWB tracks by extension
internal/config/config.go
- Added FFmpeg field to ConvertersConfig (default: "ffmpeg")
- Added .awb to default library formats

Requirements:

ffmpeg with libmp3lame for AWB→MP3 conversion
Conversion runs at ~430x realtime speed

Revision 37 - 2026-01-26

Feature: Web-Based FB2 Reader:

Implemented in-browser ebook reader for FB2, EPUB, and MOBI formats
Non-FB2 formats converted to FB2 using Calibre's ebook-convert
Route: GET /web/read/{id} - opens book in reader

Reader Features:

Full book content display with proper typography
Table of Contents navigation (collapsible sidebar)
Font size controls (+/-)
Light/dark mode toggle
Reading position saved in localStorage
Images embedded as data URIs
Responsive design (mobile-friendly TOC)

Files Created:

internal/converter/reader.go - FB2 to reader HTML conversion
- ReaderContent struct with title, authors, TOC, HTML, cover
- FB2ToReaderHTML() parses FB2 and generates reader-friendly HTML
- ConvertToFB2() converts EPUB/MOBI to FB2 via Calibre

Files Modified:

internal/server/web.go
- Added ReaderData struct for reader template
- Added handleWebReader() handler
- Added readFrom7z() for reading books from 7z archives
- Added renderReaderTemplate() with reader template
- Added readerTemplate constant with full HTML/CSS/JS
- Added "Read" button to book list template for FB2/EPUB/MOBI
- Added "Read" button to bookshelf template
internal/server/server.go - Registered /web/read/{id} route
internal/i18n/locales/*.yaml - Added reader translations (all 5 languages)

i18n Keys Added:

reader.read - Read button label
reader.toc - Table of Contents
reader.font_smaller / reader.font_larger - Font controls
reader.dark_mode - Dark mode toggle
reader.close - Close button
reader.not_supported - Error for unsupported formats
reader.conversion_failed - Conversion error message
reader.loading - Loading indicator

Bug Fix: Audiobook player link for all formats:

Changed condition from Format=="FOLDER" to IsAudiobook
Now 7z/zip single-file audiobooks properly link to audio player page
Single-track audiobooks show "Audiobook" instead of "1 Tracks"

Revision 36 - 2026-01-26

Feature: Favicon and Placeholder Covers:

Added SVG favicon with book icon in app theme colors
- Routes: /favicon.ico and /favicon.svg
- Added <link rel="icon"> to HTML templates

Feature: Placeholder Covers for Missing Book/Audiobook Covers:

Added fallback placeholder SVG images for books/audiobooks without covers
Books use book icon placeholder (3 book spines)
Audiobooks use headphones icon placeholder
- Replaces 404 responses with 200 + placeholder image
- Nordic-themed design (matching app colors) with book icon
- Aspect ratio 2:3 (200x300) - standard book cover dimensions
- "No Cover" text label
Created placeholderCoverSVG constant in handlers.go
Created servePlaceholderCover() function for serving the placeholder
Updated handleCover() to return placeholder instead of 404 for:
- Invalid book ID (non-numeric)
- Non-existent book ID
- Non-FB2 formats without embedded covers
- FB2 files without cover images
- File not found errors
Updated serveAudioCover() to return placeholder for all error cases
Updated handleAudioTrackCover() to return placeholder for all error cases:
- Invalid audiobook ID
- Missing file parameter
- Audiobook not found
- Not an audiobook
- Cover not found in track
Bug fix: Fixed nil pointer panic when GetBook returns nil book without error
- GetBook returns nil, nil for non-existent books (not an error)
- Added book == nil check alongside error check

Bug Fix: UI language switching when filtering books:

The lang URL parameter was used for both UI language AND book filtering
Filtering books by language (e.g., Japanese) would switch UI if it matched a supported language
Fix: UI language now only set via cookie (JavaScript switcher)
URL lang parameter is now exclusively for book language filtering
Removed setLangCookie() function and all 18 calls to it

Bug Fix: FOLDER button 404:

Folder-based audiobooks showed "FOLDER" as download button, which caused 404
Fix: For folder format, show headphones button with track count linking to audio detail page
Applied to both book list template and bookshelf template

Bug Fix: Subfolder track playback in archive audiobooks:

Tracks inside subfolders within ZIP/7z archives couldn't be played
Two issues fixed:
1. Archive path construction when path == filename (data bug workaround)
2. Track matching now uses multiple strategies:
  - Exact path match
  - Suffix match for relative paths
  - Basename match for just filenames
Created serveFileFromZip() and serveFileFrom7z() helper functions
Handles cases where tracks don't have full path stored (legacy data)

Bug Fix: Empty track path in audiobook template:

Legacy audiobook data has empty path field in tracks (only name stored)
Template was generating ?file= (empty) causing 400 errors
Fix: Use {{or $track.Path $track.Name}} to fallback to filename
Applied to both collection tracks and flat track list templates

Feature: Password Confirmation with Show/Hide Toggle:

Added password confirmation field to registration form
Both password fields have eye icon toggle for show/hide
Real-time validation checks passwords match before enabling submit
When password changes, confirm field re-validates automatically

Feature: Centralized i18n with Language Resource Files:

Created internal/i18n/ package for internationalization
Language files stored in internal/i18n/locales/*.yaml
To add a new language: copy en.yaml, translate, add code to supportedLanguages
All translations consolidated from web.go and server.go into YAML files
Uses Go's embed to compile translations into binary
Supported languages (5): English, Ukrainian, French, Spanish, German

Files Created:

internal/i18n/i18n.go - i18n package with YAML loading
internal/i18n/locales/en.yaml - English translations
internal/i18n/locales/uk.yaml - Ukrainian translations (Українська)
internal/i18n/locales/fr.yaml - French translations (Français)
internal/i18n/locales/es.yaml - Spanish translations (Español)
internal/i18n/locales/de.yaml - German translations (Deutsch)

Files Modified:

internal/server/web.go - Uses i18n package instead of inline translations
internal/server/server.go - Uses i18n package for auth translations
internal/server/auth_templates.go - Password confirmation with toggle

Files Created:

internal/server/favicon.go - SVG favicon with book icon

Files Modified:

internal/server/handlers.go - Added placeholderCoverSVG, servePlaceholderCover(), updated handleCover() and serveAudioCover()
internal/server/server.go - Added favicon routes (/favicon.ico, /favicon.svg)
internal/server/web.go - Added favicon link to HTML templates, updated handleAudioTrackCover() to use placeholder

Revision 35 - 2026-01-25

Feature: SMTP Email Sending for Authentication:

Added SMTPConfig to configuration (internal/config/config.go)
- Supports SMTP with STARTTLS (port 587) or implicit TLS (port 465)
- Configurable: host, port, username, password, from address
Created EmailService (internal/server/email.go)
- SendVerificationEmail() - sends email verification links
- SendPasswordResetEmail() - sends password reset links
- Falls back to logging tokens when SMTP is disabled (dev mode)
Updated auth.go to use EmailService instead of just logging
Fixed auth template rendering (was returning empty page)
- Templates define {{define "content"}} blocks
- Now properly clones base template and executes "auth" template
Fixed user dropdown hover issue (menu disappeared on hover)
- Changed from margin-top gap to padding-top with transparent outer
- Added primary color background on hover for visibility
Added users table migration (012_users.sql) to schema

Configuration (config.yaml):

smtp:
  enabled: false
  host: smtp.gmail.com
  port: 587
  username: ""
  password: ""
  from: "SOPDS Library <noreply@example.com>"
  use_tls: false
  use_starttls: true

Files Modified:

internal/config/config.go - Added SMTPConfig
internal/server/email.go - New file, email sending service
internal/server/server.go - Added emailService to Server, template fix
internal/server/auth.go - Use emailService for verification/reset emails
internal/server/web.go - Fixed dropdown hover CSS, audiodetail PageData
config.yaml - Added smtp section, fixed site.url

Revision 34 - 2026-01-25

Fix: Folder-Based Audiobook Path Calculation & M4B Grouping:

Fixed bug where folder audiobooks were stored with incorrect path
- Problem: relPath included the folder name itself (e.g., "Music/Author - Title")
- Fix: Now uses parent directory as path (e.g., "Music"), folder name as filename
- This matches the pattern used by regular files
M4B files now included in folder grouping
- Folders with multiple M4B files are grouped as audiobook collections
- Single M4B in a folder still processed as individual audiobook
- Use case: collection folders like "Author - short stories (1)" with multiple M4B files
Added cleanup of old individual audio files when creating grouped audiobook
- MarkAudioFilesInFolderDeleted() marks old individual entries as unavailable
- Affects all audio formats: mp3, m4a, m4b, flac, ogg, opus
- Prevents duplicate entries when rescanning after grouping was added
Fixed audio track serving for folder audiobooks
- serveTrackFromFolder() now handles multiple path formats:
  1. Full filesystem paths (starts with /)
  2. Paths relative to library root (FolderName/file.mp3)
  3. Just filenames
- Constructs correct full path using book.Path and trackPath
- Added debug logging for path resolution issues
Added cover extraction support for folder audiobooks
- extractTrackCoverFromFolder() reads cover from audio file metadata
Per-track covers for folder audiobooks
- Audio player updates cover image when switching tracks
- Each M4B/audio file can have its own embedded cover art
- Uses preloading to avoid flicker when switching tracks
Fixed main book cover endpoint for folder audiobooks
- serveAudioCover() now correctly constructs folder path with book.Filename
- Iterates through ALL tracks to find one with embedded cover (not just first)
- Previously returned 404 when first track had no cover
- Now serves cover from any track that has embedded art
Header now shows current track info
- Added "Now Playing" indicator with pulsing speaker icon
- Shows current track name in header when playing
- Cover image always visible (placeholder if none)
- Cover updates per-track for folder audiobooks
- Header remains sticky at top when scrolling

Files Modified:

internal/scanner/scanner.go - Fixed processAudioGroup(), include M4B in grouping
internal/infrastructure/persistence/service.go - Added MarkAudioFilesInFolderDeleted() with M4B
internal/server/web.go - Fixed serveTrackFromFolder(), added extractTrackCoverFromFolder(), added "Now Playing" header indicator, cover placeholder, per-track cover updates
internal/server/handlers.go - Fixed serveAudioCover() folder path and track iteration

Revision 33 - 2026-01-25

Taskfile for Project Automation:

Added Taskfile.yml for common project tasks
Commands: build, init, migrate, start, stop, restart, status, scan
Systemd: service-start, service-stop, service-restart, service-status, service-logs
Database: db-backup, db-restore, db-vacuum, db-stats
Duplicates: dupes, dupes-clear
Development: dev, test, vet, fmt, lint, clean
Import: import-mysql, version

Files Created:

Taskfile.yml - Task runner configuration

Revision 32 - 2026-01-25

User Authentication System:

Added complete user authentication with JWT tokens
- Registration with real-time validation (username/email availability check)
- Login by email OR username
- Logout functionality
- Password reset with token (logged to console, no SMTP)
- Email verification required before login
Rate limiting for security
- Username/email check: 150 requests/minute (prevents enumeration)
- Forgot password: 5 requests/hour (prevents abuse)
Password requirements with real-time feedback
- Minimum 8 characters
- At least 1 lowercase, 1 uppercase, 1 digit
- Color-coded validation in registration form
Username validation
- 3-30 characters
- Alphanumeric and underscore only
- Trimmed (no leading/trailing spaces)
Anonymous guest mode
- "Continue as guest" option on landing page
- Basic auth credentials work for guest mode (if configured)
- Warning banner for anonymous users (bookshelf not saved)
User dropdown in navigation
- Shows username for logged-in users
- Shows "Guest" for anonymous users
- Login/Register links for unauthenticated
- Logout link for authenticated
Bookshelf migration on login
- Anonymous bookshelf items copied to user account
- Existing items not overwritten
JWT-based sessions
- HTTP-only cookie for security
- 24-hour expiration
- Optional jwt_secret in config.yaml

Files Created:

internal/infrastructure/persistence/migrations/012_users.sql - Users table, bookshelf user_id
internal/domain/user/user.go - User entity with validation
internal/domain/repository/user_repository.go - Repository interface
internal/infrastructure/persistence/user_repository.go - GORM implementation
internal/server/auth.go - JWT utilities, rate limiting, auth handlers
internal/server/auth_templates.go - Auth page templates (landing, login, register, etc.)

Files Modified:

internal/infrastructure/persistence/models.go - Added UserID to BookshelfModel
internal/infrastructure/persistence/repositories.go - Added Users repository
internal/infrastructure/persistence/service.go - Added MigrateAnonBookshelf method
internal/config/config.go - Added JWTSecret to ServerConfig
internal/server/server.go - Added userRepo, authHandlers, auth routes, authMiddleware
internal/server/web.go - Added Auth field to PageData, auth translations, user dropdown, guest warning

New Dependencies:

github.com/golang-jwt/jwt/v5 - JWT token handling

Auth Page Templates:

Landing page with login/register/guest options
Login form with email/username support
Registration form with real-time validation
Forgot password form
Reset password form with password strength indicator
Message page for verification errors

Revision 31 - 2026-01-25

Folder-Based Audiobook Grouping:

Audio files in same folder now grouped as single audiobook entry
- Uses existing AudiobookGrouper for grouping logic
- Creates "folder" format audiobooks (similar to archive-based)
- Track structure stored in chapters JSONB field
- Properly links authors, narrators, and genres from metadata
@eaDir directories skipped during scan (Synology metadata folders)
M4B files processed individually (single-file audiobooks with chapters)

Cyrillic Encoding Fix:

Fixed Windows-1251 text incorrectly read as Latin-1 in ID3 tags
- Added fixCyrillicEncoding() to detect and convert mojibake
- Uses golang.org/x/text/encoding/charmap for Windows-1251 decoding
Fixed UTF-16 metadata misread as UTF-8 (common in OGG Vorbis comments)
- Detects "Б . А к у н и н" pattern (null bytes appearing as spaces)
- Strips null bytes and removes alternating space pattern
- Prevents PostgreSQL "invalid byte sequence for encoding UTF8: 0x00" error

OGG Duration Fix:

Fixed Vorbis sample rate detection (was reading wrong offset)
- After 8-byte codec header, skip 4 bytes (remaining version + channels)
- Previously skipped 5 bytes, reading sample rate from wrong position
- Fixed duration calculation: 163M granules / 44100 Hz = 3697s (was calculating 947881s)

Folder Audiobook Playback Fix:

Added track endpoint support for "folder" format audiobooks
- handleAudioTrackDownload now handles "folder" format (was only zip/7z)
- New serveTrackFromFolder() function serves files directly from disk
- Uses http.ServeContent() for proper HTTP range request support (seeking)
- Security check ensures file path is within library root

Folder Audiobook Author/Title Parsing:

Always parse author and title from folder name (format: "Author - Title")
- Metadata "artist" field often contains narrator, not author
- Metadata artists now treated as narrators for folder audiobooks
- parseTitleFromFolderName() extracts title after separator
- Handles " - ", " – " (en-dash), and "-" separators
- Removes year suffixes like "(2020)" or "[2020]"

Folder Audiobook Cover Fix:

Fixed cover detection for folder-based audiobooks
- serveAudioCover now handles "folder" format correctly
- Uses first track path from chapters JSON for @eaDir lookup
- Added getFolderCoverInDir() helper for folder cover detection
Added SYNOAUDIO patterns to @eaDir cover lookup
- SYNOAUDIO_01APIC_03.jpg (Audio Station album art)
- SYNOAUDIO_01APIC_00/01/02.jpg variants

Files Modified:

internal/scanner/duration.go:
- Fixed OGG Vorbis sample rate offset (skip 4 bytes, not 5)
internal/scanner/scanner.go:
- Added audioGrouper to Scanner struct
- Modified walk to collect audio files separately, skip @eaDir
- Added processAudioFolders() to group and process collected audio files
- Added processAudioGroup() to create single audiobook entry from folder
- Author linking now always parses folder name first, treats metadata as narrators
internal/scanner/audioparser.go:
- Added fixCyrillicEncoding() for Windows-1251 detection and fix
- Added looksLikeMojibake(), hasCyrillic() helper functions
- Added looksLikeUTF16AsUTF8(), fixUTF16AsUTF8() for UTF-16 fix
- All metadata fields now passed through encoding fix
internal/scanner/audiobookgrouper.go:
- createGroup() now parses title from folder name
- Added parseTitleFromFolderName() function
- Added removeYearSuffix() and isDigits() helpers
- Metadata authors treated as narrators
internal/server/handlers.go:
- Added SYNOAUDIO patterns to getEaDirCover()
- serveAudioCover() now handles "folder" format
- Added getFolderCoverInDir() helper function
- Added encoding/json import
internal/server/web.go:
- Added serveTrackFromFolder() for folder audiobook playback

New Dependencies:

golang.org/x/text/encoding/charmap - Character set conversion

Revision 30 - 2026-01-25

@eaDir and Folder Cover Support:

Added Synology @eaDir thumbnail support for audiobook covers
- getEaDirCover() checks for SYNOPHOTO_THUMB_*.jpg patterns
- Looks in @eaDir/{filename}/ and @eaDir/ directories
- Fast cover serving without parsing audio files
Added folder cover support (cover.jpg, folder.jpg)
- getFolderCover() checks for common cover image names
- Supports both .jpg and .png formats
- Scanner detects folder covers during scan
Cover priority: @eaDir → folder cover → embedded in audio → archive extraction

Duration Extraction Improvements:

Fixed memory issues with large M4B files in archives
- Only read first 10MB for M4B/M4A (moov atom often at beginning)
- Prevents memory exhaustion for 700MB+ files
Improved fallback estimation bitrate (96kbps for AAC, more accurate)
Disabled track-specific cover loading (caused 502 errors for large files)

Bug Fixes:

Fixed nil pointer panic when accessing deleted audiobook
Added null check for book in handleWebAudioDetail

Files Modified:

internal/server/handlers.go:
- getEaDirCover() - Synology thumbnail lookup
- getFolderCover() - Folder cover lookup
- Updated serveAudioCover() to check @eaDir and folder covers first
internal/scanner/scanner.go:
- hasEaDirCover() - Check @eaDir during scan
- hasFolderCover() - Check folder covers during scan
- Limited archive file reading to 10MB for M4B duration extraction
internal/server/web.go:
- Disabled track-specific cover fetching
- Added nil check for book

Revision 29 - 2026-01-25

Audiobook Player Enhancements:

Extracted real audio duration from files during scan
- New duration.go with native Go parsers for MP4, MP3, FLAC, OGG duration extraction
- MP4: Parse mvhd atom for timescale and duration
- MP3: Parse Xing/Info header for frame count, fallback to bitrate estimation
- FLAC: Parse STREAMINFO block for sample rate and total samples
- OGG: Read last granule position from end of file
- Tested accuracy: MP3 within 0.01s, M4B exact match with ffprobe
Made track rows clickable to switch playback
- Added onclick="player.onTrackClick(this, event)" to track <li> elements
- Added event.stopPropagation() on checkbox, play button, download link to prevent row click
- Added cursor:pointer and hover effect to track list items
Implemented per-track position persistence
- trackPositions map stores position for each track by path
- saveCurrentTrackPosition() saves position before switching tracks
- saveState() now includes trackPositions in cookie
- loadState() restores per-track positions from cookie
- loadTrack() restores position for specific track being loaded
Made header with cover sticky (always visible)
- Added position: sticky; top: 0; z-index: 100; to .audio-header
- Header stays visible while scrolling through track list
Added track-specific cover images support
- New endpoint /web/audio/{id}/cover?file={trackPath} for per-track covers
- handleAudioTrackCover() extracts cover from specific audio file in archive
- extractTrackCoverFromZip() and extractTrackCoverFrom7z() helpers
- extractCoverFromAudioData() uses dhowden/tag to extract embedded cover
- Player attempts to load track-specific cover when switching tracks
- Falls back to book cover if track has no embedded cover

Files Created:

internal/scanner/duration.go - Audio duration extraction for MP4, MP3, FLAC, OGG

Files Modified:

internal/scanner/audioparser.go - Uses GetAudioDurationFromReader() for real duration
internal/server/web.go - Audiodetail template changes:
- Track rows clickable with onclick handler
- Per-track position storage in AudioPlayer JS class
- Sticky header CSS
- Track cover update on switch
- New handleAudioTrackCover() handler
internal/server/server.go - Added /audio/{id}/cover route

Revision 28 - 2026-01-25

Bug Fixes:

Fixed bookshelf "Added" state not persisting after navigation
- Added OnBookshelf field to BookView struct
- Load user's bookshelf IDs and check when building book views
- Template now shows "Added" button if book is already on shelf
Fixed bookshelf button not working on mobile/audiodetail page
- Changed audiodetail bookshelf link from <a href> to JavaScript onclick
- Now uses same bookshelfAction() function as regular book list
Fixed pagination showing "Next" when books count equals page size
- Changed hasMore calculation from len(books) >= pageSize to use pagination.TotalCount
- Now correctly shows "Next" only when more pages exist
Fixed audio cover not displaying for 7z audiobooks
- Check book.IsAudiobook flag, not just audio format extension
- Handle standalone 7z/zip archive files as archives (not audio files)
- Correctly build archive path from book.Path + book.Filename

Files Modified:

internal/server/web.go - Added OnBookshelf to BookView, updated handlers
internal/server/handlers.go - Fixed audio cover extraction for 7z archives
internal/domain/repository/bookshelf_repository.go - Added GetBookIDs interface
internal/infrastructure/persistence/bookshelf_repository.go - Implemented GetBookIDs
internal/infrastructure/persistence/service.go - Added GetBookShelfIDs service method

Revision 27 - 2026-01-25

Changes:

Added missing file detection for regular files (cat_type=0)
- Scanner now detects when standalone files (e.g., 7z audiobooks) are deleted/renamed
- Prompts for confirmation before marking as unavailable (respects auto_clean config)
- Similar to existing ZIP archive cleanup logic
Added audio cover display for audiobook detail pages
- Set HasCover: true for audiobooks in handleWebAudioDetail
- Cover extraction from audio files (MP3, M4B, M4A, FLAC, OGG, OPUS)
- Supports covers embedded in files inside ZIP and 7z archives
- Template has onerror fallback to hide image if no cover found
Fixed ZIP cover extraction to buffer data for tag.ReadFrom (requires io.ReadSeeker)
Fixed duplicate detection to separate audiobooks from text books
- Audiobooks only match other audiobooks
- Non-audiobooks only match other non-audiobooks
- Prevents audiobook being marked as duplicate of FB2 with same title

Files Modified:

internal/scanner/scanner.go - Added checkMissingRegularFiles() method
internal/infrastructure/persistence/service.go - Added GetRegularFileBooks(), MarkBooksUnavailable()
internal/infrastructure/persistence/book_repository.go - Added is_audiobook to duplicate detection queries
internal/server/web.go - Set HasCover for audiobooks in handleWebAudioDetail
internal/server/handlers.go - Fixed ZIP audio cover extraction buffering

New Dependencies:

github.com/saracen/go7z - 7z archive reading for cover extraction

Revision 26 - 2026-01-25

Changes:

Optimized duplicate detection performance
- Added functional indexes on LOWER(title) for case-insensitive matching
- New index idx_books_dup_strong for strong mode (title + format + filesize)
- New index idx_books_dup_normal for normal mode (title)
Implemented incremental duplicate detection
- Only checks newly added books against existing ones
- O(n) for n new books instead of O(N log N) for N total books
- Tracks new book IDs during scan for targeted duplicate checking
Added progress reporting during duplicate detection
- Logs progress every 500 books processed
- Shows percentage completion

Files Modified:

internal/infrastructure/persistence/migrations/011_duplicate_detection_index.sql - New indexes
internal/domain/repository/book_repository.go - Added MarkDuplicatesIncremental interface
internal/infrastructure/persistence/book_repository.go - Implemented incremental method
internal/infrastructure/persistence/service.go - Added service method
internal/scanner/scanner.go - Track new book IDs, use incremental detection

Revision 25 - 2026-01-25

Changes:

Implemented full-featured audiobook player UI
- Sticky player bar at bottom of page with gradient design
- Progress bar with seek functionality (click or drag to seek)
- Time display: current time / total duration
- Playback controls: prev, -15s, play/pause, +15s, next
- Speed control: 0.5x, 0.75x, 1x, 1.25x, 1.5x, 1.75x, 2x (saved in cookie)
- Volume toggle (mute/unmute)
- Buffering indicator on progress bar
- Auto-play next track when current ends
Track list enhancements:
- Now-playing highlight with pulsing icon animation
- Auto-scroll to current track
- Auto-expand collapsed folder when track plays
- Track icon changes from music note to speaker when playing
Cookie-based progress persistence:
- Saves current track index and position per book (30 days)
- Restores position when returning to audiobook page
- Playback speed saved globally (365 days)
Keyboard shortcuts:
- Space: play/pause
- Left/Right arrows: seek -5s/+5s (Shift: -30s/+30s)
- Up/Down arrows: volume +/-10%
- M: toggle mute
- N: next track
- P: previous track (or restart if >3s played)
Mobile responsive design:
- Simplified controls on small screens
- Touch-friendly seek bar
Player bar positioned below track list (sticky, not fixed at viewport bottom)

Files Modified:

internal/server/web.go - Complete rewrite of audiodetail template with AudioPlayer class

Revision 24 - 2026-01-10

Changes:

Added more progress phases during scan
- "Loading catalogs from database..." - when loading ZIP catalog cache
- "Detecting duplicates..." - when running duplicate detection
- Previously these phases showed nothing, making scan appear stuck
Fixed progress not displaying due to buffering and throttling
- Added os.Stdout.Sync() to flush output immediately
- Only throttle "scanning" phase, always show phase changes
Skip duplicate detection if no new books added (major performance fix)
- Duplicate detection on 568K books was taking 3+ minutes
- Now skips entirely when BooksAdded == 0

Files Modified:

internal/scanner/scanner.go - Added phase reports, fixed throttle, skip duplicates if no new books
cmd/sopds/main.go - Handle new phases in printProgress, add stdout sync

Revision 23 - 2026-01-10

Changes:

Added scanner.auto_clean config option for missing archives
- ask (default) - prompt user for confirmation
- yes - auto-delete without asking
- no - skip check entirely, never delete
Fixed DeleteInCatalogs not deleting catalog entries
- Was only deleting books, leaving catalog entries in DB
- Now deletes both books AND catalog entries
- Fixes repeated prompts about missing archives on every scan

Files Modified:

internal/config/config.go - Added AutoClean field to ScannerConfig
internal/scanner/scanner.go - Check auto_clean config before prompting
internal/infrastructure/persistence/book_repository.go - DeleteInCatalogs now also deletes catalogs
config.yaml - Added auto_clean: yes setting

Revision 22 - 2026-01-10

Changes:

Added play buttons for audio tracks in audiobook detail page
- Green play button next to each track's download button
- Click to play/pause, icon toggles between play/pause
- Hidden HTML5 audio element for streaming
- Auto-stops previous track when playing new one
- Resets icon when track ends

Files Modified:

internal/server/web.go - Added track-play button, playTrack() JS function, audio element, CSS styles

Revision 21 - 2026-01-10

Changes:

Fixed audiobook filter showing all authors instead of only audiobook authors
- GetFilterOptions was missing AudioOnly filter application
- Added query.Filters.WithAudioOnly() when opts.AudioOnly is true
- Fixes Chrome crash on mobile when opening author filter dropdown

Files Modified:

internal/infrastructure/persistence/service.go - Added AudioOnly filter to GetFilterOptions

Revision 20 - 2026-01-08

Changes:

Implemented file logging from config with rolling support
- logging.file - log file path
- logging.max_size - max size in MB before rotation (default 10)
- logging.max_backups - number of old log files to keep (default 3)
- rollingLogWriter struct implements io.Writer with automatic rotation
- Rotated files named .1, .2, .3 etc (newest first)
- Falls back to stderr with warning if file/rotation fails

Files Modified:

cmd/sopds/main.go - Added rollingLogWriter type, setupLogging() function
internal/config/config.go - Added MaxSize, MaxBackups to LoggingConfig
config.yaml - Added max_size, max_backups settings

Revision 19 - 2026-01-08

Changes:

Fixed audiobook archive path bug
- Path was incorrectly storing full path including filename instead of directory
- Changed to Path: filepath.Dir(relPath) in both processAudioZip and processAudio7z
Added individual track download from archives
- New endpoint /web/audio/{id}/track?file=path serves individual files
- serveTrackFromZip() and serveTrackFrom7z() helper functions
- Tracks now store full path inside archive for download
Audiobook detail page track selection UI
- Select all checkbox with indeterminate state support
- Part-level checkboxes for collections (multi-directory audiobooks)
- Individual track checkboxes
- "Download Selected" button triggers sequential downloads
- Download link for each individual track
Added Path field to AudiobookTrack struct for storing full path inside archive
New translations: audio.selectall, audio.downloadsel (English and Ukrainian)
CSS styles for select controls, checkboxes, and track download buttons
Track count in book list now clickable - links to audiobook detail page /web/audio/{id}
- Changed from <span> to <a> with meta-link class
- Uses i18n audio.tracks translation instead of hardcoded "tracks"

Files Modified:

internal/scanner/scanner.go - Added Path to AudiobookTrack, fixed archive path bug, updated trackInfo struct
internal/server/server.go - Added /audio/{id}/track route
internal/server/web.go - Added handleAudioTrackDownload handler, serveTrackFromZip, serveTrackFrom7z, Path field to AudioTrack, checkboxes/JS/CSS in audiodetail template, new translations, clickable track count in book list

Revision 18 - 2026-01-08

Changes:

Fixed book ID not returned after GORM upsert in Save() method
- GORM populates model.ID via RETURNING clause, but domain book was not updated
- Caused book_id=0 to be passed to AddBookAuthor, resulting in FK constraint violations
- Added b.SetID(book.ID(model.ID)) after Create to propagate ID back to domain object
Added audio formats to config.yaml (mp3, m4b, m4a, flac, ogg, opus)
Audiobook titles now use folder name instead of filename (better metadata)
Added "Audiobooks" menu item in web UI to browse all audiobooks
- New /web/audio route with full filter support (language, author, genre)
- Headphones icon in both upper navigation and menu grid
- Translations for English and Ukrainian ("nav.audio", "main.audiobooks")
Audiobook author parsing from topmost folder name
- Parses "Author - Title" or "Author_-_Title" format
- Splits author name into first_name/last_name
- Strips year suffixes like "[2007]" or "(2007)" from title
- Falls back to Unknown Author if no separator found
ZIP audiobook grouping: ZIP with audio files = ONE audiobook entry
- isAudioZip() detects ZIPs containing audio files
- processAudioZip() creates single book entry with aggregated metadata
- Author/title parsed from top-level folder inside ZIP
- Total duration estimated from file sizes
- Track count stored
Audiobook detail page with tree view
- /web/audio/{id} route shows audiobook structure
- Displays parts/chapters for collections (multi-directory ZIPs)
- Displays flat track list for simple audiobooks
- Shows duration for each track/part
- Download ZIP button, add to bookshelf
- Modern collapsible UI with Font Awesome icons
Scanner stores audiobook structure in chapters JSONB field
- AudiobookStructure, AudiobookPart, AudiobookTrack types in scanner
- Type: "book" for flat structure, "collection" for multi-directory
- Each track stores name, duration, size
- Each part stores name, total duration, list of tracks
Fixed audiobook ZIP parsing to use top-level folder inside ZIP for author/title
- Expected structure: ZIP / TopFolder (Author - Title) / [SubFolders] / tracks
- Subfolders under top-level folder become parts/chapters
- Files directly under top-level folder become tracks (simple audiobook)
Added 7z archive support
- New dependency: github.com/bodgit/sevenzip
- process7z() - processes 7z archives same as ZIP
- isAudio7z() - detects audio 7z files
- processAudio7z() - handles audiobook 7z files
- Same audiobook structure parsing as ZIP (top-level folder, parts, tracks)

Files Modified:

internal/infrastructure/persistence/book_repository.go - Fixed Save() to update domain book ID
internal/scanner/scanner.go - Parse author from topmost folder, AudioMeta struct, parseAudioFolderName(), parseAuthorFullName(), isAudioZip(), processAudioZip(), process7z(), isAudio7z(), processAudio7z(), AudiobookStructure/Part/Track types
internal/server/web.go - Added handleWebAudio handler, handleWebAudioDetail handler, audiodetail template, AudiobookStructure/AudiobookPart/AudioTrack/AudioDetailData types, formatDuration template function, audio translations
internal/server/server.go - Added /web/audio and /web/audio/{id} routes
internal/infrastructure/persistence/service.go - Added AudioOnly to SearchOptions
internal/infrastructure/persistence/scopes.go - Added OnlyAudiobooks scope
internal/domain/repository/filters.go - Added AudioOnly filter and WithAudioOnly method
config.yaml - Added audio formats to library.formats
go.mod, go.sum - Added github.com/bodgit/sevenzip dependency

Revision 17 - 2026-01-08

Changes:

Fixed author race condition in concurrent scanner workers
- Added unique constraint on authors(first_name, last_name)
- Updated GetOrCreate to use ON CONFLICT DO NOTHING with re-fetch pattern
- Fixed PostgreSQL sequence not incrementing after explicit ID insert in migration
Increased series.ser column from 64 to 128 characters (some series names exceed 64)
All FB2 metadata (authors, genres, series) now properly linked to books

Files Created:

internal/infrastructure/persistence/migrations/009_author_unique_constraint.sql
internal/infrastructure/persistence/migrations/010_increase_series_name.sql

Files Modified:

internal/infrastructure/persistence/author_repository.go - Race-safe GetOrCreate with ON CONFLICT
internal/infrastructure/persistence/models.go - SeriesModel.Name size:128

Revision 16 - 2026-01-07

Changes:

Added audiobook support with full metadata extraction
Supports audio formats: MP3, M4B, M4A, FLAC, OGG, OPUS
Audio metadata extraction using dhowden/tag library (ID3, MP4 atoms, Vorbis comments)
Narrator tracking via author role ('author' vs 'narrator') in bauthors table
Duration display for audiobooks (formatted as "Xh Ym")
Track count display for multi-file audiobooks
Web UI shows headphones icon badge for audiobooks
OPDS feeds include proper audio MIME types
Fixed missing unique constraint on (filename, path) for ON CONFLICT
Fixed JSONB chapters column: changed from string to *string to allow NULL
Added audiobook fields to domain Book and all conversion layers
Fixed slow scan performance with multiple optimizations:
- Enabled SET LOCAL synchronous_commit = off within transactions
- All ZIP operations wrapped in single transaction (one commit per ZIP)
- Batch availability updates for existing books
- Added GetBookMapByCatalog() for 1-query existence check (replaces N queries per ZIP)
- Added Service.Transaction() method for transactional batch operations
- Recommended: keep scanner.workers at 4-8 to avoid I/O saturation
Fixed bauthors FK constraint violation: filter out zero IDs from failed GORM upserts before linking authors
Fixed missing FB2 author/genre/series linking (never implemented in Go version)
- Added bookWithMeta struct to store parsed FB2 metadata alongside book records
- ZIP batch processing now properly extracts and links authors, genres, and series
- Individual file processing (processFile) now also links FB2 metadata
- Added parseFB2MetadataFull() function that returns full metadata for linking
- Falls back to Unknown Author only when no authors found in FB2 metadata

New Database Fields (books table):

duration_seconds - Total duration in seconds
bitrate - Audio bitrate
is_audiobook - Boolean flag
track_count - Number of tracks (for multi-file audiobooks)
chapters - JSONB column for chapter data

New Database Fields (bauthors table):

role - Author role ('author' or 'narrator')

Files Created:

internal/infrastructure/persistence/migrations/006_audiobook_support.sql - Database migration for audio fields
internal/infrastructure/persistence/migrations/007_unique_filename_path.sql - Unique constraint for upsert
internal/infrastructure/persistence/migrations/008_fix_search_trigger.sql - Recreate trigger with UPDATE OF condition
internal/scanner/audioparser.go - Audio metadata extraction
internal/scanner/audiobookgrouper.go - Multi-file audiobook grouping logic

Files Modified:

internal/database/models.go - Added audio fields to Book struct
internal/infrastructure/persistence/models.go - Added GORM audio fields (*string for chapters), Role to BookAuthorModel
internal/domain/book/book.go - Added audiobook fields, updated Reconstruct(), added getters
internal/domain/book/value_objects.go - Added audio Format constants, IsAudio() method
internal/domain/repository/book_repository.go - Added narrator methods
internal/infrastructure/persistence/book_repository.go - Implemented narrator repository methods
internal/infrastructure/persistence/mappers.go - Added audiobook fields to BookToModel/BookToDomain
internal/infrastructure/persistence/adapters.go - Added audiobook fields to BookToLegacy
internal/infrastructure/persistence/database.go - Added SetAsyncCommit(), SetSyncCommit() methods
internal/infrastructure/persistence/service.go - Added narrator service methods, audiobook fields in Reconstruct calls, UpdateBookAvailBatch, Transaction() with SET LOCAL, GetBookMapByCatalog()
internal/infrastructure/persistence/book_repository.go - Added UpdateAvailabilityBatch, GetBookMapByCatalog for batch operations
internal/domain/repository/book_repository.go - Added UpdateAvailabilityBatch, GetBookMapByCatalog interface methods
internal/opds/feed.go - Added audio MIME types and FormatDuration helper
internal/scanner/scanner.go - Added audio file detection, metadata parsing, optimized ZIP scanning with GetBookMapByCatalog (1 query per ZIP), transaction batching
internal/server/web.go - Updated BookView struct and templates for audiobook display
internal/config/config.go - Added audio formats to defaults

Web UI Changes:

Audiobooks display headphones icon next to title
Duration shown instead of file size for audiobooks
Narrators displayed with microphone icon
Track count shown for multi-file audiobooks

Revision 15 - 2026-01-07

Changes:

Fixed field shadowing bug in i18n: BooksData.Languages renamed to FilterLangs
The old field shadowed PageData.Languages causing "can't evaluate field Code in type string" error on search page
Fixed page size, pagination and filters losing URL parameters
Converted page size buttons to use JavaScript setPageSize() function
Converted pagination links to use JavaScript goToPage() function
Converted "Clear all" filters to use JavaScript clearFilters() function
All navigation now properly preserves search queries and filter parameters
Fixed fname/lname filters not working (were parsed but never applied)
Added FirstNameFilter and LastNameFilter to SearchOptions
Added WithAuthorFirstNameExact and WithAuthorLastNameExact scopes
Added filters to all book listing pages (genre, author, series, language, new books)
All book lists now support filtering by: language, author first/last name, genre
Added NewPeriod to SearchOptions for new books page
Updated CLAUDE.md with systemd service management commands
Fixed filter dropdowns showing language codes instead of human-readable names
Fixed filter dropdowns only showing options from current page (now shows ALL options in scope)
Added GetFilterOptions method to query distinct filter values for entire scope
Added LangOption struct with Code and Name fields for proper language display
Added langsToOptions() and genresToLinkedItems() helper functions

Filters by page type:

Genre page: lang, fname, lname filters (genre already scoped)
Author page: lang, genre filters (author already scoped, fname/lname not applicable)
Series page: lang, fname, lname, genre filters
Language page: fname, lname, genre filters (lang already scoped)
New books page: lang, fname, lname, genre filters

Files Modified:

internal/server/web.go - Updated all book listing handlers to use GetFilterOptions, LangOption struct, helper functions
internal/infrastructure/persistence/service.go - Added GetFilterOptions method, FilterOptions struct
internal/infrastructure/persistence/scopes.go - Added exact name filter scopes
internal/domain/repository/filters.go - Added AuthorFirstName/AuthorLastName fields and methods
CLAUDE.md - Added systemd service commands (start/stop/restart/status)

Revision 14 - 2026-01-04

Changes:

Added internationalization (i18n) system for Web UI
Supports English (en) and Ukrainian (uk) languages
Extensible design - add new languages by editing supportedLanguages and translations map
Language switcher in navigation bar (JavaScript-based, preserves current URL)
Cookie-based language preference persistence (1 year)
Full translation of all templates: main, search, books, bookshelf, authors, genres, series, catalogs, languages, error pages
Help page with full translations

Translated UI Elements:

Navigation menu
Statistics labels
Browse menu items
Book listings (Show, Filters, All Languages, Previous, Next, No books found, Add to Shelf, Duplicates, Remove)
Empty state messages (No authors/genres/series/languages/catalogs found)
Bookshelf (title, empty message)

i18n Architecture:

Language struct with Code and Name fields
supportedLanguages slice - add new languages here
translations map[string]map[string]string - key → lang → text
T(lang, key) function for translation lookups
Template function {{t "key"}} for inline translations
newPageData() and addI18n() helpers initialize Lang and Languages fields
All handlers call setLangCookie() and addI18n() for consistent language support

Adding a New Language:

Add to supportedLanguages: {Code: "de", Name: "Deutsch"}
Add translations for all keys in translations map

Files Modified:

internal/server/web.go - i18n system, all templates translated, addI18n helper, handlers updated
internal/server/server.go - Added /web/help route

Revision 13 - 2026-01-04

Changes:

Enhanced search with separate title and author name fields
Added pattern filters for language, genre, and series (ILIKE matching)
Added language scope for searching within a language context
Schema migration: renamed doublicat to duplicate_of, favorite INT to BOOL
Fixed duplicate count display (only show when count > 1)
Added scoped search (search within current author/genre/series/catalog/language)

New Search Features:

Separate title search (q=) and author search (author=) fields
Both can be combined with AND logic
Pattern filters via URL parameters:
- lang_pattern=uk - filter by language pattern
- genre_pattern=comedy - filter by genre name
- series_pattern=Silo - filter by series name
Language scope: when browsing by language, searches stay within that language

Files Modified:

internal/infrastructure/persistence/service.go - SearchOptions with TitleQuery/AuthorQuery
internal/infrastructure/persistence/scopes.go - Added WithAuthorName, WithLangPattern, WithGenrePattern, WithSeriesPattern
internal/domain/repository/filters.go - Added AuthorNameQuery and pattern filter fields
internal/server/web.go - Updated search form with author field, language scope
internal/server/handlers.go - Updated OPDS search to use TitleQuery
internal/database/models.go - Changed Duplicate/Favorite field types

Migration:

005_schema_cleanup.sql - Renames doublicat to duplicate_of, converts favorite to boolean

Revision 12 - 2026-01-04

Changes:

Added PostgreSQL full-text search (FTS) for book searches
Search is now 440x faster (4.75ms vs 2099ms)
Uses tsvector column with GIN index
Title weighted higher than annotation in search results
Trigger auto-updates search vector on book insert/update

Files Created:

internal/infrastructure/persistence/migrations/004_fulltext_search.sql

Files Modified:

internal/infrastructure/persistence/scopes.go - WithKeywords uses FTS
internal/infrastructure/persistence/models.go - Added SearchVector field
internal/infrastructure/persistence/database.go - Multi-statement migrations

Revision 11 - 2026-01-04

Changes:

Migrated from raw pgx queries to GORM ORM with Domain-Driven Design (DDD)
Created domain layer with entities: Book, Author, Genre, Series, Catalog
Created repository interfaces in internal/domain/repository/
Implemented GORM repositories in internal/infrastructure/persistence/
Added Service layer as bridge between handlers and repositories
Moved SQL migrations to persistence package
Removed 1,500+ lines of raw SQL queries

Architecture:

internal/
├── domain/                    # Pure business logic
│   ├── book/                  # Book aggregate with value objects
│   ├── author/                # Author entity
│   ├── genre/                 # Genre entity
│   ├── series/                # Series entity
│   ├── catalog/               # Catalog entity
│   └── repository/            # Repository interfaces
│
├── infrastructure/persistence/ # GORM implementations
│   ├── database.go            # GORM connection + migrations
│   ├── models.go              # GORM models with tags
│   ├── mappers.go             # Domain <-> Model conversion
│   ├── scopes.go              # Reusable query scopes
│   ├── *_repository.go        # Repository implementations
│   ├── adapters.go            # Domain <-> Legacy type conversion
│   └── service.go             # Bridge layer for handlers
│
└── database/                  # Legacy (models only)
    └── models.go              # Legacy types for handlers

Files Created:

internal/domain/book/book.go - Book aggregate root
internal/domain/book/value_objects.go - Format, Language, Availability, Cover
internal/domain/author/author.go - Author entity
internal/domain/genre/genre.go - Genre entity
internal/domain/series/series.go - Series entity
internal/domain/catalog/catalog.go - Catalog entity
internal/domain/repository/*.go - Repository interfaces
internal/infrastructure/persistence/*.go - GORM implementations

Files Removed:

internal/database/postgres.go - Replaced by persistence/database.go
internal/database/queries.go - Replaced by repositories

Files Modified:

cmd/sopds/main.go - Uses persistence.NewDB and persistence.NewService
internal/server/server.go - Uses persistence.Service
internal/server/handlers.go - Uses service methods
internal/server/web.go - Uses service methods
internal/scanner/scanner.go - Uses persistence.Service

Revision 10 - 2026-01-03

Changes:

Added "See duplicates" feature to view all versions of a book
Books with duplicates show a "Duplicates (N)" button linking to duplicates page
Duplicate books show "Duplicates" button to find original and other copies
Duplicates page displays original book and all its duplicates for easy comparison

Files Modified:

internal/database/queries.go - Added GetDuplicateCount, GetBookDuplicates functions
internal/server/web.go - Added DuplicateOf/DuplicateCount to BookView, handleWebDuplicates handler, updated templates
internal/server/server.go - Added /web/duplicates/{id} route
CLAUDE.md - Updated documentation with new features and endpoints

Revision 9 - 2025-12-26

Changes:

Implemented cover image extraction from FB2 files (base64 decoding)
Implemented bookshelf feature (user reading lists)
Added web bookshelf page with add/remove functionality
Added "Add to Bookshelf" button on all book listings
Cover images served from /book/{id}/cover endpoint

Files Modified:

internal/server/handlers.go - Added cover extraction, bookshelf handlers
internal/server/web.go - Added bookshelf template, TotalCount field, button styles
internal/database/queries.go - Added bookshelf database functions
internal/server/server.go - Added bookshelf routes

Revision 8 - 2025-12-26

Changes:

Updated all documentation files (CLAUDE.md, README.md, PROGRESS.md)
Added comprehensive documentation for all features

Files Modified:

CLAUDE.md - Full rewrite with architecture, commands, API endpoints
README.md - Updated with all new features and configuration options
PROGRESS.md - Added all revision history

Revision 7 - 2025-12-26

Changes:

Implemented FB2 to EPUB converter in pure Go (internal/converter/)
Implemented FB2 to MOBI conversion using Calibre's ebook-convert
Wired converters to server handlers (/book/{id}/epub, /book/{id}/mobi)
EPUB conversion includes: metadata, body content, images, TOC, stylesheet

Files Created:

internal/converter/converter.go

Files Modified:

internal/server/server.go - Added converter initialization
internal/server/handlers.go - Implemented handleConvertEPUB, handleConvertMOBI
go.mod, go.sum - Added github.com/google/uuid dependency

Revision 6 - 2025-12-26

Changes:

Complete Web UI rewrite with modern design
Fixed download 404 error (OPDSPrefix in templates)
Fixed pagination 404 error (WebPrefix in URLs)
Added EPUB/MOBI download buttons (conditional on config)
Implemented hierarchical letter navigation (1→2→3 char drill-down)
Added cloud-style prefix buttons for author/series navigation
Modern CSS with gradients, Font Awesome 6 icons, responsive design
Fixed template field names (Stats.BooksCount instead of Stats.Books)

Files Modified:

internal/server/web.go - Complete rewrite
internal/database/queries.go - Added GetAuthorsByPrefix, GetSeriesByPrefix

Revision 5 - 2025-12-26

Changes:

Added MySQL to PostgreSQL migration command (import-mysql)
Fixed MySQL connection timeout during large imports (read all data to memory first)
Fixed PostgreSQL constraint violations (drop/recreate FK constraints during import)
Added --clear flag to clear PostgreSQL tables before import
Added progress reporting during migration

Files Modified:

cmd/sopds/main.go - Added import-mysql command with all migration functions

Revision 4 - 2025-12-25

Changes:

Added progress tracking and ETA to library scanner
Added ProgressInfo struct with: TotalFiles, ProcessedFiles, BooksAdded, BooksSkipped, Elapsed, Rate, ETA, Phase
Added ProgressCallback for CLI progress display
Pre-count phase before scanning for accurate progress
Progress bar display with spinner animation

Files Modified:

internal/scanner/scanner.go - Added progress tracking
cmd/sopds/main.go - Added progress display for scan command

Revision 3 - 2025-12-25

Changes:

Added sopds-go/README.md with full documentation
- Quick start guide
- Configuration reference
- API endpoints
- Project structure

Files Modified:

sopds-go/README.md (new)

Revision 2 - 2025-12-25

Changes:

Added init.sql for PostgreSQL database initialization
- Creates database and user
- Grants schema permissions (required for PostgreSQL 15+)
Fixed 002_genres.sql - escaped single quotes in Russian text

Files Modified:

sopds-go/init.sql (new)
sopds-go/internal/database/migrations/002_genres.sql (fixed)

Revision 1 - 2025-12-25

Changes:

Complete rewrite of SOPDS from Python to Go
New technology stack:
- Go 1.21+ with PostgreSQL (replacing MySQL)
- YAML configuration format (replacing INI)
- chi router for HTTP
- pgx for PostgreSQL
- encoding/xml for OPDS feeds

New Files Created (sopds-go/):

cmd/sopds/main.go - CLI with cobra (start/stop/status/scan/migrate/init)
internal/config/config.go - YAML configuration parser
internal/database/postgres.go - PostgreSQL connection pool
internal/database/models.go - Data models (Book, Author, Genre, Series, etc.)
internal/database/queries.go - All database queries
internal/database/migrations/001_initial.sql - PostgreSQL schema
internal/database/migrations/002_genres.sql - Genre data
internal/scanner/scanner.go - Parallel directory scanner with goroutines
internal/scanner/fb2parser.go - FB2 metadata extraction
internal/server/server.go - HTTP server with chi router
internal/server/handlers.go - OPDS/Web request handlers
internal/opds/feed.go - OPDS Atom feed generation
config.yaml - Default configuration template
go.mod, go.sum - Go module files
CLAUDE.md - Project documentation for Claude Code
PROGRESS.md - Development progress tracking

Key Improvements over Python version:

Parallel scanning with configurable worker count
RESTful routes instead of ?id= parameter encoding
Proper graceful shutdown with signal handling
Embedded SQL migrations
Type-safe YAML configuration
Modern Web UI with responsive design
Built-in FB2 to EPUB conversion (pure Go)
MySQL migration tool for existing databases

Feature Summary

Feature	Status
OPDS 1.2 Catalog	Done
Web Interface	Done
FB2 Metadata Extraction	Done
ZIP Archive Scanning	Done
Parallel Scanning	Done
Progress Tracking	Done
Basic Auth	Done
Scheduled Scanning	Done
FB2 to EPUB	Done
FB2 to MOBI	Done
MySQL Migration	Done
PostgreSQL Backup/Restore	Done
Cover Images	Done
Duplicate Detection	Done
Duplicates Viewer	Done
Bookshelf	Done
GORM ORM	Done
Domain-Driven Design	Done
Full-Text Search	Done
Advanced Search (title+author)	Done
Pattern Filters	Done
Scoped Search	Done
Internationalization (i18n)	Done
Help Page	Done
Audiobook Support	Done
User Authentication	Done

Database Tables

Table	Description
books	Book metadata and file info
authors	Author names
bauthors	Book-author relationships
genres	Genre definitions
bgenres	Book-genre relationships
series	Series names
bseries	Book-series relationships
catalogs	Directory structure
bookshelf	User reading lists
users	User accounts

FilesExpand file tree

PROGRESS.md

Latest commit

History

PROGRESS.md

File metadata and controls

PROGRESS.md

Project: Simple OPDS Catalog (SOPDS) - Go Version

Current Version: 1.3.3

Revision 86 - 2026-07-12

Revision 85 - 2026-07-12

Revision 84 - 2026-07-11

Revision 83 - 2026-07-11

Revision 82 - 2026-07-11

Revision 81 - 2026-07-11

Revision 80 - 2026-07-11

Revision 79 - 2026-07-11

Revision 78 - 2026-07-11

Revision 77 - 2026-07-11

Revision 76 - 2026-07-11

Revision 75 - 2026-05-11

Revision 74 - 2026-05-11

Revision 73 - 2026-05-11

Revision 72 - 2026-05-11

Revision 71 - 2026-05-10

Revision 70 - 2026-05-10

Revision 69 - 2026-05-10

Revision 68 - 2026-05-10

Revision 67 - 2026-05-10

Revision 66 - 2026-05-10

Revision 65 - 2026-05-10

Revision 64 - 2026-05-10

Revision 63 - 2026-05-10

Revision 62 - 2026-05-10

Revision 61 - 2026-05-10

Revision 60 - 2026-05-10

Revision 59 - 2026-05-10

Revision 58 - 2026-05-10

Revision 57 - 2026-05-10

Revision 56 - 2026-05-10

Revision 55 - 2026-05-10

Revision 54 - 2026-05-09

Revision 53 - 2026-05-09

Revision 52 - 2026-03-12

Revision 51 - 2026-02-08

Revision 50 - 2026-01-30

Revision 49 - 2026-01-30

Revision 48 - 2026-01-30

Revision 47 - 2026-01-30

Revision 46 - 2026-01-30

Revision 45 - 2026-01-30

Revision 44 - 2026-01-30

Revision 43 - 2026-01-30

Revision 42 - 2026-01-30

Revision 41 - 2026-01-30

Revision 40 - 2026-01-28

Revision 39 - 2026-01-28

Revision 38 - 2026-01-26

Revision 37 - 2026-01-26

Revision 36 - 2026-01-26

Revision 35 - 2026-01-25

Revision 34 - 2026-01-25

Revision 33 - 2026-01-25

Revision 32 - 2026-01-25

Revision 31 - 2026-01-25

Revision 30 - 2026-01-25

Revision 29 - 2026-01-25

Revision 28 - 2026-01-25

Revision 27 - 2026-01-25

Revision 26 - 2026-01-25

Revision 25 - 2026-01-25

Revision 24 - 2026-01-10

Revision 23 - 2026-01-10

Revision 22 - 2026-01-10

Revision 21 - 2026-01-10

Revision 20 - 2026-01-08

Revision 19 - 2026-01-08

Revision 18 - 2026-01-08

Revision 17 - 2026-01-08

Revision 16 - 2026-01-07