diff --git a/.gitignore b/.gitignore index 2691730b..c10023eb 100644 --- a/.gitignore +++ b/.gitignore @@ -26,8 +26,9 @@ static/version.json desktop/node_modules/ node_modules/ -# Playwright browser-test output (tests/e2e) +# Playwright browser-test output (tests/e2e), and the acceptance run's report test-results/ +acceptance-results/ playwright-report/ blob-report/ desktop/src-tauri/target/ diff --git a/README.md b/README.md index af779ba5..f9bb21dd 100644 --- a/README.md +++ b/README.md @@ -214,6 +214,8 @@ xattr -dr com.apple.quarantine desktop/src-tauri/target/aarch64-apple-darwin/rel Python 3.12 or newer, `ffmpeg` on your PATH, and [uv](https://github.com/astral-sh/uv). Around 170 MB of free disk for the Demucs model, which downloads automatically on first run. +Optional, for song identification with an AcoustID key: an FFmpeg built with chromaprint (Debian and Ubuntu's `ffmpeg` is), or Chromaprint's `fpcalc` on your PATH (`brew install chromaprint`, `apt install libchromaprint-tools`, or your distro's `chromaprint` package). `./run.sh setup` installs it when your FFmpeg needs it. Without either, songs are identified by their tags only. + #### macOS / Linux (one-shot) ```sh @@ -335,6 +337,7 @@ The library is persistent by default (`STEMDECK_PERSIST_LIBRARY=1`), so tracks a | `STEMDECK_FFMPEG_DIR` | (none) | Directory containing a bundled ffmpeg binary. | | `STEMDECK_FFMPEG` | `ffmpeg` | Path to the ffmpeg executable. | | `STEMDECK_FFPROBE` | `ffprobe` | Path to the ffprobe executable. | +| `STEMDECK_FPCALC` | `fpcalc` | Path to Chromaprint's fpcalc, used for song fingerprinting when FFmpeg has no chromaprint muxer. | | `STEMDECK_MAX_DURATION_SEC` | `1200` | Reject audio longer than this (seconds). | | `STEMDECK_JOB_TTL_SECONDS` | `86400` | How long to keep job dirs on disk. | | `STEMDECK_MAX_PENDING_JOBS` | `3` | Max queued jobs before returning 503. | diff --git a/app/_vendor/opencc/LICENSE b/app/_vendor/opencc/LICENSE new file mode 100644 index 00000000..9f7a310e --- /dev/null +++ b/app/_vendor/opencc/LICENSE @@ -0,0 +1,56 @@ +Apache License +Version 2.0, January 2004 +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. + +"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. + +"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. + +"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. + +"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. + +"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. + +"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). + +"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. + +"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." + +"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: + + 1. You must give any other recipients of the Work or Derivative Works a copy of this License; and + + 2. You must cause any modified files to carry prominent notices stating that You changed the files; and + + 3. You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and + + 4. If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. + +You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + diff --git a/app/_vendor/opencc/zh_t2s.txt b/app/_vendor/opencc/zh_t2s.txt new file mode 100644 index 00000000..6539201b --- /dev/null +++ b/app/_vendor/opencc/zh_t2s.txt @@ -0,0 +1,3233 @@ +# Traditional Chinese characters, each with the simplified character it folds +# to when two names are compared (app/pipeline/zh_variants.py, and its twin +# static/js/zhVariants.js, which holds the same pairs). +# +# Derived from OpenCC's TSCharacters.txt, commit 2939943b of +# https://github.com/BYVoid/OpenCC, Copyright Carbo Kuo (BYVoid) and the OpenCC +# contributors, licensed under the Apache License, Version 2.0 +# (https://www.apache.org/licenses/LICENSE-2.0). Changed from the original: +# only the first simplified form of each character is kept, and characters +# that are their own first form are left out. +# +# One pair a line: the traditional character, a space, the simplified one. +㑯 㑔 +㑳 㑇 +㑶 㐹 +㓨 刾 +㗲 𠵾 +㘚 㘎 +㜄 㚯 +㜏 㛣 +㜢 𡞱 +㠏 㟆 +㠣 𫵷 +㥮 㤘 +㩜 㨫 +㩳 㧐 +㩵 擜 +㺏 𤠋 +䁪 𥇢 +䁻 䀥 +䃮 鿎 +䊷 䌶 +䋙 䌺 +䋚 䌻 +䋹 䌿 +䋻 䌾 +䍦 䍠 +䎱 䎬 +䓣 𬜯 +䙡 䙌 +䜀 䜧 +䝼 䞍 +䡵 𫟦 +䥇 䦂 +䥑 鿏 +䥕 𬭯 +䥱 䥾 +䦛 䦶 +䦟 䦷 +䧢 𨸟 +䮄 𫠊 +䯀 䯅 +䰾 鲃 +䱷 䲣 +䱽 䲝 +䲁 鳚 +䲘 鳤 +䴉 鹮 +丟 丢 +並 并 +乾 干 +亂 乱 +亙 亘 +亞 亚 +佇 伫 +佈 布 +佔 占 +併 并 +來 来 +侖 仑 +侶 侣 +侷 局 +俁 俣 +係 系 +俔 伣 +俠 侠 +俥 伡 +俬 私 +倀 伥 +倆 俩 +倈 俫 +倉 仓 +個 个 +們 们 +倖 幸 +倫 伦 +倲 㑈 +偉 伟 +偑 㐽 +側 侧 +偵 侦 +偽 伪 +傌 㐷 +傑 杰 +傖 伧 +傘 伞 +備 备 +傢 家 +傭 佣 +傯 偬 +傳 传 +傴 伛 +債 债 +傷 伤 +傾 倾 +僂 偻 +僅 仅 +僉 佥 +僑 侨 +僕 仆 +僞 伪 +僤 𫢸 +僥 侥 +僨 偾 +僱 雇 +價 价 +儀 仪 +儁 俊 +儂 侬 +億 亿 +儈 侩 +儉 俭 +儎 傤 +儐 傧 +儔 俦 +儕 侪 +儘 尽 +償 偿 +優 优 +儲 储 +儷 俪 +儸 㑩 +儺 傩 +儻 傥 +儼 俨 +兇 凶 +兌 兑 +兒 儿 +兗 兖 +內 内 +兩 两 +冊 册 +冑 胄 +冪 幂 +凈 净 +凍 冻 +凜 凛 +凱 凯 +別 别 +刪 删 +剄 刭 +則 则 +剋 克 +剎 刹 +剗 刬 +剛 刚 +剝 剥 +剮 剐 +剴 剀 +創 创 +剷 铲 +劃 划 +劄 札 +劇 剧 +劉 刘 +劊 刽 +劌 刿 +劍 剑 +劏 㓥 +劑 剂 +劚 㔉 +勁 劲 +動 动 +務 务 +勛 勋 +勝 胜 +勞 劳 +勢 势 +勣 𪟝 +勩 勚 +勱 劢 +勳 勋 +勵 励 +勸 劝 +勻 匀 +匭 匦 +匯 汇 +匱 匮 +區 区 +協 协 +卹 恤 +卻 却 +卽 即 +厙 厍 +厠 厕 +厤 历 +厭 厌 +厲 厉 +厴 厣 +參 参 +叄 叁 +叢 丛 +吒 咤 +吳 吴 +吶 呐 +呂 吕 +咼 呙 +員 员 +唄 呗 +唸 念 +問 问 +啓 启 +啞 哑 +啟 启 +啢 唡 +喎 㖞 +喚 唤 +喪 丧 +喫 吃 +喬 乔 +單 单 +喲 哟 +嗆 呛 +嗇 啬 +嗊 唝 +嗎 吗 +嗚 呜 +嗩 唢 +嗰 𠮶 +嗶 哔 +嘆 叹 +嘍 喽 +嘓 啯 +嘔 呕 +嘖 啧 +嘗 尝 +嘜 唛 +嘩 哗 +嘮 唠 +嘯 啸 +嘰 叽 +嘵 哓 +嘸 呒 +嘽 啴 +噁 恶 +噓 嘘 +噚 㖊 +噝 咝 +噠 哒 +噥 哝 +噦 哕 +噯 嗳 +噲 哙 +噴 喷 +噸 吨 +噹 当 +嚀 咛 +嚇 吓 +嚌 哜 +嚐 尝 +嚕 噜 +嚙 啮 +嚥 咽 +嚦 呖 +嚧 𠰷 +嚨 咙 +嚮 向 +嚲 亸 +嚳 喾 +嚴 严 +嚶 嘤 +囀 啭 +囁 嗫 +囂 嚣 +囅 冁 +囈 呓 +囉 啰 +囌 苏 +囑 嘱 +囪 囱 +圇 囵 +國 国 +圍 围 +園 园 +圓 圆 +圖 图 +團 团 +垻 坝 +埡 垭 +埨 𫭢 +埰 采 +執 执 +堅 坚 +堊 垩 +堖 垴 +堝 埚 +堯 尧 +報 报 +場 场 +塊 块 +塋 茔 +塏 垲 +塒 埘 +塗 涂 +塚 冢 +塢 坞 +塤 埙 +塵 尘 +塸 𫭟 +塹 堑 +塿 𪣻 +墊 垫 +墜 坠 +墠 𫮃 +墮 堕 +墰 坛 +墳 坟 +墶 垯 +墻 墙 +墾 垦 +壇 坛 +壋 垱 +壎 埙 +壓 压 +壗 𡋤 +壘 垒 +壙 圹 +壚 垆 +壜 坛 +壞 坏 +壟 垄 +壠 垅 +壢 坜 +壩 坝 +壪 塆 +壯 壮 +壺 壶 +壼 壸 +壽 寿 +夠 够 +夢 梦 +夥 伙 +夾 夹 +奐 奂 +奧 奥 +奩 奁 +奪 夺 +奬 奖 +奮 奋 +奼 姹 +妝 妆 +姍 姗 +姦 奸 +娙 𫰛 +娛 娱 +婁 娄 +婦 妇 +婭 娅 +媧 娲 +媯 妫 +媰 㛀 +媼 媪 +媽 妈 +嫋 袅 +嫗 妪 +嫵 妩 +嫺 娴 +嫻 娴 +嫿 婳 +嬀 妫 +嬃 媭 +嬈 娆 +嬋 婵 +嬌 娇 +嬙 嫱 +嬡 嫒 +嬤 嬷 +嬪 嫔 +嬰 婴 +嬸 婶 +孃 娘 +孋 㛤 +孌 娈 +孫 孙 +學 学 +孻 𡥧 +孿 孪 +宮 宫 +寀 采 +寢 寝 +實 实 +寧 宁 +審 审 +寫 写 +寬 宽 +寵 宠 +寶 宝 +將 将 +專 专 +尋 寻 +對 对 +導 导 +尷 尴 +屆 届 +屍 尸 +屓 屃 +屜 屉 +屢 屡 +層 层 +屨 屦 +屬 属 +岡 冈 +峯 峰 +峴 岘 +島 岛 +峽 峡 +崍 崃 +崑 昆 +崗 岗 +崙 仑 +崢 峥 +崬 岽 +嵐 岚 +嵗 岁 +嵽 𫶇 +嵾 㟥 +嶁 嵝 +嶄 崭 +嶇 岖 +嶔 嵚 +嶗 崂 +嶠 峤 +嶢 峣 +嶧 峄 +嶨 峃 +嶮 崄 +嶸 嵘 +嶺 岭 +嶼 屿 +嶽 岳 +巋 岿 +巒 峦 +巔 巅 +巖 岩 +巘 𪩘 +巰 巯 +巹 卺 +帥 帅 +師 师 +帳 帐 +帶 带 +幀 帧 +幃 帏 +幓 㡎 +幗 帼 +幘 帻 +幟 帜 +幣 币 +幫 帮 +幬 帱 +幷 并 +幹 干 +幾 几 +庫 库 +廁 厕 +廂 厢 +廄 厩 +廈 厦 +廎 庼 +廕 荫 +廚 厨 +廝 厮 +廞 𫷷 +廟 庙 +廠 厂 +廡 庑 +廢 废 +廣 广 +廩 廪 +廬 庐 +廳 厅 +弒 弑 +弔 吊 +弳 弪 +張 张 +強 强 +彄 𫸩 +彆 别 +彈 弹 +彌 弥 +彎 弯 +彔 录 +彙 汇 +彠 彟 +彥 彦 +彫 雕 +彲 彨 +彿 佛 +後 后 +徑 径 +從 从 +徠 徕 +復 复 +徵 征 +徹 彻 +恆 恒 +恥 耻 +悅 悦 +悞 悮 +悵 怅 +悶 闷 +悽 凄 +惡 恶 +惱 恼 +惲 恽 +惻 恻 +愛 爱 +愜 惬 +愨 悫 +愴 怆 +愷 恺 +愾 忾 +慄 栗 +態 态 +慍 愠 +慘 惨 +慚 惭 +慟 恸 +慣 惯 +慤 悫 +慪 怄 +慫 怂 +慮 虑 +慳 悭 +慶 庆 +慺 㥪 +慼 戚 +慾 欲 +憂 忧 +憊 惫 +憐 怜 +憑 凭 +憒 愦 +憖 慭 +憚 惮 +憤 愤 +憫 悯 +憮 怃 +憲 宪 +憶 忆 +懇 恳 +應 应 +懌 怿 +懍 懔 +懞 蒙 +懟 怼 +懣 懑 +懤 㤽 +懨 恹 +懲 惩 +懶 懒 +懷 怀 +懸 悬 +懺 忏 +懼 惧 +懾 慑 +戀 恋 +戇 戆 +戔 戋 +戧 戗 +戩 戬 +戰 战 +戱 戯 +戲 戏 +戶 户 +扞 捍 +拋 抛 +拚 拼 +挩 捝 +挱 挲 +挾 挟 +捨 舍 +捫 扪 +捱 挨 +捲 卷 +掃 扫 +掄 抡 +掆 㧏 +掗 挜 +掙 挣 +掛 挂 +採 采 +揀 拣 +揚 扬 +換 换 +揮 挥 +揯 搄 +損 损 +搖 摇 +搗 捣 +搧 扇 +搵 揾 +搶 抢 +摑 掴 +摜 掼 +摟 搂 +摯 挚 +摳 抠 +摶 抟 +摺 折 +摻 掺 +撈 捞 +撏 挦 +撐 撑 +撓 挠 +撝 㧑 +撟 挢 +撣 掸 +撥 拨 +撫 抚 +撲 扑 +撳 揿 +撻 挞 +撾 挝 +撿 捡 +擁 拥 +擄 掳 +擇 择 +擊 击 +擋 挡 +擓 㧟 +擔 担 +據 据 +擠 挤 +擡 抬 +擣 捣 +擬 拟 +擯 摈 +擰 拧 +擱 搁 +擲 掷 +擴 扩 +擷 撷 +擺 摆 +擻 擞 +擼 撸 +擽 㧰 +擾 扰 +攄 摅 +攆 撵 +攏 拢 +攔 拦 +攖 撄 +攙 搀 +攛 撺 +攜 携 +攝 摄 +攢 攒 +攣 挛 +攤 摊 +攪 搅 +攬 揽 +敎 教 +敓 敚 +敗 败 +敘 叙 +敵 敌 +數 数 +斂 敛 +斃 毙 +斆 敩 +斕 斓 +斬 斩 +斷 断 +於 于 +旂 旗 +旣 既 +昇 升 +時 时 +晉 晋 +晛 𬀪 +晝 昼 +暈 晕 +暉 晖 +暐 𬀩 +暘 旸 +暢 畅 +暫 暂 +曄 晔 +曆 历 +曇 昙 +曉 晓 +曏 向 +曖 暧 +曠 旷 +曥 𣆐 +曨 昽 +曬 晒 +書 书 +會 会 +朥 𦛨 +朧 胧 +朮 术 +東 东 +枴 拐 +柵 栅 +柺 拐 +査 查 +桱 𣐕 +桿 杆 +梔 栀 +梘 枧 +梜 𬂩 +條 条 +梟 枭 +梲 棁 +棄 弃 +棊 棋 +棖 枨 +棗 枣 +棟 栋 +棡 㭎 +棧 栈 +棲 栖 +棶 梾 +椏 桠 +椲 㭏 +楊 杨 +楓 枫 +楨 桢 +業 业 +極 极 +榘 矩 +榦 干 +榪 杩 +榮 荣 +榲 榅 +榿 桤 +構 构 +槍 枪 +槓 杠 +槤 梿 +槧 椠 +槨 椁 +槮 椮 +槳 桨 +槶 椢 +槼 椝 +樁 桩 +樂 乐 +樅 枞 +樑 梁 +樓 楼 +標 标 +樞 枢 +樢 㭤 +樣 样 +樧 榝 +樫 㭴 +樳 桪 +樸 朴 +樹 树 +樺 桦 +樿 椫 +橈 桡 +橋 桥 +機 机 +橢 椭 +橫 横 +橯 𣓿 +檁 檩 +檉 柽 +檔 档 +檜 桧 +檟 槚 +檢 检 +檣 樯 +檮 梼 +檯 台 +檳 槟 +檸 柠 +檻 槛 +櫃 柜 +櫍 𬃊 +櫓 橹 +櫚 榈 +櫛 栉 +櫝 椟 +櫞 橼 +櫟 栎 +櫥 橱 +櫧 槠 +櫨 栌 +櫪 枥 +櫫 橥 +櫬 榇 +櫱 蘖 +櫳 栊 +櫸 榉 +櫻 樱 +欄 栏 +欅 榉 +權 权 +欏 椤 +欒 栾 +欓 𣗋 +欖 榄 +欞 棂 +欽 钦 +歎 叹 +歐 欧 +歟 欤 +歡 欢 +歲 岁 +歷 历 +歸 归 +歿 殁 +殘 残 +殞 殒 +殤 殇 +殨 㱮 +殫 殚 +殭 僵 +殮 殓 +殯 殡 +殰 㱩 +殲 歼 +殺 杀 +殻 壳 +殼 壳 +毀 毁 +毆 殴 +毿 毵 +氂 牦 +氈 毡 +氌 氇 +氣 气 +氫 氢 +氬 氩 +氳 氲 +氾 泛 +汎 泛 +汙 污 +決 决 +沒 没 +沖 冲 +況 况 +泝 溯 +洩 泄 +洶 汹 +浹 浃 +浿 𬇙 +涇 泾 +涗 涚 +涼 凉 +淒 凄 +淚 泪 +淥 渌 +淨 净 +淩 凌 +淪 沦 +淵 渊 +淶 涞 +淺 浅 +渙 涣 +減 减 +渢 沨 +渦 涡 +測 测 +渾 浑 +湊 凑 +湋 𣲗 +湞 浈 +湧 涌 +湯 汤 +溈 沩 +準 准 +溝 沟 +溫 温 +溮 浉 +溳 涢 +溼 湿 +滄 沧 +滅 灭 +滌 涤 +滎 荥 +滙 汇 +滬 沪 +滯 滞 +滲 渗 +滷 卤 +滸 浒 +滻 浐 +滾 滚 +滿 满 +漁 渔 +漊 溇 +漍 𬇹 +漚 沤 +漢 汉 +漣 涟 +漬 渍 +漲 涨 +漵 溆 +漸 渐 +漿 浆 +潁 颍 +潑 泼 +潔 洁 +潕 𣲘 +潙 沩 +潚 㴋 +潛 潜 +潤 润 +潯 浔 +潰 溃 +潷 滗 +潿 涠 +澀 涩 +澆 浇 +澇 涝 +澐 沄 +澗 涧 +澠 渑 +澤 泽 +澦 滪 +澩 泶 +澫 𬇕 +澮 浍 +澱 淀 +澾 㳠 +濁 浊 +濃 浓 +濄 㳡 +濆 𣸣 +濕 湿 +濘 泞 +濚 溁 +濛 蒙 +濜 浕 +濟 济 +濤 涛 +濧 㳔 +濫 滥 +濰 潍 +濱 滨 +濺 溅 +濼 泺 +濾 滤 +瀂 澛 +瀅 滢 +瀆 渎 +瀇 㲿 +瀉 泻 +瀋 沈 +瀏 浏 +瀕 濒 +瀘 泸 +瀝 沥 +瀟 潇 +瀠 潆 +瀦 潴 +瀧 泷 +瀨 濑 +瀰 弥 +瀲 潋 +瀾 澜 +灃 沣 +灄 滠 +灑 洒 +灒 𪷽 +灕 漓 +灘 滩 +灙 𣺼 +灝 灏 +灡 㳕 +灣 湾 +灤 滦 +灧 滟 +灩 滟 +災 灾 +為 为 +烏 乌 +烴 烃 +無 无 +煉 炼 +煒 炜 +煙 烟 +煢 茕 +煥 焕 +煩 烦 +煬 炀 +煱 㶽 +熅 煴 +熒 荧 +熗 炝 +熰 𬉼 +熱 热 +熲 颎 +熾 炽 +燀 𬊤 +燁 烨 +燈 灯 +燉 炖 +燒 烧 +燖 𬊈 +燙 烫 +燜 焖 +營 营 +燦 灿 +燬 毁 +燭 烛 +燴 烩 +燶 㶶 +燻 熏 +燼 烬 +燾 焘 +爍 烁 +爐 炉 +爛 烂 +爭 争 +爲 为 +爺 爷 +爾 尔 +牀 床 +牆 墙 +牘 牍 +牴 抵 +牽 牵 +犖 荦 +犛 牦 +犢 犊 +犧 牺 +狀 状 +狹 狭 +狽 狈 +猙 狰 +猶 犹 +猻 狲 +獁 犸 +獃 呆 +獄 狱 +獅 狮 +獎 奖 +獨 独 +獪 狯 +獫 猃 +獮 狝 +獰 狞 +獱 㺍 +獲 获 +獵 猎 +獷 犷 +獸 兽 +獺 獭 +獻 献 +獼 猕 +玀 猡 +現 现 +琱 雕 +琺 珐 +琿 珲 +瑋 玮 +瑒 玚 +瑣 琐 +瑤 瑶 +瑩 莹 +瑪 玛 +瑲 玱 +璉 琏 +璊 𫞩 +璕 𬍤 +璗 𬍡 +璡 琎 +璣 玑 +璦 瑷 +璫 珰 +璯 㻅 +環 环 +璵 玙 +璸 瑸 +璽 玺 +璿 璇 +瓅 𬍛 +瓊 琼 +瓏 珑 +瓔 璎 +瓚 瓒 +瓛 𤩽 +甌 瓯 +甕 瓮 +產 产 +産 产 +畝 亩 +畢 毕 +畫 画 +異 异 +畵 画 +當 当 +疇 畴 +疊 叠 +痙 痉 +痠 酸 +痾 疴 +瘂 痖 +瘋 疯 +瘍 疡 +瘓 痪 +瘞 瘗 +瘡 疮 +瘧 疟 +瘮 瘆 +瘲 疭 +瘺 瘘 +瘻 瘘 +療 疗 +癆 痨 +癇 痫 +癉 瘅 +癒 愈 +癘 疠 +癟 瘪 +癡 痴 +癢 痒 +癤 疖 +癥 症 +癧 疬 +癩 癞 +癬 癣 +癭 瘿 +癮 瘾 +癰 痈 +癱 瘫 +癲 癫 +發 发 +皁 皂 +皚 皑 +皰 疱 +皸 皲 +皺 皱 +盃 杯 +盜 盗 +盞 盏 +盡 尽 +監 监 +盤 盘 +盧 卢 +盪 荡 +眞 真 +眥 眦 +眾 众 +睍 𪾢 +睏 困 +睜 睁 +睞 睐 +瞘 眍 +瞜 䁖 +瞞 瞒 +瞶 瞆 +瞼 睑 +矇 蒙 +矓 眬 +矚 瞩 +矯 矫 +硃 朱 +硜 硁 +硤 硖 +硨 砗 +硯 砚 +碕 埼 +碩 硕 +碭 砀 +碸 砜 +確 确 +碼 码 +碽 䂵 +磑 硙 +磚 砖 +磠 硵 +磣 碜 +磧 碛 +磯 矶 +磽 硗 +磾 䃅 +礄 硚 +礎 础 +礐 𬒈 +礙 碍 +礦 矿 +礪 砺 +礫 砾 +礬 矾 +礱 砻 +祕 秘 +祿 禄 +禍 祸 +禎 祯 +禕 祎 +禡 祃 +禦 御 +禪 禅 +禮 礼 +禰 祢 +禱 祷 +禿 秃 +秈 籼 +稅 税 +稈 秆 +稏 䅉 +稜 棱 +稟 禀 +種 种 +稱 称 +穀 谷 +穇 䅟 +穌 稣 +積 积 +穎 颖 +穠 秾 +穡 穑 +穢 秽 +穩 稳 +穫 获 +穭 穞 +窩 窝 +窪 洼 +窮 穷 +窯 窑 +窵 窎 +窶 窭 +窺 窥 +竄 窜 +竅 窍 +竇 窦 +竈 灶 +竊 窃 +竪 竖 +競 竞 +筆 笔 +筍 笋 +筧 笕 +筴 䇲 +箇 个 +箋 笺 +箏 筝 +箚 札 +節 节 +範 范 +築 筑 +篋 箧 +篔 筼 +篠 筿 +篢 𬕂 +篤 笃 +篩 筛 +篳 筚 +篸 𥮾 +簀 箦 +簍 篓 +簑 蓑 +簞 箪 +簡 简 +簣 篑 +簫 箫 +簹 筜 +簽 签 +簾 帘 +籃 篮 +籅 𥫣 +籌 筹 +籔 䉤 +籙 箓 +籛 篯 +籜 箨 +籟 籁 +籠 笼 +籤 签 +籩 笾 +籪 簖 +籬 篱 +籮 箩 +籲 吁 +粵 粤 +糉 粽 +糝 糁 +糞 粪 +糧 粮 +糰 团 +糲 粝 +糴 籴 +糶 粜 +糹 纟 +糾 纠 +紀 纪 +紂 纣 +紃 𬘓 +約 约 +紅 红 +紆 纡 +紇 纥 +紈 纨 +紉 纫 +紋 纹 +納 纳 +紐 纽 +紓 纾 +純 纯 +紕 纰 +紖 纼 +紗 纱 +紘 纮 +紙 纸 +級 级 +紛 纷 +紜 纭 +紝 纴 +紞 𬘘 +紡 纺 +紬 䌷 +紮 扎 +細 细 +紱 绂 +紲 绁 +紳 绅 +紵 纻 +紹 绍 +紺 绀 +紼 绋 +紿 绐 +絀 绌 +終 终 +絃 弦 +組 组 +絅 䌹 +絆 绊 +絎 绗 +結 结 +絕 绝 +絛 绦 +絝 绔 +絞 绞 +絡 络 +絢 绚 +給 给 +絨 绒 +絪 𬘡 +絰 绖 +統 统 +絲 丝 +絳 绛 +絶 绝 +絹 绢 +絺 𫄨 +綁 绑 +綃 绡 +綄 𬘫 +綆 绠 +綈 绨 +綉 绣 +綌 绤 +綎 𬘩 +綏 绥 +綐 䌼 +綑 捆 +經 经 +綖 𫄧 +綜 综 +綝 𬘭 +綞 缍 +綠 绿 +綡 𫟅 +綢 绸 +綣 绻 +綧 𬘯 +綪 𬘬 +綫 线 +綬 绶 +維 维 +綯 绹 +綰 绾 +綱 纲 +網 网 +綳 绷 +綴 缀 +綵 彩 +綸 纶 +綹 绺 +綺 绮 +綻 绽 +綽 绰 +綾 绫 +綿 绵 +緄 绲 +緇 缁 +緊 紧 +緋 绯 +緑 绿 +緒 绪 +緓 绬 +緔 绱 +緗 缃 +緘 缄 +緙 缂 +線 线 +緝 缉 +緞 缎 +締 缔 +緡 缗 +緣 缘 +緦 缌 +編 编 +緩 缓 +緬 缅 +緯 纬 +緱 缑 +緲 缈 +練 练 +緶 缏 +緹 缇 +緻 致 +緼 缊 +縈 萦 +縉 缙 +縊 缢 +縋 缒 +縐 绉 +縑 缣 +縕 缊 +縗 缞 +縛 缚 +縝 缜 +縞 缟 +縟 缛 +縣 县 +縧 绦 +縫 缝 +縭 缡 +縮 缩 +縯 𬙂 +縱 纵 +縲 缧 +縳 䌸 +縴 纤 +縵 缦 +縶 絷 +縷 缕 +縹 缥 +總 总 +績 绩 +繃 绷 +繅 缫 +繆 缪 +繒 缯 +織 织 +繕 缮 +繚 缭 +繞 绕 +繡 绣 +繢 缋 +繩 绳 +繪 绘 +繫 系 +繭 茧 +繮 缰 +繯 缳 +繰 缲 +繳 缴 +繶 𫄷 +繸 䍁 +繹 绎 +繻 𦈡 +繼 继 +繽 缤 +繾 缱 +繿 䍀 +纁 𫄸 +纆 𬙊 +纇 颣 +纈 缬 +纊 纩 +續 续 +纍 累 +纏 缠 +纓 缨 +纔 才 +纕 𬙋 +纖 纤 +纘 缵 +纜 缆 +缽 钵 +罃 䓨 +罈 坛 +罌 罂 +罎 坛 +罰 罚 +罵 骂 +罷 罢 +羅 罗 +羆 罴 +羈 羁 +羋 芈 +羣 群 +羥 羟 +羨 羡 +義 义 +羶 膻 +習 习 +翫 玩 +翬 翚 +翹 翘 +翽 翙 +耬 耧 +耮 耢 +聖 圣 +聞 闻 +聯 联 +聰 聪 +聲 声 +聳 耸 +聵 聩 +聶 聂 +職 职 +聹 聍 +聽 听 +聾 聋 +肅 肃 +脅 胁 +脈 脉 +脛 胫 +脣 唇 +脩 修 +脫 脱 +脹 胀 +腎 肾 +腖 胨 +腡 脶 +腦 脑 +腫 肿 +腳 脚 +腸 肠 +膃 腽 +膕 腘 +膚 肤 +膞 䏝 +膠 胶 +膢 𦝼 +膩 腻 +膽 胆 +膾 脍 +膿 脓 +臉 脸 +臍 脐 +臏 膑 +臘 腊 +臚 胪 +臟 脏 +臠 脔 +臢 臜 +臥 卧 +臨 临 +臺 台 +與 与 +興 兴 +舉 举 +舊 旧 +舖 铺 +舘 馆 +艙 舱 +艤 舣 +艦 舰 +艫 舻 +艱 艰 +艷 艳 +芻 刍 +苧 苎 +茲 兹 +荊 荆 +莊 庄 +莖 茎 +莢 荚 +莧 苋 +華 华 +菴 庵 +菸 烟 +萇 苌 +萊 莱 +萬 万 +萴 荝 +萵 莴 +葉 叶 +葒 荭 +葤 荮 +葦 苇 +葯 药 +葷 荤 +蒍 𫇭 +蒐 搜 +蒓 莼 +蒔 莳 +蒕 蒀 +蒞 莅 +蒼 苍 +蓀 荪 +蓆 席 +蓋 盖 +蓮 莲 +蓯 苁 +蓴 莼 +蓽 荜 +蔄 𬜬 +蔔 卜 +蔘 参 +蔞 蒌 +蔣 蒋 +蔥 葱 +蔦 茑 +蔭 荫 +蔯 𫈟 +蔿 𫇭 +蕁 荨 +蕆 蒇 +蕎 荞 +蕒 荬 +蕓 芸 +蕕 莸 +蕘 荛 +蕢 蒉 +蕩 荡 +蕪 芜 +蕭 萧 +蕷 蓣 +薀 蕰 +薈 荟 +薊 蓟 +薌 芗 +薑 姜 +薔 蔷 +薘 荙 +薟 莶 +薦 荐 +薩 萨 +薴 苧 +薵 䓓 +薹 苔 +薺 荠 +藍 蓝 +藎 荩 +藝 艺 +藥 药 +藪 薮 +藭 䓖 +藴 蕴 +藶 苈 +藹 蔼 +藺 蔺 +蘀 萚 +蘄 蕲 +蘆 芦 +蘇 苏 +蘊 蕴 +蘋 苹 +蘚 藓 +蘞 蔹 +蘟 𦻕 +蘢 茏 +蘭 兰 +蘺 蓠 +蘿 萝 +虆 蔂 +虉 𬟁 +處 处 +虛 虚 +虜 虏 +號 号 +虧 亏 +虯 虬 +蛺 蛱 +蛻 蜕 +蜆 蚬 +蝀 𬟽 +蝕 蚀 +蝟 猬 +蝦 虾 +蝨 虱 +蝸 蜗 +螄 蛳 +螞 蚂 +螢 萤 +螮 䗖 +螻 蝼 +螿 螀 +蟄 蛰 +蟈 蝈 +蟎 螨 +蟣 虮 +蟬 蝉 +蟯 蛲 +蟲 虫 +蟳 𫊻 +蟶 蛏 +蟻 蚁 +蠁 蚃 +蠅 蝇 +蠆 虿 +蠍 蝎 +蠐 蛴 +蠑 蝾 +蠔 蚝 +蠟 蜡 +蠣 蛎 +蠨 蟏 +蠱 蛊 +蠶 蚕 +蠻 蛮 +衆 众 +衊 蔑 +術 术 +衕 同 +衚 胡 +衛 卫 +衝 冲 +袞 衮 +袷 夹 +裊 袅 +裏 里 +補 补 +裝 装 +裡 里 +製 制 +複 复 +褌 裈 +褘 袆 +褲 裤 +褳 裢 +褸 褛 +褻 亵 +襀 𫌀 +襇 裥 +襉 裥 +襏 袯 +襖 袄 +襝 裣 +襠 裆 +襤 褴 +襪 袜 +襬 摆 +襯 衬 +襲 袭 +襴 襕 +覈 核 +見 见 +覎 觃 +規 规 +覓 觅 +視 视 +覘 觇 +覡 觋 +覥 觍 +覦 觎 +親 亲 +覬 觊 +覯 觏 +覲 觐 +覷 觑 +覺 觉 +覽 览 +覿 觌 +觀 观 +觴 觞 +觶 觯 +觸 触 +訁 讠 +訂 订 +訃 讣 +計 计 +訊 讯 +訌 讧 +討 讨 +訏 𬣙 +訐 讦 +訒 讱 +訓 训 +訕 讪 +訖 讫 +託 托 +記 记 +訛 讹 +訝 讶 +訟 讼 +訢 䜣 +訣 诀 +訥 讷 +訩 讻 +訪 访 +設 设 +許 许 +訴 诉 +訶 诃 +診 诊 +註 注 +証 证 +詀 𧮪 +詁 诂 +詆 诋 +詎 讵 +詐 诈 +詒 诒 +詔 诏 +評 评 +詖 诐 +詗 诇 +詘 诎 +詛 诅 +詝 𬣞 +詞 词 +詠 咏 +詡 诩 +詢 询 +詣 诣 +試 试 +詩 诗 +詪 𬣳 +詫 诧 +詬 诟 +詭 诡 +詮 诠 +詰 诘 +話 话 +該 该 +詳 详 +詵 诜 +詷 𫍣 +詼 诙 +詿 诖 +誄 诔 +誅 诛 +誆 诓 +誇 夸 +誌 志 +認 认 +誑 诳 +誒 诶 +誕 诞 +誘 诱 +誚 诮 +語 语 +誠 诚 +誡 诫 +誣 诬 +誤 误 +誥 诰 +誦 诵 +誨 诲 +說 说 +説 说 +誰 谁 +課 课 +誶 谇 +誹 诽 +誼 谊 +誾 訚 +調 调 +諂 谄 +諄 谆 +談 谈 +諉 诿 +請 请 +諍 诤 +諏 诹 +諑 诼 +諒 谅 +諓 𬣡 +論 论 +諗 谂 +諛 谀 +諜 谍 +諝 谞 +諞 谝 +諟 𬤊 +諡 谥 +諢 诨 +諤 谔 +諦 谛 +諧 谐 +諫 谏 +諭 谕 +諮 咨 +諱 讳 +諲 𬤇 +諳 谙 +諴 𫍯 +諶 谌 +諷 讽 +諸 诸 +諺 谚 +諼 谖 +諾 诺 +謀 谋 +謁 谒 +謂 谓 +謄 誊 +謅 诌 +謊 谎 +謎 谜 +謏 𫍲 +謐 谧 +謔 谑 +謖 谡 +謗 谤 +謙 谦 +謚 谥 +講 讲 +謝 谢 +謠 谣 +謡 谣 +謨 谟 +謫 谪 +謬 谬 +謭 谫 +謳 讴 +謹 谨 +謾 谩 +譁 哗 +證 证 +譎 谲 +譏 讥 +譓 𬤝 +譖 谮 +識 识 +譙 谯 +譚 谭 +譜 谱 +譞 𫍽 +譟 噪 +譫 谵 +譭 毁 +譯 译 +議 议 +譴 谴 +護 护 +譸 诪 +譽 誉 +譾 谫 +讀 读 +讅 谉 +變 变 +讋 詟 +讌 䜩 +讎 雠 +讒 谗 +讓 让 +讕 谰 +讖 谶 +讚 赞 +讜 谠 +讞 谳 +谿 溪 +豈 岂 +豎 竖 +豐 丰 +豔 艳 +豬 猪 +豶 豮 +貍 狸 +貓 猫 +貙 䝙 +貝 贝 +貞 贞 +貟 贠 +負 负 +財 财 +貢 贡 +貧 贫 +貨 货 +販 贩 +貪 贪 +貫 贯 +責 责 +貯 贮 +貰 贳 +貲 赀 +貳 贰 +貴 贵 +貶 贬 +買 买 +貸 贷 +貺 贶 +費 费 +貼 贴 +貽 贻 +貿 贸 +賀 贺 +賁 贲 +賂 赂 +賃 赁 +賄 贿 +賅 赅 +資 资 +賈 贾 +賊 贼 +賑 赈 +賒 赊 +賓 宾 +賕 赇 +賙 赒 +賚 赉 +賜 赐 +賞 赏 +賠 赔 +賡 赓 +賢 贤 +賣 卖 +賤 贱 +賦 赋 +賧 赕 +質 质 +賫 赍 +賬 账 +賭 赌 +賰 䞐 +賴 赖 +賵 赗 +賺 赚 +賻 赙 +購 购 +賽 赛 +賾 赜 +贄 贽 +贅 赘 +贇 赟 +贈 赠 +贊 赞 +贋 赝 +贍 赡 +贏 赢 +贐 赆 +贓 赃 +贔 赑 +贖 赎 +贗 赝 +贛 赣 +贜 赃 +赬 赪 +趕 赶 +趙 赵 +趨 趋 +趲 趱 +跡 迹 +踐 践 +踰 逾 +踴 踊 +蹌 跄 +蹕 跸 +蹟 迹 +蹠 跖 +蹣 蹒 +蹤 踪 +蹺 跷 +躂 跶 +躉 趸 +躊 踌 +躋 跻 +躍 跃 +躎 䟢 +躑 踯 +躒 跞 +躓 踬 +躕 蹰 +躚 跹 +躡 蹑 +躥 蹿 +躦 躜 +躪 躏 +軀 躯 +車 车 +軋 轧 +軌 轨 +軍 军 +軏 𫐄 +軑 轪 +軒 轩 +軔 轫 +軛 轭 +軝 𬨂 +軟 软 +軤 轷 +軫 轸 +軲 轱 +軸 轴 +軹 轵 +軺 轺 +軻 轲 +軼 轶 +軾 轼 +較 较 +輄 𨐈 +輅 辂 +輇 辁 +輈 辀 +載 载 +輊 轾 +輋 𪨶 +輒 辄 +輓 挽 +輔 辅 +輕 轻 +輗 𫐐 +輛 辆 +輜 辎 +輝 辉 +輞 辋 +輟 辍 +輥 辊 +輦 辇 +輩 辈 +輪 轮 +輬 辌 +輮 𫐓 +輯 辑 +輳 辏 +輶 𬨎 +輸 输 +輻 辐 +輼 辒 +輾 辗 +輿 舆 +轀 辒 +轂 毂 +轄 辖 +轅 辕 +轆 辘 +轉 转 +轍 辙 +轎 轿 +轔 辚 +轟 轰 +轡 辔 +轢 轹 +轤 轳 +辦 办 +辭 辞 +辮 辫 +辯 辩 +農 农 +迴 回 +逕 径 +這 这 +連 连 +週 周 +進 进 +遊 游 +運 运 +過 过 +達 达 +違 违 +遙 遥 +遜 逊 +遞 递 +遠 远 +遡 溯 +適 适 +遲 迟 +遶 绕 +遷 迁 +選 选 +遺 遗 +遼 辽 +邁 迈 +還 还 +邇 迩 +邊 边 +邏 逻 +邐 逦 +郟 郏 +郵 邮 +鄆 郓 +鄉 乡 +鄒 邹 +鄔 邬 +鄖 郧 +鄧 邓 +鄩 𬩽 +鄭 郑 +鄰 邻 +鄲 郸 +鄳 𫑡 +鄴 邺 +鄶 郐 +鄺 邝 +酇 酂 +酈 郦 +醃 腌 +醖 酝 +醜 丑 +醞 酝 +醟 蒏 +醣 糖 +醫 医 +醬 酱 +醱 酦 +醲 𬪩 +釀 酿 +釁 衅 +釃 酾 +釅 酽 +釋 释 +釐 厘 +釒 钅 +釓 钆 +釔 钇 +釕 钌 +釗 钊 +釘 钉 +釙 钋 +針 针 +釣 钓 +釤 钐 +釦 扣 +釧 钏 +釩 钒 +釴 𬬩 +釵 钗 +釷 钍 +釹 钕 +釺 钎 +釾 䥺 +釿 𬬱 +鈀 钯 +鈁 钫 +鈃 钘 +鈄 钭 +鈅 钥 +鈇 𫓧 +鈈 钚 +鈉 钠 +鈍 钝 +鈎 钩 +鈐 钤 +鈑 钣 +鈒 钑 +鈔 钞 +鈕 钮 +鈞 钧 +鈡 钟 +鈣 钙 +鈥 钬 +鈦 钛 +鈧 钪 +鈮 铌 +鈰 铈 +鈳 钶 +鈴 铃 +鈷 钴 +鈸 钹 +鈹 铍 +鈺 钰 +鈽 钸 +鈾 铀 +鈿 钿 +鉀 钾 +鉅 巨 +鉆 钻 +鉈 铊 +鉉 铉 +鉊 𬬿 +鉋 铇 +鉍 铋 +鉑 铂 +鉕 钷 +鉗 钳 +鉚 铆 +鉛 铅 +鉝 𫟷 +鉞 钺 +鉢 钵 +鉤 钩 +鉥 𬬸 +鉦 钲 +鉧 𬭁 +鉬 钼 +鉭 钽 +鉮 𬬹 +鉳 锫 +鉶 铏 +鉷 𫟹 +鉸 铰 +鉺 铒 +鉻 铬 +鉿 铪 +銀 银 +銃 铳 +銅 铜 +銈 𫓯 +銍 铚 +銑 铣 +銓 铨 +銖 铢 +銘 铭 +銚 铫 +銛 铦 +銜 衔 +銠 铑 +銣 铷 +銥 铱 +銦 铟 +銨 铵 +銩 铥 +銪 铕 +銫 铯 +銬 铐 +銱 铞 +銳 锐 +銶 𨱇 +銷 销 +銹 锈 +銻 锑 +銼 锉 +鋁 铝 +鋃 锒 +鋅 锌 +鋇 钡 +鋌 铤 +鋏 铗 +鋐 𬭎 +鋒 锋 +鋗 𫓶 +鋙 铻 +鋝 锊 +鋟 锓 +鋣 铘 +鋤 锄 +鋥 锃 +鋦 锔 +鋨 锇 +鋩 铓 +鋪 铺 +鋭 锐 +鋮 铖 +鋯 锆 +鋰 锂 +鋱 铽 +鋶 锍 +鋸 锯 +鋹 𬬮 +鋼 钢 +錀 𬬭 +錁 锞 +錄 录 +錆 锖 +錇 锫 +錈 锩 +錏 铔 +錐 锥 +錒 锕 +錕 锟 +錘 锤 +錙 锱 +錚 铮 +錛 锛 +錞 𬭚 +錟 锬 +錠 锭 +錡 锜 +錢 钱 +錤 𫓹 +錦 锦 +錨 锚 +錩 锠 +錫 锡 +錮 锢 +錯 错 +録 录 +錳 锰 +錶 表 +錸 铼 +錼 镎 +鍀 锝 +鍁 锨 +鍃 锪 +鍅 钫 +鍆 钔 +鍇 锴 +鍈 锳 +鍊 炼 +鍋 锅 +鍍 镀 +鍔 锷 +鍘 铡 +鍚 钖 +鍛 锻 +鍠 锽 +鍤 锸 +鍥 锲 +鍩 锘 +鍬 锹 +鍭 𬭤 +鍰 锾 +鍵 键 +鍶 锶 +鍺 锗 +鍼 针 +鍾 钟 +鎂 镁 +鎄 锿 +鎇 镅 +鎊 镑 +鎌 镰 +鎓 𬭩 +鎔 镕 +鎖 锁 +鎘 镉 +鎚 锤 +鎛 镈 +鎝 𨱏 +鎡 镃 +鎢 钨 +鎣 蓥 +鎦 镏 +鎧 铠 +鎩 铩 +鎪 锼 +鎬 镐 +鎭 镇 +鎮 镇 +鎰 镒 +鎲 镋 +鎳 镍 +鎵 镓 +鎶 鿔 +鎸 镌 +鎿 镎 +鏃 镞 +鏇 旋 +鏈 链 +鏌 镆 +鏍 镙 +鏏 𬭬 +鏐 镠 +鏑 镝 +鏗 铿 +鏘 锵 +鏜 镗 +鏝 镘 +鏞 镛 +鏟 铲 +鏡 镜 +鏢 镖 +鏤 镂 +鏨 錾 +鏰 镚 +鏵 铧 +鏷 镤 +鏹 镪 +鏺 䥽 +鏻 𬭸 +鏽 锈 +鐃 铙 +鐄 𨱑 +鐇 𫔍 +鐋 铴 +鐍 𫔎 +鐏 𨱔 +鐐 镣 +鐒 铹 +鐓 镦 +鐔 镡 +鐘 钟 +鐙 镫 +鐝 镢 +鐠 镨 +鐥 䦅 +鐦 锎 +鐧 锏 +鐨 镄 +鐩 𬭼 +鐫 镌 +鐮 镰 +鐯 䦃 +鐲 镯 +鐳 镭 +鐵 铁 +鐶 镮 +鐸 铎 +鐺 铛 +鐽 𫟼 +鐿 镱 +鑄 铸 +鑊 镬 +鑌 镔 +鑑 鉴 +鑒 鉴 +鑔 镲 +鑕 锧 +鑞 镴 +鑠 铄 +鑣 镳 +鑥 镥 +鑪 𬬻 +鑭 镧 +鑰 钥 +鑱 镵 +鑲 镶 +鑷 镊 +鑹 镩 +鑼 锣 +鑽 钻 +鑾 銮 +鑿 凿 +钁 镢 +钂 镋 +長 长 +門 门 +閂 闩 +閃 闪 +閆 闫 +閈 闬 +閉 闭 +開 开 +閌 闶 +閎 闳 +閏 闰 +閑 闲 +閒 闲 +間 间 +閔 闵 +閘 闸 +閡 阂 +閣 阁 +閤 合 +閥 阀 +閨 闺 +閩 闽 +閫 阃 +閬 阆 +閭 闾 +閱 阅 +閲 阅 +閶 阊 +閹 阉 +閻 阎 +閼 阏 +閽 阍 +閾 阈 +閿 阌 +闃 阒 +闆 板 +闇 暗 +闈 闱 +闉 𬮱 +闊 阔 +闋 阕 +闌 阑 +闍 阇 +闐 阗 +闑 𫔶 +闒 阘 +闓 闿 +闔 阖 +闕 阙 +闖 闯 +關 关 +闞 阚 +闠 阓 +闡 阐 +闢 辟 +闤 阛 +闥 闼 +陘 陉 +陝 陕 +陞 升 +陣 阵 +陰 阴 +陳 陈 +陸 陆 +陽 阳 +隉 陧 +隊 队 +階 阶 +隑 𬮿 +隕 陨 +際 际 +隤 𬯎 +隨 随 +險 险 +隮 𬯀 +隯 陦 +隱 隐 +隴 陇 +隸 隶 +隻 只 +雋 隽 +雖 虽 +雙 双 +雛 雏 +雜 杂 +雞 鸡 +離 离 +難 难 +雲 云 +電 电 +霑 沾 +霢 霡 +霧 雾 +霽 霁 +靂 雳 +靄 霭 +靆 叇 +靈 灵 +靉 叆 +靚 靓 +靜 静 +靝 靔 +靦 腼 +靨 靥 +鞏 巩 +鞝 绱 +鞦 秋 +鞽 鞒 +韁 缰 +韃 鞑 +韆 千 +韉 鞯 +韋 韦 +韌 韧 +韍 韨 +韓 韩 +韙 韪 +韜 韬 +韝 鞲 +韞 韫 +韻 韵 +響 响 +頁 页 +頂 顶 +頃 顷 +項 项 +順 顺 +頇 顸 +須 须 +頊 顼 +頌 颂 +頍 𫠆 +頎 颀 +頏 颃 +預 预 +頑 顽 +頒 颁 +頓 顿 +頔 𬱖 +頗 颇 +領 领 +頜 颌 +頠 𬱟 +頡 颉 +頤 颐 +頦 颏 +頫 𫖯 +頭 头 +頮 颒 +頰 颊 +頲 颋 +頴 颕 +頵 𫖳 +頷 颔 +頸 颈 +頹 颓 +頻 频 +頽 颓 +顆 颗 +題 题 +額 额 +顎 颚 +顏 颜 +顒 颙 +顓 颛 +顔 颜 +顗 𫖮 +願 愿 +顙 颡 +顛 颠 +類 类 +顢 颟 +顥 颢 +顧 顾 +顫 颤 +顬 颥 +顯 显 +顰 颦 +顱 颅 +顳 颞 +顴 颧 +風 风 +颭 飐 +颮 飑 +颯 飒 +颱 台 +颳 刮 +颶 飓 +颸 飔 +颺 飏 +颻 飖 +颼 飕 +飀 飗 +飄 飘 +飆 飙 +飈 飚 +飛 飞 +飠 饣 +飢 饥 +飣 饤 +飥 饦 +飩 饨 +飪 饪 +飫 饫 +飭 饬 +飯 饭 +飱 飧 +飲 饮 +飴 饴 +飼 饲 +飽 饱 +飾 饰 +飿 饳 +餃 饺 +餄 饸 +餅 饼 +餈 糍 +餉 饷 +養 养 +餌 饵 +餎 饹 +餏 饻 +餑 饽 +餒 馁 +餓 饿 +餕 馂 +餖 饾 +餗 𫗧 +餘 余 +餚 肴 +餛 馄 +餜 馃 +餞 饯 +餡 馅 +館 馆 +餬 糊 +餱 糇 +餳 饧 +餵 喂 +餶 馉 +餷 馇 +餸 𩠌 +餺 馎 +餼 饩 +餾 馏 +餿 馊 +饁 馌 +饃 馍 +饅 馒 +饈 馐 +饉 馑 +饊 馓 +饋 馈 +饌 馔 +饑 饥 +饒 饶 +饗 飨 +饘 𫗴 +饜 餍 +饞 馋 +饢 馕 +馬 马 +馭 驭 +馮 冯 +馱 驮 +馳 驰 +馴 驯 +馹 驲 +馼 𫘜 +駁 驳 +駃 𫘝 +駉 𬳶 +駐 驻 +駑 驽 +駒 驹 +駓 𬳵 +駔 驵 +駕 驾 +駘 骀 +駙 驸 +駛 驶 +駝 驼 +駟 驷 +駡 骂 +駢 骈 +駪 𬳽 +駭 骇 +駰 骃 +駱 骆 +駸 骎 +駼 𬳿 +駿 骏 +騁 骋 +騂 骍 +騄 𫘧 +騅 骓 +騊 𫘦 +騌 骔 +騍 骒 +騎 骑 +騏 骐 +騑 𬴂 +騖 骛 +騙 骗 +騞 𬴃 +騠 𫘨 +騤 骙 +騧 䯄 +騫 骞 +騭 骘 +騮 骝 +騰 腾 +騱 𫘬 +騵 𫘪 +騶 驺 +騷 骚 +騸 骟 +騾 骡 +驀 蓦 +驁 骜 +驂 骖 +驃 骠 +驄 骢 +驅 驱 +驊 骅 +驌 骕 +驍 骁 +驎 𬴊 +驏 骣 +驕 骄 +驗 验 +驚 惊 +驛 驿 +驟 骤 +驢 驴 +驤 骧 +驥 骥 +驦 骦 +驪 骊 +驫 骉 +骯 肮 +髏 髅 +髒 脏 +體 体 +髕 髌 +髖 髋 +髮 发 +鬆 松 +鬍 胡 +鬚 须 +鬢 鬓 +鬥 斗 +鬧 闹 +鬨 哄 +鬩 阋 +鬮 阄 +鬱 郁 +鬹 鬶 +魎 魉 +魘 魇 +魚 鱼 +魛 鱽 +魟 𫚉 +魢 鱾 +魨 鲀 +魯 鲁 +魴 鲂 +魷 鱿 +魺 鲄 +鮀 𬶍 +鮁 鲅 +鮃 鲆 +鮆 𫚖 +鮈 𬶋 +鮊 鲌 +鮋 鲉 +鮍 鲏 +鮎 鲇 +鮐 鲐 +鮑 鲍 +鮒 鲋 +鮓 鲊 +鮚 鲒 +鮜 鲘 +鮝 鲞 +鮞 鲕 +鮟 𩽾 +鮠 𬶏 +鮡 𬶐 +鮣 䲟 +鮦 鲖 +鮪 鲔 +鮫 鲛 +鮭 鲑 +鮮 鲜 +鮳 鲓 +鮶 鲪 +鮸 𩾃 +鮺 鲝 +鯀 鲧 +鯁 鲠 +鯇 鲩 +鯉 鲤 +鯊 鲨 +鯒 鲬 +鯔 鲻 +鯕 鲯 +鯖 鲭 +鯗 鲞 +鯛 鲷 +鯝 鲴 +鯡 鲱 +鯢 鲵 +鯤 鲲 +鯧 鲳 +鯨 鲸 +鯪 鲮 +鯫 鲰 +鯰 鲶 +鯴 鲺 +鯷 鳀 +鯻 𬶟 +鯽 鲫 +鯿 鳊 +鰁 鳈 +鰂 鲗 +鰃 鳂 +鰆 䲠 +鰈 鲽 +鰉 鳇 +鰊 𬶠 +鰌 䲡 +鰍 鳅 +鰏 鲾 +鰐 鳄 +鰒 鳆 +鰓 鳃 +鰛 鳁 +鰜 鳒 +鰟 鳑 +鰠 鳋 +鰣 鲥 +鰤 𫚕 +鰥 鳏 +鰧 䲢 +鰨 鳎 +鰩 鳐 +鰭 鳍 +鰮 鳁 +鰱 鲢 +鰲 鳌 +鰳 鳓 +鰵 鳘 +鰶 𬶭 +鰷 鲦 +鰹 鲣 +鰺 鲹 +鰻 鳗 +鰼 鳛 +鰾 鳔 +鱀 𬶨 +鱂 鳉 +鱅 鳙 +鱇 𩾌 +鱈 鳕 +鱉 鳖 +鱒 鳟 +鱔 鳝 +鱖 鳜 +鱗 鳞 +鱘 鲟 +鱚 𬶮 +鱝 鲼 +鱟 鲎 +鱠 鲙 +鱣 鳣 +鱤 鳡 +鱧 鳢 +鱨 鲿 +鱭 鲚 +鱯 鳠 +鱲 𫚭 +鱷 鳄 +鱸 鲈 +鱺 鲡 +鳥 鸟 +鳧 凫 +鳩 鸠 +鳬 凫 +鳲 鸤 +鳳 凤 +鳴 鸣 +鳶 鸢 +鳾 䴓 +鴆 鸩 +鴇 鸨 +鴉 鸦 +鴒 鸰 +鴕 鸵 +鴛 鸳 +鴝 鸲 +鴞 鸮 +鴟 鸱 +鴣 鸪 +鴦 鸯 +鴨 鸭 +鴯 鸸 +鴰 鸹 +鴴 鸻 +鴷 䴕 +鴻 鸿 +鴿 鸽 +鵁 䴔 +鵂 鸺 +鵃 鸼 +鵏 𬷕 +鵐 鹀 +鵑 鹃 +鵒 鹆 +鵓 鹁 +鵜 鹈 +鵝 鹅 +鵟 𫛭 +鵠 鹄 +鵡 鹉 +鵪 鹌 +鵬 鹏 +鵮 鹐 +鵯 鹎 +鵰 雕 +鵲 鹊 +鵷 鹓 +鵾 鹍 +鶄 䴖 +鶇 鸫 +鶉 鹑 +鶊 鹒 +鶓 鹋 +鶖 鹙 +鶘 鹕 +鶚 鹗 +鶠 𬸘 +鶡 鹖 +鶥 鹛 +鶩 鹜 +鶪 䴗 +鶬 鸧 +鶯 莺 +鶱 𬸣 +鶲 鹟 +鶴 鹤 +鶹 鹠 +鶺 鹡 +鶻 鹘 +鶼 鹣 +鶿 鹚 +鷀 鹚 +鷁 鹢 +鷂 鹞 +鷄 鸡 +鷉 䴘 +鷊 鹝 +鷓 鹧 +鷖 鹥 +鷗 鸥 +鷙 鸷 +鷚 鹨 +鷟 𬸦 +鷥 鸶 +鷦 鹪 +鷫 鹔 +鷭 𬸪 +鷯 鹩 +鷲 鹫 +鷳 鹇 +鷴 鹇 +鷸 鹬 +鷹 鹰 +鷺 鹭 +鷽 鸴 +鸂 㶉 +鸇 鹯 +鸊 䴙 +鸌 鹱 +鸏 鹲 +鸑 𬸚 +鸕 鸬 +鸘 鹴 +鸚 鹦 +鸛 鹳 +鸝 鹂 +鸞 鸾 +鹵 卤 +鹹 咸 +鹺 鹾 +鹼 碱 +鹽 盐 +麗 丽 +麥 麦 +麩 麸 +麪 面 +麫 面 +麬 𤿲 +麯 曲 +麳 𪎌 +麴 曲 +麵 面 +麼 么 +麽 么 +黃 黄 +黌 黉 +點 点 +黨 党 +黲 黪 +黴 霉 +黶 黡 +黷 黩 +黽 黾 +黿 鼋 +鼂 鼌 +鼉 鼍 +鼕 冬 +鼴 鼹 +齊 齐 +齋 斋 +齎 赍 +齏 齑 +齒 齿 +齔 龀 +齕 龁 +齗 龂 +齘 𬹼 +齙 龅 +齜 龇 +齟 龃 +齠 龆 +齡 龄 +齣 出 +齦 龈 +齧 啮 +齪 龊 +齬 龉 +齮 𬺈 +齯 𫠜 +齲 龋 +齶 腭 +齷 龌 +齼 𬺓 +龍 龙 +龎 厐 +龐 庞 +龑 䶮 +龔 龚 +龕 龛 +龜 龟 +鿁 䜤 +鿓 鿒 +𠁞 𠀾 +𠗣 㓆 +𡃕 𠴛 +𡅏 𠲥 +𡑍 𫭼 +𡑭 𡋗 +𡓾 𡋀 +𡔖 𡍣 +𡞵 㛟 +𡠹 㛿 +𡢃 㛠 +𡮉 𡭜 +𡮣 𡭬 +𡳳 𡳃 +𡻕 岁 +𡾱 㟜 +𢣚 𢘝 +𢶫 𢫞 +𢹿 𢬦 +𣈶 暅 +𣙎 㭣 +𣞻 𣘓 +𣠩 𣞎 +𣠲 𣑶 +𣯶 毶 +𣾷 㳢 +𤁣 𣺽 +𤅶 𣷷 +𤓩 𤊰 +𤪺 㻘 +𤫩 㻏 +𤳸 𤳄 +𥊝 𥅿 +𥌃 𥅘 +𥕥 𥐰 +𥖅 𥐯 +𥗽 𬒗 +𥢢 䅪 +𥸠 𥮋 +𥼽 𥹥 +𦘧 𡳒 +𦣎 𦟗 +𦪙 䑽 +𧜗 䘞 +𧜵 䙊 +𧝞 䘛 +𧟀 𧝧 +𧩙 䜥 +𧵳 䞌 +𧶧 䞎 +𨊰 䢀 +𨊸 䢁 +𨋢 䢂 +𨤻 𨤰 +𨦫 䦀 +𨧀 𬭊 +𨧜 䦁 +𨨏 𬭛 +𨭆 𬭶 +𨭎 𬭳 +𨯅 䥿 +𩞯 䭪 +𩠴 𩠠 +𩣑 䯃 +𩶘 䲞 +𰻞 𰻝 diff --git a/app/api/discogs.py b/app/api/discogs.py new file mode 100644 index 00000000..9593602b --- /dev/null +++ b/app/api/discogs.py @@ -0,0 +1,93 @@ +"""Discogs artists by name, for a band typed in the artist box. + +GET /api/discogs/artist?q= lists the artists Discogs finds for a name +Wikipedia did not know, and GET /api/discogs/artist/{id} is the one the user +picked, in the shape GET /api/jobs/{id}/artist-extra answers in, so the box +draws it the same way (static/js/artistDiscogs.js). + +Nothing is taken on a name alone here: the user picks. Both answer 404 at once +when no Discogs token is saved, and ask Discogs nothing. The token is the +server's (settings.get_discogs_token): it goes in the Authorization header of +the requests app/pipeline/discogs.py makes and never into an answer. Each is +bounded by DISCOGS_SEARCH_BUDGET_SEC, and shares the process-wide limiter and +the on-disk cache with every other Discogs request. +""" + +from __future__ import annotations + +import asyncio +import logging +import re +import time +from typing import Any + +from fastapi import APIRouter, HTTPException + +from app.core.config import DISCOGS_SEARCH_BUDGET_SEC +from app.core.settings import get_discogs_token +from app.pipeline import discogs + +logger = logging.getLogger("stemdeck.artist") + +router = APIRouter() + +_ARTIST_ID_RE = re.compile(r"^[1-9]\d{0,11}$") + + +async def _bounded(func: Any, *args: Any) -> Any: + """``func(*args, cancelled=...)`` in a thread, given up on after + DISCOGS_SEARCH_BUDGET_SEC; the thread stops at its next request.""" + deadline = time.monotonic() + DISCOGS_SEARCH_BUDGET_SEC + + def cancelled() -> bool: + return time.monotonic() > deadline + + try: + return await asyncio.wait_for( + asyncio.to_thread(func, *args, cancelled=cancelled), DISCOGS_SEARCH_BUDGET_SEC + 1 + ) + except asyncio.TimeoutError: + logger.info("Discogs artist request timed out") + return None + + +@router.get("/artist") +async def search_discogs_artists(q: str = "") -> dict[str, list[dict[str, Any]]]: + """{"candidates": [{"id", "name", "profile"}]} for the name ``q``, empty + when Discogs has none. 422 for a name that is empty, over + discogs.QUERY_MAX_CHARS or holds control characters; 404 when no token + is saved; 503 when Discogs could not be asked.""" + query = discogs.valid_query(q) + if not query: + raise HTTPException(status_code=422, detail="invalid name") + token = get_discogs_token() + if not token: + raise HTTPException(status_code=404, detail="no Discogs token") + try: + found = await _bounded(discogs.search_artists, query, token) + except Exception: + logger.exception("Discogs artist search failed") + found = None + if found is None: + raise HTTPException(status_code=503, detail="Discogs could not be reached") + return {"candidates": found} + + +@router.get("/artist/{artist_id}") +async def get_discogs_artist(artist_id: str) -> dict[str, Any]: + """The Discogs artist ``artist_id``, as GET /api/jobs/{id}/artist-extra + answers, or 404 when the id is not one, no token is saved, or Discogs has + no such artist or could not be asked.""" + if not _ARTIST_ID_RE.match(artist_id): + raise HTTPException(status_code=404, detail="no artist details") + token = get_discogs_token() + if not token: + raise HTTPException(status_code=404, detail="no artist details") + try: + answer = await _bounded(discogs.artist_by_id, int(artist_id), token) + except Exception: + logger.exception("Discogs artist failed") + answer = None + if answer is None: + raise HTTPException(status_code=404, detail="no artist details") + return answer diff --git a/app/api/jobs.py b/app/api/jobs.py index 4c1c2a29..bb7a3ae3 100644 --- a/app/api/jobs.py +++ b/app/api/jobs.py @@ -9,6 +9,7 @@ import shutil import subprocess import threading +import time import uuid from pathlib import Path from typing import Literal @@ -18,14 +19,23 @@ from pydantic import BaseModel, Field, field_validator from app.core.config import ( + DISCOGS_LOOKUP_BUDGET_SEC, JOB_ID_RE, JOBS_DIR, + LYRICS_LOOKUP_BUDGET_SEC, + LYRICS_OFFSET_MAX_SEC, + LYRICS_TIMED_MAX_CHARS, MAX_PENDING_UPLOAD_JOBS, MAX_PENDING_URL_JOBS, + RETIME_DELETE_WAIT_SEC, STEM_NAMES, + TIMEOUT_FETCH_TAGS, + TIMEOUT_IDENTIFY_BACKFILL, + TIMEOUT_LYRICS_LOOKUP, + TIMEOUT_WORK_BACKFILL, ffprobe_executable, ) -from app.core.models import Job, _set +from app.core.models import Job, JobCancelled, _set from app.core.registry import all_jobs as registry_all_jobs from app.core.registry import get as registry_get from app.core.registry import get_proc as registry_get_proc @@ -35,14 +45,52 @@ from app.core.registry import register_if_capacity as registry_register_if_capacity from app.core.registry import remove as registry_remove from app.core.registry import set_trashed as registry_set_trashed -from app.core.settings import get_auto_sections, get_max_duration_sec +from app.core.settings import ( + get_acoustid_api_key, + get_auto_sections, + get_discogs_token, + get_max_duration_sec, +) from app.core.stems_location import is_relocating -from app.pipeline import jobqueue +from app.pipeline import discogs, jobqueue +from app.pipeline.artist_lookup import tagged_artist_name +from app.pipeline.audio_tags import probe_tags from app.pipeline.collect import merge_stem_peaks, presence_for_split -from app.pipeline.download import InvalidYouTubeURL, validate_youtube_url +from app.pipeline.download import InvalidYouTubeURL, fetch_audio_tags, validate_youtube_url from app.pipeline.errors import classify_failure +from app.pipeline.identify import can_identify_title, identify_and_find_band, release_source +from app.pipeline.lyrics_align import detect_offset +from app.pipeline.lyrics_lookup import ( + _TEXT_MAX_CHARS, + NotTheseLyrics, + build_query, + candidates_path, + copy_lyrics, + find_lyrics, + keep_answer, + lyrics_path, + lyrics_settled, + read_candidates, + read_lyrics, + saved_lyrics_belong, + set_lyrics_offset, + set_user_synced, +) +from app.pipeline.lyrics_retime import ( + RetimeFailed, + RetimeRun, + cancel_run, + claim_run, + current_run, + forget_run, + lyric_pairs, + retime_job, + wait_run_ended, +) +from app.pipeline.lyrics_retime import saved_view as saved_retime_view from app.pipeline.runner import _pipeline_lock from app.pipeline.vocal_split import split_vocals +from app.pipeline.work_lookup import find_work, might_have_work router = APIRouter(tags=["jobs"]) logger = logging.getLogger("stemdeck.api") @@ -309,6 +357,10 @@ async def _create_local_job(request: Request) -> dict[str, str]: shutil.rmtree(job_dir, ignore_errors=True) raise + # Now, while the upload still exists: it is deleted once the pipeline is + # done with it, and the stems carry no tags. Never fails the upload. + audio_tags = await asyncio.to_thread(probe_tags, source_path) + title = _sanitize_title(filename) local_source_url = f"local:{title}" job = Job( @@ -318,6 +370,7 @@ async def _create_local_job(request: Request) -> dict[str, str]: duration_sec=duration, source_url=local_source_url, source_format=ext.removeprefix("."), + audio_tags=audio_tags, auto_sections=get_auto_sections(), ) if not registry_register_if_capacity(job, MAX_PENDING_UPLOAD_JOBS): @@ -396,6 +449,13 @@ def cancel_job(job_id: str) -> dict: proc = registry_get_proc(job_id) if proc is not None and proc.poll() is None: proc.terminate() + # A line-by-line timing of its lyrics (POST .../lyrics/retime) stops + # the same way: the run sees its own flag, never the job's, and + # leaves the lyrics as they were. + elif cancel_run(job_id): + proc = registry_get_proc(job_id) + if proc is not None and proc.poll() is None: + proc.terminate() return job.to_state() job.cancel_requested = True @@ -534,6 +594,8 @@ async def resplit_job(job_id: str, body: ResplitBody) -> dict: logger.exception("[%s] resplit could not stage the source", job_id) raise HTTPException(status_code=500, detail="could not start re-split") from None + # The same recording, so the same lyrics, and LRCLIB's "nothing" too. + has_lyrics = await asyncio.to_thread(copy_lyrics, _job_dir(job_id), new_dir) new_job = Job( id=new_id, selected_stems=selected, @@ -544,6 +606,12 @@ async def resplit_job(job_id: str, body: ResplitBody) -> dict: # track it came from instead of replacing it. source_url=_resplit_source_url(job.source_url, new_id), source_format=source.suffix.lower().removeprefix("."), + # The same recording, so the same tags, identity and band. + audio_tags=job.audio_tags, + identity=job.identity, + artist=job.artist, + work=job.work, + has_lyrics=has_lyrics, auto_sections=get_auto_sections(), ) if not registry_register_if_capacity(new_job, MAX_PENDING_UPLOAD_JOBS): @@ -719,6 +787,681 @@ def update_sections(job_id: str, body: SectionsBody) -> dict: return {"job_id": job_id, "sections": validated, "sections_source": "manual"} +# Jobs whose tags are being looked up right now. Only touched on the event +# loop, with no await between the check and the add, so a plain set is enough. +_TAG_LOOKUPS: set[str] = set() + + +def _lookup_audio_tags(job: Job) -> dict[str, str] | None: + """Blocking: the tags a finished job's own source can still give, or None. + + An upload is asked through the source kept beside it, if one was. A link + is asked through its metadata, and only once the stored URL has passed the + same validator as an import: it came from the registry, not from this + request, but nothing that fails that check is ever handed to yt-dlp. + Raises when the fetch fails. + """ + source_url = job.source_url or "" + if source_url.startswith("local:"): + source = _retained_source(job.id) + return probe_tags(source) if source is not None else None + try: + url = validate_youtube_url(source_url) + except InvalidYouTubeURL: + return None + return fetch_audio_tags(url) + + +def _write_metadata_fields(job_id: str, fields: dict[str, object]) -> None: + """Put ``fields`` (the tags, the band) into metadata.json, if the job has + one, and change nothing else in it: restore() rebuilds a lost registry + entry from this file.""" + meta_path = (JOBS_DIR / job_id / "metadata.json").resolve() + if not meta_path.is_relative_to(JOBS_DIR.resolve()): + return + # The same lock as the sections editor, the other read-modify-write here. + with _SECTIONS_WRITE_LOCK: + try: + if not meta_path.is_file(): + return + meta = json.loads(meta_path.read_text(encoding="utf-8")) + if not isinstance(meta, dict): + return + meta.update(fields) + _write_json_atomic(meta_path, meta) + except (OSError, ValueError): + logger.warning("could not record audio tags in metadata for %s", job_id, exc_info=True) + + +@router.post("/{job_id}/audio-tags") +async def refresh_audio_tags(job_id: str) -> dict: + """Fill in a finished track's tags, and the band they name (#699), for + tracks imported before the pipeline found them. + + Answers {"audio_tags": {...} | null, "artist": {...} | null, + "has_lyrics": bool}. A lookup that fails or times out is "nothing found", + not an error: the page asks once per track and does not retry, and the + reason is in the server log. The band is looked up when the tags name an + artist and the job has no band yet, whether the tags were found just now + or were there already, and the lyrics (lyrics.json, GET .../lyrics) when + the job has none and anything is known to look them up by. + + Also "identity" (app/pipeline/identify.py), found here for a job that has + none, the same way the pipeline finds it: an AcoustID fingerprint of the + kept upload or of the stems summed, when the user set a key, else a + MusicBrainz search by the tags, else by the title (and LRCLIB), else the + tags. The band is then found + through it. An added key, so older clients read the answer as before. + + And "work" (app/pipeline/work_lookup.py): the musical, film or series a + soundtrack or cast recording is from, found here for a job that has none + when its identity, album tag or title says it is one. Also an added key. + + Deliberately outside _pipeline_lock. This is one metadata request, and + waiting behind a twenty-minute separation for it would leave the artist + box empty for no reason. + """ + if not JOB_ID_RE.match(job_id): + raise HTTPException(status_code=404, detail="job not found") + job = registry_get(job_id) + if job is None: + raise HTTPException(status_code=404, detail="job not found") + if job.status != "done": + raise HTTPException(status_code=409, detail="job is not finished") + job_dir = _job_dir(job_id) + if ( + job.audio_tags + and (job.artist or not tagged_artist_name(job.audio_tags.get("artist"))) + and lyrics_settled(job_dir) + and not _wants_identity(job, job.audio_tags) + and not _wants_work(job, job.identity, job.audio_tags) + ): + return { + "audio_tags": job.audio_tags, + "artist": job.artist, + "has_lyrics": True, + "identity": job.identity, + "work": job.work, + } + if job_id in _TAG_LOOKUPS: + raise HTTPException(status_code=409, detail="already looking up this track") + + _TAG_LOOKUPS.add(job_id) + try: + tags = job.audio_tags + if not tags: + try: + # A timeout abandons the wait, not the thread: yt-dlp's own + # socket timeout ends that, and its answer is then dropped. + tags = await asyncio.wait_for( + asyncio.to_thread(_lookup_audio_tags, job), timeout=TIMEOUT_FETCH_TAGS + ) + except asyncio.TimeoutError: + logger.warning( + "[%s] audio tag lookup timed out after %ds", job_id, TIMEOUT_FETCH_TAGS + ) + tags = None + except Exception: + logger.warning("[%s] audio tag lookup failed", job_id, exc_info=True) + tags = None + artist = None + identity = job.identity + # The identity when the job has none and there is something to find + # it by (a key to fingerprint with, or an artist tag), and the band + # through it, only for a job with none and a name to find it by: a + # job here for its lyrics alone may have tags naming no artist. + want_identity = _wants_identity(job, tags) + want_band = not job.artist and bool( + tagged_artist_name((tags or {}).get("artist")) + or (identity or {}).get("artist") + or want_identity + ) + if want_identity or want_band: + audio = await asyncio.to_thread(_fingerprint_audio, job) if want_identity else [] + # identify_and_find_band never raises; the wait is bounded all the + # same, since a socket timeout is per read, not per request. + try: + identity, artist = await asyncio.wait_for( + asyncio.to_thread( + identify_and_find_band, + job, + audio, + tags=tags, + want_band=want_band, + cancelled=lambda: registry_get(job_id) is not job, + ), + timeout=TIMEOUT_IDENTIFY_BACKFILL, + ) + except asyncio.TimeoutError: + logger.info("[%s] identification timed out", job_id) + work = None + # The work behind a soundtrack, once the identity says which recording + # this is. find_work never raises; the wait is bounded as above. + if _wants_work(job, identity, tags): + try: + work = await asyncio.wait_for( + asyncio.to_thread( + find_work, + identity, + tags, + title=job.title, + cancelled=lambda: registry_get(job_id) is not job, + ), + timeout=TIMEOUT_WORK_BACKFILL, + ) + except asyncio.TimeoutError: + logger.info("[%s] work lookup timed out", job_id) + lyrics = None + query = build_query(job, audio_tags=tags, band=artist, identity=identity) + # Not asked again while LRCLIB's last "nothing" is recent. + if query is not None and not lyrics_settled(job_dir): + # find_lyrics never raises, and starts no request past its budget. + try: + lyrics = await asyncio.wait_for( + asyncio.to_thread( + find_lyrics, + query, + cancelled=lambda: registry_get(job_id) is not job, + fallback_title=job.title or "", + ), + timeout=LYRICS_LOOKUP_BUDGET_SEC + TIMEOUT_LYRICS_LOOKUP + 1, + ) + except asyncio.TimeoutError: + logger.info("[%s] lyrics lookup timed out", job_id) + finally: + _TAG_LOOKUPS.discard(job_id) + + # The track was deleted while the lookup ran. + if registry_get(job_id) is not job: + return { + "audio_tags": None, + "artist": None, + "has_lyrics": False, + "identity": None, + "work": None, + } + if ( + lyrics + and not lyrics_path(job_dir).is_file() + and await asyncio.to_thread(keep_answer, job, job_dir, lyrics) + ): + registry_persist(JOBS_DIR) + found: dict[str, object] = {} + if tags and not job.audio_tags: + found["audio_tags"] = tags + if artist and not job.artist: + found["artist"] = artist + if identity and not job.identity: + found["identity"] = identity + if work and not job.work: + found["work"] = work + if found: + _set(job, **found) + registry_persist(JOBS_DIR) + await asyncio.to_thread(_write_metadata_fields, job_id, found) + return { + "audio_tags": job.audio_tags, + "artist": job.artist, + "has_lyrics": lyrics_path(job_dir).is_file(), + "identity": job.identity, + "work": job.work, + } + + +def _wants_identity(job: Job, tags: dict[str, str] | None) -> bool: + """Whether the backfill should identify ``job``: it has no identity yet, + and there is a key to fingerprint with, an artist tag to search by, or a + title naming more than a song (a YouTube upload with no music metadata).""" + if job.identity is not None: + return False + return ( + bool(get_acoustid_api_key()) + or bool(tagged_artist_name((tags or {}).get("artist"))) + or can_identify_title(tags, job.title) + ) + + +def _wants_work(job: Job, identity: dict[str, object] | None, tags: dict[str, str] | None) -> bool: + """Whether the backfill should look for the work ``job``'s song is from: + it has none yet, and its identity, album tag or title says it is from one.""" + return job.work is None and might_have_work(identity, tags, job.title) + + +def _fingerprint_audio(job: Job) -> list[Path]: + """Blocking: what to fingerprint a finished job from. The upload kept + beside it; else its stems, which add back up to the mix (a link's + download is deleted once they exist). Only whitelisted stem names, each + resolved inside the job's directory.""" + source = _retained_source(job.id) + if source is not None: + return [source] + stems_root = (JOBS_DIR / job.id / "stems").resolve() + stems = [] + for name in STEM_NAMES: + path = (stems_root / f"{name}.wav").resolve() + if path.is_file() and path.is_relative_to(JOBS_DIR.resolve()): + stems.append(path) + return stems + + +def _job_dir(job_id: str) -> Path: + """A job's directory, for an id that has passed JOB_ID_RE. Refused (404) + if it would resolve outside JOBS_DIR all the same.""" + job_dir = (JOBS_DIR / job_id).resolve() + if not job_dir.is_relative_to(JOBS_DIR.resolve()): + raise HTTPException(status_code=404, detail="job not found") + return job_dir + + +# One Discogs lookup per job at a time: a second request for the same job +# (the box opened twice, two tabs) waits for the first one's answer. +_DISCOGS_LOOKUPS: dict[str, asyncio.Task] = {} + + +@router.get("/{job_id}/artist-extra") +async def get_artist_extra(job_id: str) -> dict: + """The Discogs profile of the band a track is by (app/pipeline/discogs.py), + for the artist box, when Wikipedia has no article on it or leaves gaps. + + Answers {"id", "name", "real_name", "profile": [...], "members": + {"current", "former"}, "groups", "links": [{"kind", "url"}], "releases": + [{"year", "title"}], "url"}, or 404 when no Discogs token is set or no + artist is confidently the track's. The token never leaves the server: the + page asks here, and this asks Discogs. + + Outside _pipeline_lock, as the tag backfill is: a few metadata requests + must not wait behind a separation. Bounded by DISCOGS_LOOKUP_BUDGET_SEC. + """ + if not JOB_ID_RE.match(job_id): + raise HTTPException(status_code=404, detail="job not found") + job = registry_get(job_id) + if job is None: + raise HTTPException(status_code=404, detail="job not found") + token = get_discogs_token() + if not token: + raise HTTPException(status_code=404, detail="no artist details") + + task = _DISCOGS_LOOKUPS.get(job_id) + if task is None: + deadline = time.monotonic() + DISCOGS_LOOKUP_BUDGET_SEC + + def cancelled() -> bool: + return registry_get(job_id) is not job or time.monotonic() > deadline + + task = asyncio.create_task( + asyncio.to_thread( + discogs.artist_for_track, + job.identity, + job.audio_tags, + job.artist, + token, + cancelled=cancelled, + ) + ) + _DISCOGS_LOOKUPS[job_id] = task + task.add_done_callback(lambda _t: _DISCOGS_LOOKUPS.pop(job_id, None)) + try: + # Shielded, so one caller giving up does not cancel another's answer. + answer = await asyncio.wait_for(asyncio.shield(task), DISCOGS_LOOKUP_BUDGET_SEC + 1) + except asyncio.TimeoutError: + logger.info("[%s] Discogs lookup timed out", job_id) + answer = None + except Exception: + logger.exception("[%s] Discogs lookup failed", job_id) + answer = None + if answer is None or registry_get(job_id) is not job: + raise HTTPException(status_code=404, detail="no artist details") + return answer + + +@router.get("/{job_id}/lyrics") +def get_lyrics(job_id: str) -> Response: + """The track's lyrics, as found while it was separated (lyrics.json, see + app/pipeline/lyrics_lookup.py), or 404 when it has none. + + A 404 may carry {"others": [...]}: the versions LRCLIB had when none was + the length of the track (lyrics_candidates.json), for the tab to offer. + A transcription with no others of its own is served with those. + + Served apart from the job's state, which carries only has_lyrics: the + text runs to kilobytes, with every other version kept beside it, and the + state is sent on every change to a running job. + """ + if not JOB_ID_RE.match(job_id): + raise HTTPException(status_code=404, detail="job not found") + if registry_get(job_id) is None: + raise HTTPException(status_code=404, detail="job not found") + job_dir = _job_dir(job_id) + jobs_root = JOBS_DIR.resolve() + path = lyrics_path(job_dir).resolve() + found_path = candidates_path(job_dir).resolve() + if not path.is_relative_to(jobs_root) or not found_path.is_relative_to(jobs_root): + raise HTTPException(status_code=404, detail="no lyrics") + entry = read_lyrics(job_dir) if path.is_file() else None + if entry is None and path.is_file(): + logger.warning("unreadable lyrics for %s", job_id) + found = read_candidates(job_dir) if found_path.is_file() else None + others = found["others"] if found else [] + # Lyrics kept before a lookup held them to the track's own song and artist + # can be another song's: checked by that rule each time they are asked for, + # and dropped for good when they fail it. + job = registry_get(job_id) + query = build_query(job) if job is not None else None + others = [o for o in others if saved_lyrics_belong(o, query)] + # Never lyrics timed by the user, moved by them, or found line by line + # in the vocals: each says they are the track's, whatever the names say. + worked_on = entry is not None and any( + k in entry for k in ("user_synced", "aligned", "offset_sec") + ) + if entry is not None and not worked_on and not saved_lyrics_belong(entry, query): + logger.info("[%s] dropping kept lyrics of another song", job_id) + path.unlink(missing_ok=True) + if job is not None: + _set(job, has_lyrics=False) + registry_persist(JOBS_DIR) + entry = None + if entry is not None: + entry["others"] = [o for o in entry["others"] if saved_lyrics_belong(o, query)] + headers = {"Cache-Control": "no-cache"} + if entry is not None: + if not entry["others"]: + entry["others"] = others + return JSONResponse(entry, headers=headers) + if others: + return JSONResponse( + {"detail": "no lyrics", "others": others}, status_code=404, headers=headers + ) + raise HTTPException(status_code=404, detail="no lyrics") + + +class LyricsOffsetBody(BaseModel): + """How many seconds later than their own timing the lyrics are shown: + negative for earlier, 0 for their own timing again.""" + + offset_sec: float = Field( + ge=-LYRICS_OFFSET_MAX_SEC, le=LYRICS_OFFSET_MAX_SEC, allow_inf_nan=False + ) + + +class LyricsAlignBody(BaseModel): + """Synced lyrics kept only in the browser (found by the tab's own + LRCLIB lookup), to estimate against the track's vocals without saving + anything. Omitted for the lyrics the server keeps (lyrics.json).""" + + synced: str | None = Field(default=None, max_length=_TEXT_MAX_CHARS) + + +def _lyrics_job(job_id: str) -> tuple[Job, Path]: + """The job and its directory, for a lyrics edit: 404 for a malformed or + unknown id, or one whose lyrics.json would resolve outside JOBS_DIR.""" + if not JOB_ID_RE.match(job_id): + raise HTTPException(status_code=404, detail="job not found") + job = registry_get(job_id) + if job is None: + raise HTTPException(status_code=404, detail="job not found") + job_dir = _job_dir(job_id) + if not lyrics_path(job_dir).resolve().is_relative_to(JOBS_DIR.resolve()): + raise HTTPException(status_code=404, detail="no lyrics") + return job, job_dir + + +def _synced_lyrics(job_dir: Path) -> dict: + """Blocking: the job's lyrics.json when it has synced lyrics. 404 when it + has none, 409 when they are text only and have no timing to move.""" + entry = read_lyrics(job_dir) if lyrics_path(job_dir).is_file() else None + if entry is None: + raise HTTPException(status_code=404, detail="no lyrics") + if not entry["synced"]: + raise HTTPException(status_code=409, detail="lyrics are not synced") + return entry + + +# A lyrics edit's body: an offset, or at most _TEXT_MAX_CHARS of LRC, which +# as UTF-8 runs to four bytes a character. +_LYRICS_BODY_MAX_BYTES = 4 * _TEXT_MAX_CHARS + 1024 + + +# A manual timing's body: LYRICS_TIMED_MAX_CHARS of LRC, as UTF-8. +_TIMED_BODY_MAX_BYTES = 4 * LYRICS_TIMED_MAX_CHARS + 1024 + + +async def _lyrics_body( + request: Request, + model: type[BaseModel], + empty_ok: bool = False, + max_bytes: int = _LYRICS_BODY_MAX_BYTES, +): + """A lyrics edit's JSON body as ``model``: None for no body at all when + ``empty_ok``. 413 past _LYRICS_BODY_MAX_BYTES, else 422 for anything that + is not the model, with NaN and Infinity refused while parsing (see + _reject_non_finite) and nothing sent back of what was submitted.""" + declared = request.headers.get("content-length", "") + if declared.isdigit() and int(declared) > max_bytes: + raise HTTPException(status_code=413, detail="body too large") + raw = await request.body() + if len(raw) > max_bytes: + raise HTTPException(status_code=413, detail="body too large") + if not raw.strip() and empty_ok: + return None + try: + data = json.loads(raw, parse_constant=_reject_non_finite) + except (ValueError, UnicodeDecodeError) as exc: + raise HTTPException(status_code=422, detail="invalid JSON body") from exc + if not isinstance(data, dict): + raise HTTPException(status_code=422, detail="expected a JSON object") + try: + return model(**data) + except Exception as exc: + raise HTTPException(status_code=422, detail="invalid lyrics edit") from exc + + +@router.post("/{job_id}/lyrics/offset") +async def set_lyrics_offset_route(job_id: str, request: Request) -> dict: + """Keep the user's alignment of the track's synced lyrics (the Lyrics + tab's Align panel): {"offset_sec": seconds later, negative for earlier, + 0 for their own timing}. Kept beside the synced text in lyrics.json, which + is never rewritten, and applied by the page. Answers the offset kept.""" + job, job_dir = _lyrics_job(job_id) + body = await _lyrics_body(request, LyricsOffsetBody) + await asyncio.to_thread(_synced_lyrics, job_dir) + entry = await asyncio.to_thread(set_lyrics_offset, job, job_dir, body.offset_sec) + if entry is None: + raise HTTPException(status_code=500, detail="could not save lyrics") + return {"offset_sec": entry["offset_sec"]} + + +@router.post("/{job_id}/lyrics/align") +async def align_lyrics_route(job_id: str, request: Request) -> dict: + """Estimate from the vocals stem how much later the lyrics are sung on + this track than their stamps say, whatever timing they were saved with + (the Align panel's Auto-detect): {"confident": bool, "offset_sec": + seconds or null}. + + Without a body, for the lyrics the server keeps: a confident estimate is + kept as their offset, as POST .../lyrics/offset would, and one that is + not leaves them as they are. With {"synced": ...}, lyrics the browser + keeps: only the estimate is answered, and nothing is saved. + + Outside _pipeline_lock, like GET .../vocal-envelope: at most one streamed + read of one stem, and none when its envelope is already kept.""" + job, job_dir = _lyrics_job(job_id) + body = await _lyrics_body(request, LyricsAlignBody, empty_ok=True) + if job.status != "done": + raise HTTPException(status_code=409, detail="job not ready") + stems_dir = (job_dir / "stems").resolve() + if not stems_dir.is_relative_to(JOBS_DIR.resolve()): + raise HTTPException(status_code=404, detail="job not found") + kept_here = body is None or body.synced is None + if kept_here: + synced = (await asyncio.to_thread(_synced_lyrics, job_dir))["synced"] + else: + synced = body.synced + try: + shift = await asyncio.to_thread(detect_offset, synced, stems_dir) + except Exception: + logger.exception("lyrics alignment failed for %s", job_id) + raise HTTPException(status_code=500, detail="could not align lyrics") from None + if shift is None or abs(shift) > LYRICS_OFFSET_MAX_SEC: + return {"confident": False, "offset_sec": None} + if kept_here: + entry = await asyncio.to_thread(set_lyrics_offset, job, job_dir, shift) + if entry is None: + raise HTTPException(status_code=500, detail="could not save lyrics") + shift = entry["offset_sec"] + return {"confident": True, "offset_sec": shift} + + +# Line-by-line timing runs in flight, held so the loop does not drop them. +_retime_tasks: set[asyncio.Task] = set() + + +def _retime_ready(job_dir: Path, lyrics: dict | None) -> None: + """Blocking: 404 without lyrics (kept, or given), 409 when they have no + lines to time (an instrumental) or there is no vocals stem to time them + to.""" + if lyrics is None: + lyrics = read_lyrics(job_dir) if lyrics_path(job_dir).is_file() else None + if lyrics is None: + raise HTTPException(status_code=404, detail="no lyrics") + if not lyric_pairs(lyrics): + raise HTTPException(status_code=409, detail="lyrics have no lines") + vocals = (job_dir / "stems" / "vocals.wav").resolve() + if not vocals.is_relative_to(JOBS_DIR.resolve()) or not vocals.is_file(): + raise HTTPException(status_code=409, detail="no vocals stem") + + +async def _retime_task(job: Job, job_dir: Path, run: RetimeRun, lyrics: dict | None) -> None: + """One on-demand run: behind _pipeline_lock, so it never shares the GPU + with a separation, and stopped by a cancel or the job's deletion.""" + + def cancelled() -> bool: + return run.cancel.is_set() or registry_get(job.id) is not job + + def report(share: float) -> None: + run.progress = share + + try: + async with _pipeline_lock: + run.started = True + if cancelled(): + raise JobCancelled() + outcome = await asyncio.to_thread( + retime_job, job, job_dir, cancelled=cancelled, report=report, lyrics=lyrics + ) + run.outcome = outcome + run.state = outcome.state + except JobCancelled: + run.state = "idle" + except RetimeFailed as exc: + logger.info("[%s] lyrics not timed: %s", job.id, exc) + run.state = "failed" + except Exception: + logger.exception("[%s] lyrics timing failed", job.id) + run.state = "failed" + finally: + run.ended.set() + + +class LyricsRetimeBody(BaseModel): + """Lyrics the browser keeps (found by the tab's own LRCLIB lookup), to + time without saving anything. Omitted for the lyrics the server keeps.""" + + synced: str | None = Field(default=None, max_length=_TEXT_MAX_CHARS) + plain: str | None = Field(default=None, max_length=_TEXT_MAX_CHARS) + + +@router.post("/{job_id}/lyrics/retime") +async def start_lyrics_retime(job_id: str, request: Request) -> JSONResponse: + """Time the track's lyrics line by line to its vocals + (app/pipeline/lyrics_retime.py), in the background: 202 with the run's + state, which GET .../lyrics/retime then follows. + + Without a body, the lyrics the server keeps: the result is kept in + lyrics.json as "aligned" when it is good enough, and nothing changes + when it is not ("unsure"). With {"synced", "plain"}, lyrics the browser + keeps: the result comes back as "aligned" in GET's "done" answer and + nothing is saved but the transcript. + + Runs behind the pipeline lock, after any import in progress, and takes a + transcription of the vocals the first time (about 30 s on a GPU, a few + minutes on a CPU; asked for here, it runs on either). One run at a time + per track: 409 while one is running. POST .../cancel stops it.""" + job, job_dir = _lyrics_job(job_id) + body = await _lyrics_body(request, LyricsRetimeBody, empty_ok=True) + if job.status != "done": + raise HTTPException(status_code=409, detail="job not ready") + lyrics = None + if body is not None and (body.synced or body.plain): + lyrics = {"synced": body.synced or "", "plain": body.plain or ""} + await asyncio.to_thread(_retime_ready, job_dir, lyrics) + run = claim_run(job_id) + if run is None: + raise HTTPException(status_code=409, detail="lyrics timing already running") + task = asyncio.create_task(_retime_task(job, job_dir, run, lyrics)) + _retime_tasks.add(task) + task.add_done_callback(_retime_tasks.discard) + return JSONResponse(run.view(), status_code=202) + + +@router.get("/{job_id}/lyrics/retime") +def get_lyrics_retime(job_id: str) -> dict: + """Where the line-by-line timing of the track's lyrics stands: + {"state": "idle" | "running" | "done" | "failed" | "unsure"}, with + "progress" (0 to 1) while running and "matched" (the share of the + lyrics heard), "lines_matched" and "lines" once there is a result; for + lyrics the browser sent, "done" carries the timing itself, "aligned".""" + job, job_dir = _lyrics_job(job_id) + run = current_run(job_id) + if run is not None and ( + run.state not in ("done", "idle") + or (run.outcome is not None and run.outcome.aligned is not None) + ): + return run.view() + entry = read_lyrics(job_dir) if lyrics_path(job_dir).is_file() else None + saved = saved_retime_view(entry) + if run is not None and run.state == "idle" and saved["state"] != "done": + return run.view() + return saved + + +class UserSyncedBody(BaseModel): + """The user's own timing of the lyrics, from the sync editor: LRC whose + lines are the lyrics' own, one to one.""" + + synced: str = Field(min_length=1, max_length=LYRICS_TIMED_MAX_CHARS) + + +@router.put("/{job_id}/lyrics/user-synced") +async def put_user_synced(job_id: str, request: Request) -> dict: + """Keep the user's own timing of the track's lyrics, shown before any + other (see app/pipeline/lyrics_retime.py). 422 when it is not LRC, its + stamps go back in time, or its lines are not exactly the lyrics' own: + it can time them, never change them.""" + job, job_dir = _lyrics_job(job_id) + body = await _lyrics_body(request, UserSyncedBody, max_bytes=_TIMED_BODY_MAX_BYTES) + try: + entry = await asyncio.to_thread(set_user_synced, job, job_dir, body.synced) + except NotTheseLyrics as exc: + raise HTTPException(status_code=422, detail="not a timing of these lyrics") from exc + if entry is None: + if not lyrics_path(job_dir).is_file(): + raise HTTPException(status_code=404, detail="no lyrics") + raise HTTPException(status_code=500, detail="could not save lyrics") + return {"user_synced": entry["user_synced"]} + + +@router.delete("/{job_id}/lyrics/user-synced") +async def delete_user_synced(job_id: str) -> dict: + """Drop the user's own timing of the track's lyrics. Idempotent.""" + job, job_dir = _lyrics_job(job_id) + if not lyrics_path(job_dir).is_file(): + raise HTTPException(status_code=404, detail="no lyrics") + entry = await asyncio.to_thread(set_user_synced, job, job_dir, None) + if entry is None: + raise HTTPException(status_code=500, detail="could not save lyrics") + return {"user_synced": None} + + # Upper bound on an edited grid. A 20-minute track at 300 BPM is ~6000 beats; # 20000 leaves generous headroom while refusing a payload crafted to exhaust # memory or disk. @@ -962,6 +1705,18 @@ def delete_job(job_id: str) -> dict[str, str]: raise HTTPException(status_code=404, detail="job not found") if job.status not in ("done", "error", "cancelled"): raise HTTPException(status_code=409, detail="job is still running") + # A timing of its lyrics in flight stops rather than write into a + # directory being removed (it also checks the job is still registered). + if cancel_run(job_id) and job.vocal_split != "running": + proc = registry_get_proc(job_id) + if proc is not None and proc.poll() is None: + proc.terminate() + # The worker may still have the vocals open while it stops. + if not wait_run_ended(job_id, RETIME_DELETE_WAIT_SEC): + logger.warning("[%s] lyrics timing still stopping; removing files anyway", job_id) + forget_run(job_id) + # A fingerprint of its audio may still have a file open (Windows). + release_source(job_id) removed = _rmtree_job(job_id) # Recorded whether or not the files went away. The user asked for this job # to be gone; without the record, a directory that outlived the delete is diff --git a/app/api/router.py b/app/api/router.py index 3232a5a6..f44d7639 100644 --- a/app/api/router.py +++ b/app/api/router.py @@ -3,6 +3,7 @@ from fastapi import APIRouter from app.api.config import router as config_router +from app.api.discogs import router as discogs_router from app.api.events import router as events_router from app.api.jobs import router as jobs_router from app.api.playlist import router as playlist_router @@ -20,3 +21,4 @@ router.include_router(queue_router, prefix="/queue", tags=["queue"]) router.include_router(playlist_router, prefix="/playlist", tags=["playlist"]) router.include_router(search_router, prefix="/search", tags=["search"]) +router.include_router(discogs_router, prefix="/discogs", tags=["discogs"]) diff --git a/app/api/stems.py b/app/api/stems.py index 95bc4a86..b031ee8f 100644 --- a/app/api/stems.py +++ b/app/api/stems.py @@ -29,6 +29,7 @@ ) from app.core.registry import get as registry_get from app.core.settings import get_export_sample_rate +from app.pipeline.audio_stats import vocal_envelope from app.pipeline.click_render import cache_key as click_cache_key from app.pipeline.click_render import count_in_beats, render_click_wav, render_count_in_wav @@ -870,6 +871,61 @@ async def get_stem_peaks(job_id: str, request: Request) -> Response: return _peaks_response(job_id, request) +# The vocals stem's level every 40ms: fine enough to find the gap between two +# sung words (a breath is 150ms or more), coarse enough that a ten minute track +# is about 60 KB. peaks.json cannot serve instead: it is a fixed 3000 points +# per track, 100ms apart on a five minute song and 200ms on a ten minute one. +_VOCAL_ENVELOPE_HOP_SEC = 0.04 +_VOCAL_ENVELOPE_FILE = "vocal_envelope.json" + + +def _write_vocal_envelope(src: Path, dest: Path) -> None: + hop, levels = vocal_envelope(src, _VOCAL_ENVELOPE_HOP_SEC) + # A unique temp name: two first requests for the same track may compute at + # once, and each moves a whole file into place. + tmp = dest.with_name(f"{dest.name}.{uuid.uuid4().hex}.tmp") + try: + tmp.write_text(json.dumps({"hop": hop, "db": levels}), encoding="utf-8") + os.replace(tmp, dest) + finally: + tmp.unlink(missing_ok=True) + + +@router.get("/jobs/{job_id}/vocal-envelope") +async def get_vocal_envelope(job_id: str) -> Response: + """How loud the vocals stem is over time, for the lyrics wipe (#699): + `{"hop": seconds, "db": [whole dBFS per hop, -90 to 0]}`. + + Lyrics from LRCLIB mostly stamp only when each line starts, so the tab + uses this to tell where in a line the singer is actually singing. + + Computed the first time a track asks and kept beside its stems, so every + job has it, including those separated before it existed. Recomputed if + vocals.wav is newer than what was kept. Deliberately outside + _pipeline_lock: it is one streamed read of one stem, under a second for a + five minute track, and waiting behind a whole separation would leave the + lyrics on the rough estimate for minutes. + """ + if not JOB_ID_RE.match(job_id): + raise HTTPException(status_code=404, detail="job not found") + job = registry_get(job_id) + if job is None or job.status != "done": + raise HTTPException(status_code=404, detail="job not ready") + stems_dir = (JOBS_DIR / job_id / "stems").resolve() + src = stems_dir / "vocals.wav" + dest = stems_dir / _VOCAL_ENVELOPE_FILE + jobs_root = JOBS_DIR.resolve() + if not src.is_file() or not src.is_relative_to(jobs_root) or not dest.is_relative_to(jobs_root): + raise HTTPException(status_code=404, detail="no vocals") + try: + if not dest.is_file() or dest.stat().st_mtime_ns < src.stat().st_mtime_ns: + await asyncio.to_thread(_write_vocal_envelope, src, dest) + except Exception: + logger.exception("could not compute the vocal envelope for %s", job_id) + raise HTTPException(status_code=500, detail="vocal envelope unavailable") from None + return FileResponse(dest, media_type="application/json", headers={"Cache-Control": "no-cache"}) + + @router.get("/jobs/{job_id}/stems/beats.json") async def get_beat_grid(job_id: str) -> Response: """Return the pre-computed beat grid driving the click track. diff --git a/app/core/config.py b/app/core/config.py index 0a9f0bc9..104f67cb 100644 --- a/app/core/config.py +++ b/app/core/config.py @@ -189,6 +189,14 @@ def _stored_jobs_dir() -> Path | None: "STEMDECK_FFPROBE", FFMPEG_DIR / ("ffprobe.exe" if sys.platform.startswith("win") else "ffprobe"), ) +# Chromaprint's fpcalc, for fingerprinting when the FFmpeg in use has no +# chromaprint muxer (app/pipeline/fpcalc.py). The desktop shell downloads it +# beside FFmpeg on macOS and Linux; elsewhere it is looked for beside the FFmpeg +# in use, then on PATH. +FPCALC_BIN = _env_path( + "STEMDECK_FPCALC", + FFMPEG_DIR / ("fpcalc.exe" if sys.platform.startswith("win") else "fpcalc"), +) # JavaScript runtime for yt-dlp's YouTube challenge solver (#432). Portable # builds drop a binary here because nothing is on PATH in a portable install; # Docker ships deno on PATH and source checkouts have whatever the developer @@ -319,6 +327,395 @@ def js_solver_available() -> bool: # so it gets its own, larger budget. TIMEOUT_BEATGRID = _env_int("STEMDECK_TIMEOUT_BEATGRID", 300) +# Reading an upload's tags (artist, title, album, lyrics) with ffprobe for the +# Lyrics tab and the artist box (#699). Tags are a nicety, so a probe that +# takes longer than this is abandoned and the upload goes ahead without them. +TIMEOUT_PROBE_TAGS = _env_int("STEMDECK_TIMEOUT_PROBE_TAGS", 15) +# Looking up a finished link's tags after the fact, for tracks imported before +# tags were read: one yt-dlp metadata request (two if YouTube's bot check sends +# it back for cookies), no download. Past this the lookup answers "nothing +# found" rather than keep the page waiting. +TIMEOUT_FETCH_TAGS = _env_int("STEMDECK_TIMEOUT_FETCH_TAGS", 45) +# Caps on what a tag may carry into the job record, which is rewritten whole +# on every save: a name longer than this is not a name, and a lyrics tag +# longer than this is not lyrics. +AUDIO_TAG_MAX_CHARS = 300 +AUDIO_TAG_LYRICS_MAX_CHARS = 20000 +# An upload's tags written in a Windows codepage but read as Latin-1 (an MP3's +# ID3v1 tag, or an ID3v2 frame marked ISO-8859-1): "zapomnia³em" for +# "zapomniałem", "Êèíî" for "Кино". The codepages tried when re-reading +# them, Central European then Cyrillic. See _repair_encoding in +# app/pipeline/audio_tags.py for when a re-reading is taken. +AUDIO_TAG_LEGACY_CODEPAGES = ("cp1250", "cp1251") +# A word of at least this many letters, every one of them accented Latin, is +# taken as mis-decoded ("Êèíî"). Real names this short made only of accents +# exist ("ÆØÅ"), so three letters is not evidence on its own. +AUDIO_TAG_MOJIBAKE_MIN_WORD = 4 + +# Finding the band a job's artist tag names, on Wikidata, while the job is +# separated (#699), so the artist box and the Lyrics tab have it the moment the +# import is done. Two requests, each abandoned after this many seconds. The +# lookup runs beside separation, which takes far longer, so it costs an import +# no time unless both requests are slow and the separation was fast. +TIMEOUT_ARTIST_LOOKUP = _env_int("STEMDECK_TIMEOUT_ARTIST_LOOKUP", 8) +# How long a finished pipeline waits for a lookup still in flight before it +# lets the job finish without a band. Kept short: the page finds the band +# itself when it opens a track that has none, so the wait buys little. +ARTIST_LOOKUP_GRACE_SEC = _env_int("STEMDECK_ARTIST_LOOKUP_GRACE_SEC", 2) +# Wikimedia asks every client to name itself and a way to reach its maker. +ARTIST_LOOKUP_USER_AGENT = ( + "StemDeck (https://github.com/stemdeckapp/stemdeck; self-hosted stem separation)" +) +# A search and one entity fetch for its hits come to well under this; an +# answer bigger than it is not one worth parsing. +ARTIST_LOOKUP_MAX_BYTES = 8 * 1024 * 1024 + +# Which recording a job is, found while it separates (app/pipeline/identify.py). +# +# AcoustID: an audio fingerprint of the first FINGERPRINT_LENGTH_SEC seconds +# (fpcalc's default length), made with ffmpeg's chromaprint muxer (or fpcalc, +# FPCALC_BIN, where that FFmpeg has none), looked up with the user's own API +# key. Only runs when the user set one in Settings. TIMEOUT_FINGERPRINT bounds +# either one. +ACOUSTID_LOOKUP_URL = "https://api.acoustid.org/v2/lookup" +# A key is tried once when it is saved (identify.acoustid_key_works), so a key +# AcoustID refuses is caught in Settings rather than failing every import in +# silence. 0 turns the check off, for the browser tests, which must never +# reach AcoustID. +ACOUSTID_CHECK_KEY = _env_int("STEMDECK_ACOUSTID_CHECK_KEY", 1) != 0 +# Discogs: the user's own personal access token, set in Settings, for band +# profiles Wikipedia does not cover. A token is tried once when it is saved +# (discogs_auth.discogs_token_works), as an AcoustID key is, so one Discogs +# refuses is caught in Settings. 0 turns the check off, for the browser tests, +# which must never reach Discogs. The check is abandoned after +# TIMEOUT_DISCOGS_CHECK seconds a read, and twice that in all. +DISCOGS_IDENTITY_URL = "https://api.discogs.com/oauth/identity" +DISCOGS_CHECK_TOKEN = _env_int("STEMDECK_DISCOGS_CHECK_TOKEN", 1) != 0 +TIMEOUT_DISCOGS_CHECK = _env_int("STEMDECK_TIMEOUT_DISCOGS_CHECK", 8) +FINGERPRINT_LENGTH_SEC = 120 +# Decoding two minutes of audio for the fingerprint takes about a second; past +# this the fingerprint is abandoned and identification goes on without it. +TIMEOUT_FINGERPRINT = _env_int("STEMDECK_TIMEOUT_FINGERPRINT", 30) +# AcoustID's score is how closely the fingerprints agree, 0..1. The same +# recording scores well above 0.9; a different mix or a live cut of the song +# lands far lower. Kept high because the answer is saved with nobody looking. +ACOUSTID_MIN_SCORE = 0.8 +# AcoustID allows three requests a second per client. +ACOUSTID_MIN_INTERVAL_SEC = 0.34 +# MusicBrainz: one request a second per client, and every client names itself +# with a way to reach its maker, or it is throttled harder. +MUSICBRAINZ_API = "https://musicbrainz.org/ws/2" +MUSICBRAINZ_MIN_INTERVAL_SEC = 1.0 + + +def _musicbrainz_user_agent() -> str: + try: + from app._version import version + + release = str(version).split("+", 1)[0] or "dev" + except Exception: + release = "dev" + return f"StemDeck/{release} ( https://github.com/stemdeckapp/stemdeck )" + + +MUSICBRAINZ_USER_AGENT = _musicbrainz_user_agent() +# MusicBrainz sheds load with 503 (and now and then 429) even to a client +# keeping to its rate: measured on 2026-09-27, four of 45 benchmark songs lost +# their identity to a single 503. A refused request is asked again up to this +# many times, waiting MUSICBRAINZ_RETRY_BACKOFF_SEC, then twice that, and so +# on, or what Retry-After asks, never more than MUSICBRAINZ_RETRY_MAX_WAIT_SEC. +MUSICBRAINZ_RETRIES = 3 +MUSICBRAINZ_RETRY_BACKOFF_SEC = 1.0 +MUSICBRAINZ_RETRY_MAX_WAIT_SEC = 8.0 +# The most one request may take, its retries and their waits included: past +# it, a request still being refused gives up, so a run of busy answers never +# holds a lookup (and the shared queue behind it) for a minute. +MUSICBRAINZ_REQUEST_BUDGET_SEC = 20.0 +# A request that would wait longer than this for its turn under the rate +# limit is not made at all: a queue that long means something else is +# hammering the service, and identification is best-effort. +RATE_LIMIT_MAX_WAIT_SEC = 15 +# A search by name is kept only when MusicBrainz scores it at least this +# (0..100) and its length is within IDENTIFY_DURATION_TOLERANCE_SEC of the +# track's: a name alone can be anybody's song, the length says which. +MUSICBRAINZ_SEARCH_MIN_SCORE = 90 +IDENTIFY_DURATION_TOLERANCE_SEC = 4 +# A track its tags do not name (a YouTube upload with no music metadata) is +# looked for by its title (app/pipeline/title_parse.py): up to this many +# readings of it on MusicBrainz, then, when none is confident, on LRCLIB. +TITLE_MUSICBRAINZ_SEARCHES = 3 +TITLE_LRCLIB_SEARCHES = 2 +# A recording found that way is kept only when its credit and release titles +# (or LRCLIB's artist and album) contain at least this share of the title's +# words beyond the song's name: the performers, the show. +TITLE_MIN_COVERAGE = 0.5 +# When no reading's words are found in a credit, the artist a reading names is +# looked up on MusicBrainz by name and alias, which finds 周杰倫 from "Jay +# Chou", IU from "아이유" and 鄧麗君 from "邓丽君", and the song is searched for +# among that artist's recordings: for up to this many readings, two requests +# each. +TITLE_ARTIST_SEARCHES = 2 +# A music video runs longer than the recording on the album: an intro, a +# story, credits. Among the recordings of an artist found by name, one this +# much shorter than the upload still counts, the nearest length first. +# Measured on 2026-09-27: IU's "Good Day" video is 124 seconds longer than +# the single, BTS "Spring Day" 55, Jay Chou's "Sunny Day" 49. +IDENTIFY_VIDEO_EXTRA_SEC = 150 +# Each AcoustID or MusicBrainz request is abandoned after this many seconds. +TIMEOUT_IDENTIFY_REQUEST = _env_int("STEMDECK_TIMEOUT_IDENTIFY_REQUEST", 8) +# How long a finished pipeline waits for identification still in flight, as +# ARTIST_LOOKUP_GRACE_SEC does for the band. Separation takes minutes, so the +# answer is nearly always waiting already. +IDENTIFY_GRACE_SEC = _env_int("STEMDECK_IDENTIFY_GRACE_SEC", 2) +# POST /api/jobs/{id}/audio-tags identifies an older track before it answers; +# past this it answers with what it has. +TIMEOUT_IDENTIFY_BACKFILL = _env_int("STEMDECK_TIMEOUT_IDENTIFY_BACKFILL", 60) +# A lookup answer bigger than this is not one worth parsing. +IDENTIFY_MAX_BYTES = 4 * 1024 * 1024 +# MusicBrainz answers kept on disk, one file per recording or artist MBID, so +# re-identifying a track (a re-split, a backfill) asks nothing twice. +MUSICBRAINZ_CACHE_DIR = CACHE_DIR / "musicbrainz" +MUSICBRAINZ_CACHE_TTL_SEC = 30 * 24 * 3600 +# Discogs band profiles (app/pipeline/discogs.py), for bands Wikipedia has no +# article on, asked only with the user's own token (settings.discogs_token). +# Discogs allows 60 requests a minute to a client with a token, so one a +# second across the process. Each request is abandoned after +# TIMEOUT_DISCOGS_REQUEST seconds; a 429 (or a 502/503) is asked again up to +# DISCOGS_RETRIES times, waiting what Retry-After says up to +# DISCOGS_RETRY_MAX_WAIT_SEC, never past DISCOGS_REQUEST_BUDGET_SEC for the +# one request. GET /api/jobs/{id}/artist-extra gives the whole lookup (a +# search, a few releases to check, the artist and its releases: about seven +# requests) DISCOGS_LOOKUP_BUDGET_SEC before it answers without. +DISCOGS_API = "https://api.discogs.com" +DISCOGS_MIN_INTERVAL_SEC = 1.0 +TIMEOUT_DISCOGS_REQUEST = _env_int("STEMDECK_TIMEOUT_DISCOGS_REQUEST", 8) +DISCOGS_RETRIES = 2 +DISCOGS_RETRY_BACKOFF_SEC = 2.0 +DISCOGS_RETRY_MAX_WAIT_SEC = 10.0 +DISCOGS_REQUEST_BUDGET_SEC = 20.0 +DISCOGS_LOOKUP_BUDGET_SEC = _env_int("STEMDECK_DISCOGS_LOOKUP_BUDGET_SEC", 45) +# A name typed in the artist box, searched on Discogs (GET /api/discogs/artist): +# the search and the profiles of its first few artists, one a second, so the +# user can tell "Nihil (2)" from "Nihil (5)". What is not in by then is left +# out rather than waited for. +DISCOGS_SEARCH_BUDGET_SEC = _env_int("STEMDECK_DISCOGS_SEARCH_BUDGET_SEC", 15) +# An artist's release list at 100 a page is well under a megabyte. +DISCOGS_MAX_BYTES = 2 * 1024 * 1024 +# Answers kept on disk, one file per request, as MusicBrainz's are. +DISCOGS_CACHE_DIR = CACHE_DIR / "discogs" +DISCOGS_CACHE_TTL_SEC = 30 * 24 * 3600 +# The work behind a soundtrack or cast recording (app/pipeline/work_lookup.py): +# the musical, film or series a job's song is from, found on Wikidata beside +# the band. Past this many seconds the tag backfill answers without it. +TIMEOUT_WORK_BACKFILL = _env_int("STEMDECK_TIMEOUT_WORK_BACKFILL", 30) +# How far up Wikidata's "subclass of" a work's class is followed to find a +# kind of work it is. Three reaches "animated television series" from +# "anime television series"; further only finds classes too broad to mean it. +WORK_CLASS_DEPTH = 3 +# A cast recording comes out months after the show opens, and a soundtrack +# can come out a little before its film. A work first shown more than this +# many years after the recording came out cannot be what it is from. +WORK_YEAR_SLACK = 1 + +# A job's lyrics, found on LRCLIB (lrclib.net) while it separates and kept +# beside its stems as LYRICS_FILE (app/pipeline/lyrics_lookup.py). LRCLIB asks +# clients to name themselves and link their homepage. +LRCLIB_API = "https://lrclib.net/api" +LRCLIB_USER_AGENT = ( + "StemDeck (https://github.com/stemdeckapp/stemdeck; self-hosted stem separation)" +) +LYRICS_FILE = "lyrics.json" +# When LRCLIB was asked and no version was kept: the versions it had, for the +# tab to offer, and when it was asked. The tag backfill does not ask again for +# LYRICS_NOT_FOUND_RETRY_SEC; lyrics.json from any source settles it for good. +LYRICS_CANDIDATES_FILE = "lyrics_candidates.json" +LYRICS_NOT_FOUND_RETRY_SEC = _env_int("STEMDECK_LYRICS_NOT_FOUND_RETRY_DAYS", 30) * 24 * 3600 +# Each request is abandoned after this many seconds; /api/get can be slow when +# LRCLIB has to look further afield for an exact match. +TIMEOUT_LYRICS_LOOKUP = _env_int("STEMDECK_TIMEOUT_LYRICS_LOOKUP", 10) +# No further request is started this many seconds into a lookup. The cascade +# is at most five requests, and all of them run beside separation. +LYRICS_LOOKUP_BUDGET_SEC = _env_int("STEMDECK_LYRICS_LOOKUP_BUDGET_SEC", 30) +# LRCLIB answers 503 or 429 when it is busy, for a moment. Each request is +# asked again this many times, waiting LYRICS_LOOKUP_RETRY_SEC and then twice +# that, within the lookup's budget, so one busy moment does not cost a song +# its lyrics. +LYRICS_LOOKUP_RETRIES = 2 +LYRICS_LOOKUP_RETRY_SEC = 1.0 +# How long a finished pipeline waits for a lookup still in flight. Short for +# the same reason as the band's: the tab looks for lyrics itself when a track +# has none. +LYRICS_LOOKUP_GRACE_SEC = _env_int("STEMDECK_LYRICS_LOOKUP_GRACE_SEC", 2) +# How long the lookup waits for the identification running beside it (a +# fingerprint, then MusicBrainz at one request a second) before it goes by the +# tags instead. Both run beside separation, so the wait costs an import nothing. +LYRICS_IDENTITY_WAIT_SEC = _env_int("STEMDECK_LYRICS_IDENTITY_WAIT_SEC", 60) +# A search answers at most 20 versions of a few kilobytes each. +LYRICS_LOOKUP_MAX_BYTES = 8 * 1024 * 1024 +# LRCLIB lengths come from real releases, so the same recording lands within a +# second or two; a live cut or a radio edit does not. The page's +# SAME_RECORDING_SEC and SAME_LENGTH_SEC (static/js/lyrics*.js). +LYRICS_SAME_RECORDING_SEC = 3 +# Other versions kept beside the one chosen, for the tab to offer. +LYRICS_OTHERS_MAX = 11 +# Moving the timing of lyrics from a version of another length onto the track +# (app/pipeline/lyrics_align.py): the shifts tried, how near a line's start the +# voice's rise is looked for, and how clear the best shift must be: this many +# standard deviations above all shifts, this many times the best shift a +# second or more away, and this share of the lines on a rise of the voice. +# Measured on six library tracks' vocals stems against their LRCLIB versions: +# the same recording scored z 5.1 to 7.3, 1.24 to 1.48 and 66 to 94% of lines +# (two songs of even, repeating lines scored 1.06 and stay unverified); a +# different performance of the song z 2.7, 1.03, 55%; scattered synthetic +# singing at most z 4.9, 1.33, 35%. +LYRICS_ALIGN_MAX_OFFSET_SEC = 60 +LYRICS_ALIGN_TOLERANCE_SEC = 0.2 +LYRICS_ALIGN_MIN_LINES = 6 +LYRICS_ALIGN_MIN_Z = 4.5 +LYRICS_ALIGN_MIN_RATIO = 1.15 +LYRICS_ALIGN_MIN_HITS = 0.6 +# LRCLIB's version the track's own length ("timing": "exact") is checked the +# same way, and moved only when the voice clearly starts at least this much +# later or earlier: a copy of the right length can still be timed to another +# cut (Green Day "Basket Case": the video's lines start 16 s after the copy's). +LYRICS_ALIGN_EXACT_MIN_SHIFT_SEC = 1.0 +# How long deleting a job waits for its lyrics timing to stop and let go of +# the vocals stem (the worker is terminated, then killed after 5 s). +RETIME_DELETE_WAIT_SEC = 15 +# How far the Lyrics tab's Align panel may move a track's lyrics, either way +# ("offset_sec" in lyrics.json, POST /api/jobs/{id}/lyrics/offset): a long +# intro or a medley's later verse, and no further. +LYRICS_OFFSET_MAX_SEC = 600 +# Lyrics timed line by line to the singing (app/pipeline/lyrics_retime.py). +# The words Whisper heard in the vocals are kept beside lyrics.json in this +# file, so timing them again costs no second pass. +LYRICS_TRANSCRIPT_FILE = "lyrics_transcript.json" +# The result is kept only when at least this share of the lyrics' syllables +# was matched to heard words, and this share of the lines anchored where they +# are sung. Measured on 58 songs in ten languages: every one timed well +# matched 0.48 or more and anchored 0.52 or more of its lines; the timings +# that must not be kept (a romanised Japanese copy, a remix's lyrics, German +# transcribed as English, a transcription led astray by a prompt of the +# lyrics) matched 0.00 to 0.26 and anchored 0.00 to 0.32. +LYRICS_RETIME_MIN_MATCHED = 0.35 +LYRICS_RETIME_MIN_LINES_SHARE = 0.45 +# The alignment's size cap, lyric tokens times heard tokens: its back +# pointers take two bytes a cell. A long Chinese song is about 1500 x 1500. +LYRICS_RETIME_MAX_CELLS = 12_000_000 +# A manual timing (PUT .../lyrics/user-synced) has at most this many lines, +# and it or a line-by-line timing at most this many characters: word stamps +# add ten a word to lyrics of up to 100000. +LYRICS_USER_SYNCED_MAX_LINES = 2000 +LYRICS_TIMED_MAX_CHARS = 400_000 +# Lyrics that lost their letters outside ASCII, mended from a reference that +# has them (app/pipeline/lyrics_repair.py). A copy with no intact twin on +# LRCLIB is checked against the audio only when it has at least this many +# words, none with such a letter, and fewer than this share of them common +# English words. Measured on LRCLIB copies: four English songs (171 to 676 +# words) scored 0.23 to 0.27 on that list, eleven Polish copies (Kukulska, +# Kayah, Bisz, stripped or not) 0. +LYRICS_STRIPPED_CHECK_MIN_WORDS = 40 +LYRICS_STRIPPED_ENGLISH_MAX_SHARE = 0.1 +# Then Whisper decides the language from the vocals, and only a language +# written with such letters, detected with at least this probability, goes +# on to a transcription. Languages written in another script are left out: +# a copy of theirs in Latin letters is a transliteration, which a +# transcription cannot mend. +LYRICS_REPAIR_LANGUAGES = frozenset( + [ + "pl", + "cs", + "sk", + "sl", + "hr", + "bs", + "hu", + "ro", + "tr", + "de", + "fr", + "pt", + "es", + "ca", + "gl", + "it", + "vi", + "lt", + "lv", + "et", + "fi", + "is", + "fo", + "da", + "no", + "nn", + "sv", + "sq", + "mt", + "lb", + ] +) +LYRICS_REPAIR_MIN_LANGUAGE_PROB = 0.8 +# And the transcription must bear it out: at least this many words given +# their letters back, this share of the words paired with what Whisper heard, +# and this share of those pairs given letters. A song written without accents +# in any of those languages pairs the same way and gives back next to none. +LYRICS_REPAIR_MIN_RESTORED = 5 +LYRICS_REPAIR_MIN_ALIGNED = 0.4 +LYRICS_REPAIR_MIN_RESTORED_SHARE = 0.1 +# A word no sure pair covered (a chorus the transcription skipped, a word +# beside a misheard one) takes the spelling the song's sure pairs gave it +# elsewhere, when they gave it only that one, at least this many times, and +# never kept it as it is. On "W biegu" 1 mends 49 words against 46 for 2, the +# three being repeats of "biegne", "cigle" and "pno" whose other occurrence +# was heard. +LYRICS_REPAIR_VOCABULARY_MIN = 1 +# The in-order pairing is quadratic: words times reference words beyond this +# are not paired at all (a song is a few hundred of each). +LYRICS_REPAIR_ALIGN_MAX_CELLS = 4_000_000 + +# Lyrics transcribed from the vocals stem with Whisper when no lookup found +# any (app/pipeline/transcribe.py). Whether it runs is the transcribe_lyrics +# setting. Weights download on first use into /whisper, beside the +# Demucs checkpoints, and are verified by Whisper's own SHA-256 check. +# +# large-v3-turbo on a GPU: large-v3's encoder with a 4-layer decoder, 1.6 GB. +# small on a CPU, or on a GPU without room for turbo: 0.5 GB. +TRANSCRIBE_MODEL_GPU = os.environ.get("STEMDECK_WHISPER_MODEL_GPU", "").strip() or "turbo" +TRANSCRIBE_MODEL_CPU = os.environ.get("STEMDECK_WHISPER_MODEL_CPU", "").strip() or "small" +# Free VRAM the GPU model needs, checked in the worker before loading, since +# the Demucs worker keeps its own allocation resident on the same card. +# Measured on an RTX 3080 (Nirvana "Lithium", 255 s): turbo held in fp16 +# peaked at 2172 MB reserved by torch, 2463 MB on the card with the CUDA context. +TRANSCRIBE_GPU_MIN_FREE_MB = _env_int("STEMDECK_WHISPER_GPU_MIN_FREE_MB", 3072) +# Below this, even the small model is not tried on the GPU and runs on the CPU. +TRANSCRIBE_SMALL_GPU_MIN_FREE_MB = _env_int("STEMDECK_WHISPER_SMALL_GPU_MIN_FREE_MB", 1536) +# The whole stage, including a first-run model download on a slow connection. +TIMEOUT_TRANSCRIBE = max(60, _env_int("STEMDECK_TIMEOUT_TRANSCRIBE", 30 * 60)) +# No output for this long is a hang. Whisper reports progress once per 30 s +# window of audio, which the small model on a CPU finishes well inside it. +TIMEOUT_TRANSCRIBE_STALL = max(30, _env_int("STEMDECK_TIMEOUT_TRANSCRIBE_STALL", 300)) +# Vocals quieter than this, as stem_presence (0-100, relative to the loudest +# stem), are not worth transcribing: Whisper invents words over silence. A stem +# that is absent from a track measured 0-2 across six library tracks. +TRANSCRIBE_MIN_VOCAL_PRESENCE = 3 +# Skip silent stretches longer than this around a suspected hallucination +# (Whisper's hallucination_silence_threshold). An isolated vocal has long +# instrumental gaps, which is where Whisper makes up text. +TRANSCRIBE_HALLUCINATION_SILENCE_SEC = 2.0 +# Lines are Whisper's segments, broken further at a pause at least this long +# between two words, and at a comma or a pause once a line is this many +# characters long. Nothing is ever longer than the hard cap. +TRANSCRIBE_LINE_BREAK_GAP_SEC = 0.8 +TRANSCRIBE_LINE_SOFT_CHARS = 40 +TRANSCRIBE_LINE_HARD_CHARS = 64 +# A gap between two words at least this long is written as an end stamp, so +# the karaoke wipe waits through it instead of stretching the first word. +TRANSCRIBE_WORD_GAP_SEC = 0.3 +# A silence at least this long between two lines starts a new stanza (a blank +# line) in the plain text. +TRANSCRIBE_STANZA_GAP_SEC = 4.0 + # Beat-grid analysis parameters. 22050 Hz is plenty for onset detection (the # percussive energy that matters lives well under 11 kHz) and keeps the decode # cheap; hop 512 gives ~23 ms grid resolution, the librosa default pairing. diff --git a/app/core/models.py b/app/core/models.py index fc7ccef6..77e4abc9 100644 --- a/app/core/models.py +++ b/app/core/models.py @@ -1,10 +1,14 @@ from __future__ import annotations import dataclasses +import math +import re import time from dataclasses import dataclass, field from typing import Any, Literal +from app.core.config import AUDIO_TAG_LYRICS_MAX_CHARS, AUDIO_TAG_MAX_CHARS + class JobCancelled(Exception): """Raised inside a pipeline stage when the job's cancel flag is set.""" @@ -14,6 +18,167 @@ class JobCancelled(Exception): "queued", "downloading", "analyzing", "separating", "processing", "done", "error", "cancelled" ] +# A Wikidata item id. Anything else is not a band this app found. +_WIKIDATA_ID_RE = re.compile(r"^Q\d{1,12}$") +# A band's name, as kept on the job. Longer than this is not a name. +_ARTIST_NAME_MAX_CHARS = 300 + + +def clean_artist(value: Any) -> dict[str, str] | None: + """A band as kept on a job, {"id", "name", "englishName"}, or None. + + For anything read back from disk (the registry, metadata.json), where a + hand-edited or damaged file must not put a malformed band in front of the + page: the id has to be a Wikidata item, the names plain strings. + """ + if not isinstance(value, dict): + return None + band_id = value.get("id") + if not isinstance(band_id, str) or not _WIKIDATA_ID_RE.match(band_id): + return None + names = {} + for key in ("name", "englishName"): + name = value.get(key) + names[key] = name.strip()[:_ARTIST_NAME_MAX_CHARS] if isinstance(name, str) else "" + if not names["name"] and not names["englishName"]: + return None + return {"id": band_id, **names} + + +# The tags a file or a video names (audio_tags.py), and how long each may be: +# the lyrics a tag can carry are longer than any name. +_AUDIO_TAG_LIMITS = { + "artist": AUDIO_TAG_MAX_CHARS, + "title": AUDIO_TAG_MAX_CHARS, + "album": AUDIO_TAG_MAX_CHARS, + "lyrics": AUDIO_TAG_LYRICS_MAX_CHARS, +} + + +def clean_audio_tags(value: Any) -> dict[str, str] | None: + """A job's tags as read back from disk, or None: known keys only, each a + non-empty string within its limit. A damaged file must not reach the + lyrics lookup, which reads them as a dict of strings.""" + if not isinstance(value, dict): + return None + tags = { + key: text[:limit] + for key, limit in _AUDIO_TAG_LIMITS.items() + if isinstance(text := value.get(key), str) and text.strip() + } + return tags or None + + +# A MusicBrainz id (recording, artist, release group): a lower-case UUID. +MBID_RE = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$") +# "lrclib": named by LRCLIB's artist and album for a title MusicBrainz could +# not place (title_parse.py, lyrics_lookup.identify_on_lrclib). +IDENTITY_SOURCES = ("acoustid", "musicbrainz", "lrclib", "tags") +# More artists than this on one recording is not a credit worth keeping. +_IDENTITY_MAX_ARTISTS = 20 +_IDENTITY_MAX_TYPES = 12 +_IDENTITY_MAX_TITLE_ALIASES = 6 +# Before sound recording, a year is a typo or a placeholder. +_IDENTITY_MIN_YEAR = 1860 + + +def _identity_text(value: Any) -> str: + return value.strip()[:_ARTIST_NAME_MAX_CHARS] if isinstance(value, str) else "" + + +def _identity_mbid(value: Any) -> str | None: + return value if isinstance(value, str) and MBID_RE.match(value) else None + + +def clean_identity(value: Any) -> dict[str, Any] | None: + """What a job was identified as, in the one shape the design fixes, or None. + + {"source", "score", "recording_mbid", "title", "artist", "artist_mbids", + "album", "release_group_mbid", "release_group_type", "secondary_types", + "year", "duration"}, and "title_aliases" when there are any, with "year" + the album's first release. Used for everything written (the pipeline + builds identities through it) and everything read back from disk, so a damaged or + hand-edited record never reaches the page, the band lookup or the lyrics + lookup in any other shape. A title and an artist are required: an identity + without both names nothing. + """ + if not isinstance(value, dict) or value.get("source") not in IDENTITY_SOURCES: + return None + title = _identity_text(value.get("title")) + artist = _identity_text(value.get("artist")) + if not title or not artist: + return None + score = value.get("score") + if isinstance(score, bool) or not isinstance(score, (int, float)) or not math.isfinite(score): + score = 0.0 + duration = value.get("duration") + if ( + isinstance(duration, bool) + or not isinstance(duration, (int, float)) + or not 0 < duration < 86400 + ): + duration = None + year = value.get("year") + if ( + isinstance(year, bool) + or not isinstance(year, int) + or not _IDENTITY_MIN_YEAR <= year <= 9999 + ): + year = None + mbids = value.get("artist_mbids") + types = value.get("secondary_types") + aliases: list[str] = [] + for alias in ( + value.get("title_aliases") or [] if isinstance(value.get("title_aliases"), list) else [] + ): + text = _identity_text(alias) + if text and text != title and text not in aliases: + aliases.append(text) + aliases = aliases[:_IDENTITY_MAX_TITLE_ALIASES] + return { + "source": value["source"], + "score": round(max(0.0, min(1.0, float(score))), 4), + "recording_mbid": _identity_mbid(value.get("recording_mbid")), + "title": title, + "artist": artist, + "artist_mbids": [m for m in mbids if _identity_mbid(m)][:_IDENTITY_MAX_ARTISTS] + if isinstance(mbids, list) + else [], + "album": _identity_text(value.get("album")) or None, + "release_group_mbid": _identity_mbid(value.get("release_group_mbid")), + "release_group_type": _identity_text(value.get("release_group_type")) or None, + "secondary_types": [t for t in (_identity_text(t) for t in types) if t][ + :_IDENTITY_MAX_TYPES + ] + if isinstance(types, list) + else [], + "year": year, + "duration": round(float(duration), 3) if duration is not None else None, + # Other titles the song goes by, when any are known: the upload's own + # ("Good Day (좋은 날)") and its MusicBrainz work's. For the lyrics + # lookup, since LRCLIB files a song under any of them. Left out when + # there are none, so an identity without them keeps its old shape. + **({"title_aliases": aliases} if aliases else {}), + } + + +# What a work (app/pipeline/work_lookup.py) can be. +WORK_KINDS = ("musical", "film", "tv", "other") + + +def clean_work(value: Any) -> dict[str, str] | None: + """The work a job's song is from, {"id", "kind", "name", "englishName"}, + or None. clean_artist's rules, and a kind from WORK_KINDS.""" + work = clean_artist(value) + if work is None or not isinstance(value, dict) or value.get("kind") not in WORK_KINDS: + return None + return { + "id": work["id"], + "kind": value["kind"], + "name": work["name"], + "englishName": work["englishName"], + } + def _set(job: Job, **fields: object) -> None: """Mutate Job fields, then bump job.version so the SSE stream (#289) can @@ -72,6 +237,31 @@ class Job: # for a link, and for an upload older than this field until its state is # first served (see _job_state). source_format: str | None = None + # What the source said about itself (#699): {"artist", "title", "album", + # "lyrics"}, each only when present. An upload's container tags, read + # before the upload is deleted; a link's music metadata from yt-dlp. The + # Lyrics tab and the artist box fill themselves from it. None when the + # source had none, and for anything imported before it was read. + audio_tags: dict[str, str] | None = None + # Which recording this is, found while the job ran (app/pipeline/identify.py): + # by audio fingerprint on AcoustID when the user set a key, else a + # confident MusicBrainz search by the tags, else the tags alone. The shape + # is clean_identity's. None when nothing names the track. + identity: dict[str, Any] | None = None + # The band audio_tags' artist names, found on Wikidata while the job ran + # (#699): {"id": "Q...", "name", "englishName"}. Only ever an exact match + # for the tag, so a wrong band is never saved with nobody looking. None + # when there was no artist tag, no such band, or no connection; the page + # then looks for it itself, and a band saved there always wins over this. + artist: dict[str, str] | None = None + # The musical, film or series the song is from, when the recording is a + # soundtrack or a cast recording (app/pipeline/work_lookup.py): clean_work's + # {"id": "Q...", "kind", "name", "englishName"}. None for anything else. + work: dict[str, str] | None = None + # Whether the job has lyrics.json beside its stems (lyrics_lookup.py). The + # lyrics themselves stay in that file: they run to kilobytes, and this + # record is rewritten whole on every save. + has_lyrics: bool = False # True when a silent video track (video.mp4) was preserved from an .mp4 # upload, enabling the "Export Mix (with video)" MP4 export. has_video: bool = False @@ -162,6 +352,11 @@ def to_state(self) -> dict[str, Any]: "mix_url": self.mix_url, "source_url": self.source_url, "source_format": self.source_format, + "audio_tags": self.audio_tags, + "identity": self.identity, + "artist": self.artist, + "work": self.work, + "has_lyrics": self.has_lyrics, "has_video": self.has_video, "video_status": self.video_status, "error": self.error, @@ -201,6 +396,10 @@ def from_record(cls, data: dict[str, Any]) -> Job: job = cls(id=job_id) for key, value in fields.items(): setattr(job, key, value) + job.artist = clean_artist(job.artist) + job.identity = clean_identity(job.identity) + job.work = clean_work(job.work) + job.audio_tags = clean_audio_tags(job.audio_tags) job.cancel_requested = False return job diff --git a/app/core/registry.py b/app/core/registry.py index 9cc07c87..1a732c3e 100644 --- a/app/core/registry.py +++ b/app/core/registry.py @@ -9,8 +9,8 @@ import uuid from pathlib import Path -from app.core.config import DEMUCS_MODEL, EXTRA_STEM_NAMES, JOB_ID_RE, STEM_NAMES -from app.core.models import Job +from app.core.config import DEMUCS_MODEL, EXTRA_STEM_NAMES, JOB_ID_RE, LYRICS_FILE, STEM_NAMES +from app.core.models import Job, clean_artist, clean_audio_tags, clean_identity, clean_work logger = logging.getLogger("stemdeck.registry") @@ -368,6 +368,12 @@ def _recover_done_job(job_dir: Path) -> Job | None: sections=meta.get("sections"), sections_source=meta.get("sections_source"), tags=meta.get("tags"), + audio_tags=clean_audio_tags(meta.get("audio_tags")), + identity=clean_identity(meta.get("identity")), + artist=clean_artist(meta.get("artist")), + work=clean_work(meta.get("work")), + # The file is the record: metadata.json predates it for most jobs. + has_lyrics=(job_dir / LYRICS_FILE).is_file(), vocal_split=vocal_split, ) diff --git a/app/core/settings.py b/app/core/settings.py index da093aa2..c30aa8f7 100644 --- a/app/core/settings.py +++ b/app/core/settings.py @@ -14,6 +14,9 @@ - `cookies_file` — optional cookies.txt handed to yt-dlp for YouTube. - `auto_delete_jobs` — whether finished jobs are deleted after a while (off). - `auto_delete_days` — how long they are kept when that is on. +- `acoustid_api_key` - the user's AcoustID key, for fingerprint identification. +- `discogs_token` - the user's Discogs token, for band profiles. +- `transcribe_lyrics` - Whisper lyrics when none are found: auto | on | off. Defaults fall back to the config.py constants (which honor their env vars), so nothing changes until the user overrides a value. @@ -24,6 +27,7 @@ import json import logging import os +import re import threading import time import uuid @@ -591,3 +595,155 @@ def set_separation_quality(value: str) -> str: _ensure()["separation_quality"] = choice _save() return choice + + +# ── transcribe_lyrics ── +# Whether a job with no lyrics found gets them transcribed from its vocals stem +# with Whisper (app/pipeline/transcribe.py). "auto" (default) does it when the +# job was separated on an NVIDIA GPU (cuda) only. Anywhere else, Apple's MPS +# included, Whisper runs on the CPU, about 200 s a song rather than 30 s, so +# "auto" leaves it off; "on" and "off" force it. Read per job, so +# a change applies to the next import. STEMDECK_TRANSCRIBE_LYRICS seeds the +# default for env-based deployments. +_TRANSCRIBE_CHOICES = ("auto", "on", "off") + + +def _default_transcribe_lyrics() -> str: + env = os.environ.get("STEMDECK_TRANSCRIBE_LYRICS", "").strip().lower() + return env if env in _TRANSCRIBE_CHOICES else "auto" + + +def get_transcribe_lyrics() -> str: + with _LOCK: + v = _ensure().get("transcribe_lyrics") + return ( + v if isinstance(v, str) and v in _TRANSCRIBE_CHOICES else _default_transcribe_lyrics() + ) + + +def set_transcribe_lyrics(value: str) -> str: + choice = (value or "").strip().lower() + if choice not in _TRANSCRIBE_CHOICES: + raise ValueError("transcribe_lyrics must be one of: " + ", ".join(_TRANSCRIBE_CHOICES)) + with _LOCK: + _ensure()["transcribe_lyrics"] = choice + _save() + return choice + + +def transcribe_lyrics_enabled(device: str | None) -> bool: + """Whether a job separated on ``device`` gets its lyrics transcribed. + + ``device`` is the job's compute_device, so a GPU job that fell back to the + CPU ("cpu (fallback from cuda)") counts as a CPU one under "auto": the GPU + has just failed, and a CPU pass is the slow case "auto" exists to avoid. + """ + choice = get_transcribe_lyrics() + if choice == "auto": + return device == "cuda" + return choice == "on" + + +def lyrics_mending_enabled() -> bool: + """Whether found lyrics that lost their accents may be mended from the + vocals (transcribe.mend_lyrics), on any device unless the setting is "off". + + Unlike a transcription, "auto" does not keep this to CUDA: its gates hold + it to the rare song whose lyrics are missing letters, which otherwise + shows broken words on every CPU and Mac install.""" + return get_transcribe_lyrics() != "off" + + +# ── acoustid_api_key ── +# The user's own AcoustID application key, for identifying a track by its +# audio fingerprint (app/pipeline/identify.py). Entered in Settings and never +# shipped: AcoustID keys are per application and free to register. Without +# one, nothing is fingerprinted and identification falls back to searching +# MusicBrainz by the tags. +# +# A secret in the sense that it is the user's: never logged, and never handed +# back by the API. /api/settings publishes only whether one is set and its +# last two characters, so the field can show which key it holds. Real keys +# are about ten characters, so two is a hint and not a useful part of one. +_ACOUSTID_KEY_RE = re.compile(r"^[A-Za-z0-9_-]{8,64}$") +ACOUSTID_KEY_HINT_CHARS = 2 + + +def acoustid_key_format_ok(key: str) -> bool: + """Whether ``key`` has the shape of an AcoustID key at all.""" + return bool(_ACOUSTID_KEY_RE.match(key)) + + +def get_acoustid_api_key() -> str | None: + with _LOCK: + value = _ensure().get("acoustid_api_key") + return value if isinstance(value, str) and _ACOUSTID_KEY_RE.match(value) else None + + +def set_acoustid_api_key(value: str | None) -> str | None: + """Persist the key, or clear it when given empty/None. Raises ValueError, + without the value in the message, when it cannot be an AcoustID key.""" + with _LOCK: + if value is None or not str(value).strip(): + _ensure().pop("acoustid_api_key", None) + _save() + return None + key = str(value).strip() + if not _ACOUSTID_KEY_RE.match(key): + raise ValueError("acoustid_api_key must be 8 to 64 letters, digits, - or _") + _ensure()["acoustid_api_key"] = key + _save() + return key + + +def acoustid_api_key_hint() -> str | None: + """The last two characters of the key, for the Settings field, or None.""" + key = get_acoustid_api_key() + return key[-ACOUSTID_KEY_HINT_CHARS:] if key else None + + +# ── discogs_token ── +# The user's own Discogs personal access token, for band profiles, members and +# releases of bands Wikipedia has no article on. Entered in Settings and never +# shipped: Discogs tokens are per user and free. Without one, Discogs is never +# asked. +# +# A secret as the AcoustID key is: never logged, never put in a URL, and never +# handed back by the API, which publishes only whether one is set and its last +# two characters. Discogs tokens are 40 letters and digits; the range is kept +# loose so a change in their length does not lock anyone out. +_DISCOGS_TOKEN_RE = re.compile(r"^[A-Za-z0-9]{20,80}$") +DISCOGS_TOKEN_HINT_CHARS = 2 + + +def discogs_token_format_ok(token: str) -> bool: + """Whether ``token`` has the shape of a Discogs personal access token.""" + return bool(_DISCOGS_TOKEN_RE.match(token)) + + +def get_discogs_token() -> str | None: + with _LOCK: + value = _ensure().get("discogs_token") + return value if isinstance(value, str) and _DISCOGS_TOKEN_RE.match(value) else None + + +def set_discogs_token(value: str | None) -> str | None: + """Persist the token, or clear it when given empty/None. Raises ValueError, + without the value in the message, when it cannot be a Discogs token.""" + with _LOCK: + if value is None or not str(value).strip(): + _ensure().pop("discogs_token", None) + _save() + return None + token = str(value).strip() + if not _DISCOGS_TOKEN_RE.match(token): + raise ValueError("discogs_token must be 20 to 80 letters or digits") + _ensure()["discogs_token"] = token + _save() + return token + + +def discogs_token_hint() -> str | None: + """The last two characters of the token, for the Settings field, or None.""" + token = get_discogs_token() + return token[-DISCOGS_TOKEN_HINT_CHARS:] if token else None diff --git a/app/main.py b/app/main.py index 942f7447..eb050740 100644 --- a/app/main.py +++ b/app/main.py @@ -26,7 +26,9 @@ from app.core import tls_listener from app.core.compression import COMPRESS_LEVEL, MINIMUM_SIZE, TextGZipMiddleware from app.core.config import ( + ACOUSTID_CHECK_KEY, DEMUCS_MODEL, + DISCOGS_CHECK_TOKEN, FFMPEG_BIN, HTTPS_PORT, JOBS_DIR, @@ -50,6 +52,10 @@ AUTO_DELETE_DAYS_MIN, DURATION_MAX_SEC, DURATION_MIN_SEC, + acoustid_api_key_hint, + acoustid_key_format_ok, + discogs_token_format_ok, + discogs_token_hint, get_allow_network, get_auto_delete_days, get_auto_delete_jobs, @@ -63,19 +69,23 @@ get_playlist_max_items, get_port, get_separation_quality, + get_transcribe_lyrics, get_video_max_height, + set_acoustid_api_key, set_allow_network, set_auto_delete_days, set_auto_delete_jobs, set_auto_sections, set_cookies_file, set_demucs_device, + set_discogs_token, set_export_sample_rate, set_jobs_dir, set_max_duration_sec, set_playlist_max_items, set_port, set_separation_quality, + set_transcribe_lyrics, set_video_max_height, ) from app.core.stems_location import ( @@ -362,9 +372,20 @@ def _settings_payload() -> dict[str, object]: "video_max_height": get_video_max_height(), "export_sample_rate": get_export_sample_rate(), "separation_quality": get_separation_quality(), + # Whisper lyrics for a job no lookup found any for: "auto" (on a GPU + # only) | "on" | "off". + "transcribe_lyrics": get_transcribe_lyrics(), # Absent unless the user set one. Only the path is exposed, never the # file's contents -- those are the user's YouTube session. "cookies_file": get_cookies_file(), + # Never the key itself: whether one is set, and its last four + # characters so the field can show which one. + "acoustid_api_key_set": acoustid_api_key_hint() is not None, + "acoustid_api_key_tail": acoustid_api_key_hint(), + # The same for the Discogs token: whether one is set, and its last two + # characters, never the token. + "discogs_token_set": discogs_token_hint() is not None, + "discogs_token_tail": discogs_token_hint(), "port": get_port(), # The user's choice ("auto" | "cuda" | "mps" | "cpu") drives the UI # select; the resolved value shows what jobs will actually run on; @@ -403,6 +424,13 @@ def get_settings(request: Request) -> dict[str, object]: return {**_settings_payload(), "port": port, "lan_addresses": addresses} +# The detail for a key AcoustID refuses, which the Settings field tells apart +# from a key of the wrong shape to say which key it wants. +ACOUSTID_KEY_REFUSED = "AcoustID does not accept this key" +# The same for a Discogs token Discogs answers 401. +DISCOGS_TOKEN_REFUSED = "Discogs does not accept this token" + + @app.post("/api/settings", tags=["settings"]) async def update_settings(request: Request) -> dict[str, object]: """Update runtime settings. Reachable from the host machine always; from a @@ -444,6 +472,41 @@ async def update_settings(request: Request) -> dict[str, object]: raise HTTPException(status_code=422, detail=str(e)) from None except (TypeError, OSError): raise HTTPException(status_code=422, detail="invalid cookies file") from None + if "acoustid_api_key" in body: + value = body["acoustid_api_key"] + if value is not None and not isinstance(value, str): + raise HTTPException(status_code=422, detail="invalid AcoustID key") from None + key = (value or "").strip() + if key and ACOUSTID_CHECK_KEY and acoustid_key_format_ok(key): + # Tried once before it is kept: a key AcoustID refuses would + # otherwise fail every import without a word. + from app.pipeline.identify import acoustid_key_works + + if await asyncio.to_thread(acoustid_key_works, key) is False: + raise HTTPException(status_code=422, detail=ACOUSTID_KEY_REFUSED) from None + try: + set_acoustid_api_key(value) + except ValueError: + # A fixed message: the rejected value is the user's key, and + # nothing here echoes it back or logs it. + raise HTTPException(status_code=422, detail="invalid AcoustID key") from None + if "discogs_token" in body: + value = body["discogs_token"] + if value is not None and not isinstance(value, str): + raise HTTPException(status_code=422, detail="invalid Discogs token") from None + token = (value or "").strip() + if token and DISCOGS_CHECK_TOKEN and discogs_token_format_ok(token): + # Tried once before it is kept, as the AcoustID key is. Bounded: + # the Save button waits on it. + from app.pipeline.discogs_auth import discogs_token_works + + if await asyncio.to_thread(discogs_token_works, token) is False: + raise HTTPException(status_code=422, detail=DISCOGS_TOKEN_REFUSED) from None + try: + set_discogs_token(value) + except ValueError: + # A fixed message: the rejected value is the user's token. + raise HTTPException(status_code=422, detail="invalid Discogs token") from None if "demucs_device" in body: try: set_demucs_device(str(body["demucs_device"])) @@ -456,6 +519,11 @@ async def update_settings(request: Request) -> dict[str, object]: set_separation_quality(str(body["separation_quality"])) except ValueError as e: raise HTTPException(status_code=422, detail=str(e)) from None + if "transcribe_lyrics" in body: + try: + set_transcribe_lyrics(str(body["transcribe_lyrics"])) + except ValueError as e: + raise HTTPException(status_code=422, detail=str(e)) from None return _settings_payload() @@ -849,7 +917,11 @@ def download_logs_zip() -> StreamingResponse: # script-src has no 'unsafe-inline'/'eval': all JS is same-origin modules and the # inline scripts/onclick were moved out. 'unsafe-inline' is allowed for *styles* # only (the UI sets many style attributes). Allowances: -# connect-src -> same-origin API/SSE, the GitHub update check, Tauri IPC +# connect-src -> same-origin API/SSE, the GitHub update check, Tauri IPC, +# Wikidata/Wikipedia for the artist box the now-playing card +# opens, and LRCLIB for the Lyrics tab (#699). All four are +# read-only public APIs, asked only about the open track, and +# sent only an artist and a song name. # img-src https: -> remote YouTube/SoundCloud thumbnails # style/font -> the Google Fonts _CSP = ( @@ -863,7 +935,9 @@ def download_logs_zip() -> StreamingResponse: # track init); without them Multitrack.create throws and no audio loads # (#186). They are inline/same-origin schemes, not network endpoints, so # they add no exfiltration channel — script-src below stays locked. - "connect-src 'self' https://api.github.com ipc: http://ipc.localhost data: blob:; " + "connect-src 'self' https://api.github.com ipc: http://ipc.localhost data: blob: " + "https://www.wikidata.org https://query.wikidata.org https://*.wikipedia.org " + "https://lrclib.net; " "object-src 'none'; base-uri 'self'; frame-ancestors 'none'; form-action 'self'" ) diff --git a/app/pipeline/artist_lookup.py b/app/pipeline/artist_lookup.py new file mode 100644 index 00000000..5590117d --- /dev/null +++ b/app/pipeline/artist_lookup.py @@ -0,0 +1,401 @@ +"""The band a job's artist tag names, found on Wikidata while the job runs (#699). + +The page used to do this itself, once a finished track was opened, so the +artist box, the Lyrics tab and the now-playing card only learned the band a +moment after the track was open, and only while the page was. Done here, it +runs beside separation, which takes minutes, and the band is on the job the +moment the import is done. + +This is a port of the page's own lookup (static/js/artistLookup.js), kept to +the same rule: the first search hit that has a MusicBrainz artist id is the +band, and it is kept only when its name is the tag's name once case, Latin +accents, punctuation and spacing are set aside. Wikidata's search forgives +spelling, so a near miss still finds some band, and saved with nobody looking +the name has to agree. + +What is sent is the tag's artist name and nothing else about the track. + +Everything here is best-effort. No tag, no match, no connection or a slow +answer all leave the band unset, and the page then looks for it itself when +the track is opened. +""" + +from __future__ import annotations + +import json +import logging +import re +import ssl +import threading +import unicodedata +import urllib.parse +import urllib.request +from collections.abc import Callable +from typing import Any + +from app.core.config import ARTIST_LOOKUP_MAX_BYTES, ARTIST_LOOKUP_USER_AGENT, TIMEOUT_ARTIST_LOOKUP +from app.core.models import Job, _set, clean_artist + +logger = logging.getLogger("stemdeck.artist") + +WIKIDATA_API = "https://www.wikidata.org/w/api.php" + +# Having a MusicBrainz artist ID is what separates the band from the album +# named after it, a cinema that shares its name, or a disambiguation page. +MUSICBRAINZ_ARTIST_ID = "P434" +_QID_RE = re.compile(r"^Q\d+$") +_SEARCH_LIMIT = "10" + +# "Artist feat. Someone", "Artist (ft. Someone)": no band is called that. +_FEATURING_RE = re.compile(r"\s+[(\[]?(?:feat\.?|ft\.?|featuring)\s+", re.IGNORECASE) + +FetchJson = Callable[[dict[str, str]], Any] + + +def artist_name_key(name: Any) -> str: + """A name reduced to what tells it apart: no case, Latin accents, + punctuation, symbols or spaces. artistNameKey in artistLookup.js. + + Only the Latin combining marks are dropped: the marks in a Japanese or + Korean name are part of it, and folding them could make two names one. + """ + text = unicodedata.normalize("NFKD", str(name or "")) + text = "".join(ch for ch in text if not 0x300 <= ord(ch) <= 0x36F).lower() + return "".join( + ch for ch in text if not ch.isspace() and unicodedata.category(ch)[0] not in ("P", "S") + ) + + +def tagged_artist_name(tag: Any) -> str: + """The name to look up from an artist tag, less any featured artist.""" + return _FEATURING_RE.split(str(tag or ""), maxsplit=1)[0].strip() + + +def artist_matches_name(artist: dict[str, str] | None, name: str) -> bool: + """Whether a looked-up band's name, or its English name, is the tag's.""" + want = artist_name_key(name) + if not want or not artist: + return False + return any( + artist_name_key(candidate) == want + for candidate in (artist.get("name"), artist.get("englishName")) + ) + + +def pick_artist(entities: Any, ranked_ids: list[str]) -> dict[str, Any] | None: + """The first search hit, in the order the search ranked them, that is an + artist, or None. ``entities`` is wbgetentities' ``entities`` object.""" + if not isinstance(entities, dict): + return None + for qid in ranked_ids: + entity = entities.get(qid) + if not isinstance(entity, dict): + continue + claims = entity.get("claims") + if isinstance(claims, dict) and claims.get(MUSICBRAINZ_ARTIST_ID): + return entity + return None + + +def search_language(name: str) -> str: + """The Wikidata language to search a name in. + + The page searched in the language the app was in. The server has no such + thing, so the name's own script stands in for it: a name written in kana, + Hangul, Han, Cyrillic or Greek is searched among the labels in that + script's language, where Wikidata keeps it, and anything else in English. + """ + for ch in name: + if 0x3040 <= ord(ch) <= 0x30FF: + return "ja" + for ch in name: + code = ord(ch) + if 0xAC00 <= code <= 0xD7AF or 0x1100 <= code <= 0x11FF: + return "ko" + if 0x4E00 <= code <= 0x9FFF: + return "zh" + if 0x0400 <= code <= 0x04FF: + return "ru" + if 0x0370 <= code <= 0x03FF: + return "el" + return "en" + + +def _ssl_context() -> ssl.SSLContext: + # certifi where it is installed, as for the model downloads in + # separate.py: a packaged Python on macOS may have no CA store of its own. + try: + import certifi + + return ssl.create_default_context(cafile=certifi.where()) + except (ModuleNotFoundError, OSError): + return ssl.create_default_context() + + +def _fetch_json(params: dict[str, str]) -> Any: + """One Wikidata API request. Raises on any failure.""" + url = f"{WIKIDATA_API}?{urllib.parse.urlencode({**params, 'format': 'json'})}" + request = urllib.request.Request( + url, headers={"User-Agent": ARTIST_LOOKUP_USER_AGENT, "Accept": "application/json"} + ) + # A fixed https URL with only its query built here, never a URL from a + # request or a tag, which is what B310 exists to catch. + with urllib.request.urlopen( # nosec B310 + request, timeout=TIMEOUT_ARTIST_LOOKUP, context=_ssl_context() + ) as response: + body = response.read(ARTIST_LOOKUP_MAX_BYTES + 1) + if len(body) > ARTIST_LOOKUP_MAX_BYTES: + raise ValueError("Wikidata answer too large") + return json.loads(body) + + +def _label(entity: dict[str, Any], *codes: str) -> str: + """The entity's label in the first of ``codes`` it has one in, or "".""" + labels = entity.get("labels") + if not isinstance(labels, dict): + return "" + for code in codes: + label = labels.get(code) + if isinstance(label, dict) and isinstance(label.get("value"), str): + return label["value"] + return "" + + +# The languages a name's labels are read in, by the language it is searched +# in. A name in Chinese characters alone may be Chinese in either script +# (鄧麗君, 邓丽君) or Japanese (米津玄師): its label is looked for in all of +# them, since Wikidata keeps a separate one for each. +_LABEL_LANGUAGES = { + "zh": ("zh", "zh-hant", "zh-hans", "zh-tw", "zh-hk", "zh-cn", "ja"), +} + + +def _languages(lang: str) -> list[str]: + return [*_LABEL_LANGUAGES.get(lang, (lang,)), "en"] if lang != "en" else ["en"] + + +def _names_in(entity: dict[str, Any], codes: list[str]) -> list[tuple[str, bool]]: + """(name, is_label) for every label and alias the entity has in + ``codes``, labels first.""" + found: list[tuple[str, bool]] = [] + for field, is_label in (("labels", True), ("aliases", False)): + values = entity.get(field) + if not isinstance(values, dict): + continue + for code in codes: + entries = values.get(code) + for entry in entries if isinstance(entries, list) else [entries]: + if isinstance(entry, dict) and isinstance(entry.get("value"), str): + found.append((entry["value"], is_label)) + return found + + +def _display_name(entity: dict[str, Any], lang: str, name: str) -> str: + """What to call a band found for ``name``: its label in the script the + name was written in when one matches it ("邓丽君" searched finds the + simplified label, not the traditional 鄧麗君), else its label in + ``lang``'s languages, else English.""" + codes = _languages(lang) + want = artist_name_key(name) + for label, is_label in _names_in(entity, codes): + if is_label and want and artist_name_key(label) == want: + return label + return _label(entity, *codes) + + +def _goes_by(entity: dict[str, Any], lang: str, name: str) -> bool: + """Whether any label or alias of the entity in ``lang``'s languages, or + English, is ``name`` once case, accents and punctuation are set aside.""" + want = artist_name_key(name) + return bool(want) and any( + artist_name_key(value) == want for value, _ in _names_in(entity, _languages(lang)) + ) + + +def lookup_band( + name: str, + *, + fetch_json: FetchJson | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """The band called ``name`` on Wikidata, as {"id", "name", "englishName"}, + or None when nothing there is an artist by exactly that name. Raises when + Wikidata cannot be reached or answers nonsense.""" + search = name.strip() + if not search or cancelled(): + return None + # Resolved here rather than as the default, so a test can stand in for + # the network by replacing _fetch_json. + fetch_json = fetch_json or _fetch_json + lang = search_language(search) + found = fetch_json( + { + "action": "wbsearchentities", + "search": search, + "language": lang, + "uselang": lang, + "type": "item", + "limit": _SEARCH_LIMIT, + } + ) + hits = found.get("search") if isinstance(found, dict) else None + ids = [ + hit["id"] + for hit in (hits if isinstance(hits, list) else []) + if isinstance(hit, dict) and isinstance(hit.get("id"), str) and _QID_RE.match(hit["id"]) + ] + if not ids or cancelled(): + return None + answer = fetch_json( + { + "action": "wbgetentities", + "ids": "|".join(ids), + # Aliases too, outside English: a band written in another script + # is often known by a name its label is not (IU is 아이유). + "props": "claims|labels" if lang == "en" else "claims|labels|aliases", + "languages": "|".join(_languages(lang)), + "languagefallback": "1", + } + ) + entity = pick_artist(answer.get("entities") if isinstance(answer, dict) else None, ids) + if entity is None: + return None + band = clean_artist( + { + "id": entity.get("id"), + "name": _display_name(entity, lang, search) or search, + # Kept beside the name: LRCLIB lists artists by the name they + # release under, which the English label nearly always is. + "englishName": _label(entity, "en"), + } + ) + if not band: + return None + if artist_matches_name(band, search) or (lang != "en" and _goes_by(entity, lang, search)): + return band + return None + + +def _claim_values(entity: dict[str, Any], prop: str) -> set[str]: + """The plain string values of one of the entity's claims.""" + claims = entity.get("claims") + statements = claims.get(prop) if isinstance(claims, dict) else None + values = set() + for statement in statements if isinstance(statements, list) else []: + snak = statement.get("mainsnak") if isinstance(statement, dict) else None + datavalue = snak.get("datavalue") if isinstance(snak, dict) else None + value = datavalue.get("value") if isinstance(datavalue, dict) else None + if isinstance(value, str): + values.add(value.lower()) + return values + + +def lookup_band_by_id( + qid: str, + musicbrainz_id: str, + *, + name: str = "", + fetch_json: FetchJson | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """The band Wikidata item ``qid`` is, as {"id", "name", "englishName"}. + + For a band reached from a MusicBrainz artist (its url relationship to + Wikidata), not by searching a name, so there is no name to agree with: + the rule instead is that the item names that same MusicBrainz artist back + (P434). An item that does not is a stale or wrong link, and is not kept. + ``name`` only picks the language the label is read in. Raises when + Wikidata cannot be reached.""" + if not _QID_RE.match(qid or "") or not musicbrainz_id or cancelled(): + return None + fetch_json = fetch_json or _fetch_json + lang = search_language(name) if name else "en" + answer = fetch_json( + { + "action": "wbgetentities", + "ids": qid, + "props": "claims|labels", + "languages": "|".join(_languages(lang)), + "languagefallback": "1", + } + ) + entities = answer.get("entities") if isinstance(answer, dict) else None + entity = entities.get(qid) if isinstance(entities, dict) else None + if not isinstance(entity, dict): + return None + if musicbrainz_id.lower() not in _claim_values(entity, MUSICBRAINZ_ARTIST_ID): + return None + return clean_artist( + { + "id": entity.get("id"), + "name": _display_name(entity, lang, name) or name, + "englishName": _label(entity, "en"), + } + ) + + +def find_band( + artist_tag: Any, + *, + fetch_json: FetchJson | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """lookup_band for an artist tag, featured artists left off. Never raises: + any failure is logged and gives None.""" + name = tagged_artist_name(artist_tag) + if not name: + return None + try: + return lookup_band(name, fetch_json=fetch_json, cancelled=cancelled) + except Exception: + # Offline, a timeout, Wikidata down or answering something else. The + # page tries again itself when the track is opened. + logger.info("band lookup failed", exc_info=True) + return None + + +class BandLookup: + """One job's band lookup, in a thread of its own, so separation never + waits for it. + + The thread never touches the job. It leaves its answer here, and finish() + puts it on the job from the pipeline's own thread, once the pipeline has + got that far without being cancelled. A job cancelled or failed while the + lookup is out is therefore never written to, and an answer that arrives + after the job has finished goes nowhere. + """ + + def __init__(self, job: Job, artist_tag: str) -> None: + self._result: dict[str, str] | None = None + self._thread = threading.Thread( + target=self._run, + args=(job, artist_tag), + name=f"artist-{job.id}", + daemon=True, + ) + + def _run(self, job: Job, artist_tag: str) -> None: + # find_band never raises, so there is nothing for the thread to lose. + self._result = find_band(artist_tag, cancelled=lambda: job.cancel_requested) + + @classmethod + def start(cls, job: Job) -> BandLookup | None: + """Start looking, or None when there is nothing to look for: no artist + tag, or a band already known (a re-split inherits its source's).""" + tags = job.audio_tags or {} + if job.artist or not tagged_artist_name(tags.get("artist")): + return None + lookup = cls(job, tags["artist"]) + lookup._thread.start() + return lookup + + def finish(self, job: Job, timeout: float) -> None: + """Wait up to ``timeout`` seconds for the answer, and keep it on the + job if there is one and the job was not cancelled meanwhile.""" + self._thread.join(timeout) + if self._thread.is_alive(): + logger.info("[%s] band lookup still out after %ss; finishing without", job.id, timeout) + return + if job.cancel_requested or self._result is None: + return + _set(job, artist=self._result) diff --git a/app/pipeline/audio_stats.py b/app/pipeline/audio_stats.py index 432a6dd2..302ea26a 100644 --- a/app/pipeline/audio_stats.py +++ b/app/pipeline/audio_stats.py @@ -41,3 +41,45 @@ def scan_stem(path: Path, buckets: int = 1500) -> tuple[list[list[float]], float rms = math.sqrt(sumsq / n) if n else 0.0 return result[:buckets], rms + + +# The level a frame of pure digital silence reports, and the floor every frame +# is clamped to. Far below anything a vocals stem carries even as bleed. +ENVELOPE_FLOOR_DB = -90 + + +def vocal_envelope(path: Path, hop_sec: float = 0.04) -> tuple[float, list[int]]: + """RMS level of the WAV at `path` every `hop_sec`, in whole dBFS clamped to + [ENVELOPE_FLOOR_DB, 0], over all channels. For the lyrics wipe (#699), + which follows the vocals stem's energy to tell singing from silence. + + Returns (hop actually used, levels). The hop is a whole number of frames, + so it can differ from `hop_sec` by a fraction of a sample. Whole decibels + because the wipe needs no finer: a word boundary is a change of ten or + more, and integers keep the JSON for a ten minute track near 60 KB. + + One streamed pass in blocks of whole hops, so memory stays constant + whatever the track's length, the same way scan_stem does (#286).""" + info = sf.info(str(path)) + if info.frames == 0 or info.samplerate <= 0: + return hop_sec, [] + hop = max(1, round(info.samplerate * hop_sec)) + # A few seconds per block: large enough that the per-block overhead is + # nothing, small enough to stay a few MB for any track. + blocksize = hop * 128 + levels: list[int] = [] + for block in sf.blocks(str(path), blocksize=blocksize, dtype="float32", always_2d=True): + if block.size == 0: + continue + whole = (len(block) // hop) * hop + parts = [block[:whole].reshape(-1, hop * block.shape[1])] if whole else [] + if whole < len(block): + # The last partial hop of the file: its own frame, averaged over + # what it has. + parts.append(block[whole:].reshape(1, -1)) + for frames in parts: + mean_sq = np.mean(frames.astype(np.float64) ** 2, axis=1) + db = 10.0 * np.log10(np.maximum(mean_sq, 1e-12)) + clipped = np.clip(np.round(db), ENVELOPE_FLOOR_DB, 0).astype(int) + levels.extend(clipped.tolist()) + return hop / info.samplerate, levels diff --git a/app/pipeline/audio_tags.py b/app/pipeline/audio_tags.py new file mode 100644 index 00000000..66f9ab7c --- /dev/null +++ b/app/pipeline/audio_tags.py @@ -0,0 +1,460 @@ +"""What a source says about itself: artist, title, album, and lyrics (#699). + +Read once at import and kept on the job as ``audio_tags``, for the Lyrics tab +and the artist box. The library otherwise records only a title, and a title +names the song as often as the band, so a file's own tags are the best answer +there is to "who is this and what are the words". + +Two sources, one shape: + +* an upload's container tags, read with ffprobe before the upload is deleted + (the stems carry no tags, and the source goes once the pipeline is done); +* a link's metadata from yt-dlp, which fills artist, track and album for + releases YouTube knows as music, and otherwise the channel, when the video + itself corroborates that the channel is the artist's. + +Everything here is optional. A probe that fails, times out or finds nothing +gives ``None`` and the import goes ahead exactly as before. +""" + +from __future__ import annotations + +import json +import logging +import re +import subprocess +from itertools import groupby +from pathlib import Path +from typing import Any + +from app.core.config import ( + AUDIO_TAG_LEGACY_CODEPAGES, + AUDIO_TAG_LYRICS_MAX_CHARS, + AUDIO_TAG_MAX_CHARS, + AUDIO_TAG_MOJIBAKE_MIN_WORD, + TIMEOUT_PROBE_TAGS, + ffprobe_executable, +) + +logger = logging.getLogger(__name__) + +# ffprobe reports an MP3's lyrics frame as "lyrics-" ("lyrics-eng"), +# a FLAC's as "LYRICS" or "UNSYNCEDLYRICS", an M4A's as "lyrics". Keys are +# compared lower-cased. +_LYRICS_KEYS = ("lyrics", "unsyncedlyrics", "syncedlyrics") + + +def _is_control(ch: str) -> bool: + """C0 and C1 control characters, and lone surrogates (raw bytes that + were not UTF-8, which nothing downstream could encode).""" + code = ord(ch) + return code < 0x20 or 0x7F <= code <= 0x9F or 0xD800 <= code <= 0xDFFF + + +def _clean(value: Any, limit: int) -> str: + """A tag value as plain text: control characters other than line breaks + dropped, trimmed, and cut at ``limit``. Tags are written by anyone, and + this text ends up in the page.""" + if not isinstance(value, str): + return "" + text = "".join(ch for ch in value if ch in "\n\t" or not _is_control(ch)) + return text.replace("\r\n", "\n").strip()[:limit] + + +def _first(tags: dict[str, str], *keys: str) -> str: + for key in keys: + value = _clean(tags.get(key), AUDIO_TAG_MAX_CHARS) + if value: + return value + return "" + + +# ── Tags written in a Windows codepage ── +# +# An MP3's ID3v1 tag has no encoding at all, and an ID3v2 frame marked +# ISO-8859-1 is as often filled with whatever the tagger's Windows used: +# cp1250 across Central Europe, cp1251 for Cyrillic. Read as Latin-1, as +# ffprobe does, "zapomniałem" comes out "zapomnia³em", "Żuraw" "¯uraw", and +# "Кино" "Êèíî". A tag like that finds nothing on MusicBrainz or LRCLIB and +# looks broken on the page. +# +# Re-reading is only ever a choice between readings of the same bytes, and +# only taken when the Latin-1 one is implausible and a codepage's is clearly +# less so. Correct Latin-1 ("Björk", "Sigur Rós", "Mañana", "Ágætis byrjun") +# scores zero and is never touched. + +# Beside a letter these are ordinary punctuation, in Latin-1 and in every +# codepage tried: an apostrophe, quotes, dashes. +_BESIDE_LETTER_OK = frozenset( + "\N{NO-BREAK SPACE}\N{SOFT HYPHEN}\N{COPYRIGHT SIGN}\N{REGISTERED SIGN}" + "\N{DEGREE SIGN}\N{MIDDLE DOT}\N{ACUTE ACCENT}\N{EURO SIGN}\N{TRADE MARK SIGN}" + "\N{LEFT-POINTING DOUBLE ANGLE QUOTATION MARK}\N{RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK}" + "\N{SINGLE LEFT-POINTING ANGLE QUOTATION MARK}\N{SINGLE RIGHT-POINTING ANGLE QUOTATION MARK}" + "\N{LEFT SINGLE QUOTATION MARK}\N{RIGHT SINGLE QUOTATION MARK}\N{SINGLE LOW-9 QUOTATION MARK}" + "\N{LEFT DOUBLE QUOTATION MARK}\N{RIGHT DOUBLE QUOTATION MARK}\N{DOUBLE LOW-9 QUOTATION MARK}" + "\N{EN DASH}\N{EM DASH}\N{HORIZONTAL ELLIPSIS}\N{BULLET}" +) +# Spanish opens a question or exclamation with these, right before a letter. +_OPENING_MARKS = frozenset("\N{INVERTED QUESTION MARK}\N{INVERTED EXCLAMATION MARK}") + + +def _script(ch: str) -> str: + code = ord(ch) + if 0x0400 <= code <= 0x052F: + return "cyrillic" + if code < 0x0250 or 0x1E00 <= code <= 0x1EFF: + return "latin" + return "other" + + +def _implausibility(text: str) -> int: + """How much ``text`` looks like bytes read in the wrong codepage. Zero for + anything a person would have typed. + + Three signs, each worth the same: + + * a C1 control character, which no one types, and which is what Latin-1 + makes of cp1250's "ś", "ź", "Š", "ž" or cp1252's quotes; + * a symbol against a letter, "zapomnia³em", "¯uraw", "mo¿e" (cp1250's + "ł", "Ż", "ż"), except punctuation that belongs there; + * a word that mixes Latin and Cyrillic letters, or is at least + AUDIO_TAG_MOJIBAKE_MIN_WORD letters long without a single one from + ASCII, "Êèíî", "Ãðóïïà" (cp1251's "Кино", "Группа"). + """ + score = 0 + last = len(text) - 1 + for i, ch in enumerate(text): + if 0x80 <= ord(ch) <= 0x9F: + score += 1 + continue + if ch.isascii() or ch.isalpha() or ch.isspace() or ch in _BESIDE_LETTER_OK: + continue + before = i > 0 and text[i - 1].isalpha() + after = i < last and text[i + 1].isalpha() + if ch in _OPENING_MARKS and not before: + continue + if before or after: + score += 1 + for is_letter, run in groupby(text, str.isalpha): + if not is_letter: + continue + word = "".join(run) + scripts = {_script(ch) for ch in word} + mixed = len(scripts) > 1 + all_accents = ( + scripts == {"latin"} + and len(word) >= AUDIO_TAG_MOJIBAKE_MIN_WORD + and not any(ch.isascii() for ch in word) + ) + if mixed or all_accents: + score += 1 + return score + + +def _tagger_bytes(value: str) -> tuple[bytes, bool] | None: + """The bytes the tagger wrote, when ``value`` could be a misreading of + them, and whether they came straight from the file. + + ffprobe passes a field it cannot read as UTF-8 through as raw bytes (an + ID3v1 tag's, see probe_tags), which arrive as lone surrogates. Anything + else outside ASCII that fits Latin-1 may have been decoded as Latin-1. + Text beyond Latin-1 came from a Unicode frame and is what it says. + """ + if value.isascii(): + return None + if any(0xD800 <= ord(ch) <= 0xDFFF for ch in value): + try: + return value.encode("utf-8", "surrogateescape"), True + except UnicodeEncodeError: + return None + try: + return value.encode("latin-1"), False + except UnicodeEncodeError: + return None + + +def _as_utf8(raw: bytes, from_file: bool) -> str | None: + """``raw`` read as UTF-8 ("ż" is "ż", "é" is "é"), when it is. + + Strict, so it is only ever taken when every byte agrees, which Latin-1 + text practically never does by accident. Bytes straight from an ID3v1 + field may end in half a character, since the field is cut at 30 bytes. + """ + try: + text = raw.decode("utf-8") + except UnicodeDecodeError as exc: + if not from_file or exc.reason != "unexpected end of data": + return None + text = raw[: exc.start].decode("utf-8", "strict") + if text.isascii() or any(_is_control(ch) for ch in text if ch not in "\n\r\t"): + return None + return text + + +def _repair_encoding(tags: dict[str, str]) -> dict[str, str]: + """``tags`` with values misread from UTF-8 or a Windows codepage read again. + + UTF-8 is decided value by value, since it checks itself. A codepage is + decided for the file as a whole: one tagger wrote every field, so a title + that gives the codepage away ("Zapomnia³em") settles an artist that + cannot on its own ("Piêæ" for "Pięć"). The codepage must make the file's + fields, together, strictly more plausible than Latin-1 does; two + codepages that are equally good and disagree settle nothing. And it is + only applied to a field it makes no less plausible. + """ + repaired = dict(tags) + legacy: dict[str, tuple[bytes, int]] = {} + for key, value in tags.items(): + found = _tagger_bytes(value) + if found is None: + continue + raw, from_file = found + utf8 = _as_utf8(raw, from_file) + if utf8 is not None: + repaired[key] = utf8 + continue + # What a reader that follows the ID3 spec would have shown. + latin = raw.decode("latin-1") + repaired[key] = latin + legacy[key] = (raw, _implausibility(latin)) + before = sum(score for _, score in legacy.values()) + if not before: + return repaired + readings: list[tuple[int, dict[str, tuple[str, int]]]] = [] + for codepage in AUDIO_TAG_LEGACY_CODEPAGES: + try: + decoded = {key: raw.decode(codepage) for key, (raw, _) in legacy.items()} + except UnicodeDecodeError: + # A byte the codepage has no character for: not this codepage. + continue + scored = {key: (text, _implausibility(text)) for key, text in decoded.items()} + readings.append((sum(score for _, score in scored.values()), scored)) + if not readings: + return repaired + best_score, best = min(readings, key=lambda reading: reading[0]) + if best_score >= before: + return repaired + texts = {key: text for key, (text, _) in best.items()} + for score, scored in readings: + if score == best_score and {key: text for key, (text, _) in scored.items()} != texts: + logger.info("tags could be read in more than one codepage; left as they are") + return repaired + for key, (text, score) in best.items(): + if score <= legacy[key][1]: + repaired[key] = text + return repaired + + +def tags_from_probe(raw: dict[str, Any] | None) -> dict[str, str] | None: + """Artist, title, album and lyrics from ffprobe's ``format.tags``, or None + when the file names none of them.""" + if not isinstance(raw, dict): + return None + tags = {str(k).lower(): v for k, v in raw.items() if isinstance(v, str)} + tags = _repair_encoding(tags) + lyrics = "" + for key, value in tags.items(): + if key in _LYRICS_KEYS or key.startswith("lyrics-") or key.startswith("lyrics "): + lyrics = _clean(value, AUDIO_TAG_LYRICS_MAX_CHARS) + if lyrics: + break + found = { + "artist": _first(tags, "artist", "album_artist", "albumartist", "performer"), + "title": _first(tags, "title"), + "album": _first(tags, "album"), + "lyrics": lyrics, + } + found = {k: v for k, v in found.items() if v} + return found or None + + +# YouTube Music's auto-generated channels are named " - Topic", and +# only ever carry that artist's releases. +_TOPIC_SUFFIX = " - Topic" +# "Artist - Title", with the separators people actually type: a hyphen, an en +# or em dash or a pipe with space either side, a colon followed by one, and the +# ones East Asian titles use: "_", "/" or "~" after a space, their full-width +# forms with or without one ("YOASOBI／群青"), or a dash touching the title +# ("周杰倫 -晴天"). A title +# that goes straight on to the song in marks counts too: +# "周杰倫 Jay Chou【晴天 Sunny Day】", "YOASOBI「夜に駆ける」". +_TITLE_SEPARATOR_RE = re.compile( + r"\s+[-\N{EN DASH}\N{EM DASH}|_/~]\s*|\s*:\s+" + r"|\s*[\N{FULLWIDTH SOLIDUS}\N{FULLWIDTH VERTICAL LINE}\N{FULLWIDTH HYPHEN-MINUS}" + r"\N{FULLWIDTH COLON}]\s*" + r"|\s*(?=[\N{LEFT BLACK LENTICULAR BRACKET}\N{LEFT CORNER BRACKET}" + r"\N{LEFT WHITE CORNER BRACKET}\N{LEFT DOUBLE ANGLE BRACKET}\N{LEFT ANGLE BRACKET}" + r"\N{LEFT WHITE LENTICULAR BRACKET}])" +) +# What an artist's own channel tends to append to their name: "NirvanaVEVO", +# "Nirvana Official", "King Gnu official YouTube channel", "公式チャンネル" +# (official channel), "官方頻道". Compared on the squashed key, so spacing is +# irrelevant. +_CHANNEL_SUFFIXES = ( + "vevo", + "official", + "channel", + "youtube", + "officiel", + "oficial", + "チャンネル", + "公式", + "頻道", + "频道", + "官方", +) + + +def _name_key(text: str) -> str: + """A name reduced for comparison: no case, accents, punctuation or + spacing, and traditional Chinese as simplified (title_parse.name_key), so + a title's "Dawid Podsiadlo" is the channel's "Dawid Podsiadło".""" + # Imported here: title_parse reaches the network layer's key function + # lazily, and this module stays light to import. + from app.pipeline.title_parse import name_key + + return name_key(text) + + +def _channel_keys(channel: str) -> list[str]: + """The channel's key, then each shorter one its suffixes come off to: + "Roxy Music" stays "roxymusic" too, however many suffixes there are.""" + key = _name_key(channel) + keys = [key] + stripped = True + while stripped: + stripped = False + for suffix in _CHANNEL_SUFFIXES: + suffix_key = _name_key(suffix) + if key.endswith(suffix_key) and key != suffix_key: + key = key.removesuffix(suffix_key) + keys.append(key) + stripped = True + return keys + + +def _better_spelling(title_name: str, channel: str) -> str: + """The better of two spellings of one name: the channel's when it has the + accents a title typed without them lost ("Beyonce" on the channel + "Beyoncé") and keeps its words apart; the title's otherwise. A channel + handle runs the words together ("NataliaKukulska" for the title's + "Natalia Kukulska"), and a name with its spaces gone finds nothing on + MusicBrainz or LRCLIB.""" + + def accents(text: str) -> int: + return sum(1 for ch in text if not ch.isascii()) + + if accents(channel) > accents(title_name) and len(channel.split()) >= len(title_name.split()): + return channel + return title_name + + +def _artist_from_channel(info: dict[str, Any]) -> tuple[str, str] | None: + """(artist, title) from the channel, only when something corroborates it. + + A bare channel name is not evidence: labels, compilers and fans upload + other people's songs all the time. It counts when the channel is YouTube's + auto-generated "Artist - Topic", or when the video's own title starts with + the channel's name ("Nirvana - Lithium (Official Music Video)" on the + NirvanaVEVO channel). The title keeps whatever noise follows; the page + cleans that up. + """ + video_title = _clean(info.get("title"), AUDIO_TAG_MAX_CHARS) + channels: list[str] = [] + for field in ("channel", "uploader"): + name = _clean(info.get(field), AUDIO_TAG_MAX_CHARS) + if name and name not in channels: + channels.append(name) + for channel in channels: + if channel.endswith(_TOPIC_SUFFIX): + artist = channel.removesuffix(_TOPIC_SUFFIX).strip() + if artist: + return artist, video_title + continue + keys = [k for k in _channel_keys(channel) if k] + if not keys: + continue + for match in _TITLE_SEPARATOR_RE.finditer(video_title): + prefix = video_title[: match.start()].strip() + if prefix and _name_key(prefix) in keys: + rest = video_title[match.end() :].strip() + if rest: + # The title's spelling ("Nirvana", not "NirvanaVEVO"), + # unless the channel is the name itself spelt better. + same = _name_key(prefix) == keys[0] + return (_better_spelling(prefix, channel) if same else prefix), rest + break + return None + + +def tags_from_ytdlp(info: dict[str, Any] | None) -> dict[str, str] | None: + """Artist, track and album from yt-dlp's info, or None. + + The music fields first, and they always win. Only when they name no + artist does the channel count, and then only when corroborated (see + _artist_from_channel): the channel alone is as often a label ("Roadrunner + Records") or a fan's upload, and a wrong artist is worse than none because + it gets searched for. + """ + if not isinstance(info, dict): + return None + artists = info.get("artists") + artist = _clean(info.get("artist"), AUDIO_TAG_MAX_CHARS) or ( + _clean(artists[0], AUDIO_TAG_MAX_CHARS) if isinstance(artists, list) and artists else "" + ) + title = _clean(info.get("track"), AUDIO_TAG_MAX_CHARS) + if not artist and (corroborated := _artist_from_channel(info)) is not None: + artist = corroborated[0] + title = title or corroborated[1] + found = { + "artist": artist, + "title": title, + "album": _clean(info.get("album"), AUDIO_TAG_MAX_CHARS), + } + found = {k: v for k, v in found.items() if v} + # An album alone says nothing useful about who or what this is. + if not found.get("artist") and not found.get("title"): + return None + return found + + +def probe_tags(path: Path) -> dict[str, str] | None: + """Read an upload's tags. Never raises: tags are not worth failing an + upload over, so every failure is logged and gives None.""" + try: + result = subprocess.run( + [ + ffprobe_executable(), + "-v", + "quiet", + "-print_format", + # A field that is not UTF-8 (an ID3v1 tag, in whatever + # codepage its tagger used) as its bytes, rather than + # replacement characters: _repair_encoding reads them again. + "json=string_validation=ignore", + "-show_entries", + "format_tags", + str(path), + ], + capture_output=True, + # The same reason as _probe_duration in api/jobs.py: tags are + # arbitrary text, and the Windows locale codec would choke on it. + # Bytes that are not UTF-8 come through as lone surrogates. + text=True, + encoding="utf-8", + errors="surrogateescape", + timeout=TIMEOUT_PROBE_TAGS, + ) + except (OSError, subprocess.SubprocessError): + logger.warning("could not read tags from %s", path.name, exc_info=True) + return None + if result.returncode != 0: + logger.info("ffprobe found no readable tags in %s", path.name) + return None + try: + data = json.loads(result.stdout or "{}") + except ValueError: + logger.info("ffprobe tag output for %s was not JSON", path.name) + return None + fmt = data.get("format") if isinstance(data, dict) else None + return tags_from_probe(fmt.get("tags") if isinstance(fmt, dict) else None) diff --git a/app/pipeline/discogs.py b/app/pipeline/discogs.py new file mode 100644 index 00000000..6cfaf78b --- /dev/null +++ b/app/pipeline/discogs.py @@ -0,0 +1,889 @@ +"""A band's profile from Discogs, for bands Wikipedia has no article on. + +Small and underground bands from anywhere have a Discogs page long before +they have a Wikipedia article: a profile, members, releases and their own +links. This asks Discogs for them, and only when the user has set a Discogs +personal access token in Settings (settings.get_discogs_token). Without one, +nothing here is ever called. + +Finding the right artist follows the rule the rest of the lookups keep: no +weak guesses. A name alone is never enough, since Discogs has many artists +called NIHIL ("Nihil", "Nihil (2)", "Nihil (5)"), and a song's title can be +a band's name too. An artist is taken only when one of these says so: + +* the MusicBrainz artist the track was identified as links to it (its url + relationship to Discogs, from the cached MusicBrainz lookup); +* the Wikidata item of the band saved on the track names it (P1953); +* a Discogs release credits that exact artist name (case, accents and + scripts aside, and Discogs' "(5)" set aside as disambiguation) with the + track's song on its tracklist, or as the album the track is from. Two + different artists doing so is no answer at all. + +The token goes in the Authorization header only: never in a URL, never in a +log line, never in an answer. Every request takes its turn from +ratelimit.DISCOGS, is abandoned after TIMEOUT_DISCOGS_REQUEST, reads at most +DISCOGS_MAX_BYTES, and is kept on disk for DISCOGS_CACHE_TTL_SEC, so opening +the same band again asks Discogs nothing. Nothing here raises into its +callers: a failure is "nothing found". + +What is sent: the track's artist name, its song and album titles, and +Discogs ids; and a band name typed in the artist box that Wikipedia did not +find (search_artists), whose candidates the user picks from. +""" + +from __future__ import annotations + +import hashlib +import json +import logging +import re +import threading +import time +import unicodedata +import urllib.error +import urllib.parse +import urllib.request +import uuid +from collections.abc import Callable +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +from app.core.config import ( + DISCOGS_API, + DISCOGS_CACHE_DIR, + DISCOGS_CACHE_TTL_SEC, + DISCOGS_MAX_BYTES, + DISCOGS_REQUEST_BUDGET_SEC, + DISCOGS_RETRIES, + DISCOGS_RETRY_BACKOFF_SEC, + DISCOGS_RETRY_MAX_WAIT_SEC, + MUSICBRAINZ_USER_AGENT, + TIMEOUT_DISCOGS_REQUEST, +) +from app.core.models import MBID_RE +from app.pipeline import artist_lookup, name_aliases, ratelimit +from app.pipeline.artist_lookup import _QID_RE, _ssl_context, tagged_artist_name + +logger = logging.getLogger("stemdeck.artist") + +# Replaced by tests with a temporary directory. +CACHE_DIR: Path = DISCOGS_CACHE_DIR + +Cancelled = Callable[[], bool] + + +def _never() -> bool: + return False + + +# ── the client ── + + +def _send(request: urllib.request.Request) -> bytes: + """One request, as it goes out. Raises on any failure; an HTTP error + status raises urllib.error.HTTPError. Replaced by tests.""" + # A fixed https base with a path and query built here from ids and names, + # never a URL from a request or a tag, which is what B310 exists to catch. + with urllib.request.urlopen( # nosec B310 + request, timeout=TIMEOUT_DISCOGS_REQUEST, context=_ssl_context() + ) as response: + return response.read(DISCOGS_MAX_BYTES + 1) + + +# Replaced by tests, so a retry costs no real time. +_sleep = time.sleep + + +def _retry_wait(err: urllib.error.HTTPError, attempt: int) -> float: + """How long to wait before asking again: what Retry-After says when it is + a number of seconds, else the backoff for this attempt, capped.""" + backoff = DISCOGS_RETRY_BACKOFF_SEC * 2**attempt + try: + asked = float((err.headers or {}).get("Retry-After") or "") + except (TypeError, ValueError): + asked = backoff + return max(0.0, min(asked, DISCOGS_RETRY_MAX_WAIT_SEC)) + + +class _Refused(Exception): + """Discogs answered, and the answer was no (a 401, a 404).""" + + def __init__(self, status: int) -> None: + super().__init__(f"HTTP {status}") + self.status = status + + +def _fetch(path: str, params: dict[str, str], token: str, cancelled: Cancelled) -> Any: + """One Discogs request, after waiting for its turn. Raises on any failure. + A 429 or a 502/503 is asked again, each time on a turn of its own, up to + DISCOGS_RETRIES times and never past DISCOGS_REQUEST_BUDGET_SEC.""" + query = urllib.parse.urlencode(params) + url = f"{DISCOGS_API}/{path}" + (f"?{query}" if query else "") + request = urllib.request.Request( + url, + headers={ + "User-Agent": MUSICBRAINZ_USER_AGENT, + "Accept": "application/vnd.discogs.v2.discogs+json", + }, + ) + # The token lives here and nowhere else: not in the URL, which a proxy or + # an exception could log, and not on a redirect, which urllib would + # otherwise send it on to whatever host the redirect names. + request.add_unredirected_header("Authorization", f"Discogs token={token}") + deadline = time.monotonic() + DISCOGS_REQUEST_BUDGET_SEC + for attempt in range(DISCOGS_RETRIES + 1): + ratelimit.DISCOGS.wait(cancelled=cancelled) + try: + body = _send(request) + break + except urllib.error.HTTPError as err: + if err.code not in (429, 502, 503): + raise _Refused(err.code) from None + if attempt == DISCOGS_RETRIES: + raise + left = _retry_wait(err, attempt) + if time.monotonic() + left > deadline: + raise + # Slept in short steps, so a cancelled lookup stops promptly. + while left > 0: + if cancelled(): + raise ratelimit.RateLimited("cancelled while waiting") from None + step = min(left, 0.25) + _sleep(step) + left -= step + if len(body) > DISCOGS_MAX_BYTES: + raise ValueError("Discogs answer too large") + return json.loads(body) + + +# ── the cache ── + + +def _cache_path(path: str, params: dict[str, str]) -> Path: + # A hash of our own request, so no name from a tag ever reaches a path. + # The token is not part of the request asked for, so it is not in it. + key = path + "?" + urllib.parse.urlencode(sorted(params.items())) + return CACHE_DIR / f"{hashlib.sha256(key.encode('utf-8')).hexdigest()}.json" + + +def _cache_get(path: str, params: dict[str, str]) -> Any: + try: + kept = json.loads(_cache_path(path, params).read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + if not isinstance(kept, dict) or not isinstance(kept.get("at"), (int, float)): + return None + if time.time() - kept["at"] > DISCOGS_CACHE_TTL_SEC: + return None + return kept.get("data") + + +def _cache_put(path: str, params: dict[str, str], data: Any) -> None: + """Best-effort: a cache that cannot be written only means asking again.""" + target = _cache_path(path, params) + try: + target.parent.mkdir(parents=True, exist_ok=True) + tmp = target.with_name(f"{target.name}.{uuid.uuid4().hex}.tmp") + try: + tmp.write_text(json.dumps({"at": time.time(), "data": data}), encoding="utf-8") + tmp.replace(target) + finally: + tmp.unlink(missing_ok=True) + except OSError: + logger.info("could not cache a Discogs answer", exc_info=True) + + +@dataclass +class _Session: + """One lookup's token and cancel check, and whether any request failed + (so a "nothing found" that was really "could not ask" is not kept).""" + + token: str + cancelled: Cancelled = _never + failed: bool = False + asked: list[str] = field(default_factory=list) + + def get(self, path: str, params: dict[str, str] | None = None) -> dict[str, Any] | None: + """The answer to one request, from the cache when it is there, or + None on any failure. Never raises.""" + params = dict(params or {}) + kept = _cache_get(path, params) + if isinstance(kept, dict): + return kept + if self.cancelled(): + self.failed = True + return None + self.asked.append(path) + try: + data = _fetch(path, params, self.token, self.cancelled) + except _Refused as err: + # A 404 is an answer: that release or artist is not there. + if err.status != 404: + self.failed = True + logger.info("Discogs refused %s (HTTP %s)", path.split("/")[0], err.status) + return None + except Exception as err: + self.failed = True + # The type only: nothing about the request is worth logging more. + logger.info("Discogs request failed (%s)", type(err).__name__) + return None + if not isinstance(data, dict): + return None + _cache_put(path, params, data) + return data + + +# ── names ── + + +# Discogs tells artists who share a name apart with a number: "Nihil (5)". +_NUMBER_SUFFIX_RE = re.compile(r"\s*\(\d{1,4}\)\s*$") +_BRACKETS_RE = re.compile(r"\s*[(\[][^)\]]*[)\]]") + + +def display_name(name: Any) -> str: + """An artist's name as Discogs gives it, less its number: "Nihil (5)" is + Nihil. A trailing "*" (a name variation on a credit) goes too.""" + text = str(name or "").strip().rstrip("*").strip() + return _NUMBER_SUFFIX_RE.sub("", text).strip() + + +def name_key(name: Any) -> str: + """A Discogs artist name reduced for comparing, number set aside + (name_aliases.name_key: case, accents, punctuation, Chinese scripts).""" + return name_aliases.name_key(display_name(name)) + + +def title_key(title: Any) -> str: + """A song or album title reduced for comparing, brackets set aside + ("Barro (Remastered)" is Barro).""" + text = str(title or "") + return name_aliases.name_key(_BRACKETS_RE.sub("", text)) or name_aliases.name_key(text) + + +# ── finding the artist ── + + +# Releases a search found that are checked, at most: each is one request. +_RELEASES_CHECKED = 4 +_SEARCH_PER_PAGE = "25" +_DISCOGS_ARTIST_URL_RE = re.compile( + r"^https?://(?:www\.)?discogs\.com/(?:[a-z]{2}(?:-[a-z]{2})?/)?artist/(\d{1,12})(?:[-/?#].*)?$", + re.IGNORECASE, +) +_DISCOGS_ID_RE = re.compile(r"^[1-9]\d{0,11}$") +_WIKIDATA_DISCOGS_ARTIST = "P1953" + + +def _artist_id(value: Any) -> int | None: + if isinstance(value, bool): + return None + if isinstance(value, int) and value > 0: + return value + if isinstance(value, str) and _DISCOGS_ID_RE.match(value): + return int(value) + return None + + +def artist_id_from_musicbrainz(mbids: list[str], *, cancelled: Cancelled = _never) -> int | None: + """The Discogs artist the first of ``mbids`` that links to one does, by + its MusicBrainz url relationships, from the MusicBrainz cache when it is + there (one request a second otherwise, as every MusicBrainz caller).""" + for mbid in [m for m in mbids if isinstance(m, str) and MBID_RE.match(m)][:2]: + if cancelled(): + return None + try: + artist = name_aliases._musicbrainz_artist(mbid, cancelled=cancelled) + except Exception: + logger.info("MusicBrainz artist for Discogs failed", exc_info=True) + continue + relations = artist.get("relations") if isinstance(artist, dict) else None + for relation in relations if isinstance(relations, list) else []: + if not isinstance(relation, dict) or relation.get("type") != "discogs": + continue + url = relation.get("url") + resource = url.get("resource") if isinstance(url, dict) else None + match = _DISCOGS_ARTIST_URL_RE.match(resource) if isinstance(resource, str) else None + if match: + return int(match.group(1)) + return None + + +def artist_id_from_wikidata(qid: str, *, fetch_json: Any = None) -> int | None: + """The Discogs artist the Wikidata item ``qid`` names (P1953), when it + names exactly one. Never raises.""" + if not isinstance(qid, str) or not _QID_RE.match(qid): + return None + try: + answer = (fetch_json or artist_lookup._fetch_json)( + {"action": "wbgetentities", "ids": qid, "props": "claims"} + ) + except Exception: + logger.info("Wikidata item for Discogs failed", exc_info=True) + return None + entities = answer.get("entities") if isinstance(answer, dict) else None + entity = entities.get(qid) if isinstance(entities, dict) else None + claims = entity.get("claims") if isinstance(entity, dict) else None + statements = claims.get(_WIKIDATA_DISCOGS_ARTIST) if isinstance(claims, dict) else None + ids = set() + for statement in statements if isinstance(statements, list) else []: + if not isinstance(statement, dict) or statement.get("rank") == "deprecated": + continue + snak = statement.get("mainsnak") + datavalue = snak.get("datavalue") if isinstance(snak, dict) else None + value = datavalue.get("value") if isinstance(datavalue, dict) else None + found = _artist_id(value) + if found: + ids.add(found) + return ids.pop() if len(ids) == 1 else None + + +def _credited(artists: Any, want: str) -> set[int]: + """The ids of the artists on a Discogs credit whose name, or the name + variation they were credited as, is ``want`` (a name_key).""" + found = set() + for artist in artists if isinstance(artists, list) else []: + if not isinstance(artist, dict): + continue + artist_id = _artist_id(artist.get("id")) + if artist_id and want in {name_key(artist.get("name")), name_key(artist.get("anv"))}: + found.add(artist_id) + return found + + +def _tracks(tracklist: Any) -> list[dict[str, Any]]: + """Every track on a tracklist, the sub-tracks of an index track too.""" + tracks = [] + for track in tracklist if isinstance(tracklist, list) else []: + if not isinstance(track, dict): + continue + if track.get("type_") in (None, "", "track"): + tracks.append(track) + tracks.extend(_tracks(track.get("sub_tracks"))) + return tracks + + +def credits_on_release( + release: Any, want_artist: str, *, song: str = "", album: str = "" +) -> set[int]: + """The artists a release (GET /releases/{id}) credits by the name + ``want_artist`` with the song ``song`` on its tracklist, or, with no + song, as the album ``album``. Keys, both (name_key, title_key).""" + if not isinstance(release, dict) or not want_artist: + return set() + if song: + found: set[int] = set() + for track in _tracks(release.get("tracklist")): + if title_key(track.get("title")) != song: + continue + # A compilation credits each track's own artists. + found |= _credited(track.get("artists") or release.get("artists"), want_artist) + return found + if album and title_key(release.get("title")) == album: + return _credited(release.get("artists"), want_artist) + return set() + + +def _hit_artist_key(hit: dict[str, Any]) -> str: + """The artist part of a search hit's "Artist - Title" title, as a key.""" + title = hit.get("title") + return name_key(title.split(" - ", 1)[0]) if isinstance(title, str) else "" + + +def artist_id_from_releases( + session: _Session, name: str, *, song: str = "", album: str = "" +) -> int | None: + """The Discogs artist called ``name`` that a release credits with the + track's ``song`` (or, failing that, as its ``album``), found by searching + releases by artist and track (or album), or None. None too when two + different artists by that name both qualify: that is no answer.""" + want = name_key(name) + if not want: + return None + for mode, value in (("song", song), ("album", album)): + key = title_key(value) + if not key: + continue + params = {"type": "release", "artist": name, "per_page": _SEARCH_PER_PAGE} + params["track" if mode == "song" else "release_title"] = value + answer = session.get("database/search", params) + hits = answer.get("results") if isinstance(answer, dict) else None + hits = [h for h in hits if isinstance(h, dict)] if isinstance(hits, list) else [] + # The hits whose title names the artist first; one pressing of each + # master, since every pressing credits the same artist. + hits.sort(key=lambda h: _hit_artist_key(h) != want) + seen_masters: set[int] = set() + found: set[int] = set() + checked = 0 + for hit in hits: + release_id = _artist_id(hit.get("id")) + master = _artist_id(hit.get("master_id")) + if not release_id or (master and master in seen_masters): + continue + if master: + seen_masters.add(master) + if checked >= _RELEASES_CHECKED or session.cancelled(): + break + checked += 1 + release = session.get(f"releases/{release_id}") + found |= credits_on_release( + release, + want, + song=key if mode == "song" else "", + album=key if mode == "album" else "", + ) + if len(found) == 1: + return found.pop() + if len(found) > 1: + logger.info("Discogs: %d artists share the name and the %s", len(found), mode) + return None + return None + + +# ── what is taken from the artist ── + + +_PROFILE_MAX_CHARS = 1400 +_PROFILE_MAX_PARAGRAPHS = 8 +_MEMBERS_MAX = 40 +_GROUPS_MAX = 20 +_RELEASES_MAX = 25 +_URL_MAX_CHARS = 2048 +_NAME_MAX_CHARS = 300 +_ELLIPSIS = chr(0x2026) + +# Discogs markup: [a=Name] and [l=Name] name an artist or a label, [a123], +# [r=123], [m123] point at one by id alone, [url=...]text[/url] is a link. +_TAG_NAMED_RE = re.compile(r"\[(?:a|l)=([^\]\[]{1,200})\]", re.IGNORECASE) +_TAG_BY_ID_RE = re.compile(r"\[(?:a|l|r|m|t)=?\d{1,12}\]", re.IGNORECASE) +_TAG_URL_RE = re.compile(r"\[url=[^\]]{0,2048}\]([\s\S]{0,2000}?)\[/url\]", re.IGNORECASE) +_TAG_BARE_URL_RE = re.compile(r"\[url\]([^\[]{0,2048})\[/url\]", re.IGNORECASE) +_TAG_STYLE_RE = re.compile(r"\[/?(?:b|i|u|s|img[^\]]{0,2048})\]", re.IGNORECASE) +_SPACES_RE = re.compile(r"[ \t\u00a0]+") + + +# Kept: it joins the parts of an emoji or of a letter in some scripts. +_ZWJ = chr(0x200D) + + +def _plain(text: str) -> str: + """``text`` without control characters, bidi overrides or runs of spaces.""" + kept = "".join( + ch + for ch in text + if ch == "\n" or unicodedata.category(ch) not in ("Cc", "Cf") or ch == _ZWJ + ) + return _SPACES_RE.sub(" ", kept) + + +def profile_paragraphs(profile: Any) -> list[str]: + """A Discogs profile as plain paragraphs: its markup turned into the text + it stands for ("[a=Nihil (5)]" is "Nihil"), references by id alone + dropped, at most _PROFILE_MAX_CHARS in all, cut at a word.""" + if not isinstance(profile, str): + return [] + text = profile.replace("\r\n", "\n").replace("\r", "\n") + text = _TAG_URL_RE.sub(lambda m: m.group(1), text) + text = _TAG_BARE_URL_RE.sub(lambda m: m.group(1), text) + text = _TAG_NAMED_RE.sub(lambda m: display_name(m.group(1)), text) + text = _TAG_BY_ID_RE.sub("", text) + text = _TAG_STYLE_RE.sub("", text) + paragraphs = [] + used = 0 + for raw in _plain(text).split("\n"): + paragraph = re.sub(r"\s+([,.;:!?])", r"\1", raw).strip() + if not paragraph: + continue + room = _PROFILE_MAX_CHARS - used + if room <= 0 or len(paragraphs) >= _PROFILE_MAX_PARAGRAPHS: + break + if len(paragraph) > room: + cut = paragraph[:room].rsplit(" ", 1)[0].rstrip(",;: ") + paragraphs.append(cut + _ELLIPSIS) + break + paragraphs.append(paragraph) + used += len(paragraph) + return paragraphs + + +def _clean_name(name: Any) -> str: + return _plain(display_name(name)).strip()[:_NAME_MAX_CHARS] if isinstance(name, str) else "" + + +# Where a URL on a Discogs artist goes, by host, in the order the box shows +# them. A host is the domain itself or any subdomain of it. +_LINK_HOSTS: tuple[tuple[str, tuple[str, ...]], ...] = ( + ("bandcamp", ("bandcamp.com",)), + ("instagram", ("instagram.com",)), + ("facebook", ("facebook.com", "fb.com")), + ("youtube", ("youtube.com", "youtu.be")), + ("spotify", ("open.spotify.com",)), + ("appleMusic", ("music.apple.com", "itunes.apple.com")), +) +_LINK_ORDER = ("website", *(kind for kind, _ in _LINK_HOSTS)) +# Hosts that are somebody else's page about the band, not the band's own +# site: never taken for "Website". +_NOT_OWN_SITE = ( + "discogs.com", + "wikipedia.org", + "wikidata.org", + "musicbrainz.org", + "myspace.com", + "twitter.com", + "x.com", + "soundcloud.com", + "last.fm", + "lastfm.de", + "allmusic.com", + "rateyourmusic.com", + "metal-archives.com", + "tiktok.com", + "deezer.com", + "tidal.com", + "imdb.com", + "linktr.ee", + "reverbnation.com", + "vk.com", + "bandsintown.com", + "songkick.com", + "genius.com", + "threads.net", + "apple.com", + "spotify.com", + "amazon.com", + "google.com", + "archive.org", +) + + +def _on(host: str, domains: tuple[str, ...]) -> bool: + return any(host == d or host.endswith("." + d) for d in domains) + + +def safe_url(value: Any) -> str: + """``value`` as a plain http(s) URL, or "": no spaces or control + characters, a host straight after the scheme, no user:password@. A bare + "www.band.com" is read as https.""" + if not isinstance(value, str): + return "" + text = value.strip() + if ( + not text + or len(text) > _URL_MAX_CHARS + or any(ch.isspace() or unicodedata.category(ch) in ("Cc", "Cf") for ch in text) + ): + return "" + if not re.match(r"^[a-z][a-z0-9+.-]*:", text, re.IGNORECASE): + if not re.match( + r"^(?:www\.)?[a-z0-9-]+(?:\.[a-z0-9-]+)*\.[a-z]{2,}(?:[/?#]|$)", text, re.I + ): + return "" + text = "https://" + text + if not re.match(r"^https?://[^/\\]", text, re.IGNORECASE): + return "" + try: + parts = urllib.parse.urlsplit(text) + except ValueError: + return "" + if parts.scheme.lower() not in ("http", "https") or not parts.hostname: + return "" + if parts.username or parts.password: + return "" + return text + + +def official_links(urls: Any) -> list[dict[str, str]]: + """The band's own links among a Discogs artist's URLs, as [{kind, url}], + one of each kind, in _LINK_ORDER: its site (the first URL on no known + service), Bandcamp, Instagram, Facebook, YouTube, Spotify, Apple Music.""" + found: dict[str, str] = {} + for value in urls if isinstance(urls, list) else []: + url = safe_url(value) + if not url: + continue + host = (urllib.parse.urlsplit(url).hostname or "").lower() + kind = next((k for k, domains in _LINK_HOSTS if _on(host, domains)), "") + if not kind and not _on(host, _NOT_OWN_SITE): + kind = "website" + if kind and kind not in found: + found[kind] = url + return [{"kind": kind, "url": found[kind]} for kind in _LINK_ORDER if kind in found] + + +def main_releases(answer: Any) -> list[dict[str, str]]: + """An artist's own releases (GET /artists/{id}/releases, role "Main"), as + [{"year", "title"}], oldest first: a master once, over any release of the + same title, and at most _RELEASES_MAX.""" + items = answer.get("releases") if isinstance(answer, dict) else None + kept: dict[str, tuple[int, int, str, str]] = {} + for item in items if isinstance(items, list) else []: + if not isinstance(item, dict) or item.get("role") != "Main": + continue + title = _plain(str(item.get("title") or "")).strip()[:_NAME_MAX_CHARS] + key = title_key(title) + if not key: + continue + year = item.get("year") + year = year if isinstance(year, int) and not isinstance(year, bool) and year > 0 else 0 + rank = 0 if item.get("type") == "master" else 1 + entry = (rank, year or 9999, title, str(year) if year else "") + if key not in kept or entry[:2] < kept[key][:2]: + kept[key] = entry + ordered = sorted(kept.values(), key=lambda e: (e[1], e[2].lower())) + return [{"year": year, "title": title} for _, _, title, year in ordered[:_RELEASES_MAX]] + + +_ARTIST_PAGE_RE = re.compile(r"^https://www\.discogs\.com/(?:[a-z]{2}/)?artist/\d{1,12}[-\w%.()]*$") + + +def artist_profile(artist: Any, releases: Any) -> dict[str, Any] | None: + """What the artist box shows from a Discogs artist (GET /artists/{id}) + and its releases, or None when it has no id or name. Images are left out: + the box takes its photos from Wikimedia, and Discogs' images are theirs.""" + if not isinstance(artist, dict): + return None + artist_id = _artist_id(artist.get("id")) + name = _clean_name(artist.get("name")) + if not artist_id or not name: + return None + members: dict[str, list[str]] = {"current": [], "former": []} + for member in artist.get("members") or []: + if not isinstance(member, dict): + continue + who = _clean_name(member.get("name")) + bucket = members["former" if member.get("active") is False else "current"] + if who and who not in bucket and len(bucket) < _MEMBERS_MAX: + bucket.append(who) + groups = [] + for group in artist.get("groups") or []: + who = _clean_name(group.get("name")) if isinstance(group, dict) else "" + if who and who not in groups and len(groups) < _GROUPS_MAX: + groups.append(who) + page = artist.get("uri") + if not isinstance(page, str) or not _ARTIST_PAGE_RE.match(page): + page = f"https://www.discogs.com/artist/{artist_id}" + real_name = _plain(str(artist.get("realname") or "")).strip()[:_NAME_MAX_CHARS] + return { + "id": artist_id, + "name": name, + "real_name": real_name if real_name and name_key(real_name) != name_key(name) else "", + "profile": profile_paragraphs(artist.get("profile")), + "members": members, + "groups": groups, + "links": official_links(artist.get("urls")), + "releases": main_releases(releases), + "url": page, + } + + +# ── one track's artist ── + + +def _first(*values: Any) -> str: + for value in values: + if isinstance(value, str) and value.strip(): + return value.strip() + return "" + + +def lookup_for_track( + identity: dict[str, Any] | None, + audio_tags: dict[str, str] | None, + band: dict[str, str] | None, + token: str, + *, + cancelled: Cancelled = _never, +) -> tuple[dict[str, Any] | None, bool]: + """The Discogs profile of the artist a track is by, and whether the answer + is settled (False when a request failed, so "nothing" may be "could not + ask"). ``band`` is the Wikidata band saved on the track, if any. Never + raises.""" + session = _Session(token=token, cancelled=cancelled) + identity = identity or {} + tags = audio_tags or {} + try: + artist_id = artist_id_from_musicbrainz( + list(identity.get("artist_mbids") or []), cancelled=cancelled + ) + if not artist_id and band and not cancelled(): + artist_id = artist_id_from_wikidata(str(band.get("id") or "")) + if not artist_id and not cancelled(): + name = tagged_artist_name(_first(tags.get("artist"), identity.get("artist"))) + song = _first(identity.get("title"), tags.get("title")) + album = _first(identity.get("album"), tags.get("album")) + artist_id = artist_id_from_releases(session, name, song=song, album=album) + if not artist_id or cancelled(): + return None, not session.failed and not cancelled() + artist = session.get(f"artists/{artist_id}") + if artist is None: + return None, not session.failed + releases = session.get( + f"artists/{artist_id}/releases", + {"sort": "year", "sort_order": "asc", "per_page": "100"}, + ) + return artist_profile(artist, releases), True + except Exception: + logger.warning("Discogs lookup failed", exc_info=True) + return None, False + + +# Answers by what they were asked from, for the life of the process: a band +# found is kept a day (its pages are on disk for a month besides), "nothing" +# an hour, and a failure not at all. +_KEPT: dict[tuple, tuple[float, dict[str, Any] | None]] = {} +_KEPT_LOCK = threading.Lock() +_KEPT_MAX = 512 +_KEEP_FOUND_SEC = 24 * 3600 +_KEEP_NOTHING_SEC = 3600 + + +def _ask_key(identity: Any, tags: Any, band: Any) -> tuple: + identity = identity if isinstance(identity, dict) else {} + tags = tags if isinstance(tags, dict) else {} + band = band if isinstance(band, dict) else {} + return ( + tuple(identity.get("artist_mbids") or ()), + str(band.get("id") or ""), + _first(tags.get("artist"), identity.get("artist")), + _first(identity.get("title"), tags.get("title")), + _first(identity.get("album"), tags.get("album")), + ) + + +def artist_for_track( + identity: dict[str, Any] | None, + audio_tags: dict[str, str] | None, + band: dict[str, str] | None, + token: str, + *, + cancelled: Cancelled = _never, +) -> dict[str, Any] | None: + """lookup_for_track's profile, kept in memory as said above. Never raises.""" + if not token: + return None + key = _ask_key(identity, audio_tags, band) + with _KEPT_LOCK: + kept = _KEPT.get(key) + if kept is not None: + at, answer = kept + if time.time() - at < (_KEEP_FOUND_SEC if answer else _KEEP_NOTHING_SEC): + return answer + answer, settled = lookup_for_track(identity, audio_tags, band, token, cancelled=cancelled) + if answer is not None or settled: + with _KEPT_LOCK: + if len(_KEPT) >= _KEPT_MAX: + _KEPT.clear() + _KEPT[key] = (time.time(), answer) + return answer + + +# ── a name typed in the box ── +# +# Here the user chooses, so nothing is taken on a name alone as above: the +# artists Discogs finds are offered, each with the start of its profile so +# "Nihil (2)" can be told from "Nihil (5)", and the one picked is fetched by +# its id. + +_CANDIDATES_MAX = 8 +# Profiles fetched for the snippet, at most: each is a request, one a second. +_CANDIDATES_PROFILED = 5 +_SNIPPET_MAX_CHARS = 200 +QUERY_MAX_CHARS = 100 + + +def valid_query(value: Any) -> str: + """A typed name, trimmed, or "" when it is empty, too long, or holds a + control or format character (a bidi override, a NUL).""" + if not isinstance(value, str): + return "" + text = value.strip() + if not text or len(text) > QUERY_MAX_CHARS: + return "" + if any(unicodedata.category(ch) in ("Cc", "Cf") and ch != _ZWJ for ch in text): + return "" + return text + + +def _snippet(profile: Any) -> str: + paragraphs = profile_paragraphs(profile) + if not paragraphs: + return "" + first = paragraphs[0] + if len(first) <= _SNIPPET_MAX_CHARS: + return first + return first[:_SNIPPET_MAX_CHARS].rsplit(" ", 1)[0].rstrip(",;: ") + _ELLIPSIS + + +def search_artists( + name: str, token: str, *, cancelled: Cancelled = _never +) -> list[dict[str, Any]] | None: + """The Discogs artists a search for ``name`` finds, as [{"id", "name", + "profile"}], those named exactly ``name`` first, at most + _CANDIDATES_MAX. ``name`` keeps Discogs' number ("Nihil (5)"), since that + is how two artists of one name are told apart; ``profile`` is the start of + the artist's profile, or "" when it has none or was not in in time. None + when the search itself failed. Never raises.""" + query = valid_query(name) + if not token or not query: + return None + session = _Session(token=token, cancelled=cancelled) + try: + answer = session.get( + "database/search", {"type": "artist", "q": query, "per_page": _SEARCH_PER_PAGE} + ) + if answer is None: + return None + hits = answer.get("results") + want = name_key(query) + found: list[dict[str, Any]] = [] + for hit in hits if isinstance(hits, list) else []: + if not isinstance(hit, dict) or hit.get("type", "artist") != "artist": + continue + artist_id = _artist_id(hit.get("id")) + title = hit.get("title") + shown = _plain(title).strip()[:_NAME_MAX_CHARS] if isinstance(title, str) else "" + if not artist_id or not shown or any(c["id"] == artist_id for c in found): + continue + found.append({"id": artist_id, "name": shown, "profile": ""}) + # Stable: Discogs' own order within each half. + found.sort(key=lambda c: name_key(c["name"]) != want) + found = found[:_CANDIDATES_MAX] + for candidate in found[:_CANDIDATES_PROFILED]: + if cancelled(): + break + artist = session.get(f"artists/{candidate['id']}") + if isinstance(artist, dict): + candidate["profile"] = _snippet(artist.get("profile")) + return found + except Exception: + logger.warning("Discogs artist search failed", exc_info=True) + return None + + +def artist_by_id( + artist_id: int, token: str, *, cancelled: Cancelled = _never +) -> dict[str, Any] | None: + """The profile of the Discogs artist ``artist_id`` (artist_profile's + shape), or None when there is none or it could not be had. Never raises.""" + if not token or not isinstance(artist_id, int) or artist_id <= 0: + return None + session = _Session(token=token, cancelled=cancelled) + try: + artist = session.get(f"artists/{artist_id}") + if artist is None or cancelled(): + return None + releases = session.get( + f"artists/{artist_id}/releases", + {"sort": "year", "sort_order": "asc", "per_page": "100"}, + ) + return artist_profile(artist, releases) + except Exception: + logger.warning("Discogs artist failed", exc_info=True) + return None + + +def forget() -> None: + """Drop every answer kept in memory. For tests.""" + with _KEPT_LOCK: + _KEPT.clear() diff --git a/app/pipeline/discogs_auth.py b/app/pipeline/discogs_auth.py new file mode 100644 index 00000000..f0c1881b --- /dev/null +++ b/app/pipeline/discogs_auth.py @@ -0,0 +1,87 @@ +"""Whether Discogs accepts the user's personal access token. + +Asked once, when the token is saved in Settings (POST /api/settings), so a +token Discogs refuses is caught there rather than leaving every band lookup +to fail without a word. Asked of /oauth/identity, which answers any valid +token with its user and refuses any other with 401. + +The token is sent only in the Authorization header: never in a URL, never +logged, and never in an exception message. +""" + +from __future__ import annotations + +import logging +import threading +import urllib.error +import urllib.request + +from app.core.config import DISCOGS_IDENTITY_URL, MUSICBRAINZ_USER_AGENT, TIMEOUT_DISCOGS_CHECK +from app.pipeline.artist_lookup import _ssl_context + +logger = logging.getLogger(__name__) + +# An identity answer is a few hundred bytes; nothing past this is read. +_MAX_BYTES = 64 * 1024 + + +def _ask_identity(token: str) -> int: + """The HTTP status Discogs answers /oauth/identity with for ``token``. + Raises on anything that is not an answer (offline, timeout).""" + request = urllib.request.Request( + DISCOGS_IDENTITY_URL, + headers={ + "Accept": "application/json", + # Discogs refuses clients that do not name themselves. + "User-Agent": MUSICBRAINZ_USER_AGENT, + }, + ) + # Never sent on to where a redirect points (discogs.py _get). + request.add_unredirected_header("Authorization", f"Discogs token={token}") + try: + # A fixed https URL, never one from a request or a tag (B310). + with urllib.request.urlopen( # nosec B310 + request, timeout=TIMEOUT_DISCOGS_CHECK, context=_ssl_context() + ) as response: + response.read(_MAX_BYTES) + return int(response.status) + except urllib.error.HTTPError as err: + return int(err.code) + + +def _verdict(token: str) -> bool | None: + try: + status = _ask_identity(token) + except Exception as err: + # The type only: an exception from urllib never holds the header, but + # nothing about this request is worth the risk of logging more. + logger.info("Discogs could not be asked whether the token works (%s)", type(err).__name__) + return None + if status == 401: + return False + if 200 <= status < 300: + return True + # A 429 or a 5xx says nothing about the token. + return None + + +def discogs_token_works(token: str) -> bool | None: + """Whether Discogs accepts ``token``: True or False, or None when it + cannot be asked (offline, Discogs down or too slow), and the token is then + kept on trust. + + The Save waits for this, so it waits at most twice the request timeout: + the request's own timeout bounds each read, not the whole answer. The + request is left to finish on its own thread.""" + result: list[bool | None] = [None] + + def ask() -> None: + result[0] = _verdict(token) + + worker = threading.Thread(target=ask, name="discogs-token-check", daemon=True) + worker.start() + worker.join(TIMEOUT_DISCOGS_CHECK * 2) + if worker.is_alive(): + logger.info("Discogs took too long to say whether the token works") + return None + return result[0] diff --git a/app/pipeline/download.py b/app/pipeline/download.py index 440a3d34..27ce23be 100644 --- a/app/pipeline/download.py +++ b/app/pipeline/download.py @@ -2,6 +2,7 @@ import logging import re +import threading import time import urllib.parse from pathlib import Path @@ -11,6 +12,7 @@ from app.core.config import bundled_js_runtime, ffmpeg_dir, js_solver_available from app.core.models import Job, JobCancelled, _set from app.core.settings import get_cookies_file, get_max_duration_sec, get_video_max_height +from app.pipeline.audio_tags import tags_from_ytdlp logger = logging.getLogger("stemdeck.download") @@ -81,11 +83,53 @@ def _with_retries(job: Job, fn, *, what: str): wait, ) _set(job, stage=f"Network error — retrying ({attempt + 1}/{_MAX_RETRIES})...") - time.sleep(wait) + _sleep_unless_cancelled(job, wait) else: raise +# How often a wait on yt-dlp looks at the job's cancel flag. +_CANCEL_POLL_SEC = 0.1 + + +def _sleep_unless_cancelled(job: Job, seconds: float) -> None: + """Sleep ``seconds``, or raise JobCancelled as soon as the job is.""" + end = time.monotonic() + seconds + while (left := end - time.monotonic()) > 0: + if job.cancel_requested: + raise JobCancelled() + time.sleep(min(left, _CANCEL_POLL_SEC)) + + +def _cancellable(job: Job, fn): + """``fn()``, run on a thread of its own so a cancel stops the wait for it. + + yt-dlp's metadata probe has no progress hook to raise from, and resolving + a YouTube link takes up to twenty seconds, all of it deaf to Cancel. The + probe writes nothing, so one abandoned by a cancel is left to finish on + its own; only its answer is dropped.""" + result: dict = {} + + def run() -> None: + try: + result["value"] = fn() + # BaseException on purpose: it is raised again on the job's thread + # below. Narrower, a SystemExit from yt-dlp would end this thread + # quietly and the import would go on with no answer. + except BaseException as exc: + result["error"] = exc + + worker = threading.Thread(target=run, name=f"probe-{job.id}", daemon=True) + worker.start() + while worker.is_alive(): + if job.cancel_requested: + raise JobCancelled() + worker.join(_CANCEL_POLL_SEC) + if "error" in result: + raise result["error"] + return result.get("value") + + # YouTube refusing to serve us at all, as opposed to a video being unavailable. # Cookies are the only remedy in-tree: yt-dlp ships no PO token generator. _BOT_CHECK = ("sign in to confirm", "http error 429", "too many requests") @@ -431,6 +475,40 @@ def expand_playlist(url: str, limit: int) -> dict: } +def fetch_audio_tags(url: str) -> dict[str, str] | None: + """A link's tags from its metadata alone, for a track imported before tags + were read (#699). Nothing is downloaded. + + The URL goes through validate_youtube_url here as well as at the caller: + this is where the fetch happens, so this is where the SSRF boundary (#173) + has to hold. Raises on any failure; the caller decides what that means. + + Same cookie rule as an import (#432): without them first, with them only + once YouTube's bot check has turned the request away. _with_cookie_fallback + itself is not reused because it reports progress on a job's stage, and the + job here is finished. + """ + url = validate_youtube_url(url) + + def probe(use_cookies: bool) -> dict: + opts = { + **_base_ydl_opts(_ALLOWED_EXTRACTORS, use_cookies=use_cookies), + "noplaylist": True, + "skip_download": True, + } + with YoutubeDL(opts) as ydl: + return ydl.extract_info(url, download=False) or {} + + try: + info = probe(False) + except Exception as exc: + if not _is_bot_check(exc) or get_cookies_file() is None: + raise + logger.info("tag lookup hit YouTube's bot check; retrying with cookies") + info = probe(True) + return tags_from_ytdlp(info if isinstance(info, dict) else None) + + def _download_video_track(job: Job, url: str, job_dir: Path, *, use_cookies: bool = False) -> None: """Best-effort: download a video-only H.264/MP4 stream to video.mp4 for the MP4 export (issue #219). The audio source is downloaded separately as @@ -507,8 +585,12 @@ def download(job: Job, url: str, job_dir: Path) -> Path: # used to fail the whole job immediately (#279). def _probe(use_cookies: bool) -> dict: opts = {**_base_ydl_opts(_ALLOWED_EXTRACTORS, use_cookies=use_cookies), "noplaylist": True} - with YoutubeDL(opts) as ydl: - return ydl.extract_info(url, download=False) or {} + + def ask() -> dict: + with YoutubeDL(opts) as ydl: + return ydl.extract_info(url, download=False) or {} + + return _cancellable(job, ask) # The bot check lands on this first request, so this is where the cookie # fallback is decided. Whether it engaged is remembered below so the fetch @@ -575,6 +657,7 @@ def _fetch(use_cookies: bool) -> dict: seen: set[str] = set() deduped = [t for t in raw_tags if not (t in seen or seen.add(t))] # type: ignore[func-returns-value] _set(job, tags=deduped[:8] or None) + _set(job, audio_tags=tags_from_ytdlp(info)) # Best-effort: fetch the real video stream for the MP4 export. # Non-fatal -- on any failure the job proceeds audio-only. diff --git a/app/pipeline/fpcalc.py b/app/pipeline/fpcalc.py new file mode 100644 index 00000000..34853488 --- /dev/null +++ b/app/pipeline/fpcalc.py @@ -0,0 +1,66 @@ +"""fpcalc, Chromaprint's own fingerprinter, for an FFmpeg without chromaprint. + +Song identification fingerprints with FFmpeg's chromaprint muxer (identify.py). +The Windows (BtbN) and Docker (Debian) FFmpeg builds have it; the macOS +(shaka-project, evermeet) and Linux desktop (johnvansickle) builds do not. There +the desktop shell downloads fpcalc beside FFmpeg, and a source install can have +it from a distro package (libchromaprint-tools, chromaprint). + +fpcalc's default output is the same fingerprint the muxer prints: algorithm 2 +on its command line is the API's 1 (identify.CHROMAPRINT_ALGORITHM), compressed +and base64-encoded, so AcoustID takes either unchanged. The two were compared on +the same audio and came out byte for byte identical. + +Only the helpers live here; identify.py runs the process, so a fingerprint made +either way is stopped by the same cancel, timeout and release_source. +""" + +from __future__ import annotations + +import json +import shutil +import sys +from pathlib import Path + +from app.core.config import FPCALC_BIN, ffmpeg_executable + +FPCALC_NAME = "fpcalc.exe" if sys.platform.startswith("win") else "fpcalc" + + +def fpcalc_executable() -> str | None: + """The fpcalc to run, or None when there is none. + + FPCALC_BIN first (STEMDECK_FPCALC, else the data directory's ffmpeg folder, + where the desktop shell puts it), then beside the FFmpeg in use (setup may + have settled on one elsewhere, or data/ffmpeg/bin), then PATH. Evaluated + per call, so one downloaded while the backend runs is picked up.""" + candidates = [FPCALC_BIN] + ffmpeg = Path(ffmpeg_executable()) + if ffmpeg.is_file(): + candidates.append(ffmpeg.resolve().parent / FPCALC_NAME) + for candidate in candidates: + if candidate.is_file(): + return str(candidate) + return shutil.which("fpcalc") + + +def fpcalc_command(fpcalc: str, source: str, length: int) -> list[str]: + """fpcalc printing, as JSON, the fingerprint of the first ``length`` + seconds of ``source``: a file's absolute path, or "-" for audio on stdin.""" + return [fpcalc, "-json", "-length", str(length), source] + + +def parse_fpcalc_output(stdout: bytes | str | None) -> str | None: + """The fingerprint in fpcalc's JSON (``{"duration": .., "fingerprint": + ..}``), unchecked, or None when there is none. identify.parse_fingerprint + decides whether it is one.""" + if isinstance(stdout, bytes): + stdout = stdout.decode("utf-8", errors="replace") + try: + answer = json.loads(stdout or "") + except ValueError: + return None + if not isinstance(answer, dict): + return None + fingerprint = answer.get("fingerprint") + return fingerprint if isinstance(fingerprint, str) else None diff --git a/app/pipeline/identify.py b/app/pipeline/identify.py new file mode 100644 index 00000000..f0c72071 --- /dev/null +++ b/app/pipeline/identify.py @@ -0,0 +1,1121 @@ +"""Which recording a job is, found while it separates. + +Best source first, stopping at the first answer: + +1. AcoustID, when the user has set a key in Settings: an audio fingerprint of + the first FINGERPRINT_LENGTH_SEC seconds, made with FFmpeg's chromaprint + muxer (or fpcalc where this FFmpeg has none, see fpcalc.py), matched + against AcoustID's database. A match scoring at least + ACOUSTID_MIN_SCORE names a MusicBrainz recording, which MusicBrainz then + describes (credited title and artists, the album, its type). +2. A MusicBrainz recording search by the tags' artist and title (or the song's + name cleaned out of the video title), kept only when it is unmistakable: + see musicbrainz.search_recording. +3. The title, read every way it can be (title_parse.py: "Artist - Song", + 'Song (From "Work")', "Song Performers", "Work Cast - Song"...), each + reading searched for on MusicBrainz and kept only when a recording that + length credits the title's other words (musicbrainz.search_by_title); then + the artist a reading names, looked up by name and alias in any script, and + the song among that artist's recordings, a music video's longer length + allowed for (identify_by_artist). When MusicBrainz has none, LRCLIB is + asked the same way, and its artist and album name the track (source + "lrclib"). This is all a YouTube upload with no music metadata has, and a + second opinion when the tags named something MusicBrainz does not know. +4. The tags alone, as source "tags", when they name both an artist and a song. + +The answer is ``job.identity`` (clean_identity's shape). The band a job's +artist is (``job.artist``) is then found through it: the recording's first +credited artist's MusicBrainz id, that artist's Wikidata link, and the +Wikidata item naming the same MusicBrainz artist back. Without an identity +with artist ids, the band is searched for by name as before (artist_lookup), +then by the names the tag or the title give, on MusicBrainz (find_band_for): +a compilation matches no recording, and its artist box still shows the band. + +What is sent: to AcoustID a fingerprint (not audio) and the track's length; +to MusicBrainz, Wikidata and LRCLIB, ids, names from the track (artist, song, +album or show) and its length. Everything here is best-effort +and never raises into the pipeline: no key, no chromaprint in this FFmpeg and +no fpcalc, no connection or no match all leave the job with less, never failed. +""" + +from __future__ import annotations + +import json +import logging +import re +import subprocess +import threading +import time +import urllib.error +import urllib.parse +import urllib.request +from collections.abc import Callable +from pathlib import Path +from typing import Any + +from app.core.config import ( + ACOUSTID_LOOKUP_URL, + ACOUSTID_MIN_SCORE, + FINGERPRINT_LENGTH_SEC, + IDENTIFY_MAX_BYTES, + TIMEOUT_FINGERPRINT, + TIMEOUT_IDENTIFY_REQUEST, + TITLE_ARTIST_SEARCHES, + TITLE_LRCLIB_SEARCHES, + TITLE_MUSICBRAINZ_SEARCHES, + ffmpeg_executable, +) +from app.core.models import MBID_RE, Job, _set, clean_identity +from app.core.settings import get_acoustid_api_key +from app.pipeline import fpcalc, musicbrainz, ratelimit, title_parse +from app.pipeline.artist_lookup import ( + _ssl_context, + find_band, + lookup_band_by_id, + tagged_artist_name, +) +from app.pipeline.work_lookup import find_work, might_have_work + +logger = logging.getLogger("stemdeck.identify") + +# fpcalc's default algorithm. Chromaprint numbers its algorithms from 0 in +# its API (TEST1..TEST5) and FFmpeg passes the number straight through, so the +# muxer's 1 is CHROMAPRINT_ALGORITHM_TEST2, the one every fingerprint in +# AcoustID's database was made with. Its fingerprints start "AQ". +CHROMAPRINT_ALGORITHM = "1" +# A compressed, base64 fingerprint, as fpcalc prints and AcoustID takes it. +# URL-safe alphabet, no padding; two minutes come to a few kilobytes. +_FINGERPRINT_RE = re.compile(r"^[A-Za-z0-9_\-+/]{16,65536}$") +# FFmpeg builds without --enable-chromaprint (see the report in the PR): +# "Requested output format 'chromaprint' is not known" or, in older builds, +# "Unknown output format". +_NO_MUXER_RE = re.compile(r"output format.*chromaprint|chromaprint.*not known", re.IGNORECASE) + +# FFmpeg executables found to have no chromaprint muxer, so a machine without +# one does not spawn a doomed process for every import. +_NO_CHROMAPRINT: set[str] = set() +# The fingerprint processes running now, by job id, so the pipeline can make +# sure none still has the source open before it deletes it (release_source). +_RUNNING: dict[str, tuple[subprocess.Popen, threading.Event]] = {} +_RUNNING_LOCK = threading.Lock() + + +# ── the fingerprint ── + + +def fingerprint_command(ffmpeg: str, audio: list[Path], length: int) -> list[str]: + """The FFmpeg command that prints the fingerprint of the first ``length`` + seconds of ``audio``: one file, or several played together (the stems of + a track whose source is gone, which add back up to the mix).""" + return decode_command(ffmpeg, audio, length) + [ + "-f", + "chromaprint", + "-algorithm", + CHROMAPRINT_ALGORITHM, + "-fp_format", + "base64", + "-", + ] + + +def decode_command(ffmpeg: str, audio: list[Path], length: int) -> list[str]: + """fingerprint_command up to its output: the first ``length`` seconds of + ``audio``, summed when there are several. Also what feeds fpcalc stems, + which it cannot mix itself.""" + cmd = [ffmpeg, "-nostdin", "-hide_banner", "-loglevel", "error"] + for path in audio: + # An input option, so FFmpeg stops reading after ``length`` seconds + # instead of decoding the whole file and throwing the rest away. + cmd += ["-t", str(length), "-i", str(path)] + if len(audio) > 1: + # normalize=0: the stems are parts of one mix, so they are summed, + # not averaged. Chromaprint is level-independent either way. + inputs = "".join(f"[{i}:a]" for i in range(len(audio))) + cmd += [ + "-filter_complex", + f"{inputs}amix=inputs={len(audio)}:normalize=0[mix]", + "-map", + "[mix]", + ] + else: + cmd += ["-vn"] + return cmd + + +def parse_fingerprint(stdout: bytes | str | None) -> str | None: + """The fingerprint FFmpeg printed, or None when it printed none.""" + if isinstance(stdout, bytes): + stdout = stdout.decode("ascii", errors="replace") + text = (stdout or "").strip() + return text if _FINGERPRINT_RE.match(text) else None + + +def release_source(job_id: str, timeout: float = 5.0) -> None: + """Make sure no fingerprint of ``job_id`` still has its source open. + + Called by the pipeline before it deletes a link's source: on Windows a + file another process has open cannot be deleted, and that failure would + fail the job. A fingerprint takes about a second and the separation + before this minutes, so there is nearly always nothing to do; one still + running is stopped, and identification goes on without it.""" + with _RUNNING_LOCK: + entry = _RUNNING.get(job_id) + if entry is None: + return + proc, done = entry + if done.is_set(): + return + try: + proc.kill() + except OSError: + # Already exited between the check and the kill: nothing left open. + pass + done.wait(timeout) + + +def fingerprint( + audio: list[Path], + *, + job_id: str = "", + cancelled: Callable[[], bool] = lambda: False, +) -> str | None: + """The chromaprint fingerprint of ``audio``, or None. Never raises. + + Made by FFmpeg's chromaprint muxer, or by fpcalc once this FFmpeg is known + to have none (fpcalc.py). Neither: None, and the track is named by its + tags.""" + audio = [p for p in audio if p.is_file()] + if not audio or cancelled(): + return None + ffmpeg = ffmpeg_executable() + if ffmpeg in _NO_CHROMAPRINT: + return _fingerprint_with_fpcalc(ffmpeg, audio, job_id=job_id, cancelled=cancelled) + cmd = fingerprint_command(ffmpeg, audio, FINGERPRINT_LENGTH_SEC) + try: + proc = subprocess.Popen( + cmd, + stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + except OSError: + logger.info("could not start FFmpeg for a fingerprint", exc_info=True) + return None + output = _wait(proc, job_id=job_id, cancelled=cancelled) + if output is None: + return None + stdout, stderr = output + if proc.returncode != 0: + message = (stderr or b"").decode("utf-8", errors="replace") + if _NO_MUXER_RE.search(message): + _NO_CHROMAPRINT.add(ffmpeg) + logger.warning("this FFmpeg has no chromaprint muxer; fingerprinting with fpcalc") + return _fingerprint_with_fpcalc(ffmpeg, audio, job_id=job_id, cancelled=cancelled) + logger.info("fingerprint failed (exit %s): %s", proc.returncode, message[-300:]) + return None + return parse_fingerprint(stdout) + + +def _fingerprint_with_fpcalc( + ffmpeg: str, + audio: list[Path], + *, + job_id: str, + cancelled: Callable[[], bool], +) -> str | None: + """fingerprint() by fpcalc, for an FFmpeg without the muxer. One file is + read by fpcalc itself; stems are summed by FFmpeg (decode_command, which + needs no muxer) and piped to fpcalc as WAV. Never raises.""" + exe = fpcalc.fpcalc_executable() + if exe is None: + logger.info("no fpcalc either; tracks are identified by their tags only") + return None + upstream: subprocess.Popen | None = None + try: + if len(audio) == 1: + source = str(audio[0].absolute()) + stdin: Any = subprocess.DEVNULL + else: + upstream = subprocess.Popen( + decode_command(ffmpeg, audio, FINGERPRINT_LENGTH_SEC) + ["-f", "wav", "-"], + stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, + ) + source, stdin = "-", upstream.stdout + try: + proc = subprocess.Popen( + fpcalc.fpcalc_command(exe, source, FINGERPRINT_LENGTH_SEC), + stdin=stdin, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + finally: + # fpcalc holds its own copy; with none left here, FFmpeg sees a + # closed pipe, not a stall, if fpcalc goes first. + if upstream is not None and upstream.stdout is not None: + upstream.stdout.close() + except OSError: + if upstream is not None: + upstream.kill() + upstream.wait() + logger.info("could not start fpcalc for a fingerprint", exc_info=True) + return None + output = _wait(proc, job_id=job_id, cancelled=cancelled, upstream=upstream) + if output is None: + return None + stdout, stderr = output + if proc.returncode != 0: + message = (stderr or b"").decode("utf-8", errors="replace") + logger.info("fpcalc failed (exit %s): %s", proc.returncode, message[-300:]) + return None + return parse_fingerprint(fpcalc.parse_fpcalc_output(stdout)) + + +def _wait( + proc: subprocess.Popen, + *, + job_id: str, + cancelled: Callable[[], bool], + upstream: subprocess.Popen | None = None, +) -> tuple[bytes, bytes] | None: + """(stdout, stderr) of a fingerprint process once it exits, or None when + it was abandoned: cancelled, or still running after TIMEOUT_FINGERPRINT. + Registered for release_source meanwhile. ``upstream`` (FFmpeg feeding + fpcalc) is stopped with it, and reaped before release_source is told the + audio is closed.""" + done = threading.Event() + if job_id: + with _RUNNING_LOCK: + _RUNNING[job_id] = (proc, done) + try: + # Not registered with set_proc: that slot is the separation's, which + # runs at the same time. Cancel is polled here instead. + deadline = time.monotonic() + TIMEOUT_FINGERPRINT + while True: + try: + return proc.communicate(timeout=0.25) + except subprocess.TimeoutExpired: + if cancelled() or time.monotonic() > deadline: + proc.kill() + proc.communicate() + logger.info("fingerprint abandoned (cancelled or too slow)") + return None + finally: + if upstream is not None: + # Done with its output either way. Normally it has already exited + # (-t bounds it); after a kill of fpcalc it would only die of the + # closed pipe a moment later. + if upstream.poll() is None: + upstream.kill() + upstream.wait() + done.set() + if job_id: + with _RUNNING_LOCK: + if _RUNNING.get(job_id, (None,))[0] is proc: + _RUNNING.pop(job_id, None) + + +# ── AcoustID ── + + +class AcoustIDRefused(OSError): + """A lookup AcoustID refused, with its error code: 4 is a key it does not + know, 3 a fingerprint it cannot read.""" + + def __init__(self, code: int | None, message: str) -> None: + super().__init__(message) + self.code = code + + +# AcoustID's error code for a key it does not accept. +_ACOUSTID_BAD_KEY = 4 + + +def _acoustid_request(form: dict[str, str]) -> Any: + """One AcoustID lookup, after waiting for its turn. Raises on any failure. + + POSTed rather than put in a URL: the fingerprint is kilobytes long, and + the key then never appears in a URL that something might log. An error + AcoustID explains (a bad key, say) is raised with its message, which + never contains the key.""" + ratelimit.ACOUSTID.wait() + body = urllib.parse.urlencode(form).encode("ascii") + request = urllib.request.Request( + ACOUSTID_LOOKUP_URL, + data=body, + headers={ + "Content-Type": "application/x-www-form-urlencoded", + "Accept": "application/json", + "User-Agent": musicbrainz.MUSICBRAINZ_USER_AGENT, + }, + ) + try: + # A fixed https URL, never one from a request or a tag (B310). + with urllib.request.urlopen( # nosec B310 + request, timeout=TIMEOUT_IDENTIFY_REQUEST, context=_ssl_context() + ) as response: + answer = response.read(IDENTIFY_MAX_BYTES + 1) + except urllib.error.HTTPError as err: + # AcoustID answers a refused lookup with 400 and a JSON reason. + try: + error = json.loads(err.read(64 * 1024)).get("error", {}) + reason, code = error.get("message"), error.get("code") + except Exception: + reason, code = None, None + raise AcoustIDRefused( + code if isinstance(code, int) else None, + f"AcoustID refused the lookup ({err.code}): {reason or 'no reason'}", + ) from None + if len(answer) > IDENTIFY_MAX_BYTES: + raise ValueError("AcoustID answer too large") + return json.loads(answer) + + +def acoustid_key_works(key: str) -> bool | None: + """Whether AcoustID accepts ``key`` for lookups: True or False, or None + when it cannot be asked (offline, AcoustID down or too slow), and the key + is then kept on trust. + + Asked with a fingerprint that is not one: a key AcoustID knows is refused + for the fingerprint, one it does not for the key. The key AcoustID's site + shows on a user's profile is for submitting fingerprints, not for looking + them up, and is the one people paste; this is what tells them. + + The Save waits for this, so it waits at most twice the request timeout. + The request's own timeout bounds each read, not the whole answer: a + server sending a byte at a time would otherwise hold the Save for as long + as it went on. The request is left to finish on its own thread.""" + result: list[bool | None] = [None] + + def ask() -> None: + result[0] = _key_verdict(key) + + worker = threading.Thread(target=ask, name="acoustid-key-check", daemon=True) + worker.start() + worker.join(TIMEOUT_IDENTIFY_REQUEST * 2) + if worker.is_alive(): + logger.info("AcoustID took too long to say whether the key works") + return None + return result[0] + + +def _key_verdict(key: str) -> bool | None: + form = {"client": key, "duration": "30", "fingerprint": "AQAA", "format": "json"} + try: + answer = _acoustid_request(form) + except AcoustIDRefused as err: + # A refusal with no reason (a 503, say) says nothing about the key. + return None if err.code is None else err.code != _ACOUSTID_BAD_KEY + except Exception: + logger.info("AcoustID could not be asked whether the key works", exc_info=True) + return None + error = answer.get("error") if isinstance(answer, dict) else None + if isinstance(error, dict): + return error.get("code") != _ACOUSTID_BAD_KEY + return True + + +def best_acoustid_match( + answer: Any, duration: float | None, min_score: float = ACOUSTID_MIN_SCORE +) -> tuple[float, dict[str, Any]] | None: + """(score, recording) for the best AcoustID result scoring at least + ``min_score`` that is linked to a MusicBrainz recording, or None. + + A fingerprint can match several recordings (the same audio released + twice, merged duplicates): the one named in full whose length is nearest + the track's is taken.""" + if not isinstance(answer, dict) or answer.get("status") != "ok": + return None + results = answer.get("results") + ranked = sorted( + ( + r + for r in (results if isinstance(results, list) else []) + if isinstance(r, dict) and isinstance(r.get("score"), (int, float)) + ), + key=lambda r: -r["score"], + ) + for result in ranked: + score = float(result["score"]) + if score < min_score: + return None + recordings = [ + rec + for rec in (result.get("recordings") or []) + if isinstance(rec, dict) and isinstance(rec.get("id"), str) and MBID_RE.match(rec["id"]) + ] + if not recordings: + continue + + def rank(rec: dict[str, Any]) -> tuple: + named = bool(rec.get("title")) and bool(rec.get("artists")) + length = rec.get("duration") + off = ( + abs(float(length) - duration) + if duration and isinstance(length, (int, float)) and not isinstance(length, bool) + else 0.0 + ) + return (not named, off) + + return score, min(recordings, key=rank) + return None + + +def _recording_from_acoustid(rec: dict[str, Any]) -> dict[str, Any]: + """AcoustID's own description of a recording, in MusicBrainz's shape, for + when MusicBrainz cannot be asked.""" + credit = [] + for artist in rec.get("artists") or []: + if isinstance(artist, dict) and isinstance(artist.get("name"), str): + credit.append( + { + "name": artist["name"], + "joinphrase": artist.get("joinphrase") or "", + "artist": {"id": artist.get("id"), "name": artist["name"]}, + } + ) + releases = [] + for group in rec.get("releasegroups") or []: + if isinstance(group, dict): + releases.append( + { + "release-group": { + "id": group.get("id"), + "title": group.get("title"), + "primary-type": group.get("type"), + "secondary-types": group.get("secondarytypes") or [], + } + } + ) + duration = rec.get("duration") + return { + "id": rec.get("id"), + "title": rec.get("title"), + "length": duration * 1000 if isinstance(duration, (int, float)) else None, + "artist-credit": credit, + "releases": releases, + } + + +def identify_by_fingerprint( + audio: list[Path], + duration: float | None, + api_key: str, + *, + job_id: str = "", + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The identity AcoustID gives ``audio``, or None. Raises on a failed + request; fingerprinting itself never raises.""" + if not api_key or not duration or duration <= 0: + return None + print_ = fingerprint(audio, job_id=job_id, cancelled=cancelled) + if not print_ or cancelled(): + return None + answer = _acoustid_request( + { + "client": api_key, + "format": "json", + "meta": "recordings releasegroups compress", + "duration": str(round(duration)), + "fingerprint": print_, + } + ) + match = best_acoustid_match(answer, duration) + if match is None: + logger.info("[%s] AcoustID knows no recording for this fingerprint", job_id) + return None + score, rec = match + identity = None + if not cancelled(): + try: + recording = musicbrainz.lookup_recording(rec["id"], cancelled=cancelled) + identity = musicbrainz.identity_from_recording( + recording, source="acoustid", score=score, fallback_duration=duration + ) + except Exception: + logger.info("[%s] MusicBrainz lookup failed; using AcoustID's", job_id, exc_info=True) + return identity or musicbrainz.identity_from_recording( + _recording_from_acoustid(rec), source="acoustid", score=score, fallback_duration=duration + ) + + +# ── the title ── + + +def identify_by_title( + *, + tags: dict[str, str] | None, + title: str | None, + duration: float | None, + job_id: str = "", + lrclib: bool = True, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The identity a track's title (and its tags' title) names, or None. + + Each reading of the title (title_parse.readings_for) is searched for on + MusicBrainz, best first, up to TITLE_MUSICBRAINZ_SEARCHES; the first + confident answer wins (musicbrainz.best_title_match). When MusicBrainz has + none, or cannot be reached, LRCLIB is asked as an independent check, up + to TITLE_LRCLIB_SEARCHES readings (lyrics_lookup.identify_on_lrclib), + unless ``lrclib`` is False: tags naming artist and song already give the + lyrics lookup its names. Nothing without the track's length: a name alone + can be anybody's song. Never raises.""" + # Imported here: lyrics_lookup is the newer module and may come to + # depend on this one. + from app.pipeline.lyrics_lookup import identify_on_lrclib + + if not duration or duration <= 0: + return None + readings = title_parse.readings_for(tags, title) + reachable = True + for reading in readings[:TITLE_MUSICBRAINZ_SEARCHES]: + if cancelled(): + return None + try: + identity = musicbrainz.search_by_title(reading, duration, cancelled=cancelled) + except Exception: + # Down, or the rate limit's queue too long: no point asking again. + logger.info("[%s] MusicBrainz title search failed", job_id, exc_info=True) + reachable = False + break + if identity: + return identity + if reachable and not cancelled(): + try: + identity = identify_by_artist(readings, title, duration, cancelled=cancelled) + except Exception: + logger.info("[%s] MusicBrainz artist search failed", job_id, exc_info=True) + identity = None + if identity: + return identity + for reading in readings[:TITLE_LRCLIB_SEARCHES] if lrclib else []: + if cancelled(): + return None + try: + identity = identify_on_lrclib(reading, duration, cancelled=cancelled) + except Exception: + logger.info("[%s] LRCLIB title search failed", job_id, exc_info=True) + break + if identity: + return identity + return None + + +def identify_by_artist( + readings: list[title_parse.TitleReading], + title: str | None, + duration: float, + *, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The identity found by looking the artist up first, or None. + + For when no recording credits the title's words as they are written: the + title names 周杰倫 as "Jay Chou", 鄧麗君 as "邓丽君", or its video is longer + than the album cut. Each reading's artist is searched for by name and + alias (musicbrainz.search_artists), then the song among that artist's + recordings (musicbrainz.search_by_artist), for up to + TITLE_ARTIST_SEARCHES readings. A title with nothing else to read in it, + no separator at all ("美空ひばり 川の流れのように"), is cut every way it can + be, and each cut that names an artist is tried, the longest name first. + Raises when MusicBrainz cannot be reached.""" + tried: set[tuple[str, ...]] = set() + for reading in readings: + if len(tried) >= TITLE_ARTIST_SEARCHES or cancelled(): + break + names = title_parse.artist_names(reading) + key = tuple(title_parse.name_key(n) for n in names) + if not names or key in tried: + continue + tried.add(key) + artists = musicbrainz.search_artists(names, cancelled=cancelled) + ids = [a["id"] for a in artists] + if ids: + identity = musicbrainz.search_by_artist( + title_parse.song_names(reading), ids, duration, cancelled=cancelled + ) + if identity: + return identity + if readings: + return None + cuts = title_parse.name_splits(title) + if not cuts or cancelled(): + return None + artists = musicbrainz.search_artists([a for a, _ in cuts], cancelled=cancelled) + by_name: dict[str, list[str]] = {} + for artist in artists: + by_name.setdefault(title_parse.name_key(artist["matched"]), []).append(artist["id"]) + songs = {title_parse.name_key(a): s for a, s in cuts} + for name in sorted(by_name, key=len, reverse=True)[:TITLE_ARTIST_SEARCHES]: + if cancelled(): + return None + identity = musicbrainz.search_by_artist( + [songs[name]], by_name[name], duration, cancelled=cancelled + ) + if identity: + return identity + return None + + +def can_identify_title(tags: dict[str, str] | None, title: str | None) -> bool: + """Whether identify_by_title has a reading to go on: one naming more than + a song, or a title an artist's name could be cut from; or, for the band + alone, a compilation's title (band_from_title).""" + return ( + bool(title_parse.readings_for(tags, title)) + or bool(title_parse.name_splits(title)) + or title_parse.title_names(title)[1] + ) + + +# ── the whole answer ── + + +def identify( + *, + tags: dict[str, str] | None, + title: str | None, + duration: float | None, + audio: list[Path], + api_key: str | None, + job_id: str = "", + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The identity of a track, best source first (see the module), or None + when nothing names it, with the other titles its song goes by + (title_aliases). Never raises.""" + identity = _identify( + tags=tags, + title=title, + duration=duration, + audio=audio, + api_key=api_key, + job_id=job_id, + cancelled=cancelled, + ) + if identity is None or cancelled(): + return identity + aliases = title_aliases(identity, tags, title, cancelled=cancelled) + return clean_identity({**identity, "title_aliases": aliases}) if aliases else identity + + +def title_aliases( + identity: dict[str, Any], + tags: dict[str, str] | None, + title: str | None, + *, + cancelled: Callable[[], bool] = lambda: False, +) -> list[str]: + """The other titles an identified song goes by, for the lyrics lookup: + LRCLIB files 좋은 날 as "Good Day" as often as not. The upload's own + ("IU _ Good Day (좋은 날)": whichever of its readings names the song the + identity does), then, for a song in Chinese, Japanese or Korean, where + they matter, its MusicBrainz work's title and aliases. Never raises.""" + song = title_parse.song_key(identity.get("title")) + found: list[str] = [] + for reading in [*title_parse.readings_for(tags, title), *title_parse.parse_title(title)]: + names = title_parse.song_names(reading) + if song in {title_parse.song_key(n) for n in names}: + found += names + recording = identity.get("recording_mbid") + if recording and title_parse.is_east_asian(str(identity.get("title") or "")): + try: + found += musicbrainz.work_titles(recording, cancelled=cancelled) + except Exception: + logger.info("work titles lookup failed", exc_info=True) + aliases: list[str] = [] + keys = {song} + for name in found: + key = title_parse.song_key(name) + if key and key not in keys: + keys.add(key) + aliases.append(name.strip()) + return aliases + + +def _identify( + *, + tags: dict[str, str] | None, + title: str | None, + duration: float | None, + audio: list[Path], + api_key: str | None, + job_id: str = "", + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + # Imported here: lyrics_lookup is the newer module and may come to + # depend on this one. + from app.pipeline.lyrics_lookup import song_from_title + + tags = tags or {} + if api_key and audio: + try: + identity = identify_by_fingerprint( + audio, duration, api_key, job_id=job_id, cancelled=cancelled + ) + if identity: + return identity + except Exception: + logger.info("[%s] AcoustID lookup failed", job_id, exc_info=True) + if cancelled(): + return None + artist = tagged_artist_name(tags.get("artist")) + song = song_from_title(tags.get("title"), artist) or (tags.get("title") or "").strip() + if song and artist: + # A song in marks or among East Asian noise ("「群青」Official Music + # Video", a channel's title less its name): the title's own reading. + readings = title_parse.readings_for({"artist": artist, "title": song}) + if readings and title_parse.name_key(readings[0].artist) in { + title_parse.name_key(n) for n in title_parse.name_alternatives(artist) + }: + song = readings[0].song + if not song and artist: + song = song_from_title(title, artist) + if artist and song: + try: + identity = musicbrainz.search_recording(artist, song, duration, cancelled=cancelled) + if identity: + return identity + except Exception: + logger.info("[%s] MusicBrainz search failed", job_id, exc_info=True) + # The title: all a YouTube upload of a cast recording has, and a second + # opinion when the tags named something MusicBrainz does not know. + if cancelled(): + return None + identity = identify_by_title( + tags=tags, + title=title, + duration=duration, + job_id=job_id, + lrclib=not (artist and song), + cancelled=cancelled, + ) + if identity or not artist or not song: + return identity + return clean_identity( + { + "source": "tags", + # Nothing checked it: the file says so, and that is all. + "score": 0.0, + "title": song, + "artist": tags.get("artist") or artist, + "album": tags.get("album"), + "duration": duration, + } + ) + + +def find_band_for( + identity: dict[str, Any] | None, + artist_tag: Any, + *, + title: str | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """The band for a track, or None. Never raises. + + Through the identity's first credited artist (MusicBrainz artist -> its + Wikidata link -> the item naming it back), else by searching Wikidata for + the tag's artist name, or the identity's, with the name search's own + strict rule. When that finds nothing, or there is no name at all but a + ``title`` (a compilation, a live cut, a fan's upload of another length: + nothing a recording search could match), the names the tag or the title + give ("鄧麗君 Teresa Teng テレサ・テン", "邓丽君经典金曲 - Teresa Teng's + classic songs") are looked up on MusicBrainz by name and alias, and the + one artist they name, when it is only one, is the band.""" + if identity and identity.get("source") != "tags" and identity.get("artist_mbids"): + band = _band_by_artist_id( + identity["artist_mbids"][0], + tagged_artist_name(identity.get("artist")), + cancelled=cancelled, + ) + if band: + return band + if cancelled(): + return None + name = tagged_artist_name(artist_tag) or tagged_artist_name((identity or {}).get("artist")) + if name: + band = find_band(name, cancelled=cancelled) + if band: + return band + return _band_by_other_names(name, cancelled=cancelled) + return band_from_title(title, cancelled=cancelled) if title else None + + +def _band_by_other_names(name: str, *, cancelled: Callable[[], bool]) -> dict[str, str] | None: + """The band an artist tag or credit names by one of its other names + ("鄧麗君 Teresa Teng テレサ・テン" is three), found on MusicBrainz by name + and alias when only one artist is that likely, else on Wikidata by each + of the first two. Never raises.""" + names = title_parse.name_alternatives(name) + if len(names) < 2 or cancelled(): + return None + try: + artists = musicbrainz.search_artists(names, cancelled=cancelled) + except Exception: + logger.info("band lookup by MusicBrainz name failed", exc_info=True) + artists = [] + artist = _only_artist(artists) + if artist: + band = _band_by_artist_id(artist["id"], artist["name"], cancelled=cancelled) + if band: + return band + # Wikidata by each other name, as the tag's own name was: its label may + # be in either script ("周杰倫 Jay Chou" is neither label whole). + for other in names[1:3]: + if cancelled(): + return None + band = find_band(other, cancelled=cancelled) + if band: + return band + return None + + +def band_from_title( + title: str | None, *, cancelled: Callable[[], bool] = lambda: False +) -> dict[str, str] | None: + """The band a title names with nothing else to go by (no recording + found, no artist tag), or None. Never raises. + + Only on evidence a title cannot give by chance, since a wrong band is + worse than none ("Metropolis - Part I" once made Metropolis the band): + + * two names in different scripts that MusicBrainz knows, exactly (by + name, sort name or alias, never a near miss), as one artist and no + other: "邓丽君经典金曲 - Teresa Teng's classic songs", "周杰倫 Jay Chou"; + * or a compilation's title (its words taken off: "经典金曲", "Greatest + Hits") every part of which names that one artist and nobody else. + + That artist's Wikidata item, which must name the same MusicBrainz artist + back (and so be a musician or a band), is the band, as for a recording.""" + groups, compilation = title_parse.title_names(title) + names = [n for group in groups for n in group] + if not names or cancelled(): + return None + try: + artists = musicbrainz.search_artists(names, cancelled=cancelled) + except Exception: + logger.info("band lookup by MusicBrainz name failed", exc_info=True) + return None + ids_by_name: dict[str, set[str]] = {} + for artist in artists: + for matched in artist["matches"]: + ids_by_name.setdefault(title_parse.name_key(matched), set()).add(artist["id"]) + + def ids(name: str) -> set[str]: + return ids_by_name.get(title_parse.name_key(name), set()) + + chosen: set[str] = set() + # Two scripts: each artist id with the scripts of the names naming it. + scripts: dict[str, set[str]] = {} + for name in names: + for mbid in ids(name): + scripts.setdefault(mbid, set()).add(title_parse.script_of(name)) + chosen = {mbid for mbid, seen in scripts.items() if len(seen) > 1} + if not chosen and compilation: + # Every part names the artist: the one id all of them share. + per_part = [set().union(*(ids(n) for n in group)) for group in groups] + if all(per_part): + chosen = set.intersection(*per_part) + if len(chosen) != 1 or cancelled(): + return None + mbid = chosen.pop() + name = next(a["name"] for a in artists if a["id"] == mbid) + return _band_by_artist_id(mbid, name, cancelled=cancelled) + + +def _band_by_artist_id( + mbid: str, name: str, *, cancelled: Callable[[], bool] +) -> dict[str, str] | None: + """The band a MusicBrainz artist is on Wikidata, or None. Never raises.""" + try: + qid = musicbrainz.artist_wikidata_id(mbid, cancelled=cancelled) + if qid and not cancelled(): + return lookup_band_by_id(qid, mbid, name=name, cancelled=cancelled) + except Exception: + logger.info("band lookup by MusicBrainz id failed", exc_info=True) + return None + + +# Two artists matching the same name this close in MusicBrainz's score are +# as likely as each other: neither is taken for the band without a recording +# to say which ("BTS" the group scores 100, the next "BTS" 68). +_ARTIST_SCORE_MARGIN = 10 + + +def _only_artist(artists: list[dict[str, Any]]) -> dict[str, Any] | None: + """The one artist a name search found for the first name that matched + any, or None when none did or two are as likely.""" + if not artists: + return None + first = artists[0] + same = [a for a in artists if a["matched"] == first["matched"]] + if len(same) > 1 and same[0]["score"] - same[1]["score"] < _ARTIST_SCORE_MARGIN: + return None + return first + + +def can_identify( + tags: dict[str, str] | None, + api_key: str | None, + audio: list[Path], + title: str | None = None, +) -> bool: + """Whether identify() has anything to go on: a key and audio to + fingerprint, an artist tag, or a title naming more than a song.""" + return ( + bool(api_key and audio) + or bool(tagged_artist_name((tags or {}).get("artist"))) + or can_identify_title(tags, title) + ) + + +def identify_and_find_band( + job: Job, + audio: list[Path], + *, + tags: dict[str, str] | None, + want_band: bool, + cancelled: Callable[[], bool] = lambda: False, +) -> tuple[dict[str, Any] | None, dict[str, str] | None]: + """(identity, band) for a job, blocking: the job's identity when it has + one, else a fresh one; the band only when ``want_band``. Reads the job, + never writes it. For the tag backfill. Never raises.""" + identity = job.identity + if identity is None and can_identify(tags, get_acoustid_api_key(), audio, job.title): + identity = identify( + tags=tags, + title=job.title, + duration=job.duration_sec, + audio=audio, + api_key=get_acoustid_api_key(), + job_id=job.id, + cancelled=cancelled, + ) + band = None + if want_band and not cancelled(): + band = find_band_for( + identity, (tags or {}).get("artist"), title=job.title, cancelled=cancelled + ) + return identity, band + + +class IdentifyLookup: + """One job's identification and band lookup, in a thread of its own, so + separation never waits for it. BandLookup's contract, which this replaces + in the pipeline: the thread never touches the job, and finish() writes the + answer from the pipeline's own thread once the pipeline has got that far + uncancelled. A job cancelled or failed meanwhile is never written to. + + The identity is published the moment it is known (wait_identity), before + the band is looked up through it, for anything else running beside + separation that needs it (the lyrics lookup). Then the band, then the work + a soundtrack or cast recording is from (work_lookup.py), each kept by + finish() once its own step is done. + """ + + def __init__( + self, + job: Job, + audio: list[Path], + api_key: str | None, + want_band: bool, + want_work: bool = False, + ) -> None: + self._identity: dict[str, Any] | None = None + self._band: dict[str, str] | None = None + self._work: dict[str, str] | None = None + self._identity_ready = threading.Event() + self._band_done = threading.Event() + self._work_done = threading.Event() + self._thread = threading.Thread( + target=self._run, + args=(job, list(audio), api_key, want_band, want_work), + name=f"identify-{job.id}", + daemon=True, + ) + + def _run( + self, job: Job, audio: list[Path], api_key: str | None, want_band: bool, want_work: bool + ) -> None: + tags = dict(job.audio_tags or {}) + cancelled = lambda: job.cancel_requested # noqa: E731 + try: + identity = job.identity or identify( + tags=tags, + title=job.title, + duration=job.duration_sec, + audio=audio, + api_key=api_key, + job_id=job.id, + cancelled=cancelled, + ) + except Exception: + # identify() never raises; this is for a future change that does. + logger.info("[%s] identification failed", job.id, exc_info=True) + identity = job.identity + self._identity = identity + self._identity_ready.set() + if want_band and not cancelled(): + self._band = find_band_for( + identity, tags.get("artist"), title=job.title, cancelled=cancelled + ) + self._band_done.set() + if want_work and not cancelled(): + self._work = find_work(identity, tags, title=job.title, cancelled=cancelled) + self._work_done.set() + + @classmethod + def start(cls, job: Job, audio: list[Path]) -> IdentifyLookup | None: + """Start, or None when there is nothing to find: the identity and the + band already known (a re-split inherits both), or nothing to go on + (no key to fingerprint with, no artist tag and no title naming more + than a song).""" + api_key = get_acoustid_api_key() + tags = job.audio_tags or {} + want_identity = job.identity is None and can_identify(tags, api_key, audio, job.title) + has_name = bool( + tagged_artist_name(tags.get("artist")) or (job.identity or {}).get("artist") + ) + want_band = job.artist is None and (want_identity or has_name) + # The work behind a soundtrack: known only once the identity is, so + # asked for whenever an identity is being found, or one already known + # (or the album tag) says the recording is a soundtrack. + want_work = job.work is None and ( + want_identity or might_have_work(job.identity, tags, job.title) + ) + if not want_identity and not want_band and not want_work: + return None + lookup = cls(job, audio if api_key else [], api_key, want_band, want_work) + lookup._thread.start() + return lookup + + def wait_identity(self, timeout: float) -> dict[str, Any] | None: + """The identity, waiting up to ``timeout`` seconds for it: None when + there is none or it is not known yet. Safe from any thread.""" + self._identity_ready.wait(timeout) + return self._identity + + def finish(self, job: Job, timeout: float) -> None: + """Wait up to ``timeout`` seconds, then keep what was found on the job, + unless the job was cancelled meanwhile. An identity found in time is + kept even when the band behind it is still out, and a band even when + the work is.""" + self._thread.join(timeout) + alive = self._thread.is_alive() + if alive: + logger.info("[%s] identification still out after %ss; finishing", job.id, timeout) + if job.cancel_requested: + return + fields: dict[str, object] = {} + if self._identity_ready.is_set() and self._identity and job.identity is None: + fields["identity"] = self._identity + if self._band_done.is_set() and self._band and job.artist is None: + fields["artist"] = self._band + if self._work_done.is_set() and self._work and job.work is None: + fields["work"] = self._work + if fields: + _set(job, **fields) diff --git a/app/pipeline/lyrics_align.py b/app/pipeline/lyrics_align.py new file mode 100644 index 00000000..fd63a35a --- /dev/null +++ b/app/pipeline/lyrics_align.py @@ -0,0 +1,269 @@ +"""Moving LRCLIB's timing onto this track, for lyrics from a version of +another length (lyrics_lookup.py saves those with "timing": "unverified"), +and for a version of the same length whose lines the vocals show start at +least LYRICS_ALIGN_EXACT_MIN_SHIFT_SEC away: a copy the track's length can +still be timed to another cut, and its length alone proved nothing. + +A version of a song a few seconds longer or shorter than the track is most +often the same recording with more or less lead-in: a video's intro, a +remaster's trimmed silence, a single edit. The words are right and every line +is off by the same amount. That amount is found here without any model, from +the separated vocals stem's level over time (the envelope the Lyrics tab's +wipe follows): how sharply the voice rises at each moment is summed at the +moments the LRC's lines start, for every shift within +LYRICS_ALIGN_MAX_OFFSET_SEC, and the best shift is taken when it stands out +from the rest (LYRICS_ALIGN_MIN_Z) and beats every other shift a second or +more away (LYRICS_ALIGN_MIN_RATIO): a song's repeated rhythm lines up other +shifts too, and a different performance of the song lines up none clearly. +The stamps are rewritten by it and the lyrics marked "shifted". A fit that is +not clear leaves them as they were, still "unverified": wrong timing moved is +worse than wrong timing kept. + +Cheap CPU work: one streamed read of the vocals stem, or none when the +envelope is already kept beside it, and a few thousand candidate shifts. +""" + +from __future__ import annotations + +import json +import logging +import re +from pathlib import Path +from typing import Any + +import numpy as np +from numpy.lib.stride_tricks import sliding_window_view + +from app.core.config import ( + LYRICS_ALIGN_EXACT_MIN_SHIFT_SEC, + LYRICS_ALIGN_MAX_OFFSET_SEC, + LYRICS_ALIGN_MIN_HITS, + LYRICS_ALIGN_MIN_LINES, + LYRICS_ALIGN_MIN_RATIO, + LYRICS_ALIGN_MIN_Z, + LYRICS_ALIGN_TOLERANCE_SEC, +) +from app.core.models import Job +from app.pipeline.audio_stats import vocal_envelope + +logger = logging.getLogger("stemdeck.lyrics") + +# Kept beside the stems by GET .../vocal-envelope (app/api/stems.py). +_ENVELOPE_FILE = "vocal_envelope.json" +_ENVELOPE_HOP_SEC = 0.04 + +# What counts as singing, as the tab's wipe has it (lyricsLookup.js +# voicedPhrases): within 20 dB of the stem's loud parts, never below -50 dBFS, +# and nothing at all when its loud parts are under -45 dBFS. Its loud parts +# are its 95th percentile rather than the wipe's 90th: over a whole song the +# voice can be silent most of the time, where within one line it is not. +_VOICE_RANGE_DB = 20 +_VOICE_FLOOR_DB = -50 +_SILENT_DB = -45 +_LOUD_PERCENTILE = 95 +# A rise is measured over this many levels (120 ms at the envelope's 40 ms), +# of the level smoothed over three: a syllable's attack, not a flicker. +_RISE_FRAMES = 3 +# A line counts as starting on singing when the voice rises this much near it. +_HIT_RISE_DB = 6 +# Shifts this close to the best are the same answer, not a rival one. +_SAME_SHIFT_SEC = 1.0 + +_OFFSET_TAG = re.compile(r"^\s*\[offset:\s*([+-]?\d+)\s*\]\s*$", re.IGNORECASE) +_LINE_STAMPS = re.compile(r"^(?:\s*\[\d{1,3}:\d{1,2}(?:[.:]\d{1,3})?\])+") +_STAMP = re.compile(r"([\[<])(\d{1,3}):(\d{1,2}(?:[.:]\d{1,3})?)([\]>])") + + +def _seconds(minutes: str, rest: str) -> float: + return int(minutes) * 60 + float(rest.replace(":", ".")) + + +def _stamp(seconds: float) -> str: + centis = round(max(0.0, seconds) * 100) + return f"{centis // 6000:02d}:{(centis % 6000) / 100:05.2f}" + + +def _offset(text: str) -> float: + """The LRC's own [offset:] tag, in seconds earlier (parseLrc's rule).""" + for line in text.splitlines(): + tag = _OFFSET_TAG.match(line) + if tag: + return int(tag.group(1)) / 1000 + return 0.0 + + +def line_starts(text: str) -> list[float]: + """When each sung line starts, in seconds, earliest first: every stamp of + every line with words, the [offset:] tag applied. Lines with no words + mark an instrumental gap and start nothing.""" + offset = _offset(text) + starts = [] + for line in text.splitlines(): + stamps = _LINE_STAMPS.match(line) + if not stamps: + continue + # [^<>]: an unclosed "<" cannot make this quadratic. + words = re.sub(r"<[^<>]*>", "", line[stamps.end() :]).strip() + if not words: + continue + for m in _STAMP.finditer(stamps.group(0)): + starts.append(max(0.0, _seconds(m.group(2), m.group(3)) - offset)) + return sorted(starts) + + +def shift_lrc(text: str, seconds: float) -> str: + """``text`` with every line and word stamp ``seconds`` later (earlier + when negative), never before zero. Its [offset:] tag is folded into the + stamps and dropped, so the stamps alone say when.""" + delta = seconds - _offset(text) + lines = [line for line in text.splitlines() if not _OFFSET_TAG.match(line)] + + def move(m: re.Match[str]) -> str: + return f"{m.group(1)}{_stamp(_seconds(m.group(2), m.group(3)) + delta)}{m.group(4)}" + + return "\n".join(_STAMP.sub(move, line) for line in lines) + + +def onset_strength(db: Any) -> np.ndarray: + """How sharply singing starts at each level of an envelope: the rise of + the smoothed level over _RISE_FRAMES, where the voice is singing, else 0. + All zeros for a stem with no singing.""" + levels = np.asarray(db, dtype=float) + if levels.size < _RISE_FRAMES + 1: + return np.zeros(levels.size) + loud = float(np.percentile(levels, _LOUD_PERCENTILE)) + if loud < _SILENT_DB: + return np.zeros(levels.size) + smooth = np.convolve(levels, np.ones(3) / 3, mode="same") + rise = np.zeros(levels.size) + rise[_RISE_FRAMES:] = smooth[_RISE_FRAMES:] - smooth[:-_RISE_FRAMES] + rise[smooth < max(loud - _VOICE_RANGE_DB, _VOICE_FLOOR_DB)] = 0 + return np.clip(rise, 0, None) + + +def estimate_offset(starts: list[float], strength: np.ndarray, hop: float) -> float | None: + """The shift, in seconds, that puts the LRC's line starts where the + singing starts (``strength``, onset_strength of an envelope ``hop`` + seconds a level), or None when no shift clearly does. + + Each shift within +-LYRICS_ALIGN_MAX_OFFSET_SEC scores the strongest + rise within LYRICS_ALIGN_TOLERANCE_SEC of each moved line start, summed. + The best is kept when it is LYRICS_ALIGN_MIN_Z standard deviations above + the mean of all shifts, LYRICS_ALIGN_MIN_RATIO times the best shift a + second or more away, and puts LYRICS_ALIGN_MIN_HITS of the lines on a + rise of _HIT_RISE_DB or more: scattered singing lines a few up by chance.""" + lines = np.asarray(starts, dtype=float) + if lines.size < LYRICS_ALIGN_MIN_LINES or hop <= 0 or not strength.any(): + return None + k = max(1, round(LYRICS_ALIGN_TOLERANCE_SEC / hop)) + nearby = sliding_window_view(np.pad(strength, (k, k)), 2 * k + 1).max(axis=1) + shifts = np.arange(-LYRICS_ALIGN_MAX_OFFSET_SEC, LYRICS_ALIGN_MAX_OFFSET_SEC + hop / 2, hop) + frames = np.rint((lines[None, :] + shifts[:, None]) / hop).astype(int) + inside = (frames >= 0) & (frames < strength.size) + rises = np.where(inside, nearby[np.clip(frames, 0, strength.size - 1)], 0.0) + score = rises.sum(axis=1) + + pick = int(np.argmax(score)) + spread = float(score.std()) + far = np.abs(shifts - shifts[pick]) > _SAME_SHIFT_SEC + rival = float(score[far].max()) if far.any() else 0.0 + z = (score[pick] - score.mean()) / spread if spread > 0 else 0.0 + ratio = score[pick] / rival if rival > 0 else float("inf") + hits = float((rises[pick] >= _HIT_RISE_DB).mean()) + if z < LYRICS_ALIGN_MIN_Z or ratio < LYRICS_ALIGN_MIN_RATIO or hits < LYRICS_ALIGN_MIN_HITS: + logger.info( + "lyrics timing not confident: best %+.2fs, z %.1f, %.2f times the next, %.0f%% on", + shifts[pick], + z, + ratio, + hits * 100, + ) + return None + # Finer than the grid: where, near each moved line start, the voice rises + # most, less the lag the rise is measured over. + residuals = [] + for frame in frames[pick][inside[pick]]: + lo, hi = max(0, frame - k), min(strength.size, frame + k + 1) + window = strength[lo:hi] + if window.max() > 0: + residuals.append(lo + int(np.argmax(window)) - frame) + lag = (_RISE_FRAMES - 1) / 2 + return round(float(shifts[pick]) + (float(np.median(residuals)) - lag) * hop, 2) + + +def _envelope(stems_dir: Path) -> tuple[float, list[int]] | None: + """The vocals stem's envelope: the one kept beside it when it is not + older than the stem, else read from the stem now. None without vocals.""" + vocals = stems_dir / "vocals.wav" + if not vocals.is_file(): + return None + kept = stems_dir / _ENVELOPE_FILE + try: + if kept.is_file() and kept.stat().st_mtime_ns >= vocals.stat().st_mtime_ns: + data = json.loads(kept.read_text(encoding="utf-8")) + if isinstance(data.get("db"), list) and float(data.get("hop", 0)) > 0: + return float(data["hop"]), data["db"] + except (OSError, ValueError, TypeError, AttributeError): + logger.info("unreadable vocal envelope; reading the stem", exc_info=True) + return vocal_envelope(vocals, _ENVELOPE_HOP_SEC) + + +def detect_offset(synced: str, stems_dir: Path) -> float | None: + """How many seconds later ``synced``'s lines are sung on this track, from + the vocals stem in ``stems_dir``, whatever timing they were saved with: + the Lyrics tab's Auto-detect. None when the stem does not show it clearly, + or there is no stem. Blocking: may read the whole stem.""" + envelope = _envelope(stems_dir) + if envelope is None: + return None + hop, db = envelope + return estimate_offset(line_starts(synced), onset_strength(db), hop) + + +def align_lyrics(job: Job, job_dir: Path) -> str | None: + """Move LRCLIB lyrics' timing onto this track, when the vocals stem shows + clearly by how much: a version of another length ("unverified") by any + amount, one of the track's length ("exact") only by at least + LYRICS_ALIGN_EXACT_MIN_SHIFT_SEC. A transcription or the file's own + lyrics are the track's timing already. Returns the timing the job's + lyrics.json is left with, or None when it has none to align. Never + raises.""" + # Imported here: lyrics_lookup imports this module. + from app.pipeline.lyrics_lookup import _OFFSET_LOCK, read_lyrics, write_lyrics + + try: + entry = read_lyrics(job_dir) + if entry is None or not entry["synced"]: + return entry.get("timing") if entry else None + timing = entry.get("timing") + # Aligned by hand in the Lyrics tab: the user's timing, not to guess at. + if "offset_sec" in entry: + return timing + checked = timing == "unverified" or (timing == "exact" and entry["source"] == "lrclib") + if not checked: + return timing + envelope = _envelope(job_dir / "stems") + if envelope is None: + return timing + hop, db = envelope + shift = estimate_offset(line_starts(entry["synced"]), onset_strength(db), hop) + if shift is None: + return timing + if timing == "exact" and abs(shift) < LYRICS_ALIGN_EXACT_MIN_SHIFT_SEC: + return timing + # Written under the lock the Lyrics tab's own edits take, and only if + # nothing changed the lyrics while the shift was worked out: an + # offset or timing the user set meanwhile is theirs. + with _OFFSET_LOCK: + if read_lyrics(job_dir) != entry: + return timing + logger.info("[%s] lyrics timing moved by %+.2fs", job.id, shift) + write_lyrics( + job, + job_dir, + {**entry, "synced": shift_lrc(entry["synced"], shift), "timing": "shifted"}, + ) + return "shifted" + except Exception: + logger.warning("[%s] lyrics alignment failed", job.id, exc_info=True) + return None diff --git a/app/pipeline/lyrics_lookup.py b/app/pipeline/lyrics_lookup.py new file mode 100644 index 00000000..8e2f31f3 --- /dev/null +++ b/app/pipeline/lyrics_lookup.py @@ -0,0 +1,1376 @@ +"""A job's lyrics, found on LRCLIB while it separates, kept as lyrics.json. + +The Lyrics tab used to find them itself, in the browser, the first time it was +opened on a track. Done here, beside separation the way the band is +(artist_lookup.py), a track has its lyrics the moment the import is done, on +every device that opens it. + +What is looked up, best source first: the recording the job was identified as +(``job.identity``), else the file's own tags, else the band saved on the job +with the song's name cleaned out of the title. Lyrics the file itself carried +win over anything LRCLIB has, and need no request at all. + +The LRCLIB cascade, stopping at the first answer kept: + +(a) /api/get with artist, track, album and length: LRCLIB's exact match; +(b) a search by artist and track, ranked by length (the page's rankMatches); + when that finds nothing, the album as the artist, since LRCLIB files a + cast or soundtrack recording under the show ("Popular" by "Wicked"); +(c) a search by the track's name alone; +(d) when none of those found a version the track's length, the other names + the artist goes by: the two a name like "周杰倫 Jay Chou" gives at once, + and those MusicBrainz and Wikidata record for the artist (name_aliases.py: + 周杰倫 for Jay Chou, 아이유 for IU). What was already found is looked at + again under them, then a few are searched by; +(e) when still nothing was found, the other titles the song goes by (the + identity's title_aliases: "Good Day" for 좋은 날), by the artist. + +Every step keeps only versions of this song by a name the track is known by +(belongs_to): LRCLIB's search also answers with other artists' songs of a +similar name, and a track with no lyrics is better than one with another +song's. A track known by no name at all gets none from LRCLIB. Names and songs +are compared folded, so 紅豆 is 红豆 and a full-width letter its half-width one. + +When the track's length is known, each step first looks for a version within +LYRICS_SAME_RECORDING_SEC of it, whose timing is the track's ("timing": +"exact"). (c) is never asked without a length, since a name alone can be +anybody's song. When no version is that length but LRCLIB has the song by +artist (or show) and title, its best version is kept all the same, since a +database's words beat a transcription's: "timing": "unverified", and once the +vocals stem exists lyrics_align.py moves its stamps onto the track and marks +it "shifted" when the stem shows clearly by how much. + +Every version the searches found is kept beside the answer as ``others``, so +the tab can offer them without asking LRCLIB again. When nothing was kept at +all, they go in LYRICS_CANDIDATES_FILE instead, with when LRCLIB was asked: +the tab offers them to pick from, a transcription (transcribe.py) can be made, +and the tag backfill does not ask LRCLIB again for LYRICS_NOT_FOUND_RETRY_SEC. + +What is sent is the artist, song, album or show names and the track's length, +nothing else (and to MusicBrainz and Wikidata, for the artist's other names, +only the artist's MBID or Wikidata id). Everything here is best-effort: no connection, no match or a +slow answer leave the job without lyrics.json, and the tab then looks for them +itself as it always has. +""" + +from __future__ import annotations + +import json +import logging +import math +import os +import re +import shutil +import threading +import time +import unicodedata +import urllib.error +import urllib.parse +import urllib.request +import uuid +from collections.abc import Callable +from dataclasses import dataclass +from functools import cmp_to_key +from pathlib import Path +from typing import Any + +from app.core.config import ( + LRCLIB_API, + LRCLIB_USER_AGENT, + LYRICS_CANDIDATES_FILE, + LYRICS_FILE, + LYRICS_LOOKUP_BUDGET_SEC, + LYRICS_LOOKUP_MAX_BYTES, + LYRICS_LOOKUP_RETRIES, + LYRICS_LOOKUP_RETRY_SEC, + LYRICS_NOT_FOUND_RETRY_SEC, + LYRICS_OFFSET_MAX_SEC, + LYRICS_OTHERS_MAX, + LYRICS_SAME_RECORDING_SEC, + TIMEOUT_LYRICS_LOOKUP, + TITLE_MIN_COVERAGE, +) +from app.core.models import Job, _set, clean_identity +from app.pipeline.artist_lookup import _ssl_context, artist_name_key +from app.pipeline.lyrics_align import align_lyrics +from app.pipeline.lyrics_repair import restore_letters, words_of +from app.pipeline.lyrics_retime import clean_aligned, clean_user_synced, lyric_line_texts +from app.pipeline.name_aliases import artist_aliases, fold, has_cjk, name_key, name_weight +from app.pipeline.title_parse import TitleReading, coverage, resolvable, song_key, work_hint + +logger = logging.getLogger("stemdeck.lyrics") + +LYRICS_SOURCES = frozenset(("lrclib", "file", "whisper")) +# "exact": the track's own timing (a version its length, the file's own, a +# transcription); "unverified": a version of another length, as LRCLIB has it; +# "shifted": that, moved onto the track by lyrics_align.py. +LYRICS_TIMINGS = frozenset(("exact", "shifted", "unverified")) +# Where the letters a stripped copy lost were taken from (lyrics_repair.py): +# another LRCLIB copy of the song, or what Whisper heard in the vocals stem. +# Absent on lyrics that were never mended. +LYRICS_REPAIRS = frozenset(("lrclib", "whisper")) + +# How many album names are tried as the artist before giving up on (b). +_ALBUM_ARTIST_TRIES = 2 +# How many of the artist's other names are searched by in (d). +_OTHER_NAME_SEARCHES = 3 +# How many of the song's other titles are searched by in (e). +_OTHER_TITLE_SEARCHES = 2 +# A name, album or title longer than this is not one. +_NAME_MAX_CHARS = 300 +# Lyrics longer than this are not lyrics. A whole LRCLIB answer is capped by +# LYRICS_LOOKUP_MAX_BYTES; this caps each text kept from it. +_TEXT_MAX_CHARS = 100_000 + +FetchJson = Callable[[str, dict[str, str]], Any] +# What the job was identified as, when that is still being found beside the +# lookup: IdentifyLookup.wait_identity with a timeout. +IdentitySource = Callable[[], "dict[str, Any] | None"] +# The other names a query's artist goes by (name_aliases.artist_aliases), for +# a query and whether the lookup was cancelled. +AliasSource = Callable[["LyricsQuery", Callable[[], bool]], "list[str]"] + + +# ── ranking: rankMatches in static/js/lyricsLookup.js ── + + +def _number(value: Any) -> float: + """A JSON number as a finite float, or 0 (JS ``Number(x) || 0``).""" + if isinstance(value, bool): + return 0.0 + if isinstance(value, (int, float)): + number = float(value) + elif isinstance(value, str): + try: + number = float(value.strip() or 0) + except ValueError: + return 0.0 + else: + return 0.0 + return number if math.isfinite(number) else 0.0 + + +def _text(value: Any, limit: int) -> str: + return value[:limit] if isinstance(value, str) else "" + + +# Every line break but "\n", and the byte order mark. The page splits lyrics +# on "\n" alone (parseLrc) and Python's splitlines() on all of these: text +# carrying one parsed into different lines on each side, and the user's own +# timing of it was refused as not of these lines. +_OTHER_BREAKS = re.compile(r"\r\n?|[\x0b\x0c\x1c-\x1e\x85\u2028\u2029]") + + +def _lyric_text(value: Any, limit: int) -> str: + """Lyrics text with one kind of line break, so every reader splits it + into the same lines.""" + if not isinstance(value, str): + return "" + return _OTHER_BREAKS.sub("\n", value.replace("\ufeff", ""))[:limit] + + +def normalise(row: Any) -> dict[str, Any]: + """One LRCLIB row as a version of lyrics.json, without ``others``.""" + row = row if isinstance(row, dict) else {} + lrclib_id = int(_number(row.get("id"))) + return { + "v": 1, + "source": "lrclib", + "track": _text(row.get("trackName"), _NAME_MAX_CHARS), + "artist": _text(row.get("artistName"), _NAME_MAX_CHARS), + "album": _text(row.get("albumName"), _NAME_MAX_CHARS), + "duration": _number(row.get("duration")), + "synced": _lyric_text(row.get("syncedLyrics"), _TEXT_MAX_CHARS), + "plain": _lyric_text(row.get("plainLyrics"), _TEXT_MAX_CHARS), + "instrumental": bool(row.get("instrumental")), + "lrclib_id": lrclib_id if lrclib_id > 0 else None, + } + + +def rank_matches(rows: Any, duration: float = 0) -> list[dict[str, Any]]: + """Best first, for a track ``duration`` seconds long (0 when unknown): + closest in length, then synced over plain among versions within + LYRICS_SAME_RECORDING_SEC of each other, then intact over a copy stripped + of its accents, then whatever LRCLIB ranked first. Rows with no id, and with neither kind of lyrics and no + instrumental flag, are dropped.""" + return _rank( + [ + m + for m in map(normalise, rows if isinstance(rows, list) else []) + if m["lrclib_id"] and (m["synced"] or m["plain"] or m["instrumental"]) + ], + duration, + ) + + +def _rank(matches: list[dict[str, Any]], duration: float) -> list[dict[str, Any]]: + """rank_matches' order, over versions already normalised. Among versions + equally near in length and equally synced, one that is a stripped copy of + another (stripped_copy) goes after the rest.""" + stripped = _stripped_ids(matches) + + def off(m: dict[str, Any]) -> float: + return abs(m["duration"] - duration) if duration and m["duration"] else 0.0 + + def compare(a: tuple[int, dict[str, Any]], b: tuple[int, dict[str, Any]]) -> float: + da, db = off(a[1]), off(b[1]) + if abs(da - db) > LYRICS_SAME_RECORDING_SEC: + return da - db + if bool(a[1]["synced"]) != bool(b[1]["synced"]): + return -1 if a[1]["synced"] else 1 + sa, sb = a[0] in stripped, b[0] in stripped + if sa != sb: + return 1 if sa else -1 + return (da - db) or (a[0] - b[0]) + + return [m for _, m in sorted(enumerate(matches), key=cmp_to_key(compare))] + + +# ── stripped copies: strippedCopy in static/js/lyricsLookup.js ── +# +# LRCLIB holds many songs more than once, and some copies lost every letter +# outside ASCII on their way in: "Niewinnoci biaym niegiem" for "Niewinnością +# białym śniegiem" (Kayah, lrclib 5470091 beside the intact 10910419). Each +# such letter was either dropped or folded to its base ("się" as "sie"). Such +# a copy cannot be told from a song written without accents on its own, only +# beside the copy it was stripped from: that is what is compared here. + +# LRC time stamps and header tags, which say nothing about the words. +_LRC_TAGS = re.compile(r"\[[^\]\n]*\]|<\d{1,3}:\d{1,2}(?:[.:]\d{1,3})?>") +# The intact copy has at least this many words with a letter outside ASCII, +# so a stray "café" proves nothing. +_STRIPPED_MIN_WORDS = 5 +# The stripped copy keeps at most this share of them. +_STRIPPED_KEPT_MAX = 0.25 +# At least this share of the intact copy's accented words appear in the +# stripped one with those letters dropped or folded, +_STRIPPED_FOUND_MIN = 0.6 +# and at least this share of the stripped copy's words are the intact one's. +_STRIPPED_SAME_MIN = 0.8 + + +def _ascii_only(text: str) -> str: + return "".join(ch for ch in text if ord(ch) < 128) + + +def _word_runs(text: str) -> list[str]: + """The runs of letters and their marks in ``text``: [\\p{L}\\p{M}]+ in the + page. Marks count, or a word in Devanagari or Thai would break at each + vowel sign.""" + words: list[str] = [] + start = -1 + for i, ch in enumerate(text + " "): + if unicodedata.category(ch)[0] in "LM": + start = i if start < 0 else start + elif start >= 0: + words.append(text[start:i]) + start = -1 + return words + + +def _stripped_forms(word: str) -> set[str]: + """What an accented ``word`` becomes when its letters outside ASCII are + dropped ("każe" as "kae") or folded to their base first ("się" as "sie").""" + folded = _ascii_only(unicodedata.normalize("NFD", word)) + return {form for form in (_ascii_only(word), folded) if form} + + +class _Words: + """A version's words, lowercased, time stamps left out, with what they + would be stripped: worked out once per version, compared many times.""" + + def __init__(self, match: dict[str, Any]) -> None: + text = unicodedata.normalize("NFC", match["synced"] or match["plain"] or "") + self.words = _word_runs(_LRC_TAGS.sub(" ", text).lower()) + self.have = set(self.words) + self.accented = [w for w in self.words if not w.isascii()] + self.forms = [_stripped_forms(w) for w in self.accented] + self.known = self.have.union(*self.forms) + + def stripped_from(self, intact: _Words) -> bool: + """Whether these words are ``intact``'s with their accents lost.""" + accented = len(intact.accented) + if accented < _STRIPPED_MIN_WORDS or not self.words: + return False + if len(self.accented) > accented * _STRIPPED_KEPT_MAX: + return False + found = sum(bool(forms & self.have) for forms in intact.forms) + if found < accented * _STRIPPED_FOUND_MIN: + return False + same = sum(w in intact.known for w in self.words) + return same >= len(self.words) * _STRIPPED_SAME_MIN + + +def stripped_copy(match: dict[str, Any], other: dict[str, Any]) -> bool: + """Whether ``match`` is ``other``'s lyrics with the letters outside ASCII + lost: dropped or folded to their base letter.""" + return _Words(match).stripped_from(_Words(other)) + + +def _stripped_ids(matches: list[dict[str, Any]]) -> set[int]: + """The positions in ``matches`` of versions that are a stripped copy of + another version there.""" + words = [_Words(m) for m in matches] + intact = [(j, w) for j, w in enumerate(words) if len(w.accented) >= _STRIPPED_MIN_WORDS] + return { + i + for i, w in enumerate(words) + if any(j != i and w.stripped_from(other) for j, other in intact) + } + + +def _intact_twin( + chosen: dict[str, Any], pool: list[dict[str, Any]], duration: float +) -> dict[str, Any]: + """``chosen``, or when it is a stripped copy of a version in ``pool`` that + can stand in for it (as synced, and the track's length when ``chosen`` + is), the best of those.""" + words = _Words(chosen) + twins = [ + m + for m in pool + if m["lrclib_id"] != chosen["lrclib_id"] + and (m["synced"] or not chosen["synced"]) + and (not same_recording(chosen, duration) or same_recording(m, duration)) + and words.stripped_from(_Words(m)) + ] + return _rank(twins, duration)[0] if twins else chosen + + +def _mended(chosen: dict[str, Any], pool: list[dict[str, Any]], duration: float) -> dict[str, Any]: + """``chosen``, or when it is a stripped copy of a version in ``pool`` and + no twin could stand in for it (a plain copy, or one of another length), + ``chosen`` with its letters given back from the best of those, word by + word, its timing untouched: "repaired": "lrclib".""" + words = _Words(chosen) + intact = [ + m for m in pool if m["lrclib_id"] != chosen["lrclib_id"] and words.stripped_from(_Words(m)) + ] + if not intact: + return chosen + source = _rank(intact, duration)[0] + reference = words_of(source["synced"] or source["plain"]) + synced = restore_letters(chosen["synced"], reference) + plain = restore_letters(chosen["plain"], reference) + if not (synced.restored or plain.restored): + return chosen + return {**chosen, "synced": synced.text, "plain": plain.text, "repaired": "lrclib"} + + +def same_recording(match: dict[str, Any], duration: float) -> bool: + """Whether a version is the length of a track ``duration`` seconds long.""" + return ( + duration > 0 + and match["duration"] > 0 + and abs(match["duration"] - duration) <= LYRICS_SAME_RECORDING_SEC + ) + + +# ── the song's name: songFromTitle in static/js/lyricsLookup.js ── + +# Unicode's dash punctuation, which titles use between artist and song. Built +# from the category rather than typed out, as \p{Pd} is in the page. +_DASHES = "".join(chr(c) for c in range(0x11000) if unicodedata.category(chr(c)) == "Pd") +_DASH = f"[{re.escape(_DASHES)}:|]" +# A letter: \p{L} in the page's expressions. +_LETTER = r"[^\W\d_]" +_TITLE_NOISE = re.compile( + r"[(\[][^)\]]*(? bool: + return ch.isspace() or ch in _DASHES or ch in ":|" + + +def song_from_title(title: Any, artist: Any = "") -> str: + """The song's name from a track title, for searching lyrics: the artist + taken off either end, bracketed video noise and featured artists dropped, + quotes removed. ``artist`` may be empty, and then only the noise goes.""" + song = _TITLE_NOISE.sub(" ", str(title or "")) + # The featured artist, up to the next dash or bracket, before the artist + # is looked for, which it would otherwise stand between. + song = _FEATURING.sub(" ", song) + name = str(artist or "").strip() + if name: + literal = re.escape(name) + song = re.sub(rf"^\s*{literal}\s*{_DASH}\s*", "", song, count=1, flags=re.IGNORECASE) + song = re.sub(rf"\s*{_DASH}\s*{literal}\s*$", "", song, count=1, flags=re.IGNORECASE) + song = "".join(ch for ch in song if ch != '"' and unicodedata.category(ch) not in ("Pi", "Pf")) + song = re.sub(r"\s+", " ", song) + start, end = 0, len(song) + while start < end and _edge(song[start]): + start += 1 + while end > start and _edge(song[end - 1]): + end -= 1 + return song[start:end].strip() + + +# ── whose song a version is: belongsTo in static/js/lyricsLookup.js ── +# +# LRCLIB's search is fuzzy: asked for one artist's song it also answers with +# other artists' songs of a similar name. Lyrics that may be another song's are +# worse than none, so a version is kept only when its song is the one asked for +# and its artist is one of the names the track is known by: the artist itself, +# one of the artists credited with it ("Keala Settle" of "Keala Settle & The +# Greatest Showman Ensemble"), the show a cast recording is filed under, or +# another name the artist is recorded under (name_aliases.py: "Jay Chou" for +# 周杰倫). Names are compared folded (name_aliases.fold): full-width letters +# as half-width, traditional Chinese as simplified. +_NAME_LIST = re.compile( + r"\s*(?:[&,+/;]|\band\b|\bwith\b|\bfeat\.?|\bft\.?|\bfeaturing\b|\bvs\.?)\s*", re.IGNORECASE +) +_LEADING_THE = re.compile(r"^\s*the\s+", re.IGNORECASE) +_SONG_FEATURING = re.compile(r"\s+(?:ft\.?|feat\.?|featuring)\s.*$", re.IGNORECASE | re.DOTALL) +_SONG_BRACKETS = re.compile(r"[(\[{【][^()\[\]{}【】]*[)\]}】]") +_SONG_TAIL = re.compile(rf"\s+[{re.escape(_DASHES)}]+\s+.*$", re.DOTALL) +# What stands between the names a name is written in at once: "周杰倫 Jay +# Chou", "IU (아이유)", "五月天 (Mayday)". +_SCRIPT_BREAK = re.compile(r"[\s()\[\]{}【】「」『』〈〉《》]+") +# One credited artist matches only when its name weighs at least this much +# (name_aliases.name_weight: an ideograph counts two), so an initial or a +# stray "DJ" names nobody while 王菲 does; a run of words inside a longer name +# ("The Greatest Showman" in "The Greatest Showman Cast") needs this many. +_PART_MIN_CHARS = 4 +_RUN_MIN_WORDS = 2 + + +def _name_key(name: Any) -> str: + return name_key(_LEADING_THE.sub("", fold(name))) + + +def _name_words(name: Any) -> list[str]: + words, word = [], [] + for ch in fold(name) + " ": + if unicodedata.category(ch)[0] in ("L", "M", "N"): + word.append(ch) + elif word: + words.append(artist_name_key("".join(word))) + word = [] + return [w for w in words if w] + + +def _name_parts(name: Any) -> list[str]: + return [key for key in map(_name_key, _NAME_LIST.split(fold(name))) if key] + + +def _script_of(token: str) -> str: + """ "cjk" for a token all in Chinese, Japanese or Korean letters, "other" + for one with none, "mixed" for both ("Official髭男dism"), "" for none.""" + letters = [ch for ch in token if unicodedata.category(ch)[0] == "L"] + cjk = sum(map(has_cjk, letters)) + if not letters: + return "" + return "cjk" if cjk == len(letters) else "other" if not cjk else "mixed" + + +def script_names(name: Any) -> list[str]: + """The names a name gives in two scripts at once, each on its own: + "周杰倫 Jay Chou" as 周杰倫 and "Jay Chou", "IU (아이유)" as IU and 아이유. + [] for a name in one script, or with a word that mixes them, which is one + name ("Official髭男dism").""" + tokens = [t for t in _SCRIPT_BREAK.split(fold(name)) if t] + kinds = [_script_of(t) for t in tokens] + if "mixed" in kinds or len({k for k in kinds if k}) < 2: + return [] + groups: list[list[str]] = [] + last = "" + for token, kind in zip(tokens, kinds, strict=True): + if not kind: + last = "" + continue + if kind != last: + groups.append([]) + groups[-1].append(token) + last = kind + return [" ".join(g) for g in groups] + + +def _whole_names(name: Any) -> set[str]: + return {k for k in (_name_key(name), *map(_name_key, script_names(name))) if k} + + +def _contains_run(words: list[str], run: list[str]) -> bool: + if len(run) < _RUN_MIN_WORDS or len(run) > len(words): + return False + return any(words[i : i + len(run)] == run for i in range(len(words) - len(run) + 1)) + + +def same_artist(found: Any, names: tuple[str, ...] | list[str]) -> bool: + """Whether ``found``, a version's artist, is one of ``names``.""" + if not _name_key(found): + return False + theirs = _whole_names(found) + their_parts = _name_parts(found) + their_words = _name_words(found) + for name in names: + if not str(name or "").strip(): + continue + if theirs & _whole_names(name): + return True + if any(name_weight(p) >= _PART_MIN_CHARS and p in their_parts for p in _name_parts(name)): + return True + mine = _name_words(name) + if _contains_run(mine, their_words) or _contains_run(their_words, mine): + return True + return False + + +def _song_keys(name: Any) -> tuple[str, str]: + text = fold(name) + full = _SONG_FEATURING.sub("", _SONG_BRACKETS.sub(" ", text)) + # A name all in brackets (【白日】) is the name, not a note on it. + full = full if artist_name_key(full) else text + return artist_name_key(full), artist_name_key(_SONG_TAIL.sub("", full)) + + +def same_song(found: Any, song: Any) -> bool: + """Whether ``found``, a version's song, is ``song``: the same name less + brackets and featured artists, or one of them the other with a " - ..." + tail ("This Is Me - From The Greatest Showman"). Two tails never make a + match: "Part I - Dawn" is not "Part I - Dusk". Compared folded: + "紅豆" is "红豆", "晴天 (Sunny Day)" and "晴天（Sunny Day）" are "晴天".""" + a_full, a_head = _song_keys(found) + b_full, b_head = _song_keys(song) + if not a_full or not b_full: + return False + return a_full in (b_full, b_head) or a_head == b_full + + +def belongs_to(match: dict[str, Any], song: str, names: tuple[str, ...] | list[str]) -> bool: + """Whether a version is ``song`` by one of ``names`` (the artist, a show, + another name the artist goes by).""" + return same_song(match.get("track"), song) and same_artist(match.get("artist"), names) + + +def saved_lyrics_belong(entry: dict[str, Any], query: LyricsQuery | None) -> bool: + """Whether LRCLIB lyrics kept on a track are that track's, by the rule a + lookup keeps them by now (belongs_to). + + Lyrics kept before that rule existed were taken from a fuzzy search and + can be another song's (NIHIL "Barro" kept Renato Vianna's "Joao de + Barro"), and nothing ever looked at them again. A song that matches + neither the track's name nor any of its other titles is never this + track's. An artist in another script ("Jay Chou" for the credit 周杰倫) + was matched through the artist's aliases when the lyrics were found, and + stands; in the same script it has to match, unless the lookup found it + under another of the artist's names ("by_alias": Cat Stevens for Yusuf + Islam). Anything not from LRCLIB, or a track known by no name, is left + alone.""" + if query is None or entry.get("source") != "lrclib": + return True + songs = [s for s in (query.track, *query.track_aliases) if s] + if songs and not any(same_song(entry.get("track"), s) for s in songs): + return False + names = tuple(n for n in (query.artist, *query.album_artists) if n) + if not names or entry.get("by_alias") is True or same_artist(entry.get("artist"), names): + return True + return has_cjk(str(entry.get("artist") or "")) != has_cjk(query.artist) + + +# Spaces and tabs, not \s: \s crosses lines, and blank ones made it quadratic. +_LRC_STAMP = re.compile(r"^[ \t]*\[(\d{1,3}):(\d{1,2}(?:[.:]\d{1,3})?)\]", re.MULTILINE) + + +def has_synced_lines(text: str) -> bool: + """Whether ``text`` is LRC, with at least one timed line (parseLrc).""" + return bool(_LRC_STAMP.search(text or "")) + + +# ── what to look up ── + + +@dataclass(frozen=True) +class LyricsQuery: + artist: str + track: str + album: str + duration: float + # Lyrics the file itself carried, which win over any lookup. + embedded: str = "" + # Other names the recording may be filed under on LRCLIB: the show a cast + # or soundtrack recording belongs to. + album_artists: tuple[str, ...] = () + # Who the artist is, when known: the MusicBrainz artists the recording + # credits and the Wikidata item of the band. The other names they go by + # count as the artist's own (name_aliases.py). + artist_ids: tuple[str, ...] = () + band_id: str = "" + # Other titles the song goes by (the identity's title_aliases): LRCLIB + # may file 좋은 날 as "Good Day". + track_aliases: tuple[str, ...] = () + + +def _name(value: Any) -> str: + return value.strip()[:_NAME_MAX_CHARS] if isinstance(value, str) else "" + + +def _album_artists(album: str, work: Any, artist: str, title_work: str = "") -> tuple[str, ...]: + """Names to try as the artist when the artist finds nothing: the work the + recording belongs to, when known, or the show the track's title names + ('Dancing Through Life (From "Wicked" ...)'), and the album, less what an + album title adds to the show's name ("Wicked: The Soundtrack", "Wicked + (Original Broadway Cast Recording)", "Wicked - Original Broadway Cast").""" + names: list[str] = [] + if isinstance(work, dict): + names += [_name(work.get("englishName")), _name(work.get("name"))] + names.append(_name(title_work)) + if album: + short = re.sub(r"\s*[(\[].*?[)\]]\s*", " ", album) + short = re.split(rf"\s*:\s*|\s+{_DASH}\s+", short, maxsplit=1)[0].strip() + names += [short, album] + seen = {artist.casefold()} if artist else set() + out = [] + for name in names: + if name and name.casefold() not in seen: + seen.add(name.casefold()) + out.append(name) + return tuple(out[:_ALBUM_ARTIST_TRIES]) + + +def build_query( + job: Job, + *, + audio_tags: dict[str, str] | None = None, + band: dict[str, str] | None = None, + identity: dict[str, Any] | None = None, +) -> LyricsQuery | None: + """What the job is known to be, best source first: its identity, its + tags, or the band saved on it with the song's name from the title. + ``audio_tags``, ``band`` and ``identity`` stand in for the job's own when + given (they can be known before they are on the job). None when nothing + is known well enough to look up and the file carried no lyrics.""" + tags = audio_tags if audio_tags is not None else (job.audio_tags or {}) + band = band or job.artist or {} + identity = identity or getattr(job, "identity", None) + identity = identity if isinstance(identity, dict) else {} + band_name = _name(band.get("englishName")) or _name(band.get("name")) + duration = _number(job.duration_sec) or _number(identity.get("duration")) + embedded = tags.get("lyrics") if isinstance(tags.get("lyrics"), str) else "" + + artist_ids: tuple[str, ...] = () + track_aliases: tuple[str, ...] = () + if _name(identity.get("title")): + artist = _name(identity.get("artist")) or _name(tags.get("artist")) or band_name + if _name(identity.get("artist")): + mbids = identity.get("artist_mbids") + artist_ids = ( + tuple(m for m in mbids if isinstance(m, str)) if isinstance(mbids, list) else () + ) + track = _name(identity["title"]) + album = _name(identity.get("album")) or _name(tags.get("album")) + aliases = identity.get("title_aliases") + if isinstance(aliases, list): + track_aliases = tuple(n for n in map(_name, aliases) if n) + elif _name(tags.get("title")): + artist = _name(tags.get("artist")) or band_name + track = song_from_title(tags["title"], artist) or _name(tags["title"]) + album = _name(tags.get("album")) + else: + artist = band_name + track = song_from_title(job.title, artist) if artist else "" + album = "" + + if not track and not embedded: + return None + hint = work_hint(job.title, track) if track else None + return LyricsQuery( + artist=artist, + track=track or _name(job.title), + album=album, + duration=duration, + embedded=embedded, + album_artists=_album_artists( + album, getattr(job, "work", None), artist, hint.work if hint else "" + ), + artist_ids=artist_ids, + band_id=_name(band.get("id")), + track_aliases=track_aliases, + ) + + +# ── the lookup ── + + +def _fetch_json(endpoint: str, params: dict[str, str]) -> Any: + """One LRCLIB request to /api/. None for a 404, which is + LRCLIB's "no such track"; raises on any other failure.""" + url = f"{LRCLIB_API}/{endpoint}?{urllib.parse.urlencode(params)}" + request = urllib.request.Request( + url, headers={"User-Agent": LRCLIB_USER_AGENT, "Accept": "application/json"} + ) + try: + # A fixed https URL with only its query built here, never a URL from + # a request or a tag, which is what B310 exists to catch. + with urllib.request.urlopen( # nosec B310 + request, timeout=TIMEOUT_LYRICS_LOOKUP, context=_ssl_context() + ) as response: + body = response.read(LYRICS_LOOKUP_MAX_BYTES + 1) + except urllib.error.HTTPError as err: + if err.code == 404: + return None + raise + if len(body) > LYRICS_LOOKUP_MAX_BYTES: + raise ValueError("LRCLIB answer too large") + return json.loads(body) + + +def identify_on_lrclib( + reading: TitleReading, + duration: float | None, + *, + fetch_json: FetchJson | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """An identity (source "lrclib") for a track MusicBrainz could not place, + from one reading of its title, or None. + + One search for the reading's words. A version counts only when its track + name is the song's, its length is within LYRICS_SAME_RECORDING_SEC of the + track's, and its artist and album contain at least TITLE_MIN_COVERAGE of + the reading's words beyond the song; the most words covered, then the + nearest length, then synced lyrics, wins. Its artist and album then name + the track, for the band, the work and the lyrics to be looked up by. + Raises when LRCLIB cannot be reached.""" + want = song_key(reading.song) + if not want or not resolvable(reading) or not duration or duration <= 0 or cancelled(): + return None + fetch_json = fetch_json or _fetch_json + terms = " ".join(dict.fromkeys(t for t in (reading.song, reading.artist, reading.work) if t)) + kept = [] + for match in rank_matches(fetch_json("search", {"q": terms[:_NAME_MAX_CHARS]}), duration): + if not same_recording(match, duration) or song_key(match["track"]) != want: + continue + covered = round(coverage(reading, match["artist"], match["album"]), 3) + if covered < TITLE_MIN_COVERAGE: + continue + rank = (-covered, abs(match["duration"] - duration), not match["synced"]) + kept.append((rank, covered, match)) + if not kept: + return None + _, covered, match = min(kept, key=lambda item: item[0]) + return clean_identity( + { + "source": "lrclib", + "score": covered, + "title": match["track"], + "artist": match["artist"], + "album": match["album"] or None, + "duration": match["duration"], + } + ) + + +class _Stop(Exception): + """Cancelled, or out of time: ask nothing more.""" + + +@dataclass(frozen=True) +class LookupAnswer: + """What a lookup that ran to its end found: ``lyrics`` to keep as + lyrics.json, or None when no version qualified, and then ``others``, the + versions LRCLIB has (possibly none), to offer instead.""" + + lyrics: dict[str, Any] | None + others: list[dict[str, Any]] + + +def _from_file(query: LyricsQuery, fallback_title: str) -> dict[str, Any]: + synced = query.embedded if has_synced_lines(query.embedded) else "" + return { + "v": 1, + "source": "file", + "track": query.track or fallback_title, + "artist": query.artist, + "album": query.album, + "duration": query.duration, + "synced": synced, + "plain": "" if synced else query.embedded, + "instrumental": False, + "timing": "exact", + "others": [], + "lrclib_id": None, + } + + +def _artist_aliases(query: LyricsQuery, cancelled: Callable[[], bool]) -> list[str]: + """The names MusicBrainz and Wikidata record for the query's artist.""" + if not (query.artist_ids or query.band_id): + return [] + return artist_aliases(query.artist_ids, query.band_id, cancelled=cancelled) + + +def _other_names(query: LyricsQuery, aliases: list[str]) -> list[str]: + """Who else to search LRCLIB by, best first: the names the artist's own + name gives at once, then the recorded ones in another script than it (an + uploader writes one or the other), then the rest. Each once, and never a + name already asked by.""" + asked = {n.casefold() for n in (query.artist, *query.album_artists)} + written = has_cjk(query.artist) + ranked = [ + *script_names(query.artist), + *(n for n in aliases if has_cjk(n) != written), + *(n for n in aliases if has_cjk(n) == written), + ] + out: list[str] = [] + for name in ranked: + if name.casefold() not in asked: + asked.add(name.casefold()) + out.append(name) + return out + + +def lookup_lyrics( + query: LyricsQuery, + *, + fetch_json: FetchJson | None = None, + aliases: AliasSource | None = None, + cancelled: Callable[[], bool] = lambda: False, + budget: float | None = None, + fallback_title: str = "", +) -> LookupAnswer | None: + """What LRCLIB has for ``query``. None when nothing could be asked, or + the lookup was cancelled or ran out of time before it kept anything, so + that it is asked again later. Raises when LRCLIB cannot be reached or + answers nonsense. ``aliases`` gives the other names the artist goes by + (name_aliases.py by default), and is asked only for step (d). + + No request is started once ``cancelled()`` or ``budget`` seconds have + passed; running out of time keeps what was found.""" + if query.embedded: + return LookupAnswer(_from_file(query, fallback_title), []) + if not query.track: + return None + # Resolved here rather than as the default, so a test can stand in for + # the network by replacing _fetch_json. + fetch_json = fetch_json or _fetch_json + aliases = aliases or _artist_aliases + deadline = time.monotonic() + (LYRICS_LOOKUP_BUDGET_SEC if budget is None else budget) + duration = query.duration + found: list[dict[str, Any]] = [] + # Every version LRCLIB answered with, whoever's, to be looked at again + # under the artist's other names. + asked: list[dict[str, Any]] = [] + # The song by artist (or show) and title, whatever its length. + song: list[dict[str, Any]] = [] + chosen: dict[str, Any] | None = None + # Who the song may be filed under. Only versions of this song by one of + # them are kept or offered (belongs_to): no lyrics beat another song's. + names = [n for n in (query.artist, *query.album_artists) if n] + + def check() -> None: + if cancelled() or time.monotonic() > deadline: + raise _Stop + + def ask(endpoint: str, params: dict[str, str]) -> Any: + for attempt in range(LYRICS_LOOKUP_RETRIES + 1): + check() + try: + return fetch_json(endpoint, params) + except urllib.error.HTTPError as err: + wait = LYRICS_LOOKUP_RETRY_SEC * 2**attempt + busy = err.code in (429, 503) + if not busy or attempt == LYRICS_LOOKUP_RETRIES: + raise + if time.monotonic() + wait > deadline: + raise + logger.info("LRCLIB busy (%s); asking again in %.0fs", err.code, wait) + time.sleep(wait) + raise AssertionError("unreachable") + + def mine(rows: list[dict[str, Any]]) -> list[dict[str, Any]]: + return [m for m in rows if belongs_to(m, query.track, names)] + + def answered(rows: Any) -> list[dict[str, Any]]: + ranked = rank_matches(rows, duration) + asked.extend(ranked) + return ranked + + def search(params: dict[str, str]) -> list[dict[str, Any]]: + rows = mine(answered(ask("search", params))) + found.extend(rows) + return rows + + def held(rows: list[dict[str, Any]]) -> list[dict[str, Any]]: + """The versions the length of this track, or all when it is unknown.""" + return [m for m in rows if same_recording(m, duration)] if duration else rows + + try: + if query.artist: + # (a) The exact match. A failure here is not the end: the search + # below asks a different question of a different index. + params = {"artist_name": query.artist, "track_name": query.track} + if query.album: + params["album_name"] = query.album + if duration: + params["duration"] = str(round(duration)) + try: + exact = mine(answered([ask("get", params)])) + except urllib.error.HTTPError: + logger.info("LRCLIB exact match failed", exc_info=True) + exact = [] + song.extend(exact) + exact = held(exact) + chosen = exact[0] if exact else None + # (b) The artist's versions of the song, which are also the + # versions offered beside an exact match. + rows = search({"artist_name": query.artist, "track_name": query.track}) + song.extend(rows) + rows = held(rows) + chosen = chosen or (rows[0] if rows else None) + if chosen is None and not song: + # (b) again, under the show's name. + for name in query.album_artists: + rows = search({"artist_name": name, "track_name": query.track}) + song.extend(rows) + rows = held(rows) + if rows: + chosen = rows[0] + break + if song: + break + if chosen is None and duration and names: + # (c) The name alone, which answers with anybody's song of that + # name: only one by a name the track is known by is kept. A + # version of another length is still this artist's song, kept to + # be timed from the vocals as one the artist search found would be + # (LRCLIB's artist search can miss a name it holds under the name + # search). + rows = search({"q": query.track}) + song.extend(rows) + rows = held(rows) + chosen = rows[0] if rows else None + if chosen is None and query.artist: + # (d) The artist's other names. First what the searches above + # already answered with, looked at again under them: (c) often + # holds the song under the name the track does not go by. + check() + more = _other_names(query, aliases(query, cancelled)) + if more: + names.extend(more) + have = {m["lrclib_id"] for m in found} + again = [] + for m in _rank(asked, duration): + if m["lrclib_id"] not in have and belongs_to(m, query.track, names): + have.add(m["lrclib_id"]) + again.append(m) + found.extend(again) + rows = held(again) + chosen = rows[0] if rows else None + # Then searched by, as (b) is by the artist. + for name in more[:_OTHER_NAME_SEARCHES]: + if chosen is not None: + break + rows = search({"artist_name": name, "track_name": query.track}) + song.extend(rows) + rows = held(rows) + chosen = rows[0] if rows else None + if chosen is None and not song and query.artist: + # (e) The song's other titles, by the artist: a version counts + # when it is that title by a name the track is known by. + for alias in query.track_aliases[:_OTHER_TITLE_SEARCHES]: + rows = answered(ask("search", {"artist_name": query.artist, "track_name": alias})) + rows = [m for m in rows if belongs_to(m, alias, names)] + found.extend(rows) + song.extend(rows) + rows = held(rows) + chosen = rows[0] if rows else None + if song: + break + except _Stop: + if cancelled() or (chosen is None and not song): + return None + logger.info("lyrics lookup ran out of time; keeping what was found") + + timing = "exact" + if chosen is None and song: + # No version the track's length, but the song itself: its words, with + # timing to be moved onto the track once there is a vocals stem. A + # synced version, when there is one, since only that can be moved. + ranked = _rank(song, duration) + chosen = next((m for m in ranked if m["synced"]), ranked[0]) + timing = "unverified" if chosen["synced"] else "exact" + if chosen is not None: + # LRCLIB's exact match can be a copy that lost its accents while the + # search found the copy it was stripped from. When that copy cannot + # stand in for it, its words still can. + chosen = _intact_twin(chosen, song + found, duration) + chosen = _mended(chosen, song + found, duration) + + others: list[dict[str, Any]] = [] + seen = {chosen["lrclib_id"]} if chosen else set() + for match in _rank(found, duration): + if match["lrclib_id"] not in seen: + seen.add(match["lrclib_id"]) + others.append(match) + others = [_alias_stamped(m, query) for m in others[:LYRICS_OTHERS_MAX]] + if chosen is None: + return LookupAnswer(None, others) + return LookupAnswer({**_alias_stamped(chosen, query), "timing": timing, "others": others}, []) + + +def _alias_stamped(match: dict[str, Any], query: LyricsQuery) -> dict[str, Any]: + """``match`` marked "by_alias" when it is credited to none of the names + the track itself carries: it was held to the artist's other names, which + saved_lyrics_belong has no way to ask for again.""" + names = tuple(n for n in (query.artist, *query.album_artists) if n) + if names and not same_artist(match.get("artist"), names): + return {**match, "by_alias": True} + return match + + +def find_lyrics( + query: LyricsQuery | None, + *, + fetch_json: FetchJson | None = None, + aliases: AliasSource | None = None, + cancelled: Callable[[], bool] = lambda: False, + budget: float | None = None, + fallback_title: str = "", +) -> LookupAnswer | None: + """lookup_lyrics that never raises: any failure is logged and gives None.""" + if query is None: + return None + try: + return lookup_lyrics( + query, + fetch_json=fetch_json, + aliases=aliases, + cancelled=cancelled, + budget=budget, + fallback_title=fallback_title, + ) + except Exception: + # Offline, a timeout, LRCLIB down or answering something else. The + # tab looks for the lyrics itself when it is opened. + logger.info("lyrics lookup failed", exc_info=True) + return None + + +# ── lyrics.json ── + + +# A short BCP-47 style language code: "en", "yue", "pt-BR". +_LANGUAGE_RE = re.compile(r"^[a-z]{2,3}(?:-[A-Za-z0-9]{2,8})?$") + + +def _clean_version(value: Any) -> dict[str, Any] | None: + if not isinstance(value, dict) or value.get("source") not in LYRICS_SOURCES: + return None + lrclib_id = value.get("lrclib_id") + if not (lrclib_id is None or (type(lrclib_id) is int and lrclib_id > 0)): + return None + duration = value.get("duration") + if isinstance(duration, bool) or not isinstance(duration, (int, float)): + return None + if not math.isfinite(duration) or duration < 0: + return None + version = { + "v": 1, + "source": value["source"], + "track": _text(value.get("track"), _NAME_MAX_CHARS), + "artist": _text(value.get("artist"), _NAME_MAX_CHARS), + "album": _text(value.get("album"), _NAME_MAX_CHARS), + "duration": float(duration), + "synced": _lyric_text(value.get("synced"), _TEXT_MAX_CHARS), + "plain": _lyric_text(value.get("plain"), _TEXT_MAX_CHARS), + "instrumental": value.get("instrumental") is True, + "lrclib_id": lrclib_id, + } + if not (version["synced"] or version["plain"] or version["instrumental"]): + return None + # The language a transcription detected (transcribe.py), kept only when + # it looks like a language code; absent for every other source. + language = value.get("language") + if isinstance(language, str) and _LANGUAGE_RE.match(language): + version["language"] = language + # Found under another of the artist's names (lookup step (d)). + if value.get("by_alias") is True: + version["by_alias"] = True + return version + + +def clean_offset(value: Any) -> float | None: + """A lyrics offset in seconds, to the hundredth, or None when ``value`` + is not a finite number within +-LYRICS_OFFSET_MAX_SEC.""" + if isinstance(value, bool) or not isinstance(value, (int, float)): + return None + if not math.isfinite(value) or abs(value) > LYRICS_OFFSET_MAX_SEC: + return None + return round(float(value), 2) + 0.0 # + 0.0: never -0.0 + + +def clean_lyrics(value: Any) -> dict[str, Any] | None: + """lyrics.json as read back from disk, or None when it is not one. A + damaged or hand-edited file must not put anything malformed in front of + the page.""" + if not isinstance(value, dict) or value.get("v") != 1: + return None + entry = _clean_version(value) + if entry is None: + return None + # Missing on lyrics written before timing was recorded, and on anything + # that has no reason to doubt its own (a transcription). + timing = value.get("timing", "exact") + if timing not in LYRICS_TIMINGS: + return None + entry["timing"] = timing + # Only on lyrics whose lost letters were given back (lyrics_repair.py). + if value.get("repaired") in LYRICS_REPAIRS: + entry["repaired"] = value["repaired"] + # Only once the user has aligned them in the Lyrics tab: seconds later + # (earlier when negative) than the synced text says, applied by the page. + # A value out of bounds is dropped rather than the whole file. + offset = clean_offset(value.get("offset_sec")) + if offset is not None: + entry["offset_sec"] = offset + # Timing made for these lines by lyrics_retime.py, and the user's own + # from the sync editor: each dropped when it is malformed or is not of + # these lines any more (another version, letters mended since). + if "aligned" in value or "user_synced" in value: + lines = lyric_line_texts(entry) + aligned = clean_aligned(value.get("aligned"), lines) + if aligned is not None: + entry["aligned"] = aligned + user = clean_user_synced(value.get("user_synced"), lines) + if user is not None: + entry["user_synced"] = user + others = value.get("others") + cleaned = [_clean_version(o) for o in (others if isinstance(others, list) else [])] + entry["others"] = [o for o in cleaned if o is not None][:LYRICS_OTHERS_MAX] + return entry + + +def lyrics_path(job_dir: Path) -> Path: + return job_dir / LYRICS_FILE + + +def candidates_path(job_dir: Path) -> Path: + return job_dir / LYRICS_CANDIDATES_FILE + + +def _write_json_atomic(path: Path, data: dict[str, Any]) -> None: + """Replace ``path`` whole or not at all. Raises OSError.""" + temp = path.parent / f".{path.name}.{uuid.uuid4().hex}.tmp" + try: + with temp.open("w", encoding="utf-8", newline="\n") as handle: + handle.write(json.dumps(data, ensure_ascii=False) + "\n") + handle.flush() + os.fsync(handle.fileno()) + os.replace(temp, path) + finally: + temp.unlink(missing_ok=True) + + +def write_lyrics(job: Job, job_dir: Path, entry: dict[str, Any]) -> bool: + """Keep ``entry`` as the job's lyrics.json, whole or not at all, and mark + the job as having lyrics. False, logged, when it could not be written. + + LRCLIB lyrics carry their own others, so they replace the candidates + file. A transcription leaves it: its versions are still worth offering.""" + entry = clean_lyrics(entry) + if entry is None: + return False + try: + _write_json_atomic(lyrics_path(job_dir), entry) + if entry["source"] == "lrclib": + candidates_path(job_dir).unlink(missing_ok=True) + except OSError: + logger.warning("could not write lyrics for job %s", job.id, exc_info=True) + return False + _set(job, has_lyrics=True) + return True + + +def write_candidates(job: Job, job_dir: Path, others: list[dict[str, Any]]) -> bool: + """Record that LRCLIB was asked and nothing was kept: when, and the + versions it had, to offer. False, logged, when it could not be written.""" + cleaned = [v for v in (_clean_version(o) for o in others) if v is not None] + data = {"v": 1, "searched_at": time.time(), "others": cleaned[:LYRICS_OTHERS_MAX]} + try: + _write_json_atomic(candidates_path(job_dir), data) + except OSError: + logger.warning("could not write lyrics candidates for job %s", job.id, exc_info=True) + return False + return True + + +def read_candidates(job_dir: Path) -> dict[str, Any] | None: + """The candidates file, cleaned: {"searched_at", "others"}, or None.""" + try: + data = json.loads(candidates_path(job_dir).read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + if not isinstance(data, dict) or data.get("v") != 1: + return None + searched_at = data.get("searched_at") + if isinstance(searched_at, bool) or not isinstance(searched_at, (int, float)): + return None + if not math.isfinite(searched_at): + return None + others = data.get("others") + cleaned = [_clean_version(o) for o in (others if isinstance(others, list) else [])] + return { + "searched_at": float(searched_at), + "others": [o for o in cleaned if o is not None][:LYRICS_OTHERS_MAX], + } + + +def lyrics_settled(job_dir: Path, now: float | None = None) -> bool: + """Whether there is nothing to ask LRCLIB for this job: it has lyrics, or + LRCLIB was asked less than LYRICS_NOT_FOUND_RETRY_SEC ago and had none.""" + if lyrics_path(job_dir).is_file(): + return True + found = read_candidates(job_dir) + now = time.time() if now is None else now + return found is not None and now - found["searched_at"] < LYRICS_NOT_FOUND_RETRY_SEC + + +def keep_answer(job: Job, job_dir: Path, answer: LookupAnswer) -> bool: + """Keep a lookup's answer: the lyrics, or the candidates and when they + were asked for. True when lyrics were written (has_lyrics changed). + + Lyrics of unverified timing are then moved onto the track, which needs + the vocals stem: both callers (LyricsLookup.finish and the tag backfill) + run once separation is done.""" + if answer.lyrics is None: + write_candidates(job, job_dir, answer.others) + return False + if not write_lyrics(job, job_dir, answer.lyrics): + return False + align_lyrics(job, job_dir) + return True + + +# One edit of lyrics.json at a time from the Lyrics tab: each reads the file, +# changes the offset and writes it back whole. +_OFFSET_LOCK = threading.Lock() + + +def set_lyrics_offset(job: Job, job_dir: Path, offset: float) -> dict[str, Any] | None: + """Keep ``offset`` seconds as the user's alignment of the job's synced + lyrics (0 is their own timing again) and return the entry as written. + None when the job has no synced lyrics or they could not be written. The + synced text itself is never rewritten, so their own timing stays there.""" + cleaned = clean_offset(offset) + if cleaned is None: + return None + with _OFFSET_LOCK: + entry = read_lyrics(job_dir) + if entry is None or not entry["synced"]: + return None + entry = {**entry, "offset_sec": cleaned} + return entry if write_lyrics(job, job_dir, entry) else None + + +class NotTheseLyrics(ValueError): + """A manual timing whose lines are not the lyrics' own, or not LRC.""" + + +def set_user_synced(job: Job, job_dir: Path, synced: str | None) -> dict[str, Any] | None: + """Keep ``synced`` as the user's own timing of the job's lyrics (the sync + editor), or drop it with None, and return the entry as written. None when + the job has no lyrics or they could not be written. Raises + NotTheseLyrics when ``synced`` is not a timing of exactly their lines + (lyrics_retime.clean_user_synced): nothing else can be put in front of + the page this way.""" + with _OFFSET_LOCK: + entry = read_lyrics(job_dir) + if entry is None: + return None + entry = {k: v for k, v in entry.items() if k != "user_synced"} + if synced is not None: + cleaned = clean_user_synced(synced, lyric_line_texts(entry)) + if cleaned is None: + raise NotTheseLyrics("not a timing of these lyrics") + entry["user_synced"] = cleaned + return entry if write_lyrics(job, job_dir, entry) else None + + +def copy_lyrics(src_dir: Path, dest_dir: Path) -> bool: + """Copy a job's lyrics.json and candidates file to another job of the + same recording (a re-split), with any alignment the user gave the lyrics + (offset_sec). True when lyrics.json was copied.""" + copied = False + for path in (lyrics_path(src_dir), candidates_path(src_dir)): + if not path.is_file(): + continue + try: + shutil.copy2(path, dest_dir / path.name) + except OSError: + logger.warning("could not copy %s to %s", path.name, dest_dir.name, exc_info=True) + continue + copied = copied or path.name == LYRICS_FILE + return copied + + +def read_lyrics(job_dir: Path) -> dict[str, Any] | None: + """The job's lyrics.json, cleaned, or None when it has none.""" + try: + return clean_lyrics(json.loads(lyrics_path(job_dir).read_text(encoding="utf-8"))) + except (OSError, ValueError): + return None + + +class LyricsLookup: + """One job's lyrics lookup, in a thread of its own, so separation never + waits for it. The same contract as BandLookup: the thread never touches + the job or its directory, and finish() writes the answer from the + pipeline's own thread, once the pipeline has got that far uncancelled. + """ + + def __init__(self, job: Job, identity: IdentitySource | None) -> None: + self._result: LookupAnswer | None = None + self._thread = threading.Thread( + target=self._run, args=(job, identity), name=f"lyrics-{job.id}", daemon=True + ) + + def _run(self, job: Job, identity: IdentitySource | None) -> None: + try: + # The identity is still being found beside this (identify.py), and + # is only put on the job once the pipeline is done: it is asked of + # the identification itself. + query = build_query(job, identity=identity() if identity else None) + except Exception: + logger.info("[%s] lyrics query failed", job.id, exc_info=True) + return + # find_lyrics never raises. + self._result = find_lyrics( + query, cancelled=lambda: job.cancel_requested, fallback_title=job.title or "" + ) + + @classmethod + def start( + cls, job: Job, job_dir: Path, *, identity: IdentitySource | None = None + ) -> LyricsLookup | None: + """Start looking, or None when there is nothing to look for, or the + job already has lyrics or had none on LRCLIB lately (a re-split has + its source's). ``identity``, when given, is called in the thread for + what the job was identified as, and may wait for it.""" + if lyrics_settled(job_dir): + return None + if identity is None and build_query(job) is None: + return None + lookup = cls(job, identity) + lookup._thread.start() + return lookup + + def finish(self, job: Job, job_dir: Path, timeout: float) -> None: + """Wait up to ``timeout`` seconds for the answer, and keep it + (keep_answer) if there is one and the job was not cancelled meanwhile.""" + self._thread.join(timeout) + if self._thread.is_alive(): + logger.info( + "[%s] lyrics lookup still out after %ss; finishing without", job.id, timeout + ) + return + if job.cancel_requested or self._result is None: + return + keep_answer(job, job_dir, self._result) diff --git a/app/pipeline/lyrics_repair.py b/app/pipeline/lyrics_repair.py new file mode 100644 index 00000000..be2cbd37 --- /dev/null +++ b/app/pipeline/lyrics_repair.py @@ -0,0 +1,388 @@ +"""Letters given back to lyrics that lost them. + +Some LRCLIB copies of a song lost every letter outside ASCII on their way in: +"Znowu biegne bo co cigle kae biec" for "Znowu biegnę bo coś ciągle każe +biec" (Natalia Kukulska, "W biegu"). Each such letter was dropped ("każe" as +"kae") or folded to its base ("biegnę" as "biegne"), and both happen in one +copy. The timing of such a copy is as good as any other's, so it is kept, and +only its words are mended, from a reference that has the letters: an intact +copy of the same song on LRCLIB (lyrics_lookup.py), or failing that what +Whisper heard in the vocals stem (transcribe.py). + +A word is only ever replaced by a reference word that strips down to it, so a +reference that misheard a word, or is another song, cannot put a wrong word +in: at worst a word stays as it was. The two word lists are aligned in order +(a longest common subsequence under that match), and a pair counts only when +one of its neighbours is aligned too. + +Whether a copy with no intact reference lost its letters cannot be read from +its text alone. looks_stripped() says whether it is worth asking the audio: +no letter outside ASCII over a real text, and not English. The rest of the +decision is the transcription's (transcribe.py). +""" + +from __future__ import annotations + +import math +import re +import unicodedata +from collections import Counter +from dataclasses import dataclass +from typing import Any + +from app.core.config import ( + LYRICS_REPAIR_ALIGN_MAX_CELLS, + LYRICS_REPAIR_MIN_ALIGNED, + LYRICS_REPAIR_MIN_RESTORED, + LYRICS_REPAIR_MIN_RESTORED_SHARE, + LYRICS_REPAIR_VOCABULARY_MIN, + LYRICS_STRIPPED_CHECK_MIN_WORDS, + LYRICS_STRIPPED_ENGLISH_MAX_SHARE, +) + +# LRC time stamps and header tags, which say nothing about the words: the +# same as lyrics_lookup._LRC_TAGS, which imports this module. +_LRC_TAGS = re.compile(r"\[[^\]\n]*\]|<\d{1,3}:\d{1,2}(?:[.:]\d{1,3})?>") + +# Letters with no decomposition that a stripping tool may still write as a +# base letter ("ł" as "l"), besides dropping them. +_FOLD = { + "ł": "l", + "ø": "o", + "đ": "d", + "ħ": "h", + "ı": "i", + "ß": "ss", + "æ": "ae", + "œ": "oe", + "þ": "th", + "ð": "d", + "ŀ": "l", +} + +# Words common in English lyrics and not a word of any language whose +# spelling carries accents: a copy with enough of them is English, and +# English has nothing to lose. Short words many languages share ("a", "i", +# "to", "me", "no", "on", "die", "will", "for", "her", "was") are left out, so +# that a Polish or German copy never reads as English. +_ENGLISH = frozenset( + [ + "the", + "you", + "your", + "and", + "that", + "what", + "with", + "this", + "when", + "know", + "just", + "like", + "but", + "they", + "she", + "him", + "his", + "are", + "have", + "been", + "can", + "would", + "there", + "from", + "into", + "out", + "get", + "got", + "never", + "love", + "yeah", + "baby", + "gonna", + "wanna", + "cause", + "don", + "it", + "how", + "now", + "where", + "who", + "why", + "could", + "should", + "every", + "some", + "thing", + "things", + "think", + "feel", + "make", + "time", + "night", + "heart", + "tonight", + "away", + "down", + "again", + "our", + "their", + "them", + "then", + "than", + "only", + ] +) + + +def _ascii_only(text: str) -> str: + return "".join(ch for ch in text if ord(ch) < 128) + + +def _base(ch: str) -> str: + """What a letter outside ASCII is folded to, or "" when it has no base.""" + return _FOLD.get(ch) or _ascii_only(unicodedata.normalize("NFD", ch)) + + +def _letter(ch: str) -> bool: + return unicodedata.category(ch)[0] in "LM" + + +def word_spans(text: str) -> list[tuple[int, int]]: + """Where the words of ``text`` are, time stamps and tags left out: the + runs of letters and their marks, as in lyrics_lookup._word_runs.""" + masked = _LRC_TAGS.sub(lambda m: " " * len(m.group()), text) + spans: list[tuple[int, int]] = [] + start = -1 + for i, ch in enumerate(masked + " "): + if _letter(ch): + start = i if start < 0 else start + elif start >= 0: + spans.append((start, i)) + start = -1 + return spans + + +def words_of(text: str) -> list[str]: + """The words of ``text``, NFC, in order, time stamps left out.""" + text = unicodedata.normalize("NFC", text or "") + return [text[a:b] for a, b in word_spans(text)] + + +def looks_stripped(text: str) -> bool: + """Whether a copy is worth checking against the audio for lost letters: + at least LYRICS_STRIPPED_CHECK_MIN_WORDS words, not one letter outside + ASCII among them, and not English (fewer than + LYRICS_STRIPPED_ENGLISH_MAX_SHARE of them common English words). + + True says nothing about the language yet: a song in Indonesian, or + Russian written in Latin letters, looks the same. Only the audio can tell, + and transcribe.py asks it before anything is changed.""" + words = [w.lower() for w in words_of(text)] + if len(words) < LYRICS_STRIPPED_CHECK_MIN_WORDS: + return False + if any(not w.isascii() for w in words): + return False + english = sum(w in _ENGLISH for w in words) + return english < len(words) * LYRICS_STRIPPED_ENGLISH_MAX_SHARE + + +def _strips_to(reference: str, word: str) -> bool: + """Whether ``reference`` gives ``word`` with each of its letters outside + ASCII kept, folded to its base or dropped, and every other letter kept. + + Walked a letter at a time over the set of places in ``word`` reached so + far, so the cost is at most the product of the two lengths. A regular + expression of optional groups did the same by backtracking, which a word + of forty accented letters in someone's LRCLIB upload turned into hours of + matching with the interpreter locked.""" + reached = {0} + for ch in reference: + after: set[int] = set() + base = "" if ch.isascii() else _base(ch) + for at in reached: + if word.startswith(ch, at): + after.add(at + len(ch)) + if not ch.isascii(): + after.add(at) + if base and word.startswith(base, at): + after.add(at + len(base)) + if not after: + return False + reached = after + return len(word) in reached + + +class _Stripper: + """Whether a reference word strips down to a word.""" + + def restores(self, reference: str, word: str) -> bool: + """Whether ``reference`` (lowercase) is ``word`` (lowercase) with the + letters it lost: each of its letters outside ASCII kept, folded or + dropped gives ``word``. Never for a word left with fewer than two + letters, or half the reference's, unless it is all of it folded: a + lone "e" is not "że".""" + if reference == word or reference.isascii() or not _strips_to(reference, word): + return False + if len(word) >= max(2, math.ceil(len(reference) / 2)): + return True + return word == "".join(ch if ch.isascii() else _base(ch) for ch in reference) + + def restores_short(self, reference: str, word: str) -> bool: + """Whether ``word`` is shorter than restores() allows yet strips + down from ``reference``: "e" from "że". Taken only between two + neighbours paired in order, where no sung vowel is heard as "że".""" + return ( + reference != word + and not reference.isascii() + and _strips_to(reference, word) + and not self.restores(reference, word) + ) + + def matches(self, reference: str, word: str) -> bool: + return ( + reference == word + or self.restores(reference, word) + or self.restores_short(reference, word) + ) + + +def _align(words: list[str], reference: list[str], stripper: _Stripper) -> list[tuple[int, int]]: + """The longest in-order pairing of ``words`` with ``reference`` (both + lowercase) under _Stripper.matches, as (word index, reference index).""" + n, m = len(words), len(reference) + if not n or not m or n * m > LYRICS_REPAIR_ALIGN_MAX_CELLS: + return [] + ids: dict[str, int] = {} + wid = [ids.setdefault(w, len(ids)) for w in words] + rid = [ids.setdefault(r, len(ids)) for r in reference] + same: dict[tuple[int, int], bool] = {} + + def ok(i: int, j: int) -> bool: + key = (wid[i], rid[j]) + if key not in same: + same[key] = stripper.matches(reference[j], words[i]) + return same[key] + + table = [[0] * (m + 1) for _ in range(n + 1)] + for i in range(n - 1, -1, -1): + row, below = table[i], table[i + 1] + for j in range(m - 1, -1, -1): + if ok(i, j): + row[j] = below[j + 1] + 1 + else: + row[j] = max(below[j], row[j + 1]) + pairs: list[tuple[int, int]] = [] + i = j = 0 + while i < n and j < m: + if ok(i, j) and table[i][j] == table[i + 1][j + 1] + 1: + pairs.append((i, j)) + i, j = i + 1, j + 1 + elif table[i + 1][j] >= table[i][j + 1]: + i += 1 + else: + j += 1 + return pairs + + +def _cased_like(reference: str, word: str) -> str: + """``reference`` in the case ``word`` is written in.""" + lower = reference.lower() + if len(word) > 1 and word.isupper(): + return reference.upper() + if word[:1].isupper(): + return lower[:1].upper() + lower[1:] + return lower + + +@dataclass(frozen=True) +class Restoration: + """``text`` with letters given back to ``restored`` of its ``words`` + words, ``aligned`` of which were paired with a reference word at all.""" + + text: str + words: int + aligned: int + restored: int + + +def restore_letters(text: str, reference: list[str]) -> Restoration: + """``text`` (LRC or plain) with the letters its words lost taken from + ``reference``, the words of the same lyrics with them, in order. + + A word is replaced only by a reference word that strips down to it, + paired with it in order and with a neighbour paired too. Any other word + (the reference skipped its line, misheard a word beside it, or a chorus + comes round again) takes the spelling those pairs gave it elsewhere, when + they gave it one spelling, at least LYRICS_REPAIR_VOCABULARY_MIN times, + and never kept it as it is: "sie" is "się" all through a song, while + "co" that is sometimes "coś" is left alone. Everything else, time stamps + included, stays as it was.""" + text = unicodedata.normalize("NFC", text or "") + spans = word_spans(text) + words = [text[a:b] for a, b in spans] + lower = [w.lower() for w in words] + ref = [unicodedata.normalize("NFC", r).lower() for r in reference] + stripper = _Stripper() + pairs = _align(lower, ref, stripper) + paired = dict(pairs) + + replaced: dict[int, str] = {} + # How each word is spelled where the reference is sure of it: a pair + # with a neighbour paired too, or any pair that kept it as it is. + spelled: dict[str, Counter[str]] = {} + for i, j in pairs: + anchored = paired.get(i - 1) == j - 1 or paired.get(i + 1) == j + 1 + if anchored or ref[j] == lower[i]: + spelled.setdefault(lower[i], Counter())[ref[j]] += 1 + both = paired.get(i - 1) == j - 1 and paired.get(i + 1) == j + 1 + if (anchored and stripper.restores(ref[j], lower[i])) or ( + both and stripper.restores_short(ref[j], lower[i]) + ): + replaced[i] = ref[j] + for i, word in enumerate(lower): + if i in replaced or not word.isascii(): + continue + spellings = spelled.get(word) + if not spellings or len(spellings) != 1: + continue + ((spelling, count),) = spellings.items() + # A word paired on its own is filled only when its pair agrees. + if i in paired and ref[paired[i]] != spelling: + continue + if count >= LYRICS_REPAIR_VOCABULARY_MIN and stripper.restores(spelling, word): + replaced[i] = spelling + + out: list[str] = [] + last = 0 + for i, (a, b) in enumerate(spans): + if i in replaced: + out.append(text[last:a]) + out.append(_cased_like(replaced[i], words[i])) + last = b + out.append(text[last:]) + return Restoration("".join(out), len(words), len(pairs), len(replaced)) + + +def _borne_out(restoration: Restoration) -> bool: + """Whether a transcription agrees that a copy lost its letters: enough of + its words paired with what was heard, and enough of those given letters + back. A copy that was written without them pairs just as well and gets + next to none, since what is sung has none either.""" + return ( + restoration.restored >= LYRICS_REPAIR_MIN_RESTORED + and restoration.aligned >= restoration.words * LYRICS_REPAIR_MIN_ALIGNED + and restoration.restored >= restoration.aligned * LYRICS_REPAIR_MIN_RESTORED_SHARE + ) + + +def mend_from_transcript(entry: dict[str, Any], heard: list[str]) -> dict[str, Any] | None: + """lyrics.json ``entry`` with the letters its words lost taken from + ``heard``, the words a transcription of the song heard, in order: + "repaired": "whisper", its timing untouched. None when the transcription + does not bear out that anything was lost.""" + synced = restore_letters(entry.get("synced") or "", heard) + plain = restore_letters(entry.get("plain") or "", heard) + if not _borne_out(synced if entry.get("synced") else plain): + return None + return {**entry, "synced": synced.text, "plain": plain.text, "repaired": "whisper"} diff --git a/app/pipeline/lyrics_retime.py b/app/pipeline/lyrics_retime.py new file mode 100644 index 00000000..c9a7c669 --- /dev/null +++ b/app/pipeline/lyrics_retime.py @@ -0,0 +1,2118 @@ +"""Synced lyrics timed line by line to the track's own singing. + +lyrics_align.py moves a whole LRC by one amount, which is enough when the +copy is the same recording with more or less lead-in. It is not enough when +LRCLIB's copy was timed to another cut or arrangement: a verse the track +lacks, a chorus sung once more, a live take at another pace, or a copy whose +own stamps were never right. Here each line is found where it is sung. + +How, for a vocals stem and the lyrics' lines: + +1. Whisper hears the vocals with word timestamps, in the worker process that + transcribe.py runs (transcribe_worker.py), in the lyrics' language when + their text shows it (script_language, text_language). Its transcript is + kept beside the lyrics (LYRICS_TRANSCRIPT_FILE), so timing them again, or + after another version is picked, costs no second pass; a transcript + mend_lyrics already made is used the same way. The lyrics are not given + to Whisper as a prompt: measured on the benchmark, a prompt made it copy + the lyrics' own misspellings and lose whole lines (Dawid Podsiadlo + "Malomiasteczkowy": 99% of the lyrics matched without, 27% with), and it + ran three to four times as long. +2. The lyrics' words and the heard words become tokens, script by script: + a Latin (or Cyrillic, Greek...) word folded for case and accents, one + token per Chinese character or kana (katakana as hiragana, traditional as + simplified, zh_variants.py), one per Hangul syllable. Words in brackets + in a line with words outside them are a backing vocal sung over the lead + and count half: they take the echo Whisper hears after the lead's words, + never the lead's own. Credits at the top of a copy ("作词 : ...") and + pause marks ("♪") are not sung and are not timed. +3. A monotonic alignment (a dynamic program with free gaps on both sides and + a bonus for runs of consecutive matches) pairs lyric tokens with heard + ones: exact tokens score 1, Latin words close by edit distance and Hangul + syllables one letter off (a misheard word) a little less. Free gaps are + what let a verse the track lacks, or a chorus sung once more than the text + has, go unmatched rather than pull the rest out of place; monotonic order + keeps a repeated chorus with its own occurrence, and a small cost for + each heard word skipped between matches keeps it with its first one. +4. A line is anchored where its first matched word starts, when enough of it + matched in a run and the voice is really there (the vocal envelope, + lyrics_align._envelope): a word heard in a silence anchors nothing. Whisper + often starts the first word of a line on the held end of the line before; + the start is moved to where the voice comes back after a pause, or rises + (or, for a word held long, rises again twice as sharply), within that + word. When Whisper missed the first words of a phrase and heard nothing + else in it, the line starts where the phrase does. +5. Anchors the LRC's own stamps show are out of place (a phrase matched where + it is sung again) are dropped, and when the stamps follow the anchors, the + alignment runs again with each line's words looked for only near where + the stamps and the anchors around it put it. +6. The lines between anchors are placed with the song's own timing: by the + LRC's stamps mapped between the two anchors when those agree with them, + else spread over the singing between (not linearly across an instrumental + gap), each start put on the nearest onset of the voice. A line stamped the + same as the one before it (a translation) is shown with that one. +7. A run of lines still not found where there is singing (a chorus sung + once more at the end, which Whisper skips) is heard again on its own + (the worker's --clips), and the lines it then finds are kept when they + sit where the copy's stamps and the lines found around them say. +8. Words inside a line are stamped where they were heard, and the rest + spread between those by their syllables. + +Measured on the benchmark and left out: the lead vocal +stem of the lead/backing split (better on 4 of 16 songs, worse on 7, and 2 +to 3 minutes of CPU a song), a band-pass and levelled stem (within noise, +and fewer words matched), and a spectral-flux onset cue for legato line +starts (worse in 5 of 10 languages). + +The result is kept in lyrics.json as ``aligned``, only when it passes the +quality gate (retime_gate, no_worse): {"synced": enhanced LRC, "method": +"whisper", "matched": the share of the lyrics' syllables matched to heard +words, "lines_matched", "lines", "at": unix time}. The source ``synced`` is +never rewritten here. + +What the page shows, first present wins: + +1. ``user_synced``: a full manual timing from the sync editor (PUT + .../lyrics/user-synced, validated by clean_user_synced: its lines are the + lyrics' own, one to one). +2. ``aligned.synced``: this module's result. +3. ``synced`` moved by ``offset_sec``: the source, with the Align panel's + whole-song offset. + +When it runs: after separation for LRCLIB lyrics whose timing the voice +does not confirm line for line (a copy of another length, shifted or not, +or one of the track's length no whole-song shift confirms), under the transcribe_lyrics setting's terms (auto +means a CUDA job only), inside the pipeline's lock (retime_after_separation); +and on demand, from POST .../lyrics/retime (app/api/jobs.py), behind the same +lock. +""" + +from __future__ import annotations + +import bisect +import json +import logging +import math +import re +import threading +import time +import unicodedata +from collections.abc import Callable +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +import numpy as np + +from app.core.config import ( + LYRICS_RETIME_MAX_CELLS, + LYRICS_RETIME_MIN_LINES_SHARE, + LYRICS_RETIME_MIN_MATCHED, + LYRICS_TIMED_MAX_CHARS, + LYRICS_TRANSCRIPT_FILE, + LYRICS_USER_SYNCED_MAX_LINES, +) +from app.pipeline.lyrics_align import ( + _HIT_RISE_DB, + _LOUD_PERCENTILE, + _RISE_FRAMES, + _SILENT_DB, + _VOICE_FLOOR_DB, + _VOICE_RANGE_DB, + _offset, + onset_strength, +) +from app.pipeline.zh_variants import to_simplified + +logger = logging.getLogger("stemdeck.lyrics") + +# ── tokens ── + +_APOSTROPHES = frozenset("'’`ʼ") +# Latin letters Unicode does not build from a base letter and an accent. +_PLAIN_LETTERS = str.maketrans( + {"ł": "l", "ø": "o", "đ": "d", "ð": "d", "ß": "ss", "æ": "ae", "œ": "oe", "ı": "i", "þ": "th"} +) +_VOWELS = re.compile(r"[aeiouy]+") +# A small kana or the long-vowel mark belongs to the syllable before it. +_SMALL_KANA = frozenset("ゃゅょャュョっッーぁぃぅぇぉァィゥェォ") + + +def _han_or_kana(ch: str) -> bool: + cp = ord(ch) + return ( + 0x3400 <= cp <= 0x4DBF + or 0x4E00 <= cp <= 0x9FFF + or 0xF900 <= cp <= 0xFAFF + or 0x20000 <= cp <= 0x3FFFF + or 0x3040 <= cp <= 0x30FF + or 0x31F0 <= cp <= 0x31FF + or 0xFF66 <= cp <= 0xFF9F + ) + + +def _hangul(ch: str) -> bool: + return 0xAC00 <= ord(ch) <= 0xD7AF + + +def _cjk(ch: str) -> bool: + return _han_or_kana(ch) or _hangul(ch) + + +def _word_char(ch: str) -> bool: + return ch.isalnum() and not _cjk(ch) + + +def _latin_key(word: str) -> str: + text = unicodedata.normalize("NFKC", word).casefold().translate(_PLAIN_LETTERS) + text = unicodedata.normalize("NFKD", text) + return "".join(ch for ch in text if unicodedata.category(ch) != "Mn") + + +def _char_key(ch: str) -> str: + ch = to_simplified(unicodedata.normalize("NFKC", ch)) + cp = ord(ch[0]) if ch else 0 + # Katakana as hiragana: the same sound written either way. + if 0x30A1 <= cp <= 0x30F6: + return chr(cp - 0x60) + return ch + + +def tokens_of(text: str) -> list[tuple[str, int, int, float]]: + """The tokens of ``text`` as (key, start, end, syllables), in order, with + ``start`` and ``end`` its character offsets: a word of letters and digits + (apostrophes inside it dropped, "don't" is "dont"), else one Chinese + character, kana or Hangul syllable each.""" + out: list[tuple[str, int, int, float]] = [] + i, n = 0, len(text) + while i < n: + ch = text[i] + if _cjk(ch): + out.append((_char_key(ch), i, i + 1, 0.3 if ch in _SMALL_KANA else 1.0)) + i += 1 + continue + if _word_char(ch): + j = i + while j < n and ( + _word_char(text[j]) + or (text[j] in _APOSTROPHES and j + 1 < n and _word_char(text[j + 1])) + ): + j += 1 + key = _latin_key("".join(c for c in text[i:j] if c not in _APOSTROPHES)) + if key: + syllables = len(_VOWELS.findall(key)) + sum(c.isdigit() for c in key) + out.append((key, i, j, float(max(1, syllables)))) + i = j + continue + i += 1 + return out + + +def _has_word(text: str) -> bool: + return any(_word_char(c) or _cjk(c) for c in text) + + +def pieces_of(text: str) -> list[tuple[int, int]]: + """Where the word stamps of a line go: (start, end) character offsets of + each piece, in order. A piece is a word as the text spaces it, except + that each Chinese character or kana is one of its own (those lines do not + space their words), and a Latin word after one starts another.""" + pieces: list[tuple[int, int]] = [] + start: int | None = None + prev_cjk = False + for i, ch in enumerate(text): + if ch.isspace(): + if start is not None: + pieces.append((start, i)) + start = None + prev_cjk = False + continue + cjk = _han_or_kana(ch) + if start is None: + start = i + elif cjk or (prev_cjk and _word_char(ch)): + pieces.append((start, i)) + start = i + if cjk or _word_char(ch) or _hangul(ch): + prev_cjk = cjk + if start is not None: + pieces.append((start, len(text))) + # A piece of punctuation alone joins the one before it (or after, first). + merged: list[tuple[int, int]] = [] + for a, b in pieces: + if merged and not _has_word(text[a:b]): + merged[-1] = (merged[-1][0], b) + continue + merged.append((a, b)) + if len(merged) > 1 and not _has_word(text[merged[0][0] : merged[0][1]]): + merged[1] = (merged[0][0], merged[1][1]) + merged.pop(0) + return merged + + +# ── lines ── + +_LINE_STAMPS = re.compile(r"^(?:\s*\[\d{1,3}:\d{1,2}(?:[.:]\d{1,3})?\])+") +_LINE_STAMP = re.compile(r"\[(\d{1,3}):(\d{1,2}(?:[.:]\d{1,3})?)\]") +_WORD_STAMP = re.compile(r"<\d{1,3}:\d{1,2}(?:[.:]\d{1,3})?>") +_OPEN, _CLOSE = "([（【", ")]）】" + + +def line_text(raw: str) -> str: + """A line's text as the page shows it: word stamps out, spaces collapsed, + composed (parseLrc in static/js/lyricsLookup.js).""" + return " ".join(unicodedata.normalize("NFC", _WORD_STAMP.sub("", raw)).split()) + + +@dataclass +class Line: + text: str + prior: float | None + pieces: list[tuple[int, int]] = field(default_factory=list) + # Indices into the lyric's token list. + tokens: list[int] = field(default_factory=list) + + +def synced_lines(synced: str) -> list[tuple[float, str]]: + """Every sung line of an LRC as (seconds, text), earliest first: a line + stamped more than once (a chorus written once) is each of its times, the + [offset:] tag is applied, and lines with no words are left out.""" + offset = _offset(synced) + out: list[tuple[float, str]] = [] + for raw in synced.splitlines(): + stamps = _LINE_STAMPS.match(raw) + if not stamps: + continue + text = line_text(raw[stamps.end() :]) + if not text: + continue + for m in _LINE_STAMP.finditer(stamps.group(0)): + seconds = int(m.group(1)) * 60 + float(m.group(2).replace(":", ".")) + out.append((max(0.0, seconds - offset), text)) + out.sort(key=lambda pair: pair[0]) + return out + + +def plain_lines(plain: str) -> list[str]: + return [t for t in (line_text(raw) for raw in plain.splitlines()) if t] + + +@dataclass +class Lyric: + lines: list[Line] + keys: list[str] + weights: list[float] + token_line: list[int] + token_piece: list[int] + # Tokens in brackets in a line with words outside them: a backing vocal + # sung over the lead, which Whisper hears little of. + aside: list[bool] + + +def _bracketed(text: str) -> list[bool]: + """For each character of ``text``, whether it is inside brackets.""" + depth = 0 + out: list[bool] = [] + for ch in text: + if ch in _OPEN: + depth += 1 + out.append(depth > 0) + if ch in _CLOSE and depth: + depth -= 1 + return out + + +def build_lyric(pairs: list[tuple[float | None, str]]) -> Lyric: + """The lines (stamp or None, text) as tokens.""" + lyric = Lyric([], [], [], [], [], []) + for prior, text in pairs: + line = Line(text=text, prior=prior, pieces=pieces_of(text)) + starts = [a for a, _b in line.pieces] + toks = tokens_of(text) + bracketed = _bracketed(text) + inside = [bracketed[a] for _k, a, _b, _s in toks] + if all(inside): + inside = [False] * len(toks) + for (key, a, _b, syllables), side in zip(toks, inside, strict=True): + line.tokens.append(len(lyric.keys)) + lyric.keys.append(key) + lyric.weights.append(syllables) + lyric.token_line.append(len(lyric.lines)) + lyric.token_piece.append(max(0, bisect.bisect_right(starts, a) - 1)) + lyric.aside.append(side) + lyric.lines.append(line) + return lyric + + +# ── the voice ── + + +@dataclass +class Voice: + """The vocals stem's envelope: where it is singing and where it starts.""" + + hop: float + voiced: np.ndarray + strength: np.ndarray + cumulative: np.ndarray + + @classmethod + def of(cls, hop: float, db: Any) -> Voice: + """From an envelope, a level in dB every ``hop`` seconds. Singing is + within _VOICE_RANGE_DB of the stem's loud parts, as the page's wipe + has it (lyrics_align).""" + levels = np.asarray(db, dtype=float) + voiced = np.zeros(levels.size, dtype=bool) + if levels.size: + loud = float(np.percentile(levels, _LOUD_PERCENTILE)) + if loud >= _SILENT_DB: + voiced = levels >= max(loud - _VOICE_RANGE_DB, _VOICE_FLOOR_DB) + cumulative = np.concatenate(([0.0], np.cumsum(voiced) * hop)) + return cls(hop, voiced, onset_strength(levels), cumulative) + + def frame(self, t: float) -> int: + return int(min(max(0, round(t / self.hop)), max(0, self.voiced.size - 1))) + + def voiced_near(self, t: float, within: float) -> bool: + if not self.voiced.size: + return True + return bool(self.voiced[self.frame(t - within) : self.frame(t + within) + 1].any()) + + def voiced_between(self, a: float, b: float) -> float: + if not self.voiced.size or b <= a: + return 0.0 + return float(self.cumulative[self.frame(b)] - self.cumulative[self.frame(a)]) + + def at_voiced(self, a: float, amount: float) -> float: + """The sung moment by which ``amount`` seconds of singing follow + ``a``: with 0, where the singing next starts.""" + target = self.cumulative[self.frame(a)] + amount + k = int(np.searchsorted(self.cumulative, target, side="right")) - 1 + if k >= self.voiced.size: + return a + return max(a, k * self.hop) + + def end_of_singing(self, t: float, within: float) -> float: + """Where the singing going on at ``t`` stops, at most ``within`` + seconds on: the held end of a line belongs to it.""" + k, last = self.frame(t), self.frame(t + within) + while k < last and self.voiced[k]: + k += 1 + return max(t, k * self.hop) + + def onset_near(self, t: float, within: float) -> float | None: + """Where the voice rises most within ``within`` seconds of ``t``, + as the time the singing starts, or None when it does not clearly.""" + if not self.strength.size: + return None + lo, hi = self.frame(t - within), self.frame(t + within) + window = self.strength[lo : hi + 1] + if not window.size or window.max() < _HIT_RISE_DB: + return None + return self._rise_time(lo + int(np.argmax(window))) + + def _rise_time(self, peak: int) -> float: + # The rise is measured over _RISE_FRAMES of the level smoothed over + # three: it peaks a frame after a step in the level. + return max(0.0, (peak - (_RISE_FRAMES - 1) / 2) * self.hop) + + def rise_start(self, start: float, end: float) -> float: + """When a heard word that starts a line really starts. Whisper often + starts it on the held end of the line before: when the voice does not + rise at ``start`` but does within the word (or just past its end), + the word starts at that rise.""" + if not self.strength.size: + return start + at = self.frame(start) + here = float(self.strength[max(0, at - 2) : at + 3].max(initial=0)) + if here >= _HIT_RISE_DB: + return self._stronger_rise(start, end, here) + span = min( + _RISE_SEARCH_MAX_SEC, max(_RISE_SEARCH_MIN_SEC, end - start + _RISE_PAST_END_SEC) + ) + window = self.strength[at : self.frame(start + span) + 1] + above = np.flatnonzero(window >= _HIT_RISE_DB) + if not above.size: + return start + k = int(above[0]) + while k + 1 < window.size and window[k + 1] >= window[k]: + k += 1 + return max(start, self._rise_time(at + k)) + + def _stronger_rise(self, start: float, end: float, here: float) -> float: + """A word held longer than _HELD_WORD_SEC whose voice rises again, + _STRONGER times as sharply as at its start, before its end: Whisper + started it on the tail of the note before, and it starts at the + stronger rise.""" + if end - start < _HELD_WORD_SEC: + return start + lo, hi = self.frame(start + 0.15), self.frame(end - 0.05) + window = self.strength[lo : hi + 1] + if not window.size: + return start + k = int(np.argmax(window)) + if window[k] < max(_STRONGER * here, 2 * _HIT_RISE_DB): + return start + return self._rise_time(lo + k) + + def phrase_start(self, t: float, gap: float) -> float | None: + """Where the singing going on at ``t`` started: back through the + voice until a silence of ``gap`` seconds. None when there is no + singing at ``t``.""" + if not self.voiced.size: + return None + k = self.frame(t) + # The word may start a frame before the voice is loud enough. + while k < self.voiced.size - 1 and not self.voiced[k] and k < self.frame(t + 0.1): + k += 1 + if not self.voiced[k]: + return None + need = max(1, round(gap / self.hop)) + quiet = 0 + while k > 0: + if self.voiced[k - 1]: + quiet = 0 + else: + quiet += 1 + if quiet >= need: + return (k - 1 + quiet) * self.hop + k -= 1 + return 0.0 + + def first_voiced(self) -> float | None: + idx = np.flatnonzero(self.voiced) + return float(idx[0] * self.hop) if idx.size else None + + def last_voiced(self) -> float | None: + idx = np.flatnonzero(self.voiced) + return float((idx[-1] + 1) * self.hop) if idx.size else None + + +# How far into a line's first heard word its start is looked for. +_RISE_SEARCH_MIN_SEC = 0.4 +_RISE_SEARCH_MAX_SEC = 1.5 +# Whisper can end that word where the rise is: looked for a little past it. +_RISE_PAST_END_SEC = 0.1 +# A first word held this long, whose voice rises again this many times as +# sharply as at its start, starts at that rise. +_HELD_WORD_SEC = 0.7 +_STRONGER = 2.0 + + +# ── what was heard ── + + +@dataclass +class Heard: + keys: list[str] + starts: np.ndarray + ends: np.ndarray + # The end of the whole heard word each token is part of. + word_ends: np.ndarray + + +# A pause in the voice at least this long inside a heard word is before it... +_PAUSE_SEC = 0.15 +# ...when it starts in this first share of the word. +_PAUSE_WITHIN = 0.8 + + +def voiced_start(start: float, end: float, voice: Voice | None) -> float: + """When a heard word really starts. Whisper often starts the first word + after a pause at the end of the word before, a second or so early, and + stretches it over the pause: when the voice pauses for _PAUSE_SEC or more + in the first _PAUSE_WITHIN of the word, it starts where the voice comes + back after the last such pause.""" + if voice is None or not voice.voiced.size or end - start <= _PAUSE_SEC: + return start + lo, hi = voice.frame(start), voice.frame(end) + latest = voice.frame(start + (end - start) * _PAUSE_WITHIN) + quiet = ~voice.voiced[lo:hi] + need = max(1, round(_PAUSE_SEC / voice.hop)) + back = None + k = 0 + while k < quiet.size: + if not quiet[k]: + k += 1 + continue + first = k + while k < quiet.size and quiet[k]: + k += 1 + if k - first >= need and lo + first <= latest and k < quiet.size: + back = k + return (lo + back) * voice.hop if back is not None else start + + +def heard_tokens(words: list[dict[str, Any]], voice: Voice | None = None) -> Heard: + """Whisper's words as tokens, each word's time split between its tokens + by their syllables, each word's start moved past a pause in the voice + it begins with (voiced_start).""" + keys: list[str] = [] + starts: list[float] = [] + ends: list[float] = [] + word_ends: list[float] = [] + for word in words: + toks = tokens_of(str(word.get("text", ""))) + if not toks: + continue + start, end = float(word["start"]), max(float(word["end"]), float(word["start"])) + start = voiced_start(start, end, voice) + total = sum(t[3] for t in toks) + at = start + for key, _a, _b, syllables in toks: + span = (end - start) * syllables / total if total else 0.0 + keys.append(key) + starts.append(at) + ends.append(at + span) + word_ends.append(end) + at += span + return Heard( + keys, + np.asarray(starts, dtype=float), + np.asarray(ends, dtype=float), + np.asarray(word_ends, dtype=float), + ) + + +# ── matching ── + +# What a match of a backing vocal's word (an aside) is worth: it takes the +# echo Whisper hears after the lead's words, never the lead's own. +_ASIDE_SCALE = 0.5 +# A misheard Latin word still counts, a little less, when it is this close. +_FUZZY_MIN = 0.67 +_FUZZY_SCALE = 0.9 +# A word of two letters or fewer only counts as itself. +_FUZZY_MIN_LEN = 3 +# A Hangul syllable that shares two of its three letters with the heard one. +_HANGUL_NEAR = 0.6 +# What a match right after a match adds: runs anchor, stray words do not. +_RUN_BONUS = 0.5 +# What each heard token skipped between two matches costs. +_GAP_COST = 0.1 +_NEG = -1e9 + + +def _levenshtein(a: str, b: str) -> int: + if len(a) < len(b): + a, b = b, a + previous = list(range(len(b) + 1)) + for i, ca in enumerate(a, 1): + current = [i] + for j, cb in enumerate(b, 1): + current.append(min(previous[j] + 1, current[j - 1] + 1, previous[j - 1] + (ca != cb))) + previous = current + return previous[-1] + + +def similarity(a: str, b: str) -> float: + """How alike two Latin tokens are, 0 to 1: 1 - edit distance / length.""" + if a == b: + return 1.0 + if len(a) < _FUZZY_MIN_LEN or len(b) < _FUZZY_MIN_LEN: + return 0.0 + longest = max(len(a), len(b)) + if abs(len(a) - len(b)) > longest * (1 - _FUZZY_MIN): + return 0.0 + return 1.0 - _levenshtein(a, b) / longest + + +def _jamo(ch: str) -> tuple[int, int, int]: + """A Hangul syllable's lead consonant, vowel and final consonant.""" + code = ord(ch) - 0xAC00 + return code // 588, (code % 588) // 28, code % 28 + + +def _jamo_alike(a: str, b: str) -> bool: + return sum(x == y for x, y in zip(_jamo(a), _jamo(b), strict=True)) >= 2 + + +def score_table(lyric_keys: list[str], heard_keys: list[str]) -> tuple: + """(table, lyric ids, heard ids): table[u, v] is the score of lyric key + u against heard key v, _NEG when they do not match at all.""" + u_keys = sorted(set(lyric_keys)) + v_keys = sorted(set(heard_keys)) + u_index = {k: i for i, k in enumerate(u_keys)} + v_index = {k: i for i, k in enumerate(v_keys)} + table = np.full((len(u_keys), len(v_keys)), _NEG, dtype=np.float64) + for k, i in u_index.items(): + j = v_index.get(k) + if j is not None: + table[i, j] = 1.0 + latin_v = [(k, j) for k, j in v_index.items() if not _cjk(k[0])] + hangul_v = [(k, j) for k, j in v_index.items() if _hangul(k[0])] + for k, i in u_index.items(): + if _hangul(k[0]): + for hk, j in hangul_v: + if hk != k and _jamo_alike(k, hk): + table[i, j] = _HANGUL_NEAR + continue + if _cjk(k[0]) or len(k) < _FUZZY_MIN_LEN: + continue + for hk, j in latin_v: + if hk == k or abs(len(hk) - len(k)) > 3: + continue + sim = similarity(k, hk) + if sim >= _FUZZY_MIN: + table[i, j] = sim * _FUZZY_SCALE + lyric_ids = np.asarray([u_index[k] for k in lyric_keys], dtype=np.int64) + heard_ids = np.asarray([v_index[k] for k in heard_keys], dtype=np.int64) + return table, lyric_ids, heard_ids + + +def align_tokens( + lyric_keys: list[str], + heard_keys: list[str], + heard_starts: np.ndarray | None = None, + windows: tuple[np.ndarray, np.ndarray] | None = None, + scale: np.ndarray | None = None, +) -> list[tuple[int, int, float]]: + """The best monotonic pairing of lyric tokens with heard tokens, as + (lyric index, heard index, score), in order. Lyric tokens unheard are + free to skip, and heard tokens before the first match and after the + last; each match scores its similarity, plus _RUN_BONUS when it follows + the match of the tokens just before on both sides, less _GAP_COST for + each heard token skipped between matches. One row of the + dynamic program at a time, in numpy; only the back pointers are kept, two + bytes a cell. Empty when there is nothing to pair or the two are too + long to pair (LYRICS_RETIME_MAX_CELLS). With ``windows`` (per lyric + token, the earliest and latest start) and ``heard_starts``, a token only + pairs with heard tokens starting within its window.""" + n, m = len(lyric_keys), len(heard_keys) + if not n or not m or n * m > LYRICS_RETIME_MAX_CELLS: + return [] + table, lyric_ids, heard_ids = score_table(lyric_keys, heard_keys) + d_prev = np.zeros(m + 1) + m_prev = np.full(m + 1, _NEG) + # from_run[i, j]: the match at (i, j) follows a match. how[i, j]: what + # the best score up to (i, j) ends with: 0 a match here, 1 a lyric token + # skipped, 2 a heard token skipped. + from_run = np.zeros((n + 1, m + 1), dtype=np.bool_) + how = np.zeros((n + 1, m + 1), dtype=np.int8) + how[1:, 0] = 1 + cost = _GAP_COST * np.arange(m + 1) + for i in range(1, n + 1): + scores = table[lyric_ids[i - 1]][heard_ids] + if scale is not None: + scores = np.where(scores > 0, scores * scale[i - 1], scores) + if windows is not None and heard_starts is not None: + lo, hi = windows[0][i - 1], windows[1][i - 1] + scores = np.where((heard_starts >= lo) & (heard_starts <= hi), scores, _NEG) + run = m_prev[:-1] + _RUN_BONUS + chain = run > d_prev[:-1] + m_row = np.full(m + 1, _NEG) + m_row[1:] = np.where(scores > 0, scores + np.where(chain, run, d_prev[:-1]), _NEG) + from_run[i, 1:] = chain + here = np.maximum(m_row, d_prev) + # Heard tokens skipped between matches cost _GAP_COST each: of two + # pairings that hear the lines about as well, the one that sings them + # without a long wait between (a chorus written once is its first + # time, not one a minute on). Kept in the running maximum as + # here[k] + cost * k. + lifted = here + cost + best = np.maximum.accumulate(lifted) + d_row = best - cost + earlier = np.concatenate(([False], best[1:] == best[:-1])) + how[i] = np.where((best > lifted) | earlier, 2, np.where(m_row >= d_prev, 0, 1)).astype( + np.int8 + ) + how[i, 0] = 1 + d_prev, m_prev = d_row, m_row + pairs: list[tuple[int, int, float]] = [] + # Heard tokens after the last match cost nothing: the best end, earliest. + i, j, in_match = n, int(np.argmax(d_prev)), False + while i > 0 and j > 0: + if not in_match: + step = how[i, j] + if step == 0: + in_match = True + elif step == 1: + i -= 1 + else: + j -= 1 + continue + pairs.append((i - 1, j - 1, float(table[lyric_ids[i - 1], heard_ids[j - 1]]))) + in_match = bool(from_run[i, j]) + i, j = i - 1, j - 1 + pairs.reverse() + return pairs + + +# ── retiming ── + +# A line is anchored when at least this share of its syllables matched... +_ANCHOR_MIN_SHARE = 0.34 +# ...with at least two of its tokens matched in a run (one-token lines: one). +_ANCHOR_MIN_RUN = 2 +# The seconds a syllable takes when nothing better is known. +_DEFAULT_RATE = 0.28 +_MIN_RATE, _MAX_RATE = 0.08, 0.9 +# An anchor with no singing this close is a word heard in a silence. +_VOICE_CHECK_SEC = 0.35 +# The singing going on when a line's last word ends is that line's for at +# most this long. +_HELD_SEC = 1.0 +# A placed line starts on an onset of the voice this close to its estimate. +_SNAP_SEC = 0.6 +# LRC stamps between two anchors are trusted when their spacing agrees with +# the anchors' within this ratio. +_PRIOR_AGREE = 0.12 +# Lines this close apart are one after the other, not at once. +_MIN_GAP = 0.05 +# Two matched words of a line are sung together when no more than this many +# seconds apart, plus this many a syllable between them. +_CLUSTER_GAP_SEC = 1.5 +_CLUSTER_SEC_PER_SYLLABLE = 0.8 +# ...or any distance apart, when next to each other in what was heard and +# the voice sounds for this share of the time between them: a held note. +_HELD_VOICED = 0.8 +# An anchor this many seconds off its neighbours' shift from the stamps, on +# both sides, while those agree among themselves within _AGREE_SEC, is a +# phrase matched where it is sung again, not this line. +_OUTLIER_SEC = 3.0 +_AGREE_SEC = 1.0 +# A second pass looks for each line's words within this many seconds of +# where its neighbours' shifts from the stamps put it, given at least this +# many anchors, and is kept when it matches at least this share of what the +# first matched. +_WINDOW_SEC = 8.0 +_WINDOW_MIN_ANCHORS = 4 +_WINDOW_KEEP = 0.9 + +Anchor = tuple[float, float, float] # (start, end, seconds a syllable or 0) + + +@dataclass +class Timed: + start: float + end: float + words: list[float] + anchored: bool + + +@dataclass +class Retimed: + lines: list[Line] + timed: list[Timed] + matched: float + lines_matched: int + # The lines that were timed: all but those shown with another + # (retime_lines), which have nothing of their own to be found by. + counted: int = -1 + + def __post_init__(self) -> None: + if self.counted < 0: + self.counted = len(self.lines) + + +def _line_rate(points: list[tuple[float, float]]) -> float | None: + """Seconds a syllable from (cumulative syllables, time) points.""" + if len(points) < 2: + return None + (s0, t0), (s1, t1) = points[0], points[-1] + if s1 <= s0 or t1 <= t0: + return None + return min(_MAX_RATE, max(_MIN_RATE, (t1 - t0) / (s1 - s0))) + + +def _companions(pairs: list[tuple[float | None, str]]) -> list[int | None]: + """For each line, the line it is shown with, or None: a line stamped the + same as the one before it is its translation or romanisation (bilingual + LRC), never sung apart from it.""" + out: list[int | None] = [None] * len(pairs) + for i in range(1, len(pairs)): + here, before = pairs[i][0], pairs[i - 1][0] + if here is not None and before is not None and abs(here - before) < 0.005: + leader = out[i - 1] + out[i] = leader if leader is not None else i - 1 + return out + + +# A credit a copy lists before the singing: "作词 : 米津玄師", "Lyrics: ...". +_CREDIT = re.compile( + r"^\s*(?:作?[词詞]|作?曲|[编編]曲|制作人?|製作人?|[监監]製|[监監]制|演唱|原唱|和声|和聲|" + r"混音|母带|母帶|吉他|贝斯|貝斯|鼓|键盘|鍵盤|弦乐|弦樂|出品|发行|發行|" + r"lyrics?|music|composer|composed|written|arrange[dr]?|producer|produced)" + r"\s*(?:by)?\s*[:：]", + re.IGNORECASE, +) + + +def _credits(pairs: list[tuple[float | None, str]]) -> set[int]: + """The lines at the top of a copy that are its credits, not sung: those + matching _CREDIT, and a "Title - Artist" line just before them.""" + out: set[int] = set() + for i, (_t, text) in enumerate(pairs): + if _CREDIT.match(text): + out.add(i) + elif i > len(out) + 1: + break + if out and min(out) == 1 and " - " in pairs[0][1]: + out.add(0) + return out + + +def retime_lines( + pairs: list[tuple[float | None, str]], + words: list[dict[str, Any]], + voice: Voice | None, + duration: float, +) -> Retimed: + """The lyrics' lines as (stamp or None, text) timed to the heard words: + retime() of all but the lines shown with another (_companions), which + take that one's timing, and the credits at the top (_credits), which + are shown before the singing.""" + companions = _companions(pairs) + credits = _credits(pairs) + # A line with no word ("♪", "...") marks a pause: it keeps its place in + # the copy's own timing, moved with the line before it. + marks = {i for i, (_t, text) in enumerate(pairs) if not tokens_of(text)} + main = [ + i for i, c in enumerate(companions) if c is None and i not in credits and i not in marks + ] + heard = heard_tokens(words, voice) + result = retime(build_lyric([pairs[i] for i in main]), heard, voice, duration) + if len(main) == len(pairs): + return result + full = build_lyric(pairs) + placed = dict(zip(main, result.timed, strict=True)) + first = min((t.start for t in result.timed), default=0.0) + timed: list[Timed] = [] + for i, line in enumerate(full.lines): + if i in placed: + timed.append(placed[i]) + continue + if i in marks and i not in credits and companions[i] is None: + timed.append(_mark_timing(i, pairs, placed, line)) + continue + if i in credits or companions[i] not in placed: + # Shown before the singing, as briefly as the copy has them. + at = max(0.0, min(pairs[i][0] or 0.0, first - _MIN_GAP * (len(credits) - i))) + timed.append(Timed(at, at, [at] * len(line.pieces), anchored=True)) + continue + leader = placed[companions[i]] # type: ignore[index] + step = (leader.end - leader.start) / max(1, len(line.pieces)) + words_at = [leader.start + step * k for k in range(len(line.pieces))] + timed.append(Timed(leader.start, leader.end, words_at, anchored=leader.anchored)) + return Retimed(full.lines, timed, result.matched, result.lines_matched, len(main)) + + +def _mark_timing( + i: int, pairs: list[tuple[float | None, str]], placed: dict[int, Timed], line: Line +) -> Timed: + """A pause mark's timing: its stamp moved as the line before it was + (after it, when it leads), or just after that line's end without one.""" + before = [k for k in placed if k < i] + after = [k for k in placed if k > i] + prior = pairs[i][0] + at = 0.0 + if before: + lead = placed[max(before)] + at = lead.end + _MIN_GAP + mark_prior, lead_prior = prior, pairs[max(before)][0] + if mark_prior is not None and lead_prior is not None: + at = max(at, lead.start + (mark_prior - lead_prior)) + elif after and prior is not None and pairs[min(after)][0] is not None: + at = max(0.0, placed[min(after)].start - (pairs[min(after)][0] - prior)) # type: ignore[operator] + if after: + at = min(at, placed[min(after)].start - _MIN_GAP) + at = max(0.0, at) + return Timed(at, at, [at] * len(line.pieces), anchored=True) + + +def retime(lyric: Lyric, heard: Heard, voice: Voice | None, duration: float) -> Retimed: + """Each line's start and end, and each piece's start, on the track.""" + lines = lyric.lines + # An aside's words count for less: they are sung over the lead's. + dp_keys = lyric.keys + scale = np.where(np.asarray(lyric.aside, dtype=bool), _ASIDE_SCALE, 1.0) + total_w = sum(w for w, side in zip(lyric.weights, lyric.aside, strict=True) if not side) + pairs = align_tokens(dp_keys, heard.keys, scale=scale) + matched_at = _compact(lyric, heard, voice, {i: j for i, j, _s in pairs}) + anchors, rates = _anchor(lyric, heard, voice, matched_at) + anchors = _drop_outliers(lines, anchors) + # Again, each line's words looked for only near where its stamp and the + # anchors around it put it, when those agree: a phrase sung again + # elsewhere then cannot take a line away from its place. + windows = _windows(lyric, anchors) + if windows is not None: + pairs = align_tokens(dp_keys, heard.keys, heard.starts, windows, scale) + near = _compact(lyric, heard, voice, {i: j for i, j, _s in pairs}) + # Kept unless it loses much of what the first pass heard: stamps + # that say nothing about where lines are sung would. + if _weight(lyric, near) >= _WINDOW_KEEP * _weight(lyric, matched_at): + matched_at = near + anchors, rates = _anchor(lyric, heard, voice, matched_at) + anchors = _drop_outliers(lines, anchors) + matched_w = _weight(lyric, matched_at) + default_rate = float(np.median(rates)) if rates else _DEFAULT_RATE + if voice is not None: + anchors = _phrase_starts(heard, voice, anchors) + + # Anchors must run forward; the alignment makes them, but the back-off + # for unmatched first words can cross the line before. + kept: dict[int, Anchor] = {} + last_start = -1.0 + for li in sorted(anchors): + s, e, r = anchors[li] + if s <= last_start: + s = last_start + _MIN_GAP + kept[li] = (max(0.0, s), max(e, s), r) + last_start = kept[li][0] + anchors = kept + + starts: list[float | None] = [ + anchors[li][0] if li in anchors else None for li in range(len(lines)) + ] + song_end = duration + if voice is not None: + last = voice.last_voiced() + if last is not None: + song_end = min(duration, last + 1.0) if duration else last + 1.0 + _place_unanchored(lines, lyric, starts, anchors, voice, song_end, default_rate) + for li in range(1, len(starts)): + starts[li] = max(starts[li] or 0.0, starts[li - 1] or 0.0) + + timed: list[Timed] = [] + for li, line in enumerate(lines): + start = float(starts[li] or 0.0) + following = next((s for s in starts[li + 1 :] if s is not None and s > start), None) + limit = following - _MIN_GAP if following is not None else max(song_end, start + 1.0) + timed.append( + _words_of_line( + line, lyric, heard, matched_at, start, limit, anchors.get(li), default_rate + ) + ) + return Retimed(lines, timed, matched_w / (total_w or 1.0), len(anchors)) + + +# A line starts where its phrase does, up to this long before its first heard +# word, when nothing else was heard in between. +_PHRASE_BACK_SEC = 2.5 +# Silence this long ends a phrase; a gap shorter than this is within one. +_PHRASE_GAP_SEC = 0.25 + + +def _phrase_starts( + heard: Heard, + voice: Voice, + anchors: dict[int, Anchor], +) -> dict[int, Anchor]: + """The anchors, each moved back to the start of the phrase its first + heard word is sung in, when Whisper heard nothing of that phrase before + the word. Whisper drops the first words of a phrase often, a repeated + word most of all ("Bailando, bailando": it hears the second), and the + line then starts where the voice does, after a silence. Never back past + the end of the line before, nor further than _PHRASE_BACK_SEC.""" + out: dict[int, Anchor] = {} + floor = 0.0 + everything = np.sort(heard.starts) + for li in sorted(anchors): + start, end, rate = anchors[li] + phrase = voice.phrase_start(start, _PHRASE_GAP_SEC) + if ( + phrase is not None + and floor <= phrase < start - _PHRASE_GAP_SEC + and start - phrase <= _PHRASE_BACK_SEC + ): + between = np.searchsorted(everything, [phrase, start - 0.05]) + if between[1] == between[0]: + start = phrase + out[li] = (start, end, rate) + floor = voice.end_of_singing(end, _HELD_SEC) + return out + + +def _weight(lyric: Lyric, matched_at: dict[int, int]) -> float: + return sum(lyric.weights[i] for i in matched_at if not lyric.aside[i]) + + +def _compact( + lyric: Lyric, heard: Heard, voice: Voice | None, matched_at: dict[int, int] +) -> dict[int, int]: + """The matches, less those of a line's words heard far from the rest of + it: a line whose last words were matched to a chorus sung again later + keeps only the group of its matches that weighs most. Two words heard + one after the other with the voice going on between them are together + however long the first is held: Whisper ended Piaf's long "Non" two + seconds before "rien", and the line lost its first word.""" + by_line: dict[int, list[int]] = {} + for t in sorted(matched_at): + by_line.setdefault(lyric.token_line[t], []).append(t) + kept: dict[int, int] = {} + for toks in by_line.values(): + groups: list[list[int]] = [[toks[0]]] + for a, b in zip(toks, toks[1:], strict=False): + between = sum(lyric.weights[x] for x in range(a + 1, b)) + gap = float(heard.starts[matched_at[b]] - heard.ends[matched_at[a]]) + held = ( + voice is not None + and matched_at[b] == matched_at[a] + 1 + and voice.voiced_between( + float(heard.ends[matched_at[a]]), float(heard.starts[matched_at[b]]) + ) + >= _HELD_VOICED * gap + ) + if not held and gap > _CLUSTER_GAP_SEC + _CLUSTER_SEC_PER_SYLLABLE * between: + groups.append([]) + groups[-1].append(b) + best = max(groups, key=lambda g: sum(lyric.weights[t] for t in g)) + kept.update({t: matched_at[t] for t in best}) + return kept + + +def _anchor( + lyric: Lyric, heard: Heard, voice: Voice | None, matched_at: dict[int, int] +) -> tuple[dict[int, Anchor], list[float]]: + """The lines found where they are sung, {line: (start, end, pace)}, and + each one's pace (seconds a syllable).""" + anchors: dict[int, Anchor] = {} + rates: list[float] = [] + for li, line in enumerate(lyric.lines): + toks = [t for t in line.tokens if not lyric.aside[t]] + if not toks: + continue + cum = 0.0 + points: list[tuple[float, float, int]] = [] # (syllables before, time, token) + best_run = run = 0 + prev_j: int | None = None + for t in toks: + j = matched_at.get(t) + if j is not None: + points.append((cum, float(heard.starts[j]), t)) + run = run + 1 if prev_j is not None and j - prev_j <= 2 else 1 + best_run = max(best_run, run) + prev_j = j + else: + run, prev_j = 0, None + cum += lyric.weights[t] + share = sum(lyric.weights[p[2]] for p in points) / cum if cum else 0.0 + if share < _ANCHOR_MIN_SHARE or best_run < min(_ANCHOR_MIN_RUN, len(toks)): + continue + first_t = points[0][1] + if voice is not None: + if not voice.voiced_near(first_t, _VOICE_CHECK_SEC): + continue + first_t = voice.rise_start(first_t, float(heard.word_ends[matched_at[points[0][2]]])) + rate = _line_rate([(p[0], p[1]) for p in points]) + if rate is not None: + rates.append(rate) + end = float(heard.ends[matched_at[points[-1][2]]]) + start = _line_start(lyric, heard, matched_at, toks, points, first_t, rate, voice) + anchors[li] = (start, end, rate or 0.0) + return anchors, rates + + +def _line_start( + lyric: Lyric, + heard: Heard, + matched_at: dict[int, int], + toks: list[int], + points: list[tuple[float, float, int]], + first_t: float, + rate: float | None, + voice: Voice | None, +) -> float: + """When a line starts whose first matched token is not its first: as + many heard tokens before that one as the line has unmatched tokens + before it, when those heard tokens matched nothing and are sung with it + (misheard words, one for one), else back from the first match by the + line's pace.""" + before = toks.index(points[0][2]) + if before == 0: + return first_t + j = matched_at[points[0][2]] + used = set(matched_at.values()) + k = j - before + near = _CLUSTER_GAP_SEC + _CLUSTER_SEC_PER_SYLLABLE * points[0][0] + if ( + k >= 0 + and not any(x in used for x in range(k, j)) + and first_t - float(heard.starts[k]) <= near + ): + start = float(heard.starts[k]) + if voice is not None: + start = voice.rise_start(start, float(heard.word_ends[k])) + return min(start, first_t) + return first_t - points[0][0] * (rate or _DEFAULT_RATE) + + +def _shifts(lines: list[Line], anchors: dict[int, Anchor]) -> tuple[list[int], list[float]]: + """The anchored lines with stamps, in order, and how far each moved.""" + order = [li for li in sorted(anchors) if lines[li].prior is not None] + return order, [anchors[li][0] - lines[li].prior for li in order] # type: ignore[operator] + + +def _drop_outliers(lines: list[Line], anchors: dict[int, Anchor]) -> dict[int, Anchor]: + """The anchors, less those the LRC's own stamps show are out of place. + A song timed to another cut moves by a step where the cut is, and each + side of the step agrees with itself; a line matched to a repeat of its + words elsewhere agrees with neither side. Stamps that agree with nothing + (a copy never timed right) drop nothing.""" + order, shift = _shifts(lines, anchors) + dropped: set[int] = set() + for k, li in enumerate(order): + sides = [s for s in (shift[max(0, k - 2) : k], shift[k + 1 : k + 3]) if len(s) == 2] + if not sides or any(max(s) - min(s) > _AGREE_SEC for s in sides): + continue + if all(abs(shift[k] - float(np.median(s))) > _OUTLIER_SEC for s in sides): + dropped.add(li) + return {li: v for li, v in anchors.items() if li not in dropped} + + +def _windows(lyric: Lyric, anchors: dict[int, Anchor]) -> tuple[np.ndarray, np.ndarray] | None: + """For each lyric token, the earliest and latest time its heard match may + start: its line's stamp moved by the shifts of the anchors either side + (as far as the nearer of them disagree), _WINDOW_SEC wider. None when + there are too few anchors with stamps to tell.""" + lines = lyric.lines + order, shift = _shifts(lines, anchors) + if len(order) < _WINDOW_MIN_ANCHORS: + return None + lo = np.full(len(lyric.keys), -np.inf) + hi = np.full(len(lyric.keys), np.inf) + for li, line in enumerate(lines): + if line.prior is None: + continue + k = bisect.bisect_left(order, li) + near = shift[max(0, k - 2) : k + 2] + for t in line.tokens: + lo[t] = line.prior + min(near) - _WINDOW_SEC + hi[t] = line.prior + max(near) + _WINDOW_SEC + return lo, hi + + +def _syllables(lyric: Lyric, line: Line) -> float: + return sum(lyric.weights[t] for t in line.tokens) or 1.0 + + +def _place_unanchored( + lines: list[Line], + lyric: Lyric, + starts: list[float | None], + anchors: dict[int, Anchor], + voice: Voice | None, + song_end: float, + rate: float, +) -> None: + """Fill ``starts`` for the lines between anchors, in place.""" + n = len(lines) + li = 0 + while li < n: + if starts[li] is not None: + li += 1 + continue + a = li - 1 # the anchored line before the run, or -1 + b = li + while b < n and starts[b] is None: + b += 1 + run = list(range(li, b)) + lo = anchors[a][1] if a >= 0 else 0.0 + if a >= 0 and voice is not None and voice.voiced.size: + lo = voice.end_of_singing(lo, _HELD_SEC) + hi = float(starts[b]) if b < n else song_end # type: ignore[arg-type] + if a < 0 and voice is not None: + first = voice.first_voiced() + if first is not None and first < hi: + lo = max(lo, first - 0.2) + lo = min(lo, hi) + placed = _by_prior(lines, run, a, b, starts, lo, hi) + if placed is None: + placed = _by_voice(lines, lyric, run, lo, hi, voice, rate) + prev = starts[a] if a >= 0 else -_MIN_GAP + for k, t in zip(run, placed, strict=True): + if voice is not None: + snapped = voice.onset_near(t, _SNAP_SEC) + if snapped is not None and lo - _MIN_GAP <= snapped <= hi - _MIN_GAP: + t = snapped + t = max(t, (prev or 0.0) + _MIN_GAP) + if b < n: + # Never past the line after the run: the order is the text's. + t = min(t, hi) + starts[k] = t + prev = t + li = b + + +def _by_prior( + lines: list[Line], + run: list[int], + a: int, + b: int, + starts: list[float | None], + lo: float, + hi: float, +) -> list[float] | None: + """The run's starts from the LRC's own stamps, mapped onto the anchors + around it, when those stamps agree with the anchors: None when they do + not, or there are no stamps.""" + if any(lines[k].prior is None for k in run): + return None + pa = lines[a].prior if a >= 0 else None + pb = lines[b].prior if b < len(lines) else None + sa = starts[a] if a >= 0 else None + sb = starts[b] if b < len(lines) else None + if pa is not None and pb is not None and sa is not None and sb is not None: + if pb <= pa or sb <= sa: + return None + ratio = (sb - sa) / (pb - pa) + if abs(ratio - 1) > _PRIOR_AGREE: + return None + return [sa + (lines[k].prior - pa) * ratio for k in run] # type: ignore[operator] + # One anchor only (the run leads or ends the song): moved with it, when + # that keeps the whole run inside the time it has. + if pa is not None and sa is not None: + placed = [sa + (lines[k].prior - pa) for k in run] # type: ignore[operator] + elif pb is not None and sb is not None: + placed = [sb - (pb - lines[k].prior) for k in run] # type: ignore[operator] + else: + return None + if placed[0] < lo - 1.0 or placed[-1] > hi: + return None + return placed + + +def _by_voice( + lines: list[Line], + lyric: Lyric, + run: list[int], + lo: float, + hi: float, + voice: Voice | None, + rate: float, +) -> list[float]: + """The run's starts spread over the singing between ``lo`` and ``hi`` + by their syllables; evenly over the time when there is no singing there + (lines the track does not sing: they pass quickly).""" + weights = [_syllables(lyric, lines[k]) for k in run] + total = sum(weights) + sung = voice.voiced_between(lo, hi) if voice is not None else 0.0 + out: list[float] = [] + before = 0.0 + if voice is not None and sung >= 0.4 * len(run): + # No more singing than the lines need at the usual pace: the rest of + # the window is somebody else's (a chorus the text does not repeat). + need = min(sung, total * rate * 1.5) + for w in weights: + out.append(voice.at_voiced(lo, need * before / total)) + before += w + return out + span = max(0.0, hi - lo) + for w in weights: + out.append(lo + span * before / total) + before += w + return out + + +def _words_of_line( + line: Line, + lyric: Lyric, + heard: Heard, + matched_at: dict[int, int], + start: float, + limit: float, + anchor: Anchor | None, + default_rate: float, +) -> Timed: + """Each piece's start in a line starting at ``start`` and over by + ``limit``, and the line's end: where its words were heard, the rest + spread between those by syllables.""" + n_pieces = len(line.pieces) + limit = max(limit, start) + rate = (anchor[2] if anchor and anchor[2] else 0.0) or default_rate + piece_w = [0.0] * n_pieces + known: dict[int, float] = {} + for t in line.tokens: + p = lyric.token_piece[t] + j = matched_at.get(t) if anchor is not None else None + if j is not None and p not in known: + # The piece starts when its first matched token is heard, less + # the syllables of the piece before that token. + known[p] = float(heard.starts[j]) - piece_w[p] * rate + piece_w[p] += lyric.weights[t] + piece_w = [w or 0.5 for w in piece_w] + cum = [0.0] + for w in piece_w: + cum.append(cum[-1] + w) + points = [(0, start)] + for p in sorted(known): + if p == 0: + continue + points.append((p, min(max(known[p], points[-1][1]), limit))) + if anchor is not None: + last_known = points[-1][0] + own = cum[last_known + 1] - cum[last_known] if last_known < n_pieces else 0.0 + tail = cum[n_pieces] - cum[last_known + 1] if last_known + 1 <= n_pieces else 0.0 + end = max(anchor[1], points[-1][1] + own * rate) + tail * rate + else: + end = start + cum[n_pieces] * rate + end = max(min(end, limit), points[-1][1]) + points.append((n_pieces, end)) + words: list[float] = [] + k = 0 + for p in range(n_pieces): + while k + 1 < len(points) and points[k + 1][0] <= p: + k += 1 + p0, t0 = points[k] + if p0 == p: + words.append(t0) + continue + p1, t1 = points[k + 1] if k + 1 < len(points) else (n_pieces, end) + span_w = cum[p1] - cum[p0] + words.append(t0 + (t1 - t0) * ((cum[p] - cum[p0]) / span_w if span_w > 0 else 0.0)) + for p in range(1, n_pieces): + words[p] = max(words[p], words[p - 1]) + if words: + words[0] = start + return Timed(start=start, end=max(end, start), words=words, anchored=anchor is not None) + + +# ── LRC out ── + +# A pause this long after a line clears it (an empty stamped line). +_CLEAR_AFTER_SEC = 4.0 + + +def _stamp(seconds: float) -> str: + centis = max(0, round(seconds * 100)) + minutes, rest = divmod(centis, 6000) + return f"{minutes:02d}:{rest // 100:02d}.{rest % 100:02d}" + + +def to_lrc(result: Retimed) -> str: + """The retimed lines as enhanced LRC: each line its start, each piece its + word stamp, the last its end; an empty line where a long pause follows.""" + rows = sorted(zip(result.lines, result.timed, strict=True), key=lambda r: r[1].start) + out: list[str] = [] + for k, (line, timed) in enumerate(rows): + text = line.text + parts = [f"[{_stamp(timed.start)}]"] + cursor = 0 + for (a, _b), t in zip(line.pieces, timed.words, strict=True): + parts.append(text[cursor:a]) + parts.append(f"<{_stamp(t)}>") + cursor = a + parts.append(text[cursor:]) + if line.pieces: + parts.append(f"<{_stamp(timed.end)}>") + out.append("".join(parts)) + following = rows[k + 1][1].start if k + 1 < len(rows) else None + if following is None or following - timed.end >= _CLEAR_AFTER_SEC: + out.append(f"[{_stamp(timed.end + 0.5)}]") + return "\n".join(out) + + +# ── the gate ── + + +def onset_fit(starts: list[float], voice: Voice) -> float: + """The share of line starts with the voice rising within 0.25 s.""" + if not starts or not voice.strength.size: + return 0.0 + return sum(voice.onset_near(t, 0.25) is not None for t in starts) / len(starts) + + +def retime_gate(result: Retimed) -> bool: + """Whether a result is good enough to show: enough of the lyrics heard, + and enough lines found where they are sung.""" + lines = result.counted + return ( + lines > 0 + and result.matched >= LYRICS_RETIME_MIN_MATCHED + and result.lines_matched / lines >= LYRICS_RETIME_MIN_LINES_SHARE + ) + + +# ── the transcript ── + + +def transcript_words(result: dict[str, Any]) -> list[dict[str, Any]]: + """The timed words of a worker answer: [{"text", "start", "end"}]. + + Looser than the segments transcribe.py keeps as lyrics: a chorus sung + four times compresses as well as a loop Whisper made up, so the + compression ratio drops nothing here. Only a window Whisper itself took + for silence goes; a word made up over an instrumental anchors nothing, + since an anchor needs the voice (Voice.voiced_near).""" + from app.pipeline.transcribe import _LOGPROB_FLOOR, _NO_SPEECH_PROB, _number, _words + + words: list[dict[str, Any]] = [] + segments = result.get("segments") + for segment in segments if isinstance(segments, list) else []: + if not isinstance(segment, dict): + continue + no_speech = _number(segment.get("no_speech_prob")) or 0.0 + logprob = _number(segment.get("avg_logprob")) or 0.0 + if no_speech > _NO_SPEECH_PROB and logprob < _LOGPROB_FLOOR: + continue + words.extend(_words(segment, words[-1]["start"] if words else 0.0)) + return words + + +def transcript_path(job_dir: Path) -> Path: + return job_dir / LYRICS_TRANSCRIPT_FILE + + +def _vocals_stamp(job_dir: Path) -> list[int] | None: + """The vocals stem's size and modification time: what a kept transcript + was heard from. A stem made again (a re-separation writing it in place) + no longer matches, and its transcript is heard afresh.""" + try: + st = (job_dir / "stems" / "vocals.wav").stat() + except OSError: + return None + return [st.st_size, st.st_mtime_ns] + + +def save_transcript( + job_dir: Path, + result: dict[str, Any] | None = None, + *, + words: list[dict[str, Any]] | None = None, + language: str | None = None, + refined: bool = False, +) -> None: + """Keep heard words beside the lyrics, for timing them again without a + second pass: a worker answer's, or ``words`` already taken from one + (``refined``: with the lines a first timing missed heard again). Best + effort.""" + from app.pipeline.lyrics_lookup import _write_json_atomic + + if result is not None: + words = transcript_words(result) + language = result.get("language") if isinstance(result.get("language"), str) else None + data = { + "v": 1, + "vocals": _vocals_stamp(job_dir), + "language": language, + "refined": refined, + "words": [[round(w["start"], 3), round(w["end"], 3), w["text"]] for w in words or []], + } + try: + _write_json_atomic(transcript_path(job_dir), data) + except OSError: + logger.info("could not keep the transcript in %s", job_dir.name, exc_info=True) + + +def read_transcript(job_dir: Path) -> dict[str, Any] | None: + """The kept transcript: {"language", "refined", "words": [{"text", + "start", "end"}]}, or None when there is none or it is damaged.""" + try: + data = json.loads(transcript_path(job_dir).read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + if not isinstance(data, dict) or data.get("v") != 1 or not isinstance(data.get("words"), list): + return None + # Heard from another vocals stem than the one there now. A transcript kept + # before this was recorded has no stamp and is taken as it is. + kept = data.get("vocals") + if kept is not None and kept != _vocals_stamp(job_dir): + return None + words = [] + for row in data["words"]: + if ( + isinstance(row, list) + and len(row) == 3 + and all(isinstance(x, (int, float)) and not isinstance(x, bool) for x in row[:2]) + and isinstance(row[2], str) + and math.isfinite(row[0]) + and math.isfinite(row[1]) + ): + words.append({"start": float(row[0]), "end": float(row[1]), "text": row[2]}) + language = data.get("language") + return { + "language": language if isinstance(language, str) else None, + "refined": data.get("refined") is True, + "words": words, + } + + +# ── hearing the missed lines again ── + +# A run of at least this many lines not found is heard again when there is +# at least this much singing where it goes, heard this much wider on each +# side (only the words inside the run's own time replace the first ones). +_CLIP_MIN_LINES = 2 +_CLIP_MIN_SUNG_SEC = 3.0 +_CLIP_PAD_SEC = 0.4 + + +def missed_clips( + result: Retimed, voice: Voice | None, duration: float +) -> list[tuple[float, float]]: + """Where the runs of lines not found go, as (start, end) seconds of the + stem, when there is singing there: between the end of the line found + before and the start of the one found after.""" + if voice is None: + return [] + timed = sorted(result.timed, key=lambda t: t.start) + clips: list[tuple[float, float]] = [] + k = 0 + while k < len(timed): + if timed[k].anchored: + k += 1 + continue + j = k + while j < len(timed) and not timed[j].anchored: + j += 1 + lo = timed[k - 1].end if k > 0 else 0.0 + hi = timed[j].start if j < len(timed) else (duration or voice.hop * voice.voiced.size) + if j - k >= _CLIP_MIN_LINES and voice.voiced_between(lo, hi) >= _CLIP_MIN_SUNG_SEC: + clips.append((lo, hi)) + k = j + return clips + + +def merge_words( + words: list[dict[str, Any]], extra: list[dict[str, Any]], clips: list[tuple[float, float]] +) -> list[dict[str, Any]]: + """``words`` with those inside ``clips`` replaced by ``extra``, heard + again there, in time order.""" + + def inside(w: dict[str, Any]) -> bool: + return any(a <= w["start"] < b for a, b in clips) + + kept = [w for w in words if not inside(w)] + return sorted(kept + [w for w in extra if inside(w)], key=lambda w: w["start"]) + + +# A line found only when heard again is kept where the copy's own stamps, +# moved as the lines found around it were, put it within this much. +_CLIP_AGREE_SEC = 1.5 + + +def better(new: Retimed, old: Retimed, pairs: list[tuple[float | None, str]]) -> bool: + """Whether a timing with missed lines heard again is the better one: it + finds more lines, and every line it finds that the first did not sits + where the copy's stamps and the lines both found in the same place + around it say. A chorus + sung more often than the copy has it is heard again too, and its later + times must not take the copy's lines.""" + if (new.lines_matched, new.matched) <= (old.lines_matched, old.matched): + return False + # The lines both found in the same place: what the others go by. + shifts = [ + (k, t.start - p) + for k, (t, n, (p, _x)) in enumerate(zip(old.timed, new.timed, pairs, strict=True)) + if t.anchored and n.anchored and abs(t.start - n.start) < 0.3 and p is not None + ] + for k, (t, (prior, _x)) in enumerate(zip(new.timed, pairs, strict=True)): + if not t.anchored or old.timed[k].anchored: + continue + if prior is None or not shifts: + return False + near = sorted(shifts, key=lambda ks: abs(ks[0] - k))[:4] + expected = float(np.median([sh for _k, sh in near])) + if abs(t.start - prior - expected) > _CLIP_AGREE_SEC: + return False + return True + + +# ── the language ── + +# The commonest words of each language in Latin letters StemDeck's lyrics +# come in, only those no other of them shares: a text in one of them uses a +# handful a line. Whisper decides the language from 30 s of the vocals, and +# on a song with an English-sounding hook it can decide wrong (Nena +# "Leuchtturm": English, a transcription that matched 15% of the lyrics; in +# German, 89%). +_COMMON_WORDS = { + "en": "the you and i'm don't your that with what this love know just can't", + "de": "ich und nicht ist die der das mich dich mir dir wir auch noch wenn", + "fr": "je tu et le les est pas qui une moi toi dans pour c'est mon ne j'ai", + "es": "yo y el los las pero más como está eres tú mí porque", + "pt": "eu você e não é meu minha uma com mais tem sou quando", + "pl": "nie się że jak mnie już tylko jest ci dla czy", + "id": "aku kau kamu dan yang ini itu untuk dengan cinta hati akan", +} +_COMMON = {lang: frozenset(words.split()) for lang, words in _COMMON_WORDS.items()} +# At least this many common words, and this many times the next language's. +_LANGUAGE_MIN_HITS = 8 +_LANGUAGE_MIN_LEAD = 1.5 +_WORDS = re.compile(r"[^\W\d_]+(?:'[^\W\d_]+)?") + + +def text_language(text: str) -> str | None: + """The language of lyrics in Latin letters, from their commonest words, + or None when they do not show it clearly.""" + words = _WORDS.findall(text.casefold()) + scores = sorted( + ((sum(w in common for w in words), lang) for lang, common in _COMMON.items()), + reverse=True, + ) + (best, lang), (second, _other) = scores[0], scores[1] + if best >= _LANGUAGE_MIN_HITS and best >= _LANGUAGE_MIN_LEAD * second: + return lang + return None + + +def script_language(text: str) -> str | None: + """The language a text's script alone shows: Korean for Hangul, + Japanese for kana, Chinese for Chinese characters with neither. None for + anything else.""" + hangul = kana = han = 0 + for ch in text: + cp = ord(ch) + if 0xAC00 <= cp <= 0xD7AF: + hangul += 1 + elif 0x3040 <= cp <= 0x30FF: + kana += 1 + elif _han_or_kana(ch): + han += 1 + letters = sum(ch.isalpha() for ch in text) or 1 + if hangul / letters > 0.3: + return "ko" + if kana / letters > 0.1: + return "ja" + if han / letters > 0.3: + return "zh" + return None + + +# ── the formats ── + + +def lyric_pairs(entry: dict[str, Any]) -> list[tuple[float | None, str]]: + """The lyrics' lines as (stamp, text), in time order, one per stamp; + (None, text) for text-only lyrics.""" + if entry.get("synced"): + return synced_lines(entry["synced"]) + return [(None, text) for text in plain_lines(entry.get("plain") or "")] + + +def lyric_line_texts(entry: dict[str, Any]) -> list[str]: + """The lyrics' own lines, as any timing of them must give them.""" + return [text for _t, text in lyric_pairs(entry)] + + +_USER_ROW = re.compile(r"\s*\[(\d{1,3}):(\d{2}(?:\.\d{1,3})?)\](.*)") +_USER_WORD = re.compile(r"<(\d{1,3}):(\d{2}(?:\.\d{1,3})?)>") + + +def clean_user_synced(value: Any, lines: list[str]) -> str | None: + """A manual timing as kept, or None when it is not one of these lyrics: + LRC whose every non-empty line is one [mm:ss.xx] stamp and text (enhanced + word stamps allowed, no other tag), the texts exactly + ``lines`` in order (lyric_line_texts), and the line stamps never going + back in time. Empty stamped lines (a pause) are allowed anywhere.""" + if not isinstance(value, str) or not lines or len(value) > LYRICS_TIMED_MAX_CHARS: + return None + rows = [raw for raw in value.splitlines() if raw.strip()] + if len(rows) > LYRICS_USER_SYNCED_MAX_LINES: + return None + texts: list[str] = [] + last = -1.0 + for raw in rows: + m = _USER_ROW.fullmatch(raw) + if not m or float(m.group(2)) >= 60: + return None + seconds = int(m.group(1)) * 60 + float(m.group(2)) + if seconds < last: + return None + last = seconds + rest = m.group(3) + if any(float(w.group(2)) >= 60 for w in _USER_WORD.finditer(rest)): + return None + # Word stamps only: no other tag, and no stamp of another shape. + stripped = _WORD_STAMP.sub("", rest) + if re.search(r"\[\d|<\d", stripped): + return None + text = line_text(rest) + if text: + texts.append(text) + if texts != lines: + return None + return "\n".join(raw.strip() for raw in rows) + + +ALIGN_METHODS = frozenset(("whisper",)) + + +def _count(value: Any) -> int | None: + return value if type(value) is int and value >= 0 else None + + +def clean_aligned(value: Any, lines: list[str]) -> dict[str, Any] | None: + """The "aligned" of lyrics.json as kept, or None when it is malformed or + not a timing of ``lines`` (the lyrics' own, lyric_line_texts).""" + if not isinstance(value, dict) or value.get("method") not in ALIGN_METHODS: + return None + synced = value.get("synced") + if not isinstance(synced, str) or not synced or len(synced) > LYRICS_TIMED_MAX_CHARS: + return None + matched, at = value.get("matched"), value.get("at") + for number in (matched, at): + if isinstance(number, bool) or not isinstance(number, (int, float)): + return None + if not (math.isfinite(matched) and 0 <= matched <= 1 and math.isfinite(at) and at >= 0): + return None + lines_matched, count = _count(value.get("lines_matched")), _count(value.get("lines")) + if lines_matched is None or count is None or lines_matched > count: + return None + if not lines or [text for _t, text in synced_lines(synced)] != lines: + return None + return { + "synced": synced, + "method": value["method"], + "matched": float(matched), + "lines_matched": lines_matched, + "lines": count, + "at": float(at), + } + + +def aligned_entry(result: Retimed) -> dict[str, Any]: + return { + "synced": to_lrc(result), + "method": "whisper", + "matched": round(result.matched, 3), + "lines_matched": result.lines_matched, + "lines": len(result.lines), + "at": round(time.time(), 3), + } + + +# ── running it ── + +_STAGE = "Timing lyrics to the vocals..." +# The share of a run's progress the transcription takes; the rest is quick. +_TRANSCRIBE_SHARE = 0.9 +# A new timing is kept over the current one only when it puts at most this +# much smaller a share of its line starts on a rise of the voice. +_NO_WORSE_MARGIN = 0.05 +# A current timing counts as confirmed when a whole-song shift of at most +# this much fits it. +_CONFIRMED_SHIFT_SEC = 0.3 + + +class RetimeFailed(Exception): + """There is nothing to time, or the transcription failed.""" + + +@dataclass +class Outcome: + state: str # "done" or "unsure" + matched: float + lines_matched: int + lines: int + # The timing itself, for lyrics the browser keeps (nothing is written). + aligned: dict[str, Any] | None = None + + +def _language(entry: dict[str, Any], text: str) -> str: + """The language to transcribe in: the lyrics' own when known, else what + their script or their words show, else "" for Whisper to decide.""" + language = entry.get("language") + if isinstance(language, str) and re.fullmatch(r"[a-z]{2,3}", language): + return language + return script_language(text) or text_language(text) or "" + + +def _hear( + job: Any, + job_dir: Path, + language: str, + cancelled: Callable[[], bool], + report: Callable[[str, int | None], None] | None, + clips: list[tuple[float, float]] | None = None, +) -> dict[str, Any] | None: + """A worker answer for the job's vocals, or only ``clips`` of them; None + when the worker failed. Raises RetimeFailed without vocals, JobCancelled.""" + from app.pipeline.transcribe import _device, _run_worker, _spawn_worker_cmd + + vocals = job_dir / "stems" / "vocals.wav" + if not vocals.is_file(): + raise RetimeFailed("no vocals stem") + cmd = _spawn_worker_cmd(vocals, _device(job)) + if language: + cmd += ["--language", language] + if clips: + # Heard a little wider than the run, for the words at its edges. + padded = [(max(0.0, a - _CLIP_PAD_SEC), b + _CLIP_PAD_SEC) for a, b in clips] + cmd += ["--clips", ",".join(f"{a:.2f}:{b:.2f}" for a, b in padded)] + return _run_worker(job, cmd, cancelled=cancelled, report=report) + + +def _transcribe( + job: Any, + job_dir: Path, + language: str, + cancelled: Callable[[], bool], + report: Callable[[str, int | None], None], +) -> tuple[list[dict[str, Any]], bool]: + """The heard words, and whether missed lines were heard again already: + the kept transcript, else a new one (kept too). Raises RetimeFailed, or + JobCancelled.""" + kept = read_transcript(job_dir) + if kept is not None and kept["words"]: + return kept["words"], kept["refined"] + result = _hear(job, job_dir, language, cancelled, report) + if result is None: + raise RetimeFailed("transcription failed") + save_transcript(job_dir, result) + return transcript_words(result), False + + +def _current_starts(entry: dict[str, Any], voice: Voice) -> list[float] | None: + """The line starts the lyrics show now, when the voice confirms them: + their own stamps with the user's offset, moved by a whole-song shift of + at most _CONFIRMED_SHIFT_SEC. None when nothing confirms them.""" + from app.pipeline.lyrics_align import estimate_offset + + if not entry.get("synced"): + return None + offset = float(entry.get("offset_sec") or 0.0) + starts = sorted(t + offset for t, _text in synced_lines(entry["synced"])) + shift = estimate_offset(starts, voice.strength, voice.hop) + if shift is None or abs(shift) > _CONFIRMED_SHIFT_SEC: + return None + return [t + shift for t in starts] + + +def no_worse(entry: dict[str, Any], result: Retimed, voice: Voice | None) -> bool: + """Whether ``result`` is at least as good as the timing the lyrics have, + when the voice confirms that timing: judged by the share of line starts + on a rise of the voice. True when nothing confirms the current one.""" + if voice is None: + return True + current = _current_starts(entry, voice) + if current is None: + return True + new = onset_fit([t.start for t in result.timed], voice) + return new >= onset_fit(current, voice) - _NO_WORSE_MARGIN + + +def retime_job( + job: Any, + job_dir: Path, + *, + cancelled: Callable[[], bool], + report: Callable[[float], None] | None = None, + lyrics: dict[str, Any] | None = None, +) -> Outcome: + """Time the job's lyrics to its vocals and keep the result as their + "aligned" when it passes the gate (retime_gate, no_worse). Blocking: + may run a transcription of minutes. Raises RetimeFailed when there is + nothing to time or the transcription failed, JobCancelled when + ``cancelled()`` turned true; the lyrics are then left as they were. + + ``lyrics``, {"synced", "plain"}, are lyrics the browser keeps rather + than lyrics.json: timed the same way, and the result is only answered + (Outcome.aligned), never written.""" + from app.core.models import JobCancelled + from app.pipeline.lyrics_align import _envelope + from app.pipeline.lyrics_lookup import _OFFSET_LOCK, read_lyrics, write_lyrics + + entry = read_lyrics(job_dir) if lyrics is None else lyrics + if entry is None: + raise RetimeFailed("no lyrics") + pairs = lyric_pairs(entry) + if not pairs: + raise RetimeFailed("no lines") + + def progress(phase: str, pct: int | None) -> None: + if report is not None and phase == "transcribe" and pct is not None: + report(_TRANSCRIBE_SHARE * pct / 100) + + language = _language(entry, " ".join(text for _t, text in pairs)) + words, refined = _transcribe(job, job_dir, language, cancelled, progress) + if cancelled(): + raise JobCancelled() + envelope = _envelope(job_dir / "stems") + voice = Voice.of(*envelope) if envelope is not None else None + duration = float(getattr(job, "duration_sec", 0) or entry.get("duration") or 0) + result = retime_lines(pairs, words, voice, duration) + clips = [] if refined else missed_clips(result, voice, duration) + if clips: + # The lines not found, heard again on their own: Whisper often + # skips a chorus sung once more at the end, and hears it alone. + extra = _hear(job, job_dir, language, cancelled, None, clips) + if extra is not None: + merged = merge_words(words, transcript_words(extra), clips) + again = retime_lines(pairs, merged, voice, duration) + if better(again, result, pairs): + result, words = again, merged + # Refined either way: the same clips of the same vocals would be + # heard the same, so a re-listen that did not help is not tried + # again on the next run. + save_transcript(job_dir, words=words, language=language or None, refined=True) + if cancelled(): + raise JobCancelled() + outcome = Outcome("unsure", round(result.matched, 3), result.lines_matched, len(result.lines)) + if not retime_gate(result) or not no_worse(entry, result, voice): + logger.info( + "[%s] lyrics timing unsure: %.0f%% heard, %d of %d lines found", + job.id, + result.matched * 100, + result.lines_matched, + result.counted, + ) + return outcome + aligned = aligned_entry(result) + if lyrics is not None: + outcome.state, outcome.aligned = "done", aligned + return outcome + with _OFFSET_LOCK: + if cancelled(): + raise JobCancelled() + now = read_lyrics(job_dir) + # Other lyrics since (another version, letters mended): this timing + # is not theirs. + if now is None or lyric_line_texts(now) != [text for _t, text in pairs]: + raise RetimeFailed("the lyrics changed meanwhile") + if not write_lyrics(job, job_dir, {**now, "aligned": aligned}): + raise RetimeFailed("could not write lyrics") + logger.info( + "[%s] lyrics timed line by line: %.0f%% heard, %d of %d lines found", + job.id, + result.matched * 100, + result.lines_matched, + result.counted, + ) + outcome.state = "done" + return outcome + + +def wants_retime(job: Any, job_dir: Path) -> bool: + """Whether a job just separated gets its lyrics timed line by line: LRCLIB + lyrics that nothing has confirmed the timing of line by line (a version + of another length, moved by a whole-song shift or not, or one the + track's length the voice does not confirm), not aligned by hand nor + timed already, under the transcribe_lyrics setting (auto: a CUDA job + only).""" + from app.core.settings import transcribe_lyrics_enabled + from app.pipeline.lyrics_align import detect_offset + from app.pipeline.lyrics_lookup import read_lyrics + + if not transcribe_lyrics_enabled(getattr(job, "compute_device", None)): + return False + entry = read_lyrics(job_dir) + if entry is None or entry["source"] != "lrclib" or not entry["synced"]: + return False + if any(key in entry for key in ("aligned", "user_synced", "offset_sec")): + return False + # "shifted": a copy of another cut moved by one amount, which a cut + # inside the song (a verse more or less) leaves wrong from there on. + if entry["timing"] in ("unverified", "shifted"): + return True + return detect_offset(entry["synced"], job_dir / "stems") is None + + +def retime_after_separation(job: Any, job_dir: Path) -> bool: + """The pipeline's last step: time the lyrics line by line when + wants_retime. True when "aligned" was written. Never raises but + JobCancelled: lyrics are optional, and a failure here must not cost the + user a separation that has already succeeded.""" + from app.core.models import JobCancelled, _set + + try: + if job.cancel_requested: + raise JobCancelled() + if not wants_retime(job, job_dir): + return False + _set(job, stage=_STAGE) + outcome = retime_job(job, job_dir, cancelled=lambda: job.cancel_requested) + return outcome.state == "done" + except JobCancelled: + raise + except RetimeFailed as exc: + logger.info("[%s] lyrics not timed: %s", job.id, exc) + return False + except Exception: + logger.exception("[%s] lyrics timing failed", job.id) + return False + + +# ── on demand ── + + +@dataclass +class RetimeRun: + """One on-demand run of a job, for GET .../lyrics/retime: "running" + until it ends "done", "unsure", "failed" or, cancelled, "idle".""" + + state: str = "running" + progress: float = 0.0 + outcome: Outcome | None = None + cancel: threading.Event = field(default_factory=threading.Event) + # Set once it holds the pipeline lock: a run cancelled while waiting for + # it is over at once. + started: bool = False + # Set when its task has finished, whatever the outcome: from then on it + # holds no file of the job open (wait_run_ended). + ended: threading.Event = field(default_factory=threading.Event) + + def view(self) -> dict[str, Any]: + out: dict[str, Any] = {"state": self.state} + if self.state == "running": + out["progress"] = round(self.progress, 3) + if self.outcome is not None: + out["matched"] = self.outcome.matched + out["lines_matched"] = self.outcome.lines_matched + out["lines"] = self.outcome.lines + if self.outcome.aligned is not None: + out["aligned"] = self.outcome.aligned + return out + + +_RUNS: dict[str, RetimeRun] = {} +_RUNS_LOCK = threading.Lock() + + +def claim_run(job_id: str) -> RetimeRun | None: + """A new run for ``job_id``, or None while one is running already.""" + with _RUNS_LOCK: + current = _RUNS.get(job_id) + if current is not None and current.state == "running": + return None + run = RetimeRun() + _RUNS[job_id] = run + return run + + +def current_run(job_id: str) -> RetimeRun | None: + with _RUNS_LOCK: + return _RUNS.get(job_id) + + +def cancel_run(job_id: str) -> bool: + """Ask a running run of ``job_id`` to stop. True when one was running.""" + with _RUNS_LOCK: + run = _RUNS.get(job_id) + if run is None or run.state != "running": + return False + run.cancel.set() + if not run.started: + run.state = "idle" + return True + + +def wait_run_ended(job_id: str, timeout: float) -> bool: + """Wait up to ``timeout`` seconds for a run of ``job_id`` that got as far + as the vocals to let go of them. True when none holds them now. Deleting + a job waits on this: on Windows a file the transcription still has open + cannot be removed, and the delete would fail and leave the folder.""" + with _RUNS_LOCK: + run = _RUNS.get(job_id) + if run is None or not run.started: + return True + return run.ended.wait(timeout) + + +def forget_run(job_id: str) -> None: + with _RUNS_LOCK: + _RUNS.pop(job_id, None) + + +def saved_view(entry: dict[str, Any] | None) -> dict[str, Any]: + """The state of a job with no run since the server started: "done" when + its lyrics carry a line-by-line timing, else "idle".""" + aligned = (entry or {}).get("aligned") + if not aligned: + return {"state": "idle"} + return { + "state": "done", + "matched": aligned["matched"], + "lines_matched": aligned["lines_matched"], + "lines": aligned["lines"], + } diff --git a/app/pipeline/musicbrainz.py b/app/pipeline/musicbrainz.py new file mode 100644 index 00000000..0eaa7822 --- /dev/null +++ b/app/pipeline/musicbrainz.py @@ -0,0 +1,902 @@ +"""MusicBrainz, asked by everything in StemDeck that needs it. + +These questions, all read-only: + +* a recording by its MBID (what AcoustID matched a fingerprint to): its + credited title and artists, and the release groups it came out on; +* a recording search by artist, title and length, for a track with no + fingerprint match, kept only when it is unmistakable; +* a recording search by one reading of a track's title (title_parse.py), for + a track its tags do not name, kept only when the recording's credit and + release titles account for the title's other words; +* artists by name, sort name or alias, whatever the script (周杰倫 is also + Jay Chou, IU also 아이유), then a song among one artist's recordings: for a + title whose words no credit has as written, or a video longer than the + album cut; +* an artist's Wikidata item, from its url relationships, which is how the band + a recording credits is found without searching for its name; +* a release group's url relationships (Wikidata, IMDb), which is how the film + or musical a soundtrack is from is found (work_lookup.py). + +Every request takes its turn from ratelimit.MUSICBRAINZ (one a second across +the whole process, as MusicBrainz asks) and names StemDeck in its User-Agent. +Lookups by MBID are kept on disk, one file per MBID, so the same recording or +artist is never asked for twice within MUSICBRAINZ_CACHE_TTL_SEC. + +What is sent: an MBID, or names from the track: artist, song, album or show, +and its length. Nothing here +raises past its caller's except-clause: callers treat any failure as "not +found". +""" + +from __future__ import annotations + +import json +import logging +import re +import time +import urllib.error +import urllib.parse +import urllib.request +import uuid +from collections.abc import Callable +from pathlib import Path +from typing import Any + +from app.core.config import ( + IDENTIFY_DURATION_TOLERANCE_SEC, + IDENTIFY_MAX_BYTES, + IDENTIFY_VIDEO_EXTRA_SEC, + MUSICBRAINZ_API, + MUSICBRAINZ_CACHE_DIR, + MUSICBRAINZ_CACHE_TTL_SEC, + MUSICBRAINZ_REQUEST_BUDGET_SEC, + MUSICBRAINZ_RETRIES, + MUSICBRAINZ_RETRY_BACKOFF_SEC, + MUSICBRAINZ_RETRY_MAX_WAIT_SEC, + MUSICBRAINZ_SEARCH_MIN_SCORE, + MUSICBRAINZ_USER_AGENT, + TIMEOUT_IDENTIFY_REQUEST, + TITLE_MIN_COVERAGE, +) +from app.core.models import MBID_RE, clean_identity +from app.pipeline import ratelimit, title_parse +from app.pipeline.artist_lookup import _ssl_context +from app.pipeline.title_parse import TitleReading + +logger = logging.getLogger("stemdeck.identify") + +# Replaced by tests with a temporary directory. +CACHE_DIR: Path = MUSICBRAINZ_CACHE_DIR + +_SEARCH_LIMIT = "25" +# Words of a title put in one search, at most: a performer list can be long. +_TITLE_TERMS = 12 +_WIKIDATA_URL_RE = re.compile(r"^https?://(?:www\.)?wikidata\.org/wiki/(Q\d{1,12})$") +# What a release group is, best first, for choosing the album a recording is +# "from": the studio album over the single, and either over a compilation. +_PRIMARY_RANK = {"album": 0, "ep": 1, "single": 2} +_BRACKETS_RE = re.compile(r"\s*[(\[][^)\]]*[)\]]") + + +def _retry_wait(err: urllib.error.HTTPError, attempt: int) -> float: + """How long to wait before asking again after ``err``: what its + Retry-After says when that is a number of seconds, else the backoff for + this attempt, capped either way.""" + backoff = MUSICBRAINZ_RETRY_BACKOFF_SEC * 2**attempt + try: + asked = float((err.headers or {}).get("Retry-After") or "") + except (TypeError, ValueError): + asked = backoff + return max(0.0, min(asked, MUSICBRAINZ_RETRY_MAX_WAIT_SEC)) + + +def _fetch_json( + path: str, params: dict[str, str], *, cancelled: Callable[[], bool] = lambda: False +) -> Any: + """One MusicBrainz request, after waiting for its turn. Raises on any + failure, including a turn too far off (ratelimit.RateLimited). + + MusicBrainz answers 503 (or 429) when it sheds load, even to a client + keeping to its rate, so a refused request is asked again after a wait, up + to MUSICBRAINZ_RETRIES times, each on a turn of its own, and never past + MUSICBRAINZ_REQUEST_BUDGET_SEC in all. A job cancelled while it waits, + for its turn or to ask again, stops it (ratelimit.RateLimited).""" + query = urllib.parse.urlencode({**params, "fmt": "json"}) + url = f"{MUSICBRAINZ_API}/{path}?{query}" + request = urllib.request.Request( + url, headers={"User-Agent": MUSICBRAINZ_USER_AGENT, "Accept": "application/json"} + ) + deadline = time.monotonic() + MUSICBRAINZ_REQUEST_BUDGET_SEC + for attempt in range(MUSICBRAINZ_RETRIES + 1): + ratelimit.MUSICBRAINZ.wait(cancelled=cancelled) + try: + # A fixed https base with only a path of our own and a query built + # here, never a URL from a request or a tag, which is what B310 + # exists to catch. + with urllib.request.urlopen( # nosec B310 + request, timeout=TIMEOUT_IDENTIFY_REQUEST, context=_ssl_context() + ) as response: + body = response.read(IDENTIFY_MAX_BYTES + 1) + break + except urllib.error.HTTPError as err: + if err.code not in (429, 503) or attempt == MUSICBRAINZ_RETRIES: + raise + # Slept in short steps, so a cancelled job stops waiting promptly. + left = _retry_wait(err, attempt) + if time.monotonic() + left > deadline: + raise + while left > 0: + if cancelled(): + raise ratelimit.RateLimited("cancelled while waiting") from None + step = min(left, 0.25) + _sleep(step) + left -= step + if len(body) > IDENTIFY_MAX_BYTES: + raise ValueError("MusicBrainz answer too large") + return json.loads(body) + + +# Replaced by tests, so a retry costs no real time. +_sleep = time.sleep + + +# ── the cache ── + + +# What is kept, by the lookup it answers: "recording-works" is a recording +# with its works, asked apart from the recording's own lookup. +_CACHE_KINDS = ("recording", "artist", "release-group", "recording-works", "work") + + +def _cache_path(kind: str, mbid: str) -> Path | None: + # Both halves are ours, and the MBID is checked, so the name can never + # leave the cache directory. + if kind not in _CACHE_KINDS or not MBID_RE.match(mbid): + return None + return CACHE_DIR / f"{kind}-{mbid}.json" + + +def cache_get(kind: str, mbid: str) -> Any: + """What was kept for ``mbid``, or None when nothing fresh is.""" + path = _cache_path(kind, mbid) + if path is None: + return None + try: + kept = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + if not isinstance(kept, dict) or not isinstance(kept.get("at"), (int, float)): + return None + if time.time() - kept["at"] > MUSICBRAINZ_CACHE_TTL_SEC: + return None + return kept.get("data") + + +def cache_put(kind: str, mbid: str, data: Any) -> None: + """Keep ``data`` for ``mbid``. Best-effort: a cache that cannot be + written only means asking again next time.""" + path = _cache_path(kind, mbid) + if path is None: + return + try: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(f"{path.name}.{uuid.uuid4().hex}.tmp") + try: + tmp.write_text(json.dumps({"at": time.time(), "data": data}), encoding="utf-8") + tmp.replace(path) + finally: + tmp.unlink(missing_ok=True) + except OSError: + logger.info("could not cache MusicBrainz %s %s", kind, mbid, exc_info=True) + + +def _cached( + kind: str, + mbid: str, + path: str, + params: dict[str, str], + *, + cancelled: Callable[[], bool] = lambda: False, +) -> Any: + kept = cache_get(kind, mbid) + if kept is not None: + return kept + data = _fetch_json(path, params, cancelled=cancelled) + if isinstance(data, dict): + cache_put(kind, mbid, data) + return data + + +# ── reading an answer ── + + +def credited_artist(credit: Any) -> str: + """The artist credit as MusicBrainz prints it: each name and its join + phrase ("Queen & David Bowie").""" + if not isinstance(credit, list): + return "" + parts = [] + for entry in credit: + if not isinstance(entry, dict): + continue + artist = entry.get("artist") if isinstance(entry.get("artist"), dict) else {} + name = entry.get("name") or artist.get("name") or "" + joinphrase = entry.get("joinphrase") or "" + if isinstance(name, str) and isinstance(joinphrase, str): + parts.append(name + joinphrase) + return "".join(parts).strip() + + +def credited_artist_ids(credit: Any) -> list[str]: + """The MBIDs of every artist in a credit, in credit order.""" + if not isinstance(credit, list): + return [] + ids = [] + for entry in credit: + artist = entry.get("artist") if isinstance(entry, dict) else None + mbid = artist.get("id") if isinstance(artist, dict) else None + if isinstance(mbid, str) and MBID_RE.match(mbid) and mbid not in ids: + ids.append(mbid) + return ids + + +def _secondary_types(group: Any) -> set[str]: + types = group.get("secondary-types") if isinstance(group, dict) else None + return {t.lower() for t in types if isinstance(t, str)} if isinstance(types, list) else set() + + +def _release_group_rank(release: dict[str, Any], prefer: frozenset[str] = frozenset()) -> tuple: + group = release.get("release-group") if isinstance(release.get("release-group"), dict) else {} + types = _secondary_types(group) + primary = str(group.get("primary-type") or "").lower() + date = release.get("date") or group.get("first-release-date") or "" + return ( + # Named by the title the track came with ("Wicked", "The Greatest + # Showman"): the cast album, not a compilation it is also on. + bool(prefer) + and not prefer <= title_parse.words(group.get("title") or release.get("title")), + bool(types), + "live" in types, + "compilation" in types, + _PRIMARY_RANK.get(primary, 3 if primary else 4), + str(release.get("status") or "").lower() != "official", + str(date) if date else "9999", + ) + + +def best_release_group( + releases: Any, prefer: frozenset[str] = frozenset() +) -> dict[str, Any] | None: + """The release group a recording is best known from: a studio album over a + single or EP, any of them over a soundtrack, a soundtrack over a + compilation, anything over a live album, an official release over a + bootleg, the earliest over a reissue. A recording that only ever came out + on a soundtrack keeps the soundtrack, which is what tells the work lookup + to look for the film or show. ``prefer``, the words of a work the track's + title names, puts a release group whose title has them all first.""" + if not isinstance(releases, list): + return None + candidates = [ + r for r in releases if isinstance(r, dict) and isinstance(r.get("release-group"), dict) + ] + if not candidates: + return None + return min(candidates, key=lambda r: _release_group_rank(r, prefer))["release-group"] + + +_YEAR_RE = re.compile(r"^(\d{4})") + + +def _year(date: Any) -> int | None: + match = _YEAR_RE.match(date) if isinstance(date, str) else None + return int(match.group(1)) if match else None + + +def release_group_year(releases: Any, group: dict[str, Any]) -> int | None: + """The year a release group first came out: its first-release-date, as a + recording lookup carries it, else the earliest date among the recording's + releases in that group, since a search answer leaves the group's date + out. None when neither says.""" + year = _year(group.get("first-release-date")) + group_id = group.get("id") + if year or not group_id or not isinstance(releases, list): + return year + years = [ + _year(release.get("date")) + for release in releases + if isinstance(release, dict) + and isinstance(release.get("release-group"), dict) + and release["release-group"].get("id") == group_id + ] + return min((y for y in years if y), default=None) + + +def identity_from_recording( + recording: Any, + *, + source: str, + score: float, + fallback_duration: float | None = None, + prefer: frozenset[str] = frozenset(), +) -> dict[str, Any] | None: + """An identity (clean_identity's shape) from a MusicBrainz recording, as a + lookup or a search answers it, or None when it names nothing. ``prefer`` + as for best_release_group.""" + if not isinstance(recording, dict): + return None + credit = recording.get("artist-credit") + group = best_release_group(recording.get("releases"), prefer) or {} + length = recording.get("length") + duration = ( + length / 1000 + if isinstance(length, (int, float)) and not isinstance(length, bool) and length > 0 + else fallback_duration + ) + secondary = group.get("secondary-types") + return clean_identity( + { + "source": source, + "score": score, + "recording_mbid": recording.get("id"), + "title": recording.get("title"), + "artist": credited_artist(credit), + "artist_mbids": credited_artist_ids(credit), + "album": group.get("title"), + "release_group_mbid": group.get("id"), + "release_group_type": group.get("primary-type"), + "secondary_types": secondary if isinstance(secondary, list) else [], + "year": release_group_year(recording.get("releases"), group), + "duration": duration, + } + ) + + +# ── the questions ── + + +def lookup_recording( + mbid: str, *, cancelled: Callable[[], bool] = lambda: False +) -> dict[str, Any] | None: + """A recording by MBID, with its artist credit and its releases' release + groups, from the cache when it is there. Raises when MusicBrainz cannot + be reached.""" + if not MBID_RE.match(mbid or ""): + return None + data = _cached( + "recording", + mbid, + f"recording/{mbid}", + {"inc": "artist-credits+releases+release-groups"}, + cancelled=cancelled, + ) + return data if isinstance(data, dict) else None + + +def artist_wikidata_id(mbid: str, *, cancelled: Callable[[], bool] = lambda: False) -> str | None: + """The Wikidata item a MusicBrainz artist links to, or None. Raises when + MusicBrainz cannot be reached.""" + if not MBID_RE.match(mbid or ""): + return None + data = _cached("artist", mbid, f"artist/{mbid}", {"inc": "url-rels"}, cancelled=cancelled) + relations = data.get("relations") if isinstance(data, dict) else None + for relation in relations if isinstance(relations, list) else []: + if not isinstance(relation, dict) or relation.get("type") != "wikidata": + continue + url = relation.get("url") + resource = url.get("resource") if isinstance(url, dict) else None + match = _WIKIDATA_URL_RE.match(resource or "") if isinstance(resource, str) else None + if match: + return match.group(1) + return None + + +# Works asked about for one recording's other titles, at most: a medley +# performs several. +_WORKS_MAX = 2 + + +def work_titles(recording_mbid: str, *, cancelled: Callable[[], bool] = lambda: False) -> list[str]: + """The titles the song a recording performs goes by: each work's title + and its aliases (a Korean song's English title, a Japanese one's + romanisation), from the cache when they are there. Raises when + MusicBrainz cannot be reached.""" + if not MBID_RE.match(recording_mbid or "") or cancelled(): + return [] + data = _cached( + "recording-works", + recording_mbid, + f"recording/{recording_mbid}", + {"inc": "work-rels"}, + cancelled=cancelled, + ) + relations = data.get("relations") if isinstance(data, dict) else None + titles: list[str] = [] + works = 0 + for relation in relations if isinstance(relations, list) else []: + work = relation.get("work") if isinstance(relation, dict) else None + if not isinstance(work, dict) or relation.get("type") != "performance": + continue + if isinstance(work.get("title"), str): + titles.append(work["title"]) + work_id = work.get("id") + if not isinstance(work_id, str) or not MBID_RE.match(work_id) or cancelled(): + continue + works += 1 + found = _cached("work", work_id, f"work/{work_id}", {"inc": "aliases"}, cancelled=cancelled) + aliases = found.get("aliases") if isinstance(found, dict) else None + for alias in aliases if isinstance(aliases, list) else []: + if isinstance(alias, dict) and isinstance(alias.get("name"), str): + titles.append(alias["name"]) + if works >= _WORKS_MAX: + break + return titles + + +def lookup_release_group( + mbid: str, *, cancelled: Callable[[], bool] = lambda: False +) -> dict[str, Any] | None: + """A release group by MBID, with its url relationships (Wikidata, IMDb), + from the cache when it is there. For the work a soundtrack is from + (work_lookup.py). Raises when MusicBrainz cannot be reached.""" + if not MBID_RE.match(mbid or ""): + return None + data = _cached( + "release-group", mbid, f"release-group/{mbid}", {"inc": "url-rels"}, cancelled=cancelled + ) + return data if isinstance(data, dict) else None + + +def _lucene_phrase(text: str) -> str: + """``text`` as one quoted Lucene phrase: quotes and backslashes escaped, + so a title cannot change the shape of the query.""" + return '"' + text.replace("\\", "\\\\").replace('"', '\\"') + '"' + + +def _title_key(title: Any) -> str: + return title_parse.name_key(_BRACKETS_RE.sub("", str(title or ""))) + + +def _title_keys(title: Any) -> set[str]: + """The keys a recording's title can match by: less its brackets, and + whole ("Palette (feat. G-DRAGON)" is Palette).""" + return {k for k in (_title_key(title), title_parse.name_key(title)) if k} + + +def _recording_names(names: list[str]) -> str: + """The recording: part of a query asking for any of ``names``, each its + own quoted phrase.""" + if len(names) == 1: + return _lucene_phrase(names[0]) + return "(" + " OR ".join(_lucene_phrase(n) for n in names) + ")" + + +def search_recording( + artist: str, + title: str, + duration: float | None, + *, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The recording ``artist`` - ``title`` is, found by search, or None. + + Kept only when it is unmistakable: MusicBrainz scores it at least + MUSICBRAINZ_SEARCH_MIN_SCORE, its credited artist and its title are the + ones asked for once case, accents and punctuation are set aside, and its + length is within IDENTIFY_DURATION_TOLERANCE_SEC of the track's. Without a + length to check, nothing is kept: a name alone can be anybody's song. + Raises when MusicBrainz cannot be reached.""" + artist, title = artist.strip(), title.strip() + if not artist or not title or not duration or duration <= 0 or cancelled(): + return None + # The length and an official release narrow it down in the query: a + # popular song has hundreds of recordings under its name, most of them + # bootleg live cuts, and the one the track is would not be among the + # first page of them otherwise. + low = max(0, round((duration - IDENTIFY_DURATION_TOLERANCE_SEC) * 1000)) + high = round((duration + IDENTIFY_DURATION_TOLERANCE_SEC) * 1000) + data = _fetch_json( + "recording", + { + "query": ( + f"recording:{_lucene_phrase(title)} AND artist:{_lucene_phrase(artist)}" + f" AND dur:[{low} TO {high}] AND status:official" + ), + "limit": _SEARCH_LIMIT, + }, + cancelled=cancelled, + ) + hits = data.get("recordings") if isinstance(data, dict) else None + want_artist = title_parse.name_key(artist) + want_title = _title_key(title) + kept: list[tuple[float, float, dict[str, Any]]] = [] + for hit in hits if isinstance(hits, list) else []: + if not isinstance(hit, dict): + continue + score = hit.get("score") + length = hit.get("length") + if not isinstance(score, (int, float)) or score < MUSICBRAINZ_SEARCH_MIN_SCORE: + continue + if not isinstance(length, (int, float)) or isinstance(length, bool) or length <= 0: + continue + off = abs(length / 1000 - duration) + if off > IDENTIFY_DURATION_TOLERANCE_SEC: + continue + credit = hit.get("artist-credit") + names = {title_parse.name_key(credited_artist(credit))} + if isinstance(credit, list) and credit and isinstance(credit[0], dict): + first = credit[0] + names.add(title_parse.name_key(first.get("name"))) + if isinstance(first.get("artist"), dict): + names.add(title_parse.name_key(first["artist"].get("name"))) + if want_artist not in names: + continue + if not want_title or want_title not in ( + _title_key(hit.get("title")), + title_parse.name_key(hit.get("title")), + ): + continue + if hit.get("video") is True: + continue + kept.append((_search_rank(hit, off, float(score)), float(score), hit)) + if not kept: + return None + _, score, hit = min(kept, key=lambda item: item[0]) + return identity_from_recording(hit, source="musicbrainz", score=score / 100) + + +_LIVE_RE = re.compile(r"\blive\b", re.IGNORECASE) +# A recording that is another take on the song, not the song, as the brackets +# in its title or its disambiguation say: an instrumental, a karaoke track, a +# remix, someone else's cover. "Lemon (Kenshi Yonezu Cover)" names the artist +# a title asks for in its release title, and so would cover its words. Never +# taken for an upload of the song, whatever its length. +_NOT_THE_SONG_RE = re.compile( + r"\b(?:inst\.?|instrumental|karaoke|off\s+vocal|backing\s+track|mr(?=\s*$)" + r"|a\s+cappella|acapella|remix|cover(?:ed)?)(?:\b|\.)" + r"|カバー|翻唱|커버|伴奏|カラオケ", + re.IGNORECASE, +) +_BRACKETED_RE = re.compile(r"[(\[]([^)\]]*)[)\]]") + + +def is_another_take(hit: dict[str, Any]) -> bool: + """Whether a recording is another take on its song (_NOT_THE_SONG_RE), + by the brackets in its title or by its disambiguation.""" + title = str(hit.get("title") or "") + texts = [*_BRACKETED_RE.findall(title), str(hit.get("disambiguation") or "")] + return any(_NOT_THE_SONG_RE.search(text.strip()) for text in texts) + + +def is_live(hit: dict[str, Any]) -> bool: + """Whether a recording is a live one: its disambiguation says so ("live, + 1986"), or every release group it came out on is a live album (the best + one is, and best_release_group ranks live albums last).""" + if _LIVE_RE.search(str(hit.get("disambiguation") or "")): + return True + return "live" in _secondary_types(best_release_group(hit.get("releases"))) + + +def _search_rank(hit: dict[str, Any], off: float, score: float) -> tuple: + """Which of several recordings of the same song a search found is the + track, best first. A popular song has dozens with the same name and + artist, all scored 100: the live cuts, the karaoke and the compilations + are recordings of their own. So a live recording last, whatever else it + has going for it ("Popular" from the cast album, not from Kristin + Chenoweth's live "Coming Home"); then the studio recording first (no + disambiguation, released on something that is not a compilation, live + album or remix), then the one released most often (the original is on + every reissue and best-of), then the nearest length, then the score.""" + releases = hit.get("releases") + group = best_release_group(releases) or {} + return ( + is_live(hit), + bool(str(hit.get("disambiguation") or "").strip()), + bool(group.get("secondary-types")) or not group, + -len(releases) if isinstance(releases, list) else 0, + off, + -score, + ) + + +# ── a track named only by its title ── + + +def _length_off(hit: dict[str, Any], duration: float) -> float | None: + """How far a hit's length is from the track's, or None when it has none.""" + length = hit.get("length") + if not isinstance(length, (int, float)) or isinstance(length, bool) or length <= 0: + return None + return abs(length / 1000 - duration) + + +def _names_a_show(hit: dict[str, Any]) -> bool: + """Whether any release a recording came out on is a soundtrack or a cast + recording: by its release group's type, or by what its title says + (title_parse.kind_of).""" + for release in hit.get("releases") or []: + group = release.get("release-group") if isinstance(release, dict) else None + if not isinstance(group, dict): + continue + if "soundtrack" in _secondary_types(group): + return True + if title_parse.kind_of(group.get("title")) or title_parse.kind_of(release.get("title")): + return True + return False + + +def _release_titles(hit: dict[str, Any]) -> list[str]: + titles = [] + for release in hit.get("releases") or []: + if not isinstance(release, dict): + continue + group = release.get("release-group") + for title in ( + release.get("title"), + group.get("title") if isinstance(group, dict) else None, + ): + if isinstance(title, str) and title not in titles: + titles.append(title) + return titles + + +def best_title_match(answer: Any, reading: TitleReading, duration: float) -> dict[str, Any] | None: + """The recording a search for ``reading`` found that the track is, as an + identity, or None when none is confidently it. + + A hit counts only when its title is the song's (by any name the title + gives it), its length is within IDENTIFY_DURATION_TOLERANCE_SEC of the + track's (the Broadway cast's 457 seconds, not the film's 587), and it is + neither a video nor another take (is_another_take); and it is kept only when its credit and its release titles + contain at least TITLE_MIN_COVERAGE of the reading's words beyond the + song (the performers the title lists, the show it names), for the best of + the artist's names. Of those: the most words covered, then not live, then + the credit naming the performers (Keala Settle, not Kesha's cover on "The + Greatest Showman: Reimagined"), then a release named for the show (the + cast album, not a compilation), then _search_rank.""" + want_titles = {k for k in map(_title_key, title_parse.song_names(reading)) if k} + want = title_parse.extra_words(reading) + artist_names = [a for a in (reading.artist, *reading.artist_alts) if a] + work_words = title_parse.words(reading.work) + hits = answer.get("recordings") if isinstance(answer, dict) else None + if not want_titles or not want or not duration or duration <= 0: + return None + kept: list[tuple[tuple, float, dict[str, Any]]] = [] + for hit in hits if isinstance(hits, list) else []: + if not isinstance(hit, dict) or hit.get("video") is True: + continue + off = _length_off(hit, duration) + if off is None or off > IDENTIFY_DURATION_TOLERANCE_SEC: + continue + if not want_titles & _title_keys(hit.get("title")) or is_another_take(hit): + continue + credit = credited_artist(hit.get("artist-credit")) + releases = _release_titles(hit) + covered = round(title_parse.coverage(reading, credit, *releases), 3) + if covered < TITLE_MIN_COVERAGE: + continue + credit_words = title_parse.words(credit) + by_credit = False + for name in artist_names: + artist_words = title_parse.words(name) + if artist_words and ( + len(artist_words & credit_words) / len(artist_words) >= TITLE_MIN_COVERAGE + ): + by_credit = True + if artist_names and not work_words and not by_credit and not _names_a_show(hit): + # The artist named only in an album's title: a tribute or a cover + # ("Alexandra recorda Amália", "Tom Jobim... Os Anos 60"), unless + # the album is a show's, whose name the title can give as the + # artist ("Defying Gravity | Wicked"). + continue + named_release = bool(work_words) and any( + work_words <= title_parse.words(title) for title in releases + ) + score = hit.get("score") + score = float(score) if isinstance(score, (int, float)) else 0.0 + rank = (-covered, is_live(hit), not by_credit, not named_release) + kept.append((rank + _search_rank(hit, off, score), covered, hit)) + if not kept: + return None + _, covered, hit = min(kept, key=lambda item: item[0]) + return identity_from_recording(hit, source="musicbrainz", score=covered, prefer=work_words) + + +def search_by_title( + reading: TitleReading, + duration: float | None, + *, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The recording one reading of a track's title is, found by search, or + None (see best_title_match). The query asks for the song by any name the + title gives it, the track's length, an official release, and any of the + reading's other words in the credit or a release title, so the + recordings that could account for them come first. Raises when + MusicBrainz cannot be reached.""" + want = sorted(title_parse.search_words(reading)) + songs = title_parse.song_names(reading) + if not songs or not want or not duration or duration <= 0 or cancelled(): + return None + low = max(0, round((duration - IDENTIFY_DURATION_TOLERANCE_SEC) * 1000)) + high = round((duration + IDENTIFY_DURATION_TOLERANCE_SEC) * 1000) + # Each word its own quoted phrase: no word can change the query's shape. + terms = " ".join(_lucene_phrase(w) for w in want[:_TITLE_TERMS]) + data = _fetch_json( + "recording", + { + "query": ( + f"recording:{_recording_names(songs)}" + f" AND dur:[{low} TO {high}] AND status:official" + f" AND (artist:({terms}) OR release:({terms}))" + ), + "limit": _SEARCH_LIMIT, + }, + cancelled=cancelled, + ) + return best_title_match(data, reading, duration) + + +# ── the artist by name, then the song among theirs ── + +# Names asked for in one artist search, at most. +_ARTIST_NAMES = 10 +# Artists asked for in one recording search, at most: several can share a +# name ("Perfect"), and the song and its length tell them apart. +_ARTIST_IDS = 5 +# Artists found by name, kept for the life of the process by the names asked +# for: an import asks for the same artist twice (for the recording, then for +# the band), and a re-split once more. +_ARTISTS_FOUND: dict[tuple[str, ...], list[dict[str, Any]]] = {} +_ARTISTS_FOUND_MAX = 256 + + +def _artist_keys(artist: dict[str, Any]) -> set[str]: + """Every name an artist in a search answer goes by, as keys: its name, + its sort name both ways round ("Chou, Jay" and "Jay Chou"), and every + alias and alias sort name.""" + names: list[Any] = [artist.get("name"), artist.get("sort-name")] + sort_name = artist.get("sort-name") + if isinstance(sort_name, str) and sort_name.count(",") == 1: + last, first = sort_name.split(",") + names.append(f"{first} {last}") + for alias in artist.get("aliases") or []: + if isinstance(alias, dict): + names += [alias.get("name"), alias.get("sort-name")] + return {k for k in (title_parse.name_key(n) for n in names if isinstance(n, str)) if k} + + +def search_artists( + names: list[str], *, cancelled: Callable[[], bool] = lambda: False +) -> list[dict[str, Any]]: + """The MusicBrainz artists called one of ``names``, by name, sort name or + alias, whatever the script: "Jay Chou" and "周杰倫" both find 周杰倫, whose + aliases hold the other. One search for all of them. + + As [{"id", "name", "score", "matched", "matches"}], "matched" the first + name asked for that it goes by and "matches" every one, in the order of + ``names``, then by MusicBrainz's score. An + artist none of whose names is exactly one asked for (case, accents and + punctuation aside) is not kept: the search itself forgives spelling. + Raises when MusicBrainz cannot be reached.""" + asked: list[tuple[str, str]] = [] + for name in names: + name = str(name or "").strip() + key = title_parse.name_key(name) + if key and key not in {k for _, k in asked}: + asked.append((name, key)) + asked = asked[:_ARTIST_NAMES] + if not asked or cancelled(): + return [] + cache_key = tuple(k for _, k in asked) + if cache_key in _ARTISTS_FOUND: + return _ARTISTS_FOUND[cache_key] + query = " OR ".join( + f"artist:{_lucene_phrase(name)} OR alias:{_lucene_phrase(name)}" for name, _ in asked + ) + data = _fetch_json("artist", {"query": query, "limit": _SEARCH_LIMIT}, cancelled=cancelled) + artists = data.get("artists") if isinstance(data, dict) else None + found: list[tuple[int, float, dict[str, Any]]] = [] + for artist in artists if isinstance(artists, list) else []: + if not isinstance(artist, dict): + continue + mbid, score = artist.get("id"), artist.get("score") + if not isinstance(mbid, str) or not MBID_RE.match(mbid): + continue + score = float(score) if isinstance(score, (int, float)) else 0.0 + keys = _artist_keys(artist) + matches = [name for name, key in asked if key in keys] + if matches: + index = next(i for i, (name, _) in enumerate(asked) if name == matches[0]) + label = artist.get("name") if isinstance(artist.get("name"), str) else matches[0] + entry = {"id": mbid, "name": label, "score": score, "matched": matches[0]} + found.append((index, -score, {**entry, "matches": matches})) + answer = [entry for _, _, entry in sorted(found, key=lambda item: item[:2])] + if len(_ARTISTS_FOUND) >= _ARTISTS_FOUND_MAX: + _ARTISTS_FOUND.clear() + _ARTISTS_FOUND[cache_key] = answer + return answer + + +def search_by_artist( + songs: list[str], + artist_ids: list[str], + duration: float | None, + *, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, Any] | None: + """The recording of one of ``songs`` (names of one song) by one of + ``artist_ids``, found by search, or None (see best_artist_match). Raises + when MusicBrainz cannot be reached.""" + songs = [s for s in songs if _title_key(s)] + ids = [i for i in artist_ids if isinstance(i, str) and MBID_RE.match(i)][:_ARTIST_IDS] + if not songs or not ids or not duration or duration <= 0 or cancelled(): + return None + low = max(0, round((duration - IDENTIFY_VIDEO_EXTRA_SEC) * 1000)) + high = round((duration + IDENTIFY_DURATION_TOLERANCE_SEC) * 1000) + data = _fetch_json( + "recording", + { + "query": ( + f"recording:{_recording_names(songs)} AND arid:({' OR '.join(ids)})" + f" AND dur:[{low} TO {high}] AND status:official" + ), + "limit": _SEARCH_LIMIT, + }, + cancelled=cancelled, + ) + return best_artist_match(data, songs, ids, duration) + + +def best_artist_match( + answer: Any, songs: list[str], artist_ids: list[str], duration: float +) -> dict[str, Any] | None: + """The recording of one of ``songs`` by one of ``artist_ids`` in a search + answer that the upload is, as an identity, or None. + + The artist is known for certain here, so the length is looked at the way + a music video needs: a recording up to IDENTIFY_VIDEO_EXTRA_SEC shorter + than the upload counts (the intro, the story), never one longer. A hit + counts only when its title is one of ``songs`` once brackets are set + aside, it credits one of the artists, it is not a video, and it is not an + instrumental, karaoke or remix take. Of those: not live, then one within + IDENTIFY_DURATION_TOLERANCE_SEC of the upload's length (an audio upload + is the recording itself), then the title exactly as asked, then + _search_rank.""" + want_titles = {k for k in map(_title_key, songs) if k} + exact = {title_parse.name_key(s) for s in songs} + hits = answer.get("recordings") if isinstance(answer, dict) else None + if not want_titles or not duration or duration <= 0: + return None + kept: list[tuple[tuple, float, dict[str, Any]]] = [] + for hit in hits if isinstance(hits, list) else []: + if not isinstance(hit, dict) or hit.get("video") is True: + continue + length = hit.get("length") + if not isinstance(length, (int, float)) or isinstance(length, bool) or length <= 0: + continue + seconds = length / 1000 + if ( + not duration - IDENTIFY_VIDEO_EXTRA_SEC + <= seconds + <= duration + (IDENTIFY_DURATION_TOLERANCE_SEC) + ): + continue + title = hit.get("title") + if not want_titles & _title_keys(title) or is_another_take(hit): + continue + if not set(credited_artist_ids(hit.get("artist-credit"))) & set(artist_ids): + continue + off = abs(seconds - duration) + score = hit.get("score") + score = float(score) if isinstance(score, (int, float)) else 0.0 + rank = ( + is_live(hit), + off > IDENTIFY_DURATION_TOLERANCE_SEC, + title_parse.name_key(title) not in exact, + ) + kept.append((rank + _search_rank(hit, off, score), score, hit)) + if not kept: + return None + _, score, hit = min(kept, key=lambda item: item[0]) + return identity_from_recording(hit, source="musicbrainz", score=score / 100) diff --git a/app/pipeline/name_aliases.py b/app/pipeline/name_aliases.py new file mode 100644 index 00000000..fa62139f --- /dev/null +++ b/app/pipeline/name_aliases.py @@ -0,0 +1,264 @@ +"""The other names an artist goes by, and names reduced for comparing. + +LRCLIB files a song under whatever name its uploader used: 周杰倫, 周杰伦 or +Jay Chou; 아이유 or IU; 宇多田ヒカル or Hikaru Utada. A track known by one of +them has its lyrics under another just as often, and the lyrics lookup keeps +a version only when its artist is one the track is known by +(lyrics_lookup.belongs_to). So the names are taken from where they are +recorded as one artist's: the MusicBrainz artist the track was identified as +(its aliases) and the Wikidata item of its band (its labels and aliases). + +Never guessed from the names themselves: no transliteration, no romanisation. +Two names count as one artist's only when one of those sources says so. What +each source says is taken with care, since both also list what is only +loosely a name: + +* MusicBrainz: the artist's name, its primary alias in each language, and + its other aliases that are distinctive (see ``distinctive``). Never a + search hint (misspellings, mojibake) nor a legal name that is not primary + (the name on a passport, which other people share). +* Wikidata: every label in the languages StemDeck speaks (and the Chinese + variants), and the aliases that are distinctive. + +MusicBrainz answers are kept in its cache (musicbrainz.py) with the same +MBID, one request for the artist and its links both, and take their turn from +the shared rate limiter. Wikidata answers are kept in memory for the process. +What is sent: an MBID, or a Wikidata item id. Nothing here raises: a source +that cannot be reached gives no names. +""" + +from __future__ import annotations + +import logging +import threading +import time +import unicodedata +from collections.abc import Callable, Iterable +from typing import Any + +from app.core.models import MBID_RE +from app.pipeline import artist_lookup, musicbrainz +from app.pipeline.artist_lookup import _QID_RE, artist_name_key +from app.pipeline.zh_variants import to_simplified + +logger = logging.getLogger("stemdeck.lyrics") + +# How many of a recording's credited artists are asked about: a duet's two. +_MAX_ARTISTS = 2 +# A name that is not the artist's own or primary counts only when it weighs +# at least this much (name_weight) and, written only in Latin or another +# script with spaces, is at least two words: "Jay", "Hikki" and "VI" name too +# many people, "Jay Chou" and 周杰伦 do not. +_MIN_WEIGHT = 4 +_MIN_WORDS = 2 +# Wikidata labels and aliases are read in these languages: the ten StemDeck +# speaks, and the Chinese variants and Cantonese, whose names differ. +_WIKIDATA_LANGUAGES = "en|pl|ja|ko|zh|zh-hans|zh-hant|zh-cn|zh-tw|zh-hk|yue|de|fr|es|pt|pt-br|id" +_WIKIDATA_KEEP_SEC = 24 * 3600 +_WIKIDATA_KEEP_MAX = 256 +_NAME_MAX_CHARS = 300 + + +# ── names reduced for comparing ── + + +# Latin letters Unicode does not build from a base letter and an accent, as a +# name typed without them has them: "Podsiadlo" for Podsiadło, "Grossstadt" +# for Großstadt. The accents that are marks go in artist_name_key. +_LATIN_LETTERS = str.maketrans( + { + "ł": "l", + "Ł": "L", + "ø": "o", + "Ø": "O", + "đ": "d", + "Đ": "D", + "ð": "d", + "Ð": "D", + "ß": "ss", + "æ": "ae", + "Æ": "AE", + "œ": "oe", + "Œ": "OE", + "ı": "i", + "þ": "th", + "Þ": "TH", + } +) + + +def fold(text: Any) -> str: + """``text`` as it is compared: compatibility forms unified (full-width + Latin, half-width kana, the ideographs Unicode keeps twice), traditional + Chinese folded to simplified (zh_variants.py), and the Latin letters + above written plain. fold() in static/js/lyricsLookup.js is its twin.""" + text = unicodedata.normalize("NFKC", str(text or "")).translate(_LATIN_LETTERS) + return to_simplified(text) + + +def name_key(name: Any) -> str: + """artist_name_key of the folded name: 周杰倫 and 周杰伦 are one key.""" + return artist_name_key(fold(name)) + + +def _han(ch: str) -> bool: + cp = ord(ch) + return ( + 0x3400 <= cp <= 0x4DBF + or 0x4E00 <= cp <= 0x9FFF + or 0xF900 <= cp <= 0xFAFF + or (0x20000 <= cp <= 0x3FFFF) + ) + + +def _cjk(ch: str) -> bool: + """A Chinese, Japanese or Korean letter: an ideograph, kana or Hangul.""" + cp = ord(ch) + return ( + _han(ch) + or 0x3040 <= cp <= 0x30FF + or 0x31F0 <= cp <= 0x31FF + or 0xFF66 <= cp <= 0xFF9F + or 0x1100 <= cp <= 0x11FF + or 0x3130 <= cp <= 0x318F + or 0xAC00 <= cp <= 0xD7AF + ) + + +def name_weight(key: str) -> int: + """How much of a name ``key`` (name_key) is, in Latin letters: an + ideograph counts two, since a Chinese or Japanese name of two or three of + them is a whole name where two or three Latin letters are an initial or a + stray "DJ". Hangul counts by its letters, which a key has already split + each syllable into; marks count nothing.""" + return sum(2 if _han(ch) else 0 if unicodedata.category(ch)[0] == "M" else 1 for ch in key) + + +def has_cjk(text: str) -> bool: + return any(_cjk(ch) for ch in text) + + +def distinctive(name: Any) -> bool: + """Whether a name says who it is without its source's word for it: long + enough (name_weight), and more than one word when it is written in a + script with spaces.""" + text = str(name or "").strip() + if name_weight(name_key(text)) < _MIN_WEIGHT: + return False + return has_cjk(text) or len(text.split()) >= _MIN_WORDS + + +# ── what each source says ── + + +def _add(names: list[str], name: Any) -> None: + if isinstance(name, str) and name.strip() and name.strip() not in names: + names.append(name.strip()[:_NAME_MAX_CHARS]) + + +def musicbrainz_names(artist: Any) -> list[str]: + """The names a MusicBrainz artist (with ``inc=aliases``) goes by, best + first: its own, its primary aliases, then the other distinctive ones.""" + if not isinstance(artist, dict): + return [] + names: list[str] = [] + _add(names, artist.get("name")) + aliases = [a for a in artist.get("aliases") or [] if isinstance(a, dict)] + usable = [a for a in aliases if a.get("type") != "Search hint"] + for alias in usable: + if alias.get("primary") is True: + _add(names, alias.get("name")) + for alias in usable: + if alias.get("type") != "Legal name" and distinctive(alias.get("name")): + _add(names, alias.get("name")) + return names + + +def wikidata_names(entity: Any) -> list[str]: + """The names a Wikidata item goes by: its labels, then its distinctive + aliases.""" + if not isinstance(entity, dict): + return [] + names: list[str] = [] + labels = entity.get("labels") + for label in labels.values() if isinstance(labels, dict) else []: + if isinstance(label, dict): + _add(names, label.get("value")) + aliases = entity.get("aliases") + for group in aliases.values() if isinstance(aliases, dict) else []: + for alias in group if isinstance(group, list) else []: + if isinstance(alias, dict) and distinctive(alias.get("value")): + _add(names, alias.get("value")) + return names + + +def _musicbrainz_artist(mbid: str, *, cancelled: Callable[[], bool] = lambda: False) -> Any: + """The artist with its aliases and url relationships, from the cache when + it holds them. The answer replaces a cached one without aliases, which + artist_wikidata_id reads the same way. Raises when MusicBrainz cannot be + reached.""" + kept = musicbrainz.cache_get("artist", mbid) + if isinstance(kept, dict) and "aliases" in kept: + return kept + # Looked up on the module, so the tests' stand-in for the network holds. + data = musicbrainz._fetch_json( + f"artist/{mbid}", {"inc": "aliases+url-rels"}, cancelled=cancelled + ) + if isinstance(data, dict): + musicbrainz.cache_put("artist", mbid, data) + return data + + +_wikidata_kept: dict[str, tuple[float, list[str]]] = {} +_wikidata_lock = threading.Lock() + + +def _wikidata_item_names(qid: str) -> list[str]: + """wikidata_names for item ``qid``, kept for a day. Raises when Wikidata + cannot be reached.""" + with _wikidata_lock: + kept = _wikidata_kept.get(qid) + if kept and time.time() - kept[0] < _WIKIDATA_KEEP_SEC: + return kept[1] + answer = artist_lookup._fetch_json( + { + "action": "wbgetentities", + "ids": qid, + "props": "labels|aliases", + "languages": _WIKIDATA_LANGUAGES, + } + ) + entities = answer.get("entities") if isinstance(answer, dict) else None + names = wikidata_names(entities.get(qid) if isinstance(entities, dict) else None) + with _wikidata_lock: + if len(_wikidata_kept) >= _WIKIDATA_KEEP_MAX: + _wikidata_kept.clear() + _wikidata_kept[qid] = (time.time(), names) + return names + + +def artist_aliases( + mbids: Iterable[str], + qid: str = "", + *, + cancelled: Callable[[], bool] = lambda: False, +) -> list[str]: + """Every name the artist is known by: the MusicBrainz artists ``mbids`` + (a recording's credit) and the Wikidata item ``qid`` (its band), best + first, each once. [] when neither is known or reachable. Never raises.""" + names: list[str] = [] + for mbid in [m for m in mbids if isinstance(m, str) and MBID_RE.match(m)][:_MAX_ARTISTS]: + if cancelled(): + return names + try: + for name in musicbrainz_names(_musicbrainz_artist(mbid, cancelled=cancelled)): + _add(names, name) + except Exception: + logger.info("artist aliases from MusicBrainz failed", exc_info=True) + if isinstance(qid, str) and _QID_RE.match(qid) and not cancelled(): + try: + for name in _wikidata_item_names(qid): + _add(names, name) + except Exception: + logger.info("artist names from Wikidata failed", exc_info=True) + return names diff --git a/app/pipeline/ratelimit.py b/app/pipeline/ratelimit.py new file mode 100644 index 00000000..1190f6ad --- /dev/null +++ b/app/pipeline/ratelimit.py @@ -0,0 +1,79 @@ +"""The request rate every web service StemDeck asks is owed, per process. + +MusicBrainz allows one request a second per client and blocks a client that +goes faster; AcoustID allows three. Identification, the band lookup and the +tag backfill can all be asking at the same moment from different threads, so +the limit lives here, once per service, and every caller takes its turn from +the same instance. Anything that calls MusicBrainz goes through MUSICBRAINZ. + +A turn is reserved under the lock and slept for outside it, so callers are +served in the order they asked and never sleep while holding anything. +""" + +from __future__ import annotations + +import threading +import time +from collections.abc import Callable + +from app.core.config import ( + ACOUSTID_MIN_INTERVAL_SEC, + DISCOGS_MIN_INTERVAL_SEC, + MUSICBRAINZ_MIN_INTERVAL_SEC, + RATE_LIMIT_MAX_WAIT_SEC, +) + + +class RateLimited(Exception): + """The next turn is further off than the caller was willing to wait.""" + + +class RateLimiter: + """At most one request per ``interval`` seconds, across every thread.""" + + def __init__( + self, + interval: float, + *, + clock: Callable[[], float] = time.monotonic, + sleep: Callable[[float], None] = time.sleep, + ) -> None: + self.interval = interval + self._clock = clock + self._sleep = sleep + self._lock = threading.Lock() + self._next = 0.0 + + def wait( + self, + max_wait: float = RATE_LIMIT_MAX_WAIT_SEC, + cancelled: Callable[[], bool] = lambda: False, + ) -> None: + """Block until this caller's turn. Raises RateLimited, reserving + nothing, when that turn is more than ``max_wait`` seconds away, and + when ``cancelled`` says so while waiting (the turn is then spent).""" + with self._lock: + now = self._clock() + slot = max(now, self._next) + if slot - now > max_wait: + raise RateLimited(f"next turn in {slot - now:.1f}s") + self._next = slot + self.interval + # Slept in short steps so a cancelled job stops waiting promptly. + while True: + remaining = slot - self._clock() + if remaining <= 0: + return + if cancelled(): + raise RateLimited("cancelled while waiting") + self._sleep(min(remaining, 0.25)) + + def reset(self) -> None: + """Forget every reserved turn. For tests.""" + with self._lock: + self._next = 0.0 + + +MUSICBRAINZ = RateLimiter(MUSICBRAINZ_MIN_INTERVAL_SEC) +ACOUSTID = RateLimiter(ACOUSTID_MIN_INTERVAL_SEC) +# Discogs allows 60 requests a minute with a token (app/pipeline/discogs.py). +DISCOGS = RateLimiter(DISCOGS_MIN_INTERVAL_SEC) diff --git a/app/pipeline/runner.py b/app/pipeline/runner.py index d9832d3d..83a70b0f 100644 --- a/app/pipeline/runner.py +++ b/app/pipeline/runner.py @@ -10,7 +10,13 @@ from datetime import datetime, timezone from pathlib import Path -from app.core.config import DEMUCS_MODEL, TIMEOUT_FFMPEG +from app.core.config import ( + DEMUCS_MODEL, + IDENTIFY_GRACE_SEC, + LYRICS_IDENTITY_WAIT_SEC, + LYRICS_LOOKUP_GRACE_SEC, + TIMEOUT_FFMPEG, +) from app.core.models import Job, JobCancelled, _set from app.core.redact import redact from app.core.registry import is_upload, set_proc @@ -26,8 +32,12 @@ ) from app.pipeline.download import download from app.pipeline.errors import classify_failure +from app.pipeline.identify import IdentifyLookup, release_source +from app.pipeline.lyrics_lookup import LyricsLookup +from app.pipeline.lyrics_retime import retime_after_separation from app.pipeline.sections import detect_sections from app.pipeline.separate import separate +from app.pipeline.transcribe import transcribe_lyrics logger = logging.getLogger("stemdeck.pipeline") @@ -206,6 +216,9 @@ def _run_common(job: Job, source: Path, job_dir: Path) -> None: # at all -- and the person who imported it may no longer have the file # either. A link costs a re-download; an upload costs the recording. if not is_upload(job): + # A fingerprint still reading the source would make the delete fail + # on Windows. It finished long ago, so this is nearly always a no-op. + release_source(job.id) cleanup_source(job_dir) job.stems = [{"name": name, "url": f"/api/jobs/{job.id}/stems/{name}.wav"} for name in found] _check_cancel(job) @@ -270,12 +283,63 @@ def _run_common(job: Job, source: Path, job_dir: Path) -> None: _lap(job, "sections", mark) +def _run_with_band_lookup(job: Job, source: Path, job_dir: Path) -> None: + """_run_common, with the band the artist tag names looked up beside it (#699). + + Started as soon as the tags are known, which is now for both pipelines: + a link's come with its download, an upload's were read when it arrived. + The lookup is two small web requests and separation takes minutes, so by + the time the job is done the answer has long been waiting, and the wait + below is for nothing. When it has not arrived, the job waits at most + IDENTIFY_GRACE_SEC and finishes without it; the page then looks for + the band itself when the track is opened. + + The band is found through the recording the job is identified as + (identify.py: an AcoustID fingerprint of the source, or a MusicBrainz + search by the tags), which runs in the same thread first. + + Only a pipeline that got to the end keeps the answer: a cancel or a + failure raises out of _run_common, past finish(), and nothing is written. + """ + lookup = IdentifyLookup.start(job, [source]) + # The lyrics, on the same terms: LRCLIB beside separation, lyrics.json + # written only by a pipeline that got to the end. Looked up by the + # identity once the identification has it, else by the tags. + lyrics = LyricsLookup.start( + job, + job_dir, + identity=(lambda: lookup.wait_identity(LYRICS_IDENTITY_WAIT_SEC)) if lookup else None, + ) + _run_common(job, source, job_dir) + if lookup is not None: + mark = time.monotonic() + lookup.finish(job, IDENTIFY_GRACE_SEC) + # What the lookup cost the import, which should be nothing (#293). + _lap(job, "identify_wait", mark) + if lyrics is not None: + mark = time.monotonic() + lyrics.finish(job, job_dir, LYRICS_LOOKUP_GRACE_SEC) + _lap(job, "lyrics_wait", mark) + # Lyrics from the vocals stem, only when the lookup found none: last, + # because it needs both the stem and the lookup's answer. Never raises + # but for a cancel. + mark = time.monotonic() + transcribe_lyrics(job, job_dir) + _lap(job, "transcribe", mark) + # LRCLIB lyrics whose timing nothing confirmed, timed line by line to the + # vocals (lyrics_retime.py): after the above, whose transcript it reuses. + # Never raises but for a cancel. + mark = time.monotonic() + retime_after_separation(job, job_dir) + _lap(job, "retime", mark) + + def _run_blocking(job: Job, url: str, job_dir: Path) -> None: _check_cancel(job) mark = time.monotonic() source = download(job, url, job_dir) _lap(job, "download", mark) - _run_common(job, source, job_dir) + _run_with_band_lookup(job, source, job_dir) def _run_local_blocking(job: Job, source_path: Path, job_dir: Path) -> None: @@ -283,13 +347,17 @@ def _run_local_blocking(job: Job, source_path: Path, job_dir: Path) -> None: mark = time.monotonic() source = _prepare_local_source(job, source_path, job_dir) _lap(job, "prepare", mark) - _run_common(job, source, job_dir) + _run_with_band_lookup(job, source, job_dir) def _write_metadata(job: Job, job_dir: Path) -> None: meta = { "title": job.title, "thumbnail": job.thumbnail, + "audio_tags": job.audio_tags, + "identity": job.identity, + "artist": job.artist, + "work": job.work, "duration_sec": job.duration_sec, "bpm": job.bpm, "key": job.key, @@ -411,6 +479,8 @@ async def _run_async( if not isinstance(e, JobCancelled) and not job.cancel_requested: logger.exception("pipeline failed for job %s: %s", job.id, e) _set(job, status="error", stage="Error: Processing failed", error=error_msg) + # A fingerprint may still have the source open (Windows). + release_source(job.id) _quarantine_failed_job(job, job_dir, jobs_dir, e) persist_registry(jobs_dir) return @@ -421,6 +491,7 @@ async def _run_async( ) _set(job, status="cancelled", stage="Cancelled") persist_registry(jobs_dir) + release_source(job.id) _rmtree(job_dir) return _set(job, status="done", progress=1.0, stage="Done") diff --git a/app/pipeline/title_parse.py b/app/pipeline/title_parse.py new file mode 100644 index 00000000..a9dfbbb4 --- /dev/null +++ b/app/pipeline/title_parse.py @@ -0,0 +1,997 @@ +"""What a track's title says it is, when nothing else does. + +A YouTube upload of a cast recording or a film song usually carries no music +metadata at all: yt-dlp's artist, track and album are empty, and the channel +("WickedVEVO", "Atlantic Records", a fan's) names no artist. The title is all +there is, and titles follow a handful of patterns: + +* "Artist - Song", and as often "Song - Artist"; +* 'Song (From "Work" Original Broadway Cast Recording/2003)', "Song (from Work)"; +* "Song - Work The Musical", "Song - Work Soundtrack", "Song | Work", "Work: Song"; +* "Song Name, Name": a performer list after a run of spaces; +* "Cast of Work", "Work Cast", "Work Original Broadway Cast" as the artist; + +wrapped in video noise: "(Official Audio)", "[Lyric Video]", "HD", "4K", +"Remastered", a year after a slash. + +Titles from East Asia follow patterns of their own, and are read as carefully: + +* the song in marks rather than after a dash: "YOASOBI「夜に駆ける」", + "周杰倫 Jay Chou【晴天 Sunny Day】", "米津玄師 MV「Lemon」", "BTS 'Spring Day'"; +* one name in two scripts: "IU(아이유)", "周杰倫 Jay Chou", "晴天 Sunny Day", + each kept as an alternative, since MusicBrainz may know either; +* "_", "/", "~" and full-width punctuation ("／", "｜", "（）") as separators; +* noise in any language and bracket: "【MV】", "[MV]", "官方完整版", "高音質", + "歌詞", "뮤직비디오", "公式". Never inside a name: "Official髭男dism" is a band. + +parse_title reads a title every way it can be read, most likely first, as +TitleReading: the song, who performs it, and the work (musical, film, series) +it is from, when the title names one. Which reading is right is decided +elsewhere (musicbrainz.search_by_title, lyrics_lookup.identify_on_lrclib), by +whether a recording of that song, that length, credits those names. This +module is pure: no requests, no state. +""" + +from __future__ import annotations + +import re +import unicodedata +from dataclasses import dataclass +from typing import Any + +from app.pipeline.zh_variants import to_simplified + +# At most this many readings of one title are offered: each one costs a +# MusicBrainz search at one request a second. +MAX_READINGS = 3 +# A title longer than this is not one. +_TITLE_MAX_CHARS = 300 + +# Hyphen, figure dash, en dash, em dash, horizontal bar. Built from code +# points so no dash character has to be typed here. +_DASHES = "-" + "".join(chr(c) for c in (0x2012, 0x2013, 0x2014, 0x2015)) +# Double quotes of every kind. Single quotes are left alone: they are +# apostrophes as often as not ("Don't Stop Me Now"). +_QUOTES = '"' + "".join(chr(c) for c in (0x201C, 0x201D, 0x201E, 0x201F, 0x00AB, 0x00BB)) +# Stands for a run of two or more spaces, which some uploads put between the +# song and a performer list, so it survives the bracket removal that leaves +# spaces of its own. A control character: never in a real title. +_GAP = chr(0x1F) +# Private-use characters standing for structure found in a title: between two +# names of one thing ("IU(아이유)"), and either side of a song in marks +# ("「Lemon」"). Not whitespace, which the control characters are to Python, +# and removed from the title first, so never confused with a real character. +_ALT = chr(0xE000) +_MARK_OPEN = chr(0xE001) +_MARK_CLOSE = chr(0xE002) +_PRIVATE_USE_RE = re.compile("[" + chr(0xE000) + "-" + chr(0xF8FF) + "]") +# The marks East Asian titles put around a song's name, opening to closing: +# corner brackets, white corner brackets, double angle brackets, angle +# brackets, black lenticular brackets, white lenticular brackets. +_SONG_MARKS = {chr(o): chr(o + 1) for o in (0x300C, 0x300E, 0x300A, 0x3008, 0x3010, 0x3016)} +# The lenticular ones hold video noise as often as a song ("【MV】", "【公式】"). +_LENTICULAR = (chr(0x3010), chr(0x3016)) +# Tortoise-shell and white square brackets, read as round ones. +_ROUND_ALIASES = {chr(0x3014): "(", chr(0x3015): ")", chr(0x3018): "(", chr(0x3019): ")"} +# Where a song mark was, once found: its name between the two. +_MARKED_RE = re.compile(f"{_MARK_OPEN}([^{_MARK_OPEN}{_MARK_CLOSE}]*){_MARK_CLOSE}") +# 'Song' in single quotes, K-pop's way ("NewJeans 'Ditto'"): only a pair +# opening after a space and closing before one, at least two characters +# apart, so "Don't", "Guns N' Roses" and "Rock 'n' Roll" stay as they are. +# Straight or curly. +_SQ_OPEN = "'" + chr(0x2018) +_SQ_CLOSE = "'" + chr(0x2019) +_SQ_ANY = _SQ_OPEN + chr(0x2019) +_SINGLE_QUOTED_RE = re.compile( + rf"(?:^|(?<=[\s({_MARK_OPEN}]))[{_SQ_OPEN}]([^{_SQ_ANY}\s][^{_SQ_ANY}]*[^{_SQ_ANY}\s])" + rf"[{_SQ_CLOSE}](?=$|[\s){_MARK_CLOSE},.!?])" +) +# A Japanese subtitle between wave dashes: "夜に駆ける ~Racing Into The Night~". +_WAVE_RE = re.compile(r"~(\S[^~]*\S|\S)~") +# A hashtag ("#music", "#shorts") says nothing about the song. +_HASHTAG_RE = re.compile(r"(?:^|(?<=\s))#\S+") + +_BRACKET_RE = re.compile(r"[(\[{]([^()\[\]{}]*)[)\]}]") +_ROUND_RE = re.compile(r"[(\[]([^()\[\]{}]*)[)\]]") +_GAP_RE = re.compile("[ \t" + chr(0xA0) + "]{2,}") +_SPACES_RE = re.compile(r"\s+") +# Between artist and song: a dash with space on at least one side +# ("鄧麗君 -月亮代表我的心"), a lone "_", "/", "//" or "~" with space either +# side ("IU _ Palette"), or a pipe. +_DASH_SPLIT_RE = re.compile( + rf"\s+[{re.escape(_DASHES)}]+\s*|\s*[{re.escape(_DASHES)}]+\s+|\s+(?:_+|/{{1,2}}|~)\s+" +) +_PIPE_SPLIT_RE = re.compile(r"\s*\|\s*") +_COLON_SPLIT_RE = re.compile(r"\s*:\s+") +_GAP_SPLIT_RE = re.compile(rf"\s*{_GAP}\s*") + +_FROM_RE = re.compile(r"^\s*from\s+(.+)$", re.IGNORECASE) +_QUOTED_RE = re.compile(rf"[{_QUOTES}]([^{_QUOTES}]+)[{_QUOTES}]") +_FEAT_RE = re.compile(r"^\s*(?:ft\.?|feat\.?|featuring|with)\s", re.IGNORECASE) +_YEAR_RE = re.compile(r"(?.+)$", + re.IGNORECASE, +) +# "The Greatest Showman Cast", "Wicked Original Broadway Cast (Recording)". +_WORK_CAST_RE = re.compile( + rf"^(?P.+?)\s+{_CAST_WORDS}cast(?:\s+recording)?$", + re.IGNORECASE, +) +# "Wicked The Musical", "Wicked: The Musical", "Wicked Musical". +_WORK_MUSICAL_RE = re.compile( + rf"^(?P.+?)\s*[:{re.escape(_DASHES)}]?\s+(?:the\s+)?musical$", re.IGNORECASE +) +# "Frozen Soundtrack", "Frozen OST", "Frozen Original Motion Picture Soundtrack". +_WORK_SOUNDTRACK_RE = re.compile( + r"^(?P.+?)\s*:?\s+(?:the\s+)?(?:original\s+)?(?:motion\s+picture\s+)?" + r"(?:soundtrack|o\.?s\.?t\.?)$", + re.IGNORECASE, +) +# The last name in a list: "Keala Settle & The Greatest Showman" is the show. +_LIST_SPLIT_RE = re.compile(r"\s*(?:&|,|\+|\band\b)\s*", re.IGNORECASE) + +# Words that say nothing about which recording a title names: joins, and the +# qualifiers every cast recording and soundtrack shares. Left out of the words +# a candidate recording has to account for (extra_words). +_STOPWORDS = frozenset( + re.split( + r"\s+", + "the a an of and feat ft featuring with vs x by from original broadway london west end" + " cast recording motion picture soundtrack ost film movie musical ensemble revival" + " version edition deluxe official", + ) +) + + +@dataclass(frozen=True) +class TitleReading: + """One way to read a title: the song, who performs it, and the work it is + from. ``artist`` and ``work`` may be empty, and are then unknown.""" + + song: str + artist: str = "" + work: str = "" + # "musical", "film" or "tv": what the title says the work is. + kind: str | None = None + # The year the title gives the recording ("/2003"). + year: int | None = None + # Other names the title gives the song and the artist, in another script: + # "晴天 Sunny Day", "IU(아이유)". MusicBrainz may know either. + song_alts: tuple[str, ...] = () + artist_alts: tuple[str, ...] = () + + +# ── words ── + + +# Latin letters NFKD leaves whole, folded as the accents are, so a title +# typed without Polish or Nordic letters ("Dawid Podsiadlo") still names +# "Dawid Podsiadło": l with stroke, o with stroke, d with stroke, dotless +# i, ae, oe, thorn, eth. +_LATIN_FOLD = str.maketrans( + { + chr(0x142): "l", + chr(0xF8): "o", + chr(0x111): "d", + chr(0x131): "i", + chr(0xE6): "ae", + chr(0x153): "oe", + chr(0xFE): "th", + chr(0xF0): "d", + } +) + + +def _words(text: Any, *, simplified: bool) -> frozenset[str]: + norm = unicodedata.normalize("NFKD", str(text or "")) + # Composed again once the Latin accents are gone, so kana keep their + # voicing marks and Hangul its syllables: "が" is one letter, not two. + norm = unicodedata.normalize( + "NFC", "".join(ch for ch in norm if not 0x300 <= ord(ch) <= 0x36F) + ).casefold() + norm = norm.translate(_LATIN_FOLD).replace("'", "").replace(chr(0x2019), "") + if simplified: + norm = to_simplified(norm) + return frozenset(w for w in re.split(r"[\W_]+", norm) if w and w not in _STOPWORDS) + + +def words(text: Any) -> frozenset[str]: + """The words of ``text`` that tell one recording from another: no case, + no Latin accents, no apostrophes or punctuation, traditional Chinese as + simplified (鄧麗君 is 邓丽君), and none of _STOPWORDS.""" + return _words(text, simplified=True) + + +def extra_words(reading: TitleReading) -> frozenset[str]: + """The words beyond the song a recording must account for: the artist's + (in every name the title gives it) and the work's.""" + found = words(reading.artist) | words(reading.work) + for name in reading.artist_alts: + found |= words(name) + return found + + +def search_words(reading: TitleReading) -> frozenset[str]: + """extra_words as a search asks for them: each as written as well as + folded, since a search engine knows 周杰倫 and 周杰伦 as two words.""" + found = extra_words(reading) + for text in (reading.artist, reading.work, *reading.artist_alts): + found |= _words(text, simplified=False) + return found + + +def coverage(reading: TitleReading, *texts: Any) -> float: + """How many of the reading's extra words ``texts`` contain, 0..1, for the + best of the artist's names: "周杰倫 Jay Chou" is covered by a credit to + 周杰倫 alone. Zero when the reading has none.""" + have: set[str] = set() + for text in texts: + have |= words(text) + best = 0.0 + for artist in (reading.artist, *reading.artist_alts): + want = words(artist) | words(reading.work) + if want: + best = max(best, len(want & have) / len(want)) + return best + + +def name_key(text: Any) -> str: + """A name reduced for comparison (artist_lookup.artist_name_key), with + the Latin letters NFKD keeps whole folded too, and traditional Chinese + as simplified (zh_variants.py): 周杰倫 and 周杰伦 are one name.""" + # Imported here: artist_lookup pulls in the network layer, and this + # module stays importable without it. + from app.pipeline.artist_lookup import artist_name_key + + return to_simplified(artist_name_key(str(text or "")).translate(_LATIN_FOLD)) + + +def song_key(text: Any) -> str: + """A song's name reduced for comparison: brackets gone, then no case, + accents, punctuation or spaces (name_key).""" + return name_key(_BRACKET_RE.sub(" ", str(text or ""))) + + +def song_names(reading: TitleReading) -> list[str]: + """Every name the reading gives its song, the first one first.""" + return [n for n in (reading.song, *reading.song_alts) if n] + + +def resolvable(reading: TitleReading) -> bool: + """Whether a reading names more than a song: a song alone can be anybody's.""" + return bool(reading.song) and bool(extra_words(reading)) + + +# ── what a piece of the title says ── + + +def kind_of(text: Any) -> str | None: + """The kind of work ``text`` says a recording is from, or None.""" + text = str(text or "") + if _FILM_RE.search(text): + return "film" + if _MUSICAL_RE.search(text): + return "musical" + if _TV_RE.search(text): + return "tv" + if _SOUNDTRACK_RE.search(text): + return "film" + return None + + +def _year(text: str) -> int | None: + match = _YEAR_RE.search(text) + return int(match.group(1)) if match else None + + +def _unquote(text: str) -> str: + return text.strip().strip(_QUOTES).strip() + + +def _from_work(text: str) -> tuple[str, str | None, int | None]: + """(work, kind, year) from what follows "from" in a bracket.""" + quoted = _QUOTED_RE.search(text) + if quoted: + work = quoted.group(1).strip() + rest = text[: quoted.start()] + " " + text[quoted.end() :] + else: + work = _QUALIFIER_START_RE.split(text, maxsplit=1)[0].strip() + rest = text[len(work) :] + if not words(work): + work = "" + return work, kind_of(rest), _year(rest) + + +def cast_work(artist: str) -> str: + """The work an artist credit names as a cast ("The Greatest Showman Cast", + "Cast of Hamilton"), or "".""" + artist = artist.strip() + match = _CAST_OF_RE.match(artist) or _WORK_CAST_RE.match(artist) + if not match: + return "" + work = _LIST_SPLIT_RE.split(match.group("work"))[-1].strip() + return work if words(work) else "" + + +def _named_work(part: str) -> tuple[str, str | None]: + """(work, kind) when a whole part of the title is a work's name with its + qualifier ("Wicked The Musical", "Frozen Soundtrack"), else ("", None).""" + for pattern, kind in ((_WORK_MUSICAL_RE, "musical"), (_WORK_SOUNDTRACK_RE, "film")): + match = pattern.match(part) + if match and words(match.group("work")): + return match.group("work").strip(" :" + _DASHES), kind + return "", None + + +def _noise_words(text: str) -> list[str]: + """The words of ``text``, case-folded, once its East Asian noise is out.""" + rest = _CJK_NOISE_RE.sub(" ", text).casefold().replace("m/v", "mv") + return [w for w in _NOISE_SPLIT_RE.split(rest) if w] + + +def _is_noise(text: str) -> bool: + """Whether a bracket holding ``text`` only describes the upload: "MV", + "Official Music Video", "官方完整版", "HD", not "Sunny Day".""" + marked = bool(_CJK_NOISE_RE.search(text)) + for word in _noise_words(text): + if word in _NOISE_MARKERS: + marked = True + elif word not in _NOISE_FILLERS and not word.isdigit(): + return False + return marked + + +def _has_noise(text: str) -> bool: + """Whether ``text`` says anything about the upload at all ("Japanese + ver.", "MV Full").""" + return bool(_CJK_NOISE_RE.search(text)) or any(w in _NOISE_MARKERS for w in _noise_words(text)) + + +def _noise_token(token: str, allowed: frozenset[str]) -> bool: + """Whether one unbracketed word is noise: in ``allowed``, or made only of + _CJK_NOISE and at most one word of ``allowed`` ("官方完整版", "公式MV").""" + if token.casefold() in allowed: + return True + rest = _CJK_NOISE_RE.sub("", token) + return rest != token and (not rest or rest.casefold() in allowed) + + +def _clean_part(part: str) -> str: + part = _SPACES_RE.sub(" ", part).strip() + for _ in range(8): + before = part + part = _YEAR_SUFFIX_RE.sub("", part) + part = _TRAILING_NOISE_RE.sub("", part) + tokens = part.split(" ") + if _noise_token(tokens[-1], _TRAILING_NOISE_WORDS): + part = " ".join(tokens[:-1]) + elif len(tokens) > 1 and _noise_token(tokens[0], _LEADING_NOISE_WORDS): + part = " ".join(tokens[1:]) + part = _unquote(part).strip(" " + _DASHES + ":|_/~,;" + chr(0x3001) + chr(0x3002)) + if part == before: + break + return part + + +# ── scripts, and names in two of them ── + + +def _script(ch: str) -> str: + """Which script a character is written in, as far as telling two names + of one thing apart goes: "hangul", "hankana" (Chinese characters and + Japanese kana together, since a Japanese name mixes them), "cyrillic", + "greek", "latin", or "" for no letter at all.""" + code = ord(ch) + if 0xAC00 <= code <= 0xD7AF or 0x1100 <= code <= 0x11FF or 0x3130 <= code <= 0x318F: + return "hangul" + if ( + 0x3040 <= code <= 0x30FF + or 0x31F0 <= code <= 0x31FF + or 0x3400 <= code <= 0x4DBF + or 0x4E00 <= code <= 0x9FFF + or 0xF900 <= code <= 0xFAFF + or 0x20000 <= code <= 0x3FFFF + or code == 0x3005 + ): + return "hankana" + if not ch.isalpha(): + return "" + if 0x0400 <= code <= 0x052F: + return "cyrillic" + if 0x0370 <= code <= 0x03FF: + return "greek" + return "latin" + + +def _token_script(token: str) -> str: + """The one script a word is written in, "mixed" for more than one + ("Official髭男dism"), "" for none.""" + found = {s for s in map(_script, token) if s} + if len(found) == 1: + return found.pop() + return "mixed" if found else "" + + +def is_east_asian(text: str) -> bool: + """Whether ``text`` has a Chinese character, kana or Hangul in it.""" + return any(_script(ch) in ("hangul", "hankana") for ch in text) + + +def _script_runs(text: str) -> list[str]: + """``text`` cut where one whole word in one script is followed by one in + another: "周杰倫 Jay Chou" is two names, "晴天 Sunny Day" two titles. A + word mixing scripts ("Official髭男dism") is never cut. Empty when there is + only one run.""" + runs: list[list[str]] = [] + current = "" + for token in text.split(): + script = _token_script(token) + pure = script not in ("", "mixed") + if runs and pure and current and script != current: + runs.append([token]) + current = script + elif runs: + runs[-1].append(token) + current = current or (script if pure else "") + else: + runs.append([token]) + current = script if pure else "" + if len(runs) < 2: + return [] + return [" ".join(run) for run in runs] + + +def alternatives(text: Any) -> list[str]: + """Every name one part of a title gives a thing, the first one first: + "IU(아이유)" is IU and 아이유, "周杰倫 Jay Chou" 周杰倫 and Jay Chou, + "Official髭男dism" only itself.""" + names: list[str] = [] + keys: set[str] = set() + for piece in str(text or "").split(_ALT): + piece = _clean_part(piece) + for name in _script_runs(piece) or [piece]: + name = _clean_part(name) + key = song_key(name) + if name and key and key not in keys: + keys.add(key) + names.append(name) + return names + + +# A bracket after a name that is not another name for it: a featured artist, +# a producer, where the song is from. +_CREDIT_RE = re.compile( + r"^\s*(?:ft\.?|feat\.?|featuring|with|prod\.?|produced\s+by|cover(?:ed)?\s+by|by)\s", + re.IGNORECASE, +) + + +def _mark_alternatives(text: str) -> str: + """Round and square brackets holding another name for what they follow, + in another script ("IU(아이유)", "봄날 (Spring Day)"), turned into _ALT + and that name, so it is kept as an alternative rather than dropped with + the other brackets.""" + out: list[str] = [] + last = 0 + for match in _ROUND_RE.finditer(text): + inner = match.group(1).strip() + before = next((s for s in map(_script, reversed(text[: match.start()])) if s), "") + after = next((s for s in map(_script, inner) if s), "") + if ( + before + and after + and before != after + and not _has_noise(inner) + and not _CREDIT_RE.match(inner) + and not _FROM_RE.match(inner) + and not kind_of(inner) + ): + out.append(text[last : match.start()].rstrip() + _ALT + inner) + else: + out.append(text[last : match.end()]) + last = match.end() + out.append(text[last:]) + return "".join(out) + + +def _song_mark_patterns() -> list[tuple[re.Pattern[str], bool]]: + patterns = [] + for opening, closing in _SONG_MARKS.items(): + both = re.escape(opening + closing) + pattern = re.compile(f"{re.escape(opening)}([^{both}]*){re.escape(closing)}") + patterns.append((pattern, opening in _LENTICULAR)) + return patterns + + +_SONG_MARK_PATTERNS = _song_mark_patterns() + + +def _mark_songs(text: str) -> str: + """Each song mark with its content: noise dropped ("【MV】", 「MV」), any + other name kept between _MARK_OPEN and _MARK_CLOSE. A lenticular bracket + with any noise in it at all ("【Japanese ver.】") is dropped too: those + describe the upload far more often than they name a song.""" + + def mark(match: re.Match[str], lenticular: bool) -> str: + inner = match.group(1).strip() + if not inner or (_has_noise(inner) if lenticular else _is_noise(inner)): + return " " + return f" {_MARK_OPEN}{inner}{_MARK_CLOSE} " + + for pattern, lenticular in _SONG_MARK_PATTERNS: + text = pattern.sub(lambda m, lent=lenticular: mark(m, lent), text) + for ch in (*_SONG_MARKS, *_SONG_MARKS.values()): + text = text.replace(ch, " ") + return text + + +def _normalize(title: Any) -> str: + """``title`` with control and private-use characters gone, full-width + ASCII and half-width katakana made ordinary (NFKC, for those blocks only: + elsewhere it would change letters), and the ideographic space, the wave + dash and the tortoise-shell brackets read as a space, "~" and "()".""" + text = _PRIVATE_USE_RE.sub(" ", str(title or "")[:_TITLE_MAX_CHARS]) + out = [] + for ch in text: + code = ord(ch) + if ch < " ": + out.append(" ") + elif code in (0x3001, 0xFF0C): + # The ideographic and full-width commas, which no space follows. + out.append(", ") + elif 0xFF01 <= code <= 0xFF5E or 0xFF61 <= code <= 0xFF9F or 0xFFE0 <= code <= 0xFFE6: + out.append(unicodedata.normalize("NFKC", ch)) + elif code == 0x3000: + out.append(" ") + elif code == 0x301C: + out.append("~") + else: + out.append(_ROUND_ALIASES.get(ch, ch)) + return unicodedata.normalize("NFC", "".join(out)) + + +def _prepare(title: Any) -> tuple[str, dict[str, Any]]: + """The title ready to split: normalised, song marks and alternative + names marked, brackets read and dropped. And what the brackets said: + the work, its kind, the year.""" + text = _normalize(title) + text = _WAVE_RE.sub(r" (\1) ", _HASHTAG_RE.sub(" ", text)) + text = _GAP_RE.sub(f" {_GAP} ", text) + text = _mark_songs(text) + text = _mark_alternatives(text) + text = _SINGLE_QUOTED_RE.sub(lambda m: chr(0x201C) + m.group(1) + chr(0x201D), text) + found: dict[str, Any] = {"work": "", "kind": None, "year": None} + + def bracket(match: re.Match[str]) -> str: + inner = match.group(1).strip() + source = _FROM_RE.match(inner) + if source: + work, kind, year = _from_work(source.group(1)) + found["work"] = found["work"] or work + found["kind"] = found["kind"] or kind + found["year"] = found["year"] or year + elif not _FEAT_RE.match(inner): + kind = kind_of(inner) + if kind: + found["kind"] = found["kind"] or kind + found["year"] = found["year"] or _year(inner) + # Whatever it was, it is not part of the song's name. + return " " + + for _ in range(3): + stripped = _BRACKET_RE.sub(bracket, text) + if stripped == text: + break + text = stripped + return text, found + + +def _unmark(text: str) -> str: + return text.replace(_MARK_OPEN, " ").replace(_MARK_CLOSE, " ") + + +# ── the readings ── + + +def parse_title(title: Any) -> list[TitleReading]: + """Every reading of ``title``, most likely first, at most MAX_READINGS, + each with a song. Empty when the title names nothing.""" + text, found = _prepare(title) + kind: str | None = found["kind"] + year: int | None = found["year"] + bracket_work: str = found["work"] + readings: list[TitleReading] = [] + + def add(song: str, others: list[str]) -> None: + names = alternatives(song) or [_clean_part(song.replace(_ALT, " "))] + song = names[0] + if not song or not words(song) and not song_key(song): + return + artist_parts: list[str] = [] + artist_alts: tuple[str, ...] = () + work = bracket_work + reading_kind = kind + for other in others: + other_names = alternatives(other) or [_clean_part(other.replace(_ALT, " "))] + primary = other_names[0] + named, named_kind = _named_work(primary) + if named: + work = work or named + reading_kind = reading_kind or named_kind + continue + if not primary: + continue + artist_parts.append(primary) + artist_alts = tuple(other_names[1:]) + work = work or cast_work(primary) + artist = ", ".join(artist_parts) + if len(artist_parts) != 1: + artist_alts = () + # A title that says it is a cast recording or a soundtrack, and names + # no work of its own, names it beside the song, if anywhere. + if not work and reading_kind and artist_parts: + work = artist_parts[-1] + reading = TitleReading( + song=song, + artist=artist, + work=work, + kind=reading_kind, + year=year, + song_alts=tuple(names[1:]), + artist_alts=artist_alts, + ) + if reading not in readings: + readings.append(reading) + + # A song in marks ("YOASOBI「夜に駆ける」", "周杰倫【晴天】"): the marks say + # which part is the song, and the rest, up to a separator, who sings it. + # The first marked name, then the last (an anime's name can come first: + # "『鬼滅の刃』主題歌 LiSA「紅蓮華」"), then the first the other way round + # ("【周杰倫】晴天"). + marked = [m for m in _MARKED_RE.finditer(text) if song_key(m.group(1).replace(_ALT, " "))] + if marked: + reverse: tuple[str, str] | None = None + for match in dict.fromkeys((marked[0], marked[-1])): + # Any other marked name is left out: a work ("『鬼滅の刃』") or a + # second song, not who sings this one. + outside = _MARKED_RE.sub(" ", text[: match.start()] + " " + text[match.end() :]) + parts, _ = _split(outside) + add(match.group(1), parts[:1]) + if reverse is None and parts: + reverse = (parts[0], match.group(1)) + if reverse: + add(reverse[0], [reverse[1]]) + return readings[:MAX_READINGS] + + parts, separator = _split(text) + if not parts: + return [] + if len(parts) == 1: + raw = _SPACES_RE.sub(" ", text.replace(_GAP, " ")).strip() + quoted = _QUOTED_RE.search(raw) + outside = _clean_part(raw[: quoted.start()] + " " + raw[quoted.end() :]) if quoted else "" + if quoted and outside: + # 'Keala Settle "This Is Me"', "NewJeans (뉴진스) 'Ditto'". + add(quoted.group(1), [outside]) + else: + add(parts[0], []) + elif len(parts) == 2: + first, second = parts + if separator in ("gap", "pipe"): + # "Song Name, Name": the song, then who sings it. "Song | Work". + add(first, [second]) + add(second, [first]) + elif cast_work(first) or _named_work(first)[0]: + add(second, [first]) + elif cast_work(second) or _named_work(second)[0]: + add(first, [second]) + else: + add(second, [first]) + add(first, [second]) + else: + # "Artist - Song - Work": the middle first, then either end. + order = [1, 0, len(parts) - 1] + for index in dict.fromkeys(order): + add(parts[index], [p for i, p in enumerate(parts) if i != index]) + return readings[:MAX_READINGS] + + +def _split(text: str) -> tuple[list[str], str]: + """The parts of a title between its separators, and which separator it + was: "dash", "pipe", "colon", "gap" (a run of spaces) or "" for none. + Dashes first (and "_", "/", "~", which East Asian titles use the same + way), then pipes; else a colon ("Work: Song", unless it is "Work: The + Musical"); else a run of spaces.""" + text = text.strip() + flat = text.replace(_GAP, " ") + for pattern, name in ((_DASH_SPLIT_RE, "dash"), (_PIPE_SPLIT_RE, "pipe")): + pieces = pattern.split(flat) + parts = [p for p in (_clean_part(p) for p in pieces) if p] + if len(parts) > 1: + return parts, name + if len(pieces) > 1 and parts: + # Everything beside one part was noise: "Artist -Official Video". + flat = parts[0] + single = _clean_part(flat) + if not _named_work(single)[0]: + parts = [ + p for p in (_clean_part(p) for p in _COLON_SPLIT_RE.split(single, maxsplit=1)) if p + ] + if len(parts) > 1: + return parts, "colon" + parts = [p for p in (_clean_part(p) for p in _GAP_SPLIT_RE.split(text)) if p] + if len(parts) > 1: + return [parts[0], ", ".join(parts[1:])], "gap" + return ([single] if single else []), "" + + +def readings_for(tags: dict[str, str] | None, *titles: Any) -> list[TitleReading]: + """The readings worth resolving for a track: its tags' artist and title + first, when both are there, then each title's readings, those that name + more than a song (resolvable), duplicates dropped.""" + tags = tags if isinstance(tags, dict) else {} + artist = str(tags.get("artist") or "").strip() + tag_title = str(tags.get("title") or "").strip() + found: list[TitleReading] = [] + if artist and tag_title: + # "鄧麗君 Teresa Teng テレサ・テン", a channel's name: each is a name. + names = alternatives(artist) or [artist] + keys = {song_key(n) for n in (artist, *names)} + tag_readings = parse_title(tag_title) + marked = _MARKED_RE.search(_prepare(tag_title)[0]) + for index, reading in enumerate(tag_readings): + # A song in marks is the song whatever the rest of a tag's title + # says: the artist is the tag's ("【告白氣球】MV花絮 Making of"). + in_marks = index == 0 and marked is not None + if in_marks or not reading.artist or song_key(reading.artist) in keys: + found.append( + TitleReading( + song=reading.song, + artist=names[0], + work=reading.work or cast_work(artist), + kind=reading.kind, + year=reading.year, + song_alts=reading.song_alts, + artist_alts=tuple(names[1:]), + ) + ) + break + for title in (tag_title, *titles): + found += parse_title(title) + out: list[TitleReading] = [] + seen: set[tuple[str, frozenset[str]]] = set() + for reading in found: + key = (song_key(reading.song), extra_words(reading)) + if resolvable(reading) and key not in seen: + seen.add(key) + out.append(reading) + return out[:MAX_READINGS] + + +def work_hint(title: Any, song: Any = None) -> TitleReading | None: + """The reading of ``title`` that names a work: the one whose song is + ``song`` when that is given (the song the track was identified as), else + the first. None when the title names no work.""" + want = song_key(song) if song else "" + for reading in parse_title(title): + if not reading.work: + continue + if not want or song_key(reading.song) == want: + return reading + return None + + +# ── who a title says the artist is, for finding them by name ── + +# What a compilation or a playlist adds to its artist's name: "邓丽君经典金曲" +# is Teresa Teng's classic hits, "Teresa Teng's classic songs" the same. In +# Chinese, Japanese and Korean, found anywhere, longest first ("20首" is +# twenty songs); elsewhere as whole words. +_CJK_COMPILATION = re.split( + r"\s+", + "经典金曲 經典金曲 经典老歌 經典老歌 经典歌曲 經典歌曲 经典 經典 金曲 精选集 精選集 精选 精選" + " 合集 串烧 串燒 好听的歌 好聽的歌 好听 好聽 歌曲 名曲集 名曲 老歌 专辑 專輯 全集 大全 歌单" + " 歌單 首歌 ベストアルバム ベスト メドレー 作業用BGM 作業用 메들리 노래모음 모음 히트곡 명곡" + " 연속듣기 플레이리스트", +) +_CJK_COMPILATION_RE = re.compile( + r"\d+\s*首|" + "|".join(re.escape(w) for w in sorted(_CJK_COMPILATION, key=len, reverse=True)) +) +_COMPILATION_RE = re.compile( + r"(?:['" + chr(0x2019) + r"]s\b)?\s*\b(?:the\s+)?(?:greatest\s+hits|best\s+of|best\s+songs?" + r"|best\s+hits|classic\s+(?:songs?|hits)|golden\s+(?:hits|songs?)" + r"|top\s+(?:\d+\s+)?(?:songs?|hits)|all\s+songs|full\s+album|playlist|collection" + r"|compilation|medley|non-?stop|hits|songs|album|grandes\s+exitos|mejores\s+canciones" + r"|maiores\s+sucessos|kumpulan\s+lagu|lagu\s+terbaik|meilleures\s+chansons)\b", + re.IGNORECASE, +) +# A list of artists: "Antônio Carlos Jobim, Vinicius de Moraes", "IU x SUGA". +_MEMBERS_RE = re.compile( + r"\s*(?:&|,|\+|;|/|\band\b|\bfeat\.?|\bft\.?|\bwith\b|" + chr(0xD7) + r")\s*|\s+x\s+", + re.IGNORECASE, +) +# A title with no separator is cut into artist and song only when it has +# this many words at most: every cut is a name to look for. +_SPLIT_MAX_WORDS = 6 + + +def _without_compilation(name: str) -> str: + """``name`` less what a compilation adds to it, or "" when it adds + nothing (or is all there is).""" + stripped = _clean_part(_COMPILATION_RE.sub(" ", _CJK_COMPILATION_RE.sub(" ", name))) + return stripped if stripped != name and song_key(stripped) else "" + + +def _names(first: list[str]) -> list[str]: + names: list[str] = [] + for name in first: + if not name: + continue + names.append(name) + members = [m for m in _MEMBERS_RE.split(name) if m and m.strip()] + if len(members) > 1: + names += [m.strip() for m in members] + names += [s for s in map(_without_compilation, names) if s] + out: list[str] = [] + keys: set[str] = set() + for name in names: + key = name_key(name) + if key and key not in keys: + keys.add(key) + out.append(name) + return out + + +def artist_names(reading: TitleReading) -> list[str]: + """Every name the reading's artist might go by on MusicBrainz: each name + the title gives it, each one in a list ("A, B"), and each without what a + compilation adds to it ("邓丽君经典金曲").""" + return _names([reading.artist, *reading.artist_alts]) + + +def name_alternatives(name: Any) -> list[str]: + """artist_names for a bare artist name, a tag: "鄧麗君 Teresa Teng + テレサ・テン" is three names, "Xutos & Pontapés" one band or two people.""" + text = _normalize(name).strip() + return _names([text, *(alternatives(_mark_alternatives(text)) or [])]) + + +def name_splits(title: Any) -> list[tuple[str, str]]: + """(artist, song) for every place a title naming both with nothing + between them could be cut ("美空ひばり 川の流れのように", "Herbert + Grönemeyer Männer"), both ways round, the shortest artist first. Empty + when the title has a separator, marks or quotes, which say where the cut + is, or too many words to try them all, or too few (two, unless Chinese, + Japanese or Korean).""" + text, _found = _prepare(title) + if _MARKED_RE.search(text) or _QUOTED_RE.search(text): + return [] + parts, _separator = _split(text.replace(_ALT, " ")) + if len(parts) != 1: + return [] + tokens = parts[0].split() + # Two Latin words are as often one song's name ("Bohemian Rhapsody") as + # an artist and a song, and cost a search either way: three at least. + fewest = 2 if is_east_asian(parts[0]) else 3 + if not fewest <= len(tokens) <= _SPLIT_MAX_WORDS: + return [] + cuts: list[tuple[str, str]] = [] + for size in range(1, len(tokens)): + cuts.append((" ".join(tokens[:size]), " ".join(tokens[size:]))) + cuts.append((" ".join(tokens[-size:]), " ".join(tokens[:-size]))) + return cuts + + +def script_of(name: str) -> str: + """The scripts a name is written in, as one string ("hankana", + "latin", "hankana+latin"): two names in different scripts are two + independent spellings of whatever they both name.""" + return "+".join(sorted({s for s in map(_script, name) if s})) + + +def title_names(title: Any) -> tuple[list[list[str]], bool]: + """The names a title gives, for finding its artist when no recording was + found: per part of the title, every name it gives (alternatives), each + less what a compilation adds ("邓丽君经典金曲" is 邓丽君, "Teresa Teng's + classic songs" is Teresa Teng); and whether any part had such words. + + Nothing here says which name is the artist: find_band_for takes one only + when the title corroborates it (two scripts naming one artist, or a + compilation naming nobody else), never a part that merely is a band's + name ("Metropolis - Part I").""" + text, _found = _prepare(title) + parts, _separator = _split(_unmark(text)) + groups: list[list[str]] = [] + compilation = False + for part in parts: + names: list[str] = [] + for name in alternatives(part) or []: + rest = _clean_part(_COMPILATION_RE.sub(" ", _CJK_COMPILATION_RE.sub(" ", name))) + if rest != name: + compilation = True + if song_key(rest) and name_key(rest) not in {name_key(n) for n in names}: + names.append(rest) + if names: + groups.append(names) + return groups, compilation diff --git a/app/pipeline/transcribe.py b/app/pipeline/transcribe.py new file mode 100644 index 00000000..054cecfd --- /dev/null +++ b/app/pipeline/transcribe.py @@ -0,0 +1,611 @@ +"""Lyrics transcribed from the vocals stem when no lookup found any. + +The last stage of a job, after separation and after the lyrics lookup +(lyrics_lookup.py) has been joined: it needs the vocals stem, and it only +runs when the lookup came back empty, so that a track with published lyrics +never pays for it. Whether it runs at all is the transcribe_lyrics setting +("auto" means a CUDA job only; mending found lyrics, below, runs on any +device unless it is "off"). + +Inference happens in a fresh worker process (transcribe_worker.py), with the +same cancellation, total timeout and output-stall watchdog as the section +worker. The process exiting is what releases its GPU memory, so the persistent +Demucs worker, which shares the card, keeps its own allocation untouched. + +When the lookup did find lyrics but they look like a copy that lost its +accents (lyrics_repair.py), the same worker is asked instead to detect the +language and, only for one written with accents, transcribe: the heard words +give the letters back to LRCLIB's lines, whose timing is kept (mend_lyrics). + +Lyrics are optional. Every failure here is logged and the job finishes +without them; only a cancel propagates. + +The result is lyrics.json with source "whisper": Whisper's segments as lines, +long ones broken at pauses, written as enhanced LRC whose ```` word +stamps are what the Lyrics tab's karaoke wipe follows (parseLrc and +wordTimings in static/js/lyricsLookup.js). +""" + +from __future__ import annotations + +import json +import logging +import math +import os +import re +import subprocess +import sys +import threading +import time +from collections import deque +from collections.abc import Callable +from pathlib import Path +from typing import Any + +from app.core.config import ( + LYRICS_REPAIR_LANGUAGES, + LYRICS_REPAIR_MIN_LANGUAGE_PROB, + TIMEOUT_TRANSCRIBE, + TIMEOUT_TRANSCRIBE_STALL, + TRANSCRIBE_GPU_MIN_FREE_MB, + TRANSCRIBE_HALLUCINATION_SILENCE_SEC, + TRANSCRIBE_LINE_BREAK_GAP_SEC, + TRANSCRIBE_LINE_HARD_CHARS, + TRANSCRIBE_LINE_SOFT_CHARS, + TRANSCRIBE_MIN_VOCAL_PRESENCE, + TRANSCRIBE_MODEL_CPU, + TRANSCRIBE_MODEL_GPU, + TRANSCRIBE_SMALL_GPU_MIN_FREE_MB, + TRANSCRIBE_STANZA_GAP_SEC, + TRANSCRIBE_WORD_GAP_SEC, +) +from app.core.models import Job, JobCancelled, _set +from app.core.registry import set_proc +from app.core.settings import lyrics_mending_enabled, transcribe_lyrics_enabled +from app.pipeline.lyrics_lookup import lyrics_path, read_lyrics, write_lyrics +from app.pipeline.lyrics_repair import looks_stripped, mend_from_transcript, words_of +from app.pipeline.lyrics_retime import save_transcript + +logger = logging.getLogger("stemdeck.transcribe") + +_PHASE_PREFIX = "@@PHASE@@" +_PCT_RE = re.compile(r"(\d{1,3})%\|") +# Whisper's own rules for a window it should have skipped: silence (a likely +# no-speech token and a low log probability together) and a loop (text that +# compresses too well). A segment that still carries either is not kept. +_NO_SPEECH_PROB = 0.6 +_LOGPROB_FLOOR = -1.0 +_COMPRESSION_RATIO_MAX = 2.4 +# Punctuation a line may end on once it is long enough. +_CLAUSE_END = frozenset(",.;:!?") +# A pause long enough to end a line that is already long enough. +_SOFT_BREAK_GAP_SEC = 0.25 +# Fewer words than this is not a transcript, just Whisper hearing something. +_MIN_WORDS = 3 + +_STAGE = "Transcribing lyrics..." + + +def whisper_models_dir() -> Path: + """Where Whisper's weights are kept: /whisper, beside the + Demucs checkpoints. TORCH_HOME is set for the desktop app (to the data + directory's models/torch) and for Docker (/cache/torch); elsewhere this + is torch's own default, worked out the way torch.hub does, so the server + process does not have to import torch to know it.""" + torch_home = os.environ.get("TORCH_HOME", "").strip() + if torch_home: + base = Path(torch_home).expanduser() + else: + cache = os.environ.get("XDG_CACHE_HOME", "").strip() + base = (Path(cache).expanduser() if cache else Path.home() / ".cache") / "torch" + return base / "whisper" + + +def _spawn_worker_cmd(vocals: Path, device: str) -> list[str]: + """The worker invocation. Module-level seam so tests can swap in a stub + executable, the way separate.py exposes _spawn_worker_cmd.""" + return [ + sys.executable, + "-m", + "app.pipeline.transcribe_worker", + "--audio", + str(vocals), + "--device", + device, + "--model", + TRANSCRIBE_MODEL_GPU, + "--fallback-model", + TRANSCRIBE_MODEL_CPU, + "--download-root", + str(whisper_models_dir()), + "--min-free-mb", + str(TRANSCRIBE_GPU_MIN_FREE_MB), + "--small-min-free-mb", + str(TRANSCRIBE_SMALL_GPU_MIN_FREE_MB), + "--silence-sec", + str(TRANSCRIBE_HALLUCINATION_SILENCE_SEC), + ] + + +_FOUND = "lyrics already found" + + +def skip_reason(job: Job, job_dir: Path) -> str | None: + """Why this job gets no transcription, or None when it should get one.""" + if not transcribe_lyrics_enabled(job.compute_device): + return f"off for device {job.compute_device or 'unknown'}" + if job.has_lyrics or lyrics_path(job_dir).is_file(): + return _FOUND + tags = job.audio_tags or {} + if isinstance(tags.get("lyrics"), str) and tags["lyrics"].strip(): + return "the file carries its own lyrics" + return _vocals_reason(job, job_dir) + + +def _vocals_reason(job: Job, job_dir: Path) -> str | None: + """Why this job's vocals are not worth transcribing, or None.""" + if not (job_dir / "stems" / "vocals.wav").is_file(): + return "no vocals stem" + presence = (job.stem_presence or {}).get("vocals") + if isinstance(presence, int) and presence < TRANSCRIBE_MIN_VOCAL_PRESENCE: + return f"vocals too quiet ({presence})" + return None + + +def _device(job: Job) -> str: + # macOS has no CUDA, and Whisper's alignment buffer is a sparse tensor + # MPS cannot hold, so an MPS job transcribes on the CPU, CPU model. + return "cuda" if job.compute_device == "cuda" else "cpu" + + +def _terminate(proc: subprocess.Popen) -> None: + if proc.poll() is not None: + return + proc.terminate() + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.kill() + # wait, not communicate: the reader threads own the pipes. + proc.wait(timeout=5) + + +def _stage_for(phase: str, pct: int | None) -> str: + if phase == "download": + return f"Downloading the lyrics model {pct}%" if pct is not None else _STAGE + if phase == "transcribe" and pct is not None: + return f"Transcribing lyrics {pct}%" + return _STAGE + + +def _run_worker( + job: Job, + cmd: list[str], + *, + cancelled: Callable[[], bool] | None = None, + report: Callable[[str, int | None], None] | None = None, +) -> dict[str, Any] | None: + """Run the worker with cancellation, a total timeout and stall detection. + Returns its JSON answer, or None when it failed, timed out or stalled. + Raises JobCancelled when the job was cancelled meanwhile. + + ``cancelled`` says when to stop (by default, the job's own cancel), and + ``report`` hears each phase and percentage (by default, the job's stage + text shows them): a run on a finished job (lyrics_retime.py) keeps its own + cancel and progress and leaves the job's stage alone.""" + if cancelled is None: + + def cancelled() -> bool: + return job.cancel_requested + + env = os.environ.copy() + # The worker hard-exits when this process disappears, so a Force Quit + # cannot leave it holding the GPU (#519). + env["STEMDECK_PARENT_PID"] = str(os.getpid()) + env["PYTHONIOENCODING"] = "utf-8:replace" + proc = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + encoding="utf-8", + errors="replace", + bufsize=1, + env=env, + ) + if proc.stdout is None or proc.stderr is None: + _terminate(proc) + raise RuntimeError("transcription worker has no output pipes") + + stdout: deque[str] = deque(maxlen=4) + stderr: deque[str] = deque(maxlen=40) + lock = threading.Lock() + last_output = [time.monotonic()] + # [phase, percentage]; read by the loop below, which owns the job. + progress: list[Any] = ["load", None] + + def read_stdout() -> None: + for line in proc.stdout: + with lock: + stdout.append(line.rstrip("\n")) + last_output[0] = time.monotonic() + + def read_stderr() -> None: + # Universal newlines split tqdm's carriage-return redraws into lines. + for line in proc.stderr: + line = line.rstrip() + with lock: + last_output[0] = time.monotonic() + if line.startswith(_PHASE_PREFIX): + progress[:] = [line[len(_PHASE_PREFIX) :], None] + continue + m = _PCT_RE.search(line) + if m: + progress[1] = max(0, min(100, int(m.group(1)))) + elif line: + stderr.append(line) + + readers = [ + threading.Thread(target=read_stdout, daemon=True), + threading.Thread(target=read_stderr, daemon=True), + ] + for reader in readers: + reader.start() + + started = time.monotonic() + shown: Any = _STAGE if report is None else None + failed = None + set_proc(job.id, proc) + try: + while proc.poll() is None: + if cancelled(): + _terminate(proc) + raise JobCancelled() + now = time.monotonic() + with lock: + silent_for = now - last_output[0] + phase, pct = progress[0], progress[1] + if report is not None: + if (phase, pct) != shown: + shown = (phase, pct) + report(phase, pct) + else: + stage = _stage_for(phase, pct) + if stage != shown: + shown = stage + _set(job, stage=stage) + if now - started > TIMEOUT_TRANSCRIBE: + failed = f"timed out after {TIMEOUT_TRANSCRIBE}s" + _terminate(proc) + break + if silent_for > TIMEOUT_TRANSCRIBE_STALL: + failed = f"stalled for {TIMEOUT_TRANSCRIBE_STALL}s" + _terminate(proc) + break + time.sleep(0.1) + finally: + set_proc(job.id, None) + for reader in readers: + reader.join(timeout=2) + + if cancelled(): + raise JobCancelled() + if failed is None and proc.returncode != 0: + failed = f"exit {proc.returncode}" + if failed is not None: + logger.warning( + "[%s] lyrics transcription %s: %s", + job.id, + failed, + " | ".join(list(stderr)[-5:]) or "no diagnostic output", + ) + return None + lines = [line for line in stdout if line.startswith("{")] + if len(lines) != 1: + logger.warning("[%s] lyrics transcription returned unexpected output", job.id) + return None + try: + answer = json.loads(lines[0]) + except json.JSONDecodeError: + logger.warning("[%s] lyrics transcription returned invalid JSON", job.id) + return None + return answer if isinstance(answer, dict) else None + + +# ── the transcript as lyrics ── + + +def _number(value: Any) -> float | None: + if isinstance(value, bool) or not isinstance(value, (int, float)): + return None + value = float(value) + return value if math.isfinite(value) else None + + +def _kept(segment: Any) -> bool: + if not isinstance(segment, dict): + return False + no_speech = _number(segment.get("no_speech_prob")) or 0.0 + logprob = _number(segment.get("avg_logprob")) or 0.0 + if no_speech > _NO_SPEECH_PROB and logprob < _LOGPROB_FLOOR: + return False + return (_number(segment.get("compression_ratio")) or 0.0) <= _COMPRESSION_RATIO_MAX + + +def _words(segment: dict[str, Any], floor: float) -> list[dict[str, Any]]: + """The segment's words as {"text", "start", "end"}, in order, never + earlier than ``floor`` and never ending before they start. Whisper's + stamps can overlap by a few hundredths across words; the wipe needs them + monotonic.""" + out: list[dict[str, Any]] = [] + for word in segment.get("words") or []: + if not isinstance(word, dict): + continue + text = str(word.get("word", "")).strip() + start = _number(word.get("start")) + end = _number(word.get("end")) + if not text or start is None or end is None: + continue + start = max(start, floor) + end = max(end, start) + out.append({"text": text, "start": start, "end": end}) + floor = start + return out + + +def split_lines(words: list[dict[str, Any]]) -> list[list[dict[str, Any]]]: + """One segment's words as lines. A line ends at a long pause, at a clause + end or a short pause once it is long enough, and always before it would + pass the hard length cap. A long pause after a single word does not end + the line: a held "Yeah," before the next "yeah" is one line, not two.""" + lines: list[list[dict[str, Any]]] = [] + current: list[dict[str, Any]] = [] + length = 0 + for word in words: + if current: + gap = word["start"] - current[-1]["end"] + soft = length >= TRANSCRIBE_LINE_SOFT_CHARS and ( + current[-1]["text"][-1] in _CLAUSE_END or gap >= _SOFT_BREAK_GAP_SEC + ) + if ( + (gap >= TRANSCRIBE_LINE_BREAK_GAP_SEC and len(current) > 1) + or soft + or length + 1 + len(word["text"]) > TRANSCRIBE_LINE_HARD_CHARS + ): + lines.append(current) + current, length = [], 0 + length += len(word["text"]) + (1 if current else 0) + current.append(word) + if current: + lines.append(current) + return lines + + +def lrc_time(seconds: float) -> str: + """mm:ss.xx, the stamp LRC uses. Rounded to the hundredth first, so 59.999 + is 01:00.00 and never 00:60.00.""" + centis = max(0, round(seconds * 100)) + minutes, rest = divmod(centis, 6000) + return f"{minutes:02d}:{rest // 100:02d}.{rest % 100:02d}" + + +def lrc_line(words: list[dict[str, Any]]) -> str: + """One line as enhanced LRC: "[00:12.00]<00:12.00>Hello <00:12.60>world<00:13.10>". + + Each word carries its start. A pause of TRANSCRIBE_WORD_GAP_SEC or more + after a word is written as its end stamp, an empty piece the page skips, + so the wipe waits through the pause instead of stretching the word; the + last word always carries its end, so the wipe finishes when the singing + does rather than at the next line.""" + parts = [f"[{lrc_time(words[0]['start'])}]"] + for i, word in enumerate(words): + parts.append(f"<{lrc_time(word['start'])}>{word['text']}") + following = words[i + 1] if i + 1 < len(words) else None + if following is None: + parts.append(f"<{lrc_time(word['end'])}>") + else: + parts.append(" ") + if following["start"] - word["end"] >= TRANSCRIBE_WORD_GAP_SEC: + parts.append(f"<{lrc_time(word['end'])}>") + return "".join(parts) + + +def _names(job: Job) -> tuple[str, str, str]: + """(track, artist, album) for lyrics.json: the identified recording, else + the file's tags, else the job's title.""" + identity = job.identity or {} + tags = job.audio_tags or {} + track = identity.get("title") or tags.get("title") or job.title or "" + artist = identity.get("artist") or tags.get("artist") or "" + album = identity.get("album") or tags.get("album") or "" + return str(track), str(artist), str(album) + + +def build_lyrics(result: dict[str, Any], job: Job) -> dict[str, Any] | None: + """lyrics.json from the worker's answer, or None when it heard nothing + worth keeping.""" + segments = result.get("segments") + if not isinstance(segments, list): + return None + lines: list[list[dict[str, Any]]] = [] + floor = 0.0 + for segment in segments: + if not _kept(segment): + continue + words = _words(segment, floor) + if not words: + continue + floor = words[-1]["start"] + lines.extend(split_lines(words)) + if sum(len(line) for line in lines) < _MIN_WORDS: + return None + + plain: list[str] = [] + for i, line in enumerate(lines): + if i and line[0]["start"] - lines[i - 1][-1]["end"] >= TRANSCRIBE_STANZA_GAP_SEC: + plain.append("") + plain.append(" ".join(word["text"] for word in line)) + track, artist, album = _names(job) + duration = _number(job.duration_sec) or lines[-1][-1]["end"] + language = result.get("language") + return { + "v": 1, + "source": "whisper", + "track": track, + "artist": artist, + "album": album, + "duration": float(duration), + "synced": "\n".join(lrc_line(line) for line in lines), + "plain": "\n".join(plain), + "instrumental": False, + "others": [], + "lrclib_id": None, + "language": language if isinstance(language, str) else None, + } + + +def transcribe_lyrics(job: Job, job_dir: Path) -> bool: + """Transcribe the job's vocals into lyrics.json when it has no lyrics and + the setting allows. True when lyrics were written. Never raises except + JobCancelled: lyrics are optional, and a failure here must not cost the + user a separation that has already succeeded.""" + if job.cancel_requested: + raise JobCancelled() + started = time.monotonic() + try: + if lyrics_path(job_dir).is_file(): + # Found lyrics may have lost their accents: mended, not replaced, + # and on any device unless the setting is "off" (see + # lyrics_mending_enabled). + return lyrics_mending_enabled() and mend_lyrics(job, job_dir) + reason = skip_reason(job, job_dir) + if reason is not None: + logger.info("[%s] lyrics transcription skipped: %s", job.id, reason) + return False + _set(job, stage=_STAGE) + vocals = job_dir / "stems" / "vocals.wav" + result = _run_worker(job, _spawn_worker_cmd(vocals, _device(job))) + if result is None: + return False + entry = build_lyrics(result, job) + if entry is None: + logger.info("[%s] lyrics transcription heard no words", job.id) + return False + # A lookup still in flight is dropped by its finish(), so nothing + # else writes this file now; checked again all the same, since + # published lyrics always beat a transcript. + if lyrics_path(job_dir).is_file(): + return False + if not write_lyrics(job, job_dir, entry): + return False + except JobCancelled: + raise + except Exception: + logger.exception("[%s] lyrics transcription failed", job.id) + return False + logger.info( + "[%s] lyrics transcribed in %.1fs: model=%s device=%s language=%s (%.2f) " + "lines=%d peak_vram_mb=%s", + job.id, + time.monotonic() - started, + result.get("model"), + result.get("device"), + result.get("language"), + _number(result.get("language_probability")) or 0.0, + entry["synced"].count("\n") + 1, + result.get("peak_vram_mb"), + ) + return True + + +# ── lyrics that lost their accents ── + + +def _language_gate() -> list[str]: + """The worker arguments that make it stop once it has detected the + language, unless that is one written with letters outside ASCII.""" + return [ + "--languages", + ",".join(sorted(LYRICS_REPAIR_LANGUAGES)), + "--min-language-probability", + str(LYRICS_REPAIR_MIN_LANGUAGE_PROB), + ] + + +def heard_words(result: dict[str, Any]) -> list[str]: + """The words of the worker's answer, in order, from the segments + build_lyrics would keep.""" + segments = result.get("segments") + heard: list[str] = [] + for segment in segments if isinstance(segments, list) else []: + if _kept(segment): + for word in _words(segment, 0.0): + heard.extend(words_of(word["text"])) + return heard + + +def mend_lyrics(job: Job, job_dir: Path) -> bool: + """Give the job's LRCLIB lyrics back the letters they lost, from what + Whisper hears in its vocals. True when lyrics.json was rewritten, with + "repaired": "whisper". + + Three gates, cheapest first, so that English and lyrics that have their + accents never start the worker, and a song written without accents is + never touched: the text (lyrics_repair.looks_stripped); the language + Whisper detects in the vocals, before anything is transcribed + (LYRICS_REPAIR_LANGUAGES); then the transcription, which must give + letters back to enough words (lyrics_repair.mend_from_transcript). A word + is only replaced by a heard word that strips down to it, and the timing + stays LRCLIB's. + + Called by transcribe_lyrics once the setting allows a transcription, and + shares its contract: never raises but JobCancelled.""" + entry = read_lyrics(job_dir) + if entry is None or entry["source"] != "lrclib" or "repaired" in entry: + return False + if not looks_stripped(entry["synced"] or entry["plain"]): + return False + reason = _vocals_reason(job, job_dir) + if reason is not None: + logger.info("[%s] lyrics mending skipped: %s", job.id, reason) + return False + started = time.monotonic() + _set(job, stage=_STAGE) + vocals = job_dir / "stems" / "vocals.wav" + result = _run_worker(job, _spawn_worker_cmd(vocals, _device(job)) + _language_gate()) + if result is None: + return False + if not result.get("skipped"): + # The same words time the lyrics line by line (lyrics_retime.py) + # without a second pass. + save_transcript(job_dir, result) + language = result.get("language") + probability = _number(result.get("language_probability")) or 0.0 + if ( + result.get("skipped") + or language not in LYRICS_REPAIR_LANGUAGES + or probability < LYRICS_REPAIR_MIN_LANGUAGE_PROB + ): + logger.info( + "[%s] lyrics left as they are: sung in %s (%.2f)", job.id, language, probability + ) + return False + mended = mend_from_transcript(entry, heard_words(result)) + if mended is None: + logger.info("[%s] lyrics left as they are: the transcription does not bear it out", job.id) + return False + # Nothing else writes lyrics.json while the pipeline runs; checked all the + # same, so a mend never lands on lyrics it was not made from. + if read_lyrics(job_dir) != entry: + return False + if not write_lyrics(job, job_dir, mended): + return False + logger.info( + "[%s] lyrics mended from the vocals in %.1fs: language=%s (%.2f) model=%s", + job.id, + time.monotonic() - started, + language, + probability, + result.get("model"), + ) + return True diff --git a/app/pipeline/transcribe_worker.py b/app/pipeline/transcribe_worker.py new file mode 100644 index 00000000..3e079fb2 --- /dev/null +++ b/app/pipeline/transcribe_worker.py @@ -0,0 +1,291 @@ +"""Isolated Whisper worker: a vocals stem in, timed words out. + +Run as ``python -m app.pipeline.transcribe_worker --audio ...``. +A fresh process per job, like the section worker, rather than a persistent +one like Demucs: transcription runs once per import at most, and only when no +lyrics were found, so a resident model would hold VRAM for nothing. The +process exiting is what gives every byte of GPU memory back, which also keeps +it clear of the Demucs worker's own allocation on the same card. + +Output contract: exactly one compact JSON line on stdout, +``{"language", "language_probability", "model", "device", "segments", +"peak_vram_mb"}`` (and ``"skipped": "language"``, with no segments, when +``--languages`` did not include the one detected), each segment ``{"start", "end", "text", "words"}`` and each +word ``{"start", "end", "word", "probability"}``. Everything else, including +third-party chatter and Whisper's own progress bars, goes to stderr, where +the parent reads ``@@PHASE@@`` lines and percentages for the stage text. +""" + +from __future__ import annotations + +import argparse +import contextlib +import json +import math +import os +import sys +from pathlib import Path + +from app.core.process import arm_parent_watchdog + +# Whisper works on 16 kHz mono and decides a language from 30 s of it. +_SAMPLE_RATE = 16000 +_WINDOW_SECONDS = 30 + + +def _parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(add_help=False) + parser.add_argument("--audio", type=Path, required=True) + parser.add_argument("--device", choices=("cuda", "cpu"), required=True) + parser.add_argument("--model", required=True) + parser.add_argument("--fallback-model", required=True) + parser.add_argument("--download-root", type=Path, required=True) + parser.add_argument("--min-free-mb", type=int, required=True) + parser.add_argument("--small-min-free-mb", type=int, required=True) + parser.add_argument("--silence-sec", type=float, required=True) + # Transcribe only a song in one of these languages (comma separated), + # detected with at least this probability; anything else answers with no + # segments and "skipped": "language". Used to mend lyrics that lost their + # accents (transcribe.py), which only a language written with them can. + parser.add_argument("--languages", default="") + parser.add_argument("--min-language-probability", type=float, default=0.0) + # Known lyrics being timed (lyrics_retime.py): the language they are + # in, which skips detection. + parser.add_argument("--language", default="") + # Only these stretches of the stem ("start:end,start:end", seconds), each + # heard on its own: the lines a first pass did not find, heard again + # without the rest of the song around them. Times stay the stem's. + parser.add_argument("--clips", default="") + return parser + + +# At most this many clips, each at most this long: a clip is a few lines. +_MAX_CLIPS = 32 +_MAX_CLIP_SEC = 120.0 + + +def parse_clips(text: str) -> list[tuple[float, float]]: + """The --clips argument as (start, end) seconds, in order, each within + _MAX_CLIP_SEC; malformed pieces are left out.""" + clips: list[tuple[float, float]] = [] + for piece in text.split(",")[:_MAX_CLIPS]: + try: + a, b = (float(x) for x in piece.split(":")) + except ValueError: + continue + if math.isfinite(a) and math.isfinite(b) and 0 <= a < b: + clips.append((a, min(b, a + _MAX_CLIP_SEC))) + return sorted(clips) + + +def _phase(name: str) -> None: + print(f"@@PHASE@@{name}", file=sys.stderr, flush=True) + + +def load_audio(path: Path): + """The stem as 16 kHz mono float32. Read with soundfile and resampled + here rather than through whisper.load_audio, which shells out to whatever + ``ffmpeg`` is first on PATH instead of the binary StemDeck verified.""" + import numpy as np + import soundfile as sf + from scipy.signal import resample_poly + + data, rate = sf.read(str(path), dtype="float32", always_2d=True) + mono = data.mean(axis=1) + if rate != _SAMPLE_RATE: + g = math.gcd(int(rate), _SAMPLE_RATE) + mono = resample_poly(mono, _SAMPLE_RATE // g, int(rate) // g) + return np.ascontiguousarray(mono, dtype=np.float32) + + +def loudest_window(audio, seconds: int = _WINDOW_SECONDS) -> tuple[int, int]: + """Sample bounds of the ``seconds`` long stretch with the most energy. + + Whisper decides the language from the first 30 s it is given, which for + a vocals stem is often an intro with nobody singing, and a language + guessed from silence is a coin toss. The stretch where the singer is + loudest is where the language is actually audible. + """ + import numpy as np + + size = seconds * _SAMPLE_RATE + if len(audio) <= size: + return 0, len(audio) + hop = _SAMPLE_RATE // 2 + frames = len(audio) // hop + energy = np.square(audio[: frames * hop].astype(np.float64)).reshape(frames, hop).sum(axis=1) + span = size // hop + cumulative = np.concatenate(([0.0], np.cumsum(energy))) + sums = cumulative[span:] - cumulative[:-span] + start = int(np.argmax(sums)) * hop + return start, start + size + + +def pick_model(args: argparse.Namespace, free_mb: int | None) -> tuple[str, str]: + """(device, model) given the free VRAM, None meaning not on a GPU.""" + if args.device != "cuda" or free_mb is None: + return "cpu", args.fallback_model + if free_mb >= args.min_free_mb: + return "cuda", args.model + if free_mb >= args.small_min_free_mb: + return "cuda", args.fallback_model + return "cpu", args.fallback_model + + +def _free_vram_mb(torch) -> int | None: + try: + free, _total = torch.cuda.mem_get_info() + except Exception: + return None + return int(free // (1024 * 1024)) + + +def _model_file(whisper, name: str, root: Path) -> Path | None: + url = getattr(whisper, "_MODELS", {}).get(name) + return root / os.path.basename(url) if url else None + + +def load_model(whisper, torch, name: str, device: str, root: Path): + """The model on ``device``, in fp16 on a GPU. + + The checkpoint is stored in fp16 and on a GPU inference runs in fp16, but + whisper.load_model widens the weights to fp32 first: turbo then holds + 3.2 GB of VRAM instead of 1.6. Measured on an RTX 3080 with Nirvana's + "Lithium" (255 s), the peak reserved fell from 5254 MB to 2172 MB with + the same words. Whisper's LayerNorm computes in fp32, so it keeps its + fp32 weights. + """ + model = whisper.load_model(name, device="cpu", download_root=str(root)) + if device != "cuda": + return model + model.half() + for module in model.modules(): + if isinstance(module, torch.nn.LayerNorm): + module.float() + return model.to(device) + + +def transcribe(args: argparse.Namespace) -> dict: + import torch + import whisper + + audio = load_audio(args.audio) + free_mb = _free_vram_mb(torch) if args.device == "cuda" and torch.cuda.is_available() else None + device, model_name = pick_model(args, free_mb) + if args.device == "cuda" and device != "cuda": + print(f"not enough free VRAM ({free_mb} MB), transcribing on the CPU", file=sys.stderr) + + model_file = _model_file(whisper, model_name, args.download_root) + if model_file is not None and not model_file.is_file(): + _phase("download") + else: + _phase("load") + model = load_model(whisper, torch, model_name, device, args.download_root) + + forced = args.language if model.is_multilingual else "" + if forced: + language, probs = forced, {forced: 1.0} + elif model.is_multilingual: + start, end = loudest_window(audio) + mel = whisper.log_mel_spectrogram( + whisper.pad_or_trim(audio[start:end]), n_mels=model.dims.n_mels + ).to(model.device) + if device == "cuda": + mel = mel.half() + _tokens, probs = model.detect_language(mel) + language = max(probs, key=probs.get) + else: # an English-only model set through the environment + language, probs = "en", {"en": 1.0} + + wanted = {code for code in args.languages.split(",") if code} + if wanted and ( + language not in wanted or float(probs[language]) < args.min_language_probability + ): + return { + "language": language, + "language_probability": round(float(probs[language]), 4), + "model": model_name, + "device": device, + "segments": [], + "skipped": "language", + "peak_vram_mb": None, + } + + _phase("transcribe") + + def hear(samples): + return model.transcribe( + samples, + language=language, + word_timestamps=True, + fp16=device == "cuda", + # Progress bars (to stderr), but not every decoded line. + verbose=False, + # Carrying the previous window's text forward is what makes + # Whisper repeat a line forever over a long instrumental; lyrics + # lose little without it. + condition_on_previous_text=False, + hallucination_silence_threshold=args.silence_sec, + ) + + clips = parse_clips(args.clips) + heard: list[tuple[float, dict]] = [] + if clips: + for a, b in clips: + piece = audio[int(a * _SAMPLE_RATE) : int(b * _SAMPLE_RATE)] + if piece.size >= _SAMPLE_RATE // 2: + heard.extend((a, seg) for seg in hear(piece).get("segments", [])) + else: + heard = [(0.0, seg) for seg in hear(audio).get("segments", [])] + peak = None + if device == "cuda": + peak = int(torch.cuda.max_memory_allocated() // (1024 * 1024)) + segments = [] + for offset, seg in heard: + words = [ + { + "start": float(w["start"]) + offset, + "end": float(w["end"]) + offset, + "word": str(w["word"]), + "probability": float(w.get("probability", 0.0)), + } + for w in seg.get("words") or [] + ] + segments.append( + { + "start": float(seg["start"]) + offset, + "end": float(seg["end"]) + offset, + "text": str(seg.get("text", "")), + "words": words, + "no_speech_prob": float(seg.get("no_speech_prob", 0.0)), + "avg_logprob": float(seg.get("avg_logprob", 0.0)), + "compression_ratio": float(seg.get("compression_ratio", 0.0)), + } + ) + return { + "language": language, + "language_probability": round(float(probs[language]), 4), + "model": model_name, + "device": device, + "segments": segments, + "peak_vram_mb": peak, + } + + +def main(argv: list[str] | None = None) -> int: + # A GPU pass that outlives its parent holds VRAM with nobody to collect + # the answer (#519). + arm_parent_watchdog() + args = _parser().parse_args(argv) + if not args.audio.is_file(): + raise FileNotFoundError("vocals stem is missing") + # Third-party diagnostics stay off stdout: the parent accepts exactly one + # JSON line there, so nothing else can be mistaken for the transcript. + with contextlib.redirect_stdout(sys.stderr): + out = transcribe(args) + print(json.dumps(out, separators=(",", ":"), ensure_ascii=False), flush=True) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/app/pipeline/warmup.py b/app/pipeline/warmup.py index 9bbcaecd..ac8145bb 100644 --- a/app/pipeline/warmup.py +++ b/app/pipeline/warmup.py @@ -1,9 +1,9 @@ """Eager model pre-download for the desktop first-boot setup wizard (#275). -Run as `python -m app.pipeline.warmup`. Downloads/caches the four ML +Run as `python -m app.pipeline.warmup`. Downloads/caches the ML checkpoint families StemDeck uses: Demucs (htdemucs_6s), beat-this, -All-In-One song sections, and the on-demand lead/backing vocal-split karaoke -model. This keeps a user's first real job from paying for them mid-pipeline. Invoked by the Tauri +All-In-One song sections, the on-demand lead/backing vocal-split karaoke +model, and Whisper for lyrics transcription when that would run. This keeps a user's first real job from paying for them mid-pipeline. Invoked by the Tauri `warmup_models` command (desktop/src-tauri/src/main.rs) as one of the setup steps; Docker has no equivalent step and keeps the pre-existing lazy-download-on-first-use behavior (see docs/models.md). @@ -39,6 +39,8 @@ DEMUCS_MODEL, MODELS_DIR, SECTION_MODEL, + TRANSCRIBE_MODEL_CPU, + TRANSCRIBE_MODEL_GPU, VOCAL_SPLIT_MODEL, ) @@ -88,11 +90,33 @@ def _warm_sections() -> None: load_pretrained_model(model_name=SECTION_MODEL, device="cpu") +def _warm_whisper() -> None: + """The Whisper model the lyrics stage would use here, and only when that + stage would run: under "auto" only a CUDA install transcribes, and the + GPU model is 1.6 GB. Last in the list for the same reason, so a slow + connection costs the other models nothing. A download cut short by the + setup timeout heals itself: Whisper checks the file's SHA-256 on every + load and fetches it again when it does not match.""" + from app.core.settings import get_demucs_device, transcribe_lyrics_enabled + + device = get_demucs_device() + if not transcribe_lyrics_enabled(device): + return + + import whisper + + from app.pipeline.transcribe import whisper_models_dir + + name = TRANSCRIBE_MODEL_GPU if device == "cuda" else TRANSCRIBE_MODEL_CPU + whisper.load_model(name, device="cpu", download_root=str(whisper_models_dir())) + + _STEPS = ( ("demucs", _warm_demucs), ("beat_this", _warm_beat_this), ("sections", _warm_sections), ("vocal_split", _warm_vocal_split), + ("whisper", _warm_whisper), ) diff --git a/app/pipeline/work_lookup.py b/app/pipeline/work_lookup.py new file mode 100644 index 00000000..afe35582 --- /dev/null +++ b/app/pipeline/work_lookup.py @@ -0,0 +1,582 @@ +"""The work a soundtrack or cast recording is from: the musical, film or series. + +A song from a musical has no band to show. "Popular" is credited to Kristin +Chenoweth, but what someone opening it wants to read about is Wicked. So when +the recording a job is identified as (identify.py) came out on a soundtrack, or +its album says it is one ("Wicked (Original Broadway Cast Recording)", "The +Wizard of Oz (Original Motion Picture Soundtrack)"), the work is looked for on +Wikidata, best source first: + +1. The release group's own links on MusicBrainz. A soundtrack's Wikidata link + is sometimes the film itself (The Wizard of Oz links both the album and the + 1939 film). More often it is the album, and the work is the item that names + that album as its soundtrack release (P406). An IMDb link names the film. +2. A Wikidata search for the album's name with the qualifier taken off + ("Wicked", "The Wizard of Oz"), kept only when the name is the hit's label + or alias exactly, once case, accents and punctuation are set aside. When + no album says so, the show the track's own title names ('Dancing Through + Life (From "Wicked" Original Broadway Cast Recording/2003)'), which also + gives the kind and the year when the album does not (title_hint). + +Either way, an item is kept only when it is a kind of work a song can be from: +a musical, a film, a television series or a stage work, directly or through +Wikidata's "subclass of" (WORK_CLASS_DEPTH steps up). The novel, the video game +and the album that share the musical's name are not. Several that are (the +2003 musical and the 2024 film are both "Wicked") are told apart by what the +album says it is (a "Broadway Cast Recording" is the musical) and by year: a +work first shown well after the recording came out cannot be what it is from. + +What is sent: a release group's MBID to MusicBrainz, and to Wikidata the +album's name or ids found on MusicBrainz. Everything here is best-effort: +find_work never raises, and no answer leaves the job without a work. +""" + +from __future__ import annotations + +import logging +import re +import threading +from collections.abc import Callable +from typing import Any + +from app.core.config import WORK_CLASS_DEPTH, WORK_YEAR_SLACK +from app.core.models import clean_work +from app.pipeline import artist_lookup, musicbrainz +from app.pipeline.artist_lookup import FetchJson, artist_name_key, search_language +from app.pipeline.title_parse import TitleReading, work_hint + +logger = logging.getLogger("stemdeck.work") + +INSTANCE_OF = "P31" +SUBCLASS_OF = "P279" +SOUNDTRACK_RELEASE = "P406" +IMDB_ID = "P345" +PUBLICATION_DATE = "P577" +FIRST_PERFORMANCE = "P1191" +START_TIME = "P580" + +# The classes a work a song comes from is an instance of, and the kind each +# one makes it. The ones met most often are listed so that no walk up the +# class tree is needed for them; anything else is walked up from. +WORK_CLASSES: dict[str, str] = { + "Q11424": "film", + "Q24869": "film", # feature film + "Q202866": "film", # animated film + "Q842256": "film", # musical film + "Q24862": "film", # short film + "Q506240": "film", # television film + "Q24856": "film", # film series + "Q5398426": "tv", # television series + "Q15416": "tv", # television program + "Q7724161": "tv", # television serial + "Q1259759": "tv", # miniseries + "Q581714": "tv", # animated series + "Q117467246": "tv", # animated television series + "Q63952888": "tv", # anime television series + "Q2743": "musical", # musical play + "Q58483083": "musical", # dramatico-musical work + "Q7777570": "other", # theatrical production + "Q25379": "other", # play + "Q1344": "other", # opera +} +# When an item reaches more than one kind: a musical film is a film. +_KIND_ORDER = ("film", "tv", "musical", "other") +# What shares a work's name and is never it. Checked on the item's own +# classes, so an item typed as one of these is not walked up from at all. +NOT_WORKS = frozenset( + { + "Q7889", # video game + "Q7725634", # literary work + "Q8261", # novel + "Q482994", # album + "Q4176708", # soundtrack album + "Q134556", # single + "Q7366", # song + } +) + +_QID_RE = re.compile(r"^Q\d{1,12}$") +_IMDB_RE = re.compile(r"^(tt\d{7,10})/?$") +_IMDB_URL_RE = re.compile(r"^https?://(?:www\.)?imdb\.com/title/(tt\d{7,10})/?$") +_WIKIDATA_URL_RE = re.compile(r"^https?://(?:www\.)?wikidata\.org/wiki/(Q\d{1,12})$") +_YEAR_RE = re.compile(r"^[+]?(\d{4})") +# Links followed from one release group, at most. A soundtrack links one +# album item and sometimes the film; more is a group linking every reissue. +_MAX_LINKS = 3 +_SEARCH_LIMIT = "10" + +# What an album's name says when it is a soundtrack or a cast recording. +_WORK_WORDS_RE = re.compile( + r"soundtrack|\bost\b|\bscore\b|\bcast\b|motion picture|\bmusical\b|broadway|west end" + r"|\bfilm\b|\bmovie\b|television|\btv\b|\bseries\b", + re.IGNORECASE, +) +_BRACKETED_RE = re.compile(r"\s*[(\[][^)\]]*[)\]]") +# "Wicked: The Soundtrack", "Les Misérables - Original London Cast": the part +# after the colon or dash, when it is the qualifier. Hyphen, en dash, em dash. +_DASHES = "-" + chr(0x2013) + chr(0x2014) +_TAIL_SPLIT_RE = re.compile(rf"\s*:\s+|\s+[{_DASHES}]\s+") +# "Music from the Motion Picture Grease", "Original Soundtrack from Frozen". +_LEADING_RE = re.compile( + r"^(?:(?:the\s+)?original\s+)?(?:music|songs|soundtrack|score)\s+(?:from|of|to)\s+" + r"(?:and\s+inspired\s+by\s+)?(?:the\s+)?(?:original\s+)?" + r"(?:broadway\s+musical|motion\s+picture|film|movie|musical|television\s+series|series)\s+", + re.IGNORECASE, +) +# "Frozen Original Soundtrack", "Frozen OST". +_TRAILING_RE = re.compile( + r"\s+(?:(?:original\s+)?(?:motion\s+picture\s+)?(?:soundtrack|score)|ost|o\.s\.t\.)$", + re.IGNORECASE, +) +_HINTS = ( + ("musical", re.compile(r"broadway|west end|\bcast\b|\bmusical\b|\bstage\b", re.IGNORECASE)), + ("film", re.compile(r"motion picture|\bfilm\b|\bmovie\b", re.IGNORECASE)), + ("tv", re.compile(r"television|\btv\b|\bseries\b", re.IGNORECASE)), + # Last: a stage show's album is a cast recording, so a plain "soundtrack" + # that names no cast, series or show ("Wicked: The Soundtrack") is a film's. + ("film", re.compile(r"soundtrack|\bost\b|\bscore\b", re.IGNORECASE)), +) + +# Each class's "subclass of", as Wikidata answered it, for the life of the +# process: classes are few and do not change between two imports. +_CLASS_PARENTS: dict[str, list[str]] = {} +_CLASS_LOCK = threading.Lock() + + +# ── what the album says ── + + +def strip_album(album: Any) -> str: + """An album's name with the soundtrack or cast recording qualifier taken + off: "Wicked (Original Broadway Cast Recording)" is "Wicked". A name with + no qualifier comes back as it was.""" + text = str(album or "").strip() + text = _BRACKETED_RE.sub( + lambda m: "" if _WORK_WORDS_RE.search(m.group()) else m.group(), text + ).strip() + parts = _TAIL_SPLIT_RE.split(text, maxsplit=1) + if len(parts) == 2 and parts[0].strip() and _WORK_WORDS_RE.search(parts[1]): + text = parts[0] + text = _LEADING_RE.sub("", text) + text = _TRAILING_RE.sub("", text) + return text.strip(" \t:" + _DASHES) + + +def kind_hint(album: Any) -> str | None: + """The kind of work an album's name says it is from, or None.""" + text = str(album or "") + for kind, pattern in _HINTS: + if pattern.search(text): + return kind + return None + + +def _is_soundtrack(identity: dict[str, Any] | None) -> bool: + types = (identity or {}).get("secondary_types") or [] + return any(isinstance(t, str) and t.lower() == "soundtrack" for t in types) + + +def title_hint(identity: dict[str, Any] | None, title: Any) -> TitleReading | None: + """The work the track's title names ('Dancing Through Life (From "Wicked" + Original Broadway Cast Recording/2003)', "The Greatest Showman Cast - This + Is Me"), read the way that makes the identified song the song. None when + it names none, or when MusicBrainz placed the recording on a release that + is no soundtrack (a studio album): then a work in the title is noise.""" + identity = identity if isinstance(identity, dict) else {} + if ( + identity.get("source") in ("acoustid", "musicbrainz") + and identity.get("release_group_mbid") + and not identity.get("secondary_types") + ): + return None + return work_hint(title, identity.get("title")) + + +def work_query( + identity: dict[str, Any] | None, tags: dict[str, str] | None, title: Any = None +) -> tuple[str, str | None, bool] | None: + """(name to look for, kind hint, whether the release group is the + soundtrack) for a job, or None when nothing says it is from a work. + + The release group being a soundtrack says so, and so does an album name + carrying a qualifier: the identity's (the release group's title) or the + file's own album tag, which may name the soundtrack when the recording + was identified as a compilation it is also on. Last, the track's title + naming the show (title_hint), which is all a YouTube upload of a cast + recording has; it also says the kind when the album does not.""" + identity = identity if isinstance(identity, dict) else {} + tags = tags if isinstance(tags, dict) else {} + albums = [ + a.strip() + for a in (identity.get("album"), tags.get("album")) + if isinstance(a, str) and a.strip() + ] + from_title = title_hint(identity, title) if title else None + hint = next((h for h in (kind_hint(a) for a in albums) if h), None) or ( + from_title.kind if from_title else None + ) + if _is_soundtrack(identity) and identity.get("album"): + name = strip_album(identity["album"]) + return (name, hint, True) if name else None + for album in albums: + name = strip_album(album) + if name and name != album: + return name, hint, False + if from_title: + return from_title.work, hint, False + return None + + +def might_have_work( + identity: dict[str, Any] | None, tags: dict[str, str] | None, title: Any = None +) -> bool: + """Whether find_work has anything to go on.""" + return work_query(identity, tags, title) is not None + + +# ── Wikidata ── + + +def _claim_ids(entity: dict[str, Any], prop: str) -> list[str]: + """The item ids of an entity's claims for ``prop``, deprecated ones left out.""" + claims = entity.get("claims") + statements = claims.get(prop) if isinstance(claims, dict) else None + ids = [] + for statement in statements if isinstance(statements, list) else []: + if not isinstance(statement, dict) or statement.get("rank") == "deprecated": + continue + snak = statement.get("mainsnak") + datavalue = snak.get("datavalue") if isinstance(snak, dict) else None + value = datavalue.get("value") if isinstance(datavalue, dict) else None + qid = value.get("id") if isinstance(value, dict) else None + if isinstance(qid, str) and _QID_RE.match(qid) and qid not in ids: + ids.append(qid) + return ids + + +def work_year(entity: dict[str, Any]) -> int | None: + """The year a work first came out: published, first performed or first + aired, whichever is earliest. None when Wikidata has none of them.""" + years = [] + claims = entity.get("claims") if isinstance(entity.get("claims"), dict) else {} + for prop in (PUBLICATION_DATE, FIRST_PERFORMANCE, START_TIME): + for statement in claims.get(prop) or []: + snak = statement.get("mainsnak") if isinstance(statement, dict) else None + datavalue = snak.get("datavalue") if isinstance(snak, dict) else None + value = datavalue.get("value") if isinstance(datavalue, dict) else None + time = value.get("time") if isinstance(value, dict) else None + match = _YEAR_RE.match(time) if isinstance(time, str) else None + if match: + years.append(int(match.group(1))) + return min(years) if years else None + + +def _entities(ids: list[str], lang: str, fetch_json: FetchJson) -> dict[str, Any]: + ids = [i for i in dict.fromkeys(ids) if _QID_RE.match(i)][:50] + if not ids: + return {} + answer = fetch_json( + { + "action": "wbgetentities", + "ids": "|".join(ids), + "props": "claims|labels", + "languages": "en" if lang == "en" else f"{lang}|en", + "languagefallback": "1", + } + ) + entities = answer.get("entities") if isinstance(answer, dict) else None + return entities if isinstance(entities, dict) else {} + + +def _class_parents(ids: list[str], fetch_json: FetchJson) -> dict[str, list[str]]: + """Each class's "subclass of", from the cache or one request for all the + classes it has not seen.""" + with _CLASS_LOCK: + missing = [i for i in ids if i not in _CLASS_PARENTS] + if missing: + answer = fetch_json( + {"action": "wbgetentities", "ids": "|".join(missing[:50]), "props": "claims"} + ) + entities = answer.get("entities") if isinstance(answer, dict) else None + found = { + qid: _claim_ids(entity, SUBCLASS_OF) + for qid, entity in (entities if isinstance(entities, dict) else {}).items() + if isinstance(entity, dict) + } + with _CLASS_LOCK: + for qid in missing[:50]: + _CLASS_PARENTS[qid] = found.get(qid, []) + with _CLASS_LOCK: + return {i: _CLASS_PARENTS.get(i, []) for i in ids} + + +def _best_kind(classes: list[str]) -> str | None: + kinds = {WORK_CLASSES[c] for c in classes if c in WORK_CLASSES} + return next((k for k in _KIND_ORDER if k in kinds), None) + + +def work_kind( + entity: dict[str, Any], + fetch_json: FetchJson, + cancelled: Callable[[], bool] = lambda: False, +) -> str | None: + """The kind of work ``entity`` is ("musical", "film", "tv", "other"), or + None when it is not one a song can be from. + + Its own classes first. Then, unless one of them is a thing that is never a + work (NOT_WORKS), up through "subclass of", one request per step, up to + WORK_CLASS_DEPTH steps; the nearest step with a known class decides.""" + classes = _claim_ids(entity, INSTANCE_OF) + kind = _best_kind(classes) + if kind or not classes or NOT_WORKS.intersection(classes): + return kind + seen = set(classes) + frontier = classes + for _ in range(WORK_CLASS_DEPTH): + if cancelled() or not frontier: + return None + parents = _class_parents(frontier[:50], fetch_json) + step = [p for ps in parents.values() for p in ps if p not in seen] + kind = _best_kind(step) + if kind: + return kind + seen.update(step) + frontier = list(dict.fromkeys(step)) + return None + + +def _label(entity: dict[str, Any], *codes: str) -> str: + labels = entity.get("labels") + for code in codes: + label = labels.get(code) if isinstance(labels, dict) else None + if isinstance(label, dict) and isinstance(label.get("value"), str): + return label["value"] + return "" + + +def _work(entity: dict[str, Any], kind: str, lang: str, name: str = "") -> dict[str, str] | None: + return clean_work( + { + "id": entity.get("id"), + "kind": kind, + "name": _label(entity, lang, "en") or name, + "englishName": _label(entity, "en"), + } + ) + + +def _items_with(prop: str, value: str, fetch_json: FetchJson) -> list[str]: + """The items that have a ``prop`` statement of ``value``, by Wikidata's + search. ``value`` is a checked item id or IMDb id, never free text.""" + answer = fetch_json( + { + "action": "query", + "list": "search", + "srsearch": f"haswbstatement:{prop}={value}", + "srnamespace": "0", + "srlimit": "5", + } + ) + query = answer.get("query") if isinstance(answer, dict) else None + hits = query.get("search") if isinstance(query, dict) else None + return [ + hit["title"] + for hit in (hits if isinstance(hits, list) else []) + if isinstance(hit, dict) + and isinstance(hit.get("title"), str) + and _QID_RE.match(hit["title"]) + ] + + +def release_group_links(group: Any) -> tuple[list[str], list[str]]: + """(Wikidata item ids, IMDb title ids) a MusicBrainz release group links.""" + qids: list[str] = [] + imdb: list[str] = [] + relations = group.get("relations") if isinstance(group, dict) else None + for relation in relations if isinstance(relations, list) else []: + url = relation.get("url") if isinstance(relation, dict) else None + resource = url.get("resource") if isinstance(url, dict) else None + if not isinstance(resource, str): + continue + wikidata = _WIKIDATA_URL_RE.match(resource) + film = _IMDB_URL_RE.match(resource) + if wikidata and wikidata.group(1) not in qids: + qids.append(wikidata.group(1)) + elif film and film.group(1) not in imdb: + imdb.append(film.group(1)) + return qids[:_MAX_LINKS], imdb[:_MAX_LINKS] + + +def _first_work( + ids: list[str], lang: str, fetch_json: FetchJson, cancelled: Callable[[], bool] +) -> dict[str, str] | None: + """The first of ``ids``, in order, that is a work: one typed as a known + kind of work outright first, so no class tree is walked when one is.""" + if not ids or cancelled(): + return None + entities = _entities(ids, lang, fetch_json) + found = [entities[qid] for qid in ids if isinstance(entities.get(qid), dict)] + for entity in found: + kind = _best_kind(_claim_ids(entity, INSTANCE_OF)) + if kind: + return _work(entity, kind, lang) + for entity in found: + if cancelled(): + return None + kind = work_kind(entity, fetch_json, cancelled) + if kind: + return _work(entity, kind, lang) + return None + + +def work_from_links( + qids: list[str], + imdb: list[str], + lang: str, + fetch_json: FetchJson, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """The work a release group's links name: a linked item that is one; else + the item naming a linked album as its soundtrack release; else the item + with a linked IMDb id.""" + work = _first_work(qids, lang, fetch_json, cancelled) + if work: + return work + found: list[str] = [] + for qid in qids: + if cancelled(): + return None + found += [i for i in _items_with(SOUNDTRACK_RELEASE, qid, fetch_json) if i not in qids] + for title in imdb: + if cancelled(): + return None + if _IMDB_RE.match(title): + found += _items_with(IMDB_ID, title, fetch_json) + return _first_work(list(dict.fromkeys(found)), lang, fetch_json, cancelled) + + +def search_work( + name: str, + hint: str | None, + year: int | None, + fetch_json: FetchJson, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """The work called ``name`` on Wikidata, or None. + + Only hits whose label or alias is ``name`` exactly (artist_name_key) are + considered, and only those that are works (work_kind). Of those, a work + of the kind ``hint`` says beats one that is not, and a work first shown + more than WORK_YEAR_SLACK years after ``year`` is left out; otherwise the + search's own order, which puts the famous one first.""" + want = artist_name_key(name) + if not want or cancelled(): + return None + lang = search_language(name) + found = fetch_json( + { + "action": "wbsearchentities", + "search": name, + "language": lang, + "uselang": lang, + "type": "item", + "limit": _SEARCH_LIMIT, + } + ) + hits = found.get("search") if isinstance(found, dict) else None + ids = [] + for hit in hits if isinstance(hits, list) else []: + if not isinstance(hit, dict) or not _QID_RE.match(str(hit.get("id") or "")): + continue + match = hit.get("match") if isinstance(hit.get("match"), dict) else {} + if want in (artist_name_key(hit.get("label")), artist_name_key(match.get("text"))): + ids.append(hit["id"]) + if not ids or cancelled(): + return None + entities = _entities(ids, lang, fetch_json) + candidates = [] + for rank, qid in enumerate(ids): + entity = entities.get(qid) + if not isinstance(entity, dict) or cancelled(): + continue + born = work_year(entity) + if year and born and born > year + WORK_YEAR_SLACK: + continue + kind = work_kind(entity, fetch_json, cancelled) + if kind: + candidates.append(((bool(hint) and kind != hint, rank), entity, kind)) + if not candidates: + return None + _, entity, kind = min(candidates, key=lambda c: c[0]) + return _work(entity, kind, lang, name) + + +def _release_year(group: Any) -> int | None: + date = group.get("first-release-date") if isinstance(group, dict) else None + match = _YEAR_RE.match(date) if isinstance(date, str) else None + return int(match.group(1)) if match else None + + +def lookup_work( + identity: dict[str, Any] | None, + tags: dict[str, str] | None, + *, + title: Any = None, + fetch_json: FetchJson | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """The work a job's song is from (see the module), or None. Raises when + Wikidata cannot be reached; a MusicBrainz failure only skips its links. + + ``title``, the track's own, can name the show and its kind and year + (title_hint). When the release group's links name a work of another kind + than that (a film where the title says Broadway cast), the name search is + asked too, and a work of the kind the title says wins.""" + query = work_query(identity, tags, title) + if query is None or cancelled(): + return None + name, hint, is_soundtrack = query + from_title = title_hint(identity, title) if title else None + # Resolved here, so a test can stand in for the network by replacing + # artist_lookup._fetch_json, as conftest does for every test. + fetch_json = fetch_json or artist_lookup._fetch_json + lang = search_language(name) + year = None + linked = None + mbid = (identity or {}).get("release_group_mbid") + if is_soundtrack and mbid: + try: + group = musicbrainz.lookup_release_group(mbid, cancelled=cancelled) + except Exception: + logger.info("release group lookup failed", exc_info=True) + group = None + if cancelled(): + return None + year = _release_year(group) + qids, imdb = release_group_links(group) + linked = work_from_links(qids, imdb, lang, fetch_json, cancelled) + if linked and not (hint and linked["kind"] != hint): + return linked + if cancelled(): + return None + year = year or (from_title.year if from_title else None) + found = search_work(name, hint, year, fetch_json, cancelled) + if found and (linked is None or found["kind"] == hint): + return found + return linked or found + + +def find_work( + identity: dict[str, Any] | None, + tags: dict[str, str] | None, + *, + title: Any = None, + fetch_json: FetchJson | None = None, + cancelled: Callable[[], bool] = lambda: False, +) -> dict[str, str] | None: + """lookup_work, never raising: any failure is logged and gives None.""" + try: + return lookup_work(identity, tags, title=title, fetch_json=fetch_json, cancelled=cancelled) + except Exception: + logger.info("work lookup failed", exc_info=True) + return None diff --git a/app/pipeline/zh_variants.py b/app/pipeline/zh_variants.py new file mode 100644 index 00000000..6cb60235 --- /dev/null +++ b/app/pipeline/zh_variants.py @@ -0,0 +1,50 @@ +"""Traditional and simplified Chinese as one, for comparing names. + +The same song or singer is written two ways: 鄧麗君 in Taiwan and Hong Kong, +邓丽君 on the mainland; 紅豆 and 红豆. LRCLIB holds both, and a track's tags +or MusicBrainz credit can be either, so names are compared with every +traditional character folded to its simplified form. Folding both sides only +makes variants of one character equal: no two names that differ in anything +else become one. + +The table is app/_vendor/opencc/zh_t2s.txt, derived from OpenCC (Apache-2.0: +the file carries the attribution, and the license is beside it). +static/js/zhVariants.js holds the same pairs for the page, and a test keeps +the two equal. +""" + +from __future__ import annotations + +from functools import lru_cache +from pathlib import Path + +_TABLE_PATH = Path(__file__).resolve().parent.parent / "_vendor" / "opencc" / "zh_t2s.txt" + + +@lru_cache(maxsize=1) +def _table() -> dict[int, str]: + """{ord(traditional): simplified}, for str.translate. Empty when the data + file cannot be read: names then compare as written, as they did before.""" + table: dict[int, str] = {} + try: + text = _TABLE_PATH.read_text(encoding="utf-8") + except OSError: + return table + for line in text.splitlines(): + if line.startswith("#"): + continue + pair = line.split(" ") + if len(pair) == 2 and len(pair[0]) == 1 and len(pair[1]) == 1: + table[ord(pair[0])] = pair[1] + return table + + +def pairs() -> dict[str, str]: + """The table as {traditional: simplified}.""" + return {chr(k): v for k, v in _table().items()} + + +def to_simplified(text: str) -> str: + """``text`` with every traditional Chinese character folded to its + simplified form. Anything else is left as it is.""" + return text.translate(_table()) diff --git a/desktop/src-tauri/src/main.rs b/desktop/src-tauri/src/main.rs index b60c18fc..b1ad729c 100644 --- a/desktop/src-tauri/src/main.rs +++ b/desktop/src-tauri/src/main.rs @@ -1792,6 +1792,10 @@ fn start_backend( } apply_ffmpeg_path(&mut cmd, &data_dir)?; + // fpcalc for song fingerprinting, fetched in the background once the + // FFmpeg it stands in for is settled. Never waited for. + #[cfg(unix)] + spawn_fpcalc_setup(data_dir.clone()); #[cfg(windows)] { @@ -4790,6 +4794,193 @@ fn download_linux_ffmpeg(data_dir: &Path) -> Result<(), String> { Ok(()) } +// fpcalc, Chromaprint's own fingerprinter, for song identification where the +// FFmpeg in use has no chromaprint muxer: the shaka-project and evermeet macOS +// builds and johnvansickle's Linux build (app/pipeline/fpcalc.py). Windows gets +// none, BtbN's gpl build has the muxer. Optional: without it AcoustID +// fingerprinting stays off and tracks are identified by their tags. +// +// Pinned like the FFmpeg hashes above, since this binary is downloaded, marked +// executable and run. Each hash is the sha256 of the release tarball as +// downloaded from GitHub. To bump: change FPCALC_VERSION, download every asset +// fpcalc_asset() names from +// https://github.com/acoustid/chromaprint/releases/tag/v, run +// `sha256sum` on each, and replace all three hashes together. A stale pin fails +// closed: no fpcalc, never an unverified one. +#[cfg(any(unix, test))] +const FPCALC_VERSION: &str = "1.6.1"; +#[cfg(any(unix, test))] +const FPCALC_SHA256_LINUX_X86_64: &str = + "fc16cd37a70168040bc9ceb45f1d4d1216f5a75bc4c9cf8564bea70ac6a45733"; +#[cfg(any(unix, test))] +const FPCALC_SHA256_LINUX_ARM64: &str = + "7eaf5d655c4aa172ab28e3c870b8bb61dd2c327ac94de145676f88842cf6215a"; +#[cfg(any(unix, test))] +const FPCALC_SHA256_MACOS_UNIVERSAL: &str = + "240aeb5a8c8205af458e3625cb7487b826b711a999e491ef00111f3cebd76f00"; + +/// The release asset to fetch for this OS and CPU, with its pinned hash, or +/// None where none is fetched (Windows, or a platform with no build). +/// +/// Takes the OS and arch rather than reading `std::env::consts`, so every arm +/// is testable on any host. macOS takes the universal binary: one file that +/// runs natively on either CPU, so an Intel app under Rosetta and a native one +/// can never end up with a binary the other half cannot run (#637's shape). +#[cfg(any(unix, test))] +fn fpcalc_asset(os: &str, arch: &str) -> Option<(String, &'static str)> { + let (suffix, sha256) = match (os, arch) { + ("linux", "x86_64") => ("linux-x86_64", FPCALC_SHA256_LINUX_X86_64), + ("linux", "aarch64") => ("linux-arm64", FPCALC_SHA256_LINUX_ARM64), + ("macos", "x86_64" | "aarch64") => ("macos-universal", FPCALC_SHA256_MACOS_UNIVERSAL), + _ => return None, + }; + Some(( + format!("chromaprint-fpcalc-{FPCALC_VERSION}-{suffix}.tar.gz"), + sha256, + )) +} + +/// Whether an `ffmpeg -muxers` listing names chromaprint. Lines are +/// ` E chromaprint Chromaprint`: flags, then the name. +#[cfg(any(unix, test))] +fn lists_chromaprint_muxer(listing: &str) -> bool { + listing + .lines() + .any(|line| line.split_whitespace().nth(1) == Some("chromaprint")) +} + +/// Check `archive` against its pinned hash, then write the one file named +/// `fpcalc` in it to `target`. Nothing is extracted from an archive that does +/// not match, and only that one regular file is ever written, to `target` +/// whatever path the archive gives it. +#[cfg(any(unix, test))] +fn unpack_fpcalc(archive: &Path, expected_sha256: &str, target: &Path) -> Result<(), String> { + verify_pinned_sha256(archive, Some(expected_sha256), "fpcalc")?; + let file = fs::File::open(archive) + .map_err(|e| format!("failed to open {}: {e}", archive.display()))?; + let mut tarball = Archive::new(GzDecoder::new(file)); + let entries = tarball + .entries() + .map_err(|e| format!("failed to read the fpcalc archive: {e}"))?; + for entry in entries { + let mut entry = entry.map_err(|e| format!("failed to read the fpcalc archive: {e}"))?; + let is_fpcalc = entry.header().entry_type().is_file() + && entry + .path() + .is_ok_and(|path| path.file_name().is_some_and(|name| name == "fpcalc")); + if !is_fpcalc { + continue; + } + let mut output = fs::File::create(target) + .map_err(|e| format!("failed to create {}: {e}", target.display()))?; + std::io::copy(&mut entry, &mut output) + .map_err(|e| format!("failed to extract {}: {e}", target.display()))?; + return Ok(()); + } + Err("the fpcalc archive did not contain fpcalc".to_string()) +} + +#[cfg(unix)] +fn verify_fpcalc(path: &Path) -> Result<(), String> { + let mut command = Command::new(path); + command + .arg("-version") + .stdout(Stdio::null()) + .stderr(Stdio::null()); + let output = command_output_with_timeout(command, Duration::from_secs(15), "fpcalc check")?; + if output.status.success() { + Ok(()) + } else { + Err(format!( + "fpcalc at {} failed to run ({})", + path.display(), + output.status + )) + } +} + +#[cfg(unix)] +fn ffmpeg_has_chromaprint(ffmpeg: &Path) -> bool { + let mut command = Command::new(ffmpeg); + command + .args(["-hide_banner", "-muxers"]) + .stdout(Stdio::piped()) + .stderr(Stdio::null()); + command_output_with_timeout(command, Duration::from_secs(15), "FFmpeg muxer check") + .is_ok_and(|output| lists_chromaprint_muxer(&String::from_utf8_lossy(&output.stdout))) +} + +/// Put a working fpcalc in data/ffmpeg when song identification would need +/// one and nothing already provides it. Every failure is returned for the +/// caller to log; none is a reason to stop anything. +#[cfg(unix)] +fn ensure_fpcalc(data_dir: &Path) -> Result<(), String> { + // The user's own choice, which the backend honours; not ours to replace. + if env_path_override("STEMDECK_FPCALC").is_some() { + return Ok(()); + } + let ffmpeg_dir = data_dir.join("ffmpeg"); + let target = ffmpeg_dir.join("fpcalc"); + if target.is_file() && verify_fpcalc(&target).is_ok() { + return Ok(()); + } + // A system install (Homebrew's chromaprint, a distro's libchromaprint- + // tools), or an FFmpeg that needs no help. The backend sees the same PATH. + if verify_fpcalc(Path::new("fpcalc")).is_ok() { + return Ok(()); + } + if verified_ffmpeg_pair(data_dir).is_some_and(|(ffmpeg, _)| ffmpeg_has_chromaprint(&ffmpeg)) { + return Ok(()); + } + let (asset, sha256) = fpcalc_asset(env::consts::OS, env::consts::ARCH).ok_or_else(|| { + format!( + "no fpcalc build for {}/{}", + env::consts::OS, + env::consts::ARCH + ) + })?; + let url = format!( + "https://github.com/acoustid/chromaprint/releases/download/v{FPCALC_VERSION}/{asset}" + ); + fs::create_dir_all(&ffmpeg_dir) + .map_err(|e| format!("failed to create {}: {e}", ffmpeg_dir.display()))?; + // Beside FFmpeg rather than in downloads/, which prune_downloads may be + // emptying while this runs. + let archive = ffmpeg_dir.join("fpcalc.tar.gz"); + let staged = ffmpeg_dir.join("fpcalc.partial"); + let result = download_file(&url, &archive, Duration::from_secs(5 * 60), "fpcalc") + .and_then(|()| unpack_fpcalc(&archive, sha256, &staged)) + .and_then(|()| make_executable(&staged)) + .and_then(|()| verify_fpcalc(&staged)) + // Into place only once it runs: the backend looks for it on every + // fingerprint, and must never find a half-written one. + .and_then(|()| { + fs::rename(&staged, &target) + .map_err(|e| format!("failed to move fpcalc to {}: {e}", target.display())) + }); + let _ = fs::remove_file(&archive); + let _ = fs::remove_file(&staged); + result +} + +/// ensure_fpcalc, once per launch, on a thread of its own. It is a few +/// megabytes the app works without, so it never holds up setup or the window, +/// and an install set up before fpcalc existed gets it on its next launch. The +/// backend looks for fpcalc on every fingerprint, so one that lands after it +/// started is used from then on. +#[cfg(unix)] +fn spawn_fpcalc_setup(data_dir: PathBuf) { + static STARTED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); + if STARTED.swap(true, Ordering::SeqCst) { + return; + } + thread::spawn(move || { + if let Err(err) = ensure_fpcalc(&data_dir) { + eprintln!("[stemdeck] no fpcalc, song fingerprinting stays off: {err}"); + } + }); +} + // "arm64" on Apple Silicon, "x64" everywhere else -- takes the arch string // rather than reading std::env::consts::ARCH itself so both branches are // unit-testable on any host, not just the one they happen to be running on. @@ -4819,8 +5010,9 @@ fn override_sha256(env_var: &str) -> Option { // Unix rather than macOS: the Linux FFmpeg download calls this too (#518). // While the gate said macOS the Linux caller referred to a function that was // configured out, and nothing noticed because nothing compiles the Linux shell -// until a release builds it (#531). -#[cfg(unix)] +// until a release builds it (#531). Also under test on every host, for +// unpack_fpcalc's tests. +#[cfg(any(unix, test))] fn verify_pinned_sha256(path: &Path, expected: Option<&str>, label: &str) -> Result<(), String> { let Some(expected) = expected else { return Ok(()); @@ -7384,4 +7576,135 @@ b6052160df96b31c9b1e33854a4dcda3d4b57641b880270f31736fb9f445d384 ffmpeg-n7.1-la // No foreign responder seen: say nothing rather than guess at a cause. assert!(super::port_conflict_hint(8000, None).is_empty()); } + + // --- fpcalc, for song fingerprinting without FFmpeg's chromaprint muxer --- + + #[test] + fn fpcalc_asset_picks_the_build_for_each_platform() { + let asset = super::fpcalc_asset; + assert_eq!( + asset("linux", "x86_64"), + Some(( + "chromaprint-fpcalc-1.6.1-linux-x86_64.tar.gz".to_string(), + super::FPCALC_SHA256_LINUX_X86_64 + )) + ); + assert_eq!( + asset("linux", "aarch64"), + Some(( + "chromaprint-fpcalc-1.6.1-linux-arm64.tar.gz".to_string(), + super::FPCALC_SHA256_LINUX_ARM64 + )) + ); + for arch in ["x86_64", "aarch64"] { + assert_eq!( + asset("macos", arch), + Some(( + "chromaprint-fpcalc-1.6.1-macos-universal.tar.gz".to_string(), + super::FPCALC_SHA256_MACOS_UNIVERSAL + )) + ); + } + // Windows' FFmpeg has the muxer; nothing to fetch there or anywhere + // without a published build. + assert_eq!(asset("windows", "x86_64"), None); + assert_eq!(asset("linux", "riscv64"), None); + assert_eq!(asset("freebsd", "x86_64"), None); + } + + #[test] + fn fpcalc_pins_are_sha256_hex() { + for sha in [ + super::FPCALC_SHA256_LINUX_X86_64, + super::FPCALC_SHA256_LINUX_ARM64, + super::FPCALC_SHA256_MACOS_UNIVERSAL, + ] { + assert_eq!(sha.len(), 64); + assert!(sha + .chars() + .all(|c| c.is_ascii_hexdigit() && !c.is_ascii_uppercase())); + } + } + + #[test] + fn the_chromaprint_muxer_is_found_in_a_muxer_listing() { + let with = "File formats:\n E = Muxing supported\n --\n E caf Apple CAF\n E chromaprint Chromaprint\n E crc CRC testing\n"; + let without = "File formats:\n E = Muxing supported\n --\n E caf Apple CAF\n E crc CRC testing\n"; + assert!(super::lists_chromaprint_muxer(with)); + assert!(!super::lists_chromaprint_muxer(without)); + // A description mentioning it is not the muxer. + assert!(!super::lists_chromaprint_muxer( + " E null raw null for chromaprint tests\n" + )); + } + + /// A release-shaped tarball: `/fpcalc` beside other files. + fn fpcalc_tarball(dir: &Path, members: &[(&str, &[u8])]) -> PathBuf { + let archive = dir.join("fpcalc.tar.gz"); + let encoder = flate2::write::GzEncoder::new( + fs::File::create(&archive).unwrap(), + flate2::Compression::default(), + ); + let mut builder = tar::Builder::new(encoder); + for (path, body) in members { + let mut header = tar::Header::new_gnu(); + header.set_size(body.len() as u64); + header.set_mode(0o755); + header.set_cksum(); + builder.append_data(&mut header, path, *body).unwrap(); + } + builder.into_inner().unwrap().finish().unwrap(); + archive + } + + fn sha256_of(path: &Path) -> String { + super::sha256_file(path).unwrap() + } + + #[test] + fn unpack_fpcalc_extracts_only_fpcalc() { + let dir = make_tmp(); + let archive = fpcalc_tarball( + dir.path(), + &[ + ("chromaprint-fpcalc-1.6.1-linux-x86_64/LICENSE", b"license"), + ( + "chromaprint-fpcalc-1.6.1-linux-x86_64/fpcalc", + b"the binary", + ), + ], + ); + let target = dir.path().join("out").join("fpcalc.partial"); + fs::create_dir_all(target.parent().unwrap()).unwrap(); + let sha = sha256_of(&archive); + super::unpack_fpcalc(&archive, &sha, &target).unwrap(); + assert_eq!(fs::read(&target).unwrap(), b"the binary"); + let written: Vec<_> = fs::read_dir(target.parent().unwrap()) + .unwrap() + .map(|e| e.unwrap().file_name()) + .collect(); + assert_eq!(written, vec![std::ffi::OsString::from("fpcalc.partial")]); + } + + #[test] + fn unpack_fpcalc_rejects_a_checksum_mismatch() { + let dir = make_tmp(); + let archive = fpcalc_tarball(dir.path(), &[("x/fpcalc", b"tampered")]); + let target = dir.path().join("fpcalc.partial"); + let wrong = "0000000000000000000000000000000000000000000000000000000000000000"; + let err = super::unpack_fpcalc(&archive, wrong, &target).unwrap_err(); + assert!(err.contains("checksum mismatch"), "{err}"); + assert!(!target.exists(), "nothing extracted from a bad archive"); + assert!(!archive.exists(), "the bad archive is removed"); + } + + #[test] + fn unpack_fpcalc_fails_on_an_archive_without_fpcalc() { + let dir = make_tmp(); + let archive = fpcalc_tarball(dir.path(), &[("x/fpcalc.exe", b"wrong platform")]); + let target = dir.path().join("fpcalc.partial"); + let sha = sha256_of(&archive); + assert!(super::unpack_fpcalc(&archive, &sha, &target).is_err()); + assert!(!target.exists()); + } } diff --git a/docs/models.md b/docs/models.md index 317e6d27..1f964e5d 100644 --- a/docs/models.md +++ b/docs/models.md @@ -23,6 +23,22 @@ audit needed -- an unambiguous upstream license. StemDeck runs this model on CPU after separation and passes its existing stems. The checkpoint is not bundled in StemDeck installers. +## Whisper (lyrics transcription) + +- **Runtime**: `openai-whisper` (PyPI, MIT, `openai/whisper`), on the torch + StemDeck already ships for Demucs. +- **Checkpoints**: `large-v3-turbo` (1.6 GB) on a GPU with room for it, + `small` (0.5 GB) on a CPU or a smaller GPU. Overridable with + `STEMDECK_WHISPER_MODEL_GPU` / `STEMDECK_WHISPER_MODEL_CPU`. +- **License**: MIT for both the code and the published weights. +- **Upstream**: https://github.com/openai/whisper + +Runs only when no lyrics were found for a job and the `transcribe_lyrics` +setting allows it (by default on a GPU only). Weights download on first use +into `/whisper`, or during desktop setup when transcription would +run, and are verified by Whisper's own SHA-256 check. Not bundled in StemDeck +installers. + ## UVR-MDX-NET Karaoke 2 (on-demand lead/backing vocal split, #275) - **File**: `UVR_MDXNET_KARA_2.onnx` diff --git a/package.json b/package.json index 50645ebc..f1731c8b 100644 --- a/package.json +++ b/package.json @@ -6,6 +6,7 @@ "scripts": { "test:e2e": "playwright test", "test:e2e:headed": "playwright test --headed", + "test:acceptance": "playwright test -c tests/acceptance/playwright.acceptance.config.mjs", "test:js": "for f in tests/js/*.test.mjs; do node \"$f\"; done" }, "devDependencies": { diff --git a/packaging/linux/README-LINUX.txt b/packaging/linux/README-LINUX.txt index 5847b13f..341046a8 100644 --- a/packaging/linux/README-LINUX.txt +++ b/packaging/linux/README-LINUX.txt @@ -53,6 +53,12 @@ StemDeck downloads FFmpeg automatically on first launch (or uses a system `ffmpeg` if one is already on your PATH). The only system libraries you need are your distro's WebKitGTK + GTK, which the desktop shell links against. +Song identification with an AcoustID key also needs Chromaprint's fpcalc when +your FFmpeg has no chromaprint support. StemDeck downloads it in the background +on launch when it is needed; if that fails, songs are identified by their tags +only. A distro package works too (Debian/Ubuntu: libchromaprint-tools, +Fedora: chromaprint-tools, Arch: chromaprint). + Debian / Ubuntu: sudo apt update sudo apt install libwebkit2gtk-4.1-0 libgtk-3-0 diff --git a/packaging/linux/THIRD_PARTY_NOTICES.txt b/packaging/linux/THIRD_PARTY_NOTICES.txt index 792d5395..798f5ba7 100644 --- a/packaging/linux/THIRD_PARTY_NOTICES.txt +++ b/packaging/linux/THIRD_PARTY_NOTICES.txt @@ -56,6 +56,17 @@ soundfile License: BSD Website: https://github.com/bastibe/python-soundfile +OpenCC character table (data only) +License: Apache License 2.0 +Website: https://github.com/BYVoid/OpenCC +StemDeck folds traditional Chinese characters to simplified ones when it +compares artist and song names, so lyrics filed under either are found. +The table (app/_vendor/opencc/zh_t2s.txt, and the same pairs in +static/js/zhVariants.js) is derived from OpenCC's TSCharacters.txt, +Copyright Carbo Kuo (BYVoid) and the OpenCC contributors: only the first +simplified form of each character is kept. The license text is +app/_vendor/opencc/LICENSE. + System Requirements (Not Bundled) --------------------------------- @@ -98,3 +109,17 @@ the binary: If either link stops resolving, open an issue at https://github.com/stemdeckapp/stemdeck/issues and the source will be provided by another means. + +Chromaprint fpcalc +------------------ +For song fingerprinting, when the FFmpeg in use has no chromaprint support, +StemDeck downloads Chromaprint's fpcalc 1.6.1 (the upstream release build) +beside FFmpeg and calls it as a separate executable. It is not linked into +StemDeck. + +License: GNU Lesser General Public License v2.1 for Chromaprint as a + whole (its own code is MIT); the binary includes FFmpeg's + libraries. + https://github.com/acoustid/chromaprint/blob/v1.6.1/LICENSE.md +Source: https://github.com/acoustid/chromaprint/releases/tag/v1.6.1 + https://git.ffmpeg.org/ffmpeg.git diff --git a/packaging/macos/THIRD_PARTY_NOTICES.txt b/packaging/macos/THIRD_PARTY_NOTICES.txt index dfcd7f69..7cd3cbaf 100644 --- a/packaging/macos/THIRD_PARTY_NOTICES.txt +++ b/packaging/macos/THIRD_PARTY_NOTICES.txt @@ -12,6 +12,7 @@ Downloaded during first-run setup: - StemDeck runtime pack containing Python and Python package dependencies - FFmpeg and ffprobe +- Chromaprint's fpcalc, when the FFmpeg in use has no chromaprint support - Demucs model weights The runtime pack carries the full inventory under licenses/. INDEX.txt there @@ -40,3 +41,28 @@ Written offer of source. The complete corresponding source is: If either link stops resolving, open an issue at https://github.com/stemdeckapp/stemdeck/issues and the source will be provided by another means. + +Chromaprint fpcalc +------------------ +For song fingerprinting, StemDeck downloads Chromaprint's fpcalc 1.6.1 +(the upstream release build) beside FFmpeg and calls it as a separate +executable. It is not linked into StemDeck. + +License: GNU Lesser General Public License v2.1 for Chromaprint as a + whole (its own code is MIT); the binary includes FFmpeg's + libraries. + https://github.com/acoustid/chromaprint/blob/v1.6.1/LICENSE.md +Source: https://github.com/acoustid/chromaprint/releases/tag/v1.6.1 + https://git.ffmpeg.org/ffmpeg.git + +OpenCC character table +---------------------- +License: Apache License 2.0 +Website: https://github.com/BYVoid/OpenCC +StemDeck folds traditional Chinese characters to simplified ones when it +compares artist and song names, so lyrics filed under either are found. +The table (app/_vendor/opencc/zh_t2s.txt, and the same pairs in +static/js/zhVariants.js) is derived from OpenCC's TSCharacters.txt, +Copyright Carbo Kuo (BYVoid) and the OpenCC contributors: only the first +simplified form of each character is kept. The license text is +app/_vendor/opencc/LICENSE. diff --git a/packaging/windows/THIRD_PARTY_NOTICES.txt b/packaging/windows/THIRD_PARTY_NOTICES.txt index 7c7ab829..1cdd6662 100644 --- a/packaging/windows/THIRD_PARTY_NOTICES.txt +++ b/packaging/windows/THIRD_PARTY_NOTICES.txt @@ -56,6 +56,17 @@ soundfile License: BSD Website: https://github.com/bastibe/python-soundfile +OpenCC character table (data only) +License: Apache License 2.0 +Website: https://github.com/BYVoid/OpenCC +StemDeck folds traditional Chinese characters to simplified ones when it +compares artist and song names, so lyrics filed under either are found. +The table (app/_vendor/opencc/zh_t2s.txt, and the same pairs in +static/js/zhVariants.js) is derived from OpenCC's TSCharacters.txt, +Copyright Carbo Kuo (BYVoid) and the OpenCC contributors: only the first +simplified form of each character is kept. The license text is +app/_vendor/opencc/LICENSE. + Downloaded During First-Run Setup --------------------------------- diff --git a/pyproject.toml b/pyproject.toml index ad60d5e6..ce953120 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -88,6 +88,14 @@ dependencies = [ # today, but pin it explicitly so this survives an eventual librosa 1.x move # (#407) -- same platform gate as audio-separator itself. "audioread>=3.0; sys_platform != 'darwin' or platform_machine != 'x86_64'", + # Lyrics transcribed from the vocals stem when no published lyrics were + # found. OpenAI's reference Whisper runs on the torch Demucs already ships, + # so it adds about 3 MB of wheels (itself, tiktoken, regex, more-itertools) + # and no second CUDA runtime. faster-whisper was measured against it and + # would add ctranslate2 + PyAV (47 MB on Windows, 75 MB on Linux) plus its + # own cuBLAS/cuDNN requirement. Word timestamps come from word_timestamps=True. + # Model weights are downloaded on first use, never bundled. + "openai-whisper>=20250625", ] [project.optional-dependencies] diff --git a/run.sh b/run.sh index 00ac40c7..ae326b13 100755 --- a/run.sh +++ b/run.sh @@ -101,6 +101,15 @@ status() { fi } +# Song identification fingerprints with FFmpeg's chromaprint muxer, or with +# fpcalc where that FFmpeg has none (app/pipeline/fpcalc.py). True when there +# is neither. grep reads the whole listing, not -q, so pipefail never sees +# FFmpeg die of a closed pipe. +needs_fpcalc() { + command -v fpcalc >/dev/null 2>&1 && return 1 + ! ffmpeg -hide_banner -muxers 2>/dev/null | grep -w chromaprint >/dev/null +} + setup() { local os case "$(uname -s)" in @@ -124,6 +133,11 @@ setup() { else echo "ffmpeg: already installed ($(ffmpeg -version | head -n1))" fi + # Optional, so never fatal: without it songs are named by their tags. + if needs_fpcalc; then + echo "==> brew install chromaprint (optional, for song fingerprinting)" + brew install chromaprint || echo "chromaprint: not installed, song fingerprinting stays off" >&2 + fi if ! command -v uv >/dev/null 2>&1; then echo "==> brew install uv" brew install uv @@ -143,6 +157,12 @@ setup() { else echo "ffmpeg: already installed ($(ffmpeg -version | head -n1))" fi + # Debian and Ubuntu's ffmpeg has the muxer, so this is only for an + # FFmpeg from elsewhere. Optional, so never fatal. + if needs_fpcalc; then + echo "==> sudo apt-get install -y libchromaprint-tools (optional, for song fingerprinting)" + sudo apt-get install -y libchromaprint-tools || echo "fpcalc: not installed, song fingerprinting stays off" >&2 + fi if ! command -v uv >/dev/null 2>&1; then echo "==> installing uv via astral.sh installer" curl -LsSf https://astral.sh/uv/install.sh | sh diff --git a/static/css/daw.css b/static/css/daw.css index e256b5c6..64695e49 100644 --- a/static/css/daw.css +++ b/static/css/daw.css @@ -1287,18 +1287,63 @@ input, textarea { font-family: inherit; } .cancel-btn:hover { background: var(--panel-3); color: var(--fg); } .cancel-btn.hidden { display: none !important; } -/* Error banner */ -.error { - position: absolute; - top: 64px; left: 260px; right: 0; - z-index: 15; - padding: 12px 20px; - background: rgba(214,90,74,0.12); - border-bottom: 1px solid rgba(214,90,74,0.3); - color: #ff9090; - font-size: 13px; +/* Error banner: the studio's one alert box. By its id, never the class: + "error" is also the modifier every small status line takes (lyrics, the + artist box, settings), and the banner's rules on the class floated those + over the top bar as unreadable strips. A solid card, top right, above + everything, with its own close button (showError in job.js). */ +#error { + position: fixed; + top: 16px; right: 16px; + width: min(440px, calc(100vw - 32px)); + z-index: 1000; + display: flex; align-items: flex-start; gap: 12px; + padding: 12px 10px 12px 14px; + background: var(--panel-2); + border: 1px solid rgba(214,90,74,0.45); + border-left: 4px solid var(--danger); + border-radius: var(--radius); + box-shadow: var(--shadow-panel); + color: var(--fg); + font-size: 13px; line-height: 1.45; + animation: error-in 160ms ease-out; } -.error.hidden { display: none !important; } +#error.hidden { display: none !important; } +#error .error-icon { + flex-shrink: 0; + width: 18px; height: 18px; margin-top: 1px; + color: var(--danger); +} +#error .error-msg { flex: 1; min-width: 0; overflow-wrap: anywhere; } +#error .error-detail { + margin-top: 4px; + font-family: var(--font-mono); font-size: 11px; + color: var(--muted); +} +#error .error-actions { display: flex; align-items: center; gap: 4px; flex-shrink: 0; } +#error .retry-btn { + height: 26px; padding: 0 10px; + background: transparent; + border: 1px solid rgba(214,90,74,0.6); border-radius: var(--radius-sm); + color: var(--fg); font: inherit; font-size: 12px; + cursor: pointer; + transition: background var(--t-fast); +} +#error .retry-btn:hover { background: rgba(214,90,74,0.18); } +#error .error-close { + width: 26px; height: 26px; + display: grid; place-items: center; + background: transparent; border: none; border-radius: var(--radius-sm); + color: var(--muted); font-size: 18px; line-height: 1; + cursor: pointer; +} +#error .error-close:hover { background: var(--panel-3); color: var(--fg); } +#error :is(.retry-btn, .error-close):focus-visible { outline: 2px solid var(--danger); outline-offset: 1px; } +@keyframes error-in { + from { opacity: 0; transform: translateY(-6px); } + to { opacity: 1; transform: none; } +} +@media (prefers-reduced-motion: reduce) { #error { animation: none; } } /* ── Track header ── */ .daw-track-header { @@ -3560,6 +3605,9 @@ input, textarea { font-family: inherit; } color: var(--muted); text-decoration: none; transition: all 0.15s; } .about-social-btn:hover { background: var(--accent); border-color: var(--accent); color: #000; } +/* Where song details, lyrics and transcription come from, and their + licences: small print under the socials. */ +.about-credits { margin: 16px 0 0; font-size: 10.5px; line-height: 1.5; color: var(--muted); } /* ── Release notes dialog ── */ .release-card { @@ -3845,6 +3893,462 @@ input, textarea { font-family: inherit; } what was pushing the list past the dialog and putting a scrollbar on it. */ .friends-card { width: min(1020px, calc(100vw - 32px)); } .friends-card .lib-friends-grid { width: 100%; margin-top: 4px; } + +/* ── Lyrics tab (#699) ── + A view of the sidebar, like Favorites and Trash, but not a list: while it is + on, the search box and the library list step aside and the panel takes the + whole height, so the lyrics get the room and scroll on their own. */ +.lyrics-panel { display: none; } +.sidebar.lyrics-view .daw-search, +.sidebar.lyrics-view .clear-bin-bar, +.sidebar.lyrics-view .daw-lib-list { display: none !important; } +.sidebar.lyrics-view .lyrics-panel { + display: flex; flex-direction: column; gap: 8px; + flex: 1 1 auto; min-height: 0; +} +/* The EXTRACT label's type, like every other section head in the app. */ +.lyrics-title { + margin: 0; font-size: 10px; font-weight: 600; + letter-spacing: 0.12em; text-transform: uppercase; color: var(--muted); +} +.lyrics-note { margin: 0; font-size: 10.5px; line-height: 1.4; color: var(--muted); } +.lyrics-status { margin: 0; font-size: 12px; color: var(--fg-2); } +.lyrics-status:empty { display: none; } +.lyrics-status.muted { color: var(--muted); } +.lyrics-status.error { color: var(--danger); } + +.lyrics-versions:empty { display: none; } +.lyrics-versions { display: flex; flex-direction: column; gap: 4px; padding-top: 4px; border-top: 1px solid var(--border); } +.lyrics-match { display: flex; flex-direction: column; min-width: 0; } +.lyrics-match-title { font-size: 13px; font-weight: 600; color: var(--fg); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.lyrics-match-meta { font-size: 11px; color: var(--muted); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.lyrics-tools { display: flex; flex-wrap: wrap; gap: 4px 12px; } +.lyrics-link { + padding: 0; background: none; border: none; cursor: pointer; + color: var(--accent); font-size: 11px; font-weight: 600; +} +.lyrics-link:hover { text-decoration: underline; } +/* Align, for lyrics with no timing to move: still focusable, so its tooltip + says why (lyrics.js). */ +.lyrics-link[aria-disabled="true"] { color: var(--muted); cursor: default; } +.lyrics-link[aria-disabled="true"]:hover { text-decoration: none; } + +/* The Align panel: under the tools row, the sidebar's width, every control a + button so the keys reach them all. The nudges sit either side of the + offset they change. */ +.lyrics-align { + display: flex; flex-direction: column; gap: 6px; + padding: 8px; border: 1px solid var(--border); border-radius: 8px; background: var(--panel); +} +.lyrics-align-hint { margin: 0; font-size: 10.5px; line-height: 1.4; color: var(--muted); } +.lyrics-align-target { + margin: 0; font-size: 11px; color: var(--fg-2); + overflow: hidden; text-overflow: ellipsis; white-space: nowrap; +} +.lyrics-align-target:empty { display: none; } +.lyrics-align-btn { + min-height: 26px; padding: 0 8px; border-radius: 6px; + background: var(--panel-2); border: 1px solid var(--border-strong); + color: var(--fg-2); font-family: inherit; font-size: 11px; font-weight: 600; + cursor: pointer; transition: background var(--t-fast), color var(--t-fast); +} +.lyrics-align-btn:hover { background: var(--panel-3); color: var(--fg); } +.lyrics-align-btn:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; } +.lyrics-align-btn[aria-busy="true"] { cursor: progress; opacity: 0.6; } +.lyrics-align-start { width: 100%; color: var(--accent); border-color: var(--accent); } +.lyrics-align-nudges { display: flex; align-items: center; gap: 4px; } +.lyrics-align-nudge { flex: 0 0 auto; padding: 0 6px; font-family: var(--font-mono); font-variant-numeric: tabular-nums; } +.lyrics-align-offset { + flex: 1 1 auto; min-width: 0; text-align: center; + font-family: var(--font-mono); font-size: 12px; font-weight: 600; color: var(--fg); + font-variant-numeric: tabular-nums; white-space: nowrap; +} +.lyrics-align-actions { display: flex; gap: 6px; } +.lyrics-align-actions .lyrics-align-btn { flex: 1 1 0; min-width: 0; } +/* Done closes the panel: filled, so it reads as the way out. */ +.lyrics-align-done { background: var(--accent); border-color: var(--accent); color: var(--bg); } +.lyrics-align-done:hover { background: var(--accent); color: var(--bg); filter: brightness(1.08); } +.lyrics-align-message { margin: 0; font-size: 11px; line-height: 1.4; color: var(--fg-2); } +.lyrics-align-message:empty { display: none; } +.lyrics-align-message.muted, .lyrics-align-message.loading { color: var(--muted); } +.lyrics-align-message.error { color: var(--danger); } +/* display: flex would otherwise outrank the hidden attribute that folds it. */ +.lyrics-version-list[hidden] { display: none; } +.lyrics-version-list { list-style: none; margin: 2px 0 0; padding: 0; display: flex; flex-direction: column; gap: 2px; max-height: 180px; overflow-y: auto; } +.lyrics-version { + width: 100%; display: flex; flex-direction: column; align-items: flex-start; min-width: 0; + padding: 5px 7px; border-radius: 6px; border: 1px solid transparent; + background: none; color: var(--fg-2); text-align: left; cursor: pointer; +} +.lyrics-version:hover { background: var(--panel); border-color: var(--border); } +.lyrics-version-title { font-size: 12px; max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.lyrics-version-meta { font-size: 10.5px; color: var(--muted); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } + +/* The lyrics scroll inside the panel. The bottom padding lets the last lines + reach the middle, where the line being sung is kept. position: relative + makes this the lines' offsetParent, which lyrics.js measures from. */ +.lyrics-body { position: relative; flex: 1 1 auto; min-height: 0; overflow-y: auto; padding: 6px 2px 40%; } +.lyrics-body:empty { display: none; } +/* Synced lines are buttons, since clicking one moves the playhead there. */ +.lyrics-line { + display: block; width: 100%; margin: 0; padding: 4px 8px; + background: none; border: none; border-radius: 6px; + color: var(--muted); font: inherit; font-size: 14px; line-height: 1.45; text-align: left; + cursor: pointer; transition: color var(--t-fast), background var(--t-fast); +} +.lyrics-line:hover { color: var(--fg-2); background: var(--panel); } +/* Korean is spaced into words and wraps between them, never inside one; a + word too long for the line still breaks. lang is set per line by lyrics.js. */ +.lyrics-line:lang(ko), .lyrics-text:lang(ko) { word-break: keep-all; overflow-wrap: anywhere; } +/* A ring for keyboard focus only: a clicked line lets go of focus (lyrics.js). */ +.lyrics-line:focus { outline: none; } +.lyrics-line:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; } +.lyrics-line.current { color: var(--fg); font-weight: 600; box-shadow: inset 2px 0 0 var(--accent); } +/* The line after it, a step brighter than the rest, so the next words can be + read before they are due. */ +.lyrics-line.next { color: var(--fg-2); } +/* While the Align panel is open, the line last clicked, which Start lyrics + here moves instead of the first. */ +.lyrics-body.aligning .lyrics-line.picked { outline: 1px dashed var(--border-strong); outline-offset: -1px; } +/* The karaoke wipe, one word at a time: sung words take the accent, and the + word being sung is filled left to right up to --fill, set every frame by + lyrics.js. Per word rather than across the whole line, so a line that wraps + fills its first row before its second instead of both at once. */ +.lyrics-line.current .lw.sung { color: var(--accent); } +.lyrics-line.current .lw.singing { + color: transparent; + background: linear-gradient(90deg, var(--accent) var(--fill, 0%), var(--fg) var(--fill, 0%)); + -webkit-background-clip: text; + background-clip: text; +} +.lyrics-text { margin: 0; padding: 1px 8px; font-size: 13.5px; line-height: 1.55; color: var(--fg-2); } + +/* The Align panel's note when the lines are timed one by one, so the offset + is not what is in effect. */ +.lyrics-align-note { margin: 0; font-size: 11px; line-height: 1.4; color: var(--fg-2); } +.lyrics-align-note:empty { display: none; } +.lyrics-align-btn:disabled { opacity: 0.45; cursor: default; } +.lyrics-align-btn:disabled:hover { background: var(--panel-2); color: var(--fg-2); } + +/* ── Sync lines (lyrics.js) ── + The panel under the tools row while the mode is on: which line is next, + one big Tap, the options, the resets. Built for a glance mid-song. */ +.lyrics-sync { + display: flex; flex-direction: column; gap: 7px; + padding: 8px; border: 1px solid var(--accent); border-radius: 8px; background: var(--panel); +} +.lyrics-sync-head { display: flex; align-items: baseline; justify-content: space-between; gap: 8px; } +.lyrics-sync-title { margin: 0; font-size: 12px; font-weight: 700; color: var(--fg); } +.lyrics-sync-kind { font-size: 10.5px; color: var(--muted); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; } +.lyrics-sync-next { + margin: 0; padding: 6px 8px; border-radius: 6px; background: var(--bg-2); + font-size: 13px; line-height: 1.35; color: var(--fg); + overflow: hidden; text-overflow: ellipsis; white-space: nowrap; +} +.lyrics-sync-next-label { display: block; font-size: 9.5px; font-weight: 700; letter-spacing: 0.1em; text-transform: uppercase; color: var(--muted); } +.lyrics-sync-tap { + min-height: 44px; font-size: 15px; letter-spacing: 0.04em; + background: var(--accent); border-color: var(--accent); color: var(--bg); +} +.lyrics-sync-tap:hover { background: var(--accent); color: var(--bg); filter: brightness(1.08); } +.lyrics-sync-tap:disabled, .lyrics-sync-tap:disabled:hover { background: var(--panel-2); border-color: var(--border-strong); color: var(--muted); filter: none; } +.lyrics-sync-tap kbd { font-family: var(--font-mono); font-size: 10px; font-weight: 600; opacity: 0.75; margin-left: 6px; } +/* A tap lands: the button flashes, so a tap can be seen as well as heard. */ +.lyrics-sync-tap.tapped { animation: lyrics-tap 180ms ease-out; } +@keyframes lyrics-tap { from { filter: brightness(1.6); } to { filter: none; } } +@media (prefers-reduced-motion: reduce) { .lyrics-sync-tap.tapped { animation: none; } } +.lyrics-sync-row { display: flex; flex-wrap: wrap; align-items: center; gap: 6px; } +.lyrics-sync-row .lyrics-align-btn { flex: 1 1 auto; } +.lyrics-sync-opts { display: flex; flex-wrap: wrap; gap: 4px 12px; } +.lyrics-sync-opt { display: inline-flex; align-items: center; gap: 5px; font-size: 11px; color: var(--fg-2); cursor: pointer; } +.lyrics-sync-opt input { margin: 0; accent-color: var(--accent); } +.lyrics-sync-keys { margin: 0; font-size: 10.5px; line-height: 1.5; color: var(--muted); } +.lyrics-sync-keys kbd { + font-family: var(--font-mono); font-size: 10px; padding: 0 4px; border-radius: 3px; + border: 1px solid var(--border-strong); background: var(--panel-2); color: var(--fg-2); +} +/* In the mode each line shows when it starts, and the line in hand is framed + and carries its own nudges. The time is drawn by CSS from data-time, so it + is not part of the line's text. */ +.lyrics-body.syncing .lyrics-line::before { + content: attr(data-time); + display: inline-block; min-width: 4.6em; margin-right: 6px; + font-family: var(--font-mono); font-size: 10.5px; font-weight: 500; color: var(--muted); + font-variant-numeric: tabular-nums; +} +.lyrics-body.syncing .lyrics-line.selected { background: var(--panel); outline: 1px solid var(--accent); outline-offset: -1px; } +.lyrics-line-tools { display: flex; align-items: center; gap: 4px; padding: 2px 8px 6px; } +.lyrics-line-tools .lyrics-align-btn { min-height: 22px; padding: 0 6px; font-size: 10.5px; } +.lyrics-line-tools .lyrics-align-nudge { font-family: var(--font-mono); } +.lyrics-line-tools .lyrics-line-here { margin-left: auto; min-width: 0; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; color: var(--accent); border-color: var(--accent); } + +/* The lyrics lane over the waveform (lyricsLane.js), shaped like the + Sections ribbon above it: markers at percentages of a track that is the + zoomed waveform's width. */ +.daw-lyrics-ribbon { display: flex; flex-shrink: 0; border-bottom: 1px solid var(--border); } +.daw-lyrics-ribbon.hidden { display: none; } +.daw-lyrics-header { justify-content: space-between; gap: 8px; } +.daw-lyrics-title-group { display: inline-flex; align-items: baseline; gap: 8px; min-width: 0; overflow: hidden; } +.daw-lyrics-title-group .daw-label-sub { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; } +.lyrics-lane-done { + flex-shrink: 0; min-height: 22px; padding: 0 10px; border-radius: 5px; + background: var(--accent); border: 1px solid var(--accent); color: var(--bg); + font: inherit; font-size: 11px; font-weight: 700; cursor: pointer; +} +.lyrics-lane-done:hover { filter: brightness(1.08); } +.daw-lyrics-area { flex: 1; position: relative; height: 34px; background: var(--bg-2); overflow: clip; user-select: none; } +.daw-lyrics-track { position: absolute; inset: 0; right: auto; width: calc(100% * var(--zoom, 1)); } +.lyr-mark { + position: absolute; top: 4px; bottom: 4px; min-width: 6px; + display: flex; align-items: center; padding: 0 6px 0 7px; margin: 0; + background: none; border: none; border-left: 2px solid var(--fg-2); border-radius: 0; + color: var(--fg-2); font: inherit; font-size: 10.5px; text-align: left; + cursor: grab; overflow: hidden; +} +.lyr-mark:hover { background: rgba(255, 255, 255, 0.05); color: var(--fg); } +.lyr-mark-label { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; pointer-events: none; } +.lyr-mark.current { color: var(--fg); border-left-color: var(--fg); background: rgba(255, 255, 255, 0.06); } +.lyr-mark.selected { color: var(--accent); border-left-color: var(--accent); background: rgba(244, 183, 64, 0.12); z-index: 2; } +.lyr-mark.dragging { cursor: grabbing; z-index: 3; background: rgba(244, 183, 64, 0.2); } +.lyr-lane-playhead { position: absolute; inset: 0; pointer-events: none; will-change: transform; z-index: 4; } +.lyr-lane-playhead i { position: absolute; top: 0; bottom: 0; left: 0; width: 1px; background: var(--accent); opacity: 0.8; } + +/* ── Artist box (#699) ── + The About dialog's frame, left-aligned and wider, because it holds prose + and lists rather than a logo and two buttons. The body scrolls on its own + so the search field stays put above a long history. */ +.artist-card { + width: min(640px, calc(100vw - 32px)); + max-height: calc(100vh - 64px); + padding: 22px 22px 18px; + text-align: left; + align-items: stretch; +} +/* Room on the right for the close button, which sits in the same corner. */ +.artist-search { display: flex; gap: 8px; margin-right: 34px; } +.artist-search input { + flex: 1; min-width: 0; height: 34px; padding: 0 10px; + background: var(--bg-2); border: 1px solid var(--border-strong); border-radius: 8px; + color: var(--fg); font: inherit; font-size: 13px; +} +.artist-search input:focus { outline: none; border-color: var(--accent); } +.artist-search-btn { + height: 34px; padding: 0 14px; border-radius: 8px; + background: var(--panel); border: 1px solid var(--border); + color: var(--fg-2); font-weight: 600; cursor: pointer; +} +.artist-search-btn:hover { background: var(--accent); border-color: var(--accent); color: #000; } +.artist-search { margin-bottom: 14px; } +/* A band or a song already known opens without the search row; the body's + first line then sits in the close button's corner and leaves it room. */ +.artist-search.hidden { display: none; } +.artist-search.hidden + .artist-body > :first-child { margin-right: 34px; } +/* What is sent where: the box's foot, under the scrolling body, in the + credit line's small muted type. */ +.artist-card-foot { flex-shrink: 0; padding-top: 8px; } +.artist-privacy { margin: 0; font-size: 10.5px; line-height: 1.45; color: var(--muted); } +.artist-other { margin: -6px 0 10px; font-size: 12px; } +.artist-status + .artist-other { margin-top: -14px; text-align: center; } +.artist-other-btn { + padding: 0; background: none; border: 0; + color: var(--accent); font: inherit; cursor: pointer; +} +.artist-other-btn:hover { text-decoration: underline; } +.artist-body { flex: 1 1 auto; min-height: 0; overflow-y: auto; padding-right: 4px; } +.artist-status { margin: 24px 0; text-align: center; font-size: 13px; color: var(--fg-2); } +.artist-status.muted { color: var(--muted); } +.artist-status.error { color: var(--danger); } + +.artist-head { display: flex; gap: 14px; align-items: center; margin-bottom: 14px; } +.artist-photo { + width: 72px; height: 72px; flex-shrink: 0; + object-fit: cover; border-radius: 10px; + border: 1px solid var(--border); background: var(--panel); +} +/* Two classes deep so it outranks .about-card h2, which is the About box's + monospace wordmark and not a band name. */ +.artist-card .artist-name { + margin: 0; font-family: inherit; font-size: 22px; font-weight: 700; line-height: 1.2; +} +.artist-desc { margin: 4px 0 0; font-size: 12px; color: var(--muted); } +.artist-titles { flex: 1 1 auto; min-width: 0; } +/* A long name, or one in a script with no spaces to break at, wraps inside + the box rather than pushing the save button out of it. */ +.artist-card .artist-name, .artist-card .artist-work-name { overflow-wrap: anywhere; } +/* The name in the artist's own language after the one in the reader's + (Jay Chou 周杰倫): lighter, and kept whole on the next line when the two do + not fit on one. */ +.artist-native { + display: inline-block; max-width: 100%; + font-size: 0.72em; font-weight: 500; color: var(--muted); +} +.artist-names .artist-native { margin-left: 2px; font-size: 1em; } +/* Which Wikipedia the text below was read from, when not the reader's own. */ +.artist-edition { margin: -4px 0 6px; font-size: 11px; color: var(--muted); } +/* Korean breaks between words, as it is written, not inside them; Japanese + keeps small kana and closing marks off the start of a line. */ +.artist-card :lang(ko) { word-break: keep-all; overflow-wrap: anywhere; } +.artist-history:lang(ja) { line-break: strict; } +/* The section titles' wide tracking is for Latin capitals; between Chinese, + Japanese or Korean characters it reads as a gap inside the word. */ +.artist-section-title:lang(ja), .artist-section-title:lang(ko), .artist-section-title:lang(zh) { + letter-spacing: 0.02em; +} +/* Save for this track: the accent until it is done, then a quiet + label that says so. It sits by the name because the name is what is saved. */ +.artist-save { + flex-shrink: 0; align-self: center; + height: 30px; padding: 0 12px; border-radius: 8px; + background: var(--accent); border: 1px solid var(--accent); + color: #000; font-size: 12px; font-weight: 600; cursor: pointer; +} +.artist-save:hover { filter: brightness(1.08); } +.artist-save.saved { + background: var(--panel); border-color: var(--border); + color: var(--muted); cursor: default; filter: none; +} + +.artist-section { padding: 12px 0; border-top: 1px solid var(--border); } +/* The EXTRACT label's type, as the card's NOW PLAYING legend uses. */ +.artist-section-title { + margin: 0 0 8px; font-size: 10px; font-weight: 600; + letter-spacing: 0.12em; text-transform: uppercase; color: var(--muted); +} +.artist-history p { margin: 0 0 8px; font-size: 13px; line-height: 1.55; color: var(--fg-2); } +.artist-names { list-style: none; margin: 0; padding: 0; display: flex; flex-wrap: wrap; gap: 6px; } +.artist-names li { + padding: 3px 10px; font-size: 12px; + background: var(--panel); border: 1px solid var(--border); border-radius: 999px; +} +.artist-subtitle { margin: 10px 0 6px; font-size: 11px; font-weight: 600; color: var(--muted); } +/* Two columns: a band with twenty albums is a short box, not a long one. */ +.artist-albums { list-style: none; margin: 0; padding: 0; columns: 2; column-gap: 24px; } +.artist-albums li { display: flex; gap: 10px; padding: 3px 0; font-size: 12.5px; break-inside: avoid; } +.artist-album-year { min-width: 4ch; color: var(--muted); font-variant-numeric: tabular-nums; } +.artist-foot { + display: flex; justify-content: space-between; align-items: center; gap: 12px; + padding-top: 12px; border-top: 1px solid var(--border); + font-size: 11px; color: var(--muted); +} +.artist-more { color: var(--accent); font-weight: 600; text-decoration: none; } +.artist-more:hover { text-decoration: underline; } +/* A band filled from Discogs credits it beside Wikipedia: three items can + share the row, so it wraps rather than squeezing them. */ +.artist-foot { flex-wrap: wrap; } +.artist-discogs-credit { font-weight: 500; } +/* "Only Wikipedia was searched": a quiet aside, not a warning, with the way + to the setting as a small secondary button. */ +.artist-discogs-note { + margin: 0 0 12px; padding: 10px 12px; + background: var(--panel); border: 1px solid var(--border); border-radius: 8px; + font-size: 12px; line-height: 1.5; color: var(--fg-2); +} +.artist-discogs-note p { margin: 0 0 8px; } +.artist-discogs-open { + height: 28px; padding: 0 12px; border-radius: 8px; + background: var(--bg-2); border: 1px solid var(--border-strong); + color: var(--fg); font: inherit; font-size: 12px; font-weight: 600; cursor: pointer; +} +.artist-discogs-open:hover { border-color: var(--accent); color: var(--accent); } +.artist-status + .artist-discogs-note { margin-top: -8px; } +/* The artists Discogs has for a name Wikipedia does not know, to pick from: + each its name, with Discogs' number, and the start of its profile. */ +.artist-candidates { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 6px; } +.artist-candidate { + display: flex; flex-direction: column; gap: 2px; width: 100%; + padding: 8px 10px; text-align: start; + background: var(--panel); border: 1px solid var(--border); border-radius: 8px; + color: var(--fg); font: inherit; cursor: pointer; +} +.artist-candidate:hover, .artist-candidate:focus-visible { border-color: var(--accent); } +.artist-candidate-name { font-size: 13px; font-weight: 600; overflow-wrap: anywhere; } +.artist-candidate-profile { font-size: 12px; line-height: 1.45; color: var(--muted); } +.artist-discogs-pick + .artist-foot { justify-content: flex-end; } +@media (max-width: 560px) { .artist-albums { columns: 1; } } + +/* The musical or film a song is from: a section below the band, or the whole + box when there is no band, where it leads without the rule above it. */ +.artist-work.lead { padding-top: 0; border-top: 0; } +.artist-work-head { margin-bottom: 10px; } +.artist-work:not(.lead) .artist-work-photo { width: 56px; height: 56px; } +/* Two classes deep, as .artist-name, to outrank .about-card h2. */ +.artist-card .artist-work-name { + margin: 0; font-family: inherit; font-size: 16px; font-weight: 700; line-height: 1.25; +} +.artist-card .artist-work.lead .artist-work-name { font-size: 22px; line-height: 1.2; } +.artist-work-year { font-weight: 500; color: var(--muted); font-variant-numeric: tabular-nums; } +.artist-work-credits { + display: grid; grid-template-columns: auto 1fr; gap: 4px 12px; + margin: 4px 0 8px; font-size: 12.5px; +} +.artist-work-credits dt { color: var(--muted); } +.artist-work-credits dd { margin: 0; color: var(--fg-2); } +.artist-work-more { margin: 0; font-size: 12px; } + +/* The song, heading the box whenever the track is known as one. What follows + it (the show, or a band) then starts below a rule, as any section does. */ +.artist-song { margin-bottom: 14px; } +/* Two classes deep, as .artist-name, to outrank .about-card h2. */ +.artist-card .artist-song-title { + margin: 0; font-family: inherit; font-size: 22px; font-weight: 700; line-height: 1.2; +} +.artist-song-from { font-size: 13px; color: var(--fg-2); } +.artist-song-facts { margin: 10px 0 0; } +.artist-head.artist-band-head { padding-top: 14px; border-top: 1px solid var(--border); } +/* The performer under a show: the work section's smaller photo and name, and + the rest behind "More about ...". */ +.artist-performer-head { margin-bottom: 8px; } +.artist-performer .artist-photo { width: 56px; height: 56px; } +.artist-card .artist-performer .artist-name { font-size: 16px; line-height: 1.25; } +.artist-performer .artist-other { margin: 0 0 6px; } +.artist-performer-more:focus { outline: none; } +/* Official links under a band's or a show's heading: the member chips' pill + shape, an icon and a short label each, wrapping on a narrow box. */ +.artist-links { + list-style: none; display: flex; flex-wrap: wrap; gap: 6px; + margin: -6px 0 12px; padding: 0; +} +.artist-performer .artist-links, .artist-work .artist-links { margin: 0 0 10px; } +.artist-link { + display: inline-flex; align-items: center; gap: 6px; + padding: 3px 10px; font-size: 12px; font-weight: 500; line-height: 1.4; + color: var(--fg-2); text-decoration: none; white-space: nowrap; + background: var(--panel); border: 1px solid var(--border); border-radius: 999px; +} +.artist-link svg { flex-shrink: 0; color: var(--muted); } +.artist-link:hover { color: var(--fg); border-color: var(--accent); } +.artist-link:hover svg { color: var(--accent); } +.artist-link:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } + +/* The info button that opens the box: the heart's mirror image, a full-height + segment on the card's right edge, where the heart used to be. A button of + its own rather than the whole card being clickable, which competed with the + heart inside it and gave the keyboard nothing real to press. */ +#nowPlayingPanel .track-panel-head .daw-details-btn { + position: absolute; + right: 0; + top: 0; + bottom: 0; + width: var(--np-fav); + margin: 0; + padding: 0; + display: flex; + align-items: center; + justify-content: center; + border: none; + border-left: 1px solid var(--border); + border-radius: 0 7px 7px 0; + background: none; + color: var(--muted); + cursor: pointer; +} +#nowPlayingPanel .track-panel-head .daw-details-btn:hover { background: var(--panel-3); color: var(--fg); } +#nowPlayingPanel .track-panel-head .daw-details-btn:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; } +/* Nothing to look up before a track is open. */ +.app.no-track #nowPlayingPanel .daw-details-btn { opacity: 0.4; pointer-events: none; } +/* The text stops short of the segment. */ +#nowPlayingPanel .track-panel-info, +#nowPlayingPanel .daw-track-meta-line { padding-right: calc(var(--np-fav) + 10px); } .lib-friends-grid { display: grid; grid-template-columns: 1fr 1px 1fr; @@ -4135,6 +4639,22 @@ input, textarea { font-family: inherit; } margin-top: 6px; font-size: 11px; line-height: 1.4; color: var(--danger); } +/* AcoustID key and Discogs token: a masked field with its Save and Clear + beside it. */ +.acoustid-key, .discogs-token { + display: flex; align-items: center; gap: 8px; margin-top: 8px; +} +.acoustid-key .settings-text-input, +.discogs-token .settings-text-input { flex: 1; min-width: 0; margin-top: 0; } +.acoustid-key-msg, .discogs-token-msg { + margin-top: 6px; font-size: 11px; line-height: 1.4; color: var(--muted); +} +.acoustid-key-msg.error, .discogs-token-msg.error { color: var(--danger); } +.acoustid-register, .discogs-register { + display: inline-block; margin-top: 4px; font-size: 11px; color: var(--accent); +} +.discogs-privacy { margin-top: 4px; } + /* Logs sub-navigation: Location / Application / Setup. */ .settings-subtabs { display: flex; gap: 2px; margin: 0 0 10px; @@ -4279,8 +4799,11 @@ input, textarea { font-family: inherit; } and are therefore exactly the same width. As a flex row the search grew to whatever was left over, which is why it ran past the last chip. - Columns: content | spacer | rule | detect | split. Row one is the import - gesture, row two is what to extract. */ + Columns: content | detect | split | rule | card. Row one is the import + gesture, row two is what to extract. Split sits straight after the options + it acts on, and the loaded track comes after the rule, at the end: in the + middle of the row it read as Split's input, when it is only what is + playing (#699). */ height: auto; display: grid; /* minmax(0, 1fr), not 1fr: a grid track's automatic minimum is its @@ -4288,7 +4811,7 @@ input, textarea { font-family: inherit; } it. The card sat at its natural width and pushed Split stems off the right edge of its own container. Zero as the floor lets the column give way, and the card's own text clips with an ellipsis rather than forcing the row. */ - grid-template-columns: minmax(0, max-content) minmax(0, 1fr) auto auto auto; + grid-template-columns: minmax(0, max-content) auto auto auto minmax(0, 1fr); align-items: center; row-gap: 8px; column-gap: 10px; @@ -4351,9 +4874,9 @@ input, textarea { font-family: inherit; } min-width: 0; } -/* The rule stays, but it now separates the search from the two actions on the - same line, which is what it does in the mockup. It used to divide the row - from the Extract chips, which are on their own line now. */ +/* The rule divides the extraction, search through Split, from the track that + is playing (#699). Nothing crosses it: what is left of it is one sentence, + what is right of it is not an input to that sentence. */ /* One rule, down both rows. * * There were two: this one beside the actions, and the Collapse label's own @@ -4362,7 +4885,7 @@ input, textarea { font-family: inherit; } * that reads as broken without being nameable. Spanning the rows makes it one * line that cannot drift from itself. */ .daw-composer > .daw-composer-sep { - grid-column: 3; + grid-column: 4; grid-row: 1 / 3; display: block; justify-self: center; @@ -4379,7 +4902,7 @@ input, textarea { font-family: inherit; } Grid because the switch sits left of the label and spans both the label and the EXPERIMENTAL note under it. */ .daw-composer > .structure-toggle { - grid-column: 4; + grid-column: 2; grid-row: 1; position: relative; display: grid; @@ -4439,7 +4962,7 @@ input, textarea { font-family: inherit; } border-left: none; } .daw-composer > .daw-process-btn { - grid-column: 5; + grid-column: 3; grid-row: 1; } @@ -4460,19 +4983,19 @@ input, textarea { font-family: inherit; } and it would need the strip and the footer inset to match, or something given the freed space. */ -/* Row two, right-hand end: on the chips' line rather than below it. +/* Row two, under the two actions: on the chips' line rather than below it. - Columns 4 to 6, not 3: a spanning item's minimum lands in the first track it - covers, so starting at the separator's column handed that 1px column the - toggles' whole width and took it out of the card's. The two action columns - are already as wide as Detect structure and Split stems, which is more than - the toggles need, so spanning only those costs the row nothing. + Columns 2 to 4, the two action columns only, not the rule's: a spanning + item's minimum lands in the first track it covers, and the 1px rule column + would have been handed the toggles' whole width. The action columns are + already as wide as Detect structure and Split stems, which is more than the + toggles need, so spanning only those costs the row nothing. A grid item rather than absolutely positioned: out of flow it sat on the right line and would have landed on top of the Extract chips the moment a window was narrow enough for the two to meet. */ .daw-composer > .daw-panel-toggles { - grid-column: 4 / 6; + grid-column: 2 / 4; grid-row: 2; justify-self: end; align-self: center; @@ -4540,24 +5063,29 @@ input, textarea { font-family: inherit; } because that timeline's left edge had to line up with the lane waveforms. The timeline is gone, and this row had empty space in the middle of it, so the block moved in the markup rather than being pinned here: it is a cell in - the grid, column two, spanning both rows. That is the column the search and - the chips do not use, between them and the actions on the right. */ + the grid, spanning both rows. It was column two, between the chips and the + actions, where the row read as "file, options, this track, Split" and made + the track look like what Split would act on (#699). Now it is the last + column, after the rule, and labelled as what is playing. */ #nowPlayingPanel { position: relative; - grid-column: 2; + grid-column: 5; grid-row: 1 / 3; - /* As wide as the track it is naming, not as wide as the window. - It was stretched to fill its column, so a spare 400px of bar became 400px - of card with the same text sitting in it. It sizes to its content now and - stops; a long title clips with an ellipsis at the cap rather than dragging - the card out to meet it. */ - justify-self: start; + /* Fills its column up to the cap below. It once sized to its content, so a + spare 400px of bar would not become 400px of card; the cap does that job + now, and filling the column is what lets the card be a size container for + the artwork's container query (see .track-panel-cover below). A long + title clips with an ellipsis at the cap rather than dragging the card out + to meet it. */ + justify-self: stretch; /* A real ceiling, not a percentage of the column: 100% just meant "as wide as the bar allows", so the card was 422px at 1600 and 460 at 1920 with the - same text in it. 460 is what the meta line needs at its longest, duration - through to the extracted date, so the card is the same size at every - window width and a long title clips rather than pushing it wider. */ - max-width: min(100%, 460px); + same text in it. 460 was what the meta line needed at its longest, + duration through to the extracted date; 536 is that plus the 76px the + full-height artwork now takes from the text column (#699). So the card is + the same size at every window width and a long title clips rather than + pushing it wider. */ + max-width: min(100%, 536px); align-self: stretch; /* .footer-track-slot is `width: var(--daw-col-w)`, which is the gutter width it was built to fill in the footer. Here the column decides the width, so @@ -4565,7 +5093,11 @@ input, textarea { font-family: inherit; } out of the composer on a narrow window. */ width: auto; min-width: 0; - margin: 0 14px; + /* No side margins. In the middle of the row it needed air on both sides; + at the end, the rule and the grid gap already set it off on the left and + the bar's padding on the right, and 28px of margin was 28px taken from + the meta line, enough to clip the date in German at 1600 (#698). */ + margin: 0; display: flex; flex-direction: column; justify-content: center; @@ -4578,6 +5110,39 @@ input, textarea { font-family: inherit; } border-radius: 8px; } +/* "Now playing", set into the card's top border like a fieldset legend. + On the border rather than as a line of its own: the card's height is already + spent on the title and a two-row meta line (#698). The background is the + bar's, so the label cuts the border instead of sitting on top of it. The type + is the EXTRACT label's, so the two labels on this row read as a pair. */ +#nowPlayingPanel .np-legend { + position: absolute; + top: 0; + /* Over the text column, not the artwork: on the art it would put a band of + bar colour across the top of the picture. */ + left: calc(var(--np-fav) + var(--np-art) + 8px); + transform: translateY(-50%); + padding: 0 5px; + background: var(--panel); + font-size: 10px; + font-weight: 600; + letter-spacing: 0.12em; + line-height: 1; + text-transform: uppercase; + color: var(--muted); + white-space: nowrap; + pointer-events: none; +} +/* .footer-track-slot clips the card, a leftover from the footer gutter it was + built for, and that clip cut the legend's top half off. The lines inside + already clip themselves (title ellipsis, meta line overflow), so this keeps + the same clip on the sides and bottom and opens only the 8px above, which + is the composer's own padding and where the legend sits. */ +#nowPlayingPanel { + overflow: visible; + clip-path: inset(-8px 0 0 0); +} + /* The footer clamps the title to two lines, because its gutter is narrow and a long name would otherwise be cut after three or four words. This card is wide and short, so two lines is what makes it look cramped: the title grows @@ -4606,6 +5171,33 @@ input, textarea { font-family: inherit; } justify-content: flex-start; } #nowPlayingPanel .track-panel-info { flex: 1 1 auto; min-width: 0; } +/* The artist (#699), after the title on the title's own line, in the meta + line's quieter voice. A line of its own would make the card taller than the + bar, and in the meta line it would push the extracted date off the end. + The artist keeps its whole width up to 45% of the line, clipping past that, + and the title takes the rest, clipping as it always has. Same size and line + height as the title, so sharing a baseline adds nothing to the line. */ +#nowPlayingPanel .track-panel-info { + flex-direction: row; + align-items: baseline; + gap: 0; +} +#nowPlayingPanel .track-panel-title { flex: 0 1 auto; min-width: 0; } +#nowPlayingPanel .np-artist { + flex: 0 0 auto; + max-width: 45%; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + font-size: 13px; + font-weight: 500; + line-height: 1.25; + color: var(--muted); +} +#nowPlayingPanel .np-artist::before { + content: "\00b7"; + padding: 0 6px; +} /* A flex item's automatic minimum is its content, all the way down, so one missing min-width anywhere in this chain gives the card a floor it will not go below and the row overflows instead. The title and the meta line both @@ -4623,19 +5215,23 @@ input, textarea { font-family: inherit; } * So it gets a segment of its own, full height, separated by the same hairline * the composer uses between its clusters. The icon then has a place rather than * a position, and the card reads as two parts: what the track is, and the one - * thing you can do to it from here. */ + * thing you can do to it from here. + * + * On the left edge, before the artwork (#699): heart, picture, then the words. + * On the right it sat at the far end of the bar, away from the track it acts + * on, and the text had to stop 56px short to leave room for it. */ #nowPlayingPanel .track-panel-head .daw-fav-btn { position: absolute; - right: 0; + left: 0; top: 0; bottom: 0; transform: none; margin: 0; - width: 46px; + width: var(--np-fav); height: auto; border: none; - border-left: 1px solid var(--border); - border-radius: 0 8px 8px 0; + border-right: 1px solid var(--border); + border-radius: 7px 0 0 7px; background: none; color: var(--muted); } @@ -4647,9 +5243,62 @@ input, textarea { font-family: inherit; } width: 17px; height: 17px; } -/* The text stops where the segment starts. */ -#nowPlayingPanel .track-panel-info, -#nowPlayingPanel .daw-track-meta-line { padding-right: 56px; } +/* The artwork, as a segment like the heart's and right after it: full height, + with a hairline beside it, rather than a 40px square squeezed between + the label and the meta line. It was the smallest thing in the card while + being the quickest way to tell which song is open (#699). + + 76px is the card's inside height (78 less its borders), so the picture is + square and edge to edge. Title and meta line both start after it, which + also puts them on one left edge. That costs the text 76px, so the card's + width cap went up by the same amount; the bar has room to the right. + + Only while the card is wide enough to pay for it. A narrow card, which is a + long language in a window just above the breakpoint that hides the card, + needs those 76px for the extracted date (#698), so below 380px the art goes + back inline at 40px. A container query, not a media query: how wide the + card is depends on the language as much as on the window. + + Container queries style descendants only, which is why the room for the art + is padding on the two text rows rather than on the card itself. And a size + container ignores its content when sizing, so the card fills its column (up + to the cap) instead of shrinking to fit its text. */ +#nowPlayingPanel { + --np-fav: 34px; + --np-art: 76px; + container-type: inline-size; +} +#nowPlayingPanel .track-panel-cover { + position: absolute; + left: var(--np-fav); + top: 0; + bottom: 0; + width: var(--np-art) !important; + height: auto !important; + border-radius: 0; + border-right: 1px solid var(--border); +} +#nowPlayingPanel .track-panel-head, +#nowPlayingPanel .daw-track-meta-line { padding-left: calc(var(--np-fav) + var(--np-art)); } +@container (max-width: 379px) { + #nowPlayingPanel .track-panel-cover { + /* relative, not static: the thumbnail inside is absolutely positioned, + and against a static cover it would fill the whole card. */ + position: relative; + top: auto; + bottom: auto; + left: auto; + width: 40px !important; + height: 40px !important; + border-radius: 7px; + border-right: none; + } + #nowPlayingPanel .track-panel-head, + #nowPlayingPanel .daw-track-meta-line { padding-left: var(--np-fav); } + /* Over the title, clear of the heart and the 40px art: + heart + card padding 14 + art 40 + 6. */ + #nowPlayingPanel .np-legend { left: calc(var(--np-fav) + 60px); } +} /* Source and quality keep their footer colours, which up here is four accents in one short line. Muted to one voice; the stems dot keeps its colour, @@ -4675,7 +5324,10 @@ input, textarea { font-family: inherit; } flex-wrap: wrap; row-gap: 0; line-height: 1.35; - max-height: 2.7em; + /* Two rows at 1.35, plus 2px: at exactly 2.7em the second row's line box + rounded a pixel past the cap and was measured as clipped. A third row + would need another 1.35em, so the headroom never lets one through. */ + max-height: calc(2.7em + 2px); white-space: nowrap; overflow: hidden; } @@ -4782,6 +5434,9 @@ input, textarea { font-family: inherit; } */ @media (max-width: 1460px) { .daw-composer > #nowPlayingPanel { display: none; } + /* The rule only divides the extraction from the card. Without the card it + would divide the row from nothing. */ + .daw-composer > .daw-composer-sep { display: none; } } /* The click-track options with nothing to act on. diff --git a/static/index.html b/static/index.html index 44608e44..517587e0 100644 --- a/static/index.html +++ b/static/index.html @@ -170,6 +170,9 @@ It used to fill the gutter beside the footer timeline. That timeline is gone, and this row had empty space in the middle. --> + + +
@@ -1080,6 +1134,7 @@

StemDeck

+

Song details from AcoustID, MusicBrainz (CC0), Wikipedia (CC BY-SA) and Wikidata (CC0). Band profiles from Discogs, with a Discogs token set. Lyrics from LRCLIB. Transcription by OpenAI Whisper (MIT).

@@ -1100,6 +1155,28 @@

We Recommend

+ + + +