Prevent qbittorrent from re-downloading files after 'arr upgrade

[mokahless]

I've had everything set up for a while but recently I've started noticing double files in movie folders. Looking back over the downloads, and what's where, and seeing nothing about radarr "re-importing" (it shouldn't anyway, based on the rules, plus it still shows the better one in it's webUI) I'm pretty sure I know what's going on and I'm really not sure how to solve it.

1. Radarr downloads a Web-DL
2. Radarr imports the Web-DL
3. Qui during its dirscan, discovers the Web-DL in the movies folder (I do not use hard links. tmps from temp folder are deleted, then the plan was to eventually have them automatically re-added using qui dirscan)
4. Qui grabs the torrent files for a bunch of trackers and all is good.
5. Bluray is released
6. Radarr upgrades the Web-DL to a bluray.
7. Eventually, someone tries to download the Web-DL from one of the dirscan cross-seeds. This triggers qbit to re-download the Web-DL directly to the movie's folder, now existing in tandem with the upgrade.

So, not categorized as an "issue" because I'm not sure what really should change in the setup to fix this. I could have sworn the behaviour of qb 4.xx was to error out and stop any torrent where the files "disappeared" but this seems to have changed in 5.xx.

Edit: so I think maybe this can be resolved in qui itself with the addition of a "only process items NOT changed in the last" under the dirscan section, next to the existing "only process items changed in the last" or by converging said options into one "only process items changed in this moving range"

[Twiglet1022]

This would be solved by using hardlinks. Is there a reason you don’t use hardlinks?

If you configured qui to create a hardlinked copy to add to qbit, then when radarr deletes the original during the upgrade, qbit will still have the data and be able to seed it. Qui automations (look for outside_qbittorrent in the docs) allow you to identify files where radarr has deleted the other copy so you can do the cleanup from within qui.

[mokahless]

You can't hardlink across separate filesystems, partitions, volumes or mounts. I expose the directories of films/shows/mediacategory/temporarytorrents through mounts in proxmox CT, which are then exposed as mounts to docker. Even if I did want to manually rejigger everything to support hardlinks, the current setup is different ZFS volumes for each media category. Also, I simply don't want to mentally/managerially deal with hardlinks.

Looking at the docs, it's great the devs have added such a feature and it definitely works for one workflow, just not mine.

There are a few additions to qui options that would be helpful for other things but also help me work around this, but I think the heart of the problem is the qbittorrent behaviour that auto-re-downloads when the file is removed. This was complained about back in 2018 looking at the github, and fixed, but seems to have returned with 5.x. Though no one seems to be talking about it.

I haven't fully delved into qui automations. Do you or anyone think it might be possible to auto pause torrents if the file disappears somehow? Edit: I found "if state is missing files then pause." Hoping however it checks this and however qbittorrent reports will allow time for the automation to work before any downloads are attempted. Fastest I can run it is every 60 seconds.

[Twiglet1022]

Fair enough, no hardlinks.

So if I've understood correctly, when you add a movie via radarr, you get two copies of it. One in qbit's download directory, and another where you save you media. Dir scan then runs at some point, detects the movie in your media directory, links that file to a bunch of cross-seeds in qbit and all is good... until radarr upgrades your movie, deleting the movie from your media folder. Eventually, someone tries to download the movie from you, qbit realises you don't have the file anymore, and it starts downloading the movie back into you media folder.

The missing files detection thing might work, assuming it runs independently of qbit. That part's not clear from the docs.

Couple of other things you could try:

• Instead of using dir scan, enable search on completion. That way qui will will use the file that's in its download folder to cross-seed to other trackers, safe from any actions taken by radarr.
• If you really want to use dir scan, you could set it to scan qbit's download directory instead of your media directory. But search on completion will be more efficient in terms of making fewer API calls to your trackers because it just runs once when the file is added.

[mokahless]

Thanks a bunch for reading and your responses to this thread. I really appreciate it.

Yeah, that's mostly correct. Unfortunately, the only difference is I don't keep two copies. Everything in the download directory gets autodeleted by qbittorrent after a couple months, so I can't use dirscan directly on there, it has to be on the media directories.

Right now, I think my easiest solution is to pay more attention to the auto added torrents and just not start any that are recent movies and aren't blurays (I've had no issues, but for an abundance of caution, I've been setting all auto cross-seeds to start paused). I have a few left over that can be enough to test this "missing files" automation.

Despite all the minor issues and room for more features, the dirscan is still a godsend. With thousands of movies and shows downloaded long before that are no longer in my qbittorrent instance on any tracker, I'll be keeping the dirscan going slowly for a long time.

[Twiglet1022]

Okay I think I've grasped your setup now. So you start off with two copies, but after a couple of months you remove the ones in qbit's download directory so that you don't waste space on duplicates. Eventually you end up seeding everything directly from your media library thanks to dir scan.

Honestly, the solution you have with dir scan seems like a pretty reasonable workaround given your constraint of not using hardlinks.

You just need some way of informing qbit that the file is gone when radarr upgrades the movie.

If it works, that automation using "has missing files" would probably be the easiest way to do it, but only if it detects the missing files before qbit tries and fails to seed those missing files and instead starts downloading them again.

If that doesn't work here's another idea...

• Under your dir scan settings, make sure that you are setting a tag on all cross-seeds from dir scan to identify them e.g. x-dir
• Now set up an automation to change the download speed of all torrents with x-dir tag to 0B/s that runs say once an hour (hopefully nothing gets upgraded less than an hour after you download it).

That should prevent you from downloading any additional data on your cross-seeds from dir scan. I'm not 100% sure if 0B/s actually gets treated as 0B/s or infinity, so you could limit it to download at 1B/s instead.

To finishing this off and fully automate it, add another automation that deletes your torrents with tag x-dir if they have less than 100% progress, and you'll never need to manually intervene again.

[mokahless]

Yeah, the missing files one isn't working. It's probably far too short a window.

Those are some fantastic suggestions for the existing automations though.

For anyone coming into this in a similar situation:

• You can only limit down to 1KiB/s. qbittorrent won't let you limit if it's not currently downloading, but qui is reporting success in limiting them anyway so I guess the API doesn't have that limitation
• I throw all new auto-added DirScans into a temporary xseed category, then manually enable them and throw them into tracker categories. This is due to Dir Scan doesn't rename/ handle directories like regular scan #1591 whereby I need to manually rename folders or fix locations. They are all tagged DirScan.
• So I've set up 1. if completed and not in "xseed" category and tagged DirScan, every 15 min, limit to 1KiB/s and 2. if <100% and tagged DirScan and not in category "xseed," pause it every 5 min.
• Pausing and unfortunately interaction is to account for seedtimes, in case I autofetch a webdl a day before the BD replaces it or something. While most trackers allow something like 10% downloaded before HnR, a few don't so even that 1KiB/s for 5 min will trigger it. With the pause, I can filter for "problem" torrents once a week or so for manual review.