support file lock/unlock

[NathanFreeman]

Hello, I saw the discussion about file locking in issue #85. I'd like to confirm if the current liburing already supports file locking operations? If not, are there any other recommended approaches to achieve this? Thank you.

[axboe]

io_uring doesn't currently support file locking.

[krisman]

Looking into this...

If we call vfs without !FL_SLEEP, it could be a trivial inline execution. If it failed with FILE_LOCK_DEFERRED, throw into a io-wq. sounds reasonable?

I was thinking only BSD locks but @YoSTEALTH seems to care about POSIX locks, which I thought was less relevant today. Is that a requisite?

[axboe]

Even FS_SLEEP should be fine, as there's already notification mechanisms in there. And that's what io_uring would need to either retry or grant the lock. As a side note - always say no to io-wq. It's usually the lazy approach, which ends up being the slower approach too. Sometimes refactoring is needed on lower levels to make this feasible, but once done, it's a SO much better solution than doing the "ah that's fine, just punt anything that sleeps to io-wq". See the futex series I did back in the day for an example of how to do it right. That one could've taken the io-wq approach too, but it'd be so much more useless if it had done that.

I have some basics for file locking, I'll push them to a branch later today.

[axboe]

Kernel code: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux.git/log/?h=io_uring-filelock
liburing support (prep+man) and test cases: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/liburing.git/log/?h=io_uring-filelock

[krisman]

As a side note - always say no to io-wq. It's usually the lazy approach, which ends up being the slower approach too.

IIUC, the strategy is letting the subsystem notify io_uring when it is ready, queue a TW that will reexecute the request. Thanks, the file locking itself is a good example.

My question is if your plan is eventually doing this for every operation and dropping iowq, or if there are cases were we will never care (such as bind(2))?

Sometimes refactoring is needed on lower levels to make this feasible, but once done, it's a SO much better solution than doing the "ah that's fine, just punt anything that sleeps to io-wq". See the futex series I did back in the day for an example of how to do it right. That one could've taken the io-wq approach too, but it'd be so much more useless if it had done that.

I have some basics for file locking, I'll push them to a branch later today.

You said you have some basics, but I was taking a look and it seems fairly complete, modulo the process lock (posix flavor) which I don't know if it is relevant at all. Do you plan to send this yourself, do you want me to take over? what's your plan?

[axboe]

Feel free to take it over, and even prune out bits you don't think are interesting. I'm about to head on some vacation and got plenty of other things on my plate, would be happy to see someone else run with this, if they are interested!

[axboe]

My question is if your plan is eventually doing this for every operation and dropping iowq, or if there are cases were we will never care (such as bind(2))?

I don't think we can ever get there. Sometimes things just need to block, for resource reasons. Like reading or writing to storage, if you run out of tags, you either have to error the IO, or block waiting for some. Maybe that could be a callback at some point too and we can get rid of io-wq there as well, that'd be great. Eg instead of returning -EAGAIN when we're out of tags, if the caller provides the right information, -EIOCBQUEUED could be returned even though the IO isn't actually queued. Or some relevant error code, the exact value isn't important, but it should mean "currently busy, will trigger the callback when you can retry". And then you could drive any storage IO without needing io-wq, even when it's busy.

Buffered writes is another example. Most of them can be done inline, but sometimes balance_dirty_pages() needs calling, and we can only do that from io-wq as it blocks. It's meant to throttle the caller to avoid dirtying too much data. But there it'd also be better if it would simply return a value where where we know we'd get a callback when we can issue the next write.

But we're probably stuck with it for a while. I just don't want to add new cases for it, unless it's absolutely necessary. And particularly not when the the underlying syscall or kernel code can just be taught to do the right thing - arm a callback rather than block.