Describe the bug
Rapidly creating the same file (as seen in the reproduction section) results in this error:
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#2: Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#1: Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#3: Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#5: Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#2: Access is denied. (os error 5)
[...]
To Reproduce
use object_store::local::LocalFileSystem;
use object_store::path::Path;
use object_store::{Error, ObjectStore, PutMode, PutPayload};
#[tokio::main]
async fn main() {
let store = LocalFileSystem::new_with_prefix("./test_dir").unwrap();
for _ in 0..16 {
let store = store.clone();
tokio::spawn(async move {
loop {
match store
.put_opts(
&Path::parse("test/test.bin").unwrap(),
PutPayload::new(),
PutMode::Create.into(),
)
.await
{
Ok(_) | Err(Error::AlreadyExists { .. }) => (),
Err(err) => println!("{err}"),
}
}
});
}
tokio::time::sleep(std::time::Duration::from_secs_f32(1.5)).await;
}
with
[package]
name = "object-store-test"
version = "0.1.0"
edition = "2024"
[dependencies]
object_store = "0.13.2"
tokio = { version = "1.43.0", features = ["bytes", "default", "fs", "io-std", "io-util", "libc", "macros", "mio", "net", "rt", "rt-multi-thread", "socket2", "sync", "time", "tokio-macros"] }
and creating the test_dir directory, then cargo run.
The tokio features can likely be reduced by a lot, this is just what I copied from the Cargo.toml of the bigger project this occurred in, but I also doubt they are related to the error.
Expected behavior
The only expected error would've been Error::AlreadyExists, but it seemingly often fails before that when creating the staging file (so this is probably unrelated to the PutMode as well and would show up when trying to recreate the file rapidly, though that was not tested).
Additional context
I'm not sure if this may "really" be a bug in Rusts std or maybe even Windows & not object_store; based on a little digging I assume the error comes from CreateFileW, which does mention this:
If you call CreateFile on a file that is pending deletion as a result of a previous call to DeleteFile, the function fails. The operating system delays file deletion until all handles to the file are closed. GetLastError returns ERROR_ACCESS_DENIED.
I'm not sure if trying to rename a file (to the final path) actually counts as a deletion in Windows though, but at least on https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew I didn't find any more info that seems relevant to this exact issue, and I didn't search further than that either.
Describe the bug
Rapidly creating the same file (as seen in the reproduction section) results in this error:
To Reproduce
with
and creating the
test_dirdirectory, thencargo run.The
tokiofeatures can likely be reduced by a lot, this is just what I copied from theCargo.tomlof the bigger project this occurred in, but I also doubt they are related to the error.Expected behavior
The only expected error would've been
Error::AlreadyExists, but it seemingly often fails before that when creating the staging file (so this is probably unrelated to thePutModeas well and would show up when trying to recreate the file rapidly, though that was not tested).Additional context
I'm not sure if this may "really" be a bug in Rusts
stdor maybe even Windows & notobject_store; based on a little digging I assume the error comes fromCreateFileW, which does mention this:I'm not sure if trying to rename a file (to the final path) actually counts as a deletion in Windows though, but at least on https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew I didn't find any more info that seems relevant to this exact issue, and I didn't search further than that either.