Follow-up from #1241.
That PR changes the failed batch-write path to call Abort() instead of Close(), which is the right control flow for avoiding Parquet footer finalization after a write error. One piece it intentionally leaves alone: ParquetFileWriter.Abort() currently only closes the underlying file handle.
For local files this is mostly just an orphan cleanup concern. For object/blob backends, though, it can leave a footer-less object in storage. Readers should not see it because the snapshot is never committed, but the object can still sit around until orphan cleanup catches it.
Expected behavior:
Abort() closes the open writer/file handle.
Abort() removes the incomplete file path from the FileIO.
- If close and remove both fail, both errors are returned, for example with
errors.Join.
This also applies to RollingDataWriter's deferred abort path, since it calls the same writer-level Abort() when a stream exits with an open file.
Suggested tests:
Abort() removes the created file.
- remove errors are returned.
RollingDataWriter abort removes the incomplete current file, if practical.
Follow-up from #1241.
That PR changes the failed batch-write path to call
Abort()instead ofClose(), which is the right control flow for avoiding Parquet footer finalization after a write error. One piece it intentionally leaves alone:ParquetFileWriter.Abort()currently only closes the underlying file handle.For local files this is mostly just an orphan cleanup concern. For object/blob backends, though, it can leave a footer-less object in storage. Readers should not see it because the snapshot is never committed, but the object can still sit around until orphan cleanup catches it.
Expected behavior:
Abort()closes the open writer/file handle.Abort()removes the incomplete file path from theFileIO.errors.Join.This also applies to
RollingDataWriter's deferred abort path, since it calls the same writer-levelAbort()when a stream exits with an open file.Suggested tests:
Abort()removes the created file.RollingDataWriterabort removes the incomplete current file, if practical.