Skip to content

Feature Request: Native SWMR support for concurrent read/write #60

@14NGiestas

Description

@14NGiestas

Following my previous contributions (implementing iterate and visit), I've identified that h5fortran currently lacks high-level abstractions for SWMR (Single Writer Multiple Reader).
In scenarios where a simulation (writer) and a visualizer (reader, e.g., via h5py) access the same file, the default HDF5 locking mechanism often leads to crashes or file access errors. While this can be solved using low-level use hdf5 calls, it breaks the high-level philosophy of this library.
I am currently facing this exact issue: my visualization script (running in parallel with the main simulation) frequently crashes because of file locking/metadata contention. Providing a high-level SWMR interface would solve this real-world stability problem for users doing live data monitoring.
I'm opening this issue to keep track of the requirements for native SWMR support. Key components would include:

  • File Access: Adding a swmr logical flag to %open() or %create(). This would internally handle H5Pset_libver_bounds(fapl, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST) and the corresponding SWMR access flags.
  • Writer Side: Exposing a %flush() method for datasets (wrapping h5dflush_f) to ensure metadata/data consistency is pushed to disk so readers can see updates.
  • Reader Side: Implementing a %refresh() method (wrapping h5drefresh_f) to update the object's view without reopening the file.

Technical Context:
For the writer, the sequence usually requires:

  1. Setting libver = latest in FAPL.
  2. Creating the file and datasets.
  3. Calling h5fstart_swmr_write_f.

For the reader:

  1. Opening with H5F_ACC_RDONLY + H5F_ACC_SWMR_READ.
  2. Periodic calls to h5drefresh_f.

I believe adding this would make h5fortran much more robust for real-time data monitoring. I'm documenting this here to track the feature progress and discuss the best API design before implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions