Using shared ptr to track iodesc use#698
Draft
jayeshkrishna wants to merge 7 commits into
Draft
Conversation
Adding test that writes and immediately frees (before file close/sync) I/O decompositions. The decomposition used for read is freed after the file is closed/deleted
We no longer use custom linked lists for I/O decomposition lists. Removing unused/stale remnants (ptr to next) of the old linked list in io_desc_t.
Moving some util functions related to I/O decomposition from pioc_support.cpp to a new file (spio_iodesc_utils.cpp) No change in source code
Converting io_desc_t from a C struct to a simple C++ struct with constructor/destructor/util functions. The io_desc_t is still not a "full fledged" C++ class (this will be done in a future PR). However this minimum change is required to use shared pointers with I/O decomps. * Adding a constructor that manages constructing the io_desc_t object (instead of custom functions that alloc memory and initialize fields) * Also adding a destructor to free decomp and some util functions to free up resources. * All members of the io_desc_t struct is still public since a lot of the code base access the members directly * The member arrays/pointers are still allocated memory explicitly using malloc/calloc (these will be changed in future PRs) * A shared pointer to the io_desc_t, stored in the global io_desc list, now manages the lifetime of I/O decomps * Modified tests accordingly * Modified asserts to account for zero sized local decomp maps
Using C++ constructors and destructors for write cache info. The members are all still public (since other parts of the code access them directly) and will be changed in a future PR
I/O decompositions are separate from files and is used by files when rearranging data (before writing the data out). Since I/O decompositions can be potentially freed before it gets used by a file, we now cache a reference to the I/O decomposition in the file. * I/O decomposition objects are ref counted using shared ptrs * Using a map to cache I/O decomps in files * In the async case cache a ref to I/O decomp in the write info object
Since we now ref count I/O decompositions correctly we no longer need the hack to sync all files before freeing decomps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Using a shared pointer to track I/O decomposition
usage.
This avoids the unnecessary syncs required, to flush
data, when freeing I/O decompositions