Skip to content

Implement functions and classes in ngen to produce Catchment output netcdfs [NGWPC-10597]#177

Draft
sivasankkar wants to merge 8 commits into
developmentfrom
ngwpc_10597_netcdf_creation
Draft

Implement functions and classes in ngen to produce Catchment output netcdfs [NGWPC-10597]#177
sivasankkar wants to merge 8 commits into
developmentfrom
ngwpc_10597_netcdf_creation

Conversation

@sivasankkar
Copy link
Copy Markdown

@sivasankkar sivasankkar commented Apr 29, 2026

Enable functionality in ngen to write all catchment outputs to a netcdf file for non-MPI ngen runs. This PR is created following NGWPC-9011 PR (#106). However, that didn't make it into development because it remained in draft. We squashed all earlier commits and rebased with ngen development for this PR. This functionality does not work on MPI runs.

Additions

  • NetCDFCreator.hpp - header file for creating a class that can handle all netcdf operations.
  • NetCDFCreator.cpp - class file for methods/functions to handle all netcdf operations.
  • NetCDFCreatorTest.cpp - class containing two unit tests - one for the catchment and another for the output values.

Removals

Changes

  • NetCDFCreator.hpp and cpp: Functions to create and write catchment output variables to netcdf file.
  • Bmi_Formulation.hpp: Added get_output_variable_units and set_output_variable_units function to obtain the output variable units to write to netcdf attributes.
  • Bmi_Module_Formulation.cpp and Bmi_Multi_Formulation.cpp: Set the output variable units read from realization config.
  • NgenSimulation.hpp: Included netcdfcreator header and pointer for that class.
  • NgenSimulation.cpp: Added a function call to write the output variable values to netcdf after every time step.
  • Formulation_Manager.hpp: Added a function to retrieve all catchment formulations for iterating through for netcdf creation.
  • Layer.hpp and cpp: Added a function to gather catchment outputs to a vector for each timestep.
  • NGen.cpp: Added a message for MPI runs and a function call to create the NetCDF writer/creator instance for non-MPI runs.
  • CMakeFiles for ngen and test - Created a library for the NetCDFCreator and set up target link libraries appropriately.

Testing

  1. Tested with CNF data with 67 catchments. The NetCDF output file was created successfully for non-MPI runs.
  2. Appropriate log message was added for MPI runs.
  3. Appropriate log message was added when no output variables were mentioned in the realization config.
  4. Unit tests

Screenshots

Notes

Todos

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

  • Linux

@sivasankkar sivasankkar force-pushed the ngwpc_10597_netcdf_creation branch from cae217c to 61be6c6 Compare April 29, 2026 20:49
@sivasankkar sivasankkar marked this pull request as draft April 29, 2026 21:13
commit 971c830
Author: siva.selvanathan

Added checks for MPI and output variables in realization to prevent NetCDF creation. Added appropriate log messages.

commit b82797b
Author: siva.selvanathan
Date:   Mon Jan 26 21:46:41 2026 -0500

    Added a couple of macro If blocks to prevent attempts to write to a single NetCDF in parallel fashion.

commit cf5e694
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Mon Jan 26 14:52:17 2026 -0500

    Included num_proc if clause to prevent MPI Barrier being called for a single process. Also, removed an unused function from the test class.

commit f4b8819
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Mon Jan 26 11:29:36 2026 -0500

    Unit tests for netcdf creators. This build does not support MPI functionality.

commit fff50aa
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Tue Jan 13 21:18:32 2026 -0500

    Added dependent libraries to test_ngen_simulation for a successful build.

commit 915c0c3
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Tue Jan 13 17:24:56 2026 -0500

    Changes made to enable MPI for netcdf writing.

commit fc5eec5
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Mon Jan 12 09:19:57 2026 -0500

    Made changes to enable MPI for catchment NetCDF creation. Build is still failing.

commit 43fb963
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Sun Jan 11 21:11:14 2026 -0500

    Added pybind include directory to check pipeline build

commit 53f7022
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Sun Jan 11 17:14:34 2026 -0500

    NetCDF for catchment outputs funcitonality implemented.

commit 74a624a
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Wed Dec 24 09:17:12 2025 -0500

    Included proper headers and target directories to CMakeFile to make the program compilable.

commit 3dcac1a
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Tue Dec 23 14:26:04 2025 -0500

    Removed all class variables and constructor implementation for NetCDFCreator. Currently, the code is skeletal with the focus on getting a successful compile.

commit 4358f6a
Merge: 6fcb180 b44c038
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Thu Dec 18 13:44:24 2025 -0500

    Merge branch 'ssn_9011_netcdf_for_catchments' of https://github.com/NGWPC/ngen into ssn_9011_netcdf_for_catchments

commit 6fcb180
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Thu Dec 18 12:26:03 2025 -0500

    first commit with new netcdfcreator class

commit b44c038
Author: siva.selvanathan <sselvanathan_IE@Dewberry.com>
Date:   Thu Dec 18 12:26:03 2025 -0500

    first commit with new netcdfcreator class
… changes to NgenSimulation and NetCDF Data provider. The old C++ class NetCDFCreator hasn't been removed yet.
@sivasankkar sivasankkar force-pushed the ngwpc_10597_netcdf_creation branch from 1f1199b to f84be25 Compare May 13, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant