Bridging the Gap Between RDASApp Needs and JEDI Development

[guoqing-noaa]

PR reviews and merges in RDASApp are typically completed within a few days, whereas merging those changes into the official JEDI repositories often takes several months (and in extreme cases, over a year).

To bridge this practical gap, we propose several possible approaches:

1. Expansion of JCSDA Review Capacity

JCSDA could allocate additional resources to accelerate the review and integration of PRs, with a target average turnaround time of less than one month. However, given current resource constraints, the feasibility of this option appears limited.

2. Make all JEDI private repositories public with dedicated NOAA branches

JCSDA could make all JEDI private repositories public, allowing us to create dedicated branches (e.g., rrfs or noaa) in needed submodules. These branches would be primarily reviewed and maintained by NOAA developers, with support from JCSDA. This would enable faster progress while upstream PRs are still pending. While doable, this approach may not be feasible in the next few months.

3. Use temporary forks of some JCSDA repositories (recommended)

We can create temporary forks of relevant JCSDA repositories (e.g., SABER, GSIBEC, MPAS-JEDI, FV3-JEDI), with branches such as rrfs or noaa to support our specific needs. Changes can be developed and reviewed through standard PR review workflows within these forks. When syncing with upstream JCSDA updates, we can reapply changes via new PRs as needed. Once the relevant PRs are merged, those fork will be removed from RDASApp.

This approach appears to be the most practical in the near term. It eliminates fragile workaround practices and makes changes easier to review, track, and collaborate on. A similar practice has already been successfully adopted in RRFSx/MPAS-Model. I can provide an example PR in RDASApp to demonstrate this process.

Method 3 reflects a standard and widely adopted industry practice for working with large, slower-moving upstream projects that typically have more rigorous review processes. Representative examples include the Android Open Source Project, Red Hat Enterprise Linux, React developed by Meta Platforms, and the Chromium / Chrome ecosystem developed by Google, etc.

[ShunLiu-NOAA]

@guoqing-noaa Thank you for bringing this inquiry to the RDAS team. I will coordinate with DA management to ensure this is addressed properly.

[guoqing-noaa]

Sorry that I under-communicated the method 3 or 2 listed here and hence it might cause confusions.

What I mean is that we created temporary forks/branches to hold our pending changes so that we may continue to run realtime and retro experiments while the upstream PRs are still under review.

Once those PRs are merged, we will immediately remove those temporary fork/branches from RDASApp and point to the official repos again.

The lifetime of those temporary forks/branches depends on the review speed. Some may be just a few weeks, some may be a few months. These temporary forks/branches also remind us to actively push pending PRs move forward.

We would expect that most time, all JEDI submodules point to the official ones and in the late almost mature stage, we will have no need of those temporary forks/branches.