Skip to content

Feature/download from eumds for url adaptor#361

Merged
ecmwf-cobarzan merged 5 commits into
mainfrom
feature/download-from-eumds-for-url-adaptor
May 28, 2026
Merged

Feature/download from eumds for url adaptor#361
ecmwf-cobarzan merged 5 commits into
mainfrom
feature/download-from-eumds-for-url-adaptor

Conversation

@ecmwf-cobarzan

Copy link
Copy Markdown
Contributor

No description provided.

@ecmwf-cobarzan ecmwf-cobarzan force-pushed the feature/download-from-eumds-for-url-adaptor branch 2 times, most recently from 9eeeddd to d856482 Compare May 27, 2026 09:26
@ecmwf-cobarzan ecmwf-cobarzan force-pushed the feature/download-from-eumds-for-url-adaptor branch from d856482 to d8db1e0 Compare May 27, 2026 09:35
@ecmwf-cobarzan ecmwf-cobarzan requested a review from EddyCMWF May 27, 2026 09:57
@EddyCMWF EddyCMWF requested a review from Copilot May 27, 2026 12:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a pluggable “download interface” layer to the URL adaptor so it can download either via the existing multi-URL downloader or via EUMETSAT Data Store (EUMDAC), with EUMDAC downloads post-processed to extract NetCDF files from zip bundles.

Changes:

  • Introduces DownloadInterfaceType with MultiUrlDownloadInterface and EumdacDownloadInterface, plus a get_download_interface(...) factory.
  • Refactors UrlCdsAdaptor.retrieve_list_of_results() to delegate downloading + post-processing to the selected interface (download_interface_type, default multiurl).
  • Adds unit tests for the interface factory and EUMDAC NetCDF-only zip extraction behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
tests/test_50_url_adaptor.py Adds tests for download interface selection and EUMDAC post-processing of zip bundles.
cads_adaptors/adaptors/url/download_interface_type.py Adds the new download interface abstraction with MultiURL + EUMDAC implementations and a factory.
cads_adaptors/adaptors/url/__init__.py Switches UrlCdsAdaptor to use the new download interface abstraction during retrieval.
Comments suppressed due to low confidence (1)

cads_adaptors/adaptors/url/init.py:57

  • retrieve_list_of_results now delegates to download_interface_type (including auth decryption via prepare_credentials() and post_process() behavior), but there’s no test covering that UrlCdsAdaptor selects the correct interface and preserves the previous try_download behavior for the default case. Add a test that monkeypatches get_download_interface (or url_tools.try_download) to assert the correct calls/kwargs for the default multiurl path and for the eumdac path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cads_adaptors/adaptors/url/download_interface_type.py
Comment thread cads_adaptors/adaptors/url/download_interface_type.py Outdated
Comment thread cads_adaptors/adaptors/url/download_interface_type.py Outdated
ecmwf-cobarzan and others added 2 commits May 27, 2026 16:11
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread cads_adaptors/adaptors/url/download_interface_type.py

@EddyCMWF EddyCMWF left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with option to add per-dataset configurable credentials, and passing the e2e tests

EDIT: Just saw your comment re a single account, so ignore the option suggested above.

@ecmwf-cobarzan

Copy link
Copy Markdown
Contributor Author

All cads-e2e-tests passed wrt the URL adaptor.

@ecmwf-cobarzan ecmwf-cobarzan merged commit 7c65a2b into main May 28, 2026
31 of 36 checks passed
@ecmwf-cobarzan ecmwf-cobarzan deleted the feature/download-from-eumds-for-url-adaptor branch May 28, 2026 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants