Skip to content

Deprecated Apache Beam Version Causing Error in weather-dl tool. #362

Description

@dabhicusp

I encountered an issue while using the following command to download data from cds using weather-dl tool. Since this is an urgent task, I would greatly appreciate any assistance in resolving this error.

command used : python "weather_dl/weather-dl" weather-tools/config_files/*.cfg --runner DataflowRunner --project $PROJECT --region $REGION --temp_location "gs://$BUCKET/tmp/" --disk_size_gb 260 --sdk_container_image "gcr.io/*****/miniconda3-beam:weather-tools-with-aria2" --experiment use_runner_v2

I performed the above command in a fresh environment of weather-tools, and within the config_files directory, there is only one file with the following content:

[parameters]
client=cds
dataset=reanalysis-era5-complete
target_path=gs://gcp-public-data-arco-era5/raw/ERA5GRIB/HRES/Daily/{date:%%Y/%%Y%%m%%d}_hres_dve.grb2
partition_keys=
    date

[parameters.sahil]
api_url=https://cds.climate.copernicus.eu/api/v2
api_key=****:****-****-****

[selection]
class=ea
stream=oper
expver=1
type=an
levtype=ml
levelist=1/to/137
date=2023-04-01/to/2023-04-30
time=00/to/23
param=138/155

while researching the above issue I discovered that the version of Apache Beam used in tool is deprecated.

The error I got is ::
e1
e2
e3
e4
e5

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions