Skip to content

HTTP to Azure Blob transfers data but transfer state transitions to TERMINATED on consumer side #425

Description

@simonlet

Bug Report

Describe the Bug

When doing a transfer from HTTP data source without the name property set in the dataAddress to a Azure Blob data sink, the data is successfully transferred, but the transfer is displayed as TERMINATED on the consumer side.

Expected Behavior

Since the transfer was successful, the state should be COMPLETED.

Observed Behavior

On the consumer side, the transfer state is TERMINATED.

Steps to Reproduce

Provider:

  1. Create an HTTP endpoint data source asset without setting the name property (see json body below)
  2. Create a data offer using that asset

Consumer:
3. Negotiate that data offer
4. Start a transfer to Azure Blob storage data sink.
5. See that data is transferred but the state goes to TERMINATED.

Context Information

  • Used version: Connector version 0.14.0 (should also happen on main)

Detailed Description

The body to create the asset with the Management API (see that name is commented out):

POST /v3/assets

{
  "@context": {
    "@vocab": "https://w3id.org/edc/v0.0.1/ns/",
    "dct": "http://purl.org/dc/terms/"
  },
  "@type": "Asset",
  "@id": "{{ASSET_ID}}",
  "dataAddress": {
    "@type": "DataAddress",
    // "name": "http-source-name",
    "type": "HttpData",
    "baseUrl": "{{DATA_SOURCE_URL}}",
    "method": "GET"
  },
  "properties": {
    "http://purl.org/dc/terms/title": "{{ASSET_ID}}"
  }
}

Possible Implementation

The problem is that the HTTP data source in our case does not have a name property. Unfortunately, this is also not enforced by the EDC really. Neither in the Java Code nor when calling the Management API.

The name is taken from the name property of the data address:
https://github.com/eclipse-edc/Connector/blob/release/0.14.0/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSourceFactory.java#L73

Here, you can see that the name of the data address is not enforced:
https://github.com/eclipse-edc/Connector/blob/release/0.14.0/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSource.java#L154

This is where the HTTP data source name is then consumed. At that point it is null, which results in the error:

Solutions

I think, there are two possible solutions:

  • Enforce that every data source must have a name
  • Provide a default if the data source does not have a name

Metadata

Metadata

Assignees

No one assigned

    Labels

    wontfixThis will not be worked on

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions