Bug Report
Describe the Bug
When doing a transfer from HTTP data source without the name property set in the dataAddress to a Azure Blob data sink, the data is successfully transferred, but the transfer is displayed as TERMINATED on the consumer side.
Expected Behavior
Since the transfer was successful, the state should be COMPLETED.
Observed Behavior
On the consumer side, the transfer state is TERMINATED.
Steps to Reproduce
Provider:
- Create an HTTP endpoint data source asset without setting the
name property (see json body below)
- Create a data offer using that asset
Consumer:
3. Negotiate that data offer
4. Start a transfer to Azure Blob storage data sink.
5. See that data is transferred but the state goes to TERMINATED.
Context Information
- Used version: Connector version 0.14.0 (should also happen on main)
Detailed Description
The body to create the asset with the Management API (see that name is commented out):
POST /v3/assets
{
"@context": {
"@vocab": "https://w3id.org/edc/v0.0.1/ns/",
"dct": "http://purl.org/dc/terms/"
},
"@type": "Asset",
"@id": "{{ASSET_ID}}",
"dataAddress": {
"@type": "DataAddress",
// "name": "http-source-name",
"type": "HttpData",
"baseUrl": "{{DATA_SOURCE_URL}}",
"method": "GET"
},
"properties": {
"http://purl.org/dc/terms/title": "{{ASSET_ID}}"
}
}
Possible Implementation
The problem is that the HTTP data source in our case does not have a name property. Unfortunately, this is also not enforced by the EDC really. Neither in the Java Code nor when calling the Management API.
The name is taken from the name property of the data address:
https://github.com/eclipse-edc/Connector/blob/release/0.14.0/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSourceFactory.java#L73
Here, you can see that the name of the data address is not enforced:
https://github.com/eclipse-edc/Connector/blob/release/0.14.0/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSource.java#L154
This is where the HTTP data source name is then consumed. At that point it is null, which results in the error:
|
builder.put(ORIGINAL_NAME, part.name()) |
Solutions
I think, there are two possible solutions:
- Enforce that every data source must have a name
- Provide a default if the data source does not have a name
Bug Report
Describe the Bug
When doing a transfer from HTTP data source without the
nameproperty set in thedataAddressto a Azure Blob data sink, the data is successfully transferred, but the transfer is displayed as TERMINATED on the consumer side.Expected Behavior
Since the transfer was successful, the state should be
COMPLETED.Observed Behavior
On the consumer side, the transfer state is
TERMINATED.Steps to Reproduce
Provider:
nameproperty (see json body below)Consumer:
3. Negotiate that data offer
4. Start a transfer to Azure Blob storage data sink.
5. See that data is transferred but the state goes to
TERMINATED.Context Information
Detailed Description
The body to create the asset with the Management API (see that
nameis commented out):POST/v3/assets{ "@context": { "@vocab": "https://w3id.org/edc/v0.0.1/ns/", "dct": "http://purl.org/dc/terms/" }, "@type": "Asset", "@id": "{{ASSET_ID}}", "dataAddress": { "@type": "DataAddress", // "name": "http-source-name", "type": "HttpData", "baseUrl": "{{DATA_SOURCE_URL}}", "method": "GET" }, "properties": { "http://purl.org/dc/terms/title": "{{ASSET_ID}}" } }Possible Implementation
The problem is that the HTTP data source in our case does not have a name property. Unfortunately, this is also not enforced by the EDC really. Neither in the Java Code nor when calling the Management API.
The name is taken from the name property of the data address:
https://github.com/eclipse-edc/Connector/blob/release/0.14.0/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSourceFactory.java#L73
Here, you can see that the name of the data address is not enforced:
https://github.com/eclipse-edc/Connector/blob/release/0.14.0/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSource.java#L154
This is where the HTTP data source name is then consumed. At that point it is null, which results in the error:
Technology-Azure/extensions/data-plane/data-plane-azure-storage/src/main/java/org/eclipse/edc/connector/dataplane/azure/storage/metadata/CommonBlobMetadataDecorator.java
Line 42 in 44d010e
Solutions
I think, there are two possible solutions: