EDC as Provider with other DSP compliant connector as consumer - help needed

[IgorBalog-Eng]

Hello,

I'm testing and verifying our connector against Samples EDC connectors and have following usecase:

• started both consumer and provider connectors following instructions in Samples project
• contract negotiation and data transfer (pull) works correct EDC-EDC
• when I try to use our connector to act as consumer, I'm sending following request

curl --location 'http://localhost:19194/protocol/transfers/request' \
--header 'Authorization: {"region":"eu","audience":"http://localhost:8090","clientId":"consumer"}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "@context": {
        "dspace": "https://w3id.org/dspace/v0.8/",
        "dcat": "http://www.w3.org/ns/dcat#",
        "odrl": "http://www.w3.org/ns/odrl/2/",
        "dct": "http://purl.org/dc/terms/",
        "@vocab": "https://w3id.org/edc/v0.0.1/ns/"
    },
    "dspace:consumerPid": "urn:uuid:f94f263a-6050-4c03-a0ea-67d4893db567",
    "dspace:agreementId": "cb9f428e-6d08-4b3a-b9ba-78b16c61c179",
    "dct:format": "example:HTTP_PULL",
    "dspace:callbackAddress": "http://localhost:8090",
    "@type": "dspace:TransferRequestMessage"
}'

consumerPid is autogenerated and agreementId is obtained from negotiation (if agreement is not correct I get TransferErrorMessage saying agreement not ok)

Response received is

{
    "@id": "da788be0-f60d-47e7-a6be-ae6cf1b58678",
    "@type": "dspace:TransferProcess",
    "dspace:state": "INITIAL",
    "dspace:providerPid": "da788be0-f60d-47e7-a6be-ae6cf1b58678",
    "dspace:consumerPid": "urn:uuid:f94f263a-6050-4c03-a0ea-67d4893db567",
    "@context": {
        "@vocab": "https://w3id.org/edc/v0.0.1/ns/",
        "edc": "https://w3id.org/edc/v0.0.1/ns/",
        "dcat": "http://www.w3.org/ns/dcat#",
        "dct": "http://purl.org/dc/terms/",
        "odrl": "http://www.w3.org/ns/odrl/2/",
        "dspace": "https://w3id.org/dspace/v0.8/"
    }
}

while in provider logs I have following:

←[0;34mDEBUG 2024-09-25T15:42:21.3810235 [DataPlaneSelectorManagerImpl] DataPlaneInstance d3377496-1524-461f-a072-7119e1c0afee is now in state AVAILABLE←[0m
←[0;34mDEBUG 2024-09-25T15:42:45.2584542 DSP: Incoming CatalogRequestMessage for class org.eclipse.edc.connector.controlplane.catalog.spi.Catalog process←[0m
←[0;34mDEBUG 2024-09-25T15:43:21.886458 [DataPlaneSelectorManagerImpl] DataPlaneInstance d3377496-1524-461f-a072-7119e1c0afee is now in state AVAILABLE←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.3444126 DSP: Incoming TransferRequestMessage for class org.eclipse.edc.connector.controlplane.transfer.spi.types.TransferProcess process←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.3464106 TransferProcess da788be0-f60d-47e7-a6be-ae6cf1b58678 is now in state INITIAL←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6065026 [TransferProcessManagerImpl] TransferProcess da788be0-f60d-47e7-a6be-ae6cf1b58678 is now in state PROVISIONING←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6074946 TransferProcess: ID da788be0-f60d-47e7-a6be-ae6cf1b58678. Provisioning←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6111242 [TransferProcessManagerImpl] TransferProcess da788be0-f60d-47e7-a6be-ae6cf1b58678 is now in state PROVISIONED←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6151232 [TransferProcessManagerImpl] TransferProcess da788be0-f60d-47e7-a6be-ae6cf1b58678 is now in state STARTING←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.617129 TransferProcess: ID da788be0-f60d-47e7-a6be-ae6cf1b58678. Initiate data flow←[0m
←[0;31mSEVERE 2024-09-25T15:43:54.6181257 TransferProcess: ID da788be0-f60d-47e7-a6be-ae6cf1b58678. Fatal error while Initiate data flow. Error details: Unable to process transfer da788be0-f60d-47e7-a6be-ae6cf1b58678. No data flow controller found←[0m
←[0;33mWARNING 2024-09-25T15:43:54.6188008 Unable to process transfer da788be0-f60d-47e7-a6be-ae6cf1b58678. No data flow controller found←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6197248 [TransferProcessManagerImpl] TransferProcess da788be0-f60d-47e7-a6be-ae6cf1b58678 is now in state TERMINATING←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6217238 TransferProcess: ID da788be0-f60d-47e7-a6be-ae6cf1b58678. Terminate data flow←[0m
←[0;31mSEVERE 2024-09-25T15:43:54.6257219 TransferProcess: ID da788be0-f60d-47e7-a6be-ae6cf1b58678. Fatal error while Terminate data flow. Error details: Unable to process transfer da788be0-f60d-47e7-a6be-ae6cf1b58678. No data flow controller found←[0m
←[0;33mWARNING 2024-09-25T15:43:54.6348855 Unable to process transfer da788be0-f60d-47e7-a6be-ae6cf1b58678. No data flow controller found←[0m
←[0;34mDEBUG 2024-09-25T15:43:54.6382664 [TransferProcessManagerImpl] TransferProcess da788be0-f60d-47e7-a6be-ae6cf1b58678 is now in state TERMINATED←[0m
←[0;34mDEBUG 2024-09-25T15:44:22.3591814 [DataPlaneSelectorManagerImpl] DataPlaneInstance d3377496-1524-461f-a072-7119e1c0afee is now in state AVAILABLE←[0m

From logs provided, I was not able to figure out what was not correct. Following Dataspace Protocol documentation and example requests provided, I'm guessing that request and order of the requests are OK.

Expected behavior is that provider sends TransferStartMessage on callbackAddress provided and I can get information from this message to get the artifact (containing DataAddress elements).

Any help is appreciated.

[ndr-brt]

this looks to be the potential culprit:

"dct:format": "example:HTTP_PULL",

the format needs to be in the format <destination_type>-<flow_type>, so, in your case, if you want to achieve an http pull, the format should be: HttpData-PULL (like it's done in the samples: https://github.com/eclipse-edc/Samples/blob/main/transfer/transfer-02-consumer-pull/resources/start-transfer.json#L11)

[IgorBalog-Eng]

Yes, that was the case.
Looks I got lost in copy-paste from different files, browsers, json files.

Thanks for prompt response.