Question

I uploaded whole set of .txt files in my blob storage with folder structure Year/Month/day.

I circled the folder structure. enter image description here

Then in the pipeline I have following dataset created.

 "type": "AzureBlob",
        "linkedServiceName": "blob-store",
        "typeProperties": {
            "fileName": "{Day}.csv",
            "folderPath": "adf-data/sales/{Year}/{Month}",
            "format": {
                "type": "TextFormat",
                "columnDelimiter": "\t",
                "firstRowAsHeader": true
            },
            "partitionedBy": [
                {
                    "name": "Year",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "yyyy"
                    }
                },
                {
                    "name": "Month",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "MM"
                    }
                },
                {
                    "name": "Day",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "dd"
                    }
                }
            ]
        },
        "availability": {
            "frequency": "Day",
            "interval": 1
        },
        "external": true,
        "policy": {}
    }

When I run the pipeline I am getting an error with folder does not exist.enter image description here

Question: What can I do to troubleshoot this further or fix?

Was it helpful?

Solution

I had wrong file extension. Error message was misleading. Instead of cannot find file I thought the error was about missing folder.

I had to change this

 "typeProperties": {
            "fileName": "{Day}.csv",
            "folderPath": "adf-data/sales/{Year}/{Month}",
            "format": {
                "type": "TextFormat",
                "columnDelimiter": "\t",
                "firstRowAsHeader": true
            },

To (change file extension from csv to txt)

"typeProperties": {
            "fileName": "{Day}.txt",
            "folderPath": "adf-data/sales/{Year}/{Month}",
            "format": {
                "type": "TextFormat",
                "columnDelimiter": "\t",
                "firstRowAsHeader": true
            },
Licensed under: CC-BY-SA with attribution
Not affiliated with dba.stackexchange
scroll top