Azure Data Factory 'Failed Validation' with Folder does not exist
-
07-10-2020 - |
Domanda
I uploaded whole set of .txt
files in my blob storage with folder structure Year/Month/day.
I circled the folder structure.
Then in the pipeline I have following dataset created.
"type": "AzureBlob",
"linkedServiceName": "blob-store",
"typeProperties": {
"fileName": "{Day}.csv",
"folderPath": "adf-data/sales/{Year}/{Month}",
"format": {
"type": "TextFormat",
"columnDelimiter": "\t",
"firstRowAsHeader": true
},
"partitionedBy": [
{
"name": "Year",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "yyyy"
}
},
{
"name": "Month",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "MM"
}
},
{
"name": "Day",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "dd"
}
}
]
},
"availability": {
"frequency": "Day",
"interval": 1
},
"external": true,
"policy": {}
}
When I run the pipeline I am getting an error with folder does not exist.
Question: What can I do to troubleshoot this further or fix?
Soluzione
I had wrong file extension. Error message was misleading. Instead of cannot find file I thought the error was about missing folder.
I had to change this
"typeProperties": {
"fileName": "{Day}.csv",
"folderPath": "adf-data/sales/{Year}/{Month}",
"format": {
"type": "TextFormat",
"columnDelimiter": "\t",
"firstRowAsHeader": true
},
To (change file extension from csv to txt)
"typeProperties": {
"fileName": "{Day}.txt",
"folderPath": "adf-data/sales/{Year}/{Month}",
"format": {
"type": "TextFormat",
"columnDelimiter": "\t",
"firstRowAsHeader": true
},
Autorizzato sotto: CC-BY-SA insieme a attribuzione
Non affiliato a dba.stackexchange