Aws Datapipeline: List content of Output Bucket in ShellCommandActivity

Question

How can I list the files which are contained in my output Bucket in a Shell Script? ls ${OUTPUT1_STAGING_DIR} does not work, as I get the message that there's no file or directory by this name. I am sure there is an easy way to do this but I can't seem to find a solution.

Answer 1

From my experience using DataPipeline, that is not supported. You can only read from input bucket directory. The output bucket directory is just a place where you can write files to that will later on be copied over into S3.