Question

I have a gzipped file on a local machine and want to load it to Redshift. My command looks like this:

\COPY tablename FROM 's3://redshift.manifests/copy_from_yb01_urlinfo.txt' REGION 'us-east-1' CREDENTIALS 'aws_access_key_id=...;aws_secret_access_key=...' SSH GZIP;

But I get a message "s3:/redshift.manifests/copy_from_yb01_urlinfo.txt: No such file or directory".

But this file even public: https://s3.amazonaws.com/redshift.manifests/copy_from_yb01_urlinfo.txt. Moreover, the user whose credentials I use have a full access to S3 and Redshift: http://c2n.me/iEnI5l.png

And even more weird is the fact that I could perfectly access that file with same credentials from AWS CLI:

> aws s3 ls redshift.manifests
2014-08-01 19:32:13        137 copy_from_yb01_urlinfo.txt

How to diagnose that further?

Just in case, I connect to my Redshift cluster via psql (PostgreSQL cli):

PAGER=more LANG=C psql -h ....us-east-1.redshift.amazonaws.com -p 5439 -U ... -d ...

edit:

Uploaded file to S3 - same error on COPY... And again I uploaded it and ran COPY with same credentials.

\COPY url_info FROM 's3://redshift-datafiles/url_info_1.copy.gz' CREDENTIALS 'aws_access_key_id=...;aws_secret_access_key=...' GZIP;

I am going to despair...

Was it helpful?

Solution 2

Oh.

The fix was to remove backslash in the beginning of the command. Can't remember why I started writing it... Actually I already began writing it when I exported data from local PostgreSQL installation.

This is so stupid) One small rubber duck could have saved me a day or two.

OTHER TIPS

Since you are trying to copy to RedShift using a manifest file, you need to use the MANIFEST command at the end like :

\COPY tablename FROM 's3://redshift.manifests/copy_from_yb01_urlinfo.txt' REGION 'us-east-1' CREDENTIALS 'aws_access_key_id=...;aws_secret_access_key=...' SSH GZIP MANIFEST;
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top