Install software on AWS Datapipeline EC2 Resource

Question

I need to use a program called s3cmd in a ShellCommandActivity. How can I do this? Do I have to create a new EC2 Image ( and then install everything thats needed for pipeline etc..) and specify it in the pipeline or is there an existing Pipeline EC2 Image I can add something to? I also tried installing it in the script by running sudo su yum -y install s3cmd

which failed: "sudo: sorry, you must have a tty to run sudo"

Answer 1

If you need custom stuff on your instance, you should build a custom AMI with whatever you want already installed. This works well with data pipelines because it reduces the risk of something failing when you bring up the instance for the step by already having everything installed.

Answer 2

You can use UserData which are custom scripts run by the instance at startup. The process is outlined here