In order for Essentia to launch worker nodes or access data on S3, it requires authentication. This can be provided in three different ways.
aws configure
to save your AWS Access Credentials to a
file (enables launching worker nodes only, not used for S3 access).A downside of the last two options is that the user needs to worry about the AWS Access Credentials, maintain their security, and update them should they change. Therefore, it is recommended to use IAM Roles instead. If you do not have access to create an IAM Role, contact your AWS Administrator.
To create an IAM Role, follow these steps:
Caution
Improperly created roles pose a security risk. The above roles are very liberal in granting permissions to spin up new ec2 instances and access s3 files. Consult with your AWS experts, or contact us if there are any questions or concerns.
Note
If you plan to utilize our Redshift Integration, you need to enable Redshift access in your IAM Role. The following policies both do that:
Users can also grant the master the authorization to create other ec2 instances by logging into their master node and running the command:
aws configure
The user will then need to enter their AWS Access Credentials, which will then be stored in a file which Essentia will read when it needs them.
This method cannot be used to grant S3 access. It is recommended to use IAM Roles instead.
Essentia also offers the ability to specify AWS Access Credentials in the various commands that use them.
This can be a major benefit since it allows users or their administrators to change the level of access to suit the needs of the current analysis, by simply changing the access level of the credentials that were entered or entering new credentials with the desired access level.
Appropriate Commands to Enter Access Credentials:
Creating Worker Nodes for Scalability:
ess cluster create ... --aws_access_key=**ENTER_ACCESS_KEY** --aws_secret_access_key=**ENTER_SECRET_KEY**
Accessing or Writing S3 Data:
ess select ... --aws_access_key=**ENTER_ACCESS_KEY** --aws_secret_access_key=**ENTER_SECRET_KEY**
Note: The Access Credentials that you enter into these commands need to have the correct permissions. I.e. Essentia will not be able to access data in S3 if the user enters Access Credentials into the “ess select” command that only have access to EC2.