Add support for IAM Roles for Service Accounts in AWS EKS (Web Identity)

[Rafi Aroch-2]

Hi,

IAM Roles for Service Accounts have many advantages when deploying Flink on
AWS EKS.

From AWS documentation:

*With IAM roles for service accounts on Amazon EKS clusters, you can
> associate an IAM role with a Kubernetes service account. This service
> account can then provide AWS permissions to the containers in any pod that
> uses that service account. With this feature, you no longer need to provide
> extended permissions to the worker node IAM role so that pods on that node
> can call AWS APIs.*


As Kubernetes becomes the popular deployment method, I believe we should
support this capability.

In order for IAM Roles for Service Accounts to work, I see two necessary
changes:

   - Bump the AWS SDK version to at least:  1.11.623.
   - Add dependency to AWS STS in order for the assume-role to work.

This is relevant for S3 Filesystem & Kinesis modules.

There is already an issue open:
https://issues.apache.org/jira/browse/FLINK-14881

Can I go ahead and create a pull request to add this?

Thanks,
Rafi

[Stephan Ewen]

This sounds like a good addition.

Can you comment on the jira issue, to have the discussion in one place.
Unless anyone raises concerns, I can assign you the issue then and we could
proceed with a PR.

On Tue, Feb 11, 2020 at 4:10 PM Rafi Aroch <[hidden email]> wrote:

> Hi,
>
> IAM Roles for Service Accounts have many advantages when deploying Flink on
> AWS EKS.
>
> From AWS documentation:
>
> *With IAM roles for service accounts on Amazon EKS clusters, you can
> > associate an IAM role with a Kubernetes service account. This service
> > account can then provide AWS permissions to the containers in any pod
> that
> > uses that service account. With this feature, you no longer need to
> provide
> > extended permissions to the worker node IAM role so that pods on that
> node
> > can call AWS APIs.*
>
>
> As Kubernetes becomes the popular deployment method, I believe we should
> support this capability.
>
> In order for IAM Roles for Service Accounts to work, I see two necessary
> changes:
>
>    - Bump the AWS SDK version to at least:  1.11.623.
>    - Add dependency to AWS STS in order for the assume-role to work.
>
> This is relevant for S3 Filesystem & Kinesis modules.
>
> There is already an issue open:
> https://issues.apache.org/jira/browse/FLINK-14881
>
> Can I go ahead and create a pull request to add this?
>
> Thanks,
> Rafi
>