AWS EKS Pods Access to AWS Services

AWS EKS Pods Access to AWS Services🌱

There are instances where EKS Pods needs access to AWS services like S3, this tutorial talks about the different services like serviceaccounts, OIDC provider, IAM roles/policies and STS used to accomplish this goal.

Overview 📝

Setup Instructions 🚧

• Create a dummy S3 bucket in your specific region, I am creating it in ca-central-1.

    aws s3api create-bucket \
    --bucket my-bucket-dummy-eks \
    --region ca-central-1 \
    --create-bucket-configuration LocationConstraint=ca-central-1
  

• Create the EKS cluster using the below command which has EC2 nodes as cluster nodes. (It takes approximately 15 minutes for the cluster creation)

    eksctl create cluster --name demo-s3-cluster --region ca-central-1 --with-oidc --nodes 2 --instance-types=t2.micro
  

• Once cluster is created, run a kubectl command to get nodes, you should see the EC2 nodes.

    kubectl get nodes -o=wide
  

• Let’s create the OIDC provider, policies and roles from the console for the EKS cluster (Note: by default it will be create if you use the above EKS creation command)
  • Go to the EKS console and get the OIDC provider URL of the cluster
  • Configuring the OIDC provider using IAM console (IAM console > Access Management > Identity Providers > Add providers)
  • Create policy to grant access to S3 buckets (IAM console > Access Management > Policies ), I have named it as pod-access-to-s3.

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Sid": "VisualEditor0",
                  "Effect": "Allow",
                  "Action": [
                      "s3:ListBucket",
                      "s3:ListAllMyBuckets"
                  ],
                  "Resource": "*"
              }
          ]
      }
    

  • Create a role and attach the policy created above to it, create the role as per below screenshot
    • Make sure that the trusted policy is as mentioned below, it includes the service account name (which we are yet to create) and the OIDC ID which was created above

        {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Federated": "arn:aws:iam::<<ACCOUNTID>>:oidc-provider/oidc.eks.ca-central-1.amazonaws.com/id/<<OIDC ID>>"
                },
                "Action": "sts:AssumeRoleWithWebIdentity",
                "Condition": {
                    "StringEquals": {
                        "oidc.eks.ca-central-1.amazonaws.com/id/<<OIDC ID>>:aud": "sts.amazonaws.com",
                        "oidc.eks.ca-central-1.amazonaws.com/id/<<OIDC ID>>:sub": "system:serviceaccount:<<NAME SPACE>>:<<SERVICE-ACCOUNT>>"
                    }
                }
            }
        ]
        }        
      

• Let’s create a pod and service account attached to it in the default namespace.
  • Service account needs to be annotated with the role that we have created above:

      apiVersion: v1
      kind: ServiceAccount
      metadata:
      annotations:
          eks.amazonaws.com/role-arn: 
                  arn:aws:iam::891377161155:role/pod-access-s3-role
      name: sa-access-s3
      namespace: default
    

  • Create a Pod which will use the above service account.

      apiVersion: v1
      kind: Pod
      metadata:
      creationTimestamp: null
      labels:
              run: access-s3-pod
      name: access-s3-pod
      spec:
      containers:
      - args:
              - help
      image: amazon/aws-cli
      name: access-s3-pod
      serviceAccount: sa-access-s3
    

  • Once the pod is create run the below command to make sure that the Pod has the web identity token file mount.

      kubectl describe pod access-s3-pod | grep AWS_WEB_IDENTITY_TOKEN_FILE:
    
      #Example output
      #AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token
    

    Testing 🧪

• Let’s test and see if the pod can now access the S3 bucket or not.

    kubectl exec -it access-s3-pod -- sh
    #Run the aws s3 ls to list the buckets
    aws s3 ls
    #You will see the buckets that are present in your account
    #2024-04-20 18:28:50 my-bucket-dummy-eks
    #2024-04-20 17:16:38 s3-bucket-eks-demo
  

  Cleanup 🧹

• Make sure to delete the cluster.

    eksctl delete cluster --name demo-s3-cluster
  

  Resources 📚

• AWS EKS