Create EKS#

Create VPC with terraform#

We can create VPC only with public/private subnet for eks usage.

Create EKS with eksctl#

We can easily create eks cluster with eksct, provided by AWS.

eksctl create cluser --config-file ClusterConfig.yaml

You can find more config samples form the following link eksctl/examples at main · weaveworks/eksctl · GitHub

Deploy EBS CSI driver#

AWS eks has official add of ebs csi driver. You can enable ebs csi driver in aws console or config it in ClusterConfig.yaml before using eksctl to create the cluster.

If you want to add the ebs csi driver to the existing k8s cluster, you can follow the instruction ebs-csi-driver install.

Deploy EFS CSI Driver#

We can add efs csi driver to existing k8s cluster following the instruction efs-csi-driver install

I have summarized the following bash script to create efs csi driver:

#!/usr/bin/env bash
set -eux  
## you only need to create the policy once if you want to add efs csi driver to multiple aws eks clusters
wget https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/docs/iam-policy-example.json -O iam-policy_efs.ignore.json  
aws iam create-policy \  
--policy-name AmazonEKS_EFS_CSI_Driver_Policy \  
--policy-document file://iam-policy_efs.ignore.json || true

region="region"  
cluster_name="cluster-name"  
efs_csi_driver_image="***********.dkr.ecr.region.amazonaws.com" #get from https://docs.aws.amazon.com/eks/latest/userguide/add-ons-images.html
aws_partition="partition"  # aws, aws-us-gov ...
aws_account="aws-account-id"  
eksctl utils associate-iam-oidc-provider --region=$region --cluster=$cluster_name --approve  
eksctl create iamserviceaccount \  
--cluster "$cluster_name" \  
--namespace kube-system \  
--name efs-csi-controller-sa \  
--attach-policy-arn arn:$aws_partition:iam::$aws_account:policy/AmazonEKS_EFS_CSI_Driver_Policy \  
--approve \  
--region $region  
  
helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver/  
helm repo update  
helm upgrade -i aws-efs-csi-driver aws-efs-csi-driver/aws-efs-csi-driver \  
--namespace kube-system \  
--set image.repository=$efs_csi_driver_image \  
--set controller.serviceAccount.create=false \  
--set controller.serviceAccount.name=efs-csi-controller-sa

Deploy aws alb controller#

If you want to integrate k8s resource Ingress with alb, you need to deploy aws-load-balancer-controller first. You can get install instruction here aws-load-balancer-controller install

I have summarized the following bash script to deploy aws alb controller:

#!/usr/bin/env bash  
set -eux

wget https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.7/docs/install/iam_policy.json -O iam-policy_alb.ignore.json  
aws iam create-policy \  
--policy-name AWSLoadBalancerControllerIAMPolicy \  
--policy-document file://iam_policy_alb.ignore.json || true

#!/usr/bin/env bash  
cluster_name="simlab-eks-dev"  
aws_partition="aws"  
aws_account="343549021813"

# recommendation do not specific role-name here, because the role can only be used in the cluster since the role can only be assumed by the specific oidc provider (belongs to the eks cluster)
eksctl create iamserviceaccount \  
--cluster=$cluster_name \  
--namespace=kube-system \  
--name=aws-load-balancer-controller \  
--attach-policy-arn=arn:$aws_partition:iam::$aws_account:policy/AWSLoadBalancerControllerIAMPolicy \  
--approve

helm repo add eks https://aws.github.io/eks-charts  
helm repo update  
helm upgrade -i aws-load-balancer-controller eks/aws-load-balancer-controller \  
-n kube-system \  
--set clusterName="$cluster_name" \  
--set serviceAccount.create=false \  
--set serviceAccount.name=aws-load-balancer-controller

How to access EKS with AWS SSO account#

assign aws SSO user with system:masters group#

eksctl create iamidentitymapping \
 --cluster {cluster-name} \
 --arn "arn:aws:iam::{account-id}:role/{sso-role-name}" \
 --username {username} \
 --group system:masters

assign aws SSO user with custom group#

You can use the following command to assign aws sso user to custom group

eksctl create iamidentitymapping \
 --cluster {cluster-name} \
 --arn "arn:aws:iam::{account-id}:role/{sso-role-name}" \
 --username {username} \
 --group {group-name}

You can change {group-name} with any RoleBindings or ClusterRoleBindings.

Here is one ClusterRoleBinding sample config:

apiVersion: rbac.authorization.k8s.io/v1  
# This role binding allows group "custom-view" to have view ClusterRole
kind: ClusterRoleBinding  
metadata:  
name: custom-view  
subjects:  
# You can specify more than one "subject"  
- kind: Group  
name: custom-view # "name" is case sensitive  
apiGroup: rbac.authorization.k8s.io  
roleRef:  
# "roleRef" specifies the binding to a Role / ClusterRole  
kind: ClusterRole #this must be Role or ClusterRole  
name: view # this must match the name of the Role or ClusterRole you wish to bind to  
apiGroup: rbac.authorization.k8s.io

How to get sso-role-name#

You can use browser to login with aws sso. And in sso portal , you can click Command line or programmatic access to get corresponding AWS credentials about the aws sso account.

you can use the command aws sts get-caller-identity --query Arn --output text | cut -d/ -f2 to get sso-role-name