Amazon EKS from local kubectl does not cache MFA token/session

I have set up an EKS cluster in AWS. My user-account there is secured with MFA. And I have an assumerole, that is allowed to manage my infrastructure.

So I installed and configured my system as stated in the AWS userguide: https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html

I have kubectl in version 1.11.x installed, so I can use that binary to communicate with aws (and I do not need the patched amazon version). I then installed aws-cli and aws-iam-authenticator.

user@notebook [04:44:51 PM] [~]
-> % kubectl version --short --client
Client Version: v1.11.0
user@notebook [04:45:13 PM] [~]
-> % aws-iam-authenticator help
A tool to authenticate to Kubernetes using AWS IAM credentials
...
user@notebook [04:45:25 PM] [~]
-> % aws --version
aws-cli/1.15.51 Python/3.6.6 Linux/4.17.8-1-ARCH botocore/1.10.50
user@notebook [04:45:30 PM] [~]
-> %

I then created the configuration files from the how-to ~/.aws/credentials, ~/.aws/config, ~/.kube_aws/config and created an alias for myself to use kubectl with the new config (since I have more clusters to manage): alias kubectlaws='KUBECONFIG=/home/user/.kube_aws/config kubectl $@'.

So here comes my problem: When I connect to the cluster using the aws-cli binary, I have to enter my MFA token and then the session is cached (I read for about 15 minutes).

user@notebook [04:58:52 PM] [~]
-> % aws eks describe-cluster --profile my_profile --name ClusterName  --query cluster.status                                                                                          
Enter MFA code for arn:aws:iam:::
"ACTIVE"
user@notebook [04:59:09 PM] [~]
-> % aws eks describe-cluster --profile my_profile --name ClusterName  --query cluster.endpoint                                                                                        
"https://..."
user@notebook [04:59:30 PM] [~]
-> %

But when I do this with my kubectl alias, I have to enter the token every time I run a command! Furthermo

Indeed, this is a pain. I've been fighting with it for the whole day yesterday, but managed to get it working, let's see if our cases are similar enough.

My guess is that the usual aws configure configuration with a MFA ARN + the AWS SDK used to develop kubectl are not entirely compatible: I have two profiles set for this, one that has its mfa_serial_arn and role_arn that works as follows:

• First aws command I type, I get prompted for MFA token

• From there onwards, while session is active, I can work without typing in more tokens.

• kubectl asks for a token EVERY SINGLE TIME.

This is bad.

Now, I set a different profile, without any of the above, and I do the following:

keys=($(aws sts assume-role --role-arn \
--role-session-name \
--serial-number \
--token-code \
--query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' --output text))

# profile used here will be configured in ~/.aws/credentials
export AWS_PROFILE=

aws configure set aws_access_key_id ${temp_keys[0]}
aws configure set aws_secret_access_key ${temp_keys[1]}
aws configure set aws_session_token ${temp_keys[2]}

This creates a temporary valid aws profile, that can be used BOTH with the aws cli, and kubectl.

StackExchange DevOps Q#4616, answer score: 2