patternterraformMinor
Troubleshooting VPC flow logs with an S3 bucket using SSE-KMS encryption with CMK
Viewed 0 times
encryptionflowlogswithssebucketvpccmkusingtroubleshooting
Problem
I'm using Terraform and trying to set up automatic export of VPC flow logs into an S3 bucket in the same AWS account and region (ca-central-1) that has default encryption turned on with AWS-KMS (using a CMK). Even after trying many permutations of policies for KMS and the S3 bucket, the flow logger still always ends up in
Update: When the S3 bucket is reconfigured to use AES-256 as the default encryption (instead of KMS) the VPC flow logs get written normally. So it's definitely a KMS problem.
The
And the result of
Curiously, it works fine in my second "sandbox" AWS account where I exclusively use the AWS web console, never Terraform. This account is configured the same way with AWS-KMS on the S3 bucket.
Both accounts seem to have the same configuration, so I can't figure out why it works in the sandbox, but fails in my terraformed account.
S3 bucket
Default encryption is enabled and and Custom KMS arn is selected.
S3 bucket policy includes statements to allow VPC flow logs delivery from
Access error status.Update: When the S3 bucket is reconfigured to use AES-256 as the default encryption (instead of KMS) the VPC flow logs get written normally. So it's definitely a KMS problem.
The
aws_flow_log Terraform resource is configured exactly according to the documentation.resource "aws_flow_log" "vpc_flow_log" {
log_destination_type = "s3"
log_destination = "${aws_s3_bucket.awslogs.arn}"
traffic_type = "ALL"
vpc_id = "${aws_vpc.vpc.id}"
depends_on = ["aws_s3_bucket_policy.awslogs"]
}And the result of
aws ec2 describe-flow-logs:{
"CreationTime": "2020-05-15T16:22:40.142Z",
"DeliverLogsErrorMessage": "Access error",
"DeliverLogsStatus": "FAILED",
"FlowLogId": "fl-REDACTED",
"FlowLogStatus": "ACTIVE",
"ResourceId": "vpc-REDACTED
"TrafficType": "ALL",
"LogDestinationType": "s3",
"LogDestination": "arn:aws:s3:::REDACTED-awslogs",
"LogFormat": "${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}"
}Curiously, it works fine in my second "sandbox" AWS account where I exclusively use the AWS web console, never Terraform. This account is configured the same way with AWS-KMS on the S3 bucket.
Both accounts seem to have the same configuration, so I can't figure out why it works in the sandbox, but fails in my terraformed account.
S3 bucket
Default encryption is enabled and and Custom KMS arn is selected.
S3 bucket policy includes statements to allow VPC flow logs delivery from
delivery.logs.amazonaws.com as written in Publishing flow logs to Solution
Turns out I was missing one very important line in my KMS key policy:
Now it works fine, and my full policy looks like this:
resources = ["*"]Now it works fine, and my full policy looks like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Allow VPC Flow Logs to use the key",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": [
"kms:ReEncrypt",
"kms:GenerateDataKey",
"kms:Encrypt",
"kms:DescribeKey",
"kms:Decrypt"
],
"Resource": "*"
}
]
}Code Snippets
resources = ["*"]{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Allow VPC Flow Logs to use the key",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": [
"kms:ReEncrypt",
"kms:GenerateDataKey",
"kms:Encrypt",
"kms:DescribeKey",
"kms:Decrypt"
],
"Resource": "*"
}
]
}Context
StackExchange DevOps Q#11623, answer score: 1
Revisions (0)
No revisions yet.