HiveBrain v1.2.0
Get Started
← Back to all entries
patternterraformMinor

Troubleshooting VPC flow logs with an S3 bucket using SSE-KMS encryption with CMK

Submitted by: @import:stackexchange-devops··
0
Viewed 0 times
encryptionflowlogswithssebucketvpccmkusingtroubleshooting

Problem

I'm using Terraform and trying to set up automatic export of VPC flow logs into an S3 bucket in the same AWS account and region (ca-central-1) that has default encryption turned on with AWS-KMS (using a CMK). Even after trying many permutations of policies for KMS and the S3 bucket, the flow logger still always ends up in Access error status.

Update: When the S3 bucket is reconfigured to use AES-256 as the default encryption (instead of KMS) the VPC flow logs get written normally. So it's definitely a KMS problem.

The aws_flow_log Terraform resource is configured exactly according to the documentation.

resource "aws_flow_log" "vpc_flow_log" {
  log_destination_type = "s3"
  log_destination      = "${aws_s3_bucket.awslogs.arn}"
  traffic_type         = "ALL"
  vpc_id               = "${aws_vpc.vpc.id}"
  depends_on           = ["aws_s3_bucket_policy.awslogs"]
}


And the result of aws ec2 describe-flow-logs:

{
  "CreationTime": "2020-05-15T16:22:40.142Z",
  "DeliverLogsErrorMessage": "Access error",
  "DeliverLogsStatus": "FAILED",
  "FlowLogId": "fl-REDACTED",
  "FlowLogStatus": "ACTIVE",
  "ResourceId": "vpc-REDACTED
  "TrafficType": "ALL",
  "LogDestinationType": "s3",
  "LogDestination": "arn:aws:s3:::REDACTED-awslogs",
  "LogFormat": "${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}"
}


Curiously, it works fine in my second "sandbox" AWS account where I exclusively use the AWS web console, never Terraform. This account is configured the same way with AWS-KMS on the S3 bucket.

Both accounts seem to have the same configuration, so I can't figure out why it works in the sandbox, but fails in my terraformed account.

S3 bucket

Default encryption is enabled and and Custom KMS arn is selected.

S3 bucket policy includes statements to allow VPC flow logs delivery from delivery.logs.amazonaws.com as written in Publishing flow logs to

Solution

Turns out I was missing one very important line in my KMS key policy:

resources = ["*"]


Now it works fine, and my full policy looks like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow VPC Flow Logs to use the key",
            "Effect": "Allow",
            "Principal": {
                "Service": "delivery.logs.amazonaws.com"
            },
            "Action": [
                "kms:ReEncrypt",
                "kms:GenerateDataKey",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:Decrypt"
            ],
            "Resource": "*"
        }
    ]
}

Code Snippets

resources = ["*"]
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow VPC Flow Logs to use the key",
            "Effect": "Allow",
            "Principal": {
                "Service": "delivery.logs.amazonaws.com"
            },
            "Action": [
                "kms:ReEncrypt",
                "kms:GenerateDataKey",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:Decrypt"
            ],
            "Resource": "*"
        }
    ]
}

Context

StackExchange DevOps Q#11623, answer score: 1

Revisions (0)

No revisions yet.