patternMinor
Application calling AWS internal load balancer in same subnet is timing out
Viewed 0 times
balancersameapplicationtimingawsloadinternalcallingsubnetout
Problem
Some background:
I've created a moderately complex network using Amazon's vpc. It's a three-tiered network across two availability zones. Each layer has a subnet in zone-a and zone-b. The presentation layer is at the top, there’s an application layer in the middle, and a core layer at the bottom.
All of the security groups and ACL's for the subnets are currently allowing ALL inbound and outbound traffic to help me reduce surface area of the issue.
The presentation layer’s routing table is pointing all traffic to an internet gateway. The NAT gateway is in a segregated subnet also pointing all traffic to the internet gateway.
My application has two components, a UI (React.js) and an API (Node/Express). They are deployed as docker images. In front of each is a classic load balancer.
The UI-ELB is internet facing and resides in the presentation layer, routing traffic from 80/443 to port 8080 and is associated with my app-ec2 that is placed in the application-layer subnet.
My API has an internal load balancer in front of it. The API-ELB is in the application layer (in the same subnet as the app-ec2), and takes traffic on port 80/443 and routes it down to the api-ec2 in the core on port 3000.
Both load balancers are offloading the certificate before passing traffic to their instances.
I have both my load balancers associated as alias's in Route53, and referenced in the applications by their pretty url (https://app.website.com). Each load balancer passes the defined healthchecks and reports all ec2 instances in use.
Lastly, on the API I have enabled cors using the cors nodejs package.
Here's a quick and dirty diagram of my network.
The problem:
The APP-ELB successfully routes me to the application. However, when the app tries to send a GET request to the API-ELB, it first sends an OPTIONS request that tims out with the error code 408.
Where it gets weird
Some of the weirdest things I've encountered while debugging are:
I've created a moderately complex network using Amazon's vpc. It's a three-tiered network across two availability zones. Each layer has a subnet in zone-a and zone-b. The presentation layer is at the top, there’s an application layer in the middle, and a core layer at the bottom.
All of the security groups and ACL's for the subnets are currently allowing ALL inbound and outbound traffic to help me reduce surface area of the issue.
The presentation layer’s routing table is pointing all traffic to an internet gateway. The NAT gateway is in a segregated subnet also pointing all traffic to the internet gateway.
My application has two components, a UI (React.js) and an API (Node/Express). They are deployed as docker images. In front of each is a classic load balancer.
The UI-ELB is internet facing and resides in the presentation layer, routing traffic from 80/443 to port 8080 and is associated with my app-ec2 that is placed in the application-layer subnet.
My API has an internal load balancer in front of it. The API-ELB is in the application layer (in the same subnet as the app-ec2), and takes traffic on port 80/443 and routes it down to the api-ec2 in the core on port 3000.
Both load balancers are offloading the certificate before passing traffic to their instances.
I have both my load balancers associated as alias's in Route53, and referenced in the applications by their pretty url (https://app.website.com). Each load balancer passes the defined healthchecks and reports all ec2 instances in use.
Lastly, on the API I have enabled cors using the cors nodejs package.
Here's a quick and dirty diagram of my network.
The problem:
The APP-ELB successfully routes me to the application. However, when the app tries to send a GET request to the API-ELB, it first sends an OPTIONS request that tims out with the error code 408.
Where it gets weird
Some of the weirdest things I've encountered while debugging are:
- I can SSH into the app-ec2 instanc
Solution
So what you have actually is this:
As your API ELB is in a private zone it can't be accessed from the internet.
Your frontend in React.js just run in User's browser and not on the UI servers, those server just serve static files.
You have two options, configure your frontend servers to redirect API calls to the API ELB or just update the API ELB to be internet facing.
The usual pitfall of JavaScript apps is forgetting they run inside user's browser and not on the frontend servers as a JEE application would.
As your API ELB is in a private zone it can't be accessed from the internet.
Your frontend in React.js just run in User's browser and not on the UI servers, those server just serve static files.
You have two options, configure your frontend servers to redirect API calls to the API ELB or just update the API ELB to be internet facing.
The usual pitfall of JavaScript apps is forgetting they run inside user's browser and not on the frontend servers as a JEE application would.
Context
StackExchange DevOps Q#1298, answer score: 9
Revisions (0)
No revisions yet.