What methods are available for connecting many-to-many containers?

I have two containers (Docker):

• TensorFlow container

• NodeJS container used as a frontend to the TensorFlow container.

Both containers have TCP listeners on a port specified using environment variables.

If I want to scale the NodeJS and TensorFlow containers, for example 2 instances of NodeJS and 4 instances of TensorFlow, what are my options for connecting them to evenly spread the load across the 4 instances of TensorFlow?

Currently I'm using the TensorFlow container name inside the NodeJS container, but I'm pretty sure that would only take advantage of a single TensorFlow container... Or??

The classic answer to balancing load is, of course, a load balancer. There are many different methods of doing this, but we're interested in ones that work well with dynamic containers.

This is one of the things an orchestration layer will address. In Kubernetes, for instance, you would create a Tensor Flow service, and scale it to however many pods you need (or use autoscaling!). Kubernetes then provides an ip for the service that kube-proxy load balances across the pods.

There are also service mesh options that replace this layer and perform a similar function.

StackExchange DevOps Q#4350, answer score: 1