Java NIO Server Scaling

I wrote a Java NIO Echo Server and want to maximize the number of connections to the server.

The problem is that when I try to connect more than 10k clients the clients get their connections refused:


error Connection Refused : no further information available.

The client program that I use generates a batch of 50 clients per 5 seconds
and each client sends data to the server every 5 seconds.

I have added a backlog queue parameter to the server and want to review he server code and find any flaws which may impact scaling.

Server

```
public class Server implements Runnable {

public final static String ADDRESS = "192.168.1.3";
public final static int PORT = 8511;
public final static long TIMEOUT = 10000;
public int clients;
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
private ServerSocketChannel serverChannel;
private Selector selector;

private Map dataTracking = new HashMap();

public Server(){
init();
}

private void init(){
System.out.println("initializing server");

if (selector != null) return;
if (serverChannel != null) return;

try {

selector = Selector.open();
serverChannel = ServerSocketChannel.open();
serverChannel.configureBlocking(false);
serverChannel.socket().bind(new InetSocketAddress(ADDRESS, PORT),5000);
serverChannel.register(selector, SelectionKey.OP_ACCEPT);

} catch (IOException e) {
e.printStackTrace();
}
}

@Override
public void run() {
System.out.println("Now accepting connections...");
try{
while (!Thread.currentThread().isInterrupted()){

int ready = selector.select();
if(ready==0)
continue;
Iterator keys = selector.selectedKeys().iterator();

while (keys.hasNext()){
SelectionKey key = keys.next();
keys.remove();
if (!key.isValid()){
continue;
}

if (key.isAcceptable()){

There are a number of things in here which could be a problem, and a number of other things which are a problem....
Ephemerals

First up, you may just be running in to resource limitations. A TCP based computer has just 64K ports available for sockets. The first 1K are reserved for root, and the remainder are available for programs. You may think you have > 60K ports you can connect from, but, if your client program is all running from one computer, then each client instance will have its own source port. These source ports are allocated from the ephemeral range. You may be limited to just 15K ports or so.... This is close enough to your listed limit to be significant.

Can you push things further if you run multiple client programs from multiple computers?
Performance

Next, as I understand it, every 5 seconds you get an additional 50 connections, so, it takes > 15 minutes to get to 10,000 connections.

At that point, every connection is sending a message every 5 seconds. So, 15 minutes in, you are getting 2000 messages per second.

That's not particularly large, but, let's consider what your code does:

• it does a System.out.println(...) !!!!

That's a real problem. Println's are slow, and will likely be a bottleneck for you. Remove them.

You are also dedicating just one thread to service the entire socket. This should be enough for a fairly high volume of echo's, but it's not huge. Your echo process does in fact do a fair amount of work. A lot of byte[] buffer creation, etc. which generates a fair amount of garbage.

I would consider monitoring your garbage collection to see if you have long GC cycles. A long cycle may cause a backlog of connections to accumulate.

Speaking of that, why do you do this for every connection?

byte[] hello = new String("Hello from server").getBytes();

That should be stored as a static-final constant. No need to rebuild it (and collect it) each time.
NIO

The NIO component, at face value, looks OK. I worry that you have only one thread. I know that this is often listed as being OK for NIO, but I would recommend a small thread pool for handling the active sockets. Each active socket is handled in a new thread. One thread is plenty for handling the scheduling of the sockets, but adding the latency that the computation involves will likely limit your throughput (especially the printlns.).

StackExchange Code Review Q#92971, answer score: 9