Passing API data chunk per chunk to its socket clients every M seconds

Let's say that I want to query some API which will respond with array of random numbers:


[4, ..., 17, ..., 25]

To keep things simple enough, let's say that array has always 10 elements. For every element in that array, I need to emit via socket to all clients just ONE number every M seconds till end of array. When the end of the array is reached, fetch again.

The API that I want data from is found here (it's about true random numbers generator) and I choose some abstraction for fetching data(node package: node-random available here).

Sadly I can't find an abstraction (package) which support JSON API from the random.org. All of them just supports something that random.org calls "Old API for automated clients" (which only support HTML and text/plain responses), there is actually a new version of the API called "API for automated clients" which supports JSON response. So it would be faster or nothing at all to parse raw JSON instead text/plain response.

config.js

exports.fetch_options = {
    minimum : 0,
    maximum : 40,
    number  : 10,
    columns : 1,
    base    : 10,
    random  : "new"
};

Just some boilerplate instruction for node-random package, where I want to fetch 10 decimals number between [0, 40]. That is all. I used just exports here instead of module.exports because in the future I expect to change this file to support all application configs, like:

exports.mongodb_conn_url = "...";

so fetch_options and config about database should be logically divided.

Main logic (server.js)

```
var http = require('http'),
config = require('./config'),
evx = require('events'),
events = new evx.EventEmitter(),
io = require('socket.io'),
randGenerator = require("node-random"),
_ = require("underscore"),
fetch = true;

// Create dummy server

var server = http.createServer(function(req, res) {
res.writeHead(200, {'Content-Type'

So you're really looking to do two things:

• Be an Emitter: emit a random number from a list of random numbers to socket clients every m seconds; and

• Be a Fetcher: fetch n random numbers and add them to the list so the Emitter can get them.

I'm not sure this is a stream problem - if you were to fetch the numbers one at a time and then emit them, sure, a stream might be the right approach - rather, it is a dequeue, with the Fetcher filling it up and the Emitter draining it.

Given that, I would approach it something like this:

var dequeue = [];

function fetchNumbers() {
    // Check to see if we're getting low.  If now, just hold up.
    if (dequeue.length > 2) return setTimeout(fetchNumbers, 500);

    // Since we need data, use whatever you use to get them:
    awesomeRandomNumberFetchFunction(function (err, numbers) {
        numbers.forEach(function (num) {
            dequeue.unshift(num);
        });

        // Set reasonable timeout - if we're getting 10 and emitting one each 700 ms, 5 seconds is good.
        setTimeout(fetchNumbers, 5000);
    });
}

// Start up the Fetcher:
fetchNumbers();

function emitNumber() {
    var number = dequeue.pop();

    // If we got a number, emit it.
    if (number) io.sockets.emit('new_bid', number);

    // If we got a number, wait a random time; if we didn't, try again right quick instead
    var randomDelay = 700 + Math.round(Math.random() * 200);
    var timeout = number ? randomDelay : 50;

    // Lather, rinse, repeat
    setTimeout(emitNumber, timeout);
}

// Start up the Emitter:
emitNumber();

StackExchange Code Review Q#73619, answer score: 2