Making sure a block of code takes at least 5 seconds to execute

I am working on a personal project where I analyse Big Data by using twitter statuses. Therefore I am trying to query the Twitter API as many times as I can.

The API has a limit of 180 queries per 15 minutes, so I figured I can do 1 query every 5 seconds (at the minimum) if I want to not exceed that limit.

In the code below I do this the old-fashioned way with System.currentTimeMillis():

while (true) {
    newStatuses.clear();

    long startTime = System.currentTimeMillis();
    // Querying the API 
    newStatuses.addAll(DbTools.TWITTER_FACTORY.search(_query).getTweets());
    long totalTime = System.currentTimeMillis() - startTime;

    allStatuses.addAll(newStatuses);
    Thread.sleep((totalTime > 5000) ? 0 : 5000 - totalTime);
    System.out.println(System.currentTimeMillis());
}

Pretty much if the total querying time took more than 5 seconds then sleep() for 0 seconds, if it took less than or equal 5 seconds then sleep() for the time that is remaining.

What I basically want is the time period from between hitting the line newStatuses.addAll(DbTools.TWITTER_FACTORY.search(_query).getTweets()); each time to be at least 5 seconds.

Is there any better way to do this or do you have any suggestions of how I can improve my solution?

Rather than waiting for 5 second between calls, consider that Twitter API returns three HTTP headers that explicitly tells you your quota:

• X-Rate-Limit-Limit: the rate limit ceiling for that given request

• X-Rate-Limit-Remaining: the number of requests left for the 15 minute window

• X-Rate-Limit-Reset: the remaining window before the rate limit resets in UTC epoch seconds

You can just query as much as you like, until the X-Rate-Limit-Remaining reaches zero or until you receive a 429 Too Many Request status code, then sleep for X-Rate-Limit-Reset.

This gives you several advantages:

• You're utilising your limit to the fullest

• If twitter changes their rate limit (e.g. someone is DDoSing them and they need to stricken it up or they make a significant upgrade that allows them to increase their quota), then your application will automatically adapt.

• sleeping can't take into account of the per-user quota when you share access tokens for multiple applications/multiple purposes

Refer to Twitter documentation: https://dev.twitter.com/docs/rate-limiting/1.1

StackExchange Code Review Q#60691, answer score: 8