Client-Side Rate Limiting: Token Bucket for Outbound API Calls

An application fires API requests in tight loops or on every keystroke, exceeding the external API's rate limit and receiving 429 errors that break the user experience.

Implement a token bucket rate limiter on the client side to stay within limits proactively.

class TokenBucket {
  constructor(capacity, refillRate) {
    this.tokens = capacity;
    this.capacity = capacity;
    this.refillRate = refillRate; // tokens per ms
    this.lastRefill = Date.now();
  }

  consume(tokens = 1) {
    this.refill();
    if (this.tokens < tokens) return false;
    this.tokens -= tokens;
    return true;
  }

  refill() {
    const now = Date.now();
    const elapsed = now - this.lastRefill;
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }
}

// 100 requests/s
const bucket = new TokenBucket(100, 100 / 1000);

async function rateLimitedFetch(url) {
  while (!bucket.consume()) {
    await new Promise(r => setTimeout(r, 10));
  }
  return fetch(url);
}

Token bucket allows burst traffic up to the bucket capacity while enforcing an average rate. This is more user-friendly than a fixed-window limiter that rejects requests at the start of each window.