Java thottling mechanism

Question 1

Probably one of the best of options would be to use Semaphore http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Semaphore.html class and give it 80 permits every minute. This can be accomplished for example by using timer class http://docs.oracle.com/javase/7/docs/api/java/util/Timer.html. The caller thread will consume permits every time it performs the call to the service by calling acquire() on the semaphore, which will block if all permits have been drained already.

It would of course be possible to code this up using wait/notify and integer counter with timer or separate thread, as you mention, but that would be more complex compared to the usage of more modern java.util.concurrent API that I have outlined above.

It can look close to the following:

class Throttler implements TimerTask {
  final Semaphore s = new Semaphore(80);  
  final Timer timer = new Timer(true);

  Throttler() {
    timer.schedule(this, 0, 60*1000);   //schedule this for 1 min execution  
  }

  run() {  //called by timer 
    s.release(80 - s.availablePermits());
  }

  makeCall() {
    s.acquire();
    doMakeCall();
  }

}

This should work starting from Java 5.

Also even nicer solution would be to use com.google.common.util.concurrent.RateLimiter from Guava. It can look like this:

class Throttler {
  final RateLimiter rateLimiter = RateLimiter.create(80.0/60.0);

  makeCall() {
    rateLimiter.acquire();
    doMakeCall();
  }
}

Semantics is slightly different compared to Semaphore solution, with RateLimiter being most probably better fit in your situation.

Question 2

I have written something like this recently. The only change is that my code expects a callback when the function is done running. So if it cant run, I directly call the callback.

One additional change is that, since this call is probably async, it is possible the call is in progress when you call it the second time. In such cases, I just ignored the call.

And my throttler has a helper function called call which takes the function to be called and the callback. This was in C++, so for you that would be in the form of Action and listener interfaces.

This has the advantage over Semaphore based solutions that it serializes requests so that you dont call it too often either.

interface Callback{
    public void OnFunctionCalled();
}

class APIThrottler
   //ctor etc
   boolean CanCall();

   public boolean IsInProgress();

   public void SetInProgress(boolean inProgress = true);

   public void Mark(){/*increment counter*/; SetInProgress(false);} // couldnt think of a better name..

   public void Call(Callable event, Callback callback){
        If(IsInProgress())
            return;
        else if(CanCall())
        {
            SetInProgress();
            event.Call();
        }
        else
            callback.OnFunctionCalled();

    }
}

Once the function callsback (or within the function itself if it is synchronous) you would need to Mark() once it is done.

This is for the most part my implementation, only difference is that I was dealing with once in x seconds (or minutes).

Question 3

This can be achieved by using a simple counter. Let's call it "permit" counter initialized at 80. Before you call "the function" first attempt to acquire a permit. When you're done, release it. Then setup a repetitive timer that reset the counter to 80 every second.

class Permit extends TimerTask
{
    private Integer permit = 80;

    private synchronized void resetPermit() { this.permit = 80; }

    public synchronized boolean acquire() {
        if(this.permit == 0) return false;
        this.permit--;
        return true;
    }

    public synchronized void release() { this.permit++; }

    @Override public void run() { resetPermit(); }
}

Setup the timer to reset permit every second like this. The first parameter of schedule method is an instance of TimerTask (pass the Permit class object above). The run() method will be invoked for every period specified on the second argument (here 1000 miliseconds / 1 sec). The 'true' argument indicates this is a daemon timer which will keep repeating.

Timer timer = new Timer(true);
timer.schedule(permit, 1000);

Then everytime you need to call your function, first check if you can acquire any permit. Don't forget to release it when you're done

void myFunction() {
  if(!permit.acquire()) {
      System.out.println("Nah.. been calling this 80x in the past 1 sec");
      return;
  }

  // do stuff here

  permit.release();
}

Note the usage of synchronized keyword on Permit class methods above -- this should avoid more than 1 thread executing any of the method of the same object instance simultaneously

Question 4

You can keep records of times, and make sure no more than 80 records in the last minute.

// first=newest, last=oldest
final LinkedList<Long> records = new LinkedList<>();

synchronized public void canRun() throws InterruptedException
{
    while(true)
    {
        long now = System.currentTimeMillis();
        long oneMinuteAgo = now - 60000;

        // remove records older than one minute
        while(records.getLast() < oneMinuteAgo)
            records.removeLast();

        if(records.size()<80) // less than 80 records in the last minute
        {
            records.addFirst(now);
            return;   // can run
        }

        // wait for the oldest record to expire, then check again
        wait( 1 + records.getLast() - oneMinuteAgo);
    }
}

It is possible that at 0th second, we issue 80 calls, wait a minute, then at the 60th second, we issue another 80 calls. The other end may happen to measure a 160 calls within a minute, due to clock imprecision on both ends, or network random delays. To be safe, enlarge the time window, say do throttle at 80 calls per 70 seconds instead.