Ratelimiter

class cloudly.util.ratelimiter.RateLimiter

Bases: object

This class is used to impose rate limits in one thread or across threads. If you need rate limiting across processes, you can use this facility in a multiprocessing “manager”.

__init__(limit: int, time_window_in_seconds: float | int = 1)

Suppose limit = 10 and time_window_in_seconds=60, that specifies a rate limit on a minute basis. Specifically, it means “at most 10” in any time window of duration “60 seconds”. In other words, the minutes are not the “whole minutes” on a wall clock. The rate limit applies in any 60-second time window carved out starting anywhere on the continuous time axis. The 10 events do not need to happen evenly in the 60-second time window; they can very well happen in a burst or multiple bursts.

The methods wait() and nowait() are thread safe. This object can be passed into multiple threads and used in each thread concurrently.

wait() → None

Once the user is ready to do “the thing” (such as calling an HTTP service) that’s subject to this rate limit, call this method. This method may block. Once this method returns, user can go ahead to do the thing.

The time of this method’s return is recorded internally as one occurrence of that thing.

nowait() → bool

In contrast to wait(), this method does not wait. If user can go ahead “do it” right now because there’s still quota within the rate limit, True is returned, and user should go ahead “do it”. The current time is internally recorded as an action (to be taken by the user as expected).

If the system is currently out of quota (i.e. has reached the limit), False is returned, and user should not proceed.

Example,

if my_ratelimiter.nowait():
    do_it()
else:
    raise Exception("You have reached rate limit. Please try again later!")

Note

If there are multiple users of the same RateLimiter object, they must all use wait() or all use nowait(); they can not mix-use the two methods. Otherwise, consider this scenario: say user “A” calls nowait while user “B” has called wait and is currently holding the lock and sleeping; then “A” has to wait on the lock, which defeats the meaning of “nowait”.

class cloudly.util.ratelimiter.AsyncRateLimiter

Bases: object

An AsyncRateLimiter is used by one or more async workers in the same thread.

See RateLimiter for doc.

__init__(limit: int, time_window_in_seconds: float | int = 1)

async wait() → None

async nowait() → bool