I built a thread-safe rate limiting library for Go that's lock-free with zero allocations on the hot path. It features multiple strategies including classic token bucket and TCP-inspired AIMD algorithm for dynamic rate adjustments. Optimized for high-throughput systems with memory efficiency (packed uint64 representation) and precise per-entity limiting.
When building high-throughput services, I kept hitting the same problem with existing rate limiting libraries - they break down with high-cardinality identifiers. Try rate limiting millions of unique user IDs, API keys, or IPs and you'll face:
1. Memory consumption exploding linearly with cardinality
2. Lookup performance degrading as hash tables grow
3. Contention killing throughput under load
4. GC pressure from constant allocations
I built rate specifically to solve this:
- Fixed memory footprint regardless of cardinality (configurable power-of-2
buckets)
- Sub-microsecond, zero-allocation operations even at scale
- Lock-free design using atomic operations
- Collision probabilities you can reason about mathematically
This means you can rate limit by user ID, device ID, API key, IP address or any other high-cardinality identifier without performance penalty. The benchmark numbers tell the story: even with thousands of distinct IDs hitting the limiter concurrently, operations complete in 1-15ns with zero allocations.
When building high-throughput services, I kept hitting the same problem with existing rate limiting libraries - they break down with high-cardinality identifiers. Try rate limiting millions of unique user IDs, API keys, or IPs and you'll face:
I built rate specifically to solve this:
This means you can rate limit by user ID, device ID, API key, IP address or any other high-cardinality identifier without performance penalty. The benchmark numbers tell the story: even with thousands of distinct IDs hitting the limiter concurrently, operations complete in 1-15ns with zero allocations.