Software engineer with about 15 years of experience, former CTO, VPE, Systems Architect. My key skills are in backend development, infrastructure, computer networks.
Software engineer with about 15 years of experience, former CTO, VPE, Systems Architect. My key skills are in backend development, infrastructure, computer networks.
Location: Ukraine
Remote: yes
Willing to relocate: no
Technologies: Go, Python, JS, C, PostgreSQL, MySQL, MongoDB, Redis, Linux, Docker, kubernetes, ...
Résumé/CV: linked here https://snawoot.github.io/
Email: specified here https://snawoot.github.io/
Most recent job was CTO at crypto/VPN startup and I was working on network and security stuff in general for about last 10 years. I'm open to other fields and roles, though.
I also double that rendezvous hashing suggestion. Article mentions that it has O(n) time where n is number of nodes. I made a library[1] which makes rendezvous hashing more practical for a larger number of nodes (or weight shares), making it O(1) amortized running time with a bit of tradeoff: distributed elements are pre-aggregated into clusters (slots) before passing them through HRW.
I'm getting an impression it's just not profitable enough. For many years I get a feeling that business is considered sound only if it is superprofitable (not exactly the right term, but still) in order to cover all losses.
Probably it's because of market competition required to be at least noticed. Some companies' spendings for marketing are greater than for R&D, production and operations combined. Maybe we got ourselves into a situation where everywhere competing for low-hanging fruits or trying to make customer believe it's the service they need while all of it doesn't really overlap with real society needs.
Cuckoo filters are more efficient alternative. The algorithm of displacement is also very notable: how we can use random swaps to place data in cells optimally.
I had an attempt to improve performance of memory allocation with the use of arenas in Go and I chose freelist datastructure[1]
It almost doesn't use unsafe except one line to cast pointer types. I measured practical performance boost with "container/list" implementation hooked to my allocator. All in all it performs 2-5 times faster or up to 10 times faster if we can get rid[2] of any and allocations implied by the use of it.
All in all, heap allocations can be not that bad at all if you approach them from another angle.
> 2-6x faster than Redis (benchmark link below) yet disk persisted!
That's a false contradistinction: Redis is also disk persisted.
The benchmark you did mentions Redis benchmarking guide and this guide has following paragraph:
> Redis is, mostly, a single-threaded server from the POV of commands execution (actually modern versions of Redis use threads for different things). It is not designed to benefit from multiple CPU cores. People are supposed to launch several Redis instances to scale out on several cores if needed. It is not really fair to compare one single Redis instance to a multi-threaded data store.
Did you just benchmarked against only single Redis instance and claimed performance win? Even if so, how do benchmarks compare against source-available competitor DragonflyDB?
Finally, documentation doesn't mention how persistence exactly works and what durability guarantees should we expect?
I have addresses like [email protected] directed to my email, which I use to register myself in "someservice". This is how I know where email was leaked and needs to be disabled. I use Protonmail basic subscription to attach my domain. Before that I was using rewriting rules in Postfix.
That's great! I see you use `sync.Pool`, which is way safer alternative. However, I wasn't able to secure much of performance gains for linked list benchmark using sync.Pool. It's not even clear how much objects it will contain ready to go. On the other hand, sync.Pool docs suggest to implement own free list for small short-living objects. I wonder what do they mean and how it should look like?
Remote: yes
Willing to relocate: no
Technologies: Go, Python, JS, C, PostgreSQL, MySQL, MongoDB, Redis, Linux, Docker, kubernetes, ...
Résumé/CV: https://www.linkedin.com/in/vladislav-yarmak/
Email: [email protected]
Software engineer with about 15 years of experience, former CTO, VPE, Systems Architect. My key skills are in backend development, infrastructure, computer networks.