Having had several offers to relocate to the Bay Area all I can really say is that almost nobody pays well enough to make it worthwhile. You can make $150k-200k in Seattle, Portland, Denver, Austin, etc. and work on some really interesting problems with marketplace leaders (often even for companies based in the Bay Area). You end up living like a king comparatively. You'd have to make at least 30-40% more than that in the Bay Area just to maintain a lateral quality of life.
Having now screened over 300 candidates for openings on my team, the overwhelming majority of them being people already in the Bay Area, about the only thing I can say is that with all the easy money sloshing around the economy there it's just way, waaaaaaay to easy to be a terrible engineer and still maintain being a highly paid "professional". Most of the people I screen/interview are advanced-IDE users at best or simply framework operators (i.e. they have no idea how the things even work that they're claiming to be experts in).
Sure there are a lot of bright folks too (law of averages and all that), but it's depressing to see what people are getting paid to do glorified web scripting or whose entire job function could be replaced with 50-lines of Bash.
I'm extremely drawn to funny women. I like that they play around mentally with cultural taboos and find creative ways of walking the line between making themselves laugh and making other people uncomfortable.
When I internalized this trend ~3 years ago, it's what spurred my interest in distributed storage systems across machines, racks, and datacenters, that are only memory backed.
I'm a functional programmer (Lisp, OCaml, Erlang), and I only touch C++ when I have to wrap it in something to interface with a higher-level language.
Most of the cases where I run into this problem and end up feeling like I didn't gain much, if anything, from doing a straight C implementation are situations with deeply nested data structures.
I get that the thinking is that the benefit is that once I've made the compiler stop complaining, my memory management model should at least be sound and safe, and that is a win. Though immediately following that I start to have dream-like fantasies where this entire static analysis stage is simply bolted onto C instead of being a whole new language.
Afterall you can do some pretty impressive stuff with nothing but a pile of preprocessor macros (see Objective C).
This has been my problems with Rust for the last 10-12 months.
Lifetime elision seems thoroughly broken or extremely limited in functionality, and now when I'm writing (sometimes seemingly trivial) solutions in Rust I'm spending at least as much time and mental energy explicitly annotating lifetimes as I would be if I was just managing malloc and free in C.
As a result the pain vs. benefit curve doesn't bend nearly as far toward Rust as it theoretically should.
I realize they called this out in the article as "snark", but it really is just a blog engine, and its monthly web traffic is a small fraction of what wordpress.com gets (which is being served on one of the most craptacular platforms that exists).
They in fact could have solved this problem with Rails or even a more full-fledged CMS like BrowserCMS.
But it's a SPoF and an extra remote or local service that has to be configured, maintained, etc. The thing that's coming online has to publish a message to redis anyway, so why not just maintain the service map inside the Rust application and open up a socket that allows those things to speak straight to Nucleon for mutating its configuration?
Fast start times are a valuable thing for a service component.
Stick about 10GB of small entries in it (should be enough to create all the levels) and then see what happens.
Also, you could reserve the persisted [H|T] for controlled shutdown scenarios. Basically anything that isn't complete system failure if you're properly trapping signals.
TCP would guarantee delivery, but you're right in that you wouldn't know if the consumer actually did anything with the message. It could have crashed on parsing or something.
But moving the concern to the consumer to track the cursor doesn't make the protocol any more stable. To keep a stable cursor, the consumer would need to persist that someplace, which just pushes the acknowledgement to that persistence component instead. If a stable cursor is what you're after, then co-locating it with the durable queue provides a simpler solution with a slightly better consistency guarantee.
The garbage collection problem is a real one, but realistically how many consumers is an infrastructure service like this going to have? Tens? Hundreds? Thousands? Millions? Billions?
No matter which one of those you pick it's a trivially small secondary index to maintain even if you never reaped it. I mean it's a K/V problem (consumer_id -> queue_offest) and there's a K/V store already sitting there. If you didn't want it to grow forever then you could establish a TTL policy via configuration.
The problem you would have is consumers that don't have stable or bounded id's. Like a system that assigns a new id every time the consumer makes a request or the consumer is restarted.
Also, you have to be paying one hell of a compaction penalty if this isn't a grow-only dataset. By ordering your keys you're at least minimizing the overhead of compaction on write by utilizing the happy-path for how LevelDB moves data out of the write buffer and into the SSTs.
But deletes are going to have a big impact still, and (working from my failing memory of LevelDB internals) I think might actually be the pathologically sad case.
If you wanted the consumers to be stateless, assuming they otherwise had a deterministic identity, then you could have the queue operate like a journal internally, but present a unified queue API to consumers.
So the queue keeps track of the high-watermark on a per consumer basis and all the consumer has to do is show up, tell the queue its deterministic name/id (might be driven by imaging, configuration, or SDN), and the queue will serve up the next new item that consumer hasn't seen yet.
This would be handy for really dynamic transient worker topologies because it keeps the mutable state and state tracking concerns entirely outside the transient worker.
That said, I still wouldn't use LevelDB. Unless I was expecting to do multi-attribute range queries or something (now we're well outside queue territory), but even then you're still folding over the data for knowable start/end markers and a linear scan over a binary term file will be faster than the multiple seeks + segment scans that LevelDB requires.
Having now screened over 300 candidates for openings on my team, the overwhelming majority of them being people already in the Bay Area, about the only thing I can say is that with all the easy money sloshing around the economy there it's just way, waaaaaaay to easy to be a terrible engineer and still maintain being a highly paid "professional". Most of the people I screen/interview are advanced-IDE users at best or simply framework operators (i.e. they have no idea how the things even work that they're claiming to be experts in).
Sure there are a lot of bright folks too (law of averages and all that), but it's depressing to see what people are getting paid to do glorified web scripting or whose entire job function could be replaced with 50-lines of Bash.