Where our conversation is at, it's not about RDBMS vs. NoSQL.
On the Redis INCR operation example (or an equivalent operation in any database, even in an RDBMS). In short, you don't always have the guarantee to know the result of the operation. This is part of what the 99.9% means when services advertise their reliability. 99.9% of the time you'll get an acknowledgement back, but not 100%.
The question comes to, what happens if I miss an INCR, or if I accidentally double INCR when I retry? For some use cases, it's acceptable: like counting the number of views on a Youtube video. For others, say a financial system counting money or making a payment, that's not OK. That's where idempotency really matters (btw this is why many redis operations have the NX option).
Let's go completely off track for a little bit, say you're Strava. You let devices and users upload workouts to your service. What happens when a phone sent you the workout, then disconnected due to bad receptions? In this case, the phone doesn't know if the operation succeeded so it's forced to retry. How do we make sure the workout isn't created and counted twice on Strava? Well, each workout has a unique UUID generated by the device. Strava uses this UUID to dedup when it receives an upload. The UUID is the idempotency "checkpoint".
In this case, it doesn't matter what kind of database Strava is using. The idempotency problem is the same.
Look up the two general problem on youtube. Unless the entire end to end operation is wrapped inside a giant transaction, no system in the world can give you the confirmation.
Imagine this: you issued a write, a few things can happen:
1. The callsite crashed, maybe due to an out of memory issue or whatever. You don't know if it succeeded
2. The database returned an acknowledgement, but then the callsite crashed before storing the acknowlwedgement for the next step.
let's say you have an object like this when you started: {count: 10, etag: 1}. then for some reason, something failed.
when you retry and load the object, you get {count: 12, etag: 3}. how do you know if your previous attempt had successfully persisted or not, or if the updates to the object came from other processes/requests?
you're mixing up conflict handling vs. idempotency
> there is always more wisdom in what we throw away than what we innovate
i have found this to be true throughout my career. not that things cannot improve, they can always improve. but assuming the original work wasn't grossly incompetent, anything "new" tend to make the same [undocumented] mistakes
1. Beyond just querying, the stored proc spent a lot of time processing data. As in, looping through cursors, making business logic decisions, calculating things, etc.
2. Having the business logic (not just loading the data) inside the stored procs meant that a change in business logic that would normally only need to update application code, now invalidates the stored procedure's cached execution plan.
The KV store had etag support for conditional writes. Etags are only useful to make sure the data didn't change underneath between your read and your write.
Storing the checkpoints along with the mutation was for idempotency. If the checkpoint was in the document, that meant the mutation had succeeded and a retry should be no-op
- Go (the language) is missing the ability to declare something immutable. I don't mean making fields in a struct private then exposing getters. I mean the equivalent of an immutable reference in Rust or `const ref` in C++.
- Go mod and go workspace still can't get a cohesive story together [1]:
- - If my module wants to depend on another module in a workspace, `go mod tidy` would freak out.
- - If I use replace in my go mod file to point to the other modules in the same workspace, `go mod vendor` will copy these workspace modules to the vendor folder as well.
- Go (the language) is missing the ability to declare something immutable. I don't mean making fields in a struct private then exposing getters. I mean the equivalent of an immutable reference in Rust or `const ref` in C++.
- Go mod and go workspace still can't get a cohesive story together [1]:
- - If my module wants to depend on another module in a workspace, `go mod tidy` would freak out.
- - If I use replace in my go mod file to point to the other modules in the same workspace, `go mod vendor` will copy these workspace modules to the vendor folder as well.
Everything you said is true. C++ gives you everything you need to be safe. But that's not the point.
Rust makes it HARD (or very hard) to do things that are unsafe.
It's not what C++ can or cannot do, it's not what Rust can or cannot do. It's what the Rust language and compiler opinionatedly encourage you to do and not to do. This is what's lacking in C++.
I share exactly your sentiment. The snappy on/off is what makes the checkbox satisfying for me. The "ding" whenever you check an item off the list in Wunderlist comes to mind.
@David, thanks, and completely aligned on why we don't use nuget packages anymore.
> As for being intuitive, the default experience is to use the Web SDK. I didn't even get into SDKs but it does more than default the framework reference. It also exposes capabilities that tooling use to light up behaviors in the build and in the IDE.
It is these "does more than default framework reference" are precisely the things in discussion here. All of these hidden functionalities and dependencies are like a black box. What's the difference between what James shared here and the default Sdk=...Web, then? It's the lack of uniformity, where "ASP.NET is a first class citizen" rather than just another piece of the ecosystem that is a turn off.
Compared to other ecosystems, for example: Go, Rust, even Java for example, where everything is just code that one can pull in, and the customization is in the code, not the runtime/JVM.
It's not about if you should or should not search. It's about how often you have to do that, and how easy it is to search for what you want. It boils down to how much anti pattern a framework/ecosystem has. In this case, their default template caused me to search "dotnet core project multiple SDK", which yielded nothing, and is actually a completely wrong track.
The answer the asp.net team shared below, was instead of using the SDK attribute, use a completely different thing called FrameworkReference. Which can completely replace this SDK attribute, it seems.
Hence my question to them below was, why is framework reference not the default? Especially since it does lead to better searchability, and the template shows that one could have multiple of these per project, intuitively.
> I, and a lot of other devs, would be able to solve this particular problem without looking up the docs
Look through the rest of this comment thread and see how many failed attempts at solving this problem there were before two high profile members of the ASP.NET team came in (JamesNK and davidfowl).
It's not about looking up or not looking up. The criticism here is that the way the framework is laid out is not intuitive enough. It's very "different" from the rest of the industry (in this case, there are 3 potential ways of pulling in dependencies for ASP.NET). This requires a lot of time investment for its users to solve these slight one-off issues.
My question to them below was why isn't what they shared here the default. If they had done that, it would be intuitive to know that one can add other "FrameworkReferences" in a project, or know what to search for. Instead, the default is "each project can only target one SDK".
Where our conversation is at, it's not about RDBMS vs. NoSQL.
On the Redis INCR operation example (or an equivalent operation in any database, even in an RDBMS). In short, you don't always have the guarantee to know the result of the operation. This is part of what the 99.9% means when services advertise their reliability. 99.9% of the time you'll get an acknowledgement back, but not 100%.
The question comes to, what happens if I miss an INCR, or if I accidentally double INCR when I retry? For some use cases, it's acceptable: like counting the number of views on a Youtube video. For others, say a financial system counting money or making a payment, that's not OK. That's where idempotency really matters (btw this is why many redis operations have the NX option).
Let's go completely off track for a little bit, say you're Strava. You let devices and users upload workouts to your service. What happens when a phone sent you the workout, then disconnected due to bad receptions? In this case, the phone doesn't know if the operation succeeded so it's forced to retry. How do we make sure the workout isn't created and counted twice on Strava? Well, each workout has a unique UUID generated by the device. Strava uses this UUID to dedup when it receives an upload. The UUID is the idempotency "checkpoint".
In this case, it doesn't matter what kind of database Strava is using. The idempotency problem is the same.