Sort of. I add provenance, which helps properly identify collisions, and require a well-defined order by stacking [1] changelists.
> The doc claims it's the first with this technique.
More like the first with the particular angle on the technique. I specifically mention patch theory as another side of the same coin.
> A 30 second search reminded me of Darcs, and taught me about Pijul, and Weave.
Darcs is Pijul's ancestor, and I mentioned Pijul. I also mentioned the weave and how reference sets scale better.
> The example in the doc uses text, and unfortunately I think it's for a reason.
Readability. Nothing more. The real stuff will be a compact binary format.
> I think with large, binary game assets, the most common operation is going to be strings of "replace A with B", and depending on your chunk size relative to the distribution of changes you make on your assets, I see it as pretty close to a wash, for efficiency.
Yore will dedup change data instead because as the Lore document itself identifies, dedupping content is hard using chunks; you either get dedupping or canonical addresses. Change data doesn't have one canonical address; the address is in the commit data instead.
Dedupping changes has another benefit. If most instances are "replace A with B," and A replaces B in multiple places, Yore will be able to store just one instance of A, no matter its size. This matters because the larger the chunk, the less likely it will match any other chunk.
> Especially considering that content-addressable blocks also solves de-duplication, which for a multi-game studio is probably going to be significant. Especially if they're managing multiple releases, patches, development branches, etc.
True, but that should be table stakes. The fact that Git does not is a poor reflection on Git, not an innovation in Lore.
When it comes to large assets, wasting large chunks of space is a problem. If your chunks are 64 kib average (from the Lore document), but changes only average 1 kib (which could be a high estimate), then you will still run out of space 64 times faster and need to read 64 times more data off of the disk for certain operations.
It also makes diffing hard, as well as diff viewing.
> At this point if your VCS isn't a layer above git plumbing, nobody gonna waste time using it.
Probably true, but it's a shame because there are better ways of storing and processing the data, ways that natively handle binary files, semantics, and large files without falling over.
Bram Cohen is awesome, but this feels a little bare. I've put much more thought into version control ([1]), including the use of CRDTs (search for "# History Model" and read through the "Implementing CRDTs" section).
> For instance, you might think that big tech engineers are being deliberately demoralized as part of an anti-labor strategy to prevent them from unionizing, which is nuts. Tech companies are simply not set up to engage in these kind of conspiracies.
The title of the blog post downplays the absolute masterclass that this post is. It should be called "A Tale of Four Fuzzers: Best Practices for Advanced Fuzzing."
And if you don't have time, just go to the bullet point list at the end; that's all of the best practices, and they are fantastic.
I hope that FOSS work is never a burden, but that's too idealistic. For some, FOSS work is a burden on them and their loved ones.
To those for whom it is, please don't feel obligated to continue. I haven't, and I made this public so that others will be able to discard their feelings of obligation.
> It has been mentioned that it’s possible that the default for debug builds is that every single function is compiled as an async function. I.e. there is canonically only one function color.
But then, for those that choose to only use blocking I/O or green threads, they still pay the penalty of async merely existing.
> That’s a really bad take. It’s not much different from what they did to make allocators explicit.
I mean, Zig explicit allocators are really the same thing is Go interfaces, just dressed up as an innovative feature by a specific use case. This is what I mean by "appearing" innovative: they are taking old and tested ideas and presenting them in new ways to make them appear new.
Also, Zig could have had explicit allocators without needing to pass them to every function [1].
> They’ve moved away from designing in a bunch of fancy syntax that locks users into one particular concurrency model, and gone for a more explicit and boring design which puts power in the hands of the user.
Except that if every function is made async, they have actually removed from users the power to choose to not use async.
From my experience, the calling convention was, in 0.9.x, just an implementation detail, until it wasn't. I think I may still reserve judgment for when async is fully implemented. Then I'll torture it again.
As the author of a semi-famous post about how Zig has function colors [1], I decided to read up on this.
I see that blocking I/O is an option:
> The most basic implementation of `Io` is one that maps to blocking I/O operations.
So far, so good, but blocking I/O is not async.
There is a thread pool that uses blocking I/O. Still good so far, but blocking I/O is still not async.
Then there's green threads:
> This implementation uses `io_uring` on Linux and similar APIs on other OSs for performing I/O combined with a thread pool. The key difference is that in this implementation OS threads will juggle multiple async tasks in the form of green threads.
Okay, they went the Go route on this one. Still (sort of) not async, but there is an important limitation:
> This implementation requires having the ability to perform stack swapping on the target platform, meaning that it will not support WASM, for example.
But still no function colors, right?
Unfortunately not:
> This implementation [stackless coroutines] won’t be available immediately like the previous ones because it depends on reintroducing a special function calling convention and rewriting function bodies into state machines that don’t require an explicit stack to run.
(Emphasis added.)
And the function colors appear again.
Now, to be fair, since there are multiple implementation options, you can avoid function colors, especially since `Io` is a value. But those options are either:
* Use blocking I/O.
* Use threads with blocking I/O.
* Use green threads, which Rust removed [2] for good reasons [3]. It only works in Go because of the garbage collector.
In short, the real options are:
* Block (not async).
* Use green threads (with their problems).
* Function colors.
It doesn't appear that the function colors problem has been defeated. Also, it appears to me that the Zig team decided to have every concurrency technique in the hope that it would appear innovative.
Sort of. I add provenance, which helps properly identify collisions, and require a well-defined order by stacking [1] changelists.
> The doc claims it's the first with this technique.
More like the first with the particular angle on the technique. I specifically mention patch theory as another side of the same coin.
> A 30 second search reminded me of Darcs, and taught me about Pijul, and Weave.
Darcs is Pijul's ancestor, and I mentioned Pijul. I also mentioned the weave and how reference sets scale better.
> The example in the doc uses text, and unfortunately I think it's for a reason.
Readability. Nothing more. The real stuff will be a compact binary format.
> I think with large, binary game assets, the most common operation is going to be strings of "replace A with B", and depending on your chunk size relative to the distribution of changes you make on your assets, I see it as pretty close to a wash, for efficiency.
Yore will dedup change data instead because as the Lore document itself identifies, dedupping content is hard using chunks; you either get dedupping or canonical addresses. Change data doesn't have one canonical address; the address is in the commit data instead.
Dedupping changes has another benefit. If most instances are "replace A with B," and A replaces B in multiple places, Yore will be able to store just one instance of A, no matter its size. This matters because the larger the chunk, the less likely it will match any other chunk.
> Especially considering that content-addressable blocks also solves de-duplication, which for a multi-game studio is probably going to be significant. Especially if they're managing multiple releases, patches, development branches, etc.
True, but that should be table stakes. The fact that Git does not is a poor reflection on Git, not an innovation in Lore.
[1]: https://www.stacking.dev/