UTF-8 encodes each character into a whole number of bytes (8, 16, 24, or 32 bits), and the 10 continuation marker is only at the start of the extra continuation bytes, it is just data when that pattern occurs within a byte.
You are correct that it never occurs at the start of a byte that isn’t a continuation bytes: the first byte in each encoded code point starts with either 0 (ASCII code points) or 11 (non-ASCII).
https://arxiv.org/pdf/1811.07643 is some investigatory research describing, among other things, 4 clusters of reasons for starring: to show appreciation, bookmarking, due to usage, due to third-party recommendation.
FWIW, pex now also has options to unzip the archive to a cache directory on startup (I believe this happens by default now, but am not at a computer to confirm), to side step the zipapp limitations that you reference.
As of a few months ago, pex supports including an interpreter, either directly inline or lazily downloaded (and cached): https://docs.pex-tool.org/scie.html
(I’m under the impression that this was released in 2021, whereas the linked Tokio post is from 2020. Hopefully that frames the Tokio post’s more accurately.)
In addition to what you say, it can also be easier for a (appropriately-skilled) human to verify a small program than to verify voluminous parsing output, plus, as you say, there's the semi-automated "verification" of a very-wrong program failing to execute.
I have a feeling we’re talking at cross purposes here: I was actively trying to write incorrect code. This post isn’t about memmap2 crate specifically at all, it just happens to be a convenient way to get the exact (“incorrect”) syscall I wanted from Rust.
I see where you’re coming from but… it feels like you’re trying to convince me of something about the post? If you feel like convincing a larger audience of the limitations of the memmap2 crate specifically, I suggest writing your own blog posts and/or getting involved with it. :)
> That probably should have been the first thing to try.
The point of the post is not "how to make mmap work best with async/await" or "how to optimise mmap with async/await", but exploring the consequence of incorrect code (and thus explaining why one might need potential remedies like those). Sorry if that didn't come across!
As the author, I don't think there's a clear definition of "blocking" in this space, other some vibes about an async task not switching back to the executor for too long, for some context-dependent definition of "too long".
It's all fuzzy and my understanding is that what one use-case considers being blocked for too long might be fine for another. For instance, a web server trying to juggle many requests might use async/await for performance and find 0.1ms of blocking too much, vs. a local app that uses async/await for its programming model might be fine with 10ms of "blocking"!
Yes, it couldn't not block, it's obvious... but I've encountered non-trivial amounts of "magical thinking" around both async/await ("go-fast juice") and mmap ("go-fast juice") separately, so the intersection surely has a bunch of magical thinking too, where people haven't taken the time to properly think through what's going on.
Hence, my investigation to try to make the "obvious" conclusion obvious to more people.
Yeah, of course a synchronous call that might block the thread is blocking, I agree... but, if I didn't have the context of "we're in a comment thread about a blog post about mmap", I'm pretty sure I wouldn't flag `x[i]` on `x: &[u8]` (or any other access) as a synchronous call that is worth worrying about.
> The singled thread async "traditional IO" example is NOT single threaded
The threads backing the single-threaded IO are an implementation detail of fulfilling the `.read().await` calls. The key is that there's a single coordinator thread that's issuing all the work, with the user-space runtime multiplexing tasks on that thread. I thought the fact that the "start a request and come back when it is finished" behaviour happens to be implemented via user-space threads rather than kernel-level epoll (or similar) is unlikely to affect behaviour.
I considered scaling up the number of files and using a multi-threaded runtime, but I felt that'd make everything more complicated without fundamentally changing behaviour.
However, maybe my theory is incorrect, in which case someone else can do their own experiments to provide more concrete information.
> The issue is comparing 8 OS threads no async to 1 thread async is fundamentally not very useful as long as you didn't pin all threads to the same physical core
The point is not to benchmark async vs. non-async, but provide a general reference point for "mmap working well" for comparison. As you suggest you agree with the "minor" issue tag, I don't think the parallelism vs. concurrency distinction matters much here... but again, definitely happy to see some concrete data that suggests otherwise!
It looks like powers of larger primes are currently rendered with the exponent ring overlapping the "main" rings, e.g. 7 and 49 look the identical, while 343 has an overlapping 2 and 3 knot.
You are correct that it never occurs at the start of a byte that isn’t a continuation bytes: the first byte in each encoded code point starts with either 0 (ASCII code points) or 11 (non-ASCII).