I recommend starting with SWAR [1] before SIMD. Our registers are typically 64 bits, and one can try out SIMD patterns without taking a dependency on particular hardware.
This will only be effective if the data you’re working on is smaller than 64 bits. If you’re working with bytes, for example, you might get 8x parallelism.
That’s true, converting in either direction will typically allocate. Which it must, semantically.
One can use unsafe for a zero-copy conversion, but now you are breaking the semantics: a string becomes mutable, because its underlying bytes are mutable.
There is mention of how len() is bytes, not “characters”. A further subtlety: a rune (codepoint) is still not necessarily a “character” in terms of what is displayed for users — that would be a “grapheme”.
A grapheme can be multiple codepoints, with modifiers, joiners, etc.
(Whether I recommend it, not sure! I did it and then undid it, with suspicion that tests were taking longer due to, perhaps, worse caching of build artifacts.)
As you get into this, you’ll want to improve the tokenization to something less naive. (The naive approach is fine for the educational purpose of the article.)
I’ve made the mistake of simply splitting on spaces or punctuation, and eventually finding that will do the wrong thing.
Eventually I learned about Unicode text segmentation[0], which solves this pretty well for many languages. I believe the default Lucene tokenizer uses it. I implemented it for Go[1].
What allows a firm to stay competitive if there are many apparently wasteful tasks being done? Should that firm not be superseded by competitors offering better efficiency (reflected in prices) or consumer benefit?
An industry that accommodates a lot of bullshit work is an industry with high transaction costs. The author hints at that at the end of the article, but leaves it an open question.
If they really want to redistribute, they could reduce prices. They could return to shareholders, sure, but returning to the market (of their customers) would be more economically valuable.
Think in terms of natural selection. Behaviors (and personalities) that make more successful choices in that environment will tend to survive in the environment, regardless of whether they know what they are doing.
Most of the bumbling fails. Some of the bumbling works. And so the most “effective” politicians (and organizations) will survive, even if their success has little to do with deliberate scheming.
In this narrow case, I believe there may be a claim that those bits are not “Internet” service. They are private network traffic, within their respective networks.