AFAIK this was not a clean room reimplementation. But since it was rewritten by hand, into a different language, with not just a different internal design but a different API, I could easily buy that chardetng doesn't infringe while Python chardet 7 does.
I've worked on a system where ULIDs (not UUIDv7, but similar) were used with a cursor to fetch data in chronological order and then—surprise!—one day records had to be backdated, meaning that either the IDs for those records had to be counterfeited (potentially violating invariants elsewhere) or the fetching had to be made smarter.
You can choose to never make use of that property. But it's tempting.
The current amount of typechecking might be a net efficiency improvement AFAIK. It provides a hard runtime guarantee that variables are certain types while Python has to check whether something is supported at the last possible moment. But I don't know how much use the optimizer makes of that.
Python managed to do this by not actually checking the types at runtime. If you declare a list[int] return type but you return a list[string] then nothing happens, you're expected to prevent that by running an offline typechecker.
PHP chose to check types at runtime. To check that a value is really an array<int> the runtime could have to loop through the entire array. All the types PHP currently implements are simple and cheap to check. For more elaborate cases you need an offline checker like PHPstan and comment-based type annotations. (PHPstan catches 99% of issues before the runtime gets to it so for my own code I'd prefer the Python approach with its cleaner syntax.)
The runtime checking seems the key difference, not so much the historical strength of the type system. Python's language implementation does very little typechecking itself and PHP's third-party offline typecheckers are respectably advanced.
I don't think this shows deep thought on his part.
By Stallman's own telling a free Objective-C frontend was an unexpected outcome. Until it came up in practice he thought a proprietary compiler frontend would be legal (https://gitlab.com/gnu-clisp/clisp/blob/dd313099db351c90431c...). So his stance in this email is a reaction to specific incidents, not careful forethought.
And the harms of permissive licensing for compiler frontends seem pretty underwhelming. After Apple moved to LLVM it largely kept releasing free compiler frontends. (But maybe I'd think differently if I e.g. understood GNAT's licensing better.)
rustc is only loosely tied to LLVM. Other code generation backends exist in various states of production-readiness. There are also two other compilers, mrustc and GCC-rs.
mrustc is a bootstrap Rust compiler that doesn't implement a borrow checker but can compile valid programs, so it's similar to to your proposed subset. Rust minus verification is still a very large and complex language though, just like C++ is large and complex.
A core language that's as simple to implement as C would have to be very different and many people (I suspect most) would like it less than the Rust that exists.
RFC 3629 says surrogate codepoints are not valid in UTF-8. So if you're decoding/validating UTF-8 it's just another kind of invalid byte sequence like a 0xFF byte or an overlong encoding. AFAIK implementations tend to follow this. (You have to make a choice but you'd have to make that choice regardless for the other kinds of error.)
If you run into this when encoding to UTF-8 then your source data isn't valid Unicode and it depends on what it really is if not proper Unicode. If you can validate at other boundaries then you won't have to deal with it there.
The big problem isn't invalid UTF-8 but invalid UTF-16 (on Windows et al). AIUI Go had nasty bugs around this (https://github.com/golang/go/issues/59971) until it recently adopted WTF-8, an encoding that was actually invented for Rust's OsStr.
WTF-8 has some inconvenient properties. Concatenating two strings requires special handling. Rust's opaque types can patch over this but I bet Go's WTF-8 handling exposes some unintuitive behavior.
There is a desire to add a normal string API to OsStr but the details aren't settled. For example: should it be possible to split an OsStr on an OsStr needle? This can be implemented but it'd require switching to OMG-WTF-8 (https://rust-lang.github.io/rfcs/2295-os-str-pattern.html), an encoding with even more special cases. (I've thrown my own hat into this ring with OsStr::slice_encoded_bytes().)
The current state is pretty sad yeah. If you're OK with losing portability you can use the OsStrExt extension traits.
ripgrep exposes its (bespoke) shell completion and man page generation through a --generate option: rg --generate=man, rg --generate=complete-bash, etcetera. In xh (clap-based) we provide the same but AFAIK we're the only one to copy that interface.
Symfony (for PHP) provides some kind of runtime completion generation but I don't know the details.
A file descriptor can't be -1 but it's not 100% clear whether POSIX bans other negative numbers. So Rust's stdlib only bans -1 (for a space optimization) while still allowing for e.g. -2.
It means that anything strange that happens next isn't a language bug.
Whether something is a bug or not is sometimes hard to pin down because there's no formal spec. Most of the time it's pretty clear though. Most software doesn't have a formal spec and manages to categorize bugs anyway.
I can think of a lot of cases where it theoretically could be a problem, but `cut -d=` is the only one I've found so far where an end user ran into trouble because of this ambiguity, and I think it's the only one for which uutils bothers implementing a workaround. That's why I give it special attention.
> You write a bit as if we disagree, but I don't see any real point of contention. :-)
The `cut -d:=` spelling solves a different problem than the one I meant (and the one you're now talking about). But we're mostly on the same page!
The problem is existing shell scripts and muscle memory and command histories. `cut -d=` has always worked and works on all the other implementations so it should keep working if you switch to uutils.
This syntax is supported by argparse and clap, the most popular argument parsers for Python and Rust respectively, and it seems to have caused almost no problems for them. It's a problem for the uutils implementation of cut, since `cut -d=` is common, but that's the only instance I could find after a long time scouring search engines and bug trackers and asking for examples.
If anyone does know of other examples or other places this has been discussed I'd love to hear it though, maybe I just haven't found them.
(Also, the more reliable way to write this in general is `-a "$USER_CONTROLLED_DATA"`, since that'll behave correctly if $USER_CONTROLLED_DATA is empty. As will `-a="$USER_CONTROLLED_DATA"` if you know the command supports it.)
- C# is bytecode-first, Go targets native code. While C# does have AOT capabilities nowadays this is not as mature as Go's and not all platforms support it. Go also has somewhat better control over data layout. They wanted to get as low-level as possible while still having garbage collection.
- This is meant to be something of a 1:1 port rather than a rewrite, and the old code uses plain functions and data structures without an OOP style. This suits Go well while a C# port would have required more restructuring.
> A lot of the increase in line count can be explained by rustfmt’s formatting, as it likes to spread code out over multiple lines [...] The rest is additional features. Also note that our Rust code is in some places a straight translation of the C++, and fully idiomatic Rust might be shorter.
AFAIK this was not a clean room reimplementation. But since it was rewritten by hand, into a different language, with not just a different internal design but a different API, I could easily buy that chardetng doesn't infringe while Python chardet 7 does.