About the design model for programming in Rust, I guess only making the time to try writing something in Rust can answer the question. It is interesting how the steepness of the Rust learning curve could actually be a hook to get people to try it, which is all any well-designed language would need to gain adherents.
Back to critical systems programming, does MISRA C have a "certified" compiler like CompCert C or something? If so, Rust usage in such a niche must be a long way away - even if one could "certify" the Rust compiler code, how would one "certify" the million+ lines of C++ in LLVM!
I truly was curious about your C/C++ code - good for you that it was less C++ and more C! I don't know any masochists, so I don't know anybody who likes to work in C++ (even with just composition and generics), or even any reluctant "expert". I get away from segfaults because I am on server & can get away by more static copying of data (it pales in comparision to what alternatives offer).
I can see why you think Rust is tractable; you have worked on fairly complex stuff like automobile base software (AutoSAR, was it?). I am probably not as good at it as you are, so the cognitive load of designing at that scale with borrow-checking seems prohibitive. I hope there is a way to slice the problem which makes for less cognitive load.
The reference compiler is as free-standing as OCaml's because it too has its own back-end. Its front-end is shared with the other 2 back-ends: feeds the ubiquitous LLVM in LDC, and is included as a supported front-end in GCC since version 9.1.
Anyway, there is no way you'd give up ML-tradition pattern-matching; D doesn't have it.
I don't know. Maybe because it is GC-only? It doesn't have custom allocators, so there may be no easy way to map an object to some location in shared memory.
I wasn't talking about necessarily temporal operations like mutex ops; of course, they should be deterministically done - it is a beautiful implementation detail that it falls out of the principles of Rust, without explicit compiler support and/or "checked-delete" as is required in most other languages.
I meant lazy operations for avoiding unnecessarily making things like memory deallocation temporal. Unless you have a strict memory budget (and you might additionally need to have mitigation for memory fragmentation), freeing whenever GC deems it fit, or streams flushing whenever the OS/library deems it fit, can't be worse - but could potentially be better - for performance than being done only at points decided arbitrarily by the language and the program structure. It is simply easier to disable GC on critical paths.
But as you say, you are not (only) talking about performance. You are talking about determinism for predictability. I hope you are not implying predictability across various threads/processes that make up the system, only within 1 thread/process where that predictability leads to reliability.
Reliability across threads/processes needs strategy because failures are inevitable (yes, I have drunk the Erlang kool-aid too, among other ones). While I have no doubt that Rust's design principles would support developing such strategies, I do wonder as to what scale this would work up to ...
... as you say, I don't know the size of projects Rust has scaled to, I just know enough about Rust itself to balk at its complexity (and I am a C++ person!). Maybe we are just talking from different viewpoints. You, having done a variety of non-trivial production code in a plethora of languages, plump for Rust. But I honestly wonder about the size & longevity of your C/C++ semi-production code - semi, because it is C/C++ ;-) - I have worked on large C++ codebases for long-lived products, and I somehow can't see another complex language solving more problems net-net.
The topic was the no-GC no-runtime niche, which necessarily requires brilliant programmers, and they are stuck with C++. They would welcome tools to represent their resource control well, borrow checking or deny capabilities.
As a current C++ programmer who remembers what GCC 2.95's error messages looked like for even simple templates, I second you - it is less about the intellect, and more about experience and enough domain knowledge. But quite a few like me work on large C++ software that is not in the no-GC no-runtime niche; modules, good abstractions and native compilation would be enough to match or likely beat whatever we are putting together in C++, which doesn't even have modules yet - and the GC-based conveniences would be a big icing on the cake, so D, Nim, Pony all look interesting. (side note ... https://ponylang.io and not https://pony-lang.io as stated earlier)
Off-hand, Mirage OS comes to mind - it completely upends how you would "build" and "run" back-end services, and it is written solely in a GC'ed language - OCaml. And all GC'ed languages are not equal; see https://roscidus.com/blog about a Linux package manager ported from Python to OCaml for performance reasons (followed by an OCaml reimplementation of a component in a virtualized OS).
Some infrastructure, e.g. database engines, is ubiquitous, but all instances are of the same few software products written in C/C++/Java. Those ubiquitous instances have banged most bugs out of those codebases, so it is an uphill battle there to convince incumbents and upstarts alike, of the value of a new implementation. But Rust & Pony are marvelously positioned to march up that hill, especially if you throw in multi-threading.
My impression is that only a minority of widely-programmed back-end infrastructure that suits Rust is written in C/C++ (say, map-reduce kernels). The code-base size & the data flow complexity inside those components is pretty limited, and Rust should be tractable at that scale.
Most widely-programmed infrastructure software - in the back-end and on cell phone OS'es - have been merrily using Java/Python/Ruby/Erlang. OCaml is quoted as being used in the management component of VMware. These are much larger applications; is there any evidence or hint that Rust isn't onerous to develop larger systems in? Without that evidence, I feel (not think) that a disciplined GC'ed language (D? Pony?) has a better chance there.
As pjmlp said below, GC doesn't preclude deterministic destruction, so it isn't about resources in general; it is largely about memory. Also typical programs in any language have quite a bit of stack-allocatable data and C/C++/D/Go/Rust do support stack-allocatation, so the memory heap isn't involved everywhere.
Wherever the memory heap is involved, if the system isn't memory-starved, it can be argued that deterministic free-ing is a step too far; maybe you don't want time to be spent putting that long-ish list of heap-allocated values on the free list when exiting a function on the critical path. There is something to be said for lazy operations in such contexts, which the GC can provide (off the critical path - using GC.disable() in D) - a lot easier than a "region-based" memory management solution in C/C++/Rust. I don't know of a synthetic benchmark result to tilt the argument either way.
At a system level, it is slightly dismaying that the "back-pressure" required to trigger lazy operations is only present for memory, and only within a Unix process; when it comes to limits on open file handles or sockets or overall OS memory usage, there is no back-pressure to reclaim them - indeed no mechanism to lazily schedule files/sockets for closure.
In the no-GC no-runtime niche, you probably need brilliant programmers anyway. Most of them are banging their heads against the C++ wall right now; they would love to get back some of their lost creativity via mastering of new concepts (Rust borrow checker, or Pony's deny capabilities - https://pony-lang.io ).
This is going further & further OT, but hey - I didn't start it!
What you need is a motivating factor. Vim is almost guaranteed to be there on any Linux/BSD/Mac shell environment, which motivates people in those environments to learn it even if they learn no other editor.
Beyond such environments, like on any desktop GUI, what'd motivate you to try GUI Vim? For Emacs, I can say the integration between any set of tools you can imagine could be a motivation.
The reason Emacs users don't particularly appreciate IDE's is that Emacs' integration of code editor, compiler output, debugger control makes the development cycle extremely fast. Add any 1 more use to that. File explorer? Emacs dired. Process monitor? Emacs proced. Interactive shells? Emacs integrates Unix shell, Python sessions, anything interactive actually.
You will get all your work done in Emacs, just using editor operations! Go to the source code for a compiler error? Hit "Enter" or click on the error in the compile window. Go to the source code being debugged? Stepping in the debugger automatically opens the file at the right line number. Found the bug? The code is already there ready to be fixed; switch to the compile window & compile. Find a file in a directory? Do a text search in its dired window. Rename all files beginning with foo to begin with bar instead? Do a search-replace.
Emacs also scales way better than any desktop editor or Vim, whether it is opening GB-sized log files, or binary files viewed/edited in hexadecimal. Before long, you start managing your to-do's using org-mode, and doing all your Git operations in the insanely featureful Magit interface.
The list gets longer and longer every release. And many Linux shell environments support the same basic keyboard short-cuts (Ctrl-A to go to beginning of line, Ctrl-E to go to the end, Ctrl-R to search in the shell history), and also provide light-weight Emacs clones like 'zile', 'mg' for editing-only use in the terminal.
Is the speed bump for other-language libraries or for other-language applications?
For speeding up other-language libraries, D has a -betterC mode, which prevents you from using the subset of the language and the libraries (standard & user-defined) that relies on GC. The remaining language is a very clean C that simply works on the other language's GC'd memory (using the other language's C interface), and can use stack allocation or any heap allocation strategy of your choice for its working set (reference counting ala C++ shared_ptr could be the obvious choice, but it is your party).
For other language applications, it is a valid option to speed up the entire application by writing it in D, as it has "all" the features of those other languages + all the convenience that is afforded by a GC + threads if you don't want a multi-process design. I quote "all" because I mean useful things like blocks/closures, generic data structures, etc. - of course, neither Rust nor D give you runtime devices like monkey-patching/meta-class hackery/prototype changes.
Taking nothing away from the Rust community that has managed such a complex language so well so long, I have to say that there never was anything viable in the no-GC no-runtime niche (that is why it is a niche - otherwise why would anybody be using GC'd languages). We were/are just gritting and bearing C++.
If it is a typical program, i.e. not a device driver or OS kernel running on low-memory hardware, having a GC + runtime doesn't preclude performance, for a long-running program. Of course, for short-running programs, just malloc, don't free, is the fastest.
Do you mean "thread local passed to another thread by mistake"? There is an option to make data global instead. Unless it is a Unix pipeline kind of design, independent threads can populate their working set (from file/socket/whatever) into thread-local memory.
It depends on whether shared-memory is a design requirement or an implementation artifact (Erlang does just fine with a largely shared-nothing model).
It also depends on whether a program runs for a short time, or for a long time. If you are running for a short time, why not just avoid the GC entirely; "manage" memory manually by malloc-ing and never free-ing (free-ing too takes time, and doesn't make memory available to other processes anyway).
Well, Standard ML is a weird one too in that sense (lurking around for a couple of decades without picking up steam (outside academia?)). Languages having type inference and modules designed into them seems to have a good chance to live long.
Are you describing the D language?
https://dlang.org
C-like syntax and execution speed with high-level scripting-language-like conveniences, close to Lisp-level ability to generate code at compile time, an active user-base (https://forum.dlang.org) that continuously strives to get the language improved. Its (thread-local) memory heap is GC'd by default. They also have an LLVM back-end if that is pertinent. Recently, it even became another supported front-end in GCC, alongside Go, Ada, etc.
Does it matter which workloads always need software? If anything, the networking use-case shows that a workload, any workload, that is ubiquitous & can benefit tremendously from programmable hardware is what matters.
In a shared-memory system, just implementing libc (or the Java VM, or the Erlang VM) on an FPGA might be a win. It has to be enough bang for the buck for FPGA, but not so much that somebody would make fixed-hardware for it.
On that note, haven't networking end-points had fixed hardware also for ages now? Maybe it is inevitable that a successful application of FPGA's breeds interest in fixed-function hardware for it.
I don't know if desktops have a future, or if even local computing has one. Maybe a portion of "personal computing" ends up only done on mobile devices, and some of the rest of it moves to public clouds.
There might not be a business case for companies to produce chips & other hardware for desktops. So, lets equate buying desktops to buying low-end servers for all practical purposes. As computing gets cloud-ier, is there a business case for low-end servers either, when a high-end server can be virtualized to get the same result?
If we are stuck with power-sensitive mobile devices & virtualized cloud instances on high-end servers as the only businesses that sustain, FPGA's only play is on the servers. You'd expect lots of heterogenity in the workloads there, more than on desktops. Reconfiguring many FPGA's very frequently? Not sure it would fly, even with a small selection of applications vetted to not be damaging to the FPGA.
You can do personal computing with FPGA's with Adapteva boards. How safely can you reconfigure its FPGA, after you have managed to code for it?
About the design model for programming in Rust, I guess only making the time to try writing something in Rust can answer the question. It is interesting how the steepness of the Rust learning curve could actually be a hook to get people to try it, which is all any well-designed language would need to gain adherents.
Back to critical systems programming, does MISRA C have a "certified" compiler like CompCert C or something? If so, Rust usage in such a niche must be a long way away - even if one could "certify" the Rust compiler code, how would one "certify" the million+ lines of C++ in LLVM!