The state of low-level system programming(erik-poupaert.blogspot.com)
erik-poupaert.blogspot.com
The state of low-level system programming
http://erik-poupaert.blogspot.com/2012/12/the-state-of-low-level-system.html
18 comments
I asked a very relevant question on stackoverflow about this recently: "Writing a Python extension in Go" http://stackoverflow.com/questions/12443203/writing-a-python...
Now technically you can compile a special python which can handle Go extensions, which is the wrapper mentioned in the second answer, but you can't distribute python modules that ask people to install "this special python" to run (which will reliably work with all their other installed code..)
I would like to see a "go init" function in a C API, that would let you interop the two whichever had the `main()`, since that would let you link with all the other c-implemented scripting languages (python, ruby, etc) which would mean that Go extension support could be added to the main cpython trunk as a compile-time option.
Now technically you can compile a special python which can handle Go extensions, which is the wrapper mentioned in the second answer, but you can't distribute python modules that ask people to install "this special python" to run (which will reliably work with all their other installed code..)
I would like to see a "go init" function in a C API, that would let you interop the two whichever had the `main()`, since that would let you link with all the other c-implemented scripting languages (python, ruby, etc) which would mean that Go extension support could be added to the main cpython trunk as a compile-time option.
I really hope to see Go and Rust move in the direction of pluggable runtimes that can be embedded into a C host program. Currently, both runtimes require being seated as the main entry-point of the program to function. Both projects also have issues in their bug trackers to rectify this:
Rust issue: https://github.com/mozilla/rust/issues/3608
Go issue: http://code.google.com/p/go/issues/detail?id=2790
I wonder who will get there first.
Rust issue: https://github.com/mozilla/rust/issues/3608
Go issue: http://code.google.com/p/go/issues/detail?id=2790
I wonder who will get there first.
Also, a way to compile without kernel (system) calls, so that it can be use for kernel development. It used to be possible with go, but no more with go1. Then these languages would truly be able to replace C.
Clay[1] doesn't get as much attention as it deserves. The authors seem well-versed in programming language theory, but they also understand the practical needs of a systems language. What you end up with is a language that's sufficiently low-level to be a C replacement, but it offers a plethora of mechanisms for abstraction.
Of all the new languages billed as "systems languages", Clay is the only one that excites me.
It can also "quack like a duck" (to put it in TFA terms).
[1] http://claylabs.com/clay/
Of all the new languages billed as "systems languages", Clay is the only one that excites me.
It can also "quack like a duck" (to put it in TFA terms).
[1] http://claylabs.com/clay/
May I mention Nimrod [1]? It has a lot going for it when it comes to meta-programming.
[1] http://nimrod-code.org/
[1] http://nimrod-code.org/
I can't find any kind of getting started guide, and most of the libs have no documentation at all.
I found a "Clay for C++ programmers" tutorial
https://github.com/jckarter/clay/wiki/Clay-for-C---programme...
https://github.com/jckarter/clay/wiki/Clay-for-C---programme...
I don't think this stands up: "It must be possible for such contender to be called from C code, because that is what it takes to be called from a modern scripting engine too."
Later the author says, "If programmers wanted a statically-typed, compiled system language for their scripting efforts, they would be using the existing system languages for that already."
I think the reason that scripting languages exist is because of limitations in the established languages. What we think of as "a scripting language" is defined by the gap. If someone else could be a system programming language without the gap - maybe it doesn't need to be accessed.
The traditional unix platform is (1) C and a compiler for it, with access to the unix standard library and (2) some sh or csh derivative, and access to the likes of sed and awk.
Shell gave people a glimpse of what was possible. But shell has poor access to the unix API, and hits the wall quickly once you do anything other than very basic tool development.
So these things called scripting languages started to pop up. People wanted easy access to the unix API like C. They want easy string manipulation like shell. And they want access to functions, maps and lists without having to think about memory allocation.
If you start from the shell and awk and work into that gap you get perl - the canonical scripting languge.
TCL inhabits a similar space to perl. It's also "a scripting language". But it started at the opposite end. TCL starts as an extension language for C.
So I can understand the author's emphsais on close interaction with C. But I don't think they're important if you can take away the shortfalls they're compensating for.
I hope Go will mean I don't have to think about these things. That I'll be able to get a standard build and a go compiler/runtime and have immediate access to lots of power. No more headaches from C build tools or reaching around in the murky water underneath my scripting language.
Later the author says, "If programmers wanted a statically-typed, compiled system language for their scripting efforts, they would be using the existing system languages for that already."
I think the reason that scripting languages exist is because of limitations in the established languages. What we think of as "a scripting language" is defined by the gap. If someone else could be a system programming language without the gap - maybe it doesn't need to be accessed.
The traditional unix platform is (1) C and a compiler for it, with access to the unix standard library and (2) some sh or csh derivative, and access to the likes of sed and awk.
Shell gave people a glimpse of what was possible. But shell has poor access to the unix API, and hits the wall quickly once you do anything other than very basic tool development.
So these things called scripting languages started to pop up. People wanted easy access to the unix API like C. They want easy string manipulation like shell. And they want access to functions, maps and lists without having to think about memory allocation.
If you start from the shell and awk and work into that gap you get perl - the canonical scripting languge.
TCL inhabits a similar space to perl. It's also "a scripting language". But it started at the opposite end. TCL starts as an extension language for C.
So I can understand the author's emphsais on close interaction with C. But I don't think they're important if you can take away the shortfalls they're compensating for.
I hope Go will mean I don't have to think about these things. That I'll be able to get a standard build and a go compiler/runtime and have immediate access to lots of power. No more headaches from C build tools or reaching around in the murky water underneath my scripting language.
You are leaving out the importance of libraries and coupling of systems that were initially written independently. This is the same mistake that many Lisps have made, assuming that just because the technology is "superior", people will flock to it and eagerly rewrite all their code (starting with their dependencies).
The C ABI is not great for interacting with scripting languages though, even if it is the best we have right now. For a start there is no type information obtained with dlsym and you basically have to parse headers the interpretation of which is not clearly defined as the C language really defines an API and the ABI is defined by the compiler. Many libraries for this reason do not define stable ABIs and are a pain to use from a scripting language. Hence Python has two ffi modes an ABI interface and a compiler mode.
Now if someone produced a better solution to that it would be useful...
Now if someone produced a better solution to that it would be useful...
You know, it's strange, because the Lua folks seem to be getting along just fine.
Yeah but us Lua folks are picky about the sort of C we like to interface to.
In practice it works quite well, if you restrict the ABI to function calls and word-sized values (long and void). It especially means not to use structs. Instead you need void and getter/setter functions.
Thats very limiting. A lot of applications need arrays (and maybe complex numbers) eg for numerical applications. And it is hard to manage without some sort of int64 type nowadays.
And also say you want performance, then using getters and setters sucks compared to direct struct access, as every access needs a non inlineable function call.
And also say you want performance, then using getters and setters sucks compared to direct struct access, as every access needs a non inlineable function call.
there's also ats [http://www.ats-lang.org/]. somewhere on my long-term todo list is to write a library of efficient data strucures in ats, to take advantage of some of its safety guarantees.
It's a pity ATS doesn't get more attention in the "Systems programming language" field. It can call into C, be called from C, provides a powerful type system, and can be used to write very low level code. I've written a few posts about how ATS can interact with C on my weblog here: http://www.bluishcoder.co.nz/tags/ats/
I like reading your blog posts on ATS, but I don't think I could use it effectively.
I've had much better luck with exhaustive testing with Klee and related libraries (e.g. for testing multi-threaded programs).
For whatever reason, they map better to how I think about problems, and I've found it easier to develop new code that is testable within those constraints.
I've had much better luck with exhaustive testing with Klee and related libraries (e.g. for testing multi-threaded programs).
For whatever reason, they map better to how I think about problems, and I've found it easier to develop new code that is testable within those constraints.
If you start off treating ATS like an ML variant interfacing with C without using any of the type annotations for safety you can go far. Then you can tighten the types up for safety as you get more experienced or find you need it.
You could write tests as needed and when a test fails write types that make that failure as close to compile time as possible.
Another approach is to just write C code, either embedded in ATS files or in C files that are linked into ATS. Then slowly convert C to ATS as you feel comfortable.
You could write tests as needed and when a test fails write types that make that failure as close to compile time as possible.
Another approach is to just write C code, either embedded in ATS files or in C files that are linked into ATS. Then slowly convert C to ATS as you feel comfortable.
Vala does not seem to get much attention. It borrows C# syntax (and maybe Java), compiles down to C and offers nearly symmetrical interoperability as a result. I believe Ubuntu's Unity is written in Vala. Knowing Java, it was easy to acquire the syntax from the tutorial [https://live.gnome.org/Vala/Tutorial] in very little time.
For transparent access to compiled C libraries, external C functions need to be declared in a 'vapi' file and although these were already provided for a large part of the GNOME ecosystem, documentation on vapi syntax was nearly non-existent. Object-orientation and standard libraries are integrated with Gnome's GObject system and Glib libraries which I found unfamiliar and usually better documented in C at the time.
For transparent access to compiled C libraries, external C functions need to be declared in a 'vapi' file and although these were already provided for a large part of the GNOME ecosystem, documentation on vapi syntax was nearly non-existent. Object-orientation and standard libraries are integrated with Gnome's GObject system and Glib libraries which I found unfamiliar and usually better documented in C at the time.
If it has GObject as a dependency, then it's not designed for "low-level systems programming". Hell, just having a GC more-or-less discounts it due to the associated problems with runtime size and performance[0]. As a general rule of thumb: if you can't write a kernel in it, it's not a systems language.
[0] It is possible to write a systems language with these features, but low-level systems stuff is not what Vala was designed for.
[0] It is possible to write a systems language with these features, but low-level systems stuff is not what Vala was designed for.
GObject itself is not a dependency, Inheriting from GLib.GObject provides support for features like introspection and better integration with the collections framework and GLib libraries. However, lightweight 'non-object classes' are supported but treated differently by the compiler. There is a slight variation in constructor syntax between plain objects and GObjects.
As far as I know there is no Vala runtime or garbage collector. Memory management is by automatic reference counting but there is support for manual C++-style pointer syntax with new and delete operators.
I agree that Vala was not specifically designed for 'low-level systems stuff' but I see no reason why this is not technically possible. The Vala compiler is a source-to-source translator to C and in my previous experience worked fairly well interfacing with fairly low-level C libraries like MPI and OpenCL. I assume that a kernel writer does not require much more than a limited subset of the language that supports C pointers, structs and arrays.
As far as I know there is no Vala runtime or garbage collector. Memory management is by automatic reference counting but there is support for manual C++-style pointer syntax with new and delete operators.
I agree that Vala was not specifically designed for 'low-level systems stuff' but I see no reason why this is not technically possible. The Vala compiler is a source-to-source translator to C and in my previous experience worked fairly well interfacing with fairly low-level C libraries like MPI and OpenCL. I assume that a kernel writer does not require much more than a limited subset of the language that supports C pointers, structs and arrays.
This is called FFI.)
And yes, a language with which you cannot dload and call any shared library you wish is, how to say, handicapped language.
NodeJS, Java to name a few. Look what they do due to inability to call, say, libpq.so.)
But of course, re-monkey-patching everything from scratch by amateurs coders is much better idea. And, you know, you could create an entire "standard" with hundreds of pages of specification and then push your bloatware - JDBC.
And yes, a language with which you cannot dload and call any shared library you wish is, how to say, handicapped language.
NodeJS, Java to name a few. Look what they do due to inability to call, say, libpq.so.)
But of course, re-monkey-patching everything from scratch by amateurs coders is much better idea. And, you know, you could create an entire "standard" with hundreds of pages of specification and then push your bloatware - JDBC.
Au contrare it is C which is handicapped.
How handicapped is C++ considering it subsumes C, can use C right at the source level, has an inline FFI, and yet C++ libraries can't be called by C because C++ provides type-safe linkage.
Advanced languages are advanced because they improve on C and this inevitably means they must go beyond C. This is why such languages require setting up an environment, usually by providing main().
Even if you could set up an environment with a C function call, which is trivially achieved by replacing main with mymain(), you would simply be starting a foreign environment, not providing a library of functions which could be individually called by C.
My system Felix (http://felix-lang.org) is specifically designed to live with the compromises of C/C++ compatibility, but you still cannot put arbitrary Felix code in C callable functions because Felix provides an advanced environment supporting, for example, fibres with channels. Fibres are scheduled by returning control to a scheduler so they cannot be nested inside a function which puts its return address on the machine stack.
If you Felix write code without using features requiring advanced support that code can be made into a C callable library (both static and dynamic linking is supported).
Felix is probably better at C/C++ integration than any other language except C++.
How handicapped is C++ considering it subsumes C, can use C right at the source level, has an inline FFI, and yet C++ libraries can't be called by C because C++ provides type-safe linkage.
Advanced languages are advanced because they improve on C and this inevitably means they must go beyond C. This is why such languages require setting up an environment, usually by providing main().
Even if you could set up an environment with a C function call, which is trivially achieved by replacing main with mymain(), you would simply be starting a foreign environment, not providing a library of functions which could be individually called by C.
My system Felix (http://felix-lang.org) is specifically designed to live with the compromises of C/C++ compatibility, but you still cannot put arbitrary Felix code in C callable functions because Felix provides an advanced environment supporting, for example, fibres with channels. Fibres are scheduled by returning control to a scheduler so they cannot be nested inside a function which puts its return address on the machine stack.
If you Felix write code without using features requiring advanced support that code can be made into a C callable library (both static and dynamic linking is supported).
Felix is probably better at C/C++ integration than any other language except C++.
I believe his thesis was actually that most languages are handicapped because they can not easily be called via the usual "load a .so file and call a function" ABI. Java, for instance, would be a more useful language to me if I could easily create Python bindings for useful Java libraries like Lucene, without needing to use something like Jython that runs on the JVM.
The confusion arises when some language developers decide to ignore or break existing ABI for a particular platform.
There are general ways to call procedures - there machine instructions for this. ABI defines how exactly parameters and return values should be passed - using, registers or stack or both.
As long as you follow the rules there is no difficulty in calling everything you wish.
The whole idea of JVM as something disconnected from reality^W OS and hardware is simply wrong, and all the confusion is the consequence of that premature decision.
Of course, slogan "you don't have to know" always wins.
There are general ways to call procedures - there machine instructions for this. ABI defines how exactly parameters and return values should be passed - using, registers or stack or both.
As long as you follow the rules there is no difficulty in calling everything you wish.
The whole idea of JVM as something disconnected from reality^W OS and hardware is simply wrong, and all the confusion is the consequence of that premature decision.
Of course, slogan "you don't have to know" always wins.
tl;dr: use JNI through pyjnius of jpype.
You can run Python 2.5 on lastest Jython, then you have access to JVM libs and the JVM libs to Python libs 2.5 compatible.
Calling JVM from CPython is actually possible: - with pyjnius from kivy fame built with cython it is still in development, AFAIK it is missing the possibility to call Python code from Java code aka. submitting a Python "callback" to a Java method, I think it is called IoC pattern and Hollywood principle «Don't call me, I call you». jnius is used in Kivy to make it possible to call Android libs. As an example, I created Python bindings of Blueprints with it. Blueprints is an abstraction library for graphdb like Neo4j, OrientDB, Titan, available here https://github.com/Printemps/python-blueprints/. Tutorial: «JavaClass = autoclass('path.to.JavaClass'); JavaClass.JavaMethod(python_object)». Those bindings are merely Pythonification but they are gotchas in types conversions.
- There is also jpype which looks like an established solution used by Neo4J fame to make it possible to embed Neo4J embedded into CPython.
I choosed jnius because it's faster; Those are funny benchmarks in log scale: https://docs.google.com/spreadsheet/ccc?key=0Ajc7MnpSjCokdGt...
You can run Python 2.5 on lastest Jython, then you have access to JVM libs and the JVM libs to Python libs 2.5 compatible.
Calling JVM from CPython is actually possible: - with pyjnius from kivy fame built with cython it is still in development, AFAIK it is missing the possibility to call Python code from Java code aka. submitting a Python "callback" to a Java method, I think it is called IoC pattern and Hollywood principle «Don't call me, I call you». jnius is used in Kivy to make it possible to call Android libs. As an example, I created Python bindings of Blueprints with it. Blueprints is an abstraction library for graphdb like Neo4j, OrientDB, Titan, available here https://github.com/Printemps/python-blueprints/. Tutorial: «JavaClass = autoclass('path.to.JavaClass'); JavaClass.JavaMethod(python_object)». Those bindings are merely Pythonification but they are gotchas in types conversions.
- There is also jpype which looks like an established solution used by Neo4J fame to make it possible to embed Neo4J embedded into CPython.
I choosed jnius because it's faster; Those are funny benchmarks in log scale: https://docs.google.com/spreadsheet/ccc?key=0Ajc7MnpSjCokdGt...
I'm so happy Java can't load libpq you won't believe.
Because it means that Postgresql API gets reimplemented as a crispy JVM code that never crashes my JVM.
If we would live in the world where a Java program would use libpq and libxml and libldap and whatever, it would be so much less stable, introspective and consistent that it won't be Java anymore. It would be a particularly ugly kind of Python.
And yes, it would crash when you use threads. Since all those C libraries pretend that threads don't matter. Guess what, our Java program has 500 threads and feels fine.
And yes, it would crash when you use threads. Since all those C libraries pretend that threads don't matter. Guess what, our Java program has 500 threads and feels fine.
Technically, an instance of JVM is mere a user process, which, by definition, must obey "laws" for user processes. This could be end of story - the idea that something "reliable" could be implemented as a single user level process is, well, naive.
btw, you still must call libc.so and libpthread.so
As for pthreads, guess what, they are also bad decision, as long as they are not isolated as processes supposed to be, and crash of one thread messes up rest 499.
People who did some research, like developers of Erlang, found that neither one-process model or threaded-threaded process model is even applicable when you need a reliable system.
So, again, jvm.exe is mere yet another .exe.)
btw, you still must call libc.so and libpthread.so
As for pthreads, guess what, they are also bad decision, as long as they are not isolated as processes supposed to be, and crash of one thread messes up rest 499.
People who did some research, like developers of Erlang, found that neither one-process model or threaded-threaded process model is even applicable when you need a reliable system.
So, again, jvm.exe is mere yet another .exe.)
C code can call into rust: http://static.rust-lang.org/doc/rust.html#extern-functions
True, and so can C call back into go code with cgo. The problem is that it still requires that the main function be in a Rust/Go binary. A program written in C can't currently call into a library written in one of these two languages since the runtimes don't work unless they control the main entry point.
From the Rust doc linked: "The primary motivation of extern functions is to create callbacks for foreign functions that expect to receive function pointers."
From the Rust doc linked: "The primary motivation of extern functions is to create callbacks for foreign functions that expect to receive function pointers."
Then is it even possible to make a program that can use libraries written in Rust and Go and C++? If everyone wants to control main(), is there an escape hatch provided by any of these languages, like a C function that can be explicitly invoked near program start?
You can do this with C++ already. Rust's plan seems to be to eliminate or reduce its runtime and give up the entry point. Go doesn't appear to have a plan.
If you read what you linked to it says the main purpose is to provide C functions with callback arguments when called from Rust. It isn't clear from what you have written that this mechanism is available to generally access from C programs and scripting languages.
[deleted]
So what's the catch when it comes to D? If I can use all C libs easily, and get some more palpable syntax, what am I missing exactly and why haven't people transitioned to it?
The article is not about using C libs, is about C programs using you (your D code, that is).
That said, the problem with D: not enough marketing plus the two compilers / 2 libraries issue.
And it's close to C++ complexity enough to not let the scripting guys feel welcome (like they do in Go).
That said, the problem with D: not enough marketing plus the two compilers / 2 libraries issue.
And it's close to C++ complexity enough to not let the scripting guys feel welcome (like they do in Go).
Compatability with the C ABI I feel isn't a necesssity. On platforms built on C it is, but if you wanted to invent a new machine running a new system language with the ABI messes of C exonerated, you could. You would need to devise a translation layer in the OS executor to be able to interpret classic C ABI .so or .dll files for legacy applications and so you could compile C on these platforms, but that isn't a prohibitively hard challenge.
The need for C isn't so much restricted by the operating system, but by the hardware manufacturers. The simple fact is that C is the defacto choice of language for hardware vendors to ship a compiler for, and that doesn't look to change any time. It's a standard that can't be avoided, unless we can work cooperatively with enough major vendors to change this.
There is C specific hardware?!?
This is a myth and part of the vicious circle here. I'm not a C hater by any means but the reason hardware vendors have historically included it is because there are free compilers that are nearly C enough that they can include and they have to include something. C is just a portable assembly, you can almost write a simplistic compiler in pure yacc and more than a couple of these C compilers hardware vendors produced weren't much more than that. Software guys then wrote in C and the circle continues, intel has a great C compiler now and in fact all the best optimizers are from hardware vendors but that wasn't always the case. There was also a time (quite possibly it's still that time) when you'd buy some customed up MIPS or PowerPC chip and get an ancient version of gcc that they'd hacked to support their shit and forward porting to a modern GCC wasn't always a straight forward thing to do. They have to provide something and tools usually isn't their business.
Now so much is built on the C non-runtime that Go, Ada, Rust, anything with any runtime looks completely backwards to a lot of system folk. What's funny, you look at Linux and there is a fairly small hardware dependent directory of code where the register poking happens, it's really not that much code but its called from all the drivers and such, and that's the holdup. It's not like that stuff makes up 50% of the code or anything.
This is a myth and part of the vicious circle here. I'm not a C hater by any means but the reason hardware vendors have historically included it is because there are free compilers that are nearly C enough that they can include and they have to include something. C is just a portable assembly, you can almost write a simplistic compiler in pure yacc and more than a couple of these C compilers hardware vendors produced weren't much more than that. Software guys then wrote in C and the circle continues, intel has a great C compiler now and in fact all the best optimizers are from hardware vendors but that wasn't always the case. There was also a time (quite possibly it's still that time) when you'd buy some customed up MIPS or PowerPC chip and get an ancient version of gcc that they'd hacked to support their shit and forward porting to a modern GCC wasn't always a straight forward thing to do. They have to provide something and tools usually isn't their business.
Now so much is built on the C non-runtime that Go, Ada, Rust, anything with any runtime looks completely backwards to a lot of system folk. What's funny, you look at Linux and there is a fairly small hardware dependent directory of code where the register poking happens, it's really not that much code but its called from all the drivers and such, and that's the holdup. It's not like that stuff makes up 50% of the code or anything.
Of course there is C specific hardware!
X86 provided efficient call/return protocol instructions including display management, specifically targeting Pascal/Modula style languages. Microsoft Windows 3.1 even mandated this calling protocol. However old style C didn't use this API because it doesn't work with varargs and C doesn't have nested functions.
Instead Intel observed real code and optimised their instruction set to speed it up, locking everyone into lame languages like C. Attempts by Intel to break from this, for example with x64, have failed (x64 has two stacks).
Similarly segmented architectures, which implement extension of the Harvard machine model, have been largely unsupported in favour of linear addressing von Neumann machines, primarily because that's what C and Unix use. Interesting to see Microsoft adapt their OS designs based on this trend.
To support more advanced features in a language often requires only a few very low level primitives. This is why Linux kernel doesn't need much assembler, to do things like stack swapping. Control exchange is one of those things completely missing from C.
X86 provided efficient call/return protocol instructions including display management, specifically targeting Pascal/Modula style languages. Microsoft Windows 3.1 even mandated this calling protocol. However old style C didn't use this API because it doesn't work with varargs and C doesn't have nested functions.
Instead Intel observed real code and optimised their instruction set to speed it up, locking everyone into lame languages like C. Attempts by Intel to break from this, for example with x64, have failed (x64 has two stacks).
Similarly segmented architectures, which implement extension of the Harvard machine model, have been largely unsupported in favour of linear addressing von Neumann machines, primarily because that's what C and Unix use. Interesting to see Microsoft adapt their OS designs based on this trend.
To support more advanced features in a language often requires only a few very low level primitives. This is why Linux kernel doesn't need much assembler, to do things like stack swapping. Control exchange is one of those things completely missing from C.
I wasn't implying that there's C specific hardware, or that there's any technical restriction for shipping a C compiler with a chip. You're just reiterating my point about it being crowd effect.
Also, when talking about systems programming, we can't just limit our scope to linux - there's hundreds of other kernel projects, and embedded systems which don't have the luxury of a massive portable codebase already - they only have a basic C compiler, library, debugger.
Using a different language doesn't eliminate C, because chances are, the compiler you're using for that language has been written in C, because it's all the manufacturer supplied. Compiler vendors contribute to this cycle by expecting their compiler to run on any platform if they write it in C.
I was suggesting that if we want to liberate ourselves from C, we need the hardware vendors on board too - since they're where the cycle begins.
Of course, you don't need the vendor specific compiler if you have a cross compiler, and can build a different language compiler from another platform - however, if you look at what's available for doing that, there's nothing other than GCC - which is why new languages are unlikely to replace C as the defacto systems language, unless they can replicate that functionality.
Also, when talking about systems programming, we can't just limit our scope to linux - there's hundreds of other kernel projects, and embedded systems which don't have the luxury of a massive portable codebase already - they only have a basic C compiler, library, debugger.
Using a different language doesn't eliminate C, because chances are, the compiler you're using for that language has been written in C, because it's all the manufacturer supplied. Compiler vendors contribute to this cycle by expecting their compiler to run on any platform if they write it in C.
I was suggesting that if we want to liberate ourselves from C, we need the hardware vendors on board too - since they're where the cycle begins.
Of course, you don't need the vendor specific compiler if you have a cross compiler, and can build a different language compiler from another platform - however, if you look at what's available for doing that, there's nothing other than GCC - which is why new languages are unlikely to replace C as the defacto systems language, unless they can replicate that functionality.
Well, what sort of compiler optimizations you can make with C do have an effect on how people create microprocessors. For instance if you could be sure you could always disambiguate stack and heap accesses you could use two separate and somewhat memory pipes and get a big power or performance win, since there are a lot of N^2 structures involved in keeping everything coherent.
Also, most computers have a GPU which can't really be programmed in straight C.
Also, most computers have a GPU which can't really be programmed in straight C.
Doesn't almost all systems programming take place on platforms built on C? If creating a new machine and new language, convincing the existing systems programmers to move is going to take something very compelling.
Historically, the "turtles all the way down" approach has not been very successful outside of niches. Lisp, Forth, Java have all been used for OS's that never seemed to go anywhere.
I've always wondered about the possibility of creating a stack system, application, and scripting language all designed to work together though.
I'd much rather see three distinct languages, with a shared base syntax and style (ie, common formatting of comments, common scope delimiters, common operand behavior - something like C / Java / Javascript).
That way, you could have your low level language use pointers, nullables, inline assembly, goto and other "dangerous" things that are the most efficient ways to do stuff. You wouldn't want to have that language crippled like modern C++ though, you would still want modules, a well planned object oriented syntax, a better solution to name mangling, and better readability.
You could have an interpreted Java/C# esque applications language that would mostly be similar to modern Java/C#, maybe with auto, a better namespaces implementation (in C#, a class defined in two places with the same namespace and signature have situational behavior that gets really messy, for example). You would use this language for the application layer.
Scripting could just be an autoboxed dynamic language like Python. Maybe have a whitespace significant and insignificant dialect. I'd like to see dialects of such a language, where the context it runs from dictates available modules. Since it would be permissions based, it could function as a shell, web, and glue language depending on where you use it.
I do agree on this sentiment though. The wide base of languages in modern computing makes the whole thing a lot messier than it needs to be, and the lack of any full stack that really ties together nicely and is well thought out without the smudges would be really nice.
That way, you could have your low level language use pointers, nullables, inline assembly, goto and other "dangerous" things that are the most efficient ways to do stuff. You wouldn't want to have that language crippled like modern C++ though, you would still want modules, a well planned object oriented syntax, a better solution to name mangling, and better readability.
You could have an interpreted Java/C# esque applications language that would mostly be similar to modern Java/C#, maybe with auto, a better namespaces implementation (in C#, a class defined in two places with the same namespace and signature have situational behavior that gets really messy, for example). You would use this language for the application layer.
Scripting could just be an autoboxed dynamic language like Python. Maybe have a whitespace significant and insignificant dialect. I'd like to see dialects of such a language, where the context it runs from dictates available modules. Since it would be permissions based, it could function as a shell, web, and glue language depending on where you use it.
I do agree on this sentiment though. The wide base of languages in modern computing makes the whole thing a lot messier than it needs to be, and the lack of any full stack that really ties together nicely and is well thought out without the smudges would be really nice.
If programmers wanted a statically-typed, compiled system language for their scripting efforts, they would be using the existing system languages for that already.
I think Go has been fairly successful at Google in terms of improved code maintainability, and as an alternative to not just C but C++.
I think Go has been fairly successful at Google in terms of improved code maintainability, and as an alternative to not just C but C++.
It seems to me that they are, in fact, using "statically-typed, compiled system language[s] for their scripting efforts" - that is in fact what people are doing with server-side Java, Scala and C#, not to mention Java/Dalvik and Objective-C on mobile platforms.
Perhaps this article misses the point a bit. For several decades we have been using scripting languages to glue together sharp pointy C[++] tools (sh/Perl/Python/Ruby approach) or to puppetize monolithic C[++] hulks from within (VB/elisp approach).
More recent practice, on both mobile and server platforms, has been to abstract away from C[++] with a middle layer of memory-managed, compiled C-ish goop - JVM, .Net, Objective-C, whatever. This is the level where people are doing everything from amazing high-availability big data things to amazing little bleep-bloop handheld things. I think the answer to "what's the state of low-level systems programming?" is "not much is going on, but damn, there's a lot of fresh stuff happening in mid-level systems programming."
Go and Rust are late entrants to this party, and probably exist more as a hedge against Oracle litigating Java/JVM into the ground than anything else.
Perhaps this article misses the point a bit. For several decades we have been using scripting languages to glue together sharp pointy C[++] tools (sh/Perl/Python/Ruby approach) or to puppetize monolithic C[++] hulks from within (VB/elisp approach).
More recent practice, on both mobile and server platforms, has been to abstract away from C[++] with a middle layer of memory-managed, compiled C-ish goop - JVM, .Net, Objective-C, whatever. This is the level where people are doing everything from amazing high-availability big data things to amazing little bleep-bloop handheld things. I think the answer to "what's the state of low-level systems programming?" is "not much is going on, but damn, there's a lot of fresh stuff happening in mid-level systems programming."
Go and Rust are late entrants to this party, and probably exist more as a hedge against Oracle litigating Java/JVM into the ground than anything else.
>I think Go has been fairly successful at Google in terms of improved code maintainability, and as an alternative to not just C but C++.
Google at large barely uses the language. All the examples they've given are of peripheral stuff, like a load balancer for MySQl for YouTube and an even simpler replacement for an aged and cranky C++ system for Google Downloads. Nothing to write home about, and no replacing C++/Java for core systems at all.
Google at large barely uses the language. All the examples they've given are of peripheral stuff, like a load balancer for MySQl for YouTube and an even simpler replacement for an aged and cranky C++ system for Google Downloads. Nothing to write home about, and no replacing C++/Java for core systems at all.
The state of programmers trying to avoid coding in the language their system is coded in. C.
For example, Java and C# cannot walk like a duck, not even if that is what it would take to save themselves from drowning. Therefore, they are absolutely unsuitable as system languages.
Hadoop held the world record for sorting a Terrabyte for a while. If sorting large amounts of data as fast as possible isn't a system's programming task, then I'm a three-headed monkey.
Hadoop held the world record for sorting a Terrabyte for a while. If sorting large amounts of data as fast as possible isn't a system's programming task, then I'm a three-headed monkey.
>The Go language and the Rust language. What they have in common, though, is that both Go and Rust are entirely incapable of walking like a duck. Since you cannot call Rust -or Go code from a scripting engine, they are inadvertently positioning themselves as alternatives to the scripting engines themselves.
Before writing all these BS, maybe he should do a little research first?
Here's Rust (which is not even released yet) being capable of being called from C. It's a feature they are actively work on:
https://github.com/mozilla/rust/issues/1732
Before writing all these BS, maybe he should do a little research first?
Here's Rust (which is not even released yet) being capable of being called from C. It's a feature they are actively work on:
https://github.com/mozilla/rust/issues/1732
> Before writing all these BS, maybe he should do a little research first?
There's a subtle difference between being able to call back into Go/Rust code from C (where the main function is in Go/Rust) which both languages allow now, and allowing a call to originate from a C program, which neither currently do. The author is talking about the later.
I would agree the article could go into more detail about what's currently possible, and why it's not enough. But calling BS, I think, is going a bit too far.
There's a subtle difference between being able to call back into Go/Rust code from C (where the main function is in Go/Rust) which both languages allow now, and allowing a call to originate from a C program, which neither currently do. The author is talking about the later.
I would agree the article could go into more detail about what's currently possible, and why it's not enough. But calling BS, I think, is going a bit too far.
>and allowing a call to originate from a C program, which neither currently do.
That one is also under consideration by the Rust developers.
That one is also under consideration by the Rust developers.
it's probably an interesting read except that being hosted on google's blogspot means it only display blank pages due to content being embedded in some javascript shenanigans.
Even a screenshot would be better than this.
Please move on to some actual html website that just works or rehost the content elsewhere before posting it here. Thanks.
Even a screenshot would be better than this.
Please move on to some actual html website that just works or rehost the content elsewhere before posting it here. Thanks.
I used to be one of These Guys who never used JavaScript, too, but honestly? It's 2012 and JavaScript is a thing pretty much everywhere worth targeting. Demanding that somebody cater to your microsegment is a little much.
Although I'm the kind of guy who usually have JS disabled I could be made to see your point. But this page is actually a good example of why I run with JS disabled. It magically positioned links and boxes over the scrolling bar and caused my initial attempt at scrolling down to navigate away from the page.
Disabling JS usually have the effect of doing away with such nonsense but here that is no option.
This time it was worth the hassle since it was an interesting article, but I definitely prefer it when JS is optional.
Disabling JS usually have the effect of doing away with such nonsense but here that is no option.
This time it was worth the hassle since it was an interesting article, but I definitely prefer it when JS is optional.
> rehost the content elsewhere before posting it here
Google's blogspot allows you to do normal html blogs (maybe now they've made this their "default", but still, it's OP's choice to have it this way)
Google's blogspot allows you to do normal html blogs (maybe now they've made this their "default", but still, it's OP's choice to have it this way)
My attempts to read this on my phone were futile. Every time I tried to scroll, it just jumped back to the top immediately.
1. M:N scheduling. We wanted high scalability with hundreds of thousands or millions of tasks, which means we need the capability to have our own scheduler. OS threads aren't lightweight enough for this, especially on Windows, where over 80% of our Firefox users are. Having our own scheduler means we need our own ABI. We would like to make the scheduler optional, but getting it working was a higher priority than making it optional.
2. Safety. On many platforms (basically, anything but MSVC on Windows), the C stack is not bounds-checked; either there is no guard page or the guard page is unreliable because you can overshoot it. This is not acceptable for our Rust code, since stack overflows are a frequent source of exploitable security vulnerabilities for us. So we do our own stack management with a custom ABI that performs stack checks at every function entry, freeing us from this source of pain on all operating systems. Again, this is planned to be optional, but it was more important at this early stage for us to demonstrate advantages over C than for us to duplicate its ABI exactly.
Many libraries, however, would prefer direct interoperability with C to M:N scheduling or stack overflow safety. So we plan to provide a mechanism for Rust code to be called from C without having to set up a scheduler or perform stack checks. It's not implemented yet, but it's very much part of the current design, and I would be thrilled to see it happen.