A Module System for C++ (Revision 2) [pdf](isocpp.org)
isocpp.org
A Module System for C++ (Revision 2) [pdf]
http://isocpp.org/files/papers/n4214.pdf
8 comments
The major reason of the awfully slow compilation (and linking) of the bigger C++ projects is all the templates in the stl or (deity forbids) boost that are used in every compilation unit. So I'd like to know if using this new "sugar" can make the compilation of the existing programs faster once the compiler and linker are made to recognize that sugar. Skimming through the pdf, I wasn't able to figure out from where the speed up is supposed to come. "Compiled once" in the object code sense doesn't matter for stl or boost since everything is in headers: there's simply no object code until you start to compile the cpps which include the headers. Can anybody explain what they are trying to do? Are they attempting to solve the templates problem or are they effectively just making a new form of the "lib" files?
At a minimum this can give you a much better and more modular version of precompiled headers. Because modules are guaranteed not to be affected by macros or ordering of module imports in the importing translation units you can get a modularized version of precompiled headers.
Don't underestimate the cost of just reading in all that text and parsing it. Modules allow you to cache some pre-parsed AST or similar internal representation of any exported template definitions which should greatly reduce the cost of instantiating them in the importing translation unit due to savings in IO and parsing time. Importantly, importing translation units never even have to look at definitions they don't use whereas currently they have to fully parse everything that gets dragged in by the headers they include even if they only use a tiny subset of the functionality.
A module enabled STL with well implemented compiler and linker support should greatly reduce build times for codebases that make heavy use of STL by simply replacing #include <header> with import std.header. The proposal eliminates all the unnecessary overhead of templates (re-parsing huge amounts of text) and leaves only the necessary / desirable overhead (instantiation from some internal representation with the specific types used). It's that final instantiation that gives templates their runtime performance advantages over other models (like C# generics) so you wouldn't want to lose that.
Don't underestimate the cost of just reading in all that text and parsing it. Modules allow you to cache some pre-parsed AST or similar internal representation of any exported template definitions which should greatly reduce the cost of instantiating them in the importing translation unit due to savings in IO and parsing time. Importantly, importing translation units never even have to look at definitions they don't use whereas currently they have to fully parse everything that gets dragged in by the headers they include even if they only use a tiny subset of the functionality.
A module enabled STL with well implemented compiler and linker support should greatly reduce build times for codebases that make heavy use of STL by simply replacing #include <header> with import std.header. The proposal eliminates all the unnecessary overhead of templates (re-parsing huge amounts of text) and leaves only the necessary / desirable overhead (instantiation from some internal representation with the specific types used). It's that final instantiation that gives templates their runtime performance advantages over other models (like C# generics) so you wouldn't want to lose that.
Exactly. Essentially this reduces the algorithmic complexity of compilation which is a huge boon given that even simple programs these days result in the (repeated) parsing of millions of lines of code. Instantiation of templates obviously won't change, but recursive traversal and repeated re-parsing of includes will be completely unnecessary as the compiler can trivially[1] cache the intermediate version. Which is as it should have been from the start, but then hindsight is 20-20.
It's really an elegant solution considering the constraints they're working under.
[1] Preprocessor macros don't leak from includers to includees, so headers become independent of each other.
It's really an elegant solution considering the constraints they're working under.
[1] Preprocessor macros don't leak from includers to includees, so headers become independent of each other.
I won't say that's been solved, but extern template is supposed to address it (well, exporting templates was supposed to address it, but that never really got off the ground).
Borland came up with an approach that instantiated templates multiple times, and then weeded out duplicates. That was relatively easy to implement, so everyone else followed suit. Extern template makes it possible to make sure some instantiations are instantiated only once, so the compiler doesn't have to weed out extra instantiations (I wish I could use "instantiate" fewer times in that description, but I can't see how to pull that off).
Borland came up with an approach that instantiated templates multiple times, and then weeded out duplicates. That was relatively easy to implement, so everyone else followed suit. Extern template makes it possible to make sure some instantiations are instantiated only once, so the compiler doesn't have to weed out extra instantiations (I wish I could use "instantiate" fewer times in that description, but I can't see how to pull that off).
This is useless syntactic sugar unless there is a forward compatible ABI with comes with it. That includes modules being able to use different versions of a dynamically linked libstdc++, i.e. modules compiled with different compiler versions. Which will never happen given the resistance from compiler folks to agree on a standard ABI.
This. And the reason that C++ compilation is so painfully slow is that unlike languages who saw C++'s problems and corrected them (Java, C#), a C++ compiler can not glean definitions from compiled code.
In Java, the compiler reads a Java file once, produces a class file and the class file contains all the information needed to compile other files that refer to its contents.
In C++, the compiler has to read all of source declarations that are used by a compilation unit, over and over again. An arbitrary cpp file can transitively include tens to hundreds of header files. Modern C++ compilers can play caching tricks, but the model is intrinsically O(n^2) compilation rather than O(n) compilation where n is the number of source files.
In Java, the compiler reads a Java file once, produces a class file and the class file contains all the information needed to compile other files that refer to its contents.
In C++, the compiler has to read all of source declarations that are used by a compilation unit, over and over again. An arbitrary cpp file can transitively include tens to hundreds of header files. Modern C++ compilers can play caching tricks, but the model is intrinsically O(n^2) compilation rather than O(n) compilation where n is the number of source files.
There are a lot of reasons why compiling C++ is inherently slow, but one big one is having a text preprocessor implementing an entirely different language tacked on the front.
There is talk of defining a stable ABI within platforms [1]. I'm not particularly fond of the current proposal, but there seems to be interest. We'll have to see how it plays out.
[1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n402...
[1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n402...
The short story is that a "module" should be approximately the same thing as a "compilation unit". (Not a compilation unit, plus some fragmented-off textual header file.). Secondly, information about the public interface of that compilation unit must somehow be marked off, and provided for the compilation of dependent modules.
A more digestible description of this can be found in a Modula 2 reference manual:
http://www.modula2.org/reference/modules.php
The Modula 2 compiler I worked with years ago simply had you put the definition modules into .def files and the implementation modules into .mod files. So it's vaguely like headers, except that the .def files have structure and can be compiled; the compiled info is then referenced during compilation of the client modules. The .def file is never directly referenced as text.
A more digestible description of this can be found in a Modula 2 reference manual:
http://www.modula2.org/reference/modules.php
The Modula 2 compiler I worked with years ago simply had you put the definition modules into .def files and the implementation modules into .mod files. So it's vaguely like headers, except that the .def files have structure and can be compiled; the compiled info is then referenced during compilation of the client modules. The .def file is never directly referenced as text.
Modules go at least all the back to Algol 68 and Mesa, but C designers choose to ignore them.
C++ design goal of being able to target unmodified C toolchains as a means of being accepted by C developers, meant C++ also had to keep away from modules.
Luckily C++ can stand on its own nowadays and depart from such decisions.
C++ design goal of being able to target unmodified C toolchains as a means of being accepted by C developers, meant C++ also had to keep away from modules.
Luckily C++ can stand on its own nowadays and depart from such decisions.
Turbo Pascal used a similar scheme, but they split every source file into an "interface" and an "implementation" section. The interface would end up as information exported to other modules, but in binary form. No other part of the process had to read the original source file.
It's similar in Ada.
Fascinating reading as usual. The C++ articles that get posted on here really are enlightening.
Module system, uniform function call syntax (UFCS) -- it all starts looking more and more like D :)
I just wonder, if waiting until C++17 gets done and available across all major compilers in all platforms, not just the C++ triad that gets discussed on HN, won't be a kind of "too little, too late" kind of thing.
I think people who are invested in C++ now won't suddenly stop being invested in it by the time C++17 rolls around.
Many people are invested in C++, because in the last 20 years there was an investment in VM based runtimes, effectively leaving C and C++ as the only to go languages if one was looking for ahead of time compilation[0].
Now with Java and .NET upcoming adoption of ahead of time compilation to native code in the next versions.
Additionally with other languages like D, Rust, Go, Swift, Ada, Haskell, OCaml fighting for a place at the Sun.
There will be lots of options for those looking into languages with support for ahead-of-time compilation to native code in their reference implementations.
Thus pushing C++ deeper into the OS realm, while at the same time giving to those people other options to look for.
Hence my remark.
[0] There were other options, but they were largely ignored by the mainstream.
Now with Java and .NET upcoming adoption of ahead of time compilation to native code in the next versions.
Additionally with other languages like D, Rust, Go, Swift, Ada, Haskell, OCaml fighting for a place at the Sun.
There will be lots of options for those looking into languages with support for ahead-of-time compilation to native code in their reference implementations.
Thus pushing C++ deeper into the OS realm, while at the same time giving to those people other options to look for.
Hence my remark.
[0] There were other options, but they were largely ignored by the mainstream.
The thing is: If you use a non-C/C++ language you'll inevitably have to interface with some C or C++ code (your OS API, whether that be POSIXish or whatever.). There can be a lot of friction/impedance mismatch at this point, especially if your language uses novel ways of managing resources and/or memory. I'm guessing this is why a lot of Application-level GUI code still gets written in C/C++, e.g. in Qt, KDE or GNOME.
"Worse is Better" is Worse in the long run, I guess.
"Worse is Better" is Worse in the long run, I guess.
I agree, and am cautiously optimistic that people will have more options in or close to C++'s niche in the future. My comment was made with regards to the (surely significant?) market share of people who opted for C++ in the past and are stuck with it for better or worse.
But even just considering new projects, I'd expect that C++'s inertia in terms of reputation, ecosystem and people's skillset/experience means that C++17 or even later will have plenty of time to measure up against the competition.
But even just considering new projects, I'd expect that C++'s inertia in terms of reputation, ecosystem and people's skillset/experience means that C++17 or even later will have plenty of time to measure up against the competition.
Sure! Myself I always liked it, just prefer to explore other options as well.
You are right, though.
You are right, though.
The problem is not just ahead of time compilation. This has been available for a long time on Pascal, OCaml, Ada, and even Lisp. C++ offers access to lower level programming constructs that makes it very efficient to program things like games, numeric algorithms, device drivers, desktop applications, and many more areas that depend on these C++ features.
Apparently you didn't read my footnote, those are some of the languages ignored by the mainstream.
Funny that you list Ada and then ignore it offers the same features for systems programming as C++.
Funny that you list Ada and then ignore it offers the same features for systems programming as C++.
Cobol programmers were excited about Cobol2014 updates.
C++ will continue to be used, much like Cobol has continued to be used for an eternity.
C++ will continue to be used, much like Cobol has continued to be used for an eternity.
Is that a fair comparison, though? Do we really see people writing significant new software using Cobol as we see using C++? We don't. Right now we're all probably running a lot of software written in C++ (hello, browser!) but none in Cobol.
You'd be surprised.
I've got coworkers who's first job in the programming business was Cobol maintenance. Old programmers from the 1960s are retiring in their 80s, and they need maintainers to write new code for various ancient mainframe systems.
Do remember, IBM is still selling System/360 systems today. http://en.wikipedia.org/wiki/IBM_System_z
These systems are running Cobol code from the 1960s, and they need to be updated to include XML, REST and all the other latest developments of development.
http://www.itworldcanada.com/article/the-future-of-cobol-why...
Similarly, C++ programmers are probably going to have an even _larger_ legacy than Cobol.
Cobol is very much alive today.
http://www.veryant.com/
In fact by some measurements, Cobol is still more popular than Java and Javascript.
http://www.computerworld.com/article/2502430/data-center/cob...
I've got coworkers who's first job in the programming business was Cobol maintenance. Old programmers from the 1960s are retiring in their 80s, and they need maintainers to write new code for various ancient mainframe systems.
Do remember, IBM is still selling System/360 systems today. http://en.wikipedia.org/wiki/IBM_System_z
These systems are running Cobol code from the 1960s, and they need to be updated to include XML, REST and all the other latest developments of development.
http://www.itworldcanada.com/article/the-future-of-cobol-why...
Similarly, C++ programmers are probably going to have an even _larger_ legacy than Cobol.
Cobol is very much alive today.
http://www.veryant.com/
In fact by some measurements, Cobol is still more popular than Java and Javascript.
http://www.computerworld.com/article/2502430/data-center/cob...
Weird, IBM has been trying to replace bare COBOL with higher level languages for years. And between old crufty COBOL code, compiled COBOL as target code .. I really wonder who enjoys it nowadays.
Personally, if I had to deal with COBOL, not only would I treat COBOL as a "target language", I'd write a decompiler to turn COBOL back into a sensible language and run it over the entire codebase.
The problem with COBOL is it's more a DSL[1] than a general purpose one, except it's been abused and extended to do so.
[1] from my short readings, it's suited for simple table processing.
[1] from my short readings, it's suited for simple table processing.
Weird, IBM has been trying to replace bare COBOL with higher level languages for years.
I was trying to be conservative.
[deleted]
[deleted]
What is your point? And what is "C++ triad"?
> What is your point?
The point is that many of us, go to C++ because it is one of the few languages with ahead-of-time compilers available across multiple systems.
Now that we are experiencing a going native wave in terms of compiler toolchains for other programming languages, that reason is slowly becoming less relevant.
So many might choose another language, instead of waiting a few more years for the support to appear.
> And what is "C++ triad"?
gcc, clang and Visual C++.
HN posts about C and C++ tend to forget there are many more C and C++ compilers out there.
The point is that many of us, go to C++ because it is one of the few languages with ahead-of-time compilers available across multiple systems.
Now that we are experiencing a going native wave in terms of compiler toolchains for other programming languages, that reason is slowly becoming less relevant.
So many might choose another language, instead of waiting a few more years for the support to appear.
> And what is "C++ triad"?
gcc, clang and Visual C++.
HN posts about C and C++ tend to forget there are many more C and C++ compilers out there.
Possibly g++, clang and MSVC? Or Linux, Windows and OS X?
Both.
There are lots of compilers and OS out there besides those.
Which are what get discussed here most of the time.
There are lots of compilers and OS out there besides those.
Which are what get discussed here most of the time.
There's what, Portland, Wind River, ????? I heard intel's wrapped around clang these days.
Here are some:
http://en.wikipedia.org/wiki/Oracle_Solaris_Studio
http://en.wikipedia.org/wiki/HP_aC%2B%2B
http://en.wikipedia.org/wiki/IBM_XL_C%2B%2B
Notably, each of those can target an OS and hardware architecture made by the same vendor (Sun Solaris/SPARC, HP UX/PA-RISC, IBM AIX/POWER).
http://en.wikipedia.org/wiki/Oracle_Solaris_Studio
http://en.wikipedia.org/wiki/HP_aC%2B%2B
http://en.wikipedia.org/wiki/IBM_XL_C%2B%2B
Notably, each of those can target an OS and hardware architecture made by the same vendor (Sun Solaris/SPARC, HP UX/PA-RISC, IBM AIX/POWER).
Portland, Wind River, Intel ICC, Borland C++ Builder, Aix XL C++, HP-UX HP aC++, OS 400 XL C++, ARM DS C++, TI C++, SN Systems C++, ...
Not even half of them are already full C++11 compliant.
Not even half of them are already full C++11 compliant.
The specification looks familiar to python and go.. how it will handle backward compatibility ?
http://clang.llvm.org/docs/Modules.html