Cling: An Interactive C++ interpreter(root.cern.ch)
root.cern.ch
Cling: An Interactive C++ interpreter
https://root.cern.ch/cling
16 comments
Can somebody please shine some light on why such an endeavor was undertaken by CERN? What was so important about JIT C++ that they had to write such a sophisticated piece of software?
If you look at the root of the domain (https://root.cern.ch/) it seems to be part of work on a scientific computing framework in C++. And having a nice REPL is important for this kind of thing, and if it is in C++ you can look deeper into it? (instead of using one of the bindings to Python/... for the REPL, where you can't inspect what is going on inside the C++ code)
Interactive programming is very developer friendly workflow, something that was already possible at Xerox PARC in their Smalltalk, Interlisp-D and Mesa/CEDAR environments.
Actually there was an early attempt to provide C++ repls back in the early 90's with Lucid Energize C++, after they pivoted from Lisp Machines and applied their knowledge to C++.
https://www.youtube.com/watch?v=pQQTScuApWk
Also the majority of CERN code is written in a mix of Fortran and C++, and not all teams like to use Python. So they rather use something that provides interactivity to research their algorithms and speed at the same time.
Actually there was an early attempt to provide C++ repls back in the early 90's with Lucid Energize C++, after they pivoted from Lisp Machines and applied their knowledge to C++.
https://www.youtube.com/watch?v=pQQTScuApWk
Also the majority of CERN code is written in a mix of Fortran and C++, and not all teams like to use Python. So they rather use something that provides interactivity to research their algorithms and speed at the same time.
Wow, that's interesting. Interactive python a la jupyter notebook revolutionized my workflow; I guess it makes sense to allow the same for a codebase written largely in c++
Python notebooks rediscovered the past and it is quite good that they did so.
Now imagine if that was the standard way everyone worked by now.
Now imagine if that was the standard way everyone worked by now.
JIT compilation opens up a lot of new features and workflows in a language, from interactive development at a REPL to in some cases very powerful abilities to generate, compile and run code all at runtime (i.e. "eval" in lisps).
This is called systems research, a lost art in the US, but still very important if you want to write complex systems as as they do in CERN. By the way, that's why C++ was designed in the first place.
It is a replacement for an older C++ interpreter, Cint. Which, in turn, is a replacement for an even older Fortran interpreter (see CERNLIB).
What a lot of people aren't mentioning is the legacy factor.
We had a "C++ interpreter" called CINT for a long time, which the ROOT data analysis framework was built around. It was, to put it bluntly, a mess: it had inconsistent scoping and syntax with C++, gave worthless debugging messages, and couldn't handle a lot of standard C++ code.
It was also the way that a lot of physicists at CERN ran their analysis code. We needed an interactive language to make graphs and histograms and to fit our data, and when CINT was conceived the only other solutions were proprietary packages like MATLAB. So CERN went with a homegrown solution and wrote a C++ interpreter many years ago.
Flash forward 15 years. "Big data" is all the rage. Python, R, and the huge list of packages built around around them would easily solve the problems CINT was designed for. But the codebase for the larger experiments at CERN is maintained by physics graduate students, a good fraction of whom had no programming experience before grad school. We don't have the time or the resources to rewrite our existing code in scipy, and the older generation doesn't have the experience to supervise such a transition.
Given that so many physicists are still using CINT, the ROOT developers had to make a choice: they could continue to maintain CINT, which would have been nearly impossible with their resources and the evolution of C++, or they could rewrite something that did almost the same thing using more modern tools. They opted for the later.
We had a "C++ interpreter" called CINT for a long time, which the ROOT data analysis framework was built around. It was, to put it bluntly, a mess: it had inconsistent scoping and syntax with C++, gave worthless debugging messages, and couldn't handle a lot of standard C++ code.
It was also the way that a lot of physicists at CERN ran their analysis code. We needed an interactive language to make graphs and histograms and to fit our data, and when CINT was conceived the only other solutions were proprietary packages like MATLAB. So CERN went with a homegrown solution and wrote a C++ interpreter many years ago.
Flash forward 15 years. "Big data" is all the rage. Python, R, and the huge list of packages built around around them would easily solve the problems CINT was designed for. But the codebase for the larger experiments at CERN is maintained by physics graduate students, a good fraction of whom had no programming experience before grad school. We don't have the time or the resources to rewrite our existing code in scipy, and the older generation doesn't have the experience to supervise such a transition.
Given that so many physicists are still using CINT, the ROOT developers had to make a choice: they could continue to maintain CINT, which would have been nearly impossible with their resources and the evolution of C++, or they could rewrite something that did almost the same thing using more modern tools. They opted for the later.
actually, there was another choice besides llvm/clng: python.
cint (M. G.) was in maintenance-mode for a long time by then.
ph-sft group was more for python (P. M.), and fnal wanted llvm/c++ (P.C.).
fnal then put down the money for an IC contract (A.N.) at ph-sft for the llvm/c++ based implementation and joined the iso body.
cling is almost orthogonal to the whole ROOT mess (graphics, histogramming, partially the analysis etc.)
cint (M. G.) was in maintenance-mode for a long time by then.
ph-sft group was more for python (P. M.), and fnal wanted llvm/c++ (P.C.).
fnal then put down the money for an IC contract (A.N.) at ph-sft for the llvm/c++ based implementation and joined the iso body.
cling is almost orthogonal to the whole ROOT mess (graphics, histogramming, partially the analysis etc.)
I'm confused by all the acronyms, and I know CERN pretty well… what is M.G.? What is P.M.?…
That's interesting. Thank you for the history lesson and for the glimpse into the culture at CERN.
[deleted]
I want to see something like this available from gdb. Following code that uses shared_ptr in gdb is still awful. It would be great to be able to explore the stack frame properly, looking at STL containers without hassle and calling functions and everything.
Modern gdb does have Python plug-ins, and e.g. I'll get "std::vector of length 3, capacity 4 = {10, 20, 30}" when printing a vector, and "std::shared_ptr (count 1, weak 0) 0x8059e10" when printing a shared ptr. But oh yes, it could be so much better. For instance, often you cannot call methods on template stuff (that is, everything) because 'it may be inlined'
"May" ought to be translated as "must". Templated C++ is incredibly sluggish without inlining. People are comfortable debugging C programs with more advanced optimizations turned off because performance will still be decent. In C++, the optimizations are the only reason it ever performs reasonably.
Not really. Calling a C++ template-generated function is no different than calling a C function. If you have a lot of tiny template-generated functions then sure, it would be slow. But no slower than a C program with a bunch of tiny functions.
Maybe they are thinking of specializing for static dispatch?
Tried to build it and git is asking for username/password
git clone https://root.cern.ch/git/root.git src
Username for 'https://root.cern.ch':
Password for 'https://root.cern.ch':
fatal: Authentication failed for 'https://root.cern.ch/git/root.git/'There's a GitHub mirror of it here: https://github.com/root-mirror/root
Windows is unfortunately not yet supported and so far it doesn't look like they are even thinking about it. A few people have managed to build on Windows, but with considerable difficulty.
As an aside, I wonder if browser vendors could agree to just use something like the LLVM intermediate representation instead of javascript as a common denominator.
Everyone agrees that LLVM is awesome and everyone compiles
their personal favorite language to whatever browsers eat anyway.
WebAssembly is essentially that.
https://en.wikipedia.org/wiki/WebAssembly
https://en.wikipedia.org/wiki/WebAssembly
I suspect that sandboxing LLVM IR would not be easy. Mind you, now that browser JS runtimes grant access to GPU shaders and the ability to accept connections through webRTC, keeping JS in the sandbox isn't going to be so easy anymore.
Like WebAssembly without sandboxing or portability to different hardware platforms?
Google's Native Client was exactly this, but it never caught on. PNaCl is a proper subset of LLVM bytecode.
Cool, but I'm having trouble imagining how I'd use it in my workflow.. Is there a demo or something?
I've only used interpreters in MATLAB/R where you have to write one-liners to massage data into the format you need - which seems pretty orthogonal to the way you work in C++
I've only used interpreters in MATLAB/R where you have to write one-liners to massage data into the format you need - which seems pretty orthogonal to the way you work in C++
I use the python shell in two ways: to try/explore things in isolation, without having to muck with the larger thing I'm working on; and similarly, just learning things in general without a focus on a particular project.
And now with the ipython notebook (jupyter) I use the shell in a third way, to repeatably explore and demonstrate larger, less isolated things that are still sort of one-offs.
That said, I think the fact that there is a significant handful of C++ compiler/implementations in use, and some latitude in behavior for each, means that a C++ user is very much a user of the compiler as much as the language. Play but verify, I guess.
And now with the ipython notebook (jupyter) I use the shell in a third way, to repeatably explore and demonstrate larger, less isolated things that are still sort of one-offs.
That said, I think the fact that there is a significant handful of C++ compiler/implementations in use, and some latitude in behavior for each, means that a C++ user is very much a user of the compiler as much as the language. Play but verify, I guess.
See how it is used in ROOT.
"Cling is capable to parse everything that Clang can do. In addition, Cling can parse some interpreter-specific C++ extensions."
Punchline!
Punchline!
Kling och Klang!
That must have been pretty hard to build.
[deleted]
It was a replacement for the old C++ interpreter, CINT, which was already a central part of a lot of CERN code. Building Cling was probably easier than maintaining CINT.
Still, since C++ is a statically compiled language by design, doesn't that make it difficult to make it run as an interpreted language ?
Sure, and no one is saying it was a good idea in the first place. CINT was notoriously bad as a C++ interpreter, in fact it was marketed as being able to parse 90% of valid C++ code. Imagine someone selling a compiler that way?
But CERN developed a huge amount of CINT code anyway, and all of CERNs legacy sort-of C++ code that ran on CINT became a maintenance nightmare. Cling is a huge upgrade in the sense that it supports all this legacy code with a _lot_ less overhead than before.
But CERN developed a huge amount of CINT code anyway, and all of CERNs legacy sort-of C++ code that ran on CINT became a maintenance nightmare. Cling is a huge upgrade in the sense that it supports all this legacy code with a _lot_ less overhead than before.
Very interesting project. I am not that into clang/llvm, neither I don't have it on my machine nor know anything about clang/llvm. But a c++ interpreter is something I always wanted to use, how do I make it work for g++ 4.8.4 ?
Apart from all the awesomeness, it'll be so great to troll people who take language holywars seriously with this.
You mean those who claim C or C++ are always compiled and not interpreted? This has been around for a long time:
https://root.cern.ch/cint
https://root.cern.ch/cint
There was also the Sabre C interpreter, early releases of X11 and X10R4 came with the scripts to run it, it didn't do C++ though.
truth be told, Masaharu Goto wrote the CINT C++ interpreter at HP Japan and cern/root just embraced it and rolled it into its own floss distribution (F.R. was also an HP-CERN liaison for a very long time).
http://www.hanno.jp/gotom/Cint.html
(Saber-C, the Saber company has become Centerline, and the software itself has been renamed to CodeCenter. saber, xsaber, and sabertool have become codecenter, xcodecenter, and codetool. DEC offered Saber-C on ULTRIX. It never appeared to do very well in the marketplace, which is some kind of tragedy. It was, and is, a terrific piece of software. The fact that a product as useful as Saber-C could fail to do well in the market is an indicator of the impoverishment of software "engineering" practices industry-wide. Anyway, it ended up being ObjectCenter from Integrated Computer Solutions doing C++ and having an IDE with class browsing, with xobjectcenter)
http://www.hanno.jp/gotom/Cint.html
(Saber-C, the Saber company has become Centerline, and the software itself has been renamed to CodeCenter. saber, xsaber, and sabertool have become codecenter, xcodecenter, and codetool. DEC offered Saber-C on ULTRIX. It never appeared to do very well in the marketplace, which is some kind of tragedy. It was, and is, a terrific piece of software. The fact that a product as useful as Saber-C could fail to do well in the market is an indicator of the impoverishment of software "engineering" practices industry-wide. Anyway, it ended up being ObjectCenter from Integrated Computer Solutions doing C++ and having an IDE with class browsing, with xobjectcenter)
@Create you are confusing. Did Saber-C end up being codecenter or xobjectcenter? Is the company Centerline or Integrated Computer Solutions. Who is F.R.?
Does this work with windows? I'm still not clear where Clang stands.
I guess this will finally allow to build full-fledged IDE for C++
It allows for the same "IDE" that works great for me with Powershell - a console window on one side, and N++ on the other side of the screen. N++ Autosave plugin is invaluably rapid here, letting you write some code, then just click/tab to the console and test the module.
Eclipse CDT has gotten pretty good in the last few years. Its code scanner is practically a full C++ front-end, and if I pay attention to the red ink I practically never break my builds. That being said, _instant_ C++ is a fantastic thing
Last time I tried Eclipse (2-3 months ago) it wasn't usable on high DPI screens. Have they fixed tat?
Like Lucid Energize C++ in 1993 ?
https://www.youtube.com/watch?v=pQQTScuApWk
Or C++ Builder or C Set++ for OS/2 (might have the name wrong).
If it wasn't for tool vendors having switched their focus from C++ to Java and .NET, until LLVM and clang got created showing everyone what was wrong with their approaches, C++ IDEs might have been much more advanced already, in spite of technical issues with language.
https://www.youtube.com/watch?v=pQQTScuApWk
Or C++ Builder or C Set++ for OS/2 (might have the name wrong).
If it wasn't for tool vendors having switched their focus from C++ to Java and .NET, until LLVM and clang got created showing everyone what was wrong with their approaches, C++ IDEs might have been much more advanced already, in spite of technical issues with language.
Wasn't some of that available in LLDB for a year or two?
Any homebrew formula?
for download : https://root.cern.ch/download/cling/