A new decade, a new tool(vector-of-bool.github.io)
vector-of-bool.github.io
A new decade, a new tool
https://vector-of-bool.github.io/2020/01/06/new-decade.html
51 comments
What's your opinion on "libman", also mentioned in the author's blog post?
https://github.com/vector-of-bool/libman
The author defines it this way:
> libman is a new level of indirection between package management and build systems. If a package manager can emit libman files, then any build system that understands them can work with that package manager. If a build system can spit out a libman-export, then a package manager that understands the format can collect the build output and distribute it.
Do you see some values in this? Would that be something helpful for your project? As far as I understand, "dds" is just one emitter and consumer of a libman manifest, but the author seems to that it could also be used in combination with other tools (such as cmake as the build tool, conan as the package manager).
https://github.com/vector-of-bool/libman
The author defines it this way:
> libman is a new level of indirection between package management and build systems. If a package manager can emit libman files, then any build system that understands them can work with that package manager. If a build system can spit out a libman-export, then a package manager that understands the format can collect the build output and distribute it.
Do you see some values in this? Would that be something helpful for your project? As far as I understand, "dds" is just one emitter and consumer of a libman manifest, but the author seems to that it could also be used in combination with other tools (such as cmake as the build tool, conan as the package manager).
Let's make the comparison with pkg-config.
If a library (either as source, or as a compiled package) provides a (correct) pkg-config file, then using that library from within a project is pretty easy, assuming the build system knows how to use pkg-config files (we use waf, and it does).
But if the library doesn't provide a pkg-config file, or provides a broken one (yep, that happens), then we either have to create one or use a different method to tell the build system where to find and how to use the library.
We don't really want a package manager per se, because we're cross-platform, and package managers for compiled libraries are almost certainly going to be platform specific.
So for our release builds, and for some of our developers, we need to start from the source code, build the library using flags we provide (because we know what we want and what we're doing), install the library (somewhere) and only then set about using it from within our project. That process has to work more or less identically for our linux, macOS and mingw/windows builds (we x-compile for windows).
Consequently, even if there's a pkg-config file and even if it's contents are correct, none of that addresses (1) the version of the library (2) the compile time flags for the library (3) actually downloading the library (4) actually compiling and linking it. It does help with "installation" because the pkg-config file goes to a known location even if the installation is somewhere non-standard.
So in summary I would say that even if libman became wildly successful, and was used by the dozens of C libraries that we use in addition to the C++ ones, it would really only function as a replacement for pkg-config and would not address the harder, bigger problems we have with 3rd party library dependencies.
If a library (either as source, or as a compiled package) provides a (correct) pkg-config file, then using that library from within a project is pretty easy, assuming the build system knows how to use pkg-config files (we use waf, and it does).
But if the library doesn't provide a pkg-config file, or provides a broken one (yep, that happens), then we either have to create one or use a different method to tell the build system where to find and how to use the library.
We don't really want a package manager per se, because we're cross-platform, and package managers for compiled libraries are almost certainly going to be platform specific.
So for our release builds, and for some of our developers, we need to start from the source code, build the library using flags we provide (because we know what we want and what we're doing), install the library (somewhere) and only then set about using it from within our project. That process has to work more or less identically for our linux, macOS and mingw/windows builds (we x-compile for windows).
Consequently, even if there's a pkg-config file and even if it's contents are correct, none of that addresses (1) the version of the library (2) the compile time flags for the library (3) actually downloading the library (4) actually compiling and linking it. It does help with "installation" because the pkg-config file goes to a known location even if the installation is somewhere non-standard.
So in summary I would say that even if libman became wildly successful, and was used by the dozens of C libraries that we use in addition to the C++ ones, it would really only function as a replacement for pkg-config and would not address the harder, bigger problems we have with 3rd party library dependencies.
one thing I love about npm, it defaults to all tools and dependencies being versioned and local. No list of tools and system libs I need to install.
people complain about all the deps but deps in C/C++ and even to some degree python are hidden since they are system deps. No one seems to blink an eye when they apt-get something and it requires 60 deps and of course the second apt-get only shows the missing deps so there is this false idea that the dependencies are fewer.
for the most part npm just works. As someone with 15-35yrs C/C++ experience and a bunch of Perl and Python it's refreshing not to have to trash my system with global deps or learn non default modes of operation, that the default is all tools and deps local.
I also get why static typing people hate JS but one thing they got right IMO is there are no namespaces. You import libraries into local variables which seems like a huge win over namespaces.
Note: I also get that c/c++ is generally harder. Different compilers, different flags, different platforms, etc. I’m not saying I have a solution. getting MS (VC++), Apple(Xcode/clang), and GNU(GCC) on the same page might be a start.
people complain about all the deps but deps in C/C++ and even to some degree python are hidden since they are system deps. No one seems to blink an eye when they apt-get something and it requires 60 deps and of course the second apt-get only shows the missing deps so there is this false idea that the dependencies are fewer.
for the most part npm just works. As someone with 15-35yrs C/C++ experience and a bunch of Perl and Python it's refreshing not to have to trash my system with global deps or learn non default modes of operation, that the default is all tools and deps local.
I also get why static typing people hate JS but one thing they got right IMO is there are no namespaces. You import libraries into local variables which seems like a huge win over namespaces.
Note: I also get that c/c++ is generally harder. Different compilers, different flags, different platforms, etc. I’m not saying I have a solution. getting MS (VC++), Apple(Xcode/clang), and GNU(GCC) on the same page might be a start.
FWIW, JS does have namespaces: directories.
There's no big semantical difference between
In short, the node_modules package resolution algorithm is how JS does namespaces. It's the same thing, just with different syntax (and, IMO, a bit more weirdness). The fundamental thing isn't "no namespaces" but "namespaces are package-relative". I wonder whether they got that right through sheer genius, or by accident.
(I recognize that I'm being super pedantic here btw, hope that's OK)
Note that I agree with the rest of your comment, NPM is splendid and it's impressive how much they got right at the first go, especially given how NodeJS was on a hype train at the time (many things on hype trains only get tooling like this right after the first so many tries).
There's no big semantical difference between
var Hello = require("someorg/hey");
using Hello = SomeOrg.Hey;
In short, the node_modules package resolution algorithm is how JS does namespaces. It's the same thing, just with different syntax (and, IMO, a bit more weirdness). The fundamental thing isn't "no namespaces" but "namespaces are package-relative". I wonder whether they got that right through sheer genius, or by accident.
(I recognize that I'm being super pedantic here btw, hope that's OK)
Note that I agree with the rest of your comment, NPM is splendid and it's impressive how much they got right at the first go, especially given how NodeJS was on a hype train at the time (many things on hype trains only get tooling like this right after the first so many tries).
> ..The node_modules package resolution algorithm is how JS does namespaces.. I wonder whether they got that right through sheer genius, or by accident.
I recall hearing Ryan Dahl's regrets about design decisions in Node.js, a few of which had to do with the module/package resolution algorithm.
10 Things I Regret about Node.js - https://www.youtube.com/watch?v=M3BM9TB-8yA
There are some good aspects of it though. I like how modules/packages are just imported objects (the path resolution is a bit weird and seemingly inefficient, i.e., checking every parent folder's node_modules folder). It goes together well, that modules are also self-contained function scopes by default (I suppose a kind of anonymous namespace), that exports a plain object with assigned properties.
---
Wow, I just learned about C++20 Modules, looks like a game-changing feature.
https://modernescpp.com/index.php/c-20-modules
I recall hearing Ryan Dahl's regrets about design decisions in Node.js, a few of which had to do with the module/package resolution algorithm.
10 Things I Regret about Node.js - https://www.youtube.com/watch?v=M3BM9TB-8yA
There are some good aspects of it though. I like how modules/packages are just imported objects (the path resolution is a bit weird and seemingly inefficient, i.e., checking every parent folder's node_modules folder). It goes together well, that modules are also self-contained function scopes by default (I suppose a kind of anonymous namespace), that exports a plain object with assigned properties.
---
Wow, I just learned about C++20 Modules, looks like a game-changing feature.
https://modernescpp.com/index.php/c-20-modules
Yeah I saw that talk too, and I was baffled. He seems totally ready to throw out the baby with the bathwater. Node's package resolution algorithm might not be perfect, but it's better than nearly every other language's (which is "single big unhierarchical dump of packages, good look sorting out the dependencies kthxbye").
that might be true in c# but it's not true in C++ iirc. the library declares its "global" namespace which dicates what global symbol will be generated for the linker. "using" creates an alias for that symbol in consumers of the library but nothing you can do short of changing the source of the library will fix the issue of 2 libraries using the same namespace. This is not true for JS
Update : Verified the same issue exists in c#.
Update : Verified the same issue exists in c#.
>one thing I love about npm, it defaults to all tools and dependencies being versioned and local. No list of tools and system libs I need to install.
That is not unique to npm. Java/Maven, Ruby/Bundler, Go/Vendor also work like this and I believe that several other languages might work like this.
Comparing npm with C/C++ build/package tools is setting a very low bar
That is not unique to npm. Java/Maven, Ruby/Bundler, Go/Vendor also work like this and I believe that several other languages might work like this.
Comparing npm with C/C++ build/package tools is setting a very low bar
True, but it's nice for JS land that NPM got it right the first time. Ruby gems sucked before bundler (and last I checked, bundler still isn't cross-platform, it's nuts), I lost count of how many Go package managers get published. For at least a decade all of Java land downloaded JARs and put them into the CLASSPATH.
It really took quite a while for programming communities to accept that that local packages are a good thing, and NPM was early to the party.
And NPM does this recursively, which I think it what greggman is primarily getting at. I don't think many other package managers do this. I mean, the fact that a package's dependencies are local to it too is fundamental. You simply can't get dependency version conflicts (except if package authors do terrible global state hacks, and fortunately this is pretty unheard of). Instead, you just pull in 5 different versions of some logging library or some http client wrapper, and you don't even notice. That's not without its own downsides, but I prefer it to the steaming pile of `override` directives in our Elixir codebase has day.
It really took quite a while for programming communities to accept that that local packages are a good thing, and NPM was early to the party.
And NPM does this recursively, which I think it what greggman is primarily getting at. I don't think many other package managers do this. I mean, the fact that a package's dependencies are local to it too is fundamental. You simply can't get dependency version conflicts (except if package authors do terrible global state hacks, and fortunately this is pretty unheard of). Instead, you just pull in 5 different versions of some logging library or some http client wrapper, and you don't even notice. That's not without its own downsides, but I prefer it to the steaming pile of `override` directives in our Elixir codebase has day.
>I don't think many other package managers do this
Unless I am missing something, "go mod vendor", and "mvn package" will work in the same manner. They will download all deps of the app itself PLUS all transitive dependencies. No system libraries needed (or touched)
Unless I am missing something, "go mod vendor", and "mvn package" will work in the same manner. They will download all deps of the app itself PLUS all transitive dependencies. No system libraries needed (or touched)
I'm no expert, but given the amount of hits "maven dependency conflicts" I get on Google, strongly doubt Maven works the same.
Last I checked, it'll indeed pull in all transitive dependencies, but if multiple dependencies need the same subdependency of a different version, you need to work out conflicts. JS simply doesn't have this problem, it's splendid.
Last I checked, it'll indeed pull in all transitive dependencies, but if multiple dependencies need the same subdependency of a different version, you need to work out conflicts. JS simply doesn't have this problem, it's splendid.
"maven dependency conflicts" is when a SINGLE project has issues with its own dependencies because of bad configuration.
Multiple unrelated projects can use the same or different version of a library without any issues
Multiple unrelated projects can use the same or different version of a library without any issues
> While drafting this post, there was a text-post to /r/cpp on Reddit entitled Writing c++ is wonderful, compiling and linking it is horrible and glib. While the original author has deleted the body of their post, I can summarize that it was more-or-less a rehashing of the title itself, with some explanatory examples. The post received 400pts, with a 97% positive vote: That’s incredible, and places it as one of the most popular posts in /r/cpp for the entire year!
Here is the archived body of the reddit post if anybody was curious: http://removeddit.com/r/cpp/comments/egrzd0/writing_c_is_won...
Here is the archived body of the reddit post if anybody was curious: http://removeddit.com/r/cpp/comments/egrzd0/writing_c_is_won...
this sounds like it could be really exciting if it takes off. the one thing that puzzled me was requiring the same compiler flags and preprocessor macros all the way down a dependency tree - I'm not too immersed in the c++ world these days so I can't say for sure it is not an onerous requirement, but it seems like it could become one if you depend on a bunch of external libraries.
He is building a couple of things here. One is a way to package a C++ library and encode the dependencies. It is a subset of other similar tools and can be used as a bridge.
So rather than have CMake load special code to talk to Conan, you can have Conan export a generic description and then CMake reads that description. This approach will let libraries work with multiple different build and/or packaging tools.
The other is a 'drop dead simple' build tool. His build tool has these compile flag restrictions.
But you could still build with CMake and add all kinds of extra requirements. However, for a library that you want to share, making it simple enough to work with the simple build tool is probably a good idea.
The other is a 'drop dead simple' build tool. His build tool has these compile flag restrictions.
But you could still build with CMake and add all kinds of extra requirements. However, for a library that you want to share, making it simple enough to work with the simple build tool is probably a good idea.
It's mostly an anti-madness requirement; there are lots of flags that are safe to vary by compilation unit, but there are some that aren't, so rather than attempt to enumerate them (which may vary by compiler), just enforce uniformity.
IMO
you can have general flags (-O3/-g3) for all compilation units(CU), but there should be the capacity to add any per CU
you can have general flags (-O3/-g3) for all compilation units(CU), but there should be the capacity to add any per CU
It’s difficult in practice to distinguish between general and lib-specific flags, becuase one developer thinks is general does not always be considered the same by another, and you either need to magically read developers’ minds, or make a decision somewhere and leave a portion of your users grumpy. This is especially difficult for an ecosystem as established as C++ since there are too many interest groups all want different things.
And more good luck if you don’t control the compiler—whatever you decide may end up wrong tomorrow because the compiler developers change their minds, and of course you take all the blame because everything works except when the user uses your tool.
And more good luck if you don’t control the compiler—whatever you decide may end up wrong tomorrow because the compiler developers change their minds, and of course you take all the blame because everything works except when the user uses your tool.
Rust code just builds. a typical library will pull in hundreds of dependencies... and they will all just build (eventually). There are downsides to the approach but they are papercuts. Did I mention that Rust code builds? As in, you tell cargo to build something... And then it builds it? Type cargo build and enjoy the blinkenlights.
Does cargo build C++ packages? That's the specific problem solved by this project.
It generally manages this pretty well. You usually need a Perl interpreter installed (wtf) and then everything works.
It can, but via custom scripts.
Modern C++ is enjoyable and fast. Building and dependency management is fragmented and broken. I hope this takes off!
What are some good resources for a quick intro to modern C++? I'm somewhat interested in Qt development and C++ seems inevitable in that environment.
This looks like it addresses the exact problem I have had with getting started with c++. I only wish it had more of a ‘landing page’ layout; ie. a brief summary, a working example I can try right away, and THEN a long explanation I can peruse at my leisure.
Check out DDDS' documentation, that's what you want: https://vector-of-bool.github.io/docs/dds/index.html
It took me a few minutes following the tutorial to get my project to build on Windows, macOS, and Linux.
It took me a few minutes following the tutorial to get my project to build on Windows, macOS, and Linux.
Thank you I’ll give it a try
This is why I pat myself on the back for deciding 15 years ago to stop breaking my brain against cplusplus and learn Python.
Do I miss the challenges and rewards of working at the lowest level of the stack? A little bit. But it's more than made up for by the fact that I can produce code that's workable, readable, and extensible, and can mature and grow more powerful in months rather than years.
Of course, I now stand on the shoulders of the people who build tools like numpy and tensorflow. I have great respect for those who continue to wrestle with these demons, but I just had to tap out.
Do I miss the challenges and rewards of working at the lowest level of the stack? A little bit. But it's more than made up for by the fact that I can produce code that's workable, readable, and extensible, and can mature and grow more powerful in months rather than years.
Of course, I now stand on the shoulders of the people who build tools like numpy and tensorflow. I have great respect for those who continue to wrestle with these demons, but I just had to tap out.
Soon: there are 15 competing C++ build systems.
Anyway, I suspect that what happens is, as worse as the C++ compiling/linking situation is, it's not bad enough to warrant learning a new build system.
Anyway, I suspect that what happens is, as worse as the C++ compiling/linking situation is, it's not bad enough to warrant learning a new build system.
I wish more ppl would have experienced include files, so that they would see the advantage of lexical scoped namespace module imports. With no shared global scope. So that you can see where all variables come from at a glance. With the ability to import any library anywhere in your code with just "var foo = import lib" and it will just work. (See Dlang and nodejs commonjs modules)
Vector of bools contribution of CMake tools for VS Code is a game changer and I'm certain that any other tooling he would be working on is thought out and beneficial to the C++ community.
After reading this rather lengthy blog post, I am excited to try this out.
Currently, dependency management for C++ projects is a PITA.
After reading this rather lengthy blog post, I am excited to try this out.
Currently, dependency management for C++ projects is a PITA.
TLDR, as I see people complaining about the length of the author's blog post:
- the author mentions his work on "libman", a common format to describe C and C++ libraries. The idea is to have a manifest format that can be emitted by a package manager, then consumed by any build tools that supports it. The main example is "Conan (a C++ package manager) generates the libman file, CMake uses it for its build process". See the project at: https://github.com/vector-of-bool/libman
- then the author presents its own build and dependency management tool, named "Drop Dead Simple build and library management tool" (aka dds). It's one tool that can both consume and generate libman manifests (at least, that's what I understood), and should be simple to use in the majority of cases. It's still very alpha, you can find tutorials and documentation at: https://vector-of-bool.github.io/docs/dds/index.html
I spent the weekend playing a bit with both projects, it's really refreshing IMHO compared to CMake & co., and quite fun. It took me just a few minutes following the tutorials to get some side project build on macOS, Windows, and Linux.
Keep in mind that all of this is still very early days, and experimental.
- the author mentions his work on "libman", a common format to describe C and C++ libraries. The idea is to have a manifest format that can be emitted by a package manager, then consumed by any build tools that supports it. The main example is "Conan (a C++ package manager) generates the libman file, CMake uses it for its build process". See the project at: https://github.com/vector-of-bool/libman
- then the author presents its own build and dependency management tool, named "Drop Dead Simple build and library management tool" (aka dds). It's one tool that can both consume and generate libman manifests (at least, that's what I understood), and should be simple to use in the majority of cases. It's still very alpha, you can find tutorials and documentation at: https://vector-of-bool.github.io/docs/dds/index.html
I spent the weekend playing a bit with both projects, it's really refreshing IMHO compared to CMake & co., and quite fun. It took me just a few minutes following the tutorials to get some side project build on macOS, Windows, and Linux.
Keep in mind that all of this is still very early days, and experimental.
C++ ecosystem is beyond recovery. It's just too vast, archaic and osified. It's much more practical at this point to learn Rust and use it for new projects when possible or keep doing whatever you did before to get you C++ project built.
I tried using “TLDR this,” (trending on Show HN today) but it only summarized the first preamble. Darn.
I guess C++ is package / build hell? I’m not a C++ developer so I’m not too familiar with the dependency story there. Anyone have a short take on what’s the problem and whether DDS is a promising answer?
I guess C++ is package / build hell? I’m not a C++ developer so I’m not too familiar with the dependency story there. Anyone have a short take on what’s the problem and whether DDS is a promising answer?
Maybe it's something about being a C++ Dev that lends your output to extra verbosity, but I feel like this reads like a recipe on a blogspam site and I had to scroll down really far to get to the point.
And the point is, he's making npm/maven for C++. Did C++ not already have a fully featured package manager until now? Why not?
And the point is, he's making npm/maven for C++. Did C++ not already have a fully featured package manager until now? Why not?
> Did C++ not already have a fully featured package manager until now?
There are several competing projects, but no clear winner yet. And a lot of people a companies prefer to build their stuff themselves for additional control.
> Why not?
It's complicated. In a few words, it's practically impossible to build a general tool that could suit everybody's needs.
There are several competing projects, but no clear winner yet. And a lot of people a companies prefer to build their stuff themselves for additional control.
> Why not?
It's complicated. In a few words, it's practically impossible to build a general tool that could suit everybody's needs.
> it's practically impossible to build a general tool that could suit everybody's needs.
Indeed and that's not what npm or maven aims for either, no tool fits everybody's need, not even npm! Probably there is a deeper issue than "it's hard", or if it's "just hard", why is so much harder for C++ than for the JavaScript folks?
Indeed and that's not what npm or maven aims for either, no tool fits everybody's need, not even npm! Probably there is a deeper issue than "it's hard", or if it's "just hard", why is so much harder for C++ than for the JavaScript folks?
There is no compilation step in the js world, or any other interpreted language for that matter. Even in case of JIT, it happens completely transparently and has no effect on the package you download from npm. Nobody distributes JIT'ed code.
C++, on the other hand, doesn't work like that. Source code needs to be compiled to be usable and the exact way it is supposed to be compiled varies enormously between different projects. Not to mention that there are usually lots of compiler options and project-specific macro definitions that you might want to change to get the exact binary you need. Compilation matters, it matters a lot, and it cannot be easily abstracted away without sacrifising performance and flexibility.
C++, on the other hand, doesn't work like that. Source code needs to be compiled to be usable and the exact way it is supposed to be compiled varies enormously between different projects. Not to mention that there are usually lots of compiler options and project-specific macro definitions that you might want to change to get the exact binary you need. Compilation matters, it matters a lot, and it cannot be easily abstracted away without sacrifising performance and flexibility.
Plenty of npm/pypi modules require building platform-specific native code (see node-gyp, wheel), and a lot of JS packages require transpilation. I don't find this argument particularly compelling.
> why is so much harder for C++ than for the JavaScript folks?
There are C++ package managers that exist and work - they tend to be your OS' package manager, because of the problems they will have to work around.
Because C++ has a lot more flexibility in core components than JavaScript does, which aren't easy to resolve at the package manager level.
Things projects may _require_ to build correctly:
* Multiple compilers that don't always have compatible extensions that sometimes share the same flags, and can falsely report to be each other when queried.
* Multiple simultaneous differing versions of clib to link against. They may requiring differing specific versions of specific implementations.
* Specific kernel libraries to link against. You can't easily build and link against a specific OS kernel in isolation.
* Projects may expect specific paths, and ignore or break attempts to change these. Not being able to compile a project that has a history of twenty years isn't an option.
* Arbitrary execution with intentional side-effects. It's bad, but there is a very long history of projects that you're dealing with here.
* CPATH and CXXPATH are globals. Relative path linking is possible, but difficult and a pile of hacks. You can force a node_modules style to exist, and some projects do, but most are a mix of relative and global, and the global can override your local and break things.
There are C++ package managers that exist and work - they tend to be your OS' package manager, because of the problems they will have to work around.
Because C++ has a lot more flexibility in core components than JavaScript does, which aren't easy to resolve at the package manager level.
Things projects may _require_ to build correctly:
* Multiple compilers that don't always have compatible extensions that sometimes share the same flags, and can falsely report to be each other when queried.
* Multiple simultaneous differing versions of clib to link against. They may requiring differing specific versions of specific implementations.
* Specific kernel libraries to link against. You can't easily build and link against a specific OS kernel in isolation.
* Projects may expect specific paths, and ignore or break attempts to change these. Not being able to compile a project that has a history of twenty years isn't an option.
* Arbitrary execution with intentional side-effects. It's bad, but there is a very long history of projects that you're dealing with here.
* CPATH and CXXPATH are globals. Relative path linking is possible, but difficult and a pile of hacks. You can force a node_modules style to exist, and some projects do, but most are a mix of relative and global, and the global can override your local and break things.
> Did C++ not already have a fully featured package manager until now? Why not?
No, it doesn't. I'd say it was due to age and fragmentation; most of the more recent languages have, in effect, a single "vendor" who can coordinate something like that. Whereas C++ has historically one big proprietary and one open vendor who fought each other (Microsoft vs. GNU), plus a long tail of proprietary implementations (e.g. Intel); more recently there is clang.
The language itself doesn't have packages.
In practice, the role taken by a package manager in the Free world was occupied by autoconf, which does a good job of building across a huge variety of systems but is a pain to use. There was also much more of a tradition of "this is what you're getting, deal with it": rather than a program specifying precise version requirements, autoconf probes the capabilities and then adapts accordingly.
Build system in the Free world is make, with occasional jam if you're a big boost user. Microsoft of course do something completely different with MSBuild. Visual studio does have an integrated package manager, nuget, but it's orientated at C# use.
No, it doesn't. I'd say it was due to age and fragmentation; most of the more recent languages have, in effect, a single "vendor" who can coordinate something like that. Whereas C++ has historically one big proprietary and one open vendor who fought each other (Microsoft vs. GNU), plus a long tail of proprietary implementations (e.g. Intel); more recently there is clang.
The language itself doesn't have packages.
In practice, the role taken by a package manager in the Free world was occupied by autoconf, which does a good job of building across a huge variety of systems but is a pain to use. There was also much more of a tradition of "this is what you're getting, deal with it": rather than a program specifying precise version requirements, autoconf probes the capabilities and then adapts accordingly.
Build system in the Free world is make, with occasional jam if you're a big boost user. Microsoft of course do something completely different with MSBuild. Visual studio does have an integrated package manager, nuget, but it's orientated at C# use.
> No, it doesn't. I'd say it was due to age and fragmentation; most of the more recent languages have, in effect, a single "vendor" who can coordinate something like that.
This is a widely held meme, but I think its flat wrong. The reality is that most C++ project's have a very wide spectrum of types of dependencies and build steps. The history of competing vendors certainly adds to the non-homogeneity, but it isn't the only portion. Even if all of our C and C++ dependencies were built and installed by a common package manager, we would still have dependencies on a variety of packages in Fortran, Python, Bash, Tcl, Verilator, NGSPICE, Octave, and so on. Running the compiler and linker represents only a handful of the many steps in the process.
C++'s niche lies at an intersection of performance and complexity. If the project wasn't demanding enough for C++, folks wouldn't pick it in the first place.
This is a widely held meme, but I think its flat wrong. The reality is that most C++ project's have a very wide spectrum of types of dependencies and build steps. The history of competing vendors certainly adds to the non-homogeneity, but it isn't the only portion. Even if all of our C and C++ dependencies were built and installed by a common package manager, we would still have dependencies on a variety of packages in Fortran, Python, Bash, Tcl, Verilator, NGSPICE, Octave, and so on. Running the compiler and linker represents only a handful of the many steps in the process.
C++'s niche lies at an intersection of performance and complexity. If the project wasn't demanding enough for C++, folks wouldn't pick it in the first place.
> but it's orientated at C# use.
It's oriented for MSBuild. The package creation tooling is aimed at .Net. It's certainly possible to create high quality C++ Nuget packages (that are also huge because you have to provide multiple binaries).
It's oriented for MSBuild. The package creation tooling is aimed at .Net. It's certainly possible to create high quality C++ Nuget packages (that are also huge because you have to provide multiple binaries).
For those wondering, the content seems to start about halfway down: https://vector-of-bool.github.io/2020/01/06/new-decade.html#...
The docs similarly don't seem to include a concise explanation but at least are a bit more readable: https://vector-of-bool.github.io/docs/dds/guide/packages.htm...
I don't really see it either, my typical C++ project is okay with the library management the different *make tools offer. But C++ is not my go to language these days, I'm usually rather careful on which dependencies take on in those projects and where they come from, not really sure a npm/maven dependency model is something I would want there.
The docs similarly don't seem to include a concise explanation but at least are a bit more readable: https://vector-of-bool.github.io/docs/dds/guide/packages.htm...
I don't really see it either, my typical C++ project is okay with the library management the different *make tools offer. But C++ is not my go to language these days, I'm usually rather careful on which dependencies take on in those projects and where they come from, not really sure a npm/maven dependency model is something I would want there.
>, he's making npm/maven for C++.
Well, based on what I read, he's not actually making the equivalent of npm. Instead, he's proposing a new tool ("dds") and new syntax of config files to integrate build and package management. This distinction is key because the "package manager" for javascript implies npm the the tool & package specification but also a public repository ("https://www.npmjs.com") that is funded by the Nodejs/NPM Inc entity.
Likewise, Rust's package manager points to "crates.io" repository which I believe is funded by Mozilla.
This blog article is not talking about making a C++ repository like "npmjs.com" and "crates.io". Only a syntax specification. Arguably, the destination repository (server) is more important than the command-line tool (client) because it's easier for a funded website with diskspace+bandwidth to justify the existence of a tool rather than the other way around.
> Did C++ not already have a fully featured package manager until now? Why not?
It's a seemingly simple question but it the "answer" which hasn't happened yet is complex and has multiple parts. The 2 big parts are (1) social inertia and (2) technical complexity
(1) social inertia: C++ created ~1986 has 2+ decades of usage before the idea of "let's have a process _automatically_ reach out to the internet to download dependencies" became popular. Thus, there is no well-known repository that became a Schelling Point[1] for C++ programmers to upload their C++ libraries.
Consider popular C/C++ libraries like SQLite, Zlib, OpenSSL, etc. Why don't those authors upload their libraries to a package manager repository? Because nobody made an influential website destination for it that everybody adopted. E.g. If Google Inc funded a C++ repo with disk space and bandwidth, would corporate politics prevent Microsoft and Apple from uploading their code to it? In contrast, the Javascript quickly adopted NPM that was funded by Nodejs and it was the already default baked into Nodejs downloads. Which entity in the C++ world is analogous to NPM inc to fund a repository that Google/MS/Apple/Adobe/etc would all adopt? It needs to be an entity that cuts across all those competitors. Maybe Intel? Or maybe Epic Games (they make Unreal Engine)? Or all the big competitors agree to contribute to a non-profit entity? Nobody has stepped up to the plate like NPM Inc did.
(2) technical challenges: C++ has a compile and link steps that interpreted languages like Javascript don't have. In npm, the assets are "text" (i.e. .js files like leftpad()). In C++, many library publishers (proprietary) don't offer source code (the "text") and only binary precompiled .lib files. These libraries are binary blobs that are incompatible across cpu architectures and compilers (Intel vs ARM, Microsoft MSVC vs gcc, etc). Because C++ is low-level, there's also a different set of build configs such as 32bit vs 64bit, static link vs dynamic link, and DEBUG vs RELEASE, etc. This means C++ needs to implement a much more complicated "dependency" syntax way beyond just "versions" and nobody has created that specification that everybody has agreed to use. Since Javascript doesn't have separate compile & link phases, something like "npm" is much easier to create. If Rust also has "compile & link" step, how did they successfully get people to adopt the "cargo" tool and the "crates.io" repository? Probably because Rust's ecosystem of programmers has more emphasis on shared "open source code" than the C++ programmers of the 1980s and 1990s. Also, we emphasize again that Rust's patron (Mozilla) also funded "crates.io" and Rust programmers just adopted it so the "cargo" tool has an http destination to point to download artifacts. There's no equivalent influential entity in C++ yet. In terms of package spec complexity, I haven't studied Rust's TOML configuration spec to know if it's flexible enough to handle all of the different ways that C++ practitioners build files. A lot of C++ build flexibility is put into "./configure.sh" scripts (e.g. ffmpeg, Qt, etc) so I don't know if toml spec is a superset of all that.
[1] https://en.wikipedia.org/wiki/Focal_point_(game_theory)
Well, based on what I read, he's not actually making the equivalent of npm. Instead, he's proposing a new tool ("dds") and new syntax of config files to integrate build and package management. This distinction is key because the "package manager" for javascript implies npm the the tool & package specification but also a public repository ("https://www.npmjs.com") that is funded by the Nodejs/NPM Inc entity.
Likewise, Rust's package manager points to "crates.io" repository which I believe is funded by Mozilla.
This blog article is not talking about making a C++ repository like "npmjs.com" and "crates.io". Only a syntax specification. Arguably, the destination repository (server) is more important than the command-line tool (client) because it's easier for a funded website with diskspace+bandwidth to justify the existence of a tool rather than the other way around.
> Did C++ not already have a fully featured package manager until now? Why not?
It's a seemingly simple question but it the "answer" which hasn't happened yet is complex and has multiple parts. The 2 big parts are (1) social inertia and (2) technical complexity
(1) social inertia: C++ created ~1986 has 2+ decades of usage before the idea of "let's have a process _automatically_ reach out to the internet to download dependencies" became popular. Thus, there is no well-known repository that became a Schelling Point[1] for C++ programmers to upload their C++ libraries.
Consider popular C/C++ libraries like SQLite, Zlib, OpenSSL, etc. Why don't those authors upload their libraries to a package manager repository? Because nobody made an influential website destination for it that everybody adopted. E.g. If Google Inc funded a C++ repo with disk space and bandwidth, would corporate politics prevent Microsoft and Apple from uploading their code to it? In contrast, the Javascript quickly adopted NPM that was funded by Nodejs and it was the already default baked into Nodejs downloads. Which entity in the C++ world is analogous to NPM inc to fund a repository that Google/MS/Apple/Adobe/etc would all adopt? It needs to be an entity that cuts across all those competitors. Maybe Intel? Or maybe Epic Games (they make Unreal Engine)? Or all the big competitors agree to contribute to a non-profit entity? Nobody has stepped up to the plate like NPM Inc did.
(2) technical challenges: C++ has a compile and link steps that interpreted languages like Javascript don't have. In npm, the assets are "text" (i.e. .js files like leftpad()). In C++, many library publishers (proprietary) don't offer source code (the "text") and only binary precompiled .lib files. These libraries are binary blobs that are incompatible across cpu architectures and compilers (Intel vs ARM, Microsoft MSVC vs gcc, etc). Because C++ is low-level, there's also a different set of build configs such as 32bit vs 64bit, static link vs dynamic link, and DEBUG vs RELEASE, etc. This means C++ needs to implement a much more complicated "dependency" syntax way beyond just "versions" and nobody has created that specification that everybody has agreed to use. Since Javascript doesn't have separate compile & link phases, something like "npm" is much easier to create. If Rust also has "compile & link" step, how did they successfully get people to adopt the "cargo" tool and the "crates.io" repository? Probably because Rust's ecosystem of programmers has more emphasis on shared "open source code" than the C++ programmers of the 1980s and 1990s. Also, we emphasize again that Rust's patron (Mozilla) also funded "crates.io" and Rust programmers just adopted it so the "cargo" tool has an http destination to point to download artifacts. There's no equivalent influential entity in C++ yet. In terms of package spec complexity, I haven't studied Rust's TOML configuration spec to know if it's flexible enough to handle all of the different ways that C++ practitioners build files. A lot of C++ build flexibility is put into "./configure.sh" scripts (e.g. ffmpeg, Qt, etc) so I don't know if toml spec is a superset of all that.
[1] https://en.wikipedia.org/wiki/Focal_point_(game_theory)
Mozilla pays the Heroku bill for crates.io, and Amazon comes the s3 bucket.
Rust has the compile and link step. People adopted Cargo because it is nice to use.
Rust has the compile and link step. People adopted Cargo because it is nice to use.
>People adopted Cargo because it is nice to use.
I don't think it's that simple and I want people to really dissect what the "nice to use" means. I contend that a fully funded backend ("crates.io" on Heroku+S3) is essential to make cargo a viable tool. A programming language ecosystem needs a Schelling Point (or a disproportionately influential C++ vendor dictating industry-wide tool usage) and the C++ ecosystem doesn't have that.
We can't replay history with an alternate events since cargo and crates.io came together but let's try a thought experiment and think of cargo in isolation:
Let's say we believe cargo's nice usage syntax (and not crates.io) is the what really drives adoption. Ok, let's hypothetically clone the github/cargo project and modify it to C++ (e.g. process ".cxx", "cpp" files instead of ".rs" files. I don't think the existence of a "nice tool" on the client side will matter for C++ programmers. There are still no sponsors paying for a C++ repository with disk+bandwidth that everyone will upload their code to. Swift.org has Apple as its sponsor. Who's the logical entity to fund an industry-wide C++ repo?
[0] https://github.com/rust-lang/cargo
I don't think it's that simple and I want people to really dissect what the "nice to use" means. I contend that a fully funded backend ("crates.io" on Heroku+S3) is essential to make cargo a viable tool. A programming language ecosystem needs a Schelling Point (or a disproportionately influential C++ vendor dictating industry-wide tool usage) and the C++ ecosystem doesn't have that.
We can't replay history with an alternate events since cargo and crates.io came together but let's try a thought experiment and think of cargo in isolation:
Let's say we believe cargo's nice usage syntax (and not crates.io) is the what really drives adoption. Ok, let's hypothetically clone the github/cargo project and modify it to C++ (e.g. process ".cxx", "cpp" files instead of ".rs" files. I don't think the existence of a "nice tool" on the client side will matter for C++ programmers. There are still no sponsors paying for a C++ repository with disk+bandwidth that everyone will upload their code to. Swift.org has Apple as its sponsor. Who's the logical entity to fund an industry-wide C++ repo?
[0] https://github.com/rust-lang/cargo
> There are still no sponsors paying for a C++ repository with disk+bandwidth that everyone will upload their code to.
This already does exist: Conan, for example. And cppget for build2.
This already does exist: Conan, for example. And cppget for build2.
>This already does exist: Conan, for example. And cppget for build2.
But is there consensus that conan.io or cppget have become industry-wide Schelling Points?
For example, I searched both repos and neither have popular dependencies like Qt nor ffmpeg. Therefore, "conan install ffmpeg/4.2.2" isn't going to work. Thus, large non-trivial projects would only be able to use a package manger like Conan/cppget for some dependencies but not all.
In Javascript land, npm is so pervasive that it can be "one-stop shopping" for fetching all js dependencies. Likewise, not only is someone paying the light bills for "crates.io", it also has instant canonical status within the Rust community. Conan doesn't have comparable mindshare in the C++ ecosystem. This is a common interpretation of the question "Why doesn't C++ have a package manager?"
But is there consensus that conan.io or cppget have become industry-wide Schelling Points?
For example, I searched both repos and neither have popular dependencies like Qt nor ffmpeg. Therefore, "conan install ffmpeg/4.2.2" isn't going to work. Thus, large non-trivial projects would only be able to use a package manger like Conan/cppget for some dependencies but not all.
In Javascript land, npm is so pervasive that it can be "one-stop shopping" for fetching all js dependencies. Likewise, not only is someone paying the light bills for "crates.io", it also has instant canonical status within the Rust community. Conan doesn't have comparable mindshare in the C++ ecosystem. This is a common interpretation of the question "Why doesn't C++ have a package manager?"
Our dependency stack contains about 80 3rd party libraries. We have a custom build script to download and compile these libraries.
58 of those libraries can be built using common steps that assume the use of autotools. The remainder are riddled with special case instructions.
I wish I could say that I was more optimistic about efforts like those described in the linked-to content, but I frankly am utterly pessimistic that "dds" or any similar system will ever handle the build requirements of a project like Ardour.
Note also that a large chunk of those libraries are not even written in C++, so their authors and maintainers would not consider them part of any C++ ecosystem should one ever emerge.