Swift package manager(github.com)
github.com
Swift package manager
https://github.com/apple/swift-package-manager
7 comments
He is now employed by Apple.
It's seriously so cool that the Homebrew guy (Max Howell) is behind this. Great fit for him at Apple. =P
Lets just hope there aren't any binary trees in need of inverting at Apple ;)
Seriously though, seems like a perfect fit, much better than him working on something unrelated at Google etc
Seriously though, seems like a perfect fit, much better than him working on something unrelated at Google etc
For those who might not get the reference: https://twitter.com/mxcl/status/608682016205344768?lang=en
Having used Homebrew, I can just say I'm glad I don't write swift code, because if this turns out anything like that, ugh.
Okay, I'm gonna have to enter rant mode here, because this is aggravating me. Why in the bloody fuck do we need another package format? Okay, cool, you have a project, it has needs, you wanna track it. How hard is it to use one of the pre-existing formats? We have rpm, deb, pkg, pkgng, etc as system packaging formats. Is your language such a precious and delicate flower that it needs its own packaging format? You can even use your own binaries to manage the bloody database, but can we get people to agree on some sort of standard here?
This makes administration non-fun. It means needing to run a separate binary for every single package to audit for any sort of language change. It means that a project that uses multiple languages now needs multiple packaging databases to do its job. Which means that auditing is now much, much more difficult, because you have to go into each project's database, figure out what language it is, then call its database packager to get a list of packages and versions, and parse it with code you can't reuse for the other ten languages you've gotten on your system.
So, before you start creating the next big package manager, please, for the sake of your ops people everywhere, see if someone else has already made a sensible packaging format. Chances are that they have, and by reusing their work, you'll make your job a ton easier, and you'll make ops' job a ton easier.
This makes administration non-fun. It means needing to run a separate binary for every single package to audit for any sort of language change. It means that a project that uses multiple languages now needs multiple packaging databases to do its job. Which means that auditing is now much, much more difficult, because you have to go into each project's database, figure out what language it is, then call its database packager to get a list of packages and versions, and parse it with code you can't reuse for the other ten languages you've gotten on your system.
So, before you start creating the next big package manager, please, for the sake of your ops people everywhere, see if someone else has already made a sensible packaging format. Chances are that they have, and by reusing their work, you'll make your job a ton easier, and you'll make ops' job a ton easier.
> We have rpm, deb, pkg, pkgng, etc as system packaging formats.
Those are all general binary distribution formats: they copy certain files as root to a predefined position in the file system and execute code as root. Not at all the use case for this: Downloading library source files that are needed for building something as a user.
It actually also uses pre-existing things: "a package is a git repository with semantically versioned tags, that contains Swift sources and a Package.swift manifest file at its root."
Those are all general binary distribution formats: they copy certain files as root to a predefined position in the file system and execute code as root. Not at all the use case for this: Downloading library source files that are needed for building something as a user.
It actually also uses pre-existing things: "a package is a git repository with semantically versioned tags, that contains Swift sources and a Package.swift manifest file at its root."
As others have pointed out system package management is quite different and solves an overlapping but distinctly different set of problems to programming language module package management. In particular system package management isn't cross platform whereas programming language module package management tries to be. The programming language module package management has to integrate with whatever system package management is doing.
However, I do think you have a valid point in that there are plenty of commonalities between the software module package management needs of different languages. There is little need for each language to re-invent its own system.
It certainly would be considerably more challenging to develop a system that could work for multiple languages, and then there are the existing ecosystems to consider in terms of (the huge) cost of migration, mindshare, etc.
But in principle, you are right, even if economics means that it cannot happen in practice. No need for Yet Another One. But it will happen because economics.
However, I do think you have a valid point in that there are plenty of commonalities between the software module package management needs of different languages. There is little need for each language to re-invent its own system.
It certainly would be considerably more challenging to develop a system that could work for multiple languages, and then there are the existing ecosystems to consider in terms of (the huge) cost of migration, mindshare, etc.
But in principle, you are right, even if economics means that it cannot happen in practice. No need for Yet Another One. But it will happen because economics.
No one is suggesting using it as a system package manager, least of all Apple.
It distributes source code and the audience is developers wanting to integrate that into their projects at build time.
Using something like RPM or DEB for that task would be, uh, suboptimal.
It distributes source code and the audience is developers wanting to integrate that into their projects at build time.
Using something like RPM or DEB for that task would be, uh, suboptimal.
I agree that language specific package managers are terrible. I'm one of those ops people, and the number of package managers needed to deploy any web application is just insane. The biggest problem is that none of them can specify the full dependency graph for anything. For example, the Pg Ruby gem cannot specify that it requires the libpq shared library and headers. You have to either know that before hand, or watch the package installation fail, find the right package in your system package manager, then try again. They are also bad at deduplicating dependencies amongst many projects, but at least they allow installation as an unprivileged user. How many more pm and vm (version manager) tools must we deal with? I wish more people would realize that this is a general problem that requires a general solution.
Traditional system package managers are also terrible. They install everything into the global /usr directory so that you can't have differing sets of depdencies for different software and they require root access because /usr is only writeable by root. Because of these severe limitations and without effort being put in to fix it, I think GNU/Linux distributions have helped push everyone towards using a new package manager per language so that people can get their work done.
The solution to all of these problems is functional package management. They take the good parts of both system and language-specific package managers and add even more features and improvements on top. Functional package managers can handle software written in any language, allow unprivileged package installation, and allow different packages to depend on different versions of the same software without issue. They have additional features such as transactional upgrades and rollbacks, and system-wide deduplication. The entire dependency graph is precisely captured, all the way down to a small set of bootstrap binaries that are the roots of the graph. The GNU Guix [0] and Nix[1] projects implement the functional package management paradigm, and they are both worth taking a serious look at. In addition to standard package management features, they also can do full-system configuration management with their respective distributions GuixSD and NixOS.
[0] http://gnu.org/s/guix
[1] http://nixos.org/
Traditional system package managers are also terrible. They install everything into the global /usr directory so that you can't have differing sets of depdencies for different software and they require root access because /usr is only writeable by root. Because of these severe limitations and without effort being put in to fix it, I think GNU/Linux distributions have helped push everyone towards using a new package manager per language so that people can get their work done.
The solution to all of these problems is functional package management. They take the good parts of both system and language-specific package managers and add even more features and improvements on top. Functional package managers can handle software written in any language, allow unprivileged package installation, and allow different packages to depend on different versions of the same software without issue. They have additional features such as transactional upgrades and rollbacks, and system-wide deduplication. The entire dependency graph is precisely captured, all the way down to a small set of bootstrap binaries that are the roots of the graph. The GNU Guix [0] and Nix[1] projects implement the functional package management paradigm, and they are both worth taking a serious look at. In addition to standard package management features, they also can do full-system configuration management with their respective distributions GuixSD and NixOS.
[0] http://gnu.org/s/guix
[1] http://nixos.org/
So, do you manage cocoapods or carthage for your iOS/OS X developers right now?
Go away and rant how all languages except one your favourite should be banned, because that would make ops job so much easier. Though I am still not sure what ops are doing on developers' machines.
No, I don't want languages banned, I want them to stop reinventing the package management wheel. Why the bloody hell does apple need to release a package manager for swift, with its own new format, when, as you pointed out, cocoapods already exists? Why a new, completely incompatible format?
The current state of the world means that it is currently entirely feasible that some components will be managed by cocoapods, and some by this new format. What world-shattering difference between the two exists such that they can't share a metadata format to describe a development package? How does the new format make life easier?
The current state of the world means that it is currently entirely feasible that some components will be managed by cocoapods, and some by this new format. What world-shattering difference between the two exists such that they can't share a metadata format to describe a development package? How does the new format make life easier?
So you want every developer to produce rpm, deb, msi, depot, bff, pkg, dmg, <place missing package format here> taking into consideration OS specific paths and configuration settings for every single library, specially for packages common across OSes?!!
No. I want people to stop reinventing the metadata wheel. I used system packages as an example. We now have Package.swift for swift and package.json for node, cargo.toml for rust, etc. What problem does Package.swift solve that package.json doesn't?
How does package.json handle compiler semantics, linker, library, modules, architectures, incremental compilation, cross-compilation, dependencies to Objective-C and C libraries, ... ?
There are configuration variables for pretty much everything you listed. There are also variables for preinstall scripts, postinstall scripts, removal scripts, upgrade scripts, etc. defined in the package.json spec.
And its not called Taylor?
I certainly wouldn't mind if it was called Tailor instead... :)
Already taken:
https://github.com/izqui/Taylor
https://github.com/izqui/Taylor
I see that there is already a Heroku buildpack for Swift, based on the package manager[1] and it was ported to Cloud Foundry[2].
Speaking of my own experience in working on Cloud Foundry buildpacks, a feature I'd be keen on having is true "vendoring" of dependencies. Bundler does this correctly with a common corner case -- gems that include C/C++ code. It will keep that source code, as well as ruby. Turns out to be a very important feature when deploying to a disconnected environment.
[1] https://github.com/kylef/heroku-buildpack-swift
[2] https://blog.starkandwayne.com/2015/12/08/apple-swift-buildp...
Speaking of my own experience in working on Cloud Foundry buildpacks, a feature I'd be keen on having is true "vendoring" of dependencies. Bundler does this correctly with a common corner case -- gems that include C/C++ code. It will keep that source code, as well as ruby. Turns out to be a very important feature when deploying to a disconnected environment.
[1] https://github.com/kylef/heroku-buildpack-swift
[2] https://blog.starkandwayne.com/2015/12/08/apple-swift-buildp...
All it does is git clone tagged network git repos and invoke swiftc.
No support for tests yet, beyond ignoring folders called 'Test'. It can't even support local file paths at the moment.
Certainly, watch this space, but there's hardly anything to be excited about here yet.
No support for tests yet, beyond ignoring folders called 'Test'. It can't even support local file paths at the moment.
Certainly, watch this space, but there's hardly anything to be excited about here yet.
Finally !
https://github.com/apple/swift-package-manager/commits/maste...