BSD Owl: A build system targeting modern Unix systems(github.com)

26 pointsby eatonphil11 anni fa14 comments

BSD Owl: A build system targeting modern Unix systems

https://github.com/michipili/bsdowl

4 comments

evmar11 anni fa

It's a little strange to call yourself "highly portable" when you only support three unix-like platforms and even exclude some popular BSD variants.

michipili11 anni fa

This is a fair point. :) That said, it works on more that what is advertised. At the time of writing it also worked on the current branch of NetBSD, and OpenBSD provided you install manually a modern version of bmake.

It also used to work on Cygwin, but I am not sure of the current status.

icebraining11 anni fa

"portable" seems different from "ported" to me; it implies that it can easily support other systems, even if nobody has taken that work up.

m_mueller11 anni fa

From the examples it looks quite nice and clean. I'm missing lots of info though, maybe I didn't look deep enough:

1) What if you need additional build steps, say an additional preprocessor?

2) Are these .mk templates easy to write, e.g. do they share a lot of code?

3) What if I need different target architectures on the same machine, e.g. CPU and GPU builds? Where do you define compiler flags?

4) Do you have separate debug targets?

michipili11 anni fa

Thanks! Also note

1) It currently supports M4 and provided that there is a simple rule to infer the name of the preprocessed file from the name of the unprocessed file, it would be easy to add another one, see [1].

It is also possible to support more complicated source generation tools, like literate programming tool. We currently have support for noweb [2] which could be used to derive more.

  [1] https://github.com/michipili/bsdowl/blob/master/bps/bps.m4.mk
  [2] https://github.com/michipili/bsdowl/blob/master/bps/bps.noweb.mk

2) I would say, they are not difficult to write, but these are Makefile, and programming Makefiles is a mindset on its own, because we never write the “glorious high-level procedure doing plop-plop fuzz-fuzz on all our files” but just move from one step to the other and must save all state in inodes. :) The folder `bps` contains code which is shared by all frameworks (langc, texmf, ocaml, script).

3, 4) The system supports two variables ARCHITECTURE and CONFIGURATION which are used to selectively include a configuration file to tweak flags (like debugging, profiling, cross-compiler chain and the like) and which can be used to tweak the directory where compilation products are stored. Once it is setup it just looks like

     bmake ARCHITECTURE=Mingw32
     bmake ARCHITECTURE=Native CONFIGURATION=Release

You can see a small example in the example/langc/heat directory [3]

  [3] https://github.com/michipili/bsdowl/tree/master/example/langc/heat

m_mueller11 anni fa

Thank you! Seems to me like a sane and clean concept, i.e. how autotools and co always should have been. I'm quite fond of writing Makefiles myself, so that wouldn't be an issue. When I get around to rewriting the build system for my GPU code preprocessor framework [1] I think I'll give bsdowl another look - it's possible that it would fit quite well and reduce the amount of build system code I have to maintain myself.

[1] https://github.com/muellermichel/Hybrid-Fortran

edit: Ah there is something I missed though: BSD Make only will probably be an issue. My stuff mostly has to work on HPC GPU clusters which are pretty much exclusively Linux based, so GNU Make support is a must I think. Do you have any plans on supporting this so it becomes portable to Linux?

michipili11 anni fa

bmake is very portable and it is available on Debian and Ubuntu. I am not aware of any RPM package but installing from sources is a breeze, I had it running on the newest CentOS as well as on paleontological Solaris editions, without having to tweak a single line of code.

If you look for a more engineered installation solution, give a try to PKGSRC, which is the (portable) ports tree of NetBSD – though I never used it intensively. This even used to work in a (now obsolete) Unix-like environment on windows.

When you'll find time to experiment with the Makefiles, feel free to get in touch (my email is in the source, or with the issue tracker) I will be happy to help.

m_mueller11 anni fa

See, the problem with that is the friction it causes in a HPC environment. Usually, users have no access to a package manager - so everything you want in addition to what's provided by administrators is something you have to build from source, in your home folder, including all the dependencies. And getting administrators to install new software on such a cluster can be difficult in itself. Adoption of new programming languages and libraries is slow in this space mostly for that reason I think, and if you want to have a successful framework, you need to provide it with basically no dependencies attached apart from what you typically find on these systems. Don't forget, the users typically aren't computer/software engineers, they are scientists in my case. I.e. stubborn people who don't particularly care for software if it doesn't directly help in what they're doing ;).

michipili11 anni fa

Also, I forgot to point out, I have an autoinstall script for bmake and bsdowl, it is here:

     https://github.com/michipili/anvil/blob/master/subr/anvil_travisci_autoinstall.sh

It will install both of them under ~/.local, so that you need to tweak your path accordingly, but you can choose another prefix with the `-p` flag, like

  sh anvil_travisci_autoinstall.sh -p "${HOME}"

If you want to customise it, you can trim most of the script (autoinstall_opam*) because you do not need this. :) And if you need something more specific, just ping me on the issue tracker for instance. :)

m_mueller11 anni fa

Hey thanks, that looks promising. I appreciate your help. I'll be busy with other stuff until around end of this year, but at some point I have to make my builds more flexible (allow any number of targets in the configurations / autodetect what's available on the machine). I'll look into it then.

michipili11 anni fa

You can bribe your colleagues by showing them how cleanly it handles LaTeX documents and METAPOST figures, in case they use METAPOST for their technical drawings. :) And the clean target leaves everything you need to send your file to an editor (i.e. it does not remove bibliography files, index, and figures, which are removed by a more powerful realclean command.)

i38611 anni fa

Isn't 'highly portable' through m4, auto tools, etc what got unix build systems complicated in the first place?

IshKebab11 anni fa

Yeah I'd much prefer a not-very-portable but non-crufty build system that only worked on Windows, OSX and Linux (and whatever popular BSDs there are). We don't need to check whether a byte is 8 bits these days.

I was going to say this build system probably isn't that bad, but then in the features list it says it supports m4 and autoconf, which is a bit of a red flag.

erikb11 anni fa

I haven't got yet what it makes better than the other ten million build systems we already have. Can someone summarize?