These da Vinci style drawings were likely created using the SketchUp models from sometime ago, which you can download from the page below to play with:
At CERN, software stacks are created centrally and software distribution for experiments is done with CVMFS (https://cernvm.cern.ch/fs/), an HTTP-based read-only FUSE filesystem.
EESSI (https://eessi.io) has taken this model further by using CVMFS, Gentoo Prefix (https://prefix.gentoo.org), and EasyBuild to create full HPC environments for various architectures.
CVMFS also has a docker driver to allow only used parts of a container image to be fetched on demand, which is very good for cases in which only a small part of a fat image is used in a job. EESSI has some documentation about it here:
https://www.eessi.io/docs/tutorial/containers/
CERN is a heavy user of ceph, with about 100PB of data across cephfs, object stores (used as backend for S3), and block storage (mostly for storage for VMs). CVMFS (https://cernvm.cern.ch/fs/) is used to distribute the software stacks used by LHC experiments across the WLCG (Worldwide LHC Computing Grid), and is back by S3 with ceph for its storage needs. Physics data, however, is stored on EOS (https://eos.web.cern.ch) and CERN just recently crossed the 1EB mark of raw disk storage managed by EOS. EOS is also used as the storage solution for CERNBox (https://cernbox.web.cern.ch/), which holds user data. Data analyses use ROOT and read the data remotely from EOS using XRootD (https://github.com/xrootd/xrootd), as EOS is itself based on XRootD. XRootD is very efficient to read data across the network compared to other solutions. It is also used by other experiments beyond high energy physics, for example by LSST in its clustered database called Qserv (https://qserv.lsst.io).
I think this is good advice overall. I wrote a CMake script that does most of the heavy lifting for XRootD (see https://news.ycombinator.com/item?id=39657703). The CI is then a couple of lines, one to install the dependencies using the packaging tools, and another one calling that script. So don't underestimate the convenience that packaging can give you when installing dependencies.
I was just taking a look and couldn't help but notice the switch statement for your operator[], which likely causes a lot of unnecessary bad speculation at runtime:
Many believe the C++ compiler will magically optimize the switch away, but in some cases, like the example above for CLHEP, it doesn't happen, so you end up with bad performance.
It was a nice guest post on the website about eclipse, but most people just use gdb. It is now possible to step through ROOT macros with gdb by exporting CLING_DEBUG=1. See https://indico.jlab.org/event/459/contributions/11563/
> In general, managed tools will give you stronger governance and access controls compared to open source solutions. For businesses dealing with sensitive data that requires a robust security model, commercial solutions may be worth investing in, as they can provide an added layer of reassurance and a stronger audit trail.
There are definitely open source solutions capable of managing vast amounts of data securely. The storage group at CERN develops EOS (a distributed filesystem based on the XRootD framework), and CERNBox, which puts a nice web interface on top. See https://github.com/xrootd/xrootd and https://github.com/cern-eos/eos for more information. See also https://techweekstorage.web.cern.ch, a recent event we had along with CS3 at CERN.
Nice to see cliutils trending. Henry really understands CMake, so advice by him is always good to follow. I'm not a big fan of CMake, but after years of suffering, I mean, experience, I've learned how to use it relatively well. One feature I like is the "dashboard" model of ctest. It's somewhat painful to setup, but once you have it working, it makes your life building and testing your project much easier. Unfortunately, when I posted it nobody noticed, but if you want to have a really simple CI setup for your CMake project, checkout this link:
Also, with some advice from Henry I wrote a nice new setup.py for the Python bindings of XRootD integrated with the CMake build. I didn't want a dependency in scikit-build in the end, but his advice helped me a lot.
For me, one of the biggest leaps in how I think about performance was when I learned about the Top-Down Micro-Architecture Analysis Method, by Ahmad Yasin from Intel. You can learn the main ideas from himself in the video below:
The idea to classify cycles into front-end bound, backend bound, bad speculation or memory bound is brilliant. Once you know which one your program suffers from, it's easy to know what can be done to improve things.
I usually just set the working directory to ${CMAKE_CURRENT_SOURCE_DIR}¹ for tests that only read files, and use configure_file()² for files that could be modified or contain variables expanded by CMake. Note that you don't need to add a dependency, as configure_file() runs during configuration, and if you change the input, will copy the file again. CMake also has the notion of text fixtures for more complicated things as well, like setting up a db or other kind of service that your tests may need to run. See property FIXTURES_SETUP³ for that.