@null not as yet, but did a tiny bit of research before. If i understand it this would help mainly with batching (ie concurrent sessions/users/turns etc) to keep the pipeline more busy (vs a batch of a fixed size going together), which is interesting, but most work so far has been on latency hiding for straight line performance to start.
We did have a version of things which used expert islands for MoE parallel alternative (I think that has been mostly scrubbed from the code). It showed early promise by having trunk+hot experts together, but as models got larger, it made both that very large on its own but also didn't seem to work as well (or we just weren't good at grouping experts), the idea being to really side step latency and route sessions to those islands. It felt like as models scaled it didn't stay smart (as diverse experts activated more than I thought they would). Could be our mistake (was an exciting possibility though - if you don't mind accepting some loss).
tldr; yes this seems a very nice enhancement for smarter batching/keeping things busy and seems like most larger models we look at are MoE!
Have been using morph for a while (I am one of the authors of goose) and was surprised when introduced at the boost it gave me (much less iteration with the main expensive LLM, and I can even make the editing process simpler to take a load off the agent). Used it with claude 3.5, 3.7, 4 and currently with a o3/openai and anthropic/claude4 + morphllm combo today.
I built this using openAI codex, google cloudrun and streamlit (visualisations by graphviz - codex is not bad at translating pipelines to graphviz without really much prompting which somehow surprised me). Let me know what you think. No fine tuning required for this (as Codex already knows quite a lot).
You mention docker and it is super great for CI: it’s probably one of the widest used feature of Jenkinsfiles (you can specify what image or dockerfile you want a stage to run in - simple but powerful, as you probably know)
On kubernetes - imagine if that was already managed for you, all the powerful things that can be done on top vs your self (preview apps, owasp vulnerability testing, progressive delivery - all without writing a pipeline - this is stuff that can be done.
I note gitlab is mentioned a lot here: they noted this power as well (see auto devops) and have started building things on top of kubernetes too.
Agree Kube is not for everyone, no question, I was just trying to clarify what was in the article. (if you are offloading complexity - I would hope it is to a GKE or EKS or AKS or similar, in which case it is very much offloaded...)
I didn't mean to imply that you mix all those things together from the CDF - was just mentioning some interesting projects (some are unrelated) in the mix.
Agree also on simplicity - myself I don't like to run anything, so CodeShip is what we have for that (but it sounds like you are referring to self hosting only solutions?)
I think this "lesson" is not lost on many, including those that built and work with Jenkins and plugins. The Jenkins X project moved to a very different approach, a fresh start.
see https://cd.foundation/ & https://jenkins-x.io (it works with Jenkins if you like, but natively uses the new pipelines and no SPOF masters). Might not be for everyone but worth a look.
disclosure: cofounder of cloudbees here (wanted to comment on original article but not able to for some reason on medium).
Co-founder of cloudbees here, just some corrections:
1) there are (since 1 year) restartable builds/stages in open source (a bit over a year IIRC) - when the article mentions that they are not in open source.
2) Jenkins X is something VERY DIFFERENT from Jenkins (despite the name), it is "master less", yes requires a kube cluster to power it, but it has no Jenkins instances as you know it, in a lot of ways quite a bit different as it uses Tekton as the engine for running pipelines (which has a bunch of advantages). So I wouldn't group it in with the same challenges and constraints. It is something new that shares a name and some of the ideas.
Jenkins, along with Spinnaker and Tekton and Jenkins X are now part of https://cd.foundation - worth a look to see how they are evolving (expect some changes).
Charles Sturt Uni is a fairly well regarded Australian university. It isn't one of the so called "sandstone club" but I believe it does legitimate stuff (it isn't that easy to be called a university here). They also often specialise in rural/earth science matters (I always thought of them as a rural uni). If that helps?
Yeah it is my instinct to to use credentials but then I pull myself up and think "is this a high brow ad-hominem thing" but it probably makes sense to be curious/skeptical at first.
>That's just the thing. Why has there been so little adoption of commonly needed functionality as part of core, or at least as officially supported plugins?
Oh there are core bundled plugins, official etc - they are just core functionality that happens to be implemented by plugins.
>And why is version control not standard in 2018?
that is and always has been - "git" support used to be not in by default, but that was a while ago (it is included now).
Part of the ideas mentioned are to resolve this stability, and not depend on in process plugins (a new extensibility architecture that won't hurt stability). There are many things in plugins which should be core functionality (and will be).
I think thats a great rule of thumb. Declarative pipeline came after the script was "invented", which is slightly unfortunate, had it came before it would have encouraged the practices you describe (declarative is just for orchestration), and script would have been mainly an escape hatch (I think many people get the idea now though).
It may not work for you but "organization folders" (a feature of multi branch) were designed to discover projects from bitbucket, github etc (ie a new repo appears, with a Jenkinsfile, a new project will automatically configured). Some people have had luck with them (they don't always work how you want though, but when they do, don't need to manually create the pipeline other than dropping in a Jenkinsfile in a new repo).