Project Wisdom for Red Hat Ansible(research.ibm.com)
research.ibm.com
Project Wisdom for Red Hat Ansible
https://research.ibm.com/blog/ai-for-code-project-wisdom-red-hat
12 comments
I do not understand the incentive for this "Project Wisdom". From the official description in the project's home page:
> Project Wisdom will be able to read plain English entered by a user, and then generate automation content written in the Ansible syntax: Ansible Playbooks and Ansible Roles.
Do Ansible users really want to input plain English? I would not even know how to say what I need: "make sure journald is persistent, up to 1GB, then install zsh with my laptop config if it's in a laptop group..." That's awfully imprecise. Nobody wants to program in plain English. And the more technical the task, the less suitable plain English becomes.
And even worse: the goal is to convert plain English into yaml "code", mostly to "make it easier for beginners to learn Ansible". I think it is not helpful at all to throw blocks of generated code to a beginner.
> Project Wisdom will be able to read plain English entered by a user, and then generate automation content written in the Ansible syntax: Ansible Playbooks and Ansible Roles.
Do Ansible users really want to input plain English? I would not even know how to say what I need: "make sure journald is persistent, up to 1GB, then install zsh with my laptop config if it's in a laptop group..." That's awfully imprecise. Nobody wants to program in plain English. And the more technical the task, the less suitable plain English becomes.
And even worse: the goal is to convert plain English into yaml "code", mostly to "make it easier for beginners to learn Ansible". I think it is not helpful at all to throw blocks of generated code to a beginner.
> Nobody wants to program in plain English.
People have wanted that since at least as long as cobol has been a thing.
People have wanted that since at least as long as cobol has been a thing.
There's a reason the languages that followed COBOL ditched the faux natural language part of the syntax. The language is practical for what it does, but the syntax is impractical. The same thing can be said about SQL.
I found it ironic when viewing the live demo on stage that the examples produced didn't have "FQCN"s (full names for modules, like ansible.builtin.copy), but rather shorthand names (like "copy") even though Ansible's lint tool will complain if you use the latter.
That's likely because the training data is Ansible Galaxy, and the vast majority of existing code doesn't use FQCNs. I still prefer not to, since it adds visual clutter to the playbook (not to mention a tiny bit of extra typing even with autocomplete), and playbooks work exactly the same 99.999% of the time.
Someone mentioned they were considering adding more lint-friendly inputs into the model, but as with all things AI, the way it's done could badly affect the output, too.
I still don't feel like there's a ton of value from any AI-driven code completion tools... writing out code (especially the initial bit) is often the first 1% of work when venturing out into new programming or automation work.
That's likely because the training data is Ansible Galaxy, and the vast majority of existing code doesn't use FQCNs. I still prefer not to, since it adds visual clutter to the playbook (not to mention a tiny bit of extra typing even with autocomplete), and playbooks work exactly the same 99.999% of the time.
Someone mentioned they were considering adding more lint-friendly inputs into the model, but as with all things AI, the way it's done could badly affect the output, too.
I still don't feel like there's a ton of value from any AI-driven code completion tools... writing out code (especially the initial bit) is often the first 1% of work when venturing out into new programming or automation work.
my only takeaway from this is cringe. IBM did the same thing with Watson for about a decade. find $thing, cobble watson onto it, gin up a presser, and kick it onto the stage to be unceremoniously ripped apart by the jackals of reasoned thought on places like HN whilst cloistered C levels gobble it up like a can of Dinki Di in the wasteland of AI/ML.
IBM nerfed Ansible when they pulled the community packages off it and made them a paid premium in their Red Hat Enterprise Linux. ansible-core, as they call it, is like a shaved cat: newcomers are going to find it offensive whilst veteran cat enjoyers are going to call it out for what it is and shame you for the misplaced effort with a set of shears while they download the EPEL fur you tucked under the couch cushions.
The reason we did ansible is because we dont have a good product that competes with Github...either that or the tech writing team is particularly bad this year after the recession cuts.
IBM nerfed Ansible when they pulled the community packages off it and made them a paid premium in their Red Hat Enterprise Linux. ansible-core, as they call it, is like a shaved cat: newcomers are going to find it offensive whilst veteran cat enjoyers are going to call it out for what it is and shame you for the misplaced effort with a set of shears while they download the EPEL fur you tucked under the couch cushions.
The reason we did ansible is because we dont have a good product that competes with Github...either that or the tech writing team is particularly bad this year after the recession cuts.
> IBM nerfed Ansible when they pulled the community packages off it and made them a paid premium in their Red Hat Enterprise Linux.
Ansible was split into Ansible Core and the separate collections to separate the lifecycle of collections from Ansible itself. It was not a nerfing of Ansible by IBM. You as an admin get to pick which collections and which versions of said collections fit your needs best, rather than having to wait for new Ansible versions to be released. This is the same argument developers have in programming languages about standard libraries, you can either have a relatively large one (Python) that changes slowly, or a small one (Rust) where libraries are offloaded to the ecosystem and nobody can agree on what's best.
In RHEL, Ansible Core is only provided (in the support sense) for the use with RHEL System Roles (upstream: linux-system-roles). It still provides all the same pieces to do normal Ansible administration workloads. You don't need the EPEL Ansible package unless you want the massive standard library. Full Ansible support is provided by Ansible Automation Platform subscriptions.
Ansible was split into Ansible Core and the separate collections to separate the lifecycle of collections from Ansible itself. It was not a nerfing of Ansible by IBM. You as an admin get to pick which collections and which versions of said collections fit your needs best, rather than having to wait for new Ansible versions to be released. This is the same argument developers have in programming languages about standard libraries, you can either have a relatively large one (Python) that changes slowly, or a small one (Rust) where libraries are offloaded to the ecosystem and nobody can agree on what's best.
In RHEL, Ansible Core is only provided (in the support sense) for the use with RHEL System Roles (upstream: linux-system-roles). It still provides all the same pieces to do normal Ansible administration workloads. You don't need the EPEL Ansible package unless you want the massive standard library. Full Ansible support is provided by Ansible Automation Platform subscriptions.
"Copilot for Ansible".
I'm interested to see how this works out because unlike Copilot where you're "merely" writing pieces of regular code, Ansible is uniquely in a position to trash your entire network and operations, making correct operation absolutely essential. The stakes seem very high to me.
(Disclosure: I work for Red Hat which was acquired by IBM, but not on anything related to this)
I'm interested to see how this works out because unlike Copilot where you're "merely" writing pieces of regular code, Ansible is uniquely in a position to trash your entire network and operations, making correct operation absolutely essential. The stakes seem very high to me.
(Disclosure: I work for Red Hat which was acquired by IBM, but not on anything related to this)
I'd rather iterate over bugs in Ansible code that I created directly than fear some AI will get something I said/typed wrong and completely break my production environment. It's the kind of work you need checks/validation on your thoughts rather than mindless bootstrapping.
The disclosure does seem rather superfluous given your brutal honesty.
Ansible, for me, is the better bash.
Anytime I would typically be reaching for a shell script of medium to long length, it's typically easier to maintain in Ansible.
But, if I find my Ansible is getting rather long, I need to do things more complicated than simple loops, or there isn't an Ansible primitive to interact with the system I'm working with... time to start reaching for an actual programming language like Python or Go.
Anytime I would typically be reaching for a shell script of medium to long length, it's typically easier to maintain in Ansible.
But, if I find my Ansible is getting rather long, I need to do things more complicated than simple loops, or there isn't an Ansible primitive to interact with the system I'm working with... time to start reaching for an actual programming language like Python or Go.
This has been my experience as well.
I gravitated to Ansible because it was language independent, simple to translate line-by-line scripts and didn't require an on server agent to work. Knowing that I could invest into Ansible and take it with me anywhere that I had SSH access was very motivating.
It's not perfect, but it's so much better than my previous experiences that I won't complain about the pain points too loudly.
I gravitated to Ansible because it was language independent, simple to translate line-by-line scripts and didn't require an on server agent to work. Knowing that I could invest into Ansible and take it with me anywhere that I had SSH access was very motivating.
It's not perfect, but it's so much better than my previous experiences that I won't complain about the pain points too loudly.
Have you figured out how to use ansible with virtualenv and Django?
- create virtual env
- activate and install pip packages
- run command using that environment
- create virtual env
- activate and install pip packages
- run command using that environment
Yes, this is 100% possible with the core ansible toolkit.
I don't have a copy of any of our playbooks on this as I'm at home and it's late. But it's basically just a couple of tasks.
Check out `ansible.builtin.pip` to setup and install the virtualenv and then just call `ansible.builtin.command` to run it explicitly. Implicit use of the venv can be accomplished via common task argument `env` which also lets one fiddle with PATH.
I don't have a copy of any of our playbooks on this as I'm at home and it's late. But it's basically just a couple of tasks.
Check out `ansible.builtin.pip` to setup and install the virtualenv and then just call `ansible.builtin.command` to run it explicitly. Implicit use of the venv can be accomplished via common task argument `env` which also lets one fiddle with PATH.
I don't like it much at all. I see the value of idempotency and so on, but it is a terrible "language" shoehorned into yaml. I am now migrating my home Ansible scripts to Nix and I won't look back.
> YAML — a popular *programming* language that is human-readable and easy to understand.
What?
What?
Ansible expresses (in YAML) constructs for variables, loops, conditions and more.
I think the point here is that YAML is not a programming language.
If you're writing Ansible playbooks with YAML then you are using it as a declarative programming language. So YAML can be a programming language.
More like the encoding scheme for the programming language. YAML isn't a language, ansible is; it's like how unicode isn't a programming language.
YAML: Yet Another Meta-Language.
It's not a what now?
It's not a programming language. (Rather like HTML.) It's a markup language that can be used as the encoding for a real programming language, but it's not a programming language itself.
I find it annoying that they've apparently chosen to rebrand it. It used to just be called "Ansible". Just because Red Hat sponsors it, doesn't mean it's name has to change. It's not a UK football ground (whose names change depending on which middle-eastern despot funds the resident club).
Wisdom for ansible would be to ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters.
Instead, we get a terrible dsl baked in a bad markup language that is hard to write, painful to debug and impossible to maintain.
That you have to come up with concepts like an inventory, fact and magic variables should have been your first clue that they were reinventing badly what any language ecosystems provides for free.
And the worst is, ansible is probably one if the best deployment systems at our disposal.
Instead, we get a terrible dsl baked in a bad markup language that is hard to write, painful to debug and impossible to maintain.
That you have to come up with concepts like an inventory, fact and magic variables should have been your first clue that they were reinventing badly what any language ecosystems provides for free.
And the worst is, ansible is probably one if the best deployment systems at our disposal.
That's a hot take. In my experience Ansible is trivial to maintain and, due to the lack of better templating like Salt, ensures most playbooks look relatively similar as it's meant to be a simple, straight forward task runner. Onboarding folks into Ansible is a breeze as everything matches the standard (roles, tasks, group_vars, etc). It's dumb, reliable, and does exactly what it says on the tin.
My biggest complaint with Ansible is the execution aspect. The fact that mitogen hasn't become the standard strategy plugin and instead Ansible still opens an SSH connection per task is nuts.
My biggest complaint with Ansible is the execution aspect. The fact that mitogen hasn't become the standard strategy plugin and instead Ansible still opens an SSH connection per task is nuts.
My experience is that Ansible becomes exponentially more difficult with the difficulty of your workflows, where programmatic tends to be logarithmic. For low complexity tasks, Ansible is simpler and for high complexity tasks it gets more complicated.
Just as a random example, for complex group structures you have to know/remember that although Ansible allows nesting groups, groups are actually a flat structure. If you have prod.webservers and uat.webservers, the 'webservers' group will include members of both.
That's not hard to work around, but it can easily lead to situations where you end up deploying to prod even though you only intended to deploy to UAT.
Variables are kind of a mess too. Assuming the docs are authoritative, there are 22 different places that variables can come from and which one is used will depend on the precedence. At some point, a 'switch' or series of 'if-elif' statements start to sound much simpler.
Just as a random example, for complex group structures you have to know/remember that although Ansible allows nesting groups, groups are actually a flat structure. If you have prod.webservers and uat.webservers, the 'webservers' group will include members of both.
That's not hard to work around, but it can easily lead to situations where you end up deploying to prod even though you only intended to deploy to UAT.
Variables are kind of a mess too. Assuming the docs are authoritative, there are 22 different places that variables can come from and which one is used will depend on the precedence. At some point, a 'switch' or series of 'if-elif' statements start to sound much simpler.
RE: variables, I'm a heathen and set `hash_behaviour = merge`. This forces nested dict values to be merged instead of overwritten (lists cannot be merged, unlike Salt...). For whatever reason Ansible encourages users to not use this [1], which makes variable usage far more difficult than it should be IMO and leads to situations like you describe where folks have various top level variables for overrides (ssh:, ssh_local:, ssh_global:, etc to combine values).
[1] https://docs.ansible.com/ansible/latest/reference_appendices...
[1] https://docs.ansible.com/ansible/latest/reference_appendices...
Oh wow, I've been working with ansible for a few years now, how have I not seen this before? I've even searched for it, but I didn't know what it was called, so...
Do you know if this behavior can be enabled per-role, or even better, per-task?
Do you know if this behavior can be enabled per-role, or even better, per-task?
For context, I’ve written probably a million lines of Ansible and even maintain my own fork to add features like loopable blocks. I love it enough to curse its name.
Large Ansible code bases are painful to maintain. Roles don’t encapsulate well enough and the setting that makes their variables private is unusable if you need to “exports”/“outputs” and I guarantee you do, flow control isn’t powerful enough and you either supplement by dropping down to script/shell or by wring whole programs in Jinja, variables are so spooky action at a distance that you need an external tool to graph where they come from, vault variables aren’t greppable so good luck there, you can’t write a type checker because it’s all strings and any large codebase will dynamically generate variable names due to lack of expressiveness, you will desperately want a way to tell Ansible to eagerly evaluate some variables instead of keeping them as strings and find out how ugly the workaround is without your own patches, you eventually desperately wish you had the ability to evaluate filters and lookups on the target and might eventually, like me, write a plug-in for that. Your options for complex data transformations is writing 50 line filter pipelines with item.0 item.1 or dropping to python and just writing your own filter, you will also eventually desperately wish you could define new modules via “subroutines” by chaining a few trivial module invocations together and then find out that you must drop to an action plugins which expose all the underbelly of Ansible. You will pull your hair out getting Jinja to strip the whitespace from strings to get Ansible to recognize the results as an array since it’s all strings, you will find out that Ansible has its own string type that doesn’t play nice with str and that is somehow exposed to you in bog standard Ansible.
If Ansible was a Python library I would scream for joy at how much unnecessary complexity and necessary bullshit trivia I could drop from my brain.
> and instead Ansible still opens an SSH connection per task is nuts
Mitogen is not worth the bugs compared to using SSH ControlPersist.
Large Ansible code bases are painful to maintain. Roles don’t encapsulate well enough and the setting that makes their variables private is unusable if you need to “exports”/“outputs” and I guarantee you do, flow control isn’t powerful enough and you either supplement by dropping down to script/shell or by wring whole programs in Jinja, variables are so spooky action at a distance that you need an external tool to graph where they come from, vault variables aren’t greppable so good luck there, you can’t write a type checker because it’s all strings and any large codebase will dynamically generate variable names due to lack of expressiveness, you will desperately want a way to tell Ansible to eagerly evaluate some variables instead of keeping them as strings and find out how ugly the workaround is without your own patches, you eventually desperately wish you had the ability to evaluate filters and lookups on the target and might eventually, like me, write a plug-in for that. Your options for complex data transformations is writing 50 line filter pipelines with item.0 item.1 or dropping to python and just writing your own filter, you will also eventually desperately wish you could define new modules via “subroutines” by chaining a few trivial module invocations together and then find out that you must drop to an action plugins which expose all the underbelly of Ansible. You will pull your hair out getting Jinja to strip the whitespace from strings to get Ansible to recognize the results as an array since it’s all strings, you will find out that Ansible has its own string type that doesn’t play nice with str and that is somehow exposed to you in bog standard Ansible.
If Ansible was a Python library I would scream for joy at how much unnecessary complexity and necessary bullshit trivia I could drop from my brain.
> and instead Ansible still opens an SSH connection per task is nuts
Mitogen is not worth the bugs compared to using SSH ControlPersist.
Control persist still creates multiple SSH sessions though?
I have a SSH hook that sends a chat message whenever someone SSHs into a server, and Ansible without mitogen floods these, even with ControlPersist.
If you have it working, can you share your ansible.cfg?
I have a SSH hook that sends a chat message whenever someone SSHs into a server, and Ansible without mitogen floods these, even with ControlPersist.
If you have it working, can you share your ansible.cfg?
Well yeah, because ControlPersist is just “make session creation fast” by multiplexing them over a single SSH tunnel. So you still have “ssh internal session creation” overhead but the tcp and ssh handshake is the slow part. The speed gap between pipelining with ControlPersist and Mitogen is small enough that it’s, to me, not worth the bugs and limitations of Mitogen.
I do get the annoyance of your script though.
I do get the annoyance of your script though.
Yeah after using CloudFormation, Terraform, Pulumi, AWS CDK and k8s + YAML, I'm convinced using a real programing language idiomatically is the correct way of doing IaC.
Many languages totally support declarative idioms out of the box (like you mention), there's no need to reinvent anything here.
In particular I'm a big fan of AWS CDK, although it only works for AWS. (Yes, I know about CDKTF, but it's buggy alpha software).
Many languages totally support declarative idioms out of the box (like you mention), there's no need to reinvent anything here.
In particular I'm a big fan of AWS CDK, although it only works for AWS. (Yes, I know about CDKTF, but it's buggy alpha software).
> ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters.
Fully wholeheartedly agree but, as I stated in previous ocasions in this site, after the nightmares I've seen people, in many different places, doing with a high level tool like Ansible (and also Terraform), I apply heuristics about the probable outcome of every DevOps using a proper Turing-complete language as a general trend and I just want to leave CS and buy a hut in the middle of nowhere. Perhaps something like that must come after a shift in the mentality of the whole business where people in CS in general, assumes one needs to be a good coder, no matter if you do DevOps of software and therefore, learn about software design patterns, reusability and best coding practices.
Fully wholeheartedly agree but, as I stated in previous ocasions in this site, after the nightmares I've seen people, in many different places, doing with a high level tool like Ansible (and also Terraform), I apply heuristics about the probable outcome of every DevOps using a proper Turing-complete language as a general trend and I just want to leave CS and buy a hut in the middle of nowhere. Perhaps something like that must come after a shift in the mentality of the whole business where people in CS in general, assumes one needs to be a good coder, no matter if you do DevOps of software and therefore, learn about software design patterns, reusability and best coding practices.
The issue is that Ansible right now today is just Python but if you tied chopsticks to your fingers when typing. It’s not declarative, it’s sequentially evaluated with all the flow control and concurrency bugs to go with it.
You have to already be a good coder to write Ansible but you have all your limbs tied behind your back because you get none of the features of Python to help you reign in complexity and validate correctness.
You have to already be a good coder to write Ansible but you have all your limbs tied behind your back because you get none of the features of Python to help you reign in complexity and validate correctness.
The killer feature of Ansible vs. Python code is, noting I have a ton of both here for infrastructure related tasks (as well as shell, why not), when you have it running for three years on some box somewhere and something breaks, you go look at it for the first time in three years and it makes sense. My Python infrastructure tools OTOH, while they are great, if something goes wrong with them I have to spend half a day reading the sourcecode and remembering what their particular architcture / conventions are.
it's like the dumbness of Ansible is really the main feature it has.
it's like the dumbness of Ansible is really the main feature it has.
I agree with you but I think that dumb + size => complicated which is where I live. When you have a playbook that is thousands of tasks long the dumbness leads to exactly the problem you have with Python.
I agree with that! my ansibles are for really straight stuff deployed to a few boxes. Where I work, oddly enough at Red Hat :), the very large playbooks are definitely difficult to work with. It's hard to say though if a custom Python app of similar size is really any better though, unless your developers are Michelangelo.
I've been very slowly rebuilding all our Ansible roles in PyInfra for that exact reason.
Also been going down this path. pyinfra is overlooked I think.
It's not perfect, I still feel like I'm dropping down to just hitting `server.shell` a bunch , but at least as a programmer first, it feels more comfortable vs the mountains of yaml (and doc searching) for Ansible.
On the whole it's a less frustrating experience. I still have a vague feeling that in my case - where each server(s) is generally customised per client and I've never felt comfortable just pointing ansible (or pyinfra) and a configured server and re-running it - that perhaps I would really just be better off with a bunch of shell scripts. I don't generally tear-down and throw-up multiple servers a day, week or even month.
rrconf sort of fills this but works off a bit more infrastructure than I want to manage with its different git repos per action.
[1] https://github.com/rrconf/rrconf
It's not perfect, I still feel like I'm dropping down to just hitting `server.shell` a bunch , but at least as a programmer first, it feels more comfortable vs the mountains of yaml (and doc searching) for Ansible.
On the whole it's a less frustrating experience. I still have a vague feeling that in my case - where each server(s) is generally customised per client and I've never felt comfortable just pointing ansible (or pyinfra) and a configured server and re-running it - that perhaps I would really just be better off with a bunch of shell scripts. I don't generally tear-down and throw-up multiple servers a day, week or even month.
rrconf sort of fills this but works off a bit more infrastructure than I want to manage with its different git repos per action.
[1] https://github.com/rrconf/rrconf
I understand where you are coming from. It's interesting to see the Puppet/Chef-debate played out over and over again.
You generally want an overarching single source of truth for IT systems. One that defined which parts a system contains and how it is set up. This starts to get important when you want maintainable testing environments, and more so as the number of system integration points grow, but it generally makes tracking changes and having a controlled deployment process so much easier.
So it is natural to start with a declaration of how small pieces fit together. This is Puppet/Ansible where the language is just a list of objects, where each object is implemented in "real" programming language. But the list soon needs logic, if not for other reasons than because testing environments have subtly different requirements, and the list grows into a semi-language.
That's not enough and people new to the system soon feels constrained, why not ditch that domain specific thing and implement the whole thing in the "real" language? This is Chef/Salt. But now all you have is another object, which needs an overarching list of objects, and soon we have another simple description for that.
And then we repeat this all over again.
(There are specific issues for some parts of this, which is not part of the above story. Ansible didn't have to choose yaml for their domain specific language, they should have seen that coming. But that's not for this story. We instead get to see this repeated again in the container space with templates and Helm charts. Also Jenkins pipelines which slowly mutates to Groovy and back.)
You generally want an overarching single source of truth for IT systems. One that defined which parts a system contains and how it is set up. This starts to get important when you want maintainable testing environments, and more so as the number of system integration points grow, but it generally makes tracking changes and having a controlled deployment process so much easier.
So it is natural to start with a declaration of how small pieces fit together. This is Puppet/Ansible where the language is just a list of objects, where each object is implemented in "real" programming language. But the list soon needs logic, if not for other reasons than because testing environments have subtly different requirements, and the list grows into a semi-language.
That's not enough and people new to the system soon feels constrained, why not ditch that domain specific thing and implement the whole thing in the "real" language? This is Chef/Salt. But now all you have is another object, which needs an overarching list of objects, and soon we have another simple description for that.
And then we repeat this all over again.
(There are specific issues for some parts of this, which is not part of the above story. Ansible didn't have to choose yaml for their domain specific language, they should have seen that coming. But that's not for this story. We instead get to see this repeated again in the container space with templates and Helm charts. Also Jenkins pipelines which slowly mutates to Groovy and back.)
Pulumi could have filled this nicely but only supports a few cloud providers.
Wisdom for ansible would be to ditch ansible and any similar tool.
I feel like specialized AI code completion and hints will be the next evolution of language servers.
On a similar vein, I thought of formatters analyzing classes, variables and args names to make improvement suggestions.
Aka, linting/formattimg for the naming conventions.
Aka, linting/formattimg for the naming conventions.
There's a lot of current research around using machine learning to optimize software. One paper I'm thinking of is: https://dl.acm.org/doi/10.1145/3533028.3533308 where they use ML to optimize latency and hardware usage of Apache Flink dataflow programs.
When I look back to using and writing Ansible I experienced it as my personal dark ages.
I'm so happy I embraced immutable infrastructure. I never needed Ansible anymore.
I'm so happy I embraced immutable infrastructure. I never needed Ansible anymore.
TBH reads like COBOL.
But it won't matter, since our benevolent robotic AI overlords will create them for you, the certified IBM Project Wisdom(TM) Prompt Engineer, dual-classed to Tech-Priest (but without any Warp to go mad in - that, you will have IT for).