Hey, fair pushback, let me try to clear up a couple of points because I think there are some genuine misconceptions worth untangling.
On footguns.
Totally hear you that "Python lets you do anything" feels like a footgun. The flip side that I think gets missed: because it is real Python, you can actually test it. Pytest, mypy, ruff, jump-to-definition, refactor-rename, all of it just works. Unit-testing a 400-line YAML role with nested Jinja conditionals is genuinely hard, and that gap is what pushed me toward PyInfra in the first place.
On "importing Python libraries introduces bugs".
This one I think is worth a closer look, because the mechanics are not what they appear. PyInfra does not run Python on your servers. It runs Python on your control node to plan the change, then transpiles each operation to plain POSIX shell and pipes that over SSH. If you run with `-vvv` you can see it: `sh -c '...'` and nothing else on the wire. The target needs zero Python, zero agent, zero runtime. So whatever library you imported into your deploy script ran locally, produced a string of shell, and that string is what touches the box. A bug in some PyPI dependency cannot throw mid-operation on the host, because there is no Python on the host to throw it. Worth noting that Ansible, by contrast, ships a Python interpreter and module code to the target for most tasks, so if anything the library exposure on the executing side is larger there, not smaller.
On the control node, sure, you have dependencies, same as Ansible has Jinja2, PyYAML, paramiko, cryptography, and a long tail of Galaxy collections of varying quality. PyInfra has a stable API, solid test coverage, idempotent operations, and a real two-phase model (gather facts, then apply) so the apply phase is deterministic generated shell rather than arbitrary code running on the box.
On YAML keeping you on the paved path.
I really wanted this to be true for years, honestly. In practice, the moment you need a conditional you end up writing `{% if %}` inside a quoted string inside a map inside a list inside a role, with no type system, no debugger, and a few sharp edges in the parser (`no` as boolean, leading zeros as octal in YAML 1.1, tab/space mixing failing without a useful pointer). And the escape hatch when Jinja-in-YAML cannot express what you need is... writing a custom Python module. So you end up writing Python anyway, just with worse tooling around it.
The way I would put it: PyInfra is Python where Python helps (writing, testing, planning) and shell where shell belongs (executing on the host). Happy to dig into any specific footgun you have run into though, those are usually the most useful conversations.
Glad it clicked. The Ansible vs PyInfra docs gap isn't really preference, YAML plus Jinja plus a custom DSL is just more cognitive load than plain Python with type hints. Once you can grep the source and read it like normal code, going back feels rough.
On the restart bug: if it resurfaces, an issue on GitHub with the OS, connector (ssh/local/docker), and raw output would help a lot. The 3.x line cleaned up a bunch around connection handling and output buffering, so there's a decent chance it's already fixed.
Thanks for the video, will watch. Hands-on intro content is exactly what the project needs more of.
PyInfra is an agentless infrastructure automation tool. Same job description as Ansible, Salt, Chef. SSH into hosts, describe desired state, it diffs and converges. No agent, no central server, no daemon.
The difference: your "playbook" is just Python. Not Python cosplaying as YAML. Not Jinja smuggled inside YAML inside a Helm chart inside a Kustomize overlay. Actual Python:
from pyinfra.operations import apt, files, server
apt.packages(packages=["nginx"], update=True)
files.template(src="nginx.conf.j2", dest="/etc/nginx/nginx.conf")
server.service(service="nginx", running=True, enabled=True)
Idempotent operations. Facts gathered from hosts, branched on with normal `if` statements. Real loops, real imports, a real debugger, real type hints. Your editor autocompletes arguments because, brace yourself, they are just function signatures.
About YAML. Wonderful format. For about eleven minutes. Then someone needs an `if`, and you have `{% if %}` inside a string inside a list inside a map. Then someone types `no` as a country code for Norway and it ships to prod as `False`. Then someone indents with a tab and the parser dies without saying where. Congratulations, you reinvented a programming language. Badly. The honest move is to admit you wanted code, then write code.
PyInfra skips the eleven good minutes and goes straight to code.
Release notes in the link. Happy to answer questions.
Infrastructure as Code, not infrastructure as YAML.
Moving from classic servers to containers you get:
- Builds with fixed dependencies that never change. Rollback is easy -> what about VMs?
- Easy deployment of a prod environment on a local machine -> yep, that's a nice touch, the only valid point for me!
- Fast deployment -> lol no, Im faster with VMs.
- Easy automation (use version X with config Y) -> valid for VMs and baremetal too
With Kubernetes (or other derivates like Openshift) you get:
- Auto scaling -> you can get it with VMs too
- Fail over -> you can get it with VMs too
- Better resource usage if multiple environments are executed -> you can get it with VMs too
- Abstraction of infrastructure -> Should I really write it?
- Zero downtime deployment (biggest point for my company, we deploy >3 times per week) -> We do on some specific DC (government style) and we release 10-15 times and day with Bare metal servers and ansible
There are applications that do not need Kubernetes or even containers, but is this list really nothing oO? -> None of the arguments convinced me
I can imagine that if you use Kubernetes just like a classic cluster it could seem like an unnecesarry added complexity but you gain a lot of things. -> yes, extra cost and extra skills needed
> The sad fact is that a lot of software is difficult to compile; isn't documented well, something that is worse for building; won't work well if installed in a non-standard way, whether that is final location, different supporting libs, or different platform; and can take a long time.
So are you ready to deploy in prod a software that is so hard to compile?
> I'm happy nowadays when I see there's a binary available, no mucking around with gcc/clang/llvm - just trying to work out which one, let alone which version! - no diving down a rabbit hole of compiling dependencies that then need other dependencies compiled… no deciphering Makefiles that were written in a way that only a C guru can grok, with no comments.
But that's my job, as SRE/DevOps/whatever new fancy name!
> Whatever the benefits are, I prefer sanity.
Sanity of having a very old software, with backported features that are only on this distrib? I prefer to trust the engineers from the software that I deploy.
> The argument here I suppose is live on the bleeding edge? Or swap compilers/allocators as necessary? Or use O3? I'm not entirely sure.
My point is that when you are using a software, especially open source, you should evaluate it, and understand it. BTW, we are running arch in production, with our own repos.
> It is PITA is keeping up with compiler changes and library changes of third party software. In this case, throwing in different malloc implementations in there too, not to mention different libc implementations. With all those variables for each component you deploy you're probably less likely to understand what's going in your software.
in my example, the libc implementation is always the same! it's just that it's outdated on all main docker images. Furthermore, Redis already use a different malloc implementation (jemalloc), but in the makefile, they support also the standard malloc and tcmalloc, so throwing in another one is very easy.
> Maybe for a small app with 2-3 extra pieces or something that needs to really be optimized this is good advice, but it sounds like a lot of work for significant footprints.
Or when you need security/performance. of course, if you have 2 servers, it's useless, but we have more than 4000 servers running in production on arch, so it's worth it for us.
> It would be nice if the official docker images had some tags that were better optimized, within reason.
or just be up to date