This is the right way to do it. Whether using trunk-based development, git-flow, etc -- you're controlling the flow of merges in a particular direction.
However, I think the "larger planning issue" is harder to easily avoid when you have more devs, more changes, or the AI-boosted output we have today. If feature B requires feature A, and feature A isn't up-to-date with main, I could rebase feature A to main, then feature B to feature A. When feature A is merged and we're ready to PR feature B, I can rebase to main again then make my PR.
I agree that there's a lot of complexity today due to the process in which we write code (people, lack of understanding the problem space, etc.) vs the problem itself.
Would we say us as humans also have captured the "best" way to reduce complexity and write great code? Maybe there's patterns and guidelines but no hard and fast rules. Until we have better understanding around that, LLMs may also not arrive at those levels either. Most of that knowledge is gleamed when sticking with a system -- dealing with past choices and requiring changes and tweaks to the code, complexity and solution over time. Maybe the right "memory" or compaction could help LLMs get better over time, but we're just scratching the surface there today.
LLMs output code as good as their training data. They can reason about parts of code they are prompted and offer ideas, but they're inherently based on the data and concepts they've trained on. And unfortunately...its likely much more average code than highly respected ones that flood the training data, at least for now.
Ideally I'd love to see better code written and complexity driven down by _whatever_ writes the code. But there will always been verification required when using a writer that is probabilistic.
I think my issue with this generalization is assuming the code itself is where complexity is measured and applied.
For example, the Quake Fast Inverse Square Root[1] takes into account nuances in how floating point numbers can be manipulated. The individual operations/actions the code takes (type casts, bit shifts, etc.) are simple enough, but understanding how it all comes together is where the complexity lies, vs just looking at the graph of operations that makes up the code.
Tools like Rubocop for Ruby take an approach like you mention, measuring cyclomatic and branch complexity in your code to determine a mathematical measurement of the complexity of that code. Determining how useful this is, is another conversation I think. I usually find enforcing rules around that code complexity measurement against your code to be subjective.
Going back to the article, the visualization of with vs without abstractions can cover aggregating the mathematical representation of the code and how to tackle complexity. Abstractions lets you take a group of nodes and consider them as a single node, allowing you to build super-graphs covering the underlying structure of each part of the program.
> both syntactically and semantically
I do want to cover semantic program complexity at some point as a deeper discussion. I find that side to me to be quite interesting. How to measure it too.
> Just taking the notion of "complexity". Reducing that to _just_ cognitive load seems to be a very poor analysis, when simple/complex ought to deal with the "size" of a structure, not how easy it is to understand.
Preface: I'm likely nitpicking here; the use of "_just_" is enough for me to mostly agree with your take.
Isn't the idea that the bulk of complexity IS in the understanding of how a system works, both how it should work and how it does work? We could take the Quake Fast Inverse Square Root code, which is simple in "size" but quite complex on how it actually achieves its outcome. I'd argue it requires comments, tests, and/or clarifications to make sense of what its actually doing.
How do we measure that complexity? No idea :) But I like to believe that's why the book takes a philosophical approach to the discussion.
I agree the arguments in the book largely "make sense" to me but I found myself finding it a little hand-wavey on it actually proving its points without concrete examples. I don't recall there being any metrics or measurements on improvement either, making it a philosophical discussion to me and not a scientific exercise.
Definitely agree with this, I think ClickHouse can do a lot with joins if you don't implement them naively. Keeping the server up-to-date is a part of it too.
They've made strides in the last year or two to implement more join algorithms, and re-order your joins automatically (including whats on the "left" and "right" of the join, relating to performance of the algorithm).
Their release notes cover a lot of the highlights, and they have dedicated documentation regarding joins[1]. But we've made improvements by an order-of-magnitude before by just reordering our joins to align with how ClickHouse processes them.
100% this. I have a set of commonly-used code in a repository we use at work. AuthN/AuthZ, code specific to our infrastructure/architecture, common http middlewares, error types, DB wrapper, API clients, OpenAPI Server generation, etc.
However, my personal projects have a different set of code I reuse (more CLI- and tool-heavy focus), and I'm sure other environments would have an entirely different set of reused code.
On the opinionated library side of things, I did follow LiveBud for a while, and Go-Micro but haven't really sat well with the experiences from those, given how they lock you in to different aspects of your stack.
We've used the s3fs-fuse library for a while at work for SFTP/FTP server alternatives (AWS wants you to pay $150+/server/month last I checked!) and it's worked like a dream. We scripted the setup of new users via a simple bash script and the S3 CloudWatch events for file uploads is a dream. Its been pretty seamless and hasn't caused many headaches.
We've had to perform occasional maintenance but its operated for years with no major issues. 99% are solved with a server restart + a startup script to auto-re-mount s3fs-fuse in all the appropriate places.
Same experience for me too. I learned a lot, and integrating OAuth into a platform was pretty painless, including importing and managing users.
It was the first time I felt like I can wrangle the OAuth spec and implementation, and really understand all the ways in which things can go wrong without proper care and expertise into how an OAuth server can be attacked, and how to mitigate those issues the right way.
I haven't thought about it that way! I've not hit the GC as a performance wall when it comes to accessing nullable DB values yet. Thanks for the insight.
I think the idea is `sql.NullString` can be used to have a SQL NULL still be an empty string in Go (just avoid checking the `Valid` field, or cast to string -- no check necessary).
It seems like the intent of a string pointer vs. `sql.NullString` is their goal with this type anyways[1]?
In practice I've used a string pointer when I need a nullable value, or enforce `NOT NULL` in the DB if we can't/don't want to handle null values. Use what works for you, and keep the DB and code's expectations in-sync.
I totally agree with this too! Sometimes its nice to have a project that's meant to go nowhere, but it teaches you a ton and you get a lot of that yak shaving out of the way.
By the end you might have so little to show, but I found writing down what I learned a great way to "mark" the project with a healthy milestone :)
Comfort in the unknown helps a lot with this -- creativity is also a skill that can be trained and tested like muscles.
I usually get stuck in "cycles" where I know I'm not making progress, but I'm exploring my interests and seeing what sticks and what doesn't.
I've found forcing myself to a bare minimum has helped. The age-old "write one article a month" type of thing has forced me to find that sense of purpose to sit down and write something. Sometimes it turns out well, sometimes it doesn't. The process and lessons learned have been more important than the (immediate) results.
My one tweak would be that in projects in development, the hard rules are less applicable. Sometimes I've found hard-coding some things (e.g. FIXME items) are to make it into production when developing a prototype and testing (< 1.0.0).
Once you're reaching 1.0.0, you know these can no longer be sliding through, and you can support all of this in CI.
Totally agree. I've found that adding a timeline to _remove_ TODOs/DREAMS (whatever the team choice is) from code as a CI check is very applicable -- what you thought might be good at some point in the past may (and likely is) not applicable later down the road!
Clean the code, comment actualities later, etc. -- it helps with knowledge share to find ways to make you attend to cleaning things up.
I wish I had thought of that example too when I wrote about idempotency!
I wrote a post about idempotence as well a couple months back (https://chrisprijic.com/idempotency-is-key/) and use the definition: "its the output + side-effects that should remain the same, regardless of how many times the function with the same input is run".
I had struggled with "on the same input" when I was putting the post together until I thought about how it usually is due to side-effects that idempotency becomes desirable.
I wish it were :)
This is the right way to do it. Whether using trunk-based development, git-flow, etc -- you're controlling the flow of merges in a particular direction.
However, I think the "larger planning issue" is harder to easily avoid when you have more devs, more changes, or the AI-boosted output we have today. If feature B requires feature A, and feature A isn't up-to-date with main, I could rebase feature A to main, then feature B to feature A. When feature A is merged and we're ready to PR feature B, I can rebase to main again then make my PR.