The products we're building are more complex than they used to be, but I don't know if its true that backend development has inherently become more complex than it was in the past.
I've worked on everything from dinosaurs older than me to new greenfield projects using all the latest tech. I think over-complexity emerges in many ways, notably: too much abstraction, ignorance, dogma, and just straight up carelessness.
I think the abstraction problem is the most common and severe though. In enterprise software, I've spent an entire day in the past combing through several layers of services and repositories just to get the full picture of how a response object is being created for a single controller method.
Let me give another example: think about the great debate over ORM vs No-ORM. It's fundamentally a battle about the balances and costs of abstraction in software development. On one hand, you have a (hopefully) simple interface for defining a db's schema, but do I really want to obfuscate the developer from the implementation details of that `findBy` method and trust that the ORM will be 1) used correctly and 2) generate a performant, sensible query? What about security handling? Should I have my developers concerned about preventing potential SQL injections, or just let the ORM take care of that too? The answer depends on a lot of things.
Anyway, I believe for every abstraction:
- something significantly useful should be added (this seems like a truism as I write this, but i've seen a startling amount of useless abstractions)
- the cost should be carefully calculated
- understand where complexity is being added and subtracted
I believe thinking about abstraction in this way would've cut down a lot of the complexity I've seen in softwares past.
Vendor lock-in can be a real issue as you scale. I work at a company going through a hypergrowth phase, and we're more or less locked into Azure, which has been objectively awful for us
It's been a year since I touched an AWS lambda, but I'd bet money that cold starts are still an issue. There is a common hack that half works: have your function run every minute (you can use eventbridge rules for this); in the function handler, the first thing you should evaluate is whether or not it's a warming event or not, and exit 0 if it is. Your results may vary (mine did lol)
I came here to say this. Literally every company ever does this. In marketing it's called a value-first proposition. I don't see it as evil by any means
If you like the functional aspects of Java, check out kotlin. It's quickly become my favorite programming languages because you get all the benefits of java's strongly typed OOP system with a lot of elegant functional features, and type inference!!
Honestly, autopilot software should have checks and balances already in place to prevent use if the system is compromised. Regardless if it was compromised by human hands or general use / wear & tear
I used to take a similar view on the war on drugs- now it's more nuanced given my immersion into the problem. It's hard to say wasting countless lives and millions of dollars to combat organizations illegaly smuggling harmless product (marijuana) into the states is hardly worth the mess left behind. Take a step back and it's easier to justify when the income of that product is going to large criminal organizations that also create problems such as the meth epidemic in california or the opioid crisis across the country. And to facilitate the delivery of these criminalized products, criminal organization have no hesitation of delivering those packs covered in the blood of poorest class of people in the hemisphere (the consumers are dying too, but from consumption, another problem entirely). Lots of people against the war on drugs will say "yeah but if it was decriminalized..." A lot of things would be different if it was decriminalized, but good luck selling that pipe dream to the traditional base of america that has a strong influence across the nation. Not happening anytime soon. In my opinion everything sucks here and I haven't heard a viable solution yet to this very complicated issue
Edit: I do want to say that decriminalization is probably the most hopeful option of them all, I just think it's going to take forever, generations probably. This is really a public health issue in my opinion, it's just not treated as one :/
I've worked on everything from dinosaurs older than me to new greenfield projects using all the latest tech. I think over-complexity emerges in many ways, notably: too much abstraction, ignorance, dogma, and just straight up carelessness.
I think the abstraction problem is the most common and severe though. In enterprise software, I've spent an entire day in the past combing through several layers of services and repositories just to get the full picture of how a response object is being created for a single controller method.
Let me give another example: think about the great debate over ORM vs No-ORM. It's fundamentally a battle about the balances and costs of abstraction in software development. On one hand, you have a (hopefully) simple interface for defining a db's schema, but do I really want to obfuscate the developer from the implementation details of that `findBy` method and trust that the ORM will be 1) used correctly and 2) generate a performant, sensible query? What about security handling? Should I have my developers concerned about preventing potential SQL injections, or just let the ORM take care of that too? The answer depends on a lot of things.
Anyway, I believe for every abstraction:
- something significantly useful should be added (this seems like a truism as I write this, but i've seen a startling amount of useless abstractions)
- the cost should be carefully calculated
- understand where complexity is being added and subtracted
I believe thinking about abstraction in this way would've cut down a lot of the complexity I've seen in softwares past.