It's great to see this post I wrote years ago still being useful for people.
I agree with many here that the situation is arguably worse in many ways. However, along similar lines, I've been pleased to see a move away from cargo culting microservices (another topic I addressed in a separate post on that site).
To all those helping companies and teams improve performance, keep it up! There is hope!
Great! We (technologists) should be using machine learning approaches to practical real world problems, particularly those problems whose solutions allow humans to be more productive. That's what I mean by augmenting human intelligence/capabilities.
I gave a lecture at the University of Toronto last year with a similar point to what the speaker was getting at. Namely, we should be focusing on augmenting human intelligence, not chasing some sort of future robot overlords. See https://adamdrake.com/novel-results-considered-harmful.html
This was the approach Douglas Englebart took in the lead up to the Mother of All Demos (MOAD) and it's unfortunate people have forgotten that bit of history.
There is so much that can be done to improve the lives of people if we simply focus on how to optimize a variety of things that have to happen every day. This doesn't mean we should forsake longer-term goals like fully autonomous vehicles or other things, but it does mean that some of those who are most capable of making a very real and significant impact in the lives of thousands of people are forsaking that opportunity.
There is also often the argument that such near-term thinking will lead us to a local optimum in terms of technology advancement, but I don't see that there has been significant evidence to support that claim. After all, Englebart's demo of version control and collaborative editing (among other things) was over 50 years ago and that didn't seem to stall technological progress.
Most of the time, I've found a push to microservices within an organization to be due to some combination of:
1) Business is pressuring tech teams to deliver faster, and they cannot, so they blame current system (derogatory name: monolith) and present microservices as solution. Note, this is the same tired argument from years ago when people would refer to legacy systems/legacy code as the reason for not being able to deliver.
2) Inexperienced developers proposing microservices because they think it sounds much more fun than working on the system as it is currently designed.
3) Technical people trying to avoid addressing the lack of communication and leadership in the organization by implementing technical solutions. This is common in the case where tech teams end up trying to "do microservices" as a way to reduce merge conflicts or other such difficulties that are ultimately a problem of human interaction and lack of leadership. Technology does not solve these problems.
4) Inexperienced developers not understanding the immense costs of coordination/operations/administration that come along with a microservices architecture.
5) Some people read about microservices on the engineering blog of one of the major tech companies, and those people are unaware that such blogs are a recruiting tool of said company. Many (most?) of those posts are specifically designed to be interesting and present the company as doing groundbreaking stuff in order to increase inbound applicant demand and fill seats. Those posts should not be construed as architectural advice or _best practices_.
In the end, it's absolutely the case that a movement to microservices is something that should be evolutionary, and in direct need to technical requirements. For nearly every company out there, a horizontally-scaled monolith will be much simpler to maintain and extend than some web of services, each of which can be horizontally scaled on their own.
I also wrote https://adamdrake.com/enough-with-the-microservices.html as a way to communicate some of this, including some thoughts on when and how to structure a codebase (monolith) and when it might make sense to start moving towards microservices, etc. There are cases where it's reasonable (even advisable) to move towards microservices, but they are rare.
I typically see people try to push microservices as a "best practice" for improving the delivery rate of software teams. They carve the codebase into separate services, but still leave all of the persistence in place so that all services are communicating with the same DB(s). The result is a tangled web of interdependent services, plus additional tooling like Docker, Kubernetes, etc. that actually makes the teams even slower than they were before.
I'm glad to see us as a community getting more pragmatic about these topics, and realizing that _best practices_ are highly context dependent!
One thing I haven't seen in the comments, but have observed personally is the accounting implications and reasoning for these failed SAP projects.
Sometimes the project cannot be stopped immediately during development, even if it is already known that the project is a failure. The reason is that some companies capitalize the cost of the development over many years, and if they were to cancel the project they would have to book the entire amount as an expense in a given year, instead of depreciating it over multiple years.
I know of one company that worked on an SAP implementation for about 5 years, when it was only supposed to take 6 months or so, and the reason was that if they stopped the project they would have to book the entire cost as an expense in the current financial year. The exec team, who had financially-based bonuses, weren't keen on that.
Instead the project was put together to the extent possible so that it could be declared usable, and then promptly shelved. However, it was "completed" so in theory the sunk cost could have been spread out over multiple years.
I use it from time to time, but I certainly wouldn't consider myself an expert. When I need to do something beyond my current knowledge I search around a bit until I find the awk syntax I need to get the job done. Sorry I can't be more helpful!
I'm not of the opinion that it is dangerous as an idea, but rather that the intentional deception I see on the AI topic is dangerous (not to mention unethical).
I can't think of a single example of an "AI" company I talked with who did not know that they were explicitly exaggerating their claims and capabilities for the express purpose of attracting funding and fueling the hype fire.
The solid tech startups I've seen are focused on the problem they are solving for customers rather than the tooling (AI).
The main thesis is that there seem to be more and more companies out there solving interesting problems, which by itself is great, but they're bolting a lot of wording on top talking about AI. Most of this seems to be an attempt to differentiate themselves in the market and access funding and I find it incredibly dishonest.
The whole discussion around AI these days has become so tainted by scheisters trying to attract funding and attention that I actively try to distance myself from association.
Douglas Engelbart and others had this figured out 50 years ago (see: the Mother of All Demos). The AI hype is dangerous, since it will no doubt lead to the trough of disillusionment.
I can definitely understand the feeling of wasted time, and perhaps I didn't do a good job of explaining the point.
The main issue is that you shouldn't discuss your current salary, but you should absolutely discuss the budget for the role. The reasons I make the distinction is because many candidates will have a lower current salary, and although the company has budgeted X for the role they will be happy to pay (X-delta) instead.
Talking about your current salary is basically never a good idea, but talking about the budget and salary for the role is certainly a good idea.
1. If you're in the first 10 years or so of your tech career, you should be changing jobs every 18 to 24 months. These are the primary years in which your salary goes up, and to stay in a role for a longer period of time without significant (e.g. 20%) raises could have a dramatic effect on your lifetime earnings.
2. NEVER discuss your current salary with a recruiter or potential employer. It's currently illegal in some jurisdictions to be asked about this, but overall it's still very common. See more details and negotiating strategies in an article I wrote on this topic: https://adamdrake.com/what-is-your-current-salary.html
3. Be prepared to transition into a role where you are responsible for people and output, to some extent. This makes you more valuable to companies because you are able to increase output from a group of people instead of from just yourself. Think of it like scaling your brain. The transition from Teammate to Team Lead can be difficult, and after advising multiple companies and teams on the topic I wrote up some of my thoughts: https://adamdrake.com/teammate-to-team-lead.html
4. Maintain technical skills and make sure you're always delivering valuable technical results, or providing guidance to people so that they can do the same: https://adamdrake.com/always-be-shipping.html. People and companies will almost always value you as a member of the organization based on what you've provided lately, so you have to keep producing. Once you've been hungry, you'll never feel full. Stay hungry.
Lastly, I think it's pretty clear to all of us that this could be avoided if companies took care of people, gave raises preemptively and as close to market rates as possible, and gave people an opportunity to advance in an environment with a high degree of loyalty. Unfortunately, that's almost never the reality.
When I moved it to AWS Lambda there was some size introduced from TLS and also from some headers that AWS API Gateway adds which seem, unfortunately, impossible to easily drop (e.g., if you make the headers empty in the response, APIGW will copy them into another header and still add its own).
I might pull that service off of AWS APIGW/Lambda, in which case the size as an artifact of AGIGW would disappear again.
In short, when all startups are complaining that people don't show initiative/don't have alignment/etc. what they're really saying is that they have inadequate Command and Control!
These approaches and processes for keeping people working together with unity of purpose towards a series of objectives have been refined over thousands of years and under the most difficult circumstances. Ignore them at your (and your team's) peril.
I did something similar with https://tinysite.adamdrake.com where I have a website that fits into a single ipv4 datagram (at least it did when I was using my own HTTP server).
True. The FEIE does cut that down a bit, but anything over the FEIE limits would still be subject to tax in the US, though not in the host country, as you mentioned. I was simply making the point that having a bona fide tax residence in a country with a lower tax rate than the US does not necessarily mean that taxes will simply be those of the host country with lower tax rates (e.g. Dubai, Singapore, etc.).
Not sure if you're a US citizen/green card holder or not, but it might be worth mentioning that if so all of your worldwide income is subject to reporting and taxation. If your tax liability in your bona fide tax residence is less than what you would pay in the US, then you owe money to the US.
AFAIK, for US citizens and green card holders, there aren't any income tax advantages to working in a place like Dubai.
It's great to see this post I wrote years ago still being useful for people.
I agree with many here that the situation is arguably worse in many ways. However, along similar lines, I've been pleased to see a move away from cargo culting microservices (another topic I addressed in a separate post on that site).
To all those helping companies and teams improve performance, keep it up! There is hope!