You’re absolutely right about the organizational dynamics.
Many mangers in large companies, derive their status and power from a knowledge of existing business processes and procedures. Any substantive changes to those procedures obviously represent an existential threat to that position and they generally resist it, often very vigorously.
Kent’s original book is a masterpiece. He pretty much defined data modelling nearly fifty years ago. It’s depressing how little of real interest has been added since.
For the reasons you rightly pointed out, the current edition is a disaster.
You certainly won’t hit it with most corporate OLAP processing, which is nearly all read-only SQlite. Writes are generally batched and processed outside ‘normal’ business hours, where the limitations of SQlite writing are irrelevant.
No. The issue is whether you NEED to not whether you want to.
10 to 15 years ago, you could argue, however implausibly, that hardware constraints meant vertical scaling was impossible, and you were forced to adopt a distributed architecture. Subsequent improvement in hardware performance, means that in 2025, vertical scaling is perfect acceptable in nearly all areas, relegating distributed architecture to the most niche and marginal applications. The type of applications that the vast majority of businesses will never encounter.
I would go even further and argue that vast majority of businesses will never need to think about distributed systems. Modern hardware makes them irrelevant to all but the most niche of applications.
It’s slightly more nuanced than that. Investment banks and consultancy companies are really interested in graduates who are smart and articulate. The nature of their degree is not that relevant.
I know an extremely clever young woman who graduated in performance arts. She had no difficulty in getting a job at a top tier consultancy company. This company was far more interested in her than some mediocre plodder who hacked his way through a comp. Sci degree.
The vast majority of jobs on LinkedIn are fake. Even so your figures confirm my claim; 24k graduates versus 15k jobs. The supply is greater than the demand.
As a sibling comment has pointed out, there are too many software engineers in the UK.
The vast majority of businesses will never need more than single node architecture. Hardware advances are continually increasing that percentage.
SPARK and its modern counterpart Databricks are essentially obsolete for these organizations. Whatever justification they may have had in the past is no longer true.
I’ve recently closed down several in house SPARK clusters and replaced them with single nodes.
In addition to the simplicity of the design and reduction in cost there was a massive increase in performance. I expect this will become more common in the future; leaving distributed architecture for a small and increasingly niche group.
The market can remain depressed for longer than you can remain solvent.
We should be encouraging people to look at alternative careers to tech. Life after tech.
We should also be making it clear to students that while there are exciting things happening in tech this is not going to translate into large scale demand for people.
Large parts of technology are mature, indeed moribund. This is not a message that the technology industry wants to hear.