> With this design, it’s possible to run native SQL selects on tables with hundreds of thousands to millions of columns, with predictable (sub-second) latency when accessing a subset of columns.
I consulted for a large manufacturing firm building an application to track the logical design of a very complex product.
They modeled the parts as objects. No problem.
I was stunned to see the following pattern throughout the code base:
Class of the object
Instance #1 of the class
Instances 2,,n of the class
I politely asked why this pattern existed.
The answer was "it's always been that way."
I tracked down the Mechanical Engineer (PhD) who designed the logical parts model. He desk was, in fact, 100 feet away from mine.
I asked him what he intended, regarding the model. He responded "Blueprint, casting mold, and manufactured parts." - which I understood immediately, having studied engineering myself.
After telling him about the misunderstanding of his model by the software team, I asked him what he was going to do about it. He responded "Nothing."
I went back to the software team to explain the misunderstanding and the solution (i.e. blueprint => metaclass, casting mold => class, and manufactured parts => instances). The uniform response was "It is too late to change it now."
The result is a broken model that was wrong for more than a decade and may still be deployed. The cost of the associated technical debt is a function of 50+ team members having to delineate instance #1 from instances 2,,n for over a decade.
N.B. Most of the software team has a BS (or higher) in computer science.
P.S. Years later, I won't go anywhere near the manufactured product.
"English as a programming language" has neither well-defined syntax nor well-defined semantics.
There should be no expectation of a "correct" translation to any programming language.
N.B. Formal languages for specifying requirements and specifications have been in existence for decades and are rarely used.
From what I've observed, people creating software are reluctant to or incapable of producing [natural language] requirements and specifications that are rigorous & precise enough to be translated into correctly working software.
Software Engineering is only about 60 years old - i.e. the term has existed.
At the point in the history of civil engineering, they didn't even know what a right angle was.
Civil engineers were able to provide much utility before the underlying theory was available. I do wonder about the safety of structures at the time.
Regarding Aro (new C compiler in the Zig toochain), it seems that a C compiler written in Zig shortens considerably the path to supporting constexpr in C using Zig's comptime capabilities.
This discussion of graph search is the best thing I've seen in quite some time. Discussing and highlighting the difference between related graph search techniques made getting to A* very intuitive.
To your point, profile your data as you would your code.
A sorted array of bit locations would represent a sparse bit set well enough to start, with O(N) storage and O(log N) access. Once the sets became large and/or dense, another data structure could be considered.
What are the support costs once the software is shipped?