I think the big miss is that the former is achievable far more often than people imagine, which leads to settling for the later, aka pessimization. Reducing your allocations from millions per run to handfuls per run during initialization is effectively a "go fast" button, and is generally more reliable. It is very common for teams to spend big effort getting a 2-3x speedup on allocation-heavy code by disabling branches when a 100-1000x speedup can be had if restructuring allocations is a strategy under consideration (even before the extra 4-8x you might see if you go all the way to hand-tuned SIMD).
Disapprove and ask for a call where the author must verbally explain the changes to receive approval? This seems like a solvable problem, and one that already existed in repos with many contributors of varying skill (open source, bigco with lots of interns). Letting bugs through is an even bigger time sink.
You can setup most PR systems to squash on merge using the first commit's message, then enforce that the top commit message is prefixed with a ticket ID. This practice has many benefits: readable git log, easier git bisect for tracking down regressions, it becomes easy to find all commits associated with a block of work, more useful git blame.
> If you have a non-insignificant amount of data points to track this is going to eat just a ton of memory while also being pretty slow to encode/decode.
This is a fair critique, however, for any large GeoJSON, the coordinate arrays will dominate the size. I think it's also safe to assume this data will be gzipped at rest and over the wire, which will eliminate most of the "header" metadata size you mention. As you point out, it would be much more efficient to have a binary format, and there are good examples like these, that are ~2-3x smaller in benchmarks:
That said, I think GeoJSON should be compared against other human readable formats like KML, which has a lot of wasted space as well, while being more difficult to read/write.
IDK, it seems like old reddit did just fine without even trying that hard. The DOM and surrounding JS API is already a high level GUI framework, and the post illustrates that it's perfectly capable of doing useful interactions >60fps. I personally love working with native code, but the modern browser is capable of producing fast interfaces that saturate human senses without it. If you write JS like a C programmer would, it will usually be quite fast, even if it's not optimal. If you write native apps in C++ like a modern JS programmer - frameworks and deps with abandon - it will be a stuttery mess.
When the DOM is not enough, there's already WebGL and WASM. A vanishingly small sliver of use cases can't saturate human senses with these tools, and the slowest, jankiest websites tend to be the least deserving of them (ie: why is jira slow? It's literally a text box with a handful of buttons on the side!).
I've used duckdb a lot at this scale, and I would not expect something like this to take more than a few seconds, if that. The only slow duckdb queries I have encountered either involve complex joins or glob across many files.
I designed some parts for an enclosure over the weekend using claude opus to generate an OpenSCAD file, which I then exported to STL and sent to a 3D printer. I was able to get a visually-correct-enough STL off to the 3D printer in about 5 minutes.
I then waited about an hour for the print to finish, only to discover I wanted to make some adjustments. While I was able to iterate a couple times, I quickly realized that there were certain structures that were difficult to describe precisely enough without serious time spent on wording and deciding what to specify. It got harder as the complexity of the object grew, since one constraint affects another.
In the end, I switched to FreeCAD and did it by hand.
- notepad++: 56.4 MB (went gray-window unresponsive for 10 seconds when opening the explorer)
- notepad.exe: 54.3 MB
- emacs: 15.2 MB
- vim: 5.5MB
I would argue that notepad++ is not really comparable to VSCode, and that VSCode is closer to an IDE, especially given the context of this thread. TUIs are not offering a similar GUI app experience, but vim serves as a nice baseline.
I think that when people dump on electron, they are picturing an alternative implementation like win32 or Qt that offers a similar UI-driven experience. I'm using this benchmark, because its the most common critique I read with respect to electron when these are suggested.
It is obviously possible to beat a browser-wrapper with a native implementation. I'm simply observing that this doesn't actually happen in a typical modern C++ GUI app, where the dependency bloat and memory management is often even worse.
> If you want electron app that doesn't lag terribly
My experience with VS Code is that it has no perceptible lag, except maybe 500ms on startup. I don't doubt people experience this, but I think it comes down to which extensions you enable, and many people enable lots of heavy language extensions of questionable quality. I also use Visual Studio for Windows builds on C++ projects, and it is pretty jank by comparison, both in terms of UI design and resource usage.
I just opened up a relatively small project (my blog repo, which has 175 MB of static content) in both editors and here's the cold start memory usage without opening any files:
- Visual Studio Code: 589.4 MB
- Visual Studio 2022: 732.6 MB
update:
I see a lot of love for Jetbrains in this thread, so I also tried the same test in Android Studio: 1.69 GB!
I frequently use a docker-compose template with prometheus pushgateway + grafana for deploying on single node servers, as described at the start of the article. It works well and is trivial to setup, but the complexity explodes once your metric volume or cardinality requires more scale like prometheus alternatives a la mimir.
I think this would not need to be an issue as frequently if prometheus had a more efficient publish/scraping mechanism. iirc there was once a protobuf metric format that was dropped, and now there is just the text format. While it wouldn't handle billions of unique labels like mimir, a compact binary metric format could certainly allow for millions at reasonable resolution instead of wasting all that scale potential on repeated name strings. I should be able to push or expose a bulk blob all at once with ordered labels or at least raw int keys.
"Everything" is another that puts the default search to shame. I've also seen people who just have a script that pumps all new files into a txt file every so often and runs bruteforce ripgrep on it, which gives instant interactive results. It's really hard to imagine coming up with a search routine that is as slow and unreliable as what ships with mainstream OS file managers.
I hear this a lot, but can you really say that you're consistently saturating a 1Gbps line for netcode or 6+ Gbps nvme for disk data? In my experience this doesn't really happen with code that isn't intentionally designed to minimize unnecessary work.
A lot of slow parsing tends to get grouped in with io, and this is where python can be most limiting.
There's currently talk of adding gigawatts of data center capacity to the grid just for use cases where python dominates development. While a lot of that will be compiled into optimized kernels on CPU or GPU, it only takes a little bit of 1000x slower code to add up to a significant chunk of processing time at training or inference time.
It shouldn't take any noticable power/cycles to accomplish this task. Having flags for "performance" littered through the codebase and UI is a classic failure mode that leads to a janky slow base performance. "Do always and inhibit when not needed".
> However those tools do not have the polish that ESRI kit does, but at leas you’re not paying the licensing!
Arguably they have more polish, but less GUI. The open GIS world is more CLI, database, and library driven, which can be an advantage for many users trying to build high reliability or scaling systems.
> so now the main problem is building the hardware, there are a lot of solutions for the software part.
While cool and all, this type of sim is a tiny, tiny slice of the software stack, and not the most difficult by a long shot. For one, you need software to control the actual hardware, that runs on said hardware's specific CPU(s) stack AND in sim (making an off the shelf sim a lot less useful). Orbital/newtonian physics are not trivial to implement, but they are relatively simple compared to the software that handles integration with physical components, telemetry, command, alerting, path optimization, etc. etc. The phrase "reality has a surprising amount of detail" applies here - it takes a lot of software to model complex hardware correctly, and even more to control it safely.
As a point of comparison, I just downloaded the Windows binary for duckdb (which provides a nice TUI for similar tasks) and it was 9.84MB. People can and should expect better.
Now instead of seeing millions of lines of inscrutable code in your program bloating binary sizes, you can see it in every program (that doesn't disable stdlib).