Basically, the problem is that the Web browser doesn't give enough control to developers to be used as an application platform. Simple things like key bindings are impossible for example (because there are some key bindings that the browser will not let you modify).
Another problem is integration with other desktop applications and system utilities. The browser intentionally removes and restricts this functionality because the browser's model of the users is that users are incompetent bumbling idiots who must be protected from bad choices at the cost of being able to perform useful actions if necessary. Emacs has some safeguards of that sort, but they are intentionally made easy to disable (eg. loading directory-local code).
Emacs itself serves as a communication interface for other programs and uses operating system interfaces to the same effect. It's trivial to run a program from Emacs and feed its output to another program, while displaying the result in Emacs. In the browser, you'd have to arrange for a custom protocol, a server that listens to requests from that protocol, and write wrappers around every utility you want to run so that they can wrap their outputs into messages of that protocol... essentially re-implementing IPC system abilities in browser (but a lot less efficiently).
> The JS code won't be able to modify the browser itself, but neither can ELisp modify the actual Emacs binary code.
You misunderstood the problem. Remember when Firefox' GUI was made of XUL and that was indeed accessible from JavaScript (before they abandoned add-ons in favor of Google's extensions). Then you had add-ons like Firebug or KeySnail that could indeed take control of the browser's "chrome" (i.e. the visual elements that don't belong to the HTML page, s.a. tabs, menus, address bar, buttons around it).
If you don't remember... life was a lot better back then. I've built a tool for interactively highlighting selected DOM elements in the page from a dedicated toolbar for example (this was before the developer tools). You could completely remap all browser keys. You could call external editor to edit the text in text areas or text inputs. You could call programs on the local filesystem (there used to be an FTP add-on that did that).
I didn't mean to say it was illegal, I meant to say it was an advantage in competition against the open-source counterparts because those usually cannot afford to make discounts of any kind.
More concretely, Microsoft used to claim that the price of their software is compensated for by the operational costs (which is false, but they managed to convince a lot of people).
In the same way, due to having deep pockets, Microsoft can, temporarily, operate at a loss only to out-survive the competition only to crank up the price to the ceiling once the competition is gone.
I'm sure this is a serious problem, but I'm not sure it's the main problem. The other one, that might be an even bigger problem is the way Microsoft sells its products. While, technically, most of the stuff they sell appears as B2C products, they are sold as B2B, and increasingly more so since the emphasis shifted towards the Web versions.
What this means for the users is that their interests are overpowered by the interests of the organization supplying them with the software. An organization can be more easily bribed both "legally" by offering a (temporary) discount and straight-up illegally by paying the procurement officer. Microsoft is known to do both. This is not a battle one can win on technological merit alone, unfortunately...
To comment on my concrete (and quite miserable) situation, here's how it went (and does, even worse with the advent of AI, to this day):
* A report comes from the field (to the support team) that something is broken (or highly desired by the customer).
* The support team assigns the ticket to the lead engineer.
* The lead engineer writes the code and after a quick battle with CI pushes it all the way to the customer in need.
* Since it's the lead engineer, he has the authority to merge PRs w/o consulting anyone, which is what he does. You just pull the changes and marvel at them, or at the sunrise, whichever you like best. The "urgency" is used as a justification to skip the due process.
Now, when AI came into play, two horrible things were added into the mix:
* AI audit and code review. The audit is a 50/50 chance between finding a real problem vs misunderstanding of the purpose of the code or the context in which it is executed. The AI will also come up with a solution that is either completely wrong, unnecessary or touching too many things for a human to track. Code review acts in a similar way, except, technically it fills the role of a human reviewing the code, so it gives a sense of false security to anyone making changes.
* AI generating code that is hard for humans to follow. First and most obvious problem is the volume. In minutes AI generates changes that will take months of effort to review. But this is only the beginning of the problem. When reading code written by humans, another human builds a model of the author's approach, ability, intention and permissions. When it comes to AI, these... let's call them "dimensions" are all over the place. The generated code you read may seem very plausible for a while until you stumble into a crucial functionality that completely invalidates everything you've learned about the change so far. The change may make unnecessary segues into the territory it was never meant to touch.
So, even when there are code reviews, they now miss a lot more than they used to when only humans were writing code.
Also, and this is a pathologically bad, but a very common practice: the reviewer's name is missing from the record. You run git-blame and only see the author, not the reviewer. When things go down, the author takes the heat and the reviewer is nowhere to be found. Subsequently, reviewers don't feel like they need to care as much about the outcomes.
Oh, and one more thing. To be effective at reviewing anything you'd want to try things... as in to test them. Perhaps come up with some idea about what things must happen and what things absolutely must not. But where do you find those? Usually... in a design document, s.a. a PRD, but in the case where that design (allegedly) was solely in the head of the person writing the code, how do you even know if the behavior you conjecture the code has (because there are no tests) is the desired one?
Well, I had to be more specific instead of trying to be dramatic...
The thing is, nothing in the suggested procedure addresses the problem of how you are supposed to distinguish the decisions that matter from those that don't. Whoever came up with this idea must've thought that it's trivial to the point it doesn't need solving, but it's the opposite, unfortunately.
Also, FYO, linters aren't responsible for code formatting. It's the formatter's role. Linters advise on style. I.e. the problem of bracket placement is not solved by using linters, an example of problem that is solved by linters is the maximum number of properties a class is allowed to have.
But the meetings about formatting or using / disabling linter rules still happen. Having tools to do that didn't make the problem go away. What did happen, however, is something more like natural selection. The increased volume of newcomers which the field saw up until maybe five or so years ago were mostly steered in a particular "winning" direction. So, today, naturally, you have most people agree on a set of rules to follow because the majority came from the same background. You need a bunch of old-cadgers to work together on a project for there to be a feud over styling rules.
This is what usually happens if you let incompetent people make design decision: they stay with the first and worst version forever.
Right now I'm in the process of leaving a company where this happened over and over again. The company's product and design aspects were always neglected in this specific way: whenever a problem was discovered its solution was assigned to whoever would write the code to solve it. That's it. The QA would be notified a week or two before the release and after the solution was "finished". The QA would then not be allowed to even comment on the overall design, only the superficial bug reports were allowed to go through.
Needless to say the product's code-base is a dumpster. The morale is low because it's a daunting task to deal with this dumpster on a daily basis. Now, even if you wanted to, you wouldn't have a budget to fix the previous design mistakes because of the layers of more mistakes that were added on top of them due to the low morale and lack of budget / lack of procedure for making better design decisions. The only thing that keeps the product afloat is its uniqueness in a rather niche field and the backing of a large company that acquired it, but doesn't really depend on it and has no time for a thorough audit.
Parent doesn't have experience of working with codebases with slightly high than average code quality requirements.
In products s.a. storage, avionics, medical appliances etc. it's very typical to have a requirement for each commit to compile and to apply tests retroactively. I.e. once a test is added against an existing feature, it is run against every commit since the feature creation (this is also why git-bisect exists).
However, it's true that a lot of companies would probably do better with just rsync instead of Git. Their Git history is in such a bad state that it's basically useless. It just doesn't make sense to use such a complicated tool as Git to deal with the average workflow.
In terms of UI for Git, I've never seen anything getting remotely close to Magit. It's both in-depth and a simplified stateful interface.
Here's what I mean by "stateful": one of the huge problems with command-line tools is that their output becomes mostly inaccessible upon subsequent invocations. Suppose you ran git-log, and now you want to apply the knowledge gained from reading its output to your next action (say, you want to cherry-pick a commit). But you are lucky if the hash of the commit in question is still somewhere in the terminal. Otherwise... you aren't writing that from memory...
Stateful UI to command-line tools allows multiple interactions in parallel and preservation of information obtained in previous interactions.
Git needs a lot of state. You need to remember facts s.a. what branch you are on, what branch did you switch from, what files have been modified, what commits haven't been pushed etc. The requirement to remember all of this is overwhelming and makes the effective use of the tool difficult. This is where various tools come to help (eg. Shell plug-ins that provide auto-completion or modify the prompt to incorporate some of that state info).
I believe that people complaining about difficulties of using Git are often handicapped in their use of surrounding tools that would otherwise mitigate the problems above: they don't know how to add Shell plug-ins that display the current branch, they don't know how to have multiple tabs/panes open in a single terminal session, how to copy text between those tabs/panes, they don't know how to invoke / search auto-complete options.
Genuinely, if you don't have tools to help you use Git, something that people who are used to navigating the world of terminal UI, using Git becomes very unpleasant.
It might be uncalled for, but let me use a philosophical reference: you are trying to derive ought from is. Many tried to prove it to be possible, but most face such propositions with skepticism.
Let me give you some examples of things that were absolutely detrimental to the business of a large company that vanished in a matter of years if not months:
* Adobe Flash. Flash players was for a few years the most installed program on Earth. It vanished without a trace, dropped like a hot potato by everyone who swore they will use it for ever and ever.
* ASP Classic, especially in combination with VBScript. It was the hottest thing twenty (or am I getting too old?) years ago. Everyone was selling books and courses on ASP Classic, probably half the Internet was written in it... and it's gone.
* I expect x86 ISA to eventually die and be completely replaced by ARM or maybe something else. Some big players are already jumping off the bandwagon, and it looks like things are only going to get worse.
There were, of course, more, but I think these three examples should suffice. If tomorrow Google comes up with a better format, or, for whatever reason, tanks its business, or decides to switch to vertical farming in Arizona... say goodby to Protobuf. They owe you zilch. They never promised you anything. You were allowed to use their tech because it was convenient to them, but once it's not, you will be holding the bag.
This is the difference between a standard and a spec published by the designer. The designer is free to make any changes they see fit, up to and including completely destroying their work. They can pull the run from under you making the users pay for the use, they can make modifications to their tools that will require from you to buy things you wouldn't normally want to buy. The possibilities are endless.
It's like in that meme: it's not enough to say it, you need to declare it.
Or, in other words, what makes a standard a standard is that you declare that it is a standard. This declaration carries with it an obligation to respect the standard for ever and ever (like C89), regardless of whether in retrospect you realize you've made mistakes and want to fix them, or maybe wanting to add more stuff etc.
Having a standard is restrictive and uncomfortable for the designers, that's why many opt not to have a standard. Eg. Rust language doesn't have a standard (even though you may, of course, find documents detailing how it works), same for Python, Java and many other popular languages.
I don't know where you get the confidence... When it comes to Protobuf, we are now at version 3 of the format. It's been around for a while, but I'm old enough to have implemented v2 parser myself... v2 is partially supported in v3, even though the support isn't documented. But, emphasis on partially. Some things are no longer there.
So... I'd say that your faith is unfounded. And, in general, there's no reason to believe that a commercial entity will commit to supporting any particular technology if that doesn't generate them a profit. Standard is better.
> Why does it matter for some Python implementation if the Google C++ implementation has a lazy or eager parser?
I wrote about it in my repository, but I'll try to summarize it here: Protobuf is full of bad ideas. One such bad idea is that message fields are allowed to repeat and that the last field wins. So, if you were to write a SAX parser, you'd have a dilemma with how to handle this bizarre idea: do you accept that a callback for some property might be triggered multiple times or do you read the whole message ahead of time and then call callbacks exactly once for each property? If we accept that the parser must be lazy, we "solve" this problem by allowing the parser to read ahead (it needs to do this anyways), but this is a wasteful way to parse (uses more memory than necessary).
> The important part of protobuf is the spec of the wire format. That is what makes the standard an interop format.
I'm not sure what are you trying to say here. All messaging formats are made up of... wire format, that's what they are for. Maybe I'm not seeing it?
> Personally I also prefer code generation over dynamic parsers and generators.
I think you are trying to say that you prefer source code generation over generating supporting definitions at runtime? I wasn't talking about dynamic parser generation (eg. Lark). In my case, the parser was hand-written (using Bison + Flex) and compiled ahead of time, but the Python definitions supporting the Protobuf IML were generated at runtime.
If my guess is true, I'd like to hear your arguments in favor of generating Python source code that translates Protobuf IML into Python. To me it looks like a waste of space on disk... I really can't think of any reason to want that.
I don't know what your metric for reliability is... but I would say that the messaging / serialization format can hardly be blamed for reliability problems s.a. service uptime or service responsiveness etc. Such problems usually arise at a level where details like the choice of messaging format are not important.
Messaging format may affect application performance though. It would affect metrics s.a. throughput or bandwidth.
In a way that is more difficult to measure, a messaging format can affect the number of bugs created by developer using it and indirectly the delivery times. But this can be mitigated by tooling (i.e. Protobuf isn't self-documenting, so if you don't have the schema files, you can't interpret the messages, but you can write a tool that you can feed the schema definitions and then use the tool to interpret the messages).
> It's one of the few things Google _hasn't_ deprecated
You might be unaware of it, but there were Protobuf v1, v2, and now we are at v3. Even though it's not documented, v3 supports most of v2, but not all of it (I think v1 was never used outside of Google itself). Google never properly released Protobuf, so, they can't really deprecate it. Even their formal grammar is full of errors.
Hey. I wrote another Python implementation of Protobuf. (protopy https://gitlab.com/doodles-archive/protopy it was a while ago and haven't touched it since). I'm not saying it's better than whatever this is or that it's any good, I just post it as a proof of sorts that I'm familiar with the problem.
So, without further ado: Protobuf isn't a standard. You can't have a non-standard implementation of something that doesn't have a standard to begin with. In reality, you have Google's implementation for C++ and then everything else. Everything else was, for the most part, not written by Google. And it doesn't always align 100% with the C++ Google's stuff.
Furthermore, C++ implementation has a lot of idiosyncrasies specific to that language that can't be translated one-to-one into other languages, or, in some cases, shouldn't be, even if they could (eg. C++ implementation is all about source code generation because generating runtime entities s.a. classes in C++ is very difficult, while in languages like Python, generating classes at runtime is easy.)
Furthermore, C++ implementation has a specific way of parsing the binary payload (lazy: only the top definitions are parsed, the inner structure of messages is parsed on-demand). But, is this how every parser should behave? What if you want a SAX-like parser?
----
In the hindsight, I just think that Protobuf is not a good format for writing reliable software that aims for decades of usage. We, as in the whole programming world, don't have good formats in general, and whenever we come to the point of having to use some, we either go with an existing popular but crooked or roll our own, probably also crooked. The standard you alluded to would've been great (perhaps a refinement of ASN with more attention to parser implementation, more concrete versions etc.?) But we aren't there yet, and there isn't even a work group to try and address the issue.
I think you are putting the cart before the horse: the code was written intentionally, i.e. someone first had a thought, and then fleshed it out in the code (and maybe had forgotten all about it the minute later).
When LLM generates code, it also has a "thought process" of sorts. But it's very different from the one humans use. An LLM generating code may pretend to have a thought process similar to humans (when asked to elaborate on the reasons the code is this way or the other), but, fundamentally, it's going to be a lie, because the real reasons this LLM created the code in a particular way is very different, and is very hard to grasp for a human, even if they are familiar with the problem (there were some attempts at explaining LLMs "thinking" that would end up looking like heatmaps and other weird things that are useful for debugging them).
Imagine that instead of a stack trace, the program you are debugging spits out only the memory dump with values in registers etc. (the gibberish-looking part of the coredump file). You'd be in a similar situation receiving the "true" explanation from an LLM generating your code.
My very recent story with libvirt and secureboot resulted in blanket disabling of secureboot as part of the preparation for creation of VMs.
The reason: the VM refuses to boot when provided with an ISO (via virtual CDROM) with a meaningless error (permission denied: go figure out what permission and why was it denied and by whom).
Secureboot is meaningless / useless for most people running VMs, be it on own or rented hardware. It takes some pain and extra work to get it to work sometimes, and a huge amount of work to get it to work always. I doubt anyone was dedicated enough to get it to work always. So, I believe you are right. This is extremely unlikely to be a problem for anyone running Linux VMs, and the more VMs they need to run, the less likely it is a problem.
Oh, I've worked in more than a dozen of software companies. When it comes to planning activities and setting goals, I've rarely seen a lot of sense.
I mean, we are in the industry where it used to be a standard practice, not so long ago, to deliver daily reports about one's activities while "planking", or throwing a beach ball to another person doing some silly acrobatics...
It should come as no surprise that there's no rigorous assessment protocol for these kinds of things anywhere. Retrospectively, I will admit, that enormous amount of effort and resources are wasted due to bad planning. But it's still not done.
I can imagine that with the field becoming more competitive, eventually, the industry specialists will come together and try to address the problem, but so far and for so long the resources just kept flowing in, the huge waste wasn't really a problem.
A lot of programmers naively believe that tests prove that the program works... Even though this has been repeated over and over again: tests can only prove that the program isn't broken in some specific way.
When there's a person in the loop, you know that the program was written intentionally. It can still be wrong, but if you were to ask the author why they wrote this or another part of the program, they'd have an explanation ready. When you interact with LLM, it can generate an explanation, but, fundamentally, it doesn't work in the same way the flesh-and-blood programmer does. It doesn't really have an explanation. LLM can be right 99 times out of 100, where a human undertaking the same task might be right only 90 times out of 100, but the inability to find that 1 wrong case is scary.
LLMs and live programmers make mistakes in different ways. You, the tester, can re-trace the thought process of a live programmer and detect errors where your outcomes don't match the outcomes produced by another programmer. You, however, cannot have the same though process as an LLM... that's physically impossible. So, once it's wrong, you are on a wild goose hunt after the error.
Not the author, but as someone who frequently has to answer this question, I'll chip in:
A mistake is a mistake, whether you have a way to reproduce it right now or not. It's pretty much a given that whatever means you have right now to reproduce the problem will evolve and broaden the scope. Also, if you haven't found a way to reproduce the problem, it doesn't mean it doesn't exist: it takes a lot more effort to prove that it's impossible to reproduce than to simply not being able to reproduce the problem.
Another problem is integration with other desktop applications and system utilities. The browser intentionally removes and restricts this functionality because the browser's model of the users is that users are incompetent bumbling idiots who must be protected from bad choices at the cost of being able to perform useful actions if necessary. Emacs has some safeguards of that sort, but they are intentionally made easy to disable (eg. loading directory-local code).
Emacs itself serves as a communication interface for other programs and uses operating system interfaces to the same effect. It's trivial to run a program from Emacs and feed its output to another program, while displaying the result in Emacs. In the browser, you'd have to arrange for a custom protocol, a server that listens to requests from that protocol, and write wrappers around every utility you want to run so that they can wrap their outputs into messages of that protocol... essentially re-implementing IPC system abilities in browser (but a lot less efficiently).