AllenNLP was written for research, not for production. Many of the design choices reflect that.
As far as the vocabulary goes, a lot of AllenNLP components are about experimenting with ways to turn text into vectors. Constructing the vocabulary is part of that. When pre-trained transformers became a thing, this wasn't needed anymore. That's part of why we decided to deprecate the library: Very few people experiment with how to construct vocabularies anymore, so we don't want to live with the complexity anymore.
Maybe we need to re-work the docs if the DAG aspects stick out to you so much. The main functionality is the cache. If you have a complex experiment, you can still write the code as if all the steps were fast, and let them be slow only the first time you run it. The DAG stuff is also nice, but less important.
That said, you could execute sklearn. If that's what your experiment needs, it's the right thing to do. This is why it gives us the flexibility to also support Jax: https://github.com/allenai/tango/pull/313
The DL-specific stuff is in the components we supply. Like the trainer, dataset handling stuff, file formats, and increasingly, https://github.com/allenai/catwalk.
It depends on what you use AllenNLP for. AllenNLP has a ton of functionality for vectorizing text. Most of the tokenizer/indexer/embedder stuff is about that. But these days we all use transformers for that, so there isn't much of a need to experiment with ways to vectorize.
If you like the trainer, or the configuration language, or some of the other components you should check out Tango (https://github.com/allenai/tango). One of Tango's origins is the question "What if AllenNLP supported workflow steps other than read -> train -> evaluate?". We noticed that a lot of work in NLP no longer fit that simple pattern, so we needed a new tool that can support more complex experiments.
If you like the metrics, try torchmetrics. Torchmetrics has almost exactly the same API as AllenNLP metrics.
If you like any of the nn components, please get in touch with the Tango team (on GitHub). We recently had some discussion around rescuing a few of those, since there seems to be some excitement.
AllenNLP has only ever supported Torch. At the moment, Tango only supports Torch as well, but Jax support is well underway.
And yeah, Tango is a lot like a build script. In fact, I used to manage my experiments with Makefiles. Tango is better though. Results don't have to be single files, and they don't have to live in one filesystem either, so I can run the GPU-heavy parts of my experiments on one machine, and the CPU-heavy parts on another. The way you version your code is better than what Makefiles can do. You have actual control beyond file modification time. And of course, there is the whole Python integration stuff.
The _only_ factor to programming language popularity is how accessible it is.
Beauty and elegance don't matter. Performance doesn't matter. In fact, no measure of "fitness for purpose" matters.
If Malbolge was built into every operating system, Stack Overflow would plug the gaps, and major production systems would be built in it. People would give conference talks about "scaling Malbolge to millions of transactions per second". It would have a package manager, a deep learning framework, and several UI toolkits.
I used to have one of those. Sure it can run all that hardware, but it sounds like a jet engine when it's running. Completely ridiculous design that would never be released by Apple.
The Microsoft Surface Books also have a 1060 inside, but they are whisper quiet.
USD is backed (among other things) by the government accepting taxes in it. If you don't pay, they will come to your house with guns and take you away. This creates a demand for USD that is completely unlike any demand for crypto.
Out here in the free world, every time you want to use a library, you have to choose between 12 different options, all suck in one way or another, and you really have no way of making an informed choice. So you pick something by gut feel. Half the time you are wrong, and all of the time you worry that you have made the wrong choice.
But if you buy into Apple's cathedral, they tell you what to use, and you use it. End of story. It's always behind the times, and it won't run outside of Apple's stuff, but it works (for now, at least), it's well documented, and you don't have to spend days finding and comparing solutions. You can get back to writing code, which is the fun part.
Is it good business? Good for the world? Maybe not. But it's certainly attractive.
It is absolutely happening in enterprise software. I have met sales guys and startup advisors that prided themselves in being able to play that game well. In a certain segment of this industry, you get laughed at if you try to actually come up with a solution.
I'm getting the impression that all the freaking out is coming from non-techies. People that have been in the industry for a while are just shrugging and saying "Yeah, that's professional software development for you.".