Huh. I thought hosting one's own databases was still the norm. Guess I'm just stuck in the past, or don't consume cloud vendor marketing, or something.
Local inference users are all about sampling, but users addicted to commercial inference services are wary of sampling, because they have to pay by the token.
The author speculates that bigger/smarter models interpreting vague directives to utilize general-function tools will outperform more precise and detailed directives to utilize narrow-function tools:
> Granted to use a skill the agent needs to have general purpose access to a computer, but this is the bitter lesson in action. Giving an agent general purpose tools and trusting it to have the ability to use them to accomplish a task might very well be the winning strategy over making specialized tools for every task.
The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence.
Now OpenAI has usurped the term to refer to these inference-guiding .md files.
I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the OpenAI feature. Semantic overload sucks.
How do we deal with this? Start using "competencies" (or similar) in academic papers? Or just resign ourselves to suffering the ambiguity?
Or maybe the OpenAI feature will fall flat and nobody will talk about it at all. That would frankly be the best outcome.
Yes, you will need to employ someone with basic system administration competence. That's a given.
Cloud infra is touted as obviating the need to hire system administrators, but that's a marketing fabrication. Trying to manage infrastructure without the necessary in-house skills is a recipe for disaster, whether it's in the cloud or on-prem.
Senior software engineer with 46 years of experience (since I was 7). LLM inference hasn't been too useful for me for writing code, but it has proven very useful for explaining my coworkers' code to me.
Recently I had Gemma3-27B-it explain every Python script and library in a repo with the command:
$ find -name '*.py' -print -exec /home/ttk/bin/g3 "Explain this code in detail:\n\n`cat {}`" \; | tee explain.txt
There were a few files it couldn't figure out without other files, so I ran a second pass with those, giving it the source files it needed to understand source files that used them. Overall, pretty easy, and highly clarifying.
My shell script for wrapping llama.cpp's llama-cli and Gemma3: http://ciar.org/h/g3
That script references this grammar file which forces llama.cpp to infer only ASCII: http://ciar.org/h/ascii.gbnf
Cost: electricity
I've been meaning to check out Aider and GLM-4, but even if it's all it's cracked up to be, I expect to use it sparingly. Skills which aren't exercised are lost, and I'd like to keep my programming skills sharp.
After having worked on accounting software for a living, which did need to precisely take into account the variable nature of time measurements, I greatly appreciate that Numbat has decided not to open that can of worms.
Using averaged constants like this isn't applicable to all use-cases, but has the virtues of being simple and easy to reason about.
IMO they did exactly the right thing by not trying to shoehorn an endless supply of complexity and pain into their language's type system. Users who do need to deal with date math can use a library.
I tried very hard to like Ruby. "On paper" it looks great, but I had a lot of trouble getting used to the syntax.
I ended up going deep on the D programming language for my "new language". As a statically typed language it's not as expressive as the Python/Ruby/Perl contingent, but it's a lot more expressive than most static languages, and its C-like syntax "clicked" with me in a way Ruby never did.
It also can approach C-like run-time performance and memory footprint at times, which I appreciate. As much as I like developing in Python and Perl, I frequently wish they had better run-time performance and a smaller memory footprint. D gives me that, at the cost of a little expressiveness.
Working with Perl, two things spoiled me for other languages: JSON and DBI/DBD.
In Perl, everything serializes to JSON without fuss. It can be "lossy"; objects without an explicit JSON serialization method and coderefs can't be deserialized, but serializations at least have placeholders for them.
Compare to Python's json module, which doesn't try very hard to serialize things and throws exceptions every time it runs across something new. It's very frustrating to use.
Perl's DBI provides a universal API for all databases, with implementation-specific details in an underlying DBD module (which both provides glue between the DBI abstraction and programmer access to features specific to different database systems).
Compare to Python, where you need to import a different module and use a different API for every different kind of database. As I increasingly use Python for a living, I frequently wish they'd follow Perl's example with this.
> Rust had a lot of effort pored (and still going) into the nostd variant to be used in bare metal applications like microcontrollers and kernels.
D has had "dcompute" since 2017, for native execution on GPUs and other accelerators. People have been writing D for microcontrollers since at least 2011, possibly earlier. D is fairly mature in the systems space.
> I’m skeptical D would take.
Maybe, maybe not. Some languages "click" better than others for individual programmers. It's good to have options.
> it’s the first and only non C language in kernel mainline
Do you foresee it being the only non-C language used in the kernel forever?
> Doesn’t D require a GC to achieve the safety guarantees that Rust provides without one
No. The paper covers this. The authors used D's fat pointers, scope, slices, @safe, and ownership/borrowing features to improve code safety.
It is trivial to turn off D's GC, and the authors did not use it in the project described in their paper.
> and many third-party libraries require this GC?
Yes, a lot of D libraries depend on GC, which means they wouldn't be available for use for kernel development.
The D compiler will error out if D code marked with the @nogc attribute depends on GC (or depends on code dependent on the GC), which makes going GCless easier.
D has been around for 21 years. It has had a compiler in the gcc project (gdc) for four years. It debugs easily enough under gdb. Several editors support D syntax (including geany and vscode). It has quite a few libraries (though its interoperability with C libraries has muted native D library development somewhat), centralized at https://code.dlang.org/
Are there specific parts of the ecosystem you worry about not being mature?
As for it being a bit esoteric, yes, it's not a very widely known language, though I'm not sure why. I fell in love with it in 2018, and it has thoroughly spoiled me for writing C.
Glad my employer is still one of the sane ones.