it builds as a docker image which has stockfish and maia (maiachess.com) together with different weights so it can simulate lower-level players.
It was a fun exercise, I tried a bunch of local models with this MCP server, which isn't particularly optimized, but also doesn't seem that bad. And the results were quite disappointing, they often would invent chess related reasoning and mess up answering questions, even if you'd expect them to rely on the tools and have true evaluation available.
It was also fun to say things: fetch a random game by username 'X' from lichess, analyze it and find positions which are good puzzles for a player rated N.
and see it figure out the algorithm of tool calls:
- fetch the game
- feed the moves to stockfish
- find moves where evaluation changed sharply
- feed it to maia at strength around N and to stockfish
- if these disagree, it's probably a good puzzle.
I don't think I got to have a working setup like that even with managed cloud models. Various small issues, like timeouts on the MCP calls, general unreliability, etc. Then lost interest and abandoned the idea.
Saying CI is a developer experience thing is completely misunderstanding the point.
AI workflows, vibe coding and such only emphasize the need for proper test suites and for a measured release process, where you don't actually delete your production databases on the whim.
It took me an embarrassingly long time to internalize what the reverse proxy is. My brain got stuck on the fact that it is just proxying requests. What's so reverse about this? Silly.
> "Closed platforms are going to lose to open ones that allow innovation"
Only if people (end-users) actually care. If you're trying to build against the momentum but misjudge your values for the community values, then you're in for a disappointment.
what a strange place to link to. Even Wikipedia has a better entry [1].
The model itself is fun to think about: preventing failures by stuffing more cheese into the system. If you're interested, the classic example of the cheese failure is Chernobyl, where many different things had to fail in order to become a catastrophe.
it's the latter: "you can actually start telling it to write a Playwright Python script instead and run that".
and while running the code might faster, it's unclear whether that approach scales well. Sending an MCP tool command to click the button that says "X", is something a small local LLM can do. Writing complex code after parsing significant amount of HTML (for correct selectors for example) probably needs a managed model.
"spooky action at a distance" sounds bad, but this is how most frameworks that embrace convention over configuration work. You add a dependency to your SpringBoot application and suddenly your app actually has new endpoints and config for them and so on.
If the code generation quality is so good you trust it to generate your verification system. You might as well write the tests yourself, and generate the actual implementation.
This way you're in control of what it means the code works, and formalize the acceptance criteria. Instead of feeding it a potentially incorrect implementation, and asking it to come up with a set of requirements what it should actually do.
Here's a video of a session by Chen Li (mentioned in the post as a collaborator) from the Graal workshop with more details and some background: https://www.youtube.com/watch?v=Hepjf00LJrM
From the description it's an aggregate of the common modern JVM workloads.
The following is the complete list of benchmarks, separated into groups.
actors
akka-uct - Runs the Unbalanced Cobwebbed Tree actor workload in Akka. (default repetitions: 24)
reactors - Runs benchmarks inspired by the Savina microbenchmark workloads in a sequence on Reactors.IO. (default repetitions: 10)
apache-spark
als - Runs the ALS algorithm from the Spark MLlib. (default repetitions: 60)
chi-square - Runs the chi-square test from Spark MLlib. (default repetitions: 60)
dec-tree - Runs the Random Forest algorithm from Spark MLlib. (default repetitions: 40)
gauss-mix - Computes a Gaussian mixture model using expectation-maximization. (default repetitions: 40)
log-regression - Runs the logistic regression workload from the Spark MLlib. (default repetitions: 20)
movie-lens - Recommends movies using the ALS algorithm. (default repetitions: 20)
naive-bayes - Runs the multinomial naive Bayes algorithm from the Spark MLlib. (default repetitions: 30)
page-rank - Runs a number of PageRank iterations, using RDDs. (default repetitions: 20)
core
dummy - A dummy benchmark, which does no work. It is used only to test the harness. (default repetitions: 20)
database
db-shootout - Executes a shootout test using several in-memory databases. (default repetitions: 16)
jdk-concurrent
fj-kmeans - Runs the k-means algorithm using the fork/join framework. (default repetitions: 30)
future-genetic - Runs a genetic algorithm using the Jenetics library and futures. (default repetitions: 50)
jdk-streams
mnemonics - Solves the phone mnemonics problem using JDK streams. (default repetitions: 16)
par-mnemonics - Solves the phone mnemonics problem using parallel JDK streams. (default repetitions: 16)
scrabble - Solves the Scrabble puzzle using JDK Streams. (default repetitions: 50)
neo4j
neo4j-analytics - Executes Neo4J graph queries against a movie database. (default repetitions: 20)
rx
rx-scrabble - Solves the Scrabble puzzle using the Rx streams. (default repetitions: 80)
scala-dotty
dotty - Runs the Dotty compiler on a set of source code files. (default repetitions: 50)
scala-stdlib
scala-kmeans - Runs the K-Means algorithm using Scala collections. (default repetitions: 50)
scala-stm
philosophers - Solves a variant of the dining philosophers problem using ScalaSTM. (default repetitions: 30)
scala-stm-bench7 - Runs the stmbench7 benchmark using ScalaSTM. (default repetitions: 60)
twitter-finagle
finagle-chirper - Simulates a microblogging service using Twitter Finagle. (default repetitions: 90)
finagle-http - Sends many small Finagle HTTP requests to a Finagle HTTP server, and awaits the response. (default repetitions: 12)
The last graph is a bit hard to read with the log scale. It's 10x improvement from GNU-R to FastR+rJava and another 10x with the native GraalVM interop.
you can use profile guided optimisations when building a native image with GraalVM. You'll need to run your code in a special way under the desired load to gather the profile information, then it can be used when building the native image. YMMV, but the results can be quite interesting, here for example someone compiled http4s: https://twitter.com/lukasz_bialy/status/989091065033625606