I'm on a similar journey. I took flight 6 weeks ago and built a turn based board engine for human/agent delivery teams called Keel https://www.spoke.sh/keel. The grand vision is to apply the board engine as a control mechanism for work to be done and verified in deployed robot fleets.
I built an agent observability & orchestration toolkit over the holidays https://github.com/run-vibes/vibes. It uses Iggy as the message streaming layer and currently supports PTY streaming for Claude Code to the browser, Cloudflare tunnel, automatic Claude Code hook instrumentation into Iggy and I'm using that to build a cross harness continual learning system that I'm hoping will act as error correction for the model environment. It's using Iggy server v0.6 (io_uring on Linux) and compiles to native platform binaries for Linux & Mac (Windows support is on the list...)
We're doing Mainframe modernizations/integrations with the cloud @Mechanical Orchard. It's been really fun to see and learn the entomology of how computing got to where it is today.
As other folks have described programming on the mainframe is about learning COBOL + JCL. I would also add that you need to understand how to read/write binary files according to a defined schema (Copybook) and encoding (EBCDIC, COMP, COMP-3, COMP-5). It's also helpful to deeply understand the intricacies of Db2 and different methods of connection (ODBC, JDBC, Db2 CLI).
FWIW I came to a similar conclusion that a lot of the power in Kafka comes from the API and that eventually much of the complexity of managing the cluster will eventually be abstracted away with multiple implementations. I also felt that if I could implement Kafka persistent over the S3 keyspace then I could start with persistence direct to S3 like you've done with warpstream and then layer on a faster hot disk and in memory tiering mechanism to eventually lower end to end latencies.
I love where you're going with this so hit me up on twitter if you ever want to chat more in-depth https://twitter.com/rupurt.
I've created an ODBC DuckDB extension to query any database that has an ODBC driver. It's modeled after the fantastic official Postgres scanner extension https://github.com/duckdblabs/postgres_scanner.
It supports fetching rowsets in batches to minimize network overhead and defaults to the default DuckDB vector size of 2048.
I've tested it against the IBM DB2 & Postgres ODBC drivers and will continue to test and add support for all major databases. If you've got one you'd like to see let me know in the comments.
I've got plenty of improvements in the pipeline including:
- Write to ODBC database
- Predicate push down
- Automatic catalog discovery
- Performance benchmarks
- Configurable batch size
- Relative & absolute cursors
- OS X builds using rosetta for databases without native aarch64 drivers (IBM/Oracle)
I've found a lot of value in using Zig as my C & C++ toolchain. I really like that my build logic is more procedural, which I find easier to comprehend.
Happy LiveView user here :) Can confirm it can be used in many scenarios. One of my favorites is streaming realtime data to a chart. I serialize my data as JSON in the LiveView template and read it back in from a Hook. Saves me time by not having to create a separate API endpoint. e.g.
Why doesn't he get more involved in the project and stop complaining. Yes Ember is lacking in the prioritizing department but it's a community project and there is always room for more people.
There are a number of forks/patches out there that provide hasOne semantics and saving for multiple records in the same commit (ours are the mhelabs ones)
We have high-res PhotoMaps of the major Australian cities and allow you to see change over time by flicking through the dates of places we have captured.
I'm on a similar journey. I took flight 6 weeks ago and built a turn based board engine for human/agent delivery teams called Keel https://www.spoke.sh/keel. The grand vision is to apply the board engine as a control mechanism for work to be done and verified in deployed robot fleets.