for the internal use at my company we created a tool to partially solve this issue by building a manifest that MAY contain tags and other metadata for agentic resources lookup.
- overrepresentation in high starred buckets
- overrepresentation in rust
- if we trust proxy signal, ~50% of CC repos and growing equally as fast
it is also a harness that gets configured mainly with a toml file, single manifest is not the case but a great chunk of the configuration surfaces is sitting in `config.toml`
theta-spec is a declarative, harness-agnostic configuration standard for AI coding agents. `theta` is its canonical implementation.
We use it internally at `tamarillo.ai` and we are fully aware of its limitations, but we also feel it’s mature enough to share. This is our interpretation of what it would take to allow individuals working across different harnesses (or teams with more complex configurations) to be able to keep things consistent, or share their configurations with others.
Any feedback and contribution to the spec and reference implementation is extremely welcomed. We already support a few of the most common harnesses today.
Also when putting this together there were a lot of assumptions made, and many of those could prove to be wrong, so contributions are welcomed all the way through to the fundamentals.
At tamarillo.ai, we are building, among many other things, tooling to standardize coding harnesses. This is our humble attempt to understand one slice of their usage.
something is happening with `codex`, at tamarillo.ai we did a [little experiment](https://research.tamarillo.ai/coding-harness-inspection/), with 400K repos that have AI harnesses configured and very interesting behavior is observed
- growing fast as fuck
- overepresentation on starred repos (even though stars mean less these days, it is definitely something to look at)
At tamarillo.ai, we are building, among many other things, tooling to standardize coding harnesses. This is our humble attempt to understand one slice of their usage.
Thanks for sharing. I did not know this law existed and had a name. I know nothing about nothing but it appears to be the case that the interpretation of metrics for policies assume implicitly the "shape" of the domain. E.g. in RL for games we see a bunch of outlier behavior for policies just gaming the signal.
There seems to be 2 types
- Specification failure: signal is bad-ish, a completely broken behavior --> local optimal points achieved for policies that phenomenologically do not represent what was expected/desired to cover --> signaling an improvable reward signal definition
- Domain constraint failure: signal is still good and optimization is "legitimate", but you are prompted with the question "do I need to constraint my domain of solutions?"
- finding a bug that reduces time to completion of a game in a speedrun setting would be a new acceptable baseline, because there are no rules to finishing the game earlier
- shooting amphetamines on a 100m run would probably minimize time, but other factors will make people consider disallowing such practices.
This is a project I did not maintain much for the past few months, but recently I migrated many other codebases to be uv managed, and I find this optimal for many reasons. Happy to receive contributions and fix the quite strict requirements I set. That would probably be the faster way.
Thx for the heads up on optimizations available. The “Approximations” comment does not apply to the shortest path calculation, but rather to the distances and upper bound times estimations. This is the consequence of enabling routing for points that dont exist as nodes (closest node approximation).
"This NTAD dataset is a work of the United States government as defined in 17 U.S.C. § 101 and as such are not protected by any U.S. copyrights. This work is available for unrestricted public use."
I based my work on this, maybe the link is out, thx for testing. The dataset has already been consumed and collapsed into a smaller graph representation.
my code is not as good as anything above most probably. Ive done this exploring while studying. No linter no typechecker, grug engineer mentality. But thanks nevertheless for the comment :)
https://theta.tamarillo.ai