Author here. Want to share the results of my experiments with local inference. I failed to get decent output from my previous setup (https://news.ycombinator.com/item?id=48587865). But this stack is performing much better.
The local stack only became reliable after fixing multiple separate failures in the stack, fixing the coding harness, improving system prompt and other tweaks in different components.
Concrete repo evidence in the post is from Clawrium(https://github.com/ric03uec/clawrium) PRs #855, #859, and #861. Will continue to iterate on this till I can offload about 30% of the project work completely to the local models.
Author here. Quick context the post doesn't quite spell out:
The tool_choice="auto" failure on Qwen3-Next isn't a parser issue — the model
reasons inside <think>, decides, and never emits the tool call. No error, just
empty tool_calls. The fix was swapping the backbone from Thinking to Instruct,
not tuning any parser flag.
The "load the bigger model first, size the smaller against actual residency"
playbook generalizes to anything with shared CUDA framework overhead. The ~5 GiB
framework floor shows up even at small gpu_memory_utilization values — plan
against actuals, not targets.
Building Clawrium (https://github.com/ric03uec/clawrium), a CLI to orchestrate a fleet of autonomous agents(like openclaw, hermes etc) on multiple physical hosts.
As I started running agents on my home network I needed an easy way to maintain them without sshing every every time. I built clawrium as a way to centralize config management, secrets, templates and llm providers. Think of it like a control plane similar to kubectl.
I have 2 OpenClaws running on an old Ubuntu box, and one on a PI. Every config change meant SSHing into each one, tweaking files, restarting services, checking logs. Its all fun during experimentation but now I'm using these for some real work and don't want to spend time troubleshooting config drifts.
So I've been building Clawrium (clm) - a CLI that treats your machines as a fleet and your agents as the things you deploy onto them. Point it at a host, deploy an agent, and you get one command center instead of three terminal tabs.
What works today:
- `clm` CLI to configure instances, LLM providers, IP addresses etc
- End-to-end install + lifecycle management for OpenClaw (Zeroclaw in progress)
- Basic TUI to list all the running agent instances in one place
- Normalized config format; Clawrium translates to Openclaw native format but I intend to add support for all *Claws
What's coming: ZeroClaw, Hermes and other claw support, token-usage guardrails across the fleet, config import/export, backups and other goodies. I'm the solo dev on this so I'm building the features based on my immediate needs.
It's built on Python + Typer + ansible-runner.
Still early and rough around the edges. Would love feedback from anyone running agents across multiple machines, especially on what the CLI ergonomics should feel like.
I'm running a few instances of OpenClaw(and Zeroclaw). I use it for a few things
1. Experimenting with local models. OpenClaw uses local inference on a DGX Spark machine. Helps me understand how different, newer models work and behave
2. Bot 1 helps me maintain a personal project. It sits in discord channel and I throw ideas at it, ask it to log issues, give me priorities, update docs etc. Basically maintenance work to manage the repo
3. Bot 2 is general purpose assistant to take notes(while driving, on a walk, etc), give me daily AI news summary from trusted sources and my relevance criteria. Give me suggestions on what should I dig deeper
4. Bot 3 is setup for work environment. Sits in slack as my team's assistant. Process meeting notes automaticlaly from google and send us summary for the week/month, important notes from a project, answer questions for rest of the org and more. This is most "real" use case so far for me
<plug>
I build https://github.com/ric03uec/clawrium as a lightweight orchestration system to manage agents on a local network because i was having a hard time keeping tabs on these agents. all the ssh-ing and config file gymnastics was too much to handle.
</plug>
I'm a heavy cc user for writing code, reviewing documentation, brainstorming, updating jira tickets etc etc.
For the past few months, I started experimenting with managing a team using cc. As a team, we got together and decided to experiment with a new way to run the team and now that we're looking at some good results, I wanted to share our learnings here
> The problem with the "everyone" model being pitched here is that it may as well be a synonym for "nobody."
Can't agree with this enough!
Thanks for your inputs. A lot of it resonates with what I've observed which translates to the fact that this is as much a cultural/people problem as much it is a technical problem. If teams took ownership by just building visibility, then it'd be an easier problem to solve.
You bring up a good point of doing canary deployments for solving this problem. I'll check this out.
But its interesting that you say ".. if it is a substantial risk in your domain". Isn't this a problem that most engineering teams are struggling with, especially in last few years? Being part of a few DevOps meetups in my area(Seattle) for a while and having attended a bunch of conferences in last couple of year, I've noticed cost coming up as one of the most recurring discussion topics.
Just curious why cloud costs wont be a risk in any domain.
I agree that building visibility makes accountability easier. Its relatively trivial to build observability for individual services and we have achieved some version of it.
The problem is when 30 odd microservices (each team owning between 5-10) talking to each other. In pre-production setup, changes in few of these services might not have noticeable impact on cost which will become quite apparent in production. When this happens, we definitely notice an increase the cost and the unit metric. But then we dont know where to start fixing this problem from. Right now, this becomes a war-room situation based on the severity but I dont think this is sustainable.
In comparison, if we take API latency as a metric, the accountability and ownership are clearly defined: If an API slows down, the team that owns it, fixes it. They can work with anyone they need to but its their job to fix it.
Did you face similar concerns/issues? Not sure if this is a problem other engineering teams are struggling with or even considering as a real problem to invest it.
I'm also not sure if theres a "standard" way of doing this which we should be thinking about. So, looking for ideas and thoughts here.
Already done! This actually works pretty well when two conditions are true
1. cost becomes an engineering-org wide pain point
2. there are a bunch of low hanging fruit (another way to say you're just starting off your cost improvement journey)
Since cost is not really a "deliverable" for non-platform teams, this incentive doesnt go far. Especially after a few iterations of this are done, saving 1K is hard.
We did a short program(similar to a bug bash) for a couple of sprints to dig up all the improvements we can do to reduce to cost. This was early in our cost reduction journey. This did help us get a huge list of what we can do and we picked the most impactful items from this.
Standards enforcement like tagging, TF structure, pipelines etc is currently owned by the platform team. We also have mechanisms to figure out the cost change (approximately) with each Infra PR.
The struggle is to attribute cost changes in application-only changes and to identify them early in the lifecycle. These would be PRs for microservices that add features, fix bugs and handle tech debt.
Esper | Multiple Positions | Seattle, US | Full Time | Visa Sponsorship
We're industry’s first DevOps SaaS platform designed to provide a simple, safe and secure way for engineering and DevOps teams to release applications and manage fleets of smart Android devices in production. Our platform enables developer, mid-market orgs, and enterprise fleets of 100,000+ devices to deliver their software as a service.
Read more about our recent round of funding on Techcrunch: https://techcrunch.com/2021/05/20/esper-raises-30m-series-b-...
We take pride in being an inclusive, transparent, growth focused and customer obsessed team. We encourage learning from our (and others) mistakes, taking bold risks and challenge each other. Read more about our culture from our team: https://www.builtinseattle.com/2021/06/29/seattle-companies-...
We're hiring for following positions in our Seattle office
Our stack: Python/Django, Reacht/TS, Java, Android and Go
If you're passionate about solving challenging technical problems and working in a great startup culture, check out our other open positions at: https://jobs.lever.co/esper-3
I admit that the touchpad isn't as smooth and gesture driven as the mac but I'm not the user those are really meant for. I spend most of my time on the keyboard writing code and living in the "cloud" which makes using mouse a drag anyway. So all I'm looking for is a powerful machine that doesn't hog resources on useless stuff, with sturdy keyboard(physical keys please, none of that touch bar BS), can support dual/multi boot and works with all the peripherals I need. Thinkpads work better than most without the significant price overhead.
One of my dystopian future scenarios has no Thinkpad's in it :D
Have worked on Mac's, XPS's and HP's in the past but once I got on Thinkpad, never looked back. My current system(for past 5 years now) is a trusty X1 Carbon that has seen some rough times but has never failed me. Thinking of upgrading later this year to a newer version of the same line.
I wanted to have a working cluster from scratch which meant not using saltstack and any of kubernetes helper scripts. So I wrote a bash script to do the same here : http://blog.shippable.com/multi-node-kubernetes-cluster
At Shippable(shippable.com) we've been using docker for over an year now for the following use cases:
1. deploying all our internal components like db, message queue, middleware and frontend using containers and using a custom service discovery manager. The containerization has helped us easily deploy components separately, quickly set up dev environments, test production bugs more realistically and obviously, scale up very quickly.
2. running all the builds in custom user containers. This helps us ensure security and data isolation.
We did run into a bunch of issues till docker was "production-ready" but the use case was strong enough for us to go ahead with it
yes, adding that lets you see all remote branches, including PR's. The goal with this tool is to eliminate editing .git/config for each project and make it easier to work with pr's locally. This is just the first step. Definitely planning to add more pr-specific feature.
Do open an issue if there's any particular feature you would like to see.
Thanks :)
root access is currently only available for 'apt-get' command.
apt-get update/add-key/install will work just fine.
We're working on enabling full sudo support which will be available in coming weeks.
hi,
I'm working as a tech lead in Shippable(https://shippable.com). The production use as specified by Docker is a set of guidelines that you should/must avoid for now, like not giving the sudo access inside the containers. Also, if the application running inside of the container itself has security holes than Docker really can't help in any way. You should take a look at this presentation by Jérôme Petazzoni (http://www.slideshare.net/jpetazzo/linux-containers-lxc-dock...) which makes the same point(and more).
The local stack only became reliable after fixing multiple separate failures in the stack, fixing the coding harness, improving system prompt and other tweaks in different components.
Concrete repo evidence in the post is from Clawrium(https://github.com/ric03uec/clawrium) PRs #855, #859, and #861. Will continue to iterate on this till I can offload about 30% of the project work completely to the local models.