A maxim I’ve arrived at working w llm every day is “put deterministic things in code, non deterministic in llm”. I do wish the harnesses would be more helpful in this regard. For example I’ve seen tons of scheduled jobs that people wrote in Claude/copilot/etc that could easily have been scripts. They aren’t scripts because the author doesn’t know how to script and they stop at the point the llm does what they want. It isn’t hard to then tell the llm “make a script to do all of the tool calls this prompt needs and then pass the result back to the llm for this non-deterministic part”.
What is the “middlest” Mac one could get for this? I’m in the market but keep going back and forth between a 64gb m5 pro or “lower end m5 air and screw it I’ll just pay for cloud tokens”. At current prices the 2-3k diff to try to run something local that isn’t as powerful could buy a lot of tokens.
Yup. I’ve been evaluating several on openrouter and find token cost meaningless for my work. I haven’t found a great alternative, though the “cost per task” he uses makes some sense.
My thought process is that I don’t mind a slower model if it can work in background for me 24/7 fleshing out side gig ideas I have floating around but no time to focus on myself. I take your point though, and it’s why I haven’t bit the bullet yet. I could buy a lot of tokens for 5k. If I could make that effective then the roi of offline should be something I can calculate fairly easily.
I’ve been considering a move to local llm setup, having been underwhelmed coat vs value of various online offerings. But at the same time worried anything I get will be obsolete in a couple months. And I don’t want to have to babysit it. I really want some agents managing and creating side hustles for me and have some other things. I’m technical-have written my own harness and use gh copilot and grok daily and have a hosted openwebui+openrouter thing. I’m also torn between a 128g MacBook Pro or a framework, or spark or similar and lightweight laptop to access. Would love advice anyone has for (or against) going local. I have asked ai but have analysis paralysis as 5k would be a big investment for me so I want to make right choices
I have thought that extremely accessible, portable, non-radiation imaging would be revolutionary. Imagine every doctor - or even every person - had a handheld thing they could wave over your affected area to get a high res 3d image of the issue. Of course we have mri, xray, and ultrasound, but those are big and expensive. Obviously there are engineering reasons that is the case, but It seems like a concerted effort to make imaging more portable and accessible, coupled with ai to help analyze could bring about all sorts of follow-on health improvements. Your regular practitioner could check you for heart blockages, clots that lead to stroke, cancers etc as a matter of course. I’m not sure stepping into a golden vat of water is what I have in mind but medical imaging does seem like a possible area for drastic innovation.
I tried this but it took over my mic and people couldnt hear me on teams call until I turned it off. Nice idea but needs to share the mic w teams to be useful for me. Not sure if it’s teams fault or trace fault but either way…
Maybe dumb question: One of the use cases is instrument reading of analog instruments. My brain immediately goes to "this should have some sensor sending data, and not be analog". Is having a robot dog read analog sensors really a better fit in some cases?
This is my life goal right now. I have a bajillion ideas, know how to code them (even faster now), and just not enough time due to day job. A few questions:
How do you market them?
Is customer support an issue?
Do you see risk since ai makes it so easy to build/copy?
I get that it's annoying, but also don't know what else one would do? "FooPilot is our Office AI toolset, BarWonk is our code assist tool"? There are also a lot of Claudes and GPTs. Naming things is hard.
Git is a distributed source control system. It's open source and you can use it to version source code on your drive and/or a remote git repository.
Github is one of the most popular git repository hosts. In addition to source repositories, it has other services like issue tracking and wikis.
A while back, Microsoft bought Github.
"Github Copilot" is a service you can buy (with limited free sku) from Github that adds AI capabilities to your Github subscription.
One of the ways you can use Github Copilot is by using the GitHub Copilot extension for VSCode. This extension lets you use chat inside VSCode in such a way that it can read and write code. It lets you pick which LLM model you want to use: Claude Sonnet, Opus, OpenAI GPT, etc., from the ones they support.
Note you don't need another subscription if you only use Github Copilot. They pay Anthropic, you pay Github. You _might_ want another subscription directly with Anthropic if, say, you want to use Claude Code instead.
"VSCode Copilot" isn't a thing. Some people might call Github Copilot extension for VSCode "VSCode Copilot".
Github MCP server lets AI tools like GitHub Copilot extension for VSCode, Claude Code, or any tool that supports MCP use your Github account to do things like pull requests, read issues, etc. Just using it from Claude Code would not use Github Copilot tokens, UNLESS you used it to work against your Github Copilot service. You would not need a Github Copilot subscription to use it for example to create a pull request or read an issue. But it would use your Github Copilot tokens if, say, you used the MCP from Claude Code to assign a task to Github Copilot. It uses githubcopilot domain because they built it mostly for Github Copilot to use, though MCP is an open standard so it can be used from any MCP-supporting AI tool.
I’ve often thought this would be a good business for college age kids or teens. My parents and other I know are hesitant to send off to a 3rd party online. I think if someone local came and either did mobile or took and brought back they would trust that over even mailin services.
Similar. I regularly use Github copilot (with claude models sometimes) and it works amazingly. But I see some who struggle with them. I have sort of learned to talk to it, understand what it is generating, and routinely use to generate fixes, whole features, etc. much much faster than I could before.
I am a developer turned (reluctantly) into management. I still keep my hands in code and work w team on a handful of projects. We use GitHub copilot on a daily basis and it has become a great tool that has improved our speed and quality. I have 20+ years experience and see it as just another tool in the toolbox. Maybe I’m naive but I don’t feel threatened by it.
At least at my company the problem is the business hasn’t caught up. We can code faster but our stakeholders can’t decide what they want us to build faster. Or test faster or grasp new modalities llms make possible.
That’s where I want to go next: not just speeding up and increasing code quality but improving business analytics and reducing the amount of meetings I have to be in to get business problems understood and solved.
Similar story. I’m a bit younger, but Amiga BASIC/VB3/VB6/ASP/.NET was my path. There was a joy when “Visual Studio” meant “you can visually drag a component on and that is the app” instead of editing text files. But gradually we learned you need to be in the code. Sure you have figmas and low code tools today. But industry has gravitated back to editing curly brackets and markup in text files. And for good reasons I think.
I landed on GitHub Copilot. I now manage a team, but just last night snuck away to code some features. I find my experience and knowing how to review the output helps me adopt and know how much to prompt the agent for. Is software development changing? Absolutely. But it always has been. These tools help me get back to that first freedom I felt when I dragged a control onto a VB6 designer, but keep the benefits of code in text files. I can focus on feature, pay attention to UX detail, and pivot without taking hours.
I was just thinking about this today. Apples lack of any 3rd party integration for things like this and iMessage is really annoying sometimes. In addition to a secondary backup, I’d love to automatically sync some photos from a certain album to my parents photo frame. Or if I take a nice nature shot have it sync to a Samsung frame tv. I get the benefits of the walled garden but esp w photos and messaging it seems like opening up a little would allow for some innovation
I have used this in a “beta” feature for an enterprise app and really like it. In ~100 lines of code I have a secured OpenAI compatible endpoint that I can chat with, and write tools for in .NET. I have it doing natural language query over some data and it works quite well.
You can also expose the agents as MCP, AGUI and so it can be a tool you integrate with other AI platforms.
Kinda get what he’s saying: provide more metadata with structured logging as opposed to lots of string only logs. Ok, modern logging frameworks steer you towards that anyway. But as a counterpoint: often it can be hard to safely enrich logging like that. In the example they include subscription age, user info, etc. More than once I’ve seen logging code lookup metadata or assume it existed, only to cause perf issues or outright errors as expected data didn’t exist. Similar with sampling, it can be frustrating when the thing you need gets sampled out. In the end “it depends” on scenario, but I still find myself not logging enough or else logging too much
Fascinating llm use case I never really thought about til now. I’d love to converse with different eras and also do gap analysis with present time - what modern advances could have come earlier, happened differently etc.