I've found the local models useful for non-coding tasks, however the 8B parameter models so far have proven lacking enough for coding tasks that I'm waiting another few months for whatever the Moore's law equivalent of LLM power is to catch up. Until then, I'm sticking with Sonnet 3.7.
I guess what I was thinking was: even if you can't quantify it, you can squint at the comparative advantages and trade-offs and make a qualitative, yet objective judgement call.
This is the missing criteria in the technical decision making, IMO. How reliant is the team on the recruiting/retention of the current size and structure of the talent, both on the team and in the wider community?
Small teams trying to keep burn ultra low vs. giant companies might have similar technical goals but opposite staff capabilities. This is a crucial factor.
A second-order effect is how much time/energy/money you have to throw at maintenance. Can you afford to spend X% of your time on maintenance? Which technologies offer comparative advantages on maintenance cost? These are surprisingly often easy to answer, and nearly never explicitly considered!
I recently did a digital declutter, as described in Cal Newport's book Digital Minimalism.
One hobby I picked up much more easily than anticipated: microcontroller programming. With the spare energy and mindspace freed up from scrolling social media, I'm now expanding my skillset and reading far more books.
I'm not claiming to have boosted my skills to an unrealistic degree, but the benefits of giving up the dumb online activity are very real, very tangible, and very valuable IMO.