Kaiser is a vertically integrated healthcare provider. They provide insurance like UHC but they also require you, for the most part, to be treated in Kaiser medical facilities.
How can UHC use AI in the same way? They're an insurance company. They're not administering the actual healthcare?
usually the LLM will match against the style in the context, even if the words ask it to do otherwise. since your skill sounds like an LLM wrote it, I would be surprised if it didn't make things worse.
sorta like asking an LLM to get more creative with a visual design, that also never works.
interesting. that might be a fun intro project to MoltenVK. I hadn't dug into what was missing for Doom. I thought maybe the issue was that the intro/menu always ran in opengl mode or something. If it's just one missing op, that's way easier.
it is a separate stack, but that probably doesn't matter much. a user process (in my case, qemu) can communicate with a driverkit driver. the user process can also map memory through the driver, which is how this pci passthrough system works.
i don't think the issues with the project really are specific to driverkit.
i don't know for sure, but i suspect what makes the tinygrad stuff slow isn't the macos host driver itself. i think they're doing something very similar to what i'm doing, which is just mapping the PCI BARs to userspace, then they have a bunch of python code that drives the GPU.
this is only speculation, but i think the big thing that makes tinygrad slow is that the tinygrad inference engine has not really been optimized much for all these open LLM models. probably most of the work has gone towards optimizing the stack for george's self-driving hardware company. since you can't just run the existing CUDA kernels on their engine, that makes things a lot tougher, engineering-wise.
i am actually curious if my project could share a macos host driver with them. i think it would need some changes, but it seems like there's a lot of overlap
in fairness to the LLM critics, every time i ran into a minor speed bump in this project, it told me it probably just wasn't possible to get it to work well. the LLM did pretty actively discourage me from trying to get the whole thing working.
that said, since i was willing to ignore that aspect of it, it did accelerate getting the work done by a lot. it seems like it understands system programming really well, and did a good job navigating the qemu codebase. i have ~20 years of systems programming experience so i already knew what had to be done here. it didn't really guide the project much, but it did write a lot of the code.
1. Virtualization.framework seems to support some form of GPU passthrough from the host (granted, not eGPU - it's for the integrated GPU). I think the primary use case is having macOS guests get acceleration, while still sharing GPU time with the host. There is also a patch that recently hit QEMU mainline that supports using the "venus server" with virtio-gpu to support a similar functionality for Linux guests under Hypervisor.framework.
2. Apple internally has some kind of PCI Passthrough support available in Virtualization.framework. It seems like the code is shipped to customers in the framework, but it relies on some kind of kext or kernel component that isn't shipped in retail macOS. I can't say if that's intended to ever be released to customers, but clearly someone at Apple has thought about this the feature.
I very recently ran the numbers on these GPUs for an upcoming blog post. The token generation performance is bad, but the prefill performance is _really_ bad.
For a Qwen 3.6 35B / 3B MoE, 4-bit quant:
- parsing a 4k prompt on a M4 Macbook Air takes 17 seconds before generating a single token.
- on an M4 Max Mac Studio it's faster at 2.3 seconds
- on an RTX 5090, it's 142ms.
RTX 5090 uses more power than an M4 Max Mac Studio but it's not 16x more power.
i thought my post was already too long to include this, but to your point, you can run AI inference in this setup and the performance can be pretty good.
this is mostly right, but it's not true that you can just sign up at any time. there's an open enrollment period for the aca marketplace and if you miss it, you won't have the opportunity to buy health insurance until next year.
runway construction is only part of the story, i think. in May, there was a number of complete ATC meltdowns causing ground stops. if you look at the stats, the majority of delays at newark in May are attributed to ATC.
granted this was an especially egregious situation and not the norm, but it feels like these types of issues are on the rise based on my anecdotal experience. there were a number of full ground stops at newark due to ATC in the weeks after this. it was national news.
How can UHC use AI in the same way? They're an insurance company. They're not administering the actual healthcare?