Reddit and GitHub are littered with people launching new projects and appear to be way more feature-rich than new tool/app launches from previous years. I think it is a lot harder to get noticed with a new tool/app new because of this increase in volume of launches.
Also weekend hackathon events have completely/drastically changed as an experience in the last 2-3 years (expectations and also feature-set/polish of working code by the end of the weekend).
Was the on-device local LLM stack that you tried llama.cpp or something like MLC? I've seen better performance with MLC than llama.cpp in the past - but it has been probably at a least a year since I tested iphones and androids for local inference
I also want to add on that I really appreciate the benchmarks.
When I was working with RAG llama.cpp through RN early last year I had pretty acceptable tok/sec results up through 7-8b quantized models (on phones like the S24+ and iPhone 15pro). MLC was definitely higher tok/sec but it is really tough to beat the community support and availability in the gguf ecosystem.
I get what you mean about wanting a visual app to experience yourself and be able to point others too. I recently followed this MLX tutorial for making a small model act well for home speaker automation/tool-use that I think could potentially be used to make a good all-in-one demo: https://www.strathweb.com/2025/01/fine-tuning-phi-models-wit... (it was fast and easy to do on a MacBook pro)
Depends on what benchmarks/reports you trust I guess (and how much hardware you have for local models either in-person or in-cloud). https://aider.chat/docs/leaderboards/ has Deepseek v3 scoring higher than most closed LLMs on coding (but it is a huge local model). And https://livebench.ai has QwQ scoring quite high in the reasoning category (and that is relatively easy to run locally but it doesn't score super high in other categories).
My gut feeling is that there may be optimization you can do for faster performance (but I could be wrong since I don't know your setup or requirements). In general on a 4090 running between Q6-Q8 quants my tokens/sec have been similar to what I see on cloud providers (for open/local models). The fastest local configuration I've tested is Exllama/TabbyAPI with speculative-decoding (and quantized cache to be able to fit more context)
I love the idea of openrouter. I hadn't realized until recently though that you don't necessarily know what quantization a certain provider is running. And of course context size can vary widely from provider to provider for the same model. This blog post had great food for thought https://aider.chat/2024/11/21/quantization.html
I experimented with both Exo and llama.cpp in RPC-server mode this week. Using an M3 Max and an M1 Ultra in Exo specifically I was able to get around 13 tok/s on DeepSeek 2.5 236B (using MLX and a 4 bit quant with a very small test prompt - so maybe 140 gigs total of model+cache). It definitely took some trial and error but the Exo community folks were super helpful/responsive with debugging/advice.
The Android apk for MLC is updated frequently with recent models built-in. And a Samsung S24+ can comfortably run 7-8B models at reasonable speeds (10ish tokens/sec).
As an additional data point - I've read Anathem multiple times and have loved it every time but I read The Book of the New Sun series and really disliked it (though to be fair, I only read TBotNS series once). But I know that TBotNS gets great reviews. So as always with books, your mileage may vary.
The CapOne 2% business card is called Spark Cash and doesn't have the redemption restrictions that you're describing (I think you may be thinking of their Venture card or another personal travel card). The only two downsides to the Spark card (in comparison to the Fidelity cashback card or Citi double cash) are that applying for it likely dings you in all 3 major credit reports (as opposed to just one like most cards) and also that it carries a $59 annual fee after the 1st year.
Or somewhere with great solar power potential. For example Kihei on Maui - you can power a whole house with central air and still have enough energy to charge up an EV just with rooftop photovoltaics.
There is a "Dragon Box" on Amazon that sells for $270ish and appears to be a lower-powered Android Kodi box. I imagine that a second generation Pi could perform just as well.
My first generation Pi was significantly slower at Kodi (things like menu traversal, library updates, API consumption, etc) than a second generation Pi (I tried a friend's). So I was already considering $35-40 for a new Pi when I saw the sale price on the Nexus Player. But the bigger deal for me was Google Cast support. I wanted a single device that could run Kodi and also that the family could cast Youtube, Soundcloud, etc using phones.
Regarding remote - the Nexus Player comes with a simple, bluetooth remote (as does the Shield TV now apparently). It works great for simple Kodi interactions but doesn't work for everything (the biggest one that I've noticed so far is the lack of a context button). But as thoughtpalette mentions - I use Android and iOS remote apps when I need more control over Kodi (Yatse is what I use on Android). Also there is an Android TV control app for Android phones that has the same buttons as the simple Nexus Player bluetooth remote.
As ixnu mentioned - I use the Moonlight Android app (http://moonlight-stream.com/) paired with Geforce Experience running on my Windows box (Geforce GTX 750 Ti video card) for Steam games.
To be honest though - my Moonlight streaming experience hasn't been as smooth as using a device that can actually run the Steam client (Linux/Mac/Windows). But Moonlight is still really cool.
I picked up a $50 Nexus Player (Android TV OS) last month for this purpose. It replaced a first generation Raspberry Pi B that was actually pretty decent as a Kodi box (I used the Xbian distro).
So far I'm really happy with the Nexus Player. I like that it gets Android TV apps (like Kodi and Moonlight for game streaming from my desktop) and also supports Google Cast. It also has a usb port that supports things like keyboards, ethernet, external disks, etc with a usb hub.
I also strongly considered the Nvidia Shield TV (also Android TV OS). It's $200 with a lot better hardware, getting good reviews, supports 4k, and apparently can decode H265 in hardware.
I think there are certain stores that routinely report itemised info. I just posted this link in another comment thread but check out the https://www.visa.com/supplierlocator/ search results for the column 'Enhanced Data Level'.
If you search for the Safeway grocery store at 14444 124th Ave NE Kirkland WA 98034 - the data level cell contains 'Level III Line Item'. I believe this means that at least for Visa, that store is using level 3 credit card processing and line item receipts are reported to Visa.
Yeah, I saw that section on Wikipedia also. But I believe I've found a link indicating that level 3 is being used for consumer purchases in at least some individual stores.
If you look through Visa's supplier locator at https://www.visa.com/supplierlocator/ you can search for a business (or category of business) at a particular address or zip code. It will tell you the MCC (category) and appears to also show the data level reported to Visa.
I looked up Safeway grocery stores near Seattle. If you take a look at the one at 14444 124th Ave NE, Kirkland WA 98034 the data level is 'Fleet, Level III Line Item, Level III Summary'. I take this to mean that anyone using a Visa card at that particular Safeway will have their line item receipt sent to Visa. I would be surprised if the other credit card processors don't have setups like this as well.
Also weekend hackathon events have completely/drastically changed as an experience in the last 2-3 years (expectations and also feature-set/polish of working code by the end of the weekend).
And as another example, you see people producing CUDA kernels and MLX ports as an individual (with AI) way more these days (compared to 1-2 years ago), like this: https://huggingface.co/blog/custom-cuda-kernels-agent-skills