Also worth checking out https://github.com/saharNooby/rwkv.cpp which is based on Georgi's library and offers support for the RWKV family of models which are Apache-2.0 licensed.
I get around ~140 ms per token running a 13B parameter model on a thinkpad laptop with a 14 core Intel i7-9750 processor. Because it's CPU inference the initial prompt processing takes longer than on GPU so total latency is still higher than I'd like. I'm working on some caching solutions that should make this bareable for things like chat.
This is not true, GPT-3 can perform chain-of-thought reasoning through in-context learning either by one/few-shot examples or zero-shot by starting a prompt with "let's think step by step" (less reliable).
GPT-3.5 (what's being used here) is a little better at zero-shot in-context learning as it's been intstruction fine-tuned so it's only given the general format in the context.
I think you're focusing on a few narrow examples where LLMs are underperforming and generalising about the technology as a whole. This ignores the fact that Microsoft already has a succesful LLM-based product in the market with Github Copilot. It's a real tool (not a party-trick technology) that people actually pay for and use every day.
Search is one application, and it might be crap right now, but for Microsoft it only needs to provide incremental value, for Google it's life or death. Microsoft is still better positioned in both the enterprise (Azure, Office365, Teams) and developer (Github, VSCode) markets.
If you run a Kubernetes cluster for self-hosting software or development I highly recommend setting up a Tailscale subnet router [1]. This will allow you to access any IP (pods or services) in your cluster from any of your Tailscale-connected computers. You can even configure Tailscale DNS to point to the DNS server in your cluster to connect using the service names directly ie. http://my-service.namespace.svc.cluster.local
pikchr is awesome. A project I did recently was a WASM-compiled pikchr library to generate diagrams directly
in the browser [1]. Here's a very early demo of a live editor you can play around with [2].
Not fully-featured yet but what I'd like to eventually do is set it up in a similar way to the mermaidjs editor [3]. They encode the entire diagram in the url. That makes it really easy to link to from markdown documents and has the nice benefit that the diagram is immutable for a given url so you don't need a backend to store anything.
Cool trick, thanks for sharing. I don't get why there isn't a suitable snscanf function that takes the buffer length as an argument and returns the number of bytes parsed?
If you're getting into this stuff a great resource I found is the ZipCPU Tutorial [0] by Dan Gisselquist. The tutorial covers both Verilog design and Formal Verification methods. It uses open source tools like Verilator and SymbiYosys so getting started is pretty easy.
> I'm pretty sure you could also maintain a temp table and use some kind of "insert...where...returning" construct to squeeze that into a recursive query.
I'm not sure if this is possible in SQLite, as far as I know the WITH clause is limited to SELECT statements.
> Are they actually efficient in sqlite even for trees?
Recursive common table expressions work by adding returned rows to a queue and then performing the recursive select statement independently on each row in the queue until it's empty.
You can use WITH RECURSIVE to traverse a tree by adding the root node to the queue and recursively visiting adjacent rows until the queue is empty. This works correctly and quickly because trees have only a single path between nodes. If you try the same query on a DAG though it will return every path to a given node, you then have to perform a GROUP BY to find the shortest path outside of the recursive query. In the worst case, if you have a graph with many paths between nodes, this method is exponentially slower than a standard BFS.
The issue I found with WITH RECURSIVE queries is that they're incredibly inefficient for anything but trees. I've looked around and there doesn't seem to be any way to store a global list of visited nodes. This means that when performing a traversal of the graph the recursive query will follow all paths between two nodes.
I've actually been working on an extension to perform breadth first search queries in SQLite on general graphs [0]. The extension is actually based off of the transitive closure extension. You can use it on any existing SQLite database as long as you can wrangle your edges into a single table (real or virtual) and the node ids are integers (I'm planning on removing this constraint in the future).
My brand new laptop was stolen while I was in school and I had to finish the year with a used 10-inch netbook I borrowed from a friend. Because of the tiny display and underpowered processor I had to get creative, I installed Arch plus i3 and it worked like a charm. In fact, I noticed my productivity actually went up. Eventually I got a new laptop, I switched off Arch to Ubuntu but I've been using i3 ever since.
If you're an i3 user here are a couple cool things to check out:
- scratchpad [1]: Little known i3 feature. Let's you put individual windows in a scratch workspace and then pull them up quickly over your regular workspaces. Really useful docs and things like that, I keep my time tracker in a scratch workspace.
- rofi [2]: This is a dmenu replacement with a nicer interface that lets you combine lists of commands in a single search window. I have it configured so I can search through my open windows by title or run any program from the same interface.
I may be misreading that, but isn't the table saying that 82% of residents with dementia require assitance in their daily living activities?
From having spent time volunteering at a nursing home, that 87% number is not far off from my experience. I'd add that they tended to put volunteers with people who had less severe forms of cognitive impairment.
That being said, you would occasionally meet people who seemed mentally impervious to aging, I had the pleasure to speak with an 87 year old computer programmer who was still actively using all kinds of technology. He talked to me about punch card programming accounting systems on IBM 1401's and dealing with technology skeptical secretaries who would double check every computer calculation by hand.
I'm not sure if you're from outside North America, but I don't think nursing homes here are likely to have LAN parties unless we make some big medical breathroughs in the next few decades.
According to this survey of Canadian nursing homes, around 87% of residents suffer from some kind of cognitive impairment such as dementia, alzheimer's, or stroke related trauma [1].
Even assuming that you still have any friends around by the time your in a nursing home, the odds that neither of you suffer from cognitive impairment is about 1.7% (someone let me know if that math is wrong).
Fun fact, all doctors who plan to winter at an Antarctic station must have their appendix removed beforehand [1]. The reason is that they're usually the only doctor available.
While you can use recursive common table expressions to perform graph queries, certain queries like breadth-first traversal are not very efficient for anything but trees (not even DAGs). See my post on the sqlite forum from the other day for more details
- Human Reading Speed (English): ~250 words per minute
- Human Speaking Speed (English): ~150 words per minute
Should be treated like the Doherty Threshold [1] for generative content.
[1] https://lawsofux.com/doherty-threshold/