Cool! These are indeed very common graph-building steps.
Thinking outloud here, but some of these were supposed to be solved with RML (https://rml.io/) for the RDF paradigm. I witnessed a bit of their evolution: it started with similar operations as GraFlo and eventually they built some support for arbitrary java code. For example, say you want your node ID to be generated by concatenating the values of the firstName column and the lastName column, but only after some weird string normalization (think of making sure everything is utf8)... you woundn't want to make your schema-mappings Turing-complete, so you'd eventually have to allow for calling other functions. Any way, all of that was for RDF graphs, it's cool to see something like this for property graphs.
Somewhere in HN there's a post about disrupting or revolutionizing the laundromat industry, where some person is showered in praise (and later money) for setting up this lousy system.
In countries communicating in non-English languages which are written in the latin script, there is a very large use of Latin-1. Even when Latin-1 is "phased out", there are tons and tons of documents and databases encoded in Latin-1, not to mention millions of ill-configured terminals.
One question, did you try to replicate the other result table (Table 3)?
If I understand correctly, top-2 accuracy would be 1 if you have only 2 classes, but it will differ from "normal" accuracy less and less as the number of classes increases (on average). So this shouldn't change the results for table 3 thaaat much as the datasets have large amounts of classes (see table 1).
In any case, top-2 accuracy of 0.685 for the 20-newsgroups dataset is pretty neat for a method that doesn't even consider characters as characters[1], let alone tokens, n-grams, embeddings and all the nice stuff that those of use working on NLP have been devoting years to.
[1] In my understanding of gzip, it considers only bit sequences, which are not necessarily aligned with words (aka. bytes).
Re modules/libraries: I meant it is not easy to write a piece of SQL code, and then import it into several queries to reuse it, or lend it to someone else for use on their on schema. It is possible, yes, but seldom done, because it is hell. PostgreSQL extensions could be used for this purpose, but developing an extension requires a different set of SQL statements (or luckily, python or c) than those used by the user of the extension, which makes compounding them a bit hard. Not impossible, just hard to maintain,
About your last point, I don't think that was my line of reasoning, but, yes, for the love of what is precious, don't open SQL files as python/java file objects and then parse and rummage through them to find the data you are looking for. Not impossible, just hard to maintain.
Thanks for pointing out pgTAP, didn't know about this.
For some reason, data-science folks haven't yet caught up with ORMs.. I don't know if this is good or bad, but (as the OP shows) they are more used to rows and columns (or graphs) than objects. Maybe that will change one day.
- does not allow for easy and clean importing of modules/libraries
- is not easily to write tests for
- has limited support for a debugger
- lacks a consistent style for such large queries (plus most textbook cover fairly simple stuff) which means it's hard for a developer to start reading someone else's code (more than in other languages)
- clearly indicates in its name that it is a Query language.
Save yourself the trouble and all your collaborators the pain of working with this code in the future, of trying to add new features, of trying to reuse it in another project.
If you want to operate near the data, use PL/Python for PostgreSQL.
Among other things, a LLM can be seen as a store which you query and get results from. A chatbot is cute because it formats output text to look like conversation, and the recent applications are nice because the query (now known as prompt) can be complicated and long, and can influence the format and length of the results.
But the cool stuff is being able to link the relatively small amount of text you input as a query, into many other chunks of texts that are semantically similar (waves hands around like helicopter blades). So, an LLM is a sort of "knowledge" store, that can be used for expanding queries, and search results, to make it more likely that a good result seems similar to the input query.
What do I mean by similar? well, the first iteration of this idea is vector similarity (e.g. https://github.com/facebookresearch/DPR). The second iteration is to store the results into the model itself, so that the search operation is performed by the model itself.
This second iteration will lead, IMHO, to a different sort of search engine. Not one over "all the pages" as, in theory at least, google and the like currently work. Instead, it will be restricted to the "well learnt pages", those which, because of volume of repetition, structure of text, or just availability to the training algorithm, get picked up and encoded into the weights.
To make an analogy, is like asking a human who are the Knights of the Round Table and getting back the usual "Percival, Lanceelot and Galahad", but just because the other thousand knights mentioned in some works are not popular enough for that given human to know them.
This is a different sort of search engine than we are used to, one which might be more useful for many (most?) applications. The biases and dangers of it are things we are only starting to imagine.
1. It is expensive.
2. It moves complexity away from you and onto your providers, so it doesn't really solve the problem, only hides it from you (at a price).
3. The overall cost (energy, person-hours, material) of even the smallest project grows a lot with this approach. Even if you have the money to pay for it, you are wasting a bunch of resources around the world just for an illusion of peace of mind.
4. Most importantly, it will still fail (as all systems eventually do) and then you have no idea where it failed or how to fix it. All you can do is file some support tickets at big-corp support center and watch for updates on their twitter feed.
A lot of people complain here on HN about the sad, over-complicated, state of software-engineering, the need to know more and more concepts and to manage more and more tech "stacks" just to accomplish boring, formerly simple, tasks. One reason for this sad state is the philosophy expressed in the parent comment.
Just FYI, The largest, most-used knowledge graphs in the world (Google and LinkedIn) are not running on RDF4J or any RDF Triplestore, but on their proprietary graph stores, which also use Datalog as a query language.
Application process seems pretty nice. The questions posed for the data science positions suggest a very strong graph-theoretic team is in place. They are looking for top-notch mathematicians, not boot-camp data scientists at all.
Do be aware that in the borg collective FAQs they mention NFTs in an apparently positive note, if that is of any relevance to you.
Let us concede that in the last 20 years, the number of mentions of "I" in his writing has decreased, and there are more mentions of "We".
As much as a cellular automata, and computation in general, fan as I am, and after having read a fair amount of his work... I am still not convinced that Wolfram is not just a rich crack pot.
If you are a tourist in the global south, and you negotiate down the price of some craft/souvenir/whatever you are buying on the side of the road, you will be able to buy an extra cup of coffee back home, but the person there might have to stand in the sun and away from their family for 4 more hours.
Don't let the well-being of others all in the hands of "the market".