Cool tour.
I haven't kept up with their developments; what kind of workloads have they been pushing for? Since they don't seem to have any specialized accelerators in the Compute Sled, I am assuming they are not targeting AI workloads for now?
This is super annoying and imo, really limits the usefulness of this model.
It speaks volumes about what Anthropic's position as a company and its priorities will be going forward.
I doubt this kind of gatekeeping will prevent open-models or other innovation outside Anthropic to slow down.
I would imagine these guardrails, if needed at all, should be done at a legal framework level and students should not be a part of this blanket approach to limiting the usage of these models.
I broadly agree with the premise. As a PhD student in Computer Science, I feel there are some significant upsides to my work routine. LLM access has made many new domains more "accessible" to me which I otherwise would be too hesitant in investing my time in.
For example, my area of research is computer systems which involves operating systems, distributed systems and more recently systems for AI. Within these, there is a wide breadth of topics/techniques one can employ and up until now, I have not gone deep into theoretical aspects of things like scheduling etc. But with access to LLMs, I feel like I can at least brainstorm from a high-level about these sub-areas that I am not well-versed in and the responses give me some relevant pieces to start exploring on my own, depending on what interests me more or the amount of time I want to spend on that sub-branch of a larger tree of ideas.
However, the one thing I do have skepticism is the lack of awareness of blind-spots when dabbling into areas that I am not an expert in, and taking the LLM's lead in applying such techniques to some systems problems that I am working on. I often feel that I am not aware of what alternatives exist that the LLM has not explored for me, or if the directions it has proposed really do apply or have corner cases/assumptions that break in what I am doing. On the other hand, when working on something I have good intuitions about, I am often correcting the model's assumptions and it back-tracks what it told me. Unfortunately, I cannot do that comfortably with topics I don't have good intuition about which limits my confidence in "if this is the right direction to pursue."
Cool post. Have you looked at slicing a single GPU up for multiple VMs? Is there anything other than MIG that you have come across to partition SMs and memory bandwidth within a single GPU?
Right now it seems legal to scrape Reddit. But given their trajectory of making the API fairly expensive to use, do you think it's likely that they would also limit/prohibit scraping (assuming apps like Apollo start scraping as an alternative)?
Even if that is the case, it does not say much about their ability to evaluate what category of users (e.g., those coming from Apollo or their first-party clients) is generating more views/interactions and indirectly more "value" on their platform.
There are a few cases where `dlmopen` has issues, for example, some libraries are written with the assumption that there will only be one of them in the process (their use of globals/thread local variables etc.) which may result in conflicts across namespaces.
Specifically, `libpthread` has one such issue [1] where `pthread_key_create` will create duplicate keys in separate namespaces. But these keys are later used to index into `THREAD_SELF->specific_1stblock` which is shared between all namespaces, which can cause all sorts of weird issues.
There is a (relatively old, unmerged) patch to glibc where you can specify some libraries to be shared across namespaces [2].
> You can't load one so/dll multiple times in some sort of container
I believe you can do that with `dlmopen` in separate link maps. I have worked with multiple completely isolated Python interpreters in the same process that do not share a GIL using that approach.
I have also been working on running multiple Python interpreters in the same process by isolating them in different namespaces using `dlmopen` [1].
The objective on a high level is to receive requests for some compute intensive operations from a TCP/HTTP server and dispatch them on to different workers. In this case, a thin C++ shim receives the requests and dispatches them on to one of the Python interpreters in a namespace. This eliminates contention for the GIL amongst the interpreters and can exploit parallelism by running each interpreter on a different set of cores. The data obtained from the request does not need to be copied into the interpreter because everything is in the same address space; similarly the output produced by the Python interpreter is also just passed back without any copies to the server.
This looks like a cool project. Is there any support (or plan to support) I/O through kernel bypass technologies like RDMA? For example, the client reads the objects using 1-sided reads from the server given it knows which address the object lives in. This could be really benefitial for reducing latency and CPU load.
Here is something that has helped me a lot. It's a paper on how to read academic papers. You can extract the general idea and apply to many other technical reading material.
Reading this beautifully written piece brings to mind the recent book about Elizabeth Holmes - Bad Blood. Some of the completely unacceptable behavior by tech CEOs and "higher ups" definitely rings a bell; some startups like Theranos were doing exactly that and ended up getting a lot of negative attention.