> MIA Meeting: Low dimensional embeddings of words and documents (and how they might apply to single-cell data)
Leland McInnes
Tutte Institute for Mathematics and Computing
Over the last decade the field of Natural Language Processing (NLP) has been overtaken by neural networks and deep learning. The latest models and algorithms, from word2vec to Google’s Universal Sentence Encoder and BERT, can perform seemingly magical feats and provide powerful tools to understand and analyze text documents.
This talk will seek to pick apart word and document embedding techniques from NLP, removing the neural networks and instead working with linear algebra and dimension reduction on large sparse matrices of counts that is very similar to a large amount of single-cell data. Such an approach can achieve results on par with neural network approaches, but are both simpler to understand the inner workings of, and are generalizable to much more diverse domains than NLP, including, hopefully, single-cell research. In particular all we really need to apply these kinds of techniques are large sparse matrices of counts and some notion of locality or cooccurrence of the feature columns of that matrix. The goal of this talk is to hopefully open up some of the ideas that have been so effective in NLP and make them usable and accessible to a wider audience working with more diverse types of data, and initiate a conversation about what is required to make such techniques useful for the single-cell community.
I switched from pinboard to diigo around 2 years ago. I love the "Annotate Page" functionality. I use it on all my bookmarks to quickly save the parts that I found useful or interesting. It is particularly useful when bookmarking HN/Reddit discussions, newsletters / link aggregations like High Scalability Newsletter. The hightlights are available when I open the page later so I know what I liked. Furthermore, the highlights show up when browsing the bookmarks on diigo, which makes it really great for research.
You should totally do it... it's fantastic. Norvig makes it quite clear in the beginning that he expects you to know atleast some lisp beforehand (the "intro" chapter is a bit sparse), so you might want to brush you on your CL skills before you dive in.
Peter Norvig also has two implementations of Prolog in Paradigms of AI Programming (first one is interpreted, second is complied) in Common Lisp. The book is highly recommended because Norvig's code is very elegant and his exposition is wonderful.
Allegro Prolog [1] is based on Norvig's implementation, although I'm sure they've done a fair bit of optimizing.
The older EOPLs are much longer then the newer ones. I've found that the 1st edition requires less sophistication than the 3rd and spends more time explaining things. You might want to take a look at it.
There's also "design concepts in programming languages".
I found it worthwhile to read the erlang books concurrently. Other than a few misalignments, they present topics in roughly the same order, but the way they explained the different topics were different enough that I found myself learning something new.
Almost everyone registers for atleast 4 courses a term. You need to work on all of them because every course counts. However, when you're going to take the Junior Lab (if you're in Physics) or the digital death lab/ software labs (if you're in EECS), you take classes that are relatively easy. For example, you might take extra humanities classes to work on the HASS requirement and then in a later semester not take the humanities class and replace them with courses in your own major.
I don't know about the junior lab (i'm in eecs), but when I take classes that are difficult (which has been pretty much every semester for the past 2 years :(), I am working overtime on these classes. Besides attending lectures/recitations during the day, I spend every evening, and all of my weekend coding for the projects. Having projects and labs that took over 20-30 hours to complete were the norm, and in a particularly bad semester, I'd have one of those due almost every two weeks.
Initially, there would be people who could get by without putting in as much effort because they already knew the material, but as you go higher and start talking about lab classes (where the work is punishing regardless of your knowledge/experience) or graduate classes (very intensive, almost no one can coast), you don't see people coasting.
I think there's a misunderstanding about how the class actually works.
First off, this is for students in Aero-Astro program. It's not something that all engineers take; it's simply the starting courses for the aero astro people.
Second, even though the name suggests that it's some course that tries to teach you the intersection of all the topics, it's actually teaching you the union of the topics. From the course overview:
"Over the course of each semester, students engage in seven disciplines of study, listed below. Each discipline is taught for a fraction of the semester, through a series of lectures. When one discipline concludes, students are quizzed and begin a new discipline."
It's more for scheduling convenience and to have the students up and running with all fields in Aero-Astro than to do some grand unification of these topics.
> MIA Meeting: Low dimensional embeddings of words and documents (and how they might apply to single-cell data)
Leland McInnes Tutte Institute for Mathematics and Computing
Over the last decade the field of Natural Language Processing (NLP) has been overtaken by neural networks and deep learning. The latest models and algorithms, from word2vec to Google’s Universal Sentence Encoder and BERT, can perform seemingly magical feats and provide powerful tools to understand and analyze text documents. This talk will seek to pick apart word and document embedding techniques from NLP, removing the neural networks and instead working with linear algebra and dimension reduction on large sparse matrices of counts that is very similar to a large amount of single-cell data. Such an approach can achieve results on par with neural network approaches, but are both simpler to understand the inner workings of, and are generalizable to much more diverse domains than NLP, including, hopefully, single-cell research. In particular all we really need to apply these kinds of techniques are large sparse matrices of counts and some notion of locality or cooccurrence of the feature columns of that matrix. The goal of this talk is to hopefully open up some of the ideas that have been so effective in NLP and make them usable and accessible to a wider audience working with more diverse types of data, and initiate a conversation about what is required to make such techniques useful for the single-cell community.