Here is a more recent paper [1] from Pentti Kanerva that gives a great introduction: http://www.rctn.org/vs265/kanerva09-hyperdimensional.pdf. More papers can be found with the query "vector symbolic architecture". I find it utterly fascinating.
[1] Kanerva, Pentti. “Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors.” Cognitive Computation 1 (2009): 139-159.
While it is difficult to get a useful signal out of EEG, a PCA doesn't show that there is only one 'channel'. A lot of interesting signals have very low power. With multiple EEG sensors, one can focus on smaller spatial regions.
> But recent advances in causal discovery have actually rendered it possible to black box the structure of a DAG from data -- happy to provide citations if this is down the rabbit hole.
The reservoir can compute non-linear functions of the input over time. So, with a reservoir, you can train a linear projection of the reservoir state that responds non-linearly to the input. Without the reservoir, the output projection can only be linearly related to the input.
Further, these inputs were /optimized/ to confuse the particular classification software. If only we would be able to optimise optical illusions for a given human viewer...
> They only have enough room to tersely describe directly relevant background material and will assume you are knowledgable in the domain.
I feel like it just is the dominant style; it doesn't have to be that way. Sometimes there are these gems, that are extraordinarily well written and bring new ideas in a gentle way. Some examples:
- Zhang, Cha, John C. Platt, and Paul A. Viola. "Multiple instance boosting for object detection." Advances in neural information processing systems. 2005.
- Gelman, Andrew, et al. Bayesian data analysis. Vol. 2. Chapman & Hall/CRC, 2014.
> For those with positive coefficients we can expect that the language is associated with, ceteris paribus, a greater number of defect fixes. These languages include C, C++, JavaScript, Objective-C, Php, and Python. The languages Clojure, Haskell, Ruby, Scala, and TypeScript, all have negative coefficients implying that these languages are less likely than the average to result in defect fixing commits.
Notice how top projects in popular languages do have tendency to have more fixes than top projects in more obscure languages. Perhaps these projects have simply more users, leading to more reported bugs and community pressure? Interesting paper, but it is all to easy to jump to conclusions.
I don't know, I think I used about 40 languages. The beauty is that zip-compression captures rich statistical properties of the languages, so representation-wise it should come a long way. But counting compressed output length discretises the lang-lang distance. For shorter text this might be troubling, since this could easily result in ties. So, maybe. Perhaps I should try :).
One method that I have used in the past was über-simple, yet extremely effective. It exploits ZIP compression, based on the the insight/assumption that two concatenated texts compress beter when they share their language.
I think I found it in this paper [1]. The implementation was like 13 lines of Python code. I wonder how it would compare.
I would like to add a different perspective. I used to work on a Thinkpad, and loved the machine. Last year I changed to a MacBook Air (MBA). I also like this machine. But one of the biggest changes was that people seemed to be much more easily convinced when I showed them results on the MBA. This is a major point to consider IMHO.
Like other people, I use the 'dipping in the stream' approach. Still, vocal people drown some of the other people. I am experimenting a bit with machine learning methods to rank tweets based on how interesting I find them. Does anyone know of similar solutions?
No, not without some radical new insights :). But, why replace them? If you need something to replace them, probably you have a different need (e.g. hands-free, private communication, or expressing something that is hard to do consciously, like the level of pain, tiredness of familiarity of a face). Perhaps EEG can be used to fulfill that need instead.
Google Glass and the Oculus Rift are very interesting for us, since it puts technology right on the spot where we need to acquire signals for brain-computer interfaces. FYI, there was a successful crowd-source campaign by InteraXon for an acceptable EEG headset [1]. This is the first headset that I can imagine being worn in public spaces.
In five years we can potentially see brain-computer interfaces for consumers. I fear using the word 'solid' though, since I isn't a replacement for traditional input like the mouse or keyboard (that is what I would call solid). The biggest challenge I see is that we have to help consumers understand what it can do. I feel this technology is a game changer, but it is difficult to pinpoint what game is being changed. Therefore, it will at least take a while to get mainstream.
Hmm, difficult question. In the US there are some groups doing very advanced work with implants to restore limb or prosthesis control, and there are some very impressive movies of monkey's controlling robotic arms. But invasive (i.e. with implants) work is not really my thing. And in these studies, often the monkey is the one doing the learning — it is not the device that adapts to the user.
For non-invase (i.e. EEG measured from outside the body) EEG I think that is still far off. The problem is that the signals are measured from a distance, and that it is very hard isolate signals from a precise region in the brain which is needed for accurate control.
I typically express the performance of these brain-computer interfaces in bits/minute. Keyboard gets roughly around 300 bits/min, brain-computer interfaces 2-20 bits/min. I would not know the bandwidth (and latency) requirements for reliable prosthesis control, but that would probably depend on intended use of the prosthesis. But then again, not all the actuators need to be controlled individually; maybe it is feasible with a smart controller and a forgiving application. And of course usability plays a major role; I cannot imagine controlling a prosthesis using the keyboard, although the information throughput might be sufficient :).
I have a background in these so-called brain computer interfaces. Typically, research know some of the properties of the EEG signal during specific tasks. Using signal processing and machine learning methods (and sometimes human training), these tasks can be recognized by 'decoding' the EEG.
Some of these signals are spontaneous (realizing an error has been made), some are produced by voluntarily executing some mental task. Currently, the amount of these 'channels' that is available is limited by 1) the amount of detectors that a lab is willing to build, and 2) how many tasks the user can simultaneously execute — which is typically very low. If you really want a number, I would settle for four as the current state of the art.
I have been working on a method to make problem 1) so easy it can be solved by laymen by just collecting examples of EEG during the task of interest. Now we are founding a startup to make this happen commercially :).
PS: I think this technology does not lend itself well for analogies with channels or buttons. Buttons were invented for a physical world. Brain-computer interfaces lend itself to interact with signals there are /not/ available in normal interaction (i.e. relevance, errors, intended movements etc).
Could this training effect of a PhD perhaps be the reason that people switch fields? When starting, one's field of research seems very promising and exciting. At the end though, one knows enough to see limitations in the field. In addition, PhDs are trained to approach problems differently. I think this knowledge and a different mental attitude can make other fields seems more worthwhile.
[1] Kanerva, Pentti. “Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors.” Cognitive Computation 1 (2009): 139-159.