If you like this you may enjoy the "Pink Trombone" an interactive speech synthesizer that you control by moving around the virtual tongue and lips: https://dood.al/pinktrombone/
Like Bret Victor? Who seems to have been involved in presenting this to Jobs? I'd say he's a pretty interesting guy and probably not worth firing even if you don't like one of his proposals.
I agree with the first half of this comment, but I don't follow you here: "Until there is an element of randomness fueled by context, they aren't creating anything new. That's where genius lies."
I suspect reframing of problems, seeing things in new light, paradigm shifts, new ontologies, whatever you want to call it are not quite so simple as context-dependent randomness! I don't think we understand this process very well right now.
Finally; I think there is an artfulness in copying existing patterns, because the way in which you "abduce"[1] an observation-explaining theory out of the infinite space of possibilities is a creative and aesthetic process.
Unicode Technical Report #51, which is where Emoji are laid out, talks a bit about the current thinking of the committees on this:
> The longer-term goal for implementations should be to support embedded graphics, in addition to the emoji characters. Embedded graphics allow arbitrary emoji symbols, and are not dependent on additional Unicode encoding. Some examples of this are found in Skype and LINE—see the emoji press page for more examples.
> However, to be as effective and simple to use as emoji characters, a full solution requires significant infrastructure changes to allow simple, reliable input and transport of images (stickers) in texting, chat, mobile phones, email programs, virtual and mobile keyboards, and so on. (Even so, such images will never interchange in environments that only support plain text, such as email addresses.) Until that time, many implementations will need to use Unicode emoji instead
I really like your Hidden Unit Zoo here http://colinmorris.github.io/rbm/zoo/ as a window into what this thing is actually "thinking" about. The "top matches" for a given hidden unit are pretty helpful.
The possibilities for how language will work in the future are really exciting and interesting! We made the Minuum keyboard, too, which also explores how machine learning assistance can open up new ways of communicating.
One reason we're interested in visual communication with Dango, though, is that regular text input is pretty good already. Chorded keyboards exist and are way faster, but people mostly can't be bothered to use them. QWERTY is just good enough. But the field is wide open for rich communication with images, nothing out there is particularly good yet.
Well those are sort of equivalent. But yeah, we use cross-entropy between the projected output and the target emoji distribution as our objective to minimize.
And yes, we do the t-SNE on that pre-projection space. That's why we can visualize the targets (emoji) in it. We can also t-SNE the word embeddings themselves — the input to the RNN — which is also kind of interesting. It automatically learns all kinds of structures there. Chris Olah has a good post on word embeddings if you're interested: http://colah.github.io/posts/2014-07-NLP-RNNs-Representation...
But yeah our main focus is suggestions. You can use Dango concurrently with the normal emoji keyboard, of course! It can just sit there showing you emoji you might not know about "ambiently"
Yeah we'd need to do some more work. But this is similar to stickers and GIFs: there are many fewer examples for any given sticker or GIF, which is why we do transfer-learning approaches as discussed in the article.
So there's a good chance we could get it to work! We've not focused on that possibility… yet.
Yeah so this is a legitimate concern. Of course sometimes it's fun to say "let's eat pizza :pizza_emoji:", but that's not hugely valuable.
However, Dango's training data includes people using Emoji to augment rather than repeat their sentence. So if there are two different interpretations and an emoji could disambiguate, the ideal is that Dango has seen people use that phrase both ways and, and that it suggests both possibilities and you can pick the one that you meant. In many cases this works now, in many cases we still have work to do.
It also suggests based on messages sent to you, so if there are a couple different replies it can show you them all (although this feature still needs work).
So yeah: we can focus on vectors at different levels of the net and these are in some sense different semantic spaces. In the article I talk about a level immediately before it projects onto the emoji vectors. If you look at the output after the projection (and do a softmax) you get a probability distribution across all emoji. This would be a different space in which each axis is an emoji, rather than the emoji being points distributed around the space.
Yeah this is a real challenge with AI systems, they reflect both our worst and our best back at us, we're still struggling with the best way to deal with this.
Although in this particular case it's actually just a bug: Dango gets confused by any skin tone modifier character, since they're not supported on Android (our target platform). Try putting in a "white" arm and you'll see the same results. They're actually just our "Dango is confused by this input" results.
We start supporting them as soon as they're available! Obviously we have a lot less training data early on but we can lean on some heuristics early on until it builds up.
Unfortunately in the app we can't give you emoji that your phone doesn't support so we don't always show all the results.
Thanks! I was tempted to go way more technical but we decided it would be fun to let more people get some intuitive grasp of these kinds of algorithms.