Transformer is a patterning probabilistic machine for a sequence of identities[1]. These identities are fed to the transformer in lanes. The transformer is conditioned to shift lanes one position to the left until they make it to the output, and make a prediction in the right-most lane that got freed up. Attention adds an exponential amount of layer interconnectivity, when we compare it with a simple densely connected layers. The attention mask serves as a high-dimensional dropout, without which it would be extremely easy for the Transformer to simply repeat the inputs (and then fail to generalize when making the prediction). Each layer up until the vertical middle of the Transformer works with a higher contextual representation than the previous one, and this is again being unwound back to lower contexts from the middle layer back to the original identities (integers) on the outputs. This means that you have raw identities on the input and output which span a certain width/window of the input sequence, but in comparison the middle-most layer has a sequence of high level contexts spanning extreme lengths of the original input sequence, knowledge-wise. [1]It's important to know that modification (learning) by the Transformer, of the vector embeddings which represent the input/output identities/integers that the Transformer works with, constitute big portion of the Transformer's power, and the practical implication of that is that it's impractical to try to tell the Transformer that e.g. some of our identities are similar or there's some logical system in their similarity, because all the Transformer really cares about is the occurrence of these identities in the sequence we train the Transformer on, and the Transformer will figure out the similarities or any kind of logic in the sequence by itself.
Life (aging) is a direct cause of cancer. And we can't change that unless we have some sort of nanobots constantly repairing DNA damage in every single cell of our body.
I know I'd never be able (or wanting) to live with having left my dogs behind. I don't fear my own death _that_ much. I'm here for them until they cross, only after that I can do whatever I want with my life.
Damn, so I really need to follow up on my plans on moving to e.g. Thailand (and getting happy by following my dreams in terms of work/endeavors).
I don't have IBS but more like a colitis and I was really good for months but it came back strong after one of my dogs passed away and I got terribly sad.
What I'm getting at is that more than a vitamin D, I suspect a strong link to happiness and life satisfaction.
1. When people are financially incentivized to run servers, they always will.
2. If a crypto protocol doesn't evolve at the pace of available innovation, that particular blockchain will be superseded by a new one. That said, a (truly democratic) evolutionary process is a core part of every blockchain specification.
3. You can get blockchain data via public (and federated/proxied) API, but you can always cryptographically verify its veracity, and your edge device (e.g. your smartphone) can do that. The same the other way around, you cryptographically sign the inputs you send to the networks, so that no federated API can tamper them, because the secret key stays on your device. This is referred to as the "trust-less model".
It's interesting to see how human understanding differs when it comes to complex, yet clearly defined topics, like machine-learning/Transformers.
For comparison, my understanding of Transformers, after going through Peter Bloem's "Transformers from scratch" [1], implementing/understanding the code and the actual flow of the mathematical quantities, my understanding is that:
- Transformers consist of 3 main parts: 1. Encoders/Decoders (I/O conversion), 2. Self-attention (Indexing), 3. Feed-forward trainable network (Memory).
- The Feed-forward is the most simple kind of (an input->single-layer) neural net, actually often implemented by a Conv1d layer, which is a simple matrix multiply plus a bias and activation.
- The most interesting part is the Multi-head self-attention, which I understand as [2] a randomly-initialized multi-dimensional indexing system where different heads focus on different variations of the indexed token instance (token = initially e.g. a word or a part of a word) with respect to its containing sequence/context. Such encoded token instance contains information about all other tokens of the input sequence = a.k.a. self-attention, and these tokens vary based on how the given "attention head" was (randomly) initialized.
The part that really hits you is when you understand that for a Transformer, a token is not unique only due to its content/identity (and due to all other tokens in the given context/sentence), but also due to its position in the context -- e.g. to the Transformer, the word "the" at the first position is a completely different word to the word "the" on e.g. the second position (even if the rest of the context would be the same). (Which is obviously a massive waste of space if you think about it, but at the same time, at the moment, the only/best way of doing it, because it moves a massive amount of processing from inference time to the training time - which is what our current von-Neumann hardware architectures require.)
OP here. The second sentence came out all wrong and the downvotes are deserved. What I was trying to ponder, was - given the summer blackouts (in certain areas) due to people running their ACs (more than on average) - how do we plan to cope with the advent of EVs.
So I did some googling[1] and apparently, until 2030 there's only 1 million EVs planned to hit the roads in the US, which is quite shocking given the dire trends of climate change and toxic pollution on this planet.
Another interesting fact is that the costs of extending the power grid will be passed on to all its consumers in the form of increased rates (as it should, because if you don't drive an EV, you should be "taxed" on the extra pollution your car creates).
Forget all that. If we tried to swich half the cars to electric overnight, the power grid would collapse into a permanent blackout. Has anyone estimated the cost and time of upgrading the power grid for electric cars? Is hydrogen delivery from power plants to electric charging stations a viable option to bridge the gap until the grid is upgraded?
There should be legal repercussions against anyone even suggesting something as blatantly illegal and tyrannical/totalitarian as this. Have we learn nothing from communism?
Globally destroying web experience by mandating super-useless, super-obnoxious Cookie prompts on all web pages is one thing, but setting things up for "Equilibrium" or "1984" type of society is a whole new level, and I'm not sure if sheer stupidity can be claimed on this one.
Mankind will most likely destroy each other using super-intelligence long before super-intelligence gets any chance to do so by its own volition. We are built on conflict, disagreement, misunderstanding and exploitation - it's part of our societal DNA, no matter how sweet or reasonable we can make ourselves seem on personal level. We have avoided nuclear annihilation by luck on multiple occasions. Nuclear weapons are sticks and stones compared to the power of a runaway super-intelligence in a technological society.
Thinking that super-intelligence is containable is like thinking you can beat AlphaGo in the game of Go. And coming up with reasons why that's no so it's just you being in denial of your eventual mortality, as well as of the eventual extinction of the species. Best that we can hope for is that that extinction will be some form of transmutation to a higher level of evolution.
For some reason the Apple TV YouTube app's recommendations are pretty abysmal compared to the desktop website or the iPad app, and Watch Later isn't saving the day either. Could be that the app displays a grid of just 6 videos per screen, which is utterly ridiculous for any TV screen, god forbid a projector. Premium is still worth it though.
Governments should be forced to choose between collecting income taxes _or_ taxing everyone through the general monetary inflation (a.k.a. money printing). They currently practice both - the income tax taxes the non-rich, and the inflation also only taxes the non-rich, because the freshly-printed money goes to the rich.
A distinction should always be made between the government and the elites who influence it to benefit their interests. Being a member of the elite doesn't preclude one of being a (high-ranking) member of the government, or becoming a member of elite that way.
That "Facebook" we always speak of is never the same thing for everyone. Every Page "like", every "friendship", will change what one sees in their social bubble.
I, for one, am glad that my social bubble is utterly boring and un-engaging, and my level of engagement with Facebook is inversely proportional to my engagement/passion in life as such.
Transformer is a patterning probabilistic machine for a sequence of identities[1]. These identities are fed to the transformer in lanes. The transformer is conditioned to shift lanes one position to the left until they make it to the output, and make a prediction in the right-most lane that got freed up. Attention adds an exponential amount of layer interconnectivity, when we compare it with a simple densely connected layers. The attention mask serves as a high-dimensional dropout, without which it would be extremely easy for the Transformer to simply repeat the inputs (and then fail to generalize when making the prediction). Each layer up until the vertical middle of the Transformer works with a higher contextual representation than the previous one, and this is again being unwound back to lower contexts from the middle layer back to the original identities (integers) on the outputs. This means that you have raw identities on the input and output which span a certain width/window of the input sequence, but in comparison the middle-most layer has a sequence of high level contexts spanning extreme lengths of the original input sequence, knowledge-wise. [1]It's important to know that modification (learning) by the Transformer, of the vector embeddings which represent the input/output identities/integers that the Transformer works with, constitute big portion of the Transformer's power, and the practical implication of that is that it's impractical to try to tell the Transformer that e.g. some of our identities are similar or there's some logical system in their similarity, because all the Transformer really cares about is the occurrence of these identities in the sequence we train the Transformer on, and the Transformer will figure out the similarities or any kind of logic in the sequence by itself.