Eigenvalue are more flexible than singular values -- If you have procedure for computing eigenvalues of C, squared singular values of X are eigenvalues of C=XX'. Left and write singular vectors of X and eigenvectors of XX' and X'X. For rotation rotation matrix, singular values are all 1. Meanwhile eigenvalues tell you angles of rotation https://math.stackexchange.com/questions/4874616/obtaining-a...
I had similar experience, Claude made report of MRI for achilles tear, it measured the gap, but it was completely hallucinated. Achilles tendon is black on the MRI, it instead measured 13mm distance between two completely different things (looked white), the radiologist looked at and saw no gap at all
This project gave me motivation to build the deep learning next token prediction integration for JetBrains because I was using PyCharm at the time. (Eventually, it wasn't continued because it was kind of expensive to host)
I don't hold grudge, GPT-2 wasn't that great of a model, so releasing it would be more of a publicity value. But the blog post already had that purpose.
This was around the time I trained Transformer-XL (outside of OpenAI) with Ben Mann (https://yaroslavvb.medium.com/scaling-transformer-xl-to-128-...) . Originally we wanted to release train and release the weights as a kind of GPT-2.5, but our OpenAI friends pushed us to keep weights closed.
I used to look at all TensorFlow questions when I was on the TensorFlow team (https://stackoverflow.com/tags/tensorflow/info). Unclear where people go to interact with their users now....Reddit? But the tone on Reddit is kind of negative/complainy
Stricter (but not looser) standards can be imposed on state level. Canada has no binding national drinking water law, they leave it to territories/provinces to decide how to implement guidelines.
I've used this feature before to make my chats discoverable through search engines. I had to manually click it each time I shared, it didn't toggle on.
The difference is that for some assets you can calculate value based on fundamentals. IE, humans need shelter, hence estimate future value of shelter (real estate) based on migration trends and other factors. How do you estimate future value of bitcoin? Lack of predictability is probably why serious investment funds don't go into crypto
Realistic simulation of neurons is expensive. Back in my grad school days we ran Genesis and could afford at most 10k neurons - each neuron needs a lot of work to model the corresponding differential equations. However, it's unclear how to translate this into requirements for artificial neural networks -- the type of computation is too different.
A different metric is a more relevant goalpost -- number of synapses. If each of 125 trillion synapses in the brain can adjust its strength independently of others, it loosely corresponds to a parameter in a neural network. So if we get 100 trillion parameter networks training but still no human intelligence, we'll know conclusively that the bottleneck is something else. Currently training 1T parameter networks seem feasible
1. You don't have to sell the stock -- one could fund an ultra-wealthy lifestyle by renting expensive things, using stock as collateral. As your stock goes up in value, so does the value of things you have access to.