This is a very interesting example. I don't blame LLMs for not understanding which English text makes hard sounds. I wonder if it would get better if it were multi-modal.
I just asked Chat GPT 3.5 the following: "Please give an exponential equation that has integer solutions which is difficult to solve.", and the response is:
"
Certainly, here's an exponential equation that can be challenging to solve for integer solutions:
2^x+3^y=7^z
This equation involves three variables, x, y, and z, and requires finding integer values for these variables that satisfy the equation. This type of equation is known as a Diophantine equation, and solving it can be quite challenging, especially for larger values of x, y, and z.
Nature published their last big superconductor claim and then had to retract it. Publications getting burned by these folks makes them understandably reluctant to get worked up again.
What should the headline be? "South Korean scientists claim room temperature superconductivity again! Are they lying like their last paper? Let's celebrate!"
How much resources are required is directly related to the memory size devoted to each weight. If the weights are stored as 32-bit floating points then each weight is 32 bits which adds up when we are talking about billions of weights. But if the weights are first converted to 16-bit floating point numbers (precise to fewer decimal places) then fewer resources are needed to store and compute the numbers. Research has shown that simply chopping off some of the precision of the weights still yields good AI performance in many cases.
Note too that the numbers are standardized, e.g. floats are defined by IEEE 754 standard. Numbers in this format have specialized hardware to do math with them, so when considering which number format to use it's difficult to get outside of the established ones (foat32, float16, int8).