I don't know if this is the future, but if it is, why bother building one version of the software for everyone? We can have agents build the website for each user exactly the way they want. That would be the most exciting possibility to come out of AI-generated software.
It's not about training directly on the test set, it's about people discussing questions in the test set online (e.g., in forums), and then this data is swept up into the training set. That's what makes test set contamination so difficult to avoid.
If I'm understanding correctly, by "income", this article means "actual individual consumption", which is the amount of money spent by a household. Thus, the article is saying that for countries where households spend more, they spend more on healthcare. Given that healthcare is a huge fraction of household expenditures (almost 20%), this seems tautological. Am I misunderstanding something?
This is exactly the problem we have found in our research on generative AI for education [1]. We ran a pilot in a large high school in collaboration with math teachers, and found that students basically copy answers from ChatGPT, resulting in worse performance compared to students not given ChatGPT. If students don't want to learn, ChatGPT isn't going to fix anything.
This breaks down because it's not easy to statically reason about when a variable is a NonZeroNumber. For example, what is the type signature for subtraction of two NonZeroNumbers? You can't guarantee that it isn't zero, so it has to be a Number. Thus, you can't divide by the difference. You could use a more powerful type system to reason about these kinds of constraints, but then type checking quickly becomes undecidable (or at least very, very expensive).
Important caveat with some of the results: they are using better prompting techniques for Gemini vs GPT-4, including their top line result on MMLU (CoT@32 vs top-5). But, they do have better results on zero-shot prompting below, e.g., on HumanEval.
> So, the 2% growth rate for world energy consumption should be a conservative assumption.
An important caveat: this article assumes that energy consumption will continue to increase exponentially to get the 1000 year timeline of draining the rotational energy of the Earth.
I don’t really understand the argument that corporate profits are driving inflation. If demand for a product goes up, we would naturally expect prices to rise since there is more competition for a fixed set of goods. Since the cost of producing the product has not increased, corporate profits necessarily rise. In the long term, we would expect competition to drive prices back to marginal cost as production of that product increases, but this process can takes time.
In this case, how do you separate inflation due to increased demand from inflation due to corporate profits? The two seem inextricably tied to me.
I’m not sure I buy the conclusion. The point in the original story (at least the version I’ve read) was that Mel’s main job was to write code that could make the computer appear as fast as possible to potential customers. Of course one hack isn’t going to noticeably improve performance, but a collection of hacks can significantly improve performance.
I'm not sure about that. If Mary knows all the physics of light, she can design and set up a sensor to detect the color of the apple and the instructions, just like we have sensors that detect ultraviolet and infrared light. Just as we will never know how it feels to "see" ultraviolet light, this thought experiment is about Mary's subjective experience perceiving color, not her ability to determine the color of the world around her.
One important caveat is that in the study, as far as I can tell, the money is coming from a different country (presumably the USA) than the target country (Kenya), so there is a baseline effect of increasing the net wealth of Kenya. If the money is collected from and distributed to the same community, the results may not necessarily be the same. I think it would be very interesting to see a study along these lines, but I’m not sure how feasible it would be.
Uniformly random data means that someone’s perception of their ability is uncorrelated with their actual ability, which is exactly what DK=true is saying!
I feel like this article is severely over-complicating the analysis. Looking at the original blog post [1], their key claim appears to be that "random data produces the same curves as the DK effect, so the DK effect is a statistical artifact".
However, by "random data", the original blog means people and their self-assessments are completely independent! In fact, this is exactly what the DK effect is saying -- people are bad at self-evaluating [2]. (More precisely, poor performers overestimate their ability and high performers underestimate their ability.) In other words, the premise of the original blog post [1] is exactly the conclusion of DK!
Looking at the HN comments cited [3] by the current blog post, it appears that the main point of contention from other commenters was whether the DK effect means uncorrelated self-assessment or inversely correlated self-assessment. The DK data only supports the former, not the latter. I haven't looked at the original paper, but according to Wikipedia [2], the only claim being made appears to be the "uncorrelated" claim. (In fact, it is even weaker, since there is a slight positive correlation between performance and self-assessment.)
So, my conclusion would be that DK holds, but it does depend on exactly what is the exact claim in the original DK paper.
As far as I know, Konrad Zuse didn't prove that this strategy was a universal model of computation. In contrast, Turing proved that his universal machine could emulate any other machine, given the right program.
In my view, Turing's contribution is providing a plausible definition of computation along with a deep and comprehensive theoretical characterization of the properties of this model of computation. This is why Turing machines form the basis of theoretical computer science, and not other models such as lambda calculus. I think saying that Turing machines were adopted since they were merely more convenient is highly misleading.
I think this pattern repeats a lot: There are many cases where you can point to multiple people who invented similar ideas around the same time, but it is typically the person who provided the most deep and comprehensive treatment of the subject that ultimately gets most of the credit. This depth is not conveyed in pop science attributions such as "Turing invented computation", but this doesn't mean Turing doesn't deserve the credit.
I'm not sure what the author has in mind, but a standard way to put a topology on this space would be to use the discrete topology [1] on {0, 1}, and then use the product topology [2] to obtain a topology over the space of binary streams. This space is homeomorphic to the Cantor set (see "Examples" section in [2]), so you can think of it as being the same topology as the Cantor set.
There has been some interesting recent work [1] applying Fourier transforms (more precisely, an adaptation of the Fourier transform to the sphere) to CNNs, to automatically encode equivariance to rotational symmetries.
There is a subtle but important difference. To be more precise, consider the following two statements:
1) There exists some n such that all integers >= n satisfy the desired property.
2) For n = [a specific constant], all integers >= n satisfy the desired property.
These two statements are not the same, but both imply that there are a finite number of counterexamples. The second one is stronger, since we could prove the statement by enumerating all k < n and checking the statement for each such k; if all these checks pass, then the statement is correct.
This strategy does not work for the first strategy since we do not know n, only that such an n exists. In particular, there could be a non-constructive proof that establishes existence of such an n without providing any way to compute such an n.
From the discussion, it does sound like this paper is proving (2), not (1).
I don't think that's quite right. There are two possible precise statements for such claims:
1) There exists some n such that all integers >= n satisfy the desired property.
2) For n = [a specific constant], all integers >= n satisfy the desired property.
I'm not familiar with Artin's conjecture, but from your description, it satisfies (1) but not (2). The reason is that if there are at most 2 such primes, we can take n to be the larger of the two primes plus one. Since all primes are finite, this choice of n is also finite.
I think the key question is whether the paper described in this article proves (1) or (2). From the discussion, it sounds like it proves (2), which is the stronger result.