Thanks for trying these out :). Highlights the often subtle difference between knowing the answer and deducing the answer. Feathers could be ground into a pulp and condensed, too. I am not trying to be clever, just seems like the response is a canned answer.
I do not know whether the state of the art is able to reason or not. The textbook example you gave is admittedly not very interesting. What you are hearing from people is that parroting is not reasoning, which is true.
I wonder if the state of the art can reason its way through the following:
"Adam can count to 14000. Can Adam count to 13500?"
The response needs to be affirmative for every X1 and X2 such that X2 <= X1. That is reasoning. Anything else is not reasoning.
The response when X2 > X1 is less interesting. But, as a human it might be "Maybe, if Adam has time" or "Likely, since counting up to any number uses the same algorithm" or "I don't know".
Seems ChatGPT can cope with this. Other examples are easy to come up with, too. There must be benchmarks for this.
Input to ChatGPT:
"Adam can lift 1000 pounds of steel. Can Adam lift 1000 pounds of feathers?"
Output from ChatGPT:
"1,000 pounds of feathers would be much easier for Adam to lift compared to 1,000 pounds of steel, because feathers are much lighter and less dense."
I am not too familiar with the latest hype, but "reasoning" has a very straightforward definition in my mind. For example, can the program in question derive new facts from old ones in a logically sound manner. Things like applying modus ponens. (A and A => B) => B. Or, all men are mortal and Socrates is a man, and therefore Socrates is mortal. If the program cannot deduce new facts, then it is not reasoning, at least not by my definition.
I think it was more a PoC. I would be more impressed if it was deployed in production. "we want to reiterate that these are highly experimental results". If the dividends are massive, would they not deploy it in production and tell the world about it?
It's reasonable to perceive most of the value in math and computer science being "at the scale" where there is unpredictability arising from complexity, though scale may not really be the reason for the unpredictability.
But a lot of the trouble in these domains that I have observed comes from unmodeled effects, that must be modeled and reasoned about. GPZ work shows the same thing shown by the researcher here, which is that it requires a lot of tinkering and a lot of context in order to produce semi-usable results. SNR appears quite low for now. In security specifically, there is much value in sanitizing input data and ensuring correct parsing. Do you think LLMs are in a position to do so?
Maybe very very soft "engineering". Do you have metrics on which prompt is best? What units are you measuring this in? Can you follow a repeatable process to obtain a repeatable result?
I am not suggesting we refuse to close one window because another window is open. That would be silly. Of course we should close the window. Just pointing out that the "950X" example figure cited fails to account for the full cost (or overestimates the benefit).
I agree with the sentiment and analysis that most humans prefer short term gains over long term ones. One correction to your example, though. Dynamic bounds checking does not solve security. And we do not know of a way to solve security. So, the gains are not as crisp as you are making them seem.
This is not a stupid question. I have seen other comments on the thread that confuse the two terms and run with it. Better to ask than assume. Especially since "VM" is the same label for two or three distinct yet related notions in security.
The VM you are familiar with indeed can run an OS, and is indeed not what TikTok does.
#1 VMM - hypervisor runs VMs
#2 JVM/.NET - efficient bytecode
#3 Obfuscation - obscure bytecode
The main thing is that for #2 and #3 the machine language changes.
With "virtualization" as used in most contexts, involving a virtual machine monitor, or hypervisor, one creates zero or more new (virtual) machines, to execute on multiple software recipes. All the recipes are written in the same (machine) language, for all the machines. This can help security by introducing isolation, for example, where one VM cannot read memory belonging to another VM unless the hypervisor allows it.
With the "virtual machine" used for obfuscation, the machine language changes. The system performs the same actions as it would without obfuscation, but now it is performing those actions using a different machine language. Behaviorally, the result is the same. But, the new language makes it harder to reverse engineer the behavior.
Stupid example:
Original instruction: MOV A,B
Under hypervisor virtualization, VM0 and VM1 will perform this same instruction.
Under obfuscation virtualization, software will perform instructions that amount to the same result, but are harder to figure out. So, the MOV instruction is redefined and mapped onto a new (virtual) machine. The new machine does not simply leverage the existing instruction, rather an obfuscated sequence. For example:
A <- B + C + D * E
A <- A - C
A <- A - D * E
Obviously, the above transformation is easy to understand and undo. Others are harder to understand and undo. Look up MOVfuscator to see how crazy things may get.
That's very cool. You probably know more about it than I do, then, but my advice is to articulate the exact problem you try to solve.
I expect your field is probably teeming with AI proposals or offers on how to manage vulnerabilities, but that is doubtful the way, because again it is adding complexity, and no classifier is perfect, especially when scanners fail to understand scanned applications and their threat models or environment.
Stop selling external scanners, start simplifying code? This will never work, of course, because security vendors sell the promise of security to those willing to buy it, in the form of add-on products and capabilities.
Empower people to ignore scanner reports without so much red tape? That would never work either, because megacorp wants compliance and reduced liability.
Build secure systems as opposed to cataloging and scoring flaws? That would never work, because building secure systems is hard, nature tends to favor otherwise.
Charge people for adding complexity and credit them for removing complexity? Sadly, there is no way to do that, especially since products must ship and quality is hard to observe, since it is often invisible and only surfaces when things are broken.
Off the top of my head, would be nice to require proof of exploitation, by adding CTF-like capabilities to apps, such that only if the flag is captured do we consider the report real. This places more burden on scanners, in that it is no longer enough to report an outdated library. Requiring some proof of exploitability reduces noise and increases SNR, reducing false positives. Naturally, not all vulnerabilities have working exploits, and scanners can never fully simulate an adversary, so we may get more false negatives, but at least we would not have to waste so much time upgrading pointless modules and breaking applications to appease a false report. So the idea is "here is a dummy asset, show me how you leaked or compromised it". Adding the dummy asset should be cheap, but would force scanners to better simulate an attack.
At the very least, there ought to be a knob to decrease scanner sensitivity.