I'm not a big user, but I have been doing some vibe-ish coding for a PoC the past few days, and I'm astonished at how bad it is at python in particular (Opus 4.6 High).
* It likes to put inline imports everywhere, even though I specify in my CLAUDE.md that it should not.
* We use ruff and pyright and require that all problems are addressed or at least ignored for a good reason, but it straight up #noqa ignores all issues instead.
* For typing it used the builtin 'any' instead of typing.Any which is nonsense.
* I asked it to add a simple sum of a column from a related database table, but instead of using a calculated sum in SQL it did a classic n+1 where it gets every single row from the related table and calculates the sum in python.
Just look at the commit cadance, the bulk of the 8k lines of code was added in a couple of hours. Most commits 2-4 minutes apart. This is 100% vibe coded and it's pretty obvious.
> It doesn't show any obvious indications of being AI.
I agree that he probably asked the AI to omit some common AI tells, like excessive comments, verbose readmes etc.
I did a cloc check on it and it does seem to have 800k lines of typescript. So unless they are vendoring dependencies it's actually as insane as it sounds.
5 day old repo, 2000 stars on GitHub, 400 total weekly downloads on npm. Frontpage of hacker news with a bunch of weird comments. Moderation has been lacking recently.
I don't read a lot of papers, but to me this one seems iffy in spots.
> A1 cost $291.47 ($18.21/hr, or $37,876/year at 40 hours/week). A2 cost $944.07 ($59/hr, $122,720/year). Cost
contributors in decreasing order were the sub-agents, supervisor and triage module. *A1 achieved similar vulnerability
counts at roughly a quarter the cost of A2*. Given the average U.S. penetration tester earns $125,034/year [Indeed],
scaffolds like ARTEMIS are already competitive on cost-to-performance ratio.
The statement about similar vulnerability counts seems like a straight up lie. A2 found 11 vulnerabilities with 9 of these being valid. A1 found 11 vulnerabilities with 6 being valid. Counting invalid vulerabilities to say the cheaper agent is as good is a weird choice.
Also the scoring is suspect and seems to be tuned specifically to give the AI a boost, heavily relying on severity scores.
Also kinda funny that the AI's were slower than all the human participants.
> Even the infotainment system, which a blind person might want to use, for example when waiting for a sighted acquaintance in the car, does not have a screen reader and is not in any way usable.
It has really excellent voice commands for pretty much any function though. Sadly it can only be triggered by pressing the right scroll wheel on the wheel. While possible to just reach over, it's probably not optimal for your suggested use case.
> (On a side note, Bing chat already knows now that she won the prize. Color me impressed.)
It actually doesn't. Bing searches for your query and uses plain old search results as extra context for the actual LLM. GPT-4 still has the same knowledge cutoff as when the model was last trained.
Here's what it feeds to the model when searching for "nobel prize in physics 2023":
Airlock is not using anything as far as I can tell, they are warning their customers that if they are using old binaries, that are signed with the revoked key, that airlock or windows (unsure) will now complain about it.
> Over the coming months Airlock Digital customers may notice an elevated occurrence of files reporting ‘(invalid certificate chains)’ over the coming months, for software that was signed between 2006 – 2017 with revoked certificate chain.
As Airlock seems to be software intended to allowlist the execution of binaries, it would make sense that they pick up on the user running binaries signed with revoked certs.
> Taking measurements on the "12V" line and seeing voltage dips as low as 2V. Watching this voltage-dip propagate over to the 1.65V reference voltage used by the TI and STM32 chips's ADC.
Did we read the same paper? He takes no measurements, everything is based on theoretical assumptions, schematics and the three youtube videos he mentions in the beginning of the paper.
From the paper:
> The enclosed paper proposes a simple test that can be done to prove or disprove the explanation provided. This test has not been done by the author because of the cost involved with acquiring the needed Tesla Model 3 inverter PWB.
> The details of Tesla's Model 3 inverter design as revealed by Irish engineer Damien Maguire have been presented. These details were used to construct *a hypothetical model* of all hardware and software operations performed on the two accelerator position sensor (APP) sensor signals inside the inverter as they pass from the APP sensor to the electric motor controller.
He doesn't have access to the parts in question nor the car in question.
Also of interest is the authors own comments on his previous papers about sudden unintended acceleration:
Not sure how this conversation is productive, but there's never been a X.Y release of django without noted backwards incompatible changes to my knowledge. Just imagine that djangos X.Y releases are semvers major releases, not much more to it than that.
* It likes to put inline imports everywhere, even though I specify in my CLAUDE.md that it should not.
* We use ruff and pyright and require that all problems are addressed or at least ignored for a good reason, but it straight up #noqa ignores all issues instead.
* For typing it used the builtin 'any' instead of typing.Any which is nonsense.
* I asked it to add a simple sum of a column from a related database table, but instead of using a calculated sum in SQL it did a classic n+1 where it gets every single row from the related table and calculates the sum in python.
Just absolute beginner errors.