> The moment you run LLM generated code, any hallucinated methods will be instantly obvious: you’ll get an error. You can fix that yourself or you can feed the error back into the LLM and watch it correct itself.
But that's for methods. For libraries, the scenario is different, and possibly a lot more dangerous. For example, the LLM generates code that imports a library that does not exist. An attacker notices this too while running tests against the LLM. The attacker decides to create these libraries on the public package registry and injects malware. A developer may think: "oh, this newly generated code relies on an external library, I will just install it," and gets owned, possibly without even knowing for a long time (as is the case with many supply chain attacks).
And no, I'm not looking for a way to dismiss the technology, I use LLMs all the time myself. But what I do think is that we might need something like a layer in between the code generation and the user that will catch things like this (or something like Copilot might integrate safety measures against this sort of thing).
The problem there though, is that with PoCs like this, as an attacker you want to have a ping back to your system so that you know the attack has been successful (in this case they probably expected/hoped someone at Cursor to install the package, that's the usual objective in a dependency confusion attack). But what they could have done, is send a less sensitive thing like just the current working directory or current effective user, instead of the whole environment.
At first I was excited to see that a new tool would solve the Python "packaging" problem. But upon further reading, I realized that this was about _package management_, not so much about packaging a Python application that I've built.
Personally I haven't had many problems with package management in Python. While the ecosystem has some shortcomings (no namespaces!), pip generally works just fine for me.
What really annoys me about Python, is the fact that I cannot easily wrap my application in an executable and ship it somewhere. More often than not, I see git clones and virtualenv creation being done in production, often requiring more connectivity than needed on the target server, and dev dependencies being present on the OS. All in all, that's a horrible idea from a security viewpoint. Until that problem is fixed, I'll prefer different languages for anything that requires some sort of end user/production deployment.
1. Like you already hinted at, it is really difficult to get right, and I hardly experience any larger website (with multiple teams working on it) that implement it effectively. So while it's great in theory, I'm not sure if it's accessible enough (and therefore effective enough) for most of the world.
2. I'm assuming you're talking about Chrome's SameSite value; it's worth to note that this has been rolled back a short while ago because of compatibility issues in larger government organizations having to be accessible especially now with COVID-19. More info here: https://9to5google.com/2020/04/03/chrome-rolls-back-cookie/
I'm not sure I understand your question correctly, but I'm talking specifically about outbound network traffic. Your API's application servers (where such evil libraries could be deployed) should not be able to have any network connectivity towards the internet. So on that server, you should not be able to do even `curl www.google.com` for example.
This will keep happening, and not only will SSH And GPG keys be the target, but any interesting data will be stolen.
And the problem is much larger than these typosquatting attacks. Abandoned Github projects taken over my malicious users, rogue Maven/npm/PyPI/what have you repositories, hacked accounts on any website that is used for distributing programs, feature branches in open source projecs that are automatically built on CI servers in side corporate networks, the possibilities to grab data and send it to somewhere on the internet are endless.
One security measure that somehow grew out of fashion over the last years, is at least on application servers, to disallow any outgoing network traffic, especially to the internet (at least any cloud environment I see nowadays allows it by default). This would largely prevent these sorts of attacks from being able to actually send anything out, but also prevent XXE attacks from happening, prevent reverse connections to an attacker host from being set up, make SSRF attacks harder to verify, and so on.
I strongly recommend whitelisting only the network traffic that your application actually needs.
Lots of problems are merely I/O related, which can be solved just fine with Python threads. As for number crunching (let's assume that means CPU intensive tasks), you can always still resort to multiprocessing (which can also be combined with multithreading, I fail to see why such paradigms cannot be combined).
CVSS is a common and open approach. CVSS scores that are particularly high, are often directly exploitable. If they aren't, then probably their calculation was done wrong. You can try it for yourself here: https://www.first.org/cvss/calculator/3.0
Having said that, what can lead to a serious attack for your organization is often subject to a combination of factors. Say, there is an SSRF vulnerability in your own application, because the HTTP library you use doesn't parse the URL correctly, so now an attacker can let your application perform arbitrary HTTP requests. But fortunately, the connectivity of your application server is quite limited, so that an attacker can't reach internal systems, can't go to the internet, uses strong authentication to web services it does use, all the good stuff. So now the chance of a successful, serious attack is largely diminished.
Also, it can be quite complicated to know what the exact, real dependencies of your application are. What is the transitive/recursive list of dependencies your application uses? Which of your application's dependencies actually use libraries on your system? And what are _their_ dependencies? I think that cost wise, it is cheaper to make sure your application dependencies, containers, host system libraries, container orchestration tools, etc. are always up to date.
And yeah, I agree that the post doesn't do a good job at all to provide a sane rationale on _why_ you should update. Anyone who has ever administered an operating system knows that security vulnerabilities are found in them every day. But the awareness that a Docker container is subject to the same pace is definitely not present everywhere, and it probably should be.
I think it's important to be careful with drawing conclusions about what might or might not be used by whatever is running in your container and exposed to end users, if only for the simple fact that it might not be too obvious how a particular vulnerability gets exploited in the wild. CTF writeups often illustrate very nicely how there are quite creative ways to exploit vulnerabilities. It's not safe to assume you can predict every scenario. The path to an attack can be very complicated (but easy to execute, once the exploit is scripted).
I agree that the article doesn't emphasize what is the actual important point here (it is clickbait-ish), but the numbers they're presenting should (hopefully) trigger people to actually think about putting "continuous eyes" on the container images they're using. Just like you should continuously monitor your code, your application's dependencies, and your host system libraries.
A hacker needs to be right once, you need to be right 100% of the time. That's not marketing.
On premise deployed stacks are usually shielded off by network firewalls though, which cannot be said about most cloud based services (yes, I get it from a convenience point of view).
In order to attack an on premise application that is safely behind a firewall in a private LAN, one would have to come up with more creative, staged attacks (such as DNS rebinding or blind XSS attacks).
So, put simply, whereas the attack vector of a cloud based service is basically "the internet", this isn't usually the case with on premise deployments, and therefore they are inherently safer, or at least much harder to attack.
On top of that, cloud based services are usually multi tenant environments, so the assets to gain there are huge. One serious hole in such a service (be it Slack, logz.io, Github private repos, New Relic, etc.) will probably be a disaster.
But now you're hitting on one of the main points that was always criticized in PHP: the fact that it came with a lot of batteries included. The upside of this is that it makes PHP applications very easy to deploy, because often, you either don't need to resort to binary dependencies outside the realms of your OS's packaging system, or there will be a `php[major-version]-[library_binding]` package for your OS probably already in your OS's official package mirrors. Say all you want about PHP, but of all the application stacks I've had to deploy, PHP was and is probably still the simplest. The advantage of that shouldn't be underestimated. The only serious competitor in that space that I've ever touched is probably Go.
The article mentioned ctags, which actually gets you quite far in this regard. Combined with CtrlP (a vim plugin) and the Silver Searcher, I navigate large codebases quite quickly. I do agree that full blown IDEs probably have better support for refactoring code, but so far the advantages of using vim/tmux/unix commands are much more valuable to me:
- it's more natural for me to have a shell as a starting point and jump into an editor, rather than the other way around; it suits my workflow much better
- I can combine tools in a very convenient way, `:!ls /tmp` real quick from my vim, fuzzy search some json file hidden inside an atrocious directory structure, `:!git status -s` to see what I've changed, and so on
- IDEs are often language specific, which implies learning a new IDE for every language; I only need to know vim, which I use for every language I use but Java
- I'm more of a keyboard person rather than a mouse person, so terminal based tools that have intuitive keyboard bindings suit me better than IDEs which often require you to learn some really awkward key combinations to do things that I do very quickly in vim; just take a look at the cheatsheets for e.g. PyCharm/IntelliJ and you'll see what I mean (yes I know about the vim plugin for IntelliJ, but as an experienced vim user, it's not that great)
That's a rather quick conclusion. In the blog post she writes that at least once she helped debugging code, so perhaps she just prefers to have some association around a problem, instead of just a bare bones problem to solve. There are many people who are like that, especially the ones who tend to "deep process" problems in their thoughts in order to gain a clear picture of it. They are probably likely to be able to solve complex problems in a domain they're familiar with, but not quite so good at solving problems without any context.
Also there's the self confidence issue, common for many people, even while not a lot of people speak out about it. Ask someone with low self confidence to stand next to a white board and "implement" solutions to random problems, and he/she will freak out. At the same time, these people might very well excel in a more nurturing, friendly environment. Yet at the same time, people who are actually self confident and able to solve problems without much context on a whiteboard, might start running in circles once faced with unfamiliar problems that do not directly map to the algorithms they already know.
I found that just talking about code or problems with someone, can tell you lots of things about the ability of a person. Also, most likely, the candidate will already have some code put online, and god forbid companies are too lazy to look at it, rather than stroking their own ego by asking out of context questions that they already know the answer to.
Although you're right, one could argue that this is the world you most likely end up in when you "do Java", as most Java job openings seem to list at least one or two of the frameworks you mention, as compared to the job openings for something that would look more like core Java oriented as opposed to "Enterprise" oriented.
I have learned that when choosing a language to specialise in, you also choose the culture around it, which becomes very important when you live in it every day.
I must say that the more I read articles of this character, the more I feel that it would make sense to distinct between different types of "engineers".
A lot of the proposed advices are applicable only in certain fields, and not so much (or not at all) in different fields, whereas every field can go very deep (so it's a waste of time gaining knowledge you'll never use there). While a number of these advices are fairly generic, others definitely are not. A front end (client side) web developer won't benefit from having deep knowledge of data structures or the kernel, but he will benefit from knowing about Big O notation. Additionally, he will benefit lots from knowing how browsers work, and having deep knowledge of HTTP. Someone working on ETL processes all day will not benefit much from knowing about interpreters and compilers, but will benefit from knowing a lot about UTF-8. But also, he will benefit lots from knowing a large number of layer 7 transport protocols. A systems programmer (defined as "someone who programs close to the hardware") won't typically benefit much from knowing javascript or how browsers work, but will benefit lots from Big O notation, data structures and algorithms, and C. And so on.
I'm seeing a future in which we will be able to more distinctly treat separate specialisms within the field that we now call "software engineering" (or countless variations on it).
Probably a controversial topic, as this touches on holy grails. My personal take on this, given my own experience so far:
1) You don't strictly need to know Big O notation in order to not introduce performance bottlenecks, or mitigate them when they arise; but you should be able to use common sense resource wise, when developing things in any problem area. In other words: you should conceptually understand how computer systems work (which often implies you'll sooner or later run into Big O notation). Knowing Big O notation can also help you communicate with your peers about problems, if they know it too.
2) For a large number of problem areas, this is knowledge that is near useless I think. Know that they exist, and that they might ever help you. But for your average problem domain, you don't really need concrete knowledge there. It bothers me how much emphasis is put on this in so many job interviews, while it's hardly ever relevant in many of those cases.
3) I'm not sure how that knowledge will really help you.
4) I think this is actually the most important thing of all four. Knowledge of the kernel's main system calls will help you tremendously when your process (or any process in your stack) "just hangs" sometimes and you need to resort to systems such as strace, dtrace, sysdig and what not. That time will come, sooner or later.
I could think of a few reasons why you would like to run something in a Docker container.
- OS packaging is tedious to say the least, and "git clone and pull dependencies on production systems" processes are generally considered messy (if not evil, especially when pulling from repositories hosted on the internet); Docker solves both of these issues by offering a generic interface for shipping and deploying isolated instances of your application. You don't get that with just namespaces and apparmor, you need an API for that (which is really what makes Docker so useful)
- Docker (potentially with the help of its ecosystem) can provide a uniform interface to a couple of the most important operational aspects of an application: logging and monitoring. Especially for heterogeneous or simply large environments, this is a big win.
- When container deployment orchestration matures more, it is much easier to manage and auto-scale your application in a large scale setting since you don't need to reinvent that wheel for every specific stack out there. It will come.
- It makes setting up and understanding development environments easier. Similar to Vagrant, Docker Compose lets you describe your architecture in a config file and easily set up a full stack for you. Especially in companies supporting or developing for a complicated stack, that's very useful. It also probably makes on-boarding of new developers a lot easier.
Still you're making a valid point: you need to maintain those containers. Just like you need to maintain your application's dependencies. Let's be honest though, in many ecosystems that problem already exists: take your typical JEE application that once built, hardly ever upgrades its dependency list anymore (no one even monitors what security holes are being found in all those jars, in many cases). But yes, you should. That problem is really not solved with Docker containers. Most images/containers will be as thin as possible (as will the host OS, preferably), but the concern remains valid.