I have seen it firsthand in the CS department here at Dartmouth. It is bad.
We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.
This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.
My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.
In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.
It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.
> the OpenBSD project has not been very receptive in the past for bug reports, my sense of “this is probably fine, in practice”, and that if OpenBSD wants to weed out UB from their code base, then that’s a major project that should be done in a better way than me just being the middle man between the LLM and them for a patch here and there.
Part of the reason for all the UB in OpenBSD is that UBSan doesn't run on that platform. When I ported OpenBSD's httpd to Linux, I found that UBSan tripped before the server even came up because the config flag parsing shifts into the MSB of a signed integer.
I tried to contribute back a patch (just make the flag bitfield unsigned), but it was ignored. I think if UBSan ran natively on OpenBSD, then there would be a lot more of these patches, and the maintainers would have to take an official stance on whether they think these bugs matter.
I hope he ends up covering integer division by constants. The chapter on this in Hacker's Delight is really good but a little dense for casual readers.
It's a custom assembler built on top of the LLVM assembler (llvm-mc) that emits instrumentation code to catch ABI violations at runtime. Stuff like clobbering nonvolatile registers, misaligning the stack pointer, misusing the redzone, assuming volatile registers don't change across a function call, etc.
Hoping to finish up basic x86_64 support within the next few days. I can now reliably assemble and run unoptimized gcc output without hitting false positives, but I still have to iron out some false positives triggered by OpenSSL's handwritten assembly routines.
TODO items for the near future include porting the runtime support library into a kernel module so I can instrument Linux, and beginning ports other architectures (ideally something semi-obscure like POWER or RISC-V). I also need to figure out how to support dynamic linking, because the tool currently needs static linking to access its thread-local variables.
We encourage students to use their own machines. We even loan out old ThinkPads and MacBooks for students to run Linux on. Very few (3-5 per term) take us up on this offer. Most of our students would rather we do the sysadmin stuff for them.
The only requirement is that the code needs to run on our server, which they can easily check before submission.
I ran the servers for our networks, binary exploitation, and intro systems programming classes, and this thing is a major annoyance. It is because of this stupid RAT that students do not understand how to use the OpenSSH client.
I've tried a few things to fix this:
1., I set a motd on our class servers asking the students not to use the VSCode remote server plugin.
2. I ran `ncdu /home` in front of the class and demonstrated how, without exception, everyone with >100MB of disk usage on the class server was a VSCode user.
3. I set a user process limit of 45, because the VSCode RAT (somehow) uses ~50 Node processes. When students inevitably ignored the motd and the in-class warning, they hit the process cap and had to ask us to kill their processes in order to get back onto the system.
4. I replaced the process limit with a script that kills all the .vscode-server RATs every 10 seconds.
My experience participating in Google's program has been pretty good. The reward money is a nice supplement to my grad student stipend. I got a free trip to DEFCON out of it, too.
Yeah, I should really rewrite the README to bucket bugs by behavior instead of server; there's a decent amount of redundancy in the set of discovered bugs.
This makes sense. Nginx is reliable and well-tested, and OLS is neither.
See my recent Shmoocon talk (yes yes, self promotion) for an example of a vulnerability that would be caught immediately in Nginx, but persisted in the OLS codebase for >=10 years: https://yewtu.be/watch?v=aKPAX00ft5s&t=2h29m28s
I've spent the better part of the year messing around with every HTTP server I could get my hands on. I would not recommend OpenBSD httpd.
It's supposed to be simple, and it is simple compared to Apache, but it also has way fewer eyes on it. Ultimately, a big pile of string-handling C is likely to have some problems. There was a trivial-to-exploit server-crashing segfault in httpd's FastCGI implementation that was only fixed in the last month or so. There were also recent issues related to null bytes and line feeds in headers causing strange, exploitable misinterpretations of incoming requests in relayd. Much of this is now fixed, but I wouldn't be at all surprised if there were more low-hanging fruit remaining.
If you're looking for a lightweight HTTP server written in C, I recommend Lighttpd. It's older, more widely-used, and more standards-compliant. I'm not trying to dump on OpenBSD; I run it on my primary laptop. I just wouldn't use their HTTP tools for anything mission-critical yet.
We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.
This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.
My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.
In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.
It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.