A lot of the time once you get into multi-gig+ territory the answer isn't "make the kernel faster," it's "stop doing it in the kernel."
You end up pushing the hot path out to userland where you can actually scale across cores (DPDK/netmap/XDP style approaches), batch packets, and then DMA straight to and from the NIC. The kernel becomes more of a control plane than the data plane.
PF/ALTQ is very much in the traditional in-kernel, per-packet model, so it hits those limits sooner.
I think a more accurate version of this is: unit tests were not only per-method but also per functionality. This was often called BDD (Behavior Driven Development), e.g. Ruby's cucumber. Your intuition here is correct though.
Since the 90s, New Zealand laws have been written in clear, modern, accessible English. The end result is the broader population understands it more and can also reason about it while it’s up for debate before being passed.
I think the ambiguity in the first two amendments has to do more with the specific text rather than plain English itself being deficient.
Sometimes it’s done to fit into an existing tool/database that has a preexisting limit. Or when the hash is used only as a locator rather than for integrity.
I'm guessing the decision comes down to ease of use for people to participate in mirroring. My underestanding is IPFS tends to require more infrastructure, and still requires someone to pin the data.
Many bittorrent clients let you click a button to continue seeding the data over time.
On the first point, OPA is much older than OpenFGA. To really illustrate the point, OPA became a graduated project about a year before OpenFGA had their first code drop in the public GitHub repo. The OpenFGA people are aware of OPA and I'm sure they learned from the tradeoffs OPA made.
To the main point, what you described reflects the current trends of authorization. Define a data model, define data that adheres to that model, write declarative rules that consume that model, make a decision based on those rules.
Where things really start to differ is the kind of data that they bind against and how do you write rules. E.g. OPA is often used for either ABAC (Attribute) or RBAC (Roles) while OpenFGA is looking at ReBAC (Relationships). Each has their complexity tradeoffs, depending on the system being implemented. How easy or difficult a system makes these kinds of checks has a significant impact on how you write policies.
There are some applications where the ability to vectorize the headers and operate on them with SIMD help. These types of apps tend to pin a full core to do only packet processing though. Also, syscall are expensive. A lot of work is going into making the APIs async while avoiding syscalls.
It depends on what you are trying to do though. I don’t think the kernel has an easy path to operating on a set of packet headers as a vector at this point. Not saying it can’t happen, but it’s an area where user space is already ahead.
For reference, there was a previous test that demonstrated 40gbps with ipsec between two pods on separate nodes in k8s where the encap/decap achieved 40gbps which was the line rate for the Intel NICs used.
I do agree that io_uring will negate the need for DPDK for many use cases though, it will likely be a much simpler path and more secure path than DPDK.
Interestingly, the fastest CPU based network switches tend to do full kernel bypass. The kernel is generally slow compared to OVS and VPP, especially when they traverse over something like DPDK.
If you secure a loan for or lease a car, isn’t insurance mandatory for completion of the transaction? If so, how are most people still driving Kias off the lot if they can’t get insurance?
Ora are people getting insurance and finding their policies unrenewable?
Agreed, one way to help mitigate this is to establish Layer 7 security controls, rather than implicitly trust the network. Tailscale shouldn't be the sole security control in any environment.
Found a gap, Linux Foundation's FD.io's VPP (a high performance network virtual switch) has native wireguard support as well, all in userspace. Support here means you can do full kernel bypass from the app all the way down to the NIC card (e.g. via DPDK).
Something to consider, if you are IO constrained, compression may speed up reads because you shift some of the cost of IO to the CPU.
Ultimately, you'll need to measure this to know for sure, and those results will likely only be valid on a given hardware configuration.
OverlayFS also has a "copy_up" function, where the file is copied at the initial write. Once the copy is done, I'd expect write access to be fast. Again, you'll need to measure this.
The setup could probably look like:
container read/write -> OverlayFS([mutable fs as overlay] -> [squashfs layer as underlay] -> [squashfs layer as underlay])
Been a while since I looked. My understanding is many of these techniques such as SMPC are useful only when operations are linear (eg no exponents or tan functions) and on simple branches. Others like homomorphic encryption require operations on the whole data set.
I think the best best for some of these workflows will be differential privacy on large datasets with strict privacy budgets.
Still highly valuable, but make sure you understand the limitations and risks if you use these techniques.
You end up pushing the hot path out to userland where you can actually scale across cores (DPDK/netmap/XDP style approaches), batch packets, and then DMA straight to and from the NIC. The kernel becomes more of a control plane than the data plane.
PF/ALTQ is very much in the traditional in-kernel, per-packet model, so it hits those limits sooner.