I discovered this bash trick by chance when I was once trying to healthCheck the Envoy's official OCI image container which didn't include curl or wget while forcing the envoy admin interface to listen on localhost which breaks the traditional k8s httpGet checks.
Honestly I am happy that more remote access products are using QUIC, not WireGuard, for tunneling and realizing its technical benefits (e.g. AES hardware acceleration, dynamic endpoints, custom auth with JWT or mTLS, FIPS compliance, traffic masquerading as HTTP/3, etc.). I am a big fan of QUIC myself and I implemented it long ago in Octelium, which is a similar remote access product that's more centered around access control and zero trust rather than P2P connectivity. I believe QUIC should be the future of tunneling, especially when it comes to business and enterprise remote access use cases. Congrats on launching an I wish you the best of luck.
Yes, every resource that needs to be protected is represented by a "Service" that's implemented as a L7-aware identity-aware proxy in the Octelium Cluster, which is a distributed system that's running on top of a k8s cluster. Users simply access the protected resource/upstream through the Cluster, namely the Service, from a data-plane perspective, and the Service/identity-aware proxy does authentication/authorization/routing/visibility on a per-request basis. This upstream could be an internal resource directly accessible by the Cluster, or remotely behind NAT, or simply publicly protected SaaS resource (e.g. API protected by an access token, SaaS database protected by a password, etc.). You can read more about how Octelium works here https://octelium.com/docs/octelium/latest/overview/how-octel...
Thank you, I haven't accepted any contributions so far primarily because of this reason but things might change in the future. As mentioned in the README and docs, Octelium is designed specifically for self-hosting so the commercial side of the project is simply confined to commercial AGPLv3-alternative licensing, support, and other very enterprise-y/customized features such as SCIM, SIEM to specific providers, etc...
Well, yes, Octelium is technically a VPN from a layer-3 perspective since it uses WireGuard/QUIC tunneling, but the tunnel doesn't directly terminate to the destination like in VPNs but instead to an identity-aware proxy that does authentication and L7-aware authorization on a per-request basis with policy-as-code via CEL/OPA. From an architecture perspective, I assume it's closer to ZTNAs such as Cloudflare Access and Teleport than to traditional VPNs, even though it operates as one for the clien-based access mode. However, unlike VPNs, it does provide clientless/BeyondCorp access too as it's intended to operate as a more generic/unified access platform (e.g. API/AI/MCP gateway, ngrok-alternative, PaaS-like platform, etc.) rather than just a VPN.
(Shamless plug) I am also working on a similar FOSS, self-hosted project called Octelium https://github.com/octelium/octelium that you might find interesting if you are interested in this space. Octelium is, however, more of a generic/unified zero trust secure access platform that can operate as a remote access VPN, a ZTNA platform, API/AI/MCP gateway, a PaaS, an ngrok-alternative and a homelab infrastructure. It provides unified client-based as well as clientless access for both humans and workloads; dynamic identity-based secretless access (e.g. access to HTTP/gRPC/k8s upstreams without sharing API keys and access tokens, SSH without distributing passwords/private keys, postgres/MySQL databases without sharing passwords, etc.); dynamic L7-aware, identity-based access control ABAC via CEL and OPA as well as dynamic routing to upstreams via policy-as-code; native Passkey login/WebAuthn/TOTP MFA and support for OIDC/SAML IdPs, OpenTelemetry-native L7-aware visibility and auditing; clientless access via OAuth2 for workloads, WireGuard and QUIC tunneling with dual-stack and automatic private DNS, including in rootless mode; passwordless SSH'ing into containers and IoT without SSH servers; deploying and securing access to containers; declarative k8s-like management with horizontal scalability among other features. You can read more in the README if you're interested.
I wonder whether the current huge funding in AI will ever lead to a revolution in computer architecture. Modern PCIe/CXL is already starting to blur the difference between memory and I/O. Maybe the future is going to be that CPUs, RAM, storage devices, GPUs and other devices are going to directly address one another like a mesh network. Maybe the entire virtual memory model will change to include everything to be addressed via a unified virtual memory space from a process/CPU perspective with simple read/write syscalls that translate into network packets flowing between the CPU and the destination (e.g. RAM/GPU/NVMe) and vice versa.
Is it? I honestly kinda believe that etcd is probably the weakest point in vanilla k8s. It is simply unsuitable for heavy write environments and causes lots of consistency problems under heavy write loads, it's generally slow, it has value size constraints, it offers very primitive querying, etc... Why not replace etcd altogether with something like Postgres + Redis/NATS?
TCP over WireGuard is unavoidable because that's the whole point of tunneling. But TCP over WireGuard over QUIC just doesn't make any sense, neither from performance nor from security perspective. Not to mention that with every additional tunneling layer you need to reduce the MTU (which is already a very restricted sub-1500 value without tunneling) of all inner tunnels.
There is actually. A way more interesting re-implementation of a popular L7 is SSH over QUIC. SSH has to implement its own mutual authentication and transport embedded in the protocol implementation since it operates on top of plaintext TCP, but with QUIC you can just offload the authentication (e.g. JWT bearer tokens issued by IdPs verified at L7 or automatically via mTLS x509 certs) and transport parts to QUIC and therefore have a much more minimal implementation.
You can just do that by using QUIC-based tunneling directly instead of using WireGuard-over-QUIC and basically stacking 2 state machines on top of one another.
WireGuard-over-QUIC does not make any sense to me, this lowers performance and possibly the inner WireGuard MTUs. You can just replace WireGuard with QUIC altogether if you just want obfuscation.
While WireGuard makes every sense for an FPGA due to its minimal design, I wonder why there isn't much interest in using QUIC as a modern tunneling protocol, especially for corporate use cases. QUIC already provides an almost complete WireGuard-alternative via its datagrams that can be easily combined with TUN devices and custom authentication schemes (e.g. mTLS, bearer tokens obtained via OAuth2 and OIDC authentication, etc...) to build your own VPN. While I am not sure about performance, at least when compared to kernel-mode WireGuard, since QUIC is obviously a more complex state machine that's running in userspace and it depends on the implementation and optimizations offered by the OS (e.g. GRO/GSO), QUIC isn't just a yet another tunneling protocol, it actually offers lots of benefits such as working well with dynamic endpoints with DNS instead of just using static IP addrs, it uses modern TLSv1.3 and therefore it's compliant with FIPS for example, it uses AES which can be accelerated by the underlying hardware (e.g. AES-NI), it currently has implementations in almost every major programming language, it can work well in the future with proxies and load balancers, you can bring your own custom, more fine-grained authentication scheme (e.g. bearer tokens, mTLS, etc...), it masquerades as just another QUIC/HTTP3 traffic that's used by almost all major websites now and therefore less susceptible to dropping by any nodes in between, and other less obvious benefits such as congestion control and PMTUD.
I am working on Octelium https://github.com/octelium/octelium a FOSS unified zero trust secure access platform that is flexible enough to operate as a modern zero-config remote access VPN, a Zero Trust Network Access (ZTNA)/BeyondCorp platform, an API/AI/LLM gateway, an infrastructure for MCP gateways and agentic AI architectures/meshes, a PaaS-like platform, ngrok alternative, and even as a homelab infrastructure. It is basically a unified, generic, Kubernetes-like, zero trust architecture (ZTA) for secure access and deployment, that can operate in many human-to-workload, workload-to-workload, and hybrid environments.
I actually did a SHOW HN exactly 3 months ago and received lots of invaluable critique regarding how dense, overwhelming and unreadable the docs and repo README were. I've actually spent a lot of time trying to improve the quality of the docs and README since then. I'd love to receive any feedback, negative included, regarding the current overall quality of the docs and README from whoever is interest in that space.
As for (1) zero trust architectures don't really expose complete subnets to users, they only expose individual resources and enforce access control at each of these resources individually. As for (2) As I said above, this can actually be done with Octelium if your VPNs are able to run as SOCKS5 proxies. You can create as many Services, each representing a separate VPN/exit node SOCKS5 proxy, and then use Octelium's stable private DNS to connect to the different SOCKS5 proxies/Services depending on your domain-based rules.
Thank you again. I meant by "negative" that they are critical as opposed to insulting. As I mentioned before in this thread, I usually don't find negative/critical comments offensive at all. Believe me, the last thing that I would want for an open source project that I have been working on for years now is to show it to people, especially technical people, with poor wording or hard-to-understand terms. That was never intentional by me. In fact, the reason that README is too long is that I wanted to explain all the features and use cases in detail while add additional links to the docs for those who want to understand more which probably made it even more overwhelming for those unfamiliar with zero trust architectures. I will definitely do my best to improve the README and docs with time as I get more honest feedback like in this thread. Thank you.
Thank you really for your detailed comment. I will try to answer your questions and please don't hesitate to ask in the Slack/Discord channels or contact emails later if the answers here weren't clear enough to you.
1. Octelium Services and Namespaces are not really related or tied to Kubernetes services and namespaces. Octelium resources in general are defined in protobuf3 and compiled to Golang types, and they are stored as serialized JSON in the Postgres main data store simply as JSONB. That said, Octelium Services in specific are actually deployed on the underlying k8s cluster as k8s services/deployments. Octelium resources visually look like k8s resouces (i.e. they both have the same metadata, spec, status structure), however Octelium resources are completely independent of the underlying k8s cluster; they aren't some k8s CRDs as you might initially guess. Also Octelium has its own API server which do some kind of REST-y gRPC-based operations for the different Octelium resources to the Postgres via an intermediate component called the rscServer. As I said, Octelium and k8s resources are completely separate regardless of the visual YAML resemblance. As for managed containers, you don't really have to use it, it's an optional feature, you can deploy your own k8s services via kubectl/helm and use their hostnames as upstreams for Octelium Services to be protected like any other upstream. Managed containers are meant to automate the entire process of spinning up containers/scaling up and down and eventually cleaning up the underlying k8s pods and deployments once you're done with the owner Octelium Service.
2. Secret management in Octelium is by default stored in plaintext. That's a conscious and deliberate decision as declared in the docs because there isn't any one standard way to encrypt Secrets at rest. Mainline Kubernetes itself does exactly the same and provides a gRPC interface for interceptors to implement their own secret management (e.g. HashiCorp Vault, AWS KMS/GCP/Azure offerings, directly to software/hardware-based HSMs, some vault/secret management vendor that I've never heard of, etc...). There is simply no one way to do that, every company has its own regulatory rules, vendors and standards when it comes to secret management at rest. I provided a similar gRPC interface for everybody to intercept such REST operations and implement their own secret management according to their own needs and requirements.
3. Octelium has Session resources https://octelium.com/docs/octelium/latest/management/core/se... Every User can have one or more Sessions, where every Session is represented by an opaque JWT-like access token, which are used internally by the octelium/octeliumctl clients after a successful login, they are also set as a HTTPOnly cookies for BeyondCorp browser-based Sessions and they are used directly as bearer tokens by WORKLOAD Users for client-less access to HTTP-based resources. You can actually set different permissions to different Users and also set different permissions for different Sessions for the exact same User via the owner Credentials or even via your Policies. OAuth2 client credential flow is only intended for WORKLOAD Users. Human Users don't really use OAuth2 client credentials at all. They just login via OIDC/SAML via the web Portal or manually via issued authentication token which is not generally recommended for HUMAN Users. OAuth2 is meant for WORKLOAD Users to securely access HTTP-based Services without using any clients or SDKs. OAuth2 scopes are not really related to zero trust at all as mentioned in the docs. OAuth2 scopes are just an additional way for applications to further restrict their own permissions, not add new ones which are already set by your own Policies.
4. An Octelium Cluster runs on top of a k8s cluster. In a typical multi-node production k8s cluster, you should use at least one node for the control plane and another separate node for the data-plane and scale up if your needs grow. A knowledge with Kubernetes isn't really required to manage an Octelium Cluster. As I mentioned above, Octelium resources and k8s resources are completely separate. One exception when it comes to directly having to deal with the underlying k8s cluster, is setting/updating the Cluster TLS certificate, such cert needs to be fed to the Octelium Cluster via a specific k8s secret as mentioned in the docs. Apart from that, you don't really need to understand anything about the underlying k8s cluster.
5. To explain Octelium more technically from a control plane/data plane perspective: Octelium does with identity-aware proxies similarly to what Kubernetes itself does with containers. It simply builds a whole control plane around the identity-aware proxies which themselves represent and implement the Octelium Services, and automatically deploys them whenever you create an Octelium Service via `octeliumtl apply` commands, scales them up and down whenever you change the Octelium Service replicas and eventually cleans the up whenever you delete the Octelium Service. As I said it's similar to what k8s itself does with containers where your interactions with the k8s API server whether via kubectl or programmatically is enough for the k8s cluster to spin up the containers, do all the CRI/CNI work for you automatically without even having to know or care which node actually runs a specific container.
As for the suggestions:
1. I am not really interested in SaaS offerings myself and you can clearly see that the architecture is meant for self-hosting as opposed to having a separate cloud-based control plane or whatever. I do, however, offer both free and professional support for companies that need to self-host Octelium on their own infrastructure according to their own needs.
2. I am not sure I understand this one. But if I understood you correctly, then as I said above, Octelium resources are completely different and separate from k8s resources regardless of the visual YAML resources. Octelium has its own API server, rscServer, data store, and it does not use CRDs or mess with k8s data store whether it be etcd or something else.
Thank you really for your kind comment. Most of the links regarding how Octelium works, the quick management and installation guides, the examples (e.g. API/AI/MCP gateways, etc...) were actually included in the README itself. However, most of the criticism was supposedly coming from the terms used in the README. I was already assuming that the users are somewhat familiar with zero trust and zero trust architectures. Maybe that was the problem.