NGINX Proxy Manager(github.com)
github.com
NGINX Proxy Manager
https://github.com/NginxProxyManager/nginx-proxy-manager
12 comments
I have a bug where every time my server restarts and Nginx Proxy Manager starts up, it gives all my proxy hosts incorrect SSL certificates. In order to get my services working again I have to open NPM, manually open each host, and press edit, and (without making changes), press save.
This bug has existed for a year and a half in Github without any response from the developers.
As much as I like NPM, I need to move onto something more reliable.
This bug has existed for a year and a half in Github without any response from the developers.
As much as I like NPM, I need to move onto something more reliable.
I also prefer to run Nginx as vanilla as possible, but having automatic renewal of certificates is something which is really nice.
I looked around at some of the more popular solutions, but didn't like the mounting of the docker socket required by [acme-companion][1], or the lack of bootstrapping capability of [nginx-certbot][2], so I made [something][3] that solved both of those issues. A self contained Docker container that is able to populate the certificate request form what you write in your Nginx configuration files.
Please check it out if you have time, and I will gladly take any feedback if you have any!
[1]: https://github.com/nginx-proxy/acme-companion [2]: https://github.com/wmnnd/nginx-certbot [3]: https://github.com/JonasAlfredsson/docker-nginx-certbot
[1]: https://github.com/nginx-proxy/acme-companion [2]: https://github.com/wmnnd/nginx-certbot [3]: https://github.com/JonasAlfredsson/docker-nginx-certbot
Mine just stops renewing certificates so I stopped fighting it. Haven't replaced it with vanilla Nginx yet but that's a project on my todo list.
You might also consider using the fully OSS Traefik for a load balancer and proxy. It arrives to be friendly and approachable.
https://github.com/traefik/traefik
https://github.com/traefik/traefik
You can try nginx-proxy, its similar to trafeik but based in nginx and a bit easier.
https://github.com/nginx-proxy/nginx-proxy
https://github.com/nginx-proxy/nginx-proxy
[deleted]
I happened to need to do this in NixOS yesterday, and look how easy it is:
security.acme = {
acceptTerms = true;
email = "[email protected]";
};
services.nginx = {
enable = true;
recommendedGzipSettings = true;
recommendedOptimisation = true;
recommendedProxySettings = true;
recommendedTlsSettings = true;
virtualHosts."whatever.net" = { default = true; enableACME = true; addSSL = true; locations."/".proxyPass = "http://127.0.0.1:9955/"; };
};
here's my caddy config for a reverse proxy with TLS:
sub.domain.com { reverse_proxy localhost:8080 }
sub.domain.com { reverse_proxy localhost:8080 }
You don't even need a config file for that!
$ caddy reverse-proxy --from sub.domain.com --to localhost:8080
You mean
Because caddy currently doesn’t handle DNS names correctly, so you have to duplicate every single virtual hostname config (it’s a long-standing open bug)
[EDIT: Thanks to francislavoie for reminding me about the shorter syntax for this]
sub.domain.com, sub.domain.com. { reverse_proxy localhost:8080 }
Because caddy currently doesn’t handle DNS names correctly, so you have to duplicate every single virtual hostname config (it’s a long-standing open bug)
[EDIT: Thanks to francislavoie for reminding me about the shorter syntax for this]
Actually, we mean:
Trailing dots are complicated and really not worth the complexity it would involve to support them. See https://daniel.haxx.se/blog/2022/05/12/a-tale-of-a-trailing-...
sub.domain.com, sub.domain.com. {
reverse_proxy localhost:8080
}
Trailing dots are complicated and really not worth the complexity it would involve to support them. See https://daniel.haxx.se/blog/2022/05/12/a-tale-of-a-trailing-...
> Can you stop posting about this every single time there's even a vague mention of Caddy on HN? It's tired
It’s a genuine issue I’ve got with Caddy, and if someone recommends Caddy, I’m justified to mention the drawbacks of Caddy.
In this case, a user recommended switching from nginx to Caddy, and nginx users (who expect trailing dot domains to just work) should keep this drawback in mind.
Every time you compare Caddy or Traefik to nginx or apache, you should expect to also see the drawbacks of Caddy and Traefik mentioned.
It’s a genuine issue I’ve got with Caddy, and if someone recommends Caddy, I’m justified to mention the drawbacks of Caddy.
In this case, a user recommended switching from nginx to Caddy, and nginx users (who expect trailing dot domains to just work) should keep this drawback in mind.
Every time you compare Caddy or Traefik to nginx or apache, you should expect to also see the drawbacks of Caddy and Traefik mentioned.
You're not being a productive member of the community. You only complain about Caddy, instead of giving solid technical arguments for your case. We have presented ample evidence, experience, and technical reasoning to support our argument, but you continue to ignore the logical, technical, and complex arguments and instead complain to rack up fake Internet points here. We want to make a better product, but you need to convince us that your way is better than all the evidence, experience, and technical reasons we have already presented.
See that’s exactly the issue. You expect me to do work and spend time just to convince you to be standards-compliant.
How can I trust caddy complies with standards in other areas if caddy isn’t willing to follow the standard (which really doesn’t take much work) in this area? It’d be different if the de facto and the de jure standard diverged, as e.g., with IRC, but that’s not the case. nginx, Apache, IIS, Google’s GWS, they all follow the standard exactly, it’s just traefik and caddy that don’t.
It’s not an issue of this situation in of itself, it’s an issue of eroding trust. It’s the famous brown m&ms.
P.S.: Your arguments explain why implementing this would be a lot of work (keeping the ticket open as known issue is something I’d be fine with), but they can’t justify ignoring the standard and closing it as wontfix.
How can I trust caddy complies with standards in other areas if caddy isn’t willing to follow the standard (which really doesn’t take much work) in this area? It’d be different if the de facto and the de jure standard diverged, as e.g., with IRC, but that’s not the case. nginx, Apache, IIS, Google’s GWS, they all follow the standard exactly, it’s just traefik and caddy that don’t.
It’s not an issue of this situation in of itself, it’s an issue of eroding trust. It’s the famous brown m&ms.
P.S.: Your arguments explain why implementing this would be a lot of work (keeping the ticket open as known issue is something I’d be fine with), but they can’t justify ignoring the standard and closing it as wontfix.
> (which really doesn’t take much work)
Except it actually does take a lot of work, because it's not as simple as you make it out to be. It's very complex.
Look at the blog post I linked above from the author of cURL, looks at how much time was wasted just to support that "feature". If trailing dots weren't supported, there would be less code, and less headaches for everyone.
And either way, your argument that because we aren't "standards-compliant" here means we won't follow standards elsewhere is the slippery slope fallacy. Show us evidence of where we aren't standards compliant, then we'll talk.
Except it actually does take a lot of work, because it's not as simple as you make it out to be. It's very complex.
Look at the blog post I linked above from the author of cURL, looks at how much time was wasted just to support that "feature". If trailing dots weren't supported, there would be less code, and less headaches for everyone.
And either way, your argument that because we aren't "standards-compliant" here means we won't follow standards elsewhere is the slippery slope fallacy. Show us evidence of where we aren't standards compliant, then we'll talk.
I originally wrote a detailed response to this, but instead I’ll just say this:
It’s one thing to limit the scope of your project. cURL is trying to support every de facto and de jure standard out there, your project doesn’t have to. A lot of tail-end features have very diminishing returns. My own projects don’t do it either.
But it’s something entirely else if you just close the issues as WONTFIX, claim they’re not part of a standard (until you’re proven wrong), then claim it’s not used in the real world (it is, otherwise cURL wouldn’t implement it), then verbally attack users who point out that you’re not supporting it.
Just accept it, acknowledge that you’re not supporting it and that it’s not part of your scope. Own your faults, just be honest.
I’ve got far more persistent and far more annoying users complaining about missing features in Quasseldroid on every release or post for the past decade, but that’s no reason to get angry with them.
It’s one thing to limit the scope of your project. cURL is trying to support every de facto and de jure standard out there, your project doesn’t have to. A lot of tail-end features have very diminishing returns. My own projects don’t do it either.
But it’s something entirely else if you just close the issues as WONTFIX, claim they’re not part of a standard (until you’re proven wrong), then claim it’s not used in the real world (it is, otherwise cURL wouldn’t implement it), then verbally attack users who point out that you’re not supporting it.
Just accept it, acknowledge that you’re not supporting it and that it’s not part of your scope. Own your faults, just be honest.
I’ve got far more persistent and far more annoying users complaining about missing features in Quasseldroid on every release or post for the past decade, but that’s no reason to get angry with them.
> then verbally attack users who point out that you’re not supporting it
I didn't verbally attack you. I just asked that you stop bringing it up on every thread where Caddy is mentioned. It causes us stress and wastes our time to have to re-explain ourselves the same way every time. Yes I used an aggressive tone, but it's because we're frustrated to continue hearing it. We always have in the back of our mind "is that one person going to post about trailing dot domains _again_?"
> Just accept it, acknowledge that you’re not supporting it and that it’s not part of your scope.
We did that. That's what a WONTFIX is.
I didn't verbally attack you. I just asked that you stop bringing it up on every thread where Caddy is mentioned. It causes us stress and wastes our time to have to re-explain ourselves the same way every time. Yes I used an aggressive tone, but it's because we're frustrated to continue hearing it. We always have in the back of our mind "is that one person going to post about trailing dot domains _again_?"
> Just accept it, acknowledge that you’re not supporting it and that it’s not part of your scope.
We did that. That's what a WONTFIX is.
Then why do you comment on it at all, if it causes you so much stress? I didn't claim anything wrong, did I? I didn't post in an unfriendly tone either?
It sounds like you don't accept that it's a shortcoming, instead trying to justify and excuse it.
You don't see the authors of nginx angrily complaining at you every time you comment on a post about their httpd. (Which this thread was about originally, after all)
It sounds like you don't accept that it's a shortcoming, instead trying to justify and excuse it.
You don't see the authors of nginx angrily complaining at you every time you comment on a post about their httpd. (Which this thread was about originally, after all)
Just a quick query from someone who has beginner level experience in NGINX and Caddy. Doesn't this PR [1] for Traefik imply that trailing dots for FQDNs are fixed?
[1] https://github.com/traefik/traefik/pull/4763
[1] https://github.com/traefik/traefik/pull/4763
You can match hostnames with trailing dots in Caddy, too (and have been able to for years).
Just gonna be honest and just flat out state that I'm kind of lost on what's going on.
What is kuschku arguing against in ELI5 terms?
What is kuschku arguing against in ELI5 terms?
That it’s neither supported by default, nor mentioned in the documentation. Which leads to people misconfiguring their webservers, e.g., https://caddyserver.com./ vs https://httpd.apache.org./
It is supported by default. Out of the box you can configure it this way.
It is not _configured_ by default. There is an important distinction.
It's not mentioned in documentation because the documentation [1] does not provide an _exhaustive_ list of the kinds of addresses that Caddy can be configured to serve.
1. https://caddyserver.com/docs/caddyfile/concepts#addresses
It is not _configured_ by default. There is an important distinction.
It's not mentioned in documentation because the documentation [1] does not provide an _exhaustive_ list of the kinds of addresses that Caddy can be configured to serve.
1. https://caddyserver.com/docs/caddyfile/concepts#addresses
[deleted]
That it doesn't work implicitly in Caddy, without explicitly configuring Caddy with trailing dots.
Hey, at least caddy does case-insensetive hostname match, so you don't need to repeat the domain 2^12 times </s>
Seriously though - that's a pretty bizarre bug, absent from both nginx and apache httpd. And that attitude in sibling comment doesn't help either.
Seriously though - that's a pretty bizarre bug, absent from both nginx and apache httpd. And that attitude in sibling comment doesn't help either.
It’s not just the sibling comment that has this attitude, it’s also mholt himself displaying the same attitude. This hostile opinionated approach is part of what has kept me from using Caddy for the past few years (I went back to nginx at first, and once I started using k8s I also chose nginx as ingress controller there).
:/
:/
i have to say it is pretty weird to persistently complain about software you haven't used for years lol. i have also never used a trailing dot in a domain in 20+ years.
1. Just because I’m not running servers using caddy doesn’t mean I’m not affected by this bug, because some of my clients intentionally always postfix addresses they access with a dot to avoid internal DNS resolution.
2. I’d love to use caddy. I’m especially interested in using it for my status page, as that needs to be hosted outside my normal cluster. Ideally it should be an absolutely minimal setup (so nothing except caddy + status page, with status monitors on other servers reporting back to the status page). This would be the perfect use case for caddy, but right now I’m using an overcomplicated nginx setup because I don’t feel like I can trust caddy.
2. I’d love to use caddy. I’m especially interested in using it for my status page, as that needs to be hosted outside my normal cluster. Ideally it should be an absolutely minimal setup (so nothing except caddy + status page, with status monitors on other servers reporting back to the status page). This would be the perfect use case for caddy, but right now I’m using an overcomplicated nginx setup because I don’t feel like I can trust caddy.
That's super interesting. Automatic ACME is not a feature in nginx right? So the maintainers of the nginx NixOS package have built this integration themselves just to make NixOS more powerful for this use case?
> Automatic ACME is not a feature in nginx right?
Correct. You need separate, less reliable tooling to use ACME with nginx.
Correct. You need separate, less reliable tooling to use ACME with nginx.
Doesn’t the built-in tooling only really provide an advantage if you’re using the http or sni challenges?
If you want to use wildcard certs via the dns-01 challenge (in my case via rfc2136), the entire challenge runs out of band anyway, so there’s no difference in reliability. (At least in my tests so far, though support for the RFC 2136 standard in caddy and/or plugins is quite poor afaict)
If you want to use wildcard certs via the dns-01 challenge (in my case via rfc2136), the entire challenge runs out of band anyway, so there’s no difference in reliability. (At least in my tests so far, though support for the RFC 2136 standard in caddy and/or plugins is quite poor afaict)
How was the Nix experience? Did you try just nix or also NixOS?
I think you replied to the wrong comment
No, I'd like to know why he called it 'less reliable'.
He must have used it on nix(linux) or nix(darwin) or NixOS..
The comment wasn't about Nix, it was about ACME clients paired with Nginx being less reliable than having the ACME client built into your server (e.g. how Caddy does it). The comment was by Matt Holt, the author of Caddy.
I used NGINX Proxy Manager for ages for my VPS where I host some side projects.
But recently I switched to Caddy Docker Proxy (https://github.com/lucaslorentz/caddy-docker-proxy) and I'm happy with the switch.
Seems easier to set it up directly in the docker compose files rather than have an extra interface. I guess you can do more in NPM but for me it's enough.
Seems easier to set it up directly in the docker compose files rather than have an extra interface. I guess you can do more in NPM but for me it's enough.
Don't Traefik and Caddy already do exactly that with their official containers ? What makes NGINX Proxy Manager special in that instance ?
The setup and UI for NGINX Proxy Manager is a lot more noob-friendly than Traefik. Haven't used Caddy before, so can't compare there.
If you have the skills to use Traefik, there's no reason to use NGINX Proxy Manager.
If you have the skills to use Traefik, there's no reason to use NGINX Proxy Manager.
This has a little auth system with web UI where you can login in, create users, give them access to certain subdomains, etc. for a simple single sign on experience. Traefik and Caddy just give you the reverse proxy and auth but none of the user management or login UI.
As of Caddy v2.5.1, you can easily pair Caddy with Authelia to provide auth for all your services!
Does Traefik or Caddy have UI-built-in support for Let's Encrypt? If not, then perhaps that makes this special. If so, then probably nothing else except a relatively user-friendly UI.
No UI in Caddy, but it's only two lines of config to set up a domain with Automatic HTTPS and a reverse proxy to one of your apps. Don't really need a UI for that.
I love traefik, but getting started is kinda rough if you’re not quite technical. There are like three different places to put configuration, and even getting informative error messages out of the thing it’s not exactly trivial.
But once it works, it works.
But once it works, it works.
TLDR; It's significantly simpler to configure NGINX Proxy Manager if you have a small amount of config to do in a home lab scale setup.
I've used Apache, Traefik, and now NGINX Proxy Manager for my self hosted development environment. I've also used HAProxy in front of them as a (TCP) load balancer using a single IP address so I can run multiple backends that all deal with their own TLS. Caddy is my preferred choice for spinning up a quick webserver.
I just started using NGINX Proxy Manager, so I haven't discovered the things I don't like yet, but I can tell you why I switched to it.
Apache and Certbot were what I used initially and I should have kept using it. NGINX Proxy Manager isn't a lot different and I've spent so much time switching through solutions that I should have just managed Apache by hand. I'd be way ahead in terms of time spent.
I didn't even make a serious attempt at Caddy because I want to use wildcard TLS certificates with Cloudflare DNS challenges. The DNS plugins aren't built in to the official Docker container and finding instructions for setting up a single wildcard certificate that could be used on multiple sites was a pain.
For example, the page for wildcard certs [1] links to a simple config example [2] and a json config example [3].
The first example is like the start of the owl drawing joke where it starts out "draw a circle". It's easy to understand, but it's too simple. There are two config blocks, foo and bar, that both contain a `respond` directive. One of those could show a simple reverse proxy config and it would help a ton for people trying to learn to use Caddy for TLS termination. The second example is like the "draw the rest of the owl" part of the joke. It dumps you into a full fledged, comprehensive json config example where you'll drown in info if you don't already know how everything works.
Caddy can also be a bit of a confusing black box. For example, I recently ran across the below config. Assume you don't know anything about Caddy and tell me how it's matching requests to route the web socket traffic to the correct backend.
Here's a snippet from my HAProxy config to contrast the Caddy config:
Caddy is probably the best option for a simple, top down config because most of the sites would be super simple and it would be terse / easy to skim once you learn it. I think I would have ended up using it if I didn't have to deal with the hassle of getting the Cloudflare DNS plugin set up on my own.
I've used Traefik the most and I don't like it now. I remember the config for wildcard certificates being really hard to figure out. For http challenges you must tell services which cert resolver to use. For wildcard certificates you must omit the cert resolver config on all configs except one of them. That wasn't documented anywhere when I originally did my setup. That's v2. The v1 config was much more sane, but I don't recall exactly how it worked.
The second issue I have is the same as what you'd get with things like `caddy-docker-proxy`. Even though I have a single container with a wildcard cert doing TLS termination, I have config strewn across a dozen different places. It's really hard to get an overview of what's going on and I have at least 4 lines of config per host where I need to make up parts of the key. Ex:
The appeal of NGINX Proxy Manager is that for simple reverse proxying, which is 95% of what I've got, it's really simple. I don't have to learn some newly invented configuration language until I need an advanced config. I don't know NGINX at all and it took me about 15m to get something running behind NGINX Proxy Manager. Compare that to Caddy where it took me a couple hours just to figure out what I'd need to install/configure just to get a wildcard TLS cert or Traefik where I wasted 10s of hours getting to the point where I could use it with the main (negative) reward being that now I have something that's a pain to maintain.
With NGINX Proxy Manager you get an overview of all your TLS termination and certificates in a single place, it only takes a few minutes to get up and going, and there's an escape hatch to provide advanced configs if needed. Plus, there are tons of config examples and troubleshooting resources for NGINX.
1. https://caddyserver.com/docs/automatic-https#wildcard-certif...
2. https://caddyserver.com/docs/caddyfile/patterns#wildcard-cer...
3. https://caddyserver.com/docs/json/
4. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Up...
I've used Apache, Traefik, and now NGINX Proxy Manager for my self hosted development environment. I've also used HAProxy in front of them as a (TCP) load balancer using a single IP address so I can run multiple backends that all deal with their own TLS. Caddy is my preferred choice for spinning up a quick webserver.
I just started using NGINX Proxy Manager, so I haven't discovered the things I don't like yet, but I can tell you why I switched to it.
Apache and Certbot were what I used initially and I should have kept using it. NGINX Proxy Manager isn't a lot different and I've spent so much time switching through solutions that I should have just managed Apache by hand. I'd be way ahead in terms of time spent.
I didn't even make a serious attempt at Caddy because I want to use wildcard TLS certificates with Cloudflare DNS challenges. The DNS plugins aren't built in to the official Docker container and finding instructions for setting up a single wildcard certificate that could be used on multiple sites was a pain.
For example, the page for wildcard certs [1] links to a simple config example [2] and a json config example [3].
The first example is like the start of the owl drawing joke where it starts out "draw a circle". It's easy to understand, but it's too simple. There are two config blocks, foo and bar, that both contain a `respond` directive. One of those could show a simple reverse proxy config and it would help a ton for people trying to learn to use Caddy for TLS termination. The second example is like the "draw the rest of the owl" part of the joke. It dumps you into a full fledged, comprehensive json config example where you'll drown in info if you don't already know how everything works.
Caddy can also be a bit of a confusing black box. For example, I recently ran across the below config. Assume you don't know anything about Caddy and tell me how it's matching requests to route the web socket traffic to the correct backend.
localhost {
reverse_proxy website:8080
@rendezvous {
header Connection *Upgrade*
header Upgrade websocket
}
reverse_proxy @rendezvous rendezvous:4000
}
set-header 'Connection: Upgrade'
set-header 'Upgrade: websocket'
Here's a snippet from my HAProxy config to contrast the Caddy config:
acl is_certbot path_beg /.well-known/acme-challenge/
use_backend http-example-com if is_certbot
Caddy is probably the best option for a simple, top down config because most of the sites would be super simple and it would be terse / easy to skim once you learn it. I think I would have ended up using it if I didn't have to deal with the hassle of getting the Cloudflare DNS plugin set up on my own.
I've used Traefik the most and I don't like it now. I remember the config for wildcard certificates being really hard to figure out. For http challenges you must tell services which cert resolver to use. For wildcard certificates you must omit the cert resolver config on all configs except one of them. That wasn't documented anywhere when I originally did my setup. That's v2. The v1 config was much more sane, but I don't recall exactly how it worked.
The second issue I have is the same as what you'd get with things like `caddy-docker-proxy`. Even though I have a single container with a wildcard cert doing TLS termination, I have config strewn across a dozen different places. It's really hard to get an overview of what's going on and I have at least 4 lines of config per host where I need to make up parts of the key. Ex:
traefik.http.routers.gitea.tls=true
The appeal of NGINX Proxy Manager is that for simple reverse proxying, which is 95% of what I've got, it's really simple. I don't have to learn some newly invented configuration language until I need an advanced config. I don't know NGINX at all and it took me about 15m to get something running behind NGINX Proxy Manager. Compare that to Caddy where it took me a couple hours just to figure out what I'd need to install/configure just to get a wildcard TLS cert or Traefik where I wasted 10s of hours getting to the point where I could use it with the main (negative) reward being that now I have something that's a pain to maintain.
With NGINX Proxy Manager you get an overview of all your TLS termination and certificates in a single place, it only takes a few minutes to get up and going, and there's an escape hatch to provide advanced configs if needed. Plus, there are tons of config examples and troubleshooting resources for NGINX.
1. https://caddyserver.com/docs/automatic-https#wildcard-certif...
2. https://caddyserver.com/docs/caddyfile/patterns#wildcard-cer...
3. https://caddyserver.com/docs/json/
4. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Up...
FYI for Caddy + cloudflare DNS plugin in Docker, you just need to write a Dockerfile like this (see https://hub.docker.com/_/caddy):
Then in your Caddyfile, you configure the DNS plugin with your Cloudflare API key.
FROM caddy:2.5.1-builder AS builder
RUN xcaddy build --with github.com/caddy-dns/cloudflare
FROM caddy:2.5.1
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
Then in your Caddyfile, you configure the DNS plugin with your Cloudflare API key.
example.com {
tls {
dns cloudflare <API-KEY-HERE>
}
reverse_proxy your-app:8080
}
I appreciate the reply. I took some time to look at your example so I can give some feedback on where I end up when I think about building / maintaining my own image.
My immediate reaction is that the example is nice as a one-off build, but it's much more complex if I need to set up something I can maintain long term. I might be overthinking it, but in the context of thinking about something I can maintain my thought process is below. The questions are mostly rhetorical.
First, what versions am I getting? Does using `2.5.1-builder` result in a customer built binary that's version `2.5.1`? The command usage [1] of the `xcaddy` command says it falls back to the `CADDY_VERSION` environment variable if it's not set explicitly. Since it's not set explicitly, I go looking for that variable in the Dockerfile [2].
That's some templating language I'm not familiar with and I can't track down where the variable gets set, at least not quickly. I'd probably have to spend an hour learning how those templates work to figure it out. To make a quicker, educated guess, it most likely matches the builder version. The docs said the version can be set to any git ref, so I can explicitly set it to v2.5.1 on the command line [3] to be certain.
Now, what version of `caddy-dns/cloudflare` am I getting? The xcaddy custom builds section of the docs [4] says the version can optionally be specified, but it's not specified in the above example. There aren't any tags in the repo, so it's probably building off `master`. The doc says it functions similar to `go get`, but doesn't explain what the differences are and the default behavior isn't explained either.
The docs for `go get` [6] say it can use a revision, so maybe a specific commit can be used for that, but I'd need to test it since I'm not super familiar with Golang.
What other risks come along with building and maintaining my own custom image? I could end up with a subtly broken build that only occurs in my environment. Portability doesn't guarantee compatibility [7] and building custom images increases the risk of compatibility issues beyond what I get with official images (building and running vs just running). That blog post is a really cool read on it's own BTW.
I need to consider the potential for breakage even if it's miniscule because my Docker infrastructure is self hosted and will be sitting behind my custom built Caddy image. If my custom image breaks, I need a guaranteed way of having access to a previous, known good version. This is as simple as publishing the images externally, but adds an extra step since I'll need an account at a registry and need to integrate pushes to that registry into my build.
If I build a custom image, do I let other people I help with the odd tech thing use it or is all the effort for me only? I don't want to become the maintainer of a Docker image others rely on, so I can't even re-use any related config if I help others in the future since they won't have access to the needed image.
To be fair, I also see things I don't like in the NGINX Proxy Manager Dockerfile [7]. The two that immediately jump out at me are things I consider common mistakes. Both require unlucky timing to fail, but can technically cause failure IMO. The first is using `apt-get update` which will exit 0 on failure and has the potential to leave `apt-get install` running against obsolete versions. The second is using `apt-get update` in multiple parts of a multistage build. If I were doing it I'd run `apt-get update` in a base image and avoid it in the builder + runtime images to guarantee the versions stay the same between the build container and the runtime container.
It took me about 1h to work through that and write this comment, so it's not just a matter of building a Docker image and plugging in the config. There's a lot of nuance that goes into maintaining a Docker image (I'm sure you know that already) and not having an image with the DNS plugin(s) baked in is a show stopper for anyone like me that can't justify maintaining their own.
Also, a 4 line Docker file looks nice in terms of being simple, but explicitly declaring or even adding comments describing some of the things I pointed out above can save people a lot of time. Even comments with links to the relevant portions of the docs would be super useful.
My reason for wanting the Cloudflare DNS plugin is that I have some things I want to run 100% locally without ever exposing them to the internet. The desire for wildcard certificates is to keep things from being discoverable via CTLogs.
I hope that's useful feedback. I realize someone bemoaning the difficulty of running your stuff at home lab / small business scale isn't exactly the target audience in terms of picking up customers that pay the bills. Thanks again for the reply / example.
1. https://github.com/caddyserver/xcaddy#command-usage
2. https://github.com/caddyserver/caddy-docker/blob/master/Dock...
3. https://github.com/caddyserver/caddy/tree/v2.5.1
4. https://github.com/caddyserver/xcaddy#custom-builds
5. https://github.com/caddy-dns/cloudflare/tags
6. https://go.dev/ref/mod#go-get
7. https://www.redhat.com/en/blog/containers-understanding-diff...
8. https://github.com/NginxProxyManager/docker-nginx-full/blob/...
My immediate reaction is that the example is nice as a one-off build, but it's much more complex if I need to set up something I can maintain long term. I might be overthinking it, but in the context of thinking about something I can maintain my thought process is below. The questions are mostly rhetorical.
First, what versions am I getting? Does using `2.5.1-builder` result in a customer built binary that's version `2.5.1`? The command usage [1] of the `xcaddy` command says it falls back to the `CADDY_VERSION` environment variable if it's not set explicitly. Since it's not set explicitly, I go looking for that variable in the Dockerfile [2].
That's some templating language I'm not familiar with and I can't track down where the variable gets set, at least not quickly. I'd probably have to spend an hour learning how those templates work to figure it out. To make a quicker, educated guess, it most likely matches the builder version. The docs said the version can be set to any git ref, so I can explicitly set it to v2.5.1 on the command line [3] to be certain.
Now, what version of `caddy-dns/cloudflare` am I getting? The xcaddy custom builds section of the docs [4] says the version can optionally be specified, but it's not specified in the above example. There aren't any tags in the repo, so it's probably building off `master`. The doc says it functions similar to `go get`, but doesn't explain what the differences are and the default behavior isn't explained either.
The docs for `go get` [6] say it can use a revision, so maybe a specific commit can be used for that, but I'd need to test it since I'm not super familiar with Golang.
What other risks come along with building and maintaining my own custom image? I could end up with a subtly broken build that only occurs in my environment. Portability doesn't guarantee compatibility [7] and building custom images increases the risk of compatibility issues beyond what I get with official images (building and running vs just running). That blog post is a really cool read on it's own BTW.
I need to consider the potential for breakage even if it's miniscule because my Docker infrastructure is self hosted and will be sitting behind my custom built Caddy image. If my custom image breaks, I need a guaranteed way of having access to a previous, known good version. This is as simple as publishing the images externally, but adds an extra step since I'll need an account at a registry and need to integrate pushes to that registry into my build.
If I build a custom image, do I let other people I help with the odd tech thing use it or is all the effort for me only? I don't want to become the maintainer of a Docker image others rely on, so I can't even re-use any related config if I help others in the future since they won't have access to the needed image.
To be fair, I also see things I don't like in the NGINX Proxy Manager Dockerfile [7]. The two that immediately jump out at me are things I consider common mistakes. Both require unlucky timing to fail, but can technically cause failure IMO. The first is using `apt-get update` which will exit 0 on failure and has the potential to leave `apt-get install` running against obsolete versions. The second is using `apt-get update` in multiple parts of a multistage build. If I were doing it I'd run `apt-get update` in a base image and avoid it in the builder + runtime images to guarantee the versions stay the same between the build container and the runtime container.
It took me about 1h to work through that and write this comment, so it's not just a matter of building a Docker image and plugging in the config. There's a lot of nuance that goes into maintaining a Docker image (I'm sure you know that already) and not having an image with the DNS plugin(s) baked in is a show stopper for anyone like me that can't justify maintaining their own.
Also, a 4 line Docker file looks nice in terms of being simple, but explicitly declaring or even adding comments describing some of the things I pointed out above can save people a lot of time. Even comments with links to the relevant portions of the docs would be super useful.
My reason for wanting the Cloudflare DNS plugin is that I have some things I want to run 100% locally without ever exposing them to the internet. The desire for wildcard certificates is to keep things from being discoverable via CTLogs.
I hope that's useful feedback. I realize someone bemoaning the difficulty of running your stuff at home lab / small business scale isn't exactly the target audience in terms of picking up customers that pay the bills. Thanks again for the reply / example.
1. https://github.com/caddyserver/xcaddy#command-usage
2. https://github.com/caddyserver/caddy-docker/blob/master/Dock...
3. https://github.com/caddyserver/caddy/tree/v2.5.1
4. https://github.com/caddyserver/xcaddy#custom-builds
5. https://github.com/caddy-dns/cloudflare/tags
6. https://go.dev/ref/mod#go-get
7. https://www.redhat.com/en/blog/containers-understanding-diff...
8. https://github.com/NginxProxyManager/docker-nginx-full/blob/...
> First, what versions am I getting? Does using `2.5.1-builder` result in a customer built binary that's version `2.5.1`? The command usage [1] of the `xcaddy` command says it falls back to the `CADDY_VERSION` environment variable if it's not set explicitly. Since it's not set explicitly, I go looking for that variable in the Dockerfile [2].
Yeah - the default version it'll build with is the version embedded in the builder image, so in that case, v2.5.1. But really, you can just use always use the latest builder image and specify the version you want in the xcaddy command, i.e. 'xcaddy build v2.5.1 --with <plugin>', or any other git ref if not a version (cause we're using Go to build and you can use any git ref, like a commit hash or branch name if you want to try a WIP pull request).
We set it up with a good default so most users wouldn't need to ask that question, it should "just work" for them. But it's a valid question to ask.
> That's some templating language I'm not familiar with and I can't track down where the variable gets set, at least not quickly.
Yeah we're using Gomplate for generating the Dockerfiles for the official Docker Library builds, since we need to make builds for every CPU architecture, and even Windows docker images (I still have no idea why anyone would want those, but alas). Either way, that's an implementation detail of how we automate this stuff, doesn't matter to users.
> Now, what version of `caddy-dns/cloudflare` am I getting?
The latest, if you don't specify a version. The https://github.com/caddy-dns/cloudflare repo doesn't have tagged releases, so it'll just be the latest commit on the master branch. You can specify a specific commit like '--with github.com/caddy-dns/cloudflare@8ea1cff' for (as of this writing) the commit just before the latest.
> What other risks come along with building and maintaining my own custom image?
Honestly, none. Maybe problems with plugins not being compatible with eachother, but Caddy's plugin design means that should rarely happen, except if two plugins have the same module ID. But that's up to you to make sure you don't pick two plugins that try to do the same thing.
Because of the way Go builds work, they can always be cross-compiled. We don't use CGO, so builds of Caddy are completely static and have zero dependencies. There's really no risk that it doesn't build in a specific environment, or whatever.
> If I build a custom image, do I let other people I help with the odd tech thing use it or is all the effort for me only? I don't want to become the maintainer of a Docker image others rely on, so I can't even re-use any related config if I help others in the future since they won't have access to the needed image.
Up to you. But that's the exact reason we don't maintain builds with plugins ourselves. There's literally an infinite amount of combinations possible. Some have suggested like "caddy-lite" and "caddy-full" sort of setups where we ship just a few vetted plugins or "yolo give me all the plugins" but that's silly. We don't have the time or resources to vet all the plugins.
From your perspective it might seem like "duh, there should be an official build with Cloudflare", but really it's a pretty small percentage of users who need this.
> Also, a 4 line Docker file looks nice in terms of being simple, but explicitly declaring or even adding comments describing some of the things I pointed out above can save people a lot of time. Even comments with links to the relevant portions of the docs would be super useful.
(as I wrote in my other comment, the docs for this are on https://hub.docker.com/_/caddy)
> The desire for wildcard certificates is to keep things from being discoverable via CTLogs.
It's really trivial for someone to scan until they hit subdomains that return a successful response, if they really cared. This doesn't really protect from anything. Using wildcards for that is an antipattern.
Just make sure you have authentication on everything you care about protecting. You can easily add basicauth to a domain, or deny requests by remote IP (only allowing private IPs for example), or forward authentication to an app like Authelia which can give you single-sign on for all your apps.
Yeah - the default version it'll build with is the version embedded in the builder image, so in that case, v2.5.1. But really, you can just use always use the latest builder image and specify the version you want in the xcaddy command, i.e. 'xcaddy build v2.5.1 --with <plugin>', or any other git ref if not a version (cause we're using Go to build and you can use any git ref, like a commit hash or branch name if you want to try a WIP pull request).
We set it up with a good default so most users wouldn't need to ask that question, it should "just work" for them. But it's a valid question to ask.
> That's some templating language I'm not familiar with and I can't track down where the variable gets set, at least not quickly.
Yeah we're using Gomplate for generating the Dockerfiles for the official Docker Library builds, since we need to make builds for every CPU architecture, and even Windows docker images (I still have no idea why anyone would want those, but alas). Either way, that's an implementation detail of how we automate this stuff, doesn't matter to users.
> Now, what version of `caddy-dns/cloudflare` am I getting?
The latest, if you don't specify a version. The https://github.com/caddy-dns/cloudflare repo doesn't have tagged releases, so it'll just be the latest commit on the master branch. You can specify a specific commit like '--with github.com/caddy-dns/cloudflare@8ea1cff' for (as of this writing) the commit just before the latest.
> What other risks come along with building and maintaining my own custom image?
Honestly, none. Maybe problems with plugins not being compatible with eachother, but Caddy's plugin design means that should rarely happen, except if two plugins have the same module ID. But that's up to you to make sure you don't pick two plugins that try to do the same thing.
Because of the way Go builds work, they can always be cross-compiled. We don't use CGO, so builds of Caddy are completely static and have zero dependencies. There's really no risk that it doesn't build in a specific environment, or whatever.
> If I build a custom image, do I let other people I help with the odd tech thing use it or is all the effort for me only? I don't want to become the maintainer of a Docker image others rely on, so I can't even re-use any related config if I help others in the future since they won't have access to the needed image.
Up to you. But that's the exact reason we don't maintain builds with plugins ourselves. There's literally an infinite amount of combinations possible. Some have suggested like "caddy-lite" and "caddy-full" sort of setups where we ship just a few vetted plugins or "yolo give me all the plugins" but that's silly. We don't have the time or resources to vet all the plugins.
From your perspective it might seem like "duh, there should be an official build with Cloudflare", but really it's a pretty small percentage of users who need this.
> Also, a 4 line Docker file looks nice in terms of being simple, but explicitly declaring or even adding comments describing some of the things I pointed out above can save people a lot of time. Even comments with links to the relevant portions of the docs would be super useful.
(as I wrote in my other comment, the docs for this are on https://hub.docker.com/_/caddy)
> The desire for wildcard certificates is to keep things from being discoverable via CTLogs.
It's really trivial for someone to scan until they hit subdomains that return a successful response, if they really cared. This doesn't really protect from anything. Using wildcards for that is an antipattern.
Just make sure you have authentication on everything you care about protecting. You can easily add basicauth to a domain, or deny requests by remote IP (only allowing private IPs for example), or forward authentication to an app like Authelia which can give you single-sign on for all your apps.
> If I don't know anything about Caddy I don't even know if the `header` lines are matching that header or adding that header.
The '@' sign is pretty ubiquitous these days to mean "at" or "toward". Email addresses and screen names use this all the time. I think this is not too complicated.
> What are the asterisks for?
What asterisks are almost _always_ for: wildcards! Connection headers can contain a comma separated list of values:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Co...
Hence the need for the asterisk. Your other server configs are incompatible with HTTP spec.
> Why use a custom config language when the real HTTP headers are just as concise?
We _are_ using real HTTP headers.
> Does the difference in capitalization mean something?
Not in header keys, but values can be case sensitive. This is an HTTP spec thing, not a web server thing. Sounds like you should probably already understand this, though.
> Isn't something like this easier to skim, reason, and to learn about?
Not really, since your config isn't compatible with the HTTP spec. And they do different things.
> A simple HAProxy config can be written in a way that someone who has no idea how HAProxy works can skim the config and get a pretty good idea of what's going on.
I, for one, find that HAProxy config more confusing and complex.
> Caddy is probably the best option for a simple, top down config because most of the sites would be super simple and it would be terse / easy to skim once you learn it.
There are plenty of companies using Caddy with thousands of sites and complex configs. Caddy is great for all such use cases, not just simple ones.
Besides, it's a memory-safe web server unlike Apache, Nginx, or HAProxy.
> I think I would have ended up using it if I didn't have to deal with the hassle of getting the Cloudflare DNS plugin set up on my own.
What do you mean "set up on your own"? Our download page does it for you. Or Docker images can be automated to include it as well.
Not really disappointing, especially since no other servers come with Cloudflare-specific support baked in with automatic ACME challenge solvers. I'm not sure what bar you're holding Caddy to but it's not realistic.
> Compare that to Caddy where it took me a couple hours just to figure out what I'd need to install/configure just to get a wildcard TLS cert
You must have not read the documentation/wiki then. They just work with the DNS challenge: https://caddyserver.com/docs/automatic-https#wildcard-certif...
Takes about 2-5 minutes to set up, including getting credentials from your DNS provider.
---
You're not only losing memory safety by using a server written in C, but with external tooling managing your certs instead of having it built into the web server, you're forfeiting robust error handling, advanced revocation logic, automatic OCSP stapling, and more.
The '@' sign is pretty ubiquitous these days to mean "at" or "toward". Email addresses and screen names use this all the time. I think this is not too complicated.
> What are the asterisks for?
What asterisks are almost _always_ for: wildcards! Connection headers can contain a comma separated list of values:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Co...
Hence the need for the asterisk. Your other server configs are incompatible with HTTP spec.
> Why use a custom config language when the real HTTP headers are just as concise?
We _are_ using real HTTP headers.
> Does the difference in capitalization mean something?
Not in header keys, but values can be case sensitive. This is an HTTP spec thing, not a web server thing. Sounds like you should probably already understand this, though.
> Isn't something like this easier to skim, reason, and to learn about?
Not really, since your config isn't compatible with the HTTP spec. And they do different things.
> A simple HAProxy config can be written in a way that someone who has no idea how HAProxy works can skim the config and get a pretty good idea of what's going on.
I, for one, find that HAProxy config more confusing and complex.
> Caddy is probably the best option for a simple, top down config because most of the sites would be super simple and it would be terse / easy to skim once you learn it.
There are plenty of companies using Caddy with thousands of sites and complex configs. Caddy is great for all such use cases, not just simple ones.
Besides, it's a memory-safe web server unlike Apache, Nginx, or HAProxy.
> I think I would have ended up using it if I didn't have to deal with the hassle of getting the Cloudflare DNS plugin set up on my own.
What do you mean "set up on your own"? Our download page does it for you. Or Docker images can be automated to include it as well.
Not really disappointing, especially since no other servers come with Cloudflare-specific support baked in with automatic ACME challenge solvers. I'm not sure what bar you're holding Caddy to but it's not realistic.
> Compare that to Caddy where it took me a couple hours just to figure out what I'd need to install/configure just to get a wildcard TLS cert
You must have not read the documentation/wiki then. They just work with the DNS challenge: https://caddyserver.com/docs/automatic-https#wildcard-certif...
Takes about 2-5 minutes to set up, including getting credentials from your DNS provider.
---
You're not only losing memory safety by using a server written in C, but with external tooling managing your certs instead of having it built into the web server, you're forfeiting robust error handling, advanced revocation logic, automatic OCSP stapling, and more.
I wasn't trying to imply that Caddy is bad, so I hope it didn't come off like that. I was trying to highlight how things that seem simple might only be simple to people that already have a comprehensive understanding of the topic. I'll give you some follow up examples:
> What asterisks are almost _always_ for: wildcards!
However the link you gave doesn't include a single mention or example of using wildcards, so I can't even educate myself. Was that the link you intended to share?
The problem for me in understanding that particular config is that I don't know enough about HTTP to know if you can set headers with asterisks in them. I can't look at that config snippet and figure out immediately if it's matching incoming requests and sending the matched requests to the backend or if it's adding the headers to incoming requests (that are matched some other way) before sending them to the backend.
The root of the confusion for me is that other reverse proxies match on a location for that type of stuff (ex: /wss/) and I simply don't know enough about websockets to make an educated guess about how Caddy is deciding where the traffic gets routed. That's not the kind of thing I'd show up on your forum asking about either. I can figure it out if I need to, but that likely involves spinning up a Caddy server and watching the traffic to see what's going on.
Something as simple as `set-header` or `match-header` instead of `header` would make it easier for me to read and learn the config because it adds enough context that I can immediately figure out what the config is doing. If it's setting headers, that instantly tells me that my lack of knowledge is related to how the matching is happening and I can ignore all the other stuff like the asterisks and the capitalization, at least temporarily. In my specific case, I wanted to translate that config to nginx, so figuring out how the matching works is all I really need.
At the risk of sounding stupid, I'll say I still don't understand if it's routing traffic based on matched headers or adding headers to already matched traffic. This [1] answer seems like a similar config for nginx, but it's routing traffic based on matching those headers.
You have to remember that you can look at a config and instantly map it to the http spec. You probably know the context instantly because one of the possibilities is too stupid to consider. The thing is there's always going to be someone trying to learn that knows nothing and the contextual hints I'm talking about make a huge difference for someone that doesn't know everything.
> What do you mean "set up on your own"? Our download page does it for you.
The download page only gives me a binary, right? I need to deal with setting up a system service. It's not hard, but it's extra steps because I need to redo it every time there's an update or I need to automate it somehow. For my specific use case, running a Docker container (locked to a stable tag) that auto updates is ideal.
> Or Docker images can be automated to include it as well.
That kind of illustrates the point I was trying to make. Setting up an automated Docker build to bake in the functionality I want is significantly different than getting it out of the box. Imagine telling a young person trying to learn "start by setting up an automated Docker build that bakes Caddy + the DNS plugin into the container." Is it reasonable to expect the users of Docker images to be on the same level as the developers of them?
> Takes about 2-5 minutes to set up, including getting credentials from your DNS provider.
Unless you're an experienced sysadmin or a Docker expert, getting to the point where you can start to configure DNS challenges is a huge amount of learning and work. It would take me half a day to set up a good quality, CI built Docker image with the Cloudflare DNS plugin baked in and I've spent hundreds (if not thousands) of hours working with Linux, Docker, CI, etc.. It might take 5 minutes to *configure* if you already know what you're doing.
> no other servers come with Cloudflare-specific support baked in with automatic ACME challenge solvers
Traefik has Cloudflare DNS support baked in to the official Docker image. It's not nearly as easy as Caddy, but it has it.
I qualified my opinion up front saying "for a home lab scale setup". I agree 100% that Caddy is a better product on a technical level, but for a certain scale of setup NGINX Proxy Manager provides a better user experience for me because it's a single container with everything I need baked in, someone else maintains it, and the GUI makes simple reverse proxy configs easier than anything I've ever used (including Caddy).
I wasn't trying to say Caddy is bad and I'm sorry if it came off like that. The only constructive feedback I can give is that if there was an official Docker image with the Cloudflare DNS plugin baked in it's likely I would have put in the time to learn Caddy. I think you underestimate the amount of effort it takes for someone that isn't an expert in the space to get up to speed. For myself I would set aside a full day to familiarize myself with running Caddy, but once I add in the need to build (and maintain) my own Docker image it gets to the point where, pragmatically, I ended up choosing a "good enough" solution over what I'd consider "the best" solution (I tried Caddy first).
1. https://serverfault.com/a/923254/133882
> What asterisks are almost _always_ for: wildcards!
However the link you gave doesn't include a single mention or example of using wildcards, so I can't even educate myself. Was that the link you intended to share?
The problem for me in understanding that particular config is that I don't know enough about HTTP to know if you can set headers with asterisks in them. I can't look at that config snippet and figure out immediately if it's matching incoming requests and sending the matched requests to the backend or if it's adding the headers to incoming requests (that are matched some other way) before sending them to the backend.
The root of the confusion for me is that other reverse proxies match on a location for that type of stuff (ex: /wss/) and I simply don't know enough about websockets to make an educated guess about how Caddy is deciding where the traffic gets routed. That's not the kind of thing I'd show up on your forum asking about either. I can figure it out if I need to, but that likely involves spinning up a Caddy server and watching the traffic to see what's going on.
Something as simple as `set-header` or `match-header` instead of `header` would make it easier for me to read and learn the config because it adds enough context that I can immediately figure out what the config is doing. If it's setting headers, that instantly tells me that my lack of knowledge is related to how the matching is happening and I can ignore all the other stuff like the asterisks and the capitalization, at least temporarily. In my specific case, I wanted to translate that config to nginx, so figuring out how the matching works is all I really need.
At the risk of sounding stupid, I'll say I still don't understand if it's routing traffic based on matched headers or adding headers to already matched traffic. This [1] answer seems like a similar config for nginx, but it's routing traffic based on matching those headers.
You have to remember that you can look at a config and instantly map it to the http spec. You probably know the context instantly because one of the possibilities is too stupid to consider. The thing is there's always going to be someone trying to learn that knows nothing and the contextual hints I'm talking about make a huge difference for someone that doesn't know everything.
> What do you mean "set up on your own"? Our download page does it for you.
The download page only gives me a binary, right? I need to deal with setting up a system service. It's not hard, but it's extra steps because I need to redo it every time there's an update or I need to automate it somehow. For my specific use case, running a Docker container (locked to a stable tag) that auto updates is ideal.
> Or Docker images can be automated to include it as well.
That kind of illustrates the point I was trying to make. Setting up an automated Docker build to bake in the functionality I want is significantly different than getting it out of the box. Imagine telling a young person trying to learn "start by setting up an automated Docker build that bakes Caddy + the DNS plugin into the container." Is it reasonable to expect the users of Docker images to be on the same level as the developers of them?
> Takes about 2-5 minutes to set up, including getting credentials from your DNS provider.
Unless you're an experienced sysadmin or a Docker expert, getting to the point where you can start to configure DNS challenges is a huge amount of learning and work. It would take me half a day to set up a good quality, CI built Docker image with the Cloudflare DNS plugin baked in and I've spent hundreds (if not thousands) of hours working with Linux, Docker, CI, etc.. It might take 5 minutes to *configure* if you already know what you're doing.
> no other servers come with Cloudflare-specific support baked in with automatic ACME challenge solvers
Traefik has Cloudflare DNS support baked in to the official Docker image. It's not nearly as easy as Caddy, but it has it.
I qualified my opinion up front saying "for a home lab scale setup". I agree 100% that Caddy is a better product on a technical level, but for a certain scale of setup NGINX Proxy Manager provides a better user experience for me because it's a single container with everything I need baked in, someone else maintains it, and the GUI makes simple reverse proxy configs easier than anything I've ever used (including Caddy).
I wasn't trying to say Caddy is bad and I'm sorry if it came off like that. The only constructive feedback I can give is that if there was an official Docker image with the Cloudflare DNS plugin baked in it's likely I would have put in the time to learn Caddy. I think you underestimate the amount of effort it takes for someone that isn't an expert in the space to get up to speed. For myself I would set aside a full day to familiarize myself with running Caddy, but once I add in the need to build (and maintain) my own Docker image it gets to the point where, pragmatically, I ended up choosing a "good enough" solution over what I'd consider "the best" solution (I tried Caddy first).
1. https://serverfault.com/a/923254/133882
> The problem for me in understanding that particular config is that I don't know enough about HTTP to know if you can set headers with asterisks in them. I can't look at that config snippet and figure out immediately if it's matching incoming requests and sending the matched requests to the backend or if it's adding the headers to incoming requests (that are matched some other way) before sending them to the backend.
It's _just_ matching, when it's inside a @ named matcher. No manipulation of the headers is going on. The docs would say so if it did. See https://caddyserver.com/docs/caddyfile/matchers#header
And as for reverse_proxy, all headers are passed through as-is, with a few exceptions which are documented here: https://caddyserver.com/docs/caddyfile/directives/reverse_pr...
> The root of the confusion for me is that other reverse proxies match on a location for that type of stuff (ex: /wss/) and I simply don't know enough about websockets to make an educated guess about how Caddy is deciding where the traffic gets routed. That's not the kind of thing I'd show up on your forum asking about either. I can figure it out if I need to, but that likely involves spinning up a Caddy server and watching the traffic to see what's going on.
Actually, asking on the forums is exactly what you should do, if you're confused. We can help there.
And yes -- the initial websocket connection (which is actually HTTP first, then "upgraded" to a raw TCP duplex pipe) contains those two headers which can be used to reliably match websocket connections. It's more reliable to do it this way than by path, because you can completely multi-plex a regular HTTP app and a websocket app this way even if they both share the same request paths. Some people end up dedicating a separate port for websocket connections for their apps because of concerns of path overlap, but matching on headers let you avoid that.
I'm not sure where you found that config, but I'd hope that there was at least a mention of what it's doing. It was probably originally taken from here in the docs https://caddyserver.com/docs/caddyfile/matchers#named-matche... which does have a mention of what that matcher is doing.
> Something as simple as `set-header` or `match-header` instead of `header` would make it easier for me to read and learn the config because it adds enough context that I can immediately figure out what the config is doing.
That's redundant, because you're inside of a named matcher, i.e. a thing starting with @, so that's your hint that it's matching.
Also the 'header' directive (outside matchers) doesn't _only_ set headers, it can also delete them or make in-place replacements, or even set a default if none is already set. See https://caddyserver.com/docs/caddyfile/directives/header
> I'm talking about make a huge difference for someone that doesn't know everything.
Well, it's a balance. For the Caddyfile, we decided to focus on ergonomics rather than absolute clarity. We could make the syntax more verbose (and early betas of Caddy v2 had a much more verbose matcher syntax) but we want it to be easy to write configs once you've gotten your bearings. Yes there's a learning curve, but there's a learning curve for everything.
> The download page only gives me a binary, right? I need to deal with setting up a system service. It's not hard, but it's extra steps because I need to redo it every time there's an update or I need to automate it somehow.
If you're on debian, you can install Caddy with a systemd service by using our apt repo https://caddyserver.com/docs/install#debian-ubuntu-raspbian and then if you need to swap out the binary with a custom one, you can follow these steps https://caddyserver.com/docs/build#package-support-files-for... and then after that point if you need to upgrade versions, you can actually run "sudo caddy upgrade" which will hit the Download page to fetch a new build with the same plugins that you had in your existing binary, but with the latest version of Caddy https://caddyserver.com/docs/command-line#caddy-upgrade.
> For my specific use case, running a Docker container (locked to a stable tag) that auto updates is ideal.
See my other reply where I explain the setup for Docker. It's super easy!
> Is it reasonable to expect the users of Docker images to be on the same level as the developers of them?
If you're using Docker, it's not a stretch to expect that you can at least copy a Dockerfile from the docs and build it (either with 'docker build' or via your docker-compose.yml). You don't have to build it from CI, you can just build it in-place on the server you're running it on.
> Traefik has Cloudflare DNS support baked in to the official Docker image. It's not nearly as easy as Caddy, but it has it.
And the downside of that is that their binary is significantly more bloated because of it. It has to pull in the Cloudflare Go library for their API, as well as the APIs/SDKs for all the other DNS providers they support. That increases the attack surface a considerable amount. We decided we didn't want to have the bloat and risk by shipping them by default, so they're plugins. Also, most of them are not maintained by us, by rather by the community, because we can't be expected to actually use all these DNS providers ourselves, we can only maintain the ones for providers we actually use ourselves. Nobody ever needs all the DNS providers plugged in at the same time, you'll only ever need one at a time (outside of some weird setups where you're serving domains owned by customers or something).
We've worked hard to make sure building with plugins is as easy as possible, and I really think we've achieved that. You just need Go, and xcaddy (a little helper program we wrote), then you run 'xcaddy build --with <plugin>' and it makes a build for you. The 'builder' docker image variant gives you Go + xcaddy so you can produce a build with whatever plugins you want, very easily.
It's _just_ matching, when it's inside a @ named matcher. No manipulation of the headers is going on. The docs would say so if it did. See https://caddyserver.com/docs/caddyfile/matchers#header
And as for reverse_proxy, all headers are passed through as-is, with a few exceptions which are documented here: https://caddyserver.com/docs/caddyfile/directives/reverse_pr...
> The root of the confusion for me is that other reverse proxies match on a location for that type of stuff (ex: /wss/) and I simply don't know enough about websockets to make an educated guess about how Caddy is deciding where the traffic gets routed. That's not the kind of thing I'd show up on your forum asking about either. I can figure it out if I need to, but that likely involves spinning up a Caddy server and watching the traffic to see what's going on.
Actually, asking on the forums is exactly what you should do, if you're confused. We can help there.
And yes -- the initial websocket connection (which is actually HTTP first, then "upgraded" to a raw TCP duplex pipe) contains those two headers which can be used to reliably match websocket connections. It's more reliable to do it this way than by path, because you can completely multi-plex a regular HTTP app and a websocket app this way even if they both share the same request paths. Some people end up dedicating a separate port for websocket connections for their apps because of concerns of path overlap, but matching on headers let you avoid that.
I'm not sure where you found that config, but I'd hope that there was at least a mention of what it's doing. It was probably originally taken from here in the docs https://caddyserver.com/docs/caddyfile/matchers#named-matche... which does have a mention of what that matcher is doing.
> Something as simple as `set-header` or `match-header` instead of `header` would make it easier for me to read and learn the config because it adds enough context that I can immediately figure out what the config is doing.
That's redundant, because you're inside of a named matcher, i.e. a thing starting with @, so that's your hint that it's matching.
Also the 'header' directive (outside matchers) doesn't _only_ set headers, it can also delete them or make in-place replacements, or even set a default if none is already set. See https://caddyserver.com/docs/caddyfile/directives/header
> I'm talking about make a huge difference for someone that doesn't know everything.
Well, it's a balance. For the Caddyfile, we decided to focus on ergonomics rather than absolute clarity. We could make the syntax more verbose (and early betas of Caddy v2 had a much more verbose matcher syntax) but we want it to be easy to write configs once you've gotten your bearings. Yes there's a learning curve, but there's a learning curve for everything.
> The download page only gives me a binary, right? I need to deal with setting up a system service. It's not hard, but it's extra steps because I need to redo it every time there's an update or I need to automate it somehow.
If you're on debian, you can install Caddy with a systemd service by using our apt repo https://caddyserver.com/docs/install#debian-ubuntu-raspbian and then if you need to swap out the binary with a custom one, you can follow these steps https://caddyserver.com/docs/build#package-support-files-for... and then after that point if you need to upgrade versions, you can actually run "sudo caddy upgrade" which will hit the Download page to fetch a new build with the same plugins that you had in your existing binary, but with the latest version of Caddy https://caddyserver.com/docs/command-line#caddy-upgrade.
> For my specific use case, running a Docker container (locked to a stable tag) that auto updates is ideal.
See my other reply where I explain the setup for Docker. It's super easy!
> Is it reasonable to expect the users of Docker images to be on the same level as the developers of them?
If you're using Docker, it's not a stretch to expect that you can at least copy a Dockerfile from the docs and build it (either with 'docker build' or via your docker-compose.yml). You don't have to build it from CI, you can just build it in-place on the server you're running it on.
> Traefik has Cloudflare DNS support baked in to the official Docker image. It's not nearly as easy as Caddy, but it has it.
And the downside of that is that their binary is significantly more bloated because of it. It has to pull in the Cloudflare Go library for their API, as well as the APIs/SDKs for all the other DNS providers they support. That increases the attack surface a considerable amount. We decided we didn't want to have the bloat and risk by shipping them by default, so they're plugins. Also, most of them are not maintained by us, by rather by the community, because we can't be expected to actually use all these DNS providers ourselves, we can only maintain the ones for providers we actually use ourselves. Nobody ever needs all the DNS providers plugged in at the same time, you'll only ever need one at a time (outside of some weird setups where you're serving domains owned by customers or something).
We've worked hard to make sure building with plugins is as easy as possible, and I really think we've achieved that. You just need Go, and xcaddy (a little helper program we wrote), then you run 'xcaddy build --with <plugin>' and it makes a build for you. The 'builder' docker image variant gives you Go + xcaddy so you can produce a build with whatever plugins you want, very easily.
Thanks a ton for taking the time to write this reply. I went over it and re-read some of the docs and it makes a lot more sense to me now. The thing that made it click for me is realizing how directives act differently depending on their context/scope (I'm not sure what the right term is).
I really appreciate the explanation for the websocket matching. I spent a lot of time tying to find an explanation for that and couldn't find a good overview anywhere. Understanding the way the connection is upgraded helped me understand how everything is matched and sent to the backend. Now that I understand one scenario fairly well the docs are magically easier to read (imagine that! - lol).
> Well, it's a balance. For the Caddyfile, we decided to focus on ergonomics rather than absolute clarity. We could make the syntax more verbose (and early betas of Caddy v2 had a much more verbose matcher syntax) but we want it to be easy to write configs once you've gotten your bearings. Yes there's a learning curve, but there's a learning curve for everything.
Yes. Everything is like that. When you're learning, you're like me, complaining the config is too terse and magical. Once you know it, you're like me, complaining the config is too verbose and explicit. :-)
The plugin/build thing is a bit of a mixed bag. I get the reasoning and the build system is very good/simple (building a Docker image is trivial). I replied to your other example and gave a really unpolished thought dump of why I don't want to maintain my own image. Maybe you could convince some of the infrastructure providers like Cloudflare to maintain community images with their DNS plugins baked in.
Thanks again for taking the time to reply. I'm probably going to set up a custom Docker image and give things another try. If I do and I end up using Caddy, hopefully I can pay it forward at some point :-)
I really appreciate the explanation for the websocket matching. I spent a lot of time tying to find an explanation for that and couldn't find a good overview anywhere. Understanding the way the connection is upgraded helped me understand how everything is matched and sent to the backend. Now that I understand one scenario fairly well the docs are magically easier to read (imagine that! - lol).
> Well, it's a balance. For the Caddyfile, we decided to focus on ergonomics rather than absolute clarity. We could make the syntax more verbose (and early betas of Caddy v2 had a much more verbose matcher syntax) but we want it to be easy to write configs once you've gotten your bearings. Yes there's a learning curve, but there's a learning curve for everything.
Yes. Everything is like that. When you're learning, you're like me, complaining the config is too terse and magical. Once you know it, you're like me, complaining the config is too verbose and explicit. :-)
The plugin/build thing is a bit of a mixed bag. I get the reasoning and the build system is very good/simple (building a Docker image is trivial). I replied to your other example and gave a really unpolished thought dump of why I don't want to maintain my own image. Maybe you could convince some of the infrastructure providers like Cloudflare to maintain community images with their DNS plugins baked in.
Thanks again for taking the time to reply. I'm probably going to set up a custom Docker image and give things another try. If I do and I end up using Caddy, hopefully I can pay it forward at some point :-)
> The thing that made it click for me is realizing how directives act differently depending on their context/scope (I'm not sure what the right term is).
The Concepts page explains https://caddyserver.com/docs/caddyfile/concepts#structure. Essentially, "directives" are top-level things inside of a site block (or things within 'route' or 'handle' as well, which wrap other directives), and things inside directives are "subdirectives", i.e. options/config for directives. Matchers are just matchers. Things inside an "empty" block at the top of the Caddyfile are "global options" (similar relative placement to directives, but can only appear in the global options block at the top).
> Maybe you could convince some of the infrastructure providers like Cloudflare to maintain community images with their DNS plugins baked in.
We have no contacts at Cloudflare. They'd have to reach out to us, tbh. But I don't think they are likely to see it as a benefit to them to spend the time either. Ultimately, the plugin very rarely changes, it's pretty stable at this point.
> Thanks again for taking the time to reply. I'm probably going to set up a custom Docker image and give things another try. If I do and I end up using Caddy, hopefully I can pay it forward at some point :-)
Glad to hear. Next time you need help, the forums are the best place to ask. It actually supports markdown, and it's actually meant for Caddy support https://caddy.community/
The Concepts page explains https://caddyserver.com/docs/caddyfile/concepts#structure. Essentially, "directives" are top-level things inside of a site block (or things within 'route' or 'handle' as well, which wrap other directives), and things inside directives are "subdirectives", i.e. options/config for directives. Matchers are just matchers. Things inside an "empty" block at the top of the Caddyfile are "global options" (similar relative placement to directives, but can only appear in the global options block at the top).
> Maybe you could convince some of the infrastructure providers like Cloudflare to maintain community images with their DNS plugins baked in.
We have no contacts at Cloudflare. They'd have to reach out to us, tbh. But I don't think they are likely to see it as a benefit to them to spend the time either. Ultimately, the plugin very rarely changes, it's pretty stable at this point.
> Thanks again for taking the time to reply. I'm probably going to set up a custom Docker image and give things another try. If I do and I end up using Caddy, hopefully I can pay it forward at some point :-)
Glad to hear. Next time you need help, the forums are the best place to ask. It actually supports markdown, and it's actually meant for Caddy support https://caddy.community/
Hey. I wanted to let you know that I eventually switched my whole indie dev home lab to Caddy and it wasn't really fair for me to call the config a "black box". Once I got used to the conventions I ended up with config(s) that I'm extremely happy with and the official docs are very, very good.
I hope you have a great weekend!
I hope you have a great weekend!
Awesome! Thanks for following up! :D
Use this on my home network to expose a couple services to the internet through a domain that I bought, works great.
You do end up having to add "custom configuration" which means putting nginx configuration into a textarea without any validation. So far it's just been the occasional service that uses websockets, you need to configure nginx to handle the upgrade.
Let's encrypt integration works well enough. It says it persists your Cloudflare api key for DNS validation for cert generation, but it just doesn't or I missed a volume to mount that wasn't called out through the Unraid template for NPM.
Yeah sure you could just do this all yourself you smart person you, but on my home server I just... don't want to. I want a collection of dumb GUIs that get me where I want to be, and this did that for my reverse proxy need. If it stops doing that I'll find something else, but that hasn't happened yet.
You do end up having to add "custom configuration" which means putting nginx configuration into a textarea without any validation. So far it's just been the occasional service that uses websockets, you need to configure nginx to handle the upgrade.
Let's encrypt integration works well enough. It says it persists your Cloudflare api key for DNS validation for cert generation, but it just doesn't or I missed a volume to mount that wasn't called out through the Unraid template for NPM.
Yeah sure you could just do this all yourself you smart person you, but on my home server I just... don't want to. I want a collection of dumb GUIs that get me where I want to be, and this did that for my reverse proxy need. If it stops doing that I'll find something else, but that hasn't happened yet.
My `dns_cloudflare_api_token` is in `/data/database.sqlite`.
Thanks, was hoping someone would chime in on that :) I'll take a look at the database later, if I can hack it in I'll be happy, doesn't need to be pretty.
Assuming a container name of `nginx`, you can `grep` for it to see if it's already there.
docker exec -it nginx \
grep -a 'dns_cloudflare_api_token' /data/database.sqliteI previously used Dokku and I loved how the HTTP proxy was "automatic". I've moved past Dokku but don't want the complexity of k8s for a home network. I sort of envisioned a system that could read out running container names and create "$name.$domain" proxy entries automatically.
I never got around to it and my current method of automatically generating nginx.conf from a JSON file has a few tricks left, I guess. I guess one additional problem is that I have few things stacked up on a domain and under different "location" entries.
I never got around to it and my current method of automatically generating nginx.conf from a JSON file has a few tricks left, I guess. I guess one additional problem is that I have few things stacked up on a domain and under different "location" entries.
> I sort of envisioned a system that could read out running container names and create "$name.$domain" proxy entries automatically.
Traefik does this :) That’s what I’m using for my personal server - one traefik instance, a wildcard DNS entry, and then a docker container named “foo” is reachable at https://foo.shish.io
Traefik does this :) That’s what I’m using for my personal server - one traefik instance, a wildcard DNS entry, and then a docker container named “foo” is reachable at https://foo.shish.io
Like traefik? https://doc.traefik.io/traefik/providers/docker/
While this can be achieved using Traefik, it requires some creative combinations of static + dynamic configuration to get my_container.example.com with HTTPS and HTTP → HTTPS redirects and Let’s Encrypt. The container labels are a PITA.
I’m not sure there’s an easier solution though.
I’m not sure there’s an easier solution though.
My setup does all of those things — HTTP->HTTPS redirect and Let’s Encrypt are handled in the global static config: https://gist.github.com/shish/f346a3102f5d690be2f64f6d1eb7d7...
And then only the per-container settings need to be set in the dynamic config using container labels
And then only the per-container settings need to be set in the dynamic config using container labels
Ah, right. In the meantime, they improved the entry point config substantially. When v2 first started, there was no HTTPS redirect and no middleware config to be found on entry points.
Never bothered to upgrade my config because it still works.
Never bothered to upgrade my config because it still works.
Try this, its pretty easy, but it depends on the docker socket.
https://github.com/nginx-proxy/nginx-proxy
There is nginx-proxy for this, with auto TLS, its pretty good!
https://github.com/nginx-proxy/nginx-proxy
You can also do this with Caddy, using this plugin: https://github.com/lucaslorentz/caddy-docker-proxy
somewhat offtopic but, its sort of surprising to see just how little F5 networks has contributed to the NGINX product since its acquisition compared to the OS community.
Stop building trash heaps with nginx!
nginx is a poorly written, legacy application with insane defaults, bad documentation, has shitty frameworks (openresty) and language integrations (fuck js and lua), and owned by a fraudulent company (f5).
Why the fuck does anyone use nginx these days? Because they hate themselves?
nginx is a poorly written, legacy application with insane defaults, bad documentation, has shitty frameworks (openresty) and language integrations (fuck js and lua), and owned by a fraudulent company (f5).
Why the fuck does anyone use nginx these days? Because they hate themselves?
I prefer to use SWAG. Docker + NGINX + LetsEncrypt + fail2ban
https://github.com/linuxserver/docker-swag
Does anybody know how this compares with other reverse proxies, like Caddy?
It's very noob friendly, just click a few things on a nice webUI.
But as it's a layer doing a bunch of stuff on top of nginx, letsencrypt, etc.. There's more chance of odd bugs or things not working properly.
If you're familiar with Caddy, just keep using that.
But as it's a layer doing a bunch of stuff on top of nginx, letsencrypt, etc.. There's more chance of odd bugs or things not working properly.
If you're familiar with Caddy, just keep using that.
This is dope. Is there a way to enable caching easily?
Yes, there is an option to enable caching in the UI
I accept it's my fault for not re-testing this often, but what a huge issue. It's vanilla Nginx or Caddy from here on out, it's not worth introducing a third-party security risk.