Varnish Cache 5.0(varnish.org)
varnish.org
Varnish Cache 5.0
http://varnish.org/releases/rel5.0.0.html#rel5-0-0
6 comments
As someone who has never used Varnish but has used Nginx's cache to some degree... whats the benefit of placing varnish in the middle vs going with Nginx?
If you use it for caching, nginx is lacking something quite important at the moment: stale-while-revalidate (https://tools.ietf.org/html/rfc5861)
Aka serving a cached response while the cache is getting refreshed (even to the request who initiated the refresh). Currently when nginx has to refresh the cache it will "hang" the current response until it gets refreshed (the requests that follow, even while the cache is getting refreshed, will get the stale version tho).
It's the difference between your client being amazed by how fast your api is, to "wtf does page X takes 3s to render from time to time" comments.
Aka serving a cached response while the cache is getting refreshed (even to the request who initiated the refresh). Currently when nginx has to refresh the cache it will "hang" the current response until it gets refreshed (the requests that follow, even while the cache is getting refreshed, will get the stale version tho).
It's the difference between your client being amazed by how fast your api is, to "wtf does page X takes 3s to render from time to time" comments.
There is an openresty project that provides this https://github.com/pintsized/ledge - also provides ESI and more.
I think "proxy_cache_use_stale updating;" covers this case
Nope: "updating" does what I described, if another request initiated an update and it's in progress, it will serve a stale value, but the request who triggered the update is still "hanging" until the cache is refreshed. So you get that annoying slowdown for at least 1 user (often the CEO of your client company).
The way to work around this issue with nginx (for simple endpoints) is to write scripts that will hit the endpoints to make sure the cache is always hot, but it's a sad, half working hack (the CEO can still hit that cold cache page himself, if lucky enough).
The way to work around this issue with nginx (for simple endpoints) is to write scripts that will hit the endpoints to make sure the cache is always hot, but it's a sad, half working hack (the CEO can still hit that cold cache page himself, if lucky enough).
There's a $500 bounty for this [1], but it doesn't look like anyone has taken it on.
[1] - https://www.bountysource.com/issues/972735-proxy_cache_use_s...
[1] - https://www.bountysource.com/issues/972735-proxy_cache_use_s...
The user who posted the bounty has not had any activity on bountysource outside of that bounty as far as I can tell from their profile [0], and it was two years since he and others talked about it in the comments (not including the comment six months ago from someone else since that was not directed at the poster of the bounty). To anyone thinking of fixing the issue, might want to check up on whether the bounty is still up before getting to work. If the bounty is your goal, I mean.
[0]: https://www.bountysource.com/people/23342-bdavis
[0]: https://www.bountysource.com/people/23342-bdavis
If you're already using Nginx and its caching subsystem is working for your use case(s), then I wouldn't worry about it. A typical web stack is composed of load balancing, SSL termination, caching and compression, and static and dynamic content serving. Nginx is capable of all these things (including dynamic content serving using OpenResty).
Once you start breaking your stack apart to 1) add redundancy, 2) isolate hardware for different workloads, and 3) scale the components independently of each other, the situation gets a little more interesting. Sure, you could run different clusters of Nginx for each of SSL termination, static content, cache and compress, etc. and have them reverse proxy to each other (and some people probably do this), but when you get to this evolution of your architecture, it's worth evaluating the different options for each layer of the stack. There's an argument to be made that e.g. HAProxy is a better load balancer, Nginx is a better static content server, Pound is a better SSL terminator, Varnish is a better caching and compressing reverse proxy, etc. (N.B. that I am not saying these things, I am just saying that they can be said.) If you're not serving static content at all—let's say you're presenting an API written in Elixir/Phoenix—it may not make sense to go with Nginx in the first place!
Regarding Varnish, specifically, I've found that it offers unparalleled power and flexibility when it comes to caching and compression. Yes, that power and flexibility comes at the cost of more complexity, but it's there when you need it. It offers different storage backends, including a pure memory backend; the ability to serve gzipped content directly from cache (for supported clients); PURGE and BAN HTTP verbs; synthetic responses; query string sorting; the ability to serve stale content while updating; ESI; and probably a whole bunch of other stuff I'm forgetting and/or have never used. Nginx and Nginx Plus may support some or all of these things, built-in or as modules... I'm not sure.
Once you start breaking your stack apart to 1) add redundancy, 2) isolate hardware for different workloads, and 3) scale the components independently of each other, the situation gets a little more interesting. Sure, you could run different clusters of Nginx for each of SSL termination, static content, cache and compress, etc. and have them reverse proxy to each other (and some people probably do this), but when you get to this evolution of your architecture, it's worth evaluating the different options for each layer of the stack. There's an argument to be made that e.g. HAProxy is a better load balancer, Nginx is a better static content server, Pound is a better SSL terminator, Varnish is a better caching and compressing reverse proxy, etc. (N.B. that I am not saying these things, I am just saying that they can be said.) If you're not serving static content at all—let's say you're presenting an API written in Elixir/Phoenix—it may not make sense to go with Nginx in the first place!
Regarding Varnish, specifically, I've found that it offers unparalleled power and flexibility when it comes to caching and compression. Yes, that power and flexibility comes at the cost of more complexity, but it's there when you need it. It offers different storage backends, including a pure memory backend; the ability to serve gzipped content directly from cache (for supported clients); PURGE and BAN HTTP verbs; synthetic responses; query string sorting; the ability to serve stale content while updating; ESI; and probably a whole bunch of other stuff I'm forgetting and/or have never used. Nginx and Nginx Plus may support some or all of these things, built-in or as modules... I'm not sure.
Faster and far more configurable, the configuration language (VCL) compiles to 'C' and you can inline bits of C if you want.
https://www.varnish-cache.org/trac/wiki/ArchitectureInlineC
https://www.varnish-cache.org/trac/wiki/ArchitectureInlineC
Faster in theory maybe. In production nginx has always faster, more stable, less resources. I've used both extensively. Varnish has more caching features and configuration for sure.
If you need just a generic cache, then using varnish, nginx or httpd are all fine choices. The issue is when you start scaling at obscene amounts of traffic. At that point, having a single single attempting to do too many things just doesn't work. varnish was/is designed as an extremely fast and performant cache which, when correctly configured, simply beats the pants off of nginx or httpd. It's easily as stable as nginx and httpd and as compliant as httpd (nginx is a bit less so).
For caches, the bugaboo is latency, and event architectures are not good choices when small-as-possible latency is an issue. It is here that heavily threaded architectures really shine. Sure, you can likely get by with less resources w/ events, but (1) performance will suffer and (2) as thread implementations improve, threading will only get better and more "stingy" re: resources.
For caches, the bugaboo is latency, and event architectures are not good choices when small-as-possible latency is an issue. It is here that heavily threaded architectures really shine. Sure, you can likely get by with less resources w/ events, but (1) performance will suffer and (2) as thread implementations improve, threading will only get better and more "stingy" re: resources.
Out of interest, do you have any metrics to support this?
It would be interesting to see data that compares the two, and see how close they come in different scenarios, and how tuning/configuration might affect the performance.
It would be interesting to see data that compares the two, and see how close they come in different scenarios, and how tuning/configuration might affect the performance.
I've used both in production. They both fast enough that their performance won't be your bottleneck. One's probably faster than the other by some percentage, but whatever you do, you're going to have some other problem that's bigger than that percentage.
At that point, talking about metrics is likely procrastination.
At that point, talking about metrics is likely procrastination.
I don't think your experience alone is enough to say whether performance differences are significant or not.
While in your experience the performance differences were negligible, but that doesn't mean that will hold true in all usage scenarios. For example, maybe they perform similarly when caching many small files, but one struggles with serving longer running requests.
While in your experience the performance differences were negligible, but that doesn't mean that will hold true in all usage scenarios. For example, maybe they perform similarly when caching many small files, but one struggles with serving longer running requests.
Oh, sure. And one will be faster than the other in any given scenario. Not "faster like c++ is faster than ruby", though, it's "faster like this c++ compiler is faster than that one".
I'm sorry but I strongly disagree with the line of thought there.
Parent above was saying "both are at the same speed / none offer any speed advantage".
He is then asked for any metrics to support that statement, which is not an extravagant request and should be encouraged more.
Your answer that "At that point, talking about metrics is likely procrastination" and "one will be faster than the other in any given scenario" serves nothing other that further his statement while deriding the need for said metrics to prove that this is not turning anecdotal evidence into facts, and frankly serves nothing to the discussion short of saying the point should be accepted without verification and that asking for verification is useless.
A point of view I really cannot agree with. I do not know which is faster, nor if any of them is faster than the other or not, but if somebody claims one way or the other in a factual manner, he should not find it a waste of time to find an actual metric that shows it.
Parent above was saying "both are at the same speed / none offer any speed advantage".
He is then asked for any metrics to support that statement, which is not an extravagant request and should be encouraged more.
Your answer that "At that point, talking about metrics is likely procrastination" and "one will be faster than the other in any given scenario" serves nothing other that further his statement while deriding the need for said metrics to prove that this is not turning anecdotal evidence into facts, and frankly serves nothing to the discussion short of saying the point should be accepted without verification and that asking for verification is useless.
A point of view I really cannot agree with. I do not know which is faster, nor if any of them is faster than the other or not, but if somebody claims one way or the other in a factual manner, he should not find it a waste of time to find an actual metric that shows it.
OK, maybe I put that stupidly. Let me try it differently. "I've deployed both, I did enough load testing to learn something about their performance in the configurations that mattered to me, and the performance is so okay that I'm not even going to look it up. Choose based on other criteria."
There are other criteria.
There are other criteria.
For caches much larger than available memory nginx will win because it caches to static files and uses sendfile:
http://www.bbc.co.uk/blogs/internet/entries/17d22fb8-cea2-49...
http://www.bbc.co.uk/blogs/internet/entries/17d22fb8-cea2-49...
* ESI support is really nice (if you're an API designer, you can really go nuts on this and make a super simple API do some very complex things or expressive composition from lots of simple calls)
* More fine-grained control of your caching is possible
* Easier to express normalisation of requests (increase your cache hit rate and protect your underlying origin from malicious requests by discarding cache busters)
* Inline C means you can do things like move your authentication to the edge
* In theory if you have enough RAM you can go faster than nginx's on-disk... in practice the sweet spot for the gain is small and they're both on par
But then... it comes with disadvantages too. Like most Varnish services would never let you do the nice Inline C stuff because no-one in their right mind would run untrusted code in their environment where it could impact another customer. If you see a provider do this (at any price point), avoid them.
* More fine-grained control of your caching is possible
* Easier to express normalisation of requests (increase your cache hit rate and protect your underlying origin from malicious requests by discarding cache busters)
* Inline C means you can do things like move your authentication to the edge
* In theory if you have enough RAM you can go faster than nginx's on-disk... in practice the sweet spot for the gain is small and they're both on par
But then... it comes with disadvantages too. Like most Varnish services would never let you do the nice Inline C stuff because no-one in their right mind would run untrusted code in their environment where it could impact another customer. If you see a provider do this (at any price point), avoid them.
"In theory if you have enough RAM you can go faster than nginx's on-disk... in practice the sweet spot for the gain is small and they're both on par"
Not in theory, but in reality. Even if you don't, varnish's file on-disk implementation is just as good as nginx's if not better.
The idea behind a cache is to increase performance. If that is the priority then you would pick a design and implementation geared towards that problem set. Sure you can say "Well, nginx or httpd is 'good enough'" and that would be right. But when nginx or httpd aren't good enough, you need varnish. And even when you don't need varnish, it is often a better architectural design to break out specific functionality (best of breed).
Not in theory, but in reality. Even if you don't, varnish's file on-disk implementation is just as good as nginx's if not better.
The idea behind a cache is to increase performance. If that is the priority then you would pick a design and implementation geared towards that problem set. Sure you can say "Well, nginx or httpd is 'good enough'" and that would be right. But when nginx or httpd aren't good enough, you need varnish. And even when you don't need varnish, it is often a better architectural design to break out specific functionality (best of breed).
Since 3.0 you have VMODs, to counter for the well-founded lack of Inline C support around. These Varnish modules will extend VCL with C, C++ or even Rust libraries on a safer manner: https://varnish-cache.org/vmods/
If you make your own VMOD(can take from a few hours to days), make sure to send a PR and add it to the directory above (IOW: share it) :)
If you make your own VMOD(can take from a few hours to days), make sure to send a PR and add it to the directory above (IOW: share it) :)
Thanks for the detailed list! ESI looks pretty sweet so I'll have to take a look at that as we scale out our API in the coming months.
A one or two orders of magnitude greater speed, lots of configurations for caching policies, the ability to split a request in several subrequests (ESI) and some other nice things.
Not sure if the performance gains are still valid. There's reports that state the contrary: https://deliciousbrains.com/page-caching-varnish-vs-nginx-fa...
Nginx has something similar to ESI called SSI (http://serverfault.com/questions/406103/main-differences-bet...)
There is quite a few places for improvement in that test (e.g. tweaking number of threads in Varnish) and by no means the tests are like for like.
I will take it with a grain of salt or do your own tests, if you really care.
I will take it with a grain of salt or do your own tests, if you really care.
Yes, important point: Always test with your own infrastructure and run your own benchmark. There's nothing replacing own experimentation and experience.
What works for one company does not necessarily work for others as well.
Take a look at `varnishlog` while you have traffic flowing thru - you'll feel like you've seen the light :) Also, read the vsl-query docs: https://www.varnish-cache.org/docs/trunk/reference/vsl-query...
In terms of introspectability, Varnish (afaik) is without equal.
In terms of introspectability, Varnish (afaik) is without equal.
Totally agreed!
I believe that the power and utility of VSL are so underestimated and overlooked that I don't know where to start...
I believe that the power and utility of VSL are so underestimated and overlooked that I don't know where to start...
The ability to manually purge the cache by URL, with wildcards, is pretty handy.
1: Performance. varnish provides much less latency.
2: More control.
3: Better compliance
2: More control.
3: Better compliance
Do I understand this right?
It supports HTTP/2, but doesn't support HTTPS. Therefore it supports HTTP/2 in a mostly unusable form, because browser vendors (for good reasons) decided to support HTTP/2 only over HTTPS.
The reason I don't want to link Varnish against a SSL library, is that in my considered opinion, they all suck.
From a purely operational point of view, you are better of with two different SSL proxies in front if your Varnish (or other webserver), so that you can turn OpenSSL off in even-numbered weeks and the other (pick your poison) in odd-numbered weeks.
The code to hold safely onto your certificate and do all the songs and dances involved in SSL/TLS, is under all circumstances something which should be isolated in as small a process/protection domain as possible.
From a purely operational point of view, you are better of with two different SSL proxies in front if your Varnish (or other webserver), so that you can turn OpenSSL off in even-numbered weeks and the other (pick your poison) in odd-numbered weeks.
The code to hold safely onto your certificate and do all the songs and dances involved in SSL/TLS, is under all circumstances something which should be isolated in as small a process/protection domain as possible.
This is why i love hn. Thanks for answering this!
they do. they really do. i've read your screed about this and i completely agree. absolutely appalling. in particular a marked inability to handle thread concurrency, which is pretty fatal to varnish.
however. given that tls is operationally important. and using an additional proxy causes a lot of configuration and performance headaches. isn't there any better path forward?
however. given that tls is operationally important. and using an additional proxy causes a lot of configuration and performance headaches. isn't there any better path forward?
Yes, Varnish does h2c so for HTTPS you will need something like 1.4 or later: https://hitch-tls.org/
Made by Varnish Software to provide a "browser-vendor-compliant" HTTP/2 stack.
Made by Varnish Software to provide a "browser-vendor-compliant" HTTP/2 stack.
Hmm... you guys sure have pretty thoroughly sanitized all references to Stud at this point in code, documentation, READMEs, etc.
:-(
:-(
The reference to Bump is also on the copyright notice on the license file.
As Lasse pointed out, we never meant to take away any merit from your work. If you feel we could highlight the Hitch's origin from Stud better, let us know how. We are open to suggestions.
And yes, if you ever come to Oslo, you have an open invitation to come by an test our homebrew :)
As Lasse pointed out, we never meant to take away any merit from your work. If you feel we could highlight the Hitch's origin from Stud better, let us know how. We are open to suggestions.
And yes, if you ever come to Oslo, you have an open invitation to come by an test our homebrew :)
Hi jamwt. We haven't forgotten your(?) excellent work on stud!
Both changes.rst and the man page explain where Hitch came from.
Hitch has seen significant changes since we forked stud 0.3.2, for example proper reload/sighup support, an improved configuration format, and OCSP stapling. Running it on a large scale (cert/ip -wise) also works better now.
If you're ever in our parts of the world, let us know and we'll buy you some beers/coffee/$beverage and tell you all about how your old project is doing.
Both changes.rst and the man page explain where Hitch came from.
Hitch has seen significant changes since we forked stud 0.3.2, for example proper reload/sighup support, an improved configuration format, and OCSP stapling. Running it on a large scale (cert/ip -wise) also works better now.
If you're ever in our parts of the world, let us know and we'll buy you some beers/coffee/$beverage and tell you all about how your old project is doing.
Pssst, jamwt!
Don't let them buy you beer.
Insist on tasting their homebrew instead.
Don't let them buy you beer.
Insist on tasting their homebrew instead.
You could put a TLS proxy in front of it, I think? Though you'd have to implement ALPN or NPN in the proxy. (Apparently Hitch, above, does.)
https://www.nginx.com/blog/maximizing-drupal-8-performance-n...
In the BBC’s testing, they found that with NGINX as a drop-in replacement for Varnish, they saw five times more throughput.
In the BBC’s testing, they found that with NGINX as a drop-in replacement for Varnish, they saw five times more throughput.
I don't suppose that particular workload was very suited for Varnish in the first place, if you read carefully, you'll soon realize that most people don't have that kind of content.
I have heard from (other parts of) BBC that Varnish saved a fair amount of bacon for them during the London Olympics, obviously in different workloads.
And nobody would be more surprised than me, if Varnish was a micacle-cure for every single website in the world...
I have heard from (other parts of) BBC that Varnish saved a fair amount of bacon for them during the London Olympics, obviously in different workloads.
And nobody would be more surprised than me, if Varnish was a micacle-cure for every single website in the world...
nginx as a cache is very minimal-featured compared to Varnish. If it fits your workload, great. If not, Varnish is an exceptional tool.
PHK explained the cause of this is that the Linux kernel VM system performs poorly when overcomitted.
https://news.ycombinator.com/item?id=10752209
So the conclusion is, as long your malloc fits in your system memory, Varnish should be blazing fast on Linux. If you need a gigantic cache, please try FreeBSD.
https://news.ycombinator.com/item?id=10752209
So the conclusion is, as long your malloc fits in your system memory, Varnish should be blazing fast on Linux. If you need a gigantic cache, please try FreeBSD.
Historically, PHK was a very vocal criticizer of SPDY and HTTP/2: http://www.varnish-cache.org/docs/trunk/phk/http20.html
Of course he relented and implemented SPDY and HTTP/2 anyway.
But all the same I can't help but feel that his original criticsm still stands, and what we need is a rethink of e.g. cookies.
Of course he relented and implemented SPDY and HTTP/2 anyway.
But all the same I can't help but feel that his original criticsm still stands, and what we need is a rethink of e.g. cookies.
The original brief for H2 was to have the same semantics etc, as HTTP/1.x
From my reading PHK didn't think it went far enough, and wanted to change more.
I think he's got good points to make on session ids, cookies etc. and I suspect we'll se some of the ideas feed into future versions too.
From my reading PHK didn't think it went far enough, and wanted to change more.
I think he's got good points to make on session ids, cookies etc. and I suspect we'll se some of the ideas feed into future versions too.
I don't think we will, because most in the driving seat make their money identifying visitors. Cynical but true.
[deleted]
It's always good to see new stuff coming out for Varnish. Do these changes warrant a major jump in release numbers especially when HTTP/2 support (biggest feature) is experimental?
Anyway, I'm looking forward to testing it out and integrating v5 with Cachoid ( shameful plug: https://www.cachoid.com/ ).
Anyway, I'm looking forward to testing it out and integrating v5 with Cachoid ( shameful plug: https://www.cachoid.com/ ).
From the horse's mouth:
> Varnish 5.0 changes some (mostly) internal APIs and adds some major new features over Varnish 4.1.
> We are in the process of adding HTTP/2 support to Varnish ... we hope to have it production ready for the next major release (2017-03-15).
> Varnish 5.0 changes some (mostly) internal APIs and adds some major new features over Varnish 4.1.
> We are in the process of adding HTTP/2 support to Varnish ... we hope to have it production ready for the next major release (2017-03-15).
From the release notes http://varnish.org/docs/5.0/whats-new/relnote-5.0.html "It is important that people understand that Free and Open Source Software isn't the same as gratis software: Somebody has to pay the developers mortgages and student loans."
Varnish is an excellent piece of software, but I thought it was totally funded by the commercial side varnish software. How does this model work? It seems odd to ask for donations while also selling an expensive supported version?
Varnish is an excellent piece of software, but I thought it was totally funded by the commercial side varnish software. How does this model work? It seems odd to ask for donations while also selling an expensive supported version?
Varnish Cache Author here.
The Varnish Cache FOSS project and Varnish Software the company are two entirely different things.
By and large, the Varnish Cache FOSS project is me, and I am not employed by Varnish Software: I have my own one-man company where I "make computers to do weird things".
The time I spend on Varnish Cache is funded by "The Varnish Moral License":
http://phk.freebsd.dk/VML/index.html
(See also: http://queue.acm.org/detail.cfm?id=2636165)
Varnish Software is one of the handful of companies who pay via the VML to keep me working on Varnish.
Varnish Software has also supported the project in many other ways as well, from running the project server to donating manpower and source code.
Some of these things are being scaled down, for instance the project server had become the square peg in their round internal IT systems, so I have been migrating that off to a server kindly sponsored by RootBSD.
The Varnish Cache FOSS project and Varnish Software the company are two entirely different things.
By and large, the Varnish Cache FOSS project is me, and I am not employed by Varnish Software: I have my own one-man company where I "make computers to do weird things".
The time I spend on Varnish Cache is funded by "The Varnish Moral License":
http://phk.freebsd.dk/VML/index.html
(See also: http://queue.acm.org/detail.cfm?id=2636165)
Varnish Software is one of the handful of companies who pay via the VML to keep me working on Varnish.
Varnish Software has also supported the project in many other ways as well, from running the project server to donating manpower and source code.
Some of these things are being scaled down, for instance the project server had become the square peg in their round internal IT systems, so I have been migrating that off to a server kindly sponsored by RootBSD.
Thanks so much for replying. I did notice the Moral License and was confused as to how it works. I will attempt to get some high profile users I know to help pay for some continued development.
"Until now Varnish Software has done a lot of work for the Varnish Cache project, but for totally valid reasons, they are scaling that back and the project either needs to pick up the slack or drop some of those activities."
Why they are pulling back is anybody's guess. But it sounds like they are focusing on monetizing the existing codebase instead of further developing it.
Why they are pulling back is anybody's guess. But it sounds like they are focusing on monetizing the existing codebase instead of further developing it.