WebSocket proxy support added to nginx trunk(trac.nginx.org)
trac.nginx.org
WebSocket proxy support added to nginx trunk
http://trac.nginx.org/nginx/changeset/5073/nginx
14 comments
Can anyone explain how proxying websockets wouldn't hit the port number limit on the proxy server? If I understand correctly, when we would proxy tcp connections the workers on the backend server would able to hold only up to 64k connections to the proxy server (in the case when proxy server has only one IP-address)? Am I missing something?
Yep, you need multiple IP addresses and other tricks to create over 64K connections to a single backend: http://www.metabrew.com/article/a-million-user-comet-applica...
One workaround that comes to mind is to have each backend listen on multiple ports.
One workaround that comes to mind is to have each backend listen on multiple ports.
You don't need any special tricks in the backend. The only limits on the backend are the number of file descriptors.
You only need special tricks when you want to simulate a large number of client connections from a single machine (or bunch of machines). The client(s) have a requirement of a unique IP/source-port combination; not the server.
You only need special tricks when you want to simulate a large number of client connections from a single machine (or bunch of machines). The client(s) have a requirement of a unique IP/source-port combination; not the server.
But that case is exactly what we're talking about: a single nginx making over 64K connections to a single backend server. You either need nginx to have multiple IPs or the backend to have multiple listening ports.
But I assumed that backend isn't connected to the client directly, it's connected to the proxy server, and the proxy server holds tcp connections to the clients, and of course can hold millions of them.
In this case, nginx is simply relaying data over an active connection on each. There is an open socket between the client and nginx, and nginx and the backend, for every WebSocket.
There is not a port limit issue, however, as nginx can create more than one connection to a local backend server using unix domain sockets. But the backend server has to then manage multiple open connections, as well, one for each WebSocket.
There is not a port limit issue, however, as nginx can create more than one connection to a local backend server using unix domain sockets. But the backend server has to then manage multiple open connections, as well, one for each WebSocket.
It is a problem if you for some reason cannot use unix sockets. For example if the application server is either running on a different machine or does not have any unix socket implementation.
Thanks for correcting this. People misunderstand how tcp connections work.
Does this support sticky sessions?
Normally I think it's best for HTTP backends to be stateless, but WebSockets open up interesting possibilities for RPC solutions like NowJS (https://github.com/Flotype/now), dnode (https://github.com/substack/dnode), and others (https://github.com/kriskowal/q-connection) that benefit from keeping state in memory in the app servers.
I'd be interested to hear about other load balancing techniques for WebSockets.
Normally I think it's best for HTTP backends to be stateless, but WebSockets open up interesting possibilities for RPC solutions like NowJS (https://github.com/Flotype/now), dnode (https://github.com/substack/dnode), and others (https://github.com/kriskowal/q-connection) that benefit from keeping state in memory in the app servers.
I'd be interested to hear about other load balancing techniques for WebSockets.
WebSockets are persistent connections, not a state-management trick spanning multiple connections/requests. The only time nginx is involved in "routing" the socket is on the initial connection. Once established, nginx is simply relaying data over an active, persistent TCP/unix sockets.
For the initial connection, you can use any of nginx's normal proxy load balancing options (ip_hash, least_conn, etc).
For the initial connection, you can use any of nginx's normal proxy load balancing options (ip_hash, least_conn, etc).
HAProxy can, reportedly, load balance WebSocket connections:
http://blog.exceliance.fr/2012/11/07/websockets-load-balanci...
dotCloud's Hipache also looks like it was basically written to solve this problem:
https://github.com/dotcloud/hipache
http://blog.exceliance.fr/2012/11/07/websockets-load-balanci...
dotCloud's Hipache also looks like it was basically written to solve this problem:
https://github.com/dotcloud/hipache
I'd expect the ip_hash directive to work as usual http://wiki.nginx.org/HttpUpstreamModule#ip_hash although it's not exactly sticky sessions. There's a separate module for that https://github.com/michaelneale/nginx-sticky-module
I find it quite interesting that Android default browser (and thus WebView) is the only modern browser that still does not support WebSockets. I wish Google shed some light on why this is. Unupdatable Android browser is the modern IE6.
Isn't Browser essentially unmaintained in favor of Chrome (which itself is somewhat crippled compared to the desktop version)?
1. Even Android 4+ devices that support Chrome sold in major carrier stores still have Android browser as the default (which means that most users will use it)
2. Even in Android 4.2 WebView is based on default browser, meaning that any hybrid Android app does not have WebSockets built in.
This is the main reason why WebSockets are not ready for production.
This is the main reason why WebSockets are not ready for production.
even with proper browser support, websockets are fairly flaky. there's all kinds of issues with firewalls/proxies that can present problems.
i've been working on an app recently that was (hoping) to implement websockets, but at this stage it seems xhr-polling is still the best bet...
i've been working on an app recently that was (hoping) to implement websockets, but at this stage it seems xhr-polling is still the best bet...
> (hoping) to implement websockets, but at this stage it seems xhr-polling is still the best bet..
There are libraries like SignalR that will try websockets, but fall back to long polling if the client does not suport it.
There are libraries like SignalR that will try websockets, but fall back to long polling if the client does not suport it.
Haven't used SignalR but we've tried Socket.IO and SockJS for that, and have found SockJS to be absolutely awesome in production.
We're running an in-browser vt100 terminal, Tornado with a SockJS plugin on the backend, round-tripping every keystroke and output from the processes on the server and when SockJS uses WebSockets we get near-ssh responsiveness -- you can run vim just fine. Things are less good but still definitely usable (at least on the command line, vim can be a little frustrating) when it fails over to other protocols.
We're running an in-browser vt100 terminal, Tornado with a SockJS plugin on the backend, round-tripping every keystroke and output from the processes on the server and when SockJS uses WebSockets we get near-ssh responsiveness -- you can run vim just fine. Things are less good but still definitely usable (at least on the command line, vim can be a little frustrating) when it fails over to other protocols.
What issues with Socket.IO convinced you to go with SockJS?
No serious issues -- Socket.IO was really good. But SockJS was noticeably faster, and we saw fewer cases where things arrived out-of-order -- blog post here: http://blog.pythonanywhere.com/27/
Also, IIRC the code we had to write to handle reconnects on patchy connections with SocketIO was pretty gruesome. The problem was that if the connection dropped, it would fail down from WebSockets through various polling systems (which is correct behaviour) but would then stop at the last failover option and never recover, even when the connection came back up. We poked around in the code trying to work out how to change this, but found it too confusing. SockJS didn't have this problem, and we also found the code a bit easier to read, at least from a fairly shallow overview.
Also, IIRC the code we had to write to handle reconnects on patchy connections with SocketIO was pretty gruesome. The problem was that if the connection dropped, it would fail down from WebSockets through various polling systems (which is correct behaviour) but would then stop at the last failover option and never recover, even when the connection came back up. We poked around in the code trying to work out how to change this, but found it too confusing. SockJS didn't have this problem, and we also found the code a bit easier to read, at least from a fairly shallow overview.
Yeah, it'll be interesting to see what the Socket.IO 1.0 release brings, if it ever actually arrives...
One word: SSL. Works well, and reliably with websockets.
We're using this to reverse proxy websockets over SSL and it's simple and wonderfully stable.
https://github.com/nodejitsu/node-http-proxy
https://github.com/nodejitsu/node-http-proxy
do you know how this compares to bouncy? I use bouncy as my web proxy including websockets and SNI. I love the flexability of writing the proxy in javascript. https://github.com/substack/bouncy
It looks comparable to me... I hadn't seen bouncy before though, so I'll have to check it out if I run into any problems with node-http-proxy.
I agree, having the power to write your proxy code in javascript is really nice, especially compared to having to struggle with getting some infrastructure's DSL to work for you (e.g. nginx, varnish's).
I agree, having the power to write your proxy code in javascript is really nice, especially compared to having to struggle with getting some infrastructure's DSL to work for you (e.g. nginx, varnish's).
Or if you don't need realtime upload (only realtime download) then use Server-Sent Events, which is HTTP-compatible and thus supported by proxies already.
Could someone explain why this is helpful? What's an example of a use case where this becomes one of the better solutions?
See this series of posts:
http://www.exratione.com/2012/07/proxying-websocket-traffic-...
http://www.exratione.com/2012/08/websockets-over-ssl-stunnel...
http://www.exratione.com/2012/12/websockets-over-ssl-haproxy...
"So let us say that you are developing a Node.js / Socket.IO application that both uses websockets and serves files the old-fashioned way, such as through Express - this is a fairly common situation. You want to have an HTTP proxy server rather than Node.js field all incoming traffic, so that you can set up load balancing, route requests for static files to some other server process, avoid having to set up SSL configuration in Node.js, and so forth.
"Unfortunately at this point in time it isn't completely straightforward to proxy both websocket traffic and ordinary web requests for Node.js through a single port - which would be the ideal situation. It becomes even less straightforward if you want to use SSL. The following is a brief overview of present options as of early Q3 2012."
"The up-front summary: if you are reading this after Nginx adds support for proxying websocket traffic to Node.js (supposedly coming up in version 1.3), then everything is rainbows and unicorns - just use Nginx. If you are reading this prior to Nginx support for proxying of websocket traffic, then you will likely have to do more work and investigation in order to create a good proxy setup for your servers."
http://www.exratione.com/2012/07/proxying-websocket-traffic-...
http://www.exratione.com/2012/08/websockets-over-ssl-stunnel...
http://www.exratione.com/2012/12/websockets-over-ssl-haproxy...
"So let us say that you are developing a Node.js / Socket.IO application that both uses websockets and serves files the old-fashioned way, such as through Express - this is a fairly common situation. You want to have an HTTP proxy server rather than Node.js field all incoming traffic, so that you can set up load balancing, route requests for static files to some other server process, avoid having to set up SSL configuration in Node.js, and so forth.
"Unfortunately at this point in time it isn't completely straightforward to proxy both websocket traffic and ordinary web requests for Node.js through a single port - which would be the ideal situation. It becomes even less straightforward if you want to use SSL. The following is a brief overview of present options as of early Q3 2012."
"The up-front summary: if you are reading this after Nginx adds support for proxying websocket traffic to Node.js (supposedly coming up in version 1.3), then everything is rainbows and unicorns - just use Nginx. If you are reading this prior to Nginx support for proxying of websocket traffic, then you will likely have to do more work and investigation in order to create a good proxy setup for your servers."
Sorry if this is obvious, but you need it when nginx is your front-end and you want to use web sockets without exposing your backend. A lot of people deploy nginx in their DMZ because of its security and don't want to expose their backend servers, e.g. JEE or Ruby or what have you.
AFAIK there aren't that many robust approaches for reverse proxying web sockets. Apache mod_proxy doesn't work with web sockets, nginx (until now) only worked with websockets if you compiled and patched an unstable dev branch yourself, and not everyone was happy to get their hands dirty with HAProxy. An alternative was to build your own reverse proxy in Node.JS using node-http-proxy, but again, not everyone's cup of tea.
Use cases of websockets enabled reverse proxies are for when you have a number of websockets based apps all sitting on the same box or in the same private local network and you need a way to route incoming traffic to them based on the hostname or path or whatever.
Use cases of websockets enabled reverse proxies are for when you have a number of websockets based apps all sitting on the same box or in the same private local network and you need a way to route incoming traffic to them based on the hostname or path or whatever.
FWIW, I've found HAProxy even more joyous to work with than nginx.
Now you won't need a separate reverse proxy for Websockets. That also means, with just nginx, now you can have both websocket and http on the same url subdomain (for same origin policy and sharing ssl certificate etc).
Are WebSocket connections persistent to a single backend? That is, suppose Nginx is in front of two Unicorn processes.
Will Nginx maintain a mapping table of client browser connection to Unicorn process, so that messages will always hit the same Unicorn process? Does that mean the Unicorn processes don't behave in a "stateless" manner?
Then what happens to the websocket if a Unicorn process dies and is restarted? What does the client browser do? What does Nginx do? What happens to the websocket connections if Nginx dies and there is a failover to a backup Nginx instance?
Will Nginx maintain a mapping table of client browser connection to Unicorn process, so that messages will always hit the same Unicorn process? Does that mean the Unicorn processes don't behave in a "stateless" manner?
Then what happens to the websocket if a Unicorn process dies and is restarted? What does the client browser do? What does Nginx do? What happens to the websocket connections if Nginx dies and there is a failover to a backup Nginx instance?
All this change in nginx does is allow WebSocket connections to be proxied, so your questions don't really make sense. WebSocket's work like TCP sockets, and your backend server still needs to manage the socket directly. For example, Unicorn is not a WebSocket server, though there are WebSocket-capable servers derived from Unicorn (e.g. http://rainbows.rubyforge.org/).
If your backend dies or drops the connection, the WebSocket closes, and the client will have to reconnect. If it was just dropped, you can use nginx's ip_hash proxy option to send the reconnect to the same backend server, but it would be a new WebSocket, and the client/server will have to recreate the session state.
If your backend dies or drops the connection, the WebSocket closes, and the client will have to reconnect. If it was just dropped, you can use nginx's ip_hash proxy option to send the reconnect to the same backend server, but it would be a new WebSocket, and the client/server will have to recreate the session state.
How long does it usually take for things to go from trunk to release? This will be awesome when it's ready because I can keep my production stack really simple. All the existing solution for proxying sockets seemed way too complex, and I typically just run them on different IPs, but even that's annoying.
The next dev release with websocket proxy support is scheduled for tomorrow [1].
[1] http://trac.nginx.org/nginx/roadmap
[1] http://trac.nginx.org/nginx/roadmap
Any date/guess on a production release?
Nginx's development branch is generally considered production ready [1]. I trust the nginx core team not to introduce potentially problematic changes. As an example, the recent spdy integration [2] has lived in a parallel branch for over half a year.
The dev branch is officially upgraded to stable infrequently, maybe once a year.
[1] http://blog.bigdinosaur.org/nginx-dev-or-stable/
[2] http://nginx.org/patches/spdy/
The dev branch is officially upgraded to stable infrequently, maybe once a year.
[1] http://blog.bigdinosaur.org/nginx-dev-or-stable/
[2] http://nginx.org/patches/spdy/
Thanks, that's really helpful! I wouldn't have considered that. It still seems risky using a dev branch for my main web server, but I'll look into it.
This is nice to see. I wonder how it compares to HAProxy http://haproxy.1wt.eu/ which also works with SSL and Secure WebSockets (wss://).
Hi all. I make Ubuntu packages of the Nginx development releases, so if you want to try out the WebSocket proxying and you're using Ubuntu, these should make it pretty easy to do:
https://launchpad.net/~chris-lea/+archive/nginx-devel
I've written up a very quick tutorial on how to set things up here:
https://chrislea.com/2013/02/23/proxying-websockets-with-ngi...
Hope people find these useful.
https://launchpad.net/~chris-lea/+archive/nginx-devel
I've written up a very quick tutorial on how to set things up here:
https://chrislea.com/2013/02/23/proxying-websockets-with-ngi...
Hope people find these useful.
This should fix my problem of not being able to connect to my site though a corporate firewall because I run multiple tornado instances on a single server on different ports. Before nginx wasn't able to route port 80 requests to the chat sub-domains, but now it will be able to.. w00t
Right on!
I'm using nginx and thin, but I'm currently having to use varnish for proxying my websocket connections to thin because nginx can't do it. Then I have to use stunnel for ssl termination because varnish doesn't support it. I'll be able to cut both of those pieces of software out now.
I'm using nginx and thin, but I'm currently having to use varnish for proxying my websocket connections to thin because nginx can't do it. Then I have to use stunnel for ssl termination because varnish doesn't support it. I'll be able to cut both of those pieces of software out now.
Why not just use haproxy?
It's been over a year since I had to set this up, but I used HAProxy in a similar setup and still needed stunnel.
In the whole of my DevOps career, getting SSL working with web sockets was one of the most frustrating.
In the whole of my DevOps career, getting SSL working with web sockets was one of the most frustrating.
I second the success reports of HAProxy 1.5dev with SSL support. It's been stable for me over the last month, and more performant than using STunnel for SSL, which is what I'd been using for the last 1.5 years with Websockets. HAProxy is a remarkable, wicked fast, and very stable product.
HAProxy dev releases have had SSL support for the past 3-4 months.
http://blog.jeffzellner.com/work/2013/01/25/websockets--ssl-...
http://blog.jeffzellner.com/work/2013/01/25/websockets--ssl-...
We are running 1.5 latest dev compiled with ssl here @basho. So far so good.
https://argv0.data.riakcs.net:8443/bailis_gosling.jpg
https://argv0.data.riakcs.net:8443/bailis_gosling.jpg
At last!
Agreed! I'm excited to see this functionality making its way into nginx. Any details on when it will be officially released and make its way into repositories?