Server-Sent Events: The simplest realtime browser spec(segment.io)
segment.io
Server-Sent Events: The simplest realtime browser spec
https://segment.io/blog/2014-04-03-server-sent-events-the-simplest-realtime-browser-spec/
10 comments
Yes, the umbrella term for this HTTP server push is Comet[1], and it's been around in various forms for years. But standardizing this in the form of server-sent events is a very Good Thing, especially with support baked into modern browsers.
Also, unlike WebSockets it's also easy to add SSE support for mobile clients. I've implemented a SSE client class in just over 100 lines of Java, and it's similarly easy to do this with NSURLConnection/NSURLSession on iOS. The W3 standard provides an implementation section that reads like pseudocode.[2]
[1]: https://en.wikipedia.org/wiki/Comet_%28programming%29
[2]: http://dev.w3.org/html5/eventsource/#event-stream-interpreta...
Also, unlike WebSockets it's also easy to add SSE support for mobile clients. I've implemented a SSE client class in just over 100 lines of Java, and it's similarly easy to do this with NSURLConnection/NSURLSession on iOS. The W3 standard provides an implementation section that reads like pseudocode.[2]
[1]: https://en.wikipedia.org/wiki/Comet_%28programming%29
[2]: http://dev.w3.org/html5/eventsource/#event-stream-interpreta...
On the HTTP level it's quite similar to slow-loading iframes (although with less overhead on the wire), but the interface and usability are better.
The browser will try hard to keep the connection always on for you (unless instructed otherwise). You don't have to implement your own logic for offline/online detection, exponential backoff, etc. Thanks to id:/Last-Event-Id header the server can make error recovery transparent to the client.
You do need node/tornado/etc to take full advantage of it, but if all you have is PHP or such, then you can still make it real-time-ish. Because SSE automatically reconnects you can have any behavior ranging from a single persistent TCP/IP connection, through Comet, to regular polling. You control this with the `retry:` line and by (not) closing connection on the server (e.g. keep persistent connection when load on the server is low, set high `retry:` and close the connection when the load is high).
The browser will try hard to keep the connection always on for you (unless instructed otherwise). You don't have to implement your own logic for offline/online detection, exponential backoff, etc. Thanks to id:/Last-Event-Id header the server can make error recovery transparent to the client.
You do need node/tornado/etc to take full advantage of it, but if all you have is PHP or such, then you can still make it real-time-ish. Because SSE automatically reconnects you can have any behavior ranging from a single persistent TCP/IP connection, through Comet, to regular polling. You control this with the `retry:` line and by (not) closing connection on the server (e.g. keep persistent connection when load on the server is low, set high `retry:` and close the connection when the load is high).
I particularly love how you can get Server-Sent Events out of a CouchDB database changes feed. This is such a powerful and extremely easy way of sending real-time updates to a web client.
That sounds really neat. Do you have any links to resources or blog posts explaining how you hooked the two together? Does CouchDB support that out of the box?
Just did a bit of research, and while it doesn't seem to be documented in the guide, simply appending ?feed=eventsource to the changes URL should get you the event source URL in CouchDB >1.3.0.
SSE is great. We wrote about it in quite a bit of detail a while back.
http://blog.fastmail.fm/2012/01/09/building-the-new-ajax-mai...
There's a couple of annoying caveats
1. The "retry" nature of the connection isn't quite as reliable as suggested. See the blog post above for more details
2. NAT timeouts. Make sure you have some sort of "ping" either at application or TCP level.
http://blog.fastmail.fm/2013/05/28/push-events-nat-tcp-conne... http://blog.fastmail.fm/2011/06/28/http-keep-alive-connectio...
3. Each SSE connection is regarded as a regular site connection. So if people open up 6 tabs to your site, you might suddenly find your site is unresponsive to users because every available connection is used by an SSE connection. You might need to implement some inter-tab communication system instead.
http://blog.fastmail.fm/2012/11/26/inter-tab-communication-u...
http://blog.fastmail.fm/2012/01/09/building-the-new-ajax-mai...
There's a couple of annoying caveats
1. The "retry" nature of the connection isn't quite as reliable as suggested. See the blog post above for more details
2. NAT timeouts. Make sure you have some sort of "ping" either at application or TCP level.
http://blog.fastmail.fm/2013/05/28/push-events-nat-tcp-conne... http://blog.fastmail.fm/2011/06/28/http-keep-alive-connectio...
3. Each SSE connection is regarded as a regular site connection. So if people open up 6 tabs to your site, you might suddenly find your site is unresponsive to users because every available connection is used by an SSE connection. You might need to implement some inter-tab communication system instead.
http://blog.fastmail.fm/2012/11/26/inter-tab-communication-u...
It is a really nice things, except IE doesn't implement them so yeah, kind of useless.
It would be nice to say screw IE users but not everyone can do that.
It would be nice to say screw IE users but not everyone can do that.
+1, SSE is great. We (Twitch) use it between systems in a few key cases since it is really easy to make a client for it if you find yourself in a language that doesn't already have it baked into their HTTP client offering.
When it comes to browsers we've still not found a solution that is as solid as Flash sockets; though that could also be a bit of a hangover for us given our reliance on Flash and the subsequent tooling that we have around it.
When it comes to browsers we've still not found a solution that is as solid as Flash sockets; though that could also be a bit of a hangover for us given our reliance on Flash and the subsequent tooling that we have around it.
Here's a polyfill: https://github.com/Yaffle/EventSource/
2 issues; Last-Event-ID is not a header, but a querystring param. Needs 2kb of padding from server before any data.
Looks pretty usable.
2 issues; Last-Event-ID is not a header, but a querystring param. Needs 2kb of padding from server before any data.
Looks pretty usable.
SSE is listed as "under construction" on the IE roadmap: http://status.modern.ie/serversenteventseventsource
"Under consideration" actually.
They are very easy to shim with polling though.
MS tried to look all hip and cool with IE10/11 and SSE didn't have much buzz around it so it wasn't implemented.
Nice post, and interesting timing, just published a post on how to do SSE with Java: http://eranmedan.com/server-side-events-sse-in-java and... I must admit the Node.js version is clearly more concise and elegant :) but still this is one of the simplest ways to do server side "push", if only IE supported it... though you can always use a polyfill:
https://github.com/Yaffle/EventSource/
(though that polifill didn't work for me for that example, but I didn't spend much time trying to figure out why)
(though that polifill didn't work for me for that example, but I didn't spend much time trying to figure out why)
Yeah I love how dirt simple SSE is. WebSockets tend to be the go-to choice for realtime push, but many applications don't really need a bidirectional socket. SSE for server->client and XHR for client->server interactions is good enough for typical REST apps.
Also, (plug alert) SSE is pretty easy to throw together using https://fanout.io/ and its HTTP streaming data transport, if you want to save yourself from having to set up any new server stuff.
Also, (plug alert) SSE is pretty easy to throw together using https://fanout.io/ and its HTTP streaming data transport, if you want to save yourself from having to set up any new server stuff.
It is not immediately obvious to me what fanout.io does from your homepage. Could you add an example?
Clients register to listen with the fanout.io CDN, then my app pushes something to the CDN, and fanout.io handles pushing it to all of my end clients?
Can I determine which subset of users receive the message, or is it "all or nothing"? Are clients able to listen on a specific topic/name?
I'd love to get more info, but I didn't see any higher level explanation and don't have time to dive into docs right now.
Clients register to listen with the fanout.io CDN, then my app pushes something to the CDN, and fanout.io handles pushing it to all of my end clients?
Can I determine which subset of users receive the message, or is it "all or nothing"? Are clients able to listen on a specific topic/name?
I'd love to get more info, but I didn't see any higher level explanation and don't have time to dive into docs right now.
Yes, Fanout works as a publish-subscribe service. Clients listen on channels (topics), and you publish messages to channels which Fanout then delivers to the right clients.
In the case of HTTP streaming, Fanout acts like a reverse proxy in front of your web server. Incoming requests are relayed to your server, and you can control Fanout's behavior depending on how you reply. For example, you can reply with a "hold" instruction saying to bind the incoming request to streaming channel C. The transaction between Fanout and your server is now complete, while the connection between the client and Fanout remains open. You can then send messages to this client by publishing on channel C.
The process is explained in more detail on this post about our open source reference implementation: http://blog.fanout.io/2013/04/09/an-http-reverse-proxy-for-r...
Implementing something like SSE is just a matter of responding with a stream subscribe with initial response of Content-Type: text/event-stream, and then publishing event chunks using SSE formatting.
In the case of HTTP streaming, Fanout acts like a reverse proxy in front of your web server. Incoming requests are relayed to your server, and you can control Fanout's behavior depending on how you reply. For example, you can reply with a "hold" instruction saying to bind the incoming request to streaming channel C. The transaction between Fanout and your server is now complete, while the connection between the client and Fanout remains open. You can then send messages to this client by publishing on channel C.
The process is explained in more detail on this post about our open source reference implementation: http://blog.fanout.io/2013/04/09/an-http-reverse-proxy-for-r...
Implementing something like SSE is just a matter of responding with a stream subscribe with initial response of Content-Type: text/event-stream, and then publishing event chunks using SSE formatting.
Nice. Do you recommend any specific reading for wiring up some backend services like a pub/sub message queue into something like the SSE example you have?
Surely chunked encoding would work more reliably for sending chunks of data from the server?
http://en.wikipedia.org/wiki/Chunked_transfer_encoding
It's one of the sad facts of life that when you design a popular protocol like HTTP/1.1 so that it can do something, but don't actually do anything with it, many implementers drop it.
Many HTTP servers will not allow you to specify precisely where you break chunks; I would expect that the significant majority of slightly higher level interfaces will not allow you to do that, rather handling buffering themselves.
Many HTTP clients will not allow you to read chunks precisely.
And as for the "event" paramter—well, such things are taken care of in RFC 2616 with chunk extensions; it would end up being
Many HTTP servers will not allow you to specify precisely where you break chunks; I would expect that the significant majority of slightly higher level interfaces will not allow you to do that, rather handling buffering themselves.
Many HTTP clients will not allow you to read chunks precisely.
And as for the "event" paramter—well, such things are taken care of in RFC 2616 with chunk extensions; it would end up being
<length of data in hex> ";event=" event-name CRLF data CRLF
1d␍␊
event: poke␍␊
data: "hey!"␍␊
␍␊
␍␊
6;event=poke␍␊
"hey!"␍␊
␍␊
Any SSE server almost certainly uses chunked encoding.
Though technically a non-chunked Content-Length-less HTTP/1.0-style response should work too.
Though technically a non-chunked Content-Length-less HTTP/1.0-style response should work too.
Server-Sent Events rock; love what these guys have done with the live debugger - it's been -massive- help debugging our events (which go through segment)!
FYI not compatible with any version of IE. Source: http://caniuse.com/eventsource
That was mentioned in the article, and some polyfills were given.
Honestly, much easier and more reliable via PubNub.com or such.
Sometimes it's nice to look at what's coming down the pipeline and will be standard soon. When you spend time learning compatibility workarounds, that knowledge becomes more dated by the minute.
But for production, yea.
But for production, yea.
I have not used either service. What makes PubNub.com easier and/or more reliable?
Because they fail over several different protocols transparent to you.
And much more effective than DIY. API's are much nicer than old school.
And much more effective than DIY. API's are much nicer than old school.
Dear HN, why the downvote?
Didn't downvote, but that capability can be had without relying on an external service, and not really DIY since there are libraries (eg socket.io).
The only problem with this is c10k - you have to have something writing to the socket and this "something" was basically a process in most cases. It's a great fit for async things like Node or Twisted/Tornado, though. And for Erlang.