XMPP and Jabber(simonholywell.com)
simonholywell.com
XMPP and Jabber
http://simonholywell.com/post/2013/02/xmpp-and-jabber.html
40 comments
What I've really wanted to figure out for awhile now is how XMPP might work as a replacement for email. Even as new IMAP or Gmail clients appear on the market seemingly every few months now, I can't shake the idea that it's time for a new protocol, alas an impossibly tall order. Yet XMPP seems like the closest thing to an existing, modern standard to build upon. Am I tilting at windmills, or is there any validity to this idea?
XMPP is not that clear [0] what happens, if the recipient is offline. I'm not sure, what the popular servers (Google, Facebook, etc) do in practice.
As a proof of concept. It should be easy to implement a "shortcut send" into an email client, which sends a mail directly over XMPP, if sender and recipient are both online with this client.
[0] http://xmpp.org/extensions/xep-0160.html
As a proof of concept. It should be easy to implement a "shortcut send" into an email client, which sends a mail directly over XMPP, if sender and recipient are both online with this client.
[0] http://xmpp.org/extensions/xep-0160.html
I have witnessed somewhat normal network situations (being connected to XMPP server while connecting / disconnecting to a VPN server) that have caused a stable and mature Jabber client (psi) not actually send messages even though it seems that it did successfully send them.
For a mature messaging protocol, that smells really really bad for any serious use.
For a mature messaging protocol, that smells really really bad for any serious use.
What, exactly, do you want to do with it, and how does it replace email?
(Genuinely curious - I see suggestions for replacing email on a regular basis, but rarely ones that are deeply thought out.)
(Genuinely curious - I see suggestions for replacing email on a regular basis, but rarely ones that are deeply thought out.)
There are some fairly general reasons, like XMPP being a much newer, extensible (it's in the name!), yet mature messaging protocol with a big community, but there are also some specific ones.
One of big potential benefits I see is the XMPP concept of contact requests. Simply separating new senders from existing senders at an architectural level seems like a great start for an email replacement, both in terms of user experience ("who I know" is fundamentally separate from "who I don't know" regardless of the client), and in network utilization (spam is only ever as long as a request header instead of an entire message).
One of big potential benefits I see is the XMPP concept of contact requests. Simply separating new senders from existing senders at an architectural level seems like a great start for an email replacement, both in terms of user experience ("who I know" is fundamentally separate from "who I don't know" regardless of the client), and in network utilization (spam is only ever as long as a request header instead of an entire message).
XMPP is focused on real-time, synchronous communication. It does provision for async communication with server storage of offline messages, and even originally discriminated "chat" messages from "email"-type messages with multiple recipients and a subject. IIRC, this latter feature is deprecated as the protocol developers focused on real-time stuff.
Because in the end, and despite all its kludges, e-mail doesn't need replacement. Its biggest problem remains spam, and even that is under control by all the major providers.
Because in the end, and despite all its kludges, e-mail doesn't need replacement. Its biggest problem remains spam, and even that is under control by all the major providers.
Sometimes even the topic is what makes the difference (not even location or device) between talking synchronous or asynchronously. I'm not a very assiduous Facebook user, but at some point I've noticed it can blur the edge between sync/async. Maybe it is not perfect (and maybe it is no longer there), but I think its possible to mix both ideas. Neither should be left out.
I've thought about this, too. Certainly much communication on the modern internet is realtime, so arguably a protocol based on realtime communication makes more sense than a protocol conceived for asynchronous communication. A system that allows a conversation to move between the two seems ideal -- look at how Facebook collapsed the distinction between IMs and "messages". Trying to have a realtime conversation over email (even with a thread-aware, modern client like Gmail) is still an exercise in frustration.
The core support for the concept of presence in XMPP is also pretty compelling -- especially in remote collaboration settings, where transparency around another person's comings and goings can significantly reduce conversation friction.
That being said, the current XMPP protocol, even including the large set of approved protocol extensions, still falls very short of what would be needed for it to serve as a viable email replacement. As others in this thread have mentioned, the core protocol doesn't include any specification for how to handle messages directed to offline users, and while XEPs have been developed to address this, implementations still vary widely between implementations. Limited by the structure of JIDs, the protocol extension for multi-user chat ends up being a clumsy hack. It looks like there are also problems with large binary data transfer (see http://metajack.im/2008/06/10/binary-data-is-xmpps-achilles-...).
Obviously, a lot of these problems can be resolved with additional protocol extensions, but this risks creating a hodgepodge of conflicting standards and generating a menagerie of different implementations before drafts are settled on. The fact that the core XMPP RFC makes no allowance for offline messages, which is essentially the core of email, makes me think it might be unwise to try to hot-swap the two.
The core support for the concept of presence in XMPP is also pretty compelling -- especially in remote collaboration settings, where transparency around another person's comings and goings can significantly reduce conversation friction.
That being said, the current XMPP protocol, even including the large set of approved protocol extensions, still falls very short of what would be needed for it to serve as a viable email replacement. As others in this thread have mentioned, the core protocol doesn't include any specification for how to handle messages directed to offline users, and while XEPs have been developed to address this, implementations still vary widely between implementations. Limited by the structure of JIDs, the protocol extension for multi-user chat ends up being a clumsy hack. It looks like there are also problems with large binary data transfer (see http://metajack.im/2008/06/10/binary-data-is-xmpps-achilles-...).
Obviously, a lot of these problems can be resolved with additional protocol extensions, but this risks creating a hodgepodge of conflicting standards and generating a menagerie of different implementations before drafts are settled on. The fact that the core XMPP RFC makes no allowance for offline messages, which is essentially the core of email, makes me think it might be unwise to try to hot-swap the two.
I was expecting something a little more like this:
http://news.ycombinator.com/item?id=3355453
http://news.ycombinator.com/item?id=3355453
For the Node.js developers in the discussion, I've been hacking away on [Junction](https://github.com/jaredhanson/junction) in my spare moments.
I've long been fascinated with XMPP, because it is so flexible. Rather than focusing solely on a specific purpose (like IM), Junction exposes a middleware framework similar to Express so that you can develop applications on top of XMPP.
I've been happy with the approach, and I'd be curious to get feedback from other developers who've had similar thoughts regarding XMPP.
I've long been fascinated with XMPP, because it is so flexible. Rather than focusing solely on a specific purpose (like IM), Junction exposes a middleware framework similar to Express so that you can develop applications on top of XMPP.
I've been happy with the approach, and I'd be curious to get feedback from other developers who've had similar thoughts regarding XMPP.
Junction looks like an interesting library for XMPP and Node.js.
It might also interest Node.js developers to know that this article was written as part of a tutorial on writing a Google Chat bot with Node.js. To see part one of the article please see the following Hacker News post: http://news.ycombinator.com/item?id=5175913
It might also interest Node.js developers to know that this article was written as part of a tutorial on writing a Google Chat bot with Node.js. To see part one of the article please see the following Hacker News post: http://news.ycombinator.com/item?id=5175913
I never fully understood why everybody is using XMPP in walled gardens (e.g. Whatsapp, Facebook, Messages...). Do you think they get anything from that? Is any of those services monetized successfully?
Seemly Whatsapp is, and what they are doing is genius, I would pay even more for the service than what they are charging (99 cents year).
At first I did not understood, what it had in special, until I figured that it was the fact that it used your phone contact list, thus taking out the boring task of managing that, and making adding people to your list easy as finding someone phone number.
Yet, I would not use it as my primary messenger (neither Facebook), because of how they are tied to something in particular (phones, Facebook) and thus allow less flexibility in where or how I use them.
At first I did not understood, what it had in special, until I figured that it was the fact that it used your phone contact list, thus taking out the boring task of managing that, and making adding people to your list easy as finding someone phone number.
Yet, I would not use it as my primary messenger (neither Facebook), because of how they are tied to something in particular (phones, Facebook) and thus allow less flexibility in where or how I use them.
Why would you pay for such service if one can get a normal secure and federated XMPP service for free? Whatsapp offers no advantages of XMPP (no interoperability and no federation) and only adds problems (flawed security). Paying money for such kind of disservice is simply strange.
Adoption. That is one of the main reasons why most people won't mind paying for whatsapp ; almost everyone on their contacts list already uses the app and frankly most people don't know or care much about whatsapp's security measures.
None of my contacts use it, so I can't say anything about adoption. But if someone needs to use an IM network - there are good XMPP services. Simply invite your friends to any federated XMPP server and you have an interoperable way to communicate with them.
"Simply invite your friends to any federated XMPP server..."
I can't even get my friends to click invitation links to pre-baked apps.
I can't even get my friends to click invitation links to pre-baked apps.
Explain to them what's bad about walled IM networks (it's like not being able to send e-mails between AOL and Compuserve back in the day) and explain what is the point in the federated XMPP networks. Then explain why Whatsapp security is totally broken. If people won't care after that - just leave them alone. :) Most people understand, but claim they are too tied in the existing walled networks. Some don't mind registering on some XMPP server in addition, so you can get a range of results.
The obvious advantage is that other people use Whatsapp, which is important for a messaging app.
Wow, I didn't know they charged 99 cents/year to Android users now. It wasn't like that in the past! iPhone monetization continues as a single payment. Do you think that this is enough to maintain the company? Many other companies should be perfectly capable to reproduce the contacts-promiscuity that Whatsapp is using. And they could even mix it with compatibility to other XMPP platforms...
Tango has identical address book auto friend finding and also does video calls on top of IM. And it's free. There are several apps that do that and are free actually.
Disclaimer: I work at Tango.
Disclaimer: I work at Tango.
Because they don't care? Whatsapp is a monstrosity.
Is it me or doest this read like it's half an article? I got through to the end expecting it to be about something that you can do with XMPP and Jabber, rather than a rather basic primer on Jabber and XMPP themselves.
I don't have a problem with it, although it would've been nice if the title had made it clear that it was a primer.
I don't have a problem with it, although it would've been nice if the title had made it clear that it was a primer.
The article was originally written as separate article to appear alongside the main article I wrote for .net Magazine.
The main article is about creating a Google Talk bot with Node.js so if you want to create something with XMPP then please have a read through of that article: http://news.ycombinator.com/item?id=5175913
Also when I submitted this article to Hacker News I entitled it "A short history of XMPP and Jabber" so I am not sure how or when it got renamed.
The main article is about creating a Google Talk bot with Node.js so if you want to create something with XMPP then please have a read through of that article: http://news.ycombinator.com/item?id=5175913
Also when I submitted this article to Hacker News I entitled it "A short history of XMPP and Jabber" so I am not sure how or when it got renamed.
I would like to point the fact that Microsoft is killing live messenger, so the article is kinda outdated on something :)
Also, I wonder what networks can access the original Jabber network beside G-Talk, anyone here has a clue on that?
I really like G-Talk, and I wish it could interoperate with other interesting IM software.
Also, I wonder what networks can access the original Jabber network beside G-Talk, anyone here has a clue on that?
I really like G-Talk, and I wish it could interoperate with other interesting IM software.
Seriously though I could not even imagine my workday without GTalk.
"Google and XMPP have recently added a draft extension to the protocol called Jingle, which enables voice over IP and video conferencing."
The use of 'recently' made me do a bit of double take.
http://slashdot.org/story/05/12/16/070245/google-jabber-and-...
The use of 'recently' made me do a bit of double take.
http://slashdot.org/story/05/12/16/070245/google-jabber-and-...
The feature is still draft and only made version 1 in June 2009 (http://xmpp.org/extensions/xep-0166.html). I don't actually track changes in XMPP so I am open to correction on this point.
Around the time I actually wrote the article (many months before it was originally published in .net magazine) there had just been a libjingle release and I was told at the time it was a new thing to have this available to 3rd party developers.
Around the time I actually wrote the article (many months before it was originally published in .net magazine) there had just been a libjingle release and I was told at the time it was a new thing to have this available to 3rd party developers.
libjingle has been around for years; Wikipedia says December 2005.
(It doesn't matter, but I also raised an eyebrow and started looking for the date on the blog entry when I read that sentence.)
(It doesn't matter, but I also raised an eyebrow and started looking for the date on the blog entry when I read that sentence.)
Also, GTalk does not use libjingle but its own slightly-different pre-standard implementation, that's why it is impossible to talk to GTalk users with a XMPP client that supports Jingle via libjingle.
This is quite a mess.
This is quite a mess.
I have updated the article to reflect that this was not a very recent addition.
Wikipedia may say 2005 but it did not make the draft status that the article refers to until June 2009, which we can all agree is a lot more recent.
Wikipedia may say 2005 but it did not make the draft status that the article refers to until June 2009, which we can all agree is a lot more recent.
I'm implementing a Google Talk jabber client and I'm curious whether or not it allows sending messages as links. It doesn't seem to automatically parse them into clickable ones. Not much luck on the XML alteration....
I vaguely remember one of the authors of XMPP invented an alternative protocol which is simpler, perhaps based on JSON, anyone remember?
Using XML these days seems too heavy weight.
Using XML these days seems too heavy weight.
XMPP's XML overhead isn't as large as you might think; it compresses well, and streaming XML parsers can be pretty fast.
XML's extensibility is one of the reasons that XMPP has so many XEPs. A JSON-based protocol would need a central registry or some kind of namespaces to achieve the same thing.
XML's extensibility is one of the reasons that XMPP has so many XEPs. A JSON-based protocol would need a central registry or some kind of namespaces to achieve the same thing.
Maybe you mean this? http://xmpp.org/extensions/xep-0295.html
I'm looking at you, WhatsApp, iMessage and all others.