Google defends dropping chat federation with inaccurate comments(plus.google.com)
plus.google.com
Google defends dropping chat federation with inaccurate comments
https://plus.google.com/116276248303121270590/posts/ggNwh9eLYWu
9 comments
"You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know."
Ehhh... well... if you're willing to put some work into the matter that may not be as true as you'd think. Yes, to speak the federated protocol to another server you're going to be speaking the XML protocol, but between a server and a client that come to accommodations with each other, you don't really have to speak XML.
You'd want to speak something isomorphic to the XML, but the XMPP XML is not that verbose on a node-by-node basis or anything (minus a couple things, see below), it really is the XML, not the semantic content. It would be fairly easy for a medium-to-high skill developer to come up with an isomorphism for use with mobile clients. (It is sufficiently tricky that there would be a few traps that could catch the inexperienced or unwary (watch those namespaces!), but it isn't that difficult.) The end result is a more bandwidth efficient protocol that is a drop-in replacement at the XMPP library layer in the client for XMPP. (Of course, if your client is programmed to fling around XML all over the place internally this won't help, but, well, that's the price of failing to abstract properly.)
Where you'd still have trouble is things like roster size, but yes, there are XEPs for that, and before long you're looking at a reasonably svelte protocol (not perfectly, but reasonably svelte) that is still largely XMPP and easy to write communication with the outside world.
What's really difficult is semantic IM translation between entirely different systems, not protocol translation. ejabberd, the server whose internals I am most familiar with, essentially works by internally speaking Erlang terms with itself, and translating to the XML protocol only at the edges. It wouldn't be very hard to drop in different serialization at the edges and get a different protocol; as proof of concept of the fact this is possible, one could easily simply serialize those Erlang terms with Erlang's term serialization, and indeed, when operating ejabberd in a clustered mode, that's actually happening across the wire as the cluster nodes communicate with each other.
However much effort that may be, it seems easier than starting from scratch.
Ehhh... well... if you're willing to put some work into the matter that may not be as true as you'd think. Yes, to speak the federated protocol to another server you're going to be speaking the XML protocol, but between a server and a client that come to accommodations with each other, you don't really have to speak XML.
You'd want to speak something isomorphic to the XML, but the XMPP XML is not that verbose on a node-by-node basis or anything (minus a couple things, see below), it really is the XML, not the semantic content. It would be fairly easy for a medium-to-high skill developer to come up with an isomorphism for use with mobile clients. (It is sufficiently tricky that there would be a few traps that could catch the inexperienced or unwary (watch those namespaces!), but it isn't that difficult.) The end result is a more bandwidth efficient protocol that is a drop-in replacement at the XMPP library layer in the client for XMPP. (Of course, if your client is programmed to fling around XML all over the place internally this won't help, but, well, that's the price of failing to abstract properly.)
Where you'd still have trouble is things like roster size, but yes, there are XEPs for that, and before long you're looking at a reasonably svelte protocol (not perfectly, but reasonably svelte) that is still largely XMPP and easy to write communication with the outside world.
What's really difficult is semantic IM translation between entirely different systems, not protocol translation. ejabberd, the server whose internals I am most familiar with, essentially works by internally speaking Erlang terms with itself, and translating to the XML protocol only at the edges. It wouldn't be very hard to drop in different serialization at the edges and get a different protocol; as proof of concept of the fact this is possible, one could easily simply serialize those Erlang terms with Erlang's term serialization, and indeed, when operating ejabberd in a clustered mode, that's actually happening across the wire as the cluster nodes communicate with each other.
However much effort that may be, it seems easier than starting from scratch.
There are a whole bunch of competing binary XML formats that promise to be more efficient than XML (both in terms of space usage, and time taken to process/parse).
See: http://en.wikipedia.org/wiki/Binary_XML
Fast Infoset is a good contender: http://en.wikipedia.org/wiki/Fast_Infoset
See: http://en.wikipedia.org/wiki/Binary_XML
Fast Infoset is a good contender: http://en.wikipedia.org/wiki/Fast_Infoset
The furor is not about dropping XMPP; it is about dropping the philosophy of open communication they championed: "Client Choice, Service Choice and Platform Choice" (https://developers.google.com/talk/open_communications). If Google chose to drop XMPP in favor of some other protocol that allowed others to adopt and interoperate, the discussion would have been about the effort required to migrate. But this move by Google is a huge blow to those who hoped for a world of ip based open communication solution.
Google is in a position to define the course of internet and has used that power in several ways in the past. This move from an open communication network to a closed network, to me, is as defining as WebGL and WebRTC are, but in a very unhealthy way. Google's move therefore is clearly 'evil' (as per their own definition of evil) as it is a clear choice they made to safeguard their vested interests, stating that other corporations are unduly benefiting from their openness as a reason.(http://www.youtube.com/watch?v=AfK8h73bb-o [2:40])
They stated that major networks did not interoperate with them. But in 2007 AIM started interoperating with Google Talk. http://gmailblog.blogspot.in/2007/12/gmail-chat-aim-crazy-de.... Doesn't that count? What did Google do to promote federation, other than asking people to contact them to federate? It appears that when Google was small, they tried to be open to benefit from other big players. Now that they are the biggies, they don't see any value in being open.
I believe more than any other company you mentioned, Google benefited significantly by projecting themselves as the messiahs of internet, purportedly promoting openness and standardization. They captured the imagination of people like us by promising to be only doers of good, placing the objectives of internet and humanity ahead of the corporation's benefits. They did so, as long as it worked in their favor. Those initial adopters who stood with them for these objectives are either absorbed into Google to work as if Google's interests are internet's interests or they are too few and left with fewer alternatives to significantly influence Google and its corporate objectives. With many of the influential personalities deeply affiliated with Google, we have fewer voices calling a spade as a spade.
Google is in a position to define the course of internet and has used that power in several ways in the past. This move from an open communication network to a closed network, to me, is as defining as WebGL and WebRTC are, but in a very unhealthy way. Google's move therefore is clearly 'evil' (as per their own definition of evil) as it is a clear choice they made to safeguard their vested interests, stating that other corporations are unduly benefiting from their openness as a reason.(http://www.youtube.com/watch?v=AfK8h73bb-o [2:40])
They stated that major networks did not interoperate with them. But in 2007 AIM started interoperating with Google Talk. http://gmailblog.blogspot.in/2007/12/gmail-chat-aim-crazy-de.... Doesn't that count? What did Google do to promote federation, other than asking people to contact them to federate? It appears that when Google was small, they tried to be open to benefit from other big players. Now that they are the biggies, they don't see any value in being open.
I believe more than any other company you mentioned, Google benefited significantly by projecting themselves as the messiahs of internet, purportedly promoting openness and standardization. They captured the imagination of people like us by promising to be only doers of good, placing the objectives of internet and humanity ahead of the corporation's benefits. They did so, as long as it worked in their favor. Those initial adopters who stood with them for these objectives are either absorbed into Google to work as if Google's interests are internet's interests or they are too few and left with fewer alternatives to significantly influence Google and its corporate objectives. With many of the influential personalities deeply affiliated with Google, we have fewer voices calling a spade as a spade.
> It appears that when Google was
small, they tried to be open to
benefit from other big players.
Now that they are the biggies,
they don't see any value in being open.
Sad but true. Corporations who have to worry about keeping their share price forever in the green will have to compromise on their ideals. They will still talk about their ideals but they won't put their money where their mouth is.
Sad but true. Corporations who have to worry about keeping their share price forever in the green will have to compromise on their ideals. They will still talk about their ideals but they won't put their money where their mouth is.
This raises the question of how much we users can trust any for-profit corporation, since at some point that corporation will value its own profit above our intersts. One possible answer is that we can trust them with short-term transactions, but not with any kind of long-term relationship, especially one that's hard to get out of.
Really? There are plenty of valid reasons why XMPP is not suitable for mobile usage (most notably, the proprietary push notification systems that all platforms insist you have to use), but XML isn't one of them. If you use TLS with XMPP (as gtalk does [1]), you get the compression protocol for free, but even if you can't, there's XEP-0138 [2], which specifies the use of zlib and has been in development for over 12 years now, and finalized since 2009. XML is fugly, but it works. At the scale Google operates, it is laughable that the use of XML in chat protocols could possibly impact their bandwidth accounting.
[1] https://developers.google.com/talk/open_communications#devel...
[2] http://xmpp.org/extensions/xep-0138.html
[1] https://developers.google.com/talk/open_communications#devel...
[2] http://xmpp.org/extensions/xep-0138.html
Disclaimer: I am a Google employee, but I do not speak for Google.
I'm not offering an opinion for or against the decision to drop XMPP, but I wanted to offer some corrections to what is being said.
I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.
I do not work on anything even closely related to Hangouts, but I used to run an ejabberd server myself, and I've worked with SIP a bit.
XML does have problems, both with bandwidth, (even compressed), high CPU utilization, parsing code is more complex (which leads to more security issues). XMPP also has issues, and I'm not just talking about how hard it is to get multiple Audio/Video clients to communicate[1]. (e.g. kernel resources, the incredible number of failure states, etc.)
Cheers, Doug
#1 At least early versions of iChat used XMPP with SIP. For those who don't know SIP, it stands for Session Initiation Protocol, which means that it doesn't transfer the audio/video, it's just a language for making connections. You have to negotiate a transport layer (which was often but not always RTP or SRTP). You have to exchange connection ports via SDP. If you're lucky and manage to get by all firewalls at this point, you now have to hope you're able to use compatible codecs. Interoperability was a nightmare.
I'm not offering an opinion for or against the decision to drop XMPP, but I wanted to offer some corrections to what is being said.
I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.
I do not work on anything even closely related to Hangouts, but I used to run an ejabberd server myself, and I've worked with SIP a bit.
XML does have problems, both with bandwidth, (even compressed), high CPU utilization, parsing code is more complex (which leads to more security issues). XMPP also has issues, and I'm not just talking about how hard it is to get multiple Audio/Video clients to communicate[1]. (e.g. kernel resources, the incredible number of failure states, etc.)
Cheers, Doug
#1 At least early versions of iChat used XMPP with SIP. For those who don't know SIP, it stands for Session Initiation Protocol, which means that it doesn't transfer the audio/video, it's just a language for making connections. You have to negotiate a transport layer (which was often but not always RTP or SRTP). You have to exchange connection ports via SDP. If you're lucky and manage to get by all firewalls at this point, you now have to hope you're able to use compatible codecs. Interoperability was a nightmare.
And yet google wave ran on xmpp which was arguably a far far more sophisticated app doing way more communication than just chat
Google wave only used XMPP for federation. It was a nightmare to deploy and we never got it working reliably. Different jabber servers all had different quirks or implemented subsets of the XMPP extension spec.
The client-server protocol was simply JSON. XMPP was not involved. (Well, we translated protobuf messages to JSON, so it was ugly JSON, but still JSON).
The client-server protocol was simply JSON. XMPP was not involved. (Well, we translated protobuf messages to JSON, so it was ugly JSON, but still JSON).
(if you can't comment on this, no worries. thanks)
aside, any opinions on others choosing that as a standard way of communicating between internal services on disparate runtimes? (golang<->python, for example)
we translated protobuf messages to JSON
aside, any opinions on others choosing that as a standard way of communicating between internal services on disparate runtimes? (golang<->python, for example)
--
>I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.
I think they meant the scale of Google (total) compared to the scale of just messaging.
i.e: If you could spend a year's worth of working cutting the costs of <sevice X> by 30%, that would be fairly meaningless if it was .000001% of the total cost of operation.
I think they meant the scale of Google (total) compared to the scale of just messaging.
i.e: If you could spend a year's worth of working cutting the costs of <sevice X> by 30%, that would be fairly meaningless if it was .000001% of the total cost of operation.
Fair enough. But let's look at the scale of Google compared to messaging using your numbers to prove my point.
I don't know what Google's operating expenses are[1], but I'll use the public income numbers as a proxy, and you can adjust from there.
If you look at Google's first quarter 2013 numbers, they pulled in 13.97 billion in revenue, and I'll subtract the 2.96 billion in traffic acquisition costs to come up with 11.01 billion. We'll multiply by four[2] to get a rough estimate of yearly income: 44.04 billion.
If we assume that messaging is millionth of a percent = one hundred millionth[3] of Google's expenses, then the total cost to google is $440 per year, so saving 30% just doesn't make sense.
If, however, it's one hundred thousandth of Google's expenses (still a ridiculously small amount), then you're talking about $440k of yearly expenses, and you now need to compare a year of salary to 30%, or $146k.
At this point, depending on the availability of engineers (whether or not you have to factor in hiring costs) it starts to make a lot more sense.
Cheers, Doug
#1 and I probably wouldn't be able to divulge them if I did. :-)
#2 Each quarter is different, but x4 is a good rough estimate.
#3 Your numbers.
I don't know what Google's operating expenses are[1], but I'll use the public income numbers as a proxy, and you can adjust from there.
If you look at Google's first quarter 2013 numbers, they pulled in 13.97 billion in revenue, and I'll subtract the 2.96 billion in traffic acquisition costs to come up with 11.01 billion. We'll multiply by four[2] to get a rough estimate of yearly income: 44.04 billion.
If we assume that messaging is millionth of a percent = one hundred millionth[3] of Google's expenses, then the total cost to google is $440 per year, so saving 30% just doesn't make sense.
If, however, it's one hundred thousandth of Google's expenses (still a ridiculously small amount), then you're talking about $440k of yearly expenses, and you now need to compare a year of salary to 30%, or $146k.
At this point, depending on the availability of engineers (whether or not you have to factor in hiring costs) it starts to make a lot more sense.
Cheers, Doug
#1 and I probably wouldn't be able to divulge them if I did. :-)
#2 Each quarter is different, but x4 is a good rough estimate.
#3 Your numbers.
It's not just bandwidth, but the cost to process that XML, which is very expensive compared to other serialization formats. Then there are a whole host of other concerns that result from XML being text-based (cdata, preventing runaway parsing, etc). Not defending what Google has done, but the reality is XMPP is simply the most common among a lineup of really crappy chat/presence protocols.
Now, XML itself is one problem, but the "culture" (eg, usage norms) around XML is another -- much like Java's cultural preoccupation with abstraction and boilerplate, XML-based formats have a tendency to veer towards the unwieldy and verbose. XMPP is no exception.
While XMPP has been instrumental in propagating standardized presence and IM, its age is showing. Unless we develop a modern, open replacement that is natively suited to the needs of today's Internet (voice, video, file transfer, group messaging, etc+), we will see companies continue to develop proprietary solutions that are locked behind walled gardens.
+XEPs and extensions are insufficient; too many optional extensions hinders interop and contorts the system beyond its original design specifications. Just ask anyone who's tried to implement chat/buddylists on top of SIP presence.
Now, XML itself is one problem, but the "culture" (eg, usage norms) around XML is another -- much like Java's cultural preoccupation with abstraction and boilerplate, XML-based formats have a tendency to veer towards the unwieldy and verbose. XMPP is no exception.
While XMPP has been instrumental in propagating standardized presence and IM, its age is showing. Unless we develop a modern, open replacement that is natively suited to the needs of today's Internet (voice, video, file transfer, group messaging, etc+), we will see companies continue to develop proprietary solutions that are locked behind walled gardens.
+XEPs and extensions are insufficient; too many optional extensions hinders interop and contorts the system beyond its original design specifications. Just ask anyone who's tried to implement chat/buddylists on top of SIP presence.
Sorry but I just don't see how bandwidth and XML could ever be a problem for Google.
Youtube alone makes up a significant amount of the Internet's total traffic. Sending compressed XML isn't a problem for Google.
Youtube alone makes up a significant amount of the Internet's total traffic. Sending compressed XML isn't a problem for Google.
Just because Google has a lot of capacity doesn't mean they're wasteful with it. At scale, minor inefficiencies add up to a lot. That is the incentive for protobufs, etc. For a more concrete example of seemingly trivial bandwidth savings, check out the HTML source to their 404 page (it validates!): http://www.google.com/asdf.html
That 404 page is a relic from the past. Take a look at what makes up their search page and you'll see that it is almost 1 MB uncompressed. That's for a page that displays a logo, a navbar, a text input and a button.
Sure, the page has really become a one-page app with some advantages that come from pre-loading, but jeez, it is still 1MB!
Sure, the page has really become a one-page app with some advantages that come from pre-loading, but jeez, it is still 1MB!
The problem isn't for Google, it's for the receiving mobile devices.
> Sorry but I just don't see how bandwidth and XML could ever be a problem for Google.
Then why comment? If you have nothing to input from either side. Doubt is not a valuable commodity.
Then why comment? If you have nothing to input from either side. Doubt is not a valuable commodity.
We not here for the sole reason of providing value to you. Although we may, incidentally, hope to do so.
Some of us need our thoughts clarified by the community.
In any case, I find that learning happens in an atmosphere of uncertainty, humility and exploration. Doubt, thus, is a valuable commodity in any community which encourages learning, and I'd like to believe HN is one of them.
Some of us need our thoughts clarified by the community.
In any case, I find that learning happens in an atmosphere of uncertainty, humility and exploration. Doubt, thus, is a valuable commodity in any community which encourages learning, and I'd like to believe HN is one of them.
Yes it is. For instance, if someone spends years trying to solve a problem and they can't do it and they tell you that they doubt that you're going to be able to solve it in a year and probably shouldn't base your strategies around doing so then that information is worth something - whatever the relevant entire costs are for your having done otherwise.
If you respect people, then their opinions tell you things about the search space in which they live. Heck, even if you don't respect people it at least tells you nsomething about how widely known a particular thing is within comparable sections of society.
There are, potentially, some good arguments for filtering your information to keep newbies out of higher level discussion where people are trying to make progress on a problem - but in the context here it seems to me at the moment that you're just being somewhat mean; I don't see what you hope to gain other than to make someone feel bad.
If you respect people, then their opinions tell you things about the search space in which they live. Heck, even if you don't respect people it at least tells you nsomething about how widely known a particular thing is within comparable sections of society.
There are, potentially, some good arguments for filtering your information to keep newbies out of higher level discussion where people are trying to make progress on a problem - but in the context here it seems to me at the moment that you're just being somewhat mean; I don't see what you hope to gain other than to make someone feel bad.
Neither is being a snarky dickhead.
Maybe you should refrain from commenting until you have something "valuable" to offer, too.
Maybe you should refrain from commenting until you have something "valuable" to offer, too.
Let us say each message exchanged in XMPP is about 200-300 bytes. If in a session (time between app foreground and background on a mobile device), user exchanges 10 messages, we are talking about net data transfer of less than 3K bytes. Each of those characters might be compared to 10 tokens while parsing (thee is scope for significant optimization). I think, the battery cost of 30K comparison on CPU isn't comparable to the cost of rendering one of those messages on the screen and it might be sub 1% of the cost of keeping the screen lit for that user session.
Optimizing parsing cost, at the expense of breaking foundational philosophy of their communication services, is hardly justifiable. XMPP has several overheads, XML will not top that list.
Optimizing parsing cost, at the expense of breaking foundational philosophy of their communication services, is hardly justifiable. XMPP has several overheads, XML will not top that list.
Oh, puh-lease.
I hate XML as much as the next guy, but for a phone that can do 20 fps OpenGL scene the cost of parsing XML that is most certainly valid and as used by XMPP is effectively nothing.
I hate XML as much as the next guy, but for a phone that can do 20 fps OpenGL scene the cost of parsing XML that is most certainly valid and as used by XMPP is effectively nothing.
> Oh, puh-lease.
Don't be a dick.
> for a phone that can do 20 fps OpenGL...
It's not just the phone, there is server overhead too, which everyone seems to be ignoring. While XML has a lot of problems, I never said it was the problem with XMPP (although it certainly isn't helping).
Don't be a dick.
> for a phone that can do 20 fps OpenGL...
It's not just the phone, there is server overhead too, which everyone seems to be ignoring. While XML has a lot of problems, I never said it was the problem with XMPP (although it certainly isn't helping).
>> Oh, puh-lease.
> Don't be a dick.
Oh, puh-lease.
> there is server overhead too
It's all going over the SSL, remember?
I have a life-long passionate hate towards XML, but you have to look at a larger picture to understand why Google is killing XMPP support. It's not the performance, it's not the lack of the protocol elegance. It's the fact that they are building a wall. Facebook got their herd walled and it seems to be working well for them, so it's only natural Google is doing the same. The XMPP was in a way, so it's gotta go.
> Don't be a dick.
Oh, puh-lease.
> there is server overhead too
It's all going over the SSL, remember?
I have a life-long passionate hate towards XML, but you have to look at a larger picture to understand why Google is killing XMPP support. It's not the performance, it's not the lack of the protocol elegance. It's the fact that they are building a wall. Facebook got their herd walled and it seems to be working well for them, so it's only natural Google is doing the same. The XMPP was in a way, so it's gotta go.
Other than the unnecessary snark, I don't think we disagree fundamentally. I'm just as annoyed by walled gardens as most people here.
But NIH/control is an unlikely reason for brewing their own protocol. If they simply wanted a wall, they could just disable federation. They've done it before, and it's working for WhatsApp, it's working for Facebook, it's working for Microsoft [1].
G already has a lot of experience with XMPP and it's probably just the engineering reality between what they want to accomplish with Hangouts and what it would take to retrofit XMPP or an existing protocol. Throwing out the baby with the bathwater? Maybe. But it wouldn't be the first time, and won't be the last.
[1] Remember that FBChat, Lync etc are internally proprietary, but they do expose XMPP gateways. Intentionally preventing federation is shitty, but dropping XMPP at the core is not.
But NIH/control is an unlikely reason for brewing their own protocol. If they simply wanted a wall, they could just disable federation. They've done it before, and it's working for WhatsApp, it's working for Facebook, it's working for Microsoft [1].
G already has a lot of experience with XMPP and it's probably just the engineering reality between what they want to accomplish with Hangouts and what it would take to retrofit XMPP or an existing protocol. Throwing out the baby with the bathwater? Maybe. But it wouldn't be the first time, and won't be the last.
[1] Remember that FBChat, Lync etc are internally proprietary, but they do expose XMPP gateways. Intentionally preventing federation is shitty, but dropping XMPP at the core is not.
zlib-compressed XML cannot hold a candle to protocol buffers in the department of space efficiency.
Indeed, its the wrong tool for the job. EXI compressions would be better.
http://www.w3.org/XML/EXI/
http://exificient.sourceforge.net/?id=performance
http://xmpp.org/extensions/xep-0322.html
http://www.w3.org/XML/EXI/
http://exificient.sourceforge.net/?id=performance
http://xmpp.org/extensions/xep-0322.html
Re: "His misinformation is this: XMPP is XML based. The bandwidth that Google most likely refers to is a result of verbosity of XML. You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know."
He directly addresses the Bandwidth concern by noting that interoperability on the server side is irrelevant to mobile anyways:
"Of course none of these concerns for mobile are applicable for server federation. As far as I know, the Google Talk client on Android doesn't even use XMPP as the client-to-server protocol."
He directly addresses the Bandwidth concern by noting that interoperability on the server side is irrelevant to mobile anyways:
"Of course none of these concerns for mobile are applicable for server federation. As far as I know, the Google Talk client on Android doesn't even use XMPP as the client-to-server protocol."
I should have made the link to the document on XMPP on Mobile Devices more explicit: http://xmpp.org/extensions/xep-0286.html. In short, the use of compression in the TLS layer negates the oft stated verbosity of XMPP. It really isn't the problem you think it is. More info: http://stpeter.im/journal/1384.html. I might be a fanboy, but I do know what I'm talking about.
Compression doesn't help CPU utilization, which in turn doesn't help battery consumption. In addition, if the protocol is "chatty" (i.e., sends lots of packets even if there isn't any communication going on -- say, when users change their status/presence information, for example) it means waking up the CPU, which in turn means less power efficiency.
Please read that first document and also my remark about SIFT, which basically allows for delaying or dropping presence to cut down on antenna use and battery consumption.
Hmmm... As the author of XEP-0286, I think I ought to chip in a bit.
1) Compression does help keep the transmissions under the FACH threshold on most networks. This more than outweighs the CPU cost.
2) The bulk of the data to be compressed isn't XML, but text.
3) The chattiness of the protocol - ie, transmissions that are not needed at that point in time - will indeed change radio states, but not only did Google tackle this problem privately, but the XSF has also looked at it in detail.
4) Really, the CPU is a non-issue here. It's all about the radio power states. This is likely to be entirely different at the server end, however efficient XML parsers can and do exist to mitigate this, and make it more distributable.
1) Compression does help keep the transmissions under the FACH threshold on most networks. This more than outweighs the CPU cost.
2) The bulk of the data to be compressed isn't XML, but text.
3) The chattiness of the protocol - ie, transmissions that are not needed at that point in time - will indeed change radio states, but not only did Google tackle this problem privately, but the XSF has also looked at it in detail.
4) Really, the CPU is a non-issue here. It's all about the radio power states. This is likely to be entirely different at the server end, however efficient XML parsers can and do exist to mitigate this, and make it more distributable.
The only way to fix that specific problem is to change the protocol. Which is what Google did.
Creating new and better protocols is OK. Making them closed and non interoperable - is not. XMPP is not ideal, but it's open for participation in its development and for its usage. Whatever Google decided to create is not open, and that's their main failure.
Creating new and better protocols is OK. Making them closed and non interoperable - is not. XMPP is not ideal, but it's open for participation in its development and for its usage. Whatever Google decided to create is not open, and that's their main failure.
Nobody expects better from Microsoft, but they still expect better from Google, hence the sense of betrayal and subsequent furor.
That's exactly the point, though- why should Google have to defend themselves so much more? Apple have the most tightly locked in messaging protocol, but no-one says a word about them opening it- even after they promised to.
Well, Apple gets incredible amounts of crap from HN and other places for their lack of openness. But at this point people just assume that Apple will not open up their stuff. And for the most part Apple doesn't make promises about that.
Google, on the other hand, got a free pass for a while because they had a lot of marketing to the effect of "Look at us! We're the openest openers who ever opened openness!" and... well, they have a history of spectacularly failing to follow through on that with, say, Android ("oh, it's totally open -- except for this release that only our partners get! And except for all the bits that most people don't realize are separate and that we'll C&D you for distributing! But it's OPEN, we promise!").
So people are starting to no longer give Google the automatic pass on this. Which should be expected.
Google, on the other hand, got a free pass for a while because they had a lot of marketing to the effect of "Look at us! We're the openest openers who ever opened openness!" and... well, they have a history of spectacularly failing to follow through on that with, say, Android ("oh, it's totally open -- except for this release that only our partners get! And except for all the bits that most people don't realize are separate and that we'll C&D you for distributing! But it's OPEN, we promise!").
So people are starting to no longer give Google the automatic pass on this. Which should be expected.
I don't recall Apple ever promising that but if they did they wouldn't have explained why they can't or won't do it because they don't want to have it both ways. They are fine saying nope it's not open deal with it. Google seems to want it both ways which puts them under the microscope. Apple gets plenty of criticism from the same types of people for not being open. I don't buy into the poor lil' Google angle of this. They know exactly what they're doing.
"Upon the launch of the iPhone 4, Jobs promised that Apple would immediately start working with standards bodies to make the FaceTime protocol an "open standard." As of November 2012, it is not yet known to have been ratified by any standards body, and the extent of work by Apple with regard to this promise is unclear as Apple has not released technical specifications for the service. FaceTime is not currently supported on any non-Apple devices."
http://en.wikipedia.org/wiki/FaceTime
http://en.wikipedia.org/wiki/FaceTime
I read somewhere that the Facetime team heard about the open-source promise when Jobs announced it on stage. It could very well have been a spur of the moment decision made on stage.
(Not that it absolves Apple from failing to do it)
(Not that it absolves Apple from failing to do it)
Perhaps because nobody can seriously expect Apple to commit to openness (this is the company that forbids open source licenses in its app store), where Google was already using an open protocol?
I don't think Apple actually forbids open source from the app store, do they?
As far as I understand/interpret their policy: they only prohibit so-called copyleft licenses, and/or ones which would be violated inherently by the app store's binary-only distribution model.
As far as I understand/interpret their policy: they only prohibit so-called copyleft licenses, and/or ones which would be violated inherently by the app store's binary-only distribution model.
>And yet the furor is not about Microsoft...
The furore is partly also about Google moving away from an open protocol to a supposedly secret protocol. If Google were to publish the new protocol that Hangouts uses all/most of the animosity would vanish. And people would come up with a xmpp to hangouts bridge too.
Google has supported and even championed open source initiatives and it has served them well. And imo it will serve them well in the long term if they continue being open, instead of trying to monopolize a segment of the market.
The furore is partly also about Google moving away from an open protocol to a supposedly secret protocol. If Google were to publish the new protocol that Hangouts uses all/most of the animosity would vanish. And people would come up with a xmpp to hangouts bridge too.
Google has supported and even championed open source initiatives and it has served them well. And imo it will serve them well in the long term if they continue being open, instead of trying to monopolize a segment of the market.
Not a lack of perspective at all, more of an attempt at deflection because Google are being dicks. Skype's protocols predate Microsofts takeover, so attempting to 'blame' them is churlish, and there already has been a furore over Apple and FaceTime. The browser thing has absolutely sweet FA to do with this.
Google have spent years building a brand out of how """open""" they are; a lot of their marketing/fud is based around it.
Basically they co-opted the language of open source to rube nerds, while nobody is under any illusions re: Microsoft & Apple.
Basically they co-opted the language of open source to rube nerds, while nobody is under any illusions re: Microsoft & Apple.
This.
Chromium alone puts them in the plus column standard-wise. Usually apps are judged on merit, yet in Google's case there is the misconception that they have to defend every engineering decision they make.
I don't see how this is a hot topic seeing as every other IM is an island, and quite frankly if it means directing more resources towards projects like Google Fiber, personally I think it's more than worth it to sunset GReader or move away from a limiting standard.
Chromium alone puts them in the plus column standard-wise. Usually apps are judged on merit, yet in Google's case there is the misconception that they have to defend every engineering decision they make.
I don't see how this is a hot topic seeing as every other IM is an island, and quite frankly if it means directing more resources towards projects like Google Fiber, personally I think it's more than worth it to sunset GReader or move away from a limiting standard.
Sorry but NONE of what you said is true or frankly makes any sense.
Chromium will likely be less standards based now that they don't have the tow of the other WebKit companies keeping them in check. Case in point: SPDY.
Google is not the only company who has to defend every engineering decision. The reason it is being discussed is that it is fitting into this new narrative of Google being less interested in standards and more interested in their advertising centric walled garden.
And finally Google can do multiple things at once. It isn't an either-or scenario.
Chromium will likely be less standards based now that they don't have the tow of the other WebKit companies keeping them in check. Case in point: SPDY.
Google is not the only company who has to defend every engineering decision. The reason it is being discussed is that it is fitting into this new narrative of Google being less interested in standards and more interested in their advertising centric walled garden.
And finally Google can do multiple things at once. It isn't an either-or scenario.
SPDY is an open protocol, and Google is working towards standardization of it. Plus, many other browsers have implementations of it based on Google's protocol specification, so it's not true that SPDY is an example of Google becoming less standards based.
Exactly my point.
Google develops a proprietary protocol without ANY consultation. They then leverage the fact that they own the world's biggest website and popular browser to force it by stealth into popularity. And then afterwards decide to work to standardising it despite it not necessarily being the optimum choice for everyone.
The core protocols of the internet are fundamental. And they should be designed deliberately with as many parties as possible involved. This is not the way standards should come into existence.
Google develops a proprietary protocol without ANY consultation. They then leverage the fact that they own the world's biggest website and popular browser to force it by stealth into popularity. And then afterwards decide to work to standardising it despite it not necessarily being the optimum choice for everyone.
The core protocols of the internet are fundamental. And they should be designed deliberately with as many parties as possible involved. This is not the way standards should come into existence.
Yes. Standards should be designed by committee, ensuring as many cooks are helping the broth as possible. Much like UNIX and C came out of large standards consortia composed of many industry leaders, and like the HTML5 standards evolved incredibly quickly until those darn browser manufacturers slowed them down by going off and implementing things. A single engineer or group of engineers trying to solve a real world problem have never done anything good for the world.
(DISCLAIMER: I work for Google. All of my opinions are my own and not Google's.)
(DISCLAIMER: I work for Google. All of my opinions are my own and not Google's.)
Yes. Standards must either come from a single absolute dictator, or from a mess of gigantic pointless committees. Those are absolutely the only two options available, and there is no middle ground of any sort whatsoever.
Calling Google a single absolute dictator seems like a huge stretch. I'd put Google somewhere between there and a mess of committees myself.
History does indicate that the middle ground is fairly sparse.
I don't think you know what "proprietary" means, you don't pay Google to use SPDY.
They solved a problem and made something better, case in point Facebook and Twitter among others have already deployed it.
If you do some research you'll find that most standards started as projects that came out of a single company or lab that were later adopted by industry.
They solved a problem and made something better, case in point Facebook and Twitter among others have already deployed it.
If you do some research you'll find that most standards started as projects that came out of a single company or lab that were later adopted by industry.
No. You don't understand what proprietary means. SPDY is a trademarked technology owned by a private corporation.
The only reason companies are adopting SPDY is because it is an improvement than HTTP. SPDY is far from perfect. The issue is that with core internet protocols this is not the way it should happen.
The only reason companies are adopting SPDY is because it is an improvement than HTTP. SPDY is far from perfect. The issue is that with core internet protocols this is not the way it should happen.
>The only reason companies are adopting SPDY is because it is an improvement than HTTP.
I don't see the problem.
>The issue is that with core internet protocols this is not the way it should happen.
This is exactly how it does happen. Someone creates something, people like it, the creators don't enforce patents on it or try to keep it a secret, someone writes it up into an RFC. That's how internet standards are created. That's how they've been created as long as there has been an internet.
Look at a counterexample. Microsoft with OOXML. Here we have a so-called standard which says things like "do this the way X version of Microsoft Word does it" without actually specifying how to do it. It's not an open standard, it's a document claiming to be an open standard on paper so that Microsoft can claim their proprietary file format is an open standard. No one else is expected to use it for any purpose other than attempting interoperability with Microsoft Office and Microsoft has no particular interest in having anyone even succeed in doing that.
SPDY is not that. Google has published a reference implementation with source code. People other than Google use it for reasons having nothing to do with interoperability and Google services continue to be available to browsers without SPDY. "They didn't ask permission first" is just disingenuous griping -- who are they even supposed to have asked?
SPDY isn't a problem. Hangouts are. I don't see source code for a reference implementation. I don't see anyone but Google implementing the protocol. That's how you know something is wrong.
I don't see the problem.
>The issue is that with core internet protocols this is not the way it should happen.
This is exactly how it does happen. Someone creates something, people like it, the creators don't enforce patents on it or try to keep it a secret, someone writes it up into an RFC. That's how internet standards are created. That's how they've been created as long as there has been an internet.
Look at a counterexample. Microsoft with OOXML. Here we have a so-called standard which says things like "do this the way X version of Microsoft Word does it" without actually specifying how to do it. It's not an open standard, it's a document claiming to be an open standard on paper so that Microsoft can claim their proprietary file format is an open standard. No one else is expected to use it for any purpose other than attempting interoperability with Microsoft Office and Microsoft has no particular interest in having anyone even succeed in doing that.
SPDY is not that. Google has published a reference implementation with source code. People other than Google use it for reasons having nothing to do with interoperability and Google services continue to be available to browsers without SPDY. "They didn't ask permission first" is just disingenuous griping -- who are they even supposed to have asked?
SPDY isn't a problem. Hangouts are. I don't see source code for a reference implementation. I don't see anyone but Google implementing the protocol. That's how you know something is wrong.
The point you're missing is that people don't use it because they think its good.
They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.
That's the same thing people criticized Microsoft for.
Hangout is just much more wrong tho, as you pointed out.
They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.
That's the same thing people criticized Microsoft for.
Hangout is just much more wrong tho, as you pointed out.
The point you're missing is that people don't use it because they think its good.
They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.
That's the same thing people criticized Microsoft for.
They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.
That's the same thing people criticized Microsoft for.
SPDY isn't a trademarked technology, the name is trademarked (it should be), the protocol is not.
The way SPDY works is just like OAuth/OAuth2, HTTP, XMPP work: someone invent/improve a protocol, working on draft and get it open standard. I don't see any problem with that.
The way SPDY works is just like OAuth/OAuth2, HTTP, XMPP work: someone invent/improve a protocol, working on draft and get it open standard. I don't see any problem with that.
I don't see how you can jump to these false assumptions and then call my comment senseless.
Here is the "Blink manifesto" if you will: A proposal for un-prefixing https://groups.google.com/a/chromium.org/forum/m/?fromgroups...
As for SPDY it's not proprietary and it's being baked into HTTP 2.0: http://tools.ietf.org/html/draft-ietf-httpbis-http2-00
And I'm not sure what you mean by "advertising walled garden" other that just throwing around some senseless hostile buzzwords.
Here is the "Blink manifesto" if you will: A proposal for un-prefixing https://groups.google.com/a/chromium.org/forum/m/?fromgroups...
As for SPDY it's not proprietary and it's being baked into HTTP 2.0: http://tools.ietf.org/html/draft-ietf-httpbis-http2-00
And I'm not sure what you mean by "advertising walled garden" other that just throwing around some senseless hostile buzzwords.
Really? Can you so harshly judge Google on abandoning a (lets face it) terrible protocol?
Maybe I could get over the fact that it's so complex -- which puts an unnecessary strain on both servers and clients -- and maybe I could get over the fact that (on average) servers consume 40% more memory than their IRC counterparts. Maybe I could even get over the fact that the spec could change at any moment (the eXtensible nature of it isn't necessarily a good thing, you know).
But what I couldn't possibly in a million years get over is that... it's XML (shudder).
-- To the downvoters:
Here's some previous discussion about why XMPP sucks: https://news.ycombinator.com/item?id=2069810
I'm currently working on a project that deals with real time communication (think IRC for the new web) and after fidgeting around with XMPP for a couple of weeks, I gave up on it completely. It's awful. I doubt many people have implemented an XMPP client (or even worse.. a server). Try it some time.
Maybe I could get over the fact that it's so complex -- which puts an unnecessary strain on both servers and clients -- and maybe I could get over the fact that (on average) servers consume 40% more memory than their IRC counterparts. Maybe I could even get over the fact that the spec could change at any moment (the eXtensible nature of it isn't necessarily a good thing, you know).
But what I couldn't possibly in a million years get over is that... it's XML (shudder).
-- To the downvoters:
Here's some previous discussion about why XMPP sucks: https://news.ycombinator.com/item?id=2069810
I'm currently working on a project that deals with real time communication (think IRC for the new web) and after fidgeting around with XMPP for a couple of weeks, I gave up on it completely. It's awful. I doubt many people have implemented an XMPP client (or even worse.. a server). Try it some time.
I understand the frustration of working with a fat protocol. What is a viable, open alternative though?
IRC! :P Seriously though, there isn't a light one afaik.
I ended up making my own -- I based it on IRC but wrapped it in JSON. It's stupid simple because chat should be easy.
I ended up making my own -- I based it on IRC but wrapped it in JSON. It's stupid simple because chat should be easy.
IRC has no standardised way for handling contact lists (some servers have a bot based implementation), no proper offline messaging support (some servers have MemoServ), no easy way to find which of your contacts are online (WHOIS them all and see which are there, I guess?), and no proper permanent identification for users (they can register with NickServ, but on some servers this still allows users to log in with the same nick when the first user is offline, it just means the original owner can kick the other person when they log in).
It's very good for its intended purpose, but it's not good for IM usage.
It's very good for its intended purpose, but it's not good for IM usage.
IRC is so lightweight, it's what NASA uses for their antarctic weather flights.
http://arstechnica.com/information-technology/2013/01/antarc...
http://arstechnica.com/information-technology/2013/01/antarc...
antarctic satellite IRC?
best i:line ever.
best i:line ever.
If you can't name a single protocol that is better at what XMPP does, don't you think "(lets face it) terrible" is a bit of an overstatement? It seems more like a "democracy is the worst system, except for everything else we've tried" scenario.
I mean I can't name something that does what XMPP does is because, well, XMPP does .. like .. everything (or at least tries to). As far as a lightweight protocol is concerned, IRC is much better.
I said IRC semi in jest before because it's very old and there really should be something lightweight that ought to take IRC's place in this web 2.0 world of ours.
I said IRC semi in jest before because it's very old and there really should be something lightweight that ought to take IRC's place in this web 2.0 world of ours.
Why wrap it in JSON?
> All that talk about Client Choice, Service Choice and Platform Choice has been replaced with "if the other big players don't play, why should we?".
It seems like the impact of the other big players not playing is down played here. Did it not allow facebook users to start conversations with google talk users but not vice versa?
Playing the cooperation card first is what google did, but in the iterated prisoners dilemma you often need to play the non-cooperative card if that is what your peers are doing.
That really seemed to be the core of the issue. Now 5-25 years from now maybe google will have an entrenched culture of playing the non-cooperative card and will not reach for the cooperative card first but I do not judge this to be the case yet.
It seems like the impact of the other big players not playing is down played here. Did it not allow facebook users to start conversations with google talk users but not vice versa?
Playing the cooperation card first is what google did, but in the iterated prisoners dilemma you often need to play the non-cooperative card if that is what your peers are doing.
That really seemed to be the core of the issue. Now 5-25 years from now maybe google will have an entrenched culture of playing the non-cooperative card and will not reach for the cooperative card first but I do not judge this to be the case yet.
I believe I get to downplay that because the federation they joined helped grow their own network, and allowed them to do Google Talk for other domains. I know many companies that have moved their XMPP stuff to Google for that.
Now those outside contacts are cut off, without any warning. Worse, they see their inside contacts online, can send them messages that never show up in Hangouts!
Those big players are hardly affected, though. Also, Google could simply block those entities that don't fully federate (only).
Now those outside contacts are cut off, without any warning. Worse, they see their inside contacts online, can send them messages that never show up in Hangouts!
Those big players are hardly affected, though. Also, Google could simply block those entities that don't fully federate (only).
I believe I get to downplay that because the federation they joined helped grow their own network
Do you have any evidence that federation helped Google grow the network in a significant way?
allowed them to do Google Talk for other domains
Are you referring to Google Talk integration with Google Apps For Your Domain? Outside evidence suggests that federation is not used between these domains. For example XMPP SRV records are not required and ignored if present.
Do you have any evidence that federation helped Google grow the network in a significant way?
allowed them to do Google Talk for other domains
Are you referring to Google Talk integration with Google Apps For Your Domain? Outside evidence suggests that federation is not used between these domains. For example XMPP SRV records are not required and ignored if present.
There are a few examples though where Google has opted not to support or develop standards. For example WebM, SPDY and XMPP and not to mention their continued support for Motorola's despicable FRAND abuses.
those are all examples of standards that Google has supported and developed, two of them originating at Google (well, sort of for WebM/VP8...the standardization, at least, originated at Google). I'm not sure what you're on the right track here...just because you come up with a new format does not mean that you're undermining old ones.
That's exactly what the EFF has suggested, in fact: if they don't want to use XMPP, publish the new format for others to be able to use.
That's exactly what the EFF has suggested, in fact: if they don't want to use XMPP, publish the new format for others to be able to use.
It was ultimately a business decision - the same kind that leads to product shutdowns and all the other "evil" things that new Google does.
Oh, a _business_ decision. I guess we can just let them continue to embrace, extend, and extinguish the commons we've built over the last decades, then. No complaints necessary.
(I'm not a GOOG shareholder, so their business needs don't matter to me. I _am_ now going to be cut off from my contacts, though. Part of markets is the feedback that comes with mis-steps, and saying "it's just business" does a dis-service to those of us who feel that Google has been over-stepping their bounds.)
(I'm not a GOOG shareholder, so their business needs don't matter to me. I _am_ now going to be cut off from my contacts, though. Part of markets is the feedback that comes with mis-steps, and saying "it's just business" does a dis-service to those of us who feel that Google has been over-stepping their bounds.)
The [EEE] strategy's three phases are:
Embrace: Development of software substantially compatible with a competing product,
or implementing a public standard.
Extend: Addition and promotion of features not supported by the competing product
or part of the standard, creating interoperability problems for customers
who try to use the 'simple' standard.
Extinguish: When extensions become a de facto standard because of their dominant
market share, they marginalize competitors that do not or cannot support
the new extensions.
XMPP was essentially a non-player before Google used it, and it'll return to being a non-player after they leave. There's no big strategy needed to accomplish this.
This is not to say I support Google's recent decisions, but then again, I had already moved off their applications (Reader and Gmail) way before the shutdowns started, mostly due to the methods used to push Chrome.
They embraced standards via XMPP. They then extended them in proprietary ways, with Hangouts rather than gtalk. They then are deprecating gtalk in favor of Hangouts, and will finally extinguish it.
Hangouts is not an extension of XMPP, it is a replacement.
Google's extensions to XMPP took place via the XEP process, and were published so other vendors could implement them.
Google's extensions to XMPP took place via the XEP process, and were published so other vendors could implement them.
That's exactly the same thing. Once you own most of the implementations running "the standard" you modify it and close it so that nobody can follow your path.
It does not matter if the said modifications are called "G-XMPP" or 'Hangouts'. The result and logic are the same.
The rest is a useless fight on semantics.
It does not matter if the said modifications are called "G-XMPP" or 'Hangouts'. The result and logic are the same.
The rest is a useless fight on semantics.
The Hangouts protocol is not a modified or closed version of XMPP. They have nothing in common except that they're both used to implement IM clients.
SSH is not an extended version of Telnet.
SSH is not an extended version of Telnet.
And then again.. you didn't get the point.
SSH and Telnet do the same thing, but nobody killed off telnet by getting everyone to use their implementation of it, then kill it and propose SSH as alternative.
Thats what Hangout does. Name 5 of your friends without a google account who use internet & mobile phones. See what I mean? They were using XMPP. Everyone could talk to them using XMPP. Not anymore. Wanna talk to them? Gotta use hangout, no way around it.
It does not matter if its based on the same _code_ what matter is that it offers the same functionality AND kills off the previous product by using their market share.
That's the freaking concept.
Thats what Hangout does. Name 5 of your friends without a google account who use internet & mobile phones. See what I mean? They were using XMPP. Everyone could talk to them using XMPP. Not anymore. Wanna talk to them? Gotta use hangout, no way around it.
It does not matter if its based on the same _code_ what matter is that it offers the same functionality AND kills off the previous product by using their market share.
That's the freaking concept.
In the extinguish part, what got extinguished were the competitors', not Microsoft own products.
That is perfectly valid, to me. However, there is no reason to spread misinformation to cover such a decision and to bluntly cut off existing communications between people outside and inside of the Google Hangout domains. Without any warning to the former, even.
A lot of the shutdowns have been due to low (for google) user bases, or just no engineering staff to support it. A lot of their shutdowns I see as a way for google to become more focused as a company, which I feel like was needed. They had too many half-baked services.
As for XMPP, it was their business decision. I'm betting engineering had a good reason for it, it just may not be shared with us.
As for XMPP, it was their business decision. I'm betting engineering had a good reason for it, it just may not be shared with us.
The problem with XMPP is that it requires several open ports (5222, 5223, 5269, 5298, 8010) on the client, on the technical and practical standpoint the reason why XMPP is unpopular to big players because many businesses won't be bothered to open their ports, now if you are a solutions provider creating software for XMPP solutions for those companies, the most viable solution is to use BOSH to direct XMPP traffic to port 80, but there's another big problem, the only available BOSH library is a GPL BOSH library that you would license in order for you to not to ship your software along with it's source-code. Now if you can do the same thing over HTTP 1.1 stream, why bother with XMPP?
* The problem with XMPP is that it requires several open ports (5222, 5223, 5269, 5298, 8010) on the client.*
XMPP does not require any open ports on a client.
XMPP does not require any open ports on a client.
On the server, I mean.
source: I'm an ex-cofounder of a now very successful syncing solution startup who uses XMPP and I wrote the initial server and client code.
source: I'm an ex-cofounder of a now very successful syncing solution startup who uses XMPP and I wrote the initial server and client code.
I have no idea where you got the idea about binding all (any) of those listening ports on the client, but that is incorrect.
anecdotally:
http://wiki.xmpp.org/web/SRV_Records#Ports
anecdotally:
% lsof -i4 -T s
COMMAND PID USER FD TYPE DEVICE
imagent 158 mh 6u IPv4 0x2969ca78b2e9b6bb 0t0 TCP mac:52812->pb-in-f125.1e100.net:5223 (ESTABLISHED)
http://wiki.xmpp.org/web/SRV_Records#Ports
* On the server.
Why not? The same applies to running an HTTP or email server, which many businesses do.
[deleted]
I think I might have 2c to clarify the OP's stand, having worked on a mobile XMPP client before. This is with regards to his point about bandwidth and battery usage on mobile.
XMPP is a connection-oriented stateful protocol. The connection setup process is very expensive: Connect, authenticate, advertise the list of supported client features, get the roster (contacts), and IIRC presence exchange, etc. Therefore, once you've established a connection, you really don't want it to break. Even just reconnecting is very expensive.
However, in the mobile scenario, regular connection breakage is the norm. What most mobile based clients will then do is proxy the XMPP connection on the server-side and connect to the client over HTTP instead. This is well defined in the BoSH spec (XEP:124). BoSH was originally designed for web-based clients, but happens to meet the need of the mobile use-case.
However, when it comes down to implementation, there are several optimizations that can be applied:
1. The roster exchange at the connection time is very expensive, and usually only sends data to the client that the client probably has already stored locally from a pervious session. Connection overhead can be reduced drastically if there was some way of versioning the roster and only transferring deltas. This was the OP was referring to. (Roster versioning, XEP:273).
2. Presence packets are one of the most chatty aspects of the protocol (x went offline, x is now online, etc). I've heard someone say that it constitutes about 80% of the packets on a typical connection, but this is anecdotal. In cases where the user isn't actively interacting with the client, having presence packets being sent all the time seems wasteful. One way to tackle this is to buffer up and de-duplicate the presence packets on the server, and send them in batches only when necessary. (For example, if x goes online, then offline, then online again, and then goes 'busy', the client only needs to know that x is busy.) I think this is what the OP was referring to when he mentioned the google:queue for delayed presence delivery. This alone can be a huge win for battery and bandwidth, since transferring data and processing it can be reduced drastically.
Aside: Presence is so chatty, that some mobile clients simply don't handle presence at all. Take WhatsApp for example. It doesn't show whether every person in your list of contacts is online. The assumption is that every user is online by the nature of the device, and this assumption works in the common cases. In the edge cases is where you'd need to buffer up messages. This reduces chattiness drastically, making the protocol seem much more lightweight. (WhatsApp is based on XMPP, but have spun it off into their own thing.)
Just thought I'd add these points to the discussion here, since there seemed to be some lack of understanding of the protocol. IMHO, the XML, as verbose as it is, isn't really the problem. The protocol itself is verbose and whether the transport uses XML or whatever else is of comparatively little consequence. This verbosity of the protocol (and not so much the format of data transfer) has its effect on battery life and bandwidth usage.
XMPP is a connection-oriented stateful protocol. The connection setup process is very expensive: Connect, authenticate, advertise the list of supported client features, get the roster (contacts), and IIRC presence exchange, etc. Therefore, once you've established a connection, you really don't want it to break. Even just reconnecting is very expensive.
However, in the mobile scenario, regular connection breakage is the norm. What most mobile based clients will then do is proxy the XMPP connection on the server-side and connect to the client over HTTP instead. This is well defined in the BoSH spec (XEP:124). BoSH was originally designed for web-based clients, but happens to meet the need of the mobile use-case.
However, when it comes down to implementation, there are several optimizations that can be applied:
1. The roster exchange at the connection time is very expensive, and usually only sends data to the client that the client probably has already stored locally from a pervious session. Connection overhead can be reduced drastically if there was some way of versioning the roster and only transferring deltas. This was the OP was referring to. (Roster versioning, XEP:273).
2. Presence packets are one of the most chatty aspects of the protocol (x went offline, x is now online, etc). I've heard someone say that it constitutes about 80% of the packets on a typical connection, but this is anecdotal. In cases where the user isn't actively interacting with the client, having presence packets being sent all the time seems wasteful. One way to tackle this is to buffer up and de-duplicate the presence packets on the server, and send them in batches only when necessary. (For example, if x goes online, then offline, then online again, and then goes 'busy', the client only needs to know that x is busy.) I think this is what the OP was referring to when he mentioned the google:queue for delayed presence delivery. This alone can be a huge win for battery and bandwidth, since transferring data and processing it can be reduced drastically.
Aside: Presence is so chatty, that some mobile clients simply don't handle presence at all. Take WhatsApp for example. It doesn't show whether every person in your list of contacts is online. The assumption is that every user is online by the nature of the device, and this assumption works in the common cases. In the edge cases is where you'd need to buffer up messages. This reduces chattiness drastically, making the protocol seem much more lightweight. (WhatsApp is based on XMPP, but have spun it off into their own thing.)
Just thought I'd add these points to the discussion here, since there seemed to be some lack of understanding of the protocol. IMHO, the XML, as verbose as it is, isn't really the problem. The protocol itself is verbose and whether the transport uses XML or whatever else is of comparatively little consequence. This verbosity of the protocol (and not so much the format of data transfer) has its effect on battery life and bandwidth usage.
>Aside: Presence is so chatty, that some mobile clients simply don't handle presence at all. Take WhatsApp for example.
One could also take Hangouts for example. :)
anyways, very informative post and good anecdotes, thank you for sharing.
my understanding of the XMPP spec is that (assuming the server follows the server-client spec, anyways) there's no way to unsubscribe from your contacts' presence notifications and keep them on your roster [and therefore able to be contacted without reauthorization, usually.]
can you confirm/enlighten me? thanks!
One could also take Hangouts for example. :)
anyways, very informative post and good anecdotes, thank you for sharing.
my understanding of the XMPP spec is that (assuming the server follows the server-client spec, anyways) there's no way to unsubscribe from your contacts' presence notifications and keep them on your roster [and therefore able to be contacted without reauthorization, usually.]
can you confirm/enlighten me? thanks!
Presence subscriptions don't have to be symmetrical in XMPP, and indeed you can even have contacts on your roster that don't share presence in either direction. As for exchanging messages, as far as I know Google's implementation is the only one that doesn't allow that without presence subscription. All others do.
In fact, this caused presence subscription requests to be the only attack vector in terms of spam in Google talk, which some argue makes it harder to combat. Mostly because there is less data available to detect malicious behaviour.
In fact, this caused presence subscription requests to be the only attack vector in terms of spam in Google talk, which some argue makes it harder to combat. Mostly because there is less data available to detect malicious behaviour.
If the issue is with mobile, why not build a MQTT <-> XMPP bridge or just support MQTT as an alternate protocol?
http://en.wikipedia.org/wiki/MQ_Telemetry_Transport
http://en.wikipedia.org/wiki/MQ_Telemetry_Transport
1. Microsoft/Facebook (and maybe more corporates) didn't play by the rules. They don't believe in "give and take", they just do "take, take,..". Google didn't like it so walked out. Instead of showing the sportsman's spirit and instead d of carried on despite of all the follies they simply left having pissed off. Good decision.
2. Universities, organisations? The number is too small to be bothered about, esp when compared with the likes of the two above^. They would rather innovate and try to lock-in users into their own platforms rather than hoping someday Cuck might turn the benevolent saint.
3. Whatever you say but unless you are short of manpower and talent it's actually easier difficult to innovate in-house sth proprietary and not when you've to conform to a "standard" and that too when the rewards are [1].
4. Even I don't like the move but it was a business decision and this one made a lot more sense than killing off GR.
The surprising fact here is, we hardly see anyone criticizing the providers that chose not to share back. We, keep on saying "no problems with XMPP" and "even though MS and Fb were not giving back, Google should have just let them free-load after all it's a do no eveil corp". Huh.
2. Universities, organisations? The number is too small to be bothered about, esp when compared with the likes of the two above^. They would rather innovate and try to lock-in users into their own platforms rather than hoping someday Cuck might turn the benevolent saint.
3. Whatever you say but unless you are short of manpower and talent it's actually easier difficult to innovate in-house sth proprietary and not when you've to conform to a "standard" and that too when the rewards are [1].
4. Even I don't like the move but it was a business decision and this one made a lot more sense than killing off GR.
The surprising fact here is, we hardly see anyone criticizing the providers that chose not to share back. We, keep on saying "no problems with XMPP" and "even though MS and Fb were not giving back, Google should have just let them free-load after all it's a do no eveil corp". Huh.
At least Microsoft has XMPP federation support in Lync: http://en.wikipedia.org/wiki/Microsoft_Lync_Server#XMPP. Otherwise, yes, Skype and Facebook should add federation support. The actual protocol isn't even that important, as long it can be implemented freely.
I'm interested in how you think MSFT and FB just "took, took, took..."
Thank you for asking.
Seems you have not been paying attention recently. Certainly not in the contxt of microsoft+facebook+gtalk+xmpp+google+talk one way, just google it; but that's my assumption. This is what I meant by "took...3 times".
If it was sth else you wanted to know or were referring to then I obviously failed to grok that and you'll have to help me understand.
Seems you have not been paying attention recently. Certainly not in the contxt of microsoft+facebook+gtalk+xmpp+google+talk one way, just google it; but that's my assumption. This is what I meant by "took...3 times".
If it was sth else you wanted to know or were referring to then I obviously failed to grok that and you'll have to help me understand.
Isn't there a better question to be asked here? Federation is supposed to enable, well, federation. Google is not preventing XMPP federation in any way. Google is not "shutting down" XMPP or killing it.
Why is Google the company important to the XMPP community? I scarcely see that it should matter.
Open protocols are not exclusive clubs or infinite universes, anyone can opt in or out at any time, the rest of us will live.
I think its great when big companies such as Google or whoever participate in open standards, but I also appreciate that they are in the business of providing the best product for their users (in their eyes) and are free to do what they want.
When they do participate, we are getting a gift from them. Its not their obligation, right or even necessarily to their benefit. When they find other priorities fine, maybe someone more committed will move the project forward. And in other cases their NON-participation can be a gift as well.
So whats the big deal?
Why is Google the company important to the XMPP community? I scarcely see that it should matter.
Open protocols are not exclusive clubs or infinite universes, anyone can opt in or out at any time, the rest of us will live.
I think its great when big companies such as Google or whoever participate in open standards, but I also appreciate that they are in the business of providing the best product for their users (in their eyes) and are free to do what they want.
When they do participate, we are getting a gift from them. Its not their obligation, right or even necessarily to their benefit. When they find other priorities fine, maybe someone more committed will move the project forward. And in other cases their NON-participation can be a gift as well.
So whats the big deal?
Google Talk was the largest federated XMPP server, and it provided an easy way for people using XMPP to talk to non-technical users. "I'll add you on XMPP" = blank stares, "I'll add you on Google Talk, just use your Gmail account" works, even if you yourself are not using Google Talk.
Google said: "For example, mobile has several requirements around bandwidth and battery that are simply not part of the standard"
He says: "This glances over the the fact that the X in XMPP stands for eXtensible, which still results in proposals for new protocol extensions every month."
His misinformation is this: XMPP is XML based. The bandwidth that Google most likely refers to is a result of verbosity of XML. You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know.
The only way to fix that specific problem is to change the protocol. Which is what Google did.
This is an example a bad advocacy: badmouthing companies that decided to no longer use something you work on.
I also find troubling the pounding Google gets for every little mis-step.
When it comes to using and promoting open technologies Google is seemingly from another planet compared to other tech giants (Apple, Microsoft, Facebook, Amazon, you name it).
And yet the furor is not about Microsoft using completely secret protocol (Skype), not about Apple promising to publish their secret protocol (Facetime) and then not doing it, not about Apple preventing fair browser competition in iOS etc. but about Google no longer using XMPP.
A lack of perspective.