JSON+UDP+DHT=Freedom(telehash.org)
telehash.org
JSON+UDP+DHT=Freedom
http://telehash.org/
14 comments
This wiki page got me much closer to understanding what's really going on here: https://github.com/quartzjer/TeleHash/wiki/My-Understanding-...
This is very interesting. We looked at storing stuff in BitTorrent's Kademlia DHT to do roaming NAT-traversal in Mosh, but their performance scared us off. Before building an application on this I would want to ask:
- Who runs telehash.org? Will they still be around in five years? Is there a well-known list of anchor nodes I can hardcode into my app, or how does the program find the DHT when it first starts?
- What is the median time to retrieve a key? What will it be in 24 months?
- What is the percentage of failed requests? Can we guess what that will be in 24 months?
The "mainline" BitTorrent Kademlia DHT started off great but now the statistics are apparently abysmal. In one academic study, the median time to retrieve a key was over a minute, and 20% of retrievals failed. (See http://www.cs.rice.edu/~scrosby/tr/BTMeasure-Main.pdf)
Why won't the same fate befall this Kademlia DHT?
(Edit: is this _A_ DHT, or is the idea that if you build your program to use Telehash, you are creating your own DHT that is only used by currently-running instances of the same program?)
- Who runs telehash.org? Will they still be around in five years? Is there a well-known list of anchor nodes I can hardcode into my app, or how does the program find the DHT when it first starts?
- What is the median time to retrieve a key? What will it be in 24 months?
- What is the percentage of failed requests? Can we guess what that will be in 24 months?
The "mainline" BitTorrent Kademlia DHT started off great but now the statistics are apparently abysmal. In one academic study, the median time to retrieve a key was over a minute, and 20% of retrievals failed. (See http://www.cs.rice.edu/~scrosby/tr/BTMeasure-Main.pdf)
Why won't the same fate befall this Kademlia DHT?
(Edit: is this _A_ DHT, or is the idea that if you build your program to use Telehash, you are creating your own DHT that is only used by currently-running instances of the same program?)
> (Edit: is this _A_ DHT, or is the idea that if you build your program to use Telehash, you are creating your own DHT that is only used by currently-running instances of the same program?)
I had assumed the later, but the existence of telecast.org:42424 kind of points against it. It's possible that we've got both going on.
I had assumed the later, but the existence of telecast.org:42424 kind of points against it. It's possible that we've got both going on.
I am a programmer yet still I don't get what this is. If an app wants to send JSON to another app why not use HTTP for that. DNS will take care of the routing. The only thing that seems novel to me is their use of UDP which is faster then HTTP.
Is it to bypass DNS? With all the moves to control the internet around, then an encrypted peer-to-peer decentralised system that can deal with any kind of data would allow for DNS-less email / websites etc etc...
Though the main point of DNS is it manages a namespace, and you wouldn't want clashes, so websites would probably have to have urls like thttp://{some 64char hash}/
(Where the h stands for telehash...
Though the main point of DNS is it manages a namespace, and you wouldn't want clashes, so websites would probably have to have urls like thttp://{some 64char hash}/
(Where the h stands for telehash...
The use of UDP allows for "hole-punching".
User A wants to send and receive messages with user B. Both are identified only by their IP and Port pair. User B does not know of user As intentions. So user A send switch C an UDP packet asking for "hole-punching". Switch C has this service where users behind NAT routers connect and signup to be contacted when someone wants to "hole-punch" to them. Switch C sends a packet to B with info about A. B sends A a packet and Bs router is now expecting packets from A. Has soon has A sends its first packet to B its router is open for packets from B. Now both A and B can send packets to each other without C.
DNS is not yet entirely out of the loop. The main switch for telehash is telehash.org:42424
User A wants to send and receive messages with user B. Both are identified only by their IP and Port pair. User B does not know of user As intentions. So user A send switch C an UDP packet asking for "hole-punching". Switch C has this service where users behind NAT routers connect and signup to be contacted when someone wants to "hole-punch" to them. Switch C sends a packet to B with info about A. B sends A a packet and Bs router is now expecting packets from A. Has soon has A sends its first packet to B its router is open for packets from B. Now both A and B can send packets to each other without C.
DNS is not yet entirely out of the loop. The main switch for telehash is telehash.org:42424
This is the ideal situation. It only works, however, for a subset of routers.
Here is the bit I don't get - how do you do a lookup to get that hash?
i.e. Map from something user friendly (e.g. username) to a SHA1 hash?
i.e. Map from something user friendly (e.g. username) to a SHA1 hash?
From the Wiki:
[...]End: a SHA-1 hash key (40 hex characters, 160-bits) stored in the global DHT and distributed between switches. Switches will distribute and look up ends in the global DHT that are important to them. TeleHash defines the SHA-1 hash of the external IP:PORT of a switch as part of the protocol. Applications built on top of TeleHash can add their own ends to the DHT like hashes of files, hashes of e-mail addresses or hashes of other application-specific values (e.g. user@chat).[...]
So if i understand this right, every packet has a protocol-defined +end and the application can add it's own end on top (i.e. an Email adress?).
https://github.com/quartzjer/TeleHash/wiki/My-Understanding-...
[...]End: a SHA-1 hash key (40 hex characters, 160-bits) stored in the global DHT and distributed between switches. Switches will distribute and look up ends in the global DHT that are important to them. TeleHash defines the SHA-1 hash of the external IP:PORT of a switch as part of the protocol. Applications built on top of TeleHash can add their own ends to the DHT like hashes of files, hashes of e-mail addresses or hashes of other application-specific values (e.g. user@chat).[...]
So if i understand this right, every packet has a protocol-defined +end and the application can add it's own end on top (i.e. an Email adress?).
https://github.com/quartzjer/TeleHash/wiki/My-Understanding-...
OK, I was being a bit thick there, to look up a email address you presumably simply hash it (presumably with some kind of canonicalization) and use that as the key for your lookup for a matching +end.
I'm not sure, but this is what i think i get out of this. You really don't need an email, this could be a hash or whatever. i think of more apps or services use some protocol like this, there should be a unified why to "lookup" for an account or something.
For example, if you have a decentralized social network that is based on the same software, say "facespace", every of those software nodes would lookup for the same identifier. i guess...
For example, if you have a decentralized social network that is based on the same software, say "facespace", every of those software nodes would lookup for the same identifier. i guess...
I guess it might make sense to do a lookup based on a hash of a json query like:
{ "facespace":"arethuza" }
{ "facespace":"arethuza" }
This would be a phenomenal way to build something like app.net.
With full decentralization it could really become a true 'protocol'.
With full decentralization it could really become a true 'protocol'.
There is nothing about App.net that is decentralized. Why do people think it is decentralized? It is a walled-garden with a better gate policy.
>It is a walled-garden with a better gate policy.
...until they need money.
...until they need money.
$50 per user per year ought to be sufficient.
With a more expensive gate policy, it might not be a better one :)
This is absolutely Awesome, but the UDP part is kind of scary for any system that handles static data (not games, or stuff)
You want some kind of enterprise messaging system or message queue, then, like Rendezvous, Spread, Talend, RabbitMQ, ActiveMQ, ZeroMQ, etc.
Pendantic and off-topic, but it irks me when ZMQ gets lumped in with i.e. RabbitMQ and AMQ. ZMQ is more like an improved socket library and tools for building distributed message queues (among many other things), while RMQ and AMQ are actually message queue applications.
It is, however, much better than plain old UDP =)
I can just glimpse the promise this holds, but I don't really understand one would implement this. Does anyone have a blog post with example code?
If I've got a server that needs to receive messages, do I register it as an 'end'? And do I then send messages to this 'end' from other clients? How does it prevent two parties from registering the same 'end' hash?
If I've got a server that needs to receive messages, do I register it as an 'end'? And do I then send messages to this 'end' from other clients? How does it prevent two parties from registering the same 'end' hash?
end is a SHA1 of the ip+port of the endpoint so there won't be any collisions (hopefully)
There is some version 2.0 on its way:
https://github.com/quartzjer/TeleHash/commit/14cecbb2e58dca4...
And if you don't like reading HTML:
http://telehash.org/v2.html
http://telehash.org/v2.html
Why UDP? And Why create one implementation for each language?? Why not a local server?
To the "Why UDP?" part:
For things like P2P UDP is a much better fit, because TCP sends an ACK (ACKnowledgement) to the sender for every received packet.
Since UDP is unreliable no such ACKs will be send and it's up to the programmer to detect packet loss. The latency is therefore with UDP lower (and hence throughput higher), because the provider can push packets without waiting for a response.
For instance in P2P the receiver only needs to send a NAK (Negative AcKnowledgment) if and only if a packet/part is missing and then it will be resend by the provider.
For things like P2P UDP is a much better fit, because TCP sends an ACK (ACKnowledgement) to the sender for every received packet.
Since UDP is unreliable no such ACKs will be send and it's up to the programmer to detect packet loss. The latency is therefore with UDP lower (and hence throughput higher), because the provider can push packets without waiting for a response.
For instance in P2P the receiver only needs to send a NAK (Negative AcKnowledgment) if and only if a packet/part is missing and then it will be resend by the provider.
TCP sends an ACK (ACKnowledgement) to the sender for every received packet.
Not quite. TCP receivers only need to send ACKs when they have received enough data to fill the sliding window. That may be every packet, every other packet (the recommended by the RFC) or more.
Not quite. TCP receivers only need to send ACKs when they have received enough data to fill the sliding window. That may be every packet, every other packet (the recommended by the RFC) or more.
You could have problems with NAT, but SIP protocol has STUN and ICE which are responsible for dealing with this. Check that out.
Btw i had idea SIP + JSON/HTML + DHT which would not suffer with "man in the middle attack" that HTTP has as designed decision. You could connect, for example, with HTTP/FTP/... protocols to your buddies directly. You just have to make right SDP packet when calling them ...
Btw i had idea SIP + JSON/HTML + DHT which would not suffer with "man in the middle attack" that HTTP has as designed decision. You could connect, for example, with HTTP/FTP/... protocols to your buddies directly. You just have to make right SDP packet when calling them ...
UDP+DHT+NAT Traversal is very powerful. I implemented a new p2p radio platform called HD Radio.NET (http://hdradionet.zapto.org/) which is completely based on them. It shows to be very effective. In this news, Jason is a new idea, it may bring some benefits, for example, running something in browser.
Reminds me of peercast. Which seems to be quite dead, unfortunately.
In you FAQ it says it is cross platform, but there is only Windows software on the download page.
In you FAQ it says it is cross platform, but there is only Windows software on the download page.
This is a nice mash-up. It would have been nice to preserve Kademlia's symmetrical distance metrics, though.
> to discover its public IP:PORT
What about switches that want to run from behind a NAT? Could it be that UDP was chosen with this in mind? (http://en.wikipedia.org/wiki/UDP_hole_punching)
What about switches that want to run from behind a NAT? Could it be that UDP was chosen with this in mind? (http://en.wikipedia.org/wiki/UDP_hole_punching)
I was wondering how this approach deals with systems running behind NAT also. With IPv6 not happening anytime soon, there will be a large number of systems without individual public addresses available. Routing this UDP packets may require tweaking firewall rules, etc.
if server a is behind a nat, it first needs to send out a message to server b this will open a random port for typically 60 seconds automatically. Then another message will need to originate from server a within the 60 second time period in order to keep the port open.
http://en.wikipedia.org/wiki/NAT_traversal
http://en.wikipedia.org/wiki/NAT_traversal
Switches behind NAT either have to configure a port to receive UDP directly or depend on some other switch for hole-punching.