It is very fast because it is not anonymous first. Although it is 100% designed with being non-leaking over Tor. It already works over proxychains great. I will add SOCKS5 support soon.
Also, don't forget the distributed spam resistant automatically encrypted and transparently authenticated mail:
Dpush is distributed /unsolicited/ POST :) Solves the previously open problem perfectly.
MORPHiS hosted MORPHiS website:
morphis://sp1nara3xhndtgswh7fz
OR
localhost:4251/sp1nara3xhndtgswh7fz
URL is a hash of the data or the key that signed it. No MITM possible.
The next module I am implementing is DDS - Distributed Discussion System. It is quite easy because it is fully enabled by the existing Dpush invention that already powers MORPHiS Dmail.
I should have mentioned one very important point with the figure I provided. That is 115 requests per second on /one/ i3 /core/. The network itself isn't sweating. Every user of a distributed Reddit for example would be getting those 115 requests per second themselves if they weren't a sizable portion of the network. So 100 users and your figure is matched. The bittorrent mainline DHT has multiple millions of simultaneous nodes at any one time. MORPHiS as is already deprecates bittorrent so is destine to absorb all those nodes. Also, the network scales log, and not just log base 2. Kademlia has an 'accelerated lookup' where you can control the base with memory cost to achieve O(log base 2^b) lookups. A b=3 is a resonable value for memory usage.
I should have mentioned, it is currently limited to one core, due to python. I will make it multicore probably before 1.0. It will be relatively easy to do with the block based design of morphis and the multiprocessing pool api of python. I already use the multiprocessing to great effect in the proof of work and prefix generation.
Also, this is written in a scripting language, Python. Also it is first draft and unoptimized.
If the 115 requests is enough of a problem, which I don't see it being because that is per node, not of the whole network. If that is a problem and can't be improved enough with python, the idea was originally to port it to Rust. Rust wasn't even 1.0 when I started coding, never mind their asyncio io library didn't exist until a couple months ago. Come the Rust port using their newly released asyncio library which performs nearly as well as the libev C well known one, we will be talking the kind of performance you are talking, although 10k requests per second just isn't needed on one node. It is certainly doable though if given the time! Remember, a distributed app doesn't run on one node, and thus need all 10k requests per second on one node.
Yes, on my 2nd generation low power i3 running my normal desktop at the time, I get individual ~200ms response time, and that is with 115 32kbyte requests per second.
I am rewriting the high level protocol code that is a bit rickety because it is snowballed from the earliest code in the project other than the asyncio SSH library I implemented from scratch.
When that rewrite is done (a week or two), that will decrease latency greatly, and improve efficiency greatly, and thus should even improve throughput (although that is already max out your pipe with actual data as it is low overhead).
If I switch from 32k blocks to 512k blocks which I did some testing on (Freenet is 1meg blocks), that gives me a 10x !! throughput improvement with same CPU usage and no increase in latency of per request.
The only reason I am 32k blocks originally was the ssh protocol is 35k max packet size and I don't want to break spec so as to be able to hide as normal ssh traffic :) The 512k blocks test I did as multiple packets and was 10x faster, because that means 512k per FindNode operation instead of just 32k :) I was sitting on the fence on switching to it because I want to do the rewrite of the high level code first because the multiple data packets per request complicates that snowballed code even more :)
Also, I am using pycrypto which initial tests show is actually much slower than the other library I will likely switch to (it is called simply cryptography, it wraps platform openssl instead of implementing itself as pycrypto does). I went with pycrypto to minimize dependencies. I will have it detect if you have cryptography installed and use that optionally. I've already abstracted the pycrypto api so I can easily have it switchable at runtime. This should decrease latency a good amount as well.
Very good point. I'm not sure I ever said 'I' in that context, I certainly don't mean so.
I see myself as just the campaign manager for the World Brain, and one of the first coders of this component of it. The World Brain, is you.
Others working on open hardware, hardware mesh net (wireless, Etc.), Etc., all of these people have already done just as much as I or more in making necessary components. So it certainly isn't about me. In this post-Snowden world, so many people know exactly what to do now, it is amazing really, so many people united and working together and knowing exactly what is needed!
Thanks for that input still! I will consider that in how I word stuff in the future. I am just /very/ excited about what I've already calculated as a certain path and inevitable future for humanity.
I found out late in my coding (only 2-3 months ago) that H.G. Wells certainly beat me in 1937 to the idea, no, realization, of a unified human consciousness :) (See his essay entitled World Brain.)
FYI, your nodes ID is a SHA-512 hash (everything is SHA-512 hash) of your node's RSA-4096 public key. (All crypto is: SHA-512, RSA-4096, and DH-group14-sha1 (for compatability with openssh)).
That relation of your node's ID is one of the Sybil proofing measures designed early on.
I designed everything to be transparent and modular with crypto so I can switch to better algos as they become available (quantum proof, I am watching you!). The first addition will likely be ECC once I have decided upon a safe (as in non NSA backdoored) ECC curve which I believe the community has not proven yet as safe as RSA-4096.
Data is broken up into 32k blocks. (That might change to 512k blocks as it gives it a 10x throughput improvement with less CPU / overhead.)
Random of those blocks, based on your nodes ID and the hash of each block's data, are stored on your computer. Before they are, your node encrypts them and throws away the key. All that is ever stored on your harddrive is random encrypted data and no key. The key is the original key that the uploader is provided with. The encryption key is the hash of the data, the ID of the data is a hash of that key. Your node knows only the ID, it cannot derrive the key to decrypt the data. Your node cannot be tricked into storing unencrypted data.
Now, a soon future version will support seeding v1. This is the ability to essentially 'like' a file/website/etc, and that will ensure that your node never throws out the data blocks it does store for that file/site/files_that_site_links_directly. This is mostly UI work that is missing. This feature will also automatically seed for example your Dmails, comments, trust publications, own site, Etc. (Although a UI will let you control it).
After that, more distant, will be seeding v2. This will be more like the file sharing that is similar in concept to how https://peeriodproject.github.io/dl/peeriod_an-anonymous-app... envisions file sharing. Their paper came out as I was coding and I noticed they were the closest in many ideas to mine of any papers I've seen. Their project seems stalled, as most coming out of the academic world are. That is why I started with code and not a paper :) I will write the paper before 0.9 release.
Thank you for that feedback. You make a good point. I will reorganize the instructions to be more up front instead of buried as you discovered :)
I went and made the windows install a one click affair, but neglected to realize the Linux one, while likely easy to install, does not launch the browser like the windows batch file does and thus unless you read the RUNNING file then you wouldn't even know it had the web UI! Which is the whole point of running it :)
I will do as you suggest and move that into the README, and as well document the fact of the web UI and its port on the website. Thank you so much!
In order to be Python + High Performance I went with asyncio, which is amazing, blew me away in performance, very glad I went with it. However, that unfortunately means it needs 3.4.0 or greater.
Just download Python 3.4 and compile yourself. It is dead simple (./configure; make; make install). It then can be referred to as python3.4, pip3.4, Etc., not conflicting at all with your existing system.
Sub second latency, and it will already download as fast as your pipe will allow for most people, and it will only get orders of magnitude faster! This is first draft, tech preview, yet due to how good its design is, it is fully functional and beyond what is out there.
The end goal of my life's work of which MORPHiS is only the beginning is The World Brain. That is, humanity linked together as a unified conciousness. No matrix style needed; a keyboard is enough. An uncensorable high performance internet with an inherent web of trust acting as the synapses and our brains acting as the individual super neurons of that then neural net.
Wells is not talking about a centralized wikipedia. He is talking about a decentralized (even mentions Iceland as a safe harbour) and unified human conciousness.
Humanity can then determine its own destiny instead of having to use the stone-age system of paper ballots and inherenlty corrupt elected representatives who's representation has absolutely zero correlation with what the people they are claiming to represent actually want.
For the World Brain to function, it needs a ultra high performance and non-kill-switch able distributed datastore to operate upon. Thus, MORPHiS is the foundational layer for that end. Of course such a layer also inherently is a better file sharing app than BitTorrent, a better communication platform than Email (See Dmail, already working great!), Etc., a better comment system than Disqus, a better Forum than HN. Expect the Disqus deprecating implementation out by the end of this month is my estimate.
Distributed HTTP: Maalstroom on MORPHiS :)
GPLv2 unlike Bittorrent Inc.'s Mælström
https://morph.is
It is very fast because it is not anonymous first. Although it is 100% designed with being non-leaking over Tor. It already works over proxychains great. I will add SOCKS5 support soon.
Also, don't forget the distributed spam resistant automatically encrypted and transparently authenticated mail:
https://morph.is/v0.8/dpush-whitepaper.odt
Dpush is distributed /unsolicited/ POST :) Solves the previously open problem perfectly.
MORPHiS hosted MORPHiS website:
morphis://sp1nara3xhndtgswh7fz OR localhost:4251/sp1nara3xhndtgswh7fz
URL is a hash of the data or the key that signed it. No MITM possible.
The next module I am implementing is DDS - Distributed Discussion System. It is quite easy because it is fully enabled by the existing Dpush invention that already powers MORPHiS Dmail.