How to create a private Ethereum network(omarmetwally.wordpress.com)
omarmetwally.wordpress.com
How to create a private Ethereum network
https://omarmetwally.wordpress.com/2017/07/25/how-to-create-a-private-ethereum-network/
7 comments
A blockchain is secure because in order to "hack" into the database you need to have 51% of the computational power on the network to create the longest chain. In a large network like Bitcoin or Ethereum the massive computational power of the network ensures generating 51% of the compute power of the network is near impossibly expensive. How secure is a private blockchain? If there are only a few servers the network could be destroyed by just unplugging a few machines, or plugging in a few more.
I recommend reading this: https://blog.ethereum.org/2015/08/07/on-public-and-private-b... . In a private blockchain the reading and writing permissions are controlled, therefore only selected nodes can write to it.
"The validators are known, so any risk of a 51% attack arising from some miner collusion in China does not apply."
I guess you could still have attack from authorized nodes in your own organization?
"The validators are known, so any risk of a 51% attack arising from some miner collusion in China does not apply."
I guess you could still have attack from authorized nodes in your own organization?
Well, if you only need to maintain transactions within your own organization and there is full trust, why not just use a simple SQL database to record transactions instead of a much more expensive and complex blockchain? If there are multiple parties, what's to stop one of the nodes from, say, buying 1000 GPU instances on EC2 and massively increasing their hashing power. Even if you have some type of network control settings that only allow certain nodes access to the network, the proof of work could be distributed and you could just proxy the answer back to the machine that has access to the network.
The case for private ledgers is coordination among a moderate number of non-anonymous mostly-untrusted entities. (Whether this case ever exists is left as an exercise for the reader.) They can't use a central database but they don't need mining either; variants of BFT can be used that work as long as (n/2)+1 entities are honest.
If things are non-anonymous, you are far far better off having highly fungible but also highly audited databases.
Make it easy to do but also easy to undo anything, while also having a very large amount of transparency in the system so any "bad actor" can be dealt with swiftly.
That seems to work for the banking system, where any malicious transactions are easily "reversed" (often just reversed on one side actually, 'paid for' by insurance / banks) and a large amount of auditing which makes it possible to clamp down on the bad actors quickly.
Abusing the system then no longer becomes a technical challenge so it removes the possibility that someone with the right technical prowess can "break" the system. Robbing a bank is the easiest thing in the world, just walk in and ask for money, it's getting away with it that's difficult.
Make it easy to do but also easy to undo anything, while also having a very large amount of transparency in the system so any "bad actor" can be dealt with swiftly.
That seems to work for the banking system, where any malicious transactions are easily "reversed" (often just reversed on one side actually, 'paid for' by insurance / banks) and a large amount of auditing which makes it possible to clamp down on the bad actors quickly.
Abusing the system then no longer becomes a technical challenge so it removes the possibility that someone with the right technical prowess can "break" the system. Robbing a bank is the easiest thing in the world, just walk in and ask for money, it's getting away with it that's difficult.
> That seems to work for the banking system, where any malicious transactions are easily "reversed" (often just reversed on one side actually, 'paid for' by insurance / banks) and a large amount of auditing which makes it possible to clamp down on the bad actors quickly.
I work for a bank on blockchain projects. It's almost never this simple. You'd be suprised how much banking is paper based right now for the simple reason that no party wants a central solution controlled by any other party.
I work for a bank on blockchain projects. It's almost never this simple. You'd be suprised how much banking is paper based right now for the simple reason that no party wants a central solution controlled by any other party.
That's fascinating. Could you tell more about that, or maybe provide a link to more information? I find the topic interesting, and as a geek am inclined to find blockchain stuff cool and useful, but I know very little about the real-world situation/concerns.
A lot of companies in all kinds of sectors (finance, healthcare, etc) are looking into blockchain (or DLT) because it allows for a database model that is not centralised. This is great in situations where you don't fully trust other parties you are doing business with.
A lot of these companies are doing experiments using private blockchains (like the private ethereum network linked in OP). However depending on how much you trust the "other" parties allowed to mine (or sign blocks), you might still have the risk where one or more conspire against you and 51% attack you. Periodically putting a blockheader of your private chain into a public blockchain would be a great solution to this, as pulling off a 51% attack on bitcoin is rather hard.
A lot of these companies are doing experiments using private blockchains (like the private ethereum network linked in OP). However depending on how much you trust the "other" parties allowed to mine (or sign blocks), you might still have the risk where one or more conspire against you and 51% attack you. Periodically putting a blockheader of your private chain into a public blockchain would be a great solution to this, as pulling off a 51% attack on bitcoin is rather hard.
Thanks for the info! Do you perhaps have some suggestions for what a good 'next step' is when it comes to understanding all this in more detail?
Can you speak more to where your industry is now in adoption? If there were some "major event" would companies like yours be ready to jump ship into cryptos quickly or are they going to be too late?
I read the article and he doesn't really answer why this is beneficial. If you have an arbiter of truth, isn't it just a really inefficient database?
Exactly. A blockchain is supposed to provide a set of guarantees so the nodes don't have to trust each other. If there is already trust, why bother with a blockchain in the first place? If there isn't trust, then there's definitely a possibility that one of the untrusted parties could try and attack or disrupt and the shared ledger.
Trust is not always so black and white (as on the open internet where anonymous people try to scam each other), in a lot of business cases there are different levels of trust which needs to be handled in a different way.
> The validators are known, so any risk of a 51% attack arising from some miner collusion in China does not apply.
That's just wrong. Dangerously so. Maybe I'm just missing something in his protocol spec and he didn't really mean mining...
I've discussed this attack vector in more depth here[1]. Even if you made the mining function signing the block with your sk it'd still be wrong. Even if you forced the sk holder to have that same sk protect something really valuable, it would still be wrong... just have a larger disincentivize.
The whole point of mining is that it's non-auditable (w.r.t. who is physically mining) -- the only real metric you get is the speed of success which you can always mask by not releasing a block immediately. That's why PoW is a cockroach of a consensus protocol.
In the "we roll a dice to see who gets to append to the ledger next" metaphor for consensus, deterministic and PoS both require you to know the number of sides the dice have before you roll them. In PoW, you don't figure out the number of sides until after.
[1]: http://kadena.io/blog/MiningInPrivate.html
That's just wrong. Dangerously so. Maybe I'm just missing something in his protocol spec and he didn't really mean mining...
I've discussed this attack vector in more depth here[1]. Even if you made the mining function signing the block with your sk it'd still be wrong. Even if you forced the sk holder to have that same sk protect something really valuable, it would still be wrong... just have a larger disincentivize.
The whole point of mining is that it's non-auditable (w.r.t. who is physically mining) -- the only real metric you get is the speed of success which you can always mask by not releasing a block immediately. That's why PoW is a cockroach of a consensus protocol.
In the "we roll a dice to see who gets to append to the ledger next" metaphor for consensus, deterministic and PoS both require you to know the number of sides the dice have before you roll them. In PoW, you don't figure out the number of sides until after.
[1]: http://kadena.io/blog/MiningInPrivate.html
Not necessarily a miner collusion in China, just one of the authorized miners spinning 1,000 EC2 instances to make a 51% attack.
Private blockchains have client nodes in a closed network or otherwise require some form of "API key" to be procured from the chain's owner to be able to connect.
In this case, because there is centralisation of trust, the 51% attack is not possible as there is only one mining node or, otherwise, only one "actor" can actually add blocks to the chain with as many nodes as needed. This is the case with Ripple, for instance.
In this case, because there is centralisation of trust, the 51% attack is not possible as there is only one mining node or, otherwise, only one "actor" can actually add blocks to the chain with as many nodes as needed. This is the case with Ripple, for instance.
So why bother with a blockchain at all?
You mean what are the benefits of a private blockchain?
The trust model is different but all other capabilities are kept intact, so in the case of Ethereum you would still have the distributed singleton and immutable code that will always execute in the same way anywhere in the network, at any point in time.
For established businesses entering the space, the privacy of closed network compensates the absence of "secrets" to some extent. It is also quite useful as a training ground because of course the technology is still in its infancy.
The trust model is different but all other capabilities are kept intact, so in the case of Ethereum you would still have the distributed singleton and immutable code that will always execute in the same way anywhere in the network, at any point in time.
For established businesses entering the space, the privacy of closed network compensates the absence of "secrets" to some extent. It is also quite useful as a training ground because of course the technology is still in its infancy.
So what's the use case?
Store and transfer of digital assets?
I guess you can say it's not needed but then again what is?
Some paper and a pen might solve your business problem but you shouldn't extrapolate that all business cases would be met with those capabilities.
I do think there is plenty of room for private blockchains, but the use cases will be specific to the needs and constraints of the stakeholders.
Some paper and a pen might solve your business problem but you shouldn't extrapolate that all business cases would be met with those capabilities.
I do think there is plenty of room for private blockchains, but the use cases will be specific to the needs and constraints of the stakeholders.
> I guess you can say it's not needed but then again what is?
There's a difference between "not needed" in the sense that I don't need a calculator when I have an abacus, and "not needed" in the sense that I don't need an abacus when I have a calculator.
> the use cases will be specific to the needs and constraints of the stakeholders
Yes, and people are asking for suggestions of what those use cases might be. If you can't think of any, that suggests we might be looking at an abacus, not a calculator.
There's a difference between "not needed" in the sense that I don't need a calculator when I have an abacus, and "not needed" in the sense that I don't need an abacus when I have a calculator.
> the use cases will be specific to the needs and constraints of the stakeholders
Yes, and people are asking for suggestions of what those use cases might be. If you can't think of any, that suggests we might be looking at an abacus, not a calculator.
I do think of many use cases for private blockchains, I simply did not presume to know what you meant before asking.
- One case would be when a corporation wants to have 2500 TPS with finality in 3 seconds using the current ethereum software, unchanged. (scalability)
- One case would be when a large-enough-corporation needs to share master data and reference data of business critical assets through different subsidiaries and third parties. (privacy)
- Remember intranets? Sometimes decision makers simply want a watered down version of a great thing. (stability, control)
If you care to include commercial DLT products, then the use cases will be different again.
- One case would be when a corporation wants to have 2500 TPS with finality in 3 seconds using the current ethereum software, unchanged. (scalability)
- One case would be when a large-enough-corporation needs to share master data and reference data of business critical assets through different subsidiaries and third parties. (privacy)
- Remember intranets? Sometimes decision makers simply want a watered down version of a great thing. (stability, control)
If you care to include commercial DLT products, then the use cases will be different again.
All of those are better done using other technologies. I don't see what the benefit is of shoehorning a blockchain in there.
What's the killer app? Sorry to be annoying but I honestly don't see why an CIO would choose a blockchain over more traditional options for anything at all. I've been following Bitcoin and ETH for many years and I've been thinking about this a LOT and I just don't see it.
What's the killer app? Sorry to be annoying but I honestly don't see why an CIO would choose a blockchain over more traditional options for anything at all. I've been following Bitcoin and ETH for many years and I've been thinking about this a LOT and I just don't see it.
I wrote about how I approach blockchains from a capability point on view a few months ago on LI, see here: https://www.linkedin.com/pulse/blockchain-revolution-decentr...
I'd be happy to discuss further in private (chat/skype), but my "professional muscle memory" tells me that the only sane way of looking at a piece of technology is through its capabilities. When CIOs look for capabilities that can be met by blockchain then finding a business case becomes straightforward, otherwise it will be shoehorning as you say.
I'd be happy to discuss further in private (chat/skype), but my "professional muscle memory" tells me that the only sane way of looking at a piece of technology is through its capabilities. When CIOs look for capabilities that can be met by blockchain then finding a business case becomes straightforward, otherwise it will be shoehorning as you say.
[deleted]
[deleted]
[deleted]
Something that hasn't been done much yet, but probably will as the space matures, is for a blockchain to reuse another blockchain as its "timestamp" generator.
eg. A 'MyCoin' miner grabs the latest 10 blocks off the main, say, Ethereum blockchain, and includes those hashes as proof they began their block mining _after_ those main blocks were mined, then adds their own proof-of-work on top, winning the block-solve lottery to add the next block to the blockchain.
This could potentially reduce the amount of relative advantage a dominant miner has ( others can provide a math analysis )
This approach can also save electricity, as it puts the miners on a mutual 'duty-cycle' where MyCoin miners are only busy running cryptohashes for a fraction of the time between blocks.
Another nice side-effect of this 2-tier mining is that the block generation times are spaced more evenly over time. [ They move towards a normal distribution around the average time, rather than uniformly - a result of the 'law of large number's, which states roughly that the average of uniform 'flat' distributions approaches a normal bell curve around the mean ].
Currently Bitcoin blocks sometimes arrive after 1 minute, other times only after 19 minutes, with mean 10mins maintained by adjusting difficulty. So the bitcoin time between blocks variance is high ( Litecoin adds blocks 4x as often @ 2.5mins, Ethereum ~10secs on average, but all with quite high variance ) which can surprise users.
eg. A 'MyCoin' miner grabs the latest 10 blocks off the main, say, Ethereum blockchain, and includes those hashes as proof they began their block mining _after_ those main blocks were mined, then adds their own proof-of-work on top, winning the block-solve lottery to add the next block to the blockchain.
This could potentially reduce the amount of relative advantage a dominant miner has ( others can provide a math analysis )
This approach can also save electricity, as it puts the miners on a mutual 'duty-cycle' where MyCoin miners are only busy running cryptohashes for a fraction of the time between blocks.
Another nice side-effect of this 2-tier mining is that the block generation times are spaced more evenly over time. [ They move towards a normal distribution around the average time, rather than uniformly - a result of the 'law of large number's, which states roughly that the average of uniform 'flat' distributions approaches a normal bell curve around the mean ].
Currently Bitcoin blocks sometimes arrive after 1 minute, other times only after 19 minutes, with mean 10mins maintained by adjusting difficulty. So the bitcoin time between blocks variance is high ( Litecoin adds blocks 4x as often @ 2.5mins, Ethereum ~10secs on average, but all with quite high variance ) which can surprise users.
Proof of existence is a major problem at the moment. Everyone around me (at an EU bank) wants to do this on bitcoin or ethereum main net, but right now banks are not allowed to touch these networks from the regulators (from a "miners are service providers, they need to be KYCed" perspective).
> In a large network like Bitcoin or Ethereum the massive computational power of the network ensures generating 51% of the compute power of the network is near impossibly expensive.
This is an aside, but of the 26 currently active bitcoin mining pools, you'd only need to compromise the top 4 to have >50% mining power. Perhaps a little social engineering could bring that down to 3.
I wonder how difficult that would actually be.
This is an aside, but of the 26 currently active bitcoin mining pools, you'd only need to compromise the top 4 to have >50% mining power. Perhaps a little social engineering could bring that down to 3.
I wonder how difficult that would actually be.
Yes, but those miners have invested so much capital into their mining hardware, software, electricity, etc that they have a strong incentive to make sure that doesn't happen. If an attacker were to gain control of 51% of the network Bitcoin would quickly become worthless or at least drop sharply in price making the whole attack pretty worthless. Not only is it very difficult to get to 51%, there are strong incentives for the relevant parties to make sure such a thing doesn't happen. If entire mining pools were hijacked with malicious mining software, someone would catch on and likely pull the plug on those machines before massive damage could be done.
[deleted]
[deleted]
This is true of proof of work but this isn't necessarily true of all blockchain implementations. Under certain conditions BFT can be maintained without systems such as proof of work.
Proof of work is not the right tool for private networks since it is not sybil resistant in this context. https://github.com/paritytech/parity/wiki/Demo-PoA-tutorial is another tutorial that uses a Proof of Authority consensus for an Ethereum-like chain.
PoW is not often used in consortium networks because there is no need for the excess power usage when the blockchain network is not in the public domain and participation is limited by some form of authority or centralised management structure.
I'm not sure what you mean with sybil resistance in this context though. Do you mean to say that PoA reduces the "51% attack" vector?
I'm not sure what you mean with sybil resistance in this context though. Do you mean to say that PoA reduces the "51% attack" vector?
The way people try to use PoW for consortium networks does not cause excess power usage: low difficulty, low powered device/s.
Anyone malicious that connects to the network can easily overpower the existing validators. Sybil resistance is only the case if validators compete with computation due to economic incentives. In consortium network the native tokens have likely no/low value.
With PoA there are rules according to which validators are added/removed and limits on block issuance that ensure fault tolerance.
Anyone malicious that connects to the network can easily overpower the existing validators. Sybil resistance is only the case if validators compete with computation due to economic incentives. In consortium network the native tokens have likely no/low value.
With PoA there are rules according to which validators are added/removed and limits on block issuance that ensure fault tolerance.
I'm not sure I agree with you re: overpowering the existing validators.
On a public blockchain, you need a proof that all nodes are acting in the best interest of the network because there is no way of stopping anyone from joining. The economic incentives play a huge part here as you mention.
On a private chain, you'll have some form of registration or centralised auth which prevents "unknown" nodes from joining in the first place. If mining is centralised, that means that trust is established "off-chain" so an external sybil attack would not be a real threat.
Regarding the power usage and difficulty increase over time, I guess you could keep the resource usage low for some time, but you'd need to re-write the client software to prevent and increase in proportion to the number of blocks created, which could have a negative effect on the network synchronisation, or rate of inflation (coin issuance) if there is a token involved.
On a public blockchain, you need a proof that all nodes are acting in the best interest of the network because there is no way of stopping anyone from joining. The economic incentives play a huge part here as you mention.
On a private chain, you'll have some form of registration or centralised auth which prevents "unknown" nodes from joining in the first place. If mining is centralised, that means that trust is established "off-chain" so an external sybil attack would not be a real threat.
Regarding the power usage and difficulty increase over time, I guess you could keep the resource usage low for some time, but you'd need to re-write the client software to prevent and increase in proportion to the number of blocks created, which could have a negative effect on the network synchronisation, or rate of inflation (coin issuance) if there is a token involved.
What is the point of blockchain if there is no fault tolerance, under your private PoW scenario as soon as a single node goes rogue the security guarantees disappear and the chain is messed up.
With this PoA you can keep a well functioning chain and kick misbehaving nodes (up to 50% of validators for this consensus engine) in due time.
With this PoA you can keep a well functioning chain and kick misbehaving nodes (up to 50% of validators for this consensus engine) in due time.
I'm not sure I can agree with you. To my mind with a private blockchain there is always off-chain trust or contract limiting access to the network. (edit: regardless of the consensus algorithm used).
In order to re-write historic transactions, an attacker would need to convince other participants to accept the new chain as valid and drop the old one, which is would be prevented by the off-chain trust model in the first place.
A powerful "rogue" node that starts producing blocks with different validation rules would be automatically ignored by other validator nodes who would not recognise the blocks as valid, so here again the risk of a successful sybil attack is limited.
Fault tolerance seems to be a red herring here because but I would rather ask you to expand on what you mean.
In order to re-write historic transactions, an attacker would need to convince other participants to accept the new chain as valid and drop the old one, which is would be prevented by the off-chain trust model in the first place.
A powerful "rogue" node that starts producing blocks with different validation rules would be automatically ignored by other validator nodes who would not recognise the blocks as valid, so here again the risk of a successful sybil attack is limited.
Fault tolerance seems to be a red herring here because but I would rather ask you to expand on what you mean.
First you need to understand what is a fault in this case. If you make use of PoW the chain is picked on the basis of total difficulty, this means that if any validator (already admitted by a centralised party) is hacked, then they can very quickly produce a chain which gets accepted by all others as canonical (with higher difficulty). This reverses any existing transactions and requires significant extra-protocol action to mitigate.
With PoA there is no such ability, since the canonicality is not abusable. Furthermore addition and removal of nodes can be decentralised and determined according to any rules (since validators can be specified in a contract). This makes it possible to include validators based on something like majority voting.
With PoA there is no such ability, since the canonicality is not abusable. Furthermore addition and removal of nodes can be decentralised and determined according to any rules (since validators can be specified in a contract). This makes it possible to include validators based on something like majority voting.
I understand that if one node is compromised, it can then be used to generate malicious txs and the attacker can try to add them to a block it generates, but why would you say the other validators accept such block(s) or transactions in the first place?
It seems to me that a successful attack on a private blockchain requires the attacker gaining access to both a "pre-approved" node and having the hashrate majority needed to ensure its blocks are generated faster than the rest of the network combined.
Some background: https://coinjournal.net/vitalik-buterin-on-misconceptions-in... and https://www.reddit.com/r/ethereum/comments/48m9n6/vitalik_bu...
It seems to me that a successful attack on a private blockchain requires the attacker gaining access to both a "pre-approved" node and having the hashrate majority needed to ensure its blocks are generated faster than the rest of the network combined.
Some background: https://coinjournal.net/vitalik-buterin-on-misconceptions-in... and https://www.reddit.com/r/ethereum/comments/48m9n6/vitalik_bu...
Especially given all the warnings in the post, I'm wondering why one would want to do this? I imagine the ostensible answer is security, but given the many recent posts about Solidity's poor design and the many recent heists due to (non-obvious) smart contract bugs, why would anyone want to run a private Ethereum network?
Is there a better alternative?
Is there a better alternative?
One common use is to test out your new contract code without using real ethereum. You could even imagine a time where you could run a fuzzer on your contract to try to find bugs.
Personally I don't think there's a real risk of exposing your real Ethereum wallet by setting up a testnet, as long as you only let localhost connect to it.
Personally I don't think there's a real risk of exposing your real Ethereum wallet by setting up a testnet, as long as you only let localhost connect to it.
Corda looks pretty good for private ledgers since that's what it's designed for. I don't know if Java smart contracts are any better or worse than Solidity/EVM, though.
Pretty good, I'm really happy to see the added focus on Ethereum here :-)
Since you are building a private network, you may also add a small script so your nodes only mine on demand (i.e., when there are new transactions) which will reduce the power usage on your machine and avoids having the difficulty increase too quickly.
Save the script below as mine.js (for instance) and then in your geth startup command, include it as `geth --networkid <your_networ_id> ...other params... js ./mine.js`
`var mining_threads = 1
function checkWork() { if (eth.pendingTransactions.length > 0) { if (eth.mining) return; console.log("== Pending transactions! Mining..."); miner.start(mining_threads); } else { miner.stop(); console.log("== No transactions! Mining stopped."); } }
eth.filter("latest", function(err, block) { checkWork(); }); eth.filter("pending", function(err, block) { checkWork(); });
checkWork();`
I picked up this "trick" from Embark framework (https://github.com/iurimatias/embark-framework)
Since you are building a private network, you may also add a small script so your nodes only mine on demand (i.e., when there are new transactions) which will reduce the power usage on your machine and avoids having the difficulty increase too quickly.
Save the script below as mine.js (for instance) and then in your geth startup command, include it as `geth --networkid <your_networ_id> ...other params... js ./mine.js`
`var mining_threads = 1
function checkWork() { if (eth.pendingTransactions.length > 0) { if (eth.mining) return; console.log("== Pending transactions! Mining..."); miner.start(mining_threads); } else { miner.stop(); console.log("== No transactions! Mining stopped."); } }
eth.filter("latest", function(err, block) { checkWork(); }); eth.filter("pending", function(err, block) { checkWork(); });
checkWork();`
I picked up this "trick" from Embark framework (https://github.com/iurimatias/embark-framework)
This is appealing as a way for testing contracts without paying the rediculously high gas price. However is there any advantage of this over the test Etherium network?
[deleted]
if you're testing there are several free test eth nets that can be used either locally or disributed
Hello osmode, if you'd like you could say hello there: https://ethereumdev.io/contact/
Loved your articles and looking to write blockchain creation and more after finishing Solidity tutorials..
[deleted]