Geth Private Network POA (Clique) with Block Explorers¶

Have you ever wondered how to configure and deploy your own, enterprise ready, fully featured, private network?

In this tutoria we are doing exactly that!

At the end, you will have:

A Bootnode
Signers with Clique
Clients listening to HTTP and WebSockets
3 Different Block Explorers a) Blockscout b) AlethIO Lite-Explorer c) A simple Web3 Block Explorer
A cool network status monitor.

Then, at the end, we're dockerizing everything.

Geth and Environment¶

Environment¶

In this tutorial we're going to use Geth 1.10.17.

Throughout the tutorial I will assume you're either in a unix environment, or use WSL2. I am working in WSL2, in an ubuntu 20.04 container.

Download¶

For the first part, we're working directly with Geth in a terminal. Later we're dockerizing it. To get started, create an empty folder and download geth-all-tools:

mkdir geth-private-network
wget https://gethstore.blob.core.windows.net/builds/geth-alltools-linux-amd64-1.10.17-25c9b49f.tar.gz
tar xzvf geth-alltools-linux-amd64-1.10.17-25c9b49f.tar.gz
./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth version

This should lead to:

Let's get started with a bootnode!

Bootnode¶

When you run go-ethereum without a private network, then it will try to connect to a list of hardcoded bootnodes. In a private network that obviously doesn't make any sense and we need to run our own bootnode(s).

There are two ways to do that:

Run a normal geth node which is also a bootnode
Run a dedicated bootnode

We will run (2), a dedicated bootnode now.

Bootnode Keys and Authentication¶

A bootnode is reachable via this weirdly long url, for example enode://890b6b5367ef6072455fedbd7a24ebac239d442b18c5ab9d26f58a349dad35ee5783a0dd543e4f454fed22db9772efe28a3ed6f21e75674ef6203e47803da682@[::]:30301

Let me break it down for you: enode://[public-key]@[ip]:[port].

So, the first part is actually a public key. Where is then the private key you may ask?

That is something you provide to the bootnode.

Let's take the example private key from the geth docs:

dc90f8f7324f1cc7ba52c4077721c939f98a628ed17e51266d01c9cd0294033a

this will lead to the enode url: enode://890b6b5367ef6072455fedbd7a24ebac239d442b18c5ab9d26f58a349dad35ee5783a0dd543e4f454fed22db9772efe28a3ed6f21e75674ef6203e47803da682@[::]:30301

And if you keccak256 hash this public key web3.utils.sha3('0x890b6b5367ef6072455fedbd7a24ebac239d442b18c5ab9d26f58a349dad35ee5783a0dd543e4f454fed22db9772efe28a3ed6f21e75674ef6203e47803da682') you get: 0xdc253bda32d3cda57ba19534a79ac97a66ee081b19987b442a7469a3651d0d09. And if you take the last 20 bytes, you get the account: 0xa79AC97A66Ee081B19987b442a7469A3651D0D09

So, the bootnode url authentication is the public key. We can use that to our advantage, we just need a normal private key.

Private Key

Reminder: Ethereum Private Keys are 64 random hex characters or 32 random bytes.

We can just create our own private key or use an existing one. There is no magic behind it.

Let's use 1010101010101010101010101010101010101010101010101010101010101010 as a private key for our development environment. Of course, you could (and should!) generate a cryptographically strong key with bootnode -genkey bootnode.key, which will insert a private key into the bootnode.key file.

Booting the Bootnode¶

Now that we have a private key, let's boot the bootnode. This is quite easy, for example

./geth-alltools-linux-amd64-1.10.17-25c9b49f/bootnode --nodekeyhex 1010101010101010101010101010101010101010101010101010101010101010

This should output you

Leave that running in a terminal and carry on to the next part where we setup miner nodes!

Private Network with Clique POA¶

In this part we're setting up geth with the clique consensus algorithm and a block mining time of 1 second.

A private network with geth consists of two configurations:

A genesis.json file that configures the chain
A number of parameters configuring how geth behaves

Genesis.json file explained¶

The blockchain starts with a "block zero", that is the very first block. This block has no parent block and is generated manually. This is where the genesis.json files comes in, because that's the file that defines the blockchain and all its features.

Let's have a look at this genesis.json file:

{
    "config": {
      "chainId": 210,
      "homesteadBlock": 0,
      "eip150Block": 0,
      "eip155Block": 0,
      "eip158Block": 0,
      "byzantiumBlock": 0,
      "constantinopleBlock": 0,
      "petersburgBlock": 0,
      "istanbulBlock": 0,
      "berlinBlock": 0,
      "londonBlock": 0,
      "clique": {
        "period": 1,
        "epoch": 30000
      }
    },
    "nonce": "0x0",
    "timestamp": "0x5d6e468f",
    "extraData": "0x00000000000000000000000000000000000000000000000000000000000000000001fcd01073fe6017920a97cc384bee72c98beb0002f7067c48ef957b21009685ab69ee768e38bd0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "gasLimit": "0x59A5380",
    "difficulty": "0x1",
    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "coinbase": "0x0000000000000000000000000000000000000000",
    "alloc": {
      "0x5AD2d0Ebe451B9bC2550e600f2D2Acd31113053E": {
          "balance": "0x2000000000000000000000000000000000"
      }
    },
    "number": "0x0",
    "gasUsed": "0x0",
    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000"
  }

Let's go through these:

chainId: To prevent cross chain double spending, each transaction is signed with the chain-id. This defines the chain id of your private chain. The chainid is defined by yourself, there is no official registry of any sort, but there are at least some lists keeping track of chainids. Ideally, when running your own chain, you want to use a chainid that is unique.
homesteadblock: Homestead is the second major version release of the Ethereum platform, which includes several protocol changes and a networking change that gives us the ability to do further network upgrades:
- EIP-2 Main homestead hardfork changes
- EIP-7 Hardfork EVM update: DELEGATECALL
- EIP-8 devp2p forward compatibility
eip150Block: EIP150 fixed a few DDoS issues by increasing or changing the gas amount a few OPcodes cost.
eip155Block: EIP155 introduces the chainId being included in the transaction object, so that a reply attack becomes impossible across chains.
158Block: EIP158 removes a large number of empty accounts that have been put in the state at very low cost due to flaws in earlier versions of the Ethereum protocol, thereby greatly reducing state size and hence both reducing the hard disk load of a full client and reducing the time for a fast sync.
byzantinumBlock: According to ethereum.org, the Byzantium fork was made active on Ethereum Mainnet on Oct-16-2017 and
- Reduced block mining rewards from 5 to 3 ETH.
- Delayed the difficulty bomb by a year.
- Added ability to make non-state-changing calls to other contracts.
- Added certain cryptography methods to allow for layer 2 scaling.
constantinopleBlock: According to ethereum.org, the Constantinople fork was made active on Ethereum Mainnet on Feb-28-2019 and
- Ensured the blockchain didn't freeze before proof-of-stake was implemented.
- Optimised the gas cost of certain actions in the EVM.
- Added the ability to interact with addresses that haven't been created yet.
petersburgBlock: St. Petersburg came together with Constantinople, but rolled back EIP-1283. So, if petersburgBlock is turned on, then clients do not have EIP1283 integrated. On Mainnet constantinople and peterburg were activated in the same block.
istanbulBlock: According to ethereum.org, the Istanbul fork was made active on Ethereum Mainnet on Dec-08-2019 and
- Optimised the gas cost of certain actions in the EVM.
- Improved denial-of-service attack resilience.
- Made Layer 2 scaling solutions based on SNARKs and STARKs more performant.
- Enabled Ethereum and Zcash to interoperate.
- Allowed contracts to introduce more creative functions.
berlinBlock: According to ethereum.org, the Berlin fork was made active on Ethereum Mainnet on Apr-15-2021 and changed gas costs for a few Opcodes, most notably EIP-2929, which increased gas costs for state access opcodes
londonBlock: According to ethereum.org, the London fork waas made active on Ethereum Mainnet on Aug-05-2021 and introduced EIP-1559. This EIP addressed gas-price issues and basically did the following:
- Introduced a "Base Fee" which is calculated based on the previous blocks gas consumption and burned
- Introduced a "Priority Fee" which is given to miners as incentive to add a transaction to the block.
clique: This configures the clique consensus algorithm and defines:
- period: what's the block mining time? In this case: 1 second.
- epoch: The number of blocks after which to reset all votes.

Hyperledger Besu

This is the configuration for go-ethereum. In Hyperledger-Besu these fields are named differently. See here: https://besu.hyperledger.org/en/stable/HowTo/Configure/Consensus-Protocols/Clique/

nonce and mixhash: The nonce and mixhash are used in PoW neteworks. The mishash is a 256-bit (or 32-bytes) hash that is combined with the nonce. It proves that a block has really been mined and is valid. The whole cryptographic proof is described in the Yellowpaper, section 4.3.4. Block Header Validity. In clique, to my best knowledge, its just used to write the first block and still needs to be valid, otherwise geth will just error out on start. Best is to set these two to 0.
timestamp: A scalar value equal to the reasonable output of Unix’s time() at this block’s inception.
parentHash: The Keccak 256-bit hash of the parent block’s header, in its entirety.
number: : A scalar value equal to the number of ancestor blocks. The genesis block has a number of zero.
gasLimit: A scalar value equal to the current limit of gas expenditure per block. You will later see that this is set also through the signer nodes to let them converge to a specific gas limit.
difficulty: In PoW networks this defines "how hard" it is to mine a block. It can also be calculated from the previous block's difficulty level and the timestamp.
coinbase: Contains the address that got the block reward for mining this block. There is no mining reward in clique and you can set this to an arbitrary address (or leave it completely away).
extraData: This is normally optional, but mandatory with clique! It configures the network and enables signers. It contains three parts, lets break it down:
- 32 bytes vanity data. That means you can put into the first 32 bytes whatever you want. In our case its all zeros
  - 0x0000000000000000000000000000000000000000000000000000000000000000
- The concatenated addresses from the sealers of the clique network. It's written as hex-string without the "0x" at the beginning. It must be a multiple of 20 bytes
  - 0001fcd01073fe6017920a97cc384bee72c98beb
  - 0002f7067c48ef957b21009685ab69ee768e38bd
- The last part is a 65 bytes proposer seal, which is 65 bytes long (or 130 characters), in hex notation without the 0x. It's all zeroes in the genesis block because there are no proposers yet.
  - 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
- You can read more about it in EIP-650
- All combined is 0x00000000000000000000000000000000000000000000000000000000000000000001fcd01073fe6017920a97cc384bee72c98beb0002f7067c48ef957b21009685ab69ee768e38bd0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
alloc: This is a list of addresses and an initial balance they get. Since clique does not do any block reward, you should give one address you control enough eth to run transactions. In this case, we're giving 0x5AD2d0Ebe451B9bC2550e600f2D2Acd31113053E the initial balance of 0x2000000000000000000000000000000000, which is 10889035741470030830827987437816582766592 Wei or 10889035741470030830827.98 ETH.

Initialize a new Chain¶

The first step is to initialize a new chain wiht the genesis.json file. Make sure the genesis.json file is in the directory that we created earlier. I'll create a new folder for a signer1 and signer2 in our sample directory:

mkdir signer1data
mkdir signer2data

Before we can do that, we need to know the signers eth addresses. And before we can know the signers eth addresses, we need to generate private keys for them!

Signer Eth Vanity Addresses¶

I like to use vanityeth for that, which can be installed using the Node Package Manager (npm):

npm install -g vanity-eth

Web Service or GPU accelerated

If you do not want to install a separate NPM package or want to go faster, then I can recommend two services: 1. https://vanity-eth.tk is a Vanity-Eth-Address Generator as a service 2. If you need to go faster, then maybe try https://github.com/cenut/vanity-eth-gpu.

If you install vanity-eth, it will look like this:

Let's generate a new Ethereum Address:

vanityeth -i 0000

This will output you a new address/private key pair:

✔ {"account":{"address":"0x0000de292fedd1d5dac1726aa47098d170a2c868","privKey":"55ecb626568fcb7efdf2bcecaf7263fb3bbad5c5faa0d9ffff9e863b92478f92"}}

We are going to use this one for the bootnode, let's create two more vanity addresses for the signers.

vanityeth -i 0001

outputs this:

✔ {"account":{"address":"0x0001fcd01073fe6017920a97cc384bee72c98beb","privKey":"54c0ac98237161c4c795be8202a89e5aafba63db2d2dfb11203e2226ab021f1a"}}

And now we're putting this into a file called "pk_signer1", as wel as an account-password. Where I'm going with this is something you will see in a second:

echo "54c0ac98237161c4c795be8202a89e5aafba63db2d2dfb11203e2226ab021f1a" > pk_signer1
echo "pw1" > pw_signer1

vanityeth -i 0002

Will output:

✔ {"account":{"address":"0x0002f7067c48ef957b21009685ab69ee768e38bd","privKey":"900e629254a4717dde32d79be7f7fe1b46bbee2bf97aeeff8b1c27ed46581c98"}}

then add the files:

echo "900e629254a4717dde32d79be7f7fe1b46bbee2bf97aeeff8b1c27ed46581c98" > pk_signer2
echo "pw2" > pw_signer2

So, signer 1 and signer 2 have those eth addresses:

0x0001fcd01073fe6017920a97cc384bee72c98beb
0x0002f7067c48ef957b21009685ab69ee768e38bd

Update genesis.json file¶

Let's add those addresses to the genesis.json extraData, so it accepts our signers later.

If you have a look at the extraData field, then you will see that the addresses already match:

Prefix:     0x
32-bytes 0: 0000000...
eth-addr 1: 0001fcd01073fe6017920a97cc384bee72c98beb
eth-addr 2: 0002f7067c48ef957b21009685ab69ee768e38bd
65-bytes 0: 00000....

extraData: 0x00000000000000000000000000000000000000000000000000000000000000000001fcd01073fe6017920a97cc384bee72c98beb0002f7067c48ef957b21009685ab69ee768e38bd0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Then I'll initialize the data directories:

./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth init ./genesis.json --datadir=./signer1data
./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth init ./genesis.json --datadir=./signer2data

At the end, it should look like this:

Start Bootnode and Signers¶

Let's open 3 terminals:

Terminal 1 contains the bootnode
Terminal 2 will run signer #1
Terminal 3 will run signer #2

But before we can do that, we need to generate private/public key pairs for our miners!

If we take the privKey part and feed it into the bootnode, we can do it in two ways: "-nodekeyhex ..." which is not recommended, and "-nodekey" which expects a file.

Later we're going to dockerize all of this, and we are going to pipe the key as a file descriptor into the bootnode by doing <(echo PRIVATE_KEY), this looks like this:

./geth-alltools-linux-amd64-1.10.17-25c9b49f/bootnode -nodekey <(echo "55ecb626568fcb7efdf2bcecaf7263fb3bbad5c5faa0d9ffff9e863b92478f92")

And it will output:

enode://45e6a923bc6f76b68d4a2d2cd1031161371ea56293725906435557c816f2bc4320e815060ade47983dda483c8d5687f3aeee3630470ea911e9ba41a1c90166c4@127.0.0.1:0?discport=30301
Note: you're using cmd/bootnode, a developer tool.
We recommend using a regular node as bootstrap node for production deployments.
INFO [05-19|20:23:26.365] New local node record                    seq=1,652,984,606,364 id=220ba0658fc2abfc ip=<nil> udp=0 tcp=0

Perfect, a bootnode is running and seemignly accepting connections. If you add a verbose logging, you can later see how the nodes actually connect.

Running Signer Clique Nodes¶

First we need to import the private keys and and create accounts. We use the account module from geth:

./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth account import --datadir=./signer1data --password pw_signer1 ./pk_signer1
./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth account import --datadir=./signer2data --password pw_signer2 ./pk_signer2

Then we start geth signer 1 in one terminal:

./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth --bootnodes enode://45e6a923bc6f76b68d4a2d2cd1031161371ea56293725906435557c816f2bc4320e815060ade47983dda483c8d5687f3aeee3630470ea911e9ba41a1c90166c4@127.0.0.1:30301 --networkid=210 --datadir=./signer1data --miner.etherbase=0x0001fcd01073fe6017920a97cc384bee72c98beb --mine --unlock 0x0001fcd01073fe6017920a97cc384bee72c98beb --password ./pw_signer1 --port 30304

And singer 2 in another terminal:

./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth --bootnodes enode://45e6a923bc6f76b68d4a2d2cd1031161371ea56293725906435557c816f2bc4320e815060ade47983dda483c8d5687f3aeee3630470ea911e9ba41a1c90166c4@127.0.0.1:30301 --networkid=210 --datadir=./signer2data --miner.etherbase=0x0002f7067c48ef957b21009685ab69ee768e38bd --mine --unlock 0x0002f7067c48ef957b21009685ab69ee768e38bd --password ./pw_signer2 --port 30305

./geth-alltools-linux-amd64-1.10.17-25c9b49f/geth attach ./signer1data/geth.ipc