The Big Bang of Blockchain Consensus Algorithms: The Case of Ripple | by Klitos Christodoulou | Ripple Series | Jul, 2020 | Medium

By Klitos Christodoulou, Antonios Inglezakis, Marios Touloupou, Iosif Elias

1.1 Benchmarking the behaviour of Blockchain Consensus Algorithms

The blockchain trilemma (scalability, security, and decentralization) triggered a competition between initiatives and project teams from the wider blockchain ecosystem. These initiatives attempt to propose alternative to Bitcoin implementations for Distributed Ledgers. The big bang of implementations, as attempts to solve the trilemma challenges, led to a wide spectrum of different, heterogeneous implementations of such protocols. However, little attention has been paid in understanding their performances, especially the boundaries of their fault tolerance consensus mechanisms.

The ultimate goal of this article series is to technically understand and identify the elements that can be used to enable a generic benchmark for evaluating different network topologies and understanding the behaviour of various consensus proposals for Blockchains. As part of our exploration studies we have been using XRP Ledger’s core server software rippled.

1.2 Launching a Private Ripple Topology (or Testnet)

1.2.1 Introduction

A key phase of the software development life-cycle is to introduce a rapid testing environment at the early stages of the project. Testing a software is critical to ensure that the production software does not result into any unexpected behaviours. To this end, a testing environment is crucial for facilitating the production process of a software implementation. Developers use test-bed environments to provide the necessary tools that aim to facilitate the evolution of the source code, evaluate its intended behaviour, and observe performance at various layers.

Distributed Ledgers (e.g., Blockchains) are complex systems that build on a variety of technologies ranging from cryptography to distributed systems. Testing and deployment in such systems is most of the times cumbersome and time consuming. The process of setting up a network topology, generate key pairs and even interfacing with such systems requires a significant investment in time with a clear understanding of their internal functions (from a developer’s point of view).

Having an isolated testing environment where developers can introduce changes to the source code or interact with the protocol without affecting the function of a protocol’s main network (Mainnet) is essential. Blockchain developers, especially at the low-level, are required to set up a blockchain network topology and use it to detect any potential network anomalies, misbehaviour of nodes etc. The latter introduces several challenges while a blockchain network consists of several nodes that run the same version of source code in different, possibly distributed machines, often with a variant degree of decentralization. Thus, for developers, the challenge remains on selecting suitable tools and technologies for building private test environments and for deploying existing blockchain networks (e.g., spin-up multiple instances, configure multiple synthetic testing scenarios etc.)

1.2.2 Aim and Motivation

In this first article, we envision the automated deployment of a testing environment where developers, with minimum effort, will be able to generate a set of different synthetic test scenarios for RippleNet. The research team at the Institute for the Future (IFF) aims to complement existing work done by the XRP Ledger Project. More specifically, to provide additional tools (in the form of executable scripts, docker containers or orchestrating docker deployments) and technical documentation for providing tools that enable developers to create network topologies of n number of nodes listening to each other.

The ultimate goals of our research efforts are to provide an end-to-end testing and benchmarking framework for Blockchain/protocol implementations. In this first article we use rippled as a use-case towards the aforementioned challenges.

1.2.3 High Level Architecture

Fig 1. — High Level Architecture

Fig 1. — High Level Architecture

Fig. 1 illustrates an abstract view of the process we are automating for this first experiment using Ripple’s source code (i.e., rippled). Developers/Testers that work in a local environment, frequently push source code changes to a remote repository. At the same time, some developers build the Ripple network and pull source code from some remote location. In this article, we will show how we have configured a Docker daemon to expose its API to a TCP connection on port 2375 (default port), how to build the updated version of the rippled source code and run the docker containers. At the end of this process, n number of rippled nodes are spawned, and a Ripple Private TestNet is deployed for further testing.

1.2.4 Basic Knowledge

The XRP Ledger is a decentralized ledger protocol that is built on top of a peer-to-peer infrastructure. To coordinate the state of the shared ledger the network utilizes a BFT-like consensus algorithm. The XRP Ledger hosts its own digital asset, known as XRP, which has been designed to act as a bridge between many different currencies. The development of the XRP Ledger has been initiated and maintained by Ripple. XRP Ledger comprises Ripple’s private blockchain network, known as RippleNet. There is also the public XRP Ledger network that is open to the public to use and join. The peer-to-peer core software used for both networks is the same, and it is referred to as rippled, or the ripple daemon (https://xrpl.org/the-rippled-server.html). The same piece of software is used to set up both a local wallet server and a validator node.

Wallet nodes are used to manage accounts and issue transactions, whereas validators, are nodes that participate to the consensus of the network; at the same time storing a local copy of the complete shared global ledger. Each validator trusts only those validators that are registered in its own Unique Node List (UNL). The UNL, for each validator node, is passed as a file that includes the public keys of the other validators that are trusted. This file can be retrieved by a Web server signed with some trusted key (which is defined in the configuration file) or explicitly retrieved from a local file.

In addition to the configuration of validators and forming a UNL, running our own private network requires that we should control the connectivity of the nodes (for our private testnet). Thus, in this article we mostly focus on running an n number network of validators. For a comprehensive guide on how to run a validator peer the reader is referred to (https://xrpl.org/run-rippled-as-a-validator.html), and for setting up a wallet server to (https://xrpl.org/run-rippled-as-a-wallet-server.html).

The ripple daemon, i.e., rippled, can be built on your local machines, as it is open-source and it is available in Ripple’s Github repository (https://github.com/ripple/rippled). In subsequent sections we describe in detail on how to build your own rippled network; consisting of several rippled validator nodes. Also, along with setting up a set of rippled validators, a genesis ledger is spawned holding the amount of 100 Billion XRPs. A more comprehensive guide on how to build your own rippled server can be found in the official documentation site provide by xrpl.org (https://xrpl.org/build-run-rippled-ubuntu.html, https://xrpl.org/start-a-new-genesis-ledger-in-stand-alone-mode.html).

1.3 Spin-up a Private Ripple TestNet

1.3.1 Overview

In this section, we present an overview of the proposed process for building and running a private Ripple Testnet. More specifically, our walkthrough begins by building a rippled daemon locally, and then how to create a docker image for running it. Then, we show how to launch a private docker network using the scripts we implemented. Following from this, we then show how to create and setup a private Ripple Testnet. In more detail, how to: generate key pairs for the validators, configure the validators and deploy the Testnet. Finally, we demonstrate some basic commands for interacting and testing our network.

The deployment steps are summarised as follows:

  1. Environment Requirements
  2. Compiling the Ripple Daemon and Build rippled Docker image
  3. Launch a private Ripple TestNet
  4. Generate key-pairs for validators
  5. Generate configuration files for validators
  6. Generate Ripple Testnet docker-compose file
  7. Interacting with the Testnet

1.3.2 Environment Requirements

The deployment of our Ripple Testnet is performed on a Debian 10 “Buster” Linux machine with 12GB RAM and an SSD, using Docker containers and docker-compose utilities. The main script, which is responsible to build the network, is using “jq” utility (https://stedolan.github.io/jq/) for filtering the JSON files, while “sed” is used to replace variable and values in the template files. Since Docker Engine offers a flexible and portable architecture, we do expect this walkthrough to be applicable on every Linux machine which has the following software installed:

Regarding “jq”, this should be installed already, if not, installing it through the package manager of your distribution is also the easiest and quickest way. For all Debian-based distributions, ‘sudo apt install jq’ should install the package.

The “sed” editor is included in most Linux distributions, so installing it through the package manager of your distribution is the quickest and easiest way to do it. In Debian-based distributions, ‘sudo apt install sed’ should install the package.

We have also tested our scripts using a machine with Ubuntu 18.04 LTS without any issues. Since we make extensive use of bind-volumes, with in our docker-compose files, we expect issues in other Operating Systems, especially the ones that use the Docker Toolbox.

1.3.3 Compiling Ripple Daemon and Build Rippled Docker image

Presenting a detailed guide on how to build the rippled daemon from source code is out the scope of this article thus we briefly discuss this step. For a comprehensive guide on building the rippled daemon from source please refer to (https://xrpl.org/build-run-rippled-ubuntu.html).

For this article, we have used an existing docker image to build the rippled daemon with all the required dependencies installed. Since at this stage we are not changing anything from the original rippled source code, we can skip building the ripple daemon from scratch. We have already prepared a docker image that runs the rippled daemon which can be found in our Docker hub repository (https://hub.docker.com/u/uniciff).

1.3.4 Docker Image

To retrieve the existing Docker image with the rippled daemon, we run the following command.

$ docker pull uniciff/rippled-runner:v1.5 Quick Launch of a Private Ripple Testnet

1.3.5 Launch a Private Ripple Testnet

As already mentioned, we have implemented utility scripts to automate the process of: generating key-pairs for validators, the validators configuration files, and a docker-compose file for launching our private Ripple Testnet. Our utility scripts are available in the following GitHub repository (https://github.com/UNIC-IFF/ripple-docker-testnet).

Find below a brief walkthrough on how to launch a new Testnet using our scripts. In addition, we provide a script that allows us to stop the network and clean the environment i.e.,

$ ./stop _testnet.sh

First, we clone our repository, using the following instructions:

$ git clone https://github.com/UNIC-IFF/ripple-docker-testnet/

$ cd ripple-docker-testnet

Set executable flags to the scripts.

$ chmod +x run_testnet.sh

$ chmod +x stop_testnet.sh

Execute the following to deploy your own private Testnet with n number of validators including a Genesis Ledger.

$ ./run_testnet.sh <N>

This script automates the process, including the generation of the keys, the configuration files for the validators, and the docker-compose file for the Testnet. Finally, it deploys the Testnet creating a full mesh network connecting each validator to each other. For this instantiation, we asked the script ‘$ ./run_testnet.sh 10’ to deploy a Genesis ledger node, plus 9 validator nodes. At the end of the execution you should be able to see message “Done!!!” on the console, illustrating the end of this process.

To verify that our network is up and running we are executing the following command:

$ docker ps -a (List containers — For documentation please refer here)

To stop the Testnet and clean the environment, we execute ‘./stop_testnet.sh’.

$ ./stop_testnet.sh

Also, it is worth mentioning that since the docker compose file is automatically generated, it is available for any future deployment of our network, avoiding the execution of the ‘run_testnet.sh’ script. Thus, to run our Testnet again, using the same configuration, we can simply use docker-compose:

$ docker-compose –f docker-compose-testnet.yml up –d

1.3.6 Generating key-pairs for Validators

To make rippled software to run as a validator in the network, a validator token should be passed in its configuration file. Validator peer use this token to sign their proposals during the proposal phase, which is part of the consensus algorithm. The validator token matches the public key of the validator, which can then be used by other validator nodes in the network to identify it and ensuring that the validator is listed in their UNL. Only that way the proposals made by that validator are valid. There is a command line tool, validator-keys, that can handle all the operations related to the keys for validators. We are going to use that tool to generate key-pairs, the public key and validator token, for each validator in our Testnet. The process begins with the generation of the keys. Then, we use the keys to generate the validator token.

The command below generates a JSON file with the keys for one validator node.

$ validator-keys create_keys –-keyfile validator-keys.json

Then, we use the generated validator-keys.json file to extract the validator token; to be used later in the validator configuration file.

$ validator-keys create_token –-keyfile validator-keys.json > validator-token.txt

The ‘validator-token.txt’ file contains the validator token to be used in the configuration file under the specific section indicated by [validator_token]

Following from this step, and to setup our Testnet we repeat the above process multiple times, as the number of the validators to be used in our Testnet. To automate this process, we wrote additional scripts for generating the validator configuration files all at once. The above process is automatically triggered and executed in the ‘run_testnet.sh’ script. Thus, we do not have to worry about it!

1.3.7 Generate Configuration files for Validators

As already mentioned, the rippled software instantiates different roles in the network. Its behaviour in the network is determined/configured by the configuration files. In more detail, there are two configuration files for the rippled software. The main configuration file, that is used to config rippled, and a local file, that explicitly lists all the trusted validators of the node. This local file defined the UNL for the validator.

At first, we have used the configuration example included in the rippled source code repository. We then decided to make a few changes so that peers are able to connect to each other. The major differences are in the configuration section [peer_private], as we set the peer_private flag to 1 (enabled). For [ips_fixed] we list the <IP,port> pairs for each other validator included in our private Testnet. If the IP–Port pair of a validator that reaches our validator is not in the [ips_fixed] section, the connection cannot be established. Additionally, the port in which each peer listens is defined in [port_peer] section.

The local file defines the UNL configuration under the [validators_file] and the validator token is defined under the [validator_token] section. The [validator_token] section includes the output generated by calling the validator-keys tool and stored in the validator-token.txt. Just comply it at the end of the file.

It is required that each validator should have its own configuration file and UNL. The configuration files of the validators are similar, while they only differ at the [validator_token] section, that includes the unique validator token for each validator; that runs in the network.

As part of our exploration we have chosen to build a full mesh trusted network, so the UNL of each validator is the same; meaning, it includes all the public keys of the validators in the network.

The configuration file of each node can be viewed during runtime using the following command:

$ docker exec -it <container_name> sh -c ‘cat config/rippled.cfg

1.3.8 Generate the Ripple Testnet Docker-Compose file

Starting a new Ripple Private network requires that the first node is holding the genesis block. The Genesis block contains information regarding the first node and the initial account that holds the 100 B XRPs, and be used to top-up, and create other user accounts in the network.

To launch the first rippled node that generates the Genesis block, we pass the command line flag ‘ — start’ (https://xrpl.org/start-a-new-genesis-ledger-in-stand-alone-mode.html) while starting the node.

In order to ease ourselves while stopping and relaunching the Testnet, we are using docker-compose. Docker-compose allow us to define the configuration of the rippled nodes in one file and launch or stop them with just a single command. Our main purpose is to build an automated process that enables us to launch, with the same effort, a Testnet configured with just 2 nodes or as many as nodes our system can handle using the same docker engine. To meet that aim, we build scripts to automate the docker-compose file generation. Again, this process is handled by the ‘run_testnet.sh’ script.

To view the generated docker-compose file for our Testnet, execute:

$ cat templates/docker-compose -template.yml

1.3.9 Interacting with the Testnet

We are now in the position to interact and execute basic operations using our Testnet. Firstly, we need to test whether our Testnet is running, and whether all validator nodes are connected. A quick way to verify the above is to call the ‘crawl’ endpoint of one of the validators. For easiness, we chose to perform this test on the genesis validator, although the same command should work for each validator.

$ docker run -it — rm — network $(docker network ls -q — filter=name=ripple-testnet) curlimages/curl — insecure https://validator-genesis:51235/crawl | jq

Alternatively, to list the set of peers form one validator while running execute ‘peers’.

$ docker exec –it validator-1 sh –c “./rippled peers’’

Our scripts generate output files with useful information for the Testnet’s nodes. These files are the configfiles/ips_fixed.lst and configfiles/validators-map.json.

The file configfiles/ips_fixed.lst contains a list of all the ips/domains and the ports of our Testnet validators.

$ cat ./configfiles/ips_fixed.lst

The file configfiles/validators-map.json contains a JSON object with a mapping between validators public keys and their ips/names. To print it,

$ cat ./configfiles/validators-map.json | jq

2. Conclusions and Next Steps…

This article describes a quick process for building and running your own Ripple private Testnet. Several scripts have been developed to automate the process, mainly using the Docker engine. As a next step, we are building scripts that enable us to deploy several clusters of private networks and connect them. Finally, we aim to build processes for evaluating different network topologies and understanding the behaviour of various Blockchain consensus proposals. Stay tuned!

The Institute for the Future @ UNIC

The Institute For the Future (IFF) is an interdisciplinary research Centre hosted at the University of Nicosia (Cyprus), that aims to advance emerging technologies and contribute to their effective application in industry, government and education, as well as, evaluate their impact on employment, wealth inequality and our lives and societies in general.

Contributors

  • Marios Touloupos (@mtouloup) — Researcher / PhD Candidate, University of Nicosia — Institute For the Future (UNIC-IFF)
  • Antonios Inglezakis (@antIggl) — Researcher / Senior Software Engineer and Systems Administrator, University of Nicosia — Institute For the Future (UNIC-IFF)
  • Klitos Christodoulou (@klitoschr) — Research Manager and Faculty Member, University of Nicosia — Institute For the Future (UNIC-IFF)
  • Elias Iosif (@iosife) — Faculty Member, University of Nicosia — Institute For the Future (UNIC-IFF)

Acknowledgements

This research is funded by the Ripple’s Impact Fund, an advised fund of Silicon Valley Community Foundation (Grant id: 2018–188546).

Source: The Big Bang of Blockchain Consensus Algorithms: The Case of Ripple | by Klitos Christodoulou | Ripple Series | Jul, 2020 | Medium