Blockchain technology offers decentralised control, where no one has any ownership or control
over the network; and a tamper-proof environment, where nothing gets deleted from network storage. It also has the ability to create and transfer assets on the network without reliance on a central entity. Blockchain storage, however, requires a powerful database, and this is where BigchainDB comes in.
As single monolithic blockchain technologies now have a wide range of applications, they have been re-framed and refactored into building blocks at four levels of the stack:
- Applications
- Decentralised computing platforms
- Decentralised storage (ledgers, file systems, databases), decentralised processing (smart contracts) and decentralised communication
- Cryptographic primitives, consensus protocols and other algorithms
Blockchain storage requires a powerful database, and the traditional blockchain as a database (DB) is pathetic, as the throughput is just a few transactions per second (tps), latency before a single confirmed write is 10 minutes, and the capacity is a few dozen gigabytes. Also, adding new nodes causes problems in terms of network traffic, congestion, latency, throughput and capacity. As compared to traditional databases, a modern distributed database provides throughput exceeding 1 million tps, 1000 TB storage, latency that is a fraction of a second, with no effect on throughput and latency when increasing the number of nodes. In addition, modern databases have good capabilities for insertion, querying, access control and security.
In this article, we introduce the concept of BigchainDB, which is used for database-style decentralised storage.
BigchainDB
BigchainDB allows developers and enterprises to deploy blockchain proof-of-concepts, platforms and applications with a scalable blockchain database. It supports a wide range of industries and use cases, from identity and intellectual property to supply chains, energy, IoT and financial ecosystems — all without sacrificing scale, security or performance. With high throughput, sub-second latency and powerful crypto-condition escrow functionality to automate release-of-assets, BigchainDB looks, acts and feels like a database but has core blockchain characteristics that enterprises want.
As compared to Ethereum, Hyperledger or other blockchain systems, BigchainDB doesn’t build a full stack of blockchain technologies, but rather, offers an overlay onto existing database technologies to ‘blockchain-ify’ them. BigchainDB is designed to merge the best of the database and blockchain worlds — scale and querying from the database side; and decentralisation, immutability and assets from the blockchain side. It starts with an initial open source database, i.e., MongoDB, and adds blockchain characteristics including decentralised control, immutability, and the creation and movement of digital assets. BigchainDB easily accommodates 1 million transactions per second, and explicitly excludes having a virtual machine or other mechanism for running smart contracts.
The following are the three main characteristics of BigchainDB.
- Decentralised control: No single entity controls the network. Every node has its own locally-stored list of the public keys of other consortium members — the so-called keyring. There’s no centrally-stored or centrally-shared keyring.
- Immutability: Data once written cannot be changed or altered. For example, the decimal digits of π are immutable (3.14159…)
- Transfer of digital assets: It has the ability to create an asset and transfer it without central control.
A BigchainDB instance consists of a number of nodes, all of which contain parts (but not all) of the complete database. Decentralised control is achieved by this DNS-like federation of nodes, which have voting rights in the validation of blocks. Voting operates on a layer above the actual database and in order to achieve speed, each block of transactions is written before being validated by a quorum of nodes. Nodes vote to validate a transaction and at validation time ‘chainify’ the block, as each block provides a hash ID of the previous block. Immutability is achieved through a combination of shard replication, disallowing reversions, database backups and the cryptographic signing of all transactions.
BigchainDB technology ensures an efficient decentralised database. It supports millions of writing operations per second, storing PB-level data and delivering a sub-second response time.
Related terminology
To understand the in-depth technical functionalities of BigchainDB, it is important to understand the related terminology.
- BigchainDB node: This is a machine or logical machine running the BigchainDB server and other software, and every node is controlled by one person or organisation. A set of BigchainDB nodes can connect to each other to form a cluster. Each node in the cluster runs the same software. A cluster contains one logical RethinkDB datastore. A cluster may have additional machines to do things such as cluster monitoring.
There are several kinds of nodes:
- A dev/test node is one created by a developer working on the BigchainDB server, e.g., for testing a new or changed code. A dev/test node is typically run on the developer’s local machine.
- A bare-bones node is deployed in the cloud, either as part of a testing cluster or as a starting point before upgrading the node to production-readiness. Our cloud deployment starter templates deploy a bare-bones node, as do our scripts for deploying a testing cluster on AWS.
- A production node is part of a federation’s BigchainDB cluster. A production node has the most components and requirements.
- BigchainDB network: BigchainDB nodes joined together form a BigchainDB network, where every node runs the same software and more nodes are added from time to time. In addition, some machines are especially added to perform monitoring.
- BigchainDB consortium: All the people and organisations that run the nodes in a BigchainDB network are associated with the BigchainDB consortium, which is another organisation, similar to a government structure, for making decisions. If a BigchainDB network is run by a single company, then the consortium is just that company.
Transactions in BigchainDB
The BigchainDB framework makes use of the consensus of the used database cluster. It is also possible for the end users to make their own validation mechanism, to validate the transactions in the blockchain network.
Basically, two types of transactions are available in BigchainDB, i.e., CREATE and TRANSFER. CREATE transactions create an asset, i.e., a collection of data and link it to at least one owner — its public key. In addition, all assets are ‘divisible’ and can therefore be broken down into different proportions for different owners. In a CREATE transaction, there is no previous owner. So an input in a CREATE transaction simply specifies who the person is who is registering the object (this is usually the same as the initial owner of the asset).
An asset can represent any physical or digital object. It can be a car or a house, or it can be a digital object like a customer order or an air mile. An asset can have one or multiple owners, but it can also be its own owner.
An asset can be many different things, such as:
- A claim
- A token
- A versioned document
- A time series
- A state machine
- A permission (RBAC or role-based access control)
With TRANSFER transactions, complex instructions can be created that link existing assets (and their shares) to conditions of further transactions. In this way, assets and parts of the network can be easily moved between subscribers in the network. As is usual in a blockchain, there is no way to delete an asset or modify its properties.
Conditions for transactions
For a transaction to be validly processed in the network, several conditions must be met.
Once the transaction has been received, the nodes check it for the correct structure. For example, CREATE transactions must contain their asset data while TRANSFER transactions reference the ID of an asset created. Both types also differ in the way they have to deal with inputs and outputs. Of course, each transaction must be signed and hashed before it is transmitted to the network.
If the structure of a transaction is valid, the validity of the contained data is checked. In short, ‘double spending’ is prevented in this step. This prevents the transaction from repeatedly transferring the same asset or issuing assets that have already been issued in other transactions.
In addition, you can implement your own validation mechanisms that could check asset generation and transfers for correct functionality — for example, whether realistic dimensions have been assigned to a car part or whether the colour code of an automotive paint job exists.
Features of BigChainDB
The following are the features of BigChainDB.
- Decentralised control: BigchainDB has no single point of control and enjoys 100 per cent decentralised control with no single point of failure.
- Query engine: BigchainDB is backed by MongoDB to search for all contents in stored transactions, assets, metadata and blocks.
- Tamper-resistant: BigchainDB is 100 per cent fool-proof and tamper-resistant, i.e., once data is stored, it cannot be updated or deleted.
- Byzantine fault tolerance: Up to one third of the nodes in the network can experience arbitrary faults and the rest of the network will still come to a consensus on the next block.
- Low latency: It has a powerful database with low latency. Transactions happen very quickly.
Highly customised: Helps the end users to design their own private network with custom assets, transactions, permissions and transparency. - Advanced access control: Enables setting permissions at the transaction level to ensure the right set of duties and selective access.
- Open source: Fully open source, and anyone can build applications on top of the stack of BigchainDB.
Differences between Hyperledger Fabric and BigchainDB
BigchainDB is a blockchain based database that is decentralised, query-able, immutable, has native support or multiple assets, is Byzantine fault tolerant and much more. However, BigchainDB being a blockchain database, lacks the processing layer or the business logic (smart contract) layer for the assets it stores. While we have platforms like Hyperledger Fabric, which excel at providing smart contract services, they are not platforms that provide user friendly querying functionality the way BigchainDB does.
So, for effectively creating the smart contracts for use across industries, integrating Hyperledger Fabric (which provides the business logic layer) and BigchainDB (which provides excellent functionalities of a database and the distributed ledger technology) provides an incredible solution for the digital future.
Table 1 highlights the differences between Hyperledger Fabric and BigchainDB.
Overall, with tabular analysis, Hyperledger Fabric is suitable for complex business cases whereas BigchainDB provides basic functionalities and mostly can be used in financial organisations.
Differences between Bitcoin, distributed databases and BigchainDB
Table 2 highlights the differences between Bitcoin, distributed databases and BigchainDB.
Configuring the BigChainDB network
To configure the network, you first need to deploy a machine for the BigchainDB node. The requirements for deployment are listed below.
- This can be any standalone machine, virtual machine, VM running on a cloud server or even a Linux based computer like Raspberry Pi.
- All nodes in the network should have a public or private IP address.
- Ubuntu 18.04/19.04/19.10 server operating system.
Here are the steps you need to follow for configuration.
Step 1: Update the system using the following code:
sudo apt update sudo apt full-upgrade
Step 2: DNS setup:
a. Register a domain for the BigchainDB node, i.e., server.com.
b. Add a sub-domain for BigchainDB node, i.e., node1.server.com.
c. Create a DNS ‘A RECORD’ for node1.server.com at your machine IP address.
Step 3: Install NGINX as follows:
sudo apt update sudo apt install nginx
Step 4: Configure and reload NGINX.
a. Generate an SSL certificate for the node subdomain, i.e., node1.server.com.
b. Copy the SSL private key into /etc/nginx/ssl/cert.key.
c. Create a ‘PEM File’ by concatenating your SSL certificate with all intermediate certificates.
d. Copy that PEM file into /etc/nginx/ssl/cert.pem.
e. Under the Bigchaindb/bigchaindb repository on GitHub, locate the file nginx/nginx.conf and copy the contents into /etc/nginx/nginx.conf on the machine.
f. Edit the file /etc/nginx/nginx.conf and relate the two instances of the string example.testnet2.com with the name of the sub-domain, i.e., node1.server.com.
g. Reload NGINX – sudo service nginx reload.
Step 5: Install BigchainDB, MongoDB and Tendermint.
a. Make sure that Python 3.6+ is installed in the machine and install the other OS-level packages.
sudo apt install -y python3-pip libssl-dev
b. BigchainDB server requires gevent and to install gevent, Pip19 or later must be installed.
sudo pip3 install -U pip
c. Now install BigchainDB server:
sudo pip3 install bigchaindb==2.0.0b9
Step 6: Configure the BigchainDB server.
a. Run the following command:
bigchaindb configure
The first question is: API Server Bind? (default ‘localhost:9984’).
If you are using NGINX, then accept the default value. And if you are not using NGINX, then enter: 0.0.0.0:9984.
If you are using NGINX, edit the BigchainDB config file–$home/.bigchaindb and set the following values under “wsserver”:
“advertised_scheme”: “wss”, “advertised_host”: “node1.server.com”, “advertised_port”: 443
b. Install and start MongoDB. Use the command:
sudo apt install mongodb
c. Install Tendermint as follows:
#sudo apt install -y unzip #wget https://github.com/tendermint/tendermint/releases/download/v0.22.8/tendermint_0.22.8_linux_amd64.zip #unzip tendermint_0.22.8_linux_amd64.zip #rm tendermint_0.22.8_linux_amd64.zip #sudo mv tendermint /usr/local/bin
d. Configure Tendermint. Use the command:
tendermint int
Step 7: Set up the BigchainDB network.
a. Most of the things can be done by node operators. However, these node operators, who are also called Members, must share some information with each other so that they can form a network. There is one special Member who helps to coordinate with everyone, called the Coordinator.
Each BigchainDB node is identified by its:
- hostname, i.e. the node’s DNS sub-domain, such as bnode.example.com, or its IP address, such as 10.191.93.111
- Tendermint pub_key.value
- Tendermint node_id
The Tendermint pub_key.value is stored in the file $HOME/.tendermint/config/priv_validator.json.
b. To get your Tendermint node_id, run the following command:
tendermint show_node_id
c. Next, create and share the genesis.json file. At this point, the coordinator should have received the data from all the members and should combine them in the file $HOME/.tendermint/config/genesis.json.
d. The new genesis.json file contains the data that describes the network. The key name is the member’s moniker; it can be any valid string, but use something human-readable like ‘Alice’s Node Shop’.
At this point, the coordinator must share the new genesis.json file with all members, and each member should have received the genesis.json file.
The members must now copy the genesis.json file into the local $HOME/.tendermint/config directory. Every member now shares the same chain_id and genesis_time (used to identify the network) and the same list of validators.
All the members must edit their $HOME/.tendermint/config/config.toml file and make the following changes:
moniker = “Name of our node” create_empty_blocks = false log_level = “main:info,state:info,*:error” persistent_peers = “@:26656,\ @:26656,\ @:26656,” send_rate = 102400000 recv_rate = 102400000 recheck = false
Note: The list of persistent_peers doesn’t have to include all nodes in the network.
e. Start BigchainDB and Tendermint using Monit. To install Monit, use the following command:
sudo apt install monit
f. If you installed the BigchainDB Python package as above, you should have the bigchaindb-monit-config script in your PATH now. Run the script to build a configuration file for Monit, as follows:
bigchaindb-monit-config
Run Monit as a daemon, instructing it to wake up every second to check on processes:
monit -d 1
Monit will run the BigchainDB and Tendermint processes, and restart them when they crash. If the root bigchaindb_ process crashes, Monit will also restart the Tendermint process.
You can check the status by running monit status or monit summary. By default, it will collect program logs into the ~/.bigchaindb-monit/logs folder.
g. Access the Web portal at http://hostname(ipaddress):9984.
Your BigchainDB network is now configured, and up and running!
