NISTIR 8202 Blockchain Technology Overview Dylan Yaga Peter Mell Nik Roby Karen Scarfone This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 NISTIR 8202 Blockchain Technology Overview Dylan Yaga Peter Mell Computer Security Division Information Technology Laboratory Nik Roby G2, Inc. Annapolis Junction, MD Karen Scarfone Scarfone Cybersecurity Clifton, VA This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 October 2018 U.S. Department of Commerce Wilbur L. Ross, Jr., Secretary National Institute of Standards and Technology Walter Copan, NIST Director and Under Secretary of Commerce for Standards and Technology National Institute of Standards and Technology Internal Report 8202 66 pages (October 2018) This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose. There may be references in this publication to other publications currently under development by NIST in accordance with its assigned statutory responsibilities. The information in this publication, including concepts and methodologies, may be used by federal agencies even before the completion of such companion publications. Thus, until each publication is completed, current requirements, guidelines, and procedures, where they exist, remain operative. For planning and transition purposes, federal agencies may wish to closely follow the development of these new publications by NIST. Organizations are encouraged to review all draft publications during public comment periods and provide feedback to NIST. Many NIST cybersecurity publications, other than the ones noted above, are available at https://csrc.nist.gov/publications. Comments on this publication may be submitted to: National Institute of Standards and Technology Attn: Computer Security Division, Information Technology Laboratory 100 Bureau Drive (Mail Stop 8930) Gaithersburg, MD 20899-8930 Email: nistir8202-comments@nist.gov All comments are subject to release under the Freedom of Information Act (FOIA). NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Reports on Computer Systems Technology The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the Nation’s measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of concept implementations, and technical analyses to advance the development and productive use of information technology. ITL’s responsibilities include the development of management, administrative, technical, and physical standards and guidelines for the cost-effective security and privacy of other than national security-related information in federal information systems. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Abstract Blockchains are tamper evident and tamper resistant digital ledgers implemented in a distributed fashion (i.e., without a central repository) and usually without a central authority (i.e., a bank, company, or government). At their basic level, they enable a community of users to record transactions in a shared ledger within that community, such that under normal operation of the blockchain network no transaction can be changed once published. This document provides a high-level technical overview of blockchain technology. The purpose is to help readers understand how blockchain technology works. Keywords blockchain; consensus model; cryptocurrency; cryptographic hash function; asymmetric-key cryptography; distributed ledger; distributed consensus algorithm; proof of work; proof of stake; round robin; proof of authority; proof of identity; proof of elapsed time; soft fork, hard fork; smart contracts; data oracle. ii NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Acknowledgments The authors wish to thank all contributors to this publication, and their colleagues who reviewed drafts of this report and contributed technical and editorial additions. This includes NIST staff James Dray, Sandy Ressler, Rick Kuhn, Lee Badger, Eric Trapnell, Mark Trapnell, James Shook and Michael Davidson. Additional thanks to all the people and organizations who submitted comments during the public comment period. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Audience This publication is designed for readers with little or no knowledge of blockchain technology who wish to understand at a high level how it works. It is not intended to be a technical guide; the discussion of the technology provides a conceptual understanding. Note that some examples, figures, and tables are simplified to fit the audience. Trademark Information All registered trademarks and trademarks belong to their respective organizations. iii NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Executive Summary Blockchains are tamper evident and tamper resistant digital ledgers implemented in a distributed fashion (i.e., without a central repository) and usually without a central authority (i.e., a bank, company, or government). At their basic level, they enable a community of users to record transactions in a shared ledger within that community, such that under normal operation of the blockchain network no transaction can be changed once published. In 2008, the blockchain idea was combined with several other technologies and computing concepts to create modern cryptocurrencies: electronic cash protected through cryptographic mechanisms instead of a central repository or authority. The first such blockchain based cryptocurrency was Bitcoin. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Within the Bitcoin blockchain, information representing electronic cash is attached to a digital address. Bitcoin users can digitally sign and transfer rights to that information to another user and the Bitcoin blockchain records this transfer publicly, allowing all participants of the network to independently verify the validity of the transactions. The Bitcoin blockchain is stored, maintained, and collaboratively managed by a distributed group of participants. This, along with certain cryptographic mechanisms, makes the blockchain resilient to attempts to alter the ledger later (modifying blocks or forging transactions). Because there are countless news articles and videos describing the “magic” of blockchain technology, this paper aims to describe the method behind the magic (i.e., how blockchain technology works). Arthur C. Clarke once wrote, “Any sufficiently advanced technology is indistinguishable from magic” [1]. Clarke’s statement is a perfect representation for the emerging applications of blockchain technology. There is hype around the use of blockchain technology, yet the technology is not well understood. It is not magical; it will not solve all problems. As with all new technology, there is a tendency to want to apply it to every sector in every way imaginable. To help promote correct application, this document provides information necessary to develop a high-level understanding of the technology. Blockchain technology is the foundation of modern cryptocurrencies, so named because of the heavy usage of cryptographic functions. Users utilize public and private keys to digitally sign and securely transact within the system. For cryptocurrency based blockchain networks which utilize mining (see section 4.1), users may solve puzzles using cryptographic hash functions in hopes of being rewarded with a fixed amount of the cryptocurrency. However, blockchain technology may be more broadly applicable than cryptocurrencies. In this work, we focus on the cryptocurrency use case, since that is the primary use of the technology today; however, there is a growing interest in other sectors. Organizations considering implementing blockchain technology need to understand fundamental aspects of the technology. For example, what happens when an organization implements a blockchain network and then decides they need to make modifications to the data stored? When using a database, modifying the actual data can be accomplished through a database query and update. Organizations must understand that while changes to the actual blockchain data may be difficult, applications using the blockchain as a data layer work around this by treating later blocks and transactions as updates or modifications to earlier blocks and transactions. This software abstraction allows for modifications to working data, while providing a full history of iv NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW changes. Another critical aspect of blockchain technology is how the participants agree that a transaction is valid. This is called “reaching consensus”, and there are many models for doing so, each with positives and negatives for particular business cases. It is important to understand that a blockchain is just one part of a solution. Blockchain implementations are often designed with a specific purpose or function. Example functions include cryptocurrencies, smart contracts (software deployed on the blockchain and executed by computers running that blockchain), and distributed ledger systems between businesses. There has been a constant stream of developments in the field of blockchain technology, with new platforms being announced constantly – the landscape is continuously changing. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 There are two general high-level categories for blockchain approaches that have been identified: permissionless, and permissioned. In a permissionless blockchain network anyone can read and write to the blockchain without authorization. Permissioned blockchain networks limit participation to specific people or organizations and allow finer-grained controls. Knowing the differences between these two categories allows an organization to understand which subset of blockchain technologies may be applicable to its needs. Despite the many variations of blockchain networks and the rapid development of new blockchain related technologies, most blockchain networks use common core concepts. Blockchains are a distributed ledger comprised of blocks. Each block is comprised of a block header containing metadata about the block, and block data containing a set of transactions and other related data. Every block header (except for the very first block of the blockchain) contains a cryptographic link to the previous block’s header. Each transaction involves one or more blockchain network users and a recording of what happened, and it is digitally signed by the user who submitted the transaction. Blockchain technology takes existing, proven concepts and merges them together into a single solution. This document explores the fundamentals of how these technologies work and the differences between blockchain approaches. This includes how the participants in the network come to agree on whether a transaction is valid and what happens when changes need to be made to an existing blockchain deployment. Additionally, this document explores when to consider using a blockchain network. The use of blockchain technology is not a silver bullet, and there are issues that must be considered such as how to deal with malicious users, how controls are applied, and the limitations of the implementations. Beyond the technology issues that need to be considered, there are operational and governance issues that affect the behavior of the network. For example, in permissioned blockchain networks, described later in this document, there are design issues surrounding what entity or entities will operate and govern the network for the intended user base. v NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Blockchain technology is still new and should be investigated with the mindset of “how could blockchain technology potentially benefit us?” rather than “how can we make our problem fit into the blockchain technology paradigm?”. Organizations should treat blockchain technology like they would any other technological solution at their disposal and use it in appropriate situations. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 vi NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Table of Contents Executive Summary ................................................................................................................... iv 1 Introduction ......................................................................................................................... 1 1.1 Background and History.................................................................................. 2 1.2 Purpose and Scope ........................................................................................ 3 1.3 Notes on Terms .............................................................................................. 3 1.4 Results of the Public Comment Period ........................................................... 4 1.5 Document Structure ........................................................................................ 4 This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 2 Blockchain Categorization ................................................................................................. 5 2.1 Permissionless ................................................................................................ 5 2.2 Permissioned .................................................................................................. 5 3 Blockchain Components .................................................................................................... 7 3.1 Cryptographic Hash Functions ........................................................................ 7 3.1.1 Cryptographic Nonce ............................................................................ 9 3.2 Transactions ................................................................................................... 9 3.3 Asymmetric-Key Cryptography ..................................................................... 11 3.4 Addresses and Address Derivation ............................................................... 12 3.4.1 Private Key Storage............................................................................ 13 3.5 Ledgers ......................................................................................................... 13 3.6 Blocks ........................................................................................................... 15 3.7 Chaining Blocks ............................................................................................ 17 4 Consensus Models ........................................................................................................... 18 4.1 Proof of Work Consensus Model .................................................................. 19 4.2 Proof of Stake Consensus Model ................................................................. 21 4.3 Round Robin Consensus Model ................................................................... 23 4.4 Proof of Authority/Proof of Identity Consensus Model................................... 23 4.5 Proof of Elapsed Time Consensus Model ..................................................... 23 4.6 Consensus Comparison Matrix ..................................................................... 25 4.7 Ledger Conflicts and Resolutions ................................................................. 27 5 Forking ............................................................................................................................... 29 5.1 Soft Forks ..................................................................................................... 29 5.2 Hard Forks .................................................................................................... 29 vii NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW 5.3 Cryptographic Changes and Forks ............................................................... 30 6 Smart Contracts ................................................................................................................ 32 7 Blockchain Limitations and Misconceptions ................................................................. 34 7.1 Immutability ................................................................................................... 34 7.2 Users Involved in Blockchain Governance .................................................... 35 7.3 Beyond the Digital ......................................................................................... 36 7.4 Blockchain Death .......................................................................................... 36 7.5 Cybersecurity ................................................................................................ 36 This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 7.5.1 Cyber and Network-based Attacks ..................................................... 37 7.6 Malicious Users............................................................................................. 37 7.7 No Trust ........................................................................................................ 38 7.8 Resource Usage ........................................................................................... 38 7.9 Inadequate Block Publishing Rewards.......................................................... 39 7.10 Public Key Infrastructure and Identity ........................................................... 39 8 Application Considerations ............................................................................................. 41 8.1 Additional Blockchain Considerations ........................................................... 44 9 Conclusions....................................................................................................................... 46 List of Appendices Appendix A— Acronyms .......................................................................................................... 47 Appendix B— Glossary ............................................................................................................ 49 Appendix C— References ........................................................................................................ 55 viii NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW List of Tables and Figures Table 1: Examples of Input Text and Corresponding SHA-256 Digest Values ................ 8 Figure 1 - Example Cryptocurrency Transaction ........................................................... 10 Figure 2 - A QR code example which has encoded the text “NISTIR 8202 - Blockchain Technology Overview QR code example” .............................................................. 12 Figure 3: Generic Chain of Blocks ................................................................................. 17 Figure 4: Ledger in Conflict ........................................................................................... 27 This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Figure 5: The chain with block_n(B) adds the next block, the chain with block_n(A) is now orphaned ........................................................................................................ 28 Table 2: Impact of Quantum Computing on Common Cryptographic Algorithms .......... 31 Figure 6 - DHS Science & Technology Directorate Flowchart ....................................... 42 ix NISTIR 8202 1 BLOCKCHAIN TECHNOLOGY OVERVIEW Introduction This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Blockchains are tamper evident and tamper resistant digital ledgers implemented in a distributed fashion (i.e., without a central repository) and usually without a central authority (i.e., a bank, company or government). At their basic level, they enable a community of users to record transactions in a shared ledger within that community, such that under normal operation of the blockchain network no transaction can be changed once published. In 2008, the blockchain idea was combined with several other technologies and computing concepts to create modern cryptocurrencies: electronic cash protected through cryptographic mechanisms instead of a central repository or authority. This technology became widely known in 2009 with the launch of the Bitcoin network, the first of many modern cryptocurrencies. In Bitcoin, and similar systems, the transfer of digital information that represents electronic cash takes place in a distributed system. Bitcoin users can digitally sign and transfer their rights to that information to another user and the Bitcoin blockchain records this transfer publicly, allowing all participants of the network to independently verify the validity of the transactions. The Bitcoin blockchain is independently maintained and managed by a distributed group of participants. This, along with cryptographic mechanisms, makes the blockchain resilient to attempts to alter the ledger later (modifying blocks or forging transactions). Blockchain technology has enabled the development of many cryptocurrency systems such as Bitcoin and Ethereum 1. Because of this, blockchain technology is often viewed as bound to Bitcoin or possibly cryptocurrency solutions in general. However, the technology is available for a broader variety of applications and is being investigated for a variety of sectors. The numerous components of blockchain technology along with its reliance on cryptographic primitives and distributed systems can make it challenging to understand. However, each component can be described simply and used as a building block to understand the larger complex system. Blockchains can be informally defined as: Blockchains are distributed digital ledgers of cryptographically signed transactions that are grouped into blocks. Each block is cryptographically linked to the previous one (making it tamper evident) after validation and undergoing a consensus decision. As new blocks are added, older blocks become more difficult to modify (creating tamper resistance). New blocks are replicated across copies of the ledger within the network, and any conflicts are resolved automatically using established rules. 1 Bitcoin and Ethereum are mentioned here since they are listed as the top two cryptocurrencies on market capitalization websites 1 NISTIR 8202 1.1 BLOCKCHAIN TECHNOLOGY OVERVIEW Background and History This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 The core ideas behind blockchain technology emerged in the late 1980s and early 1990s. In 1989, Leslie Lamport developed the Paxos protocol, and in 1990 submitted the paper The PartTime Parliament [2] to ACM Transactions on Computer Systems; the paper was finally published in a 1998 issue. The paper describes a consensus model for reaching agreement on a result in a network of computers where the computers or network itself may be unreliable. In 1991, a signed chain of information was used as an electronic ledger for digitally signing documents in a way that could easily show none of the signed documents in the collection had been changed [3]. These concepts were combined and applied to electronic cash in 2008 and described in the paper, Bitcoin: A Peer to Peer Electronic Cash System [4], which was published pseudonymously by Satoshi Nakamoto, and then later in 2009 with the establishment of the Bitcoin cryptocurrency blockchain network. Nakamoto’s paper contained the blueprint that most modern cryptocurrency schemes follow (although with variations and modifications). Bitcoin was just the first of many blockchain applications. Many electronic cash schemes existed prior to Bitcoin (e.g., ecash and NetCash), but none of them achieved widespread use. The use of a blockchain enabled Bitcoin to be implemented in a distributed fashion such that no single user controlled the electronic cash and no single point of failure existed; this promoted its use. Its primary benefit was to enable direct transactions between users without the need for a trusted third party. It also enabled the issuance of new cryptocurrency in a defined manner to those users who manage to publish new blocks and maintain copies of the ledger; such users are called miners in Bitcoin. The automated payment of the miners enabled distributed administration of the system without the need to organize. By using a blockchain and consensus-based maintenance, a self-policing mechanism was created that ensured that only valid transactions and blocks were added to the blockchain. In Bitcoin, the blockchain enabled users to be pseudonymous. This means that users are anonymous, but their account identifiers are not; additionally, all transactions are publicly visible. This has effectively enabled Bitcoin to offer pseudo-anonymity because accounts can be created without any identification or authorization process (such processes are typically required by Know-Your-Customer (KYC) laws). Since Bitcoin was pseudonymous, it was essential to have mechanisms to create trust in an environment where users could not be easily identified. Prior to the use of blockchain technology, this trust was typically delivered through intermediaries trusted by both parties. Without trusted intermediaries, the needed trust within a blockchain network is enabled by four key characteristics of blockchain technology, described below: • • • Ledger – the technology uses an append only ledger to provide full transactional history. Unlike traditional databases, transactions and values in a blockchain are not overridden. Secure – blockchains are cryptographically secure, ensuring that the data contained within the ledger has not been tampered with, and that the data within the ledger is attestable. Shared – the ledger is shared amongst multiple participants. This provides transparency across the node participants in the blockchain network. 2 NISTIR 8202 • BLOCKCHAIN TECHNOLOGY OVERVIEW Distributed – the blockchain can be distributed. This allows for scaling the number of nodes of a blockchain network to make it more resilient to attacks by bad actors. By increasing the number of nodes, the ability for a bad actor to impact the consensus protocol used by the blockchain is reduced. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 For blockchain networks that allow anyone to anonymously create accounts and participate (called permissionless blockchain networks), these capabilities deliver a level of trust amongst parties with no prior knowledge of one another; this trust can enable individuals and organizations to transact directly, which may result in transactions being delivered faster and at lower costs. For a blockchain network that more tightly controls access (called permissioned blockchain networks), where some trust may be present among users, these capabilities help to bolster that trust. 1.2 Purpose and Scope This document provides a high-level technical overview of blockchain technology. It looks at different categories of implementation approaches. It discusses the components of blockchain technology and provides diagrams and examples when possible. It discusses, at a high-level, some consensus models used in blockchain networks. It also provides an overview of how blockchain technology changes (known as forking) affect the blockchain network. It provides details on how blockchain technology was extended beyond attestable transactions to include attestable application processes known as smart contracts. It also touches on some of the limitations and misconceptions surrounding the technology. Finally, this document presents several areas that organizations should consider when investigating blockchain technology. It is intended to help readers to understand the technologies which comprise blockchain networks. 1.3 Notes on Terms The terminology for blockchain technology varies from one implementation to the next – to talk about the technology, generic terms will be used. Throughout this document the following terms will be used: • • • • • • Blockchain – the actual ledger Blockchain technology – a term to describe the technology in the most generic form Blockchain network – the network in which a blockchain is being used Blockchain implementation – a specific blockchain Blockchain network user – a person, organization, entity, business, government, etc. which is utilizing the blockchain network Node – an individual system within a blockchain network o Full node – a node that stores the entire blockchain, ensures transactions are valid  Publishing node – a full node that also publishes new blocks o Lightweight node – a node that does not store or maintain a copy of the blockchain and must pass their transactions to full nodes 3 NISTIR 8202 1.4 BLOCKCHAIN TECHNOLOGY OVERVIEW Results of the Public Comment Period This document has seen substantial revision in response to the public comments received. Part of the revising process was to tighten the scope, and to provide a more foundational document as an introduction to the technology. Please note that several sections present in the draft (7.1.2 Permissioned Use Cases, 7.2.2 - Permissionless Use Cases, and 8 - Blockchain Platforms) are not present in the published version. These topics were made explicitly out of scope for this document because the rapidly changing landscape and areas of interest around this technology, as well as the ever-increasing number of platforms, would make these sections out of place in such a foundational document. The topics in these sections are still being considered for future works. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Additionally, section 8.1.2 – Bitcoin Cash contained an erroneous and unverified statement which was not identified and removed during initial editing of the draft. Since this section has been removed, this issue is now addressed. 1.5 Document Structure The rest of this document is organized as follows: • • • • • • • • • • • Section 2 discusses the high-level categorization of blockchain technology: permissionless and permissioned. Section 3 defines the high-level components of a blockchain network architecture, including hashes, transactions, ledgers, blocks, and blockchains. Section 4 discusses several consensus models employed by blockchain technology. Section 5 introduces the concept of forking. Section 6 discusses smart contracts. Section 7 discusses several limitations as well as misconceptions surrounding blockchain technology. Section 8 discusses various application considerations, as well as provides additional considerations from government, academia, and technology enthusiasts. Section 9 is the conclusion. Appendix A provides a list of acronyms and abbreviations used in the document. Appendix B contains a glossary for selected terms defined in the document. Appendix C lists the references used throughout the document. 4 NISTIR 8202 2 BLOCKCHAIN TECHNOLOGY OVERVIEW Blockchain Categorization Blockchain networks can be categorized based on their permission model, which determines who can maintain them (e.g., publish blocks). If anyone can publish a new block, it is permissionless. If only particular users can publish blocks, it is permissioned. In simple terms, a permissioned blockchain network is like a corporate intranet that is controlled, while a permissionless blockchain network is like the public internet, where anyone can participate. Permissioned blockchain networks are often deployed for a group of organizations and individuals, typically referred to as a consortium. This distinction is necessary to understand as it impacts some of the blockchain components discussed later in this document. 2.1 Permissionless This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Permissionless blockchain networks are decentralized ledger platforms open to anyone publishing blocks, without needing permission from any authority. Permissionless blockchain platforms are often open source software, freely available to anyone who wishes to download them. Since anyone has the right to publish blocks, this results in the property that anyone can read the blockchain as well as issue transactions on the blockchain (through including those transactions within published blocks). Any blockchain network user within a permissionless blockchain network can read and write to the ledger. Since permissionless blockchain networks are open to all to participate, malicious users may attempt to publish blocks in a way that subverts the system (discussed in detail later). To prevent this, permissionless blockchain networks often utilize a multiparty agreement or ‘consensus’ system (see Section 4) that requires users to expend or maintain resources when attempting to publish blocks. This prevents malicious users from easily subverting the system. Examples of such consensus models include proof of work (see Section 4.1) and proof of stake (see Section 4.2) methods. The consensus systems in permissionless blockchain networks usually promote non-malicious behavior through rewarding the publishers of protocol-conforming blocks with a native cryptocurrency. 2.2 Permissioned Permissioned blockchain networks are ones where users publishing blocks must be authorized by some authority (be it centralized or decentralized). Since only authorized users are maintaining the blockchain, it is possible to restrict read access and to restrict who can issue transactions. Permissioned blockchain networks may thus allow anyone to read the blockchain or they may restrict read access to authorized individuals. They also may allow anyone to submit transactions to be included in the blockchain or, again, they may restrict this access only to authorized individuals. Permissioned blockchain networks may be instantiated and maintained using open source or closed source software. Permissioned blockchain networks can have the same traceability of digital assets as they pass through the blockchain, as well as the same distributed, resilient, and redundant data storage system as a permissionless blockchain networks. They also use consensus models for publishing blocks, but these methods often do not require the expense or maintenance of resources (as is the case with current permissionless blockchain networks). This is because the establishment of one’s identity is required to participate as a member of the permissioned blockchain network; those maintaining the blockchain have a level of trust with each other, since they were all 5 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW authorized to publish blocks and since their authorization can be revoked if they misbehave. Consensus models in permissioned blockchain networks are then usually faster and less computationally expensive. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Permissioned blockchain networks may also be used by organizations that need to more tightly control and protect their blockchain. However, if a single entity controls who can publish blocks, the users of the blockchain will need to have trust in that entity. Permissioned blockchain networks may also be used by organizations that wish to work together but may not fully trust one another. They can establish a permissioned blockchain network and invite business partners to record their transactions on a shared distributed ledger. These organizations can determine the consensus model to be used, based on how much they trust one another. Beyond trust, permissioned blockchain networks provide transparency and insight that may help better inform business decisions and hold misbehaving parties accountable. This can explicitly include auditing and oversight entities making audits a constant occurrence versus a periodic event. Some permissioned blockchain networks support the ability to selectively reveal transaction information based on a blockchain network users identity or credentials. With this feature, some degree of privacy in transactions may be obtained. For example, it could be that the blockchain records that a transaction between two blockchain network users took place, but the actual contents of transactions is only accessible to the involved parties. Some permissioned blockchain networks require all users to be authorized to send and receive transactions (they are not anonymous, or even pseudo-anonymous). In such systems parties work together to achieve a shared business process with natural disincentives to commit fraud or otherwise behave as a bad actor (since they can be identified). If bad behavior were to occur, it is well known where the organizations are incorporated, what legal remedies are available and how to pursue those remedies in the relevant judicial system. 6 NISTIR 8202 3 BLOCKCHAIN TECHNOLOGY OVERVIEW Blockchain Components Blockchain technology can seem complex; however, it can be simplified by examining each component individually. At a high level, blockchain technology utilizes well-known computer science mechanisms and cryptographic primitives (cryptographic hash functions, digital signatures, asymmetric-key cryptography) mixed with record keeping concepts (such as append only ledgers). This section discusses each individual main component: cryptographic hash functions, transactions, asymmetric-key cryptography, addresses, ledgers, blocks, and how blocks are chained together. 3.1 Cryptographic Hash Functions This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 An important component of blockchain technology is the use of cryptographic hash functions for many operations. Hashing is a method of applying a cryptographic hash function to data, which calculates a relatively unique output (called a message digest, or just digest) for an input of nearly any size (e.g., a file, text, or image). It allows individuals to independently take input data, hash that data, and derive the same result – proving that there was no change in the data. Even the smallest change to the input (e.g., changing a single bit) will result in a completely different output digest. Table 1 shows simple examples of this. Cryptographic hash functions have these important security properties: 1. They are preimage resistant. This means that they are one-way; it is computationally infeasible to compute the correct input value given some output value (e.g., given a digest, find x such that hash(x) = digest). 2. They are second preimage resistant. This means one cannot find an input that hashes to a specific output. More specifically, cryptographic hash functions are designed so that given a specific input, it is computationally infeasible to find a second input which produces the same output (e.g., given x, find y such that hash(x) = hash(y)). The only approach available is to exhaustively search the input space, but this is computationally infeasible to do with any chance of success. 3. They are collision resistant. This means that one cannot find two inputs that hash to the same output. More specifically, it is computationally infeasible to find any two inputs that produce the same digest (e.g., find an x and y which hash(x) = hash(y)). A specific cryptographic hash function used in many blockchain implementations is the Secure Hash Algorithm (SHA) with an output size of 256 bits (SHA-256). Many computers support this algorithm in hardware, making it fast to compute. SHA-256 has an output of 32 bytes (1 byte = 8 bits, 32 bytes = 256 bits), generally displayed as a 64-character hexadecimal string (see Table 1 below). This means that there are 2256 ≈ 1077, or 115,792,089,237,316,195,423,570,985,008,687,907,853,269,984,665,640,564,039,457,584,007,913,129,639,936 possible digest values. The algorithm for SHA-256, as well as others, is specified in Federal Information Processing Standard (FIPS) 180-4 [5]. The NIST Secure Hashing website [6] contains FIPS specifications for all NIST-approved hashing algorithms. 7 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Table 1: Examples of Input Text and Corresponding SHA-256 Digest Values Input Text 1 SHA-256 Digest Value 0x6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b 2 0xd4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 Hello, World! 0xdffd6021bb2bd5b0af676290809ec3a53191dd81c7f70a4b28688a362182986f This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Since there are an infinite number of possible input values and a finite number of possible output digest values, it is possible but highly unlikely to have a collision where hash(x) = hash(y) (i.e., the hash of two different inputs produces the same digest). SHA-256 is said to be collision resistant, since to find a collision in SHA-256, one would have to execute the algorithm, on average, about 2128 times (which is 340 undecillions, or more precisely 340,282,366,920,938,463,463,374,607,431,768,211,456; roughly 3.402 x 1038). To put this into perspective, the hash rate (hashes per second) of the entire Bitcoin network in 2015 was 300 quadrillion hashes per second (300,000,000,000,000,000/s) [7]. At that rate, it would take the entire Bitcoin network roughly 35,942,991,748,521 (roughly 3.6 x 1013) years 2 to manufacture a collision (note that the universe is estimated to be 1.37 x 1010 years old) 3. Even if any such input x and y that produce the same digest, it would be also very unlikely for both inputs to be valid in the context of the blockchain network (i.e., x and y are both valid transactions). Within a blockchain network, cryptographic hash functions are used for many tasks, such as: • • • • Address derivation – discussed in section 3.4. Creating unique identifiers. Securing the block data – a publishing node will hash the block data, creating a digest that will be stored within the block header. Securing the block header – a publishing node will hash the block header. If the blockchain network utilizes a proof of work consensus model (see Section 4.1), the publishing node will need to hash the block header with different nonce values (see Section 3.1.1) until the puzzle requirements have been fulfilled. The current block header’s hash digest will be included within the next block’s header, where it will secure the current block header data. Because the block header includes a hash representation of the block data, the block data itself is 2 Calculation: 2128/((((300000000000000000×60) (hash per second -> minute) ×60) (minute -> hour) ×24) (hour -> day) ×365.25) (day -> year) = 35942991748521.060268986932617580573454677584269188193 years https://www.wolframalpha.com/input/?i=2%5E128%2F(300000000000000000+*+60+*+60+*+24+*+365.25) 3 As estimated by measurements made by the Wilkinson Microwave Anisotropy Probe https://map.gsfc.nasa.gov/universe/uni_age.html 8 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW also secured when the block header digest is stored in the next block. There are many families of cryptographic hash functions utilized in blockchain technology (SHA-256 is not the only one), such as Keccak (which was selected by NIST as the winner of a competition to create the SHA-3 hashing standard), as well as RIPEMD-160.[8] 3.1.1 Cryptographic Nonce A cryptographic nonce is an arbitrary number that is only used once. A cryptographic nonce can be combined with data to produce different hash digests per nonce: hash (data + nonce) = digest This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Only changing the nonce value provides a mechanism for obtaining different digest values while keeping the same data. This technique is utilized in the proof of work consensus model (see Section 4.1). 3.2 Transactions A transaction represents an interaction between parties. With cryptocurrencies, for example, a transaction represents a transfer of the cryptocurrency between blockchain network users. For business-to-business scenarios, a transaction could be a way of recording activities occurring on digital or physical assets. Figure 1 shows a notional example of a cryptocurrency transaction. Each block in a blockchain can contain zero or more transactions. For some blockchain implementations, a constant supply of new blocks (even with zero transactions) is critical to maintain the security of the blockchain network; by having a constant supply of new blocks being published, it prevents malicious users from ever “catching up” and manufacturing a longer, altered blockchain (see Section 4.7). The data which comprises a transaction can be different for every blockchain implementation, however the mechanism for transacting is largely the same. A blockchain network user sends information to the blockchain network. The information sent may include the sender’s address (or another relevant identifier), sender’s public key, a digital signature, transaction inputs and transaction outputs. A single cryptocurrency transaction typically requires at least the following information, but can contain more: • Inputs – The inputs are usually a list of the digital assets to be transferred. A transaction will reference the source of the digital asset (providing provenance) – either the previous transaction where it was given to the sender, or for the case of new digital assets, the origin event. Since the input to the transaction is a reference to past events, the digital assets do not change. In the case of cryptocurrencies this means that value cannot be added or removed from existing digital assets. Instead, a single digital asset can be split into multiple new digital assets (each with lesser value) or multiple digital assets can be combined to form fewer new digital assets (with a correspondingly greater value). The splitting or joining of assets will be specified within the transaction output. 9 NISTIR 8202 • BLOCKCHAIN TECHNOLOGY OVERVIEW The sender must also provide proof that they have access to the referenced inputs, generally by digitally signing the transaction – proving access to the private key. Outputs – The outputs are usually the accounts that will be the recipients of the digital assets along with how much digital asset they will receive. Each output specifies the number of digital assets to be transferred to the new owner(s), the identifier of the new owner(s), and a set of conditions the new owners must meet to spend that value. If the digital assets provided are more than required, the extra funds must be explicitly sent back to the sender (this is a mechanism to “make change”). This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Figure 1 - Example Cryptocurrency Transaction While primarily used to transfer digital assets, transactions can be more generally used to transfer data. In a simple case, someone may simply want to permanently and publicly post data on the blockchain. In the case of smart contract systems, transactions can be used to send data, process that data, and store some result on the blockchain. For example, a transaction can be used to change an attribute of a digitized asset such as the location of a shipment within a blockchain technology-based supply chain system. Regardless of how the data is formed and transacted, determining the validity and authenticity of a transaction is important. The validity of a transaction ensures that the transaction meets the protocol requirements and any formalized data formats or smart contract requirements specific to the blockchain implementation. The authenticity of a transaction is also important, as it determines that the sender of digital assets had access to those digital assets. Transactions are typically digitally signed by the sender’s associated private key (asymmetric-key cryptography is briefly discussed in Section 3.3) and can be verified at any time using the associated public key. 10 NISTIR 8202 3.3 BLOCKCHAIN TECHNOLOGY OVERVIEW Asymmetric-Key Cryptography Blockchain technology uses asymmetric-key cryptography 4 (also referred to as public key cryptography). Asymmetric-key cryptography uses a pair of keys: a public key and a private key that are mathematically related to each other. The public key is made public without reducing the security of the process, but the private key must remain secret if the data is to retain its cryptographic protection. Even though there is a relationship between the two keys, the private key cannot efficiently be determined based on knowledge of the public key. One can encrypt with a private key and then decrypt with the public key. Alternately, one can encrypt with a public key and then decrypt with a private key. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Asymmetric-key cryptography enables a trust relationship between users who do not know or trust one another, by providing a mechanism to verify the integrity and authenticity of transactions while at the same time allowing transactions to remain public. To do this, the transactions are ‘digitally signed’. This means that a private key is used to encrypt a transaction such that anyone with the public key can decrypt it. Since the public key is freely available, encrypting the transaction with the private key proves that the signer of the transaction has access to the private key. Alternately, one can encrypt data with a user’s public key such that only users with access to the private key can decrypt it. A drawback is that asymmetric-key cryptography is often slow to compute. This contrasts with symmetric-key cryptography in which a single secret key is used to both encrypt and decrypt. With symmetric-key cryptography users must already have a trust relationship established with one another to exchange the pre-shared key. In a symmetric system, any encrypted data that can be decrypted with the pre-shared key confirms it was sent by another user with access to the pre-shared key; no user without access to the pre-shared key will be able to view the decrypted data. Compared to asymmetric-key cryptography, symmetric-key cryptography is very fast to compute. Because of this, when one claims to be encrypting something using asymmetric-key cryptography, oftentimes the data is encrypted with symmetrickey cryptography and then the symmetric-key is encrypted using asymmetric-key cryptography. This ‘trick’ can greatly speed up asymmetric-key cryptography. Here is a summary of the use of asymmetric-key cryptography in many blockchain networks: • • • • 4 Private keys are used to digitally sign transactions. Public keys are used to derive addresses. Public keys are used to verify signatures generated with private keys. Asymmetric-key cryptography provides the ability to verify that the user transferring value to another user is in possession of the private key capable of signing the transaction. FIPS Publication 186-4, Digital Signature Standard [9] specifies a common algorithm for digital signing used in blockchain technologies: Elliptic Curve Digital Signature Algorithm (ECDSA). 11 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Some permissioned blockchain networks can leverage a business’s existing public key infrastructure for asymmetric-key cryptography to provide user credentials – rather than having each blockchain network user manage their own asymmetric-keys. This is done by utilizing existing directory services and using that information within the blockchain network. Blockchain networks which utilize an existing directory service can access it via existing protocols, such as the Lightweight Directory Access Protocol (LDAP) [10], and utilize the information from the directory natively, or import it into an internal certificate authority within the blockchain network. 3.4 Addresses and Address Derivation This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Some blockchain networks make use of an address, which is a short, alphanumeric string of characters derived from the blockchain network user’s public key using a cryptographic hash function, along with some additional data (e.g., version number, checksums). Most blockchain implementations make use of addresses as the “to” and “from” endpoints in a transaction. Addresses are shorter than the public keys and are not secret. One method to generate an address is to create a public key, applying a cryptographic hash function to it, and converting the hash to text: public key  cryptographic hash function  address Each blockchain implementation may implement a different method to derive an address. For permissionless blockchain networks, which allow anonymous account creation, a blockchain network user can generate as many asymmetric-key pairs, and therefore addresses as desired, allowing for a varying degree of pseudo-anonymity. Addresses may act as the public-facing identifier in a blockchain network for a user, and oftentimes an address will be converted into a QR code (Quick Response Code, a 2-dimensional bar code which can contain arbitrary data) for easier use with mobile devices. Figure 2 - A QR code example which has encoded the text “NISTIR 8202 - Blockchain Technology Overview QR code example” 12 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Blockchain network users may not be the only source of addresses within blockchain networks. It is necessary to provide a method of accessing a smart contract once it has been deployed within a blockchain network. For Ethereum, smart contracts are accessible via a special address called a contract account. This account address is created when a smart contract is deployed (the address for a contract account is deterministically computed from the smart contract creator’s address [11]). This contract account allows for the contract to be executed whenever it receives a transaction, as well as create additional smart contracts in turn. 3.4.1 Private Key Storage This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 With some blockchain networks (especially with permissionless blockchain networks), users must manage and securely store their own private keys. Instead of recording them manually, they often use software to securely store them. This software is often referred to as a wallet. The wallet can store private keys, public keys, and associated addresses. It may also perform other functions, such as calculating the total number of digital assets a user may have. If a user loses a private key, then any digital asset associated with that key is lost, because it is computationally infeasible to regenerate the same private key. If a private key is stolen, the attacker will have full access to all digital assets controlled by that private key. The security of private keys is so important that many users use special secure hardware to store them; alternatively, users may take advantage of an emerging industry of private key escrow services. These key escrow services can also satisfy KYC laws in addition to storing private keys as users must provide proof of their identity when creating an account. Private key storage is an extremely important aspect of blockchain technology. When it is reported in the news that “Cryptocurrency XYZ was stolen from…”, it almost certainly means some private keys were found and used to sign a transaction sending the money to a new account, not that the blockchain network itself was compromised. Note that because blockchain data cannot generally be changed, once a criminal steals a private key and publicly transfers the associated funds to another account, that transaction generally cannot be undone. 3.5 Ledgers A ledger is a collection of transactions. Throughout history, pen and paper ledgers have been used to keep track of the exchange of goods and services. In modern times, ledgers have been stored digitally, often in large databases owned and operated by a centralized trusted third party (i.e., the owner of the ledger) on behalf of a community of users. These ledgers with centralized ownership can be implemented in a centralized or distributed fashion (i.e., just one server or a coordinating cluster of servers). There is growing interest in exploring having distributed ownership of the ledger. Blockchain technology enables such an approach using both distributed ownership as well as a distributed physical architecture. The distributed physical architecture of blockchain networks often involve a much larger set of computers than is typical for centrally managed distributed physical architecture. The growing interest in distributed ownership of ledgers is due to possible trust, security, and reliability concerns related to ledgers with centralized ownership: 13 NISTIR 8202 • BLOCKCHAIN TECHNOLOGY OVERVIEW Centrally owned ledgers may be lost or destroyed; a user must trust that the owner is properly backing up the system. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 o A blockchain network is distributed by design, creating many backup copies all updating and syncing to the same ledger data between peers. A key benefit to blockchain technology is that every user can maintain their own copy of the ledger. Whenever new full nodes join the blockchain network, they reach out to discover other full nodes and request a full copy of the blockchain network’s ledger, making loss or destruction of the ledger difficult. Note – certain blockchain implementations provide the capability to support concepts such as private transactions or private channels. Private transactions facilitate the delivery of information only to those nodes participating in a transaction and not the entire network. • Centrally owned ledgers may be on a homogeneous network, where all software, hardware and network infrastructure may be the same. Because of this characteristic, the overall system resiliency may be reduced since an attack on one part of the network will work on everywhere. o A blockchain network is a heterogeneous network, where the software, hardware and network infrastructure are all different. Because of the many differences between nodes on the blockchain network, an attack on one node is not guaranteed to work on other nodes. • Centrally owned ledgers may be located entirely in specific geographic locations (e.g., all in one country). If network outages were to occur in that location, the ledger and services which depend on it may not be available. o A blockchain network can be comprised of geographically diverse nodes which may be found around the world. Because of this, and the blockchain network working in a peer-to-peer fashion, it is resilient to the loss of any node, or even an entire region of nodes. • The transactions on a centrally owned ledger are not made transparently and may not be valid; a user must trust that the owner is validating each received transaction. o A blockchain network must check that all transactions are valid; if a malicious node was transmitting invalid transactions, others would detect and ignore them, preventing the invalid transactions from propagating throughout the blockchain network. • The transaction list on a centrally owned ledger may not be complete; a user must trust that the owner is including all valid transactions that have been received. o A blockchain network holds all accepted transactions within its distributed ledger. To build a new block, a reference must be made to a previous block – therefore building on top of it. If a publishing node did not include a reference to the latest block, other nodes would reject it. • The transaction data on a centrally owned ledger may have been altered; a user must trust that the owner is not altering past transactions. 14 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW o A blockchain network utilizes cryptographic mechanisms such as digital signatures and cryptographic hash functions to provide tamper evident and tamper resistant ledgers. • The centrally owned system may be insecure; a user must trust that the associated computer systems and networks are receiving critical security patches and have implemented best practices for security. The system may be breached and have had personal information stolen because of insecurities. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 o A blockchain network, due to the distributed nature, provides no centralized point of attack. Generally, information on a blockchain network is publicly viewable, and offers nothing to steal. To attack blockchain network users, an attacker would need to individually target them. Targeting the blockchain itself would be met with the resistance of the honest nodes present in the system. If an individual node was not patched, it would only affect that node – not the system overall. 3.6 Blocks Blockchain network users submit candidate transactions to the blockchain network via software (desktop applications, smartphone applications, digital wallets, web services, etc.). The software sends these transactions to a node or nodes within the blockchain network. The chosen nodes may be non-publishing full nodes as well as publishing nodes. The submitted transactions are then propagated to the other nodes in the network, but this by itself does not place the transaction in the blockchain. For many blockchain implementations, once a pending transaction has been distributed to nodes, it must then wait in a queue until it is added to the blockchain by a publishing node. Transactions are added to the blockchain when a publishing node publishes a block. A block contains a block header and block data. The block header contains metadata for this block. The block data contains a list of validated and authentic transactions which have been submitted to the blockchain network. Validity and authenticity is ensured by checking that the transaction is correctly formatted and that the providers of digital assets in each transaction (listed in the transaction’s ‘input’ values) have each cryptographically signed the transaction. This verifies that the providers of digital assets for a transaction had access to the private key which could sign over the available digital assets. The other full nodes will check the validity and authenticity of all transactions in a published block and will not accept a block if it contains invalid transactions. It should be noted that every blockchain implementation can define its own data fields; however, many blockchain implementations utilize data fields like the following: • Block Header o The block number, also known as block height in some blockchain networks. o The previous block header’s hash value. o A hash representation of the block data (different methods can be used to accomplish this, such as a generating a Merkle tree (defined in Appendix B), and storing the root hash, or by utilizing a hash of all the combined block data). o A timestamp. 15 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW o The size of the block. o The nonce value. For blockchain networks which utilize mining, this is a number which is manipulated by the publishing node to solve the hash puzzle (see Section 4.1 for details). Other blockchain networks may or may not include it or use it for another purpose other than solving a hash puzzle. • Block Data o A list of transactions and ledger events included within the block. o Other data may be present. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 16 NISTIR 8202 3.7 BLOCKCHAIN TECHNOLOGY OVERVIEW Chaining Blocks This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Blocks are chained together through each block containing the hash digest of the previous block’s header, thus forming the blockchain. If a previously published block were changed, it would have a different hash. This in turn would cause all subsequent blocks to also have different hashes since they include the hash of the previous block. This makes it possible to easily detect and reject altered blocks. Figure 3 shows a generic chain of blocks. Figure 3: Generic Chain of Blocks 17 NISTIR 8202 4 BLOCKCHAIN TECHNOLOGY OVERVIEW Consensus Models A key aspect of blockchain technology is determining which user publishes the next block. This is solved through implementing one of many possible consensus models. For permissionless blockchain networks there are generally many publishing nodes competing at the same time to publish the next block. They usually do this to win cryptocurrency and/or transaction fees. They are generally mutually distrusting users that may only know each other by their public addresses. Each publishing node is likely motivated by a desire for financial gain, not the well-being of the other publishing nodes or even the network itself. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 In such a situation, why would a user propagate a block that another user is attempting to publish? Also, who resolves conflicts when multiple nodes publish a block at approximately the same time? To make this work, blockchain technologies use consensus models to enable a group of mutually distrusting users to work together. When a user joins a blockchain network, they agree to the initial state of the system. This is recorded in the only pre-configured block, the genesis block. Every blockchain network has a published genesis block and every block must be added to the blockchain after it, based on the agreed-upon consensus model. Regardless of the model, however, each block must be valid and thus can be validated independently by each blockchain network user. By combining the initial state and the ability to verify every block since then, users can independently agree on the current state of the blockchain. Note that if there were ever two valid chains presented to a full node, the default mechanism in most blockchain networks is that the ‘longer’ chain is viewed as the correct one and will be adopted; this is because it has had the most amount of work put into it. This happens frequently with some consensus models and will be discussed in detail. The following properties are then in place: • • • • The initial state of the system is agreed upon (e.g., the genesis block). Users agree to the consensus model by which blocks are added to the system. Every block is linked to the previous block by including the previous block header’s hash digest (except for the first ‘genesis’ block, which has no previous block and for which the hash of the previous block header is usually set to all zeros). Users can verify every block independently. In practice, software handles everything and the users do not need to be aware of these details. A key feature of blockchain technology is that there is no need to have a trusted third party provide the state of the system—every user within the system can verify the system’s integrity. To add a new block to the blockchain, all nodes must come to a common agreement over time; however, some temporary disagreement is permitted. For permissionless blockchain networks, the consensus model must work even in the presence of possibly malicious users since these users might attempt to disrupt or take over the blockchain. Note that for permissioned blockchain networks legal remedies may be used if a user acts maliciously. 18 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW In some blockchain networks, such as permissioned, there may exist some level of trust between publishing nodes. In this case, there may not be the need for a resource intensive (computation time, investment, etc.) consensus model to determine which participant adds the next block to the chain. Generally, as the level of trust increases, the need for resource usage as a measure of generating trust decreases. For some permissioned blockchain implementations, the view of consensus extends beyond ensuring validity and authenticity of the blocks but encompasses the entire systems of checks and validations from the proposal of a transaction, to its final inclusion on a block. In the following sections, several consensus models as well as the most common conflict resolution approach are discussed. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 4.1 Proof of Work Consensus Model In the proof of work (PoW) model, a user publishes the next block by being the first to solve a computationally intensive puzzle. The solution to this puzzle is the “proof” they have performed work. The puzzle is designed such that solving the puzzle is difficult but checking that a solution is valid is easy. This enables all other full nodes to easily validate any proposed next blocks, and any proposed block that did not satisfy the puzzle would be rejected. A common puzzle method is to require that the hash digest of a block header be less than a target value. Publishing nodes make many small changes to their block header (e.g., changing the nonce) trying to find a hash digest that meets the requirement. For each attempt, the publishing node must compute the hash for the entire block header. Hashing the block header many times becomes a computationally intensive process. The target value may be modified over time to adjust the difficulty (up or down) to influence how often blocks are being published. For example, Bitcoin, which uses the proof of work model, adjusts the puzzle difficulty every 2016 blocks to influence the block publication rate to be around once every ten minutes. The adjustment is made to the difficulty level of the puzzle, and essentially either increases or decreases the number of leading zeros required. By increasing the number of leading zeros, it increases the difficulty of the puzzle, because any solution must be less than the difficulty level – meaning there are fewer possible solutions. By decreasing the number of leading zeros, it decreases the difficulty level, because there are more possible solutions. This adjustment is to maintain the computational difficulty of the puzzle, and therefore maintain the core security mechanism of the Bitcoin network. Available computing power increases over time, as does the number of publishing nodes, so the puzzle difficulty is generally increasing. Adjustments to the difficulty target aim to ensure that no entity can take over block production, but as a result the puzzle solving computations require significant resource consumption. Due to the significant resource consumption of some proof of work blockchain networks, there is a move to add publishing nodes to areas where there is a surplus supply of cheap electricity. An important aspect of this model is that the work put into a puzzle does not influence one’s likelihood of solving the current or future puzzles because the puzzles are independent. This means that when a user receives a completed and valid block from another user, they are 19 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW incentivized to discard their current work and to start building off the newly received block instead because they know the other publishing nodes will be building off it. As an example, consider a puzzle where, using the SHA-256 algorithm, a computer must find a hash value meeting the following target criteria (known as the difficulty level): SHA256(“blockchain” + Nonce) = Hash Digest starting with “000000” In this example, the text string “blockchain” is appended with a nonce value and then the hash digest is calculated. The nonce values used will be numeric values only. This is a relatively easy puzzle to solve and some sample output follows: This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 SHA256("blockchain0") = 0xbd4824d8ee63fc82392a6441444166d22ed84eaa6dab11d4923075975acab938 (not solved) SHA256("blockchain1") = 0xdb0b9c1cb5e9c680dfff7482f1a8efad0e786f41b6b89a758fb26d9e223e0a10 (not solved) … SHA256("blockchain10730895") = 0x000000ca1415e0bec568f6f605fcc83d18cac7a4e6c219a957c10c6879d67587 (solved) To solve this puzzle, it took 10,730,896 guesses (completed in 54 seconds on relatively old hardware, starting at 0 and testing one value at a time). In this example, each additional “leading zero” value increases the difficulty. By increasing the target by one additional leading zero (“0000000”), the same hardware took 934,224,175 guesses to solve the puzzle (completed in 1 hour, 18 minutes, 12 seconds): SHA256("blockchain934224174") = 0x0000000e2ae7e4240df80692b7e586ea7a977eacbd031819d0e603257edb3a81 There is currently no known shortcut to this process; publishing nodes must expend computation effort, time, and resources to find the correct nonce value for the target. Often the publishing nodes attempt to solve this computationally difficult puzzle to claim a reward of some sort (usually in the form of a cryptocurrency offered by the blockchain network). The prospect of being rewarded for extending and maintaining the blockchain is referred to as a reward system or incentive model. Once a publishing node has performed this work, they send their block with a valid nonce to full nodes in the blockchain network. The recipient full nodes verify that the new block fulfills the puzzle requirement, then add the block to their copy of the blockchain and resend the block to their peer nodes. In this manner, the new block gets quickly distributed throughout the network of participating nodes. Verification of the nonce is easy since only a single hash needs to be done to check to see if it solves the puzzle. For many proof of work based blockchain networks, publishing nodes tend to organize 20 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW themselves into “pools” or “collectives” whereby they work together to solve puzzles and split the reward. This is possible because work can be distributed between two or more nodes across a collective to share the workload and rewards. Splitting the example program into quarters, each node can take an equal amount of the nonce value range to test: • • • • Node Node Node Node 1: 2: 3: 4: check check check check nonce nonce nonce nonce 0000000000 0536870912 1073741824 1610612736 to to to to 0536870911 1073741823 1610612735 2147483647 The following result was the first to be found to solve the puzzle: This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 SHA256("blockchain1700876653") = 0x00000003ba55d20c9cbd1b6fb34dd81c3553360ed918d07acf16dc9e75d7c7f1 This is a completely new nonce, but still one that solved the puzzle. It took 90,263,918 guesses (completed in 10 minutes, 14 seconds). Dividing up the work amongst many more machines yields much better results, as well as more consistent rewards in a proof of work model. The use of a computationally difficult puzzle helps to combat the “Sybil Attack” – a computer security attack (not limited to blockchain networks) where an attacker can create many nodes (i.e., creating multiple identities) to gain influence and exert control. The proof of work model combats this by having the focus of network influence being the amount of computational power (hardware, which costs money) mixed with a lottery system (the most hardware increases likelihood but does not guarantee it) versus in network identities (which are generally costless to create). 4.2 Proof of Stake Consensus Model The proof of stake (PoS) model is based on the idea that the more stake a user has invested into the system, the more likely they will want the system to succeed, and the less likely they will want to subvert it. Stake is often an amount of cryptocurrency that the blockchain network user has invested into the system (through various means, such as by locking it via a special transaction type, or by sending it to a specific address, or holding it within special wallet software). Once staked, the cryptocurrency is generally no longer able to be spent. Proof of stake blockchain networks use the amount of stake a user has as a determining factor for publishing new blocks. Thus, the likelihood of a blockchain network user publishing a new block is tied to the ratio of their stake to the overall blockchain network amount of staked cryptocurrency. With this consensus model, there is no need to perform resource intensive computations (involving time, electricity, and processing power) as found in proof of work. Since this consensus model utilizes fewer resources, some blockchain networks have decided to forego a block creation reward; these systems are designed so that all the cryptocurrency is already distributed among users rather than new cryptocurrency being generated at a constant pace. In such systems, the reward for block publication is then usually the earning of user provided transaction fees. The methods for how the blockchain network uses the stake can vary. Here we discuss four 21 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW approaches: random selection of staked users, multi-round voting, coin aging systems and delegate systems. Regardless of the exact approach, users with more stake are more likely to publish new blocks. When the choice of block publisher is a random choice (sometimes referred to as chain-based proof of stake), the blockchain network will look at all users with stake and choose amongst them based on their ratio of stake to the overall amount of cryptocurrency staked. So, if a user had 42 % of the entire blockchain network stake they would be chosen 42 % of the time; those with 1 % would be chosen 1 % of the time. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 When the choice of block publisher is a multi-round voting system (sometime referred to as Byzantine fault tolerance proof of stake [12]) there is added complexity. The blockchain network will select several staked users to create proposed blocks. Then all staked users will cast a vote for a proposed block. Several rounds of voting may occur before a new block is decided upon. This method allows all staked users to have a voice in the block selection process for every new block. When the choice of block publisher is through a coin age system referred to as a coin age proof of stake, staked cryptocurrency has an age property. After a certain amount of time (such as 30 days) the staked cryptocurrency can count towards the owning user being selected to publish the next block. The staked cryptocurrency then has its age reset, and it cannot be used again until after the requisite time has passed. This method allows for users with more stake to publish more blocks, but to not dominate the system – since they have a cooldown timer attached to every cryptocurrency coin counted towards creating blocks. Older coins and larger groups of coins will increase the probability of being chosen to publish the next block. To prevent stakeholders from hoarding aged cryptocurrencies, there is generally a built-in maximum to the probability of winning. When the choice of block publisher is through a delegate system, users vote for nodes to become publishing nodes – therefore creating blocks on their behalf. Blockchain network users’ voting power is tied to their stake so the larger the stake, the more weight the vote has. Nodes who receive the most votes become publishing nodes and can validate and publish blocks. Blockchain network users can also vote against an established publishing node, to try to remove them from the set of publishing nodes. Voting for publishing nodes is continuous and remaining a publishing node can be quite competitive. The threat of losing publishing node status, and therefore rewards and reputation is constant so publishing nodes are incentivized to not act maliciously. Additionally, blockchain network users vote for delegates, who participate in the governance of the blockchain. Delegates will propose changes, and improvements, which will be voted on by blockchain network users. It is worth noting that a problem known as “nothing at stake” may arise from some proof of stake algorithms. If multiple competing blockchains were to exist at some point (because of a temporary ledger conflict as discussed in Section 4.7), a staked user could act on every such competing chain – since it is essentially free to do so. The staked user may do this as a way of increasing their odds of earning a reward. This can cause multiple blockchain branches to continue to grow without being reconciled into a singular branch for extended periods of time. 22 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW Under proof of stake systems, the “rich” can more easily stake more of the digital assets, earning themselves more digital assets; however, to obtain the majority of digital assets within a system to “control” it is generally cost prohibitive. 4.3 Round Robin Consensus Model This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Round Robin is a consensus model that is used by some permissioned blockchain networks. Within this model of consensus, nodes take turns in creating blocks. Round Robin Consensus has a long history grounded in distributed system architecture. To handle situations where a publishing node is not available to publish a block on its turn, these systems may include a time limit to enable available nodes to publish blocks so that unavailable nodes will not cause a halt in block publication. This model ensures no one node creates the majority of the blocks. It benefits from a straightforward approach, lacks cryptographic puzzles, and has low power requirements. Since there is a need for trust amongst nodes, round robin does not work well in the permissionless blockchain networks used by most cryptocurrencies. This is because malicious nodes could continuously add additional nodes to increase their odds of publishing new blocks. In the worst case, they could use this to subvert the correct operation of the blockchain network. 4.4 Proof of Authority/Proof of Identity Consensus Model The proof of authority (also referred to as proof of identity) consensus model relies on the partial trust of publishing nodes through their known link to real world identities. Publishing nodes must have their identities proven and verifiable within the blockchain network (e.g., identifying documents which have been verified and notarized and included on the blockchain). The idea is that the publishing node is staking its identity/reputation to publish new blocks. Blockchain network users directly affect a publishing node’s reputation based on the publishing node’s behavior. Publishing nodes can lose reputation by acting in a way that the blockchain network users disagree with, just as they can gain reputation by acting in a manner that the blockchain network users agree with. The lower the reputation, the less likelihood of being able to publish a block. Therefore, it is in the interest of a publishing node to maintain a high reputation. This algorithm only applies to permissioned blockchain networks with high levels of trust. 4.5 Proof of Elapsed Time Consensus Model Within the proof of elapsed time (PoET) consensus model, each publishing node requests a wait time from a secure hardware time source within their computer system. The secure hardware time source will generate a random wait time and return it to the publishing node software. Publishing nodes take the random time they are given and become idle for that duration. Once a publishing node wakes up from the idle state, it creates and publishes a block to the blockchain network, alerting the other nodes of the new block; any publishing node that is still idle will stop waiting, and the entire process starts over. This model requires ensuring that a random time was used, since if the time to wait was not selected at random a malicious publishing node would just wait the minimum amount of time by default to dominate the system. This model also requires ensuring that the publishing node waited the actual time and did not start early. These requirements are being solved by executing 23 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW software in a trusted execution environment found on some computer processors (such as Intel’s Software Guard Extensions 5, or AMD’s Platform Security Processor 6, or ARM’s TrustZone 7). Verified and trusted software can run in these secure execution environments and cannot be altered by outside programs. A publishing node would query software running in this secure environment for a random time and then wait for that time to pass. After waiting the assigned time, the publishing node could request a signed certificate that the publishing node waited the randomly assigned time. The publishing node then publishes the certificate along with the block. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 5 Intel SGX - https://software.intel.com/en-us/sgx 6 AMD Secure Technology - https://www.amd.com/en/technologies/security 7 ARM TrustZone - https://www.arm.com/products/silicon-ip-security 24 NISTIR 8202 4.6 BLOCKCHAIN TECHNOLOGY OVERVIEW Consensus Comparison Matrix This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Name Goals Advantages Disadvantages Domains Implementations Proof of work (PoW) To provide a barrier to publishing blocks in the form of a computationally difficult puzzle to solve to enable transactions between untrusted participants. Difficult to perform denial of service by flooding network with bad blocks. Computationally intensive (by design), power consumption, hardware arms race. Permissionless cryptocurrencies Bitcoin, Ethereum, many more Open to anyone with hardware to solve the puzzle. Potential for 51 % attack by obtaining enough computational power. To enable a less computationally intensive barrier to publishing blocks, but still enable transactions between untrusted participants. Less computationally intensive than PoW. Stakeholders control the system. Permissionless cryptocurrencies Ethereum Casper, Krypton Open to anyone who wishes to stake cryptocurrencies. Nothing to prevent formation of a pool of stakeholders to create a centralized power. Stakeholders control the system. Potential for 51 % attack by obtaining enough financial power. To enable a more efficient consensus model through a ‘liquid democracy’ where participants vote (using cryptographically signed messages) to elect and revoke the rights of delegates to validate and secure the blockchain. Elected delegates are economically incentivized to remain honest Less node diversity than PoW or pure PoS consensus implementations Permissionless cryptocurrencies Bitshares, Steem, Cardano, EOS More computationally efficient than PoW Greater security risk for node compromise due to constrained set of operating nodes Permissioned Systems Proof of stake (PoS) Delegated PoS As all delegates are ‘known’ there may an incentive for block producers to collude and accept bribes, compromising the security of the system 25 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Name Goals Advantages Disadvantages Domains Implementations Round Robin Provide a system for publishing blocks amongst approved/trusted publishing nodes Low computational power. Requires large amount of trust amongst publishing nodes. Permissioned Systems MultiChain To create a centralized consensus process to minimize block creation and confirmation rate Fast confirmation time Relies on the assumption that the current validating node has not been compromised Permissioned Systems, Hybrid (sidechain) Systems Ethereum Kovan testnet, POA Chain, various permissioned systems using Parity To enable a more economic consensus model for blockchain networks, at the expense of deeper security guarantees associated with PoW. Less computationally expensive than PoW Permissioned Networks Hyperledger Sawtooth Proof of Authority/Identity Proof of Elapsed Time (PoET) Straightforward to understand. Allows for dynamic block production rates Leads to centralized points of failure Can be used in sidechains to blockchain networks which utilize another consensus model The reputation of a given node is subject to potential for high tail-risk as it could be compromised at any time. Hardware requirement to obtain time. Assumes the hardware clock used to derive time is not compromised Given speed-of-late latency limits, true time synchronicity is essentially impossible in distributed systems [13] 26 NISTIR 8202 4.7 BLOCKCHAIN TECHNOLOGY OVERVIEW Ledger Conflicts and Resolutions As discussed previously, for some blockchain networks it is possible that multiple blocks will be published at approximately the same time. This can cause differing versions of a blockchain to exist at any given moment; these must be resolved quickly to have consistency in the blockchain network. In this section, we discuss how these situations are generally handled. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 With any distributed network, some systems within the network will be behind on information or have alternative information. This depends on network latency between nodes and the proximity of groups of nodes. Permissionless blockchain networks are more prone to have conflicts due to their openness and number of competing publishing nodes. A major part of agreeing on the state of the blockchain network (coming to consensus) is resolving conflicting data. For example: • • • node_A creates block_n(A)with transactions #1, 2 and 3. node_A distributes it to some nodes. node_B creates block_n(B)with transactions #1, 2 and 4. node_B distributes it to some nodes. There is a conflict. o block_n will not be the same across the network.  block_n(A) contains transaction #3, but not transaction #4.  block_n(B) contains transaction #4, but not transaction #3. Conflicts temporarily generate different versions of the blockchain, which is depicted in Figure 4. These differing versions are not “wrong”; rather, they were created with the information each node had available. The competing blocks will likely contain different transactions, so those with block_n(A) may see transfers of digital assets not present in block_n(B). If the blockchain network deals with cryptocurrency, then a situation may occur where some cryptocurrency may both be spent and unspent, depending on which version of the blockchain is being viewed. Figure 4: Ledger in Conflict Conflicts are usually quickly resolved. Most blockchain networks will wait until the next block is published and use that chain as the “official” blockchain, thus adopting the “longer blockchain”. As in Figure 5, the blockchain containing block_n(B) becomes the “official” chain, as it got 27 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW the next valid block. Any transaction that was present in block_n(A), the orphaned block, but not present in the block_n(B) chain, is returned to the pending transaction pool (which is where all transactions which have not been included within a block reside). Note that this set of pending transactions is maintained locally at each node as there is no central server in the architecture. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Figure 5: The chain with block_n(B) adds the next block, the chain with block_n(A) is now orphaned Due to the possibility of blocks being overwritten, a transaction is not usually accepted as confirmed until several additional blocks have been created on top of the block containing the relevant transaction. The acceptance of a block is often probabilistic rather than deterministic since blocks can be superseded. The more blocks that have been built on top of a published block, the more likely it is that the initial block will not be overwritten. Hypothetically, a node in a proof of work blockchain network with enormous amounts of computing power could start at the genesis block and create a longer chain than the currently existing chain, thereby wiping out the entire blockchain history. This does not happen in practice due to the prohibitively large amount of resources that this would require. Also, some blockchain implementations lock specific older blocks within the blockchain software by creating checkpoints to ensure that this can never happen. 28 NISTIR 8202 5 BLOCKCHAIN TECHNOLOGY OVERVIEW Forking This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 Performing changes and updating technology can be difficult at the best of times. For permissionless blockchain networks which are comprised of many users, distributed around the world, and governed by the consensus of the users, it becomes extremely difficult. Changes to a blockchain network’s protocol and data structures are called forks. They can be divided into two categories: soft forks and hard forks. For a soft fork, these changes are backwards compatible with nodes that have not been updated. For a hard fork, these changes are not backwards compatible because the nodes that have not been updated will reject the blocks following the changes. This can lead to a split in the blockchain network creating multiple versions of the same blockchain. Permissioned blockchain networks, due to the publishing nodes and users being known, can mitigate the issues of forking by requiring software updates. Note that the term fork is also used by some blockchain networks to describe temporary ledger conflicts (e.g., two or more blocks within the blockchain network with the same block number) as described in Section 4.7. While this is a fork in the ledger, it is temporary and does not stem from a software change. 5.1 Soft Forks A soft fork is a change to a blockchain implementation that is backwards compatible. Nonupdated nodes can continue to transact with updated nodes. If no (or very few) nodes upgrade, then the updated rules will not be followed. An example of a soft fork occurred on Bitcoin when a new rule was added to support escrow 8 and time-locked refunds. In 2014, a proposal was made to repurpose an operation code that performed no operation (OP_NOP2) to CHECKLOCKTIMEVERIFY, which allows a transaction output to be made spendable at a point in the future [14]. For nodes that implement this change, the node software will perform this new operation, but for nodes that do not support the change, the transaction is still valid, and execution will continue as if a NOP 9 had been executed. A fictional example of a soft fork would be if a blockchain decided to reduce the size of blocks (for example from 1.0 MB to 0.5 MB). Updated nodes would adjust the block size and continue to transact as normal; non-updated nodes would see these blocks as valid – since the change made does not violate their rules (i.e., the block size is under their maximum allowed). However, if a non-updated node were to create a block with a size greater than 0.5 MB, updated nodes would reject them as invalid. 5.2 Hard Forks A hard fork is a change to a blockchain implementation that is not backwards compatible. At a 8 Funds placed into a third party to be disseminated based on conditions (via multi-signature transactions) 9 NOP meaning No Operation 29 NISTIR 8202 BLOCKCHAIN TECHNOLOGY OVERVIEW given point in time (usually at a specific block number), all publishing nodes will need to switch to using the updated protocol. Additionally, all nodes will need to upgrade to the new protocol so that they do not reject the newly formatted blocks. Non-updated nodes cannot continue to transact on the updated blockchain because they are programmed to reject any block that does not follow their version of the block specification. Publishing nodes that do not update will continue to publish blocks using the old format. User nodes that have not updated will reject the newly formatted blocks and only accept blocks with the old format. This results in two versions of the blockchain existing simultaneously. Note that users on different hard fork versions cannot interact with one another. It is important to note that while most hard forks are intentional, software errors may produce unintentional hard forks. This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8202 A well-known example of a hard fork is from Ethereum. In 2016, a smart contract was constructed on Ethereum called the Decentralized Autonomous Organization (DAO). Due to flaws in how the smart contract was constructed, an attacker extracted Ether, the cryptocurrency used by Ethereum, resulting in the theft of $50 million [15]. A hard fork proposal was voted on by Ether holders, and the clear majority of users agreed to hard fork and create a new version of the blockchain, without the flaw, and that also returned the stolen funds. With cryptocurrencies, if there is a hard fork and the blockchain splits then users will have independent currency on both forks (having double the number of coins in total). If all the activity moves to the new chain, the old one may eventually not be used since the two chains are not compatible (they will be independent currency systems). In the case of the Ethereum hard fork, the clear majority of support moved to the new fork, the old fork was renamed Ethereum Classic and continued operating. 5.3 Cryptographic Changes and Forks If flaws are found in the cryptographic technologies within a blockchain network, the only solution may be to create a hard fork, depending on the significance of the flaw. For example, if a flaw was found in the underlying algorithms, there could be a fork requiring all future clients to use a stronger algorithm. Switching to a new hashing algorithm could pose a significant practical problem because it could invalidate all existing specialized mining hardware. Hypothetically, if SHA-256 were discovered to have a flaw, blockchain networks that utilize SHA-256 would need a hard fork to migrate to a new hash algorithm. The block that switched over to the new hash algorithm would “lock” all previous blocks into SHA-256 (for verification), and all new blocks would need to utilize the new ...
Purchase answer to see full attachment