Upgrading Consensus

Discussion in 'Announcements' started by jy-p, Nov 16, 2016.

  1. 2017/12/15 - Decred v1.1.2 released! → Release Notes  → Downloads
  1. jy-p

    jy-p Sr. Member
    Organizer

    Jan 2, 2016
    133
    340
    Male
    [​IMG]

    Over the next few releases, Decred will enable Proof-of-Stake (“PoS”) miners to vote on proposed consensus changes. This voting will allow Decred users to exercise sovereignty over whether or not to accept hard fork changes, in addition to making other important decisions. Enabling this decision making requires versioning several subsystems: block headers, vote transactions and transaction scripts. Versioning these subsystems allows for reliable and relatively smooth hard forking on an ongoing basis, meaning Decred can continuously upgrade its consensus code provided the PoS miners agree with each of the changes. Further, we have added a series of full-block consensus tests that are automatically run on every pull request to the consensus daemon, dcrd. Before this process can commence, we must implement a soft fork to add and enforce vote transaction versioning, and enforcement will occur as part of the next release, 0.7.0. Decentralizing the larger-scale decision making is the first step of several towards making Decred into a robust and fully operational decentralized autonomous organization (“DAO”).

    Versioning consensus

    The consensus code in any cryptocurrency is the most important and most delicate code in its codebase. Correspondingly, this consensus code is the real “magic” that makes a blockchain possible: without this shared set of rules about what is and is not considered an acceptable block or transaction, a decentralized ledger cannot have a reliable distributed timestamp. As such, the consensus code can comfortably be considered the most interesting part of a cryptocurrency, however, most projects are extremely conservative about modifying consensus code because it is so delicate and typically necessitates a hard fork. Some projects, e.g. Bitcoin, go so far as to avoid hard forks under all circumstances, and instead opt for routine use of soft forks. Based on what we're doing with Decred, we obviously don't agree with soft forks as a means to upgrade consensus, mainly because it breaks the social contract that comes with a blockchain, but that is a whole separate blog entry.

    Instead of treating consensus code as a “delicate snowflake” that is only to be touched if absolutely necessary, we have instead chosen to figure out how to reinforce the snowflake and make it more resilient. Part of this reinforcement process is a system whereby all pull requests to dcrd, the consensus daemon, are subjected to automated full-block consensus tests and will not be merged to master until those tests pass. The full-block tests use a separately-coded test harness that allows for crafting custom blocks on demand to test branch conditions in the consensus code. In some ways, this could be viewed as making a “cast” of the snowflake and ensuring each change to dcrd's code does not break that cast. By having this battery of tests run on consensus code, we can be confident that both existing and new consensus code is stable and will not create unplanned forks.

    Going from one set of consensus rules to another requires signaling via versions in several subsystems. Some of the changes required are pretty obvious, like having a version in the block header, but the others are less obvious, e.g. adding versions to vote transactions and transaction scripts. The block header and vote transaction versions signal that Proof-of-Work (“PoW”) and PoS miners are on the latest version, which means PoW and PoS miners will both have the latest set of consensus rules and votebits enabled. The votebits, part of every vote transaction, encode what issues are currently being voted on by stakeholders. Transaction script versioning is required because you must know the script rules for old pay-to-script-hash (“P2SH”) transactions, which may not be spent until some time in the future. While transaction script versioning is important, the important parts for consensus versioning are the block header and vote transaction versions.

    Soft fork: vote versioning and version enforcement

    Stepwise versioning consistent with the block header for network consensus rules has been added to votes. We will be soft forking to add the vote transaction versioning and begin enforcing a particular version once 95% of both the block headers and votes show the new version in a rolling 144 block window. Blocks and votes with a version less than the latest version will be accepted until 95% of the blocks and votes both (separately) have the latest version, at which point nodes will begin to reject blocks and votes with any version less than the latest one. The vote versioning has been added in the 0.6.0 release and enforcement of the block header and vote versions will occur starting with version 0.7.0. With the soft fork complete, the process of upgrading consensus can begin.

    Upgrading to a new consensus rule set

    Once the versioning soft fork is complete and activated, it means that a supermajority of the users in the Decred network who validate blocks are upgraded to the latest version. By signaling the latest version of consensus, these PoW and PoS miners confirm that they have the latest definition of the votebits and the corresponding conditional logic to implement any consensus changes, assuming a given consensus change is voted in. After a new consensus version is activated, the voting process begins and runs for 8064 blocks, which is approximately 28 days. Over this period, up to 40320 votes are cast on the various issues encoded in the votebits for the current version. Issues receiving 75% or greater ratios of “yea” to “nay” votes over the voting period are considered approved and will go into effect after a lock-in period of 8064 blocks after the end of that voting period. Once the end of the lock-in period is reached, the issues that were approved in the prior voting cycle go into effect, meaning that consensus changes take effect at that point. This lock-in period is intended to allow users of Decred some additional time to upgrade their full nodes before the blockchain hard forks. The choice of parameters for the voting period, lock-in period and approval ratios was intentional and deserves some explanation.
    [​IMG]
    The voting period was chosen to allow for nearly a full ticket pool worth of votes, 40320 tickets, to vote, but not be so long that it would drag for several months. By having the voting period be 8064 blocks, we allow 40320 votes to be cast, which is roughly a full cycling of the ticket pool. Further, 28 days is an acceptable length of time over which to allow voting: it is neither too short nor too long a period. Similarly, the lock-in period was chosen to be 8064 blocks since it is also neither too long nor too short and gives users a reasonable amount of time to upgrade their full nodes. This means that consensus changes take 2-3 months to be encoded in votebits, voted on and activated.

    Choosing a 75% threshold for approval of an issue being voted on is based on considerations for the costs of creating a long-lived fork of Decred. In order to have a properly functioning blockchain, there must be a single chain that has only relatively short-lived forks, or else users cannot determine which transactions and blocks are valid and which are not. Decred's hybrid PoW/PoS consensus requires that blocks must be mined by PoW miners and then voted on by PoS miners, which makes voting on hard forks not only possible but rather stable. If we assume there are 2 forks of Decred, call them F and F', where F' uses a newly approved set of consensus rules and F uses the previous set of consensus rules, we can use the PoS approval threshold, call it P, to derive an expression for the orphan rate of blocks on F, the “old” chain, as a function of P. This function, graphed below, shows that if we require an approval threshold, P, equal to 75%, it means that the orphan rate for blocks on F is approximately 90%.

    [​IMG]

    This orphan rate effectively drives up the cost of PoW mining on fork F by a factor of 10, and this is assuming 100% of the hash power were to stay on fork F. In practice, this means that attempts to create a long-lived fork of Decred by a minority of dissenting stakeholders will have a drastically (10x or greater) increased cost of maintaining the chain. For other PoW-based cryptocurrencies that have a long-lived fork, such as Ethereum, the only financial barrier to a fork is a short term reduction in the speed with which blocks are mined. By having a high financial barrier to maintaining a long-lived fork, Decred ensures that spirited dissent does not lead to divisive forking of its blockchain.

    While many cryptocurrency projects are content to decentralize only select components of their system, Decred will continue pursuing decentralization as a process to build a more robust and distributed ecosystem. By formally including stakeholders in the decision making process on an ongoing basis, Decred is creating a truly multipolar system where users are governed only with their explicit cryptographic consent. We will be announcing a longer term roadmap for this process in the near future since making decisions on what constitutes consensus is only one step in the process. A major milestone in this process will be a proper DAO, built without the pitfalls associated with attempting to encode a DAO into a single monolithic smart contract.
     
  2. 418Sec

    418Sec Member
    Advocate (Twitter)

    Jan 3, 2016
    122
    65
    Male
    I guess no coin has implemented voting so prominently from the start... Its usually thought about when things are going wrong.
    This looks like a great step!
    Let's show Ethereum how DAO is done! :D
    Keep up the good work guys! :)
     
    chappjc and jy-p like this.
  3. jy-p

    jy-p Sr. Member
    Organizer

    Jan 2, 2016
    133
    340
    Male
    Rome wasn't built in a day, and neither was a DAO. :)
     
    418Sec likes this.
  4. raedah

    raedah Jr. Member

    Mar 6, 2016
    55
    36
    I like seeing everything moving foward.
     
    Tivra likes this.
  5. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    Thanks for the very insightful write up! One idea came to mind, and it's in regards to complete decentralization. The way I see it, the organization or authority who writes the issues into code can still have an influence on the outcome of a voting process. For example, let's say as a developer, I am creating the code that represents the issue at hand and the action that will be taken upon a successful 75% majority vote after the 28 day/8064 block period. Let's also say that the majority of Decred PoS miners do not keep up with these issues (most of them don't even seem to keep up with the forum), and they leave their votebit set to 1. Case in point - seven of my current dcrd peers are still running 0.5.0 or older, the oldest version being 0.0.10.

    If I'm a Decred developer and I want this issue to be addressed in my favor, I will put the action to be taken on the approve side because the majority of "set-and-forget-ers" are not likely to change their voting behavior. On the other hand, if I am personally unfavorable of the issue/action to be taken, I will put the issue on the deny side, where not as many votes are to be expected.

    Is this a correct assumption of how the voting process will be implemented? If so, how can this behavior be restricted?
     
    chappjc, jy-p and shoshin like this.
  6. davecgh

    davecgh Hero Member
    Developer Organizer

    Dec 31, 2015
    642
    788
    Male
    United States
    I think there are two issues at play here. The first is regarding potential bias based upon how the vote is implemented and the second is stakeholder complacency regarding voting and upgrading.

    For the first issue regarding potential bias, I have previously suggested the best way to handle this situation is that the rules of any votes must follow the same rules as those currently used for the yes/no voting on the validity of the previous block. In particular, this means that changes require a majority yes to be accepted as opposed to the reverse where something takes effect if not explicitly vetoed. There are several reasons for this, but the most notable is that it shouldn't be possible to effectively "sneak" in a change due to lack of voting as that very much would go against the spirit of the system. All changes should only take place if there is majority agreement. I would expect the Decred constitution to require all proposed changes to require majority yes votes to take effect. Failure to uphold the constitution would be immediate grounds for any blocks created with that version to be vetoed thereby disincentivizing their creation. This would effectively prevent any attempts to bias the results by choosing the "deny side".

    As far as the second issue of complacency, as @jy-p's article calls out, the voting can't commence at all until a majority of PoW and PoS miners (new version blocks and new version votes, respectively) have been reached. At that point, both majority PoW is prepared to enforce the new rules should they become active, and PoS understands what it is voting on. Once those majority points have been reached, old version blocks will no longer accepted, so miners will have no choice but to upgrade or they will be forked from the network and not gain anything for their efforts. Stake voters would have a little more latitude in that older version would be ignored in terms of the final vote tallying, but there still has to be a majority threshold of voters for the vote to pass (and due to the previous rule of not starting voting until the majority of PoS miners have signaled they've upgraded, this is effectively guaranteed). Further, voters on the latest version would have the option to "abstain" from voting so they are not forced into voting yes or no. That also helps combat any bias since abstaining is effectively saying "I don't understand enough about the implications of the change or I don't really care" without biasing the results.
     
    Dyrk and jy-p like this.
  7. jy-p

    jy-p Sr. Member
    Organizer

    Jan 2, 2016
    133
    340
    Male
    You've made 2 astute observations here, and we've considered and discussed both of them, despite them not making it into the blog entry. The entry was pretty long at ~1600 words, and I didn't want it to go on for too long.

    (1) You are absolutely correct to note that the ability to choose what issues are voted on in the votebits is a point of centralization. I refer to a larger roadmap of decentralization as a process and it will include issues such as this. The first step in removing ourselves as a point of centralization is to allow major decisions to be made by the stakeholders, and the next logical step would be to decentralize an open proposal process that ultimately feeds into the votebits. We already have some ideas on how to execute this step, and I will be commenting on the remaining steps in the next couple weeks.

    (2) Regarding selection bias and defaults, @davecgh and others have suggested that each votebit should really be 2 bits, encoding 4 states ("unset", "abstain", "yea", "nay"), and that the default state for any votebit is "abstain". The point of having these 4 states is that some stakeholders will vote "unset", meaning their voting wallet doesn't understand the votebits or is otherwise unupgraded, and properly upgraded stakeholders will vote "abstain", "yea" or "nay". By defaulting to "abstain", it prevents bias to the "yea" or "nay" counts on a given issue, which requires stakeholders to actively cast their votes. While we recognize some stakeholders will leave their votebits in the "abstain" state, it is our hope to have a high rate of participation with the voting so that our collective decision making is properly representative of the stakeholders.
     
    raedah, Dyrk, David and 1 other person like this.
  8. David

    David Sr. Member

    Jan 22, 2016
    359
    206
    Male
    USA
    Thanks @davecgh and @jy-p for your comments on the matter. I think you guys are doing an excellent job at working towards a fully decentralized platform. I understand that we can't do anything about apathetic PoS miners and that the possibility of exploiting the votebit consensus will be curbed to an extent... but that leads me to my next point.

    I think it is generally known that c0 has been a large stakeholder in PoS. This means they also have the potential to sway the majority vote. What role will c0 play when voting on these issues?
     
    Tivra likes this.
  9. Tivra

    Tivra Member
    Advocate (BitcoinTalk)

    Dec 30, 2015
    120
    60
    Europe
    I also wondered about the same, not about c0 specifically, but about large stakeholders in general.
     
  10. jy-p

    jy-p Sr. Member
    Organizer

    Jan 2, 2016
    133
    340
    Male
    C0 currently has 13-15% of the tickets at any given time and will use them to support whatever we feel is best for Decred. Over time, that stake will decrease as a percentage of tickets, and it is already well in the minority, which is more than can be said for most (possibly all) other cryptocurrencies that have voting infrastructure. We believe a multipolar system of governance will lead to better decisions, and that may mean that C0 is on the losing side of a given issue.

    EDIT: reply had pulled in my previous post, removed it
     
  11. davecgh

    davecgh Hero Member
    Developer Organizer

    Dec 31, 2015
    642
    788
    Male
    United States
    In addition to what @jy-p has already pointed out, I think it bears repeating that one of the goals of the initial airdrop was to ensure that C0 never had more than 50% of the coins from day 1. As was publicly announced when Decred was first launching, C0 also agreed to throttle its staking to ensure it did not have more than its proportion of coins in stake pool percentage. This combination ensured that C0 never had unilateral decision making power via stake votes, even from the initial launch.

    Given the current percentage of tickets C0 holds per the numbers above, it can also be seen that the dcrstats stake pool alone almost has as much voting power as C0 does. When you start to add all of the other stake pools and solo stake miners, it's pretty clear that C0 does not have anywhere near the voting power that would be needed to unilaterally sway the majority vote.
     
    chappjc, jy-p and Johnshpon3 like this.

Share This Page