This udpates the blockchain module to use the lastest module major
versions.
While here, it also corrects a few typos and updates some test function
names to more accurately reflect their purpose.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/blockchain/stake/v2@v2.0.1
- github.com/decred/dcrd/chaincfg/v2@v2.2.0
- github.com/decred/dcrd/database/v2@v2.0.0
- github.com/decred/dcrd/dcrutil/v2@v2.0.0
- github.com/decred/dcrd/txscript/v2@v2.0.0
This modifies the majority of the tests that make use of chain
parameters and the RPC tests to use the resurrected regression test
network.
It also bumps the affected module versions as follows:
- github.com/decred/dcrd/txscript@v1.0.2
- github.com/decred/dcrd/blockchain/stake@v1.0.3
- github.com/decred/dcrd/mempool@v1.0.2
The blockchain and dcrutil modules are also affected, but since their
version has already been bumped since their last release tags, they are
not bumped again.
This refactors and simplifies the code in blockchain to use the new more
efficient chain views.
An overview of the logic changes are as follows:
- Remove inMainChain from block nodes since that can now be efficiently
determined by using the chain view
- Track the best chain via a chain view instead of a single block node
- Use the tip of the best chain view everywhere bestNode was used
- Update chain view tip instead of updating best node
- Remove height map and associated lock in favor of chain view
- Use chain view NodeByHeight everywhere height map was used
- Change reorg logic to use more efficient chain view fork finding logic
- Change block locator code over to use more efficient chain view logic
- Remove now unused block-index-based block locator code
- Move BlockLocator definition to chain.go
- Move BlockLocatorFromHash and LatestBlockLocator to chain.go
- Update both to use more efficient chain view logic
- Rework several functions to use chain view for main chain detection
- fetchMainChainBlockByNode
- BlockByHeight
- MainChainHasBlock
- findPreviousCheckpoint
- IsCheckpointCandidate
This reworks the block index code such that it loads all of the headers
in the main chain at startup and constructs the full block index
accordingly.
Since the full index from the current best tip all the way back to the
genesis block is now guaranteed to be in memory, this also removes all
code related to dynamically loading the nodes and updates some of the
logic to take advantage of the fact traversing the block index can no
longer potentially fail. There are also many more optimizations and
simplifications that can be made in the future as a result of this.
Due to removing all of the extra overhead of tracking the dynamic state,
and ensuring the block node structs are aligned to eliminate extra
padding, the end result of a fully populated block index now takes quite
a bit less memory than the previous dynamically loaded version.
It also speeds up the initial startup process by roughly 2x since it is
faster to bulk load the nodes in order as opposed to dynamically loading
only the nodes near the tip in backwards order.
For example, here is some startup timing information before and after
this commit on a node that contains roughly 238,000 blocks:
7200 RPM HDD:
-------------
Startup time before this commit: ~7.71s
Startup time after this commit: ~3.47s
SSD:
----
Startup time before this commit: ~6.34s
Startup time after this commit: ~3.51s
Some additional benefits are:
- Since every block node is in memory, the code which reconstructs
headers from block nodes means that all headers can always be served
from memory which will be important since the network will be moving
to header-based semantics
- Several of the error paths can be removed since they are no longer
necessary
- It is no longer expensive to calculate CSV sequence locks or median
times of blocks way in the past
- It is much less expensive to calculate the initial states for the
various intervals such as the stake and voter version
- It will be possible to create much more efficient iteration and
simplified views of the overall index
An overview of the logic changes are as follows:
- Move AncestorNode from blockIndex to blockNode and greatly simplify
since it no longer has to deal with the possibility of dynamically
loading nodes and related failures
- Replace nodeAtHeightFromTopNode from BlockChain with RelativeAncestor
on blockNode and define it in terms of AncestorNode
- Move CalcPastMedianTime from blockIndex to blockNode and remove no
longer necessary test for nil
- Remove findNode and replace all of its uses with direct queries of the
block index
- Remove blockExists and replace all of its uses with direct queries of
the block index
- Remove all functions and fields related to dynamically loading nodes
- children and parentHash fields from blockNode
- depNodes from blockIndex
- loadBlockNode from blockIndex
- PrevNodeFromBlock from blockIndex
- {p,P}revNodeFromNode from blockIndex
- RemoveNode
- Replace all instances of iterating backwards through nodes to directly
access the parent now that nodes don't potentially need to be
dynamically loaded
- Introduce a lookupNode function on blockIndex which allows the
initialization code to locklessly query the index
- No longer take the chain lock when only access to the block index,
which has its own lock, is needed
- Removed the error paths from several functions that can no longer fail
- getReorganizeNodes
- findPrevTestNetDifficulty
- sumPurchasedTickets
- findStakeVersionPriorNode
- Removed all error paths related to node iteration that can no longer
fail
- Modify FetchUtxoView to return an empty view for the genesis block
This refactors the block index logic into a separate struct and
introduces an individual lock for it so it can be queried independent of
the chain lock.
It also modifies the `newBlockNode` function to accept nil for the
ticket spend information parameter and updates all of the test code that
doesn't require it to use nil.
This modifies the block node structure to include only the specifically
used fields, some of which in a more compact format, as opposed to
copying the entire header and updates all code and tests accordingly.
Not only is this a more efficient approach that helps pave the way for
future optimizations, it is also consistent with the upstream code which
helps minimize the differences to facilitate easier syncs due to less
merge conflicts.
In particular, since the merkle and stake roots, number of revocations,
size, nonce, and extradata fields aren't used currently, they are no
longer copied into the block node. Also, the block node already had a
height field, which is also in the header, so this change also removes
that duplication.
Another change is that the block node now stores the timestamp as an
int64 unix-style timestamp which is only 8 bytes versus the old
timestamp that was in the header which is a time.Time and thus 24 bytes.
It should be noted that future optimizations will very likely end up
adding most of the omitted header fields back to the block node as
individual fields so the headers can be efficiently reconstructed from
memory, however, these changes are still beneficial due to the ability
to decouple the block node storage format from the header struct which
allows more compact representations and reording of the fields for
optimal struct packing.
Ultimately, the need for the parent hash can also be removed, which will
save an additional 32 bytes which would not be possible without this
decoupling.
This merge commit adds the following code from the
github.com/decred/dcrutil package into a new
github.com/decred/dcrd/dcrutil package:
* Address handling
* Amount type
* AppDataDir func
* bitflags functions
* Block wrapper type
* Hash160 func
* Tx wrapper type
* WIF type
as well as all tests for this code.
The old github.com/decred/dcrutil/hdkeychain package has also been
merged and moved to github.com/decred/dcrd/dcrutil/hdkeychain.
dcrd packages have been updated to use the new packages and the dep
files have been updated for this change.
This repurposes the sequence number of version 2 transaction inputs to
provide consensus-enforced relative lock-time semantics so that a
transaction can require inputs to have a specified relative age before
they are allowed to be included in a block. Each relative time lock can
either specify a relative number of seconds (with a granularity of 512
seconds and a maximum value of 33,553,920) or a specific number of
blocks (max 65535).
The number of seconds is calculated relative to the past median time of
the block before the one that contains the referenced output. This is
done because, due to other changes that will also be included in the
same agenda vote, said time will become the earliest possible time the
block that contains the referenced output could have been (technically
it will be one second after that, but that complexity is ignored since
there is already a granularity involved anyways).
It is also possible to disable the behavior by setting bit 31 of the
sequence number (which all transactions currently do by default since
they are set to the max).
In order for the transaction to be permitted to the mempool, relayed,
considered for inclusion into block templates, and allowed into a block,
the specified relative time locks for all of its inputs must be
satisfied.
This only implements the required logic and tests to enforce the new
behavior. Code to enforce the new behavior when considering candidate
transactions for acceptance to the mempool, relaying, and inclusion into
block templates will be added in a separate commit.
A consensus vote is required in order to reject blocks which contain
transactions that violate the new rules at a consensus level. Code to
selectively enable consensus enforcement based on the result of an
agenda vote will be added in a separate commit.
In order to accomplish this new behavior, the concept of a sequence lock
is introduced which allows the minimum possible time and height at which
a transaction can be included into a block to be calculated from all
inputs with non-disabled relative time locks, and functions to calculate
and evaluate the sequence lock are added.
The following is an overview of the changes:
- Introduce a new struct named SequenceLock to represent the previously
described sequence lock
- Define new constants related to sequence numbers named
SequenceLockTimeDisabled, SequenceLockTimeIsSeconds,
SequenceLockTimeMask, and SequenceLockTimeGranularity
- Add a new function named calcSequenceLock to calculate the sequence
lock for a given transaction
- Add a new function named SequenceLockActive to determine if a given
sequence lock is satisfied for a given block height and past median
time
- Add a convenience function named LockTimeToSequence which can be used
to convert a relative lock time to a sequence number
- Add a comprehensive set of tests for all of the new funcs
