This updates the following modules to use the secp256k1/v2 module:
- blockchain
- chaincfg/v2
- dcrutil/v2
- hdkeychain/v2
- mempool/v3
- txscript/v2
- main
The hdkeychain/v3 and txscript/v2 modules both use types from secp256k1
in their public API.
Consequently, in order avoid forcing them to bump their major versions,
secp256k1/v1.0.3 was released with the types redefined in terms of the
secp256k1/v2 module so callers still using v1 of the module that are not
ready to upgrade to the v2 module yet can interoperate by updating to
the latest patch version.
This modifies the order in which the subsidy cache is created when not
provided by a caller to happen before the blockchain instance is created
to be more consistent.
This refactors the best chain state and block index loading code into
separate functions so they are available to upcoming database update
code to build version 2 gcs filters.
Currently, the filters provide two different serialization formats per
version. The first is the raw filter bytes without the number of items
in its data set and is implemented by the Bytes and FromBytesV1
functions. The second includes that information and is implemented by
the NBytes and FromNBytesV1 functions.
In practice, the ability to serialize the filter independently from the
number of items in its data set is not very useful since that
information is required to be able to query the filter and, unlike the
other parameters which are fixed (e.g. false positive rate and key), the
number of items varies per filter. For this reason, all usage in
practice calls NBytes and FromNBytesV1.
Consequently, this simplifies the API for working with filters by
standardizing on a single serialization format per filter version which
includes the number of items in its data set.
In order to accomplish this, the current Bytes and FromBytesV1 functions
are removed and the NBytes and FromNBytesV1 functions are renamed to
take their place.
This also updates all tests and callers in the repo accordingly.
This refactors the primary gcs filter logic into an internal struct with
a version parameter in in order to pave the way for supporting v2
filters which will have a different serialization that makes them
incompatible with v1 filters while still retaining the ability to work
with v1 filters in the interim.
The exported type is renamed to FilterV1 and the new internal struct is
embedded so its methods are externally available.
The tests and all callers in the repo have been updated accordingly.
This adds support for empty filters versus being an error along with a
full set of tests to ensure the empty filter works as intended.
It is part of the onging process to cleanup and improve the gcs module
to the quality level required by consensus code for ultimate inclusion
in header commitments.
This removes the unused and undesired FromPBytes and FromNPBytes
functions and associated tests from the gcs module in preparation for
upcoming changes aimed to support new version filters for use
in header commitments.
Since these changes, and several planned upcoming ones, constitute
breaking pubic API changes, this bumps the major version of the gcs
module, adds a replacement for gcs/v2 to the main module and updates all
other modules to make use of it.
It also bumps the rpcclient module to v5 since it makes use of the
gcs.Filter type in its API, adds a replacement for rpcclient/v5 to the
main module and updates all other modules to make use of it.
Note that this also marks the start of a new approach towards handling
module versioning between release cycles to reduce the maintenance
burden.
The new approach is as follows.
Whenever a new breaking change to a module's API is introduced, the
following will happen:
- Bump the major version in the go.mod of the affected module if not
already done since the last release tag
- Add a replacement to the go.mod in the main module if not already
done since the last release tag
- Update all imports in the repo to use the new major version as
necessary
- Make necessary modifications to allow all other modules to use the
new version in the same commit
- Repeat the process for any other modules the require a new major as a
result of consuming the new major(s)
Finally, once the repo is frozen for software release, all modules will
be tagged in dependency order to stabilize them and all module
replacements will be removed in order to ensure releases are only using
fully tagged and released code.
This modifies the blockchain Config to allow the caller to optionally
specify the subsidy cache to use rather than creating it internally and
forcing callers to query for it later.
This removes the deprecated functions along with the additional code and
tests which are no longer necessary as a result of removing the
functions.
The following is a list of exported types and functions removed along
with what they were replaced by when applicable:
- DisableLog -> UseLogger(slog.Disabled)
- BlockOneCoinbasePaysTokens -> no alternative, internal only
- IsCoinBase -> standalone.IsCoinBaseTx
- IsCoinBaseTx -> standalone.IsCoinBaseTx
- BuildMerkleTreeStore -> standalone.CalcTxTreeMerkleRoot
- BuildMsgTxMerkleTreeStore -> standalone.CalcTxTreeMerkleRoot
- SubsidyCache -> standalone.SubsidyCache
- NewSubsidyCache -> standalone.NewSubsidyCache
- CalcBlockWorkSubsidy -> standalone.SubsidyCache.CalcWorkSubsidy
- CalcStakeVoteSubsidy -> standalone.SubsidyCache.CalcStakeVoteSubsidy
- CalcBlockTaxSubsidy -> standalone.SubsidyCache.CalcTreasurySubsidy
- CoinbasePaysTax -> no alternative, internal only
- CalculateAddedSubsidy -> no alternative, internal only
- BestPrevHash -> BestSnapshot.PrevHash
- TotalSubsidy -> BestSnapshot.TotalSubsidy
- FetchSubsidyCache -> standalone.NewSubsidyCache passed to chain
- HashToBig -> standalone.HashToBig
- CompactToBig -> standalone.CompactToBig
- BigToCompact -> standalone.BigToCompact
- CalcWork -> standalone.CalcWork
- CheckProofOfWork -> standalone.CheckProofOfWork
This udpates the blockchain module to use the lastest module major
versions.
While here, it also corrects a few typos and updates some test function
names to more accurately reflect their purpose.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/blockchain/stake/v2@v2.0.1
- github.com/decred/dcrd/chaincfg/v2@v2.2.0
- github.com/decred/dcrd/database/v2@v2.0.0
- github.com/decred/dcrd/dcrutil/v2@v2.0.0
- github.com/decred/dcrd/txscript/v2@v2.0.0
This updates the blockchain dependencies and serves as a base for
blockchain/v1.2.0.
The updated direct dependencies in this commit are as follows:
- github.com/decred/dcrd/blockchain/stake@v1.2.1
- github.com/decred/dcrd/chaincfg@v1.5.2
- github.com/decred/dcrd/chaincfg/chainhash@v1.0.2
- github.com/decred/dcrd/database@v1.1.0
- github.com/decred/dcrd/dcrutil@v1.4.0
- github.com/decred/dcrd/gcs@v1.1.0
The full list of updated direct dependencies since the previous
blockchain/v1.1.1 release are as follows:
- github.com/decred/dcrd/blockchain/stake@v1.2.1
- github.com/decred/dcrd/blockchain/standalone@v1.0.0
- github.com/decred/dcrd/chaincfg@v1.5.2
- github.com/decred/dcrd/chaincfg/chainhash@v1.0.2
- github.com/decred/dcrd/database@v1.1.0
- github.com/decred/dcrd/dcrec@v1.0.0
- github.com/decred/dcrd/dcrec/secp256k1@v1.0.2
- github.com/decred/dcrd/dcrutil@v1.4.0
- github.com/decred/dcrd/gcs@v1.1.0
- github.com/decred/dcrd/txscript v1.1.0
This updates blockchain to make use of the new work functions in the
standalone module.
Finally, it also deprecates the related exported functions in the
blockchain module so they can be removed in the next major version and
rewrites them in terms of the standalone module in the mean time.
This updates blockchain to make use of the new better optimized subsidy
cache and calculation functions in the the standalone module. While
making the updates, it also does basic cleanup on some error messages.
Since the v1 chaincfg.Params does not implement the necessary interface
and moving chaincfg/v2.Params requires a major version bump, a new type
named subsidyParams is introduced which embeds v1 chaincfg.Params and
implements the interface to act as a bridge.
Also, since the SubsidyCache type is returned from blockchain, the type
is aliased and rewritten in terms of the standalone module so callers
still on the v1 module are able to interop.
Finally, this also deprecates the related exported functions in the
blockchain module so they can be removed in the next major version.
This updates blockchain to make use of the coinbase check function in
the standalone module.
It also deprecates the related exported functions in the blockchain
module so they can be removed in the next major version and rewrites
them in terms of the standalone module in the mean time.
This updates blockchain to make use of the optimized merkle root
calculation functions in the standalone module.
It also deprecates the related exported functions in the blockchain
module so they can be removed in the next major version.
Finally, the following direct dependencies are updated as required:
- github.com/decred/dcrd/blockchain/standalone@v1.0.0
This updates the main module to use the latest available minor and patch
versions of all modules and reverts the recent change that incorrectly
removed all of the blake256 references from the various go.sum files.
The following required direct dependencies are bumped as follows:
- github.com/decred/dcrd/blockchain/stake@v1.2.1
- github.com/decred/dcrd/blockchain/stake/v2@v2.0.1
- github.com/decred/dcrd/certgen@v1.1.0
- github.com/decred/dcrd/chaincfg@v1.5.2
- github.com/decred/dcrd/chaincfg/chainhash@v1.0.2
- github.com/decred/dcrd/chaincfg/v2@v2.2.0
- github.com/decred/dcrd/dcrutil/v2@v2.0.0
- github.com/decred/dcrd/gcs@v1.1.0
- github.com/decred/dcrd/hdkeychain/v2@v2.0.1
- github.com/decred/dcrd/txscript/v2@v2.0.0
- github.com/decred/dcrwallet/rpc/jsonrpc/types@v1.2.0
This updates the code to make use of the new crypto/blake256 module instead of
github.com/dchest/blake256.
* change the references in the chaincfg/chainhash and gcs modules
* update chaincfg/chainhash to use the new Sum256 API
* remove references to dchest/blake256 from all go.sum files
There is no reason to perform a bunch of additional checks when there is
no possible way a transaction can be vote due to not having at least the
minimum required number of outputs for it to possibly be one.
The copies the standalone IsCoinBaseTx function from the blockchain
module to the blockchain/standalone module.
It also updates the documentation and includes comprehensive tests.
This adds a subsidy cache with funcs to calculate subsidies for
proof-of-work, votes, and the treasury to the blockchain/standalone
module. Since the existing functions in blockchain are easy to use
improperly and have poor performance under certain access patterns,
rather than copying them over, this provide new implementations of
better optimized functions with less potential cases for misuse.
Note that because subsidy generation is consensus critical, great care
has been taken to ensure the same semantics are retained under the
scenarios which the consensus code invokes the functions, although the
semantics of the functions themselves have changed when compared to
their existing counterparts when called in circumstances that the
consensus code avoids as discussed below.
The existing subsidy code in blockchain assumes that it will never be
called in circumstances which would violate consensus such as with
negative block heights and when there are less than the minimum required
number of votes and consequently produces invalid subsidy values in
those types of cases.
This does not pose a problem for the existing consensus code itself
because it does, in fact, correctly avoid calling the functions in
scenarios that would produce incorrect results. However, since these
functions are exported for external use, forcing consumers to only call
the functions under the (previously undocumented) correct scenarios is
highly error prone.
Consequently, these new functions produce results that match the
consensus behavior even if called under those circumstances which are
currently avoided.
The following is an overview of the changes to the results of the
functions as compared to their existing counterparts:
- Negative block heights now returns 0 subsidy in all cases
- Height 0 now returns 0 subsidy in all cases
- Height 1 now returns the full amount for the PoW subsidy and 0
for the vote and treasury subsidies as compared to reduced values that
the actual consensus rules do not use
- Heights prior to one block before the point at which voting begins now
return 0 vote subsidy as compared to reduced values that the actual
consensus rules ignore at those heights
- Specifying less than the minimum number of votes required for a block
to be considered valid by consensus along with heights at or after the
point voting begins now returns 0 PoW and Treasury subsidies as
compared to further reduced values that the actual consensus rules
ignore due to the block being invalid
A few other improvements include:
- All subsidy calculation funcs are now methods on the cache instance
which means it is no longer possible to corrupt the cache by calling
the functions with different network parameters
- The cache can no longer be made to grow indefinitely
- Note that this is not an exploit in the existing code since the
internal code is very restrictive in how it uses the functions as
previously mentioned
- Improves future scalability by avoiding recalculations on all
intervals after the max potential block subsidy has been fully reduced
to 0
- Improve sparse access efficiency to O(log N) vs O(N)
Finally, it also adds a benchmark, updates the documentation, and includes
comprehensive tests.
The following is a before and after comparison of calculating subsidies
for various sparse intervals:
benchmark old ns/op new ns/op delta
-------------------------------------------------------------------
BenchmarkCalcSubsidyCacheSparse 56766 17243 -69.62%
benchmark old allocs new allocs delta
-------------------------------------------------------------------
BenchmarkCalcSubsidyCacheSparse 6 5 -16.67%
benchmark old bytes new bytes delta
-------------------------------------------------------------------
BenchmarkCalcSubsidyCacheSparse 1163 416 -64.23%
This adds functions to calculate a merkle root to the
blockchain/standalone module. Rather than copying the existing
functions from blockchain, it introduces new optimized functions that
have simpler code and are also generally easier to use. The new
functions simply return the calculated root instead of all of the
intermediate hashes which no known callers were actually using anyway.
It also adds a basic usage example, benchmarks, updates the
documentation and includes comprehensive tests.
As can be seen by the benchmarks below, the new code isn't significantly
faster in terms of execution speed since it is still dominated by the
actual hashing, however, it does significantly reduce the number of
allocations which reduces pressure on the GC, particularly during bursty
situations such as IBD. It also scales much better to larger numbers of
transactions.
The following is a before and after comparison of calculating merkle
roots for various numbers of leaves:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------
BenchmarkCalcMerkleRoot/20 25637 24287 -5.27%
BenchmarkCalcMerkleRoot/1000 1252749 1209130 -3.48%
BenchmarkCalcMerkleRoot/2000 2601754 2343198 -9.94%
BenchmarkCalcMerkleRoot/4000 4873969 4629169 -5.02%
BenchmarkCalcMerkleRoot/8000 9831116 9270095 -5.71%
BenchmarkCalcMerkleRoot/16000 19172545 19067585 -0.55%
BenchmarkCalcMerkleRoot/32000 37980323 35746500 -5.88%
benchmark old allocs new allocs delta
----------------------------------------------------------------
BenchmarkCalcMerkleRoot/20 106 64 -39.62%
BenchmarkCalcMerkleRoot/1000 5006 3004 -39.99%
BenchmarkCalcMerkleRoot/2000 10006 6004 -40.00%
BenchmarkCalcMerkleRoot/4000 20006 12004 -40.00%
BenchmarkCalcMerkleRoot/8000 40006 24004 -40.00%
BenchmarkCalcMerkleRoot/16000 80006 48004 -40.00%
BenchmarkCalcMerkleRoot/32000 160006 96004 -40.00%
benchmark old bytes new bytes delta
----------------------------------------------------------------
BenchmarkCalcMerkleRoot/20 6896 4432 -35.73%
BenchmarkCalcMerkleRoot/1000 320688 208272 -35.05%
BenchmarkCalcMerkleRoot/2000 641072 416272 -35.07%
BenchmarkCalcMerkleRoot/4000 1281840 832272 -35.07%
BenchmarkCalcMerkleRoot/8000 2563376 1664272 -35.07%
BenchmarkCalcMerkleRoot/16000 5126448 3328272 -35.08%
BenchmarkCalcMerkleRoot/32000 10252592 6656273 -35.08%
This implements a new module named blockchain/standalone which aims to
provide several of the standalone functions currently available in
blockchain and ultimately replace them in the next major version.
The primary goal of offering these functions via a separate module is to
reduce the required dependencies to a minimum as compared to the
blockchain module. It will be ideal for applications such as
lightweight clients that need to ensure basic security properties hold
and calculate appropriate vote subsidies and block explorers.
For example, some things an SPV wallet needs to prove are that the block
headers all connect together, that they satisfy the proof of work
requirements, and that a given transaction tree is valid for a given
header.
The new module currently only provides functions related to proof of
work, however, future commits will provide functions for merkle root
calculation, subsidy calculation, and coinbase transaction
identification. It is being done in stages to help ease review since it
is consensus critical code.
Finally, it also includes comprehensive tests, full package
documentation, and basic usage examples.
This modifies the AddrFromSStxPkScrCommitment function to accept a
dcrutil.AddressParams interface instead of a concrete chaincfg.Params to
remove the tight coupling between the two packages at the API boundary
and allow callers to easily provide custom values if desired. # Please
enter the commit message for your changes. Lines starting
This udpates the blockchain/stake module to use v2 of the database module.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/database/v2@v2.0.0
This udpates the blockchain/stake module to use v2 of the txscript module.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/txscript/v2@v2.0.0
This udpates the blockchain/stake module to use v2 of the dcrutil module
and v2 of the chaincfg module since dcrutil/v2 requires it.
It also updates the tests accordingly.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/chaincfg/v2@v2.1.0
- github.com/decred/dcrd/dcrutil/v2@v2.0.0
This introduces a new interface named StakeParams and updates the
functions that currently take a pointer to a chaincfg.Params struct to
accept the interface instead.
This removes the tight coupling between the two packages at the API
boundary and allows callers to easily provide custom values if desired.
It also updates the tests to make use of the interface by adding and
using a mock stake params struct the implements the new interface. The
new struct is defined in terms of a chaincfg.Params for now for
convenience, but the interface design means it is possible to define
them independently in the future as needed.
This betters CalcBlockSubsidy by allowing it to
calculate the requested block subsidy from a cached
subsidy entry regardless of its proximity to the
requested one.
This updates the blockchain/stake dependencies and serves as a base for
blockchain/stake/v1.2.0.
The updated direct dependencies in this commit are as follows:
- github.com/decred/dcrd/database@v1.1.0
- github.com/decred/dcrd/dcrutil@v1.3.0
- github.com/decred/dcrd/txscript v1.1.0
The full list of updated direct dependencies since the previous
blockchain/stake/v1.1.0 release are as follows:
- github.com/decred/dcrd/chaincfg@v1.5.1
- github.com/decred/dcrd/database@v1.1.0
- github.com/decred/dcrd/dcrec@v1.0.0
- github.com/decred/dcrd/dcrec/edwards@v1.0.0
- github.com/decred/dcrd/dcrutil@v1.3.0
- github.com/decred/dcrd/txscript v1.1.0
This refactors the consensus code which extracts the null data from the
coinbase from txscript.ExtractCoinbaseNullData so that it is performed
directly in the validation code where it more properly belongs.
The only reason the extraction was previously done in txscript is
because it was not possible to parse scripts outside of it, but that is
no longer the case now that txscript offers an exported tokenizer for
that purpose.
The extraction code is ever so slightly more efficient now that it no
longer needs to be as generic since it now has direct knowledge of the
conditions that need to be handled.
Great care was taken to ensure the semantics are not changed while
refactoring the code and no additional tests are added in this commit
because all of the conditions and code paths are covered by the tests
recently added to the full block tests.
While here, also perform some related code cleanup in the function and
improve the error messages .
Since the txscript.ExtractCoinbaseNullData is no longer necessary, this
deprecates the function and releated error code and constant so they can
be removed in the next major version of txscript.
Finally, since this relies on the script tokenizer which is not yet in a
released version of the txscript module, bump the requirement to include
an as yet unreleased version of txscript to ensure the next time the
blockchain module is released, it will require a newer version of
txscript to be released first.
This adds two new consensus tests to the full block tests for the
coinbase nulldata height commitment. The first new test ensures
coinbase transactions with the wrong script version are rejected and the
second test ensures data pushes that exceed the maximum allowed number
of bytes for the coinbase nulldata push are rejected.
This updates all of the modules that still rely on v1 of the chaincfg
module to use v1.5.1 which builds correctly with the major API bump
introduced in the edwards v1.0.0 module.
While here, it also updates all references to the v0 edwards and dcrec
modules to their tagged v1 counterparts, tidies all of the modules via
go mod tidy, and removes all unnecessary indirect entries from the mod
files to keep the modules using the versions the dependencies are tested
with.
The primary motivation for this change is that the chaincfg/v2 module
requires edwards/v1 which is not API compatible with edwards/v0.
Consequently, in order for each module to be incrementally updated to
use chaincfg/v2, all of its dependencies must also be able to build with
edwards/v1 which is the case as of chaincfg v1.5.1.
