This breaks the dependency on chaincfg.SigHashOptimization which is no
longer available in v2 of the chaincfg module. The constant is set to
false to ensure the same semantics are kept and an additional comment
has been added regarding the status.
This converts the callback function defined on the internal opcode
struct to accept the opcode and data slice instead of a parsed opcode as
the final step towards removing the parsed opcode struct and associated
supporting code altogether.
It also updates all of the callbacks and tests accordingly and finally
removes the now unused parsedOpcode struct.
This refactors the script engine to store and step through raw scripts
by making using of the new zero-allocation script tokenizer as opposed
to the less efficient method of storing and stepping through parsed
opcodes. It also improves several aspects while refactoring such as
optimizing the disassembly trace, showing all scripts in the trace in
the case of execution failure, and providing additional comments
describing the purpose of each field in the engine.
It should be noted that this is a step towards removing the parsed
opcode struct and associated supporting code altogether, however, in
order to ease the review process, this retains the struct and all
function signatures for opcode execution which make use of an individual
parsed opcode. Those will be updated in future commits.
The following is an overview of the changes:
- Modify internal engine scripts slice to use raw scripts instead of
parsed opcodes
- Introduce a tokenizer to the engine to track the current script
- Remove no longer needed script offset parameter from the engine since
that is tracked by the tokenizer
- Add an opcode index counter for disassembly purposes to the engine
- Update check for valid program counter to only consider the script
index
- Update tests for bad program counter accordingly
- Rework the NewEngine function
- Store the raw scripts
- Setup the initial tokenizer
- Explicitly check against version 0 instead of DefaultScriptVersion
which would break consensus if changed
- Check the scripts parse according to version 0 semantics to retain
current consensus rules
- Improve comments throughout
- Rework the Step function
- Use the tokenizer and raw scripts
- Create a parsed opcode on the fly for now to retain existing
opcode execution function signatures
- Improve comments throughout
- Update the Execute function
- Explicitly check against version 0 instead of DefaultScriptVersion
which would break consensus if changed
- Improve the disassembly tracing in the case of error
- Update the CheckErrorCondition function
- Modify clean stack error message to make sense in all cases
- Improve the comments
- Update the DisasmPC and DisasmScript functions on the engine
- Use the tokenizer
- Optimize construction via the use of strings.Builder
- Modify the subScript function to return the raw script bytes since the
parsed opcodes are no longer stored
- Update the various signature checking opcodes to use the raw opcode
data removal and signature hash calculation functions since the
subscript is now a raw script
- opcodeCheckSig
- opcodeCheckMultiSig
- opcodeCheckSigAlt
This converts the engine's current program counter disasembly to make
use of the standalone disassembly function to remove the dependency on
the parsed opcode struct.
It also updates the tests accordingly.
This converts the checkMinimalDataPush function defined on a parsed
opcode to a standalone function which accepts an opcode and data slice
instead in order to make it more flexible for raw script analysis.
It also updates all callers accordingly.
This converts the isConditional function defined on a parsed opcode to a
standalone function named isOpcodeConditional which accepts an opcode as
a byte instead in order to make it more flexible for raw script
analysis.
It also updates all callers accordingly.
This converts the alwaysIllegal function defined on a parsed opcode to a
standalone function named isOpcodeAlwaysIllegal which accepts an opcode
as a byte instead in order to make it more flexible for raw script
analysis.
It also updates all callers accordingly.
This converts the isDisabled function defined on a parsed opcode to a
standalone function which accepts an opcode as a byte instead in order
to make it more flexible for raw script analysis.
It also updates all callers accordingly.
This converts the DisasmString function to make use of the new
zero-allocation script tokenizer instead of the far less efficient
parseScript thereby significantly optimizing the function.
In order to facilitate this, the opcode disassembly functionality is
split into a separate function called disasmOpcode that accepts the
opcode struct and data independently as opposed to requiring a parsed
opcode. The new function also accepts a pointer to a string builder so
the disassembly can be more efficiently be built.
While here, the comment is modified to explicitly call out the script
version semantics.
The following is a before and after comparison of a large script:
benchmark old ns/op new ns/op delta
----------------------------------------------------------
BenchmarkDisasmString 288729 94157 -67.39%
benchmark old bytes new bytes delta
----------------------------------------------------------
BenchmarkDisasmString 584611 177528 -69.63%
As is already well commented in the code, the sequence number parameter
of the CHECKSEQUENCEVERIFY opcode requires 5 bytes instead of the
standard 4 bytes allowed by math opcodes. This introduces a constant
for the value instead of hardcoding 5 to increase readability and
potentially allow the value to be exported in the future.
As is already well commented in the code, the locktime parameter of the
CHECKLOCKTIMEVERIFY opcode requires 5 bytes instead of the standard 4
bytes allowed by math opcodes. This introduces a constant for the value
instead of hardcoding 5 to increase readability and potentially allow
the value to be exported in the future.
This introduces a new error named ErrCheckSigAltVerify and modifies the
opcodeCheckSigAltVerify handler to use the abstractVerify function along
with the new error. This makes the handling consistent with all other
signature checking verification opcode handlers and ensures the error
both can be programmatically detected as well as be uniquely identified
as compared to a generic verify failure.
This removes the unused curve parameter from the ParseSignature and
ParseDERSignature functions of the secp256k1 package and updates all
callers in the repository accordingly.
This modifies the PeekInt function of the stack to accept a maximum
script number length to mirror PopInt for consistency. It also updates
the two callers CLTV and CSV) which were manually performing the same
task with 5 bytes due to PeekInt enforcing 4-byte script nums to use the
modified version accordingly.
It also adds some stack tests for 5-byte encodings on both PopInt and
PeekInt.
This removes the flag to require minimal encoding when create script
numbers since since all callers now call the function with true due to
the recent removal of the minimal data script verification flag from the
script engine and updates the tests accordingly.
This removes the ScriptVerifyMinimalData flag from the txscript package,
changes the default semantics to always enforce its behavior, and
updates all callers in the repository accordingly.
This change is being made to simplify the script engine code since the
flag has always been active and required by consensus in Decred, so
there is no need to require a flag to conditionally toggle it.
It should be noted that the tests removed from script_tests.json
specifically dealt with ensuring equivalency of different ways to encode
the same numbers when the ScriptVerifyMinimalData flag is not set.
Therefore, they are no longer necessary.
A few tests which dealt with equivalency that did not already have
expected failing counterparts were converted to expected failure.
Also, several of the tests which dealt with ensuring the specific
encoding of numeric opcodes is being used have been converted to use
hashes since the minimal data requirements specifically prevent
alternate ways of pushing the same encoding which is necessary for
directly checking equality of the raw bytes.
Finally, the MINIMALDATA indicator to enable the flag in the test data
has been retained for now in order to isolate the logic changes as much
as possible.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
This cleans up the code for handling the checksig and checkmultisig
opcodes to explicitly call out any semantics that are likely not
obvious, correct some comments, and improve readability.
It also adds several tests to the reference script tests which exercise
the semantics of the check[multi]sig opcodes including both positive and
negative tests.
Finally, it corrects nearly all of the negative tests related to
signature checking of the script tests which were not properly updated
for the differences introduced by Decred so that they fail for the
intended reasons.
The malformed signatures in the tests were very carefully crafted to be
valid except for the very specific condition being tested. The majority
of the negative tests modified and added can be manually verified by
commenting out the relevant checks in the script engine, although a few
of them will pass because they fail for other reasons. In those cases,
prints can be added to ensure the expected failure path is being hit.
This removes the ScriptVerifyStrictEncoding flag from the txscript
package, changes the default semantics to always enforce its behavior
and updates all callers in the repository accordingly.
This change is being made to simplify the script engine code since the
flag has always been active and required by consensus in Decred, so
there is no need to require a flag to conditionally toggle it.
It should be noted that the tests removed from script_valid.json
specifically dealt with ensuring signatures not compliant with DER
encoding did not cause execution to halt early on invalid signatures
when neither of the ScriptVerifyStrictEncoding or
ScriptVerifyDERSignatures flags were set. Therefore, they are no longer
necessary.
For nearly the same reason, the tx test related to the empty pubkey
tx_valid.json was moved to tx_invalid.json. In particular, an empty
pubkey without ScriptVerifyStrictEncoding simply failed the signature
check and continued execution, while the same condition with the flag
halts execution. Thus, without the flag the final NOT in the script
would allow the script to succeed, while it does not under the strict
encoding rules.
Finally, the STRICTENC indicator to enable the flag in the test data has
been retained for now in order to keep the logic changes separate.
This cleans up the code for handling the mod opcode to explicitly call
out its semantics which are likely not otherwise obvious as well as
improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the div opcode including both positive and negative
tests.
This cleans up the code for handling the div opcode to explicitly call
out its semantics which are likely not otherwise obvious as well as
improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the div opcode including both positive and negative
tests.
This cleans up the code for handling the right shift opcode to
explicitly call out its semantics which are likely not otherwise obvious
as well as improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the right shift opcode including both positive and
negative tests.
This cleans up the code for handling the left shift opcode to explicitly
call out its semantics which are likely not otherwise obvious as well as
improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the left shift opcode including both positive and
negative tests.
This cleans up the code for handling the left rotation opcode to
explicitly call out its semantics which are likely not otherwise obvious
as well as improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the left rotation opcode including both positive and
negative tests.
This cleans up the code for handling the right rotation opcode to
explicitly call out its semantics which are likely not otherwise obvious
as well as improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the right rotation opcode including both positive and
negative tests.
This cleans up the code for handling the cat opcode to more closely
match the style used by the rest of the code and improves its test
coverage by adding several tests to the reference script tests which
exercise its semantics including both positive and negative tests.
This cleans up the code for handling the right opcode to explicitly call
out its semantics which are likely not otherwise obvious as well as
improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the right opcode including both positive and negative
tests.
This cleans up the code for handling the left opcode to explicitly call
out its semantics which are likely not otherwise obvious as well as
improve its readability.
It also adds several tests to the reference script tests which exercise
the semantics of the left opcode including both positive and negative
tests.
This cleans up the code for handling the substr opcode to explicitly
call out its semantics which are likely not otherwise obvious as well as
improve its readability.
This modifies the script engine to replace OP_NOP3 with
OP_CHECKSEQUENCEVERIFY and adds a flag to selectively enable its
enforcement.
The new opcode examines the top item on the stack and compares it
against the sequence number of the associated transaction input in order
to allow scripts to conditionally enforce the inclusion of relative time
locks to the transaction.
The following is an overview of the changes:
- Introduce a new flag named ScriptVerifyCheckSequenceVerify to
provide conditional enforcement of the new opcode
- Introduce a constant named OP_CHECKSEQUENCEVERIFY which has the same
value as OP_NOP3 since it is replacing it
- Update opcode to name mappings accordingly
- Abstract the logic that deals with time lock verification since it is
the same for both the new opcode and OP_CHECKLOCKTIMEVERIFY
- Implement the required opcode semantics
- Add tests to ensure the opcode works as expected including when used
both correctly and incorrectly
This reverts the changes related to the CheckSequenceVerify opcode that
were merged from upstream since additional changes are needed and it's
much cleaner to implement all of code related to the sequence locks in
the same PR which will be referenced by the DCP as opposed to being
split up in multiple.
This modifies the script engine to replace OP_UNKNOWN192 with OP_SHA256
along with a flag named ScriptVerifySHA256 to selectively enable its
enforcement.
The new opcode consumes the top item from the data stack, computes its
SHA-256, and pushes the resulting digest back to the data stack.
Since it requires an item on the data stack, execution will terminate
with an error when the stack is empty. This behavior differs from
OP_UNKNOWN192 which does not consume any elements from the data stack
and therefore makes this is hard-forking change when interpreted with
the new semantics due to the ScriptVerifySHA256 flag being set. Code to
selectively enable the opcode based on the result of an agenda vote will
be added in a separate commit.
This also includes tests to ensure the opcode works as expected
including when used both correctly and incorrectly.
Decred updated the semantics of this opcode to use blake256, but did not
rename the opcode to reflect that. This renames the opcode so that it
is clear that a blake256 hash is produced instead of a sha256 hash.
Upstream commit a6bf1d9850.
The merge commit modifies all of the encoded transactions in the test
data to use Decred native format and contains some other minor
modifications necessary to integrate with Decred.
Contains the following commits:
- 711f33450c
- b6b1e55d1e
- Reverted because Travis is already at a more recent version
- bd4e64d1d4
Also, the merge commit contains the necessary decred-specific
alterations, converts all other references to sha to hash to keep with
the spirit of the merged commits, and various other cleanup intended to
bring the code bases more in line with one another.
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing. The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.
In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs. Naturally there are significant automated tests to help
prevent consensus issues as well.
The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles. It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.
The following is an overview of the chain changes:
- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
- Main chain structure and state
- Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
- Provides efficient direct access to the unspent transaction outputs
- Uses a domain specific compression algorithm that understands the
standard transaction scripts in order to significantly compress them
- Removes reliance on the transaction index and paves the way toward
eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
- Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
- Currently using a single chain state lock as it provides a straight
forward and easy to review path forward however this can be improved
with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
chain stats in a concurrent safe fashion
- Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
code
The following is an overview of the btcd changes:
- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
view
- Rework several calls that required the block manager for safe
concurrency to use the chain package directly now that it is
concurrent safe
- Change all calls to obtain the best hash to use the new best state
snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
opposed the hash the txout was found in
- Optimize various RPC handlers:
- Change several of the RPC handlers to use the new chain snapshot
capability to avoid needlessly loading data
- Update several handlers to use new functionality to avoid accessing
the block manager so they are able to return the data without
blocking when the server is busy processing blocks
- Update non-verbose getblock to avoid deserialization and
serialization overhead
- Update getblockheader to request the block height directly from
chain and only load the header
- Update getdifficulty to use the new cached data from chain
- Update getmininginfo to use the new cached data from chain
- Update non-verbose getrawtransaction to avoid deserialization and
serialization overhead
- Update gettxout to use the new utxo store versus loading
full transactions using the transaction index
The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported
NOTE: The transaction index and address index will be reimplemented in
another commit.
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up. In particular,
this updates the code to make use of the new chainhash package.
Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.
The following is an overview of the changes:
- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
- wire.BlockHeader.BlockSha -> BlockHash
- wire.MsgBlock.BlockSha -> BlockHash
- wire.MsgBlock.TxShas -> TxHashes
- wire.MsgTx.TxSha -> TxHash
- blockchain.ShaHashToBig -> HashToBig
- peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
instead
- Update for function name changes in other dependent packages such as
btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
