flip/dcrd
1
0
Fork 0
mirror of https://github.com/FlipsideCrypto/dcrd.git synced 2026-02-06 10:56:47 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

blockchain: Reverse utxo set semantics.

Browse Source 
This modifies the way the unspent transaction output set is handled to
reverse its current semantics so that it is optimized for the typical
case, provides simpler handling, and resolves various issues with the
previous approach.  In addition, it updates the transaction, address,
and existsaddress indexes to no longer remove entries from blocks that
have been disapproved as, in all cases, the data still exists in the
blockchain and thus should be queryable via the indexes even though
there is special handling applied which treats them as if they did not
exist in certain regards.

Prior to this change, transactions in the regular tree were not applied
to the utxo set until the next block was processed and did not vote
against them.  However, that approach has several undesirable
consequences such as temporarily "invisible" utxos that are actually
spendable, disapproved transactions missing from indexes even though
they are still in the blockchain, and poor performance characteristics.

In a certain sense, the previous approach could be viewed as the
transactions not being valid until they were approved, however, that is
not really true because it was (and still is) perfectly acceptable to
spend utxos created by transactions in the regular tree of the same
block so long as they come before the transactions that spend them.
Further, utxos from a transaction in the regular tree of a block can be
spent in the next block so long as that block does not disapprove them,
which further illustrates that the utxos are actually valid unless they
are disapproved.

Consequently, this modifies that behavior to instead make the utxo set
always track the most recent block and remove the regular transactions
in the parent when a block votes against them.  This approach is
significantly more efficient for the normal case where the previous
block is not disapproved by its successor.

Also, the terminology is changed in several places to refer to
disapproved blocks and transaction trees as opposed to invalid, because
invalid implies the tree/block is malformed or does not follow the
consensus rules.  On the contrary, when a block votes against its
parent, it is only voting against regular transaction tree of the
parent.  Both the block and transaction tree are still valid in that
case, only the regular transaction tree is treated as if it never
existed in terms of effects on the utxo set and duplicate transaction
semantics.

High level overview of changes:
- Modify the utxo viewpoint to reverse semantics as previously described
  - Remove all code related to stake viewpoints
  - Change all block connection code in the viewpoint to first undo all
    transactions in the regular tree of the parent block if the current
    one disapproves it then connect all of the stake txns followed by
    the regular transactions in the block
    - NOTE: The order here is important since stake transactions are not
      allowed to spend outputs from the regular transactions in the same
      block as the next block might disapprove them
  - Change all block disconnection code in the viewpoint to first undo
    all the transactions in the regular and stake trees of the block
    being disconnected, and then resurrect the regular transactions in
    the parent block if the block being disconnected disapproved of it
  - Introduce a new type named viewFilteredSet for handling sets
    filtered by transactions that already exist in a view
  - Introduce a function on the viewpoint for specifically fetching the
    inputs to the regular transactions
  - Update mempool block connection and disconnection code to match the
    new semantics
  - Update all tests to handle the new semantics
- Modify the best state number of transactions to include all
  transactions in all blocks regardless of disapproval because they
  still had to be processed and still exist in the blockchain
- Remove include recent block parameter from mempool.FetchTransaction
  since the utxoset now always includes the latest block
  - This also has the side effect of correcting some unexpected results
    such as coinbases in the most recent block being incorrectly
    reported as having zero confirmations
- Modify mempool utxo fetch logic to use a cached disapproved view, when
  needed, rather than recreating the view for every new transaction
  added to it
- Update spend journal to include all transactions in the block instead
  of only stake transactions from the current block and regular
  transactions from the parent block
- Modify tx and address indexes to store the block index of each tx
  along with its location within the files and update the query
  functions to return the information as well
- Change the tx, address, and existsaddress indexes to index all
  transactions regardless of their disapproval
  - This also corrects several issues such as the inability to query and
    retrieve transactions that exist in a disapproved block
- Update all RPC commands that return verbose transaction information
  to set that newly available block index information properly
- Rename IsRegTxTreeKnownDisapproved in the mining.TxSource interface to
  IsRegTxTreeKnownDisapproved
  - NOTE: This will require a major bump to the mining module before
    the next release
- Rename several utxoView instances to view for consistency
- Rename several variables that dealt with disapproved trees from
  names that contained Invalid to ones that contain Disapproved

NOTE: This does not yet have database migration code and thus will
require a full chain download.  It will exit with error in the case you
attempt to run it against an existing v4 database.  The new database it
creates will be v5, so attempting to run an older version will reject
the new database to prevent corruption.

The database migration will be added in a separate commit.
This commit is contained in:
Dave Collins 2018-09-26 20:54:43 -05:00
parent 1dece72bd8
commit df1898cead
No known key found for this signature in database
GPG Key ID: B8904D9D9C93D1F2
18 changed files with 995 additions and 1191 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

185
blockchain/chain.go
View File

@ -186,6 +186,15 @@ type BlockChain struct {
mainchainBlockCache map[chainhash.Hash]*dcrutil.Block
mainchainBlockCacheSize int
// These fields house a cached view that represents a block that votes
// against its parent and therefore contains all changes as a result
// of disconnecting all regular transactions in its parent. It is only
// lazily updated to the current tip when fetching a utxo view via the
// FetchUtxoView function with the flag indicating the block votes against
// the parent set.
disapprovedViewLock sync.Mutex
disapprovedView *UtxoViewpoint
// These fields are related to checkpoint handling. They are protected
// by the chain lock.
nextCheckpoint *chaincfg.Checkpoint
@ -775,10 +784,10 @@ func (b *BlockChain) connectBlock(node *blockNode, block, parent *dcrutil.Block,
}
// Sanity check the correct number of stxos are provided.
if len(stxos) != countSpentOutputs(block, parent) {
panicf("provided %v stxos for block %v (height %v), but counted %v "+
"spent utxos", len(stxos), node.hash, node.height,
countSpentOutputs(block, parent))
if len(stxos) != countSpentOutputs(block) {
panicf("provided %v stxos for block %v (height %v) which spends %v "+
"outputs", len(stxos), node.hash, node.height,
countSpentOutputs(block))
}
// Write any modified block index entries to the database before
@ -796,26 +805,20 @@ func (b *BlockChain) connectBlock(node *blockNode, block, parent *dcrutil.Block,
return err
}
// Calculate the next stake difficulty.
nextStakeDiff, err := b.calcNextRequiredStakeDifficulty(node)
if err != nil {
return err
}
// Generate a new best state snapshot that will be used to update the
// database and later memory if all database updates are successful.
b.stateLock.RLock()
curTotalTxns := b.stateSnapshot.TotalTxns
curTotalSubsidy := b.stateSnapshot.TotalSubsidy
b.stateLock.RUnlock()
// Calculate the number of transactions that would be added by adding
// this block.
numTxns := countNumberOfTransactions(block, parent)
// Calculate the exact subsidy produced by adding the block.
subsidy := CalculateAddedSubsidy(block, parent)
// Calcultate the next stake difficulty.
nextStakeDiff, err := b.calcNextRequiredStakeDifficulty(node)
if err != nil {
return err
}
numTxns := uint64(len(block.Transactions()) + len(block.STransactions()))
blockSize := uint64(block.MsgBlock().Header.Size)
state := newBestState(node, blockSize, numTxns, curTotalTxns+numTxns,
node.CalcPastMedianTime(), curTotalSubsidy+subsidy,
@ -983,19 +986,14 @@ func (b *BlockChain) disconnectBlock(node *blockNode, block, parent *dcrutil.Blo
curTotalSubsidy := b.stateSnapshot.TotalSubsidy
b.stateLock.RUnlock()
parentBlockSize := uint64(parent.MsgBlock().Header.Size)
// Calculate the number of transactions that would be added by adding
// this block.
numTxns := countNumberOfTransactions(block, parent)
newTotalTxns := curTotalTxns - numTxns
// Calculate the exact subsidy produced by adding the block.
numParentTxns := uint64(len(parent.Transactions()) + len(parent.STransactions()))
numBlockTxns := uint64(len(block.Transactions()) + len(block.STransactions()))
newTotalTxns := curTotalTxns - numBlockTxns
subsidy := CalculateAddedSubsidy(block, parent)
newTotalSubsidy := curTotalSubsidy - subsidy
prevNode := node.parent
state := newBestState(prevNode, parentBlockSize, numTxns, newTotalTxns,
prevNode.CalcPastMedianTime(), newTotalSubsidy,
state := newBestState(prevNode, parentBlockSize, numParentTxns,
newTotalTxns, prevNode.CalcPastMedianTime(), newTotalSubsidy,
uint32(prevNode.stakeNode.PoolSize()), node.sbits,
prevNode.stakeNode.Winners(), prevNode.stakeNode.MissedTickets(),
prevNode.stakeNode.FinalState())
@ -1075,39 +1073,35 @@ func (b *BlockChain) disconnectBlock(node *blockNode, block, parent *dcrutil.Blo
return nil
}
// countSpentOutputs returns the number of utxos the passed block spends.
func countSpentOutputs(block *dcrutil.Block, parent *dcrutil.Block) int {
// We need to skip the regular tx tree if it's not valid.
// We also exclude the coinbase transaction since it can't
// spend anything.
// countSpentRegularOutputs returns the number of utxos the regular transactions
// in the passed block spend.
func countSpentRegularOutputs(block *dcrutil.Block) int {
// Skip the coinbase since it has no inputs.
var numSpent int
if headerApprovesParent(&block.MsgBlock().Header) {
for _, tx := range parent.Transactions()[1:] {
numSpent += len(tx.MsgTx().TxIn)
}
for _, tx := range block.MsgBlock().Transactions[1:] {
numSpent += len(tx.TxIn)
}
return numSpent
}
// countSpentStakeOutputs returns the number of utxos the stake transactions in
// the passed block spend.
func countSpentStakeOutputs(block *dcrutil.Block) int {
var numSpent int
for _, stx := range block.MsgBlock().STransactions {
txType := stake.DetermineTxType(stx)
if txType == stake.TxTypeSSGen || txType == stake.TxTypeSSRtx {
// Exclude the vote stakebase since it has no input.
if stake.IsSSGen(stx) {
numSpent++
continue
}
numSpent += len(stx.TxIn)
}
return numSpent
}
// countNumberOfTransactions returns the number of transactions inserted by
// adding the block.
func countNumberOfTransactions(block, parent *dcrutil.Block) uint64 {
var numTxns uint64
if headerApprovesParent(&block.MsgBlock().Header) {
numTxns += uint64(len(parent.Transactions()))
}
numTxns += uint64(len(block.STransactions()))
return numTxns
// countSpentOutputs returns the number of utxos the passed block spends.
func countSpentOutputs(block *dcrutil.Block) int {
return countSpentRegularOutputs(block) + countSpentStakeOutputs(block)
}
// reorganizeChain reorganizes the block chain by disconnecting the nodes in the
@ -1167,10 +1161,10 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
// Disconnect all of the blocks back to the point of the fork. This
// entails loading the blocks and their associated spent txos from the
// database and using that information to unspend all of the spent txos
// and remove the utxos created by the blocks.
// and remove the utxos created by the blocks. In addition, if a block
// votes against its parent, the regular transactions are reconnected.
view := NewUtxoViewpoint()
view.SetBestHash(&oldBest.hash)
view.SetStakeViewpoint(ViewpointPrevValidInitial)
var nextBlockToDetach *dcrutil.Block
for e := detachNodes.Front(); e != nil; e = e.Next() {
// Grab the block to detach based on the node. Use the fact that the
@ -1204,25 +1198,21 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
// journal.
var stxos []spentTxOut
err = b.db.View(func(dbTx database.Tx) error {
stxos, err = dbFetchSpendJournalEntry(dbTx, block, parent)
stxos, err = dbFetchSpendJournalEntry(dbTx, block)
return err
})
if err != nil {
return err
}
// Quick sanity test.
if len(stxos) != countSpentOutputs(block, parent) {
panicf("retrieved %v stxos when trying to disconnect block %v "+
"(height %v), yet counted %v many spent utxos", len(stxos),
block.Hash(), block.Height(), countSpentOutputs(block, parent))
}
// Store the loaded block and spend journal entry for later.
detachBlocks = append(detachBlocks, block)
detachSpentTxOuts = append(detachSpentTxOuts, stxos)
err = b.disconnectTransactions(view, block, parent, stxos)
// Update the view to unspend all of the spent txos and remove the utxos
// created by the block. Also, if the block votes against its parent,
// reconnect all of the regular transactions.
err = view.disconnectBlock(b.db, block, parent, stxos)
if err != nil {
return err
}
@ -1283,24 +1273,22 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
// Skip validation if the block is already known to be valid.
// However, the UTXO view still needs to be updated.
if b.index.NodeStatus(n).KnownValid() {
err = b.connectTransactions(view, block, parent, nil)
stxos := make([]spentTxOut, 0, countSpentOutputs(block))
err := view.connectBlock(b.db, block, parent, &stxos)
if err != nil {
return err
}
view.cachedStxos[n.hash] = stxos
newBest = n
continue
}
// Notice the spent txout details are not requested here and
// thus will not be generated. This is done because the state
// is not being immediately written to the database, so it is
// not needed.
//
// In the case the block is determined to be invalid due to a
// rule violation, mark it as invalid and mark all of its
// descendants as having an invalid ancestor.
err = b.checkConnectBlock(n, block, parent, view, nil)
// In the case the block is determined to be invalid due to a rule
// violation, mark it as invalid and mark all of its descendants as
// having an invalid ancestor.
stxos := make([]spentTxOut, 0, countSpentOutputs(block))
err = b.checkConnectBlock(n, block, parent, view, &stxos)
if err != nil {
if _, ok := err.(RuleError); ok {
b.index.SetStatusFlags(n, statusValidateFailed)
@ -1312,6 +1300,7 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
return err
}
b.index.SetStatusFlags(n, statusValid)
view.cachedStxos[n.hash] = stxos
newBest = n
}
@ -1348,7 +1337,6 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
// disconnected.
view = NewUtxoViewpoint()
view.SetBestHash(&oldBest.hash)
view.SetStakeViewpoint(ViewpointPrevValidInitial)
// Disconnect blocks from the main chain.
for i, e := 0, detachNodes.Front(); e != nil; i, e = i+1, e.Next() {
@ -1368,17 +1356,10 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
&n.parent.hash, parent.Hash())
}
// Load all of the utxos referenced by the block that aren't
// already in the view.
err := view.fetchInputUtxos(b.db, block, parent)
if err != nil {
return err
}
// Update the view to unspend all of the spent txos and remove
// the utxos created by the block.
err = b.disconnectTransactions(view, block, parent,
detachSpentTxOuts[i])
// Update the view to unspend all of the spent txos and remove the utxos
// created by the block. Also, if the block votes against its parent,
// reconnect all of the regular transactions.
err := view.disconnectBlock(b.db, block, parent, detachSpentTxOuts[i])
if err != nil {
return err
}
@ -1408,12 +1389,13 @@ func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error
&n.parent.hash, parent.Hash())
}
// Update the view to mark all utxos referenced by the block
// as spent and add all transactions being created by this block
// to it. Also, provide an stxo slice so the spent txout
// details are generated.
stxos := make([]spentTxOut, 0, countSpentOutputs(block, parent))
err := b.connectTransactions(view, block, parent, &stxos)
// Update the view to mark all utxos referenced by the block as spent
// and add all transactions being created by this block to it. In the
// case the block votes against the parent, also disconnect all of the
// regular transactions in the parent block. Finally, provide an stxo
// slice so the spent txout details are generated.
stxos := make([]spentTxOut, 0, countSpentOutputs(block))
err := view.connectBlock(b.db, block, parent, &stxos)
if err != nil {
return err
}
@ -1483,7 +1465,6 @@ func (b *BlockChain) forceHeadReorganization(formerBest chainhash.Hash, newBest
// Check to make sure our forced-in node validates correctly.
view := NewUtxoViewpoint()
view.SetBestHash(&formerBestNode.parent.hash)
view.SetStakeViewpoint(ViewpointPrevValidInitial)
formerBestBlock, err := b.fetchBlockByNode(formerBestNode)
if err != nil {
@ -1495,25 +1476,14 @@ func (b *BlockChain) forceHeadReorganization(formerBest chainhash.Hash, newBest
}
var stxos []spentTxOut
err = b.db.View(func(dbTx database.Tx) error {
stxos, err = dbFetchSpendJournalEntry(dbTx, formerBestBlock,
commonParentBlock)
stxos, err = dbFetchSpendJournalEntry(dbTx, formerBestBlock)
return err
})
if err != nil {
return err
}
// Quick sanity test.
if len(stxos) != countSpentOutputs(formerBestBlock, commonParentBlock) {
panicf("retrieved %v stxos when trying to disconnect block %v "+
"(height %v), yet counted %v many spent utxos when trying to "+
"force head reorg", len(stxos), formerBestBlock.Hash(),
formerBestBlock.Height(),
countSpentOutputs(formerBestBlock, commonParentBlock))
}
err = b.disconnectTransactions(view, formerBestBlock, commonParentBlock,
stxos)
err = view.disconnectBlock(b.db, formerBestBlock, commonParentBlock, stxos)
if err != nil {
return err
}
@ -1645,7 +1615,6 @@ func (b *BlockChain) connectBestChain(node *blockNode, block, parent *dcrutil.Bl
// revalidated after a restart.
view := NewUtxoViewpoint()
view.SetBestHash(parentHash)
view.SetStakeViewpoint(ViewpointPrevValidInitial)
var stxos []spentTxOut
if !fastAdd {
err := b.checkConnectBlock(node, block, parent, view,
@ -1666,13 +1635,11 @@ func (b *BlockChain) connectBestChain(node *blockNode, block, parent *dcrutil.Bl
// In the fast add case the code to check the block connection
// was skipped, so the utxo view needs to load the referenced
// utxos, spend them, and add the new utxos being created by
// this block.
// this block. Also, in the case the the block votes against
// the parent, its regular transaction tree must be
// disconnected.
if fastAdd {
err := view.fetchInputUtxos(b.db, block, parent)
if err != nil {
return 0, err
}
err = b.connectTransactions(view, block, parent, &stxos)
err := view.connectBlock(b.db, block, parent, &stxos)
if err != nil {
return 0, err
}

20
blockchain/chainio.go
View File

@ -24,7 +24,7 @@ import (
const (
// currentDatabaseVersion indicates what the current database
// version is.
currentDatabaseVersion = 4
currentDatabaseVersion = 5
// currentBlockIndexVersion indicates what the current block index
// database version.
@ -646,9 +646,7 @@ func deserializeSpendJournalEntry(serialized []byte, txns []*wire.MsgTx) ([]spen
// Calculate the total number of stxos.
var numStxos int
for _, tx := range txns {
txType := stake.DetermineTxType(tx)
if txType == stake.TxTypeSSGen {
if stake.IsSSGen(tx) {
numStxos++
continue
}
@ -676,13 +674,13 @@ func deserializeSpendJournalEntry(serialized []byte, txns []*wire.MsgTx) ([]spen
stxos := make([]spentTxOut, numStxos)
for txIdx := len(txns) - 1; txIdx > -1; txIdx-- {
tx := txns[txIdx]
txType := stake.DetermineTxType(tx)
isVote := stake.IsSSGen(tx)
// Loop backwards through all of the transaction inputs and read
// the associated stxo.
for txInIdx := len(tx.TxIn) - 1; txInIdx > -1; txInIdx-- {
// Skip stakebase.
if txInIdx == 0 && txType == stake.TxTypeSSGen {
// Skip stakebase since it has no input.
if isVote && txInIdx == 0 {
continue
}
@ -760,17 +758,13 @@ func serializeSpendJournalEntry(stxos []spentTxOut) ([]byte, error) {
// view MUST have the utxos referenced by all of the transactions available for
// the passed block since that information is required to reconstruct the spent
// txouts.
func dbFetchSpendJournalEntry(dbTx database.Tx, block *dcrutil.Block, parent *dcrutil.Block) ([]spentTxOut, error) {
func dbFetchSpendJournalEntry(dbTx database.Tx, block *dcrutil.Block) ([]spentTxOut, error) {
// Exclude the coinbase transaction since it can't spend anything.
spendBucket := dbTx.Metadata().Bucket(dbnamespace.SpendJournalBucketName)
serialized := spendBucket.Get(block.Hash()[:])
var blockTxns []*wire.MsgTx
if headerApprovesParent(&block.MsgBlock().Header) {
blockTxns = append(blockTxns, parent.MsgBlock().Transactions[1:]...)
}
blockTxns = append(blockTxns, block.MsgBlock().STransactions...)
blockTxns = append(blockTxns, block.MsgBlock().Transactions[1:]...)
if len(blockTxns) > 0 && len(serialized) == 0 {
panicf("missing spend journal data for %s", block.Hash())
}

130
blockchain/indexers/addrindex.go
View File

@ -67,10 +67,6 @@ const (
// the hash of a pubkey address might be the same as that of a script
// hash.
addrKeyTypeScriptHash = 3
// Size of a transaction entry. It consists of 4 bytes block id + 4
// bytes offset + 4 bytes length.
txEntrySize = 4 + 4 + 4
)
var (
@ -131,14 +127,15 @@ var (
//
// The serialized value format is:
//
// [<block id><start offset><tx length>,...]
// [<block id><start offset><tx length><block index>,...]
//
// Field Type Size
// block id uint32 4 bytes
// start offset uint32 4 bytes
// tx length uint32 4 bytes
// block index uint32 4 bytes
// -----
// Total: 12 bytes per indexed tx
// Total: 16 bytes per indexed tx
// -----------------------------------------------------------------------------
// fetchBlockHashFunc defines a callback function to use in order to convert a
@ -147,12 +144,13 @@ type fetchBlockHashFunc func(serializedID []byte) (*chainhash.Hash, error)
// serializeAddrIndexEntry serializes the provided block id and transaction
// location according to the format described in detail above.
func serializeAddrIndexEntry(blockID uint32, txLoc wire.TxLoc) []byte {
func serializeAddrIndexEntry(blockID uint32, txLoc wire.TxLoc, blockIndex uint32) []byte {
// Serialize the entry.
serialized := make([]byte, 12)
serialized := make([]byte, txEntrySize)
byteOrder.PutUint32(serialized, blockID)
byteOrder.PutUint32(serialized[4:], uint32(txLoc.TxStart))
byteOrder.PutUint32(serialized[8:], uint32(txLoc.TxLen))
byteOrder.PutUint32(serialized[12:], blockIndex)
return serialized
}
@ -160,7 +158,7 @@ func serializeAddrIndexEntry(blockID uint32, txLoc wire.TxLoc) []byte {
// provided region struct according to the format described in detail above and
// uses the passed block hash fetching function in order to conver the block ID
// to the associated block hash.
func deserializeAddrIndexEntry(serialized []byte, region *database.BlockRegion, fetchBlockHash fetchBlockHashFunc) error {
func deserializeAddrIndexEntry(serialized []byte, entry *TxIndexEntry, fetchBlockHash fetchBlockHashFunc) error {
// Ensure there are enough bytes to decode.
if len(serialized) < txEntrySize {
return errDeserialize("unexpected end of data")
@ -170,9 +168,11 @@ func deserializeAddrIndexEntry(serialized []byte, region *database.BlockRegion,
if err != nil {
return err
}
region := &entry.BlockRegion
region.Hash = hash
region.Offset = byteOrder.Uint32(serialized[4:8])
region.Len = byteOrder.Uint32(serialized[8:12])
entry.BlockIndex = byteOrder.Uint32(serialized[12:16])
return nil
}
@ -187,14 +187,14 @@ func keyForLevel(addrKey [addrKeySize]byte, level uint8) [levelKeySize]byte {
// dbPutAddrIndexEntry updates the address index to include the provided entry
// according to the level-based scheme described in detail above.
func dbPutAddrIndexEntry(bucket internalBucket, addrKey [addrKeySize]byte, blockID uint32, txLoc wire.TxLoc) error {
func dbPutAddrIndexEntry(bucket internalBucket, addrKey [addrKeySize]byte, blockID uint32, txLoc wire.TxLoc, blockIndex uint32) error {
// Start with level 0 and its initial max number of entries.
curLevel := uint8(0)
maxLevelBytes := level0MaxEntries * txEntrySize
// Simply append the new entry to level 0 and return now when it will
// fit. This is the most common path.
newData := serializeAddrIndexEntry(blockID, txLoc)
newData := serializeAddrIndexEntry(blockID, txLoc, blockIndex)
level0Key := keyForLevel(addrKey, 0)
level0Data := bucket.Get(level0Key[:])
if len(level0Data)+len(newData) <= maxLevelBytes {
@ -258,7 +258,7 @@ func dbPutAddrIndexEntry(bucket internalBucket, addrKey [addrKeySize]byte, block
// the given address key and the number of entries skipped since it could have
// been less in the case where there are less total entries than the requested
// number of entries to skip.
func dbFetchAddrIndexEntries(bucket internalBucket, addrKey [addrKeySize]byte, numToSkip, numRequested uint32, reverse bool, fetchBlockHash fetchBlockHashFunc) ([]database.BlockRegion, uint32, error) {
func dbFetchAddrIndexEntries(bucket internalBucket, addrKey [addrKeySize]byte, numToSkip, numRequested uint32, reverse bool, fetchBlockHash fetchBlockHashFunc) ([]TxIndexEntry, uint32, error) {
// When the reverse flag is not set, all levels need to be fetched
// because numToSkip and numRequested are counted from the oldest
// transactions (highest level) and thus the total count is needed.
@ -304,7 +304,7 @@ func dbFetchAddrIndexEntries(bucket internalBucket, addrKey [addrKeySize]byte, n
// Start the offset after all skipped entries and load the calculated
// number.
results := make([]database.BlockRegion, numToLoad)
results := make([]TxIndexEntry, numToLoad)
for i := uint32(0); i < numToLoad; i++ {
// Calculate the read offset according to the reverse flag.
var offset uint32
@ -714,20 +714,16 @@ func (idx *AddrIndex) indexPkScript(data writeIndexData, scriptVersion uint16, p
}
}
// indexBlock extract all of the standard addresses from all of the transactions
// in the parent of the passed block (if they were valid) and all of the stake
// transactions in the passed block, and maps each of them to the associated
// transaction using the passed map.
func (idx *AddrIndex) indexBlock(data writeIndexData, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) {
var parentRegularTxs []*dcrutil.Tx
if approvesParent(block) && block.Height() > 1 {
parentRegularTxs = parent.Transactions()
}
for txIdx, tx := range parentRegularTxs {
// indexBlock extracts all of the standard addresses from all of the
// regular and stake transactions in the passed block and maps each of them to
// the associated transaction using the passed map.
func (idx *AddrIndex) indexBlock(data writeIndexData, block *dcrutil.Block, view *blockchain.UtxoViewpoint) {
regularTxns := block.Transactions()
for txIdx, tx := range regularTxns {
// Coinbases do not reference any inputs. Since the block is
// required to have already gone through full validation, it has
// already been proven on the first transaction in the block is
// a coinbase.
// already been proven that the first transaction in the block
// is a coinbase.
if txIdx != 0 {
for _, txIn := range tx.MsgTx().TxIn {
// The view should always have the input since
@ -758,7 +754,7 @@ func (idx *AddrIndex) indexBlock(data writeIndexData, block, parent *dcrutil.Blo
for txIdx, tx := range block.STransactions() {
msgTx := tx.MsgTx()
thisTxOffset := txIdx + len(parentRegularTxs)
thisTxOffset := txIdx + len(regularTxns)
isSSGen := stake.IsSSGen(msgTx)
for i, txIn := range msgTx.TxIn {
@ -800,64 +796,44 @@ func (idx *AddrIndex) indexBlock(data writeIndexData, block, parent *dcrutil.Blo
//
// This is part of the Indexer interface.
func (idx *AddrIndex) ConnectBlock(dbTx database.Tx, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) error {
// The offset and length of the transactions within the serialized
// block for the regular transactions of the previous block, if
// applicable.
var parentTxLocs []wire.TxLoc
var parentBlockID uint32
if approvesParent(block) && block.Height() > 1 {
var err error
parentTxLocs, _, err = parent.TxLoc()
if err != nil {
return err
}
// NOTE: The fact that the block can disapprove the regular tree of the
// previous block is ignored for this index because even though the
// disapproved transactions no longer apply spend semantics, they still
// exist within the block and thus have to be processed before the next
// block disapproves them.
parentHash := parent.Hash()
parentBlockID, err = dbFetchBlockIDByHash(dbTx, parentHash)
if err != nil {
return err
}
}
// The offset and length of the transactions within the serialized
// block for the added stake transactions.
_, blockStxLocs, err := block.TxLoc()
// The offset and length of the transactions within the serialized block.
txLocs, stakeTxLocs, err := block.TxLoc()
if err != nil {
return err
}
// Nothing to index, just return.
if len(parentTxLocs)+len(blockStxLocs) == 0 {
return nil
}
// Get the internal block ID associated with the block.
blockHash := block.Hash()
blockID, err := dbFetchBlockIDByHash(dbTx, blockHash)
blockID, err := dbFetchBlockIDByHash(dbTx, block.Hash())
if err != nil {
return err
}
// Build all of the address to transaction mappings in a local map.
addrsToTxns := make(writeIndexData)
idx.indexBlock(addrsToTxns, block, parent, view)
idx.indexBlock(addrsToTxns, block, view)
// Add all of the index entries for each address.
stakeIdxsStart := len(parentTxLocs)
allTxLocs := append(parentTxLocs, blockStxLocs...)
stakeIdxsStart := len(txLocs)
addrIdxBucket := dbTx.Metadata().Bucket(addrIndexKey)
for addrKey, txIdxs := range addrsToTxns {
for _, txIdx := range txIdxs {
// Switch to using the newest block ID for the stake transactions,
// since these are not from the parent. Offset the index to be
// correct for the location in this given block.
blockIDToUse := parentBlockID
// Adjust the block index and slice of transaction locations to use
// based on the regular or stake tree.
txLocations := txLocs
blockIndex := txIdx
if txIdx >= stakeIdxsStart {
blockIDToUse = blockID
txLocations = stakeTxLocs
blockIndex -= stakeIdxsStart
}
err := dbPutAddrIndexEntry(addrIdxBucket, addrKey,
blockIDToUse, allTxLocs[txIdx])
err := dbPutAddrIndexEntry(addrIdxBucket, addrKey, blockID,
txLocations[blockIndex], uint32(blockIndex))
if err != nil {
return err
}
@ -873,9 +849,15 @@ func (idx *AddrIndex) ConnectBlock(dbTx database.Tx, block, parent *dcrutil.Bloc
//
// This is part of the Indexer interface.
func (idx *AddrIndex) DisconnectBlock(dbTx database.Tx, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) error {
// NOTE: The fact that the block can disapprove the regular tree of the
// previous block is ignored for this index because even though the
// disapproved transactions no longer apply spend semantics, they still
// exist within the block and thus have to be processed before the next
// block disapproves them.
// Build all of the address to transaction mappings in a local map.
addrsToTxns := make(writeIndexData)
idx.indexBlock(addrsToTxns, block, parent, view)
idx.indexBlock(addrsToTxns, block, view)
// Remove all of the index entries for each address.
bucket := dbTx.Metadata().Bucket(addrIndexKey)
@ -889,24 +871,24 @@ func (idx *AddrIndex) DisconnectBlock(dbTx database.Tx, block, parent *dcrutil.B
return nil
}
// TxRegionsForAddress returns a slice of block regions which identify each
// transaction that involves the passed address according to the specified
// number to skip, number requested, and whether or not the results should be
// reversed. It also returns the number actually skipped since it could be less
// in the case where there are not enough entries.
// EntriesForAddress returns a slice of details which identify each transaction,
// including a block region, that involves the passed address according to the
// specified number to skip, number requested, and whether or not the results
// should be reversed. It also returns the number actually skipped since it
// could be less in the case where there are not enough entries.
//
// NOTE: These results only include transactions confirmed in blocks. See the
// UnconfirmedTxnsForAddress method for obtaining unconfirmed transactions
// that involve a given address.
//
// This function is safe for concurrent access.
func (idx *AddrIndex) TxRegionsForAddress(dbTx database.Tx, addr dcrutil.Address, numToSkip, numRequested uint32, reverse bool) ([]database.BlockRegion, uint32, error) {
func (idx *AddrIndex) EntriesForAddress(dbTx database.Tx, addr dcrutil.Address, numToSkip, numRequested uint32, reverse bool) ([]TxIndexEntry, uint32, error) {
addrKey, err := addrToKey(addr, idx.chainParams)
if err != nil {
return nil, 0, err
}
var regions []database.BlockRegion
var entries []TxIndexEntry
var skipped uint32
err = idx.db.View(func(dbTx database.Tx) error {
// Create closure to lookup the block hash given the ID using
@ -918,13 +900,13 @@ func (idx *AddrIndex) TxRegionsForAddress(dbTx database.Tx, addr dcrutil.Address
var err error
addrIdxBucket := dbTx.Metadata().Bucket(addrIndexKey)
regions, skipped, err = dbFetchAddrIndexEntries(addrIdxBucket,
entries, skipped, err = dbFetchAddrIndexEntries(addrIdxBucket,
addrKey, numToSkip, numRequested, reverse,
fetchBlockHash)
return err
})
return regions, skipped, err
return entries, skipped, err
}
// indexUnconfirmedAddresses modifies the unconfirmed (memory-only) address

2
blockchain/indexers/addrindex_test.go
View File

@ -222,7 +222,7 @@ nextTest:
for i := 0; i < test.numInsert; i++ {
txLoc := wire.TxLoc{TxStart: i * 2}
err := dbPutAddrIndexEntry(populatedBucket, test.key,
uint32(i), txLoc)
uint32(i), txLoc, uint32(i%100))
if err != nil {
t.Errorf("dbPutAddrIndexEntry #%d (%s) - "+
"unexpected error: %v", testNum,

63
blockchain/indexers/existsaddrindex.go
View File

@ -203,16 +203,23 @@ func (idx *ExistsAddrIndex) ExistsAddresses(addrs []dcrutil.Address) ([]bool, er
//
// This is part of the Indexer interface.
func (idx *ExistsAddrIndex) ConnectBlock(dbTx database.Tx, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) error {
var parentTxs []*dcrutil.Tx
if approvesParent(block) && block.Height() > 1 {
parentTxs = parent.Transactions()
}
blockTxns := block.STransactions()
allTxns := append(parentTxs, blockTxns...)
// NOTE: The fact that the block can disapprove the regular tree of the
// previous block is ignored for this index because even though technically
// the address might become unused again if its only use was in a
// transaction that was disapproved, the chances of that are extremely low
// since disapproved transactions are nearly always mined again in another
// block.
//
// More importantly, the primary purpose of this index is to track whether
// or not addresses have ever been seen, so even if they technically end up
// becoming unused, they were still seen.
usedAddrs := make(map[[addrKeySize]byte]struct{})
for _, tx := range allTxns {
blockTxns := make([]*dcrutil.Tx, 0, len(block.Transactions())+
len(block.STransactions()))
blockTxns = append(blockTxns, block.Transactions()...)
blockTxns = append(blockTxns, block.STransactions()...)
for _, tx := range blockTxns {
msgTx := tx.MsgTx()
isSStx := stake.IsSStx(msgTx)
for _, txIn := range msgTx.TxIn {
@ -283,23 +290,23 @@ func (idx *ExistsAddrIndex) ConnectBlock(dbTx database.Tx, block, parent *dcruti
// then remove them from the unconfirmed map drop
// dropping the old map and reassigning a new map.
idx.unconfirmedLock.Lock()
for k := range idx.mpExistsAddr {
usedAddrs[k] = struct{}{}
for addrKey := range idx.mpExistsAddr {
usedAddrs[addrKey] = struct{}{}
}
idx.mpExistsAddr = make(map[[addrKeySize]byte]struct{})
idx.unconfirmedLock.Unlock()
meta := dbTx.Metadata()
existsAddrIndex := meta.Bucket(existsAddrIndexKey)
existsAddrIdxBucket := meta.Bucket(existsAddrIndexKey)
newUsedAddrs := make(map[[addrKeySize]byte]struct{})
for k := range usedAddrs {
if !idx.existsAddress(existsAddrIndex, k) {
newUsedAddrs[k] = struct{}{}
for addrKey := range usedAddrs {
if !idx.existsAddress(existsAddrIdxBucket, addrKey) {
newUsedAddrs[addrKey] = struct{}{}
}
}
for k := range newUsedAddrs {
err := dbPutExistsAddr(existsAddrIndex, k)
for addrKey := range newUsedAddrs {
err := dbPutExistsAddr(existsAddrIdxBucket, addrKey)
if err != nil {
return err
}
@ -314,11 +321,25 @@ func (idx *ExistsAddrIndex) ConnectBlock(dbTx database.Tx, block, parent *dcruti
//
// This is part of the Indexer interface.
func (idx *ExistsAddrIndex) DisconnectBlock(dbTx database.Tx, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) error {
// The primary purpose of this index is to track whether or not addresses
// have ever been seen, so even if they ultimately end up technically
// becoming unused due to being in a block that was disconnected and the
// associated transactions never make it into a new block for some reason,
// it was still seen at some point. Thus, don't bother removing entries.
//
// Note that this does mean different nodes may slightly disagree about
// whether or not an address that only ever existed in an orphaned side
// chain was seen, however, that is an acceptable tradeoff given the use
// case and the huge performance gained from not having to constantly update
// the index with usage counts that would be required to properly handle
// disconnecting block and disapproved regular trees.
return nil
}
// addUnconfirmedTx adds all addresses related to the transaction to the
// unconfirmed (memory-only) exists address index.
//
// This function MUST be called with the unconfirmed lock held.
func (idx *ExistsAddrIndex) addUnconfirmedTx(tx *wire.MsgTx) {
isSStx := stake.IsSStx(tx)
for _, txIn := range tx.TxIn {
@ -387,18 +408,18 @@ func (idx *ExistsAddrIndex) addUnconfirmedTx(tx *wire.MsgTx) {
}
}
// AddUnconfirmedTx is the exported form of addUnconfirmedTx.
// AddUnconfirmedTx adds all addresses related to the transaction to the
// unconfirmed (memory-only) exists address index.
//
// This function is safe for concurrent access.
func (idx *ExistsAddrIndex) AddUnconfirmedTx(tx *wire.MsgTx) {
idx.unconfirmedLock.Lock()
defer idx.unconfirmedLock.Unlock()
idx.addUnconfirmedTx(tx)
idx.unconfirmedLock.Unlock()
}
// DropExistsAddrIndex drops the exists address index from the provided
// database if it exists.
// DropExistsAddrIndex drops the exists address index from the provided database
// if it exists.
func DropExistsAddrIndex(db database.DB, interrupt <-chan struct{}) error {
return dropFlatIndex(db, existsAddrIndexKey, existsAddressIndexName,
interrupt)

109
blockchain/indexers/manager.go
View File

@ -347,7 +347,7 @@ func (m *Manager) Init(chain *blockchain.BlockChain, interrupt <-chan struct{})
var view *blockchain.UtxoViewpoint
if indexNeedsInputs(indexer) {
var err error
view, err = makeUtxoView(dbTx, block, parent,
view, err = makeUtxoView(dbTx, block,
interrupt)
if err != nil {
return err
@ -487,8 +487,7 @@ func (m *Manager) Init(chain *blockchain.BlockChain, interrupt <-chan struct{})
// index.
if view == nil && indexNeedsInputs(indexer) {
var errMakeView error
view, errMakeView = makeUtxoView(dbTx, block, parent,
interrupt)
view, errMakeView = makeUtxoView(dbTx, block, interrupt)
if errMakeView != nil {
return errMakeView
}
@ -528,16 +527,16 @@ func indexNeedsInputs(index Indexer) bool {
// loads it from the database.
func dbFetchTx(dbTx database.Tx, hash *chainhash.Hash) (*wire.MsgTx, error) {
// Look up the location of the transaction.
blockRegion, err := dbFetchTxIndexEntry(dbTx, hash)
entry, err := dbFetchTxIndexEntry(dbTx, hash)
if err != nil {
return nil, err
}
if blockRegion == nil {
if entry == nil {
return nil, fmt.Errorf("transaction %v not found in the txindex", hash)
}
// Load the raw transaction bytes from the database.
txBytes, err := dbTx.FetchBlockRegion(blockRegion)
txBytes, err := dbTx.FetchBlockRegion(&entry.BlockRegion)
if err != nil {
return nil, err
}
@ -557,74 +556,56 @@ func dbFetchTx(dbTx database.Tx, hash *chainhash.Hash) (*wire.MsgTx, error) {
// transactions in the block. This is sometimes needed when catching indexes up
// because many of the txouts could actually already be spent however the
// associated scripts are still required to index them.
func makeUtxoView(dbTx database.Tx, block, parent *dcrutil.Block, interrupt <-chan struct{}) (*blockchain.UtxoViewpoint, error) {
func makeUtxoView(dbTx database.Tx, block *dcrutil.Block, interrupt <-chan struct{}) (*blockchain.UtxoViewpoint, error) {
view := blockchain.NewUtxoViewpoint()
var parentRegularTxs []*dcrutil.Tx
if approvesParent(block) {
parentRegularTxs = parent.Transactions()
}
for txIdx, tx := range parentRegularTxs {
// Coinbases do not reference any inputs. Since the block is
// required to have already gone through full validation, it has
// already been proven on the first transaction in the block is
// a coinbase.
if txIdx == 0 {
continue
}
// Use the transaction index to load all of the referenced
// inputs and add their outputs to the view.
for _, txIn := range tx.MsgTx().TxIn {
// Skip already fetched outputs.
originOut := &txIn.PreviousOutPoint
if view.LookupEntry(&originOut.Hash) != nil {
processTxns := func(txns []*dcrutil.Tx, regularTree bool) error {
for txIdx, tx := range txns {
// Coinbases do not reference any inputs. Since the block is
// required to have already gone through full validation, it has
// already been proven on the first transaction in the block is a
// coinbase.
if regularTree && txIdx == 0 {
continue
}
msgTx := tx.MsgTx()
isVote := !regularTree && stake.IsSSGen(msgTx)
originTx, err := dbFetchTx(dbTx, &originOut.Hash)
if err != nil {
return nil, err
// Use the transaction index to load all of the referenced inputs
// and add their outputs to the view.
for txInIdx, txIn := range msgTx.TxIn {
// Ignore stakebase since it has no input.
if isVote && txInIdx == 0 {
continue
}
// Skip already fetched outputs.
originOut := &txIn.PreviousOutPoint
if view.LookupEntry(&originOut.Hash) != nil {
continue
}
originTx, err := dbFetchTx(dbTx, &originOut.Hash)
if err != nil {
return err
}
view.AddTxOuts(dcrutil.NewTx(originTx), int64(txIn.BlockHeight),
txIn.BlockIndex)
}
view.AddTxOuts(dcrutil.NewTx(originTx),
int64(wire.NullBlockHeight),
wire.NullBlockIndex)
if interruptRequested(interrupt) {
return errInterruptRequested
}
}
if interruptRequested(interrupt) {
return nil, errInterruptRequested
}
return nil
}
for _, tx := range block.STransactions() {
msgTx := tx.MsgTx()
isSSGen := stake.IsSSGen(msgTx)
// Use the transaction index to load all of the referenced
// inputs and add their outputs to the view.
for i, txIn := range msgTx.TxIn {
// Skip stakebases.
if isSSGen && i == 0 {
continue
}
originOut := &txIn.PreviousOutPoint
if view.LookupEntry(&originOut.Hash) != nil {
continue
}
originTx, err := dbFetchTx(dbTx, &originOut.Hash)
if err != nil {
return nil, err
}
view.AddTxOuts(dcrutil.NewTx(originTx), int64(wire.NullBlockHeight),
wire.NullBlockIndex)
}
if interruptRequested(interrupt) {
return nil, errInterruptRequested
}
if err := processTxns(block.STransactions(), false); err != nil {
return nil, err
}
if err := processTxns(block.Transactions(), true); err != nil {
return nil, err
}
return view, nil

136
blockchain/indexers/txindex.go
View File

@ -19,6 +19,11 @@ import (
const (
// txIndexName is the human-readable name for the index.
txIndexName = "transaction index"
// txEntrySize is the size of a transaction entry. It consists of 4
// bytes block id + 4 bytes offset + 4 bytes length + 4 bytes block
// index.
txEntrySize = 4 + 4 + 4 + 4
)
var (
@ -81,17 +86,28 @@ var (
//
// The serialized format for the keys and values in the tx index bucket is:
//
// <txhash> = <block id><start offset><tx length>
// <txhash> = <block id><start offset><tx length><block index>
//
// Field Type Size
// txhash chainhash.Hash 32 bytes
// block id uint32 4 bytes
// start offset uint32 4 bytes
// tx length uint32 4 bytes
// block index uint32 4 bytes
// -----
// Total: 44 bytes
// Total: 48 bytes
// -----------------------------------------------------------------------------
// TxIndexEntry houses information about an entry in the transaction index.
type TxIndexEntry struct {
// BlockRegion specifies the location of the raw bytes of the transaction.
BlockRegion database.BlockRegion
// BlockIndex species the index of the transaction within the array of
// transactions that comprise a tree of the block.
BlockIndex uint32
}
// dbPutBlockIDIndexEntry uses an existing database transaction to update or add
// the index entries for the hash to id and id to hash mappings for the provided
// values.
@ -169,10 +185,11 @@ func dbFetchBlockHashByID(dbTx database.Tx, id uint32) (*chainhash.Hash, error)
// described about for a transaction index entry. The target byte slice must
// be at least large enough to handle the number of bytes defined by the
// txEntrySize constant or it will panic.
func putTxIndexEntry(target []byte, blockID uint32, txLoc wire.TxLoc) {
func putTxIndexEntry(target []byte, blockID uint32, txLoc wire.TxLoc, blockIndex uint32) {
byteOrder.PutUint32(target, blockID)
byteOrder.PutUint32(target[4:], uint32(txLoc.TxStart))
byteOrder.PutUint32(target[8:], uint32(txLoc.TxLen))
byteOrder.PutUint32(target[12:], blockIndex)
}
// dbPutTxIndexEntry uses an existing database transaction to update the
@ -187,7 +204,7 @@ func dbPutTxIndexEntry(dbTx database.Tx, txHash *chainhash.Hash, serializedData
// region for the provided transaction hash from the transaction index. When
// there is no entry for the provided hash, nil will be returned for the both
// the region and the error.
func dbFetchTxIndexEntry(dbTx database.Tx, txHash *chainhash.Hash) (*database.BlockRegion, error) {
func dbFetchTxIndexEntry(dbTx database.Tx, txHash *chainhash.Hash) (*TxIndexEntry, error) {
// Load the record from the database and return now if it doesn't exist.
txIndex := dbTx.Metadata().Bucket(txIndexKey)
serializedData := txIndex.Get(txHash[:])
@ -196,7 +213,7 @@ func dbFetchTxIndexEntry(dbTx database.Tx, txHash *chainhash.Hash) (*database.Bl
}
// Ensure the serialized data has enough bytes to properly deserialize.
if len(serializedData) < 12 {
if len(serializedData) < txEntrySize {
return nil, database.Error{
ErrorCode: database.ErrCorruption,
Description: fmt.Sprintf("corrupt transaction index "+
@ -215,18 +232,28 @@ func dbFetchTxIndexEntry(dbTx database.Tx, txHash *chainhash.Hash) (*database.Bl
}
// Deserialize the final entry.
region := database.BlockRegion{Hash: &chainhash.Hash{}}
copy(region.Hash[:], hash[:])
region.Offset = byteOrder.Uint32(serializedData[4:8])
region.Len = byteOrder.Uint32(serializedData[8:12])
return &region, nil
entry := TxIndexEntry{
BlockRegion: database.BlockRegion{
Hash: new(chainhash.Hash),
Offset: byteOrder.Uint32(serializedData[4:8]),
Len: byteOrder.Uint32(serializedData[8:12]),
},
BlockIndex: byteOrder.Uint32(serializedData[12:16]),
}
copy(entry.BlockRegion.Hash[:], hash[:])
return &entry, nil
}
// dbAddTxIndexEntries uses an existing database transaction to add a
// transaction index entry for every transaction in the parent of the passed
// block (if they were valid) and every stake transaction in the passed block.
func dbAddTxIndexEntries(dbTx database.Tx, block, parent *dcrutil.Block, blockID uint32) error {
func dbAddTxIndexEntries(dbTx database.Tx, block *dcrutil.Block, blockID uint32) error {
// The offset and length of the transactions within the serialized block.
txLocs, stakeTxLocs, err := block.TxLoc()
if err != nil {
return err
}
// As an optimization, allocate a single slice big enough to hold all
// of the serialized transaction index entries for the block and
// serialize them directly into the slice. Then, pass the appropriate
@ -236,8 +263,8 @@ func dbAddTxIndexEntries(dbTx database.Tx, block, parent *dcrutil.Block, blockID
offset := 0
serializedValues := make([]byte, len(txns)*txEntrySize)
for i, tx := range txns {
putTxIndexEntry(serializedValues[offset:], blockID,
txLocs[i])
putTxIndexEntry(serializedValues[offset:], blockID, txLocs[i],
uint32(i))
endOffset := offset + txEntrySize
err := dbPutTxIndexEntry(dbTx, tx.Hash(),
serializedValues[offset:endOffset:endOffset])
@ -249,34 +276,13 @@ func dbAddTxIndexEntries(dbTx database.Tx, block, parent *dcrutil.Block, blockID
return nil
}
// Add the regular transactions of the parent if voted valid.
if approvesParent(block) && block.Height() > 1 {
// The offset and length of the transactions within the
// serialized parent block.
txLocs, _, err := parent.TxLoc()
if err != nil {
return err
}
parentBlockID, err := dbFetchBlockIDByHash(dbTx, parent.Hash())
if err != nil {
return err
}
err = addEntries(parent.Transactions(), txLocs, parentBlockID)
if err != nil {
return err
}
}
// Add the stake transactions of the current block.
//
// The offset and length of the stake transactions within the serialized
// block.
_, stakeTxLocs, err := block.TxLoc()
// Add the regular tree transactions.
err = addEntries(block.Transactions(), txLocs, blockID)
if err != nil {
return err
}
// Add the stake tree transactions.
return addEntries(block.STransactions(), stakeTxLocs, blockID)
}
@ -296,7 +302,7 @@ func dbRemoveTxIndexEntry(dbTx database.Tx, txHash *chainhash.Hash) error {
// dbRemoveTxIndexEntries uses an existing database transaction to remove the
// latest transaction entry for every transaction in the parent of the passed
// block (if they were valid) and every stake transaction in the passed block.
func dbRemoveTxIndexEntries(dbTx database.Tx, block, parent *dcrutil.Block) error {
func dbRemoveTxIndexEntries(dbTx database.Tx, block *dcrutil.Block) error {
removeEntries := func(txns []*dcrutil.Tx) error {
for _, tx := range txns {
err := dbRemoveTxIndexEntry(dbTx, tx.Hash())
@ -307,14 +313,11 @@ func dbRemoveTxIndexEntries(dbTx database.Tx, block, parent *dcrutil.Block) erro
return nil
}
// Remove all of the regular transactions of the parent if voted valid.
if approvesParent(block) && block.Height() > 1 {
if err := removeEntries(parent.Transactions()); err != nil {
return err
}
// Remove the regular and stake tree transactions from the block being
// disconnected.
if err := removeEntries(block.Transactions()); err != nil {
return err
}
// Remove the stake transactions of the block being disconnected.
return removeEntries(block.STransactions())
}
@ -428,10 +431,26 @@ func (idx *TxIndex) Create(dbTx database.Tx) error {
//
// This is part of the Indexer interface.
func (idx *TxIndex) ConnectBlock(dbTx database.Tx, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) error {
// NOTE: The fact that the block can disapprove the regular tree of the
// previous block is ignored for this index because even though the
// disapproved transactions no longer apply spend semantics, they still
// exist within the block and thus have to be processed before the next
// block disapproves them.
//
// Also, the transaction index is keyed by hash and only supports a single
// transaction per hash. This means that if the disapproved transaction
// is mined into a later block, as is typically the case, only that most
// recent one can be queried. Ideally, it should probably support multiple
// transactions per hash, which would not only allow access in the case
// just described, but it would also allow indexing of transactions that
// happen to have the same hash (granted the probability of this is
// extremely low), which is supported so long as the previous one is
// fully spent.
// Increment the internal block ID to use for the block being connected
// and add all of the transactions in the block to the index.
newBlockID := idx.curBlockID + 1
if err := dbAddTxIndexEntries(dbTx, block, parent, newBlockID); err != nil {
if err := dbAddTxIndexEntries(dbTx, block, newBlockID); err != nil {
return err
}
@ -451,8 +470,13 @@ func (idx *TxIndex) ConnectBlock(dbTx database.Tx, block, parent *dcrutil.Block,
//
// This is part of the Indexer interface.
func (idx *TxIndex) DisconnectBlock(dbTx database.Tx, block, parent *dcrutil.Block, view *blockchain.UtxoViewpoint) error {
// NOTE: The fact that the block can disapprove the regular tree of the
// previous block is ignored when disconnecting blocks because it is also
// ignored when connecting the block. See the comments in ConnectBlock for
// the specifics.
// Remove all of the transactions in the block from the index.
if err := dbRemoveTxIndexEntries(dbTx, block, parent); err != nil {
if err := dbRemoveTxIndexEntries(dbTx, block); err != nil {
return err
}
@ -465,20 +489,20 @@ func (idx *TxIndex) DisconnectBlock(dbTx database.Tx, block, parent *dcrutil.Blo
return nil
}
// TxBlockRegion returns the block region for the provided transaction hash
// from the transaction index. The block region can in turn be used to load the
// raw transaction bytes. When there is no entry for the provided hash, nil
// Entry returns details for the provided transaction hash from the transaction
// index. The block region contained in the result can in turn be used to load
// the raw transaction bytes. When there is no entry for the provided hash, nil
// will be returned for the both the entry and the error.
//
// This function is safe for concurrent access.
func (idx *TxIndex) TxBlockRegion(hash chainhash.Hash) (*database.BlockRegion, error) {
var region *database.BlockRegion
func (idx *TxIndex) Entry(hash *chainhash.Hash) (*TxIndexEntry, error) {
var entry *TxIndexEntry
err := idx.db.View(func(dbTx database.Tx) error {
var err error
region, err = dbFetchTxIndexEntry(dbTx, &hash)
entry, err = dbFetchTxIndexEntry(dbTx, hash)
return err
})
return region, err
return entry, err
}
// NewTxIndex returns a new instance of an indexer that is used to create a

8
blockchain/internal/progresslog/blocklogger.go
View File

@ -11,7 +11,6 @@ import (
"github.com/decred/slog"
"github.com/decred/dcrd/dcrutil"
"github.com/decred/dcrd/wire"
)
@ -46,12 +45,9 @@ func NewBlockProgressLogger(progressMessage string, logger slog.Logger) *BlockPr
func (b *BlockProgressLogger) LogBlockHeight(block, parent *wire.MsgBlock) {
b.Lock()
defer b.Unlock()
b.receivedLogBlocks++
regularTxTreeValid := dcrutil.IsFlagSet16(block.Header.VoteBits,
dcrutil.BlockValid)
if regularTxTreeValid {
b.receivedLogTx += int64(len(parent.Transactions))
}
b.receivedLogTx += int64(len(block.Transactions))
b.receivedLogTx += int64(len(block.STransactions))
now := time.Now()

6
blockchain/upgrade.go
View File

@ -544,5 +544,11 @@ func upgradeDB(db database.DB, chainParams *chaincfg.Params, dbInfo *databaseInf
// quickly at startup on the block nodes in memory without requiring a
// database version bump.
// TODO(davec): Replace with proper upgrade code for utxo set semantics
// reversal and index updates.
if dbInfo.version == 4 {
return errors.New("Upgrade from version 4 database not supported yet")
}
return nil
}

975
blockchain/utxoviewpoint.go
View File

File diff suppressed because it is too large Load Diff

229
blockchain/validate.go
View File

@ -98,8 +98,8 @@ func voteBitsApproveParent(voteBits uint16) bool {
return dcrutil.IsFlagSet16(voteBits, dcrutil.BlockValid)
}
// approvesParent returns whether or not the vote bits in the passed header
// indicate the regular transaction tree of the parent block should be
// headerApprovesParent returns whether or not the vote bits in the passed
// header indicate the regular transaction tree of the parent block should be
// considered valid.
func headerApprovesParent(header *wire.BlockHeader) bool {
return voteBitsApproveParent(header.VoteBits)
@ -1416,11 +1416,11 @@ func (b *BlockChain) checkDupTxs(txSet []*dcrutil.Tx, view *UtxoViewpoint) error
// Fetch utxo details for all of the transactions in this block.
// Typically, there will not be any utxos for any of the transactions.
fetchSet := make(map[chainhash.Hash]struct{})
filteredSet := make(viewFilteredSet)
for _, tx := range txSet {
fetchSet[*tx.Hash()] = struct{}{}
filteredSet.add(view, tx.Hash())
}
err := view.fetchUtxos(b.db, fetchSet)
err := view.fetchUtxosMain(b.db, filteredSet)
if err != nil {
return err
}
@ -2383,7 +2383,7 @@ func CountSigOps(tx *dcrutil.Tx, isCoinBaseTx bool, isSSGen bool) int {
// transactions which are of the pay-to-script-hash type. This uses the
// precise, signature operation counting mechanism from the script engine which
// requires access to the input transaction scripts.
func CountP2SHSigOps(tx *dcrutil.Tx, isCoinBaseTx bool, isStakeBaseTx bool, utxoView *UtxoViewpoint) (int, error) {
func CountP2SHSigOps(tx *dcrutil.Tx, isCoinBaseTx bool, isStakeBaseTx bool, view *UtxoViewpoint) (int, error) {
// Coinbase transactions have no interesting inputs.
if isCoinBaseTx {
return 0, nil
@ -2403,7 +2403,7 @@ func CountP2SHSigOps(tx *dcrutil.Tx, isCoinBaseTx bool, isStakeBaseTx bool, utxo
// Ensure the referenced input transaction is available.
originTxHash := &txIn.PreviousOutPoint.Hash
originTxIndex := txIn.PreviousOutPoint.Index
utxoEntry := utxoView.LookupEntry(originTxHash)
utxoEntry := view.LookupEntry(originTxHash)
if utxoEntry == nil || utxoEntry.IsOutputSpent(originTxIndex) {
str := fmt.Sprintf("output %v referenced from "+
"transaction %s:%d either does not exist or "+
@ -2444,7 +2444,7 @@ func CountP2SHSigOps(tx *dcrutil.Tx, isCoinBaseTx bool, isStakeBaseTx bool, utxo
// sure they don't overflow the limits. It takes a cumulative number of sig
// ops as an argument and increments will each call.
// TxTree true == Regular, false == Stake
func checkNumSigOps(tx *dcrutil.Tx, utxoView *UtxoViewpoint, index int, txTree bool, cumulativeSigOps int) (int, error) {
func checkNumSigOps(tx *dcrutil.Tx, view *UtxoViewpoint, index int, txTree bool, cumulativeSigOps int) (int, error) {
msgTx := tx.MsgTx()
isSSGen := stake.IsSSGen(msgTx)
numsigOps := CountSigOps(tx, (index == 0) && txTree, isSSGen)
@ -2455,7 +2455,7 @@ func checkNumSigOps(tx *dcrutil.Tx, utxoView *UtxoViewpoint, index int, txTree b
// transaction is a coinbase transaction rather than having to do a
// full coinbase check again.
numP2SHSigOps, err := CountP2SHSigOps(tx, (index == 0) && txTree,
isSSGen, utxoView)
isSSGen, view)
if err != nil {
log.Tracef("CountP2SHSigOps failed; error returned %v", err)
return 0, err
@ -2481,13 +2481,13 @@ func checkNumSigOps(tx *dcrutil.Tx, utxoView *UtxoViewpoint, index int, txTree b
// checkStakeBaseAmounts calculates the total amount given as subsidy from
// single stakebase transactions (votes) within a block. This function skips a
// ton of checks already performed by CheckTransactionInputs.
func checkStakeBaseAmounts(subsidyCache *SubsidyCache, height int64, params *chaincfg.Params, txs []*dcrutil.Tx, utxoView *UtxoViewpoint) error {
func checkStakeBaseAmounts(subsidyCache *SubsidyCache, height int64, params *chaincfg.Params, txs []*dcrutil.Tx, view *UtxoViewpoint) error {
for _, tx := range txs {
msgTx := tx.MsgTx()
if stake.IsSSGen(msgTx) {
// Ensure the input is available.
txInHash := &msgTx.TxIn[1].PreviousOutPoint.Hash
utxoEntry, exists := utxoView.entries[*txInHash]
utxoEntry, exists := view.entries[*txInHash]
if !exists || utxoEntry == nil {
str := fmt.Sprintf("couldn't find input tx %v "+
"for stakebase amounts check", txInHash)
@ -2525,7 +2525,7 @@ func checkStakeBaseAmounts(subsidyCache *SubsidyCache, height int64, params *cha
// getStakeBaseAmounts calculates the total amount given as subsidy from the
// collective stakebase transactions (votes) within a block. This function
// skips a ton of checks already performed by CheckTransactionInputs.
func getStakeBaseAmounts(txs []*dcrutil.Tx, utxoView *UtxoViewpoint) (int64, error) {
func getStakeBaseAmounts(txs []*dcrutil.Tx, view *UtxoViewpoint) (int64, error) {
totalInputs := int64(0)
totalOutputs := int64(0)
for _, tx := range txs {
@ -2533,7 +2533,7 @@ func getStakeBaseAmounts(txs []*dcrutil.Tx, utxoView *UtxoViewpoint) (int64, err
if stake.IsSSGen(msgTx) {
// Ensure the input is available.
txInHash := &msgTx.TxIn[1].PreviousOutPoint.Hash
utxoEntry, exists := utxoView.entries[*txInHash]
utxoEntry, exists := view.entries[*txInHash]
if !exists || utxoEntry == nil {
str := fmt.Sprintf("couldn't find input tx %v "+
"for stakebase amounts get", txInHash)
@ -2557,7 +2557,7 @@ func getStakeBaseAmounts(txs []*dcrutil.Tx, utxoView *UtxoViewpoint) (int64, err
// getStakeTreeFees determines the amount of fees for in the stake tx tree of
// some node given a transaction store.
func getStakeTreeFees(subsidyCache *SubsidyCache, height int64, params *chaincfg.Params, txs []*dcrutil.Tx, utxoView *UtxoViewpoint) (dcrutil.Amount, error) {
func getStakeTreeFees(subsidyCache *SubsidyCache, height int64, params *chaincfg.Params, txs []*dcrutil.Tx, view *UtxoViewpoint) (dcrutil.Amount, error) {
totalInputs := int64(0)
totalOutputs := int64(0)
for _, tx := range txs {
@ -2571,7 +2571,7 @@ func getStakeTreeFees(subsidyCache *SubsidyCache, height int64, params *chaincfg
}
txInHash := &in.PreviousOutPoint.Hash
utxoEntry, exists := utxoView.entries[*txInHash]
utxoEntry, exists := view.entries[*txInHash]
if !exists || utxoEntry == nil {
str := fmt.Sprintf("couldn't find input tx "+
"%v for stake tree fee calculation",
@ -2611,7 +2611,7 @@ func getStakeTreeFees(subsidyCache *SubsidyCache, height int64, params *chaincfg
// transaction inputs for a transaction list given a predetermined TxStore.
// After ensuring the transaction is valid, the transaction is connected to the
// UTXO viewpoint. TxTree true == Regular, false == Stake
func (b *BlockChain) checkTransactionsAndConnect(subsidyCache *SubsidyCache, inputFees dcrutil.Amount, node *blockNode, txs []*dcrutil.Tx, utxoView *UtxoViewpoint, stxos *[]spentTxOut, txTree bool) error {
func (b *BlockChain) checkTransactionsAndConnect(subsidyCache *SubsidyCache, inputFees dcrutil.Amount, node *blockNode, txs []*dcrutil.Tx, view *UtxoViewpoint, stxos *[]spentTxOut, txTree bool) error {
// Perform several checks on the inputs for each transaction. Also
// accumulate the total fees. This could technically be combined with
// the loop above instead of running another loop over the
@ -2625,8 +2625,8 @@ func (b *BlockChain) checkTransactionsAndConnect(subsidyCache *SubsidyCache, inp
// Ensure that the number of signature operations is not beyond
// the consensus limit.
var err error
cumulativeSigOps, err = checkNumSigOps(tx, utxoView, idx,
txTree, cumulativeSigOps)
cumulativeSigOps, err = checkNumSigOps(tx, view, idx, txTree,
cumulativeSigOps)
if err != nil {
return err
}
@ -2634,7 +2634,7 @@ func (b *BlockChain) checkTransactionsAndConnect(subsidyCache *SubsidyCache, inp
// This step modifies the txStore and marks the tx outs used
// spent, so be aware of this.
txFee, err := CheckTransactionInputs(b.subsidyCache, tx,
node.height, utxoView, true, /* check fraud proofs */
node.height, view, true, /* check fraud proofs */
b.chainParams)
if err != nil {
log.Tracef("CheckTransactionInputs failed; error "+
@ -2653,7 +2653,7 @@ func (b *BlockChain) checkTransactionsAndConnect(subsidyCache *SubsidyCache, inp
// Connect the transaction to the UTXO viewpoint, so that in
// flight transactions may correctly validate.
err = utxoView.connectTransaction(tx, node.height, uint32(idx),
err = view.connectTransaction(tx, node.height, uint32(idx),
stxos)
if err != nil {
return err
@ -2720,12 +2720,12 @@ func (b *BlockChain) checkTransactionsAndConnect(subsidyCache *SubsidyCache, inp
}
err := checkStakeBaseAmounts(subsidyCache, node.height,
b.chainParams, txs, utxoView)
b.chainParams, txs, view)
if err != nil {
return err
}
totalAtomOutStake, err := getStakeBaseAmounts(txs, utxoView)
totalAtomOutStake, err := getStakeBaseAmounts(txs, view)
if err != nil {
return err
}
@ -2791,7 +2791,7 @@ func (b *BlockChain) consensusScriptVerifyFlags(node *blockNode) (txscript.Scrip
// the bulk of its work.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.Block, utxoView *UtxoViewpoint, stxos *[]spentTxOut) error {
func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.Block, view *UtxoViewpoint, stxos *[]spentTxOut) error {
// If the side chain blocks end up in the database, a call to
// CheckBlockSanity should be done here in case a previous version
// allowed a block that is no longer valid. However, since the
@ -2800,10 +2800,10 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
// Ensure the view is for the node being checked.
parentHash := &block.MsgBlock().Header.PrevBlock
if !utxoView.BestHash().IsEqual(parentHash) {
if !view.BestHash().IsEqual(parentHash) {
return AssertError(fmt.Sprintf("inconsistent view when "+
"checking block connection: best hash is %v instead "+
"of expected %v", utxoView.BestHash(), parentHash))
"of expected %v", view.BestHash(), parentHash))
}
// Check that the coinbase pays the tax, if applicable.
@ -2833,50 +2833,35 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
}
}
// The number of signature operations must be less than the maximum
// allowed per block. Note that the preliminary sanity checks on a
// block also include a check similar to this one, but this check
// expands the count to include a precise count of pay-to-script-hash
// signature operations in each of the input transaction public key
// scripts.
// Do this for all TxTrees.
regularTxTreeValid := voteBitsApproveParent(node.voteBits)
thisNodeStakeViewpoint := ViewpointPrevInvalidStake
thisNodeRegularViewpoint := ViewpointPrevInvalidRegular
if regularTxTreeValid {
thisNodeStakeViewpoint = ViewpointPrevValidStake
thisNodeRegularViewpoint = ViewpointPrevValidRegular
utxoView.SetStakeViewpoint(ViewpointPrevValidInitial)
err = utxoView.fetchInputUtxos(b.db, block, parent)
// Disconnect all of the transactions in the regular transaction tree of
// the parent if the block being checked votes against it.
if node.height > 1 && !voteBitsApproveParent(node.voteBits) {
err := view.disconnectDisapprovedBlock(b.db, parent)
if err != nil {
return err
}
for i, tx := range parent.Transactions() {
err := utxoView.connectTransaction(tx,
node.parent.height, uint32(i), stxos)
if err != nil {
return err
}
}
}
// TxTreeStake of current block.
utxoView.SetStakeViewpoint(thisNodeStakeViewpoint)
err = b.checkDupTxs(block.STransactions(), utxoView)
// Ensure the stake transaction tree does not contain any transactions
// that 'overwrite' older transactions which are not fully spent.
err = b.checkDupTxs(block.STransactions(), view)
if err != nil {
log.Tracef("checkDupTxs failed for cur TxTreeStake: %v", err)
return err
}
err = utxoView.fetchInputUtxos(b.db, block, parent)
// Load all of the utxos referenced by the inputs for all transactions
// in the block don't already exist in the utxo view from the database.
//
// These utxo entries are needed for verification of things such as
// transaction inputs, counting pay-to-script-hashes, and scripts.
err = view.fetchInputUtxos(b.db, block)
if err != nil {
return err
}
err = b.checkTransactionsAndConnect(b.subsidyCache, 0, node,
block.STransactions(), utxoView, stxos, false)
block.STransactions(), view, stxos, false)
if err != nil {
log.Tracef("checkTransactionsAndConnect failed for "+
"TxTreeStake: %v", err)
@ -2884,13 +2869,13 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
}
stakeTreeFees, err := getStakeTreeFees(b.subsidyCache, node.height,
b.chainParams, block.STransactions(), utxoView)
b.chainParams, block.STransactions(), view)
if err != nil {
log.Tracef("getStakeTreeFees failed for TxTreeStake: %v", err)
return err
}
// Enforce all relative lock times via sequence numbers for the regular
// Enforce all relative lock times via sequence numbers for the stake
// transaction tree once the stake vote for the agenda is active.
var prevMedianTime time.Time
lnFeaturesActive, err := b.isLNFeaturesAgendaActive(node.parent)
@ -2903,64 +2888,9 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
// final.
prevMedianTime = node.parent.CalcPastMedianTime()
// Skip the coinbase since it does not have any inputs and thus
// lock times do not apply.
for _, tx := range block.Transactions()[1:] {
sequenceLock, err := b.calcSequenceLock(node, tx,
utxoView, true)
if err != nil {
return err
}
if !SequenceLockActive(sequenceLock, node.height,
prevMedianTime) {
str := fmt.Sprintf("block contains " +
"transaction whose input sequence " +
"locks are not met")
return ruleError(ErrUnfinalizedTx, str)
}
}
}
if runScripts {
err = checkBlockScripts(block, utxoView, false, scriptFlags,
b.sigCache)
if err != nil {
log.Tracef("checkBlockScripts failed; error returned "+
"on txtreestake of cur block: %v", err)
return err
}
}
// TxTreeRegular of current block. At this point, the stake
// transactions have already added, so set this to the correct stake
// viewpoint and disable automatic connection.
utxoView.SetStakeViewpoint(thisNodeRegularViewpoint)
err = b.checkDupTxs(block.Transactions(), utxoView)
if err != nil {
log.Tracef("checkDupTxs failed for cur TxTreeRegular: %v", err)
return err
}
err = utxoView.fetchInputUtxos(b.db, block, parent)
if err != nil {
return err
}
err = b.checkTransactionsAndConnect(b.subsidyCache, stakeTreeFees, node,
block.Transactions(), utxoView, stxos, true)
if err != nil {
log.Tracef("checkTransactionsAndConnect failed for cur "+
"TxTreeRegular: %v", err)
return err
}
// Enforce all relative lock times via sequence numbers for the stake
// transaction tree once the stake vote for the agenda is active.
if lnFeaturesActive {
for _, stx := range block.STransactions() {
sequenceLock, err := b.calcSequenceLock(node, stx,
utxoView, true)
view, true)
if err != nil {
return err
}
@ -2976,8 +2906,56 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
}
if runScripts {
err = checkBlockScripts(block, utxoView, true,
scriptFlags, b.sigCache)
err = checkBlockScripts(block, view, false, scriptFlags,
b.sigCache)
if err != nil {
log.Tracef("checkBlockScripts failed; error returned "+
"on txtreestake of cur block: %v", err)
return err
}
}
// Ensure the regular transaction tree does not contain any transactions
// that 'overwrite' older transactions which are not fully spent.
err = b.checkDupTxs(block.Transactions(), view)
if err != nil {
log.Tracef("checkDupTxs failed for cur TxTreeRegular: %v", err)
return err
}
err = b.checkTransactionsAndConnect(b.subsidyCache, stakeTreeFees, node,
block.Transactions(), view, stxos, true)
if err != nil {
log.Tracef("checkTransactionsAndConnect failed for cur "+
"TxTreeRegular: %v", err)
return err
}
// Enforce all relative lock times via sequence numbers for the regular
// transaction tree once the stake vote for the agenda is active.
if lnFeaturesActive {
// Skip the coinbase since it does not have any inputs and thus
// lock times do not apply.
for _, tx := range block.Transactions()[1:] {
sequenceLock, err := b.calcSequenceLock(node, tx,
view, true)
if err != nil {
return err
}
if !SequenceLockActive(sequenceLock, node.height,
prevMedianTime) {
str := fmt.Sprintf("block contains " +
"transaction whose input sequence " +
"locks are not met")
return ruleError(ErrUnfinalizedTx, str)
}
}
}
if runScripts {
err = checkBlockScripts(block, view, true, scriptFlags,
b.sigCache)
if err != nil {
log.Tracef("checkBlockScripts failed; error returned "+
"on txtreeregular of cur block: %v", err)
@ -2985,18 +2963,6 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
}
}
// Rollback the final tx tree regular so that we don't write it to
// database.
if node.height > 1 && stxos != nil {
idx, err := utxoView.disconnectTransactionSlice(block.Transactions(),
node.height, stxos)
if err != nil {
return err
}
stxosDeref := *stxos
*stxos = stxosDeref[0:idx]
}
// First block has special rules concerning the ledger.
if node.height == 1 {
err := BlockOneCoinbasePaysTokens(block.Transactions()[0],
@ -3008,7 +2974,7 @@ func (b *BlockChain) checkConnectBlock(node *blockNode, block, parent *dcrutil.B
// Update the best hash for view to include this block since all of its
// transactions have been connected.
utxoView.SetBestHash(&node.hash)
view.SetBestHash(&node.hash)
return nil
}
@ -3084,6 +3050,7 @@ func (b *BlockChain) CheckConnectBlockTemplate(block *dcrutil.Block) error {
view := NewUtxoViewpoint()
view.SetBestHash(&tip.hash)
return b.checkConnectBlock(newNode, block, parent, view, nil)
}
@ -3093,7 +3060,6 @@ func (b *BlockChain) CheckConnectBlockTemplate(block *dcrutil.Block) error {
// block template.
view := NewUtxoViewpoint()
view.SetBestHash(&tip.hash)
view.SetStakeViewpoint(ViewpointPrevValidInitial)
tipBlock, err := b.fetchMainChainBlockByNode(tip)
if err != nil {
return err
@ -3106,7 +3072,7 @@ func (b *BlockChain) CheckConnectBlockTemplate(block *dcrutil.Block) error {
// Load all of the spent txos for the tip block from the spend journal.
var stxos []spentTxOut
err = b.db.View(func(dbTx database.Tx) error {
stxos, err = dbFetchSpendJournalEntry(dbTx, tipBlock, parent)
stxos, err = dbFetchSpendJournalEntry(dbTx, tipBlock)
return err
})
if err != nil {
@ -3114,8 +3080,9 @@ func (b *BlockChain) CheckConnectBlockTemplate(block *dcrutil.Block) error {
}
// Update the view to unspend all of the spent txos and remove the utxos
// created by the tip block.
err = b.disconnectTransactions(view, tipBlock, parent, stxos)
// created by the tip block. Also, if the block votes against its parent,
// reconnect all of the regular transactions.
err = view.disconnectBlock(b.db, tipBlock, parent, stxos)
if err != nil {
return err
}

17
blockchain/validate_test.go
View File

@ -96,16 +96,6 @@ func TestBlockchainSpendJournal(t *testing.T) {
// Loop through all of the blocks and ensure the number of spent outputs
// matches up with the information loaded from the spend journal.
err = chain.db.View(func(dbTx database.Tx) error {
parentNode := chain.bestChain.NodeByHeight(1)
if parentNode == nil {
str := fmt.Sprintf("no block at height %d exists", 1)
return errNotInMainChain(str)
}
parent, err := dbFetchBlockByNode(dbTx, parentNode)
if err != nil {
return err
}
for i := int64(2); i <= chain.bestChain.Tip().height; i++ {
node := chain.bestChain.NodeByHeight(i)
if node == nil {
@ -117,9 +107,8 @@ func TestBlockchainSpendJournal(t *testing.T) {
return err
}
ntx := countSpentOutputs(block, parent)
stxos, err := dbFetchSpendJournalEntry(dbTx, block,
parent)
ntx := countSpentOutputs(block)
stxos, err := dbFetchSpendJournalEntry(dbTx, block)
if err != nil {
return err
}
@ -129,8 +118,6 @@ func TestBlockchainSpendJournal(t *testing.T) {
"calculated at "+"height %v, got %v "+
"expected %v", i, len(stxos), ntx)
}
parent = block
}
return nil

218
blockmanager.go
View File

@ -1658,6 +1658,35 @@ func (b *blockManager) notifiedWinningTickets(hash *chainhash.Hash) bool {
return beenNotified
}
// headerApprovesParent returns whether or not the vote bits in the passed
// header indicate the regular transaction tree of the parent block should be
// considered valid.
func headerApprovesParent(header *wire.BlockHeader) bool {
return dcrutil.IsFlagSet16(header.VoteBits, dcrutil.BlockValid)
}
// isDoubleSpendOrDuplicateError returns whether or not the passed error, which
// is expected to have come from mempool, indicates a transaction was rejected
// either due to containing a double spend or already existing in the pool.
func isDoubleSpendOrDuplicateError(err error) bool {
merr, ok := err.(mempool.RuleError)
if !ok {
return false
}
rerr, ok := merr.Err.(mempool.TxRuleError)
if ok && rerr.RejectCode == wire.RejectDuplicate {
return true
}
cerr, ok := merr.Err.(blockchain.RuleError)
if ok && cerr.ErrorCode == blockchain.ErrMissingTxOut {
return true
}
return false
}
// handleNotifyMsg handles notifications from blockchain. It does things such
// as request orphan block parents and relay accepted blocks to connected peers.
func (b *blockManager) handleNotifyMsg(notification *blockchain.Notification) {
@ -1785,56 +1814,81 @@ func (b *blockManager) handleNotifyMsg(notification *blockchain.Notification) {
block := blockSlice[0]
parentBlock := blockSlice[1]
// Check and see if the regular tx tree of the previous block was
// invalid or not. If it wasn't, then we need to restore all the tx
// from this block into the mempool. They may end up being spent in
// the regular tx tree of the current block, for which there is code
// below.
txTreeRegularValid := dcrutil.IsFlagSet16(block.MsgBlock().Header.VoteBits,
dcrutil.BlockValid)
// TODO: In the case the new tip disapproves the previous block, any
// transactions the previous block contains in its regular tree which
// double spend the same inputs as transactions in either tree of the
// current tip should ideally be tracked in the pool as eligibile for
// inclusion in an alternative tip (side chain block) in case the
// current tip block does not get enough votes. However, the
// transaction pool currently does not provide any way to distinguish
// this condition and thus only provides tracking based on the current
// tip. In order to handle this condition, the pool would have to
// provide a way to track and independently query which txns are
// eligible based on the current tip both approving and disapproving the
// previous block as well as the previous block itself.
// Remove all of the regular and stake transactions in the
// connected block from the transaction pool. Also, remove any
// transactions which are now double spends as a result of these
// new transactions. Finally, remove any transaction that is
// no longer an orphan. Transactions which depend on a confirmed
// transaction are NOT removed recursively because they are still
// valid. Also, the coinbase of the regular tx tree is skipped
// because the memory pool doesn't (and can't) have regular
// tree coinbase transactions in it.
if txTreeRegularValid {
for _, tx := range parentBlock.Transactions()[1:] {
b.server.txMemPool.RemoveTransaction(tx, false)
b.server.txMemPool.RemoveDoubleSpends(tx)
b.server.txMemPool.RemoveOrphan(tx)
acceptedTxs := b.server.txMemPool.ProcessOrphans(tx)
// Remove all of the regular and stake transactions in the connected
// block from the transaction pool. Also, remove any transactions which
// are now double spends as a result of these new transactions.
// Finally, remove any transaction that is no longer an orphan.
// Transactions which depend on a confirmed transaction are NOT removed
// recursively because they are still valid. Also, the coinbase of the
// regular tx tree is skipped because the transaction pool doesn't (and
// can't) have regular tree coinbase transactions in it.
//
// Also, in the case the RPC server is enabled, stop rebroadcasting any
// transactions in the block that were setup to be rebroadcast.
txMemPool := b.server.txMemPool
handleConnectedBlockTxns := func(txns []*dcrutil.Tx) {
for _, tx := range txns {
txMemPool.RemoveTransaction(tx, false)
txMemPool.RemoveDoubleSpends(tx)
txMemPool.RemoveOrphan(tx)
acceptedTxs := txMemPool.ProcessOrphans(tx)
b.server.AnnounceNewTransactions(acceptedTxs)
}
}
for _, stx := range block.STransactions()[0:] {
b.server.txMemPool.RemoveTransaction(stx, false)
b.server.txMemPool.RemoveDoubleSpends(stx)
b.server.txMemPool.RemoveOrphan(stx)
acceptedTxs := b.server.txMemPool.ProcessOrphans(stx)
b.server.AnnounceNewTransactions(acceptedTxs)
}
if r := b.server.rpcServer; r != nil {
// Now that this block is in the blockchain we can mark
// all the transactions (except the coinbase) as no
// longer needing rebroadcasting.
if txTreeRegularValid {
for _, tx := range parentBlock.Transactions()[1:] {
// Now that this block is in the blockchain, mark the
// transaction (except the coinbase) as no longer needing
// rebroadcasting.
if b.server.rpcServer != nil {
iv := wire.NewInvVect(wire.InvTypeTx, tx.Hash())
b.server.RemoveRebroadcastInventory(iv)
}
}
for _, stx := range block.STransactions()[0:] {
iv := wire.NewInvVect(wire.InvTypeTx, stx.Hash())
b.server.RemoveRebroadcastInventory(iv)
}
}
handleConnectedBlockTxns(block.Transactions()[1:])
handleConnectedBlockTxns(block.STransactions())
// In the case the regular tree of the previous block was disapproved,
// add all of the its transactions, with the exception of the coinbase,
// back to the transaction pool to be mined in a future block.
//
// Notice that some of those transactions might have been included in
// the current block and others might also be spending some of the same
// outputs that transactions in the previous originally block spent.
// This is the expected behavior because disapproval of the regular tree
// of the previous block essentially makes it as if those transactions
// never happened.
//
// Finally, if transactions fail to add to the pool for some reason
// other than the pool already having it (a duplicate) or now being a
// double spend, remove all transactions that depend on it as well.
// The dependencies are not removed for double spends because the only
// way a transaction which was not a double spend in the previous block
// to now be one is due to some transaction in the current block
// (probably the same one) also spending those outputs, and, in that
// case, anything that happens to be in the pool which depends on the
// transaction is still valid.
if !headerApprovesParent(&block.MsgBlock().Header) {
for _, tx := range parentBlock.Transactions()[1:] {
_, err := txMemPool.MaybeAcceptTransaction(tx, false, true)
if err != nil && !isDoubleSpendOrDuplicateError(err) {
txMemPool.RemoveTransaction(tx, true)
}
}
}
if r := b.server.rpcServer; r != nil {
// Filter and update the rebroadcast inventory.
b.server.PruneRebroadcastInventory()
@ -1884,48 +1938,60 @@ func (b *blockManager) handleNotifyMsg(notification *blockchain.Notification) {
block := blockSlice[0]
parentBlock := blockSlice[1]
// If the parent tx tree was invalidated, we need to remove these
// tx from the mempool as the next incoming block may alternatively
// validate them.
txTreeRegularValid := dcrutil.IsFlagSet16(block.MsgBlock().Header.VoteBits,
dcrutil.BlockValid)
if !txTreeRegularValid {
// In the case the regular tree of the previous block was disapproved,
// disconnecting the current block makes all of those transactions valid
// again. Thus, with the exception of the coinbase, remove all of those
// transactions and any that are now double spends from the transaction
// pool. Transactions which depend on a confirmed transaction are NOT
// removed recursively because they are still valid.
txMemPool := b.server.txMemPool
if !headerApprovesParent(&block.MsgBlock().Header) {
for _, tx := range parentBlock.Transactions()[1:] {
b.server.txMemPool.RemoveTransaction(tx, false)
b.server.txMemPool.RemoveDoubleSpends(tx)
b.server.txMemPool.RemoveOrphan(tx)
b.server.txMemPool.ProcessOrphans(tx)
txMemPool.RemoveTransaction(tx, false)
txMemPool.RemoveDoubleSpends(tx)
txMemPool.RemoveOrphan(tx)
txMemPool.ProcessOrphans(tx)
}
}
// Reinsert all of the transactions (except the coinbase) from the parent
// tx tree regular into the transaction pool.
for _, tx := range parentBlock.Transactions()[1:] {
_, err := b.server.txMemPool.MaybeAcceptTransaction(tx, false, true)
if err != nil {
// Remove the transaction and all transactions
// that depend on it if it wasn't accepted into
// the transaction pool.
b.server.txMemPool.RemoveTransaction(tx, true)
// Add all of the regular and stake transactions in the disconnected
// block, with the exception of the regular tree coinbase, back to the
// transaction pool to be mined in a future block.
//
// Notice that, in the case the previous block was disapproved, some of
// the transactions in the block being disconnected might have been
// included in the previous block and others might also have been
// spending some of the same outputs. This is the expected behavior
// because disapproval of the regular tree of the previous block
// essentially makes it as if those transactions never happened, so
// disconnecting the block that disapproved those transactions
// effectively revives them.
//
// Finally, if transactions fail to add to the pool for some reason
// other than the pool already having it (a duplicate) or now being a
// double spend, remove all transactions that depend on it as well.
// The dependencies are not removed for double spends because the only
// way a transaction which was not a double spend in the block being
// disconnected to now be one is due to some transaction in the previous
// block (probably the same one), which was disapproved, also spending
// those outputs, and, in that case, anything that happens to be in the
// pool which depends on the transaction is still valid.
handleDisconnectedBlockTxns := func(txns []*dcrutil.Tx) {
for _, tx := range txns {
_, err := txMemPool.MaybeAcceptTransaction(tx, false, true)
if err != nil && !isDoubleSpendOrDuplicateError(err) {
txMemPool.RemoveTransaction(tx, true)
}
}
}
handleDisconnectedBlockTxns(block.Transactions()[1:])
handleDisconnectedBlockTxns(block.STransactions())
for _, tx := range block.STransactions()[0:] {
_, err := b.server.txMemPool.MaybeAcceptTransaction(tx, false, true)
if err != nil {
// Remove the transaction and all transactions
// that depend on it if it wasn't accepted into
// the transaction pool.
b.server.txMemPool.RemoveTransaction(tx, true)
}
}
// Filter and update the rebroadcast inventory.
b.server.PruneRebroadcastInventory()
// Notify registered websocket clients.
if r := b.server.rpcServer; r != nil {
// Filter and update the rebroadcast inventory.
b.server.PruneRebroadcastInventory()
// Notify registered websocket clients.
r.ntfnMgr.NotifyBlockDisconnected(block)
}

37
mempool/mempool.go
View File

@ -597,7 +597,6 @@ func (mp *TxPool) removeTransaction(tx *dcrutil.Tx, removeRedeemers bool) {
}
// Mark the referenced outpoints as unspent by the pool.
for _, txIn := range txDesc.Tx.MsgTx().TxIn {
delete(mp.outpoints, txIn.PreviousOutPoint)
}
@ -699,23 +698,23 @@ func (mp *TxPool) checkPoolDoubleSpend(tx *dcrutil.Tx, txType stake.TxType) erro
return nil
}
// IsTxTreeKnownInvalid returns whether or not the transaction tree of the
// provided hash is knwon to be invalid according to the votes currently in the
// memory pool.
// IsRegTxTreeKnownDisapproved returns whether or not the regular tree of the
// block represented by the provided hash is known to be disapproved according
// to the votes currently in the memory pool.
//
// The function is safe for concurrent access.
func (mp *TxPool) IsTxTreeKnownInvalid(hash *chainhash.Hash) bool {
func (mp *TxPool) IsRegTxTreeKnownDisapproved(hash *chainhash.Hash) bool {
mp.votesMtx.RLock()
vts := mp.votes[*hash]
mp.votesMtx.RUnlock()
// There are not possibly enough votes to tell if the regular transaction
// tree is valid or not, so assume it's valid.
// tree is approved or not, so assume it's valid.
if len(vts) <= int(mp.cfg.ChainParams.TicketsPerBlock/2) {
return false
}
// Otherwise, tally the votes and determine if it's valid or not.
// Otherwise, tally the votes and determine if it's approved or not.
var yes, no int
for _, vote := range vts {
if vote.ApprovesParent {
@ -735,8 +734,8 @@ func (mp *TxPool) IsTxTreeKnownInvalid(hash *chainhash.Hash) bool {
//
// This function MUST be called with the mempool lock held (for reads).
func (mp *TxPool) fetchInputUtxos(tx *dcrutil.Tx) (*blockchain.UtxoViewpoint, error) {
knownInvalid := mp.IsTxTreeKnownInvalid(mp.cfg.BestHash())
utxoView, err := mp.cfg.FetchUtxoView(tx, !knownInvalid)
knownDisapproved := mp.IsRegTxTreeKnownDisapproved(mp.cfg.BestHash())
utxoView, err := mp.cfg.FetchUtxoView(tx, !knownDisapproved)
if err != nil {
return nil, err
}
@ -761,7 +760,7 @@ func (mp *TxPool) fetchInputUtxos(tx *dcrutil.Tx) (*blockchain.UtxoViewpoint, er
// orphans.
//
// This function is safe for concurrent access.
func (mp *TxPool) FetchTransaction(txHash *chainhash.Hash, includeRecentBlock bool) (*dcrutil.Tx, error) {
func (mp *TxPool) FetchTransaction(txHash *chainhash.Hash) (*dcrutil.Tx, error) {
// Protect concurrent access.
mp.mtx.RLock()
txDesc, exists := mp.pool[*txHash]
@ -771,22 +770,6 @@ func (mp *TxPool) FetchTransaction(txHash *chainhash.Hash, includeRecentBlock bo
return txDesc.Tx, nil
}
// For Decred, the latest block is considered "unconfirmed"
// for the regular transaction tree. Search that if the
// user indicates too, as well.
if includeRecentBlock {
bl, err := mp.cfg.BlockByHash(mp.cfg.BestHash())
if err != nil {
return nil, err
}
for _, tx := range bl.Transactions() {
if tx.Hash().IsEqual(txHash) {
return tx, nil
}
}
}
return nil, fmt.Errorf("transaction is not in the pool")
}
@ -984,7 +967,7 @@ func (mp *TxPool) maybeAcceptTransaction(tx *dcrutil.Tx, isNew, rateLimit, allow
}
// Don't allow the transaction if it exists in the main chain and is not
// not already fully spent.
// already fully spent.
txEntry := utxoView.LookupEntry(txHash)
if txEntry != nil && !txEntry.IsFullySpent() {
return nil, txRuleError(wire.RejectDuplicate,

12
mining.go
View File

@ -1220,7 +1220,7 @@ func (g *BlkTmplGenerator) NewBlockTemplate(payToAddress dcrutil.Address) (*Bloc
minrLog.Debugf("Considering %d transactions for inclusion to new block",
len(sourceTxns))
treeKnownInvalid := g.txSource.IsTxTreeKnownInvalid(&prevHash)
knownDisapproved := g.txSource.IsRegTxTreeKnownDisapproved(&prevHash)
mempoolLoop:
for _, txDesc := range sourceTxns {
@ -1255,7 +1255,7 @@ mempoolLoop:
// NOTE: This intentionally does not fetch inputs from the
// mempool since a transaction which depends on other
// transactions in the mempool must come after those
utxos, err := g.chain.FetchUtxoView(tx, !treeKnownInvalid)
utxos, err := g.chain.FetchUtxoView(tx, !knownDisapproved)
if err != nil {
minrLog.Warnf("Unable to fetch utxo view for tx %s: "+
"%v", tx.Hash(), err)
@ -1597,7 +1597,7 @@ mempoolLoop:
if stake.IsSSGen(msgTx) {
txCopy := dcrutil.NewTxDeepTxIns(msgTx)
if maybeInsertStakeTx(g.blockManager, txCopy, !treeKnownInvalid) {
if maybeInsertStakeTx(g.blockManager, txCopy, !knownDisapproved) {
vb := stake.SSGenVoteBits(txCopy.MsgTx())
voteBitsVoters = append(voteBitsVoters, vb)
blockTxnsStake = append(blockTxnsStake, txCopy)
@ -1701,7 +1701,7 @@ mempoolLoop:
// Quick check for difficulty here.
if msgTx.TxOut[0].Value >= best.NextStakeDiff {
txCopy := dcrutil.NewTxDeepTxIns(msgTx)
if maybeInsertStakeTx(g.blockManager, txCopy, !treeKnownInvalid) {
if maybeInsertStakeTx(g.blockManager, txCopy, !knownDisapproved) {
blockTxnsStake = append(blockTxnsStake, txCopy)
freshStake++
}
@ -1724,7 +1724,7 @@ mempoolLoop:
msgTx := tx.MsgTx()
if tx.Tree() == wire.TxTreeStake && stake.IsSSRtx(msgTx) {
txCopy := dcrutil.NewTxDeepTxIns(msgTx)
if maybeInsertStakeTx(g.blockManager, txCopy, !treeKnownInvalid) {
if maybeInsertStakeTx(g.blockManager, txCopy, !knownDisapproved) {
blockTxnsStake = append(blockTxnsStake, txCopy)
revocations++
}
@ -1884,7 +1884,7 @@ mempoolLoop:
break
}
utxs, err := g.chain.FetchUtxoView(tx, !treeKnownInvalid)
utxs, err := g.chain.FetchUtxoView(tx, !knownDisapproved)
if err != nil {
str := fmt.Sprintf("failed to fetch input utxs for tx %v: %s",
tx.Hash(), err.Error())

9
mining/mining.go
View File

@ -78,8 +78,9 @@ type TxSource interface {
// pool.
VotesForBlocks(hashes []chainhash.Hash) [][]VoteDesc
// IsTxTreeKnownInvalid returns whether or not the transaction tree of
// the provided hash is known to be invalid according to the votes
// currently in the memory pool.
IsTxTreeKnownInvalid(hash *chainhash.Hash) bool
// IsRegTxTreeKnownDisapproved returns whether or not the regular
// transaction tree of the block represented by the provided hash is
// known to be disapproved according to the votes currently in the
// source pool.
IsRegTxTreeKnownDisapproved(hash *chainhash.Hash) bool
}

28
rpcserver.go
View File

@ -3452,7 +3452,7 @@ func handleGetRawTransaction(s *rpcServer, cmd interface{}, closeChan <-chan str
var blkHash *chainhash.Hash
var blkHeight int64
var blkIndex uint32
tx, err := s.server.txMemPool.FetchTransaction(txHash, true)
tx, err := s.server.txMemPool.FetchTransaction(txHash)
if err != nil {
txIndex := s.server.txIndex
if txIndex == nil {
@ -3462,14 +3462,15 @@ func handleGetRawTransaction(s *rpcServer, cmd interface{}, closeChan <-chan str
}
// Look up the location of the transaction.
blockRegion, err := txIndex.TxBlockRegion(*txHash)
idxEntry, err := txIndex.Entry(txHash)
if err != nil {
context := "Failed to retrieve transaction location"
return nil, rpcInternalError(err.Error(), context)
}
if blockRegion == nil {
if idxEntry == nil {
return nil, rpcNoTxInfoError(txHash)
}
blockRegion := &idxEntry.BlockRegion
// Load the raw transaction bytes from the database.
var txBytes []byte
@ -3496,7 +3497,7 @@ func handleGetRawTransaction(s *rpcServer, cmd interface{}, closeChan <-chan str
context := "Failed to retrieve block height"
return nil, rpcInternalError(err.Error(), context)
}
blkIndex = wire.NullBlockIndex // TODO: Update txindex to provide.
blkIndex = idxEntry.BlockIndex
// Deserialize the transaction
var msgTx wire.MsgTx
@ -3879,8 +3880,7 @@ func handleGetTxOut(s *rpcServer, cmd interface{}, closeChan <-chan struct{}) (i
}
var txFromMempool *dcrutil.Tx
if includeMempool {
txFromMempool, _ = s.server.txMemPool.FetchTransaction(txHash,
true)
txFromMempool, _ = s.server.txMemPool.FetchTransaction(txHash)
}
if txFromMempool != nil {
mtx := txFromMempool.MsgTx()
@ -4486,7 +4486,7 @@ func fetchInputTxos(s *rpcServer, tx *wire.MsgTx) (map[wire.OutPoint]wire.TxOut,
// Attempt to fetch and use the referenced transaction from the
// memory pool.
origin := &txIn.PreviousOutPoint
originTx, err := mp.FetchTransaction(&origin.Hash, true)
originTx, err := mp.FetchTransaction(&origin.Hash)
if err == nil {
txOuts := originTx.MsgTx().TxOut
if origin.Index >= uint32(len(txOuts)) {
@ -4501,14 +4501,15 @@ func fetchInputTxos(s *rpcServer, tx *wire.MsgTx) (map[wire.OutPoint]wire.TxOut,
}
// Look up the location of the transaction.
blockRegion, err := s.server.txIndex.TxBlockRegion(origin.Hash)
idxEntry, err := s.server.txIndex.Entry(&origin.Hash)
if err != nil {
context := "Failed to retrieve transaction location"
return nil, rpcInternalError(err.Error(), context)
}
if blockRegion == nil {
if idxEntry == nil {
return nil, rpcNoTxInfoError(&origin.Hash)
}
blockRegion := &idxEntry.BlockRegion
// Load the raw transaction bytes from the database.
var txBytes []byte
@ -4790,12 +4791,17 @@ func handleSearchRawTransactions(s *rpcServer, cmd interface{}, closeChan <-chan
// are needed.
if len(addressTxns) < numRequested {
err = s.server.db.View(func(dbTx database.Tx) error {
regions, dbSkipped, err := addrIndex.TxRegionsForAddress(
idxEntries, dbSkipped, err := addrIndex.EntriesForAddress(
dbTx, addr, uint32(numToSkip)-numSkipped,
uint32(numRequested-len(addressTxns)), reverse)
if err != nil {
return err
}
regions := make([]database.BlockRegion, 0, len(idxEntries))
for i := 0; i < len(idxEntries); i++ {
entry := &idxEntries[i]
regions = append(regions, entry.BlockRegion)
}
// Load the raw transaction bytes from the database.
serializedTxns, err := dbTx.FetchBlockRegions(regions)
@ -4815,7 +4821,7 @@ func handleSearchRawTransactions(s *rpcServer, cmd interface{}, closeChan <-chan
addressTxns = append(addressTxns, retrievedTx{
txBytes: serializedTx,
blkHash: regions[i].Hash,
blkIndex: wire.NullBlockIndex,
blkIndex: idxEntries[i].BlockIndex,
})
}
numSkipped += dbSkipped

2
server.go
View File

@ -1150,7 +1150,7 @@ func (s *server) pushTxMsg(sp *serverPeer, hash *chainhash.Hash, doneChan chan<-
// Do not allow peers to request transactions already in a block
// but are unconfirmed, as they may be expensive. Restrict that
// to the authenticated RPC only.
tx, err := s.txMemPool.FetchTransaction(hash, false)
tx, err := s.txMemPool.FetchTransaction(hash)
if err != nil {
peerLog.Tracef("Unable to fetch tx %v from transaction "+
"pool: %v", hash, err)