Client-side transaction generation debugging session 2/15

#1

Yesterday, @yukan and I spent some time debugging client-side transaction generation. There were a couple of “gotchas” and lessons learned.

Generating Preorder and Register Simultaneously

First, when generating a preorder and register, those two bitcoin transactions are generated and signed at the same time. Unfortunately, generating two different transactions at the same time, with the same payment address means that a normal UTXO lookup will use the same UTXO for both transactions.

The generated transactions were:

register transaction:
`0100000001d94b07fa5352ec4a7a3677b967ddab44f8b2541423f74d7b166be323a9851a03000000006a47304402200a409872c876a6148f2c08a1c53c2642c9c3fad8e9ef5111b9f387f2c62fc35502207e5ffba10f52080b4a28c8441002aba125e2f0345a4c39e88f51a8493138227b012103d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25dffffffff0300000000000000003e6a3c69643a61746573742e6964000000000000000000000000000000000000000000000000000000000010f1d3f571a58734779bb299a12b90548436c8817c150000000000001976a91427bf9aab60f98a48363bfefbc88ad5fa19c3085a88ac6187f629010000001976a91401999117959c091ff3c2f8636258c6b552904dd788ac00000000`

preorder transaction:
0100000001d94b07fa5352ec4a7a3677b967ddab44f8b2541423f74d7b166be323a9851a03000000006b48304502210085c2d431b4ddeff6e800ea3de02301461c65bcd9c963b0ed6638d5875852f9760220278c082fb138d2b7eea32f8654253b8cc235df46026ffa73c4e8350fa280d9e3012103d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25dffffffff030000000000000000296a2769643f0b52fdd5e1e9e902c660b0c7db950874e0d3dbea8187d10e18d89bcaaa63bf8a5df24bbda552f629010000001976a91401999117959c091ff3c2f8636258c6b552904dd788ac409c0000000000001976a914000000000000000000000000000000000000000088ac00000000

You can deserialize these with either bitcoind, electrum, or a bitcoin library:

$ electrum deserialize $PREORDER
{
    "inputs": [
        {
            "address": "19TeCUPoHRrfdxHPJxrqGqnhRToi9txuW",
            "num_sig": 1,
            "prevout_hash": "031a85a923e36b167b4df7231454b2f844abdd67b977367a4aec5253fa074bd9",
            "prevout_n": 0,
            "pubkeys": [
                "03d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25d"
            ],
            "scriptSig": "48304502210085c2d431b4ddeff6e800ea3de02301461c65bcd9c963b0ed6638d5875852f9760220278c082fb138d2b7eea32f8654253b8cc235df46026ffa73c4e8350fa280d9e3012103d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25d",
            "sequence": 4294967295,
            "signatures": [
                "304502210085c2d431b4ddeff6e800ea3de02301461c65bcd9c963b0ed6638d5875852f9760220278c082fb138d2b7eea32f8654253b8cc235df46026ffa73c4e8350fa280d9e301"
            ],
            "type": "p2pkh",
            "x_pubkeys": [
                "03d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25d"
            ]
        }
    ],
....

$ electrum deserialize $REGISTER
{
    "inputs": [
        {
            "address": "19TeCUPoHRrfdxHPJxrqGqnhRToi9txuW",
            "num_sig": 1,
            "prevout_hash": "031a85a923e36b167b4df7231454b2f844abdd67b977367a4aec5253fa074bd9",
            "prevout_n": 0,
            "pubkeys": [
                "03d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25d"
            ],
            "scriptSig": "47304402200a409872c876a6148f2c08a1c53c2642c9c3fad8e9ef5111b9f387f2c62fc35502207e5ffba10f52080b4a28c8441002aba125e2f0345a4c39e88f51a8493138227b012103d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25d",
            "sequence": 4294967295,
            "signatures": [
                "304402200a409872c876a6148f2c08a1c53c2642c9c3fad8e9ef5111b9f387f2c62fc35502207e5ffba10f52080b4a28c8441002aba125e2f0345a4c39e88f51a8493138227b01"
            ],
            "type": "p2pkh",
            "x_pubkeys": [
                "03d6be49c2c5ea9344e2c441f1769b7aba80ad77b3f78033d899cbbf8ebc75b25d"
            ]
        }
    ],
...

As you can see, the same UTXO is being spent in both transactions.

blockstack.js has a solution for this, however. It provides a call, modifyUTXOSetFrom, which will adjust the library’s UTXO set with the assumption that the provided transaction is already in the mempool.

You can generate the two registration transactions as follows:

transactions.makePreorder('aaron.id', destAddress, payer)
.then((x) => {preorderTx = x; console.log('done')})
.then(() => { config.network.modifyUTXOSetFrom(preorderTx) })
.then(() => transactions.makeRegister('aaron.id', destAddress, payer, zfTest))
.then((x) => {registerTx = x; console.log('done')})

This way, the second transaction will use outputs from the first. The call resetUTXOs(address) will clear out modifications to the UTXO set of the given address.

Transaction Broadcast Service in Regtest

The next issue @yukan discovered was in the regtest transaction broadcaster included in the browser_env scenario.

The transaction broadcaster uses blockstack.js itself to communicate with blockstackd and bitcoind – the browser env runs the transaction broadcaster in regtest mode, and so it uses the default regtest configuration of the network library.

However, when the Blockstack regtest mode is configured to bind the API endpoint to 6270, instead of the default 16268, @yukan encountered this error in the docker container’s logs:

[2018-02-15 22:02:53,888] [WARNING] [namedb:153] (1.139628795741952) !!! Getting raw read/write DB instance !!!
[2018-02-15 22:02:53,888] [DEBUG] [namedb:91] (1.139628795741952) Connect to database '/tmp/blockstack-run-scenario.blockstack_integration_tests.scenarios.browser_env/blockstack-server.db'
info: 1 watched transactions, not fully confirmed
info: Broadcasting zonefile $ORIGIN te...}
error: Error processing broadcast while watching ccfcb8fb3ba92ec6a3b1cfa149328087fc797da3286dc04c70772535a3e1f375: FetchError: request to http://localhost:16268/v1/zonefile/ failed, reason: connect ECONNREFUSED 127.0.0.1:16268
error: FetchError: request to http://localhost:16268/v1/zonefile/ failed, reason: connect ECONNREFUSED 127.0.0.1:16268
   at ClientRequest.<anonymous> (/src/blockstack.js/node_modules/node-fetch/index.js:133:11)
   at emitOne (events.js:96:13)
   at ClientRequest.emit (events.js:188:7)
   at Socket.socketErrorListener (_http_client.js:310:9)
   at emitOne (events.js:96:13)
   at Socket.emit (events.js:188:7)
   at emitErrorNT (net.js:1296:8)
   at _combinedTickCallback (internal/process/next_tick.js:80:11)
   at process._tickCallback (internal/process/next_tick.js:104:9)
[2018-02-15 22:02:54,310] [DEBUG] [atlas:2188] (1.139628679304960) Missing 1 zonefiles
[2018-02-15 22:02:54,314] [DEBUG] [atlas:3353] (1.139628679304960) localhost:16264: missing 1 unique zonefiles
[2018-02-15 22:02:55,332] [DEBUG] [atlas:2188] (1.139628679304960) Missing 1 zonefiles
[2018-02-15 22:02:55,336] [DEBUG] [atlas:3353] (1.139628679304960) localhost:16264: missing 1 unique zonefiles

The transaction broadcaster doesn’t know that the API endpoint changed ports.

This was fixed by having the transaction broadcaster watch the same environment variable (BLOCKSTACK_TEST_CLIENT_RPC_PORT) that blockstack-core watches.

1 Like
#2

Thanks for writing this up @aaron!