Help needed on figuring out API endpoints for the new explorer

Core Blockchain
#1

Hey folks,

As some know, I’m working on building out our new explorer that supports the new Stacks token, and the related info around accounts and name operations that are associated with the token.

I already have pretty much all I need from existing APIs that work with our current network and explorer. But, I’m having trouble finding all the info I need for the new features.

Lots of explorer-related info can be surfaced from an Insight API provided by a Bitcore node. However, from what I’ve seen, we don’t have a Bitcore node spun up for the testnet. If we should set that up, I can try and take that on, but it won’t support many of the Stacks-related information.

Here is a sample of things I’m having trouble finding:

  • Stacks operations associated with a transaction. Things like the number of Stacks sent, and the sender and receiver. From what I see, we use our Bitcoind node for getting things like the Block height, but again no Stacks info.
  • Somehow aggregating the total number of Stacks held by all accounts. We have a genesis block that I could aggregate, but I’d need a live version.

Pinging @jude @aaron @yukan for their help. Thanks!

#2

This should probably be done client-side, and can be done with pretty straight-forward parsing of the raw transaction.

const btc = require('bitcoinjs-lib')
const bigi = require('bigi')
const c32check = require('c32check')

const tx = btc.Transaction.fromHex(rawTX)
const data = btc.script.decompile( tx.outs[0].script )[1]

const operationType = data.slice(2,3).toString() // should be == '$' for token transfers.
const consensusHash = data.slice(3, 19).toString('hex')

const tokenTypeHex =  data.slice(19, 38).toString('hex')
const tokenTypeStart = tokenTypeHex.search(/[1-9]/)
// should be == 'STACKS'
const tokenType = Buffer.from(tokenTypeHex.slice(tokenTypeStart - (tokenTypeStart % 2)), 'hex').toString()

const tokenSentHex =  data.slice(38, 46).toString('hex')
const tokenSentBigI = bigi.fromHex(tokenValueHex)

const scratchData = data.slice(46, 80).toString()

const recipientBitcoinAddress = btc.address.fromOutputScript( tx.outs[1].script )
const recipientC32Address = c32check.b58ToC32(recipientBitcoinAddress)

That gets you things like the tokens sent, scratch data, and the recipient.

For obtaining the sender, though, you’ll want to use Bitcore’s API. The sender is just the first input of the Bitcoin transaction. (You can translate the Bitcoin address to C32-check using the c32check library routine b58toC32())

That, I think, will need a new endpoint.

2018-11-21 Engineering Meeting
#3

Thanks, that’s really helpful.

That, I think, will need a new endpoint.

Yeah, I figured. If we could get an endpoint that simply iterates over Stacks addresses with balance > 0, it would solve for a few of the ‘global stats’ things we want to display.

#4

Can you open an issue for this one and describe the kinds of global statistics you need?

#5

@aaron but do we have a Bitcore node with the testnet? I’ve only seen the bitcoind server.

#6

Nope – we don’t. How much stuff in the explorer uses the Bitcore API (I’m guessing a lot)? If it’s not that much, it may be better just to program against bitcoind’s RPC interface. If it’s a lot, then I think we’ll need to spin up a testnet bitcore node.

#7

Yeah, specifically we are currently using the insight-api from explorer.blockstack.org for lots of data, although obviously that’s when pointing to production data. I could try and use the RPC api, though, for most of the same stuff.

The important point is that we can easily ‘switch’ the APIs to the mainnet when we launch. Is the mainnet RPC api public?

#8

Yes – you would be able to query bitcoin.blockstack.com

Ultimately, the decision is up to you and I think @jude – spinning up bitcore nodes is a little painful, so I’m not sure how feasible it would be to run one with our testnet.

#9

Moreover, Bitcore uses a modified bitcoind implementation. We’d have to refactor the integration test framework to use it instead of vanilla bitcoind.

@hank When running in test mode, the Blockstack Core node implements some of the /insight-api/ API methods already in order to emulate enough of a Bitcore node for UTXO clients to work in the integration test framework. Do you know which API methods you need? I can patch them into the Core node.

#10

Yeah, let’s stay away from the Bitcore idea then.

Do you know which API methods you need? I can patch them into the Core node.

Currently, /addr/:address, /tx/:id, and /blocks. But I can get most of that stuff from RPC and other endpoints, so it’s not that necessary right now.