BlockStack : How can i setup a blockstack full node (not testnet)

Community
#21

hmm, it seems like you’re hitting this error from this line:

if you’re bind-mounting the blockstack-server.ini file directly, try copying it into the data directory you’ve bind-mounted and removing the explicit mount of that ini file.
docker should be able to write to that file then.

so, from my previous compose example:
cp ./blockstack-server.ini ./data/blockstack-server.ini

    volumes:
      - ./data:/root/.blockstack-server
      # - ./blockstack-server.ini:/root/.blockstack-server/blockstack-server.ini```
#22

Thanks @jwiley a lot for your help. It’s working for me and i am able to ping the node using RPC. I tried those steps mentioned above and it worked perfectly fine.

Although, after some time i got an error for SegWit transactions and it suggested me to add “rpcserialversion=0” to my bitcoin node. I did that to my existing node and it worked.

I have one more question, i have looked for RPC on https://core.blockstack.org/#blockchain-operations link provided by you and it is working. But i think it does not have all the APIs like if i want to get the transactions of a particular account as of a block height , i am unable to find a API for this.

I can see a similar API on https://blockstack.github.io/stacks-blockchain-api/#operation/get_transaction_by_id but it is not working on my node. Could you suggest how do i get the transactions out of my existing core blockstack node?

#23

There’s no way to directly get tx data from the v1 blockchain.
Can you set something up like our explorer (https://explorer.blockstack.org) to replicate how we do it? absolutely, but it may be more trouble than it’s worth.
There are a lot of helper services that need to run properly, and I would generally not recommend it (particularly since we’re moving to v2 soon).

So, here are some methods you can use to retrieve tx data. I’ll be using core.blockstack.org, but you should replace that with your local blockstack_core instance.
The corresponding explorer page: https://explorer.blockstack.org/tx/36ec00d97748ce5ce05767dbb7c547aca0ec4fca1ca48765586ec4fd04d59865

  1. retrieve a block from the core node:
    curl -s https://core.blockstack.org/v1/blockchains/bitcoin/operations/654779 | jq 
[
  {
    "address": "1AUjfGid6qpUo1WLWorEfarmapWMLY1BJR", 
    "block_number": 598907, 
    "consensus_hash": "0b31b297596be309954aaa5e35499e20", 
    "first_registered": 598911, 
    "importer": null, 
    "importer_address": null, 
    "last_renewed": 598911, 
    "name": "zel_crux.id", 
    "name_consensus_hash": "4d3c4305cd6234048b11dc526e7a751e", 
    "name_hash128": "3493e9878d798aa4393eaa27e4157c2e", 
    "namespace_block_number": 373601, 
    "namespace_id": "id", 
    "op": "+", 
    "op_fee": 5500, 
    "opcode": "NAME_UPDATE", 
    "preorder_block_number": 598907, 
    "preorder_hash": "9866c7b04950c314993632fd87967924f71ad232", 
    "revoked": false, 
    "sender": "76a91467f7f9424342a0f1ab222ebbc1fb0192d43fe8ab88ac", 
    "sender_pubkey": "02841d6cb6500a30c88825c1352c5fa1fe7cea66b7d5f61f49f32ab0d5fdb54a3b", 
    "token_fee": "0", 
    "txid": "36ec00d97748ce5ce05767dbb7c547aca0ec4fca1ca48765586ec4fd04d59865", 
    "value_hash": "abccfd4584f10d2891a05f19a81f5d1c0298f414", 
    "vtxindex": 293
  }
]
  1. (optional) Using a returned tx id, confirm it exists:
    curl -s https://explorer-api.blockstack.org/api/v2/search/36ec00d97748ce5ce05767dbb7c547aca0ec4fca1ca48765586ec4fd04d59865 | jq 
{
  "found": true,
  "type": "btc-tx",
  "hash": "36ec00d97748ce5ce05767dbb7c547aca0ec4fca1ca48765586ec4fd04d59865"
}
  1. Retrieve the tx data from our explorer api: curl -s https://explorer-api.blockstack.org/api/transactions/36ec00d97748ce5ce05767dbb7c547aca0ec4fca1ca48765586ec4fd04d59865 | jq

Now, some blocks obviously will have multiple tx’s in them, ex: https://core.blockstack.org/v1/blockchains/bitcoin/operations/462592 - so you’d want to loop over that output array to retrieve each txid.

We do also rate-limit excessive traffic, so please don’t simply loop through all blocks, then retrieve the corresponding tx data.

#24

@jwiley … Thanks a lot again for prompt reply. I will try out above given options and keep you updated. Just wanted to check do we have a tentative date when we are releasing mainnet STACKS:2.0 ?

Since i understand that V2 is a major upgrade, can you suggest if I have to again do this exercise to setup the node and any other major changes… or the same node will be upgraded directly?

#25

I can’t give an exact date unfortunately, but this upgrade will be a hard fork so you’d have to upgrade to use the new blockchain. https://whenmainnet.co

The chaindata from v1 will be available once v2 reaches mainnet as well.
You can still use v1 after this upgrade, but it won’t retrieve new blocks or sync with the network, so i wouldn’t recommend it outside of research/curiosity.

I do have some docker/docker-compose docs already written to get it up and running easily:

https://docs.blockstack.org/mining#optional-running-with-docker
just note the commented lines here need to have the comments removed to run - https://docs.blockstack.org/mining#run-the-miner (change is PR’ed but not live).

and if you want to try running a lock mocknet: https://github.com/blockstack/stacks-local-dev

1 Like
#26

@jwiley BlockNode is running fine by following the above thread and its pointing to one of my bitcoin node running locally and the bitcoin node has synced till 413881 .I have to get the balance of one of the address - SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB which shows as zero b ut thats not the same case with explorer - https://explorer.stacks.co/address/SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB?chain=mainnet
The address has atrted some transactions by the blockheight “670060” as per the explorer.

So is it like once bitcoin node syncs to the latest blocks only the Api queries would give proper balance and account transactions ?

#27

@devd Correct, you will have to wait for your local node to sync until at least that burnchain block (and corresponding stx block) before you’ll see that address.

Since it sounds like you’re interested in running your own API, i’m going to link you some docs that might help you setup an API node:
hiro docs
github - running an API from source
github - running an API using docker

1 Like
#28

@jwiley Thanks a bunch for the info , so if i am starting an API Node does that mean it will sink all the stacks blockchain data and i dont have to point to any bitcoin node with this approach or its just a wrapper and will try to communicate with some other stacks node and bitcoin node which is not in my control.

And with the current approach with which i have set up the BlockstackNode pointing to local bitcoin node. What’s the difference between these 2 approaches . (setting up API node vs Setting up BlockStack full Node)

#29

I only mentioned the API since it seemed like that was the route you were looking to take.
There is no requirement that you run an API node though - however, it does make development “easier” since the API provides a quick way to pull data from the blockchain without any blocking requests. Indeed, if you are planning to use this node for a dapp or running a lot queries against the blockchain, having the API would make this work much easier and more consistent.

To answer your specific questions:

  1. if i am starting an API Node does that mean it will sink all the stacks blockchain data and i dont have to point to any bitcoin node: not sure i quite understand you here, but the API runs alongside the stacks blockchain, ingesting all events and storing them in a postgres database. This makes the data easier/quicker to use from other code, since some blockchain queries can take several seconds to return (at the same time blocking other requests in the queue).
  2. its just a wrapper and will try to communicate with some other stacks node and bitcoin node which is not in my control.: not quite - the API has communicate with a stacks node that you control. you can use a public bitcoin node with the stacks node as well - you don’t have to use a local bitcoin node (although this is encouraged).
  3. What’s the difference between these 2 approaches: I’ll try to summarize here:
    a. running a stacks node + local bitcoin node: this is at it’s core just the main blockchain data (bitcoin data and the stacks blockchain) - this is all you need in order to work with the stacks blockchain.
    b. running a stacks node + local bitcoin node + API: this is the same as above, but with the API it also gives you a consistent interface to query for blockchain data (which is also not subject to blocking requests since it’s stored in postgres).

Whichever route you decide to take, either one will work.
The API setup is a bit more advanced though, and it may not be quite what your goal is.
If you’re setting up a stacks node with the goal of using it from a dapp or other code where you’ll have a lot of queries - i would recommend also running an API though.

1 Like
#30

@jwiley Thanks for the detailed explanation on my previous query , makes complete sense to use the API + Stack local node + BTC Node.

Before that i was just trying to understand the json structure of couple of queries and map it with explorer to understand the the operations where STACKS is involved.

My requirement is to get the balance and transactions for stacks address at a particular blockheight:

  1. Balance at a blockheight to get balance - Did not get any Apis to get this information . Could you please validate.
  2. Transactions till a blockHeight - Did not get any direct Apis but got some Apis which gets all the transactions and from there we need to filter , but when i validate this with stacks explorer not able to map the json data . Could you please help me understand how can i use this data .

Sample address - SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB
In explorer - Total 15 transactions
Avialable Balance - 1.129834

using this Api - /v1/accounts/SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB/history?page=0

against my node and also
https://core.blockstack.org/v1/accounts/SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB/history?page=0

and

for page 1 as well

getting the json structure where credits and debits are involved and a BTC address which i am not able to relate with the stacks explorer -

Can you please help me to understand the transactions and when we do the calculations it should reach to the Stacks balance what ever shown in explorer.

i have not provided the json structure over here but you can run the Apis provided above to see the structure.

#31

This is what you’re looking for: Stacks 2.0 Blockchain API

the core.blockstack.org endpoints you’ve referenced are deprecated, any stacks blocks after bitcoin block 666053 won’t be available from that endpoint.

the explorer and the above api have all the data you’re looking for - but i will caution not to spam the those endpoints, since they are public resources we have a moderate rate-limit enforced to keep the services running.

if you need to pull a lot of data, or crawl the data - i’d encourage you to run your own API node as described in my other replies.
otherwise, depending on the volume of requests you want to make - you may see your requests throttled after a while.

now to your question about calculations - the explorer uses an API service (again, linked above already) to display the data in the webpage. the stacks-node will receive the block data initially, then send that event to the API server which then stores it in a postgres DB. the explorer will then make requests for data from that API using the endpoints described in the link above in this reply: Stacks 2.0 Blockchain API

and to be clear, all the components i’ve described here can be run by anyone - the source code is all readily available.

#32

Thanks @jwiley I have just given the core.blockstack api as a reference as i was getting same kind of response from my own node set up (blockstack + bitcoin node )
Below is the query which i am using in my node set up:

sudo curl -v http:///v1/accounts/SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB/history?page=0

So as per you response its deprecated and i should go by the Api + Blovkstack Node + Btc Node and then try the queries from “Stacks 2.0 Blockchain API” and thats where i will be able to match with the explorer data .

Is my understanding correct.

#33

yes and no - it all depends on what type of data you’re looking to retrieve and how often.
there is an RPC endpoint in the stacks-blockchain, but be aware these are blocking requests.

using that as a reference, a similar query to your curl would look like this:

curl -sL http://<local stacks node>:20443/v2/accounts/SP3DQR29HQYTGGA5287NJPM3V1WC6J0HSPXQHFJKB?page=0 | jq

I do want to stress as well that not all API endpoints you’re looking at for v1 are ported to v2 - some have been ported, some are not, and some are slightly different. they’re all described in the above links.

#34

Hi Again,

I have followed the exact steps mentioned on the below links:
stacks-blockchain-api/running_an_api.md at master · blockstack/stacks-blockchain-api · GitHub

My all the 3 nodes in the docker containers are up and running and stacks blockchain is syncing fine. But when i am verifying the node , my CURL query to stacks-blockchain node works fine and giving the expected response but query to API node is not giving a response
curl -sL localhost:20443/v2/info (works fine)
but below query does not work from the same server on which it is running:
curl -L http://localhost:3999/v2/info (Recv failure: Connection reset by peer)

I have followed the exact same process and config files provided as it is. Please help.

#35

Without seeing the logs, there could be many reasons the port is not opening for the API.
You’ll need to share some of the API’s logs so we can find what’s happening there, but I would guess it’s misconfigured since port 3999 opens after it can communicate with the stacks-blockchain on port 20443

#36

@vishalgupta96 any new info you can provide here?
particualry i’m looking for command output from the above curl’s you ran, and the API logs

#37

@jwiley : Hi Jesse, we have used the above URL for running 3 nodes (API, Stacks & Postgres) and also used the below link for standalone node: https://github.com/blockstack/stacks-blockchain-api
and in the both type of setup i am getting below docker logs and i am unable to query the node:

used below docker run command to start the standalone node:
docker run -p 3999:3999 blockstack/stacks-blockchain-api-standalone mainnet

On running this, i am getting below logs in node:
INFO [1633675537.925778] [testnet/stacks-node/src/run_loop/neon.rs:428] [main] Downloaded burnchain blocks up to height 704051; new target height is 704051; next_height = 704051, block_height = 704051 INFO [1633675537.976387] [testnet/stacks-node/src/run_loop/neon.rs:501] [main] Synchronized full burnchain up to height 704051. Proceeding to mine blocks INFO [1633675543.645034] [src/net/http.rs:1571] [p2p] Handle HTTPRequest, verb: GET, peer_addr: 127.0.0.1:60682, path: /v2/info, query: WARN [1633675543.712766] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675543.712818] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675543.712844] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675543.712869] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675543.712894] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675543.712919] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) INFO [1633675549.113535] [testnet/stacks-node/src/run_loop/neon.rs:428] [main] Downloaded burnchain blocks up to height 704051; new target height is 704051; next_height = 704051, block_height = 704051 INFO [1633675549.146951] [testnet/stacks-node/src/run_loop/neon.rs:501] [main] Synchronized full burnchain up to height 704051. Proceeding to mine blocks INFO [1633675554.133496] [src/net/http.rs:1571] [p2p] Handle HTTPRequest, verb: GET, peer_addr: 127.0.0.1:60720, path: /v2/info, query: WARN [1633675554.203447] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675554.203501] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675554.203527] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675554.203554] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675554.203578] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) WARN [1633675554.203602] [src/net/connection.rs:559] [p2p] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type")) INFO [1633675560.484671] [testnet/stacks-node/src/run_loop/neon.rs:428] [main] Downloaded burnchain blocks up to height 704051; new target height is 704051; next_height = 704051, block_height = 704051 INFO [1633675560.518253] [testnet/stacks-node/src/run_loop/neon.rs:501] [main] Synchronized full burnchain up to height 704051. Proceeding to mine blocks

But i am unable to query the node and unable to use any APIs. For ex:
Request:
sudo curl -L localhost:3999/v2/info

Response:
curl: (56) Recv failure: Connection reset by peer

Could you please help here… what’s wrong with above?

#38

Without debug logging, it’s impossible to say why you’re seeing those messages: https://github.com/blockstack/stacks-blockchain/blob/master/src/net/connection.rs#L557-L566

The messages themselves aren’t necessarily a problem - they’re simply saying they don’t understand the http request.
But, without debug logging we don’t know what the request is or even where it’s coming from.

Further, I wouldn’t recommend using the docker image you’re running - it’s mainly used for testing and I’m not sure how up to date it is.

If you’re interested in running an API node, I’d encourage you to look at the docs here: Running an API node | Stacks

Just be sure you have at least 4GB of free memory and ideally 2vcpu at a minimum.

#39

@jwiley : Hi Jesse, As i have mentioned above - i have started API node by both ways. One is the steps provided the link provided by you and another is using “standalone” image. I understand about the “standalone” that it is only for testing but the other node which i have created using the above link only. But i am getting the same logs in that node also. Could you suggest from where i can get the debug logs to share it with you?

#40

using the commands in the link above, you’d adjust it to look like this (notice the extra env vars that are added):

docker run -d --rm \
  --name stacks-blockchain \
  --net=stacks-blockchain \
  -e RUST_BACKTRACE=full \
  -e STACKS_LOG_DEBUG=1 \
  -v $(pwd)/persistent-data/stacks-blockchain:/root/stacks-node/data \
  -v $(pwd)/config:/src/stacks-node \
  -p 20443:20443 \
  -p 20444:20444 \
  blockstack/stacks-blockchain \
/bin/stacks-node start --config /src/stacks-node/Config.toml

Once you have it running until you reach the error messages, you can save the logs simply run a command a link this to save the logs:
docker logs stacks-blockchain > logs.txt 2>&1

To share the logs, it depends on the size - but you’ll likely want to compress the logs and use a public file-sharing service like google drive, dropbox etc. you can send me the link (or if the file is small enough for email) to jwiley at hiro dot so