Setting up Developer Environments

Currently, for developing apps, a potential developer has a couple different avenues they can go down.

This started in a Slack discussion with @jude @jackzampolin @larry - there’s a question about what the appropriate instructions and process for an app developer are.

The discussion seems to be framed along two different options:

  1. setup a regtest environment and include blockstack.js in your app
  2. use our release installers to get a mainnet browser, include blockstack.js in your app, and sign in with either an address-only, or buy a name.

For team No-Regtest:

  • regtest mode should be just for developers hacking on browser or core
  • process for app dev should be:
    • download & install blockstack for mac, windows or linux
    • add blockstack.js to their app
  • the ports for the regtest environment are different leading to confusion
  • regtest adds a huge amount of complexity and is even hard for us to use
  • auth and storage work with just an address and don’t require payment
  • Profiles even get stored w/o names now
  • Easier to support less options

For team Regtest:

  • Some apps want to do things like payment, weird zonefile things
  • Some people want to use namespaces for functionality

Discussion from here:

I’m personally leaning towards team no-regtest. But we wanted to move the discussion over to the forum so that we could include input from our community.

I’m going to have more info on this soon, but I think its not just some apps, it is almost all apps, especially in the case of payments. This problem is going to become more glaring as we are incentivizing developers wanting to open their own namespaces to take advantage of namespace fees.

Also I have some thoughts on the team no-regtest above:

process for app dev should be:

I have talked to most of the folks developing on the platform and the happy path described above doesn’t exist for most devs. They have more complex product requirements or have trouble understanding how the system works and end up going through the process of setting up the test environment anyway.

Giving them an easy way to do this not only lets them test out the features of our product w/o paying any money (name/namespace reg etc…) but also gives them a whole sandbox system to explore. This exploration leads to a much better understanding of the system and allows them to make informed decisions about application architectures.

the ports for the regtest environment are different leading to confusion

I think this is a bad design decision. The ports for our apps should always be the same, but configurable. I understand the decision was made to do this so people can run both at the same time, but it leads to a ton of confusion and we run into more port conflicts with other products this way leading to a worse experience for devs

Easier to support less options

We are going to need to support a regtest environment both for the usecases described above as well as onboarding new protocol devs and automated testing. I don’t see a way we get around supporting this environment.

The feature I ultimately want to see is a 1-click “Open new test environment” button, similar to the “Open new Incognito tab” option you’d see in Chrome.

Once in the test environment, you’d have a private blockchain with no namespaces and a test wallet pre-loaded with lots of regtest Bitcoin. From there you’d be able to anything/everything you’d normally be able to do in mainnet (but with no financial consequences). You’d also be able to control the rate at which new blocks get produced.

I understand the decision was made to do this so people can run both at the same time, but it leads to a ton of confusion and we run into more port conflicts with other products this way leading to a worse experience for devs

This is a hard requirement as far as I know. The reason is that users ought to be able to leave their mainnet Bitcoin, Blockstack Indexer, and Blockstack API processes running the whole time.

How about this? Could we have our CI infrastructure generate a nightly VM image that had the latest regtest environment set up? Then users who want a test environment could simply grab the VM image, boot it up in Virtualbox, and proceed to hack away on that. It wouldn’t interfere at all with the host environment, and it would already be set up. This could easily be achieved via an “Open new test environment” button as well. Thoughts?

EDIT: We’re mostly achieved for the todos app already: https://github.com/blockstack/blockstack-todos/blob/master/docker-compose.yaml

1-click regtest is what I envisioned when staring this feature: https://github.com/blockstack/blockstack-browser/issues/546

To support 1-click regtest on mac, all we would need to do (at a high level) is:

  • switch from the virtual env build that’s built from the script in the browser repo to using the CircleCI virtual env artifact
  • bundle a bitcoind binary
  • wire up the logic to stop the mainnet api endpoint and fire up the regtest version

In the future, we also want to make it so that users can enable full nodes with 1 click. It makes sense to bundle bitcoind for that use case because then they could also 1 click enable a bitcoind node and be 100% decentralized. (given that they have enough disk space).

I don’t like 1-click regtest, because it tries to obscure a bit of a complex process. Here’s my thinking on how to deploy regtest (plus an imagined future):

Today (on mainnet):

Browser -> Core Client -> blockstackd (node.blockstack.org) -> bitcoind

Today (on regtest):

Browser -> Core Client -> blockstackd (local) -> bitcoind (local, regtest)

Imagined Future (on mainnet):

Browser -> blockstackd (default: node.blockstack.org)

Imagined Future (on regtest):

Browser -> blockstackd (local) -> bitcoind (local, regtest)

An advanced user should be able to “point” their browser at a specific blockstackd instance (local or otherwise). The above communication graphs are important for a dev that needs to interact with regtest to understand, so I think the setup process should reflect that.

So, if you want to run on regtest (or testnet) or whatever, what that means is that you want to point your browser at regtest instance (or testnet) – so what do you do? Spawn a regtest instance and then point your browser at it. So the process is like “start the regtest node app” (we come up with a better name) and you get a console window that shows you scrolling text indicating you have a running regtest node. Then you point your browser at it and “voila!”

This could be done with 1-click, but this process of going and opening an app and then setting a configuration variable to direct your browser at the local node isn’t too complex, plus it gives the user more insight into what’s happening and the system’s various components.

2 Likes