Hey folks,
This is an outline of a set of proposals for the L1 Working Group to make the Stacks blockchain faster. All of these require a hard fork to execute, so there will eventually be a SIP that proposes them. I’m putting this outline here to solicit early feedback on the design. This is by no means a final draft. These proposals, if deemed acceptable, will eventually find their way into a SIP.
That said, if you need low transaction latency sooner, Hiro’s subnets system is a good candidate, and will be available shortly after the 2.1 consensus rules take effect.
Background: Microblocks
As originally designed, Stacks block miners can bridge the “time gap” between two Stacks blocks by mining microblocks. Microblocks are built upon the current miner’s block, and are confirmed by the next miner’s block. The idea is that miners can continue to accept transactions into the blockchain while waiting for the next Bitcoin block to be mined (see the diagram below).
block S microblock stream block S+1
.---. .-. .-. .-. .---.
stacks | | <-- | | <-- | | <-- | | <-- | |
*---* *-* *-* *-* *---*
| |
.---. .---.
bitcoin | | <-------------------------- | |
*...* *...*
Right now, microblock confirmation is encouraged by incentives, but is not enforced by consensus. The miner of block S+1 receives 60% of the transaction fees in the stream it confirms, and the miner of block S receives 40%. If the miner of S+1 orphans some or all of the stream, they only lose those transaction fees, and are not penalized in any other way. Furthermore, the amount of microblocks that can get confirmed by the miner of S+1 is bound by the compute budget (our version of a “block size” or “gas limit”) allotted to the miner. This budget is fixed by the consensus rules.
Proposal: Mandatory Microblock Confirmation
As it stands, if your transaction lands in a microblock, it’s not really confirmed until the next Stacks block gets mined. Only then can you be sure that your transaction was considered by the network and is anchored to Bitcoin.
This could be changed by requiring Stacks miners to pre-commit to the next block when they mine. Instead of just submitting a block-commit transaction for block S, each miner M would also promise to spend X[M] BTC when mining block S+1. Their block-commit for S+1 is only valid if it spends exactly X[M] BTC.
What this gets us is that now the network knows which miners will possibly mine block S+1. From there, we can alter the consensus rules to require that each microblock produced by the miner of S must be signed by a threshold T of miners who pre-committed to mine S+1 (as weighted by their X[M] values). If a microblock gets at least T BTC behind it, then the miner of S+1 must confirm that microblock (and by extension, its ancestors) in order to receive a block reward for S+1. If the winner of block S+1 does not do this, then block S+1 is still valid and can be built upon, but the miner of S+1 gets no STX.
This change makes it so that microblock transactions are confirmed as they are mined. Now, the latency between when your transaction is mined into a microblock and when it receives its first confirmation is the latency between microblocks, which is much smaller than 10 minutes (it’s on the order of seconds).
Proposal: Time-bound Microblock Budgets
Right now, each Stacks miner has a fixed compute budget it can use to add transactions to the Stacks blockchain. This budget is spread across the microblock stream the miner confirms, and the transactions included in its anchor block.
A compute budget is needed in order to prevent malicious miners from spamming the network to death. They cannot spend more than their allotted budget. But right now, this budget is block_bound. It is fixed per Bitcoin block – it’s the same regardless of whether or not the Bitcoin blocks for Stacks blocks S and S+1 are separated by 10 minutes, or 100 minutes. This severely impacts transaction latency: once a microblock stream exceeds the compute budget, the Stacks miner cannot append any more microblocks to it.
From a time perspective, it looks a lot like the situation below. The miner of block S can exhaust their compute budget well before block S+1 can be mined, forcing everyone to wait.
block S microblock stream (waiting) block S+1
.---. .-. .-. .-. .---.
stacks | | <-- | | <-- | | <-- | | <------------------------------------ | |
*---* *-* *-* *-* *---*
| |
.---. .---.
bitcoin | | <------------------------------------------------------------ | |
*...* *...*
===================================== time =================================>
What we can do instead is make it so that the miner’s budget is time-bound. Rather than giving miners a fixed budget per Bitcoin block, we give them a fixed budget per unit of wall-clock time. This achieves the same ends while keeping a target transaction latency. Transaction arrival over time would instead look like the situation below:
block S microblock stream budget growth block S+1
.---. .-. .-. .-. .-. .-. .-. .---.
stacks | | <-- | | <-- | | <-- | | <(tick)- | | <-- | | <(tick)- | | <-- | |
*---* *-* *-* *-* *-* *-* *-* *---*
| |
.---. .---.
bitcoin | | <------------------------------------------------------------ | |
*...* *...*
===================================== time =================================>
How might this be achieved? We’d need to require miners to run a verifiable delay function (VDF), which lets them prove that they’ve waited a certain amount of time in a way that can be verified after-the-fact.
The new consensus rules would adjust the delay parameter of the VDF so that the blockchain achieves a target average transaction latency. This would be analogous to how Bitcoin miners adjust the block difficulty every so often. If the (tick) in the diagram above got to be too long, then a subsequent miner would not be forced to wait as long. Or, if the (tick) got to be too short, then a subsequent miner would be forced to wait longer.
When combined with mandatory microblock confirmation, the resulting system would ensure that transactions get confirmed as quickly as they can be mined in microblocks, regardless of how much time passes between two subsequent Bitcoin blocks.
Proposal: Improve Throughput by Disincentivizing Missed Sortitions
(Shoutout to @friedger and peters.btc for their work on this).
The ability to pre-commit blocks also opens a path to further disincentivizing creating a fork with lots of missed sortitions, thereby improving the blockchain’s throughput. Creating a fork with lots of missed sortitions happens in two ways:
- Miners do not mine in a Bitcoin block
- Miners do not build atop the canonical Stacks tip in a Bitcoin block
The first case can happen if miners are not well-connected to the Bitcoin network, or have a slow node, or do not set a high enough transaction fee. This would cause them to miss Bitcoin blocks that arrive more quickly than they expect (“flash blocks”), and/or miss Bitcoin blocks when the Bitcoin network is congested. The second case can happen as a consequence of the first, or as a consequence of not being well-connected to the Stacks network, or as a consequence of running out-of-date node software that does not contain the latest fixes to the mining algorithm.
The fundamental problem is that Stacks, like Bitcoin, does not reward miners for producing blocks on the canonical fork that later become orphaned. This is not ideal, because it does not reward miners for trying to mine in good faith. In addition, miners who can get their blocks on the canonical chain get rewarded STX even if they orphan other miners’ blocks to do so.
By pre-committing to blocks, we can ensure that if a miner tries to mine on the canonical fork, they still get STX even if the block is orphaned later. We can also ensure that if a miner deliberately orphans a block, they do not get as much STX. We achieve this by re-distributing some STX from the latter miner to the former miner.
We can address this by requiring the miners who pre-committed to block S+1 to sign block S+1 as well. These signatures are included in block S+1 and serve as proof that the other miners believe that block S+1 can be attached to the blockchain. The other miners have not only seen the block, but also that it builds on the latest tip that these miners know about (note that miners don’t have to validate the block; they only check that it builds atop to their view of the canonical chain tip).
We use these signatures to alter the block reward rules as follows. We define a threshold T such that if block S+1 receives signatures from pre-committed miners whose committed BTC spends exceed T% of all pre-committed spends, then the ultimate winner of block S+1 receives the full block reward. If the block receives less than T% of the signatures by BTC spend, then the block reward is adjusted as follows:
-
Let T’ < T be the fraction of the BTC represented by block S+1’s signers. The winner of block S+1 receives T’ / T of the total block reward. That is, the miner of S+1 forfeits 1 - T’ / T percent of the block reward.
-
If block S+1 is canonical when its block rewards mature, then if there were N siblings of block S+1 mined before block S+1, and each sibling’s miner spent B[N] BTC, then each such sibling n’s miner gets an orphan reward equal to (B[n] / sum(B)) * e * (1 - T’ / T) percent of the block reward’s STX (where e < 1 is a protocol constant to ensure that mining siblings is never as profitable as mining on the canonical chain). That is, the forfeited block reward is shared pro-rata by all miners who tried to build on block S, but got orphaned. Their share is proportional to how much Bitcoin they spent to attempt to mine the orphan.
Importantly, the orphan reward is shared to miners who only committed to block S+1 on the Bitcoin chain. The orphaned block miners do not need to disclose their Stacks block.
This does not change the token emission in any way. It merely re-distributes some of the block reward from a miner who orphans blocks that were on the canonical chain to the miners who mined on the canonical chain in good faith.
This does not affect liveness or safety in any way either. A valid block S+1 with no signatures can still be built upon, and the difficulty of orphaning S+1 later is just as hard as it is today.
Analysis Sketch
Note to the reader: I would really appreciate it if someone with an economics background would vet this (the economics CAB perhaps?). I am not an economist.
Why does this work? In both this proposed system and the current system, the economically rational miner’s biggest payoff is to mine a Stacks block on the canonical chain. But by introducing this signing requirement, the only way the winner of block S+1 can get the full block reward is to ensure that the other miners for S+1 confirm that they also have block S. So, it’s in the miner’s best interest to ensure that every other miner has every block, so they don’t mine orphans by accident.
Would miners deliberately create orphan block-commits that correspond to no Stacks blocks at all? Because the orphan reward is proportional to the BTC spend from the miner, this is only profitable if B[n] < (B[n] / sum(B)) * e * (1 - T’ / T). Re-arranging, this is sum(B) < e * (1 - T’ / T). So, deliberately sending orphan block-commits is only profitable if all the miners who do this collectively spent less than the forfeited block reward. But, this is untenable in practice. This strategy forces such miners to play a game of chicken with the other miners: each miner stands to make money by sending a block-commit this way, but doing so risks making all such miners unprofitable if their block-commit pushes sum(B) over the value of the forfeited block reward. Because (1) miners see each other’s block-commits in the Bitcoin mempool and thus know the possible values for sum(B), (2) anyone can send these commits, and (3) no one knows in advance which block-commits will be accepted into the next Bitcoin block (so sum(B) is not known in advance), participating in this strategy is not profitable in expectation. Also, even if these deliberate orphan miners manage to send exactly sum(B) block-commits, the honest coalition of miners can send their own small orphan block-commits to force the deliberate orphan miners to lose money.
Why would miners cooperate at all to sign block S+1? They are incentivized to do so due to the tit-for-tat nature of this scheme. If a miner consistently refused to sign blocks, then other miners would refuse to sign their blocks as well (thus denying them STX block rewards). If miners don’t cooperate at all, then no one gets STX on the canonical chain. Only orphaned block miners would receive STX, and they receive less STX than they would had their blocks been canonical (even if they were the only other sibling).
Would miners ever sign S+1 if they didn’t have S? They are economically disincentivized from doing so, because if they don’t have S, then they can’t validate and build on S+1 anyway. At best, they would mine a sibling to S, and this is never as profitable as building atop S+1 (i.e. even if the sibling becomes canonical later, they still share the block reward with the other blocks at height S). So, miners remain incentivized to do their best to compel other miners to share the blocks they mine by withholding their signature for S+1 until they have validated S.
Could this scheme be weaponized by a majority of miners to prevent a minority from ever receiving block rewards? For example, could the long-running miners prevent new miners from joining by collectively withholding signatures for their blocks? This will depend on the T parameter. The higher T is, the harder it is for miners to cooperate this way. I’d recommend setting T to be at least 90%. Then, if honest miners have even 10% of the mining power, they can deny the other miners’ some of their STX in retaliation for excluding them (and it will be apparent from the chain history that deliberate exclusion is what’s happening).
Any thoughts or comments would be appreciated. Bye for now!