There have been more orphan blocks and reorgs (more than 2 blocks) on the Stacks network for a few months. Such reorgs happen when miners or sets of miners try to work on different forks potentially because they’re unaware of the presence of the other fork. Many of these reorgs appear to correspond to different mining groups, one of which seems to be a Stacks mining pool or mining service comprising a large share of the mining power on the network recently.
Based on funding patterns, we have reason to believe this mining pool or service is composed of multiple independent miners. Further, from the network behavior it seems that this mining service/pool is connected through a low-bandwidth connection or a remote geographic location (it’s hard to say exactly what region or how low the bandwidth is); this results in the mining service working on forks that other miners might not be aware of.
For the orphan blocks issue, open-source devs participating in working groups are running network diagnostics to determine values for the miner config settings that could help improve this issue in the short-term. There are also software patches under consideration that can help improve/resolve the problem in the long-term.
The main impact is on exchanges/partners and downstream, that negatively affects users directly. For exchanges, they can see transactions rolled back and possibly dropped - which causes them to increase the number of confirmations required (which in turn affects users where it takes longer to transfer tokens, etc). Secondly, it affects miners directly - they are spending bitcoin to mine a block, and then losing the reward of mining that block if they are orphaned. The orphaning of blocks also negatively impacts the performance of the chain, because as longer forks are mined there may be some transactions that are reverted to the mempool or will generally take longer to be confirmed in an anchor block.
To better understand this issue:
Lead: Jesse Wiley
Status: A significant amount of diagnostic work has been done, with more ahead.
What’s next: This is a particularly difficult issue to zero on the underlying cause for so the developers working on this issue are focused on surfacing useful network diagnostics and creating tools to gather the necessary data. The focus now is on increasing visibility into the state of the network, with first steps being to “map” the network peers, in particular, the neighbors of miners. There are also plans to launch more dedicated seed nodes in various geographical regions to see if that helps what so far appears to be a peering issue.
How you can help: Comment on the SIP proposal
For continuing updates: Please follow this thread, updates will be posted as replies below by core developers and contributors.