During the App Mining pilot, Democracy Earth was invited to become our Investor Community App Reviewer. The intention was that they would help rank apps based on the sentiment of verified Stacks token holders. We now have several months worth of ranking data allowing us share our learnings and suggest a path forward, concluding with a proposal to remove the Investor Community App Reviewer until we have data-backed solutions to some of the existing problems around voter incentives and engagement.
Democracy Earth Overview
Democracy Earth’s platform is open to verified Stacks token holders. Investors who opted in received on-boarding instructions to the Democracy Earth platform with their Blockstack ID, with guidance to evaluate all of the App Mining apps, and distribute all of their 1000 lifetime votes.
Token holders were prompted to cast their votes by voting an application either up (signifying approval) or down (disapproval). Token holders could have changed their votes at any time and had instructions to reallocate or maintain votes given the previous month on one application to different ones on a monthly basis, and received email reminders at the start and midway of each months ranking period.
Token holders could have abstained from voting as well. App Miners were able to participate in the Debate section of each app and respond to questions or feedback from the Voters. The full overview is here.
Investor engagement challenges
To start the program, we emailed 600/800 current investors (most of them twice) with a personal email to invite them to the program. We also manually on-boarded token holders over video calls to address questions and sell the value of voting. Some stats:
-
We converted 176 token holders agree to become voters via email.
-
When it came to actually voting on the platform, only 57/176 actually voted.
The breakdown by month:
-
December: 20 token holders voted
-
January: 30 token holders voted
-
February: 42 token holders voted
-
March: 49 token holders voted
Motivating additional investors to vote has felt like a slow – albeit tractable – problem. However, motivating those investors who voted once to reevaluate apps on a monthly basis has been even more challenging.
We’ve observed a clear pattern of investors setting votes once then never re-casting them. This has clear downsides for new apps. For example, in March we saw 18 token holders revote vs. the 57 total lifetime voters, or about 31%. By extension, we’re also seeing this pattern play out in the average payout for apps depending on what month they joined App Mining:
For App Mining to be successful, new apps cannot be penalized simply for joining after other apps. And as that relates to token holder voting, we need a reliable incentive to encourage reevaluation of apps on a regular basis. This challenge is not altogether unexpected since token holders aren’t paid for voting, and assessing 60+ apps requires serious time.
Addressing the problems
PBC has conferred with Democracy Earth and considered a number of solutions:
Improving the Democracy Earth interface: There may be product design improvements which will reduce the burden to evaluate all apps, and potentially revote, more frequently.
This option seems experimental and might take months to prove its effectiveness.
Resetting or decaying the votes: We considered a clean state each month and also a decay function to help nullify our challenge with reevaluation.
This change would probably help but would not address the fundamental lack of incentives to revote.
Switching the voting population: There have been public requests to switch from Token holders to another voting cohort, such as App Miners, or Blockstack forum members.
-
Longterm, we still believe token holders are the right voting cohort—they own part of the Blockstack ecosystem and are financially incentivized to help the ecosystem become successful, which means supporting high-quality apps.
-
Switching away from token holders may, or may not, solve the voter engagement challenge. We would prefer to get the incentives right vs. hope a new audience will care more.
-
App Miners have conflict of interest that we predict will produce a poor ratio of signal to noise. The current token holder voting already has this problem, and we feel it is guaranteed to increase if App Miners vote on apps.
-
We feel forum membership is an abstract cohort and an inferior voting cohort to both App Miners and token holders.
PBC’s current conclusion
We may be wrong on some of the assertions or challenges listed above, but one thing is certain—it will take months of research and development from a dedicated team to test these ideas. The PBC team doesn’t want to gamble with your payouts while we wait for those results.
The PBC team is small. The Democracy Earth team is small. Our priorities lie in quickly improving, and reducing friction in, the App Mining ranking system.
We still believe that ranking apps based on evaluation by Stacks token holders is, in principle, a high-potential app reviewer. However, given:
-
The urgency of this problem
-
The current size and breadth of our token holding population
-
And the lack of proper incentives for voting;
The PBC team feels the Investor Community App Reviewer is not delivering the value it could and should deliver and that a resolution will take months.
Democracy earth has a ton of exciting projects and work ahead, including partnerships with local and state governments, and are currently testing out quadratic voting. We are extremely close with their team and it is no fault of theirs that we feel this is the best path forward for the time being. We will continue to support them.
Next steps
Our recommendation is to pause Democracy Earth, and the Investor Community App Reviewer, from the App Mining ranking until we can address the challenges described above. This change would go into effect May 1.
Feedback and thoughts welcome.
