Based on the February App. Mining data and reviewer content, we’re now able to provide an update on the status of Blockstack’s progress as promised in this forum post (App Mining Rewards Is Totally Broken). See notes below about specific numbered issues from the table.
Issue Notes: Issue #3: Patrick Stanley mentioned in the January App Mining review that Blockstack has been talking with Product Hunt to get more information than just a number for the Product Hunt (PH) Team Score. The February data contained only a number. The Stealthy PH Team Score dropped from 8.85 in December to 6.25 in January and has remained there. Why? This is confusing. Stealthy underwent considerable UI and performance changes from January to February, which are reflected by Try My UI scoring (-.904 to .493). If the PH Team Score is about the product, why didn’t it reflect this? We believe the PH Team Score should be removed from App Mining until PH provides a detailed explanation about what influences it (note that we have no objection to continuing to use the Product Hunt Community Score in App Mining–though it’s game-ability is a separate issue to consider).
Issue #4: Stealthy’s TryMyUI score increased significantly this review period. However, in the reviewer videos we observed, we saw that testers were not referred to our web page and asked to review a blank page that had nothing to do with Stealthy. One of the testers voiced concerns that Stealthy was malware. Blockstack and Try My UI need to fix and address this problem–though we suspect it only affects mobile applications at present. See pictures of this below:
Issue #7: In the January App Mining meeting, we asked Blockstack if Apps had to be publicly available (i.e. could they be in TestFlight or invite only). Hank answered that internal discussion on the subject said “it probably shouldn’t be some sort of invite only beta”. Then it was expressed that Blockstack would like to move or get away from that. When–it doesn’t seem like something that would be difficult to do in short order? Regardless of what Blockstack does, there really should be publicly available rules around this matter and other App requirements available–especially when Blockstack is giving away $100k a month based on these ideas that are currently tribal knowledge. You can see the full discussion here at time-marker 1:20:50 https://drive.google.com/file/d/1saxZW0rZVyxYr-INjnZqkpwwA7J2NQQR/view
Issue #8: The critic.id issue appears to have been clarified by the Hermes creator on Slack (though oddly the post was deleted). Here is a screenshot of his answer:
There have been some prior discussions on App Mining for February. Here are our thoughts on those:
App mining and trymyui dont seem to provide any real value - We disagree. As we point out above, there are definitely some issues, especially as we’re observing for mobile Apps, but TryMyUI seems to actually add the most value at present. It’s not just perception but people actually using the Apps. We definitely agree with the thoughts of @nicktee in that thread.
@jeff, @hank, @patrick, & @muneeb, let me know if you have any questions about the review presented above. Stealthy will be doing this for the March App Mining results as well too. It is nice to see some progress.
I’m heading out to dinner so will come back and edit this with more complete answers later, but want to quickly point you and the community to https://github.com/blockstack/app-mining/issues. This is where we as a community are moving app mining operations to. Its where we will solve problems, and open issues to the public, rather than there being a black box.
Just going to take the time to address the statuses of items in the image:
They said they have done so and at the moment have reminded them once to ensure they do so. Seems like Product Hunt is not a terribly popular app reviewer. Let’s discuss what we should do with them in the github repo I shared.
Product provided feedback for new apps but did not for all apps. Request is out to them to do so for all apps, otherwise they are less useful.
Not sure why the label fail is used. This feels like more in progress and probably being discussed in the github repos issues.
Discussing ways to do this, and we can move into a github issue. Want to make this happen. In progress.
@larry is drafting something up as we speak as a direct way to reward founders ensuring their users have their fundamental digital rights maintained.
This needs to be improved. We are speaking with apps that are not functional and cleaning up as well as bringing on App Reviewers like Larry that can enforce certain rules through scoring.
Glad we got to a resolution and it’s exactly what we thought it was.i
Thanks for taking a look–indeed you may need more time to respond, but during this process, I’ve noticed a disturbing pattern that I’ve described and questioned on the github (and I’m repeating here because I believe the github actually narrows the discussion and excludes the broader community as I mentioned in my original post above): https://github.com/blockstack/app-mining/issues/34
"[@patrick] … it seems like concerns are raised in the forum and on github and generic, non-specific answers are provided but no action is taken. If the App literally does not do anything but let you log in, why is it in the rankings and receiving rewards? Essentially, ideas are now being rewarded instead of implementations of said ideas.
This is similar to the problem of non-publicly available apps being rewarded despite Blockstack saying this is not intended. At what point will someone actually draw a line and take action instead of stating what the rules are supposed to be and continuing to allow them to be broken? Is the desire to attract apps so strong that rules are ignored for the time being or is it just that there are no rules–just intentions that will be codified at some arbitrary point in the future?"
This might seem harsh, but keep in mind that enforcing stated rules is relatively simple in the cases cited and that it has gone on for multiple review periods. Other changes certainly require more time and do not have straightforward answers, but if an App literally does nothing and the community is reporting it, should it be getting rewarded as an App? Is it hard to verify a single case like that? It seems like a major blunder to me, but maybe I misunderstand the reward system and concepts are okay even without an implementation (that’s fine by the way–I just want clarity on that).
Me too, I do not understand what app mining trying to do also. I don’t even know where is the money come from? Who’s in charge? Who’s owner(s) of the money payout for app mining? Who can change the rules? etc
I meant IF I’m an early investor who somehow signed a contract with Blockstack PBC that I buy the Stack tokens with X, Y, Z conditions (e.g. Blockstack must use PH as app reviewer) - I will sue Blockstack if they remove PH.
As my friends told me that IF money belongs to a person A then he absolutely has the right to set/change any rules for App mining. (I believe it’s very fair)
But IF money belongs to a group then maybe someone can represent the group to make decisions or they need meeting(s) and voting(s) to decide what to do.
I’m very confuse about Blockstack right now. What’s it? is it a company? is it a decentralized group?
IF Blockstack is a company, can CEO Muneeb make any call (e.g. change Blockstack protocols, delete app mining, etc) he want? IF Blockstack is a decentralized group, do developers/early-investors/users have some powers (e.g. change app mining protocol, change Blockstack protocols, etc)?
@otis (if you’re here) please share your thoughts.
My understanding about app mining is that Blockstack PCB is trying to find a (semi-)automated algorithm to reward app developers with the goals stated on github (would be nice if we could login with blockstack there ). The program is an experiment, a challenge, there are no promises other than the rewards will be distributed according to the rules.
I also understand that the program is not suitable for all apps. However, we as the community should try as much to shape the program so that it is interesting for most of us (with the bigger goals in mind). I hope that concrete suggestions will be addressed best on github.
The forum is a good place for governance, feelings, and general exchange about this topic.
I am just a community member. You can call the app mining program as you like. I hope experiment is not insulting Blockstack.
The program pays rewards according to the rules/algorithm described in the documentation (https://docs.blockstack.org/develop/mining_intro.html): a number of reviewers provide a score for all apps, these numbers are mixed with the app’s score from the previous month and result in a final score for this month. The better the score the better the rank.
The program does not payout rewards to the most successful/best apps.
The challenge is to find a set of reviewers and a set of algorithms so the in the end the most successful/best apps are rewarded. This is and will be an ongoing process (also with a moving target). Just my 2 micro stacks.
For those folks not participating in the GitHub that @patrick mentioned, you might be interested in this proposal I created there for changing how App Mining considers Product Hunt Team Scores as a result his suggestion to take the discussion there:
Also, a reasonable answer to my earlier question above regarding rules and when they would be enforced was presented in this github issue comments:
For those community members not monitoring the github, there have been numerous updates to the FAQ addressing the rules and clarifying key points. You can see that FAQ here: https://app.co/mining#what-is-app-mining
I’ve filed an issue in the git describing Product Hunt’s team score and some suggestions for how we might address that issue going forward (@patrick and @Gina have been trying to get feedback beyond the rating number as well as reviews beyond an initial one from the PH team, but in the absence of a response to those requests, the system might benefit from the proposal): https://github.com/blockstack/app-mining/issues/55
). The program is an experiment, a challenge, there are no promises other than the rewards will be distributed according to the rules.
