App Mining Rewards Is Totally Broken

Apps App Mining
#21

+100 for making Gaia a requirement for app rewards mining!

I did have issues similar to Stealthy in regards to my app not always showing up on DE. Documented here https://blockstack.slack.com/archives/C074L9QQG/p1547739250356100

Also agree with mobile apps being at a huge disadvantage with the current process. The token bug really did destoy login for mobile. On web I was able to push quick patches and was at a huge advantage.

+100 on the positive side for taking our suggestions and using trymyui with relevent demographics of the person reviewing. The “video proof” that the reviewers did a due diligent job is great…but what happens if a reviewer does a poor job and it’s documented on video?

4 Likes
#22

Everyone, thanks a lot for your feedback. It’s very helpful.

Let’s first set the context here that we’re at a very early/experimental stage of App Mining, hence the pilot phase. This is not the “real App Mining”, this is the pilot run that’d help us iron out the issues for a smooth run of the main program. Even the main program has checks in-place for a “slow ramp up”, precisely to iterate and make the process better, smoother.

Any program like this will have issues. Period.

Decentralized app ranking is an extremely hard problem. People will try to game it. Some reviewers will not do a good job. Some app developers will occasionally get scores that should’ve been higher or lower. However, we can learn, improve, and repeat and that is precisely what we’re doing here. When we’re done, with the help of all of you, we would’ve solved an extremely hard but important problem and App Mining should rank apps on their quality in a transparent and hard to game manner, which is the goal here.

With that said, RE Gaia, I don’t think anyone is grandfathering any app in. There has been no discussion at Blockstack PBC to focus on the auth protocol as a way to reach Milestone 2. Gaia and decentralized storage is an important part of Blockstack apps and will get integrated. It’s a question of when and not if.

In the long run, any bad or low-quality reviewers will get voted out. In the short run, we can address the current concerns RE any specific reviewer. Nothing in the list of things listed in this thread is fundamentally concerning to the future of App Mining: they’re low-hanging improvements that can easily be made.

(Edited above sentence to clarify concerns about things being fundamentally broken vs quick fixes/improvements that are easily possible.)

Blockstack PBC team already pushes for transparency and already performs checks and audits. We can double down on this and push for even more transparency and audits as/if needed. Again, easy things. Remember that the goal is to make App Mining more decentralized and automated, so less reliance on PBC as that rolls out.

We’re off to an exciting start, we’re directionally going in the right direction, we’re getting valuable feedback from the pilot phase, let’s continue to improve and iterate!

7 Likes
#23

I don’t understand App Mining problem(s)?

Do we need app mining to incentives developers?
Do we need app mining to get more users?
Do we need app mining to make big investors happy?
Do we need app mining to make big promotion companies happy?
Do we need app mining to make our friends happy? ( * I make my friends happy many times for growing my company)

What’s the most important problem app mining wanna solve?

P.S. Have you guy ever think about that people like me don’t vote twice for any project? And don’t have time to vote on two systems. --> how many real users will vote on three system? (I guess None)

30%20AM
Screen Shot 2019-01-23 at 8.21.30 AM.png1240×754 114 KB

1 Like
#24

Muneeb, the voter issue is concerning. I am far from convinced that the app developer himself did not supply those votes. I think anyone who has taken the time to look at the information Prabhaav supplied would be far from convinced as well. You should be concerned about that because, good reason or not, it looks bad. I know you all are looking into it, but you, of all people in Blockstack, should be concerned about this and many of the other issues.

App Mining is a perpetual pilot unless Stacks becomes publicly available. So this is what’s real to all of us right now.

The truth is, I’m terrified of the time I go from first to last or close to last. I work my tail off, putting in close to 40 hours a week on Graphite AFTER my day job. I know what I’m doing is valuable, but the system seems so flawed that I’m just waiting for that shoe to drop and Graphite to suddenly rank behind apps that clearly aren’t as well put together or not widely adopted. Or just flat out worse.

So my point is, I need you to be concerned about the items on this list because if you’re not, it’s a red flag.

4 Likes
#25

I think we’ve addressed that issue at least three times in this thread already and confirmed that the vote is from a valid token holder. Do you recommend not counting votes of valid token holders?

2 Likes
#26

Please explain how telling people it’s valid when there is evidence to the contrary is addressing it?

2 Likes
#27

I stopped doing everything I was doing, cancelled everything on my calendar, and spent the next 5 hours getting details on every single issue. I’m not concerned that these are any fundamental flaws in App Mining that can’t be fixed. Please keep the context in mind of that line.

If I was not concerned about this thread I won’t be spending time here :slightly_smiling_face:

6 Likes
#28

The account is linked to a token holder who is verified and is not the same person as the app developer. Did he convince a token holder to vote for him? Maybe. We don’t know.

Token holders can vote however they want and we can’t do anything about it especially in the long run.

My question remains the same. Is the recommendation here to censor votes of valid token holders? If yes how do we decide who to censor and who decides who can be censored? It’s a slippery slope and we all know the answer: we cannot censor token holder votes.

2 Likes
#29

@muneeb, I’m glad you’re very optimistic about things and spent considerable time looking into our concerns.

I know you feel that the question around critic.id has been addressed, but unfortunately, it’s not quite clearly explained. I’m still stuck on @patrick’s last statement: “These are the facts we know right now and are investigating further as to how this could have happened” I don’t understand how an ID linked to the beneficiary app creator is being used by an acredited investor–I know there are scenarios that could be happening (i.e. a hack, the app creator is a delegate for the investor, the app creator is the investor), but I don’t believe that has been clarified. It’s a very bad look without more information.

When I look at the totality of this situation, I can see that our biggest problem is that we are a mobile deployment competing against a sea of desktop apps. You’ve spoken about our product and been very kind, helpful and supportive to us in our effort to create a fine decentralized messenger. Indeed we would like to see a solution to App Mining Rewards that is fair. In this review cycle, the results do not seem to meet that standard for us and it is our responsibilty to the community to point that out.

If you believe that Stealthy is the lowest quality communication App on Blockstack then I suppose you might not see much an issue with the results. However our experience thus far seems to indicate a different reality and the results are not just suffering a minor error, they are flat out agregious. It’s difficult to understand how the results have put in last place the only App available on the most common hardware that uses push notifications over an App that literally does not allow you to send messages because of thrown errors.

In the long view, I agree that things can be fixed, but I’m less optimistic than you because the short term result seems so far off the mark.

4 Likes
#30

Looks like the investor was OK with his vote being used this way. It’s the investor’s choice. We haven’t seen any evidence of a hack but can certainly try to explore more.

Let’s take a step back and consider what’s going on here. Right now Blockstack PBC checks to verify that token holders and only token holders can vote, soon this can be done in a decentralized way. Anyone can verify the votes, token holders can vote however they want, they’d just be Stacks addresses on the blockchain. We’ve double, triple confirmed that only valid token holders voted in this round. What more can be done? Are there recommendations for censoring this particular token holder?

The concern is “did someone cheat and voted when they couldn’t have”. The answer is no, only verified investor votes counted.

Let’s even consider the hypothetical example if some token holder was hacked in the long run. What are we going to do? Take a private key and return it to someone? Do a fork and roll back the blockchain? As far as the system is concerned, in the long-run, the token holder is whoever has the private key and the vote is for whoever the owner of the private key votes for.

RE app quality in ranking, that is the sole purpose of App Mining. That’s our main focus. If App Mining can’t surface top apps then it fails and there is no point in it. I have some specific feedback on Stealthy and the UX and happy to give it over a call.

7 Likes
#31

Imo the app mining process has been hopelessly flawed since the beginning. It became obvious after the very first test run.

A lot of focus was put into showing of how reputable the ranking algorithm is, as it was developed by ivy league game theorists.

But unfortunately that doesn’t matter. As with any kind of data aggregation BS in = BS out and currently the group with most skin in the game (unpaid app devs) get no say in what efforts happening in their ecosystem should get rewarded while reviews by people who don’t have any knowledge of our values or context of what we are trying to achieve get to choose what is valuable.

And the message we get is ‘this is all fine’.

I have to say this confuses me a lot. I do believe muneeb and the team genuinely want this to succeed and are taking what they consider to be reasonable action to improve things, but somehow I doubt that the long run will work in their favor here.

1 Like
#32

We’re all working to figure this out. Remember, the App Mining program is only a few months old. It’s silly to expect all kinks to be worked out by the 3rd month.

Make your voice heard by adding constructive feedback here.

1 Like
#33

@bodymindarts thanks for the input, we got game theorists to do the job of designing the game and going deep on it, if there are specific improvements to the game, please do post them as a Stacks Improvement Proposal. In fact, any changes that you or anyone else would like to see should be posted as a Stacks Improvement Proposal. The community should discuss / debate and then pick the things that work and/or have more support.

In the extreme case if App Mining is totally broken and app developers are not being heard, they’d end up going somewhere else (even to a fork of Blockstack where App Mining works differently). Decentralized systems and open-source code has this beauty that in the end it’s community owned, it lives wherever the community lives.

I’d say all of you should feel like owners of this. Blockstack PBC, or me, or Patrick, or any single entity can’t do this alone. We need to act as a community. Disagree, debate, iterate, and improve.

3 Likes
#36

Maybe make some sort of script that analyzes how efficiently it runs Like the lease amount of errors in the code the more money you make or the least amount of resources you’re using

#37

Normal people rarely use desktop to chat though. If a chat app which do not support mobile, for me it’s NOT even an app. It’s something else.

1 Like
#38

Thank you for your reply @muneeb. I know you aren’t doing an easy job, and I do believe you all are giving it your honest best.

I and my team spent 1 year designing an app that arguably:

  • is the most advanced (ie configurable) multisig Bitcoin wallet currently available
  • sets new standards in UX for multisig wallets
  • is the only wallet that provides these features in a non-custodial way

Have we achieved product market fit? Could we have done better? More clearly defined our target audience, is this a useful unit of functionality or is more needed for adoption? etc… of course there are 1000 things that we could be improving.

Building this app as an example of what can be achieved on blockstack and making it open source is what I have provided to the community and is the most constructive way I could engage. The thing is we built this with our own money under the assumption that:

  • if we prove that we are able to build a product at a professional level (= we are a mature team)
  • show that the product is innovative beyond just using some underlying building blocks of decentralization, but actually adds value to the target domain

we would be supported in a way that would enable us to sustain development, either by investment or other funding. This is the implicit promise I heard in the initial blockstack signature fund announcement https://www.youtube.com/watch?v=vQSbqwz4TPM&t=77s and one of the big reasons I chose to build on blockstack.

It turns out blockstack signature fund already has an investment in basically the only other start up trying to productize multi-sig (casa) even though they are:

  • not open-source
  • custodial
  • don’t utilize any of blockstacks infrastructure or currently have anything to do with blockstack

I’m not saying they don’t deserve the success they are having. They’re focusing on a different use-case first, high-security individual wealth storage. We are focusing on shared-ownership. They charge $10.000 a year from high net-worth individuals. With Misthos it is possible to setup an equally secure multi-sig wallet where 1 individual owns all the keys for for free.

Anyway… I’m getting side tracked. My point is after 1 year of self-financing I’ve now gone back to spending my time with things that earn money as I must support my family. I don’t have time to engage in this community and I don’t see how opening a ‘Stacks Improvement Proposal’ will have any substantial effect on our situation. Besides I don’t think my lack of engagement, given the time and money investments we made is the reason we haven’t achieved a situation that is sustainable.

In the end I take full responsibility for leading the Misthos project the way I did. Obviously it would have been cool if things had turned out another way, but a bunch of mistakes were made (including relying too heavily on promises made by blockstack) that meant we didn’t achieve what we wanted to achieve.

Wether or not the current state of things as to how you are distributing money is a success or not is not for me to decide, or fix. Thats on you. Due to time and financial constraints in my life I’ve voted by exit (at least for now).

4 Likes
#39

I love this line. We - Asians have an idiom “Blame yourself first,…” - respect!

2 Likes
#40

Good to see this discussion going on here as I figured these issues in the last run and for this run I decided not to update anything i.e. put in any more hours to see how much Blockstack PBC team does in fixing the issues. The situation came out as worst. I have actually closed the project down, going to open source it and leave it there. Already told @Gina to remove it from the app mining.

Here are my two cents, I guess I can be honest and don’t need to go anywhere else to complain.

The main problem is of trust on the team. Trust as in your ability to run such a complex experiment with no prior experience or example, instead of you favouring a project. You can’t keep saying its a pilot run, there are big holes that are fundamentally wrong metrics. It’s not about a fancy algorithm. The game happens below the algorithm!

  • DE voting is centralized, just a bunch of people who most probably belong to the same organization, deciding on the future of the apps! Who are they? What is their experience? Nobody wants to tell anyone!

  • Producthunt votes mean nothing, people don’t even download the apps before or after the vote. I gave an example about Vegan Scanner app in the slack as well.

  • TryMyUI was a good touch, reviewers comments? Most of them said they can’t understand the product, “maybe its like for work or something”? Seriously? That’s the basis of providing a grant to a project where the user doesn’t belong to the target group, isn’t even briefed or didn’t bother reading the description of the app?

Here is the scenario - I worked day and night for 5 full weeks to complete Timski in it’s current shape. Developing decentralized group chats isn’t easy, the app was for iOS, it needed to clear a certain requirements to get certified while it is pitched against Android and Desktop apps, you can literally update them with a single click at any time of the day or night.

Apps like Timski can’t go and start charging people straight away without going on a full-on battle with Slack and Discord. That’s a huge undertaking, something that can’t even be thought of without initial substantial funding, you are essentially asking us to run companies, the competitors where are unicorns already! What algorithms giving away are $32 and $1.5.

This also have been highlighted by several devs now that without business/consumer/media/entertainment etc categories, you can’t identify which app is better than the others.

All of us are experienced developers and all of us has better things to do! Even while I am writing this, I could go and work on a better project that actually pays while I am still writing it here to make you aware of the problems, just because I liked the project. I am currently working with a layer 2 network to implement whisper/swarm and storage on smart-contracts on ethereum and I am very well aware of what you have done here technically, that’s really good work and good implementation, maybe instead of tying it a closed ecosystem, it can help and become COSMOS of the storage (more on that sometime later).

You can’t go around and say it’s a pilot program and expect that devs will hang-around for 6 months or 1 year before you realize that the fundamentals are wrong and you are reluctant on even accepting them. My advise will be to go back to the drawing board and think of app-mining from the scratch again and meanwhile close this version down, you are just making devs angry and you will lose whatever apps you have currently!

Meanwhile, if you want to give grants to a few flagship apps, you can actually do that easily without running app-mining.

Wrote all of that and I actually came here to let you know of an attack vector! Think about it, I am sure your algorithms will rank it much higher:

Current app-mining requires only auth. I was stupid to think of Slack etc. I should have just built an auth for the Vegan Scanner app (useless, pointless, even while allowing the users to use a skip button). 186 PH community votes, TryMyUI would love it because there is no learning curve, the app’s design is shiny! It has about 100 daily downloads, total users above 8000, so future metrics proof as well. Wouldn’t matter if DE voters never gave it any vote, PH votes and trymyui would have got better grant than Timski anyway! Just punch-in these numbers in the algorithm to know! Until unless, Blockstack PBC would actively block the app by itself but giving what reason?

Then I have another 10 year old app on the Android with more than a million downloads, an auth there? Why are we even spending this much time on making new apps with decentralized storage?

Anyway, I will sign-off from here now! I know everyone’s intentions are really good but please do listen to what devs are saying here, this project does not have a lot of devs and new devs aren’t coming to decentralized tech during this bear run!

All the best!

5 Likes
#41

+1 to most of the issues pointed in this thread.

I am the developer of YourNote.app and the payout we got was equal to 1 hr of my time. YourNote is a project that uses the core features of Blockstack and Gaia and many people use it too (unlike many other projects). Provided that, it’s very disheartening to see such a negative response from App Mining.

App Mining aims to promote the development of quality apps on Blockstack platform but how can we ensure the same when the promotion mechanism is so flawed and unfair. Basic apps with basic (blockstack) functionality get voted at the top and a full-fledged app is ignored.

It’s very tough to continue developing it after experiencing such a response (it’s almost humiliating). I hope Blockstack fixes this soon.

4 Likes
#42

Appreciate all the work you’ve done believing in decentralization and open-source. Startups are hard and everyone attempting them takes these risks. I personally went without a salary for years, racking up credit card bills, when attempting startups (Blockstack and one effort before it).

One lesson I’ve learned and you’d find in most startup advice is that fund raising is extremely hard and never a guarantee. “It’s not done until money is in the bank.” So I, personally, wouldn’t rely on any “potential funding” until it actually happens. Deals have a tendency of falling apart. Initial checks are harder to get than you thought and so on. There are great resources available on this online. Here is one: https://playbook.samaltman.com/

A quick note on Casa, yes the Signature Fund is an investor, and we’re excited about the work they’re doing. The fund is relatively a very small investor though, their fundraising success is because of their own efforts and the fund’s investment, in my view, doesn’t move the needle much. In general, the VC market is fairly mature with many players in it and a startup’s fundraising success depends on how well you do in that market. There are many many factors to successful fundraising and getting into that would be off-topic here. I’m always happy to give advice and share my lessons (as I’ve done with several other teams).

2 Likes