TryMyUI allows us to dial in a specific audience using filters and screener questions. Assuming we get this working well, it should help make the testing more relevant and fair. We’ve discussed this with TryMyUI a few times and there are a few complications:
Information overhead: Trying to keep track of the specific, and constantly shifting, audience requests from 50+ apps will become messy and error prone.
Audience advantage: This could shift the nature of the ranking from: “Who has the best UX?” to “Who has dialed in the audience the most so their apps tests perfectly?” which is an outcome we don’t want.
No testers: Based on conversations with TryMyUI, it is pretty easy to narrow the scope of users so much that very few, or no, testers are available.
We want to solve this while maintaining the value this reviewer creates. We think we might have a solution. As a first step, we want to share some audience test data with TryMyUI and see how that affects tester numbers, test quality, and audience advantage. This won’t affect rankings we’re just gathering some info:
Only allow testers who are ____________ .
The blank is one personal descriptor and it can be professional, personal, societal, etc.
For example:
Only allow testers who are computer programers.
Only allow testers who are lawyers.
Only allow testers who are office workers.
Only allow testers who are college/university students.
Only allow testers who are parents.
Only allow testers who have been informed and understand Blockstacks mission .
I think the best tester for my app is somebody who understands Blockstacks mission. Maybe start the trymyui test by informing the user about basic Blockstack principals and setting the mood with some questions about the current state of technology:
Do you think Facebook and Google collect too much personal data? Yes
Do you think Facebook and Google have the right to sell your data without your explicit consent? No
Do you believe you should own your data? Yes
Did you know you have probably been a victim of a security breach related to personal data collected about you? Please check here: https://haveibeenpwned.com/ to see if you have been affected and note the number of breaches you have been involved in: 6
Did you know the technology behind Bitcoin, called Blockchain, can help you achieve true data ownership and could help avoid data being breaches in the future? I do now!
[Then enter into the trymyui test when you set the mood]
I think a successful tester will answer similar to above
I’d question if dialing in your audience is actually a harmful outcome. Why would that be the case?
After all UX is only important relative to the audience you cater to. Everything in business is about who your user is and catering to their needs.
In general, this could be interesting. It forces teams to think beyond targeting vague audiences like “people interested in Blockchain” (this is bad because their motives are non-uniform).
I think profession based/ role based selection could work well. eg) We’d definitely use the " Only allow testers who are college/university students option.
AFAIK, these companies don’t sell your data, they collect lots of data about you and know more about you than you’d like them to in order to sell Ads against your profile. This article does a good job of explaining what’s going on: https://motherboard.vice.com/en_us/article/8xkdz4/does-facebook-sell-data
It’s important to be clear about what centralized companies like Facebook and Google do thats harmful otherwise we lose credibility as a movement for fear mongering rather than dealing with facts.
For OI ConvertCSV: Only allow testers who are blockstack id owners AND are open source fans
For OI Timesheet: Only allow testers who are blockstack id owners AND are freelancers
For OI Chat and OI Calendar: Only allow testers who are blockstack id owners
(Is there an open issue at trymyui to allow authentication with blockstack id??)
Having provided that I would generally vote against dialing in audiences. Having tested OI ConvertCSV by a freelancer (expecting easy to use) would give me probably a much different result than having it tested by an open source fan. In the one case OI ConvertCSV might score 30 vs 70. Is OI ConvertCSV now a good or bad app globally?
I think app mining should aim for better apps that are available and interesting for the general public, (giving audience specific apps a disadvantage for this funding approach). As we are still in early days it is hard to not embrace all developers. Therefore, I would propose to have TryMyUI compose their final score by two subscores, one provided by the general audience and one by the selected audience and then weighing the two scores differently over time. Now focusing on the score of the selected audience, later on the other.
Discussing what GAFA do and do not is off topic here I guess, let’s have a discussion in a new thread.
For your startup and product, dialing in a specific audience is a great idea. I wasn’t suggesting you shouldn’t. I’m suggesting that TryMyUI testing could get distorted, to the point of being useless, by over-focusing audiences.
For example: Only allow testers who are:
Are engineers who understand blockchain
Already have a Blockstack ID
Already users of Afari
Have sent 10 messages on Afari
Currently love Afari
Hypothetically, if you could find these testers they would likely give you a near perfect score. But not very useful for evaluating apps. My suggestion from above is to hopefully split the difference: Get some narrowing down of audience without making the test pointless.
TryMyUI as a reviewer is generally oriented towards making Blockstack, and decentralized technology, easy and accessible for a broader audience. 99.9% of their reviewers don’t have a Blockstack ID.
I would probably disagree that we should focus the ecosystem solely on those who have already adopted Blockstack. That seems to suggest zero growth. But maybe there are some other compelling reasons I’m not seeing.
If you want an entity that acts as a reviewer that:
Evaluates apps based on user experience.
Only tests with existing Blockstack members.
Can somehow evaluate and filter the degree to which a person believes in data ownership.
Is impartial and generally trusted.
I think that sounds positive and we would consider it. Let me know who would be a good reviewer.
I think that in the current state the app tester should test the app after sign in. If testers have already an id, then this helps. Currently, a link to blockstack.org should be seen enough to teach users/testers about blockstack. I understand that this is not enough in the long run. However, dev tools could improve and provide better support in onboarding, something like blockstack.redirectToOnboarding().
I agreed with your logic that if we only focus on the actual user of a system, it’s very very hard to grow that system. But I think you’re missing a point here.
Blockstack don’t have much actual user. I identity myself as a tester (potential Blockstack actual user). Since the only app I often use is Stealthy and mostly use it for testing how Blockstack apps evolve. Another way to think of is I have a Mastodon ID too, but I am not a real Mastodon user.
I think more than 99% Blockstack IDs are testers or some kind of bots (I could be wrong - It’d be great if someone make a poll - Are you a real/actual Blockstack User?).
Anyhow when reviewers have Blockstack IDs, we may have a chance to convince them Blockstack is (going to be) good and the dapp XYZ is (going to be) great - therefore reviewers should (consider) become Blockstack actual user.
I’m still thinking whether we need reviewers don’t have Blockstack ID or not because I’m not sure what’s our real target user - college/university students?
In the near-term, we assume we’ll be sending more new users to Blockstack than Blockstack will send to us. As the community of apps grows, that should flip for many of us. Thus, registering for a Blockstack ID is part of the onboarding process and it would be a mistake to remove that from the review process. If your app has poor messaging around Blockstack registration, that should come out in the TMUI reviews.
I see the responsibility of a good onboarding UX more on the blockstack.org site, not on the app publishers site. In particular, in the current state all top rated blockstack apps use more or less the same messaging (“sign in (with blockstack)” nothing more or less ). The app mining program should find the best apps, not the best onboarding methods. I am not saying that the onboarding should be bad.
I’m trying to move this forward with some specific audience examples we can test with. Please share them. Until that happens I’m feeling blocked on this topic. Here is the issue, I’m moving it the Icebox for now.
Then perhaps by community input, we can come to a consensus regarding what the tester qualification questionnaire should look like, to help ensure the correct testers test the correct category of apps.
Then at random, the testers that qualify for testing a specific category, are assigned apps to test for that round.
nothing more or less ). The app mining program should find the best apps, not the best onboarding methods. I am not saying that the onboarding should be bad.