We’re constantly seeking to improve the App Mining ranking mechanism, and by extension our processes with each App Reviewer. We’re learning this can be complex and we appreciate the reviewers iterating on the process with the App Miner community.
Below is some valuable feedback from Tim at TryMyUI along with my advice. As a product designer, I’ve learned to embrace critique from both customers and colleagues. I’m sharing these thoughts with the presumption that dispassionate feedback is welcome for those looking to improve their apps.
Onboarding design
Feedback from Tim at TryMyUI
I scoured through 272 test videos… In many instances, testers not getting into the app is actually an outcome of poor design . A lot of these apps have unnecessarily complex or roundabout signup flows, they link users away to Blockstack's site, etc. and the users (who of course don't have a fully formed understanding of the Blockstack environment) get off track.
Onboarding flows are always complex and challenging. I would suggest taking an expansive, extreme-ownership view of the challenge: Onboarding starts when potential users first hear about your product. It ends once those users are happily using your app, and have abandoned whatever previous solution or product they were using. In between, there are dozens of places your users could get confused, including right before, and right after, Blockstack authentication.
It is your job to find all those confusing places, and make it impossible for them to get lost through clear communication and guidance.
Humans.name , for example, displays an explanation of Blockstack ID after the user clicked the “Create my profile” button but before it sends the user to browser.blockstack.org. This app builder anticipated the user’s hesitation and reassured them before they could become confused.
In my opinion, it is impossible to over-invest in making your app simple and enjoyable for new users. Especially new users who might not be perfectly educated on your product, or tech savvy, or fit other expectations you might have.
Browser bugs
Feedback from Tim at TryMyUI
However, there's also bugs & usability issues with Blockstack itself that are contributing. I've documented these in a Google sheet, but to summarize:
- Incompatibility with all browsers except for Chrome
- Getting stuck loading forever during account creation
- The secret recovery key process sidetracks users and then diverts them to the Blockstack apps page
- If people choose "Deny" at the last step of sign-in, they get diverted to the Blockstack apps page
- If they check the confirmation email they get during signup and follow that link, they end up on the Blockstack apps page
In general, it seems that any deviation from the one "correct" pathway ends up dumping users into the Blockstack apps page, instead of keeping them in the flow of the app they were trying to access.
We take responsibility for the list above. All are known issues, and some have already been fixed in the latest browser release. The rest are in Github:
- blockstack/blockstack-browser#1816
- blockstack/blockstack-browser#1901
- blockstack/blockstack-browser#1900
- blockstack/blockstack-browser#1899
Details from Tim’s Google sheet:
| App | Observation |
|---|---|
| Twoblocks | All users signed in successfully but many ended up at the main apps list because Twoblocks itself has almost nothing in it |
| Fupio | Blockstack signup getting stuck |
| Zinc | User did not choose Blockstack sign-in |
| Aodh | Infinite loading |
| OI Calendar | After finishing signup, gets stuck on blank loading page trying to go into app |
| Dappy Wallet | Broken in Safari; infinite load time |
| Forms.id | Infinite loading; signup (on firefox?) leads to main Blockstack page |
| DotPodcast | Sign-in button broken on Firefox |
| Sundly | Safari sign-in broken; infinite load time during account creation |
| YourNote | Safari signup opens the main Blockstack signup instead of the app signup |
| OI Chat | “Don’t have Blockstack yet?” link takes users to Blockstack, they sign up there instead |
| Block Photos | No obvious signup link; home page directions link users to Blockstack website |
| SpringRole | Most users chose email signup instead of following the “Choose Blockstack” direction |
| Kit | Browser compatibility issues |
| Cryptocracy | “Get Blockstack” - takes users to Blockstack website (actual sign-in is “Go to Dapp”) |
| Breakout | Firefox sign-in broken; signup sends users to Blockstack; users don’t understand relationship of app to Blockstack |
Many of these specific problems relate to browser compatibility. In the latest releases of blockstack.js and the browser, we’ve tested all major browsers and all of these issues should be resolved. We’ve also reviewed the March videos for the top 12 App Mining apps and can confirm:
- They have all use the latest blockstack.js
- None have any observable browser compatibility bugs.
My overall advice would be upgrade blockstack.js, then focus on debugging and browser testing. Apologies for these browser bugs taking so long clean up, we are nearly there.
App Miners viewing the videos
Feedback from Tim at TryMyUI
Of the 46 App Miners who participated in the March round:
- 21 have logged in to view the latest results
- 4 logged in after Feb results became available, but not since the Mar results
- 16 have not logged in since before Feb results (but did see Jan tests)
- 5 have never logged in
I also looked at the app scores in light of the above and found some interesting trends:
- App Miners who viewed their Feb results saw an average score increase of 5.29 points from Feb to Mar; avg Mar score of this group was 58.08
- App Miners who did not view their Feb results (but at least logged in for the Jan tests) saw a slight average decrease of 0.56 points; avg Mar score of this group was 51.53
- App Miners who never logged in at all saw an average score decrease of 18.44 points; avg Mar score of this group was 35.68
While I still feel it doesn't make a ton of sense for the scores to fall significantly over a month, you can also pretty clearly see there's a correlation between app performance and actually using the feedback from the videos. Once we increase the sample size and factor in the rolling averages, I think these numbers will start to make even more sense.
I also suspect that many of these App Miners who aren't viewing the results just don't believe in the importance of design, or are not open to hearing negative feedback of their apps. Maybe these numbers would encourage people to use the videos.
Tim’s data suggests: If you watch the videos, and try to make improvements to your app, your TryMyUI scores go up.
Good luck on the next round and please keep sharing feedback. 