Mercurius - Gaia Hub Explorer

[MichaelFedora]

Hey everyone!

Today I finished a small app I made to solve some issues that have been concerning me since I started getting involved in blockstack – real ownership in our data, even that which is stored on Gaia Hubs out of our control. I also wanted to know what was really being stored on my behalf, in an easy to get to way – and now there is! (It’s also completely client-side!)

Try it out here: https://michaelfedora.github.io/mercurius/

Source: https://github.com/MichaelFedora/mercurius

---

Mercurius is a small little file explorer that works by taking your mnemonic key and looking up many possible app-buckets on a gaia hub to see if something exists, and if so, then listing it’s contents.

In addition to seeing what apps are storing data, it can also decrypt any stored data, as well as download data (decrypted) in bulk via a zip file, all while being completely client-side.

Unfortunately this cannot “autodetect” every folder that you technically own – due to the way Gaia works, every bucket is owned by one address, which is why I have to scrub app lists and urls in order to generate many possible app buckets, and then ping the gaia hub to see if anything exists there.

Limitations

Currently this is limited to your root identity and the “official” Gaia Hub at hub.blockstack.org. You also cannot upload, move, copy, or delete files, but can only download and view.

App Naming Scheme

Currently there are a couple of weird app tags:

• Normal apps (from app.co/api/apps) are displayed normally
• Apps from the blockstack-browser repo have a * next to them
• Apps from your profile.json are shown in url form (without https://)
• Apps that I manually added to track have ** next to them (like XOR Drive)

–

Soli Deo Gloria

cc @git and @Yummyblock – a solution to your problems raised in that thread =)

[markmhendrickson]

I’m really happy to see you take the initiative here, @MichaelFedora. I’ve been meaning to sketch out some UI designs for a data explorer myself, so this is a cause dear to my heart.

I’m hesitating, though, to enter my mnemonic – not because I don’t trust you but because I’m wary in general to enter it anywhere aside from the Blockstack Browser. I imagine other users will feel the same.

Is there no way to accomplish this currently with regular Blockstack auth?

[friedger]

@markmhendrickson The mnemonic is required as this explorer works across all apps.

@MichaelFedora I share the same concerns that we should not teach users that it is ok to enter the mnemonic.

There is an open issue for the browser: https://github.com/blockstack/blockstack-browser/issues/1825 and for collections: https://github.com/blockstack/blockstack-browser/issues/1895

And there is an issue to solve the single player apps problem: https://github.com/blockstack/blockstack-browser/issues/1887

[git]

@MichaelFedora I tried it and found some issues.

1. It Seems work well on some applications with gaiahub except Xordrive. I stuck on looking up xor drive app bucket.What about your side?

2. Hope your site use blockstack auth instead of keychain sooner or later.

3. I can download jason files of each application, how can download data in bulk via a zip file?

4.After I downloaded the files from blackhole, opened it and found it all messy codes.

[MichaelFedora]

@markmhendrickson

Yes, it’s impossible to do it without entering your Mnemonic key – but if you are unsure, feel free to check out my code, or, download the release from here and open up “index.html” yourself, running it on your own machine. Also – feel free to make a dummy account, log into some apps, store some data, and then testing to see how it works (looking at the Network tab to see if I do anything weird with your mnemonic key =) (hint: I don’t!)).

The reason it’s impossible is because every bucket can only be “opened” by its corresponding private key – and these private keys are derived from the root private key, which is from the mnemonic. It’s unfortunate but necessary, then, to require the mnemonic because the traditional blockstack auth gives no ability to get the app codes (I also want to fix this in the future, but it would take about a year and a hefty sponsorship).

---

@friedger

Thanks for the links, and Blockstack is free to contract/sponsor me or just use my code (it’s under MOZ 2.0) if they feel like implementing it, but I have a few personal projects I’m juggling, one of which will use this code as well. I do agree the single player apps problem would help immensely so the scrubbing can be eliminated, saving resources and time.

---

@git

Apps store things in their own way – I don’t know why XOR Drive is broken for you, for me it’s a bunch of {uuid}_part{n}, because of how XOR stores things. I can’t make it easier to download because I don’t know the logic for how they split the files, so I can’t just stitch them back together automatically - that part you will have to do yourself.

edit: think of this like you are looking into the Program Files/{app} folder on Windows, or the {app}.app file/folder on a Mac: data is stored in there however the App decides – whether or not it is unintelligible to the end-user.

With that said, there is a download button on the top right – simply click a file, folder, or ctrl-click/shift-click/drag a selection box over multiple files/folders and click download, and it will put them all into a zip.

[hank]

Although we don’t have a solution in place for how you can grant access to one app from another, collections will at least make it possible for this app to work with regular Blockstack auth. You’ll be able to sign in from Mercurius and request various collection scopes, and get the correct keys in response.

Great work!

[git]

https://michaelfedora.github.io/mercurius/

@MichaelFedora Sorry the my words caused the misunderstands.

I mean that I clicked that download button and then download some files in that folders. e.g. blackhole.run . But, I got messy codes,

[MichaelFedora]

Yeah so that’s binary information. Again, that’s on BlackHole storing it the way they do, I can’t do anything about it (maybe they encrypt it, or maybe they split it up like XOR Drive – I dunno!).

[jeffd]

Love this @MichaelFedora thx for sharing.

Would be great to display the contents of files in a panel off to right side.

[Yummyblock]

@MichaelFedora What are the next steps for gaiahub? If I already got the Jason files from xor drive and recall photos, how to use the contents in the Jason files( unreadable sentence) to downland them(zip,png format) from gaiahub?

[MichaelFedora]

@jeffd Love this idea, thanks for letting me know - I’ll look into implementing it after the weekend.

@Yummyblock You’ll probably have to create a short script in node.js to stitch the files back together. Use fs-extra, read each file part, stitch them together in a buffer / stream and then write them back out… AFAIK.

[MichaelFedora]

@jeffd

I’m going to actually hold off on that feature until I merge this with a bigger app I’ll be developing later this year.

Mercurius is a proof of concept (I seem to be a fan of those) I made in 2-3 days, mostly for prototyping the file explorer UI and Gaia logic, and will not be worked on anymore for the forseeable future (except for minor maintenance if needed).

Sorry to those I’m disappointing, but the “other app” is much more important =)

[MichaelFedora]

Updated to version 1.1:

• Update deps, scripts, and add migration index (important for upcoming project)

Access the “release.zip” by downloading the source code and accessing /docs .

Updated to version 1.2:

• fixed some bugs (list files pagination & not catching some errors)
• disable some buttons while “working”
• add ability to lookup a specific app (a core, needed feature that I should’ve added a during the first release)