How to obfuscate file names in Gaia

Apps Storage (Gaia)
#1

Hey Blockstackers,

As you may already know, Gaia is a global decentralized storage system that Blockstack users use to store their app data. All files in Gaia are globally addressable using your username, the app name, and the filename (see gaia-gateway.com for details).

Some developers have asked me how to obfuscate file URLs in order to make them hard to guess (and thus hard to search for). To do this, you would do the following:

  • Generate a salt – a random string of >32 bytes
  • Store the salt as an encrypted file (e.g. salt.json) to the user’s Gaia hub
  • When the user signs in, load the salt.json file and store the salt in RAM.
  • Whenever you would call putFile(), generate the path to that file as sha256(salt + file_name), where salt is the salt bytes and file_name is the name of the file.
  • Whenever you would call getFile(), generate the path to that file as sha256(salt + file_name) (as above).

Then, all of the file URLs will be obfuscated, and the obfuscation is on a per-user basis. Someone that wanted to check for the existence of a file would need to know the user’s salt (which is private) and the file name.

5 Likes
#2

Would a slightly easier way for apps that have authenticated would be to sign the desired filename with the app private key, and using that instead of the random salt? That should remove the need for network round-trips.

2 Likes
#3

Yeah, that would work :). However, I’d recommend hashing the app private key first to get the salt so you’re not throwing around the private key all the time (I.e. you’re less likely to divulge the app private key via a bug).

#4

What if blockstack.js automatically obfuscates the filename using the app private key when you use putFile with encryption enabled? We’re already using the app private key in the encryption portion of that call. And presumably if you are encrypting something you probably want to make it hard for people to find the encrypted file as well.

1 Like
#5

:heart_eyes_cat: :heart_eyes_cat: :heart_eyes_cat:

#6

Ken, I think it’s great that the team is always thinking of ways to make the API easier to use by devs. It’s one of the key advantages I find in Blockstack.

Upon reading the suggested change to putfile I had the following thoughts:

  • I’m all for expanding the functionality of putfile as long as I still have access to the most basic functionality–writing raw data to gaia in an exact file of my specification.
  • Changes to putfile don’t affect existing usages of the method (i.e. the API doesn’t change in a way that breaks backward compatibility).
  • How complex will putfile get–I can think of quite a number of other things I want to do with i/o that I could put into this method?
  • As putfile gets more complex, the burden of testing it becomes more difficult as the number of permutations that are possible increases.

I find myself divided between two paradigms (these apply to the entire API, but I’m using putfile as an example):

  1. making putfile extremely capable and fully functioned with lots of functionality
  2. keeping putfile incredibly simple as a base level building block, upon which to create libraries that add specific value

I’m curious to hear your thoughts about this along with the rest of team’s?

At the moment, Stealthy’s core treats Blockstack’s API as though it were paradigm 2. This has been helpful to us by maximizing our flexibility. We’ve run into some sticky situations with paradigm 1 when backward/cross compatibility is not considered (i.e. the JSON difference between the iOS API and the Js API, or when the default to encryption in getfile/putfile was released).

edit: The problem with paradigm 2 being that the number of functions and lack of standardization can make interoperability more difficult. We’re hoping to push some of our upper layers out to Git and share them to try and address that, but it’s been slower going than I’d like.

2 Likes
#7

I’m with Alex here in being of two minds. I love how much Blockstack wants to build into its existing libraries, but I also want to make sure that things continue to work as simply as possible and can be extended as necessary by each individual developer.

I think Jude’s original solution and Will’s follow up make the most sense because then the customization is at the app level with no worries about another app having to use a layer of complexity that is not necessary.

Would love to hear more thoughts on this, though!

2 Likes
#8

Really awesome. And the gaia gateway is so cool ^ _ ^

For basic understanding. Another way to do this (but more fragile) would be to just do a fully random file name, and just keep an encrypted lookup/database file with those file names. Say:

{
  myWallet.json: "kEo34jbkajjk83eb351.json",
  secret_file.txt: "23bokJ34KLa6Kjmn4.txt"
}

Or if it’s a list of secret things being stored, an encrypted db.json with:

{
    items: ["kEo34jbkajjk83eb351", "23bokJ34KLa6Kjmn4"]
}

This salt proposal is not really all that different, only that you’d have and use well-known names and not needing to keep a lookup table?

Or are there more differences?

#9

I like Will’s proposal more than the one I wrote up specifically because you don’t have to worry about storing any extra metadata (no salt.json, no lookup tables). This makes the obfuscation approach just as robust as using Gaia without any obfuscation.

2 Likes