Persisting data to a database for a blockstack app

Apps
#1

Hi all,

I am new to Blockstack’s technology and trying to get acquainted. I looked for a topic for a question I have but wasn’t able to find one.

Let’s say I have an app that needs to store complex data - such as relational data in postgresql, graph data in a dgraph, or document data in mongodb. Does the Blockstack storage api work in such a way that instances of these databases could be running somewhere and serving data to an application?

Also, in cases where an application is collaborative, say I wanted to clone a team task management tool like Asana and run it on Blockstack, how is user ownership of data maintained when there are multiple users adding data to a single application data store?

2 Likes
#2

Because of the way Gaia works there is currently no way to “hook up” a relational, or other datastore. Gaia just supports simple getFile() and putFile() file operations. We need to build higher order datastructures on top of this simple key/value store. We have started to put together some thoughts on how this might be organized if you want to take a peek: https://github.com/blockstack/gaia/tree/master/datastores_working_group

We are calling this feature “Multi Reader Storage” and it is just getting added in. You can follow the progress here: https://github.com/blockstack/blockstack-browser/pull/1129

#3

Hi, this is a bit old thread but I am actually searching for the same answer. I am using multi-reader example in my app, so getfile, putfile, lookup etc. is working.

Now I want to store app-specific data somewhere. Let’s say I am building a chat, I need the app to store somewhere the users who have added each other. For another example, let’s say in a decentralized slack version, I would need to store the user-created channels and their participants. If I store them in a mysql database on my server, that will become a single point of failure.

Is there anything in the architecture that is supposed to act as an “app identity” for the Gaia? I saw a personal profile is being recognized as "@type": "Person", in the profile.json, is there any type available for an “app”?

Where does the application data lives?
#4

Is this work still ongoing? This is a major hurdle for building true social apps on blockstack as most of the value comes from being able to query a centralized DB for insights. How can we still use MongoDB or GUN with blockstack?

#5

Try Radiks (https://github.com/blockstack-radiks/radiks)

#6

From what I understand, that library simply copies all the user’s data to an S3 bucket ( gaia ). I understand that it means the user can ‘own’ their data, but what is the actual use case for this library? The records will have links to records in the database that other users own. The models that are saved are only useful in the context of the centralized database. In the case of a social network, the user owns all the ‘likes’ models, but not the actual ‘posts’ models. Forgive my ignorance, but I’m still trying to understand how one would actually implement a decentralized social network. The founder mentioned there are already some example social networks built with blockstack - can you share links to them?

#7

@mikestaub Radiks copies the data to a database. It also handles encryption on a model property level. Then the radiks server can be used to query the database and create a feed of public or group-owned data.

For the social network, why would the user not own the posts model? It is still created and signed by the user, it is just plain text (or encrypted with a group key).

If you search on https://app-center.openintents.org/ for “social network” you find a few apps, most of them do not implement the social aspect, but these (closed sourced) apps do:

Afari has closed down for now…
I am about to add the social aspect to app.gitix.org (social git profiles), it is an open source project!

#8

What if a user likes a public post that they do not own? When the user exports their data, there will be many empty links like this.

My point is a very important one I think. The real value today is in the graph - how is the data connected. A user being able to export all the data they own is not very valuable because most of the value is gained from the network effect.

Even if the users could all export their data at the same time, there is a coordination failure problem.

1 Like
#9

@mikestaub,

I think about this a lot.

How can we build a graph database that respects user privacy?

Radiks does indexing and group keys very well with mongodb. It also has a “central” component. The problem to solve is how a central service would link the groups encrypted data into a graph data structure, how to generate the vertex and edges, then how to query the encrypted graph data. I always wonder if Homomorphic encryption could somehow play into this.

A quick google search returns an MIT paper called CryptGraph http://www.cs.cmu.edu/~pengtaox/papers/cryptgraph.pdf

Any thoughts? @hank @jude

#10

My 2c:

Graphs are hard within a decentralized network because you never know when one vertex is going to simply up and disappear, or how to manage the ownership of said vertex.

For instance, User A may like User B’s post, but then said post may be deleted and User A’s “like entry” points to nowhere. There’s also the fact that it’s impossible to scrub every single user’s “like entries” to figure out the count on a particular post. It’s also impossible for User B to hold all the likes in case they want to refuse User A from modifying their entry and unliking something…

As far as I can understand Radiks is more of a centralized cache of decentralized data. As far as I know, when data is stored, it is stored on both the correct user’s Gaia as well as within Radiks for quick fetching (and so an actual graph can be made easily without having to reconstruct it whenever the app boots up on the user-end). Radiks can then (I’m not sure if this happens, but in theory it could) verify every so often that the data cached matches up with data stored on Gaia so the user still owns their data, etc.

I don’t think there’s any better way than this, especially due to the amount of race conditions that could occur and throughput needed if every Gaia had a datastore backend (though I working on something akin to that with Hestia, I’m still figuring things out).

I may have totally missed the discussion, but Radiks is probably what you are looking for. Let me know if you have any questions on what I said.

2 Likes
#11

Matrix.org is using something called mega-olm to handle graphs. It is a solution in the chat world with group chat and e2e encryption where users continue chatting while offline in parallel and want to delete messages. From the user perspective deleted messages are just shown as blackened messages.

This could also work for the social network. If a post is deleted the ID is still valid and the graph stays intact, it is just that the content owned by the user is not available anymore. In this context I would argue that the user also owns the post model. The graph, however, is owned by everybody/nobody and can be reconstructed even if the content was deleted because the other user has still the ID as reference.

1 Like
#12

I’ve been thinking a lot about this too. I’m attempting to configure blockstack’s JWT for authorization within dgraph’s Dgraph.Authorization object.

I’ve witnessed intimately how this works with centralized auth i.e. OAuth and asymmetric key verification of JWT claims.

It would have to work a bit differently with blockstack’s symmetric keys all client side with the following schema:

const responsePayload = {
  jti, // UUID
  iat, // JWT creation time in seconds
  exp, // JWT expiration time in seconds
  iss, // legacy decentralized identifier (string prefix + identity address) - this uniquely identifies the user
  private_key, // encrypted private key payload
  public_keys, // single entry array with public key
  profile, // profile object
  username, // Stacks username (if any)
  core_token, // encrypted core token payload
  email, // email if email scope is requested & email available
  profile_url, // url to signed profile token
  hubUrl, // url pointing to user's gaia hub
  version, // version tuple
};

dgraph does support symmetric key secret based JWT claims… but I’m struggling to figure out how to configure it via Dgraph.Authorization

The schema for dgraph symmetric key (HMAC-SHA256 JWT) authorization of claims is:

# Dgraph.Authorization {"VerificationKey":"secretkey","Header":"X-My-App-Auth","Namespace":"https://my.app.io/jwt/claims","Algo":"HS256"}

As such I imagine there would need to be a trusted user/admin who’s private key is used to verify the claims server side?

I’m really hoping to figure out the model for authorization via blockstack’s DID. This seems like a keystone for the success/failure of the entire project.