How would something like a forum page work on Blockstack?

Apps Storage (Gaia)
#1

Hi,

I love the idea of Blockstack – a developer friendly way to develop decentralized applications where users can own their own data is a great idea.

I’ve looked through the tutorials and read through the blockstack.js documentation. I have a good idea of how I could read data from and write data to specific an authorized user’s account.

But, I’m having a hard time understanding on how a Blockstack app would be able to read data from multiple users in an efficient way.

Say for example, a decentralized web forum. How would you approach displaying a list of all posts from different users?

Any suggestions on an approach for creating a list view of content created by multiple users would be appreciated.

Many thanks,
Steve

1 Like
#2

This is a great question. The real trick is going to be in structuring the data so as to minimize the number of per-user queries. I can think of a few ways that can work:

  • Have a “curator” user maintain the list of usernames with each topic. You could structure a forum topic to include the list of usernames of users who have posted at least one comment. In addition, you would store the timestamps at which they posted. This would let you efficiently determine which users’ posts to fetch based on what page of the topic the client is on. To build the topic-to-name-list mapping, you would want to design the app to inform one or more “curator” users whenever you create or post to a topic, so they can build it up for you (more on that in a moment). Then, the read path looks like:

    1. Load up and render the forum topics from your chosen curators
    2. When the user clicks on a topic, load up the list of users and timestamps who posted
    3. When loading the first page, load up the forum posts from the other users in order by timestamp, cutting off at the page boundary.
    4. When the user clicks the ‘next’ button, load up the next batch of forum posts, and so on.

  • Build an indexer over the set of forum users. You can determine whether or not a Blockstack ID owner created an account on an app by inspecting its profile. Since you can already enumerate all Blockstack IDs, you can asynchronously crawl their profiles and find out who posted which forum post, and when. The forum post body can be encrypted, but presumably the application would store at least a timestamp so the forum posts can be stitched back together as a page (and then decrypted client-side). The indexer would be able to assemble and cache pages server-side, cutting down on the HTTP requests that clients must make. It otherwise serves the (signed and optionally encrypted) forum posts unmodified. The client could be written such that whenever a user stores a topic or forum post, it also sends the indexer a hint that new data exists and should be crawled (so it will show up before the indexer makes another pass). Note that since the data and names are already out there, anyone can run an indexer—the indexer is not part of the trusted computing base, and cannot lock you in.

  • Use an existing application to bootstrap forum post discovery. Users of Graphite, Stealthy.im, Postly, and others already maintain user-to-user links in order to share data. A forum could bootstrap topic discovery from this. Similarly, the client could bootstrap discovery off of the Gaia inbox protocol we’re developing, and could bootstrap off of our decentralized social network initiative.

I think the first option is probably the best option for a forum specifically, since forums are read-mostly and usually don’t need real-time message delivery. Regarding curators, what you could do is have a “bot” user account that you hook up to a server that your forum client contacts to inform it whenever you post a new topic or reply. The curator bot assembles the list of usernames and timestamps for the forum topics, and serves them back to other clients. This architecture would be great for creating a scalable multi-topic forum system like Reddit on Blockstack, since each “sub-forum” could be managed by its own set of curator bots (i.e. running on behalf of the “sub-form admins”) and could implement its own “sub-forum”-specific moderation rules simply by being diligent about adding/removing topics and posts. Users always own their own data, and users always own their own forum posts, and admins retain the ability to fight spam and abuse.

4 Likes
#3

Thanks for the detailed and thoughtful response Jude!

I am a bit disappointed that the “curator” user approach requires a server to keep the curator’s data up to date, and puts the control of the forum data viewing onto a user account. But, it is kind of a cool idea that any message can be posted to the forum, and you could rely on different ‘curators’ to determine which content you see.

The “indexer” approach leaves something to be desired as well, as it requires a centralized hosted index server (although maybe you could develop something based on IPFS). Although, you bring up a good point about the indexer cannot lock you in, and anyone can run an indexer.

The Gaia inbox protocol idea sounds the most interesting. Is there a rough idea of the timeline for this to be released?

Thanks again,
Steve

#4

I am a bit disappointed that the “curator” user approach requires a server

Depending on how you structure the client, you wouldn’t need a server in the traditional sense. If I were to build this, I’d set up the curator bot on a Raspberry Pi on my home network, and connect it to a WebRTC relay endpoint in e.g. Google AppEngine (or some other PaaS that I don’t have to maintain myself). I’d list my WebRTC relay in the bot’s Blockstack profile, so the forum client could discover how to connect to it. When the forum client posts a new topic or message, it would also send a hint via the WebRTC relay to my Raspberry Pi, which would wake up my curator bot so it could automatically create/update the topic. That way, my private keys never leave my home, and if my WebRTC relay misbehaves, I can transparently swap in a new one.

puts the control of the forum data viewing onto a user account.

Fortunately, each user will store their posts and forum data in their own Gaia hubs, so if my curator goes offline or for whatever reason gets malicious, it’s straightforward to set up a new curator (i.e. use the indexer approach to find all the forum posts for the sub-forum, and have the new curator sign them all and re-build the sub-forum topic list). This would be an improvement over what Reddit offers, for example, because it gives users a way to revolt against bad admins without worrying about the sub-forum getting deleted. It also gives communities the ability to split and join at will, again without losing data :slight_smile:

The “indexer” approach leaves something to be desired as well, as it requires a centralized hosted index server (although maybe you could develop something based on IPFS).

I like to think of the indexer as like a “write-through CDN”. It’s not part of the trusted computing base; it’s just an optimization on the read path that saves clients from having to query every user :slight_smile: However, IPFS won’t help you here either, since you’re still in the position of having to constantly “merge” a bunch of unrelated writes into a coherent view of the forum and publish a replica of it somewhere.

Like with my curator description above, the indexer logic does not need to run on a remote server. You could run the indexer on a local computer in your home network, and replicate the indexed data to a cloud storage provider of your choice (or IPFS, if that suits you). This is what I would do if I was taking the indexer approach—I’d run a home server to build up the forum view, and I’d give it a Blockstack ID which it would use to sign the index (i.e. the indexer has its own Blockstack ID, so clients can authenticate index state without having to trust the cloud storage). Then, clients would fetch and authenticate the index as needed, and then authenticate each forum post.

The Gaia inbox protocol idea sounds the most interesting. Is there a rough idea of the timeline for this to be released?

You’ll have to ask @aaronb :slight_smile: I think he’s already working in it.

1 Like
#5

Thanks again for the insightful responses. This has helped me understand architecting for Blockstack in a much better way.

This is a great point and powerful idea.

What I would like to see is an application in which users own all of their own data, and is decentralized and censorship-free by default, without the need to rely on a curator or an indexer. Unless I’m missing something, it sounds like the only way to do this would be for a client to iterate over every single Blockstack user.

It would be a big improvement to just have to iterate over Blockstack users who have joined the app. It would be cool if the Gaia inbox protocol would allow messages to be posted to apps in addition to just users.

Thanks again,
Steve

#6

Yes, I agree.
What if Blockstack provides such an “indexer”? It would help apps in that space to get off the ground much faster.
App developers could opt-in if they want their apps to be indexed.
Maybe running such an index might make sense for Blockstack anyways in the context of running the AppStore.

#7

@digitalwaveride, this is a great idea! We have an issue for it here: https://github.com/blockstack/blockstack-core/issues/802. Do you want to take a moment and add your comments to the issue tracker as well?

1 Like