I put together an app built on Blockstack that is kind of a decentralized version of Slack. Really, I just wanted a toy example to play with the technology.
Right now my implementation replies on polling other users Gaia storage to see when they have posted new messages to channels. I’m wondering to what extent this is acceptable? Would it be a better idea to stand up a centralized service somewhere to provide pub-sub notifications? This would take a lot of load off of Gaia storage (I would only pull files when there is actually a new message) but it would add a central point of failure, and also add to my operating costs (I’m no longer only hosting static content). Thoughts?
If you want to see what I built, the app is here: https://blockslack.io/ and the source is here: https://github.com/djnicholson/blockslack It’s only a few days work so far, so it won’t hurt my feelings if the polling architecture is a terribly bad idea
This is a good idea – polling will be inefficient and slow for users. It’s okay to use a “centralized” notification mechanism, so long as the mechanism (1) can’t see any messages (you could encrypt them), and (2) all of the users’ chat data is replicated to their Gaia hubs. That way, the notification mechanism has no state and is not relied upon for application correctness – only for availability. Also, this would make it easy to swap out different notification mechanisms without breaking the application. For example, different user groups could run their own private relay servers.
Best of luck with blockslack.io! You should consider reaching out to @alexc.id and @prabhaav about their experiences building Stealthy – they have a lot of relevant experience building a decentralized encrypted chat app on Blockstack, and have had to tackle many similar problems.
As for your polling question–I could write about this for an hour. Early Stealthy (i.e. desktop) used polling exclusively but the protocol was designed with some tricks in mind to reduce the polling load–you can do quite a bit with a heartbeat, timestamps, and messaging frequency .
Mobile Stealthy changed things significantly–especially the iOS limitations on running time and interval when not in the foreground. We use a scheme similar to what Jude describes (sending push notifications through traditional means without using it to transport data–awesomely though, if both contacts have added each other, you can opt out of this and still send / receive messages).
Polling doesn’t scale well–especially for more than 50 or so users, you will run into the GAIA read/write limitations quickly too. For Stealthy channels we use push notifications to effect a polling schedule–that means that a channel only polls for messages, in-order, from relevant users (those who sent a message). All data is still transferred GAIA bucket to GAIA bucket though. This scales reasonably well as we saw with our decentralized AMA.
Another thing you might do is use polling initially to establish a P2P connection (webRTC or something) and then communicate over that–your connection establishment frequency might only have to poll a few times a minute or less if your users are cool with that.
One more thing–there is a GAIA rate limit after which you will see errors–I don’t remember the number and didn’t have any luck in this thread but if you start seeing intermittent errors based on put activity, talk to Aaron B. to see about modifying your rate limit if you’re not employing another technology.
I plan to write a very simple websocket service that will allow users to (i) subscribe to a feed by id, (ii) declare that a feed has been updated and (iii) receive notification that a feed has been updated. I’ll still hit GAIA to actually send/receive feed data though. I’ll also have polling as a fallback should the websocket service be unavailable/unacceptable to a given user (and will later add some of the other polling optimizations you alluded to).
Initially I’ll host a single instance of the websocket service myself. I’ll make the websocket service URL a per-user profile setting (you tell other users which specific websocket service they need to connect to when subscribing to your feeds). The hope being that other people may decide to spin up their own instances and share with their friends. This makes the overall service less dependent on my operational reliability and would limit my costs.
Do you know if the GAIA rate limits apply per-app or per-user?
.