Split blockstack.js into 6 separate modules & repos. See the overly colorful chart below for the gist of it. If you prefer a longer text explanation, scroll past for more details.
Most dApp libraries understand that you don’t necessarily want or need to do everything that a decentralized ecosystem offers, just some of it. For Ethereum, you have something like the EthereumJS libraries. For Bitcoin, you’ve got BitcoinJS’s various libraries. Likewise, underlying changes to something such as Gaia shouldn’t require major changes throughout the whole library. Rather, we should have individual modules for individual facets of Blockstack. My proposal for these is:
blockstack-auth: Handles the asynchronous auth flow. Has tighter couplings to the Browser, and will likely need changes as Browser changes.
blockstack-storage: Handles all things Gaia. Has tight couplings with it, and should be the only thing that needs to change if Gaia has changes.
blockstack-profile: Handles user profiles and settings. dApp developers may use this if they want to fetch things like user settings and email. Browser implementations will use this for making changes to the profile. Directly uses blockstack-storage to fetch and save the profile.
blockstack-wallet: Handles generation, encryption, decryption, and signing with the user’s private keys. This library would only be used by a Browser implementation, there should be little to no use cases for it in a dApp.
blockstack-transaction: Not yet needed. This would craft and validate stacks transactions to be signed asynchronously in the browser implementation, when the token is live and tradeable.
blockstack-common: Everything that needs to be shared between these 5. Ideally this is as minimal as possible, just constants and utility functions.
These libraries combined would be used in blockstack.js to expose their functionality. The blockstack.js module could insulate some API changes in the sub-modules to maintain a stable API. However, application developers should reasonably be able to grab any of these individual modules if they only want to use certain facets of Blockstack, but I think ultimately we’d do well to deprecate the monolith module and have people grab modules as-needed.
Well done and reasoned. Although I think this is a terrible idea when you say repos, say released packages instead and it’s not so bad.
Having more repos makes it a lot harder to work with. Blockstack already has this problem with way too many repos, and making it worse will … uh, make it worse Mono-repo is really a beautiful thing (having worked with it), commits become far easier to do and read. Less “oh, no, that’s in a different project, maybe it’s not worth fixing”.
Every added repo is added overhead when you just want to fix a bug or do a change. Projects can use tree shaking to cut down on extras, this is not something you’d want to pass over to human-level. It doesn’t scale, and the benefits are tiny.
A thing some projects have done is to release several projects from one repo. That is possible for blockstack-js. Having @blockstack/auth, @blockstack/storage etc, and maybe even still a blockstack-js for most of us who won’t care about a few extra bytes in our development environment.
Look at the systemd project as a non-web example of a project that has many different projects in one repo (even udev, which has nothing to do with systemd and are used by all other Linux startup projects live in the same repo, since it makes changes so much simpler). In the web-world lodash lets you install and import individual packages, but they didn’t split their repo just because they release many packages.
I’m not sure I understand the benefits of splitting up the library into multiple modules, whereas the potential downsides are numerous (confusing to know which modules to include, where to file bug reports, where documentation lives, etc.) I also think its a pretty sweeping interface change which is a headache for developers and will doubtlessly leave us with stale documentation, for unclear benefits.
We can still offer a mono-module that has it all, and use that for tutorials if the developer experience is confusing.
As suggested before, a monolith repo with multiple modules is probably the way to go. All of it can live together.
A lot of this is a matter of opinion, I personally find this kind of separation of concerns easier to test and refactor. You see this pattern in a lot of large libraries, like the ones mentioned above, or even more comparably with Firebase.
But more quantifiably, we’d get reduced bundle size for simple use cases. Blockstack.js comes with a lot of cruft, weighing in at 232kb after minify and gzip. A decent amount of this comes from libraries that have nothing to do with authentication, which is probably most integration’s initially want.
Your perspective seems to be from the viewpoint of existing apps that want to integrate with Blockstack for auth/identity and already have both existing storage and encryption solutions–is that correct?
Old thread, but I’m a big fan of the idea of splitting into a mono-repo. The biggest benefit, IMO, is cutting down the bundle size. Now that we have support for stacks transactions, there are a ton of things included that app developer do not need, at all. They just need the auth and storage.
gaia.js – handles Gaia connection, authentication, and storage APIs. Ships as part of the Gaia repo.
blockstack-util.js – common routines like crypto and hashing that get used everywhere
blockstack-bns.js – for doing name queries and name lookups
blockstack-profiles.js – for manipulating and signing profiles (including adding/removing “apps” entries)
blockstack-wallet.js – for deriving Blockstack private keys and encrypting/decrypting seed phrases
blockstack-tx.js – for generating and signing Blockstack transactions
blockstack-auth.js – for generating authRequest and authResponse messages and redirecting the user to their authenticator
A minimal Blockstack authenticator would take a user’s Blockstack ID and owner private key as input, and then use blockstack-bns.js, blockstack-profiles.js, gaia.js, and blockstack-auth.js to process app sign-ins and update the user’s profile.
A more advanced authenticator like the Browser would additionally incorporate blockstack-wallet.js and blockstack-bns.js to enumerate the user’s Blockstack IDs to allow the user to sign in with any of them, and would additionally ship with blockstack-tx.js to allow the user to register names.
A Stacks wallet would use only blockstack-wallet.js and blockstack-tx.js to allow the user to spend and receive Stacks tokens.
Throwing my two cents in. If everything is split up into multiple repositories, I’d still want one main repo I could include in my project. Graphite uses almost all of the blockstack.js functions (public and private) and I’d hate to have to go hunting down which new repository contains the individual pieces I need.
Agreed on that direction for what would be split up. A nitpick on the naming would be to prefer using NPM “scopes”, which is becoming a standard for organizational packages. So we’d have @blockstack/gaia.
We’d definitely keep it simple to do import blockstack from "blockstack" and get it all.
I proposed this earlier, just resurfacing for consideration.
We can split the blockstack.js code into multiple packages but retain the blockstack.js npm import which acts as an all-inclusive package that bundles together all of the Blockstack related modules. There are 2 ways to go about this:
1. We can import the submodules into blockstack.js, and write code to tie the modules together and maintain the current API.
2. We can release multiple scoped modules from the blockstack.js repo: @blockstack/auth , @blockstack/transactions with the exception of gaia which might want to be in it’s own repo.
The good thing about #1 is that each submodule is independent and can have it’s own release schedule. Changes in the submodules won’t affect blockstack.js unless the dependency version is incremented.
Both of the above preserves the API for existing developers and provides new developers with an easy way to understand Blockstack and start building apps. Advanced developers that need performance and want to pick and choose the modules they import will have the ability to do so as well.
Like @jehunter5811, I’m just tossing my two cents in as well. I almost always prefer using one package when making something, even if it includes features I never use, rather than having to import many different packages to achieve the functionality of the one larger package. This is the reason why I still prefer using jQuery to having to import and use multiple different packages to mirror jQuery’s results.
If it is split up, it definitely needs to have a way to import the whole thing at once, like @hank mentioned.
Yep that’s a good point — I think the plan would have to be that npm i blockstack still installs what you get today. It’s just that under the hood, this would just be a thin layer on top of other libraries like say, blockstack/core, blockstack/tx, blockstack/auth, gaia.js.
This breakdown would work, though I think there could be a blockstack-gaia subproject as well, which combines BNS with gaia.js.
I agree with @yukan about using scoped modules from one repo. I’m convinced that the blockstack-* libraries should all be in a mono-repo (blockstack.js), even if they have different owners/maintainers and their own test suites. A single repo makes issue reporting and tracking much simpler.
Something that’s important in the above is specifying who the owner of each of those projects would be. That’s not necessarily the person who will be doing all the implementation work, but its the person who would be reviewing PRs and planning + approving API changes — and when that person does do implementation work, they need to designate someone else to do those reviews, and approve API changes. I think some of the trouble we got into with the current setup is that there’s only one such owner for all of the client libraries, which slowed a lot of things down.
Mono-repo is really a beautiful thing (having worked with it), commits become far easier to do and read. Less “oh, no, that’s in a different project, maybe it’s not worth fixing”.