Currently, our maximum zonefile size of 4KB could support roughly 30 subdomain operations per update. However, if we increased this zonefile size, we’d be able to support many more subdomain operations.
In order to understand the decisions at work here, we can look at the two “downward pressures” on zonefile size. One thing to note is that these are forces felt by the operation of a blockstackd node — other participants should feel little to no effect. User profile loads may have to parse and fetch larger zonefiles, but these would be, at worst, 40KB fetches — possibly cached — and parses (which are very fast):
> zfStr.length
4235738 (4MB!)
> console.time("parseZF"); zf.parseZoneFile(zfStr); console.timeEnd("parseZF")
parseZF: 325.205ms
Okay – so what are the pressures of increasing zonefile size on blockstackd ?
-
Bandwidth – this will increase the number of zonefiles
blockstackdwill need to download from the network. The upper limit on this is the number of update transactions which can fit in a bitcoin block:max_transactions = 1024*1024 / 276 = 3800 max_zonefile_bandwidth = 3800 * 40KB / 10 mins. = 253 KB/sWhat is the expected throughput?
Let’s use Twitter’s new user rate as a baseline — 135,000 / day — if each of these corresponds to an entry in a zonefile, that’s 135k/300 = 450 zonefile updates or 18MB/day or 200 B/s. -
Storage – this is pretty similar story to the bandwidth. Assuming the blockstack network is processing roughly the new user / update rate of Twitter – that’s 18 MB/day of storage growth. This could become burdensome, but spinning disks are cheap (a $45 spinning plate would store ~56 years at this update rate)
Another way to frame this is the price to the user of storing this additional data. With a minimum update transaction size of roughly 276 bytes, that’s a bitcoin transaction fee of roughly $6 today. If I were to store those 40 KB on AWS standard storage (NOT glacier), I could fund 6,500,000 years of storage.
-
Blockstackd processing overhead — because, ultimately, we want to use this increased size to process more subdomains, the compute-time of additional subdomain operations must be accounted for. Because we are dealing with a spec and thinking about idealized workloads (e.g., 135k new users/day), I’m going to try to reason about what costs an optimized implementation would incur.
Verifying a subdomain may require two costly operations: a signature check, and a lookup of the previous subdomain entry. An indexed lookup in a SQL database has a throughput of ~4k / second — this is sufficiently fast that is not a real runtime cost. Signature checks, however, can be expensive. However, no one is suggesting that we’re going to have a protocol that doesn’t check signatures for user updates. So really my argument here is that if we want to support a ton of user updates and new registrations, subdomains are the cheapest way to do that, so we shouldn’t think of them as expensive.
Proposal
I think that we should upgrade to 40KB zonefiles. With zonefiles this size, we could support onboarding 150k users/day to the network, with 3-4 operations per block:
