Stacks needs a popular programming language (to build smart contracts) for faster adoption

Note: I have shared my view on discord. From a twitter space came to know that this is the right place to share ideas. I am also aware of the advantages of clarity programming language.

My Concern
I am aware that stacks uses clarity programing language and might have spent lot of resources building it. The problem with a new programming language is that it will make cost of on-boarding developers very high for the ecosystem in long run. Solidity was successful because ETH was the first one to bring programmability to blockchain.

Many of the newer L1s have realized this fact and are moving to popular programming languages

  1. ADA community building a library that will allow developers to write smart contracts in multiple programming languages

  2. Algorand coming up with AlgoKit 2.0 which will give developers the power to write smart contracts in python.

  3. Solana using a flavor of RUST for writing smart contracts.

if stacks has to the most successful L2 for bitcoin, we need a lot of developers to join the ecosystem without stacks foundation putting much effort.

I like the stacks community and would like it to be successful. Hope they will also come up with something that will help in this regard.

4 Likes

I was about to open a topic to get some feeling from the community on this same issue. I’ll use your space @vidhyanand, to expand on it if you don’t mind.

Here is a short intro to contextualize where my perspective comes from:

I’m a software engineer with 10+ years of experience. During my career, I’ve worked with a few programming languages like Python, Ruby, PHP, Java, and JavaScript in a production environment, with the last being my main one. I’ve worked with companies of various sizes in different industries, from small local start-ups to big international enterprises. In 2022, I moved to the finance industry.

Also very important, I’ve applied for a grant to kickstart the development of a new programming language targeting the STX ecosystem. You can see my application in the degens grants spreadsheet, and in this particular page I created for it https://vini-btc.gitbook.io/clarx/. Some community grant stewards recently reviewed this application (thanks for taking the time, guys) in this Twitter Space. I could only articulate some of my ideas around the topic in those documents alone, leaving the reviewers with a few understandable questions. I’m also expanding the ideas on those documents here.

The Problem

While liking a programming language is primarily a matter of taste, it’s essential to consider the long-term side-effect that some form of developer rejection could have on the ecosystem.

It’s hard to predict how the environment around Stacks will evolve, but we must agree that growth results from new functionality built on-chain. And who’ll be making this new functionality? I need to start speculating here, but it’s not a stretch to believe we’ll see a first run of new companies followed by more traditional institutional players.

When we start looking at what Stacks offers for developers in those two contexts, the software engineer on a start-up and the one in a big CEFI firm, I have a few concerns involving Clarity.

A start-up software developer is usually interested in their productivity and ability to ship quickly and reliably. That’s why frameworks like Rails and Django took off in the prime days of the start-up era. Nowadays, you see React dominating most apps’ front-ends and a rise of the back-end as a service with many services providing multi-language solutions for authentication, db, server-processing, logging, queue processing, etc.

There’s an increase in the usage of JavaScript in the back end, with the combo Nextjs + Vercel as the perfect example of where the industry is heading. Adopting JavaScript on the back end, even with all its problems, makes a lot of sense because it allows developers to do more with a single language. For a start-up, that’s essential because it means they can do more with fewer people. Another characteristic of the current ecosystem for the start-up developer is a massive emphasis on developer tooling and experience.

When you present a developer used to this environment with the task of learning a new programming language to push forward their start-up idea, you’re already placing a barrier. If you add to that the fact that the language is unfamiliar and the tooling might be lacking in many places, and you have a good technical reason to at least look for alternatives (maybe there’s a blockchain that allows me to do this in Python? or Rust? perhaps we can hire a Solidity developer and try it in multiple chains at the same time?).

The situation is similar for the developer in a bigger company, although they’d probably emphasize other aspects of a technological environment when choosing. Imagine some trading firm that wants to implement some form of on-chain hedging and is trying to understand the costs involved.

From a technical perspective alone, this company will be looking for a solution they can build reliably on top of while being able to scale. They would ask questions like: can our current team support the development of this new technology? Can we easily find developers out there who can maintain it? What do we think of this blockchain’s infrastructure, developer tools, and programming language? Within this company, most developers use Java, Scala, C/C++ and Python, and there’s a tendency for more resistance to change.

Faced with a more “niche” programming language, many of these developers would quickly turn down Clarity because it differs from what they’re used to.

The critical issue is Clarity’s syntax. Quoting David A. Wheeler on where this problem comes from:

Lisp-derived systems normally represent programs as s-expressions. S-expressions can represent a list of items by surrounding the list with parentheses (…) and separating each item by at least one whitespace, e.g., “(1 2 3)”. When using S-expressions to define programs, the operation to be performed is listed first, followed by its parameters. Thus, you would write “(f 1 2 3)” to invoke function “f” with the list of parameters “(1 2 3)”.

Note that you cannot write “2+3” with traditional s-expressions - instead you write “(+ 2 3)”. You don’t even write “f(1 2 3)” - the function’s name is inside the list! Most software developers find this nonstandard prefix notation hard to read. In addition, many definitions end up with a huge number of nested parentheses, since doing almost anything creates another list. Even if you are used to this, and have editing tools that help, this is a problem when trying to collaborate with others - it’s ugly, and many developers will refuse to use such a clumsy notation.

Notice that doesn’t mean Clarity is terrible. Quite the opposite. I appreciate Clarity’s design and its focus on writing secure smart contracts. There’s immense value in preventing several extremely costly bugs in the programming language design itself. I also think it is fair to say any experienced good developer can start writing sound Clarity code with small effort if they’re open to accepting its syntax “quirks,” like they could write C, Haskell, or Assembly code even if it’s new to them within reasonable constraints of time and resources because to me great developers are language agnostic.

But when we talk about scaling a technical community, we’re not only talking about great developers. The average developer is… average. They want to get their job done as fast as possible and don’t worry much about the code quality they’re putting out there as long as it does what it has to do. They’re also willing to spend little time learning a programming language, reading docs, or sometimes even not reading error messages. Grab a task, code it, move it to Done, and go to the next one.

Also, the fact that Clarity is currently accessible to anyone willing to put in some little effort doesn’t mean we couldn’t make things easier for the ones working on it daily. We developers spend way more time reading code than writing code, and the reality is that Clarity syntax is tricky to read.

The Solution

When I started researching what we could do to make Clarity’s syntax more accessible, I began by looking at what the Lisp-related communities thought of the matter and how people have addressed the issue in Lisp world. I stumbled upon the Readable Lisp S-expressions project by David A. Wheeler (Director of Open Source Supply Chain Security at the Linux Foundation): Github Repo. It is a tool that allows you to write S-expressions in a different, more reader-friendly way. In a short example this:


(define (factorial n)

(if (<= n 1)

1

(* n (factorial (- n 1)))))

could be written like this:


define factorial(n)

if {n <= 1}

1

{n * factorial{n - 1}}

The specification for this dialect of Lisp is very well defined in the project, so making it available to the Stacks community would be a matter of implementing it. The result would be a new programming language that compiles to Clarity. In my proposal, I called it ClarX.

But after implementing a compiler with a back-end that targets Clarity, this opens up a great set of possibilities in the front-end. First, we could implement new language features much faster than in Clarity. Imagine ClarX compiler has a feature flag that allows you to access experimental features like lambdas or workarounds to current limitations like extracting traits from tuples. This would speed up advancements in the development ecosystem and work as an essential source of feedback to what gets into the primary language in a dynamic similar to what the JavaScript community saw with Babel and other “transpilation” projects.

We could expand the compilation target to other environments and make it possible to write ClarX code that can be used in EVM-compatible blockchains. The sort of thing that would balance the scale of choosing Clarity in Stacks’ favour. Or think of a way to implement some form of module system. Implement a built-in linting and formatting tool. And maybe our community could move entirely away from S-expressions and go after something more C-like… In summary, it opens up many possibilities.

All that without compromising anything on Clarity’s side. This dynamic has played well in the past in other ecosystems. Erlang was made much more accessible with Elixir. CoffeeScript made JavaScript easier to digest for back-end engineers. TypeScript gave JavaScript superpowers. And all of them worked in a symbiotic way.

A project like this in the Stacks ecosystem could fuel healthy competition towards a better developer experience, and it will also happen anyway when the incentives align. But I think it would be wise if the community could work on it sooner rather than later so we mitigate the risk of losing great projects in our “early” days.

With a dedicated team working on this project, I’m confident it could deliver enough value that current successful companies building on Stacks would be interested in supporting it.

3 Likes

Consider Paul Gram’s Python Paradox

3 Likes

FWIW, I had the same feelings about Clarity (and I’m a lifetime emacs user!). Very difficult to read. I took a stab at creating a new language that compiles to Clarity. It’s here:

6 Likes

Wow that’s freakin’ awesome @movingforward!

1 Like

Nice @movingforward!

I do agree with the opening statements here, I also think S-expressions are difficult to read at a glance - but at the moment, Clarity is “the source of truth” on-chain, and changing that wouldn’t be a small task.

I’ve personally also landed in the conclusion that, for the time being, transpiling from other DSLs is probably the best and/or only practical solution, allowing developers to also review the generated Clarity code prior to submitting it to the blockchain. It would be really cool to see more projects like crystalscript, and even cooler to see supporting dev tools :slight_smile:

4 Likes

That’s very cool @movingforward ! Are you still using and maintaining it?

2 Likes

Crystalscript is a part-time effort, and I can only work on it when I have free time. The code is GPL so it’s available to anyone to update as well.

3 Likes

Great work here @movingforward ! What was your inspiration language when designing Crystalscript?

2 Likes

I actually was sitting down trying to decipher existing contracts and thought it would be nice to compile Clarity to some pseudo-language so it would be easier to understand. A de-compiler of sorts. I simply ended up creating a new language, that is really just Clarity, but not pure-functional and with infix notation. My background is in c, c++ and javascript, so I guess that’s what came to mind. I’m not a language expert, and I know there is room for improvement in what I’ve started.

4 Likes

Great work.

Could the clarinet-sdk support crystalscript? When I run yarn test or so, it would convert the crystalscript contracts on the fly to clarity, add it to the clarinet.toml or set of contracts and run tests.

2 Likes

Thanks! That sounds like a good idea.

2 Likes

Crystalscript is very cool @movingforward! I like @friedger’s suggestion of trying to integrate it with things like clarinet-sdk: ultimately any contract author would want to be sure to test the output of the transpilation because it would eventually be the clarity code itself that would be executed on chain.

But on the topic of Clarity alternatives as first-class languages in Stacks, this is something that would clearly require consensus-changes (therefore a fork). I understand the desire for more language support, and I agree that many L1s do seem to be moving towards existing languages or assemblies. But, there are a lot of considerations when it comes to altering the first-class language of the blockchain. Some of these are technical questions about how to transition and how to handle cross-language calls, but those are all pretty solvable. The biggest issue is “correctness”— changing the language of the blockchain requires that the new language be implemented safely and correctly. This, too, is solvable, but its something that requires a lot of careful consideration from the community and network as well.

Now, one of the major projects related to Nakamoto is the clarity-wasm project. This project aims to replace the existing Clarity VM with a WASM runtime, compiling Clarity contracts to a WASM bytecode subset. On its own, the goal of this project is to retain parity with the existing VM: a given Clarity contract should work the same in the WASM runtime as it does with the existing ClarityVM, but with much better performance. If the Clarity-WASM project is eventually activated in the stacks blockchain, that would provide a new pathway for eventual multiple-language support: a subsequent hardfork could allow contracts to be published directly as WASM. To be clear, this would still require very careful implementation updates and new protocol specifications: just because the WASM runtime safely executes Clarity code compiled to WASM doesn’t mean it could safely execute any WASM code. However, Clarity-WASM would still represent a major step towards multi-language support.

3 Likes

I think this is the money quote here. The security guarantee that Stacks offers is that smart contracts are decidable, which in this specific case means you (or your wallet) can do the following:

  • You can enumerate all the states your transaction can reach when it finishes running. Meaning, you can know exactly what your transaction can and cannot do before you send it.
  • You can determine which smart contracts can be reached from your contract-call (and which cannot be reached), meaning you can know whether or not flow control touches known-bad contracts before sending your transaction.
  • You can enumerate all the tokens and data spaces that your contract-call could potentially mutate, meaning you can determine whether or not your transaction touches tokens (or state) you don’t want it to, before you send your transaction.
  • You can determine the upper and lower bounds of compute budgets required to reach a halting state at each step in the transaction’s execution, meaning both you and miners can estimate the optimal fee/compute-budget ratio.

Clarity-WASM retains these properties, because WASM is just the intermediate representation of Clarity code. Furthermore, languages like Crystalscript retain these properties, because they compile to Clarity. But, these properties (and their security guarantees) all go out the window the minute we allow arbitrary WASM code to run.

If programming language familiarity is truly an insurmountable barrier to entry for new developers, then I think it’s worth exploring more Crystalscript-like approaches. I think these security guarantees above are underappreciated, but are highly valuable because of how they protect users from buggy code.

1 Like

What i meant from my initial post is something like a clarity-script where we can write contracts in typescript and then compile it to clarity and then deploy. This will ensure that web2 organizations that would require to build on top of stacks need not look for a clarity resource for development of smart contract. That is one way of ensuring easier adoption.

1 Like

Hi @muneeb ,
If we can get some support from Stacks foundation, we can build a library that can compile typescript to clarity. This will make it easier for the newcomers to build on top of Stacks.

1 Like

Hey there, just wanted to clarify that Muneeb is the CEO of Trust Machines and is not involved with the Stacks Foundation except as a collaborator along with many others in the ecosystem.

Now to the important bit: If you’re interested in Grants, I’d recommend visiting this page: Grants | Stacks Foundation and then contacting [email protected]. It’s important to note that most of the budget is allocated to mission-critical items for Nakamoto and sBTC as described on the site and in our previous comms, but there are always a few projects that we can squeeze in each quarter in addition.

To maximize your chances of grant funding, it’s always best to get something started, potentially via a Working Group in this case. When you can show some initial progress, funding from throughout the ecosystem tends to open up via one route or entity. 👉 Getting Started: What is a Working Group? · stacks-network · Discussion #504 · GitHub

I think most folks generally or at least loosely agree that a more popular smart contract language solution is a worthwhile experiment, however, want to be cognizant of the tradeoffs for security as well as the timing for this effort. The core developers and contributors are very focused on Nakamoto and sBTC and would likely need to be at least a bit involved here, so it wouldn’t surprise me if there were some bandwidth blockers until closer to delivering on those major upgrades.

2 Likes