Everywhere you look today, blockchain technologies are in the news. Once limited to fintech, blockchain applications are now transforming business processes in finance, hospitality, medicine, supply chains, retail, and more. In fact, blockchain, as an alternative solution to classic technology, is reshaping the entire software industry.

It’s no wonder then that this growing demand is pushing the boundaries of what’s possible for this technology and its software. With this growing demand comes new pressures for reliable testing activities, enhanced collaboration within teams, and expanded use of smart contracts in software development. At the enterprise level, the possibility of greater rewards also comes with increased risks.¹

CodeWeTrust’s research aims to develop a systematic assessment methodology of blockchain open-source frameworks considering the source code quality as an integral part of the evaluation. We do not plan to rank the blockchain platforms or propose specific ones. Our work will be presented in two parts (a) the method (this document) and (b) the comparative results (forthcoming).

Why build on the blockchain? According to IBM², Blockchain offers superior advantages to classic technology:

• enhanced security
• greater transparency
• instant traceability
• increased efficiency and speed
• Increased automation
• lower costs

The spread of blockchain technology promotes institutional adoption in the public and private sectors. Governments have begun investing more resources in blockchain technology to take advantage of cryptocurrency volatility as an investment.³

In fact, Deloitte⁴ reports that IT-based organizations are adopting blockchain to improve the performance of their operational systems. The number of related projects launched within a relatively short time span demonstrates the allure of this technology in our research community.

The IOHK (the foundation behind Cardano) recently agreed⁵ with the Ethiopian government to use Cardano’s blockchain to provide digital identity services for their education system, while investing in Bitcoin of Microstrategy (a private entity) has yielded astonishing returns. As announced by CEO Michael Saylor, 100% ROI was realized with several of their purchases. In both cases, investors need to carefully assess their entrances, value, and possible risk scenarios.

A fundamental difference with blockchain development is that it has largely been orchestrated in the open-source environment. Bitcoin, the original blockchain system, was birthed in open-source. The Ethereum platform was initially developed by two central figures in the Bitcoin project; their project has since evolved into the largest blockchain community, measured by active projects on GitHub (see the sidebar, “Understanding the Ethereum ecosystem”).

The growth of app development, and open-source community power, is fueling the demand for out-of-the-box blockchain development platforms. MakerDao, AAVE, Uniswap, Solana, and many more ecosystems have been launched and developed during the last five years – something that would not have been possible in the past with any enterprise no matter how powerful. The growth of app development boosts the increase of the blockchain platforms.

Deloitte⁶ states that the open-source distribution of blockchain software enables a critical mass of blockchain coding efforts, talent, and overlapping objectives that accelerate an ecosystem with a common standard.

Startups and established enterprises are both faced with identifying a viable framework (platform) as a basis for their visionary development. Considerations include selection methodology, available features, and the financial potential of each blockchain ecosystem⁷. Blockchain 101 lists the top 10 Enterprise Blockchain Implementation Challenges⁸.

Ironically, software quality is not among the selection criterion, even though it is a major contributing factor to most of the raised risks.

Although every blockchain ecosystem is basically a software artifact, the reader will face a real challenge searching for references on software quality evaluation of the blockchain ecosystems. Is software quality low? or high? average? How is this compared with the quality of classical frameworks that use non-blockchain software? How can we access and compare the software quality of blockchain frameworks?

H.Gao and X.Yu⁷ surveyed the proposed engineering frameworks for blockchain development. As reported, the majority of blockchain frameworks incorporate static code analysis and code reviews as part of the standard development process, but none of these surveyed models propose a systematic approach to quality assurance of blockchain codebases, as a whole,

Deloitte analyzes the GitHub footprint of major blockchain frameworks, but the analysis is limited to commitments statistics, hence ignoring the key differentiator of a codebase: the quality of the code.

Upon analyzing several ecosystems, we have observed some unique characteristics that impact the overall software quality of blockchain frameworks.

Huge diversity of used technologies: Programmers typically prefer newly developed programming languages and tech stacks. Furthermore, they often use improved versions to ensure high performance and reliability.

A Forbes survey¹⁰ states that Blockchain projects that require algorithmic or computationally intense burdens of proof have no such luxury, at least not in the current generations of the technology. Significant efforts are underway to speed up blockchain transaction speeds so they become capable of supporting near-peer to credit card processing volumes and speeds. Once this standard is achieved mass adoption of cryptocurrencies as a true means of exchange will become possible.

101 Blockchain’s survey¹¹ reports the lack of adequate skill- sets as one of the major risks of blockchain adoption. In addition to software and hardware, you must also find qualified personnel to manage blockchain technology
blockchain technology. Blockchain technology is relatively new and is still evolving. At the moment, few people have the skills to support such technology. On the other hand, the demand for qualified staff is enormous. If somebody wants to hire skilled people, they will have to pay large salaries. The right people will cost.

Sharp increase in codebase size: Blockchain ecosystems grow up faster than any other cluster of software technologies. Blockchain frameworks are getting huge as they contain many alternative versions to include test and experimental applications. It is hard to curate and maintain an acceptable level of quality. We have found many obsolete repositories that haven’t been updated during the last couple of years, although are still in use.

Developers volatility: Arguably the development team stability impacts the software quality of a software growth base. While analyzing contributor behavior and associated commitments density, it became visible that the average volatility is much higher than in other sectors of software development. Developers seem to be obsessed with the latest trends, market cap, and growth of each ecosystem, rather than the tech stack used and innovation introduced. This is especially true in the case of decentralized applications.

Trend of code commitments density: An upward trend in code commitments (new features, bug fixes) clearly indicates increased interest and good potential, while reduced activity or developer commitments is a negative sign for the growth of the framework. A sharp decrease in the average number of commitments probably indicates a lack of interest by the open-source community to continue extending, maintaining and improving the platform.

Application security: Security is of crucial importance for blockchain frameworks as any breach could lead to huge finance losses, damage to reputation, and a lack of trust from the blockchain investors. History has shown that a single breach may cause the termination of a project and shift investments to other blockchain projects.

We typically consider only the security vulnerabilities patterns detected and the reported vulnerabilities on linked libraries.

Violation of best programming practices: Forbes¹² reports that “Perhaps the biggest point of failure is the general lack of cyber hygiene present in many early blockchain projects”. This statement fully aligns with our understanding. High-quality of defects, duplicated code, and hardcode tokens is a clear obstacle for new community members’ onboarding, as it  makes the maintenance and the extension of the project slow and expensive

Libraries aging: In addition to the fact that outdated libraries incubate vulnerabilities and cause lower performance, the widespread rate of unsupervised library aging suggests a lack of effective and risky governance that raises concerns.

Licenses: Discovered licenses require careful consideration. Even though blockchain frameworks’ growth is based on communities’ contributions, non-permissive licenses might be a future threat, for companies and developers are willing to develop a proprietary version or interface a public blockchain (probably a private permissioned blockchain). Exclusive use of permissive licenses is a risk too, as it permits companies to fork and improve the code without sharing back the improvements. In this case, the license discovery results should be considered only informative and not indications of quality.

These observations led us to extend our methodology, including additional metrics but keeping our standard quality definition

Our goal is to measure key features that affect the overall quality of the software in Defi, blockchain platforms and protocols and compare the results by cluster.

By integrating the c2m Automated Source Code Quality Assessment, you can simplify a multi-stage code analysis of the codebase. The analysis covers, and provides steps to mitigate risk in all of these areas:

• Tech Stack
• Development team analysis
• Code risks review
• Static code analysis
• Security review
• License compliance review
• Contributors overview

The proposed classifiers are split in two buckets a) quantitative and qualitative. In the first bucket belong findings usually are proportional to size of a codebase , while in the second one belog the findings raise concerns independently of their density.

One of the challenges of the blockchain frameworks is the bandwidth of validated transactions per time unit. Bitcoin, for instance, takes 10-40 minutes – depending on the fees involved – whereas tokens like Cardano or Solana could be handled almost immediately. The transaction speed matters as it indicates which cryptocurrency is more efficient. A higher efficiency means that the blockchain underneath the coin is more capable of transferring data from one party to the other and confirming transactions. Transaction speed can be influenced by several factors, including block time, block size, transaction fees, and network traffic.

Even though the transaction handling mainly depends on consensus algorithms, for our static analysis we consider the programming languages and the tech stacks that impact the block time and network fees.

We consider the defects, the hardcoded risks and some key structural elements (readme me files, comments, etc). The architecture evaluation is beyond the scope of this analysis.The number of findings is normalized considering the size of the codebase

The survey of the blockchain evaluation frameworks¹⁴, considers the lack of static analysis and code review standardization as a major source of failure of major blockchain frameworks.

We suggest the consideration of major best programming practices violations (defects, structure), the distribution of hardcodens tokens (APIs, Credentials, file names etc) and structural issues (long methods and classes).

strongly believe that the number of violations is not really indicative enough but the trend of violation distribution gives a more fair picture for a blockchain platform.

Our study examines vulnerability patterns detected through SAST (static security analysis) and reported vulnerabilities for linked libraries. It must be noted that the vast majority of the available code scanning tools (open source or commercial products) support one or another. Both methods are valuable! Patterns based vulnerabilities analysis prevent deployment of risky code, while the consideration of reported vulnerabilities appear on linked libraries simplifies the application security optimization

During the last years many new platforms achieved phenomenal financial results (marketup, daily volume, coin value raise etc.). However, it has been proved over the time that the stability is widely based on massive adoption and evolution of the related software. As the majority of the successful products offer open source

code, it is reasonable to expect developer activity that, although is not proportional, reflects the platform’s scaling.

We do consider the density of the commitment (new features and bug fixes) during the last year, the distribution of commitments over the time, and the volatility of the contributors.Moreover, we use as a classifier, the standard deviation of the top contributors over the last six months. Constant performance indicates dedication and faith in product development success.

This is another important element. The majority of the blockchain frameworks have been developed during the last few years. Linking of outdated libraries and especially those that are reported for vulnerabilities requires attention and direct replacement before any use.

We do consider the licenses in conjunction with the importance of each component for the ecosystem growth. Τhe study of licenses needs a lot of attention

Blockchain technology builds trust in an untrustful world. It models the mechanism of “ trust establishment” using software . The better the last works the higher the level of the established trust and bigger the platform growth potential. The quality of used software is a non-negotiable requirement.

We have analyzed considerably big parts of the most active blockchain frameworks, selecting these frameworks based on popularity, innovation, the number of contributors, and the speed of the network.We have assessed the core parts of each platform, adding the most active repositories during the last six months.

It should be clarified that our goal is to propose a source code quality assurance methodology that could facilitate a decision over the use of a specific framework and the understanding of the involved risks. We do not rank the blockchain platforms by any means.
We have used the latest version of c2m source code assessment tool (V4.2), that covers the required steps. A subset of the used results are published in: Codewetrust.com test-cases

Startup and Enterprises decided to leverage the powerful architecture of blockchain systems are challenged to select a suitable platform that will support their product’s value proposition. Although this decision, usually based on the analysis of financial growth of each ecosystem, the technical integration complexity, the maintenance of the code, the extensions and the modifications might be proved insurmountable obstacle for their work for their project, unless
they will consider the maturity and the overall quality of the platform’s codebase as selection criteria too.