Stellar: Poor code quality with the need for improvements
The code is the basis of blockchain technologies and hence the quality of code is responsible for its success. Poor code with errors is one of the biggest enemies of DLT systems. That is why we, the Institute for Crypto Asset Analysis, analyzed the codebase of Stellar. The analysis showed several code issues and demonstrated a lower quality than the two previously analyzed cryptocurrencies, namely Ethereum and Litecoin. — Authors: Dennis Kowalzick, Philipp Sandner
Introduction
We at the Institute for Crypto Asset Analysis (ICAA) analyze the underlying code of cryptocurrencies and other DLT solutions through automated algorithms that determine a score about the quality of the code which is easy to understand and comparable. In the case of Stellar we analyzed over 100,000 lines of code and identified issues or possible improvements in over 50% of the code. For the most part those issues were minor and non-critical, but there we found also some sections that definitely require some improvements by the developers. ICAA aims to provide an innovative approach of reviewing the code of cryptocurrencies and to help investors, analysts and other blockchain enthusiasts in the decision-making process. But before we take a deep dive into the analysis of Stellar let us have a look at blockchain technology itself. For everyone who is familiar with the concept of Stellar, feel free to skip this section and continue reading in the next section.
The technology behind Stellar
Stellar was launched in 2014 by Jed McCaleb, founder of Mt. Gox and co-founder of Ripple, and former lawyer Joyce Kim. Much like Ripple, Stellar is also a payment technology that aims to connect financial institutions and drastically reduce the cost and time required for cross-border transfers. In fact, Stellar is a hard fork of the Ripple protocol.
Stellar’s basic operation is similar to that of most decentralized payment technologies. It runs a network of decentralized servers with a distributed ledger that is updated every 2 to 5 seconds among the nodes. The biggest difference between Stellar and Bitcoin however is the consensus protocols the two technologies use. While Bitcoin relies on the entire miner network to approve transactions (Proof of Work), Stellar uses the Federated Byzantine Agreement (FBA) algorithm[i], which enables faster processing of transactions by only using a portion of the network (also known as quorum slices) to approve and validate a transaction.
A transaction is only approved if all nodes within the quorum slice approve of the transaction and each node in the Stellar network chooses another set of “trustworthy” nodes. Therefore, each node has a unique quorum slice it verifies its transactions with. This shortened process has made Stellar’s network extremely fast and it is said to process as many as 1,000 network operations per second whereas Bitcoin for example process only about 7 transactions per second.
Review of the Stellar codebase
Stellar is equipped with peer-to-peer payment features, which are defined in the codebase of Stellar. The code is the backbone of the Stellar software, which runs on every node of the network. Therefore, the quality and the security of this codebase is crucial for the long-term success of the Stellar project. We used an innovative and new approach to analyze the quality of the code. With the help of a software, we automatically analyzed the C++ codebase of Stellar and determined the quality of the code. Since the Stellar code is published on GitHub it is easily accessible for everyone.[ii]
The specialized algorithms of the system automatically define a score for cryptocurrencies on a scale from -5 (worst score) to +5 (best score). This classification makes it easy to interpret the results and to compare it to other blockchain solutions. Technical remark: Our analysis is based on the Stellar code version from 2018–06–25 and draws comparisons to a release of the previous analyzed Ethereum code from 2018–03–12.
Analysis: Stellar has a low score of 0.30 on a [-5,+5] scale
In our analysis Stellar only scored a 0.30 which leaves quite some potential for further improvements. In order to classify this result, the second largest cryptocurrency, Ethereum, scored 1.69 and is ranked clearly ahead of Stellar. In the following paragraphs, we will examine Stellar’s score in detail by breaking the score down into four categories: design, metrics, duplications and code issues. Every category is again represented by an individual score, which allows us to narrow down the issues of the codebase to its components, and even go as far as to the exact lines of code. Figure 2 visualizes all of the subcategories of the Stellar version from 2018–06–25. In the following, each category is explained in detail and the results will be interpreted. As always, this part might be a little bit more technical but still plausible for non-coders.
Design issues. In this category the codebase is analyzed in terms of the design of the code. A good code design is characterized by an easy to follow and efficient structure. Even though the functionality of software with well-designed code or less well-designed code might be the same, it is desirable to develop code that every programmer understands easily. As mentioned in the previous reviews, the algorithms automatically inspect the design of the code and can detect anti-patterns. Anti-patterns are parts of code that appear to work but are not optimal constructed. Such patterns normally arise over time when new functionalities are attached or when changing developers contribute to the code. Anti-patterns may result in errors and make maintenance of the code very difficult. There are several types of anti-patterns.
The anti-pattern with the most occurrences (15) in the Stellar code is the God class (or Monster class), which contains many, often incoherent, functionalities in a single object. The result is a very complex code, which tries to solve a large problem at once, instead of breaking it into several smaller problems. Consequently, code refactoring becomes more complex and even small changes in the God class require system-wide tests of the effects. There are also several other anti-patterns but explaining all of them would be beyond the scope of this article. More important are the results of the design analysis. The Stellar code has a total number of 5,557 anti-patterns. Most of the design issues can be found in the “util” component (7), followed by “sqlite” (6). The evaluation of the design issues concludes with a score of -0.39. Figure 3 shows an example of anti-patterns within the “sqlite” component. Here we can see several Brain methods (e.g. line 7,512) within this component, which together can be classified as a Brain class. It is also important to mention, that Stellar achieved a lower score than the previous analyzed Ethereum, which scored a 2.29. This shows, that in terms of design issues Stellar has a lot of potential and needs to improve its code, especially as the amount of God classes might otherwise become a big problem.
Metric violations. The second category tracks the quality of the code with software metrics. Such a metric is for example “number of methods” (NOM). This metric counts the total number of methods (functions) in one class. It is obvious that a higher number of methods makes the code more complex and increases the risk for errors. Other metrics are for example “lack of cohesion in methods” (LOCM), which measures the cohesiveness of a class or “access to foreign data” (ATFD), which measures the frequency of access to external attributes from other classes. In order to determine the quality of the code, the system we used reports when an undesirable threshold of a metric is exceeded and calculates a score for metric violations. Within the Stellar code we found a total of 1,635 metric violations, representing a score of only -1.34, the lowest score across the four categories. Of those 1,635 violations, the majority (692) are LOCM violations, followed by “number of public attributes” with 414 violations. Compared to the Ethereum code, which scored a 0.47, Stellar scored quite significantly lower, leaving much room for code improvements.
Duplications of code. This category, as the name already suggests, searches for duplicated code within the codebase. Duplicated code is usually a problem, since it will unnecessarily increase the lines of code, which in turn lowers the performance and increases the software vulnerability. The Stellar codebase shows a good score of 4.40 with 2.59% of the code being duplicated. This score is slightly better than Ethereum’s 4.12 score. However, Ethereum’s codebase only showed 0.92% of the code being duplicated. Nonetheless both cryptocurrencies show very good scores in this category.
Code Issues. Finally, the last category focuses on code issues. In contrast to design issues, code issues apply only to a local part of the code. Depending on the characteristics of the issue, the impacts on the performance of the software may vary. Therefore, it is important to classify the implications of the detected code issues. The innovative algorithm defines the criticality within the categories of low, medium, high and critical. Stellar’s codebase unveiled a total of 1,509 code issues. However almost 94% of these issues fall into the category low and most of them are located in the “sqlite” and “soci” components. An example for a low code issue would be an unused label within the code. Such an unused label does not interfere with the correct functionality of the software but could be removed to make the code more compact. Of the remaining 6% of code issues, about 4.4% fall into the high category and less than 0.9% fall into the critical and medium category each. Of those critical and high issues, all critical and most of the high issues are again located in the “sqlite” and “soci” components. It is also interesting to point out that all of the critical issues were “uninitialized variable” issues, meaning that it is a variable that is declared but is not set to a definite known value before it is used. It will have some value, but not a predictable one. As such, it is a programming error and a common source of bugs in software. Figure 4 shows examples of critical code issues in the “sqlite” component. High and critical code issues do not necessarily lead to a dysfunction of the software but increase the risk of undesired behavior of the code. Software with less code issues tends to run more stable and should be the goal of every programmer. The occurrence of code issues is common in computer programming and is part of the development process. Nevertheless, the removal of code issues is necessary to improve the software. All in all, Stellar shows a score of 0.74 while Ethereum only received a score of -0.05 in this category. Therefore, further improvements are definitely necessary.
Summary: Overall the codebase of Stellar needs to be improved
The Institute for Crypto Asset Analysis analyzes the codebase of cryptocurrencies with automatic algorithms and translates the results into a score. In the case of Stellar, we encountered the first codebase that did not pass the quality gate as it scored an overall rating of less than 1.00. In order to pass the quality gate, a code must have less than 10 critical hotspots, less than 25 critical code issues, less than 10% duplication, and at least an overall rating of 1.00. Summarizing the results into hotspots according to the urgency of the issue, shows that only one component of the code contains critical issues and only one component is ranked as high. As Figure 5 shows, all of the hotspots can be located in the “sqlite” and “soci” component.
The majority of the issues can be traced to the category of metric violations and design issues, which unveiled by far the worst score among all of the categories in Stellar. Even though the code seems to function correctly, developers should review the critical sections of the code. The comparison of the Stellar analysis with the previous code review of Ethereum resulted in worse results for the Stellar codebase in three out of four categories. These findings can be confirmed with an overall worse score for Stellar.
In the future it will be interesting to monitor new developments of the Stellar code and to see whether the code will be improved in terms of code quality. In addition, the Institute for Crypto Asset Analysis continues to compare the results to other cryptocurrencies in order to determine, which blockchain solution has the highest code quality.
Remarks
The results shown in this paper are based upon an automatic analysis of the code. Please note that this analysis does neither represent financial advice, nor is it supposed to be understood or interpreted as solicitation to buy or sell any securities, coins or tokens.
If you like this article, we would be happy if you forward it to your colleagues or share it on social networks. If you are an expert in the field and want to criticize or endorse the article or some of its parts, feel free to leave a private note here or contextually and we will respond or address.
Do you want to learn more about how blockchain will change our world?
- Blockchain knowledge: We wrote a Medium article on how to acquire the necessary blockchain knowledge within a workload of 10 working days.
- Our two blockchain books: We have edited two books on how blockchain will change our society (Amazon link) in general and the everything related to finance (Amazon link) in particular. Both books are available in print and for Kindle — currently in German and soon in English. The authors have been more than 20 well-known blockchain experts in startups, corporations and the government from Germany, Austria, Switzerland and Liechtenstein — all contributing their expertise to these two books.
Authors
Prof. Dr. Philipp Sandner has founded the Frankfurt School Blockchain Center (FSBC). From 2018 to 2021, he was ranked among the “top 30” economists by the Frankfurter Allgemeine Zeitung (FAZ), a major newspaper in Germany. He has been a member of the FinTech Council and the Digital Finance Forum of the Federal Ministry of Finance in Germany. He is also on the Board of Directors of FiveT Fintech Fund, 21e6 Capital and Blockchain Founders Group — companies active in venture capital financing for blockchain startups and crypto asset investment management. The expertise of Prof. Sandner includes crypto assets such as Bitcoin and Ethereum, decentralized finance (DeFi), the digital euro, tokenization of assets, and digital identity. You can contact him via mail (m@philippsandner.de) via LinkedIn or follow him on Twitter (@philippsandner).
Dennis Kowalzick is research fellow at the Frankfurt School Blockchain Center and Alumni of the Frankfurt School of Finance & Management and the WHU — Otto Beisheim School of Management. You can connect with him on LinkedIn (https://www.linkedin.com/in/denniskowalzick) or contact him via mail (dennis.kowalzick@fs-blockchain.de).
Endnotes
[i] Federated Byzantine Agreement (FBA) algorithm: https://www.stellar.org/papers/stellar-consensus-protocol.pdf
[ii] Stellar codebase: https://github.com/stellar
