What is Software Quality? An Austrian Approach

The International Software Testing Qualifications Board provides an excellent preliminary definition of quality: “The degree to which a work product satisfies stated and implied needs of its stakeholders.” However, despite all its virtues, this definition still requires further clarification to fully grasp all of its business ramifications. Here is when the notion of value comes in handy.

Value Is Subjective

We learn from praxeology that individuals get to decide what they want or need (ends) and how they hope to achieve those aims (means). In other words, both “ends” and “means” are decided subjectively, meaning that they are chosen by individual humans who have different backgrounds, needs, desires, and preferences.

Since the nineteenth-century marginal revolution, all serious economists agree that the “value” of goods and services is subjective. Jesús Huerta de Soto explains in his introductory work The Austrian School: Market Order and Entrepreneurial Creativity that, properly speaking, “value” corresponds to how the acting individual assesses their ends whereas “utility” is about the degree to which the means satisfy their ends, again according to individual judgments.

Quality Is Subjective Utility

Given this theoretical framework, we can safely state that quality is a subjective notion. Stakeholders choose one product over another with the expectation that they will be better off by choosing that alternative product. The choice is not, though, a mere “whim” because ultimately every piece of software should satisfy an actual business need. However, it turns out that there are several ways to satisfy the same needs with different goods and services, like thirst can be satiated with either water or soda, and this is when individual preferences and desires come into play.

Since software development is a very human endeavor, it cannot escape the logic of human action. More precisely, the economic analysis of software quality cannot avoid touching upon the activity of those involved in bringing the software about. As such, stakeholders follow the action axiom: individuals engage in purposeful behavior when they can envision a better future situation than their current one and then make their choices accordingly. It is evident, therefore, that quality is closely related to the microeconomic notion of utility since it relates to the means. As we know from the theory of marginal utility, it comes in degrees: there are products that help us attain our goals better than others because we simply prefer some things over others (or the ordinality of value).

In such a scenario, we find an extra layer of complexity already being addressed by our starting definition: “stated and implied needs of its stakeholders.”

“Stated” basically means what the client says they need, following their mental picture of the product or the expectation of how it will look like. Although these judgments may occasionally be inaccurate, fall short of, or exceed what they actually necessitate, the resulting statements inform the very basis of the system requirements that will serve as the reference documentation for developers and testers throughout the whole software development life cycle.

“Implied,” on the other hand, is what naturally or logically derives from the stated needs, meaning that there are some needs that have yet to be discovered.

Entrepreneurial Aspect of Testing

Quality assurance (QA) engineers, also referred to as “testers,” are what Israel Kirzner calls “entrepreneurs,” although in an improper sense. Entrepreneurship, in his view, is creatively discovering human needs and desires and satisfying them with goods and services.

Now, it might seem strange at first that I am applying this concept to such an extraneous field as software testing. Entrepreneurs are the ones driving the economy forward, after all. They are the ones who coordinate the production processes, instead of simply participating in these processes as QA engineers almost always do. However, very much in the same fashion as entrepreneurs, QA engineers happen to discover implicit information that was never explicitly stated by the client nor the development team as these engineers engage in a creative process of discovery.

As the prominent quality specialist James Bach famously put it, testing is “inherently exploratory.” QA engineers unearth tacit knowledge that is unspoken, unwritten, and cannot possibly be formalized, which is also why testing cannot be fully automated since it relates to a practical know-how and not a scientific know-what. Besides unit tests and user interface tests, QA engineers mainly concentrate on integration tests, which is another way to say data transmission testing. The engineers make sure that there is a correct data exchange among components and systems and, therefore, play a crucial coordination role.

Bear in mind that all software systems are information systems at their core. These systems deal with how the relevant people obtain the information they require at the right moments. Nevertheless, one cannot simply reduce software applications to a mathematical procedure—where input is processed to return an output—as if the only relevant points of analysis and criticism were the inputs and the outputs. On the contrary, a lot of information is discovered about the process itself, as testers create new information by way of finding previously unknown defects (bugs), providing useful insights on the system state, and enhancing the communication between systems in their wake.

Testing, Data Quality, and Knowledge

Data sets are information, and interpreted information is knowledge. The “knowledge problem,” as laid out by Friedrich Hayek in his seminal work “The Use of Knowledge in Society,” indicates that the main economic problem is not simply how to best make use of the already available information (as neoclassical economics has it) but how to obtain the relevant information for the decision-making process. In Hayek’s own words: “It is a problem of the utilization of knowledge which is not given to anyone in its totality.”

Even though no one in a company knows everything about it, comprehensive data is still necessary because no business can be successful with bad data, and software exists precisely to bridge that gap. Successful businesses are those that excel at predicting the future, and data quality is essential for achieving this goal. Here, obviously, modern data science and analytics have a huge role to play in informing qualitative entrepreneurial predictions with actual, yet historical, quantitative data. At the same time, the activity of the data scientist is limited by the data quality itself. As the saying goes, “Garbage in, garbage out.”

In a sense, all quality assurance has always been—in part, if not entirely—data quality assurance. Since software systems exist to solve a business information problem, QA is mostly about ensuring that the data is created, updated, accessed, stored, transmitted, and deleted correctly and safely. Sure, there are nonfunctional aspects involved as well—such as performance or security—but the money is always primarily on the information quality, and everything else revolves around it.

Since software quality is the utility that stakeholders derive from the software product as it solves their explicit or implicit information needs, the job of QA engineers is to ensure the quality of the product but not define it—at the end of the day, that is left for the client and the market to decide.