Models and tools for FOSS quality

Martin Michlmayr

2008-01-31 04:47

There has been interest in quality models for FOSS for a long time. There are various concerns about FOSS and the quality thereof. Given that a lot of FOSS is produced by volunteers, how can we rely on the software? Is software developed in the public more secure, or can people use the source code to find exploits? It's important to have an objective assessment of the quality of a piece of software to address such questions. Furthermore, having good metrics allows users to choose between different software that offers comparable functionality. Given the large number of FOSS projects, this problem is of growing concern.

Software quality is a tough nut to crack. When you see and use a product, you will usually form a judgement as to its quality pretty quickly. However, if you try to develop rules for assessing the quality of a product you'll find that it's really hard. This is partly because there are so many different components that make up quality, and that different people put different emphasis on these components or see them in a different way. While quality has a subjective component, there are several objective components that can be measured.

There are number of researchers who are interested in developing tools and models that can be used for empirical studies of quality in FOSS. The EU has recognized the need for such models and tools and is funding not only one but several projects that study quality in FOSS:

QUALity in Open Source Software (QUALOSS): QUALOSS will develop a high level methodology to benchmark the quality of FOSS in order to ease the strategic decision of integrating adequate FOSS components into software systems. The QUALOSS platform uses tools to analyze two types of data: source code and project-repository information. The goal is to estimate the evolvability and robustness of the evaluated software products.
Software Quality Observatory for Open Source Software (SQO-OSS): This project is developing a comprehensive suite of software quality assessment tools. These tools will enable the objective analysis and benchmarking of FOSS. SQO-OSS aims to remove one of the key barriers to entry for FOSS by providing scientific proof of its quality.
Qualification and Selection of Open Source Software (QSOS): QSOS is a method designed to qualify, select and compare free and open source software in an objective, traceable and argued way. It consists of four different steps.
Quality Platform for Open Source Software (QualiPSo): QualiPSo is a huge project which focuses on a number of different areas. As part of their activities on trustworthy processes, they are interested in developing a Capability Maturity Model (CMM) like model for FOSS.

These projects are very ambitious but they certainly have the potential to make a great contribution. There is also quite a bit of overlap between these projects, which is why some of them have united and formed the Flossquality initiative (FLOSS stands for "free, libre and open source software"). These projects are relatively young, but I look forward to their results.

(Originally published on FOSSBazaar)