Software Quality Assessment of Open Source

Proceedings of the 11th Panhellenic Conference on Informatics, 2007

The open source software ecosystem comprises more than a hundred thousand applications of varying quality. Individuals and organizations wishing to use open source software packages have scarce objective data to evaluate their quality. However, open source development projects by definition allow anybody to read, and therefore evaluate their source code. In addition, most projects also publish process-related artefacts, such as bug databases, mailing lists, and configuration management system logs. The software ...

2009

Traditionally, research on quality attributes was either kept under wraps within the organization that performed it, or carried out by outsiders using narrow, black-box techniques. The emergence of open source software has changed this picture allowing us to evaluate both software products and the processes that yield them.

International Journal of Open Source Software and Processes, 2011

There is an ample debate over the quality of Free/Libre Open Source Software (FLOSS) with mixed research results. The authors show that a reason for these mixed results is that quality is being defined, measured, and evaluated differently. They report the most popular approaches including software structure measures, process measures, and maturity assessment models. The way researchers have built their samples has also contributed to the mixed results with different project properties being considered and ignored. Because FLOSS projects evolve with each release, their quality does too, and it must be measured using metrics that take into account their communities’ commitment to quality rather than just the structure of the resulting code. Challenges exist in defining what constitutes a defect or bug, and the role of modularity in affecting FLOSS quality. The authors suggest three considerations for future research on FLOSS quality models: (1) defect resolution rate, (2) kind of soft...

2011

Crowston et al. (2012) pointed out that studies that compared the quality of FLOSS with proprietary software showed mixed results. They suggested that these results vary greatly by project and proposed further research on the antecedents of quality (Crowston, Wei, Howison, & Wiggins, 2012). This paper takes a first step towards addressing this issue by reviewing the FLOSS literature in order to answer the questions of what is quality and how is it measured. If different studies evaluate quality using different measures it will be like comparing apples and oranges. This would explain the mixed results of the FLOSS quality studies. Because of quality's extreme subjectivity, it is not surprising that studies comparing the quality of FLOSS with proprietary in-house developed

IFIP Advances in Information and Communication Technology, 2011

Software quality needs to be specified and evaluated in order to determine the success of a development project, but this is a challenge with Free/Libre Open Source Software (FLOSS) because of its permanently emergent state. This has not deterred the growth of the assumption that FLOSS is higher quality than traditionally developed software, despite of mixed research results. With this literature review, we found the reason for these mixed results is that that quality is being defined, measured, and evaluated differently. We report the most popular definitions, such as software structure measures, process measures, such as defect fixing, and maturity assessment models. The way researchers have built their samples has also contributed to the mixed results with different project properties being considered and ignored. Because FLOSS projects are evolving, their quality is too, and it must be measured using metrics that take into account its community's commitment to quality rather than just its software structure. Challenges exist in defining what constitutes a defect or bug, and the role of modularity in affecting FLOSS quality.

2009

Software quality is more than just conformance to a set of requirements and represents many attributes related to each other that make up a piece of software. An important part of this measure depends on the underlying processes and methodologies used in the engineering of software. We present an early exposition towards a quality model for open source software (OSS). We describe some basic notions of quality for OSS and present a basic model, where quality notions consist of various factors that influence such quality. The purpose of this effort is ultimately to develop a quantitative metric for software quality.

2015

Abstract: Software quality is more than just conformance to a set of requirements and represents many attributes related to each other that make up a piece of soft-ware. An important part of this measure depends on the underlying processes and methodologies used in the engineering of software. We present an early exposition towards a quality model for open source software (OSS). We describe some basic notions of quality for OSS and present a basic model, where quality notions consist of various factors that influence such quality. The purpose of this effort is ultimately to develop a quantitative metric for software quality.

IERI Procedia, 2013

Open source products/projects targeting the same or similar applications are common nowadays. This makes choosing a tricky task. Quality is one factor that can be considered when choosing among similar open source solutions. In order to measure quality in software, quality models can be used. Open source quality models emerged due to the inability of traditional quality models to measure unique features (such as community) of open source software. The aim of the paper therefore is to examine the characteristic features, unique strengths, and limitations of existing open source quality models. In addition, we compare the models based on some selected attributes.

Information Systems …, 2002

2009

The validation of software quality metrics lacks statistical significance. One reason for this is that the data collection requires quite some effort. To help solve this problem, we develop tools for metrics analysis of a large number of software projects (146 projects with ca. 70.000 classes and interfaces and over 11 million lines of code). Moreover, validation of software quality metrics should focus on relevant metrics, i.e., correlated metrics need not to be validated independently. Based on our statistical basis, we identify correlation between several metrics from well-known objectoriented metrics suites. Besides, we present early results of typical metrics values and possible thresholds. 1.