A Classification of Data Quality Assessment Methods

Due to the increase in the predicaments of data handling, the need for improving the quality of data arises to reduce its insidious effects over the performance. If the cause of the hindrance is analyzed, it is better to commence the data quality improvement plan by assessing all the scenarios affected previously. Values for measuring the quality of data should be constituted in this. The quality of data should be measured so as to evaluate the importance of the information and how can it be improved. Nevertheless, the primitive matter is to understand that what and how quality should be measured as it is said, "if you can't measure it, you can't manage it ". [Peter Drucker] The foremost step here is to focus on the elements of the data that are considered critical based on the needs of the user in the business. The quality of data can be measured and improved using the metrics methodology. This writing tries to represent how quality of can be quantified for selected dimension. At first, several requirements for defining a metric for measurement are stated. Furthermore, analysis of metrics is discussed with respect to the requirement of the company to improve data quality. After that, on the basis of available approaches, new metrics for the dimensions completeness and timeliness that meets the defined requirements are derived. Lastly, evaluation of the derived metric for timeliness is done in a case study.

Nowadays, activities and decisions making in an organization is based on data and information obtained from data analysis, which provides various services for constructing reliable and accurate process. As data are significant resources in all organizations the quality of data is critical for managers and operating processes to identify related performance issues. Moreover, high quality data can increase opportunity for achieving top services in an organization. However, identifying various aspects of data quality from definition, dimensions, types, strategies, techniques are essential to equip methods and processes for improving data. This paper focuses on systematic review of data quality dimensions in order to use at proposed framework which combining data mining and statistical techniques to measure dependencies among dimensions and illustrate how extracting knowledge can increase process quality.

International Journal of …, 2011

We present a Heterogenous Data Quality Methodology (HDQM) for Data Quality (DQ) assessment and improvement that considers all types of data managed in an organization, namely structured data represented in databases, semistructured data usually represented in XML, and unstructured data represented in documents. We also define a meta-model in order to describe the relevant knowledge managed in the methodology. The different types of data are translated in a common conceptual representation. We consider two dimensions widely analyzed in the specialist literature and used in practice: Accuracy and Currency. The methodology provides stakeholders involved in DQ management with a complete set of phases for data quality assessment and improvement. A non trivial case study from the business domain is used to illustrate and validate the methodology.

Future Computing and Informatics Journal, 2021

Achieving high level of data quality is considered one of the most important assets for any small, medium and large size organizations. Data quality is the main hype for both practitioners and researchers who deal with traditional or big data. The level of data quality is measured through several quality dimensions. High percentage of the current studies focus on assessing and applying data quality on traditional data. As we are in the era of big data, the attention should be paid to the tremendous volume of generated and processed data in which 80% of all the generated data is unstructured. However, the initiatives for creating big data quality evaluation models are still under development. This paper investigates the data quality dimensions that are mostly used in both traditional and big data to figure out the metrics and techniques that are used to measure and handle each dimension. A complete definition for each traditional and big data quality dimension, metrics and handling t...

JUCS - Journal of Universal Computer Science, 2020

The paper proposes a new data object-driven approach to data quality evaluation. It consists of three main components: (1) a data object, (2) data quality requirements, and (3) data quality evaluation process. As data quality is of relative nature, the data object and quality requirements are (a) use-case dependent and (b) defined by the user in accordance with his needs. All three components of the presented data quality model are described using graphical Domain Specific Languages (DSLs). In accordance with Model-Driven Architecture (MDA), the data quality model is built in two steps: (1) creating a platform-independent model (PIM), and (2) converting the created PIM into a platform-specific model (PSM). The PIM comprises informal specifications of data quality. The PSM describes the implementation of a data quality model, thus making it executable, enabling data object scanning and detecting data quality defects and anomalies. The proposed approach was applied to open data sets, ...

Faculty of Science and Technology School of Information Technology, 2010

The assessment of data quality is a key success factor for organisational performance. It supports managers and executives to clearly identify and reveal defective data in their information systems, and consequently minimises and eliminates the risks associated with decisions based on poor data. Despite the importance of data quality assessment, limited research has been conducted on providing an objective data quality assessment. Researchers and practitioners usually rely on an error ratio metric to calculate abnormal data. However, this approach is insufficient in terms of providing a complete quality assessment since errors can be randomly and systematically distributed across databases. This study will introduce a decision rule method for providing a comprehensive quality assessment, which captures and allocates quality change at the early stage in organisational information systems. A decision rule can also be extended to answer important questions such as the randomness degree and the probability distribution of errors. These advantages will significantly reduce the time and costs associated with performing quality assessment tasks. More importantly, the efficiency and effectiveness of the decision rule for assessing data quality enables management to make accurate decisions reflecting positively on organizational values.

2015

Data quality (DQ) has been studied in significant depth over the last two decades and has received attention from both the academic and the practitioner community. Over that period of time a large number of data quality dimensions have been identified in due course of research and practice. While it is important to embrace the diversity of views of data quality, it is equally important for the data quality research and practitioner community to be united in the consistent interpretation of this foundational concept. In this paper, we provide a step towards this consistent interpretation. Through a systematic review of research and practitioner literature, we identify previously published data quality dimensions and embark on the analysis and consolidation of the overlapping and inconsistent definitions. We stipulate that the shared understanding facilitated by this consolidation is a necessary prelude to generic and declarative forms of requirements modeling for data quality.

International Journal of Business Information Systems, 2016

Data quality has significance to companies, but is an issue that can be challenging to approach and operationalise. This study focuses on data quality from the perspective of operationalisation by analysing the practices of a company that is a world leader in its business. A model is proposed for managing data quality to enable evaluation and operationalisation. The results indicate that data quality is best ensured when organisation specific aspects are taken into account. The model acknowledges the needs of different data domains, particularly those that have master data characteristics. The proposed model can provide a starting point for operationalising data quality assessment and improvement. The consequent appreciation of data quality improves data maintenance processes, IT solutions, data quality and relevant expertise, all of which form the basis for handling the origins of products.

Integrity, Internal Control and Security in Information Systems, 2002

This paper first examines various issues on data quality and provides an overview of current research in the area. Then it focuses on research at the MITRE Corporation to use annotations to manage data quality. Next some of the emerging directions in data quality including managing quality for the semantic web and the relationships between data quality and data mining will be discussed. Finally some of the directions for data quality will be provided.

Techniques for assessing data quality along different dimensions have been discussed in the data quality management (DQM) literature. In recent years, researchers and practitioners have underscored the importance of contextual quality assessment, highlighting its contribution to decision-making. The current data quality measurement methods, however, are often derived from impartial data and system characteristics, disconnected from the business and decision-making context. This paper suggests that with the increased attention to the contextual aspects, there is a need to revise current data quality measurement methods and consider alternatives that better reflect contextual evaluation. As a step in this direction, this study develops content-based measurement methods for commonly-used quality dimensions: completeness, validity, accuracy, and currency. The measurements are based on Intrinsic Value, a conceptual measure of the business value that is associated with the evaluated data. Intrinsic value is used as a scaling factor that allows aggregation of quality measurements from the single data item to higher-level data collections. The proposed value-based quality measurement models are illustrated with a few examples and their implications for data management research and practice discussed.