Data Provenance: A Categorization of Existing Approaches

2007

visibility

…

description

20 pages

link

1 file

Abstract

In many application areas like e-science and data-warehousing detailed information about the origin of data is required. This kind of information is often referred to as data provenance or data lineage. The provenance of a data item includes information about the processes and source data items that lead to its creation and current representation. The diversity of data representation models and application domains has lead to a number of more or less formal definitions of provenance. Most of them are limited to a special application domain, data representation model or data processing facility. Not surprisingly, the associated implementations are also restricted to some application domain and depend on a special data model. In this paper we give a survey of data provenance models and prototypes, present a general categorization scheme for provenance models and use this categorization scheme to study the properties of the existing approaches. This categorization enables us to distinguish between different kinds of provenance information and could lead to a better understanding of provenance in general. Besides the categorization of provenance types, it is important to include the storage, transformation and query requirements for the different kinds of provenance information and application domains in our considerations. The analysis of existing approaches will assist us in revealing open research problems in the area of data provenance.

Figures (10)

The category provenance model describes the expressive power of the conceptual model used by a provenance management system to define the provenance of a data item. We In this section we discuss data provenance from a conceptual point of view, extend the terminology introduced in section | and define a general categorization scheme for prove- nance management systems. We define several functionalities a provenance management system can provide and order these functionalities in a hierarchy of categories. The three main categories of our categorization scheme are provenance model, query and manipula- tion functionality and storage model and recording strategy. We present an overview figure for each main category (figures 1, 2 and 3). Categories and functionalities are represented by boxes and ellipses in these figures.

Figure 2: Query and manipulation funcionalities .2 Query and manipulation funcionalities

The category storage and recording includes the approaches a provenance management system uses to store provenance information, to record provenance information and to propagate provenance information recorded for source data items.

n [CWWO0], Widom et al. study the provenance of materialized views in a data warehouse. They present algorithms to create inverses of a view definition for tracing the positive contributing source of tuples in the view, based on the view definition. Transformations are represented by the relational algebra view definition and are handled at different levels of detail (by defining provenance for the basic relational operators and for the concatenation of these operators thus allowing views based on views). The provenance data is computed lazily and is not stored for further use. In the original paper the closed world model of a data warehouse is used and extended in [CW01] to allow certain openness and new transformations not representable in relational algebra by including general data warehouse transformations. 4.1 Provenance of views in a data warehouse

Trio [BSHW06, SBHW06, ABS* 06] is a database system for handling uncertain data and prove- nance, called ULDB. A ULDB presents a set of possible database instances by attaching a proba- bility to every possible alternative for a tuple in the database. The positive contributing source of tuple alternatives, which present all possibilities of a tuple, is recorded at query execution time. Trio uses a loose coupling storage strategy where source provenance is stored in an auxiliary relation. Every tuple alternative has a unique identifier consisting of a tuple identifier and an identifier for the alternative. If a tuple is not derived from other tuples alternatives in the database, but inserted from an external source, this tuple is called a base tuple and special identifiers are used to refer to the external source in the provenance of the tuple. The authors also present TriQL, a query language based on a subset of SQL (currently supporting union, join, selection and projection) extended by predicates for querying provenance and uncertainty. The restriction to this subset is based on the observation that the provenance produced by this operations is “well behaved”. A ULDB with well- behaved provenance has the advantage, that all possible instances of the database are determined by the possible instances of the base tuples in the database.

4.3. Chimera

Groth et al. have developed a protocol (PReP: P-assertion recording protocol) for recoding the provenance of interactions between services in a service-oriented architecture (SOA). This protocol requires the communicating services to send so called p-assertions, which are recorded by a prove- nance store web service. p-assertions present either inputs and outputs send by the services using a input source definition or metadata provided by the services about their internal state. Depending on the internal state data provided by the services it could by possible to identify the original or positive contributing source. The p-assertions transmitted by the services are stored by a provenance store service. To be able to correlate the multiple p-assertions sent by the communicating services, an identifier is attached to each p-assertion.

In[BCC06, BCCV06, BC05], Buneman et al. developed a provenance model for curated databases. They assume that a data item can be located by a path description in the hierarchy of the data repository it is stored in and represent data items from various data model as locations in a tree- structure. In this framework transformations are copy, paste and insertion operations on these data trees. The tree-structure enables their model to deal with source data items at different levels of detail. The authors defined four concepts of provenance which are in fact combinations of two levels of detail for source and transformation provenance respectively. In two of these concepts the user can control the transformation level of detail by manually defining the start and end of a transformation. A first prototype implementation of the model records the provenance for one database controlled by the system. The User can insert, delete and copy data in the database according to the defined transformation classes. 4.5 Copy-paste-model

4.6 DB-Notes

The DB-Notes prototype is an annotation management system for relational databases that allows a user to attach an arbitrary number of annotations to data items at the attribute value level of de- tail. Annotated databases are queried using pSQL, an extension of SQL which includes constructs for manipulating and querying annotations. The system itself generates provenance annotations. These annotations represent hierarchical positions (see section 2) of data items. The propagation of annotations is controlled by the user through the PROPAGATE clause of pSQL. When the default propagation scheme is chosen, the system propagates provenance annotations form the source data items to a result data item according to the original source definition. The authors show that the default propagation scheme may create different annotations for equivalent queries and introduce a default-all propagation scheme, which is invariant with respect to query rewriting. The current prototype implementation uses a tight-coupling storage model, where relations are extended with additional attributes for annotation storage.