Emerging database applications require the use of new indexing structures beyond B-trees and R-tr... more Emerging database applications require the use of new indexing structures beyond B-trees and R-trees. Examples are the k-D tree, the trie, the quadtree, and their variants. They are often proposed as supporting structures in data mining, GIS, and CAD/CAM applications. A common feature of all these indexes is that they recursively divide the space into partitions. A new extensible index structure, termed SP-GiST is presented that supports this class of data structures, mainly the class of space partitioning unbalanced trees. Simple method implementations are provided that demonstrate how SP-GiST can behave as a k-D tree, a trie, a quadtree, or any of their variants. Issues related to clustering tree nodes into pages as well as concurrency control for SP-GiST are addressed. A dynamic minimum-height clustering technique is applied to minimize disk accesses and to make using such trees in database systems possible and efficient. A prototype implementation of SP-GiST is presented as well as performance studies of the various SP-GiST's tuning parameters.
This paper introduces a class of join algorithms, termed W-join, for joining multiple infinite da... more This paper introduces a class of join algorithms, termed W-join, for joining multiple infinite data streams. W-join addresses the infinite nature of the data streams by joining stream data items that lie within a sliding window and that match a certain join condition. In addition to its general applicability in stream query processing, W-join can be used to track the motion of a moving object or detect the propagation of clouds of hazardous material or pollution spills over time in a sensor network environment. We describe two new algorithms for W-join and address variations and local/global optimizations related to specifying the nature of the window constraints to fulfill the posed queries. The performance of the proposed algorithms is studied experimentally in a prototype stream database system, using synthetic data streams and real time-series data. Tradeoffs of the proposed algorithms and their advantages and disadvantages are highlighted, given variations in the aggregate arrival rates of the input data streams and the desired response times per query.
The ubiquity of mobile devices, location services, and sensor pervasiveness, e.g., as in smart ci... more The ubiquity of mobile devices, location services, and sensor pervasiveness, e.g., as in smart city initiatives, call for scalable computing platforms and massively parallel architectures to process the vast amounts of the generated streamed data. Cloud computing provides some of the features needed for these massive data streaming applications. For example, the dynamic allocation of resources on an as-needed basis addresses the variability in sensor and location data distributions over time. However, today's cloud computing platforms lack very important features that are necessary in order to support the massive amounts of data streams envisioned by the massive and ubiquitous dissemination of sensors and mobile devices of all sorts in smart-city-scale applications. To support massive data stream processing, cloud platforms need to be extended in the following directions:
Databases are integral to many application domains in which the cycle of processing the data is c... more Databases are integral to many application domains in which the cycle of processing the data is complex and may involve real-world activities that are external to the database, e.g., wet-lab experiments, manual measurements, and collecting instrument readings. As a result, an update operation in the database may render dependent data items invalid or inconsistent until the real-world activities involved in deriving these items are re-performed and the output results are reflected back into the database. These real-world activities may take time to prepare for and to perform, and hence introduce inherently long time delays between the updates. The presence of these long delays between the updates along with the need for the intermediate results to be instantly available for querying makes integrating these real-world activities within the database systems a challenging task. In this paper, we address these challenges and propose techniques that reflect updates immediately into the database while keeping track of the dependent and potentially invalid data items until they are re-validated and reflecting their status in query results. The proposed system includes: (1) semantics and syntax for interfaces through which users can register real-world activities into the database system and express the dependencies of data items on these activities. (2) introducing new operator semantics that alert users when the returned query results contain potentially invalid or out-of-date data, and enable evaluating queries on either valid data only (no false positives), or both valid and potentially invalid data (including false positives), and (3) mechanisms for data invalidation and revalidation. The paper addresses several design issues and proposes directions for future work.
Biologists are increasingly using databases for storing and managing their data. Biological datab... more Biologists are increasingly using databases for storing and managing their data. Biological databases typically consist of a mixture of raw data, metadata, sequences, annotations, and related data obtained from various sources. Current database technology lacks several functionalities that are needed by biological databases. In this paper, we introduce bdbms, an extensible prototype database management system for supporting biological data. bdbms extends the functionalities of current DBMSs with: (1) Annotation and provenance management including storage, indexing, manipulation, and querying of annotation and provenance as first class objects in bdbms, (2) Local dependency tracking to track the dependencies and derivations among data items, (3) Update authorization to support data curation via content-based authorization, in contrast to identity-based authorization, and (4) New access methods and their supporting operators that support pattern matching on various types of compressed biological data types. This paper presents the design of bdbms along with the techniques proposed to support these functionalities including an extension to SQL. We also outline some open issues in building bdbms.
Context is any information that can be used to characterize the situation of an entity. Examples ... more Context is any information that can be used to characterize the situation of an entity. Examples of contexts include, but are not limited to, time, location, identity, and activity of a user. This paper proposes a general context-aware DBMS, named Chameleon, that will eliminate the need for having specialized database engines, e.g., spatial DBMS, temporal DBMS, and Hippocratic DBMS, since space, time, and identity can be treated as contexts in the general context-aware DBMS. Moreover, in Chameleon, we will be able to combine multiple contexts into more complex ones using the proposed context composition, e.g., a Hippocratic DBMS that also provides spatiotemporal and location contextual services. As a proof of concept, we construct two case studies using the same context-aware DBMS platform within Chameleon. One case study treats identity as a context to realize a privacy-aware (Hippocratic) database server while the other case study treats space as context to realize a spatial database server using the same proposed constructs and interfaces of Chameleon. To enable context-awareness and to be able to combine contexts, we need to realize various context-aware operators. We introduce a skyline operator as a necessary and important operator for efficient combination of contexts within a context-aware query processing engine. We study and address the performance bottlenecks that result from the interaction between the skyline operator and other query operators, e.g., joins. Although there is still a long way to go, the two case studies and the proposed skyline operators demonstrate that context-aware DBMSs are a viable and scalable approach.
The design of an integrated system is described for combining spatial and nonspatial attribute da... more The design of an integrated system is described for combining spatial and nonspatial attribute data to allow fast query handling. By fast it is meant that search queries can be executed in logarithmic time. Nonspatial attribute data is managed by standard database techniques. Efficient and robust spatial data structures are used to store spatial information. In addition to the standard access methods usually provided for accessing attribute data (i.e. through the data base management system), data can also be accessed through the spatial data structures in a homogeneous way that is transparent to the user. The design makes use of an SQL-like interface. The SQL query language is extended to provide efficient access methods to spatial structures, and to match functional and efficiency requirements desired from a combination of spatial and nonspatial information. The simplicity of SQL helps to clarify the design approach. However, other interfaces can be used with minor modifications.
This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing re... more This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing research and development across US agencies, industries, and universities. Spatial Computing is a set of ideas and technologies that will transform our lives by understanding the physical world, knowing and communicating our relation to places in that world, and navigating through those places.
Periodicity search in time series is a problem that has been investigated by mathematicians in va... more Periodicity search in time series is a problem that has been investigated by mathematicians in various areas, such as statistics, economics, and digital signal processing. For large databases of time series data, scalability becomes an issue that traditional techniques fail to address. In existing time series mining algorithms for detecting periodic patterns, the period length is userspecified. This is a drawback especially for datasets where no period length is known in advance. We propose an algorithm that extracts a set of candidate periods featured in a time series that satisfy a minimum confidence threshold, by utilizing the autocorrelation function and FFT as a filter. We provide some mathematical background as well as experimental results. 12
This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing re... more This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing research and development across US agencies, industries, and universities. Spatial Computing is a set of ideas and technologies that will transform our lives by understanding the physical world, knowing and communicating our relation to places in that world, and navigating through those places.
The tremendous increase in the use of cellular phones, GPS-like devices, and RFIDs results in hig... more The tremendous increase in the use of cellular phones, GPS-like devices, and RFIDs results in highly dynamic environments where objects as well as queries are continuously moving. In this paper, we present a continuous query processor designed specifically for highly dynamic environments (e.g., location-aware environments). We implemented the proposed continuous query processor inside the PLACE server (Pervasive Location-Aware Computing Environments); a scalable location-aware database server developed at Purdue University. The PLACE server extends data streaming management systems to support location-aware environments. These environments are characterized by the wide variety of continuous spatio-temporal queries and the unbounded spatio-temporal streams. The proposed continuous query processor includes: (1) New incremental spatio-temporal operators to support a wide variety of continuous spatio-temporal queries, (2) Extended semantics of sliding window queries to deal with spatial sliding windows as well as temporal sliding windows, and (3) A shared-execution framework for scalable execution of a set of concurrent continuous spatiotemporal queries. Experimental evaluation shows promising performance of the continuous query processor of the PLACE server.
The mining of periodic patterns in time series databases is an interesting data mining problem th... more The mining of periodic patterns in time series databases is an interesting data mining problem that can be envisioned as a tool for forecasting and predicting the future behavior of time series data. Discovering the rate at which the time series is periodic has always been an obstacle for fully automated periodicity mining. In existing periodic patterns mining algorithms, the period length is user-specified. This is a considerable disadvantage, especially in time series datasets where the period length is not known a priori. In this paper, we address the problem of detecting the periodicity rate of a time series database. Two types of periodicities arc defined, and a scalable and computationally efficient algorithm is proposed for each type. The algorithms require only one scan over a time series database of size n, with O(nlogn) time complexity.
The problern of frequerztly clpdating ~nulti-dimensiorzal uzdexes orises i~z man?. location-depen... more The problern of frequerztly clpdating ~nulti-dimensiorzal uzdexes orises i~z man?. location-dependent applications. While the R-tree and its varianrs are one of the donzinant choices for indexing i7izrlr1-dir17erz~io1zal objects, the R-tree exhibits i1zferiorpe1for17ioizce iiz the presence offrequent updotes. 111 this paper; we preseil? all R-tl-ee variant, ternzed the RUM-tree (stand5 for R-nee with Updote Me~iio) that 177i1zi1nizes the cost of ol7jecr crpdates. The RUM-tree processes updates in a memo-based approoch that avoids disk accesses for purging old entries dcrriizg an update process. Therefore, the cost of an ~lpdate operation in the RUM-tree reduces to the cost of 01zl11 an insert operation. The renzoval of old object entries is carrred out b j a garbage cleaner iizside the RUM-tree. In this popei; rve present the details of the RUM-tree and study its pi-operties. Theoretical a~zalyszs and experiinental evaluatio~z denlo~zstrnte that the RUMtree outpe~$ornzs other R-tree variants by up to a factor of eight in sce~zarios with frequent c~pdates.
Crowdsourcing services allow employing human intelligence in tasks that are difficult to accompli... more Crowdsourcing services allow employing human intelligence in tasks that are difficult to accomplish with computers such as image tagging and data collection. At a relatively low monetary cost and through web interfaces such as Amazon's Mechanical Turk (AMT), humans can act as a computational operator in large systems. Recent work has been conducted to build database management systems that can harness the crowd power in database operators, such as sort, join, count, etc. The fundamental problem of indexing within crowdsourced databases has not been studied. In this paper, we study the problem of tree-based indexing within crowd-enabled databases. We investigate the effect of various tree and crowdsourcing parameters on the quality of index operations. We propose new algorithms for index search, insert, and update.
Biologists are increasingly using databases for storing and managing their data. Biological datab... more Biologists are increasingly using databases for storing and managing their data. Biological databases typically consist of a mixture of raw data, metadata, sequences, annotations, and related data obtained from various sources. Current database technology lacks several functionalities that are needed by biological databases. In this paper, we introduce bdbms, an extensible prototype database management system for supporting biological data. bdbms extends the functionalities of current DBMSs to include: (1) Annotation and provenance management including storage, indexing, manipulation, and querying of annotation and provenance as first class objects in bdbms, (2) Local dependency tracking to track the dependencies and derivations among data items, (3) Update authorization to support data curation via content-based authorization, in contrast to identity-based authorization, and (4) New access methods and their supporting operators that support pattern matching on various types of compressed biological data types. This paper presents the design of bdbms along with the techniques proposed to support these functionalities including an extension to SQL. We also outline some open issues in building bdbms.
Emerging database applications require the use of new indexing structures beyond B-trees and R-tr... more Emerging database applications require the use of new indexing structures beyond B-trees and R-trees. Examples are the k-D tree, the trie, the quadtree, and their variants. They are often proposed as supporting structures in data mining, GIS, and CAD/CAM applications. A common feature of all these indexes is that they recursively divide the space into partitions. A new extensible index structure, termed SP-GiST is presented that supports this class of data structures, mainly the class of space partitioning unbalanced trees. Simple method implementations are provided that demonstrate how SP-GiST can behave as a k-D tree, a trie, a quadtree, or any of their variants. Issues related to clustering tree nodes into pages as well as concurrency control for SP-GiST are addressed. A dynamic minimum-height clustering technique is applied to minimize disk accesses and to make using such trees in database systems possible and efficient. A prototype implementation of SP-GiST is presented as well as performance studies of the various SP-GiST's tuning parameters.
This paper introduces a class of join algorithms, termed W-join, for joining multiple infinite da... more This paper introduces a class of join algorithms, termed W-join, for joining multiple infinite data streams. W-join addresses the infinite nature of the data streams by joining stream data items that lie within a sliding window and that match a certain join condition. In addition to its general applicability in stream query processing, W-join can be used to track the motion of a moving object or detect the propagation of clouds of hazardous material or pollution spills over time in a sensor network environment. We describe two new algorithms for W-join and address variations and local/global optimizations related to specifying the nature of the window constraints to fulfill the posed queries. The performance of the proposed algorithms is studied experimentally in a prototype stream database system, using synthetic data streams and real time-series data. Tradeoffs of the proposed algorithms and their advantages and disadvantages are highlighted, given variations in the aggregate arrival rates of the input data streams and the desired response times per query.
The ubiquity of mobile devices, location services, and sensor pervasiveness, e.g., as in smart ci... more The ubiquity of mobile devices, location services, and sensor pervasiveness, e.g., as in smart city initiatives, call for scalable computing platforms and massively parallel architectures to process the vast amounts of the generated streamed data. Cloud computing provides some of the features needed for these massive data streaming applications. For example, the dynamic allocation of resources on an as-needed basis addresses the variability in sensor and location data distributions over time. However, today's cloud computing platforms lack very important features that are necessary in order to support the massive amounts of data streams envisioned by the massive and ubiquitous dissemination of sensors and mobile devices of all sorts in smart-city-scale applications. To support massive data stream processing, cloud platforms need to be extended in the following directions:
Databases are integral to many application domains in which the cycle of processing the data is c... more Databases are integral to many application domains in which the cycle of processing the data is complex and may involve real-world activities that are external to the database, e.g., wet-lab experiments, manual measurements, and collecting instrument readings. As a result, an update operation in the database may render dependent data items invalid or inconsistent until the real-world activities involved in deriving these items are re-performed and the output results are reflected back into the database. These real-world activities may take time to prepare for and to perform, and hence introduce inherently long time delays between the updates. The presence of these long delays between the updates along with the need for the intermediate results to be instantly available for querying makes integrating these real-world activities within the database systems a challenging task. In this paper, we address these challenges and propose techniques that reflect updates immediately into the database while keeping track of the dependent and potentially invalid data items until they are re-validated and reflecting their status in query results. The proposed system includes: (1) semantics and syntax for interfaces through which users can register real-world activities into the database system and express the dependencies of data items on these activities. (2) introducing new operator semantics that alert users when the returned query results contain potentially invalid or out-of-date data, and enable evaluating queries on either valid data only (no false positives), or both valid and potentially invalid data (including false positives), and (3) mechanisms for data invalidation and revalidation. The paper addresses several design issues and proposes directions for future work.
Biologists are increasingly using databases for storing and managing their data. Biological datab... more Biologists are increasingly using databases for storing and managing their data. Biological databases typically consist of a mixture of raw data, metadata, sequences, annotations, and related data obtained from various sources. Current database technology lacks several functionalities that are needed by biological databases. In this paper, we introduce bdbms, an extensible prototype database management system for supporting biological data. bdbms extends the functionalities of current DBMSs with: (1) Annotation and provenance management including storage, indexing, manipulation, and querying of annotation and provenance as first class objects in bdbms, (2) Local dependency tracking to track the dependencies and derivations among data items, (3) Update authorization to support data curation via content-based authorization, in contrast to identity-based authorization, and (4) New access methods and their supporting operators that support pattern matching on various types of compressed biological data types. This paper presents the design of bdbms along with the techniques proposed to support these functionalities including an extension to SQL. We also outline some open issues in building bdbms.
Context is any information that can be used to characterize the situation of an entity. Examples ... more Context is any information that can be used to characterize the situation of an entity. Examples of contexts include, but are not limited to, time, location, identity, and activity of a user. This paper proposes a general context-aware DBMS, named Chameleon, that will eliminate the need for having specialized database engines, e.g., spatial DBMS, temporal DBMS, and Hippocratic DBMS, since space, time, and identity can be treated as contexts in the general context-aware DBMS. Moreover, in Chameleon, we will be able to combine multiple contexts into more complex ones using the proposed context composition, e.g., a Hippocratic DBMS that also provides spatiotemporal and location contextual services. As a proof of concept, we construct two case studies using the same context-aware DBMS platform within Chameleon. One case study treats identity as a context to realize a privacy-aware (Hippocratic) database server while the other case study treats space as context to realize a spatial database server using the same proposed constructs and interfaces of Chameleon. To enable context-awareness and to be able to combine contexts, we need to realize various context-aware operators. We introduce a skyline operator as a necessary and important operator for efficient combination of contexts within a context-aware query processing engine. We study and address the performance bottlenecks that result from the interaction between the skyline operator and other query operators, e.g., joins. Although there is still a long way to go, the two case studies and the proposed skyline operators demonstrate that context-aware DBMSs are a viable and scalable approach.
The design of an integrated system is described for combining spatial and nonspatial attribute da... more The design of an integrated system is described for combining spatial and nonspatial attribute data to allow fast query handling. By fast it is meant that search queries can be executed in logarithmic time. Nonspatial attribute data is managed by standard database techniques. Efficient and robust spatial data structures are used to store spatial information. In addition to the standard access methods usually provided for accessing attribute data (i.e. through the data base management system), data can also be accessed through the spatial data structures in a homogeneous way that is transparent to the user. The design makes use of an SQL-like interface. The SQL query language is extended to provide efficient access methods to spatial structures, and to match functional and efficiency requirements desired from a combination of spatial and nonspatial information. The simplicity of SQL helps to clarify the design approach. However, other interfaces can be used with minor modifications.
This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing re... more This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing research and development across US agencies, industries, and universities. Spatial Computing is a set of ideas and technologies that will transform our lives by understanding the physical world, knowing and communicating our relation to places in that world, and navigating through those places.
Periodicity search in time series is a problem that has been investigated by mathematicians in va... more Periodicity search in time series is a problem that has been investigated by mathematicians in various areas, such as statistics, economics, and digital signal processing. For large databases of time series data, scalability becomes an issue that traditional techniques fail to address. In existing time series mining algorithms for detecting periodic patterns, the period length is userspecified. This is a drawback especially for datasets where no period length is known in advance. We propose an algorithm that extracts a set of candidate periods featured in a time series that satisfy a minimum confidence threshold, by utilizing the autocorrelation function and FFT as a filter. We provide some mathematical background as well as experimental results. 12
This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing re... more This proposal outlines an effort to develop and promote a unified agenda for Spatial Computing research and development across US agencies, industries, and universities. Spatial Computing is a set of ideas and technologies that will transform our lives by understanding the physical world, knowing and communicating our relation to places in that world, and navigating through those places.
The tremendous increase in the use of cellular phones, GPS-like devices, and RFIDs results in hig... more The tremendous increase in the use of cellular phones, GPS-like devices, and RFIDs results in highly dynamic environments where objects as well as queries are continuously moving. In this paper, we present a continuous query processor designed specifically for highly dynamic environments (e.g., location-aware environments). We implemented the proposed continuous query processor inside the PLACE server (Pervasive Location-Aware Computing Environments); a scalable location-aware database server developed at Purdue University. The PLACE server extends data streaming management systems to support location-aware environments. These environments are characterized by the wide variety of continuous spatio-temporal queries and the unbounded spatio-temporal streams. The proposed continuous query processor includes: (1) New incremental spatio-temporal operators to support a wide variety of continuous spatio-temporal queries, (2) Extended semantics of sliding window queries to deal with spatial sliding windows as well as temporal sliding windows, and (3) A shared-execution framework for scalable execution of a set of concurrent continuous spatiotemporal queries. Experimental evaluation shows promising performance of the continuous query processor of the PLACE server.
The mining of periodic patterns in time series databases is an interesting data mining problem th... more The mining of periodic patterns in time series databases is an interesting data mining problem that can be envisioned as a tool for forecasting and predicting the future behavior of time series data. Discovering the rate at which the time series is periodic has always been an obstacle for fully automated periodicity mining. In existing periodic patterns mining algorithms, the period length is user-specified. This is a considerable disadvantage, especially in time series datasets where the period length is not known a priori. In this paper, we address the problem of detecting the periodicity rate of a time series database. Two types of periodicities arc defined, and a scalable and computationally efficient algorithm is proposed for each type. The algorithms require only one scan over a time series database of size n, with O(nlogn) time complexity.
The problern of frequerztly clpdating ~nulti-dimensiorzal uzdexes orises i~z man?. location-depen... more The problern of frequerztly clpdating ~nulti-dimensiorzal uzdexes orises i~z man?. location-dependent applications. While the R-tree and its varianrs are one of the donzinant choices for indexing i7izrlr1-dir17erz~io1zal objects, the R-tree exhibits i1zferiorpe1for17ioizce iiz the presence offrequent updotes. 111 this paper; we preseil? all R-tl-ee variant, ternzed the RUM-tree (stand5 for R-nee with Updote Me~iio) that 177i1zi1nizes the cost of ol7jecr crpdates. The RUM-tree processes updates in a memo-based approoch that avoids disk accesses for purging old entries dcrriizg an update process. Therefore, the cost of an ~lpdate operation in the RUM-tree reduces to the cost of 01zl11 an insert operation. The renzoval of old object entries is carrred out b j a garbage cleaner iizside the RUM-tree. In this popei; rve present the details of the RUM-tree and study its pi-operties. Theoretical a~zalyszs and experiinental evaluatio~z denlo~zstrnte that the RUMtree outpe~$ornzs other R-tree variants by up to a factor of eight in sce~zarios with frequent c~pdates.
Crowdsourcing services allow employing human intelligence in tasks that are difficult to accompli... more Crowdsourcing services allow employing human intelligence in tasks that are difficult to accomplish with computers such as image tagging and data collection. At a relatively low monetary cost and through web interfaces such as Amazon's Mechanical Turk (AMT), humans can act as a computational operator in large systems. Recent work has been conducted to build database management systems that can harness the crowd power in database operators, such as sort, join, count, etc. The fundamental problem of indexing within crowdsourced databases has not been studied. In this paper, we study the problem of tree-based indexing within crowd-enabled databases. We investigate the effect of various tree and crowdsourcing parameters on the quality of index operations. We propose new algorithms for index search, insert, and update.
Biologists are increasingly using databases for storing and managing their data. Biological datab... more Biologists are increasingly using databases for storing and managing their data. Biological databases typically consist of a mixture of raw data, metadata, sequences, annotations, and related data obtained from various sources. Current database technology lacks several functionalities that are needed by biological databases. In this paper, we introduce bdbms, an extensible prototype database management system for supporting biological data. bdbms extends the functionalities of current DBMSs to include: (1) Annotation and provenance management including storage, indexing, manipulation, and querying of annotation and provenance as first class objects in bdbms, (2) Local dependency tracking to track the dependencies and derivations among data items, (3) Update authorization to support data curation via content-based authorization, in contrast to identity-based authorization, and (4) New access methods and their supporting operators that support pattern matching on various types of compressed biological data types. This paper presents the design of bdbms along with the techniques proposed to support these functionalities including an extension to SQL. We also outline some open issues in building bdbms.
Uploads
Papers by Walid Aref