Botnet, a group of coordinated bots, is becoming the main platform of malicious Internet activiti... more Botnet, a group of coordinated bots, is becoming the main platform of malicious Internet activities like DDOS, click fraud, web scraping, spam/rumor distribution, etc. This paper focuses on design and experiment of a new approach for botnet detection from streaming web server logs, motivated by its wide applicability, real-time protection capability, ease of use and better security of sensitive data. Our algorithm is inspired by a Principal Component Analysis (PCA) to capture correlation in data, and we are first to recognize and adapt Lanczos method to improve the time complexity of PCA-based botnet detection from cubic to sub-cubic, which enables us to more accurately and sensitively detect botnets with sliding time windows rather than fixed time windows. We contribute a generalized online correlation matrix update formula, and a new termination condition for Lanczos iteration for our purpose based on error bound and non-decreasing eigenvalues of symmetric matrices. On our dataset of an ecommerce website logs, experiments show the time cost of Lanczos method with different time windows are consistently only 20% to 25% of PCA.
Cloud Computing for Data-Intensive Applications, 2014
Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and re... more Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and resources on demand, and is gaining significant adoption in the science community. At the same time, scientific workflow management systems provide essential support and functionality to scientific computing, such as management of data and task dependencies, job scheduling and execution, provenance tracking, fault tolerance. Migrating scientific workflow management systems from traditional Grid computing environments into the Cloud would enable a much broader user base to conduct their scientific research with ever increasing data scale and analysis complexity. This paper presents our experience in integrating the Swift scientific workflow management system with the OpenNebula Cloud platform, which supports workflow specification and submission, on-demand virtual cluster provisioning, high-throughput task scheduling and execution, and efficient and scalable resource management in the Cloud. We set up a series of experiments to demonstrate the capability of our integration and use a MODIS image processing workflow as a showcase of the implementation.
Sixth IEEE International Conference on Data Mining - Workshops (ICDMW'06), 2006
The reciprocal connectivity between the cerebral cortex and the thalamus in a human brain is invo... more The reciprocal connectivity between the cerebral cortex and the thalamus in a human brain is involved in consciousness and related to various brain disorders, thus, in-vivo analysis of this connectivity is critically important for brain diagnosis and surgery planning. While existing work either focuses on fiber tracking analysis or on thalamic nuclei segmentation, to our best knowledge, no techniques yet exist for performing in-vivo analysis of thalamo-corticothalamic connectivity. In this paper, (i) we propose a new partitioning paradigm, called coclustering, to model this problem. In contrast to the traditional clustering paradigm, a coclustering procedure not only simultaneously partitions cortical voxels and thalamic voxels into groups, but also identifies the corresponding strong connectivities between the two classes of groups; (ii) we develop the first coclustering algorithm, Genetic Coclustering Algorithm (GCA), to solve the coclustering problem; and (iii) we apply GCA to perform in-vivo analysis of the thalamo-cortico-thalamic connectivity and produce a strikingly clear 3-D visualization of the seven thalamic nuclei groups as well as their connectivities to the corresponding cortical regions of a human brain.
Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007), 2007
Provenance management has become increasingly important to support scientific discovery reproduci... more Provenance management has become increasingly important to support scientific discovery reproducibility, result interpretation, and problem diagnosis in scientific workflow environments. This paper proposes an approach to provenance management that seamlessly integrates the interoperability, extensibility, and reasoning advantages of Semantic Web technologies with the storage and querying power of an RDBMS. Specifically, we propose: i) two schema mapping algorithms to map an arbitrary OWL provenance ontology to a relational database schema that is optimized for common provenance queries; ii) two efficient data mapping algorithms to map provenance RDF metadata to relational data according to the generated relational database schema, and iii) a schema-independent SPARQL-to-SQL translation algorithm that is optimized on-the-fly by using the type information of an instance available from the input provenance ontology and the statistics of the sizes of the tables in the database. Experimental results are presented to show that our algorithms are efficient and scalable.
International Journal of Data Mining and Bioinformatics, 2008
As the central relay station of the human brain, the thalamus modulates sensory signals to and fr... more As the central relay station of the human brain, the thalamus modulates sensory signals to and from the cerebral cortex. The reciprocal connectivity between the cerebral cortex and the thalamus is believed to play an essential role in consciousness and various neurological disorders. Thus, in-vivo analysis of thalamo-cortical connectivity is important for our understanding of normal and pathological brain processes. In this paper • We propose a new partitioning paradigm, called coclustering, in order to segment the thalamus into thalamic nuclei based on their cortical projections. In contrast to the traditional clustering paradigm, a coclustering procedure not only simultaneously partitions cortical voxels and thalamic voxels into groups, but also identifies the corresponding strong connectivities between the two classes of groups • We develop the first coclustering algorithm, Genetic Coclustering Algorithm (GCA), to solve the coclustering problem • We apply GCA to segment the thalamus into thalamic nuclei and visualise main thalamo-cortical fibre tracts.
2011 IEEE International Conference on Service-Oriented Computing and Applications (SOCA), 2011
Due to the complexity of scientific processes, computing and storage resources in a scientific wo... more Due to the complexity of scientific processes, computing and storage resources in a scientific workflow are often needed on an uneven basis, thus, the demand of resources is elastically changing during a run of a workflow. Most existing workflow scheduling algorithms only consider a computing environment in which the number of compute resources is bounded. Resources assigned to a workflow cannot be automatically determined on demand of the size of the workflow and are not released to the environment until an execution of the workflow completes. The salient features of service-oriented computing have brought a new opportunity to schedule workflows on resources with elastically changing demand, as they allow resources to scale on demand as usage changes through dynamic provisioning. To address this issue, we firstly formalize a model of a service-oriented computing environment and a workflow graph representation for the environment. Then, we propose SCPOR, a scientific workflow scheduling algorithm that is able to schedule workflows in need of elastically changing compute resources. Our extensive experiments and comparisons for compute-intensive and dataintensive workflows have shown that SCPOR not only outperforms several representative workflow scheduling algorithms in optimizing workflow execution time, but also enables resources to scale elastically during workflow execution.
2009 IEEE International Conference on Web Services, 2009
MapReduce has recently gained a lot of attention as a parallel programming model for scalable dat... more MapReduce has recently gained a lot of attention as a parallel programming model for scalable data-intensive business and scientific analysis. In order to benefit from this powerful programming model in a scientific workflow environment, we propose a MapReduce-enabled scientific workflow composition framework consisting of: i) a dataflowbased scientific workflow model that separates the declaration of the workflow interface from the definition of its functional body; ii) a set of dataflow constructs, including Map, Reduce, Loop, and Conditional, and their composition semantics to enable MapReduce-style scientific workflows; iii) an XML-based scientific workflow specification language, called WSL, in which both Map and Reduce are fully composable with other dataflow constructs in both flat and hierarchical manners. Besides leveraging the power of MapReduce to the workflow level, our workflow composition framework is unique in that workflows are the only operands for composition; in this way, our approach elegantly solves the two-world problem of existing composition frameworks, in which composition needs to deal with both the world of tasks and the world of workflows. The proposed framework is implemented and a case study is conducted to validate our techniques.
2011 IEEE 4th International Conference on Cloud Computing, 2011
Most existing workflow scheduling algorithms only consider a computing environment in which the n... more Most existing workflow scheduling algorithms only consider a computing environment in which the number of compute resources is bounded. Compute resources in such an environment usually cannot be provisioned or released on demand of the size of a workflow, and these resources are not released to the environment until an execution of the workflow completes. To address the problem, we firstly formalize a model of a Cloud environment and a workflow graph representation for such an environment. Then, we propose the SHEFT workflow scheduling algorithm to schedule a workflow elastically on a Cloud computing environment. Our preliminary experiments show that SHEFT not only outperforms several representative workflow scheduling algorithms in optimizing workflow execution time, but also enables resources to scale elastically at runtime.
Scientific workflow management systems (SWFMSs) are facing unprecedented challenges from big data... more Scientific workflow management systems (SWFMSs) are facing unprecedented challenges from big data deluge. As revising all the existing workflow applications to fit into Cloud computing paradigm is impractical, thus migrating SWFMSs into the Cloud to leverage the functionalities of both Cloud computing and SWFMSs may provide a viable approach to big data processing. In this paper, we first discuss the challenges for scientific workflow applications and the available solutions in details, and analyze the essential requirements for a scientific computing Cloud platform. Then we propose a service framework to normalize the integration of SWFMS with Cloud computing. Meanwhile, we also present our implementation experience based on the service Framework. At last, we set up a series of experiments to demonstrate the capability of our implementation and use a Montage Image Mosaic Workflow as a showcase of the implementation.
2007 IEEE Congress on Services (Services 2007), 2007
In this demo, we present current status of our VIsual sci-Entific Workflow management system call... more In this demo, we present current status of our VIsual sci-Entific Workflow management system called VIEW, highlighting the following two features: (i) the use of Semantic Web technology to represent, store, and query provenance metadata, leading to an interoperable and extensible provenance system, and (ii) the support of visualization of various provenance graphs and intermediate or final data products of a workflow run in the form of medical images or 3-D graphical models. VIEW seamlessly integrates the interoperability, extensibility, and reasoning advantages of Semantic Web technology, the querying and storage power of a RDBMS, and the appealing visual features of visualization techniques.
Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and re... more Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and resources on demand, and is getting more and more adoption in the science community, while scientific workflow management systems provide essential support such as management of data and task dependencies, job scheduling and execution, provenance tracking, etc., to scientific computing. As we are entering into a "big data" era, it is imperative to migrate scientific workflow management systems into the Cloud to manage the ever increasing data scale and analysis complexity. We propose a reference service framework for integrating scientific workflow management systems into various Cloud platforms, which consists of eight major components, including Cloud workflow management service, Cloud resource manager, etc., and 6 interfaces between them. We also present a reference framework for the implementation of Cloud Resource Manager, which is responsible for the provisioning and management of virtual resources in the Cloud. We discuss our implementation of the framework by integrating the Swift scientific workflow management system with the OpenNebula and Eucalyptus Cloud platforms, and demonstrate the capability of the solution using a NASA MODIS image processing workflow and a production deployment on the Science@Guoshi network with support for the Montage image mosaic workflow.
Scientific workflows have recently emerged as a new paradigm for scientists to formalize and stru... more Scientific workflows have recently emerged as a new paradigm for scientists to formalize and structure complex and distributed scientific processes to enable and accelerate many scientific discoveries. In contrast to business workflows, which are typically control flow oriented, scientific workflows tend to be dataflow oriented, introducing a new set of requirements for system development. These requirements demand a new architectural design for scientific workflow management systems (SWFMSs). Although several SWFMSs have been developed that provide much experience for future research and development, a study from an architectural perspective is still missing. The main contributions of this paper are: 1) based on a comprehensive survey of the literature and identification of key requirements for SWFMSs, we propose the first reference architecture for SWFMSs; 2) according to the reference architecture, we further propose a service-oriented architecture for VIEW (a VIsual sciEntific Workflow management system); 3) we implemented VIEW to validate the feasibility of the proposed architectures; and 4) we present a VIEW-based scientific workflow application system (SWFAS), called FiberFlow, to showcase the application of our VIEW system.
The fundamental goal of computational neuroscience is to discover anatomical features that reflec... more The fundamental goal of computational neuroscience is to discover anatomical features that reflect the functional organization of the brain. Investigations of the physical connections between neuronal structures and measurements of brain activity in vivo have given rise to the concepts of anatomical and functional connectivity, which have been useful for our understanding of brain mechanisms and their plasticity. However, at present there is no generally accepted computational framework for the quantitative assessment of cortical ...
Recently, there has been an increasing need in scientific workflows to solve the shimming problem... more Recently, there has been an increasing need in scientific workflows to solve the shimming problem, the use of a special kind of adaptors, called shims, to link related but incompatible workflow tasks. However, existing techniques produce scientific workflows that are cluttered with many visible shims, which distract a scientist's focus on functional components. Moreover, these techniques do not address a new type of shimming problem that occurs due to the incompatibility between the ports of a task and the inputs/outputs of ...
With ever-improving information technologies and high performance computational power, recent tec... more With ever-improving information technologies and high performance computational power, recent techniques in granular computing, soft computing and cognitive science have allowed an increasing understanding of normal and abnormal brain functions, especially ...
Scientific workflows have recently emerged as a new paradigm for scientists to formalize and stru... more Scientific workflows have recently emerged as a new paradigm for scientists to formalize and structure complex and distributed scientific processes to enable and accelerate many scientific discoveries. In contrast to business workflows, which are typically control flow oriented, scientific workflows tend to be dataflow oriented, introducing a new set of requirements for system development. These requirements demand a new architectural design for scientific workflow management systems (SWFMSs). Although several ...
As the central relay station of the human brain, the thalamus modulates sensory signals to and fr... more As the central relay station of the human brain, the thalamus modulates sensory signals to and from the cerebral cortex. The reciprocal connectivity between the cerebral cortex and the thalamus is believed to play an essential role in consciousness and various neurological disorders. Thus, in-vivo analysis of thalamo-cortical connectivity is important for our understanding of normal and pathological brain processes. In this paper • We propose a new partitioning paradigm, called coclustering, in order to segment the thalamus into thalamic nuclei based on their cortical projections. In contrast to the traditional clustering paradigm, a coclustering procedure not only simultaneously partitions cortical voxels and thalamic voxels into groups, but also identifies the corresponding strong connectivities between the two classes of groups • We develop the first coclustering algorithm, Genetic Coclustering Algorithm (GCA), to solve the coclustering problem • We apply GCA to segment the thalamus into thalamic nuclei and visualise main thalamo-cortical fibre tracts.
One of the fundamental goals of computational neuroscience is the study of anatomical features th... more One of the fundamental goals of computational neuroscience is the study of anatomical features that reflect the functional organization of the brain. The study of physical associations between neuronal structures and the examination of brain activity in vivo have given rise to the concept of anatomical and functional connectivity, which has been invaluable for our understanding of brain mechanisms and their plasticity during development. However, at present, there is no robust and accurate computational framework for the quantitative assessment of cortical connectivity patterns. In this paper, we present a quantitative analysis and modeling tool that is able to characterize anatomical connectivity patterns based on a newly developed coclustering algorithm, termed the business model-based coclustering algorithm (BCA). We apply BCA to diffusion tensor imaging (DTI) data in order to provide an automated and reproducible assessment of the connectivity patterns between different cortical areas in human brains. BCA not only partitions the cortical mantel into well-defined clusters, but also maximizes the connectivity strength between these clusters. Moreover, BCA is computationally robust and allows both outlier detection as well as parameter-independent determination of the number of clusters. Our coclustering results have showed good performance of BCA in identifying major white matter fiber bundles in human brains and facilitate the detection of abnormal connectivity patterns in patients suffering from various neurological diseases.
Botnet, a group of coordinated bots, is becoming the main platform of malicious Internet activiti... more Botnet, a group of coordinated bots, is becoming the main platform of malicious Internet activities like DDOS, click fraud, web scraping, spam/rumor distribution, etc. This paper focuses on design and experiment of a new approach for botnet detection from streaming web server logs, motivated by its wide applicability, real-time protection capability, ease of use and better security of sensitive data. Our algorithm is inspired by a Principal Component Analysis (PCA) to capture correlation in data, and we are first to recognize and adapt Lanczos method to improve the time complexity of PCA-based botnet detection from cubic to sub-cubic, which enables us to more accurately and sensitively detect botnets with sliding time windows rather than fixed time windows. We contribute a generalized online correlation matrix update formula, and a new termination condition for Lanczos iteration for our purpose based on error bound and non-decreasing eigenvalues of symmetric matrices. On our dataset of an ecommerce website logs, experiments show the time cost of Lanczos method with different time windows are consistently only 20% to 25% of PCA.
Cloud Computing for Data-Intensive Applications, 2014
Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and re... more Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and resources on demand, and is gaining significant adoption in the science community. At the same time, scientific workflow management systems provide essential support and functionality to scientific computing, such as management of data and task dependencies, job scheduling and execution, provenance tracking, fault tolerance. Migrating scientific workflow management systems from traditional Grid computing environments into the Cloud would enable a much broader user base to conduct their scientific research with ever increasing data scale and analysis complexity. This paper presents our experience in integrating the Swift scientific workflow management system with the OpenNebula Cloud platform, which supports workflow specification and submission, on-demand virtual cluster provisioning, high-throughput task scheduling and execution, and efficient and scalable resource management in the Cloud. We set up a series of experiments to demonstrate the capability of our integration and use a MODIS image processing workflow as a showcase of the implementation.
Sixth IEEE International Conference on Data Mining - Workshops (ICDMW'06), 2006
The reciprocal connectivity between the cerebral cortex and the thalamus in a human brain is invo... more The reciprocal connectivity between the cerebral cortex and the thalamus in a human brain is involved in consciousness and related to various brain disorders, thus, in-vivo analysis of this connectivity is critically important for brain diagnosis and surgery planning. While existing work either focuses on fiber tracking analysis or on thalamic nuclei segmentation, to our best knowledge, no techniques yet exist for performing in-vivo analysis of thalamo-corticothalamic connectivity. In this paper, (i) we propose a new partitioning paradigm, called coclustering, to model this problem. In contrast to the traditional clustering paradigm, a coclustering procedure not only simultaneously partitions cortical voxels and thalamic voxels into groups, but also identifies the corresponding strong connectivities between the two classes of groups; (ii) we develop the first coclustering algorithm, Genetic Coclustering Algorithm (GCA), to solve the coclustering problem; and (iii) we apply GCA to perform in-vivo analysis of the thalamo-cortico-thalamic connectivity and produce a strikingly clear 3-D visualization of the seven thalamic nuclei groups as well as their connectivities to the corresponding cortical regions of a human brain.
Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007), 2007
Provenance management has become increasingly important to support scientific discovery reproduci... more Provenance management has become increasingly important to support scientific discovery reproducibility, result interpretation, and problem diagnosis in scientific workflow environments. This paper proposes an approach to provenance management that seamlessly integrates the interoperability, extensibility, and reasoning advantages of Semantic Web technologies with the storage and querying power of an RDBMS. Specifically, we propose: i) two schema mapping algorithms to map an arbitrary OWL provenance ontology to a relational database schema that is optimized for common provenance queries; ii) two efficient data mapping algorithms to map provenance RDF metadata to relational data according to the generated relational database schema, and iii) a schema-independent SPARQL-to-SQL translation algorithm that is optimized on-the-fly by using the type information of an instance available from the input provenance ontology and the statistics of the sizes of the tables in the database. Experimental results are presented to show that our algorithms are efficient and scalable.
International Journal of Data Mining and Bioinformatics, 2008
As the central relay station of the human brain, the thalamus modulates sensory signals to and fr... more As the central relay station of the human brain, the thalamus modulates sensory signals to and from the cerebral cortex. The reciprocal connectivity between the cerebral cortex and the thalamus is believed to play an essential role in consciousness and various neurological disorders. Thus, in-vivo analysis of thalamo-cortical connectivity is important for our understanding of normal and pathological brain processes. In this paper • We propose a new partitioning paradigm, called coclustering, in order to segment the thalamus into thalamic nuclei based on their cortical projections. In contrast to the traditional clustering paradigm, a coclustering procedure not only simultaneously partitions cortical voxels and thalamic voxels into groups, but also identifies the corresponding strong connectivities between the two classes of groups • We develop the first coclustering algorithm, Genetic Coclustering Algorithm (GCA), to solve the coclustering problem • We apply GCA to segment the thalamus into thalamic nuclei and visualise main thalamo-cortical fibre tracts.
2011 IEEE International Conference on Service-Oriented Computing and Applications (SOCA), 2011
Due to the complexity of scientific processes, computing and storage resources in a scientific wo... more Due to the complexity of scientific processes, computing and storage resources in a scientific workflow are often needed on an uneven basis, thus, the demand of resources is elastically changing during a run of a workflow. Most existing workflow scheduling algorithms only consider a computing environment in which the number of compute resources is bounded. Resources assigned to a workflow cannot be automatically determined on demand of the size of the workflow and are not released to the environment until an execution of the workflow completes. The salient features of service-oriented computing have brought a new opportunity to schedule workflows on resources with elastically changing demand, as they allow resources to scale on demand as usage changes through dynamic provisioning. To address this issue, we firstly formalize a model of a service-oriented computing environment and a workflow graph representation for the environment. Then, we propose SCPOR, a scientific workflow scheduling algorithm that is able to schedule workflows in need of elastically changing compute resources. Our extensive experiments and comparisons for compute-intensive and dataintensive workflows have shown that SCPOR not only outperforms several representative workflow scheduling algorithms in optimizing workflow execution time, but also enables resources to scale elastically during workflow execution.
2009 IEEE International Conference on Web Services, 2009
MapReduce has recently gained a lot of attention as a parallel programming model for scalable dat... more MapReduce has recently gained a lot of attention as a parallel programming model for scalable data-intensive business and scientific analysis. In order to benefit from this powerful programming model in a scientific workflow environment, we propose a MapReduce-enabled scientific workflow composition framework consisting of: i) a dataflowbased scientific workflow model that separates the declaration of the workflow interface from the definition of its functional body; ii) a set of dataflow constructs, including Map, Reduce, Loop, and Conditional, and their composition semantics to enable MapReduce-style scientific workflows; iii) an XML-based scientific workflow specification language, called WSL, in which both Map and Reduce are fully composable with other dataflow constructs in both flat and hierarchical manners. Besides leveraging the power of MapReduce to the workflow level, our workflow composition framework is unique in that workflows are the only operands for composition; in this way, our approach elegantly solves the two-world problem of existing composition frameworks, in which composition needs to deal with both the world of tasks and the world of workflows. The proposed framework is implemented and a case study is conducted to validate our techniques.
2011 IEEE 4th International Conference on Cloud Computing, 2011
Most existing workflow scheduling algorithms only consider a computing environment in which the n... more Most existing workflow scheduling algorithms only consider a computing environment in which the number of compute resources is bounded. Compute resources in such an environment usually cannot be provisioned or released on demand of the size of a workflow, and these resources are not released to the environment until an execution of the workflow completes. To address the problem, we firstly formalize a model of a Cloud environment and a workflow graph representation for such an environment. Then, we propose the SHEFT workflow scheduling algorithm to schedule a workflow elastically on a Cloud computing environment. Our preliminary experiments show that SHEFT not only outperforms several representative workflow scheduling algorithms in optimizing workflow execution time, but also enables resources to scale elastically at runtime.
Scientific workflow management systems (SWFMSs) are facing unprecedented challenges from big data... more Scientific workflow management systems (SWFMSs) are facing unprecedented challenges from big data deluge. As revising all the existing workflow applications to fit into Cloud computing paradigm is impractical, thus migrating SWFMSs into the Cloud to leverage the functionalities of both Cloud computing and SWFMSs may provide a viable approach to big data processing. In this paper, we first discuss the challenges for scientific workflow applications and the available solutions in details, and analyze the essential requirements for a scientific computing Cloud platform. Then we propose a service framework to normalize the integration of SWFMS with Cloud computing. Meanwhile, we also present our implementation experience based on the service Framework. At last, we set up a series of experiments to demonstrate the capability of our implementation and use a Montage Image Mosaic Workflow as a showcase of the implementation.
2007 IEEE Congress on Services (Services 2007), 2007
In this demo, we present current status of our VIsual sci-Entific Workflow management system call... more In this demo, we present current status of our VIsual sci-Entific Workflow management system called VIEW, highlighting the following two features: (i) the use of Semantic Web technology to represent, store, and query provenance metadata, leading to an interoperable and extensible provenance system, and (ii) the support of visualization of various provenance graphs and intermediate or final data products of a workflow run in the form of medical images or 3-D graphical models. VIEW seamlessly integrates the interoperability, extensibility, and reasoning advantages of Semantic Web technology, the querying and storage power of a RDBMS, and the appealing visual features of visualization techniques.
Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and re... more Cloud computing is an emerging computing paradigm that can offer unprecedented scalability and resources on demand, and is getting more and more adoption in the science community, while scientific workflow management systems provide essential support such as management of data and task dependencies, job scheduling and execution, provenance tracking, etc., to scientific computing. As we are entering into a "big data" era, it is imperative to migrate scientific workflow management systems into the Cloud to manage the ever increasing data scale and analysis complexity. We propose a reference service framework for integrating scientific workflow management systems into various Cloud platforms, which consists of eight major components, including Cloud workflow management service, Cloud resource manager, etc., and 6 interfaces between them. We also present a reference framework for the implementation of Cloud Resource Manager, which is responsible for the provisioning and management of virtual resources in the Cloud. We discuss our implementation of the framework by integrating the Swift scientific workflow management system with the OpenNebula and Eucalyptus Cloud platforms, and demonstrate the capability of the solution using a NASA MODIS image processing workflow and a production deployment on the Science@Guoshi network with support for the Montage image mosaic workflow.
Scientific workflows have recently emerged as a new paradigm for scientists to formalize and stru... more Scientific workflows have recently emerged as a new paradigm for scientists to formalize and structure complex and distributed scientific processes to enable and accelerate many scientific discoveries. In contrast to business workflows, which are typically control flow oriented, scientific workflows tend to be dataflow oriented, introducing a new set of requirements for system development. These requirements demand a new architectural design for scientific workflow management systems (SWFMSs). Although several SWFMSs have been developed that provide much experience for future research and development, a study from an architectural perspective is still missing. The main contributions of this paper are: 1) based on a comprehensive survey of the literature and identification of key requirements for SWFMSs, we propose the first reference architecture for SWFMSs; 2) according to the reference architecture, we further propose a service-oriented architecture for VIEW (a VIsual sciEntific Workflow management system); 3) we implemented VIEW to validate the feasibility of the proposed architectures; and 4) we present a VIEW-based scientific workflow application system (SWFAS), called FiberFlow, to showcase the application of our VIEW system.
The fundamental goal of computational neuroscience is to discover anatomical features that reflec... more The fundamental goal of computational neuroscience is to discover anatomical features that reflect the functional organization of the brain. Investigations of the physical connections between neuronal structures and measurements of brain activity in vivo have given rise to the concepts of anatomical and functional connectivity, which have been useful for our understanding of brain mechanisms and their plasticity. However, at present there is no generally accepted computational framework for the quantitative assessment of cortical ...
Recently, there has been an increasing need in scientific workflows to solve the shimming problem... more Recently, there has been an increasing need in scientific workflows to solve the shimming problem, the use of a special kind of adaptors, called shims, to link related but incompatible workflow tasks. However, existing techniques produce scientific workflows that are cluttered with many visible shims, which distract a scientist's focus on functional components. Moreover, these techniques do not address a new type of shimming problem that occurs due to the incompatibility between the ports of a task and the inputs/outputs of ...
With ever-improving information technologies and high performance computational power, recent tec... more With ever-improving information technologies and high performance computational power, recent techniques in granular computing, soft computing and cognitive science have allowed an increasing understanding of normal and abnormal brain functions, especially ...
Scientific workflows have recently emerged as a new paradigm for scientists to formalize and stru... more Scientific workflows have recently emerged as a new paradigm for scientists to formalize and structure complex and distributed scientific processes to enable and accelerate many scientific discoveries. In contrast to business workflows, which are typically control flow oriented, scientific workflows tend to be dataflow oriented, introducing a new set of requirements for system development. These requirements demand a new architectural design for scientific workflow management systems (SWFMSs). Although several ...
As the central relay station of the human brain, the thalamus modulates sensory signals to and fr... more As the central relay station of the human brain, the thalamus modulates sensory signals to and from the cerebral cortex. The reciprocal connectivity between the cerebral cortex and the thalamus is believed to play an essential role in consciousness and various neurological disorders. Thus, in-vivo analysis of thalamo-cortical connectivity is important for our understanding of normal and pathological brain processes. In this paper • We propose a new partitioning paradigm, called coclustering, in order to segment the thalamus into thalamic nuclei based on their cortical projections. In contrast to the traditional clustering paradigm, a coclustering procedure not only simultaneously partitions cortical voxels and thalamic voxels into groups, but also identifies the corresponding strong connectivities between the two classes of groups • We develop the first coclustering algorithm, Genetic Coclustering Algorithm (GCA), to solve the coclustering problem • We apply GCA to segment the thalamus into thalamic nuclei and visualise main thalamo-cortical fibre tracts.
One of the fundamental goals of computational neuroscience is the study of anatomical features th... more One of the fundamental goals of computational neuroscience is the study of anatomical features that reflect the functional organization of the brain. The study of physical associations between neuronal structures and the examination of brain activity in vivo have given rise to the concept of anatomical and functional connectivity, which has been invaluable for our understanding of brain mechanisms and their plasticity during development. However, at present, there is no robust and accurate computational framework for the quantitative assessment of cortical connectivity patterns. In this paper, we present a quantitative analysis and modeling tool that is able to characterize anatomical connectivity patterns based on a newly developed coclustering algorithm, termed the business model-based coclustering algorithm (BCA). We apply BCA to diffusion tensor imaging (DTI) data in order to provide an automated and reproducible assessment of the connectivity patterns between different cortical areas in human brains. BCA not only partitions the cortical mantel into well-defined clusters, but also maximizes the connectivity strength between these clusters. Moreover, BCA is computationally robust and allows both outlier detection as well as parameter-independent determination of the number of clusters. Our coclustering results have showed good performance of BCA in identifying major white matter fiber bundles in human brains and facilitate the detection of abnormal connectivity patterns in patients suffering from various neurological diseases.
Uploads
Papers by Cui Lin