Skip to main content

Florin Pop

Followers

26

Following

22

Co-authors

6

Public Views

Dr. Florin Pop is Professor at the Department of Computer Science and Engineering at the University Politehnica of Bucharest. His general research interests are: Large-Scale Distributed Systems (design and performance), Grid Computing and Cloud Computing, Peer-to-Peer Systems, Big Data Management, Data Aggregation, Information Retrieval and Ranking Techniques, Bio-Inspired Optimization Methods.

less

University of Cambridge

Rob van Nieuwpoort

Choopan Rattanapoka

Anura Jayasumana

Guillaume Pierre

Irisa

Columbia University

Kaladzavi Guidedi

Interests

Uploads

Papers by Florin Pop

Resource Management based on Gossip Monitoring Algorithm for Large Scale Distributed Systems

Scalable Comput. Pract. Exp., 2011

The optimization of resource management in large scale distributed systems (LSDS) with the capabi... more The optimization of resource management in large scale distributed systems (LSDS) with the capability of self-organization is a complex process. LSDS are highly dynamic systems, with permanent changes in their configurations, as peers may join and leave the system with no restriction or control. This paper present the architecture for monitoring and resource management based on existing middleware solutions through the design of algorithms and methods inspired by natural models. The architecture is decentralized and it aims to optimize resource management in different types of distributed systems such as Grid, P2P, and Cloud. The important components considered for the architecture are: allocation of resources, task scheduling, resource discovery, monitoring resources and provide fault tolerance. As the system may have a large number of nodes, we need a scalable algorithm for monitoring process, able to guarantee a fast convergence no matter what the structure of the network is. In ...

Implementing Replication of Objects in DOORS—The Object-Oriented Runtime System for Edge Computing

Sensors, 2021

Aiming for simplicity and efficiency in the domain of edge computing, DOORS is a distributed syst... more Aiming for simplicity and efficiency in the domain of edge computing, DOORS is a distributed system expected to scale up to hundreds of nodes, which encapsulates application state and behavior into objects and gives them the ability to exchange asynchronous messages. DOORS offers semi-synchronous replication and the ability to explicitly move objects from one node to another, as methods to achieve scalability and resilience. The present paper gives an outline of the system structure, describes how DOORS implements object replication, and describes a basic set of measurements, yielding an initial set of conclusions for the improvements of the design.

New scheduling approach using reinforcement learning for heterogeneous distributed systems

Journal of Parallel and Distributed Computing, 2017

Computer clusters, cloud computing and the exploitation of parallel architectures and algorithms ... more Computer clusters, cloud computing and the exploitation of parallel architectures and algorithms have become the norm when dealing with scientific applications that work with large quantities of data and perform complex and time-consuming calculations. With the rise of social media applications and smart devices, the amount of digital data and the velocity at which it is produced have increased exponentially, determining the development of distributed system frameworks and platforms that increase productivity, consistency, faulttolerance and security of parallel applications. The performance of such systems is mainly influenced by the architectural disposition and composition of the physical machines, the resource allocation and the scheduling of jobs and tasks. This paper proposes a reinforcement learning algorithm for the scheduling problem in distributed systems. The machine learning technique takes into consideration the heterogeneity of the nodes and their disposition within the grid, and the arrangement of tasks in a directed acyclic graph of dependencies, ultimately determining a scheduling policy for a better execution time. This paper also proposes a platform, in which the algorithm is implemented, that o↵ers scheduling as a service to distributed systems.

An integrated architecture for future studies in data processing for smart cities

Microprocessors and Microsystems, 2017

Data processing for Smart Cities become more challenging, facing with different handling steps: d... more Data processing for Smart Cities become more challenging, facing with different handling steps: data collection from different heterogeneous sources, processing sometimes in real-time and then delivered to high level services or applications used in Smart Cities. Applications used for intelligent transportation systems, crowd management, water resources management, noise and air pollution management, require different data processing techniques. The main subject of this paper is to propose an architecture for data processing in Smart Cities. The architecture is oriented on the flow of data from the source to the end user. We describe seven steps of data processing: collection of data from heterogeneous sources, data normalization, data brokering, data storage, data analysis, data visualization and decision support systems. We consider two case studies on crowd management in smart cities and on Intelligent Transportation Systems (ITS) and we provide experimental highlights.

MLBox: Machine learning box for asymptotic scheduling

Information Sciences, 2017

As the usability of Cloud-based solutions has increased for various types of users with different... more As the usability of Cloud-based solutions has increased for various types of users with different needs, from scientists that want to process big data sets collected from sensors or business analysts that want to take decisions based on the huge amount of gathered data to simple users that store or share documents via a Cloud platform, the generated data is increasing more and more. For example, the ATLAS and other detectors at CERN generate petabytes of data and Facebook stores data with a rate of around 600 TB daily. In the current context, efficient scheduling for Big Data applications is a challenge and an appropriate scheduling technique is required for different types of incoming requests. In this paper we propose a scheduling algorithm for different types of computation requests: independent tasks, like bag of tasks (BoT) model or tasks with dependencies modeled as directed acyclic graphs (DAG), and they will be scheduled for execution in a Cloud datacenter. The tasks in the requests are scheduled on the available resources using the suitable scheduling algorithm for each request. We rely on a machine learning toolbox, named as MLBox, to find what algorithm should be used for a certain request. We implemented four heuristics for scheduling BoTs and four heuristics for DAGs scheduling and generated the training data for the machine learning algorithm by running multiple traditional scheduling algorithms and selecting the 'best' one for a given request. We evaluate the performance by comparing the scheduling of different tasks requests using some of the traditional algorithms and our machine learning based scheduling algorithm.

Simulation Framework for the Evaluation of Dependable Distributed Systems

Scalable Computing Practice and Experience, Jan 3, 2001

The use of discrete-event simulators in the design and development of distributed systems is appe... more The use of discrete-event simulators in the design and development of distributed systems is appealing due to their efficiency and scalability. Their core abstractions of process and event map neatly to the components and interactions of modernday distributed systems and allow designing realistic simulation scenarios. MONARC, a multithreaded, process oriented simulation framework designed for modelling large scale distributed systems, allows the realistic simulation of a wide-range of distributed system technologies, with respect to their specific components and characteristics. In this paper we present an innovative solution to the problem of evaluating the dependability characteristic of distributed systems. Our solution is based on several proposed extensions to the simulation model of the MONARC simulation framework. These extensions refer to fault tolerance and system orchestration mechanisms being added in order to asses the reliability and availability of distributed systems. The extended simulation model includes the necessary components to describe various actual failure situations and provides the mechanisms to evaluate different strategies for replication and redundancy procedures, as well as security enforcement mechanisms.

Load-Balancing Metric for Service Dependability in Large Scale Distributed Environments

Scalable Computing: Practice and Experience, 2011

Today we live in a World-as-a-Service society, where electronic services are used everywhere, to ... more Today we live in a World-as-a-Service society, where electronic services are used everywhere, to support from large collaborations to ever more scalable business applications. In this era, specific service capabilities such as dependability and availability are more than ever needed to sustain ever more critical business-and service-oriented quality metrics. For example, today businesses need to know they can rely on a banking service called from used inside a complex client-oriented application. Service composition could not work unless services can reliably expect results from one another. In this context, we present a load-balancing metric designed to provide increased levels of availability for a wide-range of services running inside globallyscale distributed environments. We propose the use of "'service containers"', special mechanisms designed to encapsulate sets of replicated and distributed services. These mechanisms make faults transparent for the end-user based on smart decisions based on monitored information and specific metrics. We propose a set of service-oriented metrics designed to increase the use of resources in such a distributed approach. The proposed container-based approach is part of an architecture designed to increase dependability (resilience, fault tolerance, security) of a wide-range of distributed systems. We present evaluation experiments over real-world scenarios. We show how the proposed container-based mechanism can not only deliver increased tolerance to failures, but also it leads to improvements in the time-of-response, scalability or load-balancing. The solution is able to cope with various scenarios involving different failure-injection patterns in the system. The novelty of proposed solution is represented by the new metric for multi-criteria load-balancing within a fault tolerant distributed environment based on replication.

A Virtualization-based Approach to Dependable Service Computing

Scalable Computing: Practice and Experience, Sep 30, 2011

Dependability represents a critical requirement for modern distributed systems. Today new require... more Dependability represents a critical requirement for modern distributed systems. Today new requirements emerged. Among them reliability, safety, availability, security and maintainability are needed by more and more modern distributed service computing architectures. In this paper we present an approach to ensuring dependability in distributed infrastructures using virtualization for fault-tolerance, augmented with advanced security models. The proposed solution is part of a hierarchical architectural model that allows a unitary and aggregate approach to dependability requirements while preserving scalability of large scale distributed systems. In this context we propose dependability solutions based on the use of virtualization and modern security models, combined together to automatically protect services and applications belonging to the distributed system. We also present evaluation results for several scenarios, involving applications and services with different characteristics.

ESIP—grid based satellite data processing platform

Experiments on ESIP-Environment oriented satellite data processing platform

Earth Science Informatics, 2010

The paper describes the features and the architectures of the ESIP (Environment oriented Satellit... more The paper describes the features and the architectures of the ESIP (Environment oriented Satellite Data Processing) and gProcess platforms, supporting the Grid based satellite imagery processing. Features such as Grid services, interactive components, workflow execution, process monitoring, and service composition provided by gProcess, and satellite image processing operators provided by ESIP are described. The paper exemplifies the workflow based description of the GEMI vegetation index in ESIP. The experiments highlight and evaluate as well the performance of the GEMI workflow execution over the SEE-GRID infrastructure.

An Advanced Simulation Model For Dependable Distributed Systems

ECMS 2011 Proceedings edited by: T. Burczynski, J. Kolodziej, A. Byrski, M. Carvalho, 2011

We present a simulation model designed for evaluation of dependability in distributed systems. Th... more We present a simulation model designed for evaluation of dependability in distributed systems. The model is a modification of the MONARC simulation model by adding new capabilities for capturing the reliability, safety, availability, security, and maintainability requirements. It includes components for failures injection, and it provides evaluation mechanisms for different replication strategies, redundancy procedures, and security enforcement mechanisms. The model is implemented as an extension of the multi-threaded, process oriented simulator MONARC, which allows the realistic simulation of a wide-range of distributed system technologies, with respect to their specific components and characteristics. The experimental results show that the application of the discrete-event simulators in the design and development of the dependable distributed systems is appealing due to their efficiency and scalability

A fault tolerance approach for distributed systems using monitoring based replication

Proceedings of the 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing, 2010

Abstract High availability is a desired feature of a dependable distributed system. Replication i... more Abstract High availability is a desired feature of a dependable distributed system. Replication is a well-known technique to achieve fault tolerance in distributed systems, thereby enhancing availability. We propose an approach relying on replication techniques and based on monitoring information to be applied in distributed systems for fault tolerance. Our approach uses both active and passive strategies to implement an optimistic replication protocol. Using a proxy to handle service calls and relying on service replication strategies ...

An Agent Based Architecture for DAG Scheduling

Distributed and Parallel Systems

This paper presents an efficient agent based DAG scheduling system. The proposed system has a dec... more This paper presents an efficient agent based DAG scheduling system. The proposed system has a decentralized architecture. It is able to manage tasks with dependencies, and bases its decisions on Grid resources' status captured dynamically and used at schedule time. Fault tolerance mechanisms can also be easily implemented, providing a great degree of certainty that the schedule results will be correct and delivered before the imposed deadline. The system has been integrated and tested with MonAlisa farms and the ApMon, which is a MonAlisa extension. The results obtained so far in the performed experiments show that the system introduces a reasonable overhead while producing higher quality mappings than the centralized ones.

MedioGrid: A Grid-based Platform for Satellite Image Processing

2007 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, 2007

Abstract Recently a new Grid-based software platform was developed to allow remote sensing image ... more Abstract Recently a new Grid-based software platform was developed to allow remote sensing image processing for preventing disasters like river floods and forest fires. While several innovative components were presented in research papers in the last year, this paper presents a state-of-the-art of the whole platform and exposes the novelty elements that distinguish it from other similar approaches.

A Hybrid Algorithm for Scheduling Workflow Applications in Grid Environments (ICPDP)

Lecture Notes in Computer Science

In this paper, based on a thorough analysis of different policies for DAG scheduling, an improved... more In this paper, based on a thorough analysis of different policies for DAG scheduling, an improved algorithm ICPDP (Improved Critical Path using Descendant Prediction) is introduced. The algorithm performs well with respect to the total scheduling time, the schedule length and load balancing. In addition, it provides efficient resource utilization, by minimizing the idle time on the processing elements. The algorithm has a quadratic polynomial time complexity. Experimental results are provided to support the performance ...

Algoritmi. Culegere de probleme de programare pentru clasa a 11-a

A dependability layer for large-scale distributed systems

International Journal of Grid and Utility Computing, 2011

Ensuring dependability in large-scale distributed systems represents an important research subjec... more Ensuring dependability in large-scale distributed systems represents an important research subject today. Despite the fact that many projects obtained valuable results in this domain, no acceptable solution was yet found that could integrate all the requirements for designing a dependable system and that could exploit all the capabilities of modern systems. We present a unitary and aggregate approach to ensuring reliability, availability, safety and security of distributed systems. Starting from the proposed architecture, we present implementation details for two solutions designed to ensure fault tolerance, using virtualisation and containerbased replication of services. We also present an approach to enhance security using combined modern security models in large-scale distributed systems. The results and implementation details can serve as a methodology to assist distributed infrastructures in adopting such a middleware layer designed to enforce dependability in large-scale distributed systems.

Dynamic meta-scheduling architecture based on monitoring in distributed systems

International Journal of Autonomic Computing, 2010

The Scheduling process in Large Scale Distributed Systems (LSDS) became more important due to inc... more The Scheduling process in Large Scale Distributed Systems (LSDS) became more important due to increases in the number of users and applications. This paper presents a dynamic meta-scheduling architecture model for LSDS based on monitoring. The dynamic scheduling process tries to perform task allocation on the fly as the application executes. The monitoring has an important role in this process because it can offer a full view of nodes in distributed systems. The proposed architecture is an agent framework and contains a Grid Monitoring Service, an Execution Service and a Discovery Service. The performance of the used monitoring system, MonALISA, is very important for dynamic scheduling because it ensures the realtime process. The experimental results validate our architecture and scheduling model.

A dynamic rescheduling algorithm for resource management in large scale dependable distributed systems

Computers & Mathematics with Applications, 2012

Scheduling is a key component for performance guarantees in the case of distributed applications ... more Scheduling is a key component for performance guarantees in the case of distributed applications running in large scale heterogeneous environments. Another function of the scheduler in such system is the implementation of resilience mechanisms to cope with possible faults. In this case resilience is best approached using dedicated rescheduling mechanisms. The performance of rescheduling is very important in the context of large scale distributed systems and dynamic behavior. The paper proposes a generic rescheduling algorithm. The algorithm can use a wide variety of scheduling heuristics that can be selected by users in advance, depending on the system's structure. The rescheduling component is designed as a middleware service that aims to increase the dependability of large scale distributed systems. The system was evaluated in a real-world implementation for a Grid system. The proposed approach supports fault tolerance and offers an improved mechanism for resource management. The evaluation of the proposed rescheduling algorithm was performed using modeling and simulation. We present experimental results confirming the performance and capabilities of the proposed rescheduling algorithm.

gProcess and ESIP Platforms for Satellite Imagery Processing over the Grid

The Environment oriented Satellite Data Processing Platform (ESIP) is developed through the SEE-G... more The Environment oriented Satellite Data Processing Platform (ESIP) is developed through the SEE-GRID-SCI (SEE-GRID eInfrastructure for regional eScience) co-funded by the European Commission through FP7 [1]. The gProcess Platform [2] is a set of tools and services supporting the development and the execution over the Grid of the workflow based processing, and particularly the satelite imagery processing. The ESIP [3],[4] is build on top of the gProcess platform by adding a set of satellite image processing software modules ...

Resource Management based on Gossip Monitoring Algorithm for Large Scale Distributed Systems

Scalable Comput. Pract. Exp., 2011

The optimization of resource management in large scale distributed systems (LSDS) with the capabi... more The optimization of resource management in large scale distributed systems (LSDS) with the capability of self-organization is a complex process. LSDS are highly dynamic systems, with permanent changes in their configurations, as peers may join and leave the system with no restriction or control. This paper present the architecture for monitoring and resource management based on existing middleware solutions through the design of algorithms and methods inspired by natural models. The architecture is decentralized and it aims to optimize resource management in different types of distributed systems such as Grid, P2P, and Cloud. The important components considered for the architecture are: allocation of resources, task scheduling, resource discovery, monitoring resources and provide fault tolerance. As the system may have a large number of nodes, we need a scalable algorithm for monitoring process, able to guarantee a fast convergence no matter what the structure of the network is. In ...

Implementing Replication of Objects in DOORS—The Object-Oriented Runtime System for Edge Computing

Sensors, 2021

Aiming for simplicity and efficiency in the domain of edge computing, DOORS is a distributed syst... more Aiming for simplicity and efficiency in the domain of edge computing, DOORS is a distributed system expected to scale up to hundreds of nodes, which encapsulates application state and behavior into objects and gives them the ability to exchange asynchronous messages. DOORS offers semi-synchronous replication and the ability to explicitly move objects from one node to another, as methods to achieve scalability and resilience. The present paper gives an outline of the system structure, describes how DOORS implements object replication, and describes a basic set of measurements, yielding an initial set of conclusions for the improvements of the design.

New scheduling approach using reinforcement learning for heterogeneous distributed systems

Journal of Parallel and Distributed Computing, 2017

Computer clusters, cloud computing and the exploitation of parallel architectures and algorithms ... more Computer clusters, cloud computing and the exploitation of parallel architectures and algorithms have become the norm when dealing with scientific applications that work with large quantities of data and perform complex and time-consuming calculations. With the rise of social media applications and smart devices, the amount of digital data and the velocity at which it is produced have increased exponentially, determining the development of distributed system frameworks and platforms that increase productivity, consistency, faulttolerance and security of parallel applications. The performance of such systems is mainly influenced by the architectural disposition and composition of the physical machines, the resource allocation and the scheduling of jobs and tasks. This paper proposes a reinforcement learning algorithm for the scheduling problem in distributed systems. The machine learning technique takes into consideration the heterogeneity of the nodes and their disposition within the grid, and the arrangement of tasks in a directed acyclic graph of dependencies, ultimately determining a scheduling policy for a better execution time. This paper also proposes a platform, in which the algorithm is implemented, that o↵ers scheduling as a service to distributed systems.

An integrated architecture for future studies in data processing for smart cities

Microprocessors and Microsystems, 2017

Data processing for Smart Cities become more challenging, facing with different handling steps: d... more Data processing for Smart Cities become more challenging, facing with different handling steps: data collection from different heterogeneous sources, processing sometimes in real-time and then delivered to high level services or applications used in Smart Cities. Applications used for intelligent transportation systems, crowd management, water resources management, noise and air pollution management, require different data processing techniques. The main subject of this paper is to propose an architecture for data processing in Smart Cities. The architecture is oriented on the flow of data from the source to the end user. We describe seven steps of data processing: collection of data from heterogeneous sources, data normalization, data brokering, data storage, data analysis, data visualization and decision support systems. We consider two case studies on crowd management in smart cities and on Intelligent Transportation Systems (ITS) and we provide experimental highlights.

MLBox: Machine learning box for asymptotic scheduling

Information Sciences, 2017

As the usability of Cloud-based solutions has increased for various types of users with different... more As the usability of Cloud-based solutions has increased for various types of users with different needs, from scientists that want to process big data sets collected from sensors or business analysts that want to take decisions based on the huge amount of gathered data to simple users that store or share documents via a Cloud platform, the generated data is increasing more and more. For example, the ATLAS and other detectors at CERN generate petabytes of data and Facebook stores data with a rate of around 600 TB daily. In the current context, efficient scheduling for Big Data applications is a challenge and an appropriate scheduling technique is required for different types of incoming requests. In this paper we propose a scheduling algorithm for different types of computation requests: independent tasks, like bag of tasks (BoT) model or tasks with dependencies modeled as directed acyclic graphs (DAG), and they will be scheduled for execution in a Cloud datacenter. The tasks in the requests are scheduled on the available resources using the suitable scheduling algorithm for each request. We rely on a machine learning toolbox, named as MLBox, to find what algorithm should be used for a certain request. We implemented four heuristics for scheduling BoTs and four heuristics for DAGs scheduling and generated the training data for the machine learning algorithm by running multiple traditional scheduling algorithms and selecting the 'best' one for a given request. We evaluate the performance by comparing the scheduling of different tasks requests using some of the traditional algorithms and our machine learning based scheduling algorithm.

Simulation Framework for the Evaluation of Dependable Distributed Systems

Scalable Computing Practice and Experience, Jan 3, 2001

The use of discrete-event simulators in the design and development of distributed systems is appe... more The use of discrete-event simulators in the design and development of distributed systems is appealing due to their efficiency and scalability. Their core abstractions of process and event map neatly to the components and interactions of modernday distributed systems and allow designing realistic simulation scenarios. MONARC, a multithreaded, process oriented simulation framework designed for modelling large scale distributed systems, allows the realistic simulation of a wide-range of distributed system technologies, with respect to their specific components and characteristics. In this paper we present an innovative solution to the problem of evaluating the dependability characteristic of distributed systems. Our solution is based on several proposed extensions to the simulation model of the MONARC simulation framework. These extensions refer to fault tolerance and system orchestration mechanisms being added in order to asses the reliability and availability of distributed systems. The extended simulation model includes the necessary components to describe various actual failure situations and provides the mechanisms to evaluate different strategies for replication and redundancy procedures, as well as security enforcement mechanisms.

Load-Balancing Metric for Service Dependability in Large Scale Distributed Environments

Scalable Computing: Practice and Experience, 2011

Today we live in a World-as-a-Service society, where electronic services are used everywhere, to ... more Today we live in a World-as-a-Service society, where electronic services are used everywhere, to support from large collaborations to ever more scalable business applications. In this era, specific service capabilities such as dependability and availability are more than ever needed to sustain ever more critical business-and service-oriented quality metrics. For example, today businesses need to know they can rely on a banking service called from used inside a complex client-oriented application. Service composition could not work unless services can reliably expect results from one another. In this context, we present a load-balancing metric designed to provide increased levels of availability for a wide-range of services running inside globallyscale distributed environments. We propose the use of "'service containers"', special mechanisms designed to encapsulate sets of replicated and distributed services. These mechanisms make faults transparent for the end-user based on smart decisions based on monitored information and specific metrics. We propose a set of service-oriented metrics designed to increase the use of resources in such a distributed approach. The proposed container-based approach is part of an architecture designed to increase dependability (resilience, fault tolerance, security) of a wide-range of distributed systems. We present evaluation experiments over real-world scenarios. We show how the proposed container-based mechanism can not only deliver increased tolerance to failures, but also it leads to improvements in the time-of-response, scalability or load-balancing. The solution is able to cope with various scenarios involving different failure-injection patterns in the system. The novelty of proposed solution is represented by the new metric for multi-criteria load-balancing within a fault tolerant distributed environment based on replication.

A Virtualization-based Approach to Dependable Service Computing

Scalable Computing: Practice and Experience, Sep 30, 2011

Dependability represents a critical requirement for modern distributed systems. Today new require... more Dependability represents a critical requirement for modern distributed systems. Today new requirements emerged. Among them reliability, safety, availability, security and maintainability are needed by more and more modern distributed service computing architectures. In this paper we present an approach to ensuring dependability in distributed infrastructures using virtualization for fault-tolerance, augmented with advanced security models. The proposed solution is part of a hierarchical architectural model that allows a unitary and aggregate approach to dependability requirements while preserving scalability of large scale distributed systems. In this context we propose dependability solutions based on the use of virtualization and modern security models, combined together to automatically protect services and applications belonging to the distributed system. We also present evaluation results for several scenarios, involving applications and services with different characteristics.

ESIP—grid based satellite data processing platform

Experiments on ESIP-Environment oriented satellite data processing platform

Earth Science Informatics, 2010

The paper describes the features and the architectures of the ESIP (Environment oriented Satellit... more The paper describes the features and the architectures of the ESIP (Environment oriented Satellite Data Processing) and gProcess platforms, supporting the Grid based satellite imagery processing. Features such as Grid services, interactive components, workflow execution, process monitoring, and service composition provided by gProcess, and satellite image processing operators provided by ESIP are described. The paper exemplifies the workflow based description of the GEMI vegetation index in ESIP. The experiments highlight and evaluate as well the performance of the GEMI workflow execution over the SEE-GRID infrastructure.

An Advanced Simulation Model For Dependable Distributed Systems

ECMS 2011 Proceedings edited by: T. Burczynski, J. Kolodziej, A. Byrski, M. Carvalho, 2011

We present a simulation model designed for evaluation of dependability in distributed systems. Th... more We present a simulation model designed for evaluation of dependability in distributed systems. The model is a modification of the MONARC simulation model by adding new capabilities for capturing the reliability, safety, availability, security, and maintainability requirements. It includes components for failures injection, and it provides evaluation mechanisms for different replication strategies, redundancy procedures, and security enforcement mechanisms. The model is implemented as an extension of the multi-threaded, process oriented simulator MONARC, which allows the realistic simulation of a wide-range of distributed system technologies, with respect to their specific components and characteristics. The experimental results show that the application of the discrete-event simulators in the design and development of the dependable distributed systems is appealing due to their efficiency and scalability

A fault tolerance approach for distributed systems using monitoring based replication

Proceedings of the 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing, 2010

Abstract High availability is a desired feature of a dependable distributed system. Replication i... more Abstract High availability is a desired feature of a dependable distributed system. Replication is a well-known technique to achieve fault tolerance in distributed systems, thereby enhancing availability. We propose an approach relying on replication techniques and based on monitoring information to be applied in distributed systems for fault tolerance. Our approach uses both active and passive strategies to implement an optimistic replication protocol. Using a proxy to handle service calls and relying on service replication strategies ...

An Agent Based Architecture for DAG Scheduling

Distributed and Parallel Systems

This paper presents an efficient agent based DAG scheduling system. The proposed system has a dec... more This paper presents an efficient agent based DAG scheduling system. The proposed system has a decentralized architecture. It is able to manage tasks with dependencies, and bases its decisions on Grid resources' status captured dynamically and used at schedule time. Fault tolerance mechanisms can also be easily implemented, providing a great degree of certainty that the schedule results will be correct and delivered before the imposed deadline. The system has been integrated and tested with MonAlisa farms and the ApMon, which is a MonAlisa extension. The results obtained so far in the performed experiments show that the system introduces a reasonable overhead while producing higher quality mappings than the centralized ones.

MedioGrid: A Grid-based Platform for Satellite Image Processing

2007 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, 2007

Abstract Recently a new Grid-based software platform was developed to allow remote sensing image ... more Abstract Recently a new Grid-based software platform was developed to allow remote sensing image processing for preventing disasters like river floods and forest fires. While several innovative components were presented in research papers in the last year, this paper presents a state-of-the-art of the whole platform and exposes the novelty elements that distinguish it from other similar approaches.

A Hybrid Algorithm for Scheduling Workflow Applications in Grid Environments (ICPDP)

Lecture Notes in Computer Science

In this paper, based on a thorough analysis of different policies for DAG scheduling, an improved... more In this paper, based on a thorough analysis of different policies for DAG scheduling, an improved algorithm ICPDP (Improved Critical Path using Descendant Prediction) is introduced. The algorithm performs well with respect to the total scheduling time, the schedule length and load balancing. In addition, it provides efficient resource utilization, by minimizing the idle time on the processing elements. The algorithm has a quadratic polynomial time complexity. Experimental results are provided to support the performance ...

Algoritmi. Culegere de probleme de programare pentru clasa a 11-a

A dependability layer for large-scale distributed systems

International Journal of Grid and Utility Computing, 2011

Ensuring dependability in large-scale distributed systems represents an important research subjec... more Ensuring dependability in large-scale distributed systems represents an important research subject today. Despite the fact that many projects obtained valuable results in this domain, no acceptable solution was yet found that could integrate all the requirements for designing a dependable system and that could exploit all the capabilities of modern systems. We present a unitary and aggregate approach to ensuring reliability, availability, safety and security of distributed systems. Starting from the proposed architecture, we present implementation details for two solutions designed to ensure fault tolerance, using virtualisation and containerbased replication of services. We also present an approach to enhance security using combined modern security models in large-scale distributed systems. The results and implementation details can serve as a methodology to assist distributed infrastructures in adopting such a middleware layer designed to enforce dependability in large-scale distributed systems.

Dynamic meta-scheduling architecture based on monitoring in distributed systems

International Journal of Autonomic Computing, 2010

The Scheduling process in Large Scale Distributed Systems (LSDS) became more important due to inc... more The Scheduling process in Large Scale Distributed Systems (LSDS) became more important due to increases in the number of users and applications. This paper presents a dynamic meta-scheduling architecture model for LSDS based on monitoring. The dynamic scheduling process tries to perform task allocation on the fly as the application executes. The monitoring has an important role in this process because it can offer a full view of nodes in distributed systems. The proposed architecture is an agent framework and contains a Grid Monitoring Service, an Execution Service and a Discovery Service. The performance of the used monitoring system, MonALISA, is very important for dynamic scheduling because it ensures the realtime process. The experimental results validate our architecture and scheduling model.

A dynamic rescheduling algorithm for resource management in large scale dependable distributed systems

Computers & Mathematics with Applications, 2012

Scheduling is a key component for performance guarantees in the case of distributed applications ... more Scheduling is a key component for performance guarantees in the case of distributed applications running in large scale heterogeneous environments. Another function of the scheduler in such system is the implementation of resilience mechanisms to cope with possible faults. In this case resilience is best approached using dedicated rescheduling mechanisms. The performance of rescheduling is very important in the context of large scale distributed systems and dynamic behavior. The paper proposes a generic rescheduling algorithm. The algorithm can use a wide variety of scheduling heuristics that can be selected by users in advance, depending on the system's structure. The rescheduling component is designed as a middleware service that aims to increase the dependability of large scale distributed systems. The system was evaluated in a real-world implementation for a Grid system. The proposed approach supports fault tolerance and offers an improved mechanism for resource management. The evaluation of the proposed rescheduling algorithm was performed using modeling and simulation. We present experimental results confirming the performance and capabilities of the proposed rescheduling algorithm.

gProcess and ESIP Platforms for Satellite Imagery Processing over the Grid

The Environment oriented Satellite Data Processing Platform (ESIP) is developed through the SEE-G... more The Environment oriented Satellite Data Processing Platform (ESIP) is developed through the SEE-GRID-SCI (SEE-GRID eInfrastructure for regional eScience) co-funded by the European Commission through FP7 [1]. The gProcess Platform [2] is a set of tools and services supporting the development and the execution over the Grid of the workflow based processing, and particularly the satelite imagery processing. The ESIP [3],[4] is build on top of the gProcess platform by adding a set of satellite image processing software modules ...