Numerical Relativity in a Distributed Environment

2001

Abstract The Cactus software package is representative for a class of scientific applications that are tightly coupled, have regular space decompositions, and involve huge memory and processor time requirements. Cactus has proved to be a valuable tool for astrophysicists, who first initiated its development. However, today's fastest supercomputers are not powerful enough to perform realistic large-scale astrophysics simulations with Cactus.

International Journal of …, 2001

The Cactus software package is suitable for a class of scientific applications that are tightly coupled, have regular space decompositions, and involve huge memory and processor time requirements. Cactus has proved to be a valuable tool for astrophysicists, who first initiated its development. However, today's fastest supercomputers are not powerful enough to perform realistic large-scale astrophysics simulations with Cactus. Instead, astrophysicists must turn to innovative resource environments-in particular, computational grids-to satisfy this need for computational power. This paper addresses issues related to the execution of applications such as Cactus in grid environments. The authors focus on two types of grids: a set of geographically distributed supercomputers and a collection of one million Internet-connected workstations. The authors study the application performance on traditional systems, validate the theoretical results against experimental data, and predict performance in the two new environments.

2001

Improvements in the performance of processors and networks make it both feasible and interesting to treat collections of workstations, servers, clusters, and supercomputers as integrated computational resources, or Grids. However, the highly heterogeneous and dynamic nature of such Grids can make application development difficult. Here we describe an architecture and prototype implementation for a Grid-enabled computational framework based on Cactus, the MPICH-G2 Grid-enabled message-passing library, and a variety of specialized features to support efficient execution in Grid environments. We have used this framework to perform record-setting computations in numerical relativity, running across four supercomputers and achieving scaling of 88% (1140 CPU's) and 63% (1500 CPUs). The problem size we were able to compute was about five times larger than any other previous run. Further, we introduce and demonstrate adaptive methods that automatically adjust computational parameters during run time, to increase dramatically the efficiency of a distributed Grid simulation, without modification of the application and without any knowledge of the underlying network connecting the distributed computers.

Proceedings. Frontiers '99. Seventh Symposium on the Frontiers of Massively Parallel Computation, 1999

We are developing a system for collaborative research and development for a distributed group of researchers at different institutions around the world. In a new paradigm for collaborative computational science, the computer code and supporting infrastructure itself becomes the collaborating instrument, just as an accelerator becomes the collaborating tool for large numbers of distributed researchers in particle physics. The design of this "Collaboratory" allows many users, with very different areas of expertise, to work coherently together, on distributed computers around the world. Different supercomputers may be used separately, or for problems exceeding the capacity of any single system, multiple supercomputers may be networked together through high speed gigabit networks. Central to this Collaboratory is a new type of community simulation code, called "Cactus". The scientific driving force behind this project is the simulation of Einstein's equations for studying black holes, gravitational waves, and neutron stars, which has brought together researchers in very different fields from many groups around the world to make advances in the study of relativity and astrophysics. But the system is also being developed to provide scientists and engineers, without expert knowledge of parallel or distributed computing, mesh refinement, and so on, with a simple framework for solving any system of partial differential equations on many parallel computer systems, from traditional supercomputers to networks of workstations.

The advent of the Petascale era provides a great opportunity as well as a great challenge for computational science and engineering. In order to fully leverage the resources available, scientific applications need to scale to unprecedented numbers of processing cores and adapt to multicore architectures with complex memory and network hierarchies. In the numerical relativity community, the XiRel project has been funded by NSF to help prepare for these upcoming resources. The central goal of XiRel is to develop a highly scalable, efficient computational infrastructure based on the Carpet adaptive mesh refinement library that is fully integrated into the Cactus framework and optimized for numerical relativity applications. This paper presents our work towards building and benchmarking such an infrastructure which will benefit a wide spectrum of scientific applications that are based on Cactus.

IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference, 2008

We presented the technologies and algorithms to build a web-based visualization and steering system to monitor the dynamics of remote parallel simulations executed on a Linux Cluster. The polynomial time based algorithm to optimally utilize distributed computing resources over a network to achieve maximum frame-rate was also proposed. Keeping up with the advancements in modern web technologies, we have developed an Ajax-based web frontend which allows users to remotely access and control ongoing computations via a web browser facilitated by visual feedbacks in real-time. Experimental results are also given from sample runs mapped to distributed computing nodes and initiated by users at different geographical locations. Our preliminary results on frame-rates illustrated that system performance was affected by network conditions of the chosen mapping loop including available network bandwidth and computing capacities. The underlying programming framework of our system supports mixed-programming mode and is flexible to integrate most serial or parallel simulation code written in different programming languages such as Fortran, C and Java.

2008

This is the first in a series of technical reports from the NSF XiRel project which is developing the basis for a next-generation infrastructure for numerical relativity, based on the Cactus framework and the Carpet Adaptive Mesh Refinement infrastructure. In this report, we construct an initial set of weak scaling benchmarks for numerical relativity codes to measure the scaling and performance of adaptive mesh refinement on current supercomputers. Initial results are provided for these benchmarks with Cactus and Carpet, providing a basis for measuring and understanding performance gains in the future development of XiRel.

Future Generation Computer Systems, 2005

We describe the Astrophysics Simulation Collaboratory (ASC) portal, a collaborative environment in which distributed projects can perform research. The ASC project seeks to provide a web-based problem solving framework for the astrophysics community to harness computational Grids. To facilitate collaboration amongst distributed researchers within a virtual organization, the ASC Portal supplies specialized tools for the management of large-scale numerical simulations and the resources on which they are performed. The ASC Virtual Organization uses the Cactus framework for studying numerical relativity and astrophysics phenomena. We describe the architecture of the ASC Portal and present its components with emphasis on elements related to the Cactus framework.

2002

This past decade has seen rapid growth in the size, resolution, and complexity of Grand Challenge simulation codes. This trend is accompanied by a trend towards multinational, multidisciplinary teams who carry out this research in distributed teams, and the corresponding growth of Grid infrastructure to support these widely distributed Virtual Organizations. As the number and diversity of distributed teams grow, the need for visualization tools to analyze and display multi-terabyte, remote data becomes more pronounced and more urgent. One such tool that has been successfully used to address this problem is Visapult. Visapult is a parallel visualization tool that employs Grid-distributed components, latency tolerant visualization and graphics algorithms, along with high performance network I/O in order to achieve effective remote analysis of massive datasets. In this paper we discuss improvements to network bandwidth utilization and responsiveness of the Visapult application that result from using connectionless protocols to move data payload between the distributed Visapult components and a Grid-enabled, highperformance physics simulation used to study gravitational waveforms of colliding black holes; The Cactus code. These improvements have boosted Visapult's network efficiency to 88-96% of the maximum theoretical available bandwidth on multi-gigabit Wide Area Networks, and greatly enhanced interactivity. Such improvements are critically important for future development of effective interactive Grid applications.

1999

We are developing a system for collaborative research and development for a distributed group of researchers at different institutions around the world. In a new paradigm for collaborative computational science, the computer code and supporting infrastructure itself becomes the collaborating instrument, just as an accelerator becomes the collaborating tool for large numbers of distributed researchers in particle physics. The design of this “collaboratory” allows many users, with very different areas of expertise, to work coherently together, on distributed computers around the world. Different supercomputers may be used separately, or for problems exceeding the capacity of any single system, multiple supercomputers may be networked together through high speed gigabit networks. Central to this collaboratory is a new type of community simulation code, called “Cactus”. The scientific driving force behind this project is the simulation of Einstein's equations for studying black holes, gravitational waves, and neutron stars, which has brought together researchers in very different fields from many groups around the world to make advances in the study of relativity and astrophysics. But the system is also being developed to provide scientists and engineers, without expert knowledge of parallel or distributed computing, mesh refinement, and so on, with a simple framework for solving any system of partial differential equations on many parallel computer systems, from traditional supercomputers to networks of workstations