Algorithm Design: Parallel and Sequential

GPUs have become a buzzword for Artificial Intelligence and Machine learning, other than this they have a lot more applications, this is because of the parallel architecture support provided by GPUs. With the help of GPU architecture, which consists of several computation-capable cores, is well-suited for parallel processing. Parallel processing is a technique that involves dividing a large computational task into multiple smaller tasks that can be executed concurrently. GPUs work better than CPUs when algorithms are highly parallelizable and allow for the division of calculations into separate jobs. Algorithm complexity is just one element that affects how CPUs and GPUs compare in terms of performance.

IEEE Transactions on Computers, 1989

1996

Texts in Computational Science and Engineering, 2010

This chapter comes in two parts. The first part gives a general definition of parallelism and concurrency. Concurrent programming was largely developed in the context of operating systems. Some important steps in the early development of operating systems are highlighted to show how this occurred. It will be shown that concurrent programs can increase the throughput of programs even on single processors.

Architectural Design, 2006

Today you have interaction with hardware. You also have the opposite, that is, hardware changes according to the human being and the human being is interacting with the hardware-you have design, you have clothes, and we know that in the future expo of the 21st Century, it will be people. 1

Journal of Parallel and Distributed Computing, 1993

There are several metrics that characterize the performance of a parallel system, such as, parallel execution time, speedup and e ciency. A number of properties of these metrics have been studied. For example, it is a well known fact that given a parallel architecture and a problem of a xed size, the speedup of a parallel algorithm does not continue to increase with increasing number of processors. It usually tends to saturate or peak at a certain limit. Thus it may not be useful to employ more than an optimal number of processors for solving a problem on a parallel computer. This optimal number of processors depends on the problem size, the parallel algorithm and the parallel architecture. In this paper we study the impact of parallel processing overheads and the degree of concurrency of a parallel algorithm on the optimal number of processors to be used when the criterion for optimality is minimizing the parallel execution time. We then study a more general criterion of optimality and show how operating at the optimal point is equivalent to operating at a unique value of e ciency which is characteristic of the criterion of optimality and the properties of the parallel system under study. We put the technical results derived in this paper in perspective with similar results that have appeared in the literature before and show how this paper generalizes and/or extends these earlier results.

1995

Description/Abstract The best enterprises have both a compelling need pulling them forward and an innovative technological solution pushing them on. In high-performance computing, we have the need for increased computational power in many applications and the inevitable long-term solution is massive parallelism. In the short term, the relation between pull and push may seem unclear as novel algorithms and software are needed to support parallel computing.

Parallel computing is critical in many areas of computing, from solving complex scientific problems to improving the computing experience for smart device and personal computer users. This study investigates different methods of achieving parallelism in computing, and presents how parallelism can be identified compared to serial computing methods. Various uses of parallelism are explored. The first parallel computing method discussed relates to software architecture, taxonomies and terms, memory architecture, and programming. Next parallel computing hardware is presented, including Graphics Processing Units, streaming multiprocessor operation, and computer network storage for high capacity systems. Operating systems and related software architecture which support parallel computing are discussed, followed by conclusions and descriptions of future work in ultrascale and exascale computing.

Parallel programming is an extension of sequential programming; today, it is becoming the mainstream paradigm in day-to-day information processing. Its aim is to build the fastest programs on parallel computers. The methodologies for developing a parallel program can be put into integrated frameworks. Development focuses on algorithm, languages, and how the program is deployed on the parallel computer.