Hardware Complexity and Parallel Computation (Preliminary Version)

Symposium on the Theory of Computing, 1982

2007

1979

A simple model of concurrent computations is presented in which disjoint instructions /processes/ of program are executed concurrently by processors /in a sufficiently large number/ under a shared memory environment. The semantics of such a program specifies the tree of configuration sequences which are acceptable as possible computations of the program. We do not agree with the existing literature /e.g. [2]/ that every sharing one processor among processes can be conceived as a concurrency. We claim that the other meaning of concurrency can be defined as well. The difference between these two meanings turns out to be essential. We do not assume that each configuration is obtained from its predecessor in the computation by exactly one processor performing an atomic step /assignment or test/ in a process. On the contrary, we assume that a processor cannot be delayed during his activities. The length of a step is indefinite, it must be finite only. This reflects various speeds of processors. Hence, for the configuration in which several processors are able to start the execution of their subsequent steps, a maximal number of atomic steps will be started, the choice being nondeterministic. We discuss semantical phenomena of concurrent computations. It is argued that they can be expressed in the language of an algorithmic logic. The problem of complete axiomatization of the latter remains open. The comparison with another model of concurrency — Petri nets — is given and, we hope, it is interesting. For, our approach offers a structured /algebraic/ restriction of the language of nets and new variants of semantics. From the results obtained in the theory of vector addition systems we learn an important property of concurrent computations — there is no faithful one processor simulation of them.

1991

In recent years, powerful theoretical techniques have been developed for supporting communication, synchronization and fault tolerance in general purpose parallel computing. The proposition of this thesis is that different techniques should be used to support different algorithms. The determining factor is granularity, or the extent to which an algorithm uses long blocks for communication between processors. We consider the Block PRAM model of Aggarwal, Chandra and Snir, a synchronous model of parallel computation in which the processors communicate by accessing a shared memory. In the Block PRAM model, there is a time cost for each access by a processor to a block of locations in the shared memory. This feature of the model encourages the use of long blocks for communication. In the thesis we present Block PRAM algorithms and lower bounds for specific problems on arrays, lists, expression trees, graphs, strings, binary trees and butterflies. These results introduce useful basic techniques for parallel computation in practice, and provide a classification of problems and algorithms according to their granularity. Also presented are optimal algorithms for universal hashing and skewing, which are techniques for supporting conflict-free memory access in general-and special-purpose parallel computations, respectively. We explore the Block PRAM model as a theoretical basis for the design of scalable general purpose parallel computers. Several simulation results are presented which show the Block PRAM model to be comparable to, and competitive with, other models that have been proposed for this role. Two major advantages of machines based on the Block PRAM model is that they are able to preserve the granularity properties of individual algorithms and can efficiently incorporate a significant degree of fault tolerance. The thesis also discusses methods for the design of algorithms that do not use synchronization. We apply these methods to define fast circuits for several fundamental Boolean functions.

Information Processing Letters, 1977

Mathematical Foundations of Computer Science 1992, 1992

Weak parallel machines represent a new class of physically feasible parallel machine models whose prominent representative is the so-called Parallel Turing Machine (PTM) as introduced by the author in 1984. Except PTMs, further members of this class are e.g. various kinds of systolic machines, cellular automata, orthogonal iterative arrays, etc. From the computational point of view the main common feature of weak parallel machines is their ability to perform pipelined computations efficiently, what is used in characterizing the corresponding machine class by so-caJled Pipelined Computation Thesis. This thesis states that on these machines the period of computation is polynomiaJly related to the space of sequential Turing machine computations. The paper gives a brief overview of the most important known results concerning PTMs and extends them by new results stressing the significance of PTMs in the context of physically feasible parallel computations.

Theoretical Computer Science, 2003

This paper contains answers to several problems in the theory of the computational complexity of inÿnite words. We show that the problem whether all inÿnite words generated by iterating deterministic generalized sequential machines have logarithmic space complexity is equivalent to the open problem asking whether the unary classes of languages in P and in DLOG are equivalent. Similarly, the problem to ÿnd a concrete inÿnite word which cannot be generated in logarithmic space is equivalent to the problem to ÿnd a concrete language which does not belong to DSPACE(n). Finally, we separate classes of inÿnite words generated by double and triple D0L TAG systems.

Journal of Parallel and Distributed Computing, 1991

The many revolutionary changes brought about by the integrated chip-in the form of significant improvements in processing, storage, and communications-have also brought about a host of related problems for designers and users of parallel and distributed systems. These systems develop and proliferate at an amazing momentum, motivating research in the understanding and testing of complex distributed systems. Unfortunately, these relatively expensive systems are being designed, built, used, refined, and rebuilt (at perhaps an avoidable expense) even before we have developed methodology for understanding the underlying principles of their behavior. Though it is not realistic to expect that the current rate of manufacturing can be slowed down to accommodate research in design principles, it behooves us to bring attention to the importance of design methodology and performance understanding of such systems and, in this way, to attempt to influence parallel system design in a positive manner. At the present time, there is considerable debate among various schools of thought on parallel machine architectures, with different schools proposing different architectures and design philosophies. Consider, for example, one such debate involving tightly coupled systems. Early on, Minsky [l] conjectured a somewhat pessimistic bound of log y1 for typical speedup on n processors. Since then, researchers [2 ] have shown that certain characteristics of programs, such as the DO loops in Fortran, can often be exploited to yield more optimistic levels of speedup. Other researchers [ 31 counter this kind of optimism by pointing out that parallel and vector processing has limitations in potential speedup (i.e., Amdahl's law) to the extent that speedup is bounded from above by n/ (s. n + 1-s), where s is the fraction of a computation that must be done serially. This suggests that it makes more sense to first concentrate on achieving the maximum speedup possible with a single powerful processor. In this view, the distributed approach is not as attractive an option. More recently, work on hypercubes [4] appears to indicate that

… Proceedings of the Conference Held at …, 1986

Journal of Parallel and Distributed Computing, 1994

We discuss issues pertinent to performance analysis of massively parallel systems. We first argue that single parameter characterization of parallel software or of parallel hardware rarely provides insight into the complex interactions among the soft,ware and the hard ware components of a parallel system. In particular, bounds for the speed up based upon simple models of parallelism are violated when a model ignores the effects of communication.

New Generation Computing, 1999

We formalize the implementation mechanisms required to support or-parallel execution of logic programs in terms of operations on dynamic data structures. Upper and lower bounds are derived, in terms of the number of operations n performed on the data structure, for the problem of guaranteeing correct semantics during or-parallel execution.

Foundations of Software Technology and …, 1997

We study several data-structures and operations that commonly arise in parallel implementations of logic programming languages. The main problems that arise in implementing such parallel systems are abstracted out and precisely stated. Upper and ...

1992

Theoretical Computer Science, 1989

We present efficient time-bounded ATM and space-bounded TM simulations of one-way conglomerates (OWCs), which are interconnesrcti networks of finite-state machines tnat allow only one-way communication between adjacent nodes. In particular, we show that OWCs with depth D(n) and operating in time T(n) can be simulated by ATMs in time 0(D(n). log T(n)) (and hence by a TM with the same amount of space). This extends Ruzzo's result that boolean circuits of depth D(n) can be simulated by 0(D(n))-tims bounded ATMs, and refines Goldschlager's result that two-way conglomerates operating in T(o) time can be simulated by T(n)space bounded TMs. By exploiting the regularity of interconnections in some OWCs, we obtain more efficient space-bounded TM simulations. For example, using the ATM result, a kdimensional one-way mesh array of n'-nodes would require n* space on a TM (such an array can run in c" time in the worst case). WC show that the space can be reduced to n'-aih.