High Performance Computing

Question Bank
Q1. What is parallel computing, and how does it differ from sequential computing?
Q2. Explain the concept of task granularity in the context of parallel algorithms. How does it
impact the performance of parallel computations?
Q3. Describe the difference between fine-grained and coarse-grained parallelism. Provide
examples of applications that are suitable for each type.
Q4. How does load balancing contribute to the efficiency of parallel algorithms? What
challenges may arise if load balancing is not achieved?
Q5. What are the key considerations for designing parallel algorithms that exhibit scalability?
Why is scalability important in parallel computing?
Q6. How does data dependence between tasks impact the design of parallel algorithms? What
strategies can be employed to handle data dependence efficiently?
Q7. Explain the concept of fault tolerance in parallel computing. What mechanisms can be used
to detect and recover from faults in a parallel system?
Q8. Describe a real-world application that benefits significantly from parallel computing. What
challenges or optimizations might be specific to parallelizing this application?
Q9. What are the four categories in Flynn's taxonomy, and how are they defined?
Q10. Explain the characteristics of Single Instruction Single Data (SISD) architecture. Provide
an example of a computing system that fits into this category.
Q11. Describe the key features of Multiple Instruction Single Data (MISD) architecture. Can you
provide a practical application scenario where MISD architecture might be beneficial?
Q12. Discuss the advantages and challenges associated with Single Instruction Multiple Data
(SIMD) architecture. Provide an example of a SIMD architecture and a typical application that
benefits from it.
Q13. Examine the characteristics of Multiple Instruction Multiple Data (MIMD) architecture.
How does MIMD differ from other categories in Flynn's taxonomy, and what are its common
implementations in modern computing systems?
Q14. What is a multicore processor, and how does it differ from a single-core processor? Explain
the advantages and challenges associated with multicore architectures.
Q15. Discuss the concept of parallelism in multicore processors. How can software developers
harness the power of multiple cores to enhance performance in applications?
Q16. Explain the concept of multithreading and how it differs from multiprocessing. What are
the potential benefits of multithreading in terms of program responsiveness and resource
utilization?
Q17. Describe the difference between hardware and software multithreading. How does
multithreading enhance performance in applications, and what considerations should be taken
into account when designing multithreaded programs?
Q18. Define the concept of N-wide superscalar architecture. How does it differ from a scalar or a
superscalar architecture, and what advantages does it offer in terms of instruction execution and
throughput?
Q19. Examine the challenges associated with designing and implementing N-wide superscalar
processors. How does the width of instruction issue impact the complexity of the processor, and
what techniques are commonly used to address these challenges?
Q20. Define communication costs in the context of parallel machines. How do communication
costs impact the overall performance of parallel algorithms, and what strategies can be employed
to minimize these costs?
Q21. Discuss the trade-offs between computation and communication costs in parallel systems.
Provide examples of scenarios where communication costs dominate and others where
computation costs are more significant.
Q22. Differentiate between the various levels of parallelism, including instruction, transaction,
task, thread, memory, and function. How do these levels interact in a parallel computing
environment, and why is it important to consider multiple levels of parallelism in system design?
Q23. Provide real-world examples of applications that exploit different levels of parallelism.
Explain how each level contributes to the overall efficiency and performance of the parallelized
application.
Q24. Compare and contrast task-level parallelism and thread-level parallelism. How do they
differ in terms of granularity, coordination, and communication requirements?
Q25. Discuss the impact of communication costs on the scalability of parallel algorithms based
on task and thread parallelism. How can the design of communication patterns influence the
effectiveness of parallelization at these levels?
Q26. Define implicit parallelism in the context of microprocessor architectures. How does
implicit parallelism differ from explicit parallelism, and what are the advantages and challenges
associated with leveraging implicit parallelism in modern processors?
Q27. Discuss the techniques and approaches used to extract implicit parallelism in
microprocessor designs. Provide examples of how compilers and hardware architectures
contribute to uncovering and exploiting parallelism in applications.
Q28. Identify and describe current trends in microprocessor architectures that aim to enhance
parallelism. How are modern processors designed to handle increasing levels of parallelism, and
what architectural features contribute to improved performance in parallel workloads?
Q29. a. Explain the concept of task decomposition in the context of parallel algorithm design.
What are the key considerations when decomposing a problem into tasks for parallel execution?
Q30. Provide examples of real-world problems and discuss how task decomposition can be
applied to achieve parallelism. How does the granularity of tasks impact the overall performance
of parallel algorithms?
Q31. Define data decomposition and discuss its role in parallel computing. How does data
decomposition differ from task decomposition, and under what circumstances is data
decomposition more suitable?
Q32. Illustrate the application of data decomposition in parallel algorithms. What challenges
might arise when dealing with irregular or dynamic data structures during the decomposition
process?
Q33. What is functional decomposition, and how does it contribute to the design of parallel
algorithms? Provide examples of problems that can be effectively solved using functional
decomposition.
Q34. Discuss the relationship between functional decomposition and task decomposition. How
can the decomposition of functions lead to the identification of parallelizable tasks in an
algorithm?
Q35. Explore the concept of domain decomposition in the context of parallel numerical
simulations or scientific computing. How does domain decomposition facilitate the
parallelization of computations over spatial or problem domains?
Q36. Describe the challenges associated with load balancing in domain decomposition. What
techniques can be employed to address load imbalance in parallel algorithms using domain
decomposition?
Q37. Define hierarchical decomposition and its significance in designing scalable parallel
algorithms. How does hierarchical decomposition enable the organization of tasks at multiple
levels of abstraction?
Q38. Provide examples of algorithms where hierarchical decomposition is particularly effective.
Discuss the advantages and potential drawbacks of using hierarchical decomposition in parallel
computing.
Q39. Explain the concept of task granularity in the context of parallel computing. How does the
granularity of tasks impact the performance and efficiency of parallel algorithms? Provide
examples of scenarios where fine-grained tasks and coarse-grained tasks are appropriate.
Q40. Discuss the challenges associated with balancing task granularity in parallel algorithms.
How does the choice of task granularity influence load balancing and the overall scalability of
parallel systems?
Q41. Define inter-task communication in parallel computing. Why is it essential, and what role
does it play in coordinating the execution of parallel tasks?
Q42. Compare and contrast message passing and shared memory as communication models
between parallel tasks. Under what circumstances would you prefer one communication model
over the other?
Q43. Describe the importance of synchronization in parallel computing. What types of
synchronization mechanisms are commonly used to manage interactions between parallel tasks?
Q44. Discuss the impact of data dependence between tasks on the design of parallel algorithms.
How can dependencies be managed to avoid race conditions and ensure the correctness of
parallel computations?
Q45. Explain the concept of load balancing in parallel computing. What is the significance of
load balancing, and how does it impact the overall performance of parallel algorithms?
Q46. Discuss various mapping techniques used for load balancing. Provide examples of static
and dynamic load balancing strategies and explain the advantages and challenges associated with
each.
Q47. Describe dynamic load balancing in the context of parallel algorithms. How does dynamic
load balancing differ from static load balancing, and under what circumstances is dynamic load
balancing more suitable?
Q48. Explore specific algorithms or approaches used for dynamic load balancing. What criteria
are considered when deciding to dynamically redistribute the workload among processors, and
how does it contribute to improved efficiency in parallel systems?