1. Parallel and Distributed Computing
Zhiguo Zhang
Winter, 2009

2. Course Objectives:
As multithreaded and multi-core processors become the mainstream computing platform,
writing parallel programs to exploit these processors’ computing potential becomes
increasingly important. This class will teach students concepts and techniques in
parallel and distributed computing. This course will concentrate on the basic
concepts, models, programming, system design and software engineering issues
associated with the development of parallel/distributed (enterprise) applications
or system. Many interesting programming projects will be assigned. Parallel and
distributed programming tools such as Pthreads, OpenMP, MPI, nVidia / CUDA will be
involved.
WWW: http://sist.sysu.edu.cn/~lnszzg/09w.PDC
Email: lnszzg@mail.sysu.edu.cn
Submission: 070000000.name.chapter01.hw.ver01.zip

3. Prerequisites:
This course requires courses in operating system, programming (C, C++). The
mastering of computer networks and computer architectures is preferred.
In general, you should have basic C/C++ programming skills and concepts such as
threads, cache, etc.

4. Principal Texts:
Foundations of Multithreaded, Parallel, and Distributed Programming, Gregory R.
Andrews, Addison Wesley, 2000, ISBN 0-201-35752-6.
Supplemental Texts:
Parallel Programming: Techniques and Applications Using Networked
Workstations and Parallel Computers, Barry Wilkinson and Michael Allen, Prentice
Hall, 1999, ISBN 0-13-671710-1. (China Machine Press, 2002)
Distributed Computing Principles and Applications, M. L. Liu, 清华大学
Scalable Parallel Computing: Technology, Architecture, Programming, Kai Hwang
and Zhiwei Xu, McGraw-Hill, 1999

5. Parallel Programming with MPI, Peter Pacheco, Morgan Kaufmann, 1996, ISBN
1-55860-339-5.
Java Network Programming and Distributed Computing, David Reilly and Michael
Reilly, Addison Wesley, 2002, ISBN 0-201-71037-4.
Java Distributed Computing Jim Farley, O'Reilly Publishers, 1998. ISBN
1-56592-206-9.
Designing and Building Parallel Programs: Concepts and Tools for Parallel Software
Engineering, Ian Foster, Addison Wesley, ISBN 0-201-57594-9.
Introduction to Parallel Computing: Design and Analysis of Algorithms, Vipin
Kumar, Ananth Grama, Anshul Gupta, and George Karypis, Benjamin Cummings,
1993, ISBN 0-8053-3170-0.

6. Grading Policy:
1. Homework, Lab And Class Projects (Individual): 60%
2. Final Exam: 40%
Bonus (10%): I will randomly give 10 simple quizzes in classes. The major purpose of quizzes is to
check your attendance.

7. Tentative Lecture Outline:
Parallel and Distributed Computing – Basic concepts
Computing Platforms
Processes and synchronization
Critical sections and locks
Semaphores and Pthreads library
Monitors and Java
Multiprocessor implementations
Parallel Computing
Overview of parallel computing
Barriers, data parallel algorithms, bag of tasks
Semaphores and Pthreads library
Message passing and MPI library
Lab 1 (Parallel Programming)
Distributed Computing

8. Distributed parallel computing
Clients and servers using message passing
RPC and rendezvous
Programming distributed systems: Ada, SR, Ada, Java sockets and RMI
Distributed implementations
Distributed computing paradigms
Lab 2 (Distributed Computing)
Networked Computing
Cluster Computing
Internet Applications (Client-Server and Distributed Objects)
Enterprise Computing (J2EE)
Assignment 2 (Distributed Computing)
Formal Models
CSP, SDL, Lotos, CTL and model checking

9. Writing (Report) Guideline:
Title
Introduction
Conceptual Model
Detailed Computing Model(s) or Architecture
Software Architecture (Implementation)
Programming Considerations and Examples
Summary
References
Final Exam:
Multiple Choices
Programming
Explanation and design