# Difference between revisions of "HPC - Algorithms and Applications - Winter 13"

Line 30: | Line 30: | ||

* Oct 21, Oct 28, Nov 4: [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/fundamentals.pdf Fundamentals - Parallel Architectures, Models, and Languages] | * Oct 21, Oct 28, Nov 4: [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/fundamentals.pdf Fundamentals - Parallel Architectures, Models, and Languages] | ||

** additional material: [http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-134.html Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures] (technical report by Williams et al.) | ** additional material: [http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-134.html Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures] (technical report by Williams et al.) | ||

− | ** MPI | + | ** MPI examples: Cannon's Algorithm [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/mpi_cannon.c mpi_cannon.c] (unsafe send/receive), [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/mpi_cannon_sr.c mpi_cannon_sr.c] (using MPI_Sendrecv), [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/mpi_cannon_nbl.c mpi_cannon_nbl.c] (non-blocking communication) |

* Oct 28, Nov 4, Nov 11: Dwarf No. 1 - [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/denseLA.pdf Dense Linear Algebra]; | * Oct 28, Nov 4, Nov 11: Dwarf No. 1 - [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS13/denseLA.pdf Dense Linear Algebra]; | ||

− | |||

** additional material: [http://epubs.siam.org/sirev/resource/1/siread/v46/i1/p3_s1 article by Elmroth et al.] in [http://epubs.siam.org/sirev/ SIAM Review] | ** additional material: [http://epubs.siam.org/sirev/resource/1/siread/v46/i1/p3_s1 article by Elmroth et al.] in [http://epubs.siam.org/sirev/ SIAM Review] | ||

** chapter [http://netlib.org/scalapack/slug/node74.html In-core dense matrices] of the [http://netlib.org/scalapack/slug/scalapack_slug.html ScaLAPACK User's Guide] | ** chapter [http://netlib.org/scalapack/slug/node74.html In-core dense matrices] of the [http://netlib.org/scalapack/slug/scalapack_slug.html ScaLAPACK User's Guide] | ||

+ | <!-- | ||

* Nov 19, Nov 26: Dwarf No. 5 - [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS12/structured.pdf Structured Grids] | * Nov 19, Nov 26: Dwarf No. 5 - [http://www5.in.tum.de/lehre/vorlesungen/hpc/WS12/structured.pdf Structured Grids] | ||

** articles by M. Frigo and V. Strumpen:<br> [http://www.fftw.org/~athena/papers/ics05.pdf Cache oblivious stencil operations] (preprint);<br> [http://link.springer.com/article/10.1007%2Fs11227-007-0111-y The memory behavior of cache oblivious stencil operations] (preprint can be found via Google) | ** articles by M. Frigo and V. Strumpen:<br> [http://www.fftw.org/~athena/papers/ics05.pdf Cache oblivious stencil operations] (preprint);<br> [http://link.springer.com/article/10.1007%2Fs11227-007-0111-y The memory behavior of cache oblivious stencil operations] (preprint can be found via Google) |

## Revision as of 17:16, 4 November 2013

**Term**- Winter 13/14
**Lecturer**- Prof. Dr. Michael Bader
**Time and Place**- Lecture: Monday, 14.15-15.45, MI 02.07.023 (starts Oct 21);

Tutorial: Wednesday, 10-12, MI 02.07.023 (starts Oct 23, roughly bi-weekly) **Audience**- Elective topic in Informatics Bachelor/Master: students in mathematics or in any science or engineering discipline are welcome!
**Tutorials**- Oliver Meister
**Exam**- written or oral exam at end of semester
**Semesterwochenstunden / ECTS Credits**- 3 SWS (2V + 1Ü) / 4 ECTS
**TUMonline**- https://campus.tum.de/tumonline/lv.detail?clvnr=950111465 (lecture)

https://campus.tum.de/tumonline/wbStpModHB.detailPage?&pKnotenNr=705979 (module description)

# Content

The lecture will have a focus on parallel algorithms and implementation techniques in the field of numerical simulation and high performance computing, such as:

- linear algebra problems on dense and sparse matrices
- simulation on structured and unstructured meshes
- particle-based simulations (with long-range and short-range interactions)
- spectral methods (parallel FFT and related algorithms)
- Monte Carlo and statistical methods

(a.k.a. *the seven dwarfs of HPC*).

The accompanying tutorials will include practical assignments, and will concentrate on the programming of GPU and accelerator platforms.

# Lecture Material

Slides and exercise sheets/solutions will be made available during the lecture.

Lecture slides will be published here after the lessons: See also the lecture from winter term 2012/13.

- Oct 21: Intro
- Oct 21, Oct 28, Nov 4: Fundamentals - Parallel Architectures, Models, and Languages
- additional material: Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures (technical report by Williams et al.)
- MPI examples: Cannon's Algorithm mpi_cannon.c (unsafe send/receive), mpi_cannon_sr.c (using MPI_Sendrecv), mpi_cannon_nbl.c (non-blocking communication)

- Oct 28, Nov 4, Nov 11: Dwarf No. 1 - Dense Linear Algebra;
- additional material: article by Elmroth et al. in SIAM Review
- chapter In-core dense matrices of the ScaLAPACK User's Guide

# Tutorials

Roughly every second week a two hour tutorial will take place (details at page top; days and time will be announced here and in the lectures). The assignments and their solutions will be gradually posted here.

- Oct 23rd: Organizational remarks

# Literature and Online Material

- R.H. Bisseling: Parallel Scientific Computing - A structured approach using BSP and MPI, Oxford University Press, 2004.
- Course notes on Rob Bisseling's lecture on Parallel Algorithms (based on the text book)

- T.G. Mattson, B.A. Sanders, B.L. Massingill: Patterns for Parallel Programming, Addison-Wesley, 2005
- D.B. Kirk, W.W. Hwu: Programming Massively Parallel Processors - A Hands-on Approach, Morgan-Kaufman, 2010
- J. Sanders, E. Kandrot: CUDA by Example, Addison-Wesley, 2011

# Prerequisites

Helpful, but not strictly required is knowledge in:

- basics of numerical methods (e.g.: lecture
*IN0019 Numerical Programming*or similar) - basics of parallel programming (lecture
*Parallel Programming*,*HPC - Programming Paradigms and Scalability*, or similar)

Most important is a certain interest in problems from scientific computing and numerical simulation!