Loading Events

« All Events

  • This event has passed.

Multi-GPU Programming Bootcamp

May 6 09:00 May 7 14:30 CEST

Overview

Welcome to the Multi-GPU programming bootcamp!

Scaling applications to multiple GPUs across multiple nodes requires one to be adept at programming models and optimisation techniques, and proficient at performing root-cause analysis using in-depth profiling to identify and minimise bottlenecks.

Topics on GPU programming covers single node multi-GPU (P2P) and multi-Node multi-GPU (GPU Direct) programming, the NVIDIA Collectives Communications Library (NCCL), and the NVSHMEM, which is a parallel programming interface based on OpenSHMEM that provides efficient and scalable communication for NVIDIA GPU clusters).

The Multi-GPU Programming Bootcamp covers step-by-step ways to improve application performance using cues from profilers and provides an understanding of the underlying technologies and communication topology needed to leverage powerful NVIDIA® libraries to extract more performance from the system.

Prerequisites

  • Experience with C/C++, NVIDIA CUDA®, MPI

Agenda

Friday, May 3, 2024

Time (CET)Topic
11:00–12:00Cluster Dry Run Session

Monday, May 6, 2024

09:00-09:05Welcome
09:05-09:30Introduction to Nsight Systems and GPU Computing (Lecture and Read-only Lab)
09:30 -10:30Overview of a Single GPU Code
10:30 -12:30Parallelizing a Single GPU Code using CUDA memcpy, Streams, and an Introduction to Single Node Multi-GPU ( P2P ) (Lecture and Lab)
12:30-13:00Lunch
13:00-15:00Introduction to Multi-Node Multi-GPU (GPU Direct ) programming (Lecture and Lab) [2 hrs]
15:00-16:00LLM Projects Discussion (Optional)

Tuesday, May 7, 2024

Time (CET)Topic
09:00-11:00Introduction to NVIDIA Collectives Communications Library (NCCL) (Lecture and Lab)
11:00-13:00Introduction to NVSHMEM (Lecture and Lab)
13:00-13:30Demo of the Nsight Systems Profiler Report for a Multi-Node Application
13:30-14:30LLM Projects Discussion (Optional)

Compute Resource

Attendees will have the opportunity to use a GPU cluster for the duration of the bootcamp. 

Registration

Please register using this link https://www.openhackathons.org/s/siteevent/a0C5e000008AbEfEAK/se000316

Disclaimer

Due to EuroCC2 regulations, we cannot except generic or private email addresses. Please use your official university or company email address.

This training is intended for users established in the European Union or a country associated with Horizon 2020. You can read more about the countries associated with Horizon2020 here https://ec.europa.eu/info/research-and-innovation/statistics/framework-programme-facts-and-figures/horizon-2020-country-profiles_e

Organisers

This Bootcamp is co-organized by ENCCS, EuroCC Austria, EuroCC Germany in association with (HLRS)Jülich Supercomputing Centre (JSC)Leibniz Supercomputing Centre (LRZ)Vienna Scientific Cluster (VSC)OpenACC organization, and NVIDIA.

ENCCS Lesson Materials

Have a look at ENCCS lesson materials where you can find multiple lessons on GPU programming, data analysis and HPC optimisation.