ENCCS & NSC Megatron Bootcamp

Summary

The ENCCS & NSC Megatron Bootcamp provides exciting opportunities for participants to train GPT models with NVIDIA Megatron-LM under the guidance of expert mentors from National Labs, Universities, and Industry leaders in a collaborative environment.

This 3-day Bootcamp is designed to give NLP researchers a step-wised overview of the fundamentals of NVIDIA Megatron-LM ( NVIDIA open-source framework for training very large language models). The event will have many hands-on exercises complemented by tutorials, code snippets, and presentations to help you kick-start training your GPT Megatron models for your own language.

Event Format

This 3-day Bootcamp will be hosted online in the Central European Summer Time (CEST) on October 25th to 27th. All communication will be done through Zoom, Slack, and email.

Prerequisites

Experience with training distributed deep learning models with Python and Pytorch is preferred, no GPU programming knowledge is required. 
This event has limited capacity, so please make sure that prerequisites are met before applying. Participants will receive an acceptance email a week before the event.

GPU Compute Resource

Teams attending the event will be given access to "Berzelius" for the duration of the hackathon. Berzelius is the premier AI/ML cluster at NSC. To learn more about Berzelius, please visit the links below for more information.

Berzelius Overview
Berzelius - Getting Started Guide

Agenda

Day 1: October 25, 2021 (9:00 AM to 12:00 PM CEST)

Day 2: October 26, 2021 (9:00 AM to 1:30 PM CEST)

Day 3: October 27, 2021 (9:00 AM to 2:00 PM CEST)

Registration

Please register by following this link https://www.gpuhackathons.org/event/enccs-nsc-megatron-bootcamp

Follow our Events Schedule

Follow us on Twitter and subscribe to our Newsletter to stay tuned to our events and other news.