The Luxembourg SuperComputing Competence Center will host a half-day online course on containers and environment management on HPC systems. The course involves presentation of the main concepts and practical session where participants can setup their environments in Meluxina supercomputer.
Who should attend?
The course is ideal for users of HPC systems that need to setup reproducible or portable environments. This includes administrators, but also researchers who need methods to reliably and consistently distribute their workflows among colleagues and systems.
What will you learn?
Users of programming languages like Python, or software tools like R, often require packages to extend the functionality of the base system. Linux/GNU distributions offer various packages, however, packages are often missing, or the available versions are old as administrators try to reduce the size of distributions and the associated maintenance.
Containers and package and environment management systems provide a method for installing missing packages, and otherwise modify the base system environment. However, these tools are now been used to serve a variety of needs, such as
- create environments in isolation of the base system and other environments,
- source packages from a variety of sources, and
- build environments in a consistent and reproducible manner.
Training outcomes
By the end of the course, the participants will be able to:
- create environments to manage systems environments;
- use environments to manage package dependencies;
- generate reproducible descriptions of environments and containers;
- natively use underlying system resource in containers, such as network (InfiniBand interconnect) drivers.
Presentations of the various concepts will be followed by practical sessions where participants can setup and try their environments.
Prerequisites
The users focuses on package management on HPC systems. Participants should be familiar with basic HPC usage including connecting to the cluster and launching jobs. The techniques covered are applicable both in personal machines and on HPC clusters.
Compute Resources
Participants will have access to the MeluXina supercomputer during the training. For more information about MeluXina, please refer to the Meluxina overview and the MeluXina – Getting Started Guide.
Content
Session 1
An introduction to containers. Basic concepts and a hands on demo of a basic container structure. Introduction to Apptainer and Singularity, the main container platforms used in HPC systems.
Session 2
Software environments. How to create and manage environments in HPC systems. Both generic binary package and environment managers like Conda and language specific package management like pip will be covered.
Session 3
Reproducibility considerations. How to create reproducible environments with container and environment management systems.
Session 4
Interoperability with system modules. Build environments with on top of environment modules. Load containers using the module system (shpc).
Agenda
This course will be hosted online in Central European Time (CET) on 18 July, 2025 (9:00 AM – 1:00 PM CET).
Schedule
09:00 AM – 09:30 AM: Lecture Part 1 - Linux environment and containers for HPC systems
09:30 AM – 09:50 AM: Practical session 1
09:50 AM – 10:20 AM: Lecture Part 2 - The Conda environment management and package management for Python and R
10:20 AM – 10:40 AM: Practical session 2
10:40 AM – 11:00 AM: Break
11:00 AM – 11:30 AM: Lecture Part 3 - Reproducibility with Conda environments and Apptainer type containers
11:30 AM – 11:50 AM: Practical session 3
11:50 AM – 12:20 PM: Lecture Part 4 - Interoperability with Module systems
12:20 PM – 12:40 PM: Practical session 4
12:40 PM – 01:00 PM: Q&A
Important: Limited spots available (20 participants max)!
Contact person for more information:
Georgios Kafanas – Georgios.kafanas[at]uni.lu
