This online workshop is meant to give an overview of working with research data in Python using general libraries for storing, processing, analysing and sharing data. The focus is on improving performance. After covering tools for performant processing (netcdf, numpy, pandas, scipy) on single workstations the focus shifts to parallel, distributed and GPU computing (snakemake, numba, dask, multiprocessing, mpi4py).
This online training course aims to provide a basic understanding of HPDA challenges for eScience and how these are addressed by the Ophidia HPDA framework. Ophidia is a CMCC Foundation research effort targeting scientific data-intensive analysis, by joining HPC paradigms and Big Data approaches. The framework specifically targets the analysis on top of HPC systems and is currently involved in the ESiWACE2 CoE and the eFlows4HPC EuroHPC JU projects for large-scale scientific data analytics.