Nek5000 is an open source code for simulations of incompressible flows in complex geometries. It is widely used in a broad range of applications, including the study of thermal hydraulics in nuclear reactor cores, the modelling of ocean currents and the simulation of combustion in mechanical engines etc.
Nek5000 is written in mixed Fortran 77/C and designed to employ fully large-scale parallelism. The code has a long history of HPC development. In Nek5000, the incompressible Navier-Stokes equations are discretized in space by using high order weighted residual techniques employing tensor-product bases. The tensor-product based operator evolution can be implemented as matrix-matrix produces. With this approach, we have previously presented serial studies of partially porting Nek5000 to parallel GPU-accelerated systems (https://doi.org/10.1016/j.jpdc.2019.05.010).
Within ENCCS, together with the developers from Department of Mechanics and PDC Center for High Performance Computing, we will follow on from our previously developed work and take advantage of the optimized results to port the full version of Nek5000 to GPU-accelerated systems applied to real-life physical problems. The project will mainly focus on:
- continuing to develop and optimizing the kernels
- redesign the gather-scatter method with MPI operation to decrease the amount of data transferred between the hosts and devices and
- develop two-level multi-grid preconditioner using the technique of overlapping the GPU kernels and CPU.