User Story Mpacts: leveraging expertise

The Mpacts simulation software is designed to simulate the behaviour of granular materials, assemblies of large numbers of particles, e.g. sand, ore, coal, grains, apples, potatoes, corn flakes etc., in industrial processes, e.g. conveyor belts, harvesting machines, excavators, rotating drums,… 

The complexity of granular dynamics 
While granular dynamics is conceptually very similar to molecular dynamics - particles are moving because of interaction forces and external forces, it is also much more complex. Contrary to atoms, particles have shape, which complicates contact detection. Their rotational degrees of freedom imply more complex equations of motion. Interaction forces are dissipative (energy is not conserved) and determined by material and surface properties (which are hard to characterize). The range of these interaction forces is also much shorter compared to the size of the particles. As a consequence, the amount of data and computations per interaction is much larger than in molecular dynamics. All these factors make it hard to develop computationally efficient simulation software for granular processes.

Memory access and impact on performance
As the particles move in (simulated) space, interacting particles don’t stay close together in memory. This complicates memory access which in turn has a direct impact on performance. The hierarchical structure of memory on modern CPUs (L1, L2, L3 cache levels, RAM) is based on the assumption that computationally expensive loops will access data through arrays in a more or less contiguous way. Memory access patterns that deviate from this will suffer from cache misses, which means that data needed by the CPU has to be fetched from more distant regions of the memory. This can cause a significant slowdown of the code. By the nature of particle flows, the neighbourhood of a particle changes continuously and inevitably, the processing of the contacts between particles will, after some time, access the particle data in a random manner, causing many cache misses and slowing down the application.

Running software efficiently thanks to VSC
Besides computing infrastructure, The Flemish Supercomputing Center (VSC) also offers its users also a wide range of services. One of these services is helping users improve their software so it can run more efficiently on the VSC infrastructure. 
In the case of Mpacts, VSC recommended sorting the particles so that particles close in (simulated) space are also close together in computer memory. This programming technique made the software more efficient and, thus, faster.

Bert Tijskens (VSC): “To optimize memory access patterns, we implemented a spatial sorting mechanism based on space-filling curves (Hilbert curves). In this way, particles that are spatially close together, and thus are likely to have contact, will also be close together in memory. This drastically increases the likelihood that cached values can be used. This typically improves the performance by a factor two.” 

Benefits for Mpacts 

Simon Vanmaercke (Co-Founder of Mpacts): “By sorting the particles in memory, we obtain maximum efficiency from the existing hardware, which translates into faster computation times by factor two. The increase by factor 2 means we are able to deliver a simulation in one month instead of two. When we use GPU, the difference is even more pronounced: performance then increases by a factor of five! This improvement has a high impact on our responsiveness in solving engineering problems.” 
 

The effect on the computational time is illustrated in this YouTube movie for single-core performance:

When using GPU’s also the available hardware capabilities are used to the maximum. 
The case of MPacts clearly illustrates the importance of good programming knowledge to get the most out of GPUs. Bringing in the expertise of VSC can be helpful in this.  

Python, C++ and Fortran in one project
For VSC, this was the first time we combined Python, C++ and Fortran programming into one project. As a spin-off, VSC built a tool that automates the compilation of Python modules from C++/C/Fortran.

Bert Tijskens (VSC): “I also introduce this tool into other industrial projects and my students. When students start programming in Python, and hit a performance problem, they simply can replace the Python module with C++ code which is more efficient.”  

A fruitful collaboration between Mpacts and VSC 

Bert Tijskens: “I used to work for the DEM Research Group at the KU Leuven, which eventually spun off Mpacts. So, I am very familiar with the computational complexity of Granular Dynamics. It is nice to be able to accelerate their business with VSC expertise”

Simon Vanmaercke (Co-Founder Mpacts): “The ability to work with the highly skilled experts in the field of HPC at VSC was a very interesting and rewarding experience. I would recommend anyone involved in supercomputing to contact the VSC and benefit from their available expertise to improve your calculations." 

Watch the VSC Success story about MPacts: 

Publised on 21/02/2023