top | item 44400193

(no title)

elmolino89 | 8 months ago

The whole idea of maintaining module systems in a perfect sync on several systems as compared to i.e. just rsync-ing SIFs sounds strange to me. Often HPC systems (or rather their admins) are fairly (and for a good reason) conservative, keeping old system and libraries versions. Your mileage may vary, but in a small benchmark of a bioinformatics program called samtools depending on the version the fastest binaries were either run in a conda environment or inside singularity container using Clear Linux distro. Binaries compiled using either system's GCC or from a module were slower.

One would have to repeat it throwing in at least Spack to see if this still holds water.

discuss

order

IshKebab|8 months ago

> maintaining module systems in a perfect sync on several systems

We use NFS. It's a fairly common solution. I don't think it's perfect but for this sort of thing you can turn on aggressive caching and then it works pretty well. In a previous company I believe all the modules were rsync'd nightly.